Troubleshooting & FAQ
Common questions and first checks when something is not behaving as expected in Lifeboat.
A server will not start. What should I check first?
Section titled “A server will not start. What should I check first?”In order: read the server logs in the UI — launch failures almost always explain themselves there; confirm the model fits the selected GPUs using the fit check and model inspection; confirm the quantization format matches what the checkpoint actually contains; and for multi-GPU servers confirm the tensor-parallel width divides evenly into the GPU selection. If a model requires a draft model for speculative decoding, its absence is reported at start rather than silently ignored.
Why am I getting 503, 504, or 413 responses?
Section titled “Why am I getting 503, 504, or 413 responses?”These come from the load balancer, and each has a distinct meaning — that specificity is deliberate:
| Response | Meaning | Usual fix |
|---|---|---|
| 503 with no available backend | No running server is serving the requested model | Start a server for that model, or correct the model name in the request |
| 503 from capacity gating | Servers are at their configured capacity | Add capacity, raise the capacity limit, or increase queue depth |
| 504 queue timeout | The request waited longer than the queue timeout allows | Add capacity or raise the timeout — sustained occurrences mean under-provisioning |
| 413 request too large | The request exceeds the configured size limit | Reduce context size or raise the limit for that server |
Rejection counts by reason are exposed in Prometheus metrics, which is the fastest way to see which of these is happening at volume.
Traffic is not split the way I configured it. Why?
Section titled “Traffic is not split the way I configured it. Why?”Check the load-balancer pools view: it shows each server’s actual traffic share against its configured share, with a drift indicator. Sustained drift above roughly 10% under steady load is the signal to investigate. The usual causes are servers of unequal real capacity carrying equal weights, one server intermittently hitting its capacity gate, or a workload whose request sizes are skewed enough that token-aware balancing correctly produces an uneven request count.
A GGUF model failed to download or serve. What went wrong?
Section titled “A GGUF model failed to download or serve. What went wrong?”Three common cases, all reported explicitly rather than failing silently:
- Whole-repository download when you wanted one quantization — use the multi-select variant picker to choose the specific file, which is frequently the difference between a 100 GB and a 14 GB download.
- Split or sharded archive — Lifeboat detects these and reports the exact merge step needed rather than failing later at server start.
- Unsupported architecture — caught at preflight, before the download rather than after it.
Do the optimizations change my model’s answers?
Section titled “Do the optimizations change my model’s answers?”No. Model weights are never quantized in order to obtain Lifeboat’s concurrency gains, and the inference mathematics are unchanged, so output quality matches the baseline engine for the same model. If you want to verify this in your own environment, run the same prompts against a server with optimizations on and one with them off — the toggle is per server precisely so that comparison is easy.
Can I run Lifeboat on AMD or Intel accelerators?
Section titled “Can I run Lifeboat on AMD or Intel accelerators?”No. Lifeboat’s optimization layer is CUDA-only and requires NVIDIA GPUs. Teams on AMD Instinct, Intel Gaudi, or Intel XPU can run the baseline upstream engine directly on those platforms, but without Lifeboat’s concurrency and latency gains.
GPU access broke after a host reboot or driver upgrade. Why?
Section titled “GPU access broke after a host reboot or driver upgrade. Why?”This is the failure mode the one-time host-setup step is designed to prevent: it installs a refresh service so GPU device passthrough survives reboots and driver upgrades. If GPU access is lost after either event, re-run host setup — it is idempotent and safe to repeat — and confirm the container toolkit and driver versions still meet the minimums in Deployment.
How does Lifeboat compare to vLLM, TGI, Triton, or Ollama?
Section titled “How does Lifeboat compare to vLLM, TGI, Triton, or Ollama?”Those are inference engines; Lifeboat is an inference engine plus an operational control plane, tuned for agent workloads. The practical differences are the concurrency gain on identical hardware at unchanged model quality, the browser-based control plane covering models, servers, clusters, keys, users, and audit, the token-aware load balancer with capacity gating built in rather than assembled separately, and multi-node fleet management as a product feature. For a single model on a single GPU with a script in front of it, a plain engine is simpler; the difference shows up at fleet scale and under agent-shaped load.
Where do I get help?
Section titled “Where do I get help?”Contact your Iterate.ai representative with your Lifeboat version, GPU model and driver version, the relevant server log excerpt, and — if the issue is load-related — the metrics or load-balancer snapshot covering the period.
Sources and references
Section titled “Sources and references”- Requirements and prerequisites: Deployment.
- Routing behavior in depth: Clusters and load balancing.