Skip to content
Docs

Troubleshooting & FAQ

This page is the summary. Each section below has a page of its own:

Organised by symptom, because that is what you have when you arrive here. Each entry names the first thing to check rather than a list of possibilities.

A server will not start

Section titled “A server will not start”

The reason on the server row is classified, not generic — read it first. Full table of causes and fixes: Why a server will not start.

The three most common:

  1. Out of memory. Check the Dashboard GPU panel for what is holding VRAM, and delete server rows you do not use — Lifeboat holds memory back for servers that might start, so stale duplicates cost real headroom.
  2. A reasoning or tool-call parser on a model that cannot support it. Clear both in Advanced. They only shape generated text and are never right for an embedding model.
  3. insufficient_shared_memory with tensor parallelism above 1. Raise LIFEBOAT_SHM_SIZE and recreate the container — it is fixed at creation.

A server sits in “starting” for a long time

Section titled “A server sits in “starting” for a long time”

Usually correct behaviour. The engine loads weights and then captures execution graphs, which on a large mixture-of-experts model takes minutes and logs a line per shape. Lifeboat only treats a server as stuck after 15 minutes and 5 minutes of silence in its log — silence is the signal, not elapsed time.

Open Show Log. If lines are still appearing, it is working.

Requests are failing

Section titled “Requests are failing”

Read X-Lifeboat-Reject-Reason on the response. Every code and its remedy: Diagnosing a rejection.

Quick map: 503 with no reason = every backend at capacity, queue off. 504 queue_timeout = not enough capacity for the load. 413 = raise max input tokens. 503 no_backends = the model name matches nothing running. 503 embedding_only_pool = a chat request aimed at an embedding model.

Everything is slow

Section titled “Everything is slow”

Check in this order:

  1. Routing mode. Superfast caps each server at 4 concurrent requests. Excellent for latency, limiting for throughput. Switch to Hybrid or Max concurrency for batch work.
  2. Queue wait. X-Lifeboat-Wait-ms, or the Dashboard’s load-balancer card. A large wait is a capacity problem, not a tuning one.
  3. GPU utilisation. Near zero while requests are slow means the bottleneck is elsewhere — queueing, or prompt processing.
  4. The model and format. Decode speed is bound by memory bandwidth, so bytes-per-token is what sets the ceiling. A 4-bit quantization reads half the bytes of 8-bit and is usually faster on memory-constrained hardware.
  5. Speculative decoding. 2-4x on single-stream responses. Set Algorithm to Auto.

Slow on a GGUF server specifically

Section titled “Slow on a GGUF server specifically”

Check max concurrency is 16 or below. That engine has fixed slots; above them the surplus queues inside the engine, where Lifeboat cannot see it — so the console shows no wait while requests are waiting.

Every turn of a long conversation is slow

Section titled “Every turn of a long conversation is slow”

Some model families cannot use prefix caching, so each turn reprocesses the whole conversation instead of just the new part. For an agent resending a long transcript every turn this is the dominant cost. If long multi-turn work is the use case, prefer a model family that caches prefixes.

Tool calls silently do nothing

Section titled “Tool calls silently do nothing”

The classic symptom: the model clearly intends to call a tool, the reply is empty or the raw markup appears as visible text, and no tool call reaches the client.

Lifeboat detects and corrects this at server start when the model’s own chat template proves the format, and records server_tool_call_parser_corrected in the audit log. If it persists:

  • Raw <tool_call> or <function= text in the reply — no parser is set, or it does not match. Re-apply the launch recommendation for that model.
  • Empty reply, no tool call, finish_reason: stop — a parser is set that matched the opening token but cannot read the body. This is the silent case; the engine logs a parse failure on that server.
  • Works unstreamed, fails streamed — a client-side accumulation issue on block-structured protocols. Fixed in current releases; upgrade.

To see what a model actually emits, send the request without tools — no parser runs, so the raw form is visible.

Thinking text appears in replies

Section titled “Thinking text appears in replies”

<think> blocks in the content mean no reasoning parser is set for that model. Set Reasoning Parser in the server’s Advanced settings and the thinking moves to its own field.

If a model spends its whole output budget thinking and returns nothing, disable thinking for that request (chat_template_kwargs) or raise the output limit. For a coding assistant, disabling it is usually right.

A download fails

Section titled “A download fails”
Symptom Check
model_download_failed A gated or private repository. Add a Hugging Face token and accept the licence on the Hub with that account.
model_too_large / remote_disk_full Disk in the models directory.
Refused before the button The repository has no format this machine can serve. Choose a GGUF build.
It pulled far more than expected Use the variant picker and tick one quantization.

Details: Model management and Integrations.

A node shows offline

Section titled “A node shows offline”

Agent not running, port not reachable from the control plane, or a mismatched cluster secret. If the agent is up but every remote start fails, check the agent container’s home directory is writable. Nodes.

An embeddings request returns a chat-model error

Section titled “An embeddings request returns a chat-model error”

The model name did not match a server, so the request was routed by position. Send a name from GET /v1/models. Current releases refuse this by direction rather than guessing.

I cannot sign in

Section titled “I cannot sign in”

No default password ships with the product. A fresh deployment asks you to create the first administrator; if no administrator can sign in, set LIFEBOAT_ADMIN_PASSWORD and restart the container. Configuration and admin.

“Licence required” when starting a server

Section titled ““Licence required” when starting a server”

The 24-hour grace period has elapsed. The console keeps working — activate from the License page. Only starting a server is gated. Licensing.

Confidential computing does not report “attested”

Section titled “Confidential computing does not report “attested””

Both halves are required: a TDX or SEV-SNP CPU with the workload inside the confidential VM, and a supported NVIDIA GPU in confidential mode. Intel TME is not TDX, workstation Blackwell cards are not the Server edition, and AMD GPUs cannot do it at all. The License page states which half is missing. A state other than attested is correct on hardware that cannot attest. Security and compliance.

What each alert means

Section titled “What each alert means”

Full table with severities: the alerts bell.

The one most often dismissed too quickly is server_auto_restarted: it means something crashed. Once after a transient is the system working. Repeatedly is a server that cannot stay up, and its log says why.

Why is the model list shorter than expected?

Section titled “Why is the model list shorter than expected?”

The recommended catalogue is filtered to formats this machine can serve, in preference order. A model missing from it is not unavailable — it is not servable here. You can still register any model by repository id or path.

Frequently asked

Section titled “Frequently asked”

Does Lifeboat quantize model weights? No. It compresses the key-value cache while weights stay at full precision, so output is unchanged.

Can I run it without a GPU? Yes — the :lite image or the desktop app, serving quantized models on the CPU or an integrated GPU. The GPU optimization layer does not apply there.

Can I run it on AMD or Intel? AMD Instinct, yes, with one image per CDNA generation. Integrated Intel and AMD GPUs, yes, through Lite. Intel Gaudi and Arc dedicated accelerators, no. Deployment has the per-generation detail.

Which NVIDIA driver? 580.65.06 or newer — the image ships the CUDA 13 runtime. An older driver passes every other check and then fails when a model loads.

Where do I download it? Container images on Docker Hub at hub.docker.com/r/iterateai/lifeboat; desktop installers and checksums at github.com/IterateAI/lifeboat-releases.

Which image tag? :latest is NVIDIA. AMD needs :latest-rocm or a generation-specific tag; a machine with no GPU needs :lite. A container image can only select on CPU architecture, not GPU vendor, so one tag cannot serve all three.

Why do the container and desktop version numbers differ? They are released on separate cadences. A desktop version is not a pullable image tag.

How does it compare to vLLM, TGI, Triton or Ollama? Those are engines (or, for Ollama, a local-development tool). Lifeboat is an engine plus the operational layer — console, roles, audit, registry, multi-node orchestration, load balancing — and adds roughly 2x concurrent capacity per GPU without touching weight precision. Choose Lifeboat when you need production operations and maximum users per GPU.

What does it send to Iterate.ai? A periodic licence check-in and, when unlicensed, a daily census; both carry a short hardware description sent once per configuration. Never model names, prompts, completions or token counts. The census is disableable and an air-gapped licence sends nothing. Licensing.

Sources and references

Section titled “Sources and references”