Configuration Reference
Lifeboat separates deployment-time configuration (environment variables, set once — see Environment variables) from runtime administration, which is this page and takes effect on save.
Routing mode
Section titled “Routing mode”The most consequential runtime setting: it changes how the deployment behaves under load, fleet-wide, immediately. Four modes and their trade-offs: Routing modes.
The card polls live, so you can watch hybrid switch between its two states per pool rather than inferring it.
Unset or unrecognised means superfast. Fresh installs and upgrades both land there deliberately.
System prompt
Section titled “System prompt”Injected into every chat request that does not carry its own. It is applied correctly on both the OpenAI-compatible and the Anthropic-compatible endpoint, which express a system prompt incompatibly — so it does not silently vanish on the second.
Per-server settings
Section titled “Per-server settings”Set on the server, applied at its next start unless noted.
| Setting | Default | Notes |
|---|---|---|
| Tensor parallel size | Auto | Tensor parallelism |
| Memory fraction | Auto | Leave on Auto — Memory and sizing |
| Context length | Auto | Context length |
| Speculative decoding | Auto | Speculative decoding |
| Weight | 1.0 | Router share. 0 drains. Immediate. |
| Max concurrency | 32 | In-flight ceiling. Immediate. ≤16 on GGUF. |
| Max input tokens | 0 | Fit filter. Immediate. |
| Routing tier | empty | Pool isolation. Immediate. |
| Reasoning parser | auto | Separates thinking from the answer. |
| Tool call parser | auto | Corrected automatically when the model’s template proves the format. |
| Extra arguments | empty | Passed to the engine. Quote JSON in single quotes. |
Extra arguments
Section titled “Extra arguments”Split like a shell command line. JSON must be in single quotes, or the double quotes are stripped and the engine fails with a bare parse error naming nothing useful. Lifeboat rejects that shape up front.
Per-cluster
Section titled “Per-cluster”The load-balancing algorithm, changeable while servers are attached — it is a router knob, not part of the deploy shape. Membership changes need the servers detached. See Clusters.
What needs what
Section titled “What needs what”| Change | Effect |
|---|---|
| Routing mode, system prompt, users, keys, weights, concurrency, tiers | Immediate on save |
| Per-server engine settings | Next start of that server |
| Environment variables | Container restart |
LIFEBOAT_SHM_SIZE |
Container recreation |
Sources and references
Section titled “Sources and references”- Deployment-time: Environment variables
- Roles: Users and roles