API & Automation

This page is the summary. Each section below has a page of its own:

• API authentication
• OpenAI-compatible API
• Anthropic-compatible API
• Management API
• API errors

Two API surfaces: an inference API that applications call, and a management API covering everything the console can do. Both are versioned and documented per release.

API keys

API Keys → Create Key. Copy it then — it is shown once.

Use a key, not a user account, for an application. Keys are scoped to inference and revocable individually, so rotating one does not disturb anything else. Send it as a bearer token:

curl http://<host>:8001/v1/chat/completions \
  -H "Authorization: Bearer $LIFEBOAT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"my-model","messages":[{"role":"user","content":"hello"}]}'

An administrator can reveal a stored key when it has genuinely been lost, and the reveal is written to the audit log. That is the trade: recoverable, but never quietly.

Which model name do I send?

Whatever GET /v1/models lists. That is the model’s alias if one is set, and it is deliberately not the filesystem path the server was launched with — that path is scrubbed from the listing and from streamed responses.

GET /v1/models/{id} is answered by the control plane from the same aggregated listing rather than proxied, so the list and the single-model lookup can never disagree about names or capabilities.

Endpoints

Everything under /v1/* is proxied to whichever backend the router picks, so the whole surface is load-balanced identically.

OpenAI-compatible

Endpoint	Notes
POST /v1/chat/completions	Streaming supported. The configured system prompt is injected if the request has none.
POST /v1/completions
POST /v1/embeddings	Needs an embedding server; has its own pool.
POST /v1/rerank	Also accepts PUT.
POST /v1/score	Tensor engine only — the GGUF engine returns 404.
POST /v1/classify
POST /v1/responses, GET /v1/responses/{id}, POST /v1/responses/{id}/cancel	Retrieval searches the pool, because response state is per-backend.
POST /v1/audio/transcriptions	multipart/form-data.
POST /v1/audio/speech	Text-to-speech. See the caveat below.
GET /v1/models, GET /v1/models/{id}

Anthropic-compatible

POST /v1/messages and POST /v1/messages/count_tokens. Usable by clients that speak the Anthropic protocol, including coding agents. The system prompt is applied as a top-level field, as that schema requires.

Generation

POST /v1/images/generations, /v1/images/edits, /v1/videos, /v1/meshes — these need a diffusion server; without one they return 501 no_diffusion_backend. /v1/images/variations is not implemented.

Text-to-speech caveat

Each request loads the model, so expect seconds of fixed overhead — it suits batch narration, not interactive use. Output is WAV or raw PCM; mp3 and opus are rejected explicitly rather than returned as mislabelled WAV. It does not go through the router, and has its own concurrency cap; over-cap requests queue rather than fail.

Streaming

Server-sent events on the chat, completions and messages endpoints. Streams relayed from a remote node are identical in shape to local ones, and the filesystem path is scrubbed from the bytes as they pass.

Response headers

Every response names the backend, pool, weight, in-flight count, queue wait and algorithm used — see Clusters and load balancing. Logging X-Lifeboat-Server and X-Lifeboat-Wait-ms on the client side makes a later “the API was slow” answerable.

Errors

Status	Body	What to do
401	—	Missing or wrong bearer token.
402	license_required	Grace expired. Activate a licence; the console keeps working.
404	—	On chat or completions, a genuine error. On rerank, score or audio it means that engine does not implement it, and the request has already been retried elsewhere.
413	request_too_large	No backend’s input-token cap covers it. Raise the cap or shorten the prompt.
501	no_embedding_backend	/v1/embeddings with no embedding server running.
501	no_diffusion_backend	An image or video endpoint with no diffusion server.
501	tts_engine_unavailable	This image has no text-to-speech engine.
503	no_backends	Nothing running for that model — usually a name that matches nothing.
503	embedding_only_pool	A generation request aimed at an embedding model.
503	—	Every backend at capacity, queue off.
503	queue_full	Queue at its ceiling.
504	queue_timeout	Waited longer than the queue allows.

503 and 504 bodies also report the routing mode and the effective per-backend limit, so “rejected at 4 because Superfast is on” is distinguishable from “rejected at 32 because the pool is full” without cross-referencing anything.

Management API

The same REST API the console uses: servers, models, clusters, nodes, keys, users, configuration, alerts, audit, licence. Authenticate as a user. Each release ships an OpenAPI specification, an HTML reference and a Postman collection; the live specification is always at /openapi.json.

Useful without reading any of it:

• GET /api/version — liveness and version.
• GET /api/lb/state — router snapshot: in-flight, dispatch and rejection counts, queue depth, wait percentiles.
• GET /metrics — Prometheus format, unauthenticated, for scraping.
• GET /api/hardware/profile — what this host can run, including any container memory limit.

Automating a deployment

Create servers through POST /api/servers, but omit tensor_parallel_size and mem_fraction_static rather than sending numbers — an omitted field means “recommend it”, and the recommendation re-derives at every start. A number pins the server to whatever the GPU looked like once. Note that PUT keys off the fields present in the body, so to switch a pinned server back to automatic you must send an explicit null rather than omitting it.

Speculative decoding has no silent default on the API path: a server created programmatically gets it only if the request asks for it. "speculative_algorithm": "auto" is the equivalent of the console’s default.

Sources and references

• Routing, headers and rejections: Clusters and load balancing
• Keys, roles and reveal: Configuration and admin
• Trying a request by hand: Playground