Skip to content
Docs

OpenAI-Compatible API

Existing OpenAI SDKs work against Lifeboat with a base-URL change and a Lifeboat key.

Which model name?

Section titled “Which model name?”

Whatever GET /v1/models lists — the model’s alias if one is set, deliberately not the filesystem path the server was launched with. That path is scrubbed from the listing and from streamed responses.

GET /v1/models/{id} is answered by the control plane from the same aggregated listing rather than proxied, so the list and the single lookup cannot disagree about names or capabilities.

Endpoints

Section titled “Endpoints”
Endpoint Methods Notes
/v1/chat/completions POST Streaming. Configured system prompt injected if absent.
/v1/completions POST
/v1/embeddings POST Needs an embedding server; its own pool.
/v1/rerank POST, PUT
/v1/score POST Tensor engine only — the GGUF engine 404s and Lifeboat fails over.
/v1/classify POST
/v1/responses POST
/v1/responses/{id}, /cancel GET, POST Searches the pool — response state is per-backend.
/v1/audio/transcriptions POST multipart/form-data.
/v1/audio/speech POST See below.
/v1/images/generations, /edits POST Needs a diffusion server.
/v1/videos, /v1/meshes POST Needs a diffusion server.
/v1/models, /v1/models/{id} GET

/v1/images/variations is not implemented.

Streaming

Section titled “Streaming”

Server-sent events on chat, completions and messages. A stream relayed from a remote node is identical in shape to a local one.

Things that differ from a hosted provider

Section titled “Things that differ from a hosted provider”
  • Routing headers on every response name the backend, pool, weight, in-flight count and queue wait. Log them.
  • Capacity is finite and visible. 503 and 504 carry a reason and the effective limit. See API errors.
  • Direction is enforced. An embeddings request cannot be answered by a chat model, and vice versa — it is refused rather than guessed.
  • Text-to-speech reloads per request, so expect seconds of fixed overhead; it suits batch narration, not interactive use. Output is WAV or raw PCM; mp3 and opus are rejected explicitly rather than returned as mislabelled WAV. It does not go through the router and has its own concurrency cap — over-cap requests queue, they do not fail.
  • Audio uploads are sized by bytes for the fit filter, which under-estimates compressed formats. Only relevant if a backend sets an input cap.

Sources and references

Section titled “Sources and references”