Skip to content
Docs

API & Automation

Lifeboat has two API surfaces: an inference API that applications call, and a management API that covers everything the control plane can do. Both are documented and versioned per release.

What does the inference API look like?

Section titled “What does the inference API look like?”

An OpenAI-compatible surface served through Lifeboat’s load balancer, so existing clients work with only a base-URL change. It covers chat completions, legacy completions, embeddings, and model listing, with Anthropic-compatible and Google-compatible shapes available for supported endpoints.

  • Streaming is fully supported over server-sent events, including token-level timing and time-to-first-token measurement.
  • Authentication is by bearer API key issued in the control plane; keys can be paused, resumed, and revoked without redeploying anything.
  • Routing is automatic — a request names a model and the load balancer selects a healthy server serving it, across the whole fleet. See Clusters and load balancing.
  • Dispatch is observable — every proxied response carries headers identifying how it was routed, which makes per-request debugging possible without server-side correlation.

What does the management API cover?

Section titled “What does the management API cover?”

Roughly 90 REST operations spanning the full operational surface: server lifecycle; the model registry including downloads, cancellation, probing, and GGUF variant selection; multi-GPU and tensor-parallel configuration; fleet nodes and named clusters including one-call multi-node deploy; the credential vault; API-key administration; users and roles; the audit log; GPU fleet operations; configuration; and Prometheus metrics.

Management endpoints are authenticated with an administrative session token and are subject to the same role checks as the UI, so an automation account can be scoped to exactly what it needs.

Is there an API specification?

Section titled “Is there an API specification?”

Yes. Every release ships a versioned OpenAPI specification, a rendered HTML API reference, and a Postman collection — so client generation, contract testing, and manual exploration all work without reverse-engineering the surface. A bundled smoke-test harness exercises authentication and the critical read paths and is intended to be run in CI or as an acceptance check after an upgrade.

What can be automated?

Section titled “What can be automated?”

The common patterns:

  • Model rollout — register a model, download it, fit-check it, and deploy it across a named cluster, driven from a pipeline.
  • Capacity changes — start and stop servers on a schedule, for example shifting GPUs between interactive and batch pools by time of day.
  • Key lifecycle — issue, rotate, and revoke inference API keys as part of an existing secrets workflow.
  • Health gating — poll server state and the load-balancer snapshot before shifting traffic during an upgrade.
  • Compliance export — pull the audit log on a schedule into a SIEM or archive.

Sources and references

Section titled “Sources and references”