Management API

Everything the console can do is a REST call. Authenticate as a user with the appropriate role.

Specification

Each release ships an OpenAPI specification, an HTML reference and a Postman collection. The live specification is always at /openapi.json — regenerate from there rather than trusting a snapshot if you need the current shape.

Useful without reading any of it

Endpoint	Returns
GET /api/version	Liveness and version. The first thing to check.
GET /api/lb/state	Router snapshot: in-flight, dispatch and rejection counts, queue depth, wait percentiles, per-pool state.
GET /metrics	Prometheus format, unauthenticated.
GET /api/hardware/profile	What this host can run, including any container memory limit.

Creating servers correctly

The one thing worth getting right when automating.

Omit tensor_parallel_size and mem_fraction_static rather than sending numbers. An omitted field means “recommend it”, and the recommendation re-derives at every start from live GPU state and measured footprints. A number pins the server to whatever the GPU looked like once, which is how a server ends up taking the whole card on every start.

Note the update verb keys off the fields present in the body, so switching a pinned server back to automatic means sending an explicit null, not omitting the field. Omitting it means “leave unchanged”.

Speculative decoding has no silent default on the API path. A server created programmatically gets it only if the request asks. "speculative_algorithm": "auto" is the equivalent of the console’s default and is what you usually want.

Set backend explicitly only if you mean to override the automatic engine choice.

Roles

Viewer for reads, operator for servers and models, admin for configuration, users, keys and licensing. See Users and roles.

Sources and references

• Inference: OpenAI-compatible API
• Auth: API authentication
• Sizing: Memory and sizing