Skip to content
Docs

Your First Model and Server

Fifteen minutes from a running control plane to a served model.

1. Register a model

Section titled “1. Register a model”

Models → Add Model. Three sources: a repository id from the public hub, a local path, or the Recommended list — which is filtered to formats your machine can actually serve, in preference order.

Set an alias. It becomes the name the API advertises and the name applications send, and it keeps the filesystem path out of /v1/models.

2. Download it

Section titled “2. Download it”

Download on the row. If the repository holds several quantizations, the variant picker lists them with real sizes and you tick one. The button stays disabled until the listing has loaded and you have chosen — without that, an early click pulls the whole repository, which is where the 100 GB surprises came from.

See Downloading models and GGUF variants.

3. Create a server

Section titled “3. Create a server”

Servers → Create Server.

  • Model — what you just registered.
  • Name — appears in logs, alerts and routing headers.
  • Port — must be free on the host.
  • Tensor Parallel Size, Memory Fraction, Context Length — leave on Auto.

Auto is not a placeholder. It re-derives from live GPU state at every start and uses a previously measured footprint for the same weights when it has one. A pinned number is a number that was right once; pin a high memory fraction on an empty GPU and that server takes the card every time, and the next one will not start. See Memory and sizing.

4. Start it

Section titled “4. Start it”

Start, then watch. The status goes starting while the engine loads weights and captures execution graphs — legitimately minutes on a large model, with a log line per captured shape. It is only treated as stuck after 15 minutes and 5 minutes of silence in the log. Use Show Log; the panel stays open across the transition.

If it lands in error, the message on the row is classified rather than generic: Why a server will not start.

5. Talk to it

Section titled “5. Talk to it”

Create a key under API Keys, then:

Terminal window
curl http://<host>:8001/v1/chat/completions \
  -H "Authorization: Bearer $LIFEBOAT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"my-alias","messages":[{"role":"user","content":"hello"}]}'

Send the name that GET /v1/models lists. Or skip the key and use the Playground, which goes through the same router and shows you which backend answered.

What to do next

Section titled “What to do next”

Sources and references

Section titled “Sources and references”