Skip to content
Docs

Clusters

A cluster is a named pool of GPUs across one or more nodes, aligned to a workload rather than to hardware — interactive, batch, one per tenant.

Why name pools by workload

Section titled “Why name pools by workload”

Because that is the thing you actually want to control. Two workloads with different latency expectations on the same GPUs interfere with each other; two clusters let you give them different routing behaviour, different weights and different capacity, without partitioning the hardware physically.

Deploying a model to a cluster

Section titled “Deploying a model to a cluster”

One action creates one server row per member, sharing model, port and settings. Each is then started individually, locally or through a node’s agent.

Per member, Lifeboat resolves sizing the same way it does for a standalone server: tensor parallelism and memory fraction are derived per member unless pinned. A GGUF member gets tensor parallelism 1, since that engine ignores it.

What you can change, and when

Section titled “What you can change, and when”
Change While servers are attached
Load-balancing algorithm Yes. It is a runtime router knob, not part of the deploy shape.
Membership — GPUs, weights, member names No. Detach the servers first, because it changes what would be deployed.

Routing tiers

Section titled “Routing tiers”

A routing tier is a label on a server that carves out an isolated pool. The same model in the same cluster can be split across tiers, and a request only sees the tier it was routed to.

Use it when two workloads must not share capacity even though they share a model — a batch job that should never delay an interactive session, for example. It is also the answer when you want two clusters’ algorithms to apply independently: a request whose candidates span several clusters falls back to plain weighted round-robin, because there is no principled way to combine two algorithms. Tiers keep them apart.

Cluster-level alerts

Section titled “Cluster-level alerts”

cluster_member_down (warning) when a member stops answering; cluster_created and cluster_edited are informational and recorded in the audit log.

Sources and references

Section titled “Sources and references”