Skip to content
Docs

Nodes & Agents

A node is another GPU host running the same image in agent mode. Multi-node is one control plane and many agents.

There is no self-registration

Section titled “There is no self-registration”

The control plane is the single source of truth. There is no gossip and no discovery — an administrator adds a node explicitly with its hostname, address and agent port, and the control plane calls out to it. Nodes never start anything on their own.

That is a deliberate design choice: a fleet where a machine can join itself is a fleet where an unexpected machine can join itself.

Authentication

Section titled “Authentication”

A shared secret (LIFEBOAT_CLUSTER_SECRET) sent as a header, which must match on both sides. An unsigned or wrong secret is rejected, and the control plane surfaces the agent’s rejection naming the node, rather than leaving you with a node that silently never appears.

Heartbeats

Section titled “Heartbeats”

The control plane polls each node every 30 seconds to refresh its status and GPU inventory, reconcile that node’s server rows, and clear a node_offline alert when the node returns.

What is NOT synchronized

Section titled “What is NOT synchronized”

Almost everything. Configuration, the model registry, licensing and routing all live in the control plane’s own store. Nothing is pushed to a node in the background.

Model weights are replicated explicitly — you choose the nodes when starting a download. Before any remote launch the control plane asks the node whether the model is present.

Deploying across nodes

Section titled “Deploying across nodes”

A cluster deploy creates one server row per member, sharing model, port and settings, and each is started individually — locally by the control plane, or remotely via that node’s agent. See Clusters.

All client traffic goes to the control plane’s port, including streaming: a stream relayed from a remote node is identical in shape to a local one, with the filesystem path scrubbed from the bytes.

A node shows offline

Section titled “A node shows offline”

In order of likelihood:

  1. The agent is not running, or its port is not reachable from the control plane.
  2. The cluster secret does not match.
  3. The agent is up but every server start on it fails — check that the agent container has a writable home directory. An agent whose home is not writable fails at engine import with a permission error mentioning a cache path, on the first launch only.

Current limits on remote nodes

Section titled “Current limits on remote nodes”
  • GGUF servers are local-node only — a remote start is refused with llamacpp_remote_unsupported.
  • Encrypted model weights are local-node only — remote node attestation is not wired up.
  • Speculative decoding is not validated on remote nodes.
  • Auto tensor parallelism cannot introspect a remote box and falls back to the node’s GPU count.

Sources and references

Section titled “Sources and references”