Skip to content
Docs

Observability & Metrics

Four sources, each answering a different question.

/metrics — Prometheus

Section titled “/metrics — Prometheus”

Plain text, unauthenticated, scraped directly by an existing Prometheus or compatible agent.

Series Type Question it answers
lifeboat_lb_dispatch_total{server_id} counter Is traffic reaching the backends I expect, in the proportions I set?
lifeboat_lb_rejected_total{reason} counter Are we rejecting, and for which reason?
lifeboat_lb_inflight{server_id} gauge Is a backend saturated?
lifeboat_lb_queue_depth gauge Is the queue absorbing bursts or filling up?
lifeboat_lb_queue_wait_seconds summary How long are requests waiting?

What to alert on, in priority order:

  1. lifeboat_lb_rejected_total{reason="queue_timeout"} rising — the pool cannot serve its load.
  2. lifeboat_lb_queue_depth sustained near its ceiling — the same, earlier.
  3. Dispatch share diverging from configured weight — a backend is unhealthy or drained.
  4. lifeboat_lb_inflight pinned at the limit — add capacity, or the mode is capping you.

/api/lb/state — live snapshot

Section titled “/api/lb/state — live snapshot”

Admin-authenticated. In-flight, dispatch and rejection counts, queue depth, wait percentiles, per-pool router state, the active routing mode, the effective caps, and the per-cluster algorithms. The Configuration and Dashboard cards render from this.

Use it for a point-in-time answer; use Prometheus for trends.

Per-request headers

Section titled “Per-request headers”

Every /v1/* response names the backend, pool, weight, in-flight count, queue wait and algorithm. Logging X-Lifeboat-Server and X-Lifeboat-Wait-ms in your client turns a later “the API was slow” into a query rather than a guess. See Load-balancing algorithms.

Server logs

Section titled “Server logs”

Per-server logs are readable from the console and are the authoritative account of a start failure or a crash. Mount the log path to a host volume for retention beyond the container’s life, and ship it to your existing log platform.

At the default log level the control plane also explains its own automatic decisions — derived memory fractions and why, measured footprints, engine auto-routing, embedding detection, parser corrections. If you are asking why Lifeboat chose something, that is where the answer is.

The audit trail

Section titled “The audit trail”

Who did what, and which decisions Lifeboat made on its own. See Audit log.

Alerts

Section titled “Alerts”

The console raises categorised alerts with severities, visible on the bell and the Alerts page. See Alerts and health.

Sources and references

Section titled “Sources and references”