Dashboard & Telemetry

Every LifeboatCache store serves a dashboard and Prometheus metrics from the process that owns it. Nothing extra to install or run.

Where does the dashboard run?

Engine	Process serving it
vLLM, native mode	vLLM’s scheduler process
vLLM, LMCache mode	the LMCache server process
SGLang / Lifeboat	the tensor-parallel rank 0 scheduler (other ranks log that the port is taken)

It binds to 127.0.0.1:9400 by default and logs its address at startup: Lifeboat Cache dashboard: http://127.0.0.1:9400/. From a laptop, tunnel to the node:

ssh -L 9400:localhost:9400 <gpu-node>
# then open http://localhost:9400

What does it show?

Cache effectiveness: hit rate (lookups that found KV in LifeboatCache), loads served, KV bytes served, with sampled history (30 minutes at the default 2-second interval).
Compression: live codec, ratio, raw versus stored bytes, and the sampled reconstruction error.
Tiers: each storage target with its circuit-breaker state, objects, bytes, queue, which one takes new writes, and any tier skipped at startup with the reason.
Eviction and capacity: evictions, used bytes against the capacity budget.
Recovery: what was rebuilt from disk at startup and how long it took.

The Configure tab edits storage targets, the local fallback, the codec per ladder rung, the eviction policy and the write mode. Changes are validated, applied live and saved to the configuration file; sections bound at startup (engine, dashboard, telemetry, control_plane, device) are flagged restart required.

How do I expose it safely beyond localhost?

Set both a bind address and a token — never one without the other:

dashboard:
  host: 0.0.0.0
  auth_token: "<long random string>"     # or set LIFEBOAT_DASHBOARD_TOKEN in the environment
  allowed_hosts: [kvcache.example.internal]   # Host names a reverse proxy forwards, for config edits
  allow_config_edit: true                # false makes the dashboard read-only

Open it once with ?token=<token>, or send Authorization: Bearer <token>. Configuration writes additionally require Content-Type: application/json and, on a loopback bind, a loopback or allowed Host header, which blocks cross-site requests and DNS rebinding. The dashboard saves the configuration file with its original permissions (owner-only if new), because the file can hold the token.

Which HTTP endpoints are there?

Method and path	Returns
`GET /`	the dashboard page
`GET /api/stats`	live snapshot and sampled history (JSON)
`GET /api/config`	effective configuration (JSON and YAML)
`POST /api/config`	validate, apply and save a new configuration
`GET /metrics`	Prometheus text format (also on `telemetry.prometheus.port`, 9402)
`GET /healthz`	liveness — `{"ok": true}`, never needs the token

Changing a setting from a script — the body takes patch (deep-merged onto the running configuration), yaml (a whole file) or config (a whole JSON object):

curl -s -X POST http://127.0.0.1:9400/api/config \
  -H 'Content-Type: application/json' -H "Authorization: Bearer $LIFEBOAT_DASHBOARD_TOKEN" \
  -d '{"patch": {"storage": {"fallback_local": {"capacity_gb": 200}}}}'
# {"ok": true, "applied": ["storage"], "restart_required": [], "saved_to": "/home/<you>/.lifeboat_cache/lifeboat.yaml"}

An invalid change returns 400 with the validation error and changes nothing. The saved file is fully resolved: an extends: line is replaced by the values it brought in.

Status	Meaning
`401 {"error": "unauthorized"}`	token missing or wrong
`403 {"error": "bad Host header"}`	a config edit through a host name not in `allowed_hosts`
`403 … config editing is disabled`	`dashboard.allow_config_edit: false`
`415`	config edit without `Content-Type: application/json`

How do I check a node from the command line?

lifeboat-cache status                 # reads the dashboard address from the config
lifeboat-cache status --url http://10.0.0.5:9400 --token "$LIFEBOAT_DASHBOARD_TOKEN"
lifeboat-cache status --json          # the /api/stats snapshot without history

engine vllm-native  up …s  policy lru
hit rate      …%  (… / … lookups, … loads served)
compression   fp8  …x  … MiB -> … MiB
index         … chunks, … MiB
 * local-fallback   filesystem  closed   … objs   … MiB  /home/<you>/.lifeboat_cache/kv

The * marks the tier taking new writes; closed is a healthy circuit breaker (open means the tier is being skipped).

no Lifeboat node answering at http://127.0.0.1:9400: … means no store is running on that address — the engine was not started with LIFEBOAT_CACHE=1, it is a different rank’s process, or the dashboard is disabled.

Which Prometheus metrics should I scrape?

Scrape port 9402 (telemetry.prometheus.port), or /metrics on the dashboard port.

Metric	Type	Labels	Use
`lifeboat_hit_rate`	gauge	—	share of lookups served; alert on a sustained drop
`lifeboat_lookups_total`	counter	`result` (`hit`, `miss`)	hit rate over any window
`lifeboat_hits_total` / `lifeboat_misses_total`	counter	`tenant`, `tier`	where hits come from
`lifeboat_store_total`	counter	`outcome` (`stored`, `dedup`, `shed`, `shed_<reason>`)	rising `shed_*` = writes dropped under pressure
`lifeboat_tier_writes_total`	counter	`tier`	write traffic per tier
`lifeboat_tier_bytes`	gauge	`tier`	bytes held per tier
`lifeboat_tier_breaker_open`	gauge	`tier`	1 = tier is being skipped — alert on it
`lifeboat_capacity_bytes`	gauge	—	effective budget of the write tier
`lifeboat_index_entries` / `lifeboat_index_bytes`	gauge	—	what the index holds
`lifeboat_compression_raw_bytes` / `_stored_bytes`	gauge	—	compression ratio
`lifeboat_evictions_total`	counter	`tenant`, `policy`	eviction churn
`lifeboat_corrupt_total`	counter	`tenant`	blocks that failed their integrity check (recomputed)
`lifeboat_recovered_entries_total`	counter	—	entries rebuilt at startup
`lifeboat_op_latency_ms`	summary	`op`, `tier`	operation latency per tier

Label cardinality is capped by telemetry.prometheus.max_label_cardinality (10,000).

Sources and references

Configuration reference — dashboard and telemetry sections
Troubleshooting
Eviction & capacity