Configuration Reference

LifeboatCache is configured by one YAML file, lifeboat.yaml, plus a few environment variables read by the engine plugins. Every key is optional; the file is validated strictly, so a misspelt key fails at startup with Extra inputs are not permitted instead of being ignored.

Where is the configuration file read from?

In this order, first match wins:

1. an explicit path — lifeboat_config in the engine’s connector or backend config, or a command argument such as lifeboat-cache config show <path>;
2. $LIFEBOAT_CACHE_CONFIG;
3. $LIFEBOAT_CACHE_HOME/lifeboat.yaml (default ~/.lifeboat_cache/lifeboat.yaml), if it exists;
4. built-in defaults: local tier under $LIFEBOAT_CACHE_HOME/kv (50 GB), fp8, LRU eviction, dashboard on 127.0.0.1:9400.

Within the file, values resolve in layers, lowest first: schema defaults → presets named by extends: → the file → overrides: layers (fleet, region, cluster, pool, node, model, tenant) → environment overrides.

How do I override a key from the environment?

LIFEBOAT__<section>__<key>=<value> sets any key and wins over the file:

LIFEBOAT__engine__native_cpu_gb=40 \
LIFEBOAT__storage__fallback_local__capacity_gb=500 \
LIFEBOAT_CACHE=1 vllm serve <model>

Check the result without starting an engine:

LIFEBOAT__engine__native_cpu_gb=40 lifeboat-cache config show ~/.lifeboat_cache/lifeboat.yaml --section engine

How do I use a preset?

Presets ship with the package; name one with extends: and override what you need:

extends: single_node_gpu
storage:
  tiers:
    - {name: nvme, type: nvme, enabled: true, path: /data/kv, capacity_gb: 1500}

Preset	For
single_node_gpu	one GPU node, a local NVMe tier, score-based eviction
max_concurrency	throughput-first: fp8 locally, LBQ on shared / archive tiers, ARC eviction
max_accuracy	lossless codec on local rungs, stricter accuracy sampling
storage_efficient	lbq3 on every rung, global de-duplication
dev_cpu	a laptop: CPU, small local tier, no Prometheus
fleet_50k	large fleets: control plane, ONTAP + S3 tiers, OpenTelemetry

Which keys matter for most deployments?

Key	Default	What it does
storage.tiers	[]	storage targets, fastest first — see Tiers & storage backends
storage.fallback_local.path	$LIFEBOAT_CACHE_HOME/kv	local tier used when no target is usable
storage.fallback_local.capacity_gb	50	its budget
storage.fallback_local.min_free_gb	10	free space always left on the disk
storage.shared_lookup	false	several processes sharing one tier directory
compression.ladder	fp8 on every stored rung	codec per tier type — see Compression
compression.lossless_8bit	remote	fp8 KV: raw locally, lossless on shared / archive
eviction.l1_policy	episodic in a file, lru with no file	see Eviction & capacity
engine.vllm_mode	native	native or lmcache — see vLLM modes
engine.native_cpu_gb	20	vLLM’s host-RAM tier (native mode)
engine.native_block_tokens	auto (256 where valid)	vLLM offload block size; 0 = vLLM’s default
engine.native_promotion_wait_s	5.0	wait for CPU-tier room before recomputing
engine.native_async_scheduling	auto	vLLM async scheduling: auto = off on sliding-window models (gpt-oss, Gemma), vLLM’s default elsewhere; true / false force it
engine.io_workers / read_workers	8 / 0 (auto)	store and restore threads
dashboard.host / port / auth_token	127.0.0.1 / 9400 / empty	see Dashboard & telemetry
telemetry.prometheus.port	9402	metrics port

Which environment variables do the engine plugins read?

Variable	Engines	Default	Effect
LIFEBOAT_CACHE	all	unset	1 (or true, yes, on) turns LifeboatCache on
LIFEBOAT_CACHE_HOME	all	~/.lifeboat_cache	home for the config file and the default KV directory
LIFEBOAT_CACHE_CONFIG	all	unset	path of the configuration file
LIFEBOAT_DASHBOARD_TOKEN	all	unset	dashboard token when dashboard.auth_token is empty; also read by lifeboat-cache status
LIFEBOAT_SGLANG_HOST_GB	SGLang, Lifeboat	10% of RAM, 2–16 GB	host pool per rank, in GB, or auto
LIFEBOAT_SGLANG_PAGE_SIZE	SGLang, Lifeboat	64	page size when the launch sets none
LIFEBOAT_SGLANG_WRITE_POLICY	SGLang, Lifeboat	unset	write_through, write_through_selective or write_back
LIFEBOAT_SGLANG_ZERO_COPY	SGLang, Lifeboat	1	0 = copying page interface
LIFEBOAT_SGLANG_STAGING_TOKENS	SGLang, Lifeboat	1024	bound on the GPU write-back staging buffer
LIFEBOAT_CACHE_FORCE_RADIX	SGLang, Lifeboat	1	0 = keep a disabled prefix cache disabled (LifeboatCache stays off)
LIFEBOAT_SIDECAR_TRACE	SGLang, Lifeboat	unset	1 logs hybrid-model state pages stored and requested
LIFEBOAT_MLX_PREFETCH_GB	SGLang on Apple silicon	8	prefetch buffer cap; 0 = off
LIFEBOAT__<section>__<key>	all	—	override any lifeboat.yaml key

How do I validate and inspect a configuration?

lifeboat-cache config validate /etc/lifeboat.yaml
# OK: /etc/lifeboat.yaml is valid (device=auto)
lifeboat-cache config show /etc/lifeboat.yaml --section storage --format yaml
lifeboat-cache config diff /etc/lifeboat.yaml /etc/lifeboat.new.yaml   # after resolution

Typical validation errors:

Message	Cause
Extra inputs are not permitted	misspelt or misplaced key
compression.ladder.<rung> references codec '<x>' not defined in compression.codecs	unknown codec name
storage.tiers names must be unique	two tiers with the same name
storage.coalesce requires min_tokens <= target_tokens <= max_tokens	coalescing bounds out of order
no config path given and $LIFEBOAT_CACHE_CONFIG is unset	a config command run without a path

Edits to the file take effect when the engine next starts. To change a running node, use the dashboard’s Configure tab: it applies the same validation, keeps the running configuration if the new one is invalid, applies the change live and saves it to the file (the engine, dashboard, telemetry, control_plane and device sections still need a restart).

What are all the keys?

The complete reference, generated from the configuration schema of the current release. Keys under topology, control_plane, reuse, tenancy, sessions and overrides serve fleet and multi-tenant deployments; their defaults suit a single server.

Top-level keys

Key	Type	Default	Limits
extends	str or null	null
device	auto / cpu / cuda / rocm	auto
model_profiles_dir	str	./profiles

topology

Key	Type	Default	Limits
discovery	auto / static / k8s / slurm	auto
node_id	str	auto
nvlink_domain	str	auto
numa_aware	bool	true

topology.labels

Key	Type	Default	Limits
region	str	""
az	str	""
cluster	str	""
pod	str	""
rack	str	""

control_plane

Key	Type	Default	Limits
enabled	bool	false
endpoints	list[str]	[]
membership	gossip / etcd / static	gossip
metadata_store	memory / etcd / redis	memory
index_shards	int	256	≥ 1
replication_factor	int	3	≥ 1
heartbeat_ms	int	1000	≥ 1
offline_mode	cached_placement / local_only	cached_placement

control_plane.config_push

Key	Type	Default	Limits
enabled	bool	true
canary_percent	float	5	≥ 0, ≤ 100
auto_rollback_on_regression	bool	true

compression

Key	Type	Default	Limits
recompress_on_demote	bool	true
lossless_8bit	off / remote / on	remote
decode_device	auto / cpu / cuda / rocm / mps	auto
codecs	dict[str, Codec]	the built-in set: fp8, lossless, lbq4, lbq3, lbq2, kvtc and aliases

compression.ladder

Key	Type	Default	Limits
gpu	str	lbq4
cpu	str	fp8
local_nvme	str	fp8
shared	str	fp8
archive	str	fp8

compression.accuracy_guard

Key	Type	Default	Limits
enabled	bool	true
sample_rate	float	0.01	≥ 0, ≤ 1
metric	logit_kl / cosine / mse	logit_kl
threshold	float	0.05	≥ 0
action	raise_precision / warn / evict	raise_precision

eviction

Key	Type	Default	Limits
l1_policy	lru / arc / isolated_lru / episodic / predictive	episodic

eviction.episodic

Key	Type	Default	Limits
enabled	bool	true
segment_tokens	int	2048	≥ 1
clusters_max	int	8	≥ 1
embed_dim	int	1024	≥ 8
sim_threshold	float	0.8	> 0, ≤ 1
topic_ordering	bool	false
freq_weight_s	float	10.0	≥ 0
freq_half_life_s	float	50.0	≥ 0
per_layer_budget	sensitivity / uniform / none	sensitivity
persist_evicted_episodes	bool	true
prefetch_on_topic_return	bool	true

eviction.predictive

Key	Type	Default	Limits
enabled	bool	false
horizon_s	int	300	≥ 1

eviction.admission

Key	Type	Default	Limits
enabled	bool	true
min_reuse_score	float	0.2	≥ 0, ≤ 1
write_behind	idle / always / never	idle

reuse

Key	Type	Default	Limits
cross_model_handoff	dict[str, Any]	{'enabled': False}

reuse.position_independent

Key	Type	Default	Limits
enabled	bool	true
recompute_ratio	float	0.15	≥ 0, ≤ 1
apply_to	list[str]	['documents', 'tool_outputs']

reuse.hybrid_state

Key	Type	Default	Limits
enabled	bool	true
snapshot_every_tokens	int	512	≥ 1
compress	str	turboquant4

reuse.cross_node_p2p

Key	Type	Default	Limits
enabled	bool	true
transport	nixl / ucx / tcp	nixl
max_hops	int	1	≥ 0
prefer_nvlink_domain	bool	true

storage

Key	Type	Default	Limits
enabled	bool	true
write_mode	auto / low_latency / efficient	auto
tiers	list[Tier]	[]
shared_lookup	bool	false
shared_evict_grace_s	float	30.0	≥ 0

storage.coalesce

Key	Type	Default	Limits
target_tokens	int	2048	≥ 1
min_tokens	int	256	≥ 1
max_tokens	int	8192	≥ 1
adaptive	bool	true
flush_interval_ms	int	50	≥ 0

storage.placement

Key	Type	Default	Limits
write_local_first	bool	true
replicate_to_shared_async	bool	true
read_nearest	bool	true
hot_object_replicas	int	3	≥ 1
hotspot_threshold_rps	int	500	≥ 1
hotspot_track_max	int	100000	≥ 1

storage.backpressure

Key	Type	Default	Limits
queue_depth_per_tier	int	4096	≥ 1
shed_below_reuse_score	float	0.1	≥ 0, ≤ 1
max_write_bandwidth_mbps_per_node	int	0	≥ 0

storage.backpressure.circuit_breaker

Key	Type	Default	Limits
error_rate	float	0.2	≥ 0, ≤ 1
open_s	int	30	≥ 1

storage.index

Key	Type	Default	Limits
content_addressed	bool	true
dedup	bool	true
dedup_scope	tenant / global / none	tenant
tenant_salted_hashing	bool	true

storage.fallback_local

Key	Type	Default	Limits
enabled	bool	true
path	str	$LIFEBOAT_CACHE_HOME/kv
capacity_gb	float	50	≥ 0
min_free_gb	float	10	≥ 0
direct_io	bool	true

tenancy

Key	Type	Default	Limits
enabled	bool	true
default_qos	latency / throughput / batch	throughput
fair_share	bool	true

tenancy.quotas

Key	Type	Default	Limits
capacity_gb	int	0	≥ 0
write_mbps	int	0	≥ 0
read_mbps	int	0	≥ 0
iops	int	0	≥ 0

sessions.snapshot

Key	Type	Default	Limits
enabled	bool	true
include_hybrid_state	bool	true

sessions.fork

Key	Type	Default	Limits
enabled	bool	true
copy_on_write	bool	true

security

Key	Type	Default	Limits
integrity_hash	bool	true
residency_region	str	""

security.encryption

Key	Type	Default	Limits
enabled	bool	false
kms	str	""

telemetry

Key	Type	Default	Limits
chargeback_per_tenant	bool	true

telemetry.prometheus

Key	Type	Default	Limits
enabled	bool	true
port	int	9402	≥ 1, ≤ 65535
max_label_cardinality	int	10000	≥ 1

telemetry.otel

Key	Type	Default	Limits
enabled	bool	false
endpoint	str	""
trace_sample_rate	float	0.001	≥ 0, ≤ 1

telemetry.export_locality_to_router

Key	Type	Default	Limits
enabled	bool	false
target	dynamo / llm-d / none	dynamo

dashboard

Key	Type	Default	Limits
enabled	bool	true
host	str	127.0.0.1
port	int	9400	≥ 1, ≤ 65535
auth_token	str	""
allow_config_edit	bool	true
allowed_hosts	list[str]	[]
sample_interval_s	float	2.0	> 0

engine

Key	Type	Default	Limits
vllm_mode	native / lmcache	native
native_cpu_gb	float	20	> 0
native_cpu_fit_shm	bool	true
native_block_tokens	int or null	auto (256 where every KV-cache group shares one block size)	≥ 0
native_promotion_wait_s	float	5.0	≥ 0
native_async_scheduling	bool or null	null
lmcache_host	str	localhost
lmcache_port	int	6555	≥ 1, ≤ 65535
lmcache_l1_size_gb	float or auto	auto
lmcache_eviction_policy	LRU / ARC / IsolatedLRU / noop	LRU
lmcache_chunk_size	int	256	≥ 1
lmcache_prefetch_max_in_flight	int	1	≥ 1
lmcache_extra_args	str	""
autostart_wait_timeout_s	float	180	> 0
io_workers	int	8	≥ 1
read_workers	int	0	≥ 0
l2_inflight_gb	float	4	> 0

format

Key	Type	Default	Limits
version	int	1	≥ 1
header	self_describing / minimal	self_describing
allow_tp_resplit	bool	true

overrides

Key	Type	Default	Limits
fleet	dict[str, Any]	{}
regions	dict[str, dict[str, Any]]	{}
clusters	dict[str, dict[str, Any]]	{}
pools	dict[str, dict[str, Any]]	{}
nodes	dict[str, dict[str, Any]]	{}
models	dict[str, dict[str, Any]]	{}
tenants	dict[str, dict[str, Any]]	{}

Sources and references

• Install
• Tiers & storage backends
• Compression
• Eviction & capacity
• Supported versions