Skip to content
Docs

Release Notes

Lifeboat’s current release line is 2.x. This page summarizes changes at a product level; each release also ships a versioned OpenAPI specification, an HTML API reference, and a Postman collection. Contact your Iterate.ai representative for the detailed per-build changelog for your deployment.

Lifeboat 2.2 — highlights

Section titled “Lifeboat 2.2 — highlights”

Speculative decoding. Interactive, single-stream responses get a 2–4x speedup with identical output through draft-model speculation. For recognized models this is zero-configuration: Lifeboat fetches and registers the matching draft model automatically and pre-fills the setting when a server is created. It stacks with both the cache optimizations and low-precision weight formats.

Blackwell-native 4-bit weights. NVFP4 and MXFP4 checkpoints are auto-detected and selected, roughly halving per-token memory reads for a direct decode speedup on supported GPUs.

Upstream engine refresh. The underlying inference runtime is tracked and re-based on current upstream releases, so newly added model architectures land in Lifeboat without waiting for a Lifeboat-specific port.

Lifeboat 2.1 — highlights

Section titled “Lifeboat 2.1 — highlights”

First-class GGUF support. Register, preflight, download, and serve GGUF quantized models, with a multi-select variant picker showing real per-file sizes so a single quantization can be pulled from a multi-quant repository. Split archives and unsupported architectures are detected and reported clearly instead of failing at server start.

Cancellable downloads with real progress. Download progress reflects true byte and file totals, and a download can be cancelled mid-flight with the registry reset immediately so a corrected download can be queued right away.

Multi-GPU tensor parallelism. Serve across 1, 2, 4, or 8 GPUs with interconnect-topology awareness, composable with quantization.

Named clusters and one-call multi-node deploy. Carve fleet GPUs into business-aligned pools such as interactive, batch, or per-tenant, and deploy the same model to every node in a cluster in a single action.

Token-aware weighted load balancer. Per-model pools, per-server weights and capacity gating, an optional waiter queue, idle-pool eviction, per-request dispatch headers, and an actual-versus-configured traffic-share drift indicator in the dashboard.

Observability. A Prometheus metrics endpoint covering dispatch counts, rejections by reason, in-flight requests, queue depth, and queue wait times; a live load-balancer state snapshot; and per-server log access from the UI.

Encrypted credential vault. Hugging Face tokens stored encrypted at rest, selectable and pinnable per model, with an environment-variable fallback for simple single-tenant deployments.

Audited API-key reveal. An administrator can retrieve a stored inference key when required, and the reveal is recorded in the audit log.

GPU fleet operations. Fleet-wide GPU visibility with live utilization per node, plus optional disk deletion of model files restricted to approved model directories.

Capability probing. Probe a model to report what it actually supports — streaming, real maximum output length, effective context window, JSON mode, tool calling, vision, and reasoning traces — rather than relying on its model card.

Operational notes

Section titled “Operational notes”

Upgrades are an image swap that preserves the data volume, so the registry, users, API keys, audit log, and downloaded models carry across. Host setup is idempotent and can be safely re-run after a reboot or driver upgrade. A bundled smoke-test harness exercises authentication and the critical read paths and is intended as a post-upgrade acceptance check.

Sources and references

Section titled “Sources and references”