Release Notes

Lifeboat’s current release line is 2.x. This page summarizes changes at a product level; each release also ships a versioned OpenAPI specification, an HTML API reference, and a Postman collection. Contact your Iterate.ai representative for the detailed per-build changelog for your deployment.

Lifeboat 2.2.61 — current

Two fixes for servers that could not start. Both shipped in earlier releases and were found upgrading a production deployment to 2.2.60, which should be skipped for the container images.

A server with Lifeboat’s optimizations switched off works again. From 2.2.58 through 2.2.60, a tensor-engine server whose optimizations were turned off loaded its model and then stopped on its first request. Servers with the optimizations on — the default — were not affected.

FP4 models start on workstation Blackwell GPUs. On an RTX PRO 6000 or a GB10, Lifeboat adds a setting so FP4 (NVFP4 or MXFP4) checkpoints start cleanly. Since 2.2.39 that setting used a name the inference engine no longer accepts, so every FP4 server on these GPUs failed before loading the model. Lifeboat now uses the current setting, and a test checks every engine setting Lifeboat adds against the engine itself.

Every channel is on 2.2.61. The container images, the desktop apps for macOS and Windows, and the pip package (pip install -U lifeboat) all carry this release. Both fixes above are for the tensor engine, which the pip package does not include, so a pip install behaves as it did on 2.2.60 — upgrading keeps it on the same version as the rest of a deployment, and brings the 2.2.60 defaults (Max concurrency routing, 64 requests per server) to installs that were still on 2.2.59 or earlier.

Lifeboat 2.2.60

A new server uses the whole GPU by default. Until now the default routing mode, Superfast, held every server to four requests at once and queued the rest. That keeps the first token fast for a few users, but on a large GPU it left most of the card idle under load. The default is now Max concurrency, and a new server accepts up to 64 requests at once (32 before). Measured on one 96 GB RTX PRO 6000 Blackwell with a 30B mixture-of-experts model, 256-token answers:

Concurrent requests	2.2.59 defaults	2.2.60 defaults
16	1,418 tok/s	4,168 tok/s
32	1,419 tok/s	8,182 tok/s
64	1,446 tok/s	7,708 tok/s

An install that never chose a routing mode moves to the new default on upgrade. Existing servers keep the concurrency limit they were created with. Superfast and Hybrid are still on the Configuration page for deployments where the first token for a handful of users matters more than total throughput.

Speculative decoding on hybrid models keeps the full context window. On models whose layers are mostly linear attention, Lifeboat already capped concurrent requests so speculative decoding would fit. It now reserves the cache one request needs at the server’s full context length first, and gives the rest to concurrent slots. A 27B model on a 96 GB card gets more slots and still serves a 262,144-token prompt.

Benchmarks you can trust. The console’s Benchmarks page now:

• authenticates its own requests, so it works on deployments that require an API key — before, every request was refused and the run still reported as complete;
• marks a run as failed when its requests fail, instead of reporting zeros;
• counts output tokens from the engine’s own report, which matters with speculative decoding, where one streamed update can carry several tokens;
• sends its sampling settings in every request, so two engines are compared doing the same work.

GGUF vision models keep their full context with speculative decoding. A new server option, Run the projector on the CPU, moves the image encoder off the GPU. On a 24 GB card that freed enough memory for a 27B model to serve its full 262,144-token window with speculative decoding on, at 59–63 tokens per second. Image input still works. Off by default, because it makes image processing slower.

Lifeboat 2.2.59

A burst of requests waits instead of being refused. By default each server takes four requests at once, for the fastest first token, and the fifth used to get an immediate 503. Reported from a production deployment: two agent bursts of about 50 and 26 requests produced 75 rejections against a healthy server. Requests over the limit now queue and run as soon as a slot frees, for up to 120 seconds. LIFEBOAT_LB_QUEUE_ENABLED=false restores the old behaviour. An .env file created from an older template may set LIFEBOAT_LB_QUEUE_ENABLED=false explicitly — remove that line to get the new default.

Speculative decoding starts on hybrid models instead of running out of memory. On models whose layers are mostly linear attention — the Qwen3.5 to 3.8 family among them — the tensor engine keeps recurrent state for every draft token of every concurrent request, and speculative decoding raises the request limit to 48. Reported from a 96 GB card with a 27B model:

	Memory
left for cache after the weights and the draft	13 GB (at the 0.75 memory setting tried)
recurrent state the engine wanted for 48 requests	30.5 GB

So every speculative launch failed, at every memory setting tried, while the same server started at once with speculation off. Lifeboat now sizes the concurrent-request limit to what fits — 10, 21 and 24 requests at the 0.75, 0.88 and 0.92 settings that failed — capped at the server’s own Max Concurrency. If not even one request fits, the server starts without speculation and the log says why, rather than failing. Nothing to configure.

Long context on the GGUF engine reaches the engine. A 262,144-token window set on a GGUF server came up at a few thousand tokens per request: a context flag in the advanced arguments switched Lifeboat’s own sizing off, and any flag Lifeboat manages made the whole argument string be ignored. Each flag is now handled on its own; the memory estimate counts only the layers that hold a cache and includes the per-request recurrent state of hybrid models; and a shared cache pool keeps several requests at the full window. Verified at 262,144 tokens per request on a 24 GB card, answering correctly from a 165,000-token prompt. When a window does not fit, the server log names the cache precision that would reach it, and the server form now offers that precision.

GGUF repositories behave. Calibration (imatrix) files are no longer offered as the default download or registered as models; the vision projector auto-pick prefers F16 and the form shows which file Auto resolves to; and an alias that is already taken names the model holding it and how to release it.

Lifeboat 2.2.58

A stopped server no longer holds memory back from a new one. When a server is left on automatic memory sizing, Lifeboat sets aside room on the GPU for the other servers configured against that card, so a model started later does not take everything and leave its neighbours unable to start.

That reservation was also being charged for servers you had deliberately stopped — which hold no memory at all. Reported from a 96 GB card: an 8B model was deployed and then stopped, and a 27B then refused to start with an out-of-memory error on a card that was in fact free. Deleting the stopped server, and losing its settings, was the only way around it.

Measured with those exact numbers, for the 27B landing on that GPU:

The 8B peer is…	Memory reserved	Cache the 27B gets
running	0 GB (already counted)	25.9 GB
stopped — before	38.2 GB	3.4 GB — cannot serve
stopped — now	0 GB	25.9 GB

Stopping is now equivalent to deleting, without losing the configuration. A server that is running still reserves, and one that is configured but has never run still reserves — it genuinely may start.

Stopping a server now tells you if it did not release the card. A server can be recorded as stopped while a process it launched still holds GPU memory, after a crash or when a restart lost track of it. The stop result now says so plainly, and the Servers page offers Force kill to reclaim it — previously the first sign was the next model failing to start, pointing at the wrong thing.

The log panel keeps the previous run. The log of a server that crashed used to be overwritten the moment it was restarted — exactly the moment anyone wants to read it. One previous run is preserved and is one click away, under Previous run.

Settings that would not take effect are flagged when you save them, rather than being silently ignored, and every server has a Verify config action showing side by side what you asked for and what the running server reports.

The Dashboard says when a pool is turning requests away and roughly what it would take to absorb them. It reports only — it never changes your deployment, and it deliberately does not suggest more servers for a problem more servers would not solve.

One client can no longer spend another’s share of the prompt-routing cache. The record of which server holds which prompt is bounded, and a single client sending many distinct prompts could displace everyone else’s entries — so their next request repeated work already done. Each client now has a fair share of it.

Prompt caching now spans three tiers instead of one. Processing a prompt is the expensive half of serving it, and that result — the key-value cache — was held only in GPU memory. GPU memory is small, so on agent, retrieval and chat traffic it is evicted within seconds and the same work is done again.

Lifeboat now keeps it in host memory, and optionally on disk behind that. Measured on a workload deliberately larger than the GPU could hold, with the GPU allocation pinned identical on both arms so the tier was the only variable:

	Prompt reuse	Response time, median
GPU memory only	0.14%	82.1 ms
With host memory behind it	99.73%	32.9 ms

On the GGUF engine the equivalent host-memory cache was capped at a laptop-sized default on hosts with hundreds of gigabytes free. Sizing it for the machine measured 40.7x faster prompt processing — median prompt time 250.1 ms to 6.1 ms on the same prompts.

Switched on per server under Server Settings → Performance. Size the tier above your working set or it does nothing: a cache smaller than the thing being cached thrashes and returns nothing.

Requests can be routed to the server that already holds the prompt. Session affinity keeps one client on one backend, which does not help the case that dominates agent and retrieval traffic — many different clients sharing one long prefix. The new KV-aware cluster algorithm decides from the prompt itself and measured 14.3x more cache reuse (3.37% → 48.30%) while traffic stayed evenly spread across backends.

Load always comes first. A busy backend is never chosen for cache reasons, so the worst case is ordinary least-connections balancing and the best case is that plus a prompt nobody had to re-process. Off by default; selected per cluster.

The disk cache tier writes far fewer files for the same data. The standard layout writes one file per cache page — about 12 KB each — so a terabyte of cached prompts becomes tens of millions of files. On a local disk that is unremarkable; on shared storage it is the dominant cost, because each one is a network round trip.

Lifeboat can now pack the same pages into a few large append-only files. Measured end to end on a real workload: 4,902x fewer files for the same bytes (within 0.2%), with cache reuse unchanged. Every record is checksummed and verified on every read, so a damaged page becomes a cache miss and is re-computed rather than served. Choose it when the cache directory is on a network share.

Also in this release. Three pip-package fixes specific to Windows: the startup banner no longer exits on a default console encoding, lifeboat doctor can now read host memory, and restarting the console after a model has run no longer fails. A stalled model download now names the Hugging Face transfer backend and the environment variable that avoids it. Eight bundled dependencies were raised to current security floors.

Lifeboat 2.2.57

NVIDIA GPUs in ordinary x86-64 servers now have a GPU path in the pip package. They ran on the CPU there before. Measured on an H100: the package installed, the control plane served, and every token was generated on the processor with 80 GB of accelerator memory idle.

The cause is the one that affected AMD Instinct, arriving from the other side. The portable engine reaches GPUs through Vulkan, and NVIDIA’s Vulkan driver ships with the graphics stack — a server or container carrying only the GPU’s compute driver does not have it. Installing a Vulkan loader does not help, because the part that is missing is the vendor’s own.

Lifeboat now publishes an engine built for NVIDIA accelerators and selects it automatically when it finds one, measured at 7.6x the single-request throughput on an RTX PRO 6000 Blackwell — 116 to 874 tokens per second on the same binary and the same model. One build covers T4, A100, A10, L4, L40S, H100, H200 and Blackwell cards, and parts released later are covered as they appear.

It is a larger download than the portable engine — about 725 MB against 31 MB — because it carries the accelerator maths libraries that a server with only a driver does not have. Bundling them is the point: without them the engine loads, finds no GPU, and reports exactly the problem this release fixes. It is fetched once, and only on machines that have an NVIDIA GPU.

Lifeboat 2.2.56

AMD Instinct accelerators now have a GPU path in the pip package. They ran on the CPU there before, and the reason was not a missing optimization: the portable engine reaches GPUs through Vulkan, and Vulkan’s open driver targets graphics cards. A data-center compute card has no graphics engine, so it was never enumerated — the engine reported no GPU at all and every model ran on the processor, on a machine with 64 GB of accelerator memory sitting idle.

Lifeboat now publishes an engine built against ROCm and selects it automatically when it finds an Instinct card, measured at 1.9x the single-request throughput on an MI210. One build covers MI210, MI250, MI300X, MI325X and MI350X. Radeon cards are unaffected — the portable engine already reaches those, so they keep it.

Selection is keyed on the exact chip rather than “an AMD GPU is present”, so a card the build does not contain keeps the engine that works for it today.

Lifeboat 2.2.55

The downloadable serving engine now runs on older Linux, which is most of it. Lifeboat previously used a general-purpose build needing Ubuntu 24.04 or newer, which left out long-term releases still in wide use — Ubuntu 22.04, Debian 12, the RHEL 9 family, and the systems most edge hardware ships with. On those, the engine installed and then could not start. Lifeboat now publishes its own builds against a much older system runtime, so one download works on Linux releases going back to 2018, on both 64-bit Intel/AMD and ARM.

Small and embedded hardware is a supported target rather than an accident. The engine carries every processor variant and picks at run time, so one download serves a Raspberry Pi 4 or 5, a mini-PC or thin client, and low-power embedded chips with no modern vector instructions — as well as a current server. Integrated graphics are used automatically when present.

NVIDIA Jetson boards get a CUDA engine automatically, rather than the general Vulkan one. Worth roughly 2x on an Orin Nano Super: 47 → 88 tok/s for a single request and 112 → 180 tok/s under concurrency. Nothing to configure.

Concurrent slots are now sized from the machine. They were fixed at four regardless of hardware, which left most of a large server idle. On a 64-core host this took aggregate throughput from 630 to 1330 tok/s. Small boards are unchanged — more slots cost cache memory, which is the binding constraint there.

lifeboat doctor reports what the machine can actually run — cores, memory, the accelerator it found and the largest model it can comfortably serve, before you download any weights. It also names a GPU that is present but unreachable by the installed engine, which otherwise looks like everything working while inference quietly runs on the CPU.

Lifeboat 2.2.54

Apple Silicon Macs can serve safetensors models natively. Installing with pip install 'lifeboat[mlx]' adds MLX, Apple’s own inference engine, which runs on the Mac’s GPU and is the only way to serve safetensors weights on a Mac. GGUF still works and is still the default; MLX is a second engine beside it.

Lifeboat 2.2.53

New Benchmarks page. Run a concurrency sweep against any running model and get throughput, time-to-first-token and end-to-end latency at each level, measured on your own hardware rather than quoted from someone else’s. Each run is saved, so a setting change can be compared against the run before it.

Every model now carries an estimated cost per million tokens. It is derived from what the model actually costs to serve — for a mixture-of-experts model that is the active parameter count rather than the total, which is why a 35B MoE is cheaper to run than its size suggests. The figure is seeded when a server is created and is fully editable: correct it for your own electricity or hardware cost and it stays corrected.

Prefill/decode disaggregation can be enabled from the control plane and the Helm chart. It splits a model across machines so prompt processing and token generation scale independently, which helps when one of the two is the bottleneck. It is off by default and changes nothing unless enabled; the KV-transfer backend is auto-detected from the hardware, and a machine serving only half of a split model is kept out of normal request routing.

Speculative decoding now works on the GGUF engine, and Auto picks the best method a model file supports. Previously the setting was offered but had no effect on that engine, and Auto chose the weaker method for every GGUF model. Existing servers pick both fixes up on their next start.

AMD Radeon graphics cards are supported for GGUF models, from the same image that serves the Instinct accelerators.

Fixed: models that reason before answering were reported as not supporting tool calling, so clients stopped offering them tools. Fixed: downloading a model into a folder Lifeboat cannot write to now fails immediately and names the folder, its owner and the command that fixes it.

Lifeboat 2.2.50

Maintenance and security release. The third-party components bundled in the product have been updated, several of them carrying security fixes — among them the library that handles encryption and certificate validation. Nothing to configure and no action required; the updates take effect as soon as you move to this version.

Windows now installs from a signed installer. Lifeboat-2.2.50-setup.exe replaces the previous .zip: it is signed, adds a Start-menu entry, can start Lifeboat when you sign in, registers an uninstaller, and checks for the .NET 8 Desktop Runtime it needs rather than installing an application that then does not appear.

Windows is a supported desktop platform, and the documentation now says so. The installation command issued with a licence is a shell script, which cannot run on Windows — pasting it into PowerShell fails with “The term ‘bash’ is not recognized”. Windows has a native app instead, and both the welcome email and the installer itself now point at it rather than leaving the command to fail. Two prerequisites that otherwise make the app look broken are called out: the free .NET 8 Desktop Runtime, and unblocking the downloaded archive before expanding it so Windows will load the app’s libraries.

A working NVIDIA GPU is no longer reported as broken. On the desktop app the Dashboard could show a red “NVIDIA access broken” banner while the GPU fleet card simultaneously showed the card, its memory and its temperature — both describing the same healthy GPU seconds apart. The health panel insisted on a diagnostic library the desktop app deliberately does not ship; it now reads the driver directly when that library is absent, and only reports a fault when the driver itself does not answer. The banner also no longer refers to a container on a machine that is not running one.

Lifeboat 2.2.49

A port conflict now means a listener, not any socket. Stopping a server and starting it again could fail with “Port 30001 is already in use by PID …” naming Lifeboat’s own process, for a port nothing was listening on. The check counted every socket referencing the port — including the spent outbound connection the control plane keeps after it stops proxying to an engine. It now counts listeners only. Container deployments were never affected.

Lifeboat 2.2.48

Inference on a Mac could fail with an out-of-memory error from the graphics system. A model would start, report itself healthy, appear in the model listing, and then fail every request. Lifeboat was reserving far more memory for the conversation cache than the machine can give the GPU — on a 64 GB Mac it asked for 56 GB against a 55 GB ceiling — and then restarted the server automatically, so the state repeated. It now sizes that cache against the memory actually available, reducing how many requests it handles at once rather than shortening the conversation window, and records which limit applied. Nothing to configure; existing servers pick it up on their next start.

The same arithmetic under-counted the cache cost of many current models by half, because it derived a figure the model file states outright. Any memory estimate on those models was optimistic by a factor of two.

MLX models can now be served on Apple Silicon. Lifeboat recommended the MLX build of a model and let you download it, and then had no way to run it — on a Mac that was the only format it offered for those models, so the recommendation ended in a dead end after a multi-gigabyte download. Choosing the model is now enough; the engine is selected for you.

The model listing reports a context window for MLX models instead of leaving it blank, so gateways and IDE clients can size prompts against it.

Capacity rejections are counted in the metrics endpoint. When every backend is busy and the queue is off, Lifeboat answers 503 — and that, the most common rejection, was the one case the counters missed, so monitoring showed nothing during exactly the capacity event it exists to surface.

Lifeboat 2.2.46

Three first-run defects on the desktop app, all reported from one Mac.

MLX models could not be downloaded. An MLX checkpoint is safetensors — mlx-community/Qwen3-8B-4bit holds model.safetensors and no GGUF — so the download gate that refuses unservable safetensors matched it exactly and disabled the button, while its own message advised looking for “an mlx-community build”. The browser copy of that gate had been stricter than the server’s, which had exempted MLX from the start.

An MLX download was allowed where MLX cannot run. The name-based exemption was unconditional, so pasting an mlx-community repo on an Intel Mac, on Windows, or in the Lite image began a download that could never load. Both gates now require MLX to actually be present. GGUF keeps no such condition — that engine ships in every shape.

A recommended model pointed at a repository that does not exist. Qwen/Qwen3-4B-Instruct-2507-GGUF returns “Repository Not Found”, which reads as a token problem because the dialog has a token selector under the error. It was a wrong repo id in our catalogue; it now points at a verified one. Every repository Lifeboat recommends is existence-checked before release, and checked against its own format tag — a repository that exists can still be the wrong one.

Lifeboat 2.2.45

Help on every console page. Each page in the console links the documentation page that answers its questions — the Servers page to the server lifecycle, the Configuration page to the configuration reference, and so on, rather than to a product overview.

Documentation rewritten and expanded, from 15 pages to 70: quickstarts for each deployment shape, a page per capability, five API pages, and troubleshooting organised by symptom rather than by subsystem — a server that will not start, slow responses, failing requests, downloads, tool calls, nodes and clusters, and the desktop app. Every classified failure the product can report now has an entry naming its cause and its fix.

Lifeboat 2.2.44

Native desktop app for macOS and Linux. No container runtime. Signed with an Apple Developer ID, notarized and stapled, so it opens with a double-click and works offline; Linux ships as a .deb and a distro-agnostic tarball for x64 and arm64. It bundles the GGUF engine and the same control plane, console and API as the container. Downloads and checksums: github.com/IterateAI/lifeboat-releases.

Lifeboat Lite — the no-accelerator image. Around 720 MB against 17 GB, running on any machine with no GPU at all, and using an integrated Intel or AMD GPU via Vulkan when one is present. Every CPU instruction-set variant is compiled in and selected at runtime, so one image covers old and new hardware rather than crashing on the old. The control plane, load balancer, routing modes, queueing, model registry, licensing and audit are all present; the GPU optimization layer is not, because it hooks the tensor engine.

AMD Instinct support. ROCm images per CDNA generation, validated on MI210 hardware. See the FAQ for what is and is not available per generation.

Hardware-aware model choices. The deployment reports what the host can actually run — accelerator, memory, CPU, and any container memory limit — and the model catalogue is filtered to formats that machine can serve, in preference order. A model the host could never load is refused before the download starts rather than after it, which on a 60 GB checkpoint is the difference between a sentence and an hour.

Free tier and fleet visibility. Up to 2 concurrent servers on a single node for non-commercial and evaluation use. Deployments report a short hardware description once per configuration, and never model names, prompts or token counts. See Licensing.

Console. A Help link on every page, pointing at the documentation for that page. Server status updates immediately on start rather than flickering back through the previous state, and an open log panel stays open.

Lifeboat 2.2 — highlights

Speculative decoding. Interactive, single-stream responses get a 2–4x speedup with identical output through draft-model speculation. For recognized models this is zero-configuration: Lifeboat fetches and registers the matching draft model automatically and pre-fills the setting when a server is created. It stacks with both the cache optimizations and low-precision weight formats.

Blackwell-native 4-bit weights. NVFP4 and MXFP4 checkpoints are auto-detected and selected, roughly halving per-token memory reads for a direct decode speedup on supported GPUs.

Upstream engine refresh. The underlying inference runtime is tracked and re-based on current upstream releases, so newly added model architectures land in Lifeboat without waiting for a Lifeboat-specific port.

Lifeboat 2.1 — highlights

First-class GGUF support. Register, preflight, download, and serve GGUF quantized models, with a multi-select variant picker showing real per-file sizes so a single quantization can be pulled from a multi-quant repository. Split archives and unsupported architectures are detected and reported clearly instead of failing at server start.

Cancellable downloads with real progress. Download progress reflects true byte and file totals, and a download can be cancelled mid-flight with the registry reset immediately so a corrected download can be queued right away.

Multi-GPU tensor parallelism. Serve across 1, 2, 4, or 8 GPUs with interconnect-topology awareness, composable with quantization.

Named clusters and one-call multi-node deploy. Carve fleet GPUs into business-aligned pools such as interactive, batch, or per-tenant, and deploy the same model to every node in a cluster in a single action.

Token-aware weighted load balancer. Per-model pools, per-server weights and capacity gating, an optional waiter queue, idle-pool eviction, per-request dispatch headers, and an actual-versus-configured traffic-share drift indicator in the dashboard.

Observability. A Prometheus metrics endpoint covering dispatch counts, rejections by reason, in-flight requests, queue depth, and queue wait times; a live load-balancer state snapshot; and per-server log access from the UI.

Encrypted credential vault. Hugging Face tokens stored encrypted at rest, selectable and pinnable per model, with an environment-variable fallback for simple single-tenant deployments.

Audited API-key reveal. An administrator can retrieve a stored inference key when required, and the reveal is recorded in the audit log.

GPU fleet operations. Fleet-wide GPU visibility with live utilization per node, plus optional disk deletion of model files restricted to approved model directories.

Capability probing. Probe a model to report what it actually supports — streaming, real maximum output length, effective context window, JSON mode, tool calling, vision, and reasoning traces — rather than relying on its model card.

Where to get each artifact

• Container images — hub.docker.com/r/iterateai/lifeboat
• Desktop app, installers, checksums, per-release notes — github.com/IterateAI/lifeboat-releases

Operational notes

Upgrades are an image swap that preserves the data volume, so the registry, users, API keys, audit log, and downloaded models carry across. On NVIDIA hosts the host-setup step is idempotent and can be safely re-run after a reboot or driver upgrade; AMD and Lite deployments have no host-setup step. Note that the container images and the desktop app are released on separate cadences, so their version numbers are not expected to match — a desktop version is not a pullable image tag. A bundled smoke-test harness exercises authentication and the critical read paths and is intended as a post-upgrade acceptance check.

Sources and references

• Product page: iterate.ai — Lifeboat
• Capabilities in depth: the Capabilities section in this documentation.