Skip to content
Docs

Choose Your Deployment

Lifeboat ships in five shapes. They serve the same console and the same API, so the choice is about hardware and operations, not features.

Decide in one table

Section titled “Decide in one table”
You have Use Why
An NVIDIA GPU server iterateai/lifeboat:latest The full optimization layer and speculative decoding.
An AMD Instinct server :amd / :amd-mi300x / :amd-mi355x One image per CDNA generation — GPU vendors.
Kubernetes The Helm chart Same images; gpu.vendor switches everything at once.
No GPU, an integrated GPU, or Arm :lite (~720 MB) Serves quantized models on CPU or via Vulkan.
A laptop or workstation, no Docker The desktop app Signed, notarized, tray icon, nothing to configure.
Any machine with Python, trying it out pip install lifeboat One command, no container. Quickest path to a served model.

The one mistake worth avoiding

Section titled “The one mistake worth avoiding”

:latest is the NVIDIA image. A container image can only select on CPU architecture, not GPU vendor, so one tag cannot serve NVIDIA, AMD and CPU hosts. Pulling :latest onto an AMD or CPU-only machine downloads ~17 GB that cannot run there.

The one-line installer detects the host and picks the right tag, including choosing Lite when there is no accelerator. If you pull by hand, pick deliberately.

What you give up with Lite, the desktop app and pip

Section titled “What you give up with Lite, the desktop app and pip”

All three serve models through the GGUF engine, which does not carry the GPU optimization layer — that hooks the tensor engine. You keep the control plane, the load balancer and all four routing modes, the capacity gate and queue, sticky sessions, the full API surface, the model registry, licensing and audit. You lose the ~2x concurrency gain and speculative decoding.

That is the honest trade. On a machine with no GPU there was never a tensor engine to hook, so nothing is being withheld — it is a statement about the hardware.

Sizing before you start

Section titled “Sizing before you start”

Decode speed is bound by memory bandwidth, not core count, so the model is what matters. On an 8 GB machine a 1.5B–4B model at 4-bit is comfortable, 8B is the ceiling, and 14B will not fit. Ask the deployment itself rather than guessing — it reports what the host can run, including any container memory limit. See Hardware requirements.

Next

Section titled “Next”

Sources and references

Section titled “Sources and references”