Skip to content
Docs

Deployment

This page is the summary. Each section below has a page of its own:

Lifeboat runs on infrastructure you own. The same artifacts cover a laptop, a single workstation-class GPU host, and a multi-node data-centre fleet, and there is no dependency on an Iterate.ai-operated service at runtime.

What are the deployment options?

Section titled “What are the deployment options?”
  • Docker container (recommended for servers) — a hardened image carrying the control plane, the node agent and the inference runtime. Deployed with Compose or plain docker run. Separate images target NVIDIA, AMD and no-accelerator hosts.
  • Helm chart on Kubernetes — the same image, with GPU vendor selected by a single value.
  • Desktop app — a signed native application for macOS and Linux, with a tray icon and no container runtime at all. The console, API and model registry are the same as the container’s, so nothing is learned twice.
  • pip package — pip install lifeboat on macOS, Linux or Windows with Python 3.10+. The console, the API and the model registry again unchanged; it carries the GGUF engine only, so the GPU optimization layer stays with the container. See pip quickstart.
  • Multi-node — the same image on every machine; one node runs the control plane and the others run node agents joined to it.

Which image do I need?

Section titled “Which image do I need?”

A container image can only select on CPU architecture, not GPU vendor, so one tag cannot serve every host. Pick the tag that names your hardware.

Target Tag Size
NVIDIA iterateai/lifeboat:latest ~17 GB
AMD Instinct MI210 / MI250 (CDNA2) iterateai/lifeboat:latest-rocm or :amd ~28 GB
AMD Instinct MI300X / MI325X (CDNA3) iterateai/lifeboat:amd-mi300x ~28 GB
AMD Instinct MI350X / MI355X (CDNA4) iterateai/lifeboat:amd-mi355x ~28 GB
No GPU, integrated GPU, or Arm (Graviton) iterateai/lifeboat:lite ~720 MB

The AMD images are tagged per CDNA generation on purpose: the tensor engine’s kernels are compiled for one generation at a time, because the 4-bit and 8-bit number formats differ between CDNA3 and CDNA4. Running the wrong one fails inside a GPU kernel rather than at startup, so the container checks the GPU it finds against the generation it was built for and says so.

Version tags such as :2.2.44 are immutable; :latest, :latest-rocm and :lite roll forward. All tags are on Docker Hub at hub.docker.com/r/iterateai/lifeboat.

What are the hardware requirements?

Section titled “What are the hardware requirements?”

GPU host (NVIDIA or AMD)

Section titled “GPU host (NVIDIA or AMD)”
Component Minimum Recommended
OS Linux x86_64 or arm64 (Ubuntu 22.04+, RHEL 9+) Ubuntu 24.04 LTS
CPU 4 cores 8+ cores
RAM 16 GB 32 GB+ (memory tiering benefits from more)
Disk 50 GB free 200 GB+ SSD (models are large)
NVIDIA GPU compute capability 7.0+, driver 580.65.06 or newer L40S, A100, H100, or RTX PRO 6000 Blackwell
AMD GPU Instinct MI210 or newer, ROCm 6.3+ MI300X or newer

The NVIDIA driver floor is not a rounding figure: the image ships the CUDA 13 runtime, whose documented minimum driver is 580.65.06. An older driver passes every other check and then fails when a model loads.

No accelerator

Section titled “No accelerator”

The Lite image and the desktop app need no GPU, no driver and no container toolkit. 8 GB of RAM is a usable machine.

Decode speed is limited by memory bandwidth rather than core count, so size by model rather than by CPU: on an 8 GB machine a 1.5B–4B model at 4-bit is comfortable, 8B is the ceiling, and 14B will not fit. Time-to-first-token, not generation speed, is what a low-core host notices. Ask the deployment itself — the control plane reports what the host can actually run, including any container memory limit.

Are AMD and Intel supported?

Section titled “Are AMD and Intel supported?”

AMD Instinct: yes, and validated on real hardware. Most of the optimization layer is hardware-independent; two items depend on silicon that CDNA2 does not have (the FP8 KV cache needs CDNA3 or newer, and GPU partitioning has no ROCm equivalent), and the launch profile disables exactly those and says why in the log rather than failing. On an MI210, one card serves a 30B mixture-of-experts model at roughly 480 tokens/second aggregate.

Integrated Intel and AMD GPUs: yes, through the Lite image, which uses them via Vulkan when one is present and the CPU otherwise.

Intel Gaudi and Intel Arc dedicated accelerators: no.

Two caveats worth stating before an AMD purchase. On CDNA2 the 4-bit and 8-bit paths available are narrower than on NVIDIA, so a large model is best served as a quantized GGUF; from CDNA3 the full set applies. And confidential computing is not available on AMD GPUs at all — the control plane detects this and stands the feature down rather than refusing to start servers on a host that could never attest.

What software prerequisites are needed?

Section titled “What software prerequisites are needed?”

NVIDIA: Docker Engine 25.0+, Docker Compose 2.21+ (Compose only), NVIDIA Container Toolkit 1.17+, and driver 580.65.06+. A one-time, idempotent host-setup step configures GPU device passthrough and installs a refresh service so reboots and driver upgrades do not break GPU access.

AMD: Docker Engine 25.0+ and a ROCm-capable kernel driver. There is no container toolkit and no host-setup step; the container is given the ROCm device nodes directly.

Lite: Docker Engine 25.0+ and about 3 GB of disk. Nothing else.

What does a quick start look like?

Section titled “What does a quick start look like?”
Terminal window
curl -sSL https://raw.githubusercontent.com/IterateAI/lifeboat-releases/main/install/get-lifeboat.sh | bash

The installer checks the host, detects the accelerator, selects the matching image, writes an environment file and brings the stack up. It selects Lite automatically when there is no GPU, rather than pulling a CUDA image onto a machine that cannot use it.

By hand: fetch the Compose file and the example environment file, set at minimum the admin password and the models directory, and bring the stack up. Change the default administrative credentials before first production use. Then open the control plane in a browser, register a model and create a server.

On Kubernetes:

Terminal window
helm repo add lifeboat https://raw.githubusercontent.com/IterateAI/lifeboat-releases/main/helm
helm install lifeboat lifeboat/lifeboat --set gpu.vendor=nvidia --set admin.password=<password>

gpu.vendor takes nvidia, amd or lite, and switches the image, the GPU resource and the runtime wiring together. admin.password is required — the chart refuses to render without it, rather than deploying something with a default credential.

Upgrades are an image swap: stop the stack, pull the new image, start again, keeping the data volume so the registry, users, keys, audit log and downloaded models are preserved.

Can Lifeboat run air-gapped?

Section titled “Can Lifeboat run air-gapped?”

Yes, and this is a primary use case. Container images and model weights are staged once from an internal registry and an internal model mirror, after which the deployment has no outbound dependency — the reason Lifeboat is deployed in defence, government and regulated-industry environments where no traffic may leave the site. Registering models from local paths, rather than from a public model hub, is the standard air-gapped workflow, and an air-gapped licence contacts nothing at all. See Licensing for how to obtain one.

Which ports and volumes matter?

Section titled “Which ports and volumes matter?”

The control plane serves the UI, the management API and the inference proxy on a single configurable port; node agents listen on their own port for control-plane commands; and each model process serves on its own port behind the proxy. The desktop app uses port 30800.

Persist two paths: the data directory (registry, audit log, logs, download cache) and the models directory. Mounting the log path to a host volume gives longer log retention than the container’s own lifetime.

How is it fronted in production?

Section titled “How is it fronted in production?”

Put your own reverse proxy or ingress in front of the control plane for TLS termination, hostname routing and network access control; Lifeboat can also terminate TLS natively when supplied with a certificate and key. Point Prometheus at the metrics endpoint and ship logs to your existing log platform. See Security and compliance.

Sources and references

Section titled “Sources and references”