Deployment
Lifeboat runs on infrastructure the customer owns. The same artifacts cover a single workstation-class GPU host and a multi-node data-center fleet, and there is no dependency on an Iterate.ai-operated service at runtime.
What are the deployment options?
Section titled “What are the deployment options?”- Docker container (recommended) — a single hardened multi-architecture image, deployed with Compose or plain
docker run, that carries the control plane, the node agent, and the inference runtime. - Bare-metal installer — for hosts where containers are not used, with a command-line entry point for bring-up.
- Multi-node — the same image on every machine; one node runs the control plane and the others run node agents joined to it.
What are the hardware requirements?
Section titled “What are the hardware requirements?”Lifeboat requires NVIDIA GPUs (CUDA).
| Component | Minimum | Recommended |
|---|---|---|
| OS | Linux x86_64 (Ubuntu 22.04, RHEL 8+) | Ubuntu 22.04 LTS |
| CPU | 4 cores | 8+ cores |
| RAM | 16 GB | 32 GB+ (memory tiering benefits from more) |
| Disk | 50 GB free | 200 GB+ SSD (models are large) |
| GPU | One NVIDIA GPU, compute capability 7.0 or newer | L40S, A100, H100, or RTX PRO 6000 Blackwell |
Lifeboat is optimized for air-cooled data-center GPUs, and its published benchmarks were measured on RTX PRO 6000 Blackwell hardware.
AMD and Intel accelerators are not supported by the optimization layer, which is CUDA-only. Teams on those platforms can run the baseline upstream engine directly, but without Lifeboat’s concurrency gains.
What software prerequisites are needed?
Section titled “What software prerequisites are needed?”A current container stack: Docker Engine 25.0+, Docker Compose 2.21+ (only if using Compose), the NVIDIA Container Toolkit 1.17+, and an NVIDIA driver of at least the 535 series — 575 or newer for Blackwell GPUs. A one-time, idempotent host-setup step configures GPU device passthrough and installs a refresh service so reboots and driver upgrades do not break GPU access. Verify the host with a quick check of Docker, Compose, container-toolkit, and driver versions before deploying.
What does a quick start look like?
Section titled “What does a quick start look like?”- Fetch the Compose file and the example environment file.
- Set at minimum the admin password and the models directory. Change the default administrative credentials before first production use.
- Bring the stack up — the host-setup step runs once automatically and then exits.
- Open the control plane in a browser, register a model, and create a server.
Upgrades are an image swap: stop the stack, pull the new image, and start again, keeping the data volume so the registry, users, keys, audit log, and downloaded models are preserved.
Can Lifeboat run air-gapped?
Section titled “Can Lifeboat run air-gapped?”Yes, and this is a primary use case. Container images and model weights are staged once from an internal registry and an internal model mirror, after which the deployment has no outbound dependency — the reason Lifeboat is deployed in defense, government, and regulated-industry environments where no traffic may leave the site. Registering models from local paths, rather than from Hugging Face, is the standard air-gapped workflow.
Which ports and volumes matter?
Section titled “Which ports and volumes matter?”The control plane serves the UI, the management API, and the inference proxy on a single configurable port; node agents listen on their own port for control-plane commands; and each model process serves on its own port behind the proxy. Persist two paths: the data directory (registry, audit log, logs, download cache) and the models directory. Mounting the log path to a host volume gives longer log retention than the container’s own lifetime.
How is it fronted in production?
Section titled “How is it fronted in production?”Put the organization’s own reverse proxy or ingress in front of the control plane for TLS termination, hostname routing, and network access control; Lifeboat can also terminate TLS natively when supplied with a certificate and key. Point Prometheus at the metrics endpoint, and ship logs to the existing log platform. See Security and compliance.
Sources and references
Section titled “Sources and references”- Configuration reference: Configuration and admin.
- Deployment-time security posture: Security and compliance.