Skip to content
Docs

Hardware Requirements

Split by deployment shape, because they differ enormously.

GPU server

Section titled “GPU server”
Component Minimum Recommended
OS Linux x86_64 or arm64 (Ubuntu 22.04+, RHEL 9+) Ubuntu 24.04 LTS
CPU 4 cores 8+
RAM 16 GB 32 GB+ (memory tiering benefits from more)
Disk 50 GB free 200 GB+ SSD — models are large
NVIDIA GPU compute capability 7.0+, driver 580.65.06+ L40S, A100, H100, RTX PRO 6000 Blackwell
AMD GPU Instinct MI210 or newer, ROCm 6.3+ MI300X or newer
Software Docker Engine 25.0+, Compose 2.21+, NVIDIA Container Toolkit 1.17+ (NVIDIA only)

The driver floor is exact, not approximate

Section titled “The driver floor is exact, not approximate”

The image ships the CUDA 13 runtime, whose documented minimum driver is 580.65.06. An older driver passes every other preflight and then fails when a model loads, with an error that names neither the driver nor the version. The installer checks it and refuses, which is the better of the two failures.

No accelerator

Section titled “No accelerator”

The Lite image and the desktop app need no GPU, no driver and no container toolkit. Docker 25.0+ and about 3 GB of disk, or just the installer for desktop.

8 GB of RAM is a usable machine. An integrated Intel or AMD GPU is used via Vulkan when present, and passed through with LIFEBOAT_GPU_DEVICE.

Desktop

Section titled “Desktop”

macOS 13+ (Apple Silicon or Intel), or Linux with glibc 2.31+ (Debian 11+, Ubuntu 20.04+, RHEL 9+), python3 >= 3.10 and GTK. 8 GB RAM minimum.

Sizing a model to a machine

Section titled “Sizing a model to a machine”

Decode speed is bound by memory bandwidth, not core count. The ceiling is roughly the machine’s bandwidth divided by the bytes read per token, so the model’s size in memory is the number that matters.

On an 8 GB machine: 1.5B–4B at 4-bit is comfortable, 8B is the ceiling, 14B will not fit — roughly 8.1 GB of weights before the operating system takes its share.

Expect time-to-first-token, not generation speed, to dominate on a low-core host: prefill is the compute-bound half, and a smaller model improves it disproportionately.

Rather than estimating, ask the deployment — it reports what the host can run, including any container memory limit, which is the figure people most often miss.

Shared memory

Section titled “Shared memory”

Tensor parallelism above 1 needs /dev/shm sized for it (LIFEBOAT_SHM_SIZE, default 8 GB). It is fixed at container creation. See Tensor parallelism.

Sources and references

Section titled “Sources and references”