Skip to content
Docs

GPU Vendors & Accelerators

One product, several accelerators, and the differences are hardware facts rather than porting gaps.

NVIDIA

Section titled “NVIDIA”

iterateai/lifeboat:latest. The full optimization layer, speculative decoding, and confidential computing on supported parts.

Driver floor 580.65.06 — the image ships the CUDA 13 runtime.

AMD Instinct

Section titled “AMD Instinct”

One image per CDNA generation, and this is not a packaging preference: the tensor engine’s kernels are compiled for one generation at a time, because the low-precision number formats differ between generations. Running the wrong one fails inside a GPU kernel rather than at startup, so the container checks the GPU it finds against the generation it was built for and says so.

Cards Tag
MI210 / MI250 (CDNA2) :latest-rocm or :amd
MI300X / MI325X (CDNA3) :amd-mi300x
MI350X / MI355X (CDNA4) :amd-mi355x

What is disabled on CDNA2, and why: the FP8 cache path needs FP8 tensor hardware, which starts at CDNA3; and GPU partitioning has no ROCm equivalent. The launch profile disables exactly those two, logs the reason, and leaves the rest of the optimization layer running. From CDNA3 the full set applies.

Two caveats before an AMD purchase:

  • On CDNA2 the 8-bit path is emulated in software and is very slow — serve a large model as a quantized GGUF there. From CDNA3 the 8-bit path is native.
  • Confidential computing is unavailable on AMD GPUs, full stop. An EPYC SEV-SNP host satisfies only the CPU half. Lifeboat detects this and stands the feature down rather than refusing every server start on a host that could never attest.

Validated on MI210 hardware. On one card, a 30B mixture-of-experts model serves at roughly 480 tokens/second aggregate.

Integrated Intel and AMD GPUs

Section titled “Integrated Intel and AMD GPUs”

Supported through :lite, via Vulkan, falling back to the CPU. Pass the render device through (LIFEBOAT_GPU_DEVICE=/dev/dri:/dev/dri) — without it the container cannot open the GPU even though the host has one.

The optimization layer does not apply; the control plane, router and API are identical.

Apple Silicon

Section titled “Apple Silicon”

Through the desktop app, using Metal and MLX. MLX is the fast path on a Mac and is what the recommended catalogue offers first. The tensor engine does not run on macOS, so safetensors checkpoints are not servable there — which is why a download of one is refused before it starts.

Arm servers

Section titled “Arm servers”

:lite is multi-architecture and runs on Arm, including Graviton. The NVIDIA image is also published for arm64 for Grace-Hopper class hardware.

Not supported

Section titled “Not supported”
  • Intel Gaudi and Intel Arc dedicated accelerators.
  • NPUs in “AI PC” class machines. They are detected and reported as present but unusable, with the reason: there is no supported tensor path for them, and an NPU shares system memory anyway, so it would not raise the decode ceiling.

Reporting them rather than ignoring them is deliberate — an operator who sees an idle NPU will ask, and silence reads as a bug.

Sources and references

Section titled “Sources and references”