Supported Devices
One question first: will it run on my machine? This page answers that in a table and says which download to take. The detail behind each row lives in GPU Vendors & Accelerators and Hardware Requirements.
Two engines matter throughout:
- the tensor engine — the fast path, and the only one with the throughput optimizations, FP8 and speculative decoding. Linux, with CUDA or ROCm.
- the GGUF engine — quantized models on anything, including CPU-only. No optimization layer, and it is what every non-Linux and every accelerator-free machine uses.
The table
Section titled “The table”| Machine | Get | Engines | Accelerated by | Status |
|---|---|---|---|---|
| NVIDIA GPU server (Linux) | iterateai/lifeboat:latest |
tensor + GGUF | CUDA | Verified |
| AMD Instinct MI210 / MI250 | iterateai/lifeboat:amd |
tensor + GGUF | ROCm (gfx90a) | Verified on MI210 |
| AMD Instinct MI300X / MI325X | iterateai/lifeboat:amd-mi300x |
tensor + GGUF | ROCm (gfx942) | Built, not run on hardware |
| AMD Instinct MI350X / MI355X | iterateai/lifeboat:amd-mi355x |
tensor + GGUF | ROCm (gfx950) | Built, not run on hardware |
| AMD Radeon (RDNA) | iterateai/lifeboat:lite |
GGUF | Vulkan | Compiles; not run on hardware |
| Intel Arc / Xe / integrated | iterateai/lifeboat:lite |
GGUF | Vulkan | Built; not run on hardware |
| CPU-only server or mini-PC | iterateai/lifeboat:lite or pip install lifeboat |
GGUF | CPU | Verified |
| Apple Silicon Mac | Desktop app or pip install lifeboat |
GGUF + MLX | Metal | Verified |
| Intel Mac | Desktop app | GGUF | Metal ships; engagement depends on the GPU | Partly verified |
| Windows | Desktop app | GGUF | Vulkan | Verified |
| Arm64 Linux server | pip install lifeboat |
GGUF | CPU | Verified |
| NVIDIA Jetson (JetPack 6) | pip install lifeboat |
GGUF | CUDA, selected automatically | Verified on Orin |
| Raspberry Pi 4 / 5 (64-bit OS) | pip install lifeboat |
GGUF | CPU | Code path verified on Arm; not on Pi hardware |
| Android phone (arm64) | Not downloadable yet | GGUF | CPU, and Adreno or Mali where it helps | Verified on hardware |
“Verified” means a model was served on that hardware and measured, not that it compiles.
What is explicitly not supported
Section titled “What is explicitly not supported”- 32-bit Arm (
armhf, the 32-bit Raspberry Pi OS). No wheel and no engine build. - Microcontrollers — Arduino, ESP32 and similar. This is not a porting gap: an ESP32 has around half a megabyte of RAM against a several-hundred-megabyte model, before you reach the absence of an operating system.
- Neural processing units — the NPU in an “AI PC” (Intel VPU, AMD XDNA) and the one in a phone chipset. They are detected and reported as unusable with the reason. An NPU also shares system memory, so it would not lift the ceiling that actually binds.
Choosing by what you have
Section titled “Choosing by what you have”A GPU server you own. Take the image for your vendor. NVIDIA needs driver 580.65.06 or newer — the image ships the CUDA 13 runtime, and an older driver fails inside the container rather than at install.
A laptop, mini-PC or anything without a datacenter GPU. pip install lifeboat, or the desktop app if you want a tray icon and no terminal. Both give you the console, the model registry, the load balancer and the full OpenAI-compatible API.
An edge board. The Linux engines link an old system runtime deliberately — glibc 2.27, so distributions back to 2018 — and carry every processor variant, chosen at run time. One download runs from an Atom to a current server. Jetson boards get a CUDA build automatically.
Not sure. Install and ask it: lifeboat doctor reports the cores, memory and accelerator it found, which engines it can use, and the largest model this machine can serve — before you download any weights.
Sizing, briefly
Section titled “Sizing, briefly”Decode speed is bound by memory bandwidth, so the model size is the decision that matters most. On an 8 GB machine, 1.5B–4B at 4-bit is comfortable and 8B is the ceiling. A 2 GB Raspberry Pi is a 0.5B–1.5B machine. On a low-core board expect time-to-first-token, not generation speed, to dominate.