pip Quickstart

The fastest way to try Lifeboat on a laptop or an ordinary server. No container, no GPU, no configuration.

pip install "lifeboat[hub]"
lifeboat engine install
lifeboat up

The console is then on http://127.0.0.1:8001. Create the first administrator when it asks, and the 24-hour grace period starts — you can serve a model immediately without a licence.

[hub] pulls huggingface_hub, which is what downloads a model by repository id — both from the console and from lifeboat serve. Plain pip install lifeboat installs and runs fine, but the first thing most people do after it is fetch a model, and without the extra that step stops with “is not a local path and huggingface_hub is not installed”. lifeboat doctor reports the gap before you hit it.

Prerequisites

Python 3.10 – 3.13. Nothing else. No Docker, no accelerator, no driver.
About 3 GB of disk for the engine and a small model.

On Debian, Ubuntu and their derivatives — including NVIDIA JetPack — a virtual environment needs a package the base image does not install:

sudo apt install python3-venv     # or python3.10-venv, matching your interpreter

Without it python3 -m venv fails with “ensurepip is not available” and names the package to install, which is the one good thing about that failure. Measured on a Jetson Orin running JetPack 6.

Some distributions still ship an older interpreter as python3 — Rocky, RHEL, AlmaLinux and Oracle Linux 9 ship 3.9, which is below the floor. pip install lifeboat there reports “No matching distribution found”, which reads as “the package does not exist” rather than “your Python is too old”. Install a supported interpreter (dnf install python3.12) and use that.

Windows

Making `lifeboat` runnable

Installed into a virtual environment, lifeboat is not on PATH, so the commands above fail with “The term ‘lifeboat’ is not recognized”. The usual answer — activating the environment — does not work on a default Windows install either:

C:\lifeboat\Scripts\Activate.ps1
# Activate.ps1 cannot be loaded because running scripts is disabled
# on this system.   (Get-ExecutionPolicy -> Restricted)

Two things do work, because neither is a script:

$env:PATH = "C:\lifeboat\Scripts;" + $env:PATH   # then: lifeboat doctor
C:\lifeboat\Scripts\lifeboat.exe doctor          # or call it by full path

Both last for that window only. To use the normal workflow instead, allow local scripts once — Set-ExecutionPolicy -Scope CurrentUser RemoteSigned — after which Activate.ps1 runs. That is a persistent change to your user profile, so it is worth making deliberately rather than as part of an install.

Required environment variables

Two are required today. Set them in the same PowerShell session you start Lifeboat from, before lifeboat up:

$env:PYTHONIOENCODING = "utf-8"
$env:HF_HUB_DISABLE_XET = "1"

PYTHONIOENCODING — without it lifeboat up exits with UnicodeEncodeError: 'charmap' codec can't encode characters. A Windows console encodes output as cp1252, which cannot represent the box-drawing characters in the startup banner, and the banner is printed before the server binds. It prints the console URL first and then raises, so the symptom is an address that never answers rather than an obvious crash.

HF_HUB_DISABLE_XET — without it a model download can stall at zero bytes and end in “CAS Client Error” naming a CDN URL. Xet is Hugging Face’s newer transfer backend; it arrives as hf-xet inside the [hub] extra and is used automatically when present. Setting this falls back to the classic HTTP path. Measured on Windows 11: two attempts stalled with .incomplete at 0 bytes while a plain curl of the same file ran normally; with Xet disabled the same 491 MB download completed at about 2 MB/s.

Two more things worth knowing on Windows:

Use 127.0.0.1, not localhost. The engine binds IPv4 only, and Windows resolves localhost to ::1 first, so every request pays a fixed fallback of roughly two seconds. Measured on the same host: 2.08 s against 0.002 s.
Do not restart lifeboat up while a server is running. Startup checks whether each known server is still alive using a POSIX call that is invalid on Windows, and exits with OSError: [WinError 87]. See Troubleshooting: server start. Your model keeps serving on its own port; it is the console that stops.

Neither the desktop installer nor the container image is affected — this section is specific to the pip package.

Serve one model without the console

lifeboat serve Qwen/Qwen2.5-0.5B-Instruct-GGUF

That downloads a single quantisation — not the whole repository — and serves it on http://127.0.0.1:8000 behind the OpenAI-compatible API. --quant picks a different one, --ctx-size sets the per-request context, and anything after the model is passed through to the engine.

Apple Silicon: add MLX

MLX is Apple’s native engine and the only way to serve safetensors weights on a Mac. It is not installed by default because it adds roughly 320 MB:

pip install 'lifeboat[mlx]'

lifeboat doctor tells you which state you are in. Create the server with engine MLX (or "backend": "mlx" through the API) and point it at an mlx-community model. GGUF still works and remains the default; MLX is a second engine, not a replacement.

The extra is a no-op on anything that is not an Apple Silicon Mac — mlx-lm publishes no wheel for other platforms, so the marker simply installs nothing rather than failing.

What this machine can actually run

lifeboat doctor

It reports the platform, whether the engine is installed and usable, and whether a tensor engine is present — so a machine that cannot run something is told so before a multi-gigabyte download rather than after.

What the pip package does NOT include

It carries the GGUF engine (plus MLX on Apple Silicon via the mlx extra). You get the console, the model registry and variant picker, the load balancer and all four routing modes, the capacity gate and queue, sticky sessions, the full OpenAI and Anthropic API surface, API keys, licensing and audit.

You do not get the tensor engine, and therefore none of the GPU optimization layer that hooks it — no TurboQuant, fair scheduling, admission control or FP8 KV cache — and no safetensors weights. Those are in the container image. On a machine with no GPU there was never a tensor engine to hook, so nothing is being withheld; it is a statement about the hardware. See Choose your deployment.

Where things are stored

Variable	What it holds
`LIFEBOAT_DATA_DIR`	Registry, audit log, licence, the engine. Persist it — the grace clock and cluster identity live here.
`LIFEBOAT_MODELS_DIR`	Downloaded weights.

Both default to a per-user application directory. If you set either, point it at a path you intend to keep.

Platform support

Published for Python 3.10, 3.11, 3.12 and 3.13 on macOS (Apple Silicon), Linux x86-64, Linux ARM64 and Windows x64.

Two situations need the container image instead:

Intel Macs. No wheel is published, so pip reports no matching distribution rather than installing something that cannot run.
Linux older than Ubuntu 24.04. Upstream builds every prebuilt Linux engine on the current Ubuntu, so it needs glibc 2.38 and GCC 13 (GLIBCXX_3.4.32). That excludes most long-term releases in service today: Ubuntu 22.04 – and NVIDIA JetPack 6, which is built on it – Debian 12, and RHEL 9 with its rebuilds Rocky, AlmaLinux and Oracle Linux. None of them offers a newer runtime from its own repositories. lifeboat doctor names the missing symbols. Use the container image, which carries its own runtime, or build the engine on the host and point LIFEBOAT_LLAMACPP_BIN at it.

Verify

curl -s localhost:8001/api/version
curl -s localhost:8001/metrics | head

Sources and references

The trade against the container: Choose your deployment
Sizing a model to the machine: Hardware requirements