Skip to content
Docs

pip Quickstart

The fastest way to try Lifeboat on a laptop or an ordinary server. No container, no GPU, no configuration.

Terminal window
pip install lifeboat
lifeboat engine install
lifeboat up

The console is then on http://127.0.0.1:8001. Create the first administrator when it asks, and the 24-hour grace period starts — you can serve a model immediately without a licence.

Prerequisites

Section titled “Prerequisites”
  • Python 3.10 – 3.13. Nothing else. No Docker, no accelerator, no driver.
  • About 3 GB of disk for the engine and a small model.

Some distributions still ship an older interpreter as python3 — Rocky, RHEL, AlmaLinux and Oracle Linux 9 ship 3.9, which is below the floor. pip install lifeboat there reports “No matching distribution found”, which reads as “the package does not exist” rather than “your Python is too old”. Install a supported interpreter (dnf install python3.12) and use that.

Serve one model without the console

Section titled “Serve one model without the console”
Terminal window
lifeboat serve Qwen/Qwen2.5-0.5B-Instruct-GGUF

That downloads a single quantisation — not the whole repository — and serves it on http://127.0.0.1:8000 behind the OpenAI-compatible API. --quant picks a different one, --ctx-size sets the per-request context, and anything after the model is passed through to the engine.

What this machine can actually run

Section titled “What this machine can actually run”
Terminal window
lifeboat doctor

It reports the platform, whether the engine is installed and usable, and whether a tensor engine is present — so a machine that cannot run something is told so before a multi-gigabyte download rather than after.

What the pip package does NOT include

Section titled “What the pip package does NOT include”

It carries the GGUF engine only. You get the console, the model registry and variant picker, the load balancer and all four routing modes, the capacity gate and queue, sticky sessions, the full OpenAI and Anthropic API surface, API keys, licensing and audit.

You do not get the tensor engine, and therefore none of the GPU optimization layer that hooks it — no TurboQuant, fair scheduling, admission control or FP8 KV cache — and no safetensors weights. Those are in the container image. On a machine with no GPU there was never a tensor engine to hook, so nothing is being withheld; it is a statement about the hardware. See Choose your deployment.

Where things are stored

Section titled “Where things are stored”
Variable What it holds
LIFEBOAT_DATA_DIR Registry, audit log, licence, the engine. Persist it — the grace clock and cluster identity live here.
LIFEBOAT_MODELS_DIR Downloaded weights.

Both default to a per-user application directory. If you set either, point it at a path you intend to keep.

Platform support

Section titled “Platform support”

Published for Python 3.10, 3.11, 3.12 and 3.13 on macOS (Apple Silicon), Linux x86-64, Linux ARM64 and Windows x64.

Two hosts need the container image instead:

  • Intel Macs. No wheel is published, so pip reports no matching distribution rather than installing something that cannot run.
  • RHEL 9 and its rebuilds (Rocky, AlmaLinux, Oracle Linux). The prebuilt engine needs a newer C++ runtime than those distributions ship, and there is no upgrade available from their repositories. lifeboat doctor names this specifically. Use the container image, or build the engine on the host and point LIFEBOAT_LLAMACPP_BIN at it.

Verify

Section titled “Verify”
Terminal window
curl -s localhost:8001/api/version
curl -s localhost:8001/metrics | head

Next

Section titled “Next”

Sources and references

Section titled “Sources and references”