Model Management

Lifeboat keeps a registry of every model the deployment knows about, with the full lifecycle — register, download, inspect, serve, retire — handled from the browser rather than by hand on the filesystem.

Which model architectures are supported?

168+ architectures, covering the major open-weight families:

Family	Coverage
LLaMA	1, 2, 3, 3.1, 3.2, 4; Code Llama; multimodal/vision variants
Qwen	1 through 3.5, including VL (vision), audio, MoE, and omni variants
DeepSeek	V2, V3, VL2, and OCR variants
GLM	ChatGLM, GLM-4, GLM-4V, GLM-4-MoE, plus OCR and ASR variants
Gemma	1, 2, 3, and 3n
Mistral	Mistral, Mistral Large, Mixtral (MoE)
Phi	Phi-3, Phi-4 multimodal, PhiMoE
Multimodal & others	Command-R, Falcon, InternVL, LLaVA, Pixtral, DBRX, Baichuan, Yi, Granite, OLMo, Cohere, MiniCPM, and many more

Architectures not explicitly implemented fall back to a general-purpose loader, so the long tail still serves — with standard rather than optimized kernels.

How do you add a model?

Two sources:

• Local — point the registry at a path inside a mounted models volume, or scan a directory to register everything in it at once.
• Hugging Face — register by repository name. Public repositories can be probed before registering, so you can check size and architecture without committing a download.

Downloads run in the background with live progress (bytes and files completed against real totals) and can be cancelled mid-flight, with the registry reset immediately so a corrected download can be queued right away. Deleting a model removes the registry entry by default, or optionally the files on disk as well — restricted to approved model directories so a stray request cannot delete anything else.

What does capability probing tell you?

Rather than trusting a model card, Lifeboat can probe a model and report what it actually does: whether streaming works and how fast the first byte arrives, the maximum output length it will really produce, its effective context window against the declared one, whether JSON mode works, whether tool/function calling completes a full round trip, whether it accepts images, and whether it emits reasoning traces. The probe runs against the model directly, so the result reflects the model’s own behavior rather than any proxy configuration in front of it.

Which quantization formats are supported?

Quantization is chosen per server and is independent of Lifeboat’s cache optimizations — the two compose.

Format	Notes
None (BF16/FP16)	Default; weights as published
FP8	8-bit float weights
AWQ / GPTQ / Marlin	Common community quantization formats
INT4 / INT8	4-bit and 8-bit integer weights
NVFP4 / MXFP4	Blackwell-native 4-bit float; roughly halves per-token memory reads for a direct decode speedup on supported GPUs, and is auto-detected from the checkpoint
GGUF	Auto-detected from the model files

How does Lifeboat handle GGUF models?

GGUF support is first-class, and the notable piece is the multi-select variant picker: a Hugging Face repository often contains many quantizations of the same model totalling hundreds of gigabytes. Lifeboat lists the individual files with their real sizes so you can download only the one quantization you want — turning a 100 GB repository into a 14 GB download. Unsupported GGUF architectures are caught before download rather than at server start, and split/sharded archives are detected and reported with the exact remedy instead of failing silently later.

Are private models supported?

Yes. Lifeboat includes a token vault for Hugging Face credentials so gated and private repositories can be downloaded. Tokens are encrypted at rest, can be scoped or pinned per model, and are managed from the UI. A single environment-provided token is supported as a fallback for simple single-tenant deployments.

Sources and references

• Running a model once registered: Servers and multi-GPU.
• How optimization interacts with quantization: Inference optimization.