Self hosters running multiple local models who need automatic swapping without manual server restarts.

llama-swap is a lightweight reverse proxy that automatically starts, stops, and swaps backend LLM processes such as llama.cpp or vLLM based on the incoming request's model name.

Runs as a single Go binary or Docker container in front of one or more local inference servers.

Actively maintained since 2024 with frequent releases.

Lets a single machine host many large models without keeping them all loaded in VRAM simultaneously.

Wraps any OpenAI or Anthropic compatible backend, most commonly llama.cpp, vLLM, or TabbyAPI.

Open source under MIT on GitHub, no account required, runs entirely on the operator's own hardware.