LLM Router

AI coding tools send too many prompts to premium models by default.

- You waste paid tokens on simple questions
- You burn through Claude, Gemini, or OpenAI quota faster than necessary
- You stop working when one provider is rate-limited or down

llm-routersits between your coding tool and your model providers. It classifies each prompt, tries the cheapest capable model first, and falls back automatically when needed.

You keep the same workflow. The router changes the model choice underneath.

pip install llm-routing llm-router install

Package name:llm-routingon PyPI. CLI command:llm-router.

export OPENAI_API_KEY="sk-..." # GPT-4o, o3 export GEMINI_API_KEY="AIza..." # Gemini Flash/Pro (free tier available) export OLLAMA_BASE_URL="http://localhost:11434" # Local models (free) export OPENROUTER_API_KEY="sk-or-v1-…" # 343 OpenRouter models (qwen, deepseek, grok, …)

Works withzero API keyson Claude Code Pro/Max subscriptions — routing uses MCP tools that call external models only when beneficial. AddOPENROUTER_API_KEYto unlock the open-weight workhorse pool used by thecost_aggressivepolicy.

llm-router health # Check provider connectivity

If you already use Claude Code, Codex, or Gemini CLI, keep your existing workflow and letllm-routerchoose models underneath it.

The exact chain depends on your configured providers, budget profile, and routing policy.

- Full auto-routingmeans hooks intercept prompts and route automatically with no workflow change.
- Manual MCP toolsmeans routing is available on demand through tools such asllm_query.

llm-router install # Claude Code (default) llm-router install --host codex # Codex CLI llm-router install --host gemini-cli # Gemini CLI llm-router install --host vscode # VS Code llm-router install --host cursor # Cursor

User prompt │ ▼ ┌──────────────────────┐ │ Complexity Classifier │ ← Heuristic (free, instant) or Ollama/Flash ($0.0001) └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ Free-First Router │ ← Tries cheapest model first, walks up the chain │ │ │ Ollama (free) │ │ → Codex (prepaid) │ │ → Gemini Flash │ │ → GPT-4o / Claude │ └──────────┬───────────┘ │ ▼ ┌──────────────────────┐ │ Guards (parallel) │ ← Circuit breaker, budget pressure, quality check └──────────┬───────────┘ │ ▼ Response + cost logged to local SQLite

Classification is free for many tasks (regex heuristics catch ~70%) or near-free for ambiguous prompts when using local Ollama or Gemini Flash.

Beyond "send cheap prompts to cheap models":

- Secrets never leave your machine.A prompt containing an API key, token or private key routes to local models only — fail-closed, so it cannot reach an external provider.
- Cost-inverted subscription routing.Free/local first for simple and moderate prompts, your one paid seat first for complex ones, and the seat demoted when its quota is strained. Opt in withLLM_ROUTER_SUBSCRIPTION_PROVIDER.
- Automatic fallback with circuit breakers.A provider that fails or rate-limits is skipped, not retried into the ground.
- You can see it working.A status line, terminal title and OS notification show the last model routed, savings and health — for hosts with no native statusline.
- Session-end summary.Savings vs baseline, tier mix, per-provider cost, latency p50/p95/p99 and top routes.
- Media and pipelines too.llm_image/llm_video/llm_audio, andllm_orchestratefor multi-step research.

llm-router install # wire up your host (Claude Code by default) llm-router health # provider connectivity llm-router status # savings + quota at a glance llm-router doctor # diagnose a broken setup

20+ providers, free-first.Ollama(local, free) leads the chain;OpenRouter(343 models behind one key) is the biggest single unlock;GeminiandGroqhave usable free tiers. Anthropic works via your existing Claude subscription — no API key needed.

A policy sets how eagerly the router routes away from your premium model —conservative(10–15% savings) throughbalanced(the default, 35–45%) tocost_aggressive(70–85%, needsOPENROUTER_API_KEY).

60 tools across routing, analysis, code, media, budget and diagnostics — exposed to any MCP host. The defaultconsolidatedsurface shows 11 front-door tools; setLLM_ROUTER_SLIM=fullfor all 60.

Savings are calculated by comparing actual spend against a baseline of routing every task to Claude Sonnet/Opus.
- Each routed task logs: model used, tokens consumed, estimated cost
- A baseline cost is computed as if the same tokens were processed by the most expensive model in the chain
- Savings =(baseline - actual) / baseline

- Baseline assumes you would have used Opus/Sonnet for everything (worst case)
- Token estimates uselen(text) / 4approximation, not exact tokenizer counts
- Cost data comes from LiteLLM's pricing tables (may lag provider price changes)
- Savings vary significantly by workload — code-heavy sessions route more to cheap models
- The router itself adds small overhead (classification costs ~$0.0001 per ambiguous task)

Observed range:35–80% savings depending on policy and task mix. The "87%" figure in some docs represents a single-user peak over a specific development period, not a guaranteed outcome.

llm-router runs entirely on your machine. There is no hosted proxy, no telemetry, no account required.

- Scrub API keys from structured logs
- Detect hook deadlocks before installation
- Store all data locally in~/.llm-router/
- Respect provider rate limits and TOS

- Prompts are sent to whichever provider the router selects — review your provider's privacy policy
- Usage logs (SQLite) are not encrypted at rest — use full-disk encryption if needed
- The router cannot prevent model jailbreaks or prompt injection at the provider level

Everything is environment variables — no config file required to start:

export OPENROUTER_API_KEY="sk-or-v1-..." # biggest single unlock export OLLAMA_BASE_URL="http://localhost:11434" # local, free export LLM_ROUTER_POLICY="cost_aggressive" # routing policy export LLM_ROUTER_ENFORCE="smart" # off | advise | smart | hard

git clone https://github.com/ypollak2/llm-router.git cd llm-router uv sync --extra dev uv run pytest tests/ -q # Run tests (1900+) uv run ruff check src/ tests/ # Lint

-|-----------| |llm-routing| Current PyPI package (pip install llm-routing) | |llm-router| CLI command and GitHub repo name | |claude-code-llm-router| Deprecated legacy package (redirects tollm-routing) |

⭐ If llm-router saved you money, star the repo — it helps other developers discover it.

An MCP server with an LLMling backend that uses YAML files to configure LLM applications.

Translate JSON i18n files using Google Gemini or local Ollama models, with incremental caching support.

An MCP server for AI video generation. MCP server for AI video generation. Lets Claude, ChatGPT, OpenClaw , Hermes & other agents create AI videos and publish them to YouTube, TikTok, Instagram etc..

HumanDesign.ai MCP is the official account-connected Human Design server for Claude, ChatGPT, Codex, Cursor, and VS Code.

3D Cartoon Generator & File System Tools

Generates 3D-style cartoon images using Google's Gemini AI and provides secure file system operations.

A Model Context Protocol (MCP) server written in Go that wraps the APsystems OpenAPI, giving AI assistants like Claude direct access to your solar monitoring data. Includes an optional web dashboard for visual monitoring.

MCP server for interacting with the APVISO AI-powered penetration testing platform from Claude Code, Cursor, Windsurf, Codex, and other MCP-compatible tools.

Detect fabrication and hallucination in any LLM output. Score responses from GPT-4o, Claude, Gemini, Llama and 30+ models. Free tier included.

Identity, continuity & trust for AI agents — prove you're an LLM (not human), hold a spirit key for signing/encryption/memory, and issue offline-verifiable signed documents.