llmtrim

Proxy · CLI · MCP · library (Python · Ruby · Swift · Kotlin · JS/WASM)

You run Claude Code, Codex, Cursor, or your own app. Every turn, the tool sends a large request: system prompt, tools, history, raw command output. You pay for every token of that, including the parts that do not help the model.

A 200-line build log with two errors. Tool schemas resent on every call. JSON with hundreds of near-identical rows. That bulk is still billed.

llmtrim sits on your machine as a local proxy, trims the waste, and forwards a smaller request. The reply is unchanged. You keep the same tools and answers; you spend less.

before: your tool ───── full request ─────▶ OpenAI / Anthropic / … ◀──────── reply ────────── after: your tool ──▶ llmtrim ──smaller──▶ OpenAI / Anthropic / … (on your machine) ◀──────── reply ────────── (same answer)

npm install -g @llmtrim/cli@latest && llmtrim setup # open a new terminal, then keep working llmtrim status

That's it.setupstarts a local proxy, wires your shell, and enables recoverable tool-output shaping. When Claude Code is present, it also turns on/sub. You do not run a separate install for that.

# Linux / macOS curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh # Windows (PowerShell) irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex # Package managers brew install fkiene/tap/llmtrim cargo binstall llmtrim scoop install llmtrim docker run -d -p 43117:43117 -v llmtrim-state:/data ghcr.io/fkiene/llmtrim

llmtrim ca openssl x509 -in ~/.llmtrim/ca.pem -noout -text | grep -A3 "Name Constraints"

llmtrim status # savings + health (aliases: monitor, gain) llmtrim update # new release, restart daemon, refresh integrations llmtrim ensure # match the recommended install state on this machine

Aftersetup,update, orensure, owned Claude Code/substays in sync with the binary.

Time series:llmtrim status --daily·--weekly·--monthly·--json·--csv.

An agent ran a build. The tool returned 58 lines; two were errors. All 58 would have been billed.

4,662 chars → 978 (−79%). Errors stay verbatim. Repeated INFO lines fold into a template plus the values (lossless when the range is regular).

When~/.claudeexists,setup,update, andensurewire/sub(and routed subagents). No separate install command for that.

Claude Code treats any API-key auth as overriding claude.ai login, soclaude.ai connectors are disabledwhile the dummy token is set. That is expected. To keep connectors (and accept Anthropic/loginwhen the session expires):

llmtrim sub anthropic-login keep # connectors OK; Anthropic login required llmtrim sub anthropic-login skip # default: no Anthropic /login; connectors off

Restart Claude Code aftersub on/sub off/sub mode/sub anthropic-loginfor the settings change to take effect. Fallback whose first hop is "what's in use" still needs a real Anthropic login.

Same engine, no proxy required. No extra model calls; compress runs in-process.

llmtrim mcp install # Claude Code llmtrim mcp install --print # paste into any client

Default is fine for most traffic.autoinspects each request and picks compressors by shape (tools →agent, code →code, long Q&A →rag, elseaggressive).

Override withLLMTRIM_PRESET=<name>orpreset = "<name>"in$XDG_CONFIG_HOME/llmtrim/config.toml:

autocomposes these per request shape, so most users never set them directly. Pick one when you know your traffic and want to skip shape detection:

Env:LLMTRIM_PRESET(preset),LLMTRIM_CONFIG(config-file path).

These knobs are orthogonal to compression. Each resolves env-first, then from the config file, so set whichever fits. The env var wins when both are present.

extra_hostsentries must be exact hostnames (llm.acme.com, never a bareacme.com): each one widens the name-constrained MITM CA, which regenerates automatically on the next launch to cover them.

export LLMTRIM_UPSTREAM_PROXY=http://host:port # or with auth: http://user:pass@host:port (redacted in logs)

Outbound calls useCONNECT+ verifying TLS; the upstream only sees the encrypted stream. Looping to llmtrim's own listen address is rejected. Put the variable in thedaemon'slaunch environment (launchd / systemd), not only your interactive shell. Profile secrets sit in plaintext.

Every case is sent twice, once original and once compressed, then both answers are scored and billed at real rates. Cost and quality are measured together, not estimated, across 112 cases:

The token cuts are model-independent (−31% input, −74% output). The dollar saving tracks the model's output-to-input price ratio: −66% here, projecting to −57% at GPT-4o rates and −59% at Claude Sonnet rates. The proxy compresses only the new-content surface and never rewrites the cache-controlled prefix, so your prompt-cache discount survives.

The same A/B on the standard academic suites, at a conservative shape-matched preset (qwen3-next-80b, paired 95% CI). Quality is the score on the original request vs the compressed one. GSM8K comes from the frontier above (n=12); the other three are the named benchmarks readers compare against (n=20 each):

Three rows compress with no quality loss; GSM8K is the one dip:

- BFCLdrops the tool schemas the query doesn't need (a menu of 2 to 37 candidates per call).
- SQuAD v2still answers its unanswerable questions correctly.
- TruthfulQAholds factual accuracy exactly: its ~75-token prompts are almost all answer text, so the safe preset finds nothing to cut.
- GSM8Ktrades −8pp of accuracy for −71% cost, so measure per workload before enabling its reasoning preset. ¹Its input goes negative because that preset injects a Chain-of-Draft instruction whose payoff is output-side (see the frontier table).

make -C crates/llmtrim-cli/bench data (cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \ --corpus bench/data/squad2.jsonl --preset rag \ --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

make -C crates/llmtrim-cli/bench data # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …) (cd crates/llmtrim-cli && cargo run -q --features live -- bench suite) # live A/B across all corpora (needs OPENROUTER_API_KEY) (cd crates/llmtrim-cli/bench/scripts && PYTHONPATH=. python3 -m benchkit.tools.chart) # regenerate the chart + table

Each tool compresses one slice of the request. llmtrim compresses input and output, leaves the cached prefix untouched to keep the prompt cache stable, and scores on whether the answer survives the cut, not on tokens removed. Both axes below use theo200k_baseencoder and reproduce from this repo.