mcp-backpressure

Problem:LLMs can generate hundreds of parallel tool calls, causing resource exhaustion, server crashes, and no structured feedback for clients to retry.

Solution:Middleware that limits concurrent executions, queues excess requests with timeout, and returns structured JSON-RPC overload errors.

from fastmcp import FastMCP from mcp_backpressure import BackpressureMiddleware mcp = FastMCP("MyServer") mcp.add_middleware(BackpressureMiddleware( max_concurrent=5, # Max parallel executions queue_size=10, # Bounded queue for waiting requests queue_timeout=30.0, # Queue wait timeout (seconds) ))

- Concurrency limiting: Semaphore-based control of parallel executions
- Bounded queue: Optional FIFO queue with configurable size
- Queue timeout: Automatic timeout for queued requests with cleanup
- Structured errors: JSON-RPC compliant overload errors with detailed metrics
- Metrics: Real-time counters for active, queued, and rejected requests
- Callback hook: Optional notification on each overload event
- Zero dependencies: Only requires FastMCP and Python 3.10+

from mcp_backpressure import BackpressureMiddleware mcp.add_middleware(BackpressureMiddleware( max_concurrent=5, # Required: max parallel tool executions queue_size=10, # Optional: bounded queue (0 = no queue) queue_timeout=30.0, # Optional: seconds to wait in queue overload_error_code=-32001, # Optional: JSON-RPC error code on_overload=callback, # Optional: called on each overload ))

When the server is overloaded, requests are rejected with a structured JSON-RPC error:

{ "code": -32001, "message": "SERVER_OVERLOADED", "data": { "reason": "queue_full", "active": 5, "queued": 10, "max_concurrent": 5, "queue_size": 10, "queue_timeout_ms": 30000, "retry_after_ms": 1000 } }

Get real-time metrics from the middleware:

metrics = middleware.get_metrics() # Synchronous print(f"Active: {metrics.active}") print(f"Queued: {metrics.queued}") print(f"Total rejected: {metrics.total_rejected}") print(f"Rejected (concurrency): {metrics.rejected_concurrency_limit}") print(f"Rejected (queue full): {metrics.rejected_queue_full}") print(f"Rejected (timeout): {metrics.rejected_queue_timeout}")

For async contexts, useawait middleware.get_metrics_async().

Register a callback to be notified of each overload event:

def on_overload(error: OverloadError): print(f"OVERLOAD: {error.reason} (active={error.active})") # Log to monitoring system, update metrics, etc. middleware = BackpressureMiddleware( max_concurrent=5, queue_size=10, on_overload=on_overload, )

This simulates 30 concurrent requests against a server limited to 5 concurrent executions with a queue of 10, demonstrating how the middleware handles overload.

The middleware provides two-level limiting:
- Semaphore(max_concurrent): Controls active executions
- Bounded queue(queue_size): Holds waiting requests with timeout

- If execution slot available → execute immediately
- If execution slots full and queue not full → wait in queue with timeout
- If queue full → reject withqueue_full
- If timeout in queue → reject withqueue_timeout

Invariants(guaranteed under all conditions):

- active <= max_concurrentALWAYS
- queued <= queue_sizeALWAYS
- Cancellation correctly frees slots and decrements counters
- Queue timeout removes item from queue

- Global limits only(v0.1): Per-client and per-tool limits deferred to v0.2+
- Simple counters: No Prometheus/OTEL dependencies by default
- JSON-RPC errors: Follows MCP protocol conventions
- Monotonic time: Queue timeouts usetime.monotonic()for reliability

Contributions welcome! Please open an issue before submitting PRs.

This is a web browser that enables your coding agent, such as Claude Code, to visit websites on your behalf and assist you in identifying bugs or creating UI test cases.

One shared context layer for AI agents and humans — live API specs, DB schemas, and versioned contracts across repos so every agent and teammate works from the same source of truth.

The MCP server for Bitrix24 provides AI assistants with structured access to the Bitrix24 API. It delivers up-to-date method descriptions, parameters, and valid values, allowing assistants to work with precise data instead of guesswork. This reduces code errors and accelerates Bitrix24 integration development.

Tool platform by IBM to build, test and deploy tools for any data source

One remote MCP server for 500+ production APIs — Stripe, HubSpot, Postgres, Gmail, and more. OAuth and API key auth, credential management, and a CLI.

An MCP server for interacting with the Postman API, requiring an API key.

Arbitrary code execution and tool-use platform for LLMs by Riza

A command-line tool for interacting with Shopify's Admin GraphQL API, Functions, and Polaris Web Components.

Single tool to control all 100+ API integrations, and UI components

Agent-native developer Q&A API with MCP + A2A endpoints for citations, job pickup, and answer submission.