MCP Waifu Queue

This project implements an MCP (Model Context Protocol) server for a conversational AI "waifu" character, leveraging the OpenRouter API via a Redis queue for asynchronous processing. It utilizes the FastMCP library for simplified server setup and management.

Table of Contents

- Features
- Architecture
- Prerequisites
- Installation
- Configuration
- Running the Service
- MCP API
- Testing
- Troubleshooting
- Contributing
- License

Features

Text generation via OpenRouter using model from ~/.model-openrouter or openrouter/free.
Request queuing using Redis for handling concurrent requests asynchronously.
MCP-compliant API using FastMCP.
Job status tracking via MCP resources.
Configuration via environment variables (.env file).
API key loading:
- OpenRouter: OPENROUTER_API_KEY or ~/.api-openrouter
Model selection files in home directory:
- ~/.model-openrouter for OpenRouter model name

main.py: The main entry point, initializing the FastMCP application and defining MCP tools/resources.
respond.py: Contains the core text generation logic using the OpenRouter API.
task_queue.py: Handles interactions with the Redis queue (using python-rq), enqueuing generation requests.
utils.py: Contains utility functions, specifically call_predict_response which is executed by the worker to call the generation logic in respond.py.
worker.py: A Redis worker (python-rq) that processes jobs from the queue, calling call_predict_response.
config.py: Manages configuration using pydantic-settings.
models.py: Defines Pydantic models for MCP request and response validation.

The flow of a request is as follows:

1. A client sends a request to the generate_text MCP tool (defined in main.py).
2. The tool enqueues the request (prompt) to a Redis queue (handled by task_queue.py).
3. A worker.py process picks up the job from the queue.
4. The worker executes the call_predict_response function (from utils.py).
5. call_predict_response calls the predict_response function (in respond.py), which interacts with the OpenRouter API.
6. The generated text (or an error message) is returned by predict_response and stored as the job result by RQ.
7. The client can retrieve the job status and result using the job://{job_id} MCP resource (defined in main.py).

graph LR
    subgraph Client
        A[User/Client] -->|1. Send Prompt via MCP Tool| B(mcp-waifu-queue: main.py)
    end
    subgraph mcp-waifu-queue Server
        B -->|2. Enqueue Job (prompt)| C[Redis Queue]
        B -->|7. Return Job ID| A
        D[RQ Worker (worker.py)] --|>| C
        D -->|3. Dequeue Job & Execute| E(utils.call_predict_response)
        E -->|4. Call Generation Logic| F(respond.predict_response)
        F -->|5. Call OpenRouter API| G[OpenRouter API]
        G -->|6. Return Response| F
        F --> E
        E -->|Update Job Result in Redis| C
        A -->|8. Check Status via MCP Resource| B
        B -->|9. Fetch Job Status/Result| C
        B -->|10. Return Status/Result| A
    end

Prerequisites

Python 3.7+
pip or uv (Python package installer)
Redis server (installed and running)
An OpenRouter API Key

You can find instructions for installing Redis on your system on the official Redis website: https://redis.io/docs/getting-started/
You can obtain an OpenRouter API key from: https://openrouter.ai/

Installation

1. Clone the repository:

    git clone <YOUR_REPOSITORY_URL>
    cd mcp-waifu-queue
    

2. Create and activate a virtual environment using uv:

    python -m uv venv .venv
    .venv/Scripts/python.exe -m ensurepip
    .venv/Scripts/python.exe -m pip install uv
    

3. Install dependencies:

    .venv/Scripts/python.exe -m uv pip install -r requirements.txt
    .venv/Scripts/python.exe -m uv pip install -r requirements-dev.txt
    

Configuration

1. Model Names via files in $HOME:
- OpenRouter model file:

      echo "openrouter/free" > ~/.model-openrouter

      

    echo "YOUR_API_KEY_HERE" > ~/.api-openrouter

    

3. Other Settings: Copy the .env.example file to .env:

    cp .env.example .env
    

4. Modify the .env file to set the remaining configuration values:

MAX_NEW_TOKENS: Maximum number of tokens for the response (default: 2048).
REDIS_URL: The URL of your Redis server (default: redis://localhost:6379).
FLASK_ENV, FLASK_APP: Optional, related to Flask if used elsewhere, not core to the MCP server/worker operation.

    python -m mcp_waifu_queue.worker

    

3. Start the MCP Server:
Open another terminal, activate the virtual environment, and run the MCP server using a tool like uvicorn (you might need to install it: pip install uvicorn or uv pip install uvicorn):

    uvicorn mcp_waifu_queue.main:app --reload --port 8000 # Example port

    

    # Ensure the script is executable: chmod +x ./scripts/start-services.sh

    ./scripts/start-services.sh

    # Then start the MCP server manually as shown above.

    

generate_text
Description: Sends a text generation request to the OpenRouter API via the background queue.
Input: {"prompt": "Your text prompt here"} (Type: GenerateTextRequest)
Output: {"job_id": "rq:job:..."} (A unique ID for the queued job)

job://{job_id}
Description: Retrieves the status and result of a previously submitted job.
URI Parameter: job_id (The ID returned by the generate_text tool).
Output: {"status": "...", "result": "..."} (Type: JobStatusResponse)
status: The current state of the job (e.g., "queued", "started", "finished", "failed"). RQ uses slightly different terms internally ("started" vs "processing", "finished" vs "completed"). The resource maps these.
result: The generated text if the job status is "completed", otherwise null. If the job failed, the result might be null or contain error information depending on RQ's handling.

pytest tests

Error: OpenRouter API key not available: Ensure OPENROUTER_API_KEY is set or ~/.api-openrouter exists with your key on a single line (no whitespace).
Jobs stuck in "queued": Verify that the RQ worker (python -m mcp_waifu_queue.worker) is running in a separate terminal and connected to the same Redis instance specified in .env. Check the worker logs for errors.
ConnectionRefusedError (Redis): Make sure your Redis server is running and accessible at the REDIS_URL specified in .env.

• MCP Server Connection Issues: Ensure the MCP server (uvicorn ...) is running and you are connecting to the correct host/port.

Contributing

1. Fork the repository.
2. Create a new branch for your feature or bug fix (git checkout -b feature/your-feature-name).
3. Make your changes and commit them (git commit -am 'Add some feature').
4. Push your branch to your forked repository (git push origin feature/your-feature-name).
5. Create a Pull Request on the original repository.

Please adhere to the project's coding standards and linting rules (ruff).

License

This project is licensed under the MIT-0 License - see the LICENSE file for details.