Multi-LLM Cross-Check

by lior-ps

10 stars

234 downloads

Not rated

GitHub

About

Provides a unified interface for querying multiple LLM providers simultaneously, enabling side-by-side response comparison for fact-checking, gathering diverse perspectives, or evaluating different models' capabilities.

Details

Author: lior-ps
Repository: lior-ps/multi-llm-cross-check-mcp-server
GitHub stars: 10
Downloads: 234
License: MIT License
Categories: AI, Design, Developer Tools, Search, Frontend, API, Security, Infrastructure

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below
Plugin Name Multi-LLM Cross-Check

Command (node, npx, python, etc.) uv
Arguments
- Argument 1 --directory
- Argument 2 /multi-llm-cross-check-mcp-server
- Argument 3 run
- Argument 4 main.py
Environment
- GEMINI_API_KEY your_gemini_key
- OPENAI_API_KEY your_openai_key
- ANTHROPIC_API_KEY your_anthropic_key
- PERPLEXITY_API_KEY your_perplexity_key
Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

From the repository

To install Multi LLM Cross-Check Server for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @lior-ps/multi-llm-cross-check-mcp-server --client claude

1. Clone this repository:

git clone https://github.com/lior-ps/multi-llm-cross-check-mcp-server.git
cd multi-llm-cross-check-mcp-server

2. Initialize uv environment and install requirements:

uv venv
uv pip install -r requirements.txt

3. Configure in Claude Desktop:
Create a file named claude_desktop_config.json in your Claude Desktop configuration directory with the following content:

   {
     "mcp_servers": [
       {
         "command": "uv",
         "args": [
           "--directory",
           "/multi-llm-cross-check-mcp-server",
           "run",
           "main.py"
         ],
         "env": {
           "OPENAI_API_KEY": "your_openai_key",  // Get from https://platform.openai.com/api-keys
           "ANTHROPIC_API_KEY": "your_anthropic_key",  // Get from https://console.anthropic.com/account/keys
           "PERPLEXITY_API_KEY": "your_perplexity_key",  // Get from https://www.perplexity.ai/settings/api
           "GEMINI_API_KEY": "your_gemini_key"  // Get from https://makersuite.google.com/app/apikey
         }
       }
     ]
   }

Notes:

1. You only need to add the API keys for the LLM providers you want to use. The server will skip any providers without configured API keys.
2. You may need to put the full path to the uv executable in the command field. You can get this by running which uv on MacOS/Linux or where uv on Windows.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "multi-llm cross-check": {
            "env": {
                "GEMINI_API_KEY": "your_gemini_key",
                "OPENAI_API_KEY": "your_openai_key",
                "ANTHROPIC_API_KEY": "your_anthropic_key",
                "PERPLEXITY_API_KEY": "your_perplexity_key"
            },
            "args": [
                "--directory",
                "/multi-llm-cross-check-mcp-server",
                "run",
                "main.py"
            ],
            "command": "uv"
        }
    }
}

Linux

{
    "env": {
        "GEMINI_API_KEY": "your_gemini_key",
        "OPENAI_API_KEY": "your_openai_key",
        "ANTHROPIC_API_KEY": "your_anthropic_key",
        "PERPLEXITY_API_KEY": "your_perplexity_key"
    },
    "args": [
        "--directory",
        "/multi-llm-cross-check-mcp-server",
        "run",
        "main.py"
    ],
    "command": "uv"
}

Macos

{
    "env": {
        "GEMINI_API_KEY": "your_gemini_key",
        "OPENAI_API_KEY": "your_openai_key",
        "ANTHROPIC_API_KEY": "your_anthropic_key",
        "PERPLEXITY_API_KEY": "your_perplexity_key"
    },
    "args": [
        "--directory",
        "/multi-llm-cross-check-mcp-server",
        "run",
        "main.py"
    ],
    "command": "uv"
}

Windows

{
    "env": {
        "GEMINI_API_KEY": "your_gemini_key",
        "OPENAI_API_KEY": "your_openai_key",
        "ANTHROPIC_API_KEY": "your_anthropic_key",
        "PERPLEXITY_API_KEY": "your_perplexity_key"
    },
    "args": [
        "--directory",
        "/multi-llm-cross-check-mcp-server",
        "run",
        "main.py"
    ],
    "command": "uv"
}

Multi LLM Cross-Check MCP Server

A Model Control Protocol (MCP) server that allows cross-checking responses from multiple LLM providers simultaneously. This server integrates with Claude Desktop as an MCP server to provide a unified interface for querying different LLM APIs.

Features

Prerequisites

- Python 3.8 or higher
- API keys for the LLM providers you want to use
- uv package manager (install with pip install uv)

Installation

Installing via Smithery

To install Multi LLM Cross-Check Server for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @lior-ps/multi-llm-cross-check-mcp-server --client claude

Manual Installation

1. Clone this repository:

git clone https://github.com/lior-ps/multi-llm-cross-check-mcp-server.git
cd multi-llm-cross-check-mcp-server

2. Initialize uv environment and install requirements:

uv venv
uv pip install -r requirements.txt

3. Configure in Claude Desktop:
Create a file named claude_desktop_config.json in your Claude Desktop configuration directory with the following content:

   {
     "mcp_servers": [
       {
         "command": "uv",
         "args": [
           "--directory",
           "/multi-llm-cross-check-mcp-server",
           "run",
           "main.py"
         ],
         "env": {
           "OPENAI_API_KEY": "your_openai_key",  // Get from https://platform.openai.com/api-keys
           "ANTHROPIC_API_KEY": "your_anthropic_key",  // Get from https://console.anthropic.com/account/keys
           "PERPLEXITY_API_KEY": "your_perplexity_key",  // Get from https://www.perplexity.ai/settings/api
           "GEMINI_API_KEY": "your_gemini_key"  // Get from https://makersuite.google.com/app/apikey
         }
       }
     ]
   }

Notes:

Using the MCP Server

Once configured:

1. The server will automatically start when you open Claude Desktop
2. You can use the cross_check tool in your conversations by asking to "cross check with other LLMs"
3. Provide a prompt, and it will return responses from all configured LLM providers

API Response Format

The server returns a dictionary with responses from each LLM provider:

{
    "ChatGPT": { ... },
    "Claude": { ... },
    "Perplexity": { ... },
    "Gemini": { ... }
}

Error Handling

- If an API key is not provided for a specific LLM, that provider will be skipped
- API errors are caught and returned in the response
- Each LLM's response is independent, so errors with one provider won't affect others

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Details

Jump to

Setting up with Highlight

From the repository

Claude Desktop / Cursor

Linux

Macos

Windows

Multi LLM Cross-Check MCP Server

Features

Prerequisites

Installation

Installing via Smithery

Manual Installation

Using the MCP Server

API Response Format

Error Handling

Contributing

License

Install

More like this

Sentry

Github

Git

GitLab

Context 7

1Panel