IntraIntel.ai - Multi-LLM Agent Coding Challenge

This project implements a Multi-LLM Agent system in Python designed to answer medical questions. It leverages a custom Model Context Protocol (MCP) to interact with distinct tool servers for information retrieval (Web Search and PubMed Search). The core agent orchestrates multiple free Large Language Models (LLMs) from the Hugging Face Inference API to perform tasks such as query refinement, context snippet summarization, and final answer synthesis using a Retrieval-Augmented Generation (RAG) approach. The system is designed to provide separate, synthesized answers based on the context retrieved from Web Search and PubMed, including links to the original source materials.

Video Demonstration link

System Architecture

Two llm models used distillbart and mistral instruct from free tier hugging face api

Core Approach & Design

The system follows a multi-step, agentic RAG pipeline:

1. User Input: The agent takes a medical question from the user.
2. (Optional) Query Refinement (LLM1): The user's question can be passed to a first LLM (e.g., mistralai/Mistral-7B-Instruct-v0.3) to generate a more concise and effective search query, focusing on key medical terms.
3. Information Retrieval (MCP Tools):
The (potentially refined) search query is sent to one or both MCP tool servers:
Web Search MCP Server: Queries the general web using duckduckgo-search.
PubMed MCP Server: Queries the PubMed database using BioPython's Entrez utilities.
These servers return structured search results (title, snippet, URL/ID).
4. Context Processing:
(Optional) Snippet Summarization (LLM2): For each search result, if the snippet is long, it can be passed to a specialized summarization LLM (e.g., sshleifer/distilbart-cnn-6-6) to create a more concise version.
Context Formatting: The (summarized or original) snippets from each source (Web Search, PubMed) are formatted separately into structured context strings. Each item in the context is labeled (e.g., "Item 1", "Item 2") and includes its title, content, and reference (URL or PubMed ID).
5. Answer Synthesis (LLM3 - RAG):
For each information source (Web Search, PubMed), the agent makes a separate call to a powerful instruction-tuned LLM (e.g., mistralai/Mistral-7B-Instruct-v0.3).
The prompt to this LLM includes:
The original user question.
The formatted context from that specific source.
An instruction to answer based only on the provided context and to cite item numbers if possible.
This LLM generates a synthesized answer for that source.
6. Output: The system presents the original question, followed by the synthesized answer derived from Web Search (with source links) and then the synthesized answer derived from PubMed (with source links).

Key Technologies:
Python 3.9+ with asyncio for concurrency.
FastAPI & Uvicorn: For building the asynchronous MCP tool servers.
httpx: For asynchronous HTTP requests from the agent to MCP servers and the Hugging Face API.
Hugging Face Inference API: To access free LLMs for refinement, summarization, and synthesis.
duckduckgo-search: For API-key-free web searches.
BioPython: For querying NCBI's PubMed database.
python-dotenv: For managing API keys and environment variables.

Request to MCP Server:
Endpoint: POST /execute
JSON Body: {"query": "user's search query string"}
Response from MCP Server:
JSON Body (Success):

        {

          "source": "server_name_string (e.g., WebSearchServer, PubMedServer)",

          "status": "success",

          "results": [

            {

              "title": "string",

              "snippet": "string",

              "url": "string_url_if_web_search", // or missing

              "id": "string_pubmed_id_if_pubmed" // or missing

            }

            // ... more results

          ]

        }

        

        {

          "source": "server_name_string",

          "status": "error",

          "error_message": "string_describing_the_error"

        }

        

1. Clone/Download:
Obtain the project files and place them in a directory named intra_intel_multi_llm_challenge.

2. Create and Activate Python Virtual Environment:
Navigate to the project root directory (intra_intel_multi_llm_challenge).
Create the environment:

        python -m venv venv

        

    pip install -r requirements.txt

    

4. Configure Environment Variables:
In the project root, copy the file .env.example to a new file named .env.
Open the .env file and fill in your details:

        # .env

        HF_API_TOKEN="your_huggingface_api_token_here"

        NCBI_EMAIL="your_email_for_pubmed@example.com"

        

5. Accept Terms for Gated Models (Crucial for Mistral):
The default text generation LLM used is mistralai/Mistral-7B-Instruct-v0.3. This is a gated model.
Log in to your Hugging Face account on their website.
Go to the model card page: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3
Read and accept the license terms to gain access to this model. Your HF_API_TOKEN will then be authorized to use it. If you skip this, API calls to this model will fail.

1. Terminal 1: Start the Web Search MCP Server:
Navigate to the project root (intra_intel_multi_llm_challenge).
Ensure your virtual environment is activated.
Run:

        python -m uvicorn mcp_servers.web_search_server:app --reload --port 8001

        

2. Terminal 2: Start the PubMed MCP Server (for Bonus Functionality):
Navigate to the project root.
Ensure your virtual environment is activated.
Run:

        python -m uvicorn mcp_servers.pubmed_search_server:app --reload --port 8002

        

3. Terminal 3: Run the Main Agent:
Navigate to the project root.
Ensure your virtual environment is activated.
Run:

        python agent/main_agent.py

        

LLM Choices: You can change VERIFIED_INSTRUCT_MODEL_ID and SUMMARIZATION_MODEL_ID in agent/main_agent.py if you find other models that work better or are more reliably available on the Hugging Face free inference API (always verify with the widget on the model's card page first!).
Toggle Features: In the if __name__ == "__main__": block of agent/main_agent.py, you can change the refine_queries and summarize_snippets boolean flags passed to main_agent_workflow to test the pipeline with these steps enabled or disabled. You can also change use_pubmed_for_this_question for individual calls.
Verbose Logging: To see detailed debug information (API payloads, full contexts, etc.):
1. Open agent/main_agent.py.
2. Change LOG_LEVEL = logging.INFO to LOG_LEVEL = logging.DEBUG.
3. Save and re-run the agent.

Dependencies

The main dependencies are listed in requirements.txt:
fastapi: For building the MCP servers.
uvicorn[standard]: ASGI server for FastAPI.
httpx: For making asynchronous HTTP requests.
duckduckgo-search: For web search functionality.
biopython: For querying PubMed via NCBI Entrez.
python-dotenv: For managing environment variables.

This README should cover the necessary details for understanding, setting up, and running the project.

Sample medical questions output screenshot

Web search

Pubmed search