RAG-MCP Pipeline Research

by dzikrisyairozi

598 downloads

Not rated

GitHub

About

A learning repository exploring Retrieval-Augmented Generation (RAG) and Multi-Cloud Processing (MCP) server integration using free and open-source models.

Details

Author: dzikrisyairozi
Downloads: 598
Categories: Search, Knowledge Base

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name RAG-MCP Pipeline Research

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

From the repository

Clone the repository, run python src/setup_environment.py to prepare the environment, and activate the virtual environment. Start with Module 0 (Prerequisites) and progress sequentially through the modules, completing the practical exercises in each section.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "rag-mcp pipeline research": {
            "rag-mcp-pipeline-research": {
                "command": "python",
                "args": [
                    "src/setup_environment.py"
                ]
            }
        }
    }
}

McpServers

{
    "rag-mcp-pipeline-research": {
        "command": "python",
        "args": [
            "src/setup_environment.py"
        ]
    }
}

RAG-MCP Pipeline Research

A comprehensive research project exploring Retrieval-Augmented Generation (RAG) and Multi-Cloud Processing (MCP) server integration using free and open-source models.

Project Overview

This repository serves as a structured learning and research path for understanding how to integrate Large Language Models (LLMs) with external services through MCP servers, with a focus on practical business applications such as accounting software integration (e.g., QuickBooks).

🌟 Key Features

- No paid API keys required - uses free Hugging Face models - Run everything locally without external dependencies - Comprehensive step-by-step documentation for beginners - Practical examples with working code

Research Modules

Module 0: Prerequisites

Establish a solid foundation before diving into specific areas: - Programming & Tools: Python, Git/GitHub, Docker - Basic Concepts: Machine learning, RESTful APIs, cloud services - AI & LLM Foundations: Understanding transformers, RAG, and prompt engineering - Development environment setup with free models

Module 1: AI Modeling & LLM Integration

- Understanding different LLM architectures and capabilities - Integration methods with various LLM providers (Hugging Face, open-source models) - Fine-tuning strategies for domain-specific tasks - Evaluation metrics and performance optimization

Module 2: Hosting & Deployment Strategies for AI

- Scalable infrastructure for AI applications - Cost optimization techniques - Model serving options (serverless, container-based, dedicated instances) - Monitoring and observability for LLM applications

Module 3: Deep Dive into MCP Servers

- Architecture and components of MCP servers - Building secure API gateways for external service integration - Authentication and authorization patterns - Command execution protocols and standardization

Module 4: API Integration & Command Execution

- Integration with business software APIs (QuickBooks, etc.) - Data transformation and normalization - Error handling and resilience strategies - Testing and validation methodologies

Module 5: RAG (Retrieval Augmented Generation) & Alternative Strategies

- Vector database selection and optimization - Document processing pipelines - Hybrid retrieval approaches - Alternative augmentation strategies for LLMs

Project Goals

1. Gain comprehensive understanding of RAG and MCP server concepts
2. Build prototype integrations with popular business software
3. Develop a framework for AI-powered data entry and processing
4. Create documentation and best practices for future implementations

Getting Started

1. Clone this repository to your local machine

   git clone https://github.com/your-username/rag-mcp-pipeline-research.git

   cd rag-mcp-pipeline-research

2. Run the setup script to prepare your environment

   # Navigate to the project directory

   python src/setup_environment.py

3. Activate the virtual environment

   # On Windows

   venv\Scripts\activate

   

   # On macOS/Linux

   source venv/bin/activate

4. Start with Module 0: Prerequisites
5. Progress through each module sequentially
6. Complete the practical exercises in each section

Why Free Models?

This project intentionally uses free, open-source models from Hugging Face instead of commercial APIs like OpenAI for several reasons:

1. Accessibility - Anyone can follow along without financial barriers
2. Educational Value - Better understanding of how models work internally
3. Privacy - All processing happens locally on your machine
4. Flexibility - Easier to customize and fine-tune models for specific needs
5. Future-Proofing - Skills transfer to any model, not tied to specific providers

For production applications, you may choose to use commercial APIs for better performance, but the concepts learned here apply universally.

License

MIT

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Details

Jump to

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

RAG-MCP Pipeline Research

Project Overview

🌟 Key Features

Research Modules

Module 0: Prerequisites

Module 1: AI Modeling & LLM Integration

Module 2: Hosting & Deployment Strategies for AI

Module 3: Deep Dive into MCP Servers

Module 4: API Integration & Command Execution

Module 5: RAG (Retrieval Augmented Generation) & Alternative Strategies

Project Goals

Getting Started

Why Free Models?

License

Install

More like this

Aws Kb Retrieval Server

Memory

Brave Search

Context 7

🔎 GPT Researcher

MaxKB