mcp_llm_inferencer

Introduction

The mcp_llm_inferencer is an open-source library designed to leverage the power of Large Language Models (LLMs) such as Claude and OpenAI's GPT to convert prompt-mapped inputs into concrete components for MCP servers. These components include tools, resource templates, and prompt handlers, making it a versatile tool for developers working with MCP server environments.

Features

- LLM Call Engine: Efficiently calls LLMs with built-in retry and fallback logic to ensure reliable responses.
- Interchangeable Claude & OpenAI Support: Seamlessly switch between Claude and OpenAI APIs based on your preference or availability.
- Streaming Support for Claude Desktop: Stream responses directly from Claude Desktop, providing real-time feedback.
- Tool and Resource Response Validation: Ensures that the generated tools and resources meet predefined criteria before deployment.
- Structured Output Bundling: Organizes output into structured bundles per component, simplifying integration and use.

Installation Instructions

Prerequisites

Installing mcp_llm_inferencer

1. Clone the repository:

    git clone https://github.com/your-repo/mcp_llm_inferencer.git

    cd mcp_llm_inferencer

    

2. Install the package using pip:

    pip install .

    

3. Set up your API keys as environment variables:
- For Claude:

        export CLAUDE_API_KEY='your-claude-api-key'

        

        export OPENAI_API_KEY='your-openai-api-key'

        

Usage Examples

Basic Example

Here is a simple example demonstrating how to use mcp_llm_inferencer with the OpenAI API:

```python
from mcp_llm_inferencer import MCPInferencer