Flama
About
The production framework for Predictive and Generative AI. Serve any model as an API in one line, with OpenAI/Anthropic/Ollama-compatible endpoints, a built-in chat UI, and native MCP.
Details
- Author
- vortico
- Categories
- Other, AI
Jump to
Setup
Install Flama in your MCP client (Claude Desktop, Cursor, Windsurf, and others).
Repository: https://github.com/vortico/flama
Follow the installation instructions in the repository README, then restart your MCP client.
The production framework for Predictive and Generative AI.
Turn any model into a production API in a single line of code. Serve predictive and generative models on a Rust-powered core, and expose your tools to AI agents over the Model Context Protocol (MCP).
Flama is theFramework forLightweightApplications, artificial intelligenceModels, andAutomation. It packages a model from any of the mainstream frameworks into a single portable format (the.flmfile), so every model looks the same to your API no matter where it came from, and serves it over HTTP in seconds.
- 📦Any framework, one format.Package scikit-learn, TensorFlow, PyTorch, or an LLM into a single portable.flmartifact.
- ⬇️Models on demand.Download and package any model from the HuggingFace Hub with one command.
- 🤖Generative AI serving.Serve LLMs with OpenAI-, Anthropic-, and Ollama-compatible endpoints, side by side.
- 💬Chatbot out of the box.Every served model ships a polished streaming chat UI at/chat/, with Markdown, LaTeX, and Mermaid.
- 🔌Native MCP.Expose tools, resources, and prompts to AI agents with a single decorator, schemas derived from your type hints.
- ⚡Rust-powered core.Routing, JSON encoding, request parsing, and compression compiled to native code, shipped as plain wheels.
- 🚀Production-ready first.Go from a packaged model to a running service over the CLI, in Python, with a spec file, or inside a container.
Flama is published on PyPI and ships native wheels for every supported Python version (3.10 to 3.14) on Linux, macOS, and Windows. No Rust toolchain required.
Schema, database, and LLM support are optional extras, so you install only what you need:
pip install "flama[pydantic]" # schema validation (also: typesystem, marshmallow) pip install "flama[database]" # SQLAlchemy-backed resources pip install "flama[llm]" # generative AI serving (vLLM on Linux, MLX on Apple Silicon) pip install "flama[full]" # everything
From zero to a production API with a built-in chat UI in three commands, no Python code required:
pip install "flama[llm,pydantic]" # 1. Download and package a model from HuggingFace into a portable .flm flama get --family llm --source huggingface mlx-community/gemma-4-E2B-it-qat-4bit # 2. Try it straight from your terminal echo "What is Flama?" | flama model mlx-community_gemma-4-E2B-it-qat-4bit.flm stream --system "Be concise." # 3. Serve it over HTTP flama serve --model file=mlx-community_gemma-4-E2B-it-qat-4bit.flm,url=/,name=gemma
That is it: a full HTTP API, a streaming chat interface athttp://127.0.0.1:8000/chat/, and multi-dialect endpoints. The same.flmfile runs onvLLM(Linux with CUDA) orMLX(Apple Silicon), with Flama selecting the backend at load time.
You do not even need a server to try a model. Pipe a prompt intoflama model ... streamand the response streams straight into your shell:
Speak the protocols your clients already use
A single model can serve multiple wire protocols simultaneously, so existing OpenAI, Anthropic, and Ollama clients work without code changes, just point them at your server.
The same workflow serves classic ML models. Package a model trained in any mainstream framework:
import flama from sklearn.neural_network import MLPClassifier model = MLPClassifier(activation="tanh", hidden_layer_sizes=(10,)) # ... training ... flama.dump(model, "model.flm")
Or fetch one straight from the Hub, then serve it:
flama get --family ml --source huggingface scikit-learn/Fish-Weight flama serve --model file=scikit-learn_Fish-Weight.flm,url=/model,name=fish
Flama ships native, first-class support for the Model Context Protocol. Declare a capability with a single decorator, mount the server, and Flama derives the JSON Schema from your type hints and serves it over a stateless protocol:
from flama import Flama app = Flama() app.mcp.add_server("/mcp/tools/", "tools") @app.mcp.tool("add", description="Add two integers", mcp="tools") def add(a: int, b: int) -> int: return a + b
Any MCP-capable client (Claude, Cursor, VS Code Copilot, or a custom agent) can discover and invoke it. Tasks, Elicitation, and MCP Apps are included. Learn more in theMCP docs.
Flama is also a complete toolkit for building production APIs:
- Resourceswith standard CRUD methods over SQLAlchemy tables.
- Dependency injectionviaComponents, the base of the plugin ecosystem.
- Adaptable schemaswith Pydantic, Typesystem, or Marshmallow, all optional extras.
- Auto-generated OpenAPIschema plus Swagger UI and ReDoc.
- Pagination, background tasks, lifespan events, and JWT authentication.
- Streaming-first HTTPwith Server-Sent Events and NDJSON responses.
- Domain-Driven Designpatterns: repositories, workers, and domain models.
- flama upgradecodemods that rewrite imports and renamed symbols across major versions.
A curated, documentation-aligned set of runnable examples lives invortico/flama-examples, covering fundamentals, the CLI, advanced topics, predictive AI, generative AI, and domain-driven design.
Visithttps://flama.dev/docs/for the full documentation, including thequickstartand theCLI guide.
Dropskill.mdinto your AI coding assistant and let it build Flama apps with full framework knowledge.
- José Antonio Perdiguero López (@perdy)
- Miguel Durán-Olivencia (@migduroli)
This project is absolutely open to contributions, so if you have a nice idea, please read ourcontributing docsbefore submittinga pull request. Questions and ideas are welcome inGitHub Discussions.
If you find Flama useful for building robust Machine Learning and Generative AI APIs, the best way to support our work is togive us a ⭐ on GitHub, it is the best fuel for our development efforts. You can also followVorticofor updates.
Flama is released under theApache 2.0license.
Access GPT-5, Claude, Gemini and other models through a single MCP connection. Save development time and money on subscriptions.
Universal tool adapter — @tool decorator exports Python functions to OpenAI, Claude, Gemini, MCP, JSON Schema. Audit token costs.
Standing review layer for coding agents: Claude, GPT and Gemini debate each answer and return one recommendation plus the strongest dissent.
614 million contacts, live SMTP-verified. Invalid emails cost you $0. Connect Argorant with any Agent through CLI, MCP or API and build verified lists in Claude, GPT or Hermes.
Deepseek Thinking & Claude 3.5 Sonnet
Combines DeepSeek's reasoning capabilities with Claude 3.5 Sonnet's response generation through OpenRouter.
An MCP server that orchestrates Google Gemini and Claude Code models via the OpenRouter API.
Delegate bounded work from Claude to any OpenAI-compatible LLM endpoint (LM Studio, Ollama, OpenRouter), preserving your Claude context and quota.
Stop AI Hallucinations Before They Start Run models from OpenAI, Google, Anthropic, xAI, Perplexity, and OpenRouter in parallel. They check each other's work, debate solutions, and catch errors before you see them.
Enables voice interactions with Claude and other LLMs using an OpenAI API key for STT/TTS services.
Access multiple AI models like Claude, Gemini, and OpenAI through a single server using your own API keys.
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
