VisionAgent MCP Server
About
VisionAgent MCP Server is a lightweight, side-car MCP server that runs locally on STDIN/STDOUT, translating tool calls from MCP‑compatible clients (Claude Desktop, Cursor, Cline) into authenticated HTTPS requests to Landing AI’s VisionAgent REST APIs. It enables natural‑language…
Details
- Author
- landing-ai
- Downloads
- 292
- Categories
- AI
Jump to
- Translates MCP tool calls into authenticated VisionAgent API requests.
- Supports agentic document analysis, text‑to‑object detection, text‑to‑instance segmentation, activity recognition, and depth estimation.
- Auto‑generates tool definitions from a live OpenAPI spec via npm run generate-tools.
- Renders masks, bounding boxes, and depth maps to files or inline previews.
- Validates arguments with Zod schemas derived from the OpenAPI spec.
- Outputs JSON and media results streamed back to the MCP client.
Install globally via npm install -g vision-tools-mcp, set the VISION_AGENT_API_KEY environment variable, then configure your MCP client with npx vision-tools-mcp as the command and the required environment variables. Prerequisites are Node.js 20 LTS, a VisionAgent account with an API key, and an MCP‑compatible client.
VisionAgent MCP Server
> Beta – v0.1
> This project is early access and subject to breaking changes until v1.0.
VisionAgent MCP Server v0.1 - Overview
Modern LLM “agents” call external tools through the Model Context Protocol (MCP). VisionAgent MCP is a lightweight, side-car MCP server that runs locally on STDIN/STDOUT, translating each tool call from an MCP-compatible client (Claude Desktop, Cursor, Cline, etc.) into an authenticated HTTPS request to Landing AI’s VisionAgent REST APIs. The response JSON, plus any images or masks, is streamed back to the model so that you can issue natural-language computer-vision and document-analysis commands from your editor without writing custom REST code or loading an extra SDK.
📸 Demo
https://github.com/user-attachments/assets/2017fa01-0e7f-411c-a417-9f79562627b7
🧰 Supported Use Cases (v0.1)
| Capability | Description |
| ----------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| agentic-document-analysis | Parse PDFs / images to extract text, tables, charts, and diagrams taking into account layouts and other visual cues. Web Version here.|
| text-to-object-detection | Detect free-form prompts (“all traffic lights”) using OWLv2 / CountGD / Florence-2 / Agentic Object Detection (Web Version here); outputs bounding boxes. |
| text-to-instance-segmentation | Pixel-perfect masks via Florence-2 + Segment-Anything-v2 (SAM-2). |
| activity-recognition | Recognise multiple activities in video with start/end timestamps. |
| depth-pro | High-resolution monocular depth estimation for single images. |
> Run npm run generate-tools whenever VisionAgent releases new endpoints. The script fetches the latest OpenAPI spec and regenerates the local tool map automatically.
🗺 Table of Contents
1. Quick Start 2. Configuration 3. Example Prompts 4. Architecture & Flow 5. Developer Guide 6. Troubleshooting 7. Contributing 8. Security & Privacy🚀 Quick Start
Get Your VisionAgent API Key
If you do not have a VisionAgent API key, create an account and obtain your API key.```bash
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
