MCP YOLOE: Zero-Shot Object Detection & Segmentation
About
Provide your AI agents with "eyes." This server enables open-vocabulary object detection and instance segmentation using naturally phrased text prompts (e.g., "detect the laptop next to the coffee").
Details
- Author
- rjn32s
- Downloads
- 278
- Categories
- Other, AI
Jump to
- Zero-shot detection with natural language prompts.
- Precision segmentation with exact polygon masks.
- Flexible inputs: local paths, URLs, Base64.
- Agent-first design for Claude, IDEs, and workspace agents.
- Uses YOLOE26‑L architecture (55.0 mAP, ~6.2ms on T4).
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
MCP YOLOE: Zero-Shot Object Detection & SegmentationCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
You interact through an AI agent (e.g., Claude) by providing a natural language prompt such as “Find the ‘vintage typewriter’ in this image and give me its exact coordinates.” The server accepts local file paths, remote image URLs, and Base64-encoded images as input.
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"mcp yoloe: zero-shot object detection & segmentation": {
"mcp-yolo": {
"command": "uvx",
"args": [
"mcp-yolo"
]
}
}
}
}McpServers
{
"mcp-yolo": {
"command": "uvx",
"args": [
"mcp-yolo"
]
}
}MCP-YOLO
MCP-YOLO is a powerful Model Context Protocol server that grants AI agents advanced computer vision capabilities. Unlike traditional YOLO models that only detect a fixed list of objects, this server uses Zero-Shot Learning to detect and segment anything you describe.Key Features
- Zero-Shot Detection: Detect arbitrary objects using natural language prompts. - Precision Segmentation: Get exact polygon masks for every detected object. - Flexible Inputs: Works with local file paths, remote image URLs, and Base64 strings. - Agent-First: Designed specifically for integration with Claude, IDEs, and autonomous workspace agents.Example Usage
Ask your agent to: > "Find the 'vintage typewriter' in this image and give me its exact coordinates."Performance
Uses the state-of-the-art YOLOE26-L architecture, providing a perfect balance of high precision (55.0 mAP) and rapid inference (~6.2ms on T4 GPUs).Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
