Calypso Multimodal RAG

Tags:multimodal-rag,easiest-mcp-rag,gemini-rag,hosted-rag-mcp,mcp-server

Instead of wiring each agent or workflow to a one-off document search stack, use this MCP as the agent-facing entry point to one reusable answer layer: upload source material once, retrieve across text and visual content, and return answers with evidence users can verify.

# One-liner with npx npx -y @calypsohq/multimodal-rag-mcp-server --api-key "sk-your-key-here"

CALYPSO_API_KEY="sk-..." npx -y @calypsohq/multimodal-rag-mcp-server

Then add the same command to Claude Desktop, Cursor, or Smithery using the configuration examples below.

Calypso is built for teams that want the easiest hosted multimodal RAG MCP server: no local vector stack, no Docker compose, and no custom OCR or image-processing pipeline before agents can ask grounded questions.

Start here if you want the easiest hosted multimodal RAG MCP server.

- Production multimodal RAG agent with multi-turn memory
- Built-in upload tools for single files and batch knowledge uploads
- Automatic discovery of your team's RAG variants and knowledge buckets
- Verifiable citations with source references and retrieval metadata
- Read-only resources and reusable prompts for safe workflows

Most company knowledge is not only text. The answer often lives across a setup screenshot, a PDF table, a product diagram, a help-center page, or a chart inside a report. Calypso packages that full knowledge surface into a single retrieval layer so agents can ask grounded questions without guessing from generic model memory.

- Search the formats users actually rely on: documentation, PDFs, screenshots, charts, diagrams, product images, support articles, manuals, policies, FAQs, and reports.
- Ground answers before the model writes: Gemini File Search retrieves relevant text and visual context first, then the RAG agent answers from that source material.
- Show the evidence trail: responses can include source references, page-aware grounding, and retrieval metadata so people can verify before they trust.
- Scope retrieval with metadata: use workspace, team, customer, language, file type, status, or other metadata to keep answers relevant without duplicating knowledge bases.
- Reuse the same knowledge layer everywhere: connect Cursor, Claude Desktop, AI agents, n8n workflows, product UI, support flows, and website experiences to the same source-backed layer.

In practice, this means your agent can answer questions like:

- "Explain this setup screenshot and the attached policy PDF. What should the support rep do next?"
- "What does this onboarding PDF say about approval rules?"
- "Why is this setup screen failing?"
- "Compare the pricing chart with our plan documentation and recommend the right tier."
- "Summarize the policy that applies to this support ticket."
- "Which product plan fits this customer based on our pricing docs?"
- "Compare the diagrammed ingestion flow with the retrieval flow."

- Ask grounded questions against the configured Calypso knowledge base
- Select any discovered team RAG variant with the optionalmodelargument
- Continue a multi-turn conversation via the native/v1/responsesconversation model
- Reset the conversation context with/new
- Use the same OpenAI-compatible Responses endpoint that servescalypso-rag-agent
- Discover built-in resources and prompts for the supported Calypso workflows

- Node.js 18+
- The server uses Web Fetch API primitives (fetch,Headers,Request,Response,FormData,Blob, andFile) for API calls and SDK compatibility. The package bootstraps missing globals at startup for MCP runtimes that expose only part of the Node 18+ Web API surface.
- A Calypso API endpoint that exposes:

- POST /v1/responses
- GET /v1/rag-agent/models
- GET /v1/knowledge/buckets
- POST /v1/knowledge/files/upload-session
- POST /v1/knowledge/files/upload-session/{session_id}/finalize
- POST /v1/knowledge/files:batch/upload-session
- POST /v1/knowledge/files:batch/upload-session/{batch_id}/finalize
- GET /v1/knowledge/batches/{batch_id}

- CALYPSO_API_KEY(required)
- CALYPSO_API_BASE_URL(optional, defaulthttps://api.calypso.so/v1)
- CLI flags / Smithery-provided command arguments
- Environment variables
- Default base URL (https://api.calypso.so/v1)

npx -y @calypsohq/multimodal-rag-mcp-server --api-key "sk-..."

env CALYPSO_API_KEY="sk-..." CALYPSO_API_BASE_URL="https://api.calypso.so/v1" npx -y @calypsohq/multimodal-rag-mcp-server

Add a new MCP server (command type) like:

npx -y @calypsohq/multimodal-rag-mcp-server --api-key sk-... --api-base-url https://api.calypso.so/v1

Claude -> Settings -> Developer -> Edit Config

~/Library/Application Support/Claude/claude_desktop_config.json

Claude Desktop usesclaude_desktop_config.jsonfor desktop MCP servers. Claude Code uses separate config locations such as~/.claude.jsonor project-level.mcp.json.

Paste this intoclaude_desktop_config.json:

- calypsoApiKey(required)
- calypsoApiBaseUrl(optional, defaults tohttps://api.calypso.so/v1)

The Smithery launch path is equivalent to:

npx -y @calypsohq/multimodal-rag-mcp-server --api-key sk-... --api-base-url https://api.calypso.so/v1

UsecalypsoApiBaseUrlonly when targeting a self-hosted Calypso-compatible deployment. The cloud default does not need an override.

- Missing API key: provide--api-keyorCALYPSO_API_KEY
- Wrong API host: make sure--api-base-url/CALYPSO_API_BASE_URLends in/v1
- Self-hosted deployment: only override the base URL if you are not usinghttps://api.calypso.so/v1
- Smithery launch mismatch: use the packagednpx -y @calypsohq/multimodal-rag-mcp-serverpath instead of runningnode dist/index.jsfrom a fresh clone
- FormData is not definedorHeaders is not defined: upgrade to the latest package. The MCP server bootstraps missing Web Fetch API globals before upload tools run.
- ENOENT for/mnt/user-data/uploads/...: that path belongs to a hosted agent or attachment sandbox, not necessarily to the MCP server. Retry withcontentBase64instead offilePath.
- Local artifact saving: Calypso answers can be used with Claude Desktop's filesystem MCP server, but Claude must be explicitly asked to save the final output to an allowed local path.

UsefilePathfor local MCP installs, including Claude Desktop and Cursor configs that launch this package with a local command such as:

npx -y @calypsohq/multimodal-rag-mcp-server

In that setup, the MCP server process runs on your machine and can read paths available to the same user account. Pass the local path directly; the server reads raw bytes and uploads them through the Calypso upload-session URL. You do not need to base64-encode local files.

UsecontentBase64for hosted or remote MCP clients, including Smithery-hosted servers, browser/cloud runtimes, generated in-memory content, and agent containers. In those environments, a path on your Mac or workstation is not readable by the MCP server process.

If a path starts with/mnt/user-data/uploads,/mnt/data,/mnt/attachments, or another hosted sandbox prefix, do not send it asfilePathunless this MCP server is running in that same sandbox. UsecontentBase64or another inline byte source instead.

- It does not auto-route to other personas or agents.
- It automatically discovers the API key's availablecalypso-rag-agentvariants at startup.
- Use the optionalmodelargument to choose a named variant such ascalypso-rag-agent:pricing.
- Each model variant keeps its own MCP conversation chain, so switching variants does not continue the wrong thread.
- It usesPOST /v1/responsesinstead ofPOST /v1/chat/completions.
- First turns create a named conversation, and follow-up turns chain withprevious_response_id.
- OptionalfileIdsare supported for retrieval-scoped questions. New uploads should use the durable knowledge upload tools and wait for indexing before asking.
- Use/newas the prompt to reset the MCP conversation.

Lists buckets for the team tied to the configured Calypso API key.

- UsesGET /v1/knowledge/buckets.
- Does not acceptteam_id; Calypso derives team scope from the API key.
- Returns bucket ids, slugs, names, status, member counts, source counts, and bucket-store readiness.
- Defaults to active buckets only. PassincludeArchived: truewhen you need archived buckets for audits or cleanup.
- Use this beforecalypso-upload-fileorcalypso-upload-files-batchwhen you need to choose a destination bucket.
- calypso://rag-agent-modelsanswers which buckets are bound to each RAG variant.calypso-list-bucketsanswers which buckets exist for the API key's team.

Fetches one knowledge file's metadata and indexing status by file id.

- UsesGET /v1/knowledge/files/{file_id}.
- Resolves the opaquefileIdsfromcalypso-list-bucketsinto filename, mime type, size, indexing status, and per-bucket sync state.
- Passverify: trueto cross-check against the provider (?verify=gemini) — slower, but authoritative when a status looks stale.

{ "fileId": "file_123", "verify": true }

Creates an empty bucket for the create-then-fill workflow.

- UsesPOST /v1/knowledge/buckets(requires a backend with this endpoint deployed and theknowledge:bucket:createcapability on the API key).
- The server normalizes the slug; collisions return a typedbucket_slug_existserror.
- idempotencyKeyreplays return the existing bucket instead of erroring.
- Uploads can still create buckets implicitly viabucketSlugs+createMissingBuckets; this tool exists for creating the destination first.

{ "name": "Product docs", "slug": "product-docs", "idempotencyKey": "create-product-docs-1" }

Creates a RAG agent variant bound to one or more buckets.

- UsesPOST /v1/rag-agent/agents(requires a backend with this endpoint deployed and therag:agent:createcapability on the API key).
- The success payload leads withmodel— pass it straight tocalypso-rag-agent.
- Bucket bindings are validated server-side: unknown or archived buckets returnbucket_not_found; agent id collisions returnagent_id_exists; the plan's agent cap returnsagent_limit_reached.
- Creating an agent refreshes the model catalog and emits MCPtools/list_changed+resources/list_changednotifications, so the new variant is usable without reconnecting.