Okareo
About
Simulate and evaluate voice & text agents from your editor. Generate synthetic-user "Drivers" from your codebase, run multi-turn simulations against your agents, and pull back transcripts, traces, and scores without leaving your AI assistant.
Details
- Author
- okareo-ai
- Downloads
- 306
- Categories
- Other, Developer Tools, AI
Jump to
- Generate synthetic users (Drivers) from your agent's system prompt and tools.
- Run voice and text simulations in parallel across multiple languages.
- Replay production failures as reusable test Drivers.
- Run judge, symbolic, and audio-based evaluation checks.
- Re-score past runs without re-running the model.
- Query analytics and pin dashboards.
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
OkareoCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Point your MCP editor at the hosted endpoint https://tools.okareo.com/mcp. For Claude Code, run claude mcp add --transport http okareo https://tools.okareo.com/mcp. For any MCP client, add the server URL to your configuration JSON. Authentication uses OAuth on first connect; if OAuth discovery is not supported, set the OKAREO_API_KEY environment variable for a Bearer-header fallback.
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"okareo": {
"okareo": {
"type": "http",
"url": "https://tools.okareo.com/mcp"
}
}
}
}McpServers
{
"okareo": {
"type": "http",
"url": "https://tools.okareo.com/mcp"
}
}Okareo
> Agent simulation and evaluation for voice and text agents — run dozens of synthetic conversations against your agent from inside Claude Code, Cursor, or any MCP-ready editor. The Okareo MCP server turns your AI coding assistant into an evaluation and testing workbench. Instead of writing SDK scripts or switching to a dashboard, you describe what you want in natural language and your copilot does the rest: build synthetic users from your codebase, run multi-turn simulations against your agent, and pull back transcripts, traces, and scores for analysis. Okareo is hosted — there's nothing to install and nothing to run locally. Point your editor at the endpoint, sign in once through your browser, and the tools appear. ``
https://tools.okareo.com/mcp
`
What you can do from a prompt
- Generate Drivers from your code. Okareo reads your agent's system prompt, tools, and policy logic and proposes synthetic users (Drivers) that stress the parts most likely to break — not generic personas.
- Run voice and text simulations in parallel. Spin up dozens of synthetic callers across languages, run them against your target, and get results back in moments.
- Replay production failures as tests. Pull a failed call from a connected voice integration, turn it into a reusable Driver, and add it to your suite.
- Evaluate and re-score. Run judge, symbolic, and audio-based checks; re-score a past run against a new check without re-running the model.
- Query analytics. Trend pass rates across models and pin dashboards.
Why Okareo
- Voice depth. Synthetic callers across many languages, with the noise, crosstalk, and clipping real customers call from.
- One timeline, three views. Simulation transcript, observability trace, and audio land together — so Conversation Designers and Engineers debug the same call.
- Agent-first, multi-turn-first. The unit of evaluation is a full conversation, not a single prompt/response pair.
Supported editors
Claude Code · Claude Desktop · Cursor · VS Code 1.101+ · Cline · Gemini Code Assist · GitHub Copilot · Windsurf — and any client that speaks MCP's streamable-HTTP transport.
Connect
Claude Code (one line):
`
claude mcp add --transport http okareo https://tools.okareo.com/mcp
`
Any MCP client (config):
`json
{
"mcpServers": {
"okareo": {
"type": "http",
"url": "https://tools.okareo.com/mcp"
}
}
}
`
Authentication
OAuth on first connect — a browser tab opens to app.okareo.com, you sign in and consent, and your editor stores the token. If your client doesn't yet implement MCP OAuth discovery, set OKAREO_API_KEY in your environment and use the Bearer-header fallback (see the docs).
Example prompts
- "Run my refund scenarios against the voice agent in Spanish and Portuguese. Show me what regressed."
- "Pull this week's calls where the language switch failed and turn them into Drivers."
- "Re-score last night's evaluation against my new pii_leak check without re-running the model."
- "Read my agent in src/ and create 12 Drivers that stress the policy boundaries, then run them."
Tools
- Scenarios: save_scenario, list_scenarios, get_scenario, create_scenario_version
- Drivers: create_or_update_driver, list_driver_voices, get_driver, list_drivers
- Targets: create_or_update_target, get_target, list_targets
- Simulations: run_simulation, list_simulations
- Tests & results: run_test, list_test_runs, get_test_run_results, get_conversation_transcript, reevaluate_test_run
- Checks: create_or_update_check, generate_check
- Voice: ingest_conversations, connect_voice_integration, get_voice_webhook_url
- Models & analytics: list_available_llms, register_generation_model, query_analytics, save_dashboard`
Links
- Docs: https://docs.okareo.com/mcp/introduction - Configuration: https://docs.okareo.com/mcp/configuration - Website: https://okareo.com/mcp - GitHub: https://github.com/okareo-ai/okareo-mcp - Python SDK: https://pypi.org/project/okareo/ - TypeScript SDK: https://www.npmjs.com/package/okareo-ts-sdkSign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
