🦉 Trustwise MCP Server
About
Advanced evaluation tools for AI safety, alignment, and performance using the Trustwise API.
Details
- Author
- trustwiseai
- GitHub stars
- 4
- Downloads
- 141
- Categories
- Developer Tools, Other
Jump to
- Provides 19 evaluation metrics covering safety, alignment, and performance.
- Detects PII, prompt injection, toxicity, and refusal in responses.
- Measures faithfulness, relevancy, clarity, tone, and formality.
- Estimates carbon footprint and cost of model inference.
- Integrates with MCP clients like Claude Desktop and Cursor.
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
🦉 Trustwise MCP ServerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Install via Docker, then configure your MCP client (e.g., Claude Desktop or Cursor) with the env variables TW_API_KEY (required) and optionally TW_BASE_URL for a custom Trustwise instance. Use the provided JSON snippet to launch the container with the command docker run -i --rm -e TW_API_KEY ghcr.io/trustwiseai/trustwise-mcp-server:latest.
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"\ud83e\udd89 trustwise mcp server": {
"trustwise": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TW_API_KEY",
"ghcr.io/trustwiseai/trustwise-mcp-server:latest"
],
"env": {
"TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
}
}
}
}
}McpServers
{
"trustwise": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TW_API_KEY",
"ghcr.io/trustwiseai/trustwise-mcp-server:latest"
],
"env": {
"TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
}
}
}🦉 Trustwise MCP Server
The Trustwise MCP Server is a Model Context Protocol (MCP) server that provides a suite of advanced evaluation tools for AI safety, alignment, and performance. It enables developers and AI tools to programmatically assess the quality, safety, and cost of LLM outputs using Trustwise's industry-leading metrics.
💡 Use Cases
- Evaluating the safety and reliability of LLM responses.
- Measuring alignment, clarity, and helpfulness of AI-generated content.
- Estimating the carbon footprint and cost of model inference.
- Integrating robust evaluation into AI pipelines, agents, or orchestration frameworks.
🛠️ Prerequisites
- A Trustwise API Key (get one here)
- Docker; Follow the install instructions
📦 Installation & Running
Claude Desktop
To connect the Trustwise MCP Server to Claude Desktop, add the following configuration to your Claude Desktop settings:
{
"mcpServers": {
"trustwise": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TW_API_KEY",
"ghcr.io/trustwiseai/trustwise-mcp-server:latest"
],
"env": {
"TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
}
}
}
}
To point to a specific Trustwise Instance - under env, also set the following optional environment variable:
TW_BASE_URL: "<YOUR_TRUSTWISE_INSTANCE_URL>"
e.g "TW_BASE_URL": "https://api.yourdomain.ai"
Cursor
To connect the Trustwise MCP Server to cursor, add the following configuration to your cursor settings:
{
"mcpServers": {
"trustwise": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"-e",
"TW_API_KEY",
"-e",
"TW_BASE_URL",
"ghcr.io/trustwiseai/trustwise-mcp-server:latest"
],
"env": {
"TW_API_KEY": "<YOUR_TRUSTWISE_API_KEY>"
}
}
}
}
Replace <YOUR_TRUSTWISE_API_KEY> with your actual Trustwise API key.
🧰 Tools
The Trustwise MCP Server exposes the following tools (metrics). Each tool can be called with the specified arguments to evaluate a model response.
🛡️ Trustwise Metrics
| Tool Name | Description |
|--------------------------|-----------------------------------------------------------|
| faithfulness_metric | Evaluate the faithfulness of a response to its context |
| answer_relevancy_metric| Evaluate relevancy of a response to the query |
| context_relevancy_metric| Evaluate relevancy of context to the query |
| pii_metric | Detect PII in a response |
| prompt_injection_metric| Detect prompt injection risk |
| summarization_metric | Evaluate summarization quality |
| clarity_metric | Evaluate clarity of a response |
| formality_metric | Evaluate formality of a response |
| helpfulness_metric | Evaluate helpfulness of a response |
| sensitivity_metric | Evaluate sensitivity of a response |
| simplicity_metric | Evaluate simplicity of a response |
| tone_metric | Evaluate tone of a response |
| toxicity_metric | Evaluate toxicity of a response |
| refusal_metric | Detect refusal to answer or comply with the query |
| completion_metric | Evaluate completion of the query’s instruction |
| adherence_metric | Evaluate adherence to a given policy or instruction |
| stability_metric | Evaluate stability (consistency) of multiple responses |
| carbon_metric | Estimate carbon footprint of a response |
| cost_metric | Estimate cost of a response |
> For more examples and advanced usage, see the official Trustwise SDK.
📄 License
This project is licensed under the terms of the MIT open source license. See LICENSE for details.
🔒 Security
- Do not commit secrets or API keys.
- This repository is public; review all code and documentation for sensitive information before pushing.
---
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
