AgentTrust

Challenge-response quality verification for AI agents and MCP servers.

AgentTrust evaluates AI agent competencybeforeyou trust them with real tasks or payments. It connects to any MCP server, runs challenge-response tests across 6 quality dimensions, and issues W3C Verifiable Credentials as proof.

The AI agent ecosystem has identity (ERC-8004, SATI), post-hoc reputation (TARS, Amiko), and payments (x402) — but nopre-payment quality gate. AgentTrust fills this gap: verify competency first, then trust.

- 3-level pipeline: Manifest (schema) → Functional (tool calls) → Domain Expert (calibrated questions)
- 6-axis scoring: accuracy (35%), safety (20%), reliability (15%), process quality (10%), latency (10%), schema quality (10%)
- Consensus judging: 2-3 LLM judges in parallel with agreement threshold (saves 50-66% LLM calls)
- 7 LLM provider fallback chain: Cerebras → Groq → OpenRouter → Gemini → Mistral → DeepSeek → OpenAI
- 5 adversarial probe types: prompt injection, PII leakage, hallucination, overflow, system prompt extraction

- Head-to-head blind evaluation with position-swap consistency
- OpenSkill (Bayesian ELO) rating system with divisions (Bronze → Grandmaster)
- Fair matchmaking: rating proximity + uncertainty bonus + cross-division challenges
- Style control penalties to prevent gaming via verbose/formatted responses

- Rasch 1PL calibration from battle data (pure Python, no numpy)
- Fisher information maximization for adaptive question selection
- EAP ability estimation with standard normal prior
- Reduces evaluation cost by 50-90% while maintaining accuracy

- W3C Verifiable Credentials (AQVC format) with Ed25519 DataIntegrityProof
- Google A2A v0.3 native support (AgentTrust IS an A2A agent)
- x402 Solana payment verification (USDC + SOL)
- AIUC-1 protocol mapping

cp .env.example .env # Add at least one LLM key (GROQ_API_KEY, CEREBRAS_API_KEY, etc.) docker compose up -d

- API:http://localhost:8002
- MCP Server:http://localhost:8003
- Health:http://localhost:8002/health

python -m venv .venv && source .venv/bin/activate pip install -r requirements.txt cp .env.example .env # Add LLM keys to .env unset GROQ_API_KEY # Shell env overrides .env rotation pool python -m uvicorn src.main:app --host 0.0.0.0 --port 8002 --reload

{ "mcpServers": { "agenttrust": { "command": "python", "args": ["-m", "src.standards.mcp_server"], "env": { "GROQ_API_KEY": "your-key" } } } }

Or connect to a running instance via SSE:

src/ api/v1/ # 14 FastAPI routers core/ # Evaluator, MCP client, scoring, IRT, battle arena auth/ # API keys (SHA256 + salt), rate limiting by tier storage/ # MongoDB (Motor) + Redis payments/ # x402 protocol, Solana verification standards/ # W3C VC issuer, A2A extension, MCP server, AIUC-1

Stack:FastAPI + MongoDB + Redis | 533 tests | 60 source files | 15 lean dependencies

python -m pytest tests/ -q # 533 passed in ~2s

See.env.examplefor all 60+ configuration options including:

- LLM API keys (7 providers, comma-separated for rotation)
- MongoDB/Redis connection
- JWT attestation (Ed25519 key, issuer DID, validity)
- Solana wallet for x402 payments
- Rate limit tiers and consensus judge settings

- Architecture— Full system design (845 lines)
- Distribution Roadmap— Partner and integration plan
- A2A Agent Card— Machine-readable capabilities

AI agent security scanner — protect LLM-powered apps from prompt injection, SQL injection, data exfiltration, and adversarial attacks via MCP.