verify-action-mcp

A small post-action verification service for AI agent tool calls. Submit (claim, evidence), get back a 4-value verdict and a tamper-evident hosted receipt that downstream agents or CI steps can use to continue, block, or escalate.

> 🇯🇵 日本語版は ↓ ページ後半 を参照してください。

---

English

Problem / Use case / Decision

Problem. AI agents commonly claim an action succeeded when the underlying state did not change consistently with the claim.

Use case. Before a downstream agent, CI job, or deployment step trusts an action, call verify_action against the supplied evidence.

Decision (per receipt).

| aar_verdict | Recommended action |
|---|---|
| verified | continue |
| contradicted | block |
| insufficient_evidence | request more evidence |
| unsafe_to_verify | escalate to human |

This service does not independently access your database, code repository, or APIs. It checks whether supplied evidence is internally consistent with a supplied claim, then issues a tamper-evident receipt under its key. The trust boundary is "agent-produced artifacts as evidence" — strongest for code_diff (the diff is itself the evidence), weaker for evidence shapes that depend on the caller faithfully reporting external state.

Why

These silent successes don't show up in benchmarks (which score "did the model say it succeeded?"). They surface when something downstream breaks — sometimes hours or days later. Typical patterns:

- "I added a null check for user.email" — but the diff also rewrote 5 unrelated functions. (code_diff — primary)
- "I deleted user 12345" — but the affected_rows field or SQL operation actually targeted id 99999. (db_op — experimental)
- "I posted the webhook update successfully" — but the API returned HTTP 503 with an error body. (api_call — experimental)

verify-action-mcp runs after the agent has done the work, with the artifacts. Existing pre-action policy admission control products from major vendors operate on a different lane.

Quick start

MCP (Claude Code, Cursor, Cline, Codex, etc.)

// claude_desktop_config.json or your harness's MCP config
{
  "mcpServers": {
    "verify-action": {
      "transport": {"type": "http", "url": "https://verify.armadalab.dev/mcp"}
    }
  }
}

The agent now has a verify_action tool available. It can self-call before reporting completion, or you can invoke it from your harness logic.

REST

curl -X POST https://verify.armadalab.dev/verify \
  -H 'Content-Type: application/json' \
  -d '{
    "claim": "Added null check for user.email in src/user.py",
    "evidence": {
      "diff": "--- a/src/user.py\n+++ b/src/user.py\n@@ -10,3 +10,5 @@\n def get_email(user):\n+    if user.email is None:\n+        return None\n     return user.email"
    }
  }'

Response (receipt truncated; full shape below):

{
  "verdict": "ok",
  "aar_verdict": "verified",
  "reasoning": "Coherent: claim references 1/1 paths actually in diff; claim implies addition/modification; diff added 2 lines; 2/2 identifier(s) present in diff",
  "confidence": 0.8,
  "verifier_used": "code_diff_v1",
  "kind_dispatched": "code_diff",
  "receipt": {
    "schema": "verify_action_receipt.v0",
    "verdict": "verified",
    "claim_hash": "sha256:<64-hex>",
    "evidence_manifest_hash": "sha256:<64-hex>",
    "kid": "v0-default",
    "issued_by": "aar:reference-impl@v0",
    "signature": "hmac-sha256:<base64>",
    "_full": "(see Receipts section)"
  }
}

Self-host

git clone https://github.com/Armada735/verify-action-mcp
cd verify-action-mcp
./start.sh   # binds 127.0.0.1:8092
./stop.sh

Pure Python stdlib. No pip install. Tested on Linux.

What it verifies

A dispatcher routes by kind (or auto-infers from evidence shape):

| Kind | Status | Evidence shape | Critical signal that forces contradicted |
|---|---|---|---|
| code_diff | primary | {diff: "<unified diff>"} | All claimed paths absent from diff |
| db_op | experimental | {before_count, after_count, operation, affected_rows} | Claim ID not in SQL ID |
| file_op | experimental | {path, exists_before, exists_after, line_count?, size_bytes?} | Numeric divergence > 50% or > 50 absolute |
| api_call | experimental | {request, response_status, response_body} | HTTP failure status / error-body under success claim |
| generic | experimental | any object | (conservative; usually returns insufficient_evidence) |

code_diff is the v0 primary integration target — the agent itself produces the diff that is the evidence, so the trust boundary is clean. The other kinds are useful but rely on the caller to construct a faithful evidence object describing external state this service does not independently observe.

Each verifier looks at:
- Verb in claim ↔ direction of state change (delete = -1, insert = +1, update = 0)
- Specific identifiers / paths / emails / URLs
- Counts / line counts / sizes
- HTTP status semantics
- "Critical signals" that force mismatch regardless of pos/neg balance

Verdicts (dual format)

| Field | Values | Notes |
|---|---|---|
| aar_verdict | verified / contradicted / insufficient_evidence / unsafe_to_verify | 4-value canonical (verify_action_receipt.v0) |
| verdict | ok / mismatch / uncertain | 3-value legacy alias for backwards compatibility |

unsafe_to_verify is returned when the verifier itself raised an exception (cannot examine evidence) — distinct from insufficient_evidence (evidence examined, ambiguous).

Receipts (verify_action_receipt.v0)

Every /verify call also issues an HMAC-SHA256-attested receipt as a nested receipt field. Full shape:

| Field | Type | Description |
|---|---|---|
| schema | string | "verify_action_receipt.v0" |
| kid | string | Key id; v0 ships with "v0-default". Operators rotate keys with fresh kids. |
| issued_by | string | Issuer identifier (this reference impl: "aar:reference-impl@v0") |
| issued_at | string | RFC 3339 UTC timestamp |
| verifier_id | string | "verify-action-mcp@<version>" |
| verifier_method | string | "rule_based.<kind>" (e.g. rule_based.db_op) |
| claim_hash | string | "sha256:<64-hex>" — content-addressed; raw claim is not stored |
| evidence_manifest_hash | string | "sha256:<64-hex>" — same |
| verdict | string | One of the 4 aar_verdict values |
| confidence | number | 0–1 |
| reason_codes | array of strings | Free-form diagnostic codes (v0 unrestricted) |
| policy_or_oracle_refs | array of strings | Optional refs to policy / oracle inputs (usually []) |
| caller_context | object | Optional caller_context echoed back (max 8 keys, 64-char strings) |
| signature | string | "hmac-sha256:<base64-no-padding>" |

What the receipt asserts: that this specific service issued this specific verdict for this content-addressed (claim, evidence) pair at this time, signed under a known key id (kid).

What the receipt does NOT assert: factual truth of the claim, legal admissibility in any forum, or warranty of any kind.

Trust model in v0: HMAC is symmetric — the receipt verifies that a private key under our control signed it. It is not a third-party attestation in the cryptographic sense, and you cannot today hand a receipt to a third party and have them verify it without involving this service. Treat v0 receipts as a tamper-evident hosted log entry from this service. Ed25519 public-key signed receipts ship within 30 days (see Roadmap); schema upgrade path for v1 (asymmetric, multi-issuer) is documented in aar/SCHEMA_UPGRADES.md.

API

| Method | Path | Purpose |
|---|---|---|
| GET | / /about | Project description (HTML) |
| GET | /healthcheck | Liveness probe |
| GET | /spec | Tool schema + verifier kinds (JSON) |
| GET | /stats | Aggregate counters since process start |
| GET | /privacy | Privacy notice (HTML) |
| GET | /tos | Terms of service (HTML) |
| GET | /feedback | Feedback endpoint description (HTML) |
| POST | /verify | REST: {claim, evidence, kind?, context?, caller_context?} → verdict + receipt |
| POST | /mcp | MCP JSON-RPC 2.0 endpoint |
| POST | /feedback | Anonymous free-form feedback: {message, category?, harness?, trace_ref?} |

MCP methods

- initialize → {protocolVersion: "2024-11-05", capabilities: {tools: {}}, serverInfo: {name, version}}
- tools/list → {tools: [{name: "verify_action", description, inputSchema}]}
- tools/call (name=verify_action) → {content: [...], isError, _structured_result: {verdict, aar_verdict, reasoning, confidence, receipt, ...}}
- notifications/initialized, ping → empty result

Examples

code_diff — coherent

```bash
curl -X POST https://verify.armadalab.dev/verify -H 'Content-Type: application/json' -d '{
"claim": "Added null check for user.email in src/user.py",
"evidence": {
"diff": "--- a/src/user.py\n+++ b/src/user.py\n@@ -10,3 +10,5 @@\n def get_email(user):\n+ if user.email is None:\n+ return None\n return user.email"
}
}'