Ai 測試大師 / Mcp Test Runner

AModel Context Protocolserver that lets Claude Desktop / Cursor / any MCP client drive your test suite end-to-end: run tests, inspect failures (screenshot + video + trace), analyze a live URL to draft test cases, and — after each run — produce a prioritized action plan telling you exactly what to fix or write next.

-

Run testsacross multiple frameworks (web + mobile + API) via a single MCP surface

Mobile via Maestro(since v0.3.0): same MCP tools, iOS Simulator / Android Emulator / real device; YAML flows; cross-platform without rewrites

Native API testing — two runners(since v0.6.0 / v0.6.1): two peers now share the API testing slot, each fed by the artifact your team already maintains.

- Schemathesis(QA_RUNNER=schemathesis, since v0.6.0): point at an OpenAPI 3.x / Swagger 2.0 URL orfile://schema and get property-based fuzzed tests covering status codes, response schemas, content types, and5xx-on-fuzz violations.
- Newman(QA_RUNNER=newman, since v0.6.1): point at an exported Postman 2.x collection (plus optional environment / globals files) and Newman replays every request, runs the embeddedpm.test(...)assertions, and returns one mk-qa-master nodeid per assertion. Newman is asystem prerequisite(npm install -g newman) — it's an npm package, not pip, so it doesn't ship as a Python extra.

Both drop into the same MCP tool surface as the web / mobile runners, and both feed the samereport.json/ history / flake / optimizer pipeline. Existing API tests written in pytest+httpx, Jest+supertest, Cypresscy.request(), or Gonet/http/httpteststill ride their existing runners — no migration needed. Pact provider verification stays on the v0.7.0 conditional roadmap.

Failure artifacts: screenshot (base64-inlined), video, Playwright trace.zip / Maestro recordings

Run history: every run snapshotted; HTML report shows a sparkline trend

DOM / Screen analyzer—analyze_urlfor web (forms / nav / dialogs / CTAs + the API endpoints the page hits) andanalyze_screenfor mobile (maestro hierarchy→ form / cta / tab_bar modules)

Smart test generation(generate_test): hand it an analyzer module and it writes a runnable Playwright.pyor Maestro.yamlwith concrete selectors, not# TODOstubs

Auto-retry flakes— pytest side viapytest-rerunfailures; Maestro side via custom retry wrapper (no native--reruns); flaky tests surfaced separately from real failures

Self-improvement coach(get_optimization_plan): post-run analysis across three lenses — suite quality, MCP usability, AI generation effectiveness

JUnit XML outputfor CI integrations (GitHub Actions / Jenkins / GitLab)

Two paths — pick the one that matches how you'll use it.

A. Run viauvx(zero install, recommended for end users)

"env": { "QA_RUNNER": "schemathesis", "QA_OPENAPI_URL": "https://api.example.com/openapi.json" }

StandardQA_TIMEOUT_SECONDSstill applies (default 600s).

Point the runner at any exported Postman 2.x collection and Newman 6.x replays every request, runs the embeddedpm.test(...)assertions, and returns one mk-qa-master "test" per assertion. Results flow through the samereport.json/ history / flake / optimizer pipeline as the Schemathesis and UI runners.

System prerequisite: Newman ships via npm, not pip. Install once:

"env": { "QA_RUNNER": "newman", "QA_POSTMAN_COLLECTION": "/absolute/path/to/your-collection.json" }

StandardQA_TIMEOUT_SECONDSstill applies (default 600s).

When backend bypass isn't an option: Claude looks at the CAPTCHA, mk-qa-master does the clicks.

Supports reCAPTCHA v2 (since v0.7.0) and hCaptcha (since v0.7.1).

The first capability in the family where the AI client's vision is load-bearing, not optional. Two new MCP tools (inspect_visual_challenge+solve_visual_challenge) detect a reCAPTCHA v2 or hCaptcha image-grid challenge on the active Playwright page, screenshot it for the multimodal AI client, accept the tile-selection the AI returns, and execute the click chain. The runner is the eyes and hands; the AI client (Claude / Cursor / Gemini / GPT-4o) is the actual solver.

The built-in QA knowledge layer (get_qa_context section="CAPTCHA") codifies three tiers. Reach for them in order:

The solver does nothing until you explicitly opt in. Two env vars drive it:

"env": { "QA_RUNNER": "pytest", "QA_PROJECT_ROOT": "/path/to/project", "QA_VISUAL_CHALLENGE_CONSENT": "true", "QA_VISUAL_CHALLENGE_AUTHORIZED_DOMAINS": "client-staging.example.com" }

Then, when arun_testscall surfaces anexternal_dependencyfailure that points at a CAPTCHA, the AI client can escalate:

Regardless of consent or allowlist, the solver refuses to operate on known third-party identity providers (accounts.google.com,login.microsoftonline.com,id.apple.com,facebook.com,login.live.com, etc.). No legitimate QA scenario justifies a CAPTCHA solver against someone else's login portal.

No screenshot retention beyond the active inspect→solve cycle. Telemetry logs the boolean outcome only — never the screenshot, never the challenge text, never the tile selection. The 5-minute LRU cache holds at most 10 outstanding challenges per process and never touches disk.

The AI client's vision model does the actual judging — Claude Sonnet 4, GPT-4o, and Gemini 2.5 all ship with native vision but their accuracy on a 3x3 reCAPTCHA varies. Plan for at least one retry per challenge (reCAPTCHA gives you three before locking out).get_telemetrywill eventually surface aggregate pass-rate so you can size that expectation per-client.

Scope: reCAPTCHA v2 image-grid only in v0.7.0. hCaptcha lands in v0.7.1. reCAPTCHA v3 / Cloudflare Turnstile are permanently out of scope — they don't surface a visible challenge to inspect.

Schemathesis catches correctness drift. v0.8.0 adds the layer that catches the security drift hiding behind a passing schema.

v0.8.0 ships anOWASP API Security Top 10 (2023) rule-based scanneras a new MCP tool:run_api_security_scan. It loads an OpenAPI 3.x spec, walks each (path × method), and dispatches five purely-HTTP- observable rules:

Mirrors the v0.7 visual-challenge consent model:

The scanner runs adversarial test cases. Do not point it at production systems you don't own, and do not point it at any system where you don't have authorization. The two env vars above are the contract.

Use as a Claude Code / Codex / Hermes / OpenClaw skill (v0.9.0)

v0.9.0 packages mk-qa-master as across-host agent skillin addition to its MCP-server form. Theskills/mk-qa-master/folder is the single source of truth — the sameSKILL.md, slash commands, and reference docs load into:

- Claude Code— via.claude-plugin/plugin.json(this repo is a plugin marketplace).
- OpenAI Codex— via.codex-plugin/plugin.json(Codex reads Claude- style marketplaces).
- OpenClaw— install from local checkout:openclaw plugins install /path/to/mk-qa-master.
- Hermes Agent— symlink the skill folder into~/.hermes/skills/.

# Inside Claude Code: /plugin marketplace add kao273183/mk-qa-master /plugin install mk-qa-master@mk-qa-master

Restart Claude Code so the skill registers. Then any QA testing prompt auto-activates the skill — or explicitly invoke a slash command:

/mk-qa-master:run-tests login /mk-qa-master:generate https://staging.example.com /mk-qa-master:api-security https://api.staging.example.com/openapi.yaml

The skill is a single-file operating contract that teaches the host how to drive mk-qa-master's 22 MCP tools coherently. It encodes:

- When to auto-activate— phrases like "run my tests", "why did this test fail", "scan this API for OWASP issues" trigger it.
- Five flows— run tests / generate tests / debug failures / solve CAPTCHAs / scan APIs.
- Hard rules— surface consent errors verbatim, don't silently re-run with relaxed filters, confirm before destructive runs.

22 tools. Frozen schema. Versioned drift. Pin and go.

mk-qa-master shipped v1.0.0 on 2026-06-02. The MCP tool surface is locked:22 tools, the consent gate env vars, the plan / bookend shapes, and the hard-stop blacklistsdon't change without a deprecation cycle.

Breaking changes require a v2.0 bump. Deprecations get≥ 1 minor of warningwithDeprecationWarningraised at runtime, "Deprecated:" in the MCP tool description, and an entry indocs/MIGRATION-1.x-to-2.0.md(created when v2.0 work opens).

A CI snapshot test (tests/test_v1_schema_snapshot.py) freezes the 22-tool surface intests/snapshots/v1/tool_surface.json. Any drift fails CI unless the PR setsBREAKING_CHANGE_ACK=trueAND bothdocs/MIGRATION-0.x-to-1.0.mdanddocs/DEPRECATION-POLICY.mdexist. The ack alone isn't a free pass — the docs must be in place.

A second test (tests/test_v1_doc_sync.py) scans every public doc for tool-count claims and fails if any disagree with the live server.