talkthrough-mcp

<!-- mcp-name: io.github.korovin-aa97/talkthrough-mcp -->

Quickstart · Tools · FAQ ·
Troubleshooting · Changelog ·
Contributing

Feedback ingestion for AI agents. Record your screen and talk; your agent
does the rest — files the bugs, writes the spec, builds the backlog.

talkthrough-mcp is a local-first MCP server that turns a narrated screen
recording (or any video/audio file) into agent-ready structured data:
timestamped transcript segments, scene-change keyframes, OCR'd on-screen text,
and wall-clock anchoring. Everything is served through lazy retrieval tools, so
a 30-minute recording never floods the model context — the agent pulls exactly
the transcript slice, moment bundle, or frame it needs.

There is no LLM inside the server and no cloud anywhere in the path: ffmpeg,
faster-whisper, and RapidOCR run on your machine, and the calling agent brings
the intelligence. What makes it different from screen-recorder SaaS and
video-analyzer MCPs: it works on arbitrary local files, it ships the agent
workflows (server prompts + example agents), and it anchors every timestamp to
wall-clock time — so "the moment I said the checkout hung" maps straight to
the right window of your server logs.

Quickstart

One command, no system dependencies: ffmpeg falls back to a bundled build,
OCR is pip-only, and whisper models download themselves on first use.

<!-- gen:install — generated by scripts/gen_integrations.py; do not hand-edit -->

Claude Code

Two install paths — pick one, not both (the plugin already includes
the server; installing both would register it twice):

Server only — the 7 tools + 5 prompts, and nothing else on your
system. Choose this for a minimal setup, or when you manage MCP servers
yourself across several clients:

claude mcp add -s user talkthrough -- uvx talkthrough-mcp

Full plugin — the same server, plus native slash commands
(/talkthrough:triage-recording, …) that handle the ceremony for you,
a ready-made triage subagent, and an agent skill that teaches Claude the
workflow. Choose this for the best out-of-the-box experience:

/plugin marketplace add korovin-aa97/talkthrough-mcp
/plugin install talkthrough@talkthrough

Every other MCP client

<details>
<summary><b>Claude Desktop</b></summary>

claude_desktop_config.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: integrations/claude-desktop/

</details>

<details>
<summary><b>Cursor</b></summary>

~/.cursor/mcp.json (or project .cursor/mcp.json):

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: integrations/cursor/

</details>

<details>
<summary><b>OpenAI Codex CLI</b></summary>

~/.codex/config.toml (or project-scoped .codex/config.toml in trusted projects):

[mcp_servers.talkthrough]
command = "uvx"
args = ["talkthrough-mcp"]

More: integrations/codex/

</details>

<details>
<summary><b>Gemini CLI</b></summary>

~/.gemini/settings.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: integrations/gemini-cli/

</details>

<details>
<summary><b>Cline / Roo Code</b></summary>

cline_mcp_settings.json (via MCP Servers UI):

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: integrations/cline/

</details>

<details>
<summary><b>OpenClaw</b></summary>

~/.openclaw/openclaw.json:

{
  "mcp": {
    "servers": {
      "talkthrough": {
        "command": "uvx",
        "args": [
          "talkthrough-mcp"
        ]
      }
    }
  }
}

More: integrations/openclaw/

</details>

<details>
<summary><b>OpenCode</b></summary>

opencode.json (project) or ~/.config/opencode/opencode.json:

{
  "mcp": {
    "talkthrough": {
      "type": "local",
      "command": [
        "uvx",
        "talkthrough-mcp"
      ],
      "enabled": true
    }
  }
}

More: integrations/opencode/

</details>

<details>
<summary><b>Goose</b></summary>

~/.config/goose/config.yaml:

extensions:
  talkthrough:
    enabled: true
    type: stdio
    cmd: uvx
    args: ["talkthrough-mcp"]

More: integrations/goose/

</details>

<details>
<summary><b>GitHub Copilot CLI</b></summary>

~/.copilot/mcp-config.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: integrations/copilot-cli/

</details>

<details>
<summary><b>Windsurf</b></summary>

~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "talkthrough": {
      "command": "uvx",
      "args": [
        "talkthrough-mcp"
      ]
    }
  }
}

More: integrations/windsurf/

</details>

<details>
<summary><b>Zed</b></summary>

settings.json (Zed):

{
  "context_servers": {
    "talkthrough": {
      "source": "custom",
      "command": {
        "path": "uvx",
        "args": [
          "talkthrough-mcp"
        ]
      }
    }
  }
}

More: integrations/zed/

</details>

Any other MCP stdio client uses the same server command: uvx talkthrough-mcp.
Per-engine folders with exactly these snippets plus verification steps live
in integrations/; agents can self-install via
llms-install.md.

<!-- /gen:install -->

Local checkout (development)

git clone https://github.com/korovin-aa97/talkthrough-mcp
claude mcp add talkthrough -- uv run --directory /path/to/talkthrough-mcp talkthrough-mcp

Then, in your agent:

> Process ~/Desktop/recording.mov and triage it — or just invoke the
> triage-recording server prompt.

Tools

| Tool | What it does |
|---|---|
| process_media(path, recorded_at?, vocabulary?, language?, model?, force?) | Ingest a video/audio file: local STT, keyframes, OCR, wall-clock. Returns a compact summary. Idempotent by content hash — re-calls are instant. |
| get_transcript(job_id, start_ms?, end_ms?, format?) | Paginated transcript as segments, text, or srt; truncation returns next_start_ms. |
| get_frames(job_id, at_ms? \| start_ms?+end_ms?, max_frames?, include_duplicates?) | Keyframe images nearest a timestamp or evenly thinned across a range (unique frames by default, max 6/call). |
| get_moment(job_id, start_ms, end_ms) | The "one remark" bundle: transcript slice + up to 3 frames + their OCR text + wall-clock range. |
| search(job_id, query) | Substring search over the transcript AND on-screen OCR text; hits carry t_ms/t_wall and frame refs. |
| extract_frame(job_id, at_ms, crop?) | Exact-timestamp full-resolution re-extract from the source video (optional crop) when keyframes miss the instant. |
| list_jobs() | Recent processed recordings with durations, wall-clock starts, and counts. |

Every tool description ships 10+ usage examples, so agents pick the right tool
without extra prompting.

Server prompts (slash commands in MCP clients)

| Prompt | Workflow |
|---|---|
| triage-recording | Narrated screencast → precise findings JSON (bug/feature/question routing, frame evidence) |
| spec-from-workshop | Recorded workshop → structured spec with quoted decisions and open questions |
| backlog-from-demo | Product demo → prioritized backlog with timestamped evidence |
| meeting-actions | Meeting audio → action items, decisions, open questions |
| correlate-with-logs | Recording remarks ↔ system logs via wall-clock windows |

The same prompts live as plain files in examples/prompts/
if your client doesn't surface MCP prompts. The findings contract used by
triage-recording is examples/output-contract.schema.json.

Works as a skill too (no MCP required)

The same workflow ships as a cross-engine Agent Skill
at .agents/skills/talkthrough/ — Claude Code,
Codex CLI ($talkthrough), Cursor, Copilot, Gemini CLI, Goose and other
SKILL.md-compatible tools read it. Agents without MCP wiring can drive the
CLI directly: talkthrough-mcp process recording.mov --json prints the
same summary the MCP tool returns, and the job store is shared either way.

Wall-clock anchoring

Every timestamped result carries both t_ms (video-relative) and t_wall
(ISO 8601 real time) once the recording start is known. Resolution ladder:

1. recorded_at parameter (agent/user override) → confidence exact
2. QuickTime com.apple.quicktime.creationdate tag, carries the local
timezone (QuickTime Player recordings; ⌘⇧5 wrote it before macOS 26) → high
3. Container creation_time tag (UTC) → medium — macOS 26+ ⌘⇧5/ReplayKit
screen recordings land here (no creationdate tag anymore); pass
recorded_at= when local-tz t_wall matters
4. File mtime minus duration (recorders finalize files at recording END) → low
5. Nothing → tools still work with relative t_ms only

Why it matters: "the upload spinner froze here" becomes a ±30 s grep window
in your server logs.

Privacy

Everything runs locally: your recordings never leave your machine, speech is
transcribed by a local whisper model, OCR is local ONNX inference, and there is
no telemetry. The only network access is one-time tool/model downloads (ffmpeg
build, whisper model, OCR models).

Languages

Narration in any of Whisper's ~99 languages works: the language is
auto-detected per recording, and the summary reports both language and
language_probability so agents can tell a confident detection from a shaky
one (silence or music at the start can fool the detector — pin it with
language="ru" and force=true when that happens).

Pick the model for your languages — per call (model= parameter, agents do
this themselves when a transcript comes back garbled) or as the server
default (TALKTHROUGH_WHISPER_MODEL):

| Model | Size | Best for |
|---|---|---|
| small (default) | 464 MB | English and major-language narration on CPU |
| large-v3-turbo | ~1.5 GB | recommended for non-English — near-large quality at near-small speed |
| medium | ~1.5 GB | conservative alternative to turbo |
| tiny / base | 75–145 MB | quick drafts, CI |
| .en variants | — | English-only, slightly faster/better for EN |

talkthrough-mcp process ~/Videos/long-session.mov   # prints the summary
talkthrough-mcp process demo.mov --json             # machine-readable
talkthrough-mcp gc --keep-days 30                   # clean the job store
talkthrough-mcp serve                               # stdio MCP server (default)

- One speaker stream. No diarization yet — "who said it" isn't tracked
(#4).
- Local files only. No URL/YouTube ingestion
(#5) — download
first.
- Keyframes + transcript, not motion analysis. A glitch between scene
changes can be invisible in the frame set; extract_frame re-checks any
instant, but frame-by-frame motion reasoning is your multimodal model's job.
- STT quality tracks the model you pick. The default small favors speed;
non-English narration wants model="large-v3-turbo" (see
Languages).
- OCR reads crisp UI text well; tiny or low-contrast print is best-effort.
- Wall-clock confidence depends on recorder metadata — worst case pass
recorded_at= (see the ladder above).
- Windows is best-effort (see above).

Why not just upload the video to a multimodal model (e.g. Gemini)?
For a short, non-sensitive clip — do that. The trade-offs appear with length
and sensitivity: an hour of screen recording costs on the order of a million
tokens per question, the file leaves your machine, and you still can't map a
remark to 14:32:07 UTC to grep your server logs. talkthrough indexes once,
locally, then answers any number of follow-ups from the index.

Why not screenpipe?
Different job. screenpipe is an always-on recorder of your machine going
forward (commercial license). It can't open the .mov a teammate or customer
just sent you. talkthrough analyzes any file it's handed — the two compose
fine.

I use Jam for bug reports — do I need this?
Keep Jam for browser bugs: console+network captured at record time is great
evidence. talkthrough covers what a browser extension can't — desktop apps,
mobile screencasts, ops incidents, meetings, any file — with no account, and
correlates with server-side* logs via wall-clock time.

Can't I just script ffmpeg + whisper myself?
Yes — that's exactly this pipeline. What you'd be rebuilding: scene-change
detection with perceptual dedup, OCR, transcript+OCR search, the wall-clock
ladder, MCP tools with embedded usage examples, five workflow prompts, and a
findings contract. One uvx command instead of an afternoon of glue.

Is it really local? What leaves my machine?
Nothing at runtime. The network is used only for one-time downloads (ffmpeg
build, whisper/OCR models). No telemetry. See Privacy — and
SECURITY.md treats a violation of this promise as a
vulnerability.

For agents & tooling

Machine-readable entry points, so AI agents can install and use this server
without a human reading docs:

- llms-install.md — step-by-step install instructions for agents
- llms.txt — index of the documentation
- .agents/skills/talkthrough/SKILL.md — an Agent Skill teaching the tool workflow; discovered automatically inside a checkout by Codex CLI ($talkthrough) and readable by Claude Code, Cursor, Copilot, Gemini CLI and other SKILL.md-compatible tools
- AGENTS.md — instructions for coding agents contributing to this repo
- server.json — MCP registry manifest
- integrations/ — per-engine adapters, all generated from one source of truth and drift-tested (incl. the Claude Code plugin under integrations/claude-code/)

Roadmap (not in v1)

URL/YouTube ingestion · speaker diarization · cloud STT · embeddings/semantic
search · hosted/remote mode · .mcpb bundle · whisper.cpp backend

License

MIT