Squish

Agents don't consume videos — they navigate them.Real run: a scene cut pinned to0.2 sby retrieving34 frames — not 3,088(overview → zoom → zoom). Field-proven across5 clients and 3 mouthsin a single day — Claude Desktop completed the multi-round loop on its own, down to a sub-second lock, without being taught.

The demo is the primitive.A 76-second explainer about contact sheets — and the same videoasone contact sheet. One needs a play button; the other you just read:

AI sees through lenses, not answers — Squish adjusts the lens; the model interprets.Video is continuous; reasoning is sparse. Most questions touch a tiny fraction of the timeline. Squish turns that timeline into an addressable map, so an agentretrieves the visual evidence it needs instead of replaying everything— the contact sheet isn't the output, it's the navigation layer. Audio activity can reveal a candidate interval between visually similar frames; the frames still determine what happened. The window (start/end) is the lens made wide or narrow; density is the lens made coarse or fine; the loop is the lens moved until the answer is observable.

npm install -g @getsquish/squish # or one-shot: npx -y @getsquish/squish <video>

Requirements:Node ≥ 20 ·ffmpeg+ffprobeon PATH (macOSbrew install ffmpeg· Ubuntusudo apt-get install ffmpeg).

Bring a clip whose answer you already know. Ask AI to find one specific momentwithout giving it the original video:
- Runnpx -y @getsquish/squish clip.mov --json.
- Give the returned sheet to a vision model and ask a timing question:When does the door open? When does an object first appear? Where is the unusual audio activity, and what do the nearby frames show?
- Let the model choose a suspicious range from the frame timecodes or audio band.
- Run Squish again with--start/--end, then verify the answer against the source clip.

The index proposes; the zoomed visual evidence confirms. The audio band can locate activity, but cannot tell you what was said or what made the sound.

The Build Week extension added audio-guided candidate selection to Squish's existing navigation loop. Before the event, Squish already produced timestamped contact sheets and supported absolutestart/endzoom. Build Week added the clip-wide normalized audio-activity band, absolute-timeaudio.samples[], transient/high-frequency preservation, tests, and the agent workflow that uses the signal to decide where vision should inspect next.

The demo keeps two proof layers separate:

- Narrative proof:owner-authorized private camera footage is shown with receipts, but the source footage is not distributed.
- Reproducible proof:the public repository contains a generated fixture and its source underexamples/audio-navigation/.

git clone https://github.com/getsquish/squish.git cd squish ./examples/audio-navigation/generate-sample.sh npx -y @getsquish/squish@0.3.1 examples/audio-navigation/sample.mp4 --json --out /tmp/squish-overview npx -y @getsquish/squish@0.3.1 examples/audio-navigation/sample.mp4 \ --density 6x6 --start 11.5 --end 13.5 --json --out /tmp/squish-zoom

squish clip.mov # sheets land beside the input squish clip.mov --density 5x5 --json # denser grid + machine-readable output squish clip.mov --start 1:00 --end 1:30 --density 5x5 # zoom into a range

{ "input": "/abs/path/clip.mov", "duration": 20.275, "frames": 9, "sheets": 1, "files": ["/abs/path/clip.sheet-1.jpg"], "audio": { "present": true, "normalization": "clip_peak", "window": { "start": 0, "end": 20.275 }, "samples": [ { "time": 0.106, "level": 0.08 }, { "time": 0.317, "level": 1 } ] }, "warnings": [], "contract": "squish-cli-v0" }

The example shortensaudio.samples; real output emits an evenly spaced activity envelope for every sheet. Sample times are absolute source seconds. Levels are0..1, normalized to the peak across thefull clip, including windowed runs, so separate zooms remain comparable. Exit0success ·1failure (message on stderr). Temp frames are always cleaned up. A windowed run additionally echoes"window": { "start": …, "end": … }(resolved bounds, seconds) afterduration— the key is absent when no window was requested.

One tool,squish_video—{ video_path, density?, start?, end?, out_dir? }→ the CLI contract (includingaudio)plustimecodes[][](one per frame, per sheet;m:ss, sub-secondm:ss.dwhen a window is short), stamped"contract": "squish-mcp-v0".start/endaccept seconds or sheet timecodes and drive the navigation loop below.

Works with Claude Code, Claude Desktop, Cursor, Hermes, and any stdio MCP client:

{ "mcpServers": { "squish": { "command": "npx", "args": ["-y", "@getsquish/squish", "mcp"] } } }

Remote MCP — official AI apps, zero install

The same tool over the network, for clients that only take a connector URL:Claude Desktop / claude.ai → Settings → Connectors → Add custom connector →https://api.getsquish.app/mcp. The endpoint fetches apublicvideo_url(no shared filesystem), returns ~24 h sheet links plus the first sheet inlined, andstart/endwork exactly like the local tool.

Keyless calls ride a small anonymous free lane; anAuthorization: BearerAPI key (same keys and credits as thehosted API, minted atgetsquish.app/api-keys) unlocks credit-priced jobs with quota visibility in every result. Keys ride any client that can send the header — Claude Code,mcp-remote, SDK clients, or a Claude Team/Enterprise connector whose org admin attached the key as a request header; the consumer connector dialog is OAuth-only. Full reference:remote MCP docs.
- Overview— callsquish_video(MCP) orsquish clip.mov --json(CLI) and read the sheet(s) with vision. Cells run in time order, left→right, top→bottom.
- Navigate— spot the regions that matter; every cell carries an absolute timecode. Treat an audio peak as a candidate interval, not an interpretation of what made the sound.
- Zoom— call again withstart/endset to the timecodes you spotted, only where uncertainty remains: denser sheets of a narrower window, addresses still absolute.
- Repeatuntil the answer is observable — never re-read the whole clip at high density when one range matters.
- Citeabsolute timestamps ("at 0:07 the press comes down").

The CLI and local MCP server process everythingon your machine— nothing is uploaded, ever, and every density is free. Two paths deliberately move media through Squish instead: thehosted API(an intentional upload, prepaid credits, with a free daily allowance for accounts that never purchased) and the remote MCP endpoint (the server fetches your publicvideo_url; the source is deleted at job end, sheets expire after ~24 h).

This is theengine— the CLI + MCP mouths of Squish, published to npm as@getsquish/squish. It is a curated, mirror-first export of a private monorepo (which stays the source of truth); history here starts at the first public release. SeeCONTRIBUTING.mdfor how changes flow.

src/ CLI (main/args) · engine (probe → plan → extract → compose → write) · MCP server · sheet renderer src/core/ pure planners shared with the web app: density · sampling · grid layout · timecode format tests/ node:test suite + a real-MCP-client e2e skills/ agent skills — npx skills add getsquish/squish installs video-navigation

Apache-2.0(withNOTICE). The Squish name, logo, mascot, and getsquish.app brand assets arenotlicensed by this repository.

Fetch and summarize YouTube videos by extracting their titles, descriptions, and transcripts.