FrameFetch

Any social-video URL → answers, transcript, metadata, insights, frames & on-screen text (OCR).
Agent-first video data API + MCP server. Pay per call, or with x402 (USDC) — no account.

FrameFetch turns oneYouTube, YouTube Shorts, TikTok, Instagram Reels, Pinterest, or Redditvideo URL into a single JSON response: adirect answer to a question about the video,metadata,engagement insights, atranscript(captions or Whisper), anLLM digest(text or spoken mp3),structured JSON(chapters/entities/products/claims),comments + sentiment,parametrically-sampled frames(every Nth / 1-per-second / a time range, at any width), and theon-screen text burned into those frames(OCR — captions, price tags, signage). Plus keywordsearchwhen you don't have a URL yet, andbatchfor up to 10 URLs in one call. Built API-first and MCP-first for AI agents.

This repo is theopen-source client + docs. The service itself runs atframefetch.net— you bring a free API key (or pay per call with x402); the backend stays hosted.

Node 18+ (uses built-infetch). Get a free key:framefetch.net.

Version note.This repo is at0.4.0. The newest version currently on npm is0.3.0—npm install framefetchstill gives you that one, and it has onlyextract/metadata/transcript/frames/platforms/status/demo/createKey. Everything else documented below is live on the API today and available from this repo; from npm 0.3.0 you can reach the same data throughextract({ fields: [...] }).

Ask a question — get an answer, not a transcript dump

A direct question about a video returns a short, grounded answer with timestamped quotes, instead of you parsing a 25,000-token transcript yourself.

import { FrameFetch } from 'framefetch'; const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY }); const { ask } = await ff.ask( 'https://www.youtube.com/watch?v=jNQXAC9IVRw', 'What does the presenter say to do first?', ); console.log(ask.answer); // short, direct answer console.log(ask.confidence); // 'high' | 'medium' | 'low' for (const q of ask.quotes) { // verbatim, timestamped supporting quotes console.log([${q.t_sec}s] ${q.text}); } console.log(ask.coverage); // which part of the transcript was analyzed

Charged only when an answer is actually produced. A repeat question about an already-extracted video reuses the cached transcript, so it answers fast without a re-download or re-transcription — but the answer itself is always freshly generated, never cache-served.

Frames-based answers:when a video has no transcript (e.g. Pinterest, or transcription failed), the answer is grounded in sampled keyframe images instead. Thencoverage.modeis"frames",quotesis[](no transcript text to quote), andconfidenceis capped at"medium".

import { FrameFetch } from 'framefetch'; const ff = new FrameFetch({ apiKey: process.env.FRAMEFETCH_API_KEY }); const r = await ff.extract({ url: 'https://www.youtube.com/watch?v=jNQXAC9IVRw', fields: ['metadata', 'transcript', 'frames', 'text_overlay'], frames: { mode: 'fps', fps: 1, width: 480 }, }); console.log(r.metadata.title); // "Me at the zoo" console.log(r.transcript.text); // "All right, so here we are, in front of the elephants…" console.log(r.frames.length); // 19 — frames is an array console.log(r.textOverlay?.[0]?.text); // on-screen text detected in the first frame, if any

Note the two spellings:text_overlayis therequestfield name,textOverlayis theresponsekey.

await ff.metadata(url); // title, author, duration, views, likes… await ff.transcript(url); // captions, else Whisper await ff.frames(url, { mode: 'fps', fps: 1, width: 512 }); await ff.ask(url, 'What product is being reviewed?'); // grounded Q&A, see above await ff.digest(url); // LLM summary of the transcript await ff.audioDigest(url, { voice: 'nova' }); // spoken mp3 briefing (signed URL, 24h) await ff.structured(url); // chapters/entities/products/claims/key_moments await ff.comments(url, { comments_cap: 50 }); // top-level comments await ff.commentSentiment(url); // aggregated audience-mood rollup (+ the comments) await ff.platforms(); // capability matrix (no key) await ff.status(); // live service health (no key) // on-screen text (OCR) — requires "frames" alongside it, use extract() directly: await ff.extract({ url, fields: ['frames', 'text_overlay'], frames: { mode: 'fps', fps: 1 } });

Every helper above is a thin wrapper overextract(), so anythingextract()accepts (translate,format, extrafields, …) can be passed as the last argument and is forwarded unchanged.

// Find something to extract when you don't have a URL yet const s = await ff.search('how to make sourdough', { limit: 5 }); for (const hit of s.results) { console.log(hit.title, hit.url, hit.durationSec); } // Then extract up to 10 of them in ONE call. Shared options apply to every url. const b = await ff.batch(s.results.slice(0, 3).map((r) => r.url), { fields: ['metadata', 'digest'], }); for (const item of b.results) { if (!item.ok) { console.error(item.url, item.error?.code); continue; } console.log(item.metadata.title, '→', item.digest.gist); }

One failing URL never fails the batch — each entry carries its ownokflag and, whenokis false, anerrorwithcode/message/hint. Per-URLframesspecs are not accepted in a batch; useextract()for those.

Translate the transcript, export subtitles

// translate the transcript into 1 of 25 languages (surfaced as transcript_translated) const r = await ff.transcript(url, { translate: 'ja' }); console.log(r.transcript_translated.text); // export subtitles directly — format is sent as a query param, response comes back as a string const srt = await ff.transcript(url, { format: 'srt' }); // source-language subtitles const vttJa = await ff.transcript(url, { translate: 'ja', format: 'vtt' }); // translated subtitles

const ff = new FrameFetch(); // no key await ff.demo('https://youtu.be/jNQXAC9IVRw'); // instant metadata, rate-limited const { key } = await ff.createKey('you@example.com'); // self-serve key + free credit

FrameFetch ships an MCP server (Streamable HTTP) with four tools:framefetch_extract,framefetch_platform_capabilities,framefetch_searchandframefetch_account. Add it to Claude, Cursor, or any MCP client:

{ "mcpServers": { "framefetch": { "url": "https://framefetch.net/mcp", "headers": { "Authorization": "<YOUR_FRAMEFETCH_KEY>" } } } }

claude mcp add --transport http framefetch https://framefetch.net/mcp \ --header "Authorization: <YOUR_FRAMEFETCH_KEY>"

MCP lives athttps://framefetch.net/mcpand speaks JSON-RPC over Streamable HTTP. REST lives under/v1/*and takes plain JSON ({"url": "…"}). Crossing the two is the single most common first-call mistake, so both directions answer clearly: a REST body POSTed to/mcpcomes back as a JSON-RPC parse error, and a JSON-RPC body POSTed to/v1/extractcomes back as400 WRONG_ENDPOINTnaming the right URL for your client.

Prefer a local stdio server (Claude Desktop, sandboxes, no inbound HTTP)? This package shipsframefetch-mcp, a zero-dependency stdio↔HTTP bridge that exposes the same tools and forwards calls toframefetch.net:

{ "mcpServers": { "framefetch": { "command": "npx", "args": ["-y", "framefetch-mcp"], "env": { "FRAMEFETCH_API_KEY": "<YOUR_FRAMEFETCH_KEY>" } } } }

tools/listworks with no key; tool calls useFRAMEFETCH_API_KEY(or x402). Override the endpoint withFRAMEFETCH_MCP_URL.

Autonomous agents can pay per call inUSDC via x402on Base — no signup, no human in the loop. Discoverable in the x402 Bazaar and at/.well-known/x402.json. Humans can use a free tier, prepaid credits, or a Stripe card.

import { FrameFetchError } from 'framefetch'; try { await ff.transcript(url); } catch (e) { if (e instanceof FrameFetchError && e.status === 402) { // out of credit — top up at framefetch.net or via x402 } }

ff.extract({ url: string, fields?: Field[], // 'metadata' | 'insights' | 'transcript' | 'frames' | 'text_overlay' // | 'digest' | 'audio_digest' | 'structured' | 'comments' // | 'comment_sentiment' | 'delta' frames?: { mode, n, fps, from, to, format, width }, translate?: string, // ISO-639-1 target language (25 supported) voice?: string, // TTS voice for audio_digest comments_cap?: number, // 1-200, default 100 ask?: string, // 3-500 char question — see ff.ask() above publish?: boolean, // opt in to a public per-video SEO page format?: 'md' | 'markdown' | 'srt' | 'vtt', // alternate egress rendering (returns a string, not JSON) });

Seeindex.d.tsfor the complete typed response shape (ExtractResult,Ask,VideoStructured,VideoComments,CommentSentiment,AudioDigest,SearchResult,BatchResult, …).

Full OpenAPI:framefetch.net/openapi.json· Docs:framefetch.net/docs

Website·Docs·Pricing·Status·Guide: giving an agent video data·Compare vs alternatives

YouTube transcript extraction for AI agents. Clean text, timestamps, or structured JSON from any video. No API keys required.