TinySearch
About
Token-efficient web search for AI chat apps and agents.
Details
- Author
- marcellm01
- Categories
- Search, Knowledge Base, Other
Jump to
Setup
Install TinySearch in your MCP client (Claude Desktop, Cursor, Windsurf, and others).
Repository: https://github.com/marcellm01/TinySearch
Follow the installation instructions in the repository README, then restart your MCP client.
TinySearch searches, crawls, and reranks the web locally, then gives your agent only the evidence worth putting in its context.
Documentation·Quick start·Python·Discord
TinySearch is a self-hosted web-research tool for AI agents. It searches the web, reads the best pages, removes low-value content, and returns compact evidence with source URLs.
Your model receives the useful passages instead of paying to process entire webpages.
TinySearch is part ofTinySuite, a suite of focused tools designed to make agentic operations cheaper by minimizing token usage through smart retrieval, selection, and context-management techniques.
Tiers 1 and 2 need no search service. Tier 3 adds a dedicated SearXNG service, persistent model storage, and a network MCP endpoint. See theinstallation guidefor the Docker setup.
The expensive part of agent research is context
A search result is not yet useful evidence. Agents often have to open several pages, ingest navigation and boilerplate, and spend paid input tokens deciding which passages matter.
TinySearch moves that work in front of the model:
flowchart LR A[Question] --> B[Search and crawl] B --> C[Local hybrid reranking] C --> D[Compact evidence<br/>with source URLs] D --> E[Your agent]
- Smaller model context.Only the best-ranked evidence chunks are returned, within a controlled evidence budget.
- No metered search API required by default.TinySearch can search through DDGS without a paid search provider.
- Local retrieval by default.ONNX embeddings and hybrid reranking run on your machine instead of creating embedding API charges.
Search broadly. Read locally. Pay the model only for the evidence that matters.
This is retrieval, not summarization: TinySearch selects the passages worth keeping with local BM25 and embedding rerank, it doesn't run a model over the page to rewrite or condense it. Every returned chunk is the original page text, unedited, so what you cite is what the page actually said. That keeps the pipeline fast and free to run locally, at the cost of not compacting as aggressively as a dedicated reduction model could. A learned reduction step is a direction we may explore later; it isn't part of TinySearch today.
Actual savings depend on the pages, evidence limits, client model, and provider pricing. TinySearch reduces the web content sent to the model; it does not control what the client does with that evidence afterward.
The cost panel uses an illustrative $3.00 per million input-token rate and excludes search, crawling, model output, and downstream agent use.
The naive baseline isn't a strawman product, it's the same pages TinySearch crawled for each query, fed to the model unfiltered, the way a generic "search, then fetch the page" tool (a plain web-search-plus-fetch loop, the kind built into most coding agents) would. Measured against the current recommended flow (searchthenscrape_urls, not the deprecated all-in-oneresearchtool) and counted on the actual MCP tool-result text, TinySearch's primary interface. Reproduce or rerun it yourself:
python scripts/benchmark_token_savings.py --json-out report.json
Withuvinstalled, add TinySearch to any MCP client:
{ "mcpServers": { "tinysearch": { "command": "uvx", "args": [ "--python", "3.12", "--from", "tinysuite-search[server]", "tinysearch" ] } } }
The client launches TinySearch over stdio when it needs it. No repository clone, hosted account, or paid search key is required.
Fastsearchstarts without Chromium or an embedding model. The first scrape initializes Chromium; focused scraping and the legacyresearchtool also initialize the configured embedding model. Pre-warm both ahead of time if you will use those workflows:
uvx --from "tinysuite-search[server]" tinysearch setup
Prefer Docker, a remote MCP endpoint, or a source checkout? Follow theinstallation guide.
TinySearch deliberately stays focused. It is a retrieval layer, not another agent, chat interface, hosted search product, or permanent web index.
See the completeMCP tool referencefor parameters and response contracts.
TinySearch does not spend another model call writing the final answer. The recommended flow issearchfor lightweight discovery, thenscrape_urlsfor the pages worth reading.
Successful MCP tool-result text is XML. A search result looks like this:
<search_results> <query>Python asyncio cancellation</query> <results> <result index="1"> <title>Coroutines and Tasks</title> <url>https://docs.python.org/3/library/asyncio-task.html</url> <search_preview>Tasks can be cancelled...</search_preview> </result> </results> </search_results>
scrape_urlsreturns each page's selected Markdown chunks under one<url_grounded_answers>batch root, andget_current_datetimereturns<current_datetime>. Dynamic values are escaped so retrieved content cannot forge the XML boundaries around it.
MCP still uses its standard JSON-RPC transport envelope, including protocol-level errors and optionalstructuredContent. Python and FastAPI keep their structured JSON contracts for applications that need to store, inspect, or transform the evidence.
- searchreturns backend-ordered titles, URLs, previews, and upstream dates without starting Chromium or an embedding model.
- scrape_urlsreads one to five known pages concurrently. Omit an item's query or use""to keep clean Markdown in page order within the configured token budget.
- Supply a focused item query when TinySearch should chunk and hybrid-rank that page before returning evidence.
The deprecated MCPresearchtool retains the older all-in-one search, crawl, and rerank pipeline for compatibility. New MCP integrations should composesearchwithscrape_urlsinstead.
TinySearch also works as a regular Python package:
import asyncio from tinysearch import scrape_urls, search async def main(): results = await search("Python async tasks") print(results["results"]) page_url = results["results"][0]["url"] evidence = await scrape_urls([{ "url": page_url, "query": "How does asyncio cancellation work?", }]) print(evidence["results"]) asyncio.run(main())
The Python API returns stable, JSON-serializable results.searchaccepts a per-calllimitfrom 1 to 50.scrape_urlsaccepts a per-callmax_tokensbudget (4,000 by default); omit an item's scrape query or use""for page-order mode. Rendering structured evidence into an LLM prompt is explicit, so applications can store, inspect, transform, or budget the result first.
The optional FastAPI app mirrors these surfaces.POST /searchandPOST /researchacceptoutput_format(promptorjson) and always respond with JSON; prompt mode places rendered text in theanswerfield.POST /scrapeaccepts one to five{ "url", "query" }items and always returns structured per-item outcomes. The app also exposes/health,/current_datetime, and read-only/config; configuration writes require explicit environment opt-in.
TinySearch selects a web-search backend from config, so you can start with no search service and add one later without changing code.
Set theBRAVE_SEARCH_API_KEYenvironment variable to add Brave's official Web Search API as a keyed fallback for theddgsandduckduckgobackends. Brave is only consulted when the primary call errors or returns no results.
Full key reference, SearXNG JSON-output setup, and Compose details live in theconfiguration reference.
TinySearch uses its bundled Playwright Chromium by default. To use a browser that you operate separately, set its Chrome DevTools Protocol endpoint in the config file:
{ "browser_cdp_url": "http://browser:9222" }
Server processes also acceptTINYSEARCH_BROWSER_CDP_URL. When either setting is present, TinySearch connects through Crawl4AI instead of installing or launching the bundled Chromium. The external browser owns its executable, profile, proxy, and fingerprint configuration; TinySearch does not select or install a particular browser backend.
Treat a CDP endpoint as privileged remote control of the browser. Keep it on a private network or loopback interface, require authentication when it crosses a host boundary, and do not expose port 9222 directly to the public internet. When TinySearch itself runs in Docker,localhostrefers to the TinySearch container, so use an endpoint reachable from that container.
The CDP endpoint is operator-managed and cannot be changed through the HTTPPUT /configendpoint, even when configuration writes are enabled. Set it in the startup environment or the file selected byTINYSEARCH_CONFIG_PATH, then restart TinySearch. HTTP clients can continue updating other settings by omittingbrowser_cdp_urlfrom their partial update.
- No vendor in the loop.No TinySearch account, no required API key, no per-request billing, no analytics service or hosted scraped-data cache. The infrastructure you'd otherwise pay a search API for runs on your machine.
- Source-grounded by construction.Every evidence chunk is the original page text, still attached to its originating URL, so a claim in your agent's answer traces back to one specific passage instead of stopping at "the vendor's model said this."
- Built around token efficiency.Page selection and passage selection happen locally, before content enters model context.
- Useful without paid infrastructure.DDGS search and local ONNX embeddings are the defaults.
- Bring your own stack when needed.SearXNG and OpenAI-compatible embedding providers remain optional.
- Works where agents already work.Use MCP over stdio, Streamable HTTP, Python, FastAPI, or Docker.
Each tool focuses on a different part of the agent workflow and uses targeted techniques to reduce unnecessary context before it reaches the model. TinySearch handles the web-research layer by turning pages into a small, ranked, source-grounded evidence packet.
The README is the product overview. Detailed setup and operational material lives in the TinySuite documentation:
- TinySearch overview and installation
- Configuration reference
- MCP tools
- Troubleshooting
The repository also contains an annotated example configuration atconfigs/tinysearch_config.json.
TinySearch is intentionally lightweight. Use a commercial search API, persistent crawler, or full search index when you need:
- guaranteed search coverage or an SLA
- large-scale or scheduled indexing
- long-term page storage and change history
- enterprise observability and access controls
git clone https://github.com/TinySuiteHQ/TinySearch cd TinySearch python -m venv .venv source .venv/bin/activate pip install -e ".[server]" python -m unittest discover tests
TinySearch supports Python 3.12 and newer. CI tests Python 3.12, 3.13, and 3.14 across Linux, macOS, and Windows.
- tinysearch.searchandtinysearch.scrape_urls: structured Python API
- tinysearch.research: legacy all-in-one structured Python research pipeline
- tinysearch.get_current_datetime: structured UTC date and time
- tinysearch.to_prompt: pure structured-evidence prompt renderer
- tinysearch mcp: stdio MCP server (also the no-argument default)
- tinysearch serve: Streamable HTTP MCP server
- tinysearch.servers.fastapi_server:app: optional FastAPI application
Questions, ideas, and bug reports are welcome:
- Join the TinySearch Discord
- Open a GitHub issue
- Email the maintainer
TinySearch reads public pages and returns selected excerpts to the calling client. Search, crawling, local embeddings, and reranking can run without sending page content to an embedding provider. If you choose an OpenAI-compatible embedding backend, that provider receives the text sent for vectorization.
TinySearch is available under theMIT License. Downloaded model weights remain subject to their respective model-card licenses. SeeNOTICEfor third-party distribution details.
Search global news using natural language. Webz.io News Search API returns the most relevant articles and content, with filters for source, country, language, date, sentiment, and category.
Fetch, convert, and search AWS documentation pages, with recommendations for related content.
Search campgrounds around the world on campertunity, check availability, and provide booking links.
The Ferryhopper MCP Server exposes ferry routes, schedules and booking redirects so an AI assistant can discover connections across Europe and the Mediterranean and send users to Ferryhopper to complete bookings.
All-in-One SEO & Web Intelligence Toolkit API from FetchSERP.
MCP server that provides read-only access to HyperKitty, the web-based email archive component of Mailman 3.
At Sunrise Apps, we believe AI agents should be limitless, especially when it comes to visual data. We created ImageSorcery to bridge the critical gap in AI's ability to interact with and manipulate images directly, all while upholding the highest standards of privacy and security.
Just Domain is the domain registrar for businesses built with AI. Its remote MCP server checks availability and returns first-year and renewal pricing, plus a link to register on justdomain.ai, with DNS and WHOIS privacy in the same place. No account, no API key, read only. Endpoint: https://mcp.justdomain.ai/
Research tools, including a Sqlite-backed document stash
Semantic search over 9 free-license stock photo sources. Hosted remote server with OAuth — no API key to paste.
SerpApi MCP Server for Google and other search engine results
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.




