MCP for Docs

An MCP (Model Context Protocol) server that automatically downloads and converts documentation from various sources into organized markdown files.

mcp-for-docs is designed to crawl documentation websites, convert their content to markdown format, and organize them in a structured directory system. It can also generate condensed cheat sheets from the downloaded documentation.

- 🕷️Smart Documentation Crawler: Automatically crawls documentation sites with configurable depth
- 📝HTML to Markdown Conversion: Preserves code blocks, tables, and formatting
- 📁Automatic Categorization: Intelligently organizes docs into tools/APIs categories
- 📄Cheat Sheet Generator: Creates condensed reference guides from documentation
- 🔍Smart Discovery System: Automatically detects existing documentation before crawling
- 🚀Local-First: Uses existing downloaded docs when available
- ⚡Rate Limiting: Respects server limits and robots.txt
- ✅User Confirmation: Prevents accidental regeneration of existing content
- ⚙️Comprehensive Configuration: JSON-based configuration with environment variable overrides
- 🧪Test Suite: 94 tests covering core functionality

- Node.js 18+
- npm or yarn
- Claude Desktop or Claude Code CLI

git clone https://github.com/shayonpal/mcp-for-docs.git cd mcp-for-docs

For Claude Desktop(~/Library/Application Support/Claude/claude_desktop_config.json):

{ "mcpServers": { "mcp-for-docs": { "command": "node", "args": ["/path/to/mcp-for-docs/dist/index.js"], "env": {} } } }

{ "mcpServers": { "mcp-for-docs": { "command": "node", "args": ["/path/to/mcp-for-docs/dist/index.js"], "env": {} } } }

To download documentation from a website:

await crawl_documentation({ url: "https://docs.n8n.io/", max_depth: 3, // Optional, defaults to 3 force_refresh: false // Optional, set to true to regenerate existing docs });

The tool will first check for existing documentation and show you what's already available. To regenerate existing content, useforce_refresh: true.

- Tools:/Users/shayon/DevProjects/~meta/docs/tools/[tool-name]/
- APIs:/Users/shayon/DevProjects/~meta/docs/apis/[api-name]/

To create a cheat sheet from documentation:

await generate_cheatsheet({ url: "https://docs.anthropic.com/", use_local: true, // Use local files if available (default) force_regenerate: false // Optional, set to true to regenerate existing cheatsheets });

Cheat sheets are saved to:/Users/shayon/DevProjects/~meta/docs/cheatsheets/

The tool will check for existing cheatsheets and show you what's already available. To regenerate existing content, useforce_regenerate: true.

To see what documentation is available locally:

await list_documentation({ category: "all", // Options: "tools", "apis", "all" include_stats: true });

- n8n documentation
- Anthropic API docs
- Obsidian Tasks plugin docs
- Apple Swift documentation

Most documentation sites following standard patterns should work automatically.

- Configuration System(v0.4.0): Added comprehensive JSON-based configuration with environment variable support
- Smart Discovery: Automatically finds and reports existing documentation before crawling
- Improved Conversion: Fixed HTML to Markdown issues including table formatting and inline code preservation
- Dynamic Categorization: Intelligent detection of tools vs APIs based on URL patterns and content analysis
- Test Coverage: 94 tests passing with comprehensive unit and integration testing

For detailed changes, seeCHANGELOG.md.
- Editconfig.jsonand update thedocsBasePathfor your machine:

{ "docsBasePath": "/Users/yourusername/path/to/docs" }

export DOCS_BASE_PATH="/different/path/on/this/machine"

# Install dependencies npm install # Run in development mode npm run dev # Run tests npm test # Build for production npm run build # Lint code npm run lint

- URL Structure Preservation(#15): Currently flattens URL structure when saving docs
- Large Documentation Sites(#14): No document limit for very large sites
- GitHub Repository Docs(#9): Specialized crawler for GitHub repos not yet implemented

See allopen issuesfor the complete roadmap.
- Fork the repository
- Create a feature branch
- Make your changes
- Update CHANGELOG.md
- Submit a pull request

This project is licensed under the GPL 3.0 License - see theLICENSEfile for details.

- Built with theModel Context Protocol SDK
- UsesPlaywrightfor web scraping
- Markdown conversion powered byTurndown

A Retrieval-Augmented Generation (RAG) server for document processing, vector storage, and intelligent Q&A, powered by the Model Context Protocol.