Web Fetcher

[中文|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=zh)[Deutsch|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=de)[Español|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=es)[français|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=fr)[日本語|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ja)[한국어|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ko)[Português|](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=pt)[Русский MCP server for fetch web page content using Playwright headless browser. 🌟**Recommended**:](https://www.readme-i18n.com/jae-jae/fetcher-mcp?lang=ru)[OllaMan- Powerful Ollama AI Model Manager. - **JavaScript Support**: Unlike traditional web scrapers, Fetcher MCP uses Playwright to execute JavaScript, making it capable of handling dynamic web content and modern web applications. **Intelligent Content Extraction**: Built-in Readability algorithm automatically extracts the main content from web pages, removing ads, navigation, and other non-essential elements. **Flexible Output Format**: Supports both HTML and Markdown output formats, making it easy to integrate with various downstream applications. **Parallel Processing**: The`fetch_urls`tool enables concurrent fetching of multiple URLs, significantly improving efficiency for batch operations. **Resource Optimization**: Automatically blocks unnecessary resources (images, stylesheets, fonts, media) to reduce bandwidth usage and improve performance. **Robust Error Handling**: Comprehensive error handling and logging ensure reliable operation even when dealing with problematic web pages. **Configurable Parameters**: Fine-grained control over timeouts, content extraction, and output formatting to suit different use cases. First time setup - install the required browser by running the following command in your terminal: Use the`--transport=http`parameter to start both Streamable HTTP endpoint and SSE endpoint services simultaneously: ``` `npx -y fetcher-mcp --log --transport=http --host=0.0.0.0 --port=3000` ``` After startup, the server provides the following endpoints: - `/mcp`- Streamable HTTP endpoint (modern MCP protocol) - `/sse`- SSE endpoint (legacy MCP protocol) Clients can choose which method to connect based on their needs. Run with the`--debug`option to show the browser window for debugging: Configure this MCP server in Claude Desktop: On MacOS:`~/Library/Application Support/Claude/claude_desktop_config.json` On Windows:`%APPDATA%/Claude/claude_desktop_config.json` ``` `{ "mcpServers": { "fetcher": { "command": "npx", "args": ](https://ollaman.com/)["-y", "fetcher-mcp"] } } }` ``` ``` `docker run -p 3000:3000 ghcr.io/jae-jae/fetcher-mcp:latest` ``` ``` `version: "3.8" services: fetcher-mcp: image: ghcr.io/jae-jae/fetcher-mcp:latest container_name: fetcher-mcp restart: unless-stopped ports: - "3000:3000" environment: - NODE_ENV=production # Using host network mode on Linux hosts can improve browser access efficiency # network_mode: "host" volumes: # For Playwright, may need to share certain system paths - /tmp:/tmp # Health check healthcheck: test: ["CMD", "wget", "--spider", "-q", "http://localhost:3000"] interval: 30s timeout: 10s retries: 3` ``` - `fetch_url`- Retrieve web page content from a specified URL - Uses Playwright headless browser to parse JavaScript - Supports intelligent extraction of main content and conversion to Markdown - Supports the following parameters: - `url`: The URL of the web page to fetch (required parameter) - `timeout`: Page loading timeout in milliseconds, default is 30000 (30 seconds) - `waitUntil`: Specifies when navigation is considered complete, options: 'load', 'domcontentloaded', 'networkidle', 'commit', default is 'load' - `extractContent`: Whether to intelligently extract the main content, default is true - `maxLength`: Maximum length of returned content (in characters), default is no limit - `returnHtml`: Whether to return HTML content instead of Markdown, default is false - `waitForNavigation`: Whether to wait for additional navigation after initial page load (useful for sites with anti-bot verification), default is false - `navigationTimeout`: Maximum time to wait for additional navigation in milliseconds, default is 10000 (10 seconds) - `disableMedia`: Whether to disable media resources (images, stylesheets, fonts, media), default is true - `debug`: Whether to enable debug mode (showing browser window), overrides the --debug command line flag if specified `fetch_urls`- Batch retrieve web page content from multiple URLs in parallel - Uses multi-tab parallel fetching for improved performance - Returns combined results with clear separation between webpages - Supports the following parameters: - `urls`: Array of URLs to fetch (required parameter) - Other parameters are the same as`fetch_url` `browser_install`- Install Playwright Chromium browser binary automatically - Installs required Chromium browser binary when not available - Automatically suggested when browser installation errors occur - Supports the following parameters: - `withDeps`: Install system dependencies required by Chromium browser, default is false - `force`: Force installation even if Chromium is already installed, default is false - **Wait for Complete Loading**: For websites using CAPTCHA, redirects, or other verification mechanisms, include in your prompt: This will use the`waitForNavigation: true`parameter. **Increase Timeout Duration**: For websites that load slowly: ``` `Please set the page loading timeout to 60 seconds` ``` This adjusts both`timeout`and`navigationTimeout`parameters accordingly. - **Preserve Original HTML Structure**: When content extraction might fail: ``` `Please preserve the original HTML content` ``` Sets`extractContent: false`and`returnHtml: true`. **Fetch Complete Page Content**: When extracted content is too limited: ``` `Please fetch the complete webpage content instead of just the main content` ``` **Return Content as HTML**: When HTML format is needed instead of default Markdown: ``` `Please return the content in HTML format` ``` - **Dynamic Debug Activation**: To display the browser window during a specific fetch operation: ``` `Please enable debug mode for this fetch operation` ``` - **Manual Login**: To login using your own credentials: ``` `Please run in debug mode so I can manually log in to the website` ``` Sets`debug: true`or uses the`--debug`flag, keeping the browser window open for manual login. **Interacting with Debug Browser**: When debug mode is enabled: - The browser window remains open - You can manually log into the website using your credentials - After login is complete, content will be fetched with your authenticated session **Enable Debug for Specific Requests**: Even if the server is already running, you can enable debug mode for a specific request: ``` `Please enable debug mode for this authentication step` ``` Sets`debug: true`for this specific request only, opening the browser window for manual login. Install the browsers needed for Playwright: You can also enable visible browser mode for debugging: - [g-search-mcp: A powerful MCP server for Google search that enables parallel searching with multiple keywords simultaneously. Perfect for batch search operations and data collection. A browser screenshot tool to capture scrolling screenshots of webpages using Playwright, with support for intelligent section identification and multiple output formats. Take screenshots and read console logs from web pages using Playwright. CloakBrowser MCP server for AI agents: Playwright-powered browsing, clean tool forwarding, Docker support, and multi-session HTTP transport. Control a browser for web automation tasks using Playwright on Cloudflare Workers. Control a browser for web automation tasks like navigation, typing, clicking, and taking screenshots using Playwright on Cloudflare Workers. A server for web crawling and content extraction using the Crawl4AI library. Scrape any URL with JavaScript rendering and get back clean markdown — built for AI agents, LLM pipelines, and autonomous research workflows. Ultra-fast web scraper and deep discussion crawler delivering clean Markdown for AI Agents Query financial web tables from sources like iwencai, tdx, and eastmoney using Playwright.](https://github.com/jae-jae/g-search-mcp)