MCP Deep Web Research Server

An advanced web research server with intelligent search queuing, enhanced content extraction, and deep research capabilities.

A Model Context Protocol (MCP) server for advanced web research.

- Added visit_page tool for direct webpage content extraction
- Optimized performance to work within MCP timeout limits

- Reduced default maxDepth and maxBranching parameters
- Improved page loading efficiency
- Added timeout checks throughout the process
- Enhanced error handling for timeouts

This project is a fork ofmcp-webresearchbymzxrai, enhanced with additional features for deep web research capabilities. We're grateful to the original creators for their foundational work.

- Node.js>= 18 (includesnpmandnpx)
- Claude Desktop app

To install Deep Web Research Server for Claude Desktop automatically viaSmithery:

npx -y @smithery/cli install @PedroDnT/mcp-deepwebresearch --client claude

# Install globally using npm npm install -g mcp-deepwebresearch # Or using yarn yarn global add mcp-deepwebresearch # Or using pnpm pnpm add -g mcp-deepwebresearch

# Using npm npm install mcp-deepwebresearch # Using yarn yarn add mcp-deepwebresearch # Using pnpm pnpm add mcp-deepwebresearch

{ "mcpServers": { "deepwebresearch": { "command": "mcp-deepwebresearch", "args": [] } } }

Location:%APPDATA%\Claude\claude_desktop_config.json

{ "mcpServers": { "deepwebresearch": { "command": "mcp-deepwebresearch", "args": [] } } }

Location:~/Library/Application Support/Claude/claude_desktop_config.json

This config allows Claude Desktop to automatically start the web research MCP server when needed.

After installation, run this command to install required browser dependencies:

Simply start a chat with Claude and send a prompt that would benefit from web research. If you'd like a prebuilt prompt customized for deeper web research, you can use theagentic-researchprompt that we provide through this package. Access that prompt in Claude Desktop by clicking the Paperclip icon in the chat input and then selectingChoose an integration→deepwebresearch→agentic-research.

- Performs comprehensive research with content analysis
- Arguments:

{ topic: string; maxDepth?: number; // default: 2 maxBranching?: number; // default: 3 timeout?: number; // default: 55000 (55 seconds) minRelevanceScore?: number; // default: 0.7 }

{ findings: { mainTopics: Array<{name: string, importance: number}>; keyInsights: Array<{text: string, confidence: number}>; sources: Array<{url: string, credibilityScore: number}>; }; progress: { completedSteps: number; totalSteps: number; processedUrls: number; }; timing: { started: string; completed?: string; duration?: number; operations?: { parallelSearch?: number; deduplication?: number; topResultsProcessing?: number; remainingResultsProcessing?: number; total?: number; }; }; }

- Performs multiple Google searches in parallel with intelligent queuing
- Arguments:{ queries: string[], maxParallel?: number }
- Note: maxParallel is limited to 5 to ensure reliable performance

- Visit a webpage and extract its content
- Arguments:{ url: string }
- Returns:

{ url: string; title: string; content: string; // Markdown formatted content }

A guided research prompt that helps Claude conduct thorough web research. The prompt instructs Claude to:

- Start with broad searches to understand the topic landscape
- Prioritize high-quality, authoritative sources
- Iteratively refine the research direction based on findings
- Keep you informed and let you guide the research interactively
- Always cite sources with URLs

The server can be configured through environment variables:

- MAX_PARALLEL_SEARCHES: Maximum number of concurrent searches (default: 5)
- SEARCH_DELAY_MS: Delay between searches in milliseconds (default: 200)
- MAX_RETRIES: Number of retry attempts for failed requests (default: 3)
- TIMEOUT_MS: Request timeout in milliseconds (default: 55000)
- LOG_LEVEL: Logging level (default: 'info')

- Symptom: "Too many requests" error
- Solution: IncreaseSEARCH_DELAY_MSor decreaseMAX_PARALLEL_SEARCHES

- Symptom: "Request timed out" error
- Solution: Ensure requests complete within the 60-second MCP timeout

- Symptom: "Browser failed to launch" error
- Solution: Ensure Playwright is properly installed (npx playwright install)

This is beta software. If you run into issues:

# On macOS tail -n 20 -f ~/Library/Logs/Claude/mcp.log # On Windows Get-Content -Path "$env:APPDATA\Claude\logs\mcp.log" -Tail 20 -Wait

# Install dependencies pnpm install # Build the project pnpm build # Watch for changes pnpm watch # Run in development mode pnpm dev

# Run all tests pnpm test # Run tests in watch mode pnpm test:watch # Run tests with coverage pnpm test:coverage

# Run linter pnpm lint # Fix linting issues pnpm lint:fix # Type check pnpm type-check

- Fork the repository
- Create your feature branch (git checkout -b feature/amazing-feature)
- Commit your changes (git commit -m 'Add some amazing feature')
- Push to the branch (git push origin feature/amazing-feature)
- Open a Pull Request

- Follow TypeScript best practices
- Maintain test coverage above 80%
- Document new features and APIs
- Update CHANGELOG.md for significant changes
- Follow semantic versioning

- Use batch operations where possible
- Implement proper error handling and retries
- Consider memory usage with large datasets
- Cache results when appropriate
- Use streaming for large content

- Node.js >= 18
- Playwright (automatically installed as a dependency)

This project builds upon the excellent work ofmcp-webresearchbymzxrai. The original codebase provided the foundation for our enhanced features and capabilities.

Search, view details, and download 3D models from the Sketchfab platform.