Scrapfly MCP Server
About
Get real-time web data, automatically handle anti-bot systems, extract structured data, and capture screenshots with your favorite tools, such as Cursor or LangChain.
Details
- Author
- scrapfly
- Downloads
- 329
- Categories
- Web Scraping, Automation, Other
Jump to
- Live web data access with automatic JavaScript rendering
- Anti‑bot handling and proxy infrastructure
- Structured data extraction and clean markdown output
- Full‑page or element‑level screenshot capture (PNG)
- Secure authentication via OAuth2 or API key
- Works with popular AI agents and automation platforms
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
Scrapfly MCP ServerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Configure the server by adding an entry to your AI tool's MCP configuration (e.g., claude_desktop_config.json). Use npx mcp-remote https://mcp.scrapfly.io/mcp for OAuth2 authentication, or append ?key=YOUR-API-KEY for API key authentication. After restarting the tool, invoke scraping through five available MCP tools: instructions (guidance and required parameters), web_get_page (quick fetch), web_scrape (full‑control scraping), screenshot (visual capture), and info_account (account details).
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"scrapfly mcp server": {
"scrapfly": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.scrapfly.io/mcp"
]
}
}
}
}McpServers
{
"scrapfly": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.scrapfly.io/mcp"
]
}
}Key Features & Capabilities
- Live web data access - Anti-bot handling & proxy infrastructure - Structured data extraction - Screenshots capture - Secure Auth options Works with your favorite AI tools: Claude Desktop, Cursor, Cline, Windsurf, LangChain, LlamaIndex, CrewAI, OpenAI function calling, n8n, Make, and Zapier.Practical Use Cases
- Job aggregation - Price monitoring - Content research & aggregation - Competitor intelligence & market analysis - Real-estate analysis - RAG & LLM data ingestion - Automated multi-step workflowsTools
instructions
Purpose: Provide guidance and required parameters for successful scraping calls.
Features: Supplies best-practice recommendations, parameter selection hints, error-handling guidance, and the mandatory pow value needed before calling web_get_page or web_scrape.
Example usage:
{ "tool": "scraping_instruction_enhanced" }
web_get_page
Purpose: Fetch a webpage quickly using optimized defaults.
Features: Automatic JavaScript rendering, anti-bot handling, clean text/markdown output, and sensible default parameters for most sites.
Example usage:
``
{
"tool": "web_get_page",
"parameters": {
"url": "https://news.ycombinator.com",
"pow": "obtained_from_instruction_tool",
"format": "markdown",
"format_options": ["only_content"]
}
}
`
web_scrape
Purpose: Provide full-control, customizable scraping for complex sites and workflows.
Features: Supports browser automation, login/auth flows, custom HTTP methods and headers, cookie management, multi-step interactions, and AI-powered data extraction.
Example usage:
`
{
"tool": "web_scrape",
"parameters": {
"url": "https://web-scraping.dev/login",
"pow": "obtained_from_instruction_tool",
"render_js": true,
"js_scenario": [
{ "fill": { "selector": "input[name='username']", "value": "myuser" } },
{ "fill": { "selector": "input[name='password']", "value": "mypass" } },
{ "click": { "selector": "button[type='submit']" } },
{ "wait_for_navigation": { "timeout": 5000 } }
]
}
}
`
screenshot
Purpose: Capture visual snapshots of webpages.
Features: Supports full-page screenshots or element-level captures using CSS selectors.
Example usage:
`
{
"tool": "screenshot",
"parameters": {
"url": "https://web-scraping.dev/pricing",
"capture": ".pricing-table",
"format": "png",
"options": ["load_images", "block_banners"]
}
}
`
info_account
Purpose: Retrieve real-time details about your Scrapfly account and project.
Features: Returns account metadata, project configuration, subscription status, credit usage, remaining quota, and concurrency limits.
Example usage:
{ "tool": "info_account" }
Setup Guide
Claude Desktop
1. Locate Your Configuration File
Claude Desktop stores its configuration in a JSON file. Open the file for your operating system:
macOS:
~/Library/Application Support/Claude/claude_desktop_config.json
Windows:
%APPDATA%\Claude\claude_desktop_config.json
2. Choose Authentication Method
Select your preferred authentication method:
OAuth2 (recommended):
Add the following configuration to your claude_desktop_config.json file:
`
{
"mcpServers": {
"scrapfly": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.scrapfly.io/mcp"
]
}
}
}
`
API Key:
Add the following configuration to your claude_desktop_config.json file:
`
{
"mcpServers": {
"scrapfly": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.scrapfly.io/mcp?key=YOUR-API-KEY"
]
}
}
}
``
3. Restart Claude Desktop
After saving the configuration file, completely quit and restart Claude Desktop to apply the changes.
4. Verify the Integration
After restarting, check that the MCP tools are available:
- Look for the hammer icon (🔨) in the bottom right corner of the chat window
- Click the hammer icon to see available MCP tools - you should see Scrapfly tools listed
- Try a test prompt: "Can you scrape https://news.ycombinator.com and show me the top 5 posts?"
Documentation
https://scrapfly.io/docs/mcp/getting-startedSign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
