Web Scraping API
Description
# 🚀 **We’ve just launched our new MCP server!** If you're building or using AI agents, you can now plug our web scraping API directly into your autonomous workflows. Just add the following to your MCP config: ``` { "mcpServers": { "web-scraping-api-by-instantapi-ai": {…
About
# 🚀 **We’ve just launched our new MCP server!** If you're building or using AI agents, you can now plug our web scraping API directly into your autonomous workflows. Just add the following to your MCP config: ``` { "mcpServers": { "web-scraping-api-by-instantapi-ai": { "command": "npx", "args": [ "-y", "mcp-remote"…
Details
- Author
- zeeb0tt
- GitHub stars
- 2
- Downloads
- 386
- Categories
- Automation, Web Scraping, API, AI
Jump to
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
Web Scraping APICommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Add the provided configuration to your MCP settings, using the command npx -y mcp-remote https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse with an Authorization header set via the AUTH_HEADER environment variable
scrape
Extract structured data from any webpage with one call: provide the URL and a mock JSON object as the fields parameter, using natural-language placeholder values to describe the data you want (not selectors or HTML details). Example Usage Scenario: User wants to extract the product details from the Amazon product page. ``` "url": "https://www.amazon.com.au/MSI-PRO-MP341CQW-UltraWide-Compatible/dp/B09Y19TRQ2", "fields": { "monitor_name": "< The product name of the monitor. >", "brand": "< The brand or manufacturer name. >", "display_size_in_inches": "< Numeric only. >", "resolution": "< Example format: 1920x1080. >", "panel_type": "< Type of panel. >", "refresh_rate_hz": "< Numeric only. >", "aspect_ratio": "< Example format: 16:9. >", "ports": "< A comma-delimited list of available ports (e.g., HDMI, DisplayPort, etc.). >", "features": "< Key selling points or capabilities, comma-delimited (e.g., LED, Full HD, etc.). >", "price": "< Numeric price (integer or float). >", "price_currency": "< Price currency (3 character alphabetic ISO 4217). >", "review_count": "< Total number of customer reviews, numeric only. >", "average_rating": "< Float or numeric star rating (e.g., 4.3). >", "review_summary": "< A 50 words or less summary of all the written customer feedback. >" } ``` Sample response: ``` { "monitor_name": "MSI PRO MP341CQW 34" UltraWide Business Monitor", "brand": "MSI", "display_size_in_inches": 34, "resolution": "3440x1440", "panel_type": "VA", "refresh_rate_hz": 100, "aspect_ratio": "21:9", "ports": "HDMI, DisplayPort", "features": "1500R Curved, Frameless, 1ms, Tilt Compatible, HDR Ready, Built-in Speakers, VESA", "price": 299, "price_currency": "AUD", "review_count": 443, "average_rating": 4.5, "review_summary": "Customers appreciate the monitor's excellent color accuracy and immersive curved design, making it ideal for both work and entertainment." } ```
links
Extract all links on a webpage that match a specific description with one call: Provide the URL and a natural-language description of the kinds of links to extract. Example Usage Scenario: User wants to extract individual product URLs from an Ikea product listing page. ``` "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/?page=3", "description": "individual product urls" ``` Sample response: ``` [ "https://www.ikea.com/au/en/p/strandlummer-duvet-cover-and-pillowcase-multicolour-floral-pattern-00579802/", "https://www.ikea.com/au/en/p/gullnattljus-duvet-cover-and-2-pillowcases-white-floral-pattern-60600075/", ... ] ```
next
Extract "Next Page" links from a paginated web page with one call: Provide the URL only. Example Usage Scenario: User wants to extract the "next page" links from a paginated Ikea product listing result. ``` "url": "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/" ``` Sample response: ``` [ "https://www.ikea.com/au/en/cat/quilt-cover-sets-10680/?page=2#products-page-2", ... ] ```
search
Extract relevant Google search result URLs with one call: Provide the Google domain, search query, and page number of the results. Example Usage Scenario: User wants to extract the first page of search result URLs for the search query "white sofas" on the www.google.com domain. ``` "google_domain": "www.google.com", "query": "white sofas", "page": 1 ``` Sample response: ``` [ "https://www.ashleyfurniture.com/p/maitelynn_sofa/4820238.html?google_pla=true&utm_source=google&utm_medium=unpaidShopping®ion_id=564100", "https://www.livingspaces.com/pdp-isabella-sand-97-inch-sofa-320111?utm_test=spotlight®ion_id=808080", ... ] ```
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"web scraping api": {
"web-scraping-api-by-instantapi-ai": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse",
"--header",
"Authorization:${AUTH_HEADER}"
],
"env": {
"AUTH_HEADER": "Bearer <YOUR_API_KEY>"
}
}
}
}
}McpServers
{
"web-scraping-api-by-instantapi-ai": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse",
"--header",
"Authorization:${AUTH_HEADER}"
],
"env": {
"AUTH_HEADER": "Bearer <YOUR_API_KEY>"
}
}
}🚀 We’ve just launched our new MCP server!
If you're building or using AI agents, you can now plug our web scraping API directly into your autonomous workflows. Just add the following to your MCP config:
{
"mcpServers": {
"web-scraping-api-by-instantapi-ai": {
"command": "npx",
"args": [
"-y",
"mcp-remote",
"https://web-scraping-api-by-instantapi-ai.help-052.workers.dev/sse",
"--header",
"Authorization:${AUTH_HEADER}"
],
"env": {
"AUTH_HEADER": "Bearer <YOUR_API_KEY>"
}
}
}
}
🔐 Just replace <YOUR_API_KEY> with your API key from InstantAPI.ai. Sign up for your own API key here: https://web.instantapi.ai/#pricing-03-254921
Once connected, your AI agent can automatically invoke any of our scraping endpoints—/scrape, /links, /next, /search—to complete complex web data tasks, without needing any HTML selectors or manual logic.
🧠 Example real-world task from a user:
Go to https://www.domain.com.au/sale/bowral-nsw-2576/?bedrooms=1-any&bathrooms=1-any&excludeunderoffer=1&establishedtype=established&sort=dateupdated-desc
Extract each property link.
For each, get the full physical address and how many days since it was listed. If not known, assume 0 (today).
Keep paginating until no properties are listed in the last 7 days.
Return all matching addresses.
✅ The agent completed the task autonomously—scraping new leads for a mortgage broker with zero code written.
If you’re building AI workflows, this massively cuts down on glue code and edge case handling. Just give your agent a goal and let it fetch the data.
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
