Librecrawl Technical Seo Audit Mcp

by adityaarsharma

463 downloads

Not rated

GitHub

Developer Tools

About

Open-source self-hosted technical SEO audit MCP server built on the LibreCrawl engine. Runs full audits inside Claude, Cursor, or Codex — 37 tools, 50+ checks per audit (hreflang, schema.org Rich Results, security headers, image performance, WAF detection on 200-OK pages), chunke

Details

Author: adityaarsharma
Downloads: 463
Categories: Developer Tools

- Unlimited pages with chunked-progressive crawling (no timeout)
- 50+ technical SEO checks including hreflang, schema, and redirects
- WAF/bot-block detection (Cloudflare, Akamai, DataDome, Imperva, PerimeterX)
- Auto-adaptive crawl delay using AIMD algorithm
- Ephemeral by design — session data deleted after download
- Branded PDF reports ready for client delivery
- Google Search Console merge support

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name Librecrawl Technical Seo Audit Mcp

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

From the repository

Install with the provided curl command or manually using Python and Docker. The installer configures your MCP client. To use it, simply ask your AI assistant to audit a website — the assistant calls the server’s 37 MCP tools to run the crawl and return results.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "librecrawl technical seo audit mcp": {
            "librecrawl": {
                "command": "npx",
                "args": [
                    "-y",
                    "mcp-remote",
                    "http://127.0.0.1:5081/mcp"
                ]
            }
        }
    }
}

McpServers

{
    "librecrawl": {
        "command": "npx",
        "args": [
            "-y",
            "mcp-remote",
            "http://127.0.0.1:5081/mcp"
        ]
    }
}

🕷️ librecrawl-technical-seo-audit-mcp

The AI-native technical SEO crawler.

Run a complete on-site SEO audit on any website — straight from Claude, Cursor, Codex, or any Model Context Protocol (MCP) client. Unlimited pages · 50+ checks · PDF + CSVs · MIT-licensed · self-hosted · ephemeral by design.

Built on the open-source LibreCrawl engine, exposed through 37 MCP tools your AI assistant calls directly.

⚡ Install in 60s · 🪄 What it does · 🚀 50+ checks · 🆚 Compare · 📖 Quick start

</div>

---

🤔 Don't know what an MCP is? Read this 30-second explainer

> Model Context Protocol (MCP) is the open standard that lets AI assistants like Claude, Cursor, or Codex call external tools. Think of it as "USB for AI assistants" — you plug a tool in, the AI can use it. librecrawl-technical-seo-audit-mcp is one of those tools. Once installed, you just ask your AI assistant to audit a site, and it does. No GUI. No dashboard. No exports.

New to all this?
- Don't have Claude Code yet? → Install Claude Code (free for individuals).
- Prefer Cursor? → Get Cursor.
- Already have one of those? → Skip to Install in 60s.

---

🪄 The whole pitch in 4 lines

You:    Audit https://acme.com — full site, no caps, give me the zip
Agent:  → librecrawl_start_chunked_audit · polls until done · saves zip locally
You:    Show me broken pages + broken external links + hreflang errors
Agent:  → reads CSVs, prints filtered tables. Server already forgot the audit.

That's the product. Your AI assistant runs a full technical SEO audit for you. You get a branded PDF + 7 CSVs covering 50+ technical checks, ready to hand a client. The server wipes everything the moment you download.

---

🔥 Why this exists

There are great desktop SEO crawlers (you know the ones). There are great cloud SEO suites. There was no AI-native crawler. librecrawl-technical-seo-audit-mcp fills that gap with five things no comparable open-source MCP server does:

⚡ It runs inside your AI assistant

37 MCP tools your agent calls directly. No GUI app to babysit, no SaaS dashboard to log into, no CSV exports to upload to ChatGPT. You just ask.

🚀 Chunked-progressive crawler that never times out

Most SEO MCP servers (SiteAudit MCP, AgentAEO, SE Ranking MCP) run synchronously and disconnect on sites over a few hundred pages. librecrawl-technical-seo-audit-mcp runs the crawl in a background worker thread, persists progress to SQLite WAL, and returns a session_id in under 2 seconds. Your agent polls a tiny status tool until done. 10,000-page enterprise sites work the same as 50-page blogs. Survives PM2 / MCP-client restarts mid-crawl.

🛡️ Catches WAF challenges other crawlers silently misreport

Cloudflare, Akamai, DataDome, Imperva, and PerimeterX challenge pages are served as 200 OK but contain a JavaScript challenge instead of your content. Most crawlers report these as "page OK, all good". librecrawl-technical-seo-audit-mcp fingerprints the challenge in the response body and flags bot_block_challenge_detected. You see what's actually broken.

🤖 An AIMD controller tunes crawl delay live

Additive-Increase / Multiplicative-Decrease — the same algorithm TCP congestion control uses. Error rate > 10% → halve chunk, double delay. p95 latency > 1.5× target → 1.5× delay. Clean signals → additive decrease. Polite by construction. No rate-limit blow-ups. No manual tuning. Respects robots.txt Crawl-Delay floor.

🧹 Ephemeral by design — the agency-safe default

Once you download the zip, the server deletes the session row, every artifact file on disk, AND the upstream LibreCrawl crawl record. Per-audit server footprint after cleanup: 0 bytes, 0 rows. Auditing 50 client sites? Zero data persists where another operator could see it.

📄 Branded PDF reports ready to hand a client

WeasyPrint, A4, page numbers, footer on every page. Open in any PDF viewer. No SaaS watermark. Hand it to a client as your work.

---

⚡ Install in 60 seconds

curl -fsSL https://raw.githubusercontent.com/adityaarsharma/librecrawl-technical-seo-audit-mcp/main/install.sh | bash

The installer asks 3 questions (target client, optional Google PageSpeed API key, optional GSC integration) and writes a ready-to-use MCP entry into your Claude / Cursor / Codex / Windsurf config. Done.

<details>
<summary><strong>What if I'm not a developer?</strong></summary>

You don't need to be. If you can:
1. Open a terminal (macOS: Cmd+Space → "Terminal" · Windows: Win+R → "powershell")
2. Paste the curl command above
3. Answer 3 yes/no questions

…you're done. The installer handles Python, Docker, the LibreCrawl backend, and your AI client config. First-audit-to-zip is under 10 minutes from cold start.

</details>

<details>
<summary><strong>Manual install (Python 3.10+, Docker for LibreCrawl backend)</strong></summary>

```bash
git clone https://github.com/adityaarsharma/librecrawl-technical-seo-audit-mcp.git
cd librecrawl-technical-seo-audit-mcp
python3 -m venv venv && source venv/bin/activate
pip install httpx mcp weasyprint markdown fpdf2

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Details

Jump to

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

🕷️ librecrawl-technical-seo-audit-mcp

The AI-native technical SEO crawler.

🤔 Don't know what an MCP is? Read this 30-second explainer

🪄 The whole pitch in 4 lines

🔥 Why this exists

⚡ It runs inside your AI assistant

🚀 Chunked-progressive crawler that never times out

🛡️ Catches WAF challenges other crawlers silently misreport

🤖 An AIMD controller tunes crawl delay live

🧹 Ephemeral by design — the agency-safe default

📄 Branded PDF reports ready to hand a client

⚡ Install in 60 seconds

Install

More like this

Sentry

Github

Git

GitLab

Context 7

Playwright