MCP-RSS-Crawler
About
Fetches and caches RSS feeds using a SQLite database for use with LLMs via the MCP protocol.
Details
- Author
- mshk
- GitHub stars
- 19
- Downloads
- 366
- Categories
- Web Scraping, Other, Automation
Jump to
- Fetches and caches RSS feeds using a SQLite database
- Implements the MCP protocol for seamless LLM integration
- Supports filtering feeds by category, source, or keywords
- Provides API endpoints to add, update, and delete feeds
- Supports fetching articles from Firecrawl
Setting up with Highlight
This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:
- Download and install Highlight from highlightai.com/download
- Navigate to the plugins tab and select "Add Custom Plugin"
-
Configure the plugin with the settings below
Plugin Name
MCP-RSS-CrawlerCommand (node, npx, python, etc.)Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
- Enable "Start Automatically" if you want the plugin to start when Highlight launches
From the repository
Clone the repository, configure a claude_desktop_config.json with the path to Bun, the server script, and environment variables (PORT, DB_DIR, FIRECRAWL_API_KEY). Install dependencies with bun install, then start Claude Desktop. The MCP endpoint accepts POST requests with a JSON body containing a messages array.
Claude Desktop / Cursor
Paste into your MCP client config file to install this server.
{
"mcpServers": {
"mcp-rss-crawler": {
"rss-crawler": {
"command": "/path/to/bun",
"args": [
"run",
"/path/to/mcp-rss-crawler/apps/mcp-server/src/mcp-cli.ts"
],
"cwd": "/path/to/mcp-rss-crawler",
"env": {
"PORT": "5556",
"DB_DIR": "/path/to/mcp-rss-crawler",
"FIRECRAWL_API_KEY": "fc-<YOUR_FIRECRAWL_API_KEY>"
}
}
}
}
}McpServers
{
"rss-crawler": {
"command": "/path/to/bun",
"args": [
"run",
"/path/to/mcp-rss-crawler/apps/mcp-server/src/mcp-cli.ts"
],
"cwd": "/path/to/mcp-rss-crawler",
"env": {
"PORT": "5556",
"DB_DIR": "/path/to/mcp-rss-crawler",
"FIRECRAWL_API_KEY": "fc-<YOUR_FIRECRAWL_API_KEY>"
}
}
}Fetches and caches RSS feeds using a SQLite database for use with LLMs via the MCP protocol.
An MCP (Message Chain Protocol) server that fetches RSS feeds and shares them with LLMs.
- Fetching and caching of RSS feeds (SQLite database)
- MCP protocol implementation for seamless LLM integration
- Support for filtering feeds by category, source, or keywords
- Comprehensive API endpoints for feed management
- Add, update, and delete feeds
- Bun
- Firecrawl API key
- Claude Desktop or other MCP client
- Clone this repository
- Create aclaude_desktop_config.jsonfile based onclaude_desktop_config.json.examplewith your configuration
{ "mcpServers": { "rss-crawler": { "command": "/path/to/bun", "args": ["run", "/path/to/mcp-rss-crawler/apps/mcp-server/src/mcp-cli.ts"], "cwd": "/path/to/mcp-rss-crawler", "env": { "PORT": "5556", "DB_DIR": "/path/to/mcp-rss-crawler", "FIRECRAWL_API_KEY": "fc-<YOUR_FIRECRAWL_API_KEY>" } } } }
The server implements the Message Chain Protocol (MCP) which allows LLMs to access your latest RSS feeds. The MCP endpoint accepts POST requests with a JSON body containing a messages array and returns a response with the latest feed items.
{ "messages": [ { "role": "user", "content": "What are the latest news from my RSS feeds?" } ] }
{ "messages": [ { "role": "assistant", "content": "Here are the latest articles from your RSS feeds:", "name": "rss-mcp" }, { "role": "tool", "content": "[{\"title\":\"Article Title\",\"summary\":\"Article summary...\",\"published\":\"2025-03-16T04:30:00.000Z\",\"origin\":\"Feed Name\",\"link\":\"https://example.com/article\"}]", "name": "rss-feeds" } ] }
The server can be configured through environment variables or a.envfile:
- PORT- Server port (default: 5556)
- FIRECRAWL_API_KEY- Firecrawl API key
- DB_DIR- Database directory (default:~/.mcp-rss-crawler)
- For connection issues, check your network settings and firewall configuration
- Logs are available in the console and can be used to diagnose problems
- For more detailed logging, set theDEBUG=mcp-rss:*environment variable
Data scraping, conversion, and extraction tools from Dumpling AI.
Fetches and caches daily articles from GeekNews using web scraping.
Fetches and parses stories from Hacker News, providing structured data for top, new, ask, show, and job posts.
Real-time news aggregation from AP, BBC, NPR, Hacker News, and Google News
A web crawler and content extractor that supports multiple output formats like text, markdown, and JSON.
Fetches RSS feeds and returns entries as formatted text, with support for OPML configuration.
Collects and summarizes technical articles from sources like Qiita, Dev.to, NewsAPI, and Hacker News using the Gemini API.
Aggregates trending topics from over 20 sources in real-time, with customizable fields and RSS feed support.
Interact with Yahoo Finance to get stock data, market news, and financial information using the yfinance Python library.
Fetch stock data, news, and financial information from Yahoo Finance.
Sign in to leave a review
Use Google, GitHub, or an email account so ratings stay tied to real people.
No reviews posted yet.
