MCP-Airflow-API

Key Architecture: Single MCP server with shared common tools (43) plus v2-exclusive asset tools (2) - dynamically loads appropriate toolset based onAIRFLOW_API_VERSIONenvironment variable!

curl -X GET "http://localhost:8080/api/v1/dags?limit=100&offset=0" \ -H "Authorization: Basic YWlyZmxvdzphaXJmbG93"

MCP-Airflow-API approach (natural language):

"Show me the currently running DAGs"

🎯 Recommended: Docker Compose (Complete Demo Environment)

git clone https://github.com/call518/MCP-Airflow-API.git cd MCP-Airflow-API # Configure your Airflow credentials cp .env.example .env # Edit .env with your Airflow API settings # Start all services docker-compose up -d # Access OpenWebUI at http://localhost:3002/ # API documentation at http://localhost:8002/docs

Getting Started with OpenWebUI (Docker Option)

Method 2: Claude-Desktop MCP Client Integration

git clone https://github.com/call518/MCP-Airflow-API.git cd MCP-Airflow-API pip install -e . # Run in stdio mode python -m mcp_airflow_api

-

Natural Language Queries
No need to learn complex API syntax. Just ask as you would naturally speak:

- "What DAGs are currently running?"
- "Show me the failed tasks"
- "Find DAGs containing ETL"

Comprehensive Monitoring Capabilities
Real-time cluster status monitoring:

- Cluster health monitoring
- DAG status and performance analysis
- Task execution log tracking
- XCom data management

Dynamic API Version Support
Single MCP server adapts to your Airflow version:

- API v1: 43 shared tools for Airflow 2.x compatibility
- API v2: 43 shared tools + 2 asset management tools for Airflow 3.0+
- Environment Variable Control: Switch versions instantly withAIRFLOW_API_VERSION
- Zero Configuration Changes: Same tool names, enhanced capabilities
- Efficient Architecture: Shared common codebase eliminates duplication

Comprehensive Tool Coverage
Covers almost all Airflow API functionality:

- DAG management (trigger, pause, resume)
- Task instance monitoring
- Pool and variable management
- Connection configuration
- Configuration queries
- Event log analysis

Large Environment Optimization
Efficiently handles large environments with 1000+ DAGs:

- Smart pagination support
- Advanced filtering options
- Batch processing capabilities

-

Leveraging Model Context Protocol (MCP)
MCP is an open standard for secure connections between AI applications and data sources, providing:

- Standardized interface
- Secure data access
- Scalable architecture

- stdiomode: Direct MCP client integration for local environments
- streamable-httpmode: HTTP-based deployment for Docker and remote access

FASTMCP_TYPE=stdio # Default: Direct MCP client mode FASTMCP_TYPE=streamable-http # Docker/HTTP mode FASTMCP_PORT=8000 # HTTP server port (Docker internal)

Comprehensive Airflow API Coverage
Full implementation of official Airflow REST APIs:

Complete Docker Support
Full Docker Compose setup with 3 separate services:

- Open WebUI: Web interface (port3002)
- MCP Server: Airflow API tools (internal port8000, exposed via18002)
- MCPO Proxy: REST API endpoint provider (port8002)

# Required - Dynamic API Version Selection (NEW!) # Single server supports both v1 and v2 - just change this variable! AIRFLOW_API_VERSION=v1 # v1 for Airflow 2.x, v2 for Airflow 3.0+ AIRFLOW_API_BASE_URL=http://localhost:8080/api # Test Cluster Connection Examples: # For Airflow 2.x test cluster (from Airflow-Docker-Compose) AIRFLOW_API_VERSION=v1 AIRFLOW_API_BASE_URL=http://localhost:38080/api # For Airflow 3.x test cluster (from Airflow-Docker-Compose) AIRFLOW_API_VERSION=v2 AIRFLOW_API_BASE_URL=http://localhost:48080/api # Authentication AIRFLOW_API_USERNAME=airflow AIRFLOW_API_PASSWORD=airflow # Optional - MCP Server Configuration MCP_LOG_LEVEL=INFO # DEBUG/INFO/WARNING/ERROR/CRITICAL FASTMCP_TYPE=stdio # stdio/streamable-http FASTMCP_PORT=8000 # HTTP server port (Docker mode) # Bearer Token Authentication for streamable-http mode # Enable authentication (recommended for production) # Default: false (when undefined, empty, or null) # Values: true/false, 1/0, yes/no, on/off (case insensitive) REMOTE_AUTH_ENABLE=false # true/false REMOTE_SECRET_KEY=your-secure-secret-key-here

Forstreamable-httpmode, this MCP server supports Bearer token authentication to secure remote access. This is especially important when running the server in production environments.

# In .env file REMOTE_AUTH_ENABLE=true REMOTE_SECRET_KEY=your-secure-secret-key-here

python -m mcp_airflow_api --type streamable-http --auth-enable --secret-key your-secure-secret-key-here

- stdio mode(Default): Local-only access, no authentication needed
- streamable-http + REMOTE_AUTH_ENABLE=false: Remote access without authentication ⚠️NOT RECOMMENDED for production
- streamable-http + REMOTE_AUTH_ENABLE=true: Remote access with Bearer token authentication ✅RECOMMENDED for production

Note:REMOTE_AUTH_ENABLEdefaults tofalsewhen undefined, empty, or null. Supported values aretrue/false,1/0,yes/no,on/off(case insensitive).

When authentication is enabled, MCP clients must include the Bearer token in the Authorization header:

{ "mcpServers": { "mcp-airflow-api": { "type": "streamable-http", "url": "http://your-server:8000/mcp", "headers": { "Authorization": "Bearer your-secure-secret-key-here" } } } }

- Always enable authenticationwhen using streamable-http mode in production
- Use strong, randomly generated secret keys(32+ characters recommended)
- Use HTTPSwhen possible (configure reverse proxy with SSL/TLS)
- Restrict network accessusing firewalls or network policies
- Rotate secret keys regularlyfor enhanced security
- Monitor access logsfor unauthorized access attempts

When authentication fails, the server returns:

- 401 Unauthorizedfor missing or invalid tokens
- Detailed error messagesin JSON format for debugging

version: '3.8' services: mcp-server: build: context: . dockerfile: Dockerfile.MCP-Server environment: - FASTMCP_PORT=8000 - AIRFLOW_API_VERSION=v1 - AIRFLOW_API_BASE_URL=http://your-airflow:8080/api - AIRFLOW_API_USERNAME=airflow - AIRFLOW_API_PASSWORD=airflow

git clone https://github.com/call518/MCP-Airflow-API.git cd MCP-Airflow-API pip install -e . # Run in stdio mode python -m mcp_airflow_api

git clone https://github.com/call518/Airflow-Docker-Compose.git cd Airflow-Docker-Compose

For testingAPI v1compatibility with stable production features:

# Navigate to Airflow 2.x environment cd airflow-2.x # (Optional) Customize environment variables cp .env.template .env # Edit .env file as needed # Deploy Airflow 2.x cluster ./run-airflow-cluster.sh # Access Web UI # URL: http://localhost:38080 # Username: airflow / Password: airflow

- Image:apache/airflow:2.10.2
- Port:38080(configurable viaAIRFLOW_WEBSERVER_PORT)
- API:/api/v1/endpoints
- Authentication: Basic Auth
- Use case: Production-ready, stable features

# Navigate to Airflow 3.x environment cd airflow-3.x # (Optional) Customize environment variables cp .env.template .env # Edit .env file as needed # Deploy Airflow 3.x cluster ./run-airflow-cluster.sh # Access API Server # URL: http://localhost:48080 # Username: airflow / Password: airflow

- Image:apache/airflow:3.0.6
- Port:48080(configurable viaAIRFLOW_APISERVER_PORT)
- API:/api/v2/endpoints + Assets management
- Authentication: JWT Token (FabAuthManager)
- Use case: Development, testing new features

Option 3: Deploy Both Versions Simultaneously

For comprehensive testing across different Airflow versions:

# Start Airflow 2.x (port 38080) cd airflow-2.x && ./run-airflow-cluster.sh # Start Airflow 3.x (port 48080) cd ../airflow-3.x && ./run-airflow-cluster.sh

To stop and clean up the test environments:

# For Airflow 2.x cd airflow-2.x && ./cleanup-airflow-cluster.sh # For Airflow 3.x cd airflow-3.x && ./cleanup-airflow-cluster.sh

- Scalable design and modular structure for easy addition of new features
- Standards-compliant protocol for integration with other tools
- Cloud-native operations and LLM-ready interface
- Context-aware query processing and automated workflow management capabilities

- Data Engineers— Reduce debugging time, improve productivity, minimize learning curve
- DevOps Engineers— Automate infrastructure monitoring, reduce incident response time
- System Administrators— User-friendly management without complex APIs, real-time cluster status monitoring

🚀 Open Source Contribution and Community

- Bug reports and feature suggestions
- Documentation improvements
- Code contributions

Please consider starring the project if you find it useful.

MCP-Airflow-API changes the paradigm of data engineering and workflow management:
No need to memorize REST API calls — just ask in natural language:

"Show me the status of currently running ETL jobs."