ScreenPilot

by Mtehabsim

54 stars

367 downloads

Not rated

GitHub

About

Tool that allows the AI to control your device in the same way you do, enabling automation for everything!

Details

Author: Mtehabsim
GitHub stars: 54
Downloads: 367
Categories: Other, Automation

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name ScreenPilot

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

From the repository

Install Python 3.12, clone the repository, create and activate a virtual environment, install the required packages from requirements.txt, then configure the server in Claude AI Desktop’s MCP settings by adding the full paths to python.exe and main.py in the config file. After saving and restarting Claude, ScreenPilot’s tools become available.

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "screenpilot": {
            "ScreenPilot": {
                "command": "python",
                "args": [
                    "-m",
                    "venv",
                    "venv"
                ]
            }
        }
    }
}

McpServers

{
    "ScreenPilot": {
        "command": "python",
        "args": [
            "-m",
            "venv",
            "venv"
        ]
    }
}

ScreenPilot

MCP server to let LLM take full control on your device by providing screen automation toolkit for controlling and interacting with graphical user interfaces. Good for automation, education and having fun.

Main Features

- 📷 Screen capture and analysis
- 🖱️ Mouse control (clicking, positioning)
- ⌨️ Keyboard input (typing, key presses, hotkeys)

watch demo

https://github.com/user-attachments/assets/c18380c0-b3dd-4b7c-925d-28ef205ca11f

Installation

0. Install python 3.12 1. Clone the repository:

   git clone https://github.com/Mtehabsim/ScreenPilot.git

2. create virtiual environment


python -m venv venv

3. activate the env

venv\Scripts\activate

4. Install the required packages:

   pip install -r requirements.txt

5. Open Claude AI desktop 6. file -> settings -> developer -> edit config 7. open config file and paste this

{
    "mcpServers": {
        "device-controll": {
            "command": "pathToEnv\\venv\\Scripts\\python.exe",
            "args": [
                "pathToProject\\ScreenPilot\\main.py"
            ]
        }
    }
}

8. Replace
"pathToEnv\\venv\\Scripts\\python.exe" → with the full path to your python.exe
"pathToProject\\ScreenPilot\\main.py" → with the full path to your main.py file

9. Save the config file.

10. Open Claude AI Desktop.

11. Go to File → Exit

12. You can now open Claude AI Desktop and enjoy ScreenPilot.

Available Tools

- Screen Capture: Take screenshots and get screen information
- Mouse Control: Move the mouse and perform clicks
- Keyboard Actions: Type text, press keys, and use hotkey combinations
- Scrolling: Scroll in different directions and to specific positions
- Element Detection: Check if elements exist on screen and wait for them to appear
- Action Sequences: Perform multiple actions in sequence

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Details

Jump to

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

ScreenPilot

Main Features

watch demo

Installation

Available Tools

Contributing

Install

More like this

@mendableai/firecrawl Mcp Server

Umut Naci

Playwright

Sequential Thinking

Filesystem

EverArt