PyAutoGUI

by hetaobackend

8 stars

1.9k downloads

Not rated

GitHub

About

Enables automated GUI testing and control across operating systems by wrapping PyAutoGUI to perform mouse movements, keyboard input, screenshot capture, and image recognition tasks.

Details

Author: hetaobackend
Repository: hetaoBackend/mcp-pyautogui-server
GitHub stars: 8
Downloads: 1,852
License: MIT License
Categories: Developer Tools, Automation, Productivity, Design, AI, Media, Search, Frontend, Project Management, Other

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below
Plugin Name PyAutoGUI

Command (node, npx, python, etc.) uv
Arguments
- Argument 1 --directory
- Argument 2 /path/to/mcp-pyautogui-server
- Argument 3 run
- Argument 4 mcp-pyautogui-server
Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uvx"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uvx"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uvx"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uvx"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uvx"
        }
    }
}

JSON

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

From the repository

Install the package:

pip install mcp-pyautogui-server

On MacOS:

~/Library/Application\ Support/Claude/claude_desktop_config.json

On Windows:

%APPDATA%/Claude/claude_desktop_config.json

Development/Unpublished Servers Configuration:

{

  "mcpServers": {

    "mcp-pyautogui-server": {

      "command": "uv",

      "args": [

        "--directory",

        "/path/to/mcp-pyautogui-server",

        "run",

        "mcp-pyautogui-server"

      ]

    }

  }

}

Published Servers Configuration:

{

  "mcpServers": {

    "mcp-pyautogui-server": {

      "command": "uvx",

      "args": [

        "mcp-pyautogui-server"

      ]

    }

  }

}

`move_mouse`

Move mouse to specific coordinates. Parameters: x (int), y (int)

`click_mouse`

Click at current or specified position. Parameters: position (optional tuple of int), button (optional string)

`drag_and_drop`

Perform drag and drop operations. Parameters: start_position (tuple of int), end_position (tuple of int)

`get_mouse_position`

Get current mouse position. Returns: (x, y) coordinates

`type_text`

Simulate typing text. Parameters: text (string)

`press_key`

Press individual keys. Parameters: key (string)

`hotkey`

Simulate hotkey combinations. Parameters: keys (list of strings)

`take_screenshot`

Take a screenshot of the current screen. Returns: image data

`get_screen_size`

Get screen size. Returns: (width, height) tuple

`find_image`

Find image locations on screen. Parameters: image (string), confidence (optional float)

`get_pixel_color`

Get pixel colors at specific coordinates. Parameters: x (int), y (int) Returns: color value

The server implements the following tools:

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "pyautogui": {
            "cwd": "optional",
            "env": {},
            "args": [
                "--directory",
                "/path/to/mcp-pyautogui-server",
                "run",
                "mcp-pyautogui-server"
            ],
            "shell": false,
            "command": "uv"
        }
    }
}

Linux

{
    "cwd": "optional",
    "env": [],
    "args": [
        "--directory",
        "/path/to/mcp-pyautogui-server",
        "run",
        "mcp-pyautogui-server"
    ],
    "shell": false,
    "command": "uv"
}

Macos

{
    "cwd": "optional",
    "env": [],
    "args": [
        "--directory",
        "/path/to/mcp-pyautogui-server",
        "run",
        "mcp-pyautogui-server"
    ],
    "shell": false,
    "command": "uv"
}

Windows

{
    "cwd": "optional",
    "env": [],
    "args": [
        "mcp-pyautogui-server"
    ],
    "shell": false,
    "command": "uvx"
}

An MCP server for PyAutoGUI that enables automated GUI testing and control, including mouse and keyboard actions, screenshots, and screen information.

A MCP (Model Context Protocol) server that provides automated GUI testing and control capabilities through PyAutoGUI.

- Control mouse movements and clicks
- Simulate keyboard input
- Take screenshots
- Find images on screen
- Get screen information
- Cross-platform support (Windows, macOS, Linux)

The server implements the following tools:

- Move mouse to specific coordinates
- Click at current or specified position
- Drag and drop operations
- Get current mouse position

- Type text
- Press individual keys
- Hotkey combinations

- Take screenshots
- Get screen size
- Find image locations on screen
- Get pixel colors

- Python 3.12+
- PyAutoGUI
- Other dependencies will be installed automatically

~/Library/Application\ Support/Claude/claude_desktop_config.json

%APPDATA%/Claude/claude_desktop_config.json

Development/Unpublished Servers Configuration:

{ "mcpServers": { "mcp-pyautogui-server": { "command": "uv", "args": [ "--directory", "/path/to/mcp-pyautogui-server", "run", "mcp-pyautogui-server" ] } } }

{ "mcpServers": { "mcp-pyautogui-server": { "command": "uvx", "args": [ "mcp-pyautogui-server" ] } } }

Note: Set PyPI credentials via environment variables or command flags:

- Token:--tokenorUV_PUBLISH_TOKEN
- Username/password:--username/UV_PUBLISH_USERNAMEand--password/UV_PUBLISH_PASSWORD

For the best debugging experience, use the MCP Inspector.

npx @modelcontextprotocol/inspector uv --directory /path/to/mcp-pyautogui-server run mcp-pyautogui-server

The Inspector will display a URL that you can access in your browser to begin debugging.

This project is licensed under the MIT License - see the LICENSE file for details.

This is a web browser that enables your coding agent, such as Claude Code, to visit websites on your behalf and assist you in identifying bugs or creating UI test cases.

Browser automation via MCP for Chrome and Firefox

Bring the full power of BrowserStack’s Test Platform to your AI tools, making testing faster and easier for every developer and tester on your team.

Official Chrome DevTools MCP server for controlling and inspecting a live Chrome browser from coding agents such as Gemini, Claude, Cursor, and Copilot.

AI 測試大師 — MCP server driving pytest / Jest / Cypress / Go / Maestro. Analyze, generate, run, advise. Web + Mobile (iOS/Android/BlueStacks).

A Playwright-based MCP server that exposes a live browser as a traceable, inspectable, debuggable and controllable execution environment for AI agents.

This MCP allows you to record browser interactions with a chrome extension that include screenshots and console logs. The data is then saved to a local database that feeds the data to an AI system like claude code to search.

Control Chrome with debugging capabilities, userscript injection, and extension support.

Drive Electron apps from AI agents via MCP - click, type, drag, screenshot, eval JS, and more.

AI-powered E2E testing for 10 platforms. 253 MCP tools. Zero config. Test Flutter, React Native, iOS, Android, Web, Electron, Tauri, KMP, .NET MAUI from natural language.

Programmatically control Windows mouse, keyboard, window management, screen capture, and clipboard operations.

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

About

Details

Jump to

Setting up with Highlight

From the repository

move_mouse

click_mouse

drag_and_drop

get_mouse_position

type_text

press_key

hotkey

take_screenshot

get_screen_size

find_image

get_pixel_color

Claude Desktop / Cursor

Linux

Macos

Windows

Install

More like this

@mendableai/firecrawl Mcp Server

Umut Naci

Playwright

Time

Sentry

Github

`move_mouse`

`click_mouse`

`drag_and_drop`

`get_mouse_position`

`type_text`

`press_key`

`hotkey`

`take_screenshot`

`get_screen_size`

`find_image`

`get_pixel_color`