Desktop Automation

Automate desktop actions and interact with your local environment using LLM applications.

A Model Context Protocol (MCP) server that exposes desktop automation capabilities, allowing LLM applications to interact with your desktop environment through standardized tools.

This MCP server provides a bridge between LLM applications and desktop automation functionality. It exposes mouse, keyboard, and screen automation capabilities through the MCP protocol, enabling AI assistants to:

- Click, right-click, and double-click at specific coordinates
- Move the mouse cursor with smooth or instant movement
- Type text with optional character delays
- Press individual keys or key combinations
- Get current mouse cursor position
- Capture screenshots of the desktop
- Get screen dimensions

- Click: Left click at specified coordinates
- Right Click: Right click at specified coordinates
- Double Click: Double click at specified coordinates
- Move Mouse: Move cursor with optional smooth animation
- Get Position: Retrieve current mouse coordinates

- Type Text: Type text at current cursor position
- Press Key: Press individual keys or key combinations
- Delayed Typing: Type with configurable delays between characters

- Take Screenshot: Capture the full screen and save to file
- Get Screen Size: Retrieve screen dimensions

- Go 1.23.0 or later
- Task runner (optional, for using Taskfile commands)

# Install Task (optional) go install github.com/go-task/task/v3/cmd/task@latest

# Clone the repository (if not already available) git clone <repository-url> cd desktop-automation-mcp # Download dependencies go mod download go mod tidy # Build the server go build -o mcp-server ./cmd/mcp-server # Or using Task task build

The server uses stdio transport for communication:

# Run directly ./mcp-server # Or using Task task run

Configure your LLM application (Claude Desktop, etc.) to connect to this MCP server:

{ "mcpServers": { "desktop-automation": { "command": "/path/to/mcp-server" } } }

- x(number, required): X coordinate
- y(number, required): Y coordinate

Right click at specified screen coordinates.

- x(number, required): X coordinate
- y(number, required): Y coordinate

Double click at specified screen coordinates.

- x(number, required): X coordinate
- y(number, required): Y coordinate

Move mouse cursor to specified coordinates.

- x(number, required): X coordinate
- y(number, required): Y coordinate
- smooth(boolean, optional): Use smooth movement animation
- duration(number, optional): Duration for smooth movement in seconds (default: 1.0)

- text(string, required): Text to type
- delay(number, optional): Delay between characters in milliseconds

- key(string, required): Key to press (e.g., 'enter', 'space', 'ctrl')
- modifiers(array, optional): Modifier keys (e.g., ['ctrl', 'shift'])

- path(string, optional): Path to save the screenshot (if not provided, saves to temp directory)

desktop-automation-mcp/ ├── cmd/ │ └── mcp-server/ # MCP server entry point │ └── main.go ├── internal/ │ └── automation/ # Desktop automation logic (copied from desktop-automation) │ ├── keyboard.go │ ├── mouse.go │ └── screen.go ├── go.mod # Go module definition ├── Taskfile.yml # Task runner configuration ├── .gitignore # Git ignore rules └── README.md # This file

# Build the server task build # Run the server task run # Clean build artifacts task clean # Download and tidy dependencies task deps # Run tests task test # Install to GOPATH/bin task install

# Build go build -o mcp-server ./cmd/mcp-server # Run ./mcp-server # Test go test ./... # Clean go clean rm -f mcp-server

- github.com/mark3labs/mcp-go: MCP protocol implementation for Go
- github.com/go-vgo/robotgo: Cross-platform desktop automation library

- desktop-automation: The original CLI-based desktop automation tool
- mcp-go: Go implementation of the Model Context Protocol

An MCP server for integrating with the Claude Desktop application on macOS. Requires the Claude Desktop app to be installed and configured.