Computer Use AI SDK

by m13v

195 stars

313 downloads

Not rated

GitHub

Developer Tools

Description

# Computer Use AI SDK * We've built an MCP server that controls computer * You've heard of OpenAI's operator, you've heard of Claude's computer use. Now the open source alternative: Computer Use SDK from screenpipe. * It's native on macOS—no virtual machine bs, no guardrails…

About

Details

Author: m13v
GitHub stars: 195
Downloads: 313
Categories: Developer Tools

- Launches apps, reads content, clicks, enters text, and presses keys.
- Uses desktop‑rendered elements for speed and reliability.
- Native on macOS with no virtual machine overhead.
- No guardrails — full control over any app or website.
- Provides both CLI and web‑based client interfaces.
- Integrates with Anthropic’s API for AI agent capabilities.

Setting up with Highlight

This MCP is not yet compatible with Highlight’s one-click setup. However, you can still use it with Highlight by following these steps:

Download and install Highlight from highlightai.com/download
Navigate to the plugins tab and select "Add Custom Plugin"
Configure the plugin with the settings below

Plugin Name Computer Use AI SDK

Command (node, npx, python, etc.)

Please refer to the README for specific instructions on how to obtain API keys or other required environment variables.
Enable "Start Automatically" if you want the plugin to start when Highlight launches

Setting up with Cursor

Open Cursor's settings
Navigate to the MCP section
Click "Add new Global MCP server"
Copy and paste the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Claude Desktop

Open Claude Desktop's settings
Go to the Integrations tab
Click "Add MCP Server"
Use the configuration below based on your operating system

Note: Make sure Claude Desktop is running before attempting to connect.

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Windsurf

Open Windsurf's preferences
Go to the Integrations section
Click "Add MCP Integration"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Cline

Open Cline's settings
Navigate to the Integrations section
Click "Add MCP Server"
Copy and paste the configuration below based on your operating system

Note: Make sure to restart Cline after adding the configuration.

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

Setting up with Goose

Open Goose's preferences
Go to the Integrations tab
Select "MCP Servers"
Click "Add New Server"
Use the configuration below based on your operating system

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

JSON

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

From the repository

Clone the repository, install Rust and Node.js, then run the backend server with cargo run --bin server in the mcp-server-os-level directory. Choose between a CLI interface (set the ANTHROPIC_API_KEY environment variable and run npx tsx main.ts in mcp-client-cli-interface) or a web‑app interface (set the key in a .env file and run npm run dev in mcp-client-nextjs).

Claude Desktop / Cursor

Paste into your MCP client config file to install this server.

{
    "mcpServers": {
        "computer use ai sdk": {
            "MCP-server-client-computer-use-ai-sdk": {
                "command": "npx",
                "args": [
                    "tsx",
                    "main.ts"
                ]
            }
        }
    }
}

McpServers

{
    "MCP-server-client-computer-use-ai-sdk": {
        "command": "npx",
        "args": [
            "tsx",
            "main.ts"
        ]
    }
}

Computer Use AI SDK

We've built an MCP server that controls computer

You've heard of OpenAI's operator, you've heard of Claude's computer use. Now the open source alternative: Computer Use SDK from screenpipe.

It's native on macOS—no virtual machine bs, no guardrails. Use it with any app or website however you want.

No pixel-based bs—it relies on underlying desktop-rendered elements, making it much faster and far more reliable than pixel-based vision models.

You can now build your own agents getting started with our simple Hello World Template using our MCP server and client.

There are tools that our MCP Server provides out of the box:
Launch apps
Read content
Click
Enter text
Press keys

These will be computational primitives to allow the AI to control your computer and do your tasks for you. What will you build? Come check us out at https://screenpi.pe

Demos

agent sending a message

https://github.com/user-attachments/assets/f8687500-9a8c-4a96-81b6-77562feff093

get latest whatsapp messages

open arc browser

Get started

git clone https://github.com/m13v/computer-use-ai-sdk.git
cd MCP-server-client-computer-use-ai-sdk

```bash

No reviews yet — be the first

Use Google, GitHub, or an email account so ratings stay tied to real people.

Email sign in

No reviews posted yet.

Computer Use AI SDK

Description

About

Details

Jump to

Setting up with Highlight

Setting up with Cursor

JSON

JSON

JSON

Setting up with Claude Desktop

JSON

JSON

JSON

Setting up with Windsurf

JSON

JSON

JSON

Setting up with Cline

JSON

JSON

JSON

Setting up with Goose

JSON

JSON

JSON

From the repository

Claude Desktop / Cursor

McpServers

Computer Use AI SDK

Demos

Get started

Install

Description

About

Details

Jump to

Setting up with Highlight

From the repository

Claude Desktop / Cursor

McpServers

Computer Use AI SDK

Demos

Get started

Install

More like this

Sentry

Github

Git

GitLab

Context 7

Playwright