Microsoft Paint MCP Server for Node.js

Semantic MCP interface for Microsoft Paint using UI Automation, Win32 and a drawing DSL.

Microsoft Paint is probably one of the least intimidating pieces of software ever shipped with Windows.

A blank canvas. A pencil. A few shapes. A color palette.

For millions of people, it may have been one of the first places where a computer became something you could experiment with rather than simply operate.

But because I wanted to understand a more interesting question:

How far can an ordinary piece of software go when an AI is given a meaningful way to understand and interact with it?

The obvious way to automate Paint would have been to move a mouse to known screen coordinates and click buttons.

But it doesn't understand anything.

Change the window position, the display scale, the toolbar layout, or the environment, and coordinate-based automation quickly reveals what it really is: a script replaying gestures.

I wanted to approach the problem differently.

Before teaching an AI to draw with Paint, I wanted to understand Paint itself.

So the experiment moved downward before it moved upward.

What does Paint expose about itself that software can actually understand?

That led to Windows UI Automation, inspection of the Paint accessibility tree, canvas discovery, window lifecycle management, coordinate transformations, Win32 input, and eventually a semantic interface between an AI and the application.

And structure eventually became an MCP server.

Once the interaction layer existed, another question appeared.

If an AI no longer had to think primarily in mouse coordinates, what should it think in?

move pointer to x=412, y=287 press mouse button move pointer to x=650, y=410 release

{ "kind": "circle", "cx": 300, "cy": 220, "radius": 120 }

{ "solid": "tesseract", "size": 110, "projection": "perspective" }

It became the rendering surface beneath an emerging visual language.

The MCP server translates semantic and mathematical descriptions into geometry, geometry into logical canvas coordinates, and those coordinates into real mouse interaction with Microsoft Paint.

Idea ↓ Semantic description ↓ Geometry ↓ Canvas coordinates ↓ Windows interaction ↓ Microsoft Paint

That is the part of the experiment I find most interesting.

The drawing layer gradually became a small mathematical DSL.

It can currently express things such as:

- circles, ellipses and arcs;
- rectangles and rounded rectangles;
- regular and star polygons;
- logarithmic spirals;
- grids and repeated structures;
- arbitrary polylines;
- Platonic solids;
- toruses and torus knots;
- surfaces of revolution;
- custom wireframe meshes;
- and even a tesseract projected from 4D → 3D → 2D.

The shapes are not created through Paint's native Shape buttons.

They are generated mathematically as points and strokes and then physically drawn onto the canvas.

It keeps the experiment focused on the boundary betweenabstract descriptionandreal application interaction.

If this experiment had started with Blender, AutoCAD, Mathematica or another sophisticated visual environment, much of the capability could be attributed to the application itself.

It forces the intelligence and abstraction to live somewhere else.

The result is a surprisingly flexible visual sandbox.

An AI can potentially use the same primitive canvas to explain:

Construct polygons, transformations, projections and geometric relationships.

Turn equations and mathematical structures into visible objects.

Explore three-dimensional objects through two-dimensional projections.

Express flows, relationships, timelines, maps and simple conceptual models.

Build visual explanations step by step using one of the simplest applications available on Windows.

The interesting question is therefore no longer:

What can an AI explain when drawing becomes part of its language?

An example: a four-dimensional object in Paint

One of the available generators is atesseract.

A tesseract is the four-dimensional analogue of a cube.

The implementation begins with the sixteen vertices of the hypercube, projects them from four dimensions into three, projects the result again into two dimensions, and finally converts its thirty-two edges into strokes that Paint can physically draw.

From the MCP client's perspective, the request remains semantic:

{ "mode": "generator", "tool": "pencil", "fit": "contain", "generators": [ { "kind": "solid", "solid": "tesseract", "size": 110, "rotX": 15, "rotY": -30, "projection": "perspective" } ] }

Paint knows nothing about four-dimensional geometry.

It is simply the final surface where an abstract idea becomes visible.

AI was used extensively while building this project.

But using AI was never the interesting part.

The interesting part was decidingwhat had to be understood before something should be automated.

Throughout the experiment the workflow repeatedly looked like this:

observe ↓ question ↓ inspect ↓ model ↓ experiment ↓ specify ↓ implement ↓ verify

For example, the server does not assume that the Paint canvas lives at one permanent screen coordinate.

It discovers and resolves the application context, maintains the Paint session, maps logical drawing coordinates to the actual drawable canvas and uses Windows APIs to execute the interaction.

Drawing results can also be verified after execution rather than assuming that the absence of an exception means that ink actually appeared.

The goal is not to hide AI from the engineering process.

Nor is it to celebrate AI for producing code.

using AI to explore a system more deeply while keeping understanding, architecture and verification explicit.

LLM / MCP Client │ ▼ MCP Server │ ▼ PaintController │ ├── Paint session management ├── UI Automation discovery ├── Semantic canvas resolution ├── Mathematical generators └── Coordinate transformation │ ▼ Win32 / SendInput adapter │ ▼ Microsoft Paint

The codebase follows a layered / hexagonal structure that keeps the mathematical domain independent from Windows-specific automation.

The pure geometry layer knows nothing about Paint.

The Paint orchestration layer knows nothing about MCP clients.

The Windows infrastructure implements the mechanics required to make the final interaction real.

For the deeper technical architecture, see the documentation below.

The server currently exposes seven MCP tools.

The productive drawing APIs operate in logical drawing space rather than requiring the caller to know where Paint happens to be on the screen.

The 2D DSL currently includes generators such as:

ellipse circle disk arc rectangle roundedRectangle polyline logarithmicSpiral regularPolygon starPolygon grid dotsAlongPath

tetrahedron cube octahedron dodecahedron icosahedron greatIcosahedron starOctangula tesseract torus torusKnot revolution wireframe

These descriptions are validated before reaching the drawing layer and transformed into pure geometry before any Windows interaction occurs.

The project uses Windows UI Automation to discover application controls instead of relying entirely on fixed toolbar coordinates.

That investigation became useful enough to document independently.

If you are interested in understanding how a modern Windows application can be inspected and automated through its accessibility model, see:

There is also a practical PowerShell walkthrough:

These documents preserve the engineering investigation behind the MCP implementation rather than hiding it behind the final API.

- Windows
- Microsoft Paint
- Node.js
- PowerShell
- an MCP-compatible client

git clone https://github.com/miguelcespedes/mcp-server-microsoft-paint-nodejs.git cd mcp-server-microsoft-paint-nodejs