WinGuiMcp is a Model Context Protocol (MCP) server for Windows GUI automation. It provides a Playwright-inspired "snapshot + ref" API for agentic AI control of the Windows Desktop.
Start at docs/README.md for reading order and the full document map. Authority map: docs/CANONICAL_SOURCES.md.
| Document | Purpose |
|---|---|
| SPEC.md | Executive summary — principles, stack, phases |
| stack.md | C# / .NET 10 LTS, NuGet baseline, alternatives considered |
| architecture.md | System layers, session model, data flows |
| data-model.md | Unified Element Model (UEM) and ref format |
| tools.md | Normative MCP tool definitions and JSON schemas |
| phases.md | Phased delivery plan and acceptance gates |
| errors.md | Structured error codes and recovery hints |
| security.md | UIPI, elevation, deployment constraints |
Exploratory design notes live in docs/research/ (non-normative):
- design-research.md — comprehensive design doc
- deep-research-report.md — open-source landscape survey
- raw-design-report.txt — original long-form report
- windows_gui_mcp_design_html.html — interactive dashboard
Coding agents: see AGENTS.md.
- Repository: https://github.com/synthet/win-gui-mcp
This project is currently in the Phase 1: MVP development stage.
- .NET 10 LTS.0 SDK
- Windows OS (Windows 10/11 recommended)
dotnet build WinGuiMcp.sln
dotnet run --project src/WinGuiMcp/WinGuiMcp.csprojWinGuiMcp uses a hybrid perception model:
- Primary: Microsoft UI Automation (UIA) via FlaUI.
- Fallback: Computer Vision (OCR + object detection) for custom-rendered apps (Phase 3).
The server exposes MCP tools so agents work from structured accessibility trees with stable element refs—not pixel coordinates.