Skip to content

fix(computer-use): bound Windows UIA requests and preserve control status - #3437

Merged
bobleer merged 4 commits into
GCWing:mainfrom
bobleer:bob/windows-cua-hang
Oct 17, 2026
Merged

bobleer merged 4 commits into
GCWing:mainfrom
bobleer:bob/windows-cua-hang

Conversation

@bobleer

@bobleer bobleer commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Windows Computer Use can wait indefinitely when an application’s UI Automation provider stalls. Bound every UIA request on one MTA worker, fence input after cancellation, and keep optional preview failures separate from control status.

Type and Areas

Type: Bug fix / focused CI coverage

Areas: Desktop Windows Computer Use, Web UI control card, focused tests, Windows CI coverage and owning documentation.

Motivation / Impact

A 1.0.6 diagnostic trace showed an app click waiting 133.8 seconds until user cancellation; the existing UIA receiver had no deadline. The logs cannot identify the exact blocked COM call. Requests now have a ten-second caller deadline, 50 ms cancellation polling, configured UIA connection/transaction limits, and automatic focus disabled. A still-running native call retains the sole worker slot; subsequent requests fail explicitly instead of queuing or spawning replacement threads. Input already submitted has an unknown outcome on failure and is never replayed automatically. Retained observations require caller acceptance, so a native cache published just before a rejected reply cannot authorize later node-index input. Stop releases idle UIA caches asynchronously on their owning thread, and an immediately following request waits within its deadline for that cleanup.

AX-only observations and missing, stale or failed previews no longer make a valid control status appear unavailable. Stop remains available.

Verification

  • cargo test -p openbitfun-desktop --lib computer_use — 188 passed, 9 native fixtures ignored. Includes nine blocking-provider, reply-acceptance and idle-cleanup regressions; rerun after the review fix.
  • cargo build -p openbitfun-desktop — passed on macOS.
  • node scripts/test-windows-computer-use-native.mjs --check --target x86_64-pc-windows-msvc — passed for the production Windows modules, including UIA and shortcut discovery.
  • pnpm --dir src/web-ui run test:run src/app/components/ComputerUseControlCard.test.tsx src/app/components/computer-use src/infrastructure/api/adapters/peer-device-adapter.test.ts — 85 passed.
  • pnpm run check:web, pnpm run fmt:rs, pnpm run check:github-config, and git diff --check — passed.
  • Windows CI independently executed the production native-module suite: 77 passed, 1 interactive fixture ignored, including all nine new provider-worker regressions. The existing full Tauri probe still encounters STATUS_ENTRYPOINT_NOT_FOUND before tests start; this is not counted as native test coverage.
  • Upstream CI run 39020427422 completed successfully on 731d1e9092ed3e8565344ba1a8d92a6128b145f2; all 14 PR checks passed, including the three-platform Rust/CLI matrix, frontend, Windows packaging, and Linux x64/ARM64 release validation.

Reviewer Notes

No persisted or wire shapes changed. The nullable TypeScript preview result now matches the existing Rust optional response. Native input already inside a provider may still finish after Stop; a permanently hung provider remains explicitly busy until recovery or host restart. Real Windows replay was unavailable. Windows interactive fixtures and all four remote scenarios were not exercised end to end; peer response isolation has unit coverage only. No diagnostic contents or personal application data are included.

Checklist

  • This PR is focused and does not include secrets, temporary prompts, generated scratch files, or unrelated artifacts.
  • Relevant verification is recorded above, or skipped checks are explained.
  • User-facing strings, docs, and locales are updated where applicable.

@bobleer
bobleer merged commit 725aa85 into GCWing:main Oct 17, 2026
14 checks passed
@bobleer
bobleer deleted the bob/windows-cua-hang branch October 18, 2026 09:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant