Reduce Windows CI runner time without dropping validation

[JeremyKuhne]

Summary

• Preserve all 1,913 Windows Release tests: run non-ETW modules concurrently, then run the full CLI/ETW module alone. Discover modules from the solution and require the established 1,514 + 399 test counts.
• Keep the complete portable command-capture contract on Linux ARM64. Focus its Windows gate on the native 0/1/8-argument proof, both negative encoder mutations, and an authenticated-spec failure; the original combined invocation remains callable.
• Check pure fake-agent CLI, skill, and usage shapes with the evaluator's production validators instead of repeating isolated host startups; retain native policy, process, failure-artifact, timeout, and comparator witnesses.
• Run the real native-symbol ETW gate after the test hosts exit. Then overlap the isolated fake-agent contract with the full PS7 and Windows PowerShell 5.1 capture checks, project capture, focused native argv, local activation, and deterministic eval within the same Windows job. Bound the owned child and fail the job on any failed gate. The required ci aggregate, job count, and 20-minute cap are unchanged.

Evidence

• The previous hosted Windows baseline was 845 seconds. An exact-source clean local sequence took 601.10 seconds; the fresh candidate's measured phases totaled 282.86 seconds on this faster machine. Only the contract-overlap phase was constrained to four logical CPUs. The hosted runs below pass CI but have not met the ≤422-second target.
• Two four-CPU overlap pilots and the fresh candidate passed in approximately 177–180 seconds each. A forced failure returned nonzero and killed the owned fake process; a one-second fake deadline rejected late completion without leaving that process running.
• Local Release build and all 1,913 Windows test cases passed discovery (1,911 passed, two environment-dependent skips). The full fake-agent contract, complete PS7/PS5.1 benchmark checks, portable command-capture contract, native-symbol ETW gate, 31 deterministic eval tasks, CLI help, MCP, docs, and pinned skills gates passed locally. Linux ARM64 and hosted Windows checks passed at the published head; see the hosted results below.
• At GitHub's published runner rates, the old four-job CI run prices at $0.204. If Windows finishes in the proposed seven billed minutes (≤420 seconds) with other job times unchanged, the rate-weighted CI cost would be $0.124, saving $0.080 per equivalent run. The observed runner-rate figures for both published heads are below.

No raw ETW captures or private investigation artifacts are included.

Hosted results and review follow-up

• Initial PR head 599e968: Windows 479s (7m59s), Linux ARM64 7m43s, agent-files 1m24s, and required ci passed; published-rate four-job cost $0.134.
• Review-fix head da73210: Windows 519s (8m39s), Linux ARM64 7m56s, agent-files 1m16s, and required ci passed; published-rate four-job cost $0.144. Relative to the 845s / $0.204 baseline, this head saves 326s (38.6%) and $0.060 per equivalent CI run. It misses the ≤422s half-time goal by 97s.
• The resolved review thread identified a real flaw: the fake-agent deadline was checked only after synchronous gates, with a natural-exit/Kill race. da73210 monitors each owned foreground gate and output drain, interrupts an expired fake deadline, and makes cleanup idempotent. Locally a one-second deadline interrupted a running gate in 1.51s with both owned PIDs gone; forced foreground failure and the complete seven-gate path also passed.

The second hosted run took 40s longer than the first (test step +10s, overlapped contracts +30s). That is an observed difference across hosted runs, not an isolated causal estimate of the review fix's overhead. No merge, release, or raw investigation trace is part of this PR.

[Copilot]

Copilot review overview

🟡 Changes recommended

The fake-agent deadline is not enforced during other gates, and cleanup contains a process-exit race.

Review effort: Balanced
Findings: 1

Open (1)

• Enforce fake process timeout while synchronous steps are running · New

What changed in this PR

Reduces Windows CI duration while preserving Windows-specific ETW, test, capture, and evaluator coverage.

Changes:

• Parallelizes non-ETW test modules while isolating CLI/ETW tests.
• Overlaps fake-agent evaluation with focused Windows contracts.
• Replaces redundant fake-host processes with direct production-validator checks.

The hosted run passed in ~479 seconds, improving on 845 seconds but missing the ≤422-second target.

File	Description
tools/​Test-WindowsDotNet.ps1	Partitions and validates Windows test modules.
tools/​Test-WindowsContracts.ps1	Orchestrates overlapping Windows gates.
tools/​Test-CaptureCommandTrace.ps1	Adds focused native-argv validation.
tools/​Test-AgentEval.ps1	Moves schema cases to direct validator tests.
tools/​fixtures/​Fake-CopilotEvalHost.ps1	Removes redundant fake-host modes.
eval/​README.md	Documents revised evaluation coverage.
.github/​workflows/​ci.yml	Integrates the optimized Windows workflow.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.