feat: delegate-test coverage through BSP (#1890)

[wenyt (wenytang-ms)]

Summary

First opt-in P0 slice of #1890: add Delegate Test to Gradle (Coverage) without changing the user's build files. The profile remains non-default so we can onboard users and observe adoption before changing the default test experience.

Design

BSP-first execution

Coverage uses the existing BSP buildTarget/test path and Tooling API TestLauncher, preserving Gradle's test environment and streamed test results. A per-run init script:

• applies JaCoCo when the project does not already apply it;
• writes execution data and XML reports to a unique temporary directory;
• finalizes each executed Test task with its project's jacocoTestReport.

The build server sends the finish notification after TestLauncher.run() and its finalizers complete. Coverage is attached to the VS Code TestRun before the run ends.

Task-server fallback

When BSP delegation is unavailable, the existing task-server fallback runs cleanTest test --tests ... jacocoTestReport and parses JUnit XML. It uses the same isolated JaCoCo data directory.

Coverage rendering

JaCoCo XML is converted to VS Code FileCoverage, StatementCoverage, and BranchCoverage. Detailed coverage is cached per TestRun.

Source resolution

The JDTLS command java.gradle.getBuildTargetInfo returns:

• the project's Gradle version;
• BSP source roots with test/generated metadata.

Coverage prefers production, non-generated roots and falls back to a workspace glob when BSP metadata is unavailable.

Compatibility and concurrency

• Coverage requires Gradle 6.1 or newer. Known older versions are rejected with a clear error before dispatch.
• Delegated runs are serialized because current BSP callbacks do not contain a run id. The previous idle-timeout release was removed because elapsed silence cannot safely distinguish a dropped connection from a long-running test.

Onboarding

This PR intentionally adds an explicit profile rather than failure-driven automatic retry. Existing java.test.runTests telemetry records the selected profile label, so adoption can be measured without making the profile default.

Validation

• TypeScript compile and lint
• Extension unit suite: 118 passing, 1 pending
• JDTLS plugin package build
• Manual JUnit 5 coverage run

Follow-ups

• Thread BSP originId through test callbacks to support per-run routing and safe concurrent runs.
• Evaluate making delegation the project-level default after onboarding data and reliability feedback.
• Consider BSP-native coverage result transport to remove client-side XML parsing.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[wenyt (wenytang-ms)]

delegatetogradle.mp4

[Copilot]

Pull request overview

Implements the first P0 slice of “Delegate Test to Gradle” default-level test experience by adding coverage support to the delegated Gradle test runner, primarily through BSP (with a task-server fallback). This extends the existing delegated Run/Debug execution path to also support Coverage runs without requiring user build-script changes.

Changes:

• Register a new Delegate Test to Gradle (Coverage) run profile and add coverage-aware execution paths.
• Add JaCoCo init-script generation + XML parsing to surface vscode.FileCoverage/detail coverage (including branch counters).
• Add a new JDTLS delegate command to fetch build target source roots (and Gradle version) for more accurate source mapping.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
extension/src/test/unit/GradleTestRunner.test.ts	Adds unit coverage for BSP coverage runs, Gradle version gating, and run serialization behavior.
extension/src/test/unit/delegateTestP0.test.ts	Adds unit tests for JaCoCo XML parsing and source match selection logic.
extension/src/Extension.ts	Registers the Coverage-kind delegate test run profile.
extension/src/bs/groovy.ts	Adds a shared Groovy single-quote escaping helper for init-script generation.
extension/src/bs/GradleTestRunner.ts	Adds coverage orchestration (BSP + fallback), per-run coverage collection, and serialization gate for delegated runs.
extension/src/bs/coverage.ts	Introduces JaCoCo init-script generation, XML parsing, source resolution, and VS Code coverage attachment/detail caching.
extension/package.json	Bumps VS Code engine + @types/vscode to support the coverage APIs/profile kind.
extension/jdtls.ext/.../GradleDelegateCommandHandler.java	Implements the new delegate command that returns Gradle version + BSP source roots.
extension/jdtls.ext/.../plugin.xml	Registers the new delegate command id.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

[azure-pipelines]

Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

[wenyt (wenytang-ms)]

This reconfigures the user's existing jacocoTestReport task, overriding its execution data and report outputs. It can also trigger user-defined dependencies or actions when delegated coverage runs. Could we register a uniquely named report task owned by the extension and finalize the selected Test tasks with that instead? This would keep the coverage run isolated from the user's JaCoCo configuration.

[wenyt (wenytang-ms)]

Revised convergence: land the exec-only, JDTLS-analyzed coverage model

Updating my earlier A/B/C convergence comment on #1890. After tracing the actual code paths, the conclusion changes:

Once coverage analysis is owned by JDTLS, no report task is needed, and a pure BSP buildTarget/test + init-script path is sufficient — no protocol change, no two-launcher bridge.

The .exec-boundary model in the #1890 body therefore strictly dominates options A/B/C, and this PR should converge on it rather than on B.

1. The "coverage needs a report task" premise was wrong

My earlier comment claimed coverage "inherently needs a report task that TestLauncher won't run, so any BSP-native solution must bridge the two launchers." That does not hold under the body's own design:

• JDTLS's CoverageHandler loads .exec and analyzes against the IJavaProject output locations. We never run jacocoTestReport.
• No report task ⇒ no BuildLauncher.forTasks ⇒ nothing to bridge.
• TestLauncher (BSP buildTarget/test) + a per-run --init-script that only attaches the JaCoCo agent and redirects its destfile into the run directory is the entire execution path — fully on BSP, zero protocol change.

Consequently:

• A (client two-phase w/ task-server report) — unnecessary; keeps a redundant gradle shell-out.
• B (BSP dataKind + server-side report phase) — unnecessary protocol + versioning work to build an XML we don't consume.
• C / build-server-for-gradle#226 — keep as an independent run/debug-environment capability, but for the test path it re-assembles the JVM on the client, reintroducing the [BUG] The extension uses raw data. vscode-java-test#1880 / Design BSP lifecycle management for Gradle server JVM-only recovery #1878 root cause and losing streamed test events. Not for coverage.

Target = the #1890 body model. This PR converges there, not on B.

2. Concrete changes for this PR (half-baked B → exec-only)

Today this PR rides BSP but still reconfigures and finalizedBy-runs jacocoTestReport to produce XML, then parses that XML client-side. To reach the target:

• Delete client-side XML handling in coverage.ts: the jacocoTestReport reconfiguration in getCoverageInitScriptLines (the finalizedBy block), plus parseJacocoXml, resolveSourceFileUri / pickBestSourceMatch, and collectCoverage.
• Init-script only attaches JaCoCo: set the agent destination into the run's exec dir; never touch jacocoTestReport.
• Emit isolated .exec per run / Gradle project / Test task into the directory Java Test provides.
• Drop the task-server coverage invocation; keep the task-server path strictly as the non-coverage fallback.
• Prefer a raw -javaagent on the Test JVM over JacocoTaskExtension, so coverage does not require the user to apply the jacoco plugin and the Gradle 6.1 report-DSL requirement can be dropped. (Validate agent attachment on TestLauncher.run().)

3. Cross-repo contract (vscode-java-test)

The analysis seam already runs for delegated runs — provideFileCoverage is invoked after the external runner in testController.ts, independent of who launched the JVM. Remaining work there:

• Multi-.exec merge (blocking). CoverageHandler hard-codes a single jacoco.exec (JACOCO_EXEC). It must accept a directory / file list and call ExecFileLoader.load(...) repeatedly. This is now the primary path, not an edge case.
• Coverage artifact contract. Add the run-scoped descriptor to IRunTestContext (coverage?: { format: "jacoco-exec"; outputDirectory }), plus the fork-level file/naming strategy (single append destfile vs per-fork files the handler merges).
• Agent/analyzer version ownership. The agent version must be injected by the integration and pinned to JDTLS's bundled org.jacoco.core, not taken from the user's jacoco plugin. Define behavior on IncompatibleExecDataVersionException. This becomes a hard prerequisite for any future default-on (P2).

4. Make three invariants explicit in the design

These are currently implicit or buried in the validation table, but they are the make-or-break assumptions:

1. JDT output location == Gradle class output. The importer sets each source entry's output to Gradle's build/classes via buildTargetOutputPaths (GradleBuildServerBuildSupport), which is exactly why JaCoCo class-id matching works. State it as an invariant: if a future importer change breaks the alignment, coverage silently goes empty.
2. Analysis still requires the JDT project model. We move execution to Gradle but analysis stays JDTLS-coupled; the project must be imported.
3. No clean between run and analysis. Build-dir classes are marked optional (cleanable); analyzing after a clean yields empty coverage.

5. Reliability / completeness

• Detect silent-empty coverage. A class-id mismatch doesn't throw — the analyzer just returns nothing. Add a sanity check: if the .exec contains execution data but the analyzer matches zero classes across all output dirs, emit a distinct diagnostic and a dedicated telemetry reason (don't surface it as "0% covered").
• Escalation stance. Resolve the contradiction between the body's "do not auto-retry a failed standard run with Gradle" and the escalation.ts classifier: make it an explicit, manual one-click "Retry with Gradle", or drop it. No inert half-state.
• Debug + Coverage. Decide whether coverage-while-debugging is in scope; if so, validate two agents (jdwp + jacoco) coexisting on the forked Test JVM.

---

Net: execution is ~90% there (BSP-first is right). The change is to stop producing XML and hand .exec to JDTLS, and to land the CoverageHandler multi-exec + artifact contract on the Java Test side. That removes the two-path divergence, keeps everything on TestLauncher, and needs no BSP protocol change.