Skip to content

Surface out-of-process data collector messages on the MTP execution path - #16272

Merged
Azat Mukhametshin (azat-msft) merged 6 commits into
microsoft:mainfrom
azat-msft:fix/mtp-datacollector-hang
Jul 15, 2026
Merged

Surface out-of-process data collector messages on the MTP execution path#16272
Azat Mukhametshin (azat-msft) merged 6 commits into
microsoft:mainfrom
azat-msft:fix/mtp-datacollector-hang

Conversation

@azat-msft

Copy link
Copy Markdown
Member

Problem

Running a pure Microsoft.Testing.Platform (MTP) app under vstest.console with a generic out-of-process data collector (Blame, Event Log, or a custom /Collect: collector) completed the run but silently dropped the collector's run-time and session-end messages and attachments from the console — only the startup SessionStarted line appeared.

The ~90s shutdown hang for Blame under MTP was already fixed by #16235 (MtpDataCollectionForwarder). This is the remaining downstream half of that same wiring gap.

Root cause

On the MTP path there is no testhost, so MtpProxyExecutionManager owns the out-of-process data-collection lifecycle directly. The data collector runs in datacollector.exe and its log messages reach the console in two hops:

  1. IPC — during the AfterTestRunEnd handshake, DataCollectionRequestSender.SendAfterTestRunEndAndGetResult pulls each DataCollectionMessage off the socket and buffers it into the in-process _dataCollectionEventsHandler.Messages list.
  2. In-process — draining that list into the run's event handler (the console logger).

The MTP path performed hop 2 only once, in BeforeTestRun (startup messages). The run-time and session-end batch (TestHostLaunched, TestCaseStarted/Ended, SessionEnded, warnings, Dispose) arrives via hop 1 after the run, so the buffer filled up and was never flushed. The classic path relays these live via DataCollectionTestRunEventsHandler; the MTP path had no live pump and no post-run drain.

Fix

Extract a SurfaceDataCollectionMessages helper and flush the buffer again after AfterTestRun completes. General fix for any out-of-process collector — no per-collector special case. The classic testhost path and the working MTP Code Coverage integration are untouched.

Test

New acceptance test RunMtpApplicationWithGenericOutOfProcDataCollectorCompletesRun runs a pure-MTP app with the generic SampleDataCollector and asserts the run completes with the normal summary and that the collector lifecycle (SessionStarted, TestHostLaunched, per-test-case events, SessionEnded) and per-test-case attachments are surfaced. Fails on main (missing TestHostLaunched), passes with this change.

Validation

  • MtpUnderVstestTests full suite: 10/10 pass (incl. Code Coverage path — no regression).
  • New test + existing /Blame test: 4/4 pass, run completes in ~1s (no hang).
  • Release build of Microsoft.TestPlatform.CrossPlatEngine clean.

…path

When an MTP application runs under vstest.console there is no testhost, so
MtpProxyExecutionManager owns the out-of-process data-collection lifecycle
directly. Data collector log messages are buffered on the run events handler and
delivered during the AfterTestRunEnd exchange, but the MTP path only drained
that buffer once, in BeforeTestRun (startup messages). The messages produced
during the run and at session end (per-test-case notifications, warnings/errors,
the Blame sequence-file path, disposal) were therefore silently dropped -- unlike
the classic path, which relays them live via DataCollectionTestRunEventsHandler.

Flush the buffer again after AfterTestRun completes so any out-of-process
collector (Blame, Event Log, custom ones) has its full lifecycle surfaced.

Add an acceptance test that runs a pure-MTP app with a generic out-of-process
data collector and asserts the run completes with the normal summary and that
the collector lifecycle (SessionStarted, TestHostLaunched, per-test-case events,
SessionEnded) and per-test-case attachments are surfaced.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 08a0cb32-b521-4cee-9cb4-485c25bb1ec0
Copilot AI review requested due to automatic review settings July 13, 2026 20:06

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a gap on the Microsoft.Testing.Platform (MTP) execution path where out-of-process data collector messages (and related lifecycle diagnostics) were buffered during/after the run but not forwarded to the console logger, causing them to be silently dropped.

Changes:

  • Flush buffered out-of-process data collector log messages after AfterTestRunEnd completes in MtpProxyExecutionManager.
  • Refactor message forwarding into a SurfaceDataCollectionMessages helper used both pre-run (startup) and post-run (session end).
  • Add an acceptance test covering MTP + generic out-of-proc data collector (SampleDataCollector) message surfacing and per-test-case attachment production.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
test/Microsoft.TestPlatform.Acceptance.IntegrationTests/MtpUnderVstestTests.cs Adds an end-to-end acceptance test to verify collector lifecycle messages and attachments are surfaced for MTP runs.
src/Microsoft.TestPlatform.CrossPlatEngine/Client/MTP/MtpProxyExecutionManager.cs Drains buffered datacollector log messages after the run completes to avoid silent drops on the MTP path.

Comment thread test/Microsoft.TestPlatform.Acceptance.IntegrationTests/MtpUnderVstestTests.cs Outdated
Comment thread test/Microsoft.TestPlatform.Acceptance.IntegrationTests/MtpUnderVstestTests.cs Outdated
@azat-msft
Azat Mukhametshin (azat-msft) marked this pull request as draft July 13, 2026 21:18
Copilot AI review requested due to automatic review settings July 13, 2026 21:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Comment thread src/Microsoft.TestPlatform.CrossPlatEngine/Client/MTP/MtpProxyExecutionManager.cs Outdated
Copilot AI review requested due to automatic review settings July 13, 2026 21:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comment thread src/Microsoft.TestPlatform.CrossPlatEngine/Client/MTP/MtpProxyExecutionManager.cs Outdated
Copilot AI review requested due to automatic review settings July 14, 2026 10:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

- SurfaceDataCollectionMessages now also flushes and clears the buffered raw
  (telemetry) messages, not just log messages, mirroring
  ProxyExecutionManagerWithDataCollection on the classic path so MTP runs no longer
  drop datacollector raw messages or retain them for the manager lifetime.
- Acceptance test: assert the full datacollector message prefix for the
  TestCaseStarted/TestCaseEnded checks so they cannot match unrelated output; point
  the collector source files at a dedicated subdirectory instead of the results
  directory; and assert the exact per-test-case attachment count while excluding
  that source directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: fb7de7a6-b8a1-4ebb-8266-5c5a948f245c
Copilot AI review requested due to automatic review settings July 14, 2026 11:08
@azat-msft

Copy link
Copy Markdown
Member Author

Addressed the review feedback (commit 00ac892):

  • SurfaceDataCollectionMessages now also flushes and clears the buffered raw (telemetry) messages (DataCollectionRunEventsHandler.RawMessages), not just log messages, mirroring ProxyExecutionManagerWithDataCollection on the classic path so MTP runs no longer drop datacollector raw messages or retain them for the manager's lifetime.
  • The acceptance test now asserts the full Data collector 'SampleDataCollector' message: prefix for the TestCaseStarted/TestCaseEnded checks (no longer matchable by unrelated output), points the collector's source files at a dedicated subdirectory (not the results directory), and asserts the exact per-test-case attachment count while excluding that source directory.

Verified: RunMtpApplicationWithGenericOutOfProcDataCollectorCompletesRun passes 2/2 matrix cases.

Note: the two review comments about BuildPropertyProvider and the discovery completed.Wait(...) refer to the /TestCaseFilter code, which is now a separate PR (#16282); both are addressed there.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 14, 2026 13:43
@azat-msft
Azat Mukhametshin (azat-msft) marked this pull request as ready for review July 14, 2026 13:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

Comment on lines +238 to +239
// The run must complete with the usual summary rather than hang at shutdown.
ValidateSummaryStatus(2, 1, 1);
Comment on lines +251 to +254
// The collector emits one attachment per test case that reports a start through the forwarded
// TestCaseStart events. All four MtpMSTestProject test cases report a start on this path (the
// skipped one still surfaces a start node), so four attachments must land in the results
// directory. Exclude the collector's own source directory so only the moved attachments are counted.
The MtpMSTestProject asset now has five test cases (after the runsettings
env-var test merged from main): three pass, one fails, one is skipped.
Update the new RunMtpApplicationWithGenericOutOfProcDataCollectorCompletesRun
test to assert the 3/1/1 summary and five per-test-case attachments, and
correct the stale comment wording. Addresses PR review feedback and fixes
the failing Windows Release integration test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e19db317-61c7-4dff-8adf-e0d44088c5eb
Copilot AI review requested due to automatic review settings July 14, 2026 20:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

…ndardOutput

The MtpMSTestProject asset gained a fifth test (RunSettingsEnvironmentVariableIsInjected)
via a concurrently-merged PR, but this test's summary assertion still expected
four tests. All MtpApp acceptance tests now consistently assert 3 passed / 1 failed
/ 1 skipped. Also update the stale class-level asset description comment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e19db317-61c7-4dff-8adf-e0d44088c5eb
Copilot AI review requested due to automatic review settings July 14, 2026 20:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 15, 2026 08:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.

@azat-msft
Azat Mukhametshin (azat-msft) merged commit f0c2756 into microsoft:main Jul 15, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants