Skip to content

Prepare affected-test selection rollout - #10450

Draft
Evangelink wants to merge 1 commit into
mainfrom
dev/amauryleve/adopt-affected-tests
Draft

Prepare affected-test selection rollout#10450
Evangelink wants to merge 1 commit into
mainfrom
dev/amauryleve/adopt-affected-tests

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Prepares testfx to adopt the experimental Microsoft.Testing.Platform affected-test workflow from dotnet/sdk#55574, using the composable filter-provider support already merged in testfx#10235.

The rollout is intentionally disabled. The SDK change has not flowed into this repository's pinned SDK, and the affected-test extension package and public storage schema are not available yet. Existing test runs therefore keep their current command and behavior.

Prepared now

  • adds repository-specific test.affectedTests change and instrumentation scopes to global.json;
  • prepares trusted main Windows Debug for future --collect-test-map collection;
  • prepares PR Windows Debug for future --affected-tests selection;
  • keeps both call sites at enableAffectedTests: false;
  • scopes DOTNET_CLI_ENABLE_AFFECTED_TESTS=1 to the inactive affected-test branches;
  • prevents pipelines from setting the SDK-owned DOTNET_CLI_TEST_AFFECTED_TESTS_MODE marker;
  • adds a validation script and CI check for disabled/activated state consistency, argument construction, secret-free configuration, and gate isolation;
  • documents storage, activation, validation, and rollback.

Intentionally gated

The storage property is intentionally absent from global.json rather than inventing a provider name, package, credential, or private endpoint. The intended CI design is Azure Blob Storage as the durable cross-run map, authenticated through secure pipeline facilities, with Azure DevOps artifacts used only for non-secret diagnostics or snapshots.

Activation requires:

  1. an SDK build containing Add affected test selection to dotnet test dotnet/sdk#55574;
  2. a publicly available affected-test extension package flowed through normal dependency management;
  3. the package's published Azure Blob storage schema and secure pipeline authentication;
  4. a successful compatible mapping collection before PR selection becomes required.

Rollout and rollback

After the prerequisites flow, add the documented storage configuration, enable collect on trusted main, validate the produced map and diagnostics, then enable run for PR Windows Debug. Validate documentation-only, narrowly affected, force-all, missing/incompatible-map, fork-PR, and collection-failure scenarios before making selection required.

Rollback is explicit and immediate: set enableAffectedTests back to false. The unchanged full test command remains the disabled branch.

Validation

  • Debug repository build: 0 warnings, 0 errors.
  • Disabled, collect activation, missing-storage, job-level gate leakage, and SDK child-marker leakage scenarios passed.
  • global.json and modified YAML parse successfully.
  • PSScriptAnalyzer and markdownlint pass.
  • git diff --check passes.
  • Four fresh bug-focused review rounds completed; fixes covered activation deadlock and environment-variable leakage bypasses, and the final review found no significant issues.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings August 5, 2026 10:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prepares an intentionally disabled rollout of affected-test selection for Microsoft.Testing.Platform CI.

Changes:

  • Adds affected-test scopes and gated pipeline modes.
  • Adds rollout validation and safety checks.
  • Documents activation, storage, validation, and rollback.
Show a summary per file
File Description
global.json Defines affected-test scopes.
azure-pipelines.yml Configures disabled collect/run call sites and validation.
eng/pipelines/steps/test-windows-debug-coverage.yml Adds gated affected-test commands.
eng/validate-affected-tests.ps1 Validates rollout configuration.
docs/affected-test-selection.md Documents rollout procedures.
docs/README.md Links the new guide.

Review details

  • Files reviewed: 6/6 changed files
  • Comments generated: 3
  • Review effort level: Balanced

Comment on lines +46 to +47
- ${{ if and(eq(parameters.enableAffectedTests, true), eq(parameters.affectedTestsMode, 'run')) }}:
- script: dotnet test -c $(_BuildConfig) --no-build -bl:$(BUILD.SOURCESDIRECTORY)\artifacts\TestResults\$(_BuildConfig)\TestStep.binlog -p:UsingDotNetTest=true -p:TestingPlatformCaptureOutput=false --affected-tests
Comment on lines +55 to +56
if (-not $affectedTestsEnabled -and $null -ne $affectedTests.storage) {
throw "Remove test.affectedTests.storage while affected-test pipeline execution is disabled."
Comment on lines +27 to +29
if ($null -eq $value -or @($value).Count -eq 0) {
throw "global.json test.affectedTests.$path must be a non-empty array."
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants