Add manual testing instructions (#592)#3004
Open
LarryOsterman wants to merge 3 commits into
Open
Conversation
Create docs/manual-testing.md with comprehensive manual testing instructions including environment setup, server configuration, test execution steps for all/namespace and consolidated modes, validation criteria, and result reporting guidelines. Reference the new guide from the End-to-end Tests section in CONTRIBUTING.md. Fixes microsoft#592 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
LarryOsterman
requested review from
KarishmaGhiya,
RickWinter,
anannya03,
sandeep-sen,
tmeschter and
xirzec
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new documentation page that explains how to run manual end-to-end validation of Azure MCP tool invocation, and links to it from the contribution guide so contributors can find a single, consistent set of steps.
Changes:
- Added
docs/manual-testing.mdwith environment setup, prompt execution steps, and validation/reporting criteria. - Linked the new manual testing guide from the “End-to-end Tests” section in
CONTRIBUTING.md.
Invoking Livetests
Copilot submitted PRs are not trustworthy by default. Users with write access to the repo need to validate the contents of this PR before leaving a comment with the text /azp run mcp - pullrequest - live. This will trigger the necessary livetest workflows to complete required validation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| docs/manual-testing.md | Introduces a manual testing guide for Azure MCP Server tool invocation validation. |
| CONTRIBUTING.md | Adds a cross-reference to the manual testing guide from the e2e testing section. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes issue #592
[Add additional context, screenshots, or information that helps reviewers]GitHub issue number?
#592
Pre-merge Checklist
servers/Azure.Mcp.Server/README.mdand/orservers/Fabric.Mcp.Server/README.mddocumentationREADME.mdchanges running the script./eng/scripts/Process-PackageReadMe.ps1. See Package READMEToolDescriptionEvaluatorand obtained a score of0.4or more and a top 3 ranking for all related test promptsconsolidated-tools.jsonbreaking-changelabelservers/Azure.Mcp.Server/docs/azmcp-commands.md./eng/scripts/Update-AzCommandsMetadata.ps1to update tool metadata inazmcp-commands.md(required for CI)servers/Azure.Mcp.Server/docs/e2eTestPrompts.mdcrypto mining, spam, data exfiltration, etc.)/azp run mcp - pullrequest - liveto run Live Test Pipeline