MAINT: Standardize system prompts on prepended_conversation#2040
Open
adrian-gavrila wants to merge 15 commits into
Open
MAINT: Standardize system prompts on prepended_conversation#2040adrian-gavrila wants to merge 15 commits into
adrian-gavrila wants to merge 15 commits into
Conversation
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Behavior is grep-discoverable, runtime-enforced, and test-covered; the section did not clear the bar this slim instruction file sets. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…prompt' into adrian-gavrila/standardize-system-prompt
Contributor
There was a problem hiding this comment.
⚠️ Human review recommended
It changes a core execution chokepoint and parameter contract across many attacks, so a human reviewer should validate compatibility and any downstream behavioral impact beyond the added unit coverage.
Pull request overview
Standardizes system_prompt= as a first-class, consumed attack argument by lifting it into AttackParameters and reliably lowering it into a single leading system message at the shared AttackStrategy.execute_with_context_async entrypoint, ensuring delivery for both direct (execute_async) and executor-driven (AttackExecutor) runs.
Changes:
- Add
system_prompt: str | NonetoAttackParametersand lower it intocontext.prepended_conversationatAttackStrategy.execute_with_context_async, with a conflictValueErrorwhen a system-role prepended message is already present. - Remove the dead
SingleTurnAttackContext.system_promptfield and update tests to assert behavioral delivery rather than no-op state. - Explicitly exclude
system_promptfrom self-seeding / internally-constructed prompt attacks’params_typeand add unit coverage for rejection and executor-path regression.
File summaries
| File | Description |
|---|---|
pyrit/executor/attack/core/attack_parameters.py |
Adds system_prompt to the canonical attack parameter contract. |
pyrit/executor/attack/core/attack_strategy.py |
Lowers system_prompt into a prepended system message at the shared chokepoint and enforces conflict rules. |
pyrit/executor/attack/single_turn/single_turn_attack_strategy.py |
Removes unused SingleTurnAttackContext.system_prompt field. |
pyrit/executor/attack/single_turn/flip_attack.py |
Excludes system_prompt from a self-seeding attack’s accepted params. |
pyrit/executor/attack/single_turn/skeleton_key.py |
Excludes system_prompt from a self-seeding attack’s accepted params. |
pyrit/executor/attack/single_turn/many_shot_jailbreak.py |
Excludes system_prompt from a self-seeding attack’s accepted params. |
pyrit/executor/attack/single_turn/context_compliance.py |
Excludes system_prompt from a self-seeding attack’s accepted params. |
pyrit/executor/attack/single_turn/role_play.py |
Excludes system_prompt from a self-seeding attack’s accepted params. |
pyrit/executor/attack/compound/sequential_attack.py |
Excludes system_prompt from compound attack per-call overrides. |
tests/unit/executor/attack/core/test_attack_strategy.py |
Adds unit coverage for lowering behavior, ordering, conflict errors, and executor-bypass simulation. |
tests/unit/executor/attack/core/test_attack_executor.py |
Regression test ensuring executor-path lowering happens via the shared chokepoint. |
tests/unit/executor/attack/single_turn/test_prompt_sending.py |
Updates assertions to validate lowering behavior and adds delivery-to-conversation-manager test. |
tests/unit/executor/attack/single_turn/test_role_play.py |
Verifies system_prompt exclusion from params_type and explicit rejection at runtime. |
Copilot's findings
- Files reviewed: 13/13 changed files
- Comments generated: 0
Note
Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.
rlundeen2
reviewed
Jun 19, 2026
…field Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8ae0d01b-261f-41fc-a331-179f8574732b
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8ae0d01b-261f-41fc-a331-179f8574732b
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 8ae0d01b-261f-41fc-a331-179f8574732b
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 65558ff3-57b6-4216-982c-e8ad36c2416c
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 65558ff3-57b6-4216-982c-e8ad36c2416c
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Standardizes objective-target system prompts on
prepended_conversation, the existing path that supports one or more system messages and composes with seeded conversation history.SingleTurnAttackContext.system_promptwas exposed but never consumed, so setting it silently did nothing. This PR keeps the field temporarily for compatibility, emits aDeprecationWarningwhen it is used, and schedules its removal for 0.17.0.Callers should instead use:
For multiple system prompts, this PR adds
Message.from_system_prompts(...), which returns the samelist[Message]callers could build manually.Changes
SingleTurnAttackContext.system_promptfield.Message.from_system_prompts(...).prepended_conversationas the standard system-prompt channel.Implements ADO #9697’s framework-standardization track. The CoPyRIT GUI work was delivered separately in #2056.
Validation
.pyand.ipynbfiles were synchronized and executed.