Skip to content

Commit 1914f40

Browse files
committed
Keep a delegating Bot's brief whole: drop the line that told it to end at what to check
Told to end its brief at what to check, with no line on how to report, Claude wrote shorter briefs that also dropped the work's own checks. Replaying the delegating step of four captured multi-step runs six times each, briefs for "emails from the last day" kept a rolling 24 hours in 4 of 24 with that line and 15 of 24 without it (17 of 24 on main). Live, the task then counted a 30-hour-old email as the last day's in 2 of 3 runs on the Claude Agent SDK. The report is already shaped after the brief, whatever the brief asks it to include, so the line is not needed for length: the offline report lengths were measured on briefs written without it.
1 parent 5fe4c35 commit 1914f40

2 files changed

Lines changed: 11 additions & 11 deletions

File tree

‎shared/orchestration.ts‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -77,11 +77,11 @@ export const REACTIONS = ["👍", "❤️", "🎉", "👀", "✅", "🙏", "😂
7777
* the conversation had, the venue's, in 2 of 3 runs. Stopping work the person still wants is worse
7878
* than stopping nothing, so a stop that names no task of this conversation stops nothing.
7979
*
80-
* THE BRIEF SAYS WHAT TO DO AND CHECK, NOT HOW TO REPORT. The task is told how to report by its own
81-
* frame (`frameBrief`). A Claude Bot's briefs ended "Report back with the sheet link, how many emails
82-
* were included, and anything you could not confirm", and the task answered that list item by item
83-
* beside its frame's. Told only "never how to report back", Claude still wrote such a line in 4 of 4
84-
* replayed briefs; told to end the brief at what to check, in 2 of 4. What the work must check stays.
80+
* NOTHING HERE ON HOW THE TASK REPORTS. Claude's briefs often end "report back with ... anything you
81+
* could not confirm", and the task's report is shaped after the brief instead (`BRIEF_CLOSE`). Told
82+
* here to end a brief at what to check, Claude wrote shorter briefs that dropped the work's own
83+
* checks: of 24 replayed briefs for "emails from the last day", 4 kept a rolling 24 hours (17 before),
84+
* and the task then took a 30-hour-old email as the last day's.
8585
*/
8686
export const DELEGATING_GUIDANCE = [
8787
`When you hold ${DELEGATE_TOOL}, you are the person's point of contact and the work they ask for runs in task threads of your own, so this conversation stays short. Each turn, do exactly one of these three things, never two:`,

‎shared/tests/orchestration.test.ts‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -160,12 +160,12 @@ describe("how a Bot answers a short ask itself", () => {
160160
* order itself, and wrote 👍 as a reply beside its 👍 reaction.
161161
*/
162162
describe("how a Bot that holds delegate is told to work", () => {
163-
test("a brief says what to do and check, and ends there, leaving how to report to the task", () => {
164-
// A Claude Bot's briefs ended "Report back with the sheet link, how many emails were included,
165-
// and anything you could not confirm", and the task answered that list item by item.
166-
expect(DELEGATING_GUIDANCE).toContain(
167-
"Say in it what to do and what to check, and end it there: no line on what to report back, mention or confirm in the result, since the task is told how to report already.",
168-
);
163+
test("a brief is not told to stop short: the work's own checks stay in it", () => {
164+
// Told to end a brief at what to check, Claude wrote shorter briefs: of 24 replayed briefs for
165+
// "emails from the last day", 4 kept a rolling 24 hours (17 without the line), and the task then
166+
// took a 30-hour-old email as the last day's. How the task reports is shaped after the brief.
167+
expect(DELEGATING_GUIDANCE).not.toContain("end it there");
168+
expect(DELEGATING_GUIDANCE).not.toContain("how to report");
169169
expect(DELEGATING_GUIDANCE).toContain(
170170
"every figure, name, link and constraint from this conversation the work needs",
171171
);

0 commit comments

Comments
 (0)