Skip to content

ADR-008: Agentic graph construction — the Builder - #307

Merged
dovvnloading merged 15 commits into
mainfrom
adr008-agentic-builder
Aug 10, 2026
Merged

ADR-008: Agentic graph construction — the Builder#307
dovvnloading merged 15 commits into
mainfrom
adr008-agentic-builder

Conversation

@dovvnloading

Copy link
Copy Markdown
Owner

Problem

The graph had no agentic construction path: no tool-use loop, no way for an
agent to create/connect/run nodes on the canvas under budget and approval
control, and no way to save or replay a build as a reusable recipe.

Change

Adds the Builder: a bounded, checkpointed tool-use loop that plans a goal
into a checklist, then executes it step by step by calling scoped,
approval-gated tools against the real graph.

  • Tool surface: graph.create_node / connect / set_node_content /
    read_subgraph, and run_node (invoke a node's own action — execute,
    reply, chart, research), all going through the same domain calls and
    undo-tracked commands a user's own actions do.
  • Provider primitive: api_provider.chat_turn_with_tools, a sibling of
    the existing streaming dispatch that collects tool-call events instead of
    dropping them, with a capability gate for tools-incapable models.
  • The loop (backend/builder.py): a new plan node kind is the single
    resume point — pause, Stop, a budget breach, and an app restart all
    rebuild from the plan node's own state, never from a held transcript. Hard
    step/token/wall-clock budgets are checked before every turn and every tool
    call. Loop control (complete a step, replan, finish, abort) rides in-band
    as ordinary auto-approved tools.
  • Oversight modes: co-pilot prompts for every non-auto tool call;
    autopilot auto-approves a disclosed scope set (graph edits, code
    execution) while net.fetch always prompts, even in autopilot.
  • Undo: every tool call records its own run-id-stamped command (not a
    held-open composite, which would swallow concurrent user edits), so "Undo
    build" reverts a whole run and ordinary Ctrl+Z still works step by step.
  • Recipes: two built-in recipes plus save-your-build, seeding a run's
    checklist without a planning call.
  • MCP: ADR-007's deferred wiring lands here — configured MCP servers'
    tools are registered into the builder's own tool registry.

Includes a review-fix pass (final commit) that closed an autopilot scope gap
(an MCP tool with no declared scopes was being auto-approved — an empty set
is a subset of every set), fixed token-budget accounting that silently
undercounted on tool-call turns across three providers, fixed a step-id
collision across repeated replans, made a transient provider fault
resumable instead of a permanent dead end, capped read_subgraph's node
count, fixed a capability-cache poisoning bug, and closed a stranded-run
leak when a plan node is deleted mid-run.

Test plan

  • pytest -q from repo root: 2396 passed, 17 skipped. The 5 failures in
    backend/tests/test_native_dialogs.py are pre-existing and unrelated to
    this branch — the installed pywebview 5.4 lacks webview.FileDialog in
    this environment.
  • npm run check (web_ui): 1619/1619 passed, bundle-size gate OK.
  • Live end-to-end verification in the browser: a co-pilot build rendered
    the plan node's budget gauges, tool approval panel, and status
    transitions correctly, including a real failure path (unreachable
    provider) landing an actionable error and offering Undo build.

dovvnloading and others added 15 commits August 9, 2026 15:09
…ipeline

Adds the local SQLite knowledge store (documents/chunks, WAL mode, corrupt-db
rescue, backup cadence - mirrors chat_library.py's own connection hygiene),
structure-aware token-budgeted chunking with offset-exact citation support,
and an extract -> chunk -> store ingestion pipeline that reuses attachments.py's
pdf/docx/text extraction plus a new markup-stripping HTML extractor. Ingestion
is idempotent by content hash, scoped per collection.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds an FTS5 external-content index over chunks.text (insert/delete
triggers keep it in sync, migration backfills pre-existing rows), a
bm25-ranked search_chunks() with safe query sanitization against FTS5
operator syntax, and registers knowledge.search on ToolRegistry under a
new knowledge.read scope - auto-approval, read-only.

Tested via direct registry.invoke() calls: ADR-008's tool-use loop (the
piece that offers tools to a live model and processes ToolCallEvents) is
not built yet, so this tool is registered and fully working but not yet
wired into a live conversation - matching ADR-007's own established
precedent for capabilities built ahead of their eventual caller.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds an `embedding` capability to ProviderCapabilities and a concrete
.embed() method on OllamaProvider (per-model probe via ollama.show(),
mirroring ollama_supports_tools) and OpenAIProvider (client-derived,
mirroring image_generation) - the local-first-default-plus-API-option pair
ADR-017 names. Anthropic/Gemini/llama.cpp declare embedding=False for this
stage.

The embeddings table (chunk_id, model_id) migration adds the vector index;
knowledge_embeddings.py owns the numpy pack/unpack and provider calls:
embed_pending_chunks() only ever embeds chunks with no cached vector for
that model (the exit criterion's "cache prevents re-embedding"), and
vector_search() is a brute-force cosine-similarity scan (numpy, already a
dependency - no sqlite-vec or new pip package) matching the ADR's own
"flat index file" alternative.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…untrusted context

Adds backend/knowledge_retrieval.py: reciprocal_rank_fusion() merges FTS5
and vector search by rank (never by raw score - bm25 and cosine similarity
aren't comparable scales), hybrid_search() runs both and fuses when an
embedding-capable provider/model is supplied, degrading gracefully to
lexical-only otherwise. select_within_budget() greedily trims a ranked
result list to a token allowance instead of a fixed k, so retrieval can
never overflow a small local model's context window. format_untrusted_
context() builds the labeled, instruction-resistant evidence block for
automatic chat-turn augmentation, reusing Web Research's own established
spotlighting convention with a distinct [k...] citation marker.

knowledge.search (stage 17.2's tool) now runs hybrid_search() when an
embedding provider/model is registered, unchanged (lexical-only) otherwise.
search_chunks()/vector_search() now also return each chunk's token_count so
budget-aware selection needs no second round-trip.

Fixture-set test proves the exit criterion directly: a query that only
lexical search answers, a paraphrase that only vector search answers, both
correctly resolved by hybrid_search() where either index alone misses one.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…) + citations UI

Backend:
- knowledge_ingest.ingest_text(): a sibling to ingest_file() for already-
  in-memory text with no file path to extract from - web-research retention
  and branch indexing both need this, not extract_text()'s file dispatch.
- Web Research retention: WebResearchRequest.retain_to_knowledge (opt-in,
  default False) makes WebResearchService.run() ingest each accepted
  source document via ingest_text() after synthesis - a retention failure
  is logged and swallowed, never breaks the actual research result.
- Branch indexing: ChatState.index_into_knowledge (per-node opt-in, set on
  whichever node's branch history should be indexed) + SceneDocument.
  set_chat_index_into_knowledge() + the new "knowledge" topic's two WS
  intents (backend/api/intents_knowledge.py): search (hybrid_search(),
  read-only, the frontend-reachable counterpart to backend/tools_knowledge.
  py's ADR-008-future tool) and setChatIndexIntoKnowledge (indexes the
  branch via chat_branch_history()+ingest_text() BEFORE flipping the flag,
  so a stored true always means the write actually happened).

Frontend: a new "Knowledge" search panel (AppBar chip + Dialog, mirroring
DiagnosticsDialog's request/reply shape) - search results render as
"N sources used", each expandable to the exact cited excerpt
(document[offsetStart:offsetEnd], byte-for-byte) with an Open source link
for real http(s) sources. No fabricated "jump to this local file at a byte
offset" mechanism - none exists anywhere in this codebase, so none is
claimed here either; the excerpt itself IS the content at that offset.

Contract: ChatState.index_into_knowledge -> indexIntoKnowledge (codegen
regenerated). Guard updates: node-state-migration wire-key list, undo-
classification intent count (143->145) + two new classifications.

Verified live: real WS round-trip through the Knowledge panel against a
freshly-started backend renders the correct empty-state UI with no console
or server errors.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…race

A 5-dimension adversarial review of the ADR-017 diff surfaced 10 confirmed
findings. This fixes the six safely-scoped ones:

- embed_pending_chunks() now raises instead of silently mispairing
  vectors to chunk_ids when a provider returns a mismatched batch length.
- vector_search() validates query-embedding count and stored-vector
  dimension before np.stack(), turning a would-be opaque numpy crash into
  a clear, diagnosable error.
- KnowledgeSearchDialog's search request had no in-flight guard: the
  Enter-key handler could fire a second request while the first was still
  pending, and an out-of-order response would silently overwrite fresher
  results. A sequence token now discards stale responses, and the guard
  makes a second Enter press during an active search a no-op.
- The search input now has an aria-label, matching every other search
  input in the app.
- Both knowledge WS intent handlers (search, setChatIndexIntoKnowledge)
  now run their blocking SQLite calls via asyncio.to_thread instead of
  inline on the event loop, matching every other blocking-I/O intent
  handler in the codebase.

The remaining four findings are real but out of scope for a review-fix
pass and are documented as known gaps in the (local, gitignored) ADR-017
doc: OpenAIProvider's client-derived (not model-derived) embedding
capability check, the Ollama capability cache's pre-existing
permanent-negative-caching behavior, the WS search intent being
lexical-only in production pending a model-selection surface for
embeddings, and format_untrusted_context's lack of escaping (dormant,
matches Web Research's own already-shipped precedent).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…stamping

The graph becomes a tool surface (the ADR's decision #1). Three pieces:

- api_provider.chat_turn_with_tools(): one model turn that RETURNS tool
  calls instead of dropping them - the exact gap ADR-007 left (the
  streaming dispatch loop never set ChatRequest.tools and silently
  discarded tool_call events). Mirrors _chat_stream_dispatch's provider
  construction, model_ref precedence, transport retry, and error
  translation; gates tools-capability authoritatively at construction;
  collects tool calls + final text + usage. No 18.5 fallback wrapper -
  a mid-build silent model swap is what ADR-018 rules out.

- backend/tools_graph.py: graph.create_node / graph.connect /
  graph.set_node_content (scope graph.mutate, approval "once") and
  graph.read_subgraph (graph.read, auto). Every mutating handler drives
  the same domain factories the WS intents call, wrapped in the same
  record_command - an agent-created node is undoable, patch-published,
  and persisted exactly like a user-created one. Placement is
  parent-relative and model-free; read results are excerpt-capped so
  reads don't eat the build's own token budget.

- run_id stamping: record_command() and composite() accept a run_id
  kwarg (Command.run_id existed since ADR-010 stage 10.5 but nothing in
  production ever assigned it). Handlers read run_id off the RunContext,
  per-call rather than via a long-open composite - the composite buffer
  is document-global, so holding one across an approval await would
  swallow concurrent user commands into the builder's undo entry.

Stage exit criterion covered in backend/tests/test_tools_graph.py: a
scripted agent turn through the real primitive and real registry creates
and connects two nodes, approval-gated (three prompts observed), and the
whole turn reverts as one undo_run while the user's own node survives.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
run_node(node_id, action?) executes a node's action INLINE under the
builder's run - deliberately not re-entering the fire-and-forget
AgentDispatcher surfaces (own busy kinds, intent-wired callbacks, no
awaitable completion). Actions:

- execute (pycoder default): runs the node's current code in the SAME
  REPL a manual Run uses (dispatcher.get_pycoder_repl), under the same
  240 s timeout, landing results through the same complete_pycoder_run /
  fail_pycoder_run domain methods - the node renders identically to a
  manual run. No analysis turn: the builder model is the analyst; it
  reads the output in this same loop.
- reply (chat default): generates an assistant reply child from the
  node's branch history via _call_chat_agent, with branch System-Prompt
  resolution and provider/model provenance stamped, recorded as a
  run_id-stamped command.
- chart (explicit, on any content node): chart generation is an action
  ON a source node, not a node kind's own run - dispatching purely on
  kind would let the chat action shadow it, which the first test run
  caught. Generates via _call_chart_agent into add_chart_node.

Scope enforcement is dynamic per action (execute -> code.execute,
reply/chart -> provider.call) inside the handler - the ADR's "run_node
additionally carries the scope of what it runs" - since the registry's
own scope check is static per-tool. The target node carries the
builder's request_id as pending_request_id for the run's duration:
per-node conflict guards, the live-run undo refusal, and the spinner UI
all come free. web_research runs are a named not-yet error until the
network-gating stage.

ToolRegistry.invoke now re-raises RequestCancelledError from HANDLERS
instead of swallowing it into an error ToolResult - a long-running
handler observing the same cancel event must follow the same contract
as invoke's own checkpoints (cancellation propagates to the loop, never
fed back to the model as a tool "error" to reason about).

Stage exit criterion covered in backend/tests/test_run_node_tool.py:
the scripted agent turn runs a code node for real (fake REPL) and the
tool result it reads back carries the execution output.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plan node - the ADR's "planning is explicit and visible" decision -
lands end-to-end: PlanState (goal/steps/status machine/budgets/spent
counters/approval surface, all plan_-/builder_-prefixed so the
bare-attribute ban needs zero exemptions), add_plan_node/set_plan_steps
domain mutators (completed steps are immutable history - neither a user
edit nor a model replan can rewrite them), 15 wire fields + a typed
PlanStepRow (codegen regenerated), session save/load with the one
load-time normalization: a non-terminal builder_status restores as
"interrupted" - terminal, honest, resumable - since no RunHandle
survives a restart.

backend/builder.py is the loop itself: planning via respond_json
(provider-universal, works where the executor's tools-capable gate would
refuse), then per pending step a bounded turn loop alternating
chat_turn_with_tools with registry invocations. Loop control is in-band
through four auto-approval builder.* tools (complete_step / replan /
finish_build / abort) - a per-step turn cap stops a step that never
completes. Budgets are hard: tokens/wall-clock checked before every turn
and every tool call, the step budget at the outer pre-start point
(spent_steps increments at step START, so an in-flight step must not
trip its own check - caught by this stage's own test). A breach pauses
with all state on the plan node; resume is canvas-sourced.

The approval router is mode-aware: copilot parks every non-auto call on
the run's approval_future (the pycoder swap-per-round mechanics) behind
plan-node awaiting/summary fields; autopilot auto-approves calls whose
registered scopes fit the disclosed set and still prompts for net.fetch.
ToolRegistry gains scopes_for() so the router keys on registration truth.

AgentDispatcher.start_builder_run claims the new "builder" kind (Stop =
release-on-cancel + finalize landing "stopped", the 6.2 posture) and
lazily builds the session's ToolRegistry - ADR-007's registry finally
gets its first production constructor. Six new intents (builder/start|
startExecution|cancel|approveTool|denyTool + scene/setPlanSteps),
classified, gate 145 -> 151; builder/start is A - it genuinely records
the plan-node creation, stamped post-claim with the run id so undo_run
reverts the plan node too. Planner/executor prompts join the pinned
registry inventory (terse by design - every token recurs per turn; the
executor carries the untrusted-content spotlighting language).

Stage exit criterion covered in backend/tests/test_builder.py: a
scripted co-pilot build lands a 4-node branch with every mutating call
individually approved, control tools never prompting, budgets/replan/
abort/deny paths each proven, Stop verified as slot-release-immediate,
and the whole build reverting via one undo_run. Mid-step replan keeping
the running step live (and the loop re-resolving it from the replaced
list) was a real bug this suite caught before it shipped.

Frontend (PlanNodeView + launcher) follows in the next commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PlanNodeView renders the build live off the wire row: goal, the
checklist with per-status markers and step summaries, the three budget
gauges (spent/max steps, tokens, seconds), status + detail (failed
details are role=alert), the autopilot chip, and mode-appropriate
controls - Start build / Resume for awaiting_start|paused|interrupted,
Stop while running. The tool-approval panel copies
CodeExecutionApprovalPanel's architecture: per-node, zero passive
dismissal (a dismissal would strand the run's parked approval future),
and zero-argument Approve/Deny closed over the current snapshot's
pendingRequestId.

BuilderLaunchDialog (AppBar, next to Knowledge) collects goal, oversight
mode, and the three hard budgets, and starts the build through the
value-returning builder/start intent - selecting autopilot surfaces the
disclosure sentence inline before launch (per-run, disclosed choice per
the ADR). Six new sceneStore methods wire the run controls.

Fixture ripple: the 15 new required wire fields land in the five
SceneNodeRow fixture builders (the 17.5 indexIntoKnowledge precedent).
Styling uses only the design-system variables - the no-raw-colors gate
caught the first draft's hex fallbacks.

Verified: 10 new component tests; full frontend check green (1612
tests, typecheck, lint, build, bundle gate); full backend suite green
(2364 passed; the 5 pre-existing test_native_dialogs.py environment
failures are untouched by this branch).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Undo this build" closes the loop ADR-010 stage 10.5 left open: the
undo_run machinery and the scene/undoRun intent shipped with zero
callers, and stage 8.1 made the builder stamp every command with its
run id - this adds the affordance. PlanNodeView offers an "Undo build"
button once the run is over (done/failed/stopped/interrupted/paused,
with a stamped run id), wired through the existing sceneStore.undoRun.
The domain's live-run guard already refuses undo mid-run, so
Stop-then-undo remains the enforced sequence with no new code.

Live browser verification caught a real bug: the plan node's buttons
lacked the `nodrag` class, so React Flow's drag handler swallowed every
click (the ChatNodeView-pinned convention). Fixed on all five buttons
and pinned by a new test. Verified end-to-end in the running app: a
build launched from the AppBar dialog landed a plan node (with the
Ollama-unreachable failure surfacing honestly as role=alert), and Undo
build reverted the run-stamped creation.

The stage's revert-the-whole-build exit criterion is covered by
test_builder.py's exit test (undo_run reverts a 4-node scripted build,
the user's own node surviving).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
run_node gains the "research" action (web_research's default): the SAME
sync pipeline the dedicated surface runs - WebResearchService with all
of ADR-004's SSRF/IP-pinning/robots machinery inside its fetcher -
landing through the same complete/fail domain methods, inline under the
builder's run. The builder's threading.Event cancel bridges onto the
service's own CancellationToken via a watcher task (the pipeline stages
checkpoint on the token, not our event) - proven by a test that hits
Stop mid-run and watches the token trip.

Designing it surfaced a real autopilot hole: the mode router keyed
auto-approval on a tool's REGISTERED scope, and run_node registers only
graph.read - autopilot would have silently auto-approved a net.fetch
research run. run_node_effective_scope() now derives the scope a call
actually exercises (target kind + action) and the router unions it in;
malformed run_node calls route to the human. Two tests pin the exit
criterion's "no network unless approved": a net.fetch tool prompts in
autopilot, and run_node(research) prompts via the derived scope while
graph mutations still auto-approve.

ADR-007's deferred MCP runtime wiring lands in its designated consumer:
builder_tool_registry reads the persisted server list, connects each
enabled server, and registers its tools (namespaced, per-server scopes,
approval="always") with per-server failure tolerance - one broken
config never costs the Builder its graph tools.

The interrupted-on-load normalization (shipped in 8.3's restorer) gets
its round-trip tests: a "running" build restores as interrupted with
its mid-flight step failed - never a spinner no run backs - while
terminal states and spent budgets round-trip verbatim.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recipes are data (the ADR's decision #4): named plans - goal, step
titles, default mode - that seed a build. The settings store gains
get_recipes/set_recipes (the get_mcp_servers posture exactly: JSON-safe
dicts, shared normalization so a round trip is always well-formed,
malformed entries dropped, whole-list replace). Two built-ins ship as
constants in backend/builder.py, merged read-only at list time so a
rename never desyncs a user's file.

builder/start gains an optional recipe argument: a recipe-seeded plan
lands its checklist immediately at awaiting_start with NO planning
model call (proven: the planner is monkeypatched to fail loudly and is
never reached). builder/listRecipes (request/reply) and
builder/saveRecipe complete the loop - save-your-build captures a
terminal plan node's goal + step titles (statuses deliberately dropped:
a recipe is the plan, not this run's history), refuses built-in names,
and replaces same-named user recipes. Classification gate 151 -> 153.

Launcher: a recipe picker (built-ins labeled) that enables launch
without a typed goal and relabels the action "Start from recipe";
PlanNodeView offers "Save as recipe" on done builds.

The 8.6 exit criterion is covered both ways in test_builder.py: a
shipped recipe and a user-saved build each seed a run through the real
WS intents. Full backend suite: 2378 passed (the 5 pre-existing
test_native_dialogs.py environment failures are untouched by this
branch); full frontend check green (1618 tests, bundle gate OK).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…, canonicalize chart data

Fixes 13 bugs found by re-reading the branch's own diff after an adversarial
review pass: autopilot treated an empty MCP tool scope set as auto-approved
(any unscoped server bypassed human approval entirely); Ollama/Anthropic/
Gemini all dropped already-collected token usage on tool-call turns, so the
builder's token budget went effectively unenforced; a second plan replan
could mint a step id that collided with one an earlier replan already used,
killing the build with an unresumable error; a watchdog timeout or any other
provider exception left the in-flight step wedged at "running" forever, and
"failed" was not a resumable status even though the plan node's state lives
entirely on the canvas; a budget breach discovered on a later tool call in a
turn could stomp a step an earlier call in that same turn had just completed,
or silently drop a declared finish; graph.read_subgraph had no cap on node
count, so a hub node's full read could overflow a turn's context window; a
transient Ollama capability-probe failure was cached as a permanent negative,
silently blocking the builder forever with a false "no tool support" error;
the run_node schema still advertised research as unsupported; the web
research service's own cancellation exception was swallowed into an ordinary
tool error instead of propagating as a real cancellation; chart generation
skipped canonicalize_chart_data before storing the model's raw output,
violating the chart state's documented invariant; deleting a plan node with
a live builder run never cancelled it, permanently locking the builder for
the rest of the session; and undo/redo of a plan-node command recorded
mid-run could resurrect a stale "running"/"awaiting_approval" status with no
live run behind it.

Also splits register_node_intents's live-run teardown capture into its own
function to stay under the register* 300-line cap after the new plan-cancel
branch pushed it over.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts:
#	backend/domain/node_states.py
#	contracts/graphlink_scene_payload.py
#	tests/test_undo_classification_gate.py
#	web_ui/src/app/App.tsx
#	web_ui/src/app/canvas/SceneCanvas.pinSearchJump.test.tsx
#	web_ui/src/app/canvas/SceneCanvas.test.tsx
#	web_ui/src/app/canvas/SceneCanvas.virtualization.test.tsx
#	web_ui/src/app/canvas/renderCountGate.test.tsx
#	web_ui/src/app/canvas/sceneStore.test.ts
#	web_ui/src/app/chrome/AppBar.tsx
#	web_ui/src/lib/bridge-core/generated/scene-state.schema.json
#	web_ui/src/lib/bridge-core/generated/scene-state.ts
@dovvnloading
dovvnloading merged commit ca32814 into main Aug 10, 2026
3 checks passed
@dovvnloading
dovvnloading deleted the adr008-agentic-builder branch August 10, 2026 17:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant