RFC: Enterprise-Grade Roadmap & Architectural Specifications for AutoMaintainer 2.0

[purvanshjoshi]

RFC: Enterprise-Grade Architecture & Feature Roadmap for AutoMaintainer 2.0

As AutoMaintainer transitions from a multi-agent proof-of-concept into a production-ready, industry-standard Autonomous AI Software Engineering Platform (competing with Devin, OpenHands, and GitHub Copilot Workspace), we are establishing this Request for Comments (RFC).

This proposal outlines the architectural specifications, data flows, security sandboxing, graph/loop engineering principles, harness engineering for CLI/SDK/IDE, and technical blueprints for the next phase of development.

---

1. Dynamic Sandboxed Test Execution and Self-Healing Loop (test_node)

Problem Statement

Currently, the Implementer Node modifies files and relies purely on LLM static review to verify correctness. Without runtime execution, agent changes risk syntax errors, broken imports, and regression bugs escaping into pull requests.

Architectural Blueprint

Integrate an automated execution node (test_runner_node) between the Implementer and Reviewer in the LangGraph agent state graph.

flowchart TD
    A["Architect Node"] --> B["Implementer Node"]
    B --> C["Sandboxed Test Runner Node"]
    C -->|"Tests Pass"| D["Reviewer Node"]
    C -->|"Tests Fail (iterations < 3)"| E["Traceback and Failure Extractor"]
    E -->|"Error Context and Diff"| B
    C -->|"Tests Fail (iterations >= 3)"| D
    D -->|"Approved"| F["Maintainer / PR Node"]
    D -->|"Iterate"| B

Loading

Technical Specifications:

1. Test Suite Auto-Discovery:
   • Detect project language and framework heuristics via AST and root metadata:
     • Python: pytest, python -m unittest, tox, poetry run pytest
     • TypeScript / JavaScript: npm test, yarn test, pnpm test, jest, vitest
     • Rust / Go: cargo test, go test ./...
2. Containerized Sandboxing:
   • Execute test suites inside isolated Docker/gVisor micro-containers with strict resource limits (max 2 vCPUs, 2GB RAM, 60s timeout, non-root user, disabled outbound network access except local database mocks).
3. Structured Failure Parsing:
   • Parse stdout, stderr, assertion diffs, and file tracebacks.
   • Extract exact failing test names, line numbers, and expected vs. actual values.
4. Self-Correction Iteration:
   • Inject the structured test error traceback directly into the Implementer's prompt with instructions: "The previous changes failed with the following traceback. Fix the root cause without breaking existing test suites."

---

2. Human-in-the-Loop (HITL) Plan Approval and Interactive Checkpoints

Problem Statement

In mission-critical enterprise codebases, engineers require supervisory control to review and refine technical designs before an AI agent touches code.

Architectural Blueprint

Leverage LangGraph's native breakpoint capability (interrupt_before=["implementer"]) with asynchronous state persistence in Supabase.

sequenceDiagram
    autonumber
    actor User as Engineer / Tech Lead
    participant UI as Next.js Dashboard
    participant API as FastAPI Backend
    participant Graph as LangGraph Engine
    
    User->>UI: Trigger Issue / Task
    UI->>API: POST /start (repo, issue_id)
    API->>Graph: Execute architect_node
    Graph-->>API: Yield Plan and Interrupt
    API->>UI: Broadcast Plan via Supabase (WAITING_APPROVAL)
    UI->>User: Render Interactive Plan Approval Modal
    User->>UI: Modify Constraints / Whitelist Files / Approve
    UI->>API: POST /runs/{run_id}/resume (approved_plan)
    API->>Graph: Resume Graph Execution (implementer_node)

Loading

Technical Specifications:

• Interactive Plan Modal in Next.js:
  • Displays proposed files to create/edit/delete with checkboxes.
  • Allows developers to inject prompt constraints (e.g., "Do not modify the database schema, use in-memory caching instead").
  • One-click actions: Approve and Run, Edit Plan, or Abort Task.
• State Checkpointing:
  • Save full snapshot of AgentState to Supabase checkpoints table so runs can resume seamlessly across browser restarts.

---

3. Native GitHub App and ChatOps Engine (@automaintainer)

Problem Statement

Requiring developers to navigate to a separate dashboard creates friction. Integrating directly into the GitHub issue/PR workflow delivers zero-friction developer adoption.

Architectural Blueprint

A lightweight GitHub App webhook listener (/api/webhooks/github) with cryptographic payload verification.

Technical Specifications:

1. Event Triggers:
   • issues.labeled: Automatically starts an agent run when an issue is labeled automaintain or needs-ai.
   • issue_comment.created: Listens for maintainer commands on PRs or issues.
2. Supported Slash Commands:
   • @automaintainer /fix-ci — Reads GitHub Actions failure logs, replicates the issue, and pushes a fixing commit to the PR branch.
   • @automaintainer /add-tests [file] — Generates unit and integration tests with high code coverage.
   • @automaintainer /explain — Summarizes architectural intent and potential blast radius for reviewers.
   • @automaintainer /refactor — Applies clean code, performance optimizations, and typing improvements.
3. Live Interactive GitHub Timeline Updates:
   • Posts a single collapsible, dynamically updated comment on the issue/PR showing real-time agent phase progression.

---

4. Persistent Codebase Knowledge Graph and Vector Memory

Problem Statement

Re-parsing repository files from scratch on every run causes high token overhead and forgets historical architectural context, coding styles, and previous PR solutions.

Architectural Blueprint

A hybrid retrieval architecture combining AST Tree-Sitter Knowledge Graphs with Dense Semantic Vector Search stored in Supabase pgvector.

flowchart LR
    subgraph Ingestion["Ingestion Pipeline"]
        A["Git Repository"] --> B["Tree-Sitter AST Parser"]
        A --> C["Docstrings and ADRs"]
        A --> D["Historical Merged PRs"]
        B --> E["Symbol and Dependency Graph"]
        C --> F["Embedding Model (text-embedding-3)"]
        D --> F
        E --> G[("Supabase pgvector + Graph")]
        F --> G
    end
    
    subgraph Query["Query Pipeline"]
        H["Target Issue / Prompt"] --> I["Hybrid Search Engine"]
        G --> I
        I --> J["Enriched Context for Agents"]
    end

Loading

Technical Specifications:

• Symbol Dependency Graph: Map class hierarchies, interface implementations, callers, and callees across Python/TypeScript.
• Coding Style Memory: Index CONTRIBUTING.md, linting configurations, and merged PR conventions so generated code matches the exact idiom of the host project.
• Sub-symbolic Context Pruning: Retrieve only relevant functions/classes rather than dumping entire 500-line files into the prompt, reducing token consumption by ~70%.

---

5. Multi-Model Intelligent Routing and Automatic Failover Engine

Problem Statement

Hardcoding a single model provider (e.g., Groq Llama 3) creates vulnerability to rate limits (HTTP 429), context length limits, and API outages.

Architectural Blueprint

An intelligent, cost-aware LLM Gateway that routes agent tasks based on reasoning complexity.

Agent Role	Primary Model	Fallback Model	Rationale
Architect	Claude 3.7 Sonnet / GPT-4o	Gemini 2.0 Pro	High reasoning, large context planning, architectural foresight
Implementer	Groq Llama 3.3 70B	Claude 3.5 Sonnet / DeepSeek V3	High-speed code synthesis, low latency
Test Runner / Syntax Fixer	Groq Llama 3.3 70B	GPT-4o-mini	Fast syntax adjustments
Reviewer and Maintainer	Claude 3.7 Sonnet / GPT-4o	Gemini 2.0 Pro	Deep security analysis, AST inspection, blast-radius assessment

---

6. Visual PR Impact Engine with Mermaid.js Diagrams

Problem Statement

Reviewing multi-file AI pull requests can be mentally exhausting for maintainers without visual architectural diffs.

Architectural Blueprint

The Maintainer Node automatically constructs standardized PR descriptions containing:

1. Mermaid.js Flowcharts: Visualizing changed components and modified execution paths.
2. Blast Radius and Security Matrix:

   Area	Impact Level	Description
   API Endpoints	Low	Added optional run_id parameter to /stop
   Database Schema	None	No table migrations required
   Security Scope	Hardened	File descriptor ownership exclusively managed
   Dependencies	None	No new third-party packages introduced

3. Automated Test Coverage Deltas: Real-time diff of code coverage before vs. after changes.

---

7. Graph Engineering: Formalizing Stateful Multi-Agent Topologies

Core Principles of Graph Engineering

Graph Engineering is the disciplined design of stateful, deterministic, and modular computational graphs for autonomous AI systems. Moving beyond unstructured agent loops, Graph Engineering treats agent interactions as explicit finite state machines (FSMs) with strictly defined state transitions, schemas, and reducers.

flowchart TD
    subgraph GlobalGraph["Global AutoMaintainer Graph"]
        StartNode(["START"]) --> ArchNode["Architect Node"]
        ArchNode --> RouterBranch{"Plan Approved?"}
        
        RouterBranch -->|"Yes"| SubgraphExec["Nested Implementation Subgraph"]
        RouterBranch -->|"No / Interrupted"| CheckpointWait["State Checkpoint (HITL Wait)"]
        CheckpointWait -->|"Resume"| SubgraphExec
        
        subgraph SubgraphExec["Nested Implementation Subgraph"]
            direction TB
            CodeSynth["Code Synthesizer"] --> SandboxTest["Sandboxed Test Runner"]
            SandboxTest --> QualityGate{"Tests Pass?"}
            QualityGate -->|"No (iter < k)"| ReflectionNode["Failure Reflection Engine"]
            ReflectionNode --> CodeSynth
            QualityGate -->|"Yes / Exhausted"| SubgraphOut(["Yield to Reviewer"])
        end
        
        SubgraphExec --> ReviewNode["Reviewer Node"]
        ReviewNode --> FinalGate{"Approval Status"}
        FinalGate -->|"LGTM"| PRNode["Maintainer / PR Node"]
        FinalGate -->|"Needs Work"| ArchNode
        PRNode --> EndNode(["END"])
    end

Loading

Key Graph Engineering Constructs in AutoMaintainer 2.0:

1. Strict Typed State Schemas and State Reducers:
   • Define immutable state interfaces using TypedDict and Pydantic v2 models.
   • Use custom state reducers (such as append-only message channels and commutative patch sets) to eliminate race conditions across asynchronous node dispatches.
2. Hierarchical Nested Subgraphs:
   • Deconstruct monolithic agent logic into isolated, modular subgraphs (e.g., an Implementation Subgraph and an Evaluation Subgraph).
   • Scoped subgraphs operate on local state variables and project consolidated results back to the global execution context upon completion.
3. Time-Travel Checkpointing and State Replay:
   • Every graph super-step persists a cryptographic state hash and snapshot into Supabase / PostgreSQL.
   • Enables time-travel debugging: if an agent takes a wrong path, the system can roll back to a specific checkpoint and re-branch execution.

---

8. Loop Engineering: Convergence, Bounded Iterations, and Feedback Dynamics

Core Principles of Loop Engineering

Loop Engineering governs the iterative dynamics of autonomous agents. Without rigorous loop engineering, self-correcting agents suffer from context poisoning, infinite cyclic regressions, and divergent code degeneration.

flowchart LR
    subgraph LoopDynamics["Tri-Phase Convergence Loop"]
        direction TB
        S1["1. Synthesis Phase: Generates Unified Diff"] --> S2["2. Execution Phase: Sandboxed Verification"]
        S2 --> S3["3. Reflection Phase: Root-Cause Analysis"]
        S3 --> S4{"Convergence Check"}
        S4 -->|"Pass"| ExitSuccess["Exit Loop (Converged)"]
        S4 -->|"Fail (Monotonic Progress)"| S1
        S4 -->|"Fail (Degraded / Max Iter)"| RollbackState["Rollback to Best Known Checkpoint"]
    end

Loading

Key Loop Engineering Mechanisms in AutoMaintainer 2.0:

1. Bounded Iteration Horizons ($k_{max}$) and Cost Ceilings:
   • Enforce hard iteration limits ($k=3$) and dynamic token burn quotas per task.
   • If an agent does not converge within $k_{max}$, the graph gracefully terminates and produces a diagnostic triage report rather than burning unbounded tokens.
2. High-Entropy, Compact Error Ingestion (Zero Context Poisoning):
   • When a test fails, do not dump raw 10,000-line terminal logs into the context window.
   • Extract only high-entropy signal: failing test symbol, assertion mismatch, traceback root frame, and unified git diff. This maintains a clean signal-to-noise ratio in LLM attention layers.
3. Monotonic Progress Assertion and Checkpoint Rollbacks:
   • Track test suite pass counts across iterations ($P_t \ge P_{t-1}$).
   • If iteration $t$ breaks previously passing tests (regression), the loop automatically triggers a Git Branch Rollback to the best-known checkpoint before attempting a new synthesis strategy.
4. Structural Reflection Before Mutation:
   • Enforce a dedicated Reflection Step prior to code generation. The agent must first answer: "Why did the previous patch fail? What assumption was violated?" before generating new code tokens.

---

9. Harness Engineering: Core Abstraction for CLI, SDK, and WebIDE

Core Principles of Harness Engineering

Harness Engineering is the systematic construction of execution adapters, runtime boundaries, and interface protocols that decouple the core agent reasoning engine from the host environment. By creating a unified Core Agent Protocol (CAP), the exact same agent brain powers the AutoMaintainer CLI, the Enterprise SDK, the WebIDE, and the SWE-bench Evaluation Testbed.

flowchart TD
    subgraph ClientSurfaces["Multi-Surface Distribution Layer"]
        CLI["AutoMaintainer CLI (TUI / Headless)"]
        SDK_Py["Enterprise Python SDK"]
        SDK_TS["TypeScript / Node.js SDK"]
        WebIDE["Next.js WebIDE (Monaco + PTY)"]
        Bench["SWE-bench Evaluation Harness"]
    end

    subgraph CAP_Layer["Unified Core Agent Protocol (CAP / gRPC / JSON-RPC)"]
        Adapter["Streaming Event Bridge & Lifecycle Multiplexer"]
    end

    subgraph AgentCore["AutoMaintainer Core Agent Engine"]
        GraphEngine["LangGraph Execution Topology"]
        SandboxBridge["Sandboxed Execution Bridge"]
        MemoryStore["pgvector Semantic Memory"]
    end

    ClientSurfaces --> Adapter
    Adapter --> AgentCore

Loading

Key Harness Specifications in AutoMaintainer 2.0:

A. The AutoMaintainer CLI Harness (automaintainer-cli)

• Interactive Terminal User Interface (TUI): Built with Python Rich / Textual providing live split-view agent tree visualization, terminal logs, and interactive plan approval inside developer shells.
• Headless CI Mode: Designed for GitHub Actions / GitLab CI pipelines:

  automaintainer run --repo . --issue 142 --non-interactive --output-format sarif

• Exit Code Contracts: Strict exit codes (0 for success/PR created, 1 for execution failure, 2 for quality gate/test failure, 3 for budget exhaustion) enabling deterministic CI gating.

B. The AutoMaintainer Enterprise SDK Harness (@automaintainer/sdk & automaintainer-sdk)

• Strongly-Typed Client Libraries: Asynchronous Python (asyncio) and TypeScript client libraries exposing declarative run management, event streams, and token budgeting:

  import { AutoMaintainerClient } from '@automaintainer/sdk';
  
  const client = new AutoMaintainerClient({ apiKey: process.env.AUTOMAINTAINER_API_KEY });
  const run = await client.runs.create({
    repo: 'PxA-Labs/AutoMaintainer',
    targetIssue: 142,
    onEvent: (event) => console.log(`[${event.phase}] ${event.message}`)
  });

• Streaming Observability: Native WebSocket and Server-Sent Event (SSE) subscriptions for real-time token telemetry, latency monitoring, and step-by-step diff streaming.

C. The WebIDE Engine Harness

• Virtualized Filesystem & LSP Bridge: Monaco editor integrated with Language Server Protocol (LSP) multiplexers for TypeScript, Python, Rust, and Go.
• PTY WebSocket Terminal Harness: Cross-platform pseudoterminal multiplexer with process tree tracking, automatic orphan SIGTERM cleanup, and VT100 escape sequence streaming.
• Live Staging & Interactive Diffs: Monaco side-by-side diff editor allowing developers to visually inspect and adjust AI patches before commit creation.

D. The Evaluation & Benchmark Test Harness (SWE-bench)

• Deterministic Sandbox Harness: Disposable Docker container fixtures isolating repository environments for quantitative benchmarking against SWE-bench Lite and SWE-bench Verified tasks.
• Continuous Regression Harness: Automatically runs daily end-to-end regression suites against historic resolved issues to guarantee new prompt or model updates never degrade code generation accuracy.

---

10. Phased Roadmap and Implementation Plan

flowchart LR
    subgraph P1["Phase 1: Quality and Self-Healing"]
        direction TB
        A1["Sandboxed Test Runner Node"] --> A2["Traceback and Self-Healing Loop"]
    end
    
    subgraph P2["Phase 2: Graph, Loop and Harness Foundations"]
        direction TB
        B1["Nested Subgraphs and Reducers"] --> B2["Core Agent Protocol and Checkpoints"]
    end
    
    subgraph P3["Phase 3: Multi-Surface Ecosystem"]
        direction TB
        C1["CLI and Enterprise SDK Release"] --> C2["WebIDE LSP and GitHub App"]
    end
    
    subgraph P4["Phase 4: Intelligence and Benchmarks"]
        direction TB
        D1["Persistent pgvector Memory"] --> D2["SWE-bench Evaluation Harness"]
    end

    P1 --> P2
    P2 --> P3
    P3 --> P4

Loading

---

Community Feedback and Discussion

We invite maintainers, contributors, and the open-source community to weigh in:

1. Which client interface (CLI, Python/TS SDK, or WebIDE) is most critical for your engineering workflow?
2. Which graph constructs and loop termination heuristics do you find most reliable in agent deployments?
3. What additional commands or integrations would you like to see supported in the CLI and SDK?

Please share your thoughts, suggestions, and feedback below.

[archittmittal]

Community Feedback on RFC: AutoMaintainer 2.0 Architectural Specifications

Thank you for putting together this comprehensive RFC. The transition toward a formal graph topology with deterministic loop and harness engineering is the exact evolution needed for AutoMaintainer 2.0.

Here is our perspective and feedback on the core architectural questions and execution priorities:

---

1. Client Interface Priority: AutoMaintainer CLI First (automaintainer-cli)

• Rationale: Software engineering happens natively on the local filesystem and in CI/CD pipelines. A rich terminal interface (built on Rich / Textual) provides immediate utility to developers without context-switching to a web browser.
• Key Requirements:
  • Interactive TUI Mode: Split-pane agent DAG visualization, live token telemetry, and keyboard-driven HITL plan approval.
  • Headless CI Mode: Deterministic exit codes (0 for clean PR created, 1 for execution failure, 2 for test regression, 3 for budget exhaustion) with SARIF/JSON output formats to enable seamless GitHub Actions integration.
• Phasing: Prioritizing the CLI in Phase 3 establishes the Core Agent Protocol (CAP) adapters that can subsequently power the WebIDE and Enterprise SDK harnesses.

---

2. Sandboxed Test Runner: Local Docker Container Sandboxing

• Rationale: Local containerized sandboxing (docker run with micro-container fixtures, strict resource quotas of 2 vCPUs / 2GB RAM, 60s hard timeouts, and disabled non-local network access) provides reproducible isolation across Python, TypeScript, Go, and Rust without requiring remote SaaS dependencies or cloud infrastructure lock-in.
• High-Entropy Extraction: As outlined in the RFC, keeping the error ingestion compact (parsing root tracebacks and failing assertion deltas rather than dumping full 10k-line logs) is vital to preserve attention fidelity and prevent context poisoning.

---

3. Loop Engineering & Convergence: Bounded Horizon ($k \le 3$) + Monotonic Assertion & Git Rollback

• Convergence Contract: Enforcing hard $k=3$ iteration horizons prevents infinite repair loops and runaway token expenditure.
• Monotonic Progress & Rollbacks: Tracking passing test counts ($P_t \ge P_{t-1}$) is critical. If iteration $t$ breaks previously passing test suites, the loop should automatically trigger a Git branch rollback to the best-known checkpoint before attempting a new synthesis strategy with structural reflection.

---

4. Multi-Model Routing: Hybrid Role-Based Gateway

• Routing Strategy:
  • Architect & Reviewer Nodes: Claude 3.7 Sonnet / GPT-4o for high-reasoning planning, AST blast-radius evaluation, and security checks.
  • Implementer & Test Runner Nodes: Groq Llama 3.3 70B for high-throughput, ultra-low latency code synthesis and quick syntax corrections.
• Resilience: Automatic fallback routing to secondary endpoints upon encountering HTTP 429 rate limits or provider downtime ensures uninterrupted pipeline execution.

---

Looking forward to collaborating on the Phase 1: Quality and Self-Healing implementation!

[purvanshjoshi]

Hi @archittmittal,

Thank you for the detailed feedback on the RFC.

1. CLI-First Distribution: Fully aligned on prioritizing the CLI harness (automaintainer-cli). Establishing a robust terminal interface with deterministic exit codes (0 for success, 1 for error, 2 for regression, 3 for budget limits) and SARIF output provides immediate value for local development and CI pipelines while defining the Core Agent Protocol (CAP) for downstream consumers.
2. Sandboxed Test Runner: Local containerized execution with strict resource caps (2 vCPUs, 2GB RAM, 60s timeout) and non-local network isolation ensures reproducible, dependency-free execution across all supported languages.
3. Convergence Invariants: The k=3 horizon paired with monotonic pass assertions (P_t >= P_t-1) and automated Git branch rollbacks guarantees that iterative fixes converge without introducing regressions.
4. Model Gateway: The hybrid routing matrix (Claude 3.7 Sonnet / GPT-4o for Architect and Reviewer; Groq Llama 3.3 for high-throughput Implementer loops) delivers optimal reasoning fidelity while keeping latency low.

We will formalize these priorities directly into the Phase 1 milestones.

[krrish2803]

Here is my feedback :

Which interface is most important? (CLI, SDK, or WebIDE)
We recommend focusing on the CLI + WebIDE:
Command Line (CLI): Super important for running AI checks automatically inside GitHub Actions (CI/CD) so code is checked on every push.
Web Dashboard (WebIDE): Super important for developers to visually inspect and approve code changes before the AI touches main files.

What rules work best to keep the AI safe and prevent loops?
3-Attempt Limit (k=3): Limit the AI to 3 attempts max per bug fix so it doesn't get stuck in infinite loops or burn API tokens.
Smart Error Cleaning: When tests fail, send only the exact error message and line number to the AI instead of dumping huge 10,000-line logs.
Automatic Rollback on Regressions: If a new AI fix breaks old tests that were previously working, automatically roll back the code to the last working checkpoint.

What new features should be added?
🛡️ automaintainer audit: A command that scans code for security vulnerabilities and dangerous dependency updates.
⏪ automaintainer replay: A command to rewind and inspect past AI execution steps if something goes wrong.
🧪 automaintainer bench: A tool to measure how accurately the AI solves real-world coding bugs.

[purvanshjoshi]

Hi @krrish2803,

Thank you for reviewing the roadmap and sharing your perspective.

1. Interface Balance: We agree with the dual focus on the CLI (for CI/CD pipelines and local terminal workflows) and the WebIDE (for visual code diff inspection and HITL plan approvals).
2. Convergence and Error Distillation: Enforcing the 3-attempt ceiling and compact error distillation (passing only failing assertion diffs and root traceback frames) ensures we maintain a clean signal-to-noise ratio in LLM attention layers without context poisoning.
3. Proposed Commands:
   • automaintainer audit: A dedicated security and dependency audit command will be incorporated into the CLI specification.
   • automaintainer replay: Step-by-step checkpoint time-travel inspection will leverage the Supabase state snapshot logs.
   • automaintainer bench: Standardized SWE-bench evaluation runner for measuring resolution accuracy across versioned releases.

These command interfaces will be integrated into the CLI harness specifications.

[archittmittal]

@purvanshjoshi we can also get hands on Harness Deepseek opensource model

[purvanshjoshi]

Hi @archittmittal,

Regarding DeepSeek integration:

We agree that integrating open-weight reasoning models like DeepSeek-R1 and DeepSeek-V3 is a strong addition to our model gateway. Their reasoning-to-cost efficiency makes them ideal candidates for the Implementer and Reflection nodes, whether hosted locally via vLLM / Ollama or accessed via inference endpoints. We will include DeepSeek provider configurations in the Multi-Model Routing specifications.

[manyagkarle13]

Hi — I’m @manyagkarle13 (contributor).
The existing Persistent Codebase Knowledge Graph + Vector Memory can be extended to retain maintainer feedback and decisions from HITL checkpoints.

For example, when a maintainer approves, rejects, or modifies an AI-generated plan, the system could optionally store that decision along with the relevant context and reason. Future agent runs could retrieve these decisions when working on similar changes, helping the agent follow established project-specific preferences and avoid repeating previously rejected approaches.

This could also work alongside the existing reflection and rollback mechanisms, giving the agent not only knowledge of the codebase, but also a history of human-approved engineering decisions.

I’d be interested to know if this could fit into the proposed Knowledge Graph/Vector Memory layer, or if maintaining this type of feedback history would introduce too much complexity.

[purvanshjoshi]

Hi @manyagkarle13,

Thank you for this valuable proposal.

Capturing human feedback and decision rationales from HITL checkpoints fits naturally into our persistent memory architecture.

1. Schema & Ingestion: When a maintainer approves, modifies, or rejects a plan, we can record a structured decision record containing the task context, proposed plan summary, user feedback/constraints, and decision outcome (APPROVED, REJECTED, MODIFIED).
2. Vector Indexing: Storing embeddings of these decisions in pgvector allows the Architect and PM nodes to perform semantic similarity lookups on historical approvals and rejections during planning.
3. Low Overhead: Because this involves lightweight metadata writes triggered only at user interaction checkpoints, it introduces negligible computational overhead while significantly improving alignment with repository-specific conventions over time.

We will incorporate this Human Decision Memory model into Section 4 (Persistent Codebase Knowledge Graph and Vector Memory).

[archittmittal]

Feedback: hosting model + what web → CLI → SDK sequencing costs us

Reviewed the current tree against this RFC. Two things worth folding in.

1. HF Spaces won't carry a public launch — and the terminal is why

The single container in Dockerfile is web server, agent runtime, and sandbox at once. On Spaces that's fine (one tenant). Publicly it breaks on four points:

Issue	Where	Impact
Shell for every visitor	backend/main.py:369-374 (pty.fork() → os.execv("/bin/bash"))	Any user gets a real shell in the server's own container — can read GITHUB_TOKEN, GROQ_API_KEY, SUPABASE_SERVICE_KEY from /proc/1/environ, plus every other tenant's clone in /tmp
One global run slot	main.py:154 active_task	User B's /start and /stop clobber user A. Ceiling of 1 concurrent pipeline per process
Shared workspace	agents.py:319, main.py:33 — /tmp/{owner}_{repo}	Two users on the same repo share a git working tree
One shared identity	agents.py:22-24, module-level gh = Github(...)	Everyone's agents push as our token; no per-user credential path exists

The Web IDE terminal means we aren't hosting a web app, we're hosting untrusted RCE — which decides the container runtime. §1 already specifies gVisor micro-containers for test_runner_node; that same bar has to cover the PTY, and plain runc doesn't meet it (shared kernel, one LPE = all tenants' tokens).

Proposal: control plane (stateless FastAPI: auth, quotas, run records — no git, no bash, no /tmp) + one ephemeral Firecracker microVM per run, 5–15 min TTL, where clone/gitnexus/LangGraph/PTY all live. Buy this rather than build it — E2B or Modal both expose a PTY API, so terminal_ws becomes an authenticated proxy and os.execv("/bin/bash") leaves our codebase. Self-host fallback: Kata > gVisor > never runc. Per-sandbox: 2 vCPU, 2–4GB, egress allowlisted to GitHub/Groq/Supabase/registries (a cloned repo can ship a malicious postinstall, and gitnexus runs npm-installed grammars).

Credentials: replace the PAT at agents.py:323 with a GitHub App — per-run installation tokens, single repo, 1h expiry. Nothing long-lived ever enters a sandbox, and it fixes rate limits. The GROQ_API_KEY_{i} rotation loop at agents.py:76 is already a signal we're at the ceiling of one shared key.

Our Supabase Realtime log bus is the piece that already scales correctly — keep it.

2. CLI is a second runtime, not a thin client

Re @krrish2803's CLI+WebIDE priority — worth being explicit that these don't share an execution path. A CLI user has the repo checked out locally and expects agents to run against their working tree, including uncommitted changes. They won't accept "we clone from GitHub into our cloud VM."

	Web	CLI	SDK
Runs where	our sandbox	user's machine	caller's process/CI
Repo source	git clone	local working tree	either
Isolation	critical	none	theirs

If agents.py only knows how to clone into /tmp (:319-334), the CLI becomes a fork of the pipeline and the two drift. Needs a Runner interface — LocalRunner / SandboxRunner — landed before the web launch even though web only uses one.

Related: agents.py is the SDK, and currently can't be one — GROQ_API_KEY/GITHUB_TOKEN/gh are import-time globals (:22-24) and workflow is built at import (:907). A library can't be configured by process env, and a multi-tenant server can't hold one gh for all users — same blocker, both directions. AutoMaintainer(config).run(...) + a build_graph() factory makes the SDK the thing our own backend imports. Build order is core package → CLI → web, not web → CLI → SDK.

Cheap now, expensive after launch

1. Routes become permanent contracts the day a CLI pins them. Today: POST /start, POST /stop (singletons, no run_id), /repo/{repo_name:path}/file (keyed by repo, not run or user), unversioned. Move to /v1/, resource-oriented, run-scoped — CLI and SDK both need to address a specific run, which /start//stop can't express. Dashboard should consume the same public API.
2. supabase_schema.sql has no tenancy. runs has no user_id/org_id, and RLS is USING (true) — every user reads every other user's runs and logs. Add orgs + real RLS predicates now, even at one member per org; retrofitting teams touches every query and ships as a disclosure.
3. Don't make Supabase Realtime the public contract. Great browser transport, but as the log API it means shipping anon keys inside the CLI and SDK. Keep Postgres canonical, add GET /v1/runs/{id}/events (SSE) for CLI/SDK. Same rows, two transports.
4. Typed events. broadcast_log writes color into the row (agents.py:33-54). Presentation belongs at render time; SDK consumers want {type: "pr_opened", pr_number: 42}. Typed events are what make §2's HITL checkpoints addressable from CLI at all.
5. Token model alongside the GitHub App — browser session vs. CLI refresh token vs. SDK API key, all hitting /v1/. The automaintainer login flow the README promises implies device-code OAuth; worth designing once.

Verdict: the sandbox/control-plane split is a sound long-run bet and serves all three surfaces. Items 1–4 are days now and quarters later. The Runner abstraction is the one that decides whether the CLI is a client or a second codebase.

One caveat on the local runner: agents would execute on user machines with user credentials against a real working tree — different security and support story than the sandbox. Suggest --dry-run as the CLI default with explicit confirmation before any push.

[purvanshjoshi]

Hi @archittmittal,

Thank you for this critical architectural review. The points raised regarding multi-tenancy, security isolation, and SDK/CLI runtime decoupling are exact and foundational.

1. Control Plane vs. Sandboxed Execution Plane:

   • We fully agree that in-process PTY execution (pty.fork() / os.execv("/bin/bash")) and shared working trees in /tmp cannot support public multi-tenancy.
   • Moving to a decoupled architecture where the FastAPI backend serves as a stateless control plane (handling authentication, routing, and quotas) while dispatching ephemeral sandboxes (via Firecracker microVMs / E2B / Modal) for workspace cloning, LangGraph execution, and PTY bridging is the right architectural model.
   • Migrating from static PATs to GitHub App installation tokens (ephemeral, repository-scoped, 1-hour expiry) resolves both token exfiltration risks and shared API rate limits.

2. Core Package, Runner Abstraction, and Build Order:

   • We agree that the build sequence must be: Core Engine / SDK -> CLI -> Web.
   • Decoupling agents.py from module-level environment globals into an instantiable factory (AutoMaintainer(config).build_graph()) and introducing a unified Runner interface (LocalRunner for local working trees vs. SandboxRunner for remote cloud execution) ensures the CLI and Web interfaces share the same execution logic without code divergence.
   • Setting --dry-run as the local CLI default provides necessary safety for developers running agents against their local working directory.

3. Contract and Tenancy Standardization:

   • Versioned, run-scoped API contracts (/v1/runs/{id}/...).
   • Adding multi-tenant schema isolation (org_id / user_id with active Postgres RLS policies) in Supabase.
   • Adding Server-Sent Events (GET /v1/runs/{id}/events) as the public log transport for the CLI and SDK, while keeping Postgres canonical.
   • Decoupling presentation concerns (color) from the event bus in favor of strongly-typed domain events ({type: "pr_opened", ...}).

These architectural items will be prioritized as foundational prerequisites across Phases 1 and 2.