Skip to content

test(e2e): add recent-release upgrade qualification - #14335

Merged
rsliter merged 2 commits into
mainfrom
codex/14268-recent-upgrade
Nov 16, 2026
Merged

rsliter merged 2 commits into
mainfrom
codex/14268-recent-upgrade

Conversation

@rsliter

@rsliter rsliter commented Nov 9, 2026 •

Copy link
Copy Markdown
Collaborator

Outcome

The upgrade catalogue gains a pinned v0.0.141 fixture with one running and one stopped sandbox. It reuses the existing scenario, which requires workspace preservation, authenticated agent turns, usable gateway access, and terminal cleanup. Live qualification remains pending.

Reason

Issue #14268 requires recent-release upgrade evidence in addition to historical component-replacement migrations. The new fixture covers reuse of the compatible OpenShell 0.0.116 runtime. The v0.0.89 and v0.0.123 fixtures retain their distinct migration coverage.

Related issues

Refs #14268. Builds on merged #14276 for the destination and workspace-marker repairs. ACP prefix migration is already merged through #12830.

Changes

  • Pin the v0.0.141 commit, bootstrap checksum, OpenShell version, and immutable OpenClaw 2026.9.2 archive identity.
  • Register one target in the existing typed catalogue. Reuse its fixture validation, installer adapter, live scenario, stopped-state checks, and cleanup.
  • Use the existing native agent RPC for the recent fixture's legacy phase. Its convenience command requests administrator scope; the native RPC retains the ordinary write scope and requires successful responses and authenticated provider traffic.
  • Extend support cases for release identity, bootstrap checksum, archive parity, and planner selection. Preserve historical fixtures and live assertions.
  • Record dependency and trusted-catalogue sequencing in the owning E2E README. Consolidate one comment to keep the planner test at its unchanged 1,500-line limit.

Verification

Candidate: 3a7f9b223704e6a3026327053e57e33294895fb6. The branch is synchronized with canonical main 2822077e58a57d160b567886a939fefe5ecacef7, which contains #14276. The two mechanical conflicts retain main's private executable preparation and current catalogue inventory while adding the recent fixture.

  • node_modules/.bin/vitest run --project e2e-support test/e2e/support/openshell-gateway-upgrade-old-installer.test.ts test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts test/e2e/support/workflow-plan.test.ts --maxWorkers=1: 212 passed across three files in a credential-isolated clone. Canonical main was reverified immediately before execution.

  • git diff --check --cached passed. The merged files match the independently reviewed seven-file proposal. No live assertion, timeout, retry policy or assertion budget changed.

  • Installed pre-commit and commit-message hooks passed inside the verified credential-isolation boundary. The signed merge commit includes DCO. The actual installed pre-push hook passed publication validation and plugin, JavaScript-config and CLI type checks inside the same boundary. GitHub verifies both published commits.

  • Independent review of the seven-file merge proposal found no actionable defect. The v0.0.141 release commit and bootstrap checksum, OpenClaw archive integrity and native RPC compatibility were verified. Historical fixtures, credential custody, successful authenticated responses, MCP cleanup, workspace checks and stopped-sandbox restoration remain intact.

  • Diff review found no secrets, API keys, credentials or unrelated candidate changes.

  • Code CI passed, including all 12 CLI shards and their aggregate.

  • All nine specialist artifacts in Advisor run 43311296576 were verified against this candidate, base, trusted workflow and v26 plan. Eight finding ledgers are clear. The single builder refactor was declined as optional after source review and independent review found no incorrect metadata or behavior. Complete disposition.

  • Focused E2E run 43313902305 passed: cloud-onboard, full-e2e, both security-posture variants and both trusted openshell-gateway-upgrade historical rows. All six source-bound archives, required assertions, phases and registered cleanup callbacks were verified. The relevant-E2E aggregate passed; full release qualification was not selected.

  • The new v0.0.141 live qualification remains a post-catalogue-merge step. Pre-merge historical results do not qualify that new row.

Review notes

Sensitive paths: tools/e2e/openshell-gateway-upgrade-fixture.mts and tools/e2e/target-catalogue.mts in NVIDIA/NemoClaw. An independent subagent reviewed all seven files against canonical main 2822077e58a57d160b567886a939fefe5ecacef7, including release/bootstrap/archive pins, native RPC compatibility and preservation of the historical fixture and cleanup contracts. It found no actionable issue. This review is pre-publication evidence; it does not replace CI or eligible maintainer review.

Validation runs in a private clone with private dependencies and cache. Host credential files, credential helpers, secret environment variables, and outside sockets are blocked. Installed packages and hook files are read-only; generated source-cache writes are permitted only inside the private clone. Git signing and authenticated transport remain outside this boundary. No validation policy is waived.

Manual PR selection reads the catalogue from trusted main. The new selector openshell-gateway-upgrade-v0-0-141-x86-64 becomes selectable after catalogue admission. Its first live qualification follows that admission; keep #14268 open until qualification succeeds. #14276 is merged and its destination/workspace-marker fixes are included. No planner override is used. The current Advisor aggregate remains failed for its optional builder finding and seven unresolved requests for the same gateway qualification boundary. The source and independent-review disposition is linked above. The manual-only gateway job is supported; the pre-merge run uses the trusted v0.0.89/v0.0.123 rows. The new v0.0.141 row follows catalogue admission and its first trusted live qualification. No planner override or status-check waiver is used.


Signed-off-by: Rebecca Sliter 571084+rsliter@users.noreply.github.com

Summary by CodeRabbit

  • Tests

    • Added gateway-upgrade coverage for release v0.0.141, including running and stopped sandboxes and native agent communication checks.
    • Expanded validation of installer integrity, upgrade configuration, and zero-advisory results across reviewed releases.
    • Added the new release to automated workflow planning and target coverage.
  • Documentation

    • Updated the historical upgrade-fixture guide with v0.0.141 compatibility details and prerequisites.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@rsliter rsliter self-assigned this Nov 9, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 9, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The change adds a reviewed v0.0.141 gateway-upgrade fixture and catalogue target. It updates live upgrade checks to use native agent RPC for the matching legacy version and extends supporting release, workflow, and documentation checks.

Changes: Gateway upgrade coverage

Layer / File(s) Summary
Fixture and target registration
tools/e2e/openshell-gateway-upgrade-fixture.mts, tools/e2e/target-catalogue.mts, test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts, test/e2e/support/workflow-plan.test.ts, test/e2e/README.md
Adds the reviewed v0.0.141 fixture and Docker-only x86-64 target. Updates catalogue and workflow-plan expectations, boundary checks, and fixture documentation.
Live upgrade and release checks
test/e2e/live/openshell-gateway-upgrade.test.ts, test/e2e/support/openshell-gateway-upgrade-old-installer.test.ts
Selects native agent RPC for the upgraded phase or when the legacy OpenClaw version matches the recent fixture. Checks the reviewed installer hash and zero-advisory boundary for the registration and recent fixtures.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Possibly related issues

  • NVIDIA/NemoClaw issue 14268 — Adds the requested recent-version-to-candidate live upgrade scenario with running and stopped sandboxes. The PR states that the issue remains open until the new live qualification succeeds and does not fix the reported ACP shim or destination installer failures.

Suggested reviewers: cjagwani, ericksoa, cv

Merge Risk: Minimal · up to 3a7f9

This change adds a pinned v0.0.141 gateway-upgrade fixture and an x86-64 E2E target, and routes the matching legacy phase through native agent RPC. Production code is not affected. The first live qualification of the new row will happen after catalogue admission, so the maintainers' plan to keep the tracking issue open until it passes is appropriate.

Security architecture risk: Low · up to 3a7f9

Risk is limited to upgrade qualification on test runners. Existing release pins, credential checks, and workflow authorization gates remain in place, but the new legacy RPC path still requires live qualification.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • inferred — The added execution exposes the selected test runner, its Docker gateway and managed volumes, and one running plus one stopped fixture sandbox to the historical installer and agent turn. These are test resources; the change does not establish a new production-facing entrypoint. Runner credentials remain relevant because execution inherits the existing workflow environment.

Trust Boundaries and Controls

  • observed — Selector lookup resolves an existing catalogue entry and rejects unknown identifiers; changed-file selection filters catalogue entries rather than accepting arbitrary fixture environments. The new target does not change the workflow gates controlling GitHub-token and Docker Hub credential delivery.
  • observed — Both qualification phases require the fixture credential to be absent from the sandbox environment and managed OpenClaw files. Agent success requires successful execution, expected response text, newly recorded provider requests, and authentication on every selected request. The recent legacy native-RPC branch retains credential redaction and additionally requires a successful RPC response.
Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "test(e2e): add recent-release upgrade qualification" matches the main change. The PR adds a pinned v0.0.141 recent-release fixture to the OpenShell gateway-upgrade E2E catalogue, with a new…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "test(e2e): add recent-release upgrade qualification" matches the main change. The PR adds a pinned v0.0.141 recent-release fixture to the OpenShell gateway-upgrade E2E catalogue, with a new target, live scenario wiring, and support tests. The title is short, specific, and follows a conventional commit style. It does not describe the separate pending live qualification result, which the PR body says has not yet run successfully, but a title does not need to cover that.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

markdownlint-cli2 (0.24.0)

test/e2e/README.md

markdownlint-cli2 did not write JSON output
file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:4776
if (!t.more()) throw new RE2JSSyntaxException(Parser.ERR_MISSING_BRACKET, t.from(wholeClassPos));
^

RE2JSSyntaxException: error parsing regexp: missing closing ]: [^]*?\n---\n?
at Parser.parseClassChar (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:4776:24)
at Parser.parseClass (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:5579:22)
at Parser.parseInternal (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:5258:11)
at Parser.parse (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:4646:37)
at RE2.compileImpl (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:5656:19)
at RE2JS.compile (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/node_modules/re2js/build/index.js:6470:20)
at lintInput (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/lib/markdownlint.mjs:812:49)
at lintAsync (file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/lib/markdownlint.mjs:916:10)
at file:///usr/local/lib/node_modules/markdownlint-cli2/node_modules/markdownlint/lib/markdownlint.mjs:927:5
at new Promise () {
error: 'missing closing ]',
input: '[^]*?\n---\n?'
}

Node.js v24.22.3

markdownlint-cli2 v0.24.0 (markdownlint v0.42.0)
Finding: :test/e2e/README.md !node_modules/** !skills/**
Linting: 1 file

Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 2822077 and 3a7f9b2.

Files selected for processing: 7

  • test/e2e/README.md
  • test/e2e/live/openshell-gateway-upgrade.test.ts
  • test/e2e/support/openshell-gateway-upgrade-old-installer.test.ts
  • test/e2e/support/openshell-gateway-upgrade-workflow-boundary.test.ts
  • test/e2e/support/workflow-plan.test.ts
  • tools/e2e/openshell-gateway-upgrade-fixture.mts
  • tools/e2e/target-catalogue.mts

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

@rsliter

rsliter commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author
Authoring session — opening
Field Status Value Evidence or limit
Harness recorded Codex Desktop; CLI runtime 0.166.0-alpha.6 Persisted authoring metadata identifies the active subagent. Recorded repository, branch, and command activity match this change.
Skills partial nemoclaw-maintainer-audit-e2e-assertions; nemoclaw-maintainer-e2e; nemoclaw-contributor-implement-issue; nemoclaw-contributor-create-pr Complete reads appear in the authoring record. A combined CI-classification guidance read was truncated and is excluded. Skill catalogue entries are excluded.
Instruction files partial AGENTS.md; CONTRIBUTING.md; WRITING.md; test/README.md; .agents/references/e2e-authoring.md; .agents/skills/_shared/documentation-writing-review.md; .github/PULL_REQUEST_TEMPLATE.md; .github/SESSION_REPORT_TEMPLATE.md Complete reads or injections are recorded. Partial E2E README reads and truncated combined guidance reads are excluded.
Tool calls recorded Wrapper layer: exec 99, wait 30, send_message 25. Separately, observable nested layer: exec_command 118. Attempted invocations include failures. Inherited parent calls are excluded. Shell subprocesses are not counted as tool invocations. Layers are not added together.
Session window recorded 2026-11-06T14:24:29.741Z to 2026-11-09T13:43:57.300Z; 71 h 19 min 27.559 s Whole active subagent session through the source handoff, including idle time and intervening ACP review and validation work. These are not recent-fixture-only totals. Reporting work after cutoff is excluded.
Compactions recorded 1 Persisted record through cutoff, including the compaction event and retained earlier events.
Models recorded gpt-6.1-sol Recorded turn contexts; no model changes.
Reasoning levels recorded medium Recorded turn contexts; no effort changes.

The active authoring scope started with the upgrade-contract task after inherited parent history.
It includes the original November 6 fixture work and resumed November 9 work on codex/14268-recent-upgrade.
The resumed patch is based on canonical main 4d5d613f73b96d51a92e7ae88df5da5103836067.
The report uses the authoring record, not a later publication session.
A later comment-only line-budget repair is outside this cutoff.

@github-code-quality

github-code-quality Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: TypeScript

TypeScript / code-coverage/plugin

main 63002cd codex/14268-recent-u... 3a7f9b2 +/-
96% 97% +1%
View coverage of the 10 most impacted files.
File main 63002cd codex/14268-recent-u... 3a7f9b2 +/-
nemoclaw/src/sh...cy-boundary.cts 100% 97% -3%
nemoclaw/src/onboard/config.ts 98% 96% -2%
nemoclaw/src/ru...time-context.ts 100% 99% -1%
nemoclaw/src/bl...print/runner.ts 95% 95% 0%
nemoclaw/src/bl...ime-identity.ts 97% 97% 0%
nemoclaw/src/bl...t-management.ts 100% 100% 0%
nemoclaw/src/index.ts 94% 99% +5%
nemoclaw/src/de...search-state.ts 0% 100% +100%
nemoclaw/src/on...native-route.ts 0% 100% +100%
nemoclaw/src/search-config.ts 0% 100% +100%

TypeScript / code-coverage/cli

main 63002cd codex/14268-recent-u... 3a7f9b2 +/-
84% 86% +2%
View coverage of the 10 most impacted files.
File main 63002cd codex/14268-recent-u... 3a7f9b2 +/-
src/lib/onboard.ts 62% 50% -12%
src/lib/actions...ess-recovery.ts 65% 83% +18%
src/lib/inferen...ollama/proxy.ts 41% 61% +20%
src/lib/onboard.../application.ts 55% 84% +29%
src/lib/domain/...y/dimensions.ts 0% 75% +75%
src/lib/inferen...er/lifecycle.ts 0% 75% +75%
src/lib/actions...ry/operation.ts 0% 76% +76%
src/lib/actions...try/snapshot.ts 0% 84% +84%
src/lib/inferen...ovider/index.ts 0% 90% +90%
src/lib/hermes-...yard-routing.ts 0% 95% +95%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.


Updated November 12, 2026 21:32 UTC

@rsliter

rsliter commented Nov 9, 2026

Copy link
Copy Markdown
Collaborator Author

Complete review for 5dcb30090d984355457b92c46c89550f2d3d3300, base 4d5d613f73b96d51a92e7ae88df5da5103836067: code CI passed. All nine specialists in Advisor run 42636246568 succeeded, all nine finding ledgers are clear, and every required review artifact is present and bound to this commit/base.

The Advisor aggregate fails only for one operability E2E request that treats a manual-only job as unsupported. The captured inventory includes openshell-gateway-upgrade; migration and security recommend that same supported job explicitly. I deduplicate this as required live qualification, not a code or planner defect. The deterministic floor additionally selects cloud-onboard, full-e2e, and security-posture for the changed E2E evidence paths.

The new v0.0.141 target is intentionally unavailable in trusted-main selection until catalogue admission. PR #14276 also remains a prerequisite. I did not add a new test, override the trusted planner, request CodeRabbit, retry a policy rejection, or change code to satisfy the unsupported-selector claim. No additional revision cycle was consumed.

Live qualification and eligible human review remain outstanding. This comment records the disposition; it does not convert the failed Advisor aggregate into a passing check or claim successful runtime evidence.

@wscurran wscurran added area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure area: security Security controls, permissions, secrets, or hardening integration: openclaw OpenClaw integration behavior labels Nov 12, 2026
pasogott pushed a commit to pasogott/NemoClaw that referenced this pull request Nov 12, 2026
…nt (NVIDIA#14276)

Prepare and revalidate a safe OpenShell replacement destination before retiring the old gateway. Preserve the selected executable handoff, historical fixture bytes, and upgrade-state assertions.

Refs NVIDIA#14268. Recent-release qualification remains in NVIDIA#14335.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
Consume the merged OpenShell destination and workspace-marker fixes.
Retain main executable preparation and planner inventory while adding the
pinned v0.0.141 running and stopped sandbox upgrade fixture.

Signed-off-by: Rebecca Sliter <571084+rsliter@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Review Advisor finished for commit 3a7f9b2. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

Request review only when Require no Advisor blockers is green.

All previous runs

@rsliter

rsliter commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review disposition for 3a7f9b223704e6a3026327053e57e33294895fb6, base 2822077e58a57d160b567886a939fefe5ecacef7. Code CI passed. All nine specialist artifacts in Advisor run 43311296576 were collected and verified against the candidate, base, trusted workflow and v26 plan 09c9e249ac119a4c16616c1c93b8e25b5bdd8cf2a8a9056bac439b76d8382c85.

Eight finding ledgers are clear. I decline reduction finding F-reduction-simplification-52af6cb0d2c341056a84 as optional refactoring. It proposes a builder for three catalogue declarations but identifies no incorrect metadata or runtime behavior. Source review and an independent review confirmed the correct fixture identities, pins, runner, profile, live owner, environment and intentional 70/120/120-minute budgets. Existing fixture-pin tests and exact-target validation remain intact. Refactoring the two historical declarations is unnecessary for this qualification change.

Seven unresolved E2E requests identify the same accepted recent-upgrade boundary. The existing manual-only openshell-gateway-upgrade job is supported: the trusted receipt validator accepts manual-only job IDs. E2E run 43313902305 is running cloud-onboard, full-e2e, security-posture, and openshell-gateway-upgrade for this commit, through trusted workflow f1865746e22b1eeec6e25b9a0d27af831c57f9db. Trusted main currently contains only the v0.0.89/v0.0.123 upgrade rows. The new v0.0.141 row follows the recorded catalogue-merge then first-live-qualification sequence. Keep #14268 open until that qualification succeeds.

The Advisor aggregate remains failed for its reported finding and unresolved recommendations. This disposition does not claim a passing aggregate or completed live qualification. No additional code revision was made.

Independent security review of all seven changed files on this commit passed all nine repository rubric categories. It verified immutable archive/bootstrap pins, server-enforced ordinary operator.write RPC authorization, credential custody and redaction, validated and quoted inputs, failure propagation, and retained cleanup controls. It found no vulnerability or required repair and retains the same post-merge v0.0.141 runtime limit.

@rsliter

rsliter commented Nov 12, 2026

Copy link
Copy Markdown
Collaborator Author

E2E evidence for 3a7f9b223704e6a3026327053e57e33294895fb6, PR base 2822077e58a57d160b567886a939fefe5ecacef7, trusted workflow f1865746e22b1eeec6e25b9a0d27af831c57f9db.

E2E run 43313902305, attempt 1: SUCCESS, including Relevant E2E. All six selected live executions passed.

Selector Result Execution evidence
cloud-onboard PASS Cloud onboarding
full-e2e PASS Full OpenClaw
security-posture PASS OpenClaw, Hermes
openshell-gateway-upgrade PASS for the trusted historical rows v0.0.89, v0.0.123

All six execution archives were hash-verified against GitHub metadata and bound to this source commit and run. Required assertion outcomes, phases and registered cleanup callbacks passed. The v0.0.123 case restored both additional stopped sandboxes, verified their workspace markers and returned them to stopped state. Cleanup verification covers each fixture’s named resources and registered callbacks.

The global dispatch input was mock; cloud/full/security retain their configured NVIDIA credential profiles, and the historical upgrades use authenticated host-local fixture inference. The runtime images come from the reviewed 2822077 publication; tested CLI and E2E source is 3a7f9b2. This focused run is not full release qualification.

The new v0.0.141 row is absent from trusted pre-merge selection. Its first live qualification follows catalogue admission under the recorded sequence. These historical passes do not qualify v0.0.141. Keep #14268 open until the new case passes.

@rsliter
rsliter marked this pull request as ready for review November 12, 2026 22:38

@prekshivyas prekshivyas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit 3a7f9b2. No actionable code findings. The new v0.0.141 fixture still requires live qualification after catalogue admission; historical E2E passes do not qualify it.

@wscurran wscurran added the chore Build, CI, dependency, or tooling maintenance label Nov 13, 2026
@rsliter
rsliter merged commit 07d67de into main Nov 16, 2026
134 of 135 checks passed
@rsliter
rsliter deleted the codex/14268-recent-upgrade branch November 16, 2026 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci CI workflows, checks, release automation, or GitHub Actions area: e2e End-to-end tests, nightly failures, or validation infrastructure area: security Security controls, permissions, secrets, or hardening chore Build, CI, dependency, or tooling maintenance integration: openclaw OpenClaw integration behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants