docs(24_miracle): document and verify research lifecycle - #15
Draft
thugongheng wants to merge 1 commit into
Draft
Annotations
3 errors and 3 warnings
|
Run blocking AI review
Process completed with exit code 1.
|
|
Run blocking AI review:
tools/miracle_iteration.py#L114
The legacy flow runs and writes baseline validation results and copies the rank09 validation opponent before invoking Codex. `workspace-write` limits writes but is not a read-isolation boundary%2C so the coding agent can inspect sibling session files%2C validation artifacts%2C opponent source%2C or the external AgentBench root despite the prompt saying not to. This contaminates the held-out validation set and makes the claim that only training evidence is exposed false. Suggestion%3A Do not materialize or run validation assets until after learning%2C and execute the provider in an OS-level sandbox whose readable filesystem is allowlisted to only `main.py` and approved training evidence. Add a test that attempts reads of validation/session/AgentBench paths and verifies they are denied.
|
|
Run blocking AI review:
tools/miracle_iteration.py#L204
The `--legacy-smoke` dispatch bypasses every approval/preflight boundary and directly starts real Judge/opponent processes plus a Codex provider. An ordinary CLI flag therefore re-enables exactly the runtime execution that this tool and protocol claim remains blocked%2C creating an unreviewed process and scientific-evidence path. Suggestion%3A Remove the executable legacy route from this production preflight tool%2C or require the same independently approved manifests and runtime authorization as the normal lifecycle before any process or output directory is created. Preserve historical code only as inert documentation or in a separately quarantined tool that cannot be mistaken for an approved run path.
|
|
Complete job
Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/checkout@v4. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/
|
|
Run blocking AI review:
tests/miracle/test_results_pipeline.py#L36
The Results integration test silently substitutes the vendored historical aggregate whenever the configured Results checkout lacks `aggregate.py`%2C while still using that checkout's `report_builder.py`. A partial or incompatible Results checkout can therefore pass against a component combination that does not exist in Results%2C contrary to the stated exact cross-repository integration boundary. Suggestion%3A Require both aggregate and report builder from the same configured Results checkout for integration tests and skip or fail with an explicit dependency reason if either is absent. Test the vendored aggregate separately as historical compatibility evidence.
|
|
Run blocking AI review:
tools/miracle_iteration.py#L28
The directory digest is ambiguous because it hashes each relative path immediately followed by its contents without lengths or separators. For example%2C a file named `a` containing `bc` hashes identically to a file named `ab` containing `c`%2C so the recorded source%2C workspace%2C and opponent SHA-256 values do not uniquely bind the asset tree. It also follows symlinks%2C which can unintentionally include files outside the declared root. Suggestion%3A Reject symlinks and hash a canonical%2C length-delimited manifest containing each normalized relative path%2C file type%2C size%2C and per-file SHA-256%2C then hash that manifest. Reuse the framework's validated canonical asset-digest implementation if one exists.
|
background
wait
wait-all
cancel
parallel
Loading