Skip to content

ci: AGENTS.md and the pull-request gates - #14

Open
thesprockee wants to merge 2 commits into
mainfrom
agents-and-gates
Open

thesprockee wants to merge 2 commits into
mainfrom
agents-and-gates

Conversation

@thesprockee

@thesprockee thesprockee commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

PR 2 of 3 of Spritz's order (after #8). Closes #11.

What

  • AGENTS.md: what the repo is, logging, conventional commits, the credit rule, review, issues/branches/PRs, workflows (pins, trust boundary), entries, commands, never.
  • scripts/gates.py with four gates, each printing one JSON record (event, at, ok, errors, what was checked) and exiting 0/1, or 2 with a cannot_run record when it can't run:
    • title: the PR title is a Conventional Commits header (adapted from nevr-runtime's tools/check_pr_title.py).
    • commits: every commit in base..head is conventional, and each Co-Authored-By is Name <email> with a GitHub noreply address or a@sprock.io/agents@sprock.io (the credit rule).
    • review: a PR that changes anything but README/docs has a current approval (on the head commit) from an owner/member/collaborator other than its author, and no open change request (the design's review gate).
    • pins: every uses: is pinned by a 40-hex commit. check-submission.yml's five actions are pinned in this PR.
  • .github/workflows/gates.yml: runs them on pull_request (and review also on pull_request_review), read-only token, no secrets, records kept 90 days as artifacts.
  • tests/test_gates.py: 24 tests, a passing and a failing case per gate.

The history before this is free text ("Form: ..."); the gates only look at new PRs.

Not in this PR

The gates are advisory until the ruleset below exists. AGENTS.md mentions the commands for the validator from #8, so merge #8 first.

Note for Mia: making the gates required (owner setting)

The gates only report until you add a ruleset; there is none today. So review stays red on a PR until someone other than its author approves, and nothing stops a merge meanwhile. To make them bind, in Settings → Rules → Rulesets → New branch ruleset, target main (default branch):

  • Require a pull request before merging: 1 required approval; "Dismiss stale pull request approvals when new commits are pushed".
  • Require status checks to pass, with these checks (the names as they show on a PR): title, commits, review, pins, tests. Add validate once feat: entry schema, validator and CI check for listed mods #8 is merged and package-layout-tests once feat: the submission check reads the plugin-release layout #17 is, if you want those required too.
  • Block force pushes and restrict deletions (the defaults).
    Owners can be left as bypass actors if you want to merge your own PRs; the review gate is red for a PR whose only approver is its author, whatever the ruleset says.

Commands run

$ python3 -I -m unittest discover -s tests -k test_gates
Ran 24 tests
OK

Mutants (one line changed at a time, tests run, file restored from a saved copy):

M1 unpin actions/checkout in check-submission.yml      -> FAIL test_the_repos_workflows_are_pinned
M2 own approval counts                                  -> FAIL test_own_approval_does_not_count
M3 stale approval (other commit) counts                 -> FAIL test_stale_approval_does_not_count
M4 any Co-Authored-By address accepted                  -> FAIL test_credit_must_be_a_noreply_address
M5 title/commit header check removed                    -> ERROR test_other_titles_fail, test_free_text_subject_fails
M6 CHANGES_REQUESTED ignored                            -> FAIL test_changes_requested_blocks_even_with_another_approval
restored tree: Ran 23 tests ... OK

Real records from this branch (and the review gate against the open #8, which has no approval yet):

$ python3 -I scripts/gates.py title --title "ci: AGENTS.md and the pull-request gates"
{"at": "2026-10-26T12:29:18Z", "errors": [], "event": "gate.title", "ok": true, "title": "ci: AGENTS.md and the pull-request gates"}
$ python3 -I scripts/gates.py commits --base origin/main --head HEAD
{"base": "origin/main", "commits": 1, "errors": [], "event": "gate.commits", "head": "HEAD", "ok": true}
$ python3 -I scripts/gates.py review --repo EchoTools/echo-mods --pr 8
{"author": "thesprockee", "errors": ["no approval of baf9a763c336 from an owner, member or collaborator other than thesprockee (changed: ...)"], "event": "gate.review", "files": 8, "ok": false, "pr": 8, ...}
$ python3 -I scripts/gates.py pins
{"errors": [], "event": "gate.pins", "files": [".github/workflows/check-submission.yml", ".github/workflows/gates.yml"], "ok": true}

The review job on this PR is expected to be red until someone other than the author approves it.

Found and fixed on this PR (0770bf9)

The first push ran the gates as gates.py ... | tee; GitHub's default bash has no pipefail, so review printed "ok": false and the check was green (run 40322956555). Fixed with defaults: run: shell: bash; test_a_piped_gate_still_fails_the_job reads the workflow for it (mutant: remove the default -> that test fails; 24 tests OK restored). On the new tip review is red as intended (run 40323161570); title, commits, pins, tests and CodeQL pass.

Co-Authored-By: Andrew Bates a@sprock.io

AGENTS.md sets the repo's rules (logging, conventional commits, the credit rule, review, workflows,
entries). scripts/gates.py enforces what a machine can: a conventional PR title and commit headers,
well-formed Co-Authored-By trailers, a current approval from someone other than the author on
anything but docs, and every action pinned by commit. Each gate prints a JSON record and fails loudly;
the existing submission workflow's actions are pinned. The header check is adapted from
nevr-runtime's tools/check_pr_title.py.

Closes #11

Co-Authored-By: Andrew Bates <a@sprock.io>
The gates ran as `gates.py ... | tee`, and GitHub's default bash has no pipefail, so the job took
tee's exit code: the review gate printed ok:false and the check was green. Run every step with
`shell: bash` (-eo pipefail), and test that the workflow says so.

Refs #11

Co-Authored-By: Andrew Bates <a@sprock.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AGENTS.md and CI gates for the catalog repo

1 participant