ci(cd): opt-in re-run-safe freshness guard on the apply steps#27
Open
rplusq wants to merge 3 commits into
Open
ci(cd): opt-in re-run-safe freshness guard on the apply steps#27rplusq wants to merge 3 commits into
rplusq wants to merge 3 commits into
Conversation
Adds an opt-in `enforce-fresh` compare-and-set guard as a STEP inside the
side-effecting jobs (deploy-app's ECS deploy, deploy-infra's terraform apply),
not as an upstream gate job — so it re-evaluates against live state on ANY re-run
("re-run failed jobs" / "re-run this job"), making a deploy idempotent and unable
to roll the environment back from a superseded/stale run. The step reads the live
version (app: /health banner via stage-url; infra: deployed_infra_commit SSM
marker via a caller-supplied app-account role) and skips the apply (green no-op)
unless that commit is an ancestor-or-equal of the target — fail-open on an
unreadable version, fail-closed on a live commit unresolvable in git history,
`allow-stale` to override. Default false → zero behaviour change for consumers
that don't opt in. cd.yml threads the inputs through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…able, allow-stale bypass - Missing freshness-module-id when enforce-fresh now FAILS CLOSED (was a silent fail-open that didn't protect infra). - allow-stale fully bypasses the guard (steps gated `enforce-fresh && !allow-stale`) so a deliberate rollback can't be wedged by the guard's own machinery. - Portable extraction (grep|awk, not GNU sed `T`) + explicit `shell: bash`. - App target that isn't a git commit → warn + fail-open (skip=false), not a hard block; blind live-version reads emit `::warning` so an inert guard is visible. - continue-on-error on the infra freshness-creds step so a transient STS blip fails the guard OPEN rather than wedging the deploy. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The grep|awk parser exits 1 when /health is unreachable or lacks the `commit hash:` token (common when /health flaps during an ECS task cycle); under set -euo pipefail that aborted the step before the fail-open branch, turning warn-and-proceed into a red deploy. Add `|| true` so an unreadable banner proceeds UNGUARDED with a warning, as intended. (Both re-review passes flagged this as the sole remaining HIGH.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds an opt-in, re-run-safe freshness guard to the shared deploy workflows. When a caller sets
enforce-fresh: true, the ECS deploy (deploy-app.yml) and theterraform apply(deploy-infra.yml) each re-read the live version immediately before applying and skip (green no-op) if it is not an ancestor-or-equal of the commit this run intends to ship — i.e. a newer/divergent version is already live, so applying would roll it back.Default
enforce-fresh: false→ zero behaviour change for every current consumer. Only a caller that opts in (and passes the params below) gets the guard.Why
For pay-core's new content-addressed CD, the freshness decision was a separate up-front job whose
stale=falseoutput is cached across GitHub re-runs ("re-run failed jobs" / "re-run this job") — so a stale/superseded run could re-apply an older commit and roll prod back, and no operator-discipline note ("don't click re-run") is an acceptable guarantee. The fix is idempotency-by-construction: a compare-and-set STEP inside the side-effecting job re-evaluates on any path that reaches the apply.How
cd.ymlthreadsenforce-fresh,freshness-module-id,freshness-allow-stale→deploy-app.yml/deploy-infra.yml(app target =versionviastage-url/health; infra target =github.shavia thedeployed_infra_commitSSM marker, read with a caller-supplied app-account role)./healthcommit hash:banner; infra guard: reads the SSM marker (placed before the monitoring-creds step so Grafana still gets monitoring creds).version(loud::warning, never a false block); fail-CLOSED on a live commit unresolvable in git history (possible divergent deploy) and on a missingfreshness-module-idwhen enforcing;freshness-allow-stalefully bypasses the guard for a deliberate rollback.grep|awk, explicitshell: bash);continue-on-erroron the infra freshness-creds step so a transient STS blip fails open rather than wedging a deploy.Testing
Not exercised end-to-end (no Actions/AWS runtime here); YAML parses, the fail-open/portable bash was unit-checked under
set -euo pipefail. Reviewed by two independent passes (a Claude review + a Codex review), each run twice — final round: no findings beyond the one fail-open fix now included.Consumer
WalletConnect/pay-corePR #1095 opts in (cd-staging/cd-prod); it is blocked on this PR merging intofix/dont_block_on_trivyfirst (pay-core pins that branch, so it picks up the new inputs automatically once merged).🤖 Generated with Claude Code