Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .github/workflows/release-ghcr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,12 @@ on:
image_group:
type: string
default: all
reuse_publication_artifacts:
type: boolean
default: false

permissions:
actions: read
contents: write
packages: write
id-token: write
Expand Down Expand Up @@ -132,6 +136,15 @@ jobs:
raise SystemExit('Release registry credential preflight failed')
print('Docker Hub release credential preflight passed')
PY
- name: Download this run's qualified Docker Hub evidence
if: ${{ inputs.reuse_publication_artifacts }}
shell: bash
env:
VERSION: ${{ steps.source.outputs.version }}
run: |
set -euo pipefail
gh run download "$GITHUB_RUN_ID" --repo "$GITHUB_REPOSITORY" \
--pattern "release-images-*-$VERSION" --dir target/release/source-proofs
- name: Qualify and mirror the shared release catalog
shell: bash
env:
Expand All @@ -140,10 +153,12 @@ jobs:
NAMESPACE: ${{ steps.source.outputs.namespace }}
GROUP: ${{ inputs.image_group }}
DRY_RUN: ${{ inputs.dry_run }}
REUSE_PUBLICATION_ARTIFACTS: ${{ inputs.reuse_publication_artifacts || false }}
run: |
set -euo pipefail
args=()
if [[ "$DRY_RUN" == false ]]; then args+=(--publish); fi
if [[ "$REUSE_PUBLICATION_ARTIFACTS" == true ]]; then args+=(--source-evidence target/release/source-proofs); fi
python scripts/publish_ghcr_images.py --group "$GROUP" --version "$VERSION" \
--source-commit "$SOURCE_COMMIT" --namespace "$NAMESPACE" --output target/release/ghcr "${args[@]}"
echo "GHCR group \`$GROUP\` passed; dry-run: \`$DRY_RUN\`. Digests match Docker Hub." >> "$GITHUB_STEP_SUMMARY"
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -373,6 +373,7 @@ jobs:
needs: [prepare, images]
if: ${{ always() && needs.prepare.result == 'success' && needs.images.result == 'success' && inputs.publish_images && !inputs.dry_run }}
permissions:
actions: read
contents: write
packages: write
id-token: write
Expand All @@ -381,4 +382,5 @@ jobs:
tag: ${{ inputs.tag }}
image_group: ${{ inputs.image_group }}
dry_run: false
reuse_publication_artifacts: true
secrets: inherit
21 changes: 14 additions & 7 deletions rocketmq-doc/en/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,14 @@ generates CycloneDX SBOMs; Trivy blocks every CRITICAL finding, including those
without a fix. New images are pushed to per-run staging tags, scanned again by
registry digest, signed with Cosign, and accompanied by verified SBOM
attestations. Only then is the stable version tag promoted while
preserving the digest. Newly uploaded signatures and attestations may take time
to become discoverable. Temporary discovery failures are retried up to nine times
with bounded backoff (at most 375 seconds of waiting per verification).
Authorization, certificate, issuer, and cryptographic failures stop immediately;
tags are promoted only after successful verification.
preserving the digest. Cosign verifies the exact uploaded Sigstore bundles locally
against the image digest, release workflow certificate identity, issuer, source and
component annotations, and complete scanned SBOM. This avoids waiting for registry
indexing or downloading every historical signature again. Authorization, certificate,
issuer, and cryptographic failures stop immediately; tags are promoted only after
successful verification. Registry uploads retry throttling and server failures with
bounded backoff. Registry clients use the authenticated `registry-1.docker.io` host;
the public image names remain unchanged.
The publisher deletes its temporary tag after promotion and on handled publication
failures. An `always()` workflow step also removes only the current run's staging
tags, including failed or cancelled runs when the runner remains available.
Expand All @@ -132,8 +135,12 @@ Before copying any selected image, the mirror verifies the immutable tag/source
and OCI labels, the Docker Hub keyless signature with its release workflow
identity and source/component annotations, and the signed CycloneDX SBOM's exact
repository/digest subject. It accepts in-toto Statement v0.1 and v1 and rejects
unknown versions. All selected images receive fresh SBOMs and Trivy scans with
zero CRITICAL findings required before the first copy.
unknown versions. The automatic mirror downloads evidence produced by successful
Docker Hub jobs in the same Actions run. It re-verifies both signed bundles and
reuses that run's fresh SBOM and zero-CRITICAL Trivy report after checking the exact
image subject. Missing, incomplete, conflicting or invalid evidence blocks every
copy. Independent GHCR dispatches verify registry evidence and generate fresh SBOMs
and Trivy scans, with zero CRITICAL findings required before the first copy.

The complete publication record binds the image digests and hashes of the SBOMs,
scans, and source proofs. Cosign signs this record as a blob; evidence is uploaded
Expand Down
65 changes: 43 additions & 22 deletions scripts/publish_dockerhub_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -196,36 +196,57 @@ def qualify(reference: str, prefix: Path, *, remote: bool) -> Path:
return sbom


def verify_bundle(bundle: Path, digest: str, predicate_type: str) -> tuple[dict, dict]:
# Verify the exact uploaded bundle, including certificate, transparency proof and subject
# digest. Registry discovery otherwise downloads every historical bundle for this image.
run(["cosign", "verify-blob-attestation", "--bundle", str(bundle), "--type", predicate_type,
"--certificate-identity-regexp", IDENTITY, "--certificate-oidc-issuer", ISSUER,
"--check-claims=true", "--digest", digest.removeprefix("sha256:"), "--digestAlg", "sha256"])
envelope = json.loads(bundle.read_text(encoding="utf-8"))["dsseEnvelope"]
encoded = envelope["payload"]
statement = json.loads(base64.b64decode(encoded + "=" * (-len(encoded) % 4), validate=True))
subjects = statement.get("subject", [])
if (statement.get("_type") not in {"https://in-toto.io/Statement/v0.1", "https://in-toto.io/Statement/v1"}
or statement.get("predicateType") != predicate_type or len(subjects) != 1
or subjects[0].get("digest") != {"sha256": digest.removeprefix("sha256:")}):
raise ReleaseError("verified bundle does not bind the exact release digest and predicate type")
return statement, envelope


def verify_release_bundles(reference: str, sbom: Path, commit: str, name: str, prefix: Path) -> tuple[str, str]:
repository, digest = reference.split("@", 1)
signature, _ = verify_bundle(prefix.with_suffix(".signature-bundle.json"), digest,
"https://sigstore.dev/cosign/sign/v1")
annotations = signature["subject"][0].get("annotations", {})
if annotations.get("source_commit") != commit or annotations.get("component") != name:
raise ReleaseError(f"verified signature annotations do not bind the release source to {name}")
statement, envelope = verify_bundle(prefix.with_suffix(".attestation-bundle.json"), digest,
"https://cyclonedx.org/bom")
canonical = registry_reference(repository)
names = {repository, canonical}
if canonical.startswith("registry-1.docker.io/"):
path = canonical.removeprefix("registry-1.docker.io/")
names.update({f"docker.io/{path}", f"index.docker.io/{path}"})
if (statement["subject"][0].get("name") not in names
or statement.get("predicate") != json.loads(sbom.read_text(encoding="utf-8"))):
raise ReleaseError(f"verified attestation does not bind the scanned SBOM to {name}")
# Preserve Cosign's image-verification JSON view for existing evidence consumers.
signature_view = [{"critical": {"identity": {"docker-reference": repository},
"image": {"docker-manifest-digest": digest},
"type": "cosign container image signature"}, "optional": annotations}]
return json.dumps(signature_view), json.dumps(envelope) + "\n"


def sign(reference: str, sbom: Path, commit: str, name: str, prefix: Path) -> None:
reference = registry_reference(reference)
annotations = ["--annotations", f"source_commit={commit}", "--annotations", f"component={name}"]
upload_cosign_evidence(["cosign", "sign", "--yes", "--bundle",
str(prefix.with_suffix(".signature-bundle.json")), *annotations, reference])
verification = ["--certificate-identity-regexp", IDENTITY, "--certificate-oidc-issuer", ISSUER]
signature = verify_registry_evidence(["cosign", "verify", *verification, *annotations, reference])
prefix.with_suffix(".signature.json").write_text(signature, encoding="utf-8")
upload_cosign_evidence(["cosign", "attest", "--yes", "--bundle",
str(prefix.with_suffix(".attestation-bundle.json")),
"--type", "cyclonedx", "--predicate", str(sbom), reference])
expected = json.loads(sbom.read_text(encoding="utf-8"))
digest = reference.rsplit("@sha256:", 1)[1]

def validate_sbom(attestation: str) -> None:
for line in attestation.splitlines():
if not line.strip():
continue
envelope = json.loads(line)
encoded = envelope["payload"]
statement = json.loads(base64.b64decode(encoded + "=" * (-len(encoded) % 4), validate=True))
if (statement.get("predicate") == expected
and statement.get("predicateType") == "https://cyclonedx.org/bom"
and any(subject.get("digest", {}).get("sha256") == digest
for subject in statement.get("subject", []))):
return
raise ReleaseError(f"verified attestation does not bind the scanned SBOM to {name}")

attestation = verify_registry_evidence(["cosign", "verify-attestation", *verification,
"--type", "cyclonedx", reference], validate=validate_sbom)
signature, attestation = verify_release_bundles(reference, sbom, commit, name, prefix)
prefix.with_suffix(".signature.json").write_text(signature, encoding="utf-8")
prefix.with_suffix(".attestation.jsonl").write_text(attestation, encoding="utf-8")


Expand Down
67 changes: 61 additions & 6 deletions scripts/publish_ghcr_images.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,13 +24,15 @@
import os
from pathlib import Path
import re
import shutil
import subprocess
import sys
import tomllib
from urllib.parse import quote

from publish_dockerhub_images import (IDENTITY, ISSUER, ReleaseError, check_identity,
qualify, registry_reference, run, verify_registry_evidence)
qualify, registry_reference, run, verify_registry_evidence,
verify_release_bundles)


ROOT = Path(__file__).resolve().parents[1]
Expand Down Expand Up @@ -95,8 +97,20 @@ def verify_sbom_subject(attestations: str, repository: str, digest: str) -> None
raise ReleaseError("verified source SBOM does not bind the exact repository/digest")


def verify_source(repository: str, digest: str, commit: str, name: str, prefix: Path) -> None:
def verify_source(repository: str, digest: str, commit: str, name: str, prefix: Path,
*, evidence_prefix: Path | None = None) -> None:
reference = registry_reference(f"{repository}@{digest}")
if evidence_prefix is not None:
signature, attestation = verify_release_bundles(
f"{repository}@{digest}", evidence_prefix.with_suffix(".cdx.json"), commit, name, evidence_prefix,
)
verify_sbom_subject(attestation, repository, digest)
for kind in ("signature", "attestation"):
shutil.copyfile(evidence_prefix.with_suffix(f".{kind}-bundle.json"),
prefix.with_suffix(f".source-{kind}-bundle.json"))
prefix.with_suffix(".source-signature.json").write_text(signature, encoding="utf-8")
prefix.with_suffix(".source-attestation.jsonl").write_text(attestation, encoding="utf-8")
return
verification = ["--certificate-identity-regexp", IDENTITY, "--certificate-oidc-issuer", ISSUER]
signature = verify_registry_evidence([
"cosign", "verify", *verification, "--annotations", f"source_commit={commit}",
Expand All @@ -118,6 +132,40 @@ def scan(reference: str, prefix: Path) -> None:
raise ReleaseError(f"CRITICAL findings block {reference}")


def publication_evidence(directory: Path, version: str, commit: str, name: str,
repository: str, digest: str) -> Path:
matches = []
for path in directory.rglob("publication.json"):
publication = json.loads(path.read_text(encoding="utf-8"))
if (not publication.get("complete") or publication.get("dry_run")
or publication.get("version") != version or publication.get("source_commit") != commit):
raise ReleaseError("source publication evidence is incomplete or belongs to a different release")
for image in publication.get("images", []):
if image.get("component") != name:
continue
if (image.get("repository") != repository or image.get("digest") != digest
or image.get("tags") != [version] or not image.get("published")
or image.get("checks") != "passed"):
raise ReleaseError(f"source publication evidence conflicts with {name}")
matches.append(path.parent / name)
if len(matches) != 1:
raise ReleaseError(f"expected one complete source publication record for {name}")
return matches[0]


def reuse_scan(reference: str, evidence_prefix: Path, prefix: Path) -> None:
findings = json.loads(evidence_prefix.with_suffix(".trivy.json").read_text(encoding="utf-8"))
names = {reference, registry_reference(reference), reference.replace("docker.io/", "index.docker.io/", 1)}
if (findings.get("SchemaVersion") != 2 or findings.get("ArtifactType") != "container_image"
or findings.get("ArtifactName") not in names or not isinstance(findings.get("Results"), list)):
raise ReleaseError("source vulnerability scan does not bind the exact image")
if any(vulnerability.get("Severity") == "CRITICAL"
for result in findings["Results"] for vulnerability in result.get("Vulnerabilities", [])):
raise ReleaseError(f"CRITICAL findings block {reference}")
for suffix in (".cdx.json", ".trivy.json"):
shutil.copyfile(evidence_prefix.with_suffix(suffix), prefix.with_suffix(suffix))


def selected_images(group: str) -> list[dict]:
policy = json.loads((ROOT / "docker/release-images.json").read_text(encoding="utf-8"))
if policy["schema_version"] != 1 or policy["platform"] != "linux/amd64":
Expand Down Expand Up @@ -147,7 +195,7 @@ def validate_source(version: str, commit: str) -> None:


def execute(group: str, version: str, commit: str, namespace: str, owner: str,
publish: bool, output: Path) -> dict:
publish: bool, output: Path, *, source_evidence: Path | None = None) -> dict:
validate_source(version, commit)
if any(not re.fullmatch(r"[a-z0-9][a-z0-9_-]*", value) for value in (namespace, owner)):
raise ReleaseError("registry namespace/owner must be lowercase")
Expand Down Expand Up @@ -180,8 +228,13 @@ def save() -> None:
# Every selected source passes identity, signature, SBOM and fresh scan checks before any copy.
for name, source, _, digest, _ in plan:
print(f"Qualifying {name} {digest}", flush=True)
verify_source(source, digest, commit, name, output / name)
scan(f"{source}@{digest}", output / name)
if source_evidence is not None:
evidence_prefix = publication_evidence(source_evidence, version, commit, name, source, digest)
verify_source(source, digest, commit, name, output / name, evidence_prefix=evidence_prefix)
reuse_scan(f"{source}@{digest}", evidence_prefix, output / name)
else:
verify_source(source, digest, commit, name, output / name)
scan(f"{source}@{digest}", output / name)
for name, source, destination, digest, present in plan:
reference = f"{destination}:{version}"
if publish:
Expand Down Expand Up @@ -216,10 +269,12 @@ def main() -> int:
parser.add_argument("--owner", default="mxsm")
parser.add_argument("--publish", action="store_true")
parser.add_argument("--output", type=Path, default=ROOT / "target/release/ghcr")
parser.add_argument("--source-evidence", type=Path,
help="reuse freshly verified Docker Hub artifacts from this same workflow run")
args = parser.parse_args()
try:
result = execute(args.group, args.version, args.source_commit, args.namespace,
args.owner, args.publish, args.output)
args.owner, args.publish, args.output, source_evidence=args.source_evidence)
except (ReleaseError, OSError, KeyError, ValueError, TypeError, subprocess.CalledProcessError) as error:
print(f"GHCR_RELEASE_FAILED: {error}", file=sys.stderr)
return 1
Expand Down
Loading
Loading