Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,10 @@ A test that only covers the exact instance provides weaker protection: the same
- The green step **proves** the fix resolves it.
- A class-scoped test acts as a durable regression guard against future reintroduction of the same pattern.

## API compatibility guard

PRs to `stable/**` must pass the `API compatibility` check (`scripts/api_compat.py`, tests in `tests/acceptance/test_api_compat.py`), and the Publish workflow repeats it before releasing from a stable branch. It compares `generated/` with the newest published `N.x.y` on PyPI. Only an intended break may be approved, with the `api-break-approved` label on the PR (or the `api_break_approved` input of a manual Publish run for upstream drift). Never apply the label to make a check pass; make the change backward compatible instead. See CONTRIBUTING.md for what counts as breaking.

## README Code Examples

### API spec examples: prefer ergonomic helpers
Expand Down
95 changes: 95 additions & 0 deletions .github/workflows/api-compat.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
name: API compatibility

# Blocks PRs to stable branches that change the public Python API in a way that breaks
# customer code (see scripts/api_compat.py). A PR is compared with its base branch, so it
# answers only for its own changes: an approved break already merged does not fail every
# later PR. When a break is intended (e.g. a bug fix that must change the API), apply the
# `api-break-approved` label: the check still runs and lists the breaks, but passes. The
# Publish workflow compares the code it is about to release with the last release.

on:
pull_request:
branches:
- 'stable/**'
# labeled/unlabeled: re-evaluate when api-break-approved is added or removed. edited:
# re-evaluate when the PR is retargeted to another stable branch. The job must run for
# every such event: a skipped run would replace a failed one and the required check
# would count as passing.
types: [opened, synchronize, reopened, labeled, unlabeled, edited]

permissions:
contents: read

concurrency:
group: api-compat-${{ github.event.pull_request.number }}
cancel-in-progress: true

jobs:
api-compat:
name: API compatibility
env:
UV_PYTHON: "3.12"
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v6

- name: Check out the base branch
uses: actions/checkout@v6
with:
ref: ${{ github.event.pull_request.base.sha }}
path: .api-compat-base
persist-credentials: false

- name: Install uv
uses: astral-sh/setup-uv@v7

- name: Set up Python
uses: actions/setup-python@v6
with:
python-version: "3.12"

# Both the baseline wheel and generated/ are imported to read their API, so the
# SDK's runtime dependencies must be installed.
- name: Install dependencies
run: uv sync --all-extras --dev

# Regenerate both sides from their checked-in bundled specs, so a generator change is
# checked even when its regenerated output was not committed. Upstream spec drift is
# left to the release gate, which checks what Publish actually generates.
- name: Generate SDK
run: |
# stable/9 has no `make generate-only`; generate inside each checkout.
for root in . .api-compat-base; do
(
cd "$root"
uv run generate.py --generator openapi-python-client --config generator-config-python-client.yaml --skip-tests --bundled-spec external-spec/bundled/rest-api.bundle.json
uv run ruff format generated/
uv run ruff check generated/ --fix
)
done

- name: Check API compatibility
env:
# stable/<N> with no N.x release yet (just cut from main) has no published API
# to protect and passes.
MAJOR: ${{ github.base_ref }}
APPROVED: ${{ contains(github.event.pull_request.labels.*.name, 'api-break-approved') }}
run: |
set -uo pipefail
released=$(uv run python scripts/api_compat.py --print-baseline-major "${MAJOR#stable/}") || exit 2
if [ -z "$released" ]; then
echo "::notice::No ${MAJOR#stable/}.x release published yet; nothing to protect."
exit 0
fi
rc=0
uv run python scripts/api_compat.py --baseline-dir .api-compat-base/generated || rc=$?
if [ "$rc" -eq 1 ] && [ "$APPROVED" = "true" ]; then
echo "::warning::Breaking API changes approved with the api-break-approved label."
echo "Approved with the \`api-break-approved\` label." >> "$GITHUB_STEP_SUMMARY"
exit 0
fi
if [ "$rc" -eq 1 ]; then
echo "::error::If this break is intended (e.g. a bug fix that must change the API), apply the api-break-approved label. Otherwise make the change backward compatible."
fi
exit "$rc"
26 changes: 24 additions & 2 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,12 @@ on:
description: "Git ref/branch/tag/SHA in camunda/camunda.git to fetch the OpenAPI spec from (temporary override)."
required: false
default: ""
api_break_approved:
description: >-
Release even though the public API breaks compatibility with the last release on
this stable branch (for breaks with no PR to label, e.g. from upstream spec drift).
type: boolean
default: false

concurrency:
group: release-${{ github.ref_name }}
Expand Down Expand Up @@ -213,6 +219,8 @@ jobs:
git push

release:
env:
UV_PYTHON: "3.12"
needs: generate
runs-on: ubuntu-latest
permissions:
Expand Down Expand Up @@ -257,7 +265,7 @@ jobs:
- name: Set up Python
uses: actions/setup-python@v6
with:
python-version-file: "pyproject.toml"
python-version: "3.12"

- name: Install dependencies
run: uv sync --all-extras --dev --group ci
Expand Down Expand Up @@ -299,12 +307,26 @@ jobs:

echo "Version $NEW_VERSION matches branch constraint ${BRANCH_MAJOR}.x"

# Compares the public API about to be released (generated/ as committed by the
# generate job) with the newest release of this stable major on PyPI. This covers
# generated code from upstream spec drift that no PR reviewed. Override: the
# api-break-approved label on any PR merged since that release, or the
# api_break_approved input of a manual run. scripts/semantic-release-with-retry.sh
# repeats it if a retry moves to a newer branch tip.
- name: API compatibility gate (stable branches)
if: steps.check_release.outputs.release_needed == 'true' && startsWith(github.ref_name, 'stable/')
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
DISPATCH_APPROVED: ${{ github.event_name == 'workflow_dispatch' && inputs.api_break_approved }}
run: bash scripts/api-compat-gate.sh "$GITHUB_REF_NAME"

- name: Create release
if: steps.check_release.outputs.release_needed == 'true'
env:
GH_TOKEN: ${{ steps.app-token.outputs.token }}
DISPATCH_APPROVED: ${{ github.event_name == 'workflow_dispatch' && inputs.api_break_approved }}
run: |
uv run semantic-release version
bash scripts/semantic-release-with-retry.sh
uv run semantic-release publish

- name: Build package
Expand Down
11 changes: 11 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,12 +115,23 @@ make itest
- `make itest` – generate SDK and run integration tests (requires running server)
- `make lint` – run ruff linter
- `make typecheck` – run pyright type checker
- `make api-compat MAJOR=<N>` – compare the public API of `generated/` with the newest published `N.x.y` release
- `make docs-api` – generate API documentation to `./public/`
- `make preview-docs` – generate and serve docs locally
- `make clean` – remove the `generated/` directory
- `make clean_spec` – remove the `.openapi-cache/` directory
- `make clean-docs` – remove generated documentation

### API compatibility on stable branches

`stable/N` regenerates from a moving upstream spec, so a release can change the public API without anyone reviewing it. `scripts/api_compat.py` compares the API of `generated/` with the newest non-prerelease `N.x.y` on PyPI and lists every change that would break code written against it: removed modules, classes, functions, attributes or enum members; new required parameters; narrowed parameter types; and result values that became wider (for example a response field turning `str | None` or `str | Unset`). Request models may widen; response models may narrow. Exit codes: `0` compatible, `1` breaking, `2` setup error.

- The `API compatibility` check runs on every PR to `stable/**` and compares it with its base branch, both regenerated from their checked-in bundled specs, so a PR answers only for its own changes.
- The Publish workflow compares what it is about to release with the newest `N.x.y` on PyPI before releasing from `stable/**`.
- If a break is intended (e.g. a bug fix that must change the API), apply the `api-break-approved` label to the PR. The check still lists the breaks but passes. The release gate accepts the label only on a PR merged into the release branch since the baseline release, and only for the API as of that PR's merge: a break added afterwards needs its own approval. For a break with no PR to label (upstream spec drift), re-run Publish manually with `api_break_approved`.

Run it locally with `make api-compat MAJOR=10`.

### README Code Examples

Code blocks in `README.md` are **injected from compilable example files** — do not edit them inline.
Expand Down
8 changes: 7 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: install generate clean test itest docs-api docs-md docs-link-check bundle-spec typecheck-examples clean-docs preview-docs sync-readme sync-readme-check check
.PHONY: install generate clean test itest docs-api docs-md docs-link-check bundle-spec typecheck-examples clean-docs preview-docs sync-readme sync-readme-check check api-compat

# Git ref/branch/tag/SHA in https://github.com/camunda/camunda.git to fetch the OpenAPI spec from.
# Override like: `make generate SPEC_REF=45369-fix-spec`
Expand Down Expand Up @@ -57,6 +57,12 @@ lint:
typecheck:
uv run pyright

# Compare the public API of generated/ with the newest published MAJOR.x.y release.
# Usage: make api-compat MAJOR=10 (exit 1 lists breaking changes)
api-compat:
@test -n "$(MAJOR)" || { echo "usage: make api-compat MAJOR=<N>"; exit 2; }
uv run python scripts/api_compat.py --baseline-major $(MAJOR)

check: lint typecheck

typecheck-examples:
Expand Down
30 changes: 30 additions & 0 deletions scripts/api-break-approvals.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
# Usage: api-break-approvals.sh <baseline-tag> <release-branch>
#
# Prints `#N <merge-sha>` for each PR labelled api-break-approved that was merged into
# <release-branch> after <baseline-tag>, newest merge first. A label on a PR merged elsewhere
# (say `main`, whose commits a backport shares) or before the baseline approves nothing here.
# Exits non-zero if a lookup fails, so the caller can tell "none" from "unknown".
set -euo pipefail

tag="$1"
branch="$2"
shas=$(git log --format=%H "${tag}..HEAD")

# A file, not `$(...)`: command substitution does not inherit `set -e`, so a failed lookup
# on any but the last commit would be swallowed.
approved=$(mktemp)
trap 'rm -f "$approved"' EXIT
for sha in $shas; do
gh api "repos/${GITHUB_REPOSITORY}/commits/${sha}/pulls" \
--jq '.[] | select(.merged_at != null) | select(any(.labels[]; .name == "api-break-approved")) | "\(.number) \(.base.ref) \(.merge_commit_sha)"' \
| while read -r number base merge_sha; do
if [ "$base" = "$branch" ] && grep -qx "$merge_sha" <<<"$shas"; then
echo "#${number} ${merge_sha}"
fi
done
done > "$approved"

for sha in $shas; do
grep " ${sha}$" "$approved" || true
done | awk '!seen[$0]++'
71 changes: 71 additions & 0 deletions scripts/api-compat-gate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
#!/usr/bin/env bash
# Release gate for stable/<major>: compare the public API at HEAD (generated/) with the
# newest release of the major on PyPI, and fail unless any break is approved.
#
# Called by the "API compatibility gate" step of .github/workflows/publish.yml, and again
# by scripts/semantic-release-with-retry.sh after it resets to a newer branch tip, so the
# commit that gets released is always the commit that was checked.
#
# Approval: the api-break-approved label on a PR merged into <branch> since that release
# (scripts/api-break-approvals.sh), or DISPATCH_APPROVED=true (the api_break_approved
# input of a manual run). A label approves the API as of that PR's merge: breaks added
# after the newest approved merge need their own approval.
# Needs GH_TOKEN and GITHUB_REPOSITORY. Exit codes: 0 pass, 1 blocked, 2 setup error.
set -uo pipefail

here=$(dirname "$0")
BRANCH="${1:?usage: api-compat-gate.sh <branch>}"
major="${BRANCH#stable/}"
# The newest published release of this stable major, not the nearest tag: a stable
# branch is cut from main at a prerelease.
version=$(uv run python scripts/api_compat.py --print-baseline-major "$major") || exit 2
if [ -z "$version" ]; then
echo "::notice::No ${major}.x release published yet; nothing to compare against."
exit 0
fi
tag="v${version}"
git rev-parse -q --verify "refs/tags/${tag}" >/dev/null || {
echo "::error::Baseline ${version} has no ${tag} tag to list the PRs merged since."
exit 2
}
echo "Baseline: ${tag} (checking $(git rev-parse --short HEAD))"
rc=0
uv run python scripts/api_compat.py --baseline-version "$version" || rc=$?
[ "$rc" -eq 1 ] || exit "$rc"
if [ "${DISPATCH_APPROVED:-false}" = "true" ]; then
echo "::warning::Breaking API changes approved by the api_break_approved input of this run."
exit 0
fi
approved=$("$here/api-break-approvals.sh" "$tag" "$BRANCH") || {
echo "::error::cannot list the approved PRs merged since ${tag}."
exit 2
}
if [ -z "$approved" ]; then
echo "::error::Release blocked: the public API breaks compatibility with ${tag}. If intended, label the PR that made the change api-break-approved (or re-run this workflow manually with api_break_approved), then re-run."
exit 1
fi
prs=$(cut -d' ' -f1 <<<"$approved" | xargs)
read -r latest_pr latest <<<"$approved"
if [ "$latest" != "$(git rev-parse HEAD)" ]; then
dir=$(mktemp -d)
trap 'git worktree remove --force "$dir" >/dev/null 2>&1; rm -rf "$dir"' EXIT
git worktree add -q --detach "$dir" "$latest" || exit 2
# From that commit's own spec and generator, so only later changes show up.
(
cd "$dir" &&
uv run generate.py --generator openapi-python-client --config generator-config-python-client.yaml --skip-tests --bundled-spec external-spec/bundled/rest-api.bundle.json &&
uv run ruff format generated/ &&
uv run ruff check generated/ --fix
) >/dev/null || {
echo "::error::cannot generate the SDK at ${latest}."
exit 2
}
rc=0
uv run python scripts/api_compat.py --baseline-dir "$dir/generated" || rc=$?
if [ "$rc" -eq 1 ]; then
echo "::error::Release blocked: breaking API changes were added after ${latest_pr} (${latest}), the newest approved PR. Label the PR that made them api-break-approved (or re-run this workflow manually with api_break_approved), then re-run."
exit 1
fi
[ "$rc" -eq 0 ] || exit "$rc"
fi
echo "::warning::Breaking API changes approved with the api-break-approved label on: ${prs}"
Loading
Loading