Skip to content

Shared checks.yml in this repo with a derived Python matrix #95

Closed as not planned

Description

@lesnik512

Why

The standard, section 7: every library's ci.yml and scheduled.yml call one reusable checks.yml living here, referenced at main. Today 22 repos carry a copy of _checks.yml; 12 are byte-identical and the rest differ in service containers, the lint-job Python pin, and a docs job.

Design (settled)

  • File: .github/workflows/checks.yml in modern-python/.github, on: workflow_call. Callers: uses: modern-python/.github/.github/workflows/checks.yml@main with secrets: inherit. Referenced at main deliberately: one merge here reaches every repo; a change is first exercised from a branch ref in one repo.
  • Jobs: lint on the repo's floor Python (just install lint-ci), pytest matrix (just install then just test-ci, fail-fast: false), links (lychee --offline with the blob/main remap), docs behind a boolean input.
  • Matrix derived at run time: floor from requires-python, ceiling from the newest released cycle on https://endoflife.date/api/python.json, plus that cycle's free-threaded build. No repo edits a matrix when a Python ships.
  • Service containers: GitHub's context table lists inputs as available in jobs.<job_id>.services, so the shared workflow can take the image and health command as inputs for the four repos that need one (db-retry: postgres; faststream-redis-timers, modern-di-arq: redis; faststream-concurrent-aiokafka: redpanda). Verify this on the first of those four before migrating the others.
  • release.yml stays per repo: PyPI Trusted Publishing does not accept a reusable workflow (Trusted publishing: Support for GitHub reusable workflows pypi/warehouse#11096, open).

Done when

Every library's ci.yml and scheduled.yml call the shared workflow and _checks.yml is deleted in each; the four service repos run through the inputs path.

Activity

  1. lesnik512 commented on Sep 12, 2026

    @lesnik512
    MemberAuthor

    Input from #91 (adding 3.14t to the per-repo _checks.yml): the derived matrix needs two per-entry knobs that the shared workflow has to expose, otherwise two repos go red the day it switches on.

    • Job-level env that applies only to the free-threaded entry. db-retry sets DISABLE_SQLALCHEMY_CEXT_RUNTIME=1 on 3.14t (SQLAlchemy 2.0.x cyext re-enables the GIL; fixed upstream for 2.1), faststream-concurrent-aiokafka sets AIOKAFKA_NO_EXTENSIONS=1 (no cp314t wheel; the C codecs re-enable the GIL). Both use ${{ matrix.python-version == '3.14t' && '1' || '' }} today, so an env input (or a free-threaded-env map) that the workflow spreads onto the pytest job covers it.
    • A way to drop the free-threaded entry per repo. modern-di-arq (hiredis, free-threading support redis/hiredis-py#229) and modern-di-grpc (grpcio, Support no-GIL (free-threaded) Python grpc/grpc#38762) cannot keep the GIL disabled because of a dependency and carry exemption rows on the standard. A boolean input like free-threaded: false is the minimum; the exemption row is the record that justifies setting it.

    Also worth carrying over: the 3.14t entry in every migrated repo runs python -c "import sys, <package>; assert not sys._is_gil_enabled()" before the suite. Pytest only emits a RuntimeWarning when an import re-enables the GIL, so without that step a GIL-enabled run still goes green.

  2. lesnik512 commented on Sep 15, 2026

    @lesnik512
    MemberAuthor

    Closing without migrating. #101 built the workflow as designed and db-retry#46 proved it end to end from a branch ref, including the service-container path. The decision is against it, on the shape of the interface rather than on whether it works.

    Why not

    workflow_call inputs are scalars. Anything structured a repo varies, a service container or an env map, has to be flattened into a string: first a health command plus a baked-in POSTGRES_PASSWORD, then raw docker create options. Both are image knowledge leaking into the org repo, and neither reads well. That is a GitHub limit, not a design slip, so it does not go away with better input names.

    What sharing buys is real but bounded: the run-time matrix (the only part section 6 needs) and one-PR policy changes instead of a per-repo sweep. Sweeps are how the org already works, so that saving does not pay for the interface.

    The idea, if this is ever reopened

    A reusable workflow with zero inputs, everything read from the repo:

    • services: the pytest job runs just services-up when just --summary lists it; the recipe is docker compose up -d --wait <service>, so image, env and healthcheck stay in the compose file the repo already keeps for local dev (modern-di-arq has this today as redis-up)
    • test env: tests default to localhost, as three of the five service repos already do
    • free-threaded exemption: derived from the Free Threading classifier section 11 requires; today exactly the two exempt repos lack it
    • docs job: when the justfile has docs-build
    • the t-build switch two repos need (DISABLE_SQLALCHEMY_CEXT_RUNTIME, AIOKAFKA_NO_EXTENSIONS): exported by test-ci when the interpreter reports the GIL disabled

    Every caller is then the same two lines. The matrix script and its tests in #101 are the reusable part of that work.

    Section 7 of the standard still says the shared checks.yml "will live" here; that sentence needs its own change now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions