Merge develop into infrahub-develop - #1221
Merged
Merged
Conversation
Add an opinionated, offline formatter for Infrahub schema YAML files whose job is to normalise the ordering of keys within each node, generic, attribute, relationship and dropdown choice, so hand-authored schemas read consistently and produce small diffs. - New infrahub_sdk/ctl/schema_format.py with the pure formatting logic: canonical key orders, restricted-namespace filtering (core nodes only), list-item order preserved, a PyYAML dumper matching the schema-library layout, literal-block multiline handling, and a semantic-equality guard that aborts rather than risk changing a file's meaning. - New `format` subcommand in schema.py: in-place by default, plus --check (CI gate) and --diff, with warnings for comments that PyYAML cannot preserve. - Unit + CLI tests and the regenerated infrahubctl CLI reference.
Switch the schema formatter from PyYAML to ruamel.yaml round-trip mode so that reordering keys no longer discards comments. This also preserves quoting and inline (flow) sequences (e.g. `[manufacturer, name__value]`) for free, so the diff a format run produces is now purely key-ordering. - schema_format.py: reorder keys in place with move_to_end so the comments ruamel attaches to each key travel with it; keep the semantic-equality guard, restricted-namespace filtering, and canonical key orders. The header is preserved (or added when missing). - Drop the comment-drop warning and count_droppable_comments, which existed only because PyYAML lost comments. - schema.py: format from the raw file text; update the command help. - Add ruamel.yaml to the `ctl` / `all` dependency sets.
The formatter's canonical key ordering is written against a known set of schema properties. When Infrahub adds or removes a property in the published JSON schema, that ordering may need updating (an unrecognised key is preserved, but not ideally placed). Track this without gating releases: - infrahub_sdk/ctl/schema_drift.py compares the live schema's property sets to a committed baseline (schema_properties.json) and reports added/removed properties. It never raises on drift. - `invoke schema-drift-check` emits GitHub ::warning:: annotations for any drift and always exits 0; `invoke schema-drift-update` refreshes the baseline. - .github/workflows/schema-drift.yml runs the check on release publish and manual dispatch, warn-only. - Baseline snapshot + offline unit tests for the drift logic.
…er detection Address code-review findings: - _format_entity: only iterate `attributes`/`relationships` when they are lists, so a parseable-but-malformed schema (e.g. `attributes: 5`) is left untouched instead of crashing. - _ensure_schema_header: detect a real `# yaml-language-server:` directive line via regex rather than an arbitrary substring, so the header is still added when the string only appears in a scalar or unrelated comment. - test_format_preserves_comments: assert exit_code == 0 so the test can no longer pass silently if the format command fails. Add regression tests for the malformed-section and substring-in-scalar cases.
CI lints with ruff 0.15.12 (develop's pinned version), which enforces pydocstyle D413; the branch's local ruff 0.15.0 did not, so this passed locally but failed in CI. Add the required blank line after the final docstring section in the schema formatter/drift modules and the schema format command.
Sync the 361-commit gap so CI (which validates the merge with develop) uses the same ruff (0.15.12) and doc generator as develop.
_print_schema_diff used markup=False (needed so bracketed diff content like [manufacturer, name] stays literal) together with inline [green]/[red] tags, which then printed verbatim instead of colouring the line. Apply the colour with the style= argument instead.
…a format Three off-by-default transforms for `infrahubctl schema format`, keeping the base command purely key-ordering: - --strip-defaults: remove node/attribute/relationship keys whose value equals the schema default (context-aware; grounded in the published JSON-schema defaults). Consequential/internal fields (branch, state, inherited, display) are intentionally not stripped. - --sort-by-order-weight: sort attributes and relationships ascending by order_weight; items without one keep their authored order and go last. - --backfill-order-weight: give attributes/relationships lacking an order_weight a single constant value (1000). The semantic guard now neutralises exactly the requested transforms on both sides of the comparison, so an intended change is allowed while any unintended corruption still aborts. Verified guard-safe and idempotent across all schema-library files for every flag combination.
…guard Address PR review comments: - _format_one_schema_file now returns a FormatOutcome enum instead of loose string literals, tightening the contract with the format command loop (per review feedback). - The semantic guard neutralised list reordering by sorting on `name`, which spuriously aborted when two items shared a name but differed in weight. Sort by full item content instead — a total, content-based order permits any intended reorder while still catching a dropped or corrupted item.
Raise patch coverage on the new modules: - schema_drift: test fetch_live_properties (mocked HTTP) and the write/load baseline round-trip — module now fully covered. - schema format CLI: cover the multi-document, invalid-file, and FormatError branches, plus non-dict list items / extension entries. - Drop a now-dead isinstance guard in _strip_default_keys (both callers already pass a mapping). Type-only test imports moved under TYPE_CHECKING to match the repo convention; no private helpers are imported from tests.
Round-trip ruamel YAML is stricter than the PyYAML safe_load used to discover schema files — notably it rejects duplicate keys, which `schema load` tolerates (last wins). That raised ruamel's YAMLError, which escaped the per-file `except FormatError`, hit @catch_exception, printed a traceback and exited 1 — aborting a whole folder run midway. Catch YAMLError on load and re-raise as FormatError so it is reported per file and the remaining files still format.
feat(ctl): add `infrahubctl schema format` command
github-actions
Bot
added
group/ci
Codecov Report❌ Patch coverage is
@@ Coverage Diff @@
## infrahub-develop #1221 +/- ##
====================================================
+ Coverage 83.98% 85.21% +1.23%
====================================================
Files 139 146 +7
Lines 14390 15605 +1215
Branches 2381 2631 +250
====================================================
+ Hits 12085 13298 +1213
+ Misses 1654 1642 -12
- Partials 651 665 +14
Flags with carried forward coverage won't be shown. Click here to find out more.
... and 20 files with indirect coverage changes 🚀 New features to boost your workflow:
|
infrahub-github-bot-app
Bot
merged commit Jul 30, 2026
c656727
into
infrahub-develop
36 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merging develop into infrahub-develop after merging pull request #1189.
Summary by cubic
Add a new
infrahubctl schema formatcommand to canonically reorder keys in schema YAML while preserving comments and semantics, plus a warn-only schema drift check to flag JSON schema changes after releases.New Features
infrahubctl schema format:ruamel.yaml.--check,--diff,--strip-defaults,--sort-by-order-weight, and--backfill-order-weight.$schemaheader when missing; guards against semantic changes.infrahub_sdk/ctl/schema_properties.jsonand tasksinvoke schema-drift-check(warn-only) andinvoke schema-drift-update..github/workflows/schema-drift.ymlruns on release to surface added/removed JSON schema properties.Dependencies
ruamel.yaml>=0.18to thectlandallextras.Written for commit aa0cb26. Summary will update on new commits.