Conversation
Temporary [tool.uv.sources] override to GridTools/gt4py#2912 at 22aab76b9, the last PR of stack #2917 (#2899 -> #2907 -> #2910 -> #2912). Pinned by rev, not branch, so a rebase of the stack cannot silently move what this is built against. Revert once the stack is released. The manifest `gt4py==` pins are left alone; a source override does not enforce them. gt4py pins `dace==2.0.0a9`, which moves dace from 2.0.0a7; nothing else in the resolution changes.
Output of gt4py's `scripts/python/migrate_connectivities.py` (from #2910, at the pinned rev) run over model/, tools/ and bindings/, then `ruff format`. Dimensions become `DimensionIndex` / `LocalDimensionIndex` subclasses, whose tag is now their qualified name. Each `FieldOffset` becomes a `NeighborConnectivity[Domain, Codomain]` that adopts the existing local dimension (`Local: typing.TypeAlias = E2CDim`), so every `dims.XxxDim` stays valid. `Koff` is removed and its uses become `KDim + i` / `as_offset(KDim, ...)`. `Local` must stay an annotated `TypeAlias`: the PEP 695 form ruff's UP040 asks for yields a `TypeAliasType` rather than the class, so the rule is silenced on those lines.
`gtx.Dimension` is now a PEP 695 alias for `type[DimensionIndex]`, so `isinstance(d, gtx.Dimension)` raises TypeError. That made `dimension.horizontal_dims()` fail on first call, and with it everything built on it, including the GHEX domain descriptors in `mpi_decomposition`. `DimensionMeta` is not exported from `gt4py.next`, so it is reached through `gt4py.next.common`, which these modules already import. The helpers yield 3 horizontal, 2 vertical (`KDim`, `Staggered[KDim]`) and 15 local dimensions, as before.
#2910 keys offset providers by the connectivity declaration and rejects string
keys at every program entry point, and removes `FieldOffset`. `Grid` passes its
`connectivities` straight through as the provider, so it is now keyed by the
connectivity class:
- `Grid.connectivities` is `Mapping[type[NeighborConnectivity], NeighborTable]`
and `get_connectivity` takes the class. `construct_connectivity` reads
`domain`, `codomain` and `local_dimension_of` from the declaration instead of
`FieldOffset.source/target`.
- The halo constructor and the grid manager's neighbor tables are class-keyed;
the halo's `str | FieldOffset` normalisation is gone, since both callers now
pass classes.
- Field providers take `connectivities={"e2c": dims.E2C}`, the connectivity
itself, where they took its local dimension and read its name.
- The stencil-test connectivity view is class-keyed and no longer needs a cast
over a name-keyed mapping; its tests check the class-keyed contract.
- String provider keys and `get_connectivity("X")` calls become `dims.X`, and
the `Mapping[gtx.FieldOffset, np.ndarray]` annotations of the reference
functions become `Mapping[type[gtx.NeighborConnectivity], np.ndarray]`.
havogt
force-pushed
the
gt4py-2844-dimensions
branch
from
September 25, 2026 08:16
f357a1a to
b0e4a57
Compare
havogt
changed the title
Contributor
Author
|
cscs-ci run default |
The first CI run on this branch failed 45 common unit tests and 54 mypy checks. All were icon4py-side: - A dimension's name is `.tag`; `.value` on a dimension class raises. Log messages and the RNG seed in `test_parallel_io` read `.tag`; pytest ids and test messages use `__name__`, since the tag is now a qualified path. `test_icon` reached a connectivity through its local dimension's name and now uses `dim.owner`. mypy does not flag any of these: `value` is declared on the index instances, so reading it on the class type-checks. - `setup_program` annotated `offset_provider` as `gtx_typing.OffsetProvider`, which is the tag-keyed internal form; it now takes `OffsetProviderLike`, what the programs themselves accept. That type is not re-exported from `gt4py.next.typing`, so it comes from `gt4py.next.common`. - There is no string factory for dimensions any more: the test-only dimensions in `test_vertical` are declared classes, and `test_parallel_grid_manager` takes the local dimension from the connectivity it iterates. - `test_halo` indexed the class-keyed neighbor tables by name. - One dimension-keyed dict literal in `test_factory` needed an annotation. `model/common/tests/common` with `--datatest-skip`: 727 passed, 0 failed. mypy: no issues found in 420 source files.
Contributor
Author
|
cscs-ci run default |
Three test calls still passed `"E2C"` to `connectivity_field`, which now takes
the connectivity class; two are tracer-advection stencil tests CI caught, the
third is the stencil-test framework's own unit test. The earlier sweep only
covered `get_connectivity("...")` and `connectivities["..."]`, and mypy does
not check the test trees these live in.
No string literal naming a connectivity is left in model/, tools/ or bindings/.
Contributor
Author
|
cscs-ci run default |
|
When developing, you can test your changes on CSCS CI before merge with the You can pass options to override pipeline variables, for example:
Avoid running the pipeline for all tests when you are developing. Available options are:
For each option, Multiple values can be given to each option with See The Merging Once your PR is approved and ready for merging, add it to the merge queue. The Optional Tests To run benchmarks you can use:
For more detailed information please look at CI in the EXCLAIM universe. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Not for merge. This branch pins an unmerged gt4py stack. It exists to evaluate the connectivities as types proposal (GridTools/gt4py stack #2917: #2899 → #2907 → #2910 → #2912) on ICON4Py, and to feed the result back to its author.
It replaces this PR's earlier content, which targeted GridTools/gt4py#2844; that PR was closed unmerged in favour of the stack.
What the stack changes for ICON4Py
NeighborConnectivity[Domain, Codomain]class.FieldOffsetis removed, and a Cartesian shift isKDim + i(ADR 0029).Commits
Pin gt4py to #2912 at
22aab76b9. This moves dace from 2.0.0a7 to 2.0.0a9, which gt4py pins.Declare dimensions and connectivities as classes: the output of the stack's own
migrate_connectivities.pyovermodel/,tools/andbindings/. Each connectivity adopts its existing local dimension, so everydims.XxxDimstays valid.Koffis gone.isinstance(…, DimensionMeta):isinstance(d, gtx.Dimension)now raises, which brokedimension.horizontal_dims()and everything built on it.Key offset providers by connectivity class.
Grid.connectivities,get_connectivity, the halo constructor, the field providers'connectivities=and the stencil-test connectivity view are all class-keyed, and string keys becomedims.X. The script reports these sites but cannot rewrite them:Gridbuilds its providers dynamically from names, which is the bulk of the hand work.Finish the migration where CI found gaps: the dimension
.valuereads,setup_program's provider annotation, the test-only dimensions that used the removed string factory, and two test sites.Status
The first CI run found gaps, all on the icon4py side: 45 common unit tests and 54 mypy errors. Commit 5 fixes them. Locally on this head:
model/common/tests/commonwith--datatest-skip: 727 passed, 0 failed.The CSCS run on santis is pending.
Feedback for the stack
DimensionMetais still not exported fromgt4py.next.isinstance(x, gtx.Dimension)raises, soDimensionMetais the only way to ask "is this a dimension", and it is reachable only throughgt4py.next.common. The migration script's own advice names it by that path.Gridbuildsconnectivitiesfrom names at run time and hands the dict to programs asoffset_provider. Nothing flags that dict until a program call rejects it.Local: typing.TypeAlias = E2CDimto a PEP 695type Local = E2CDim, which yields aTypeAliasTyperather than the class. Every downstream running ruff with pyupgrade rules will need# noqa: UP040on each adopted local dimension, 14 lines here..value→.tagis invisible to mypy.valueis declared on the index instances, so reading it on a dimension class type-checks and only fails at run time, once per executed path. Here the full mypy gate was green while twelve such reads remained; the test suite found them.gt4py.next.typingexports onlyOffsetProvider, the tag-keyed internal form that the strict entry points reject.OffsetProviderLike, the form users actually pass, exists only ingt4py.next.common, so a downstream annotating a provider reaches for the public name and gets the wrong type.