Require marshmallow 4 or later - #107
Conversation
github-actions
Bot
added
part:docs
bb481ba to
8245824
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds backwards-compatible support for marshmallow 4.x by migrating from schema context-based serialization configuration to Python's ContextVar. The changes maintain full API compatibility while deprecating the old constructor-based approach.
Key changes:
- Introduces
serialize_as_string_defaultas aContextVarfor controlling serialization format globally - Deprecates the
serialize_as_string_defaultconstructor parameter with aDeprecationWarning - Adds backwards compatibility handling in
_QuantityField.__init__to check metadata for per-field configuration
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| src/frequenz/quantities/experimental/marshmallow.py | Adds QuantitySchema.__init__ with deprecation logic and backwards compatibility for metadata-based field configuration |
| tests/experimental/test_marshmallow.py | Adds two new tests to verify the deprecated constructor API works correctly with both True and False values |
| RELEASE_NOTES.md | Documents the API changes, migration path, and rationale for the marshmallow 4.x compatibility update |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
8245824 to
9c30b12
Compare
github-actions
Bot
added
the
part:tooling
6b536ab to
8cca58a
Compare
llucax
added
the
scope:breaking-change
llucax
left a comment
llucax
left a comment
There was a problem hiding this comment.
OK, this is a tricky one, and a reason why we might want to split marshmallow support into a separate package at some point.
For people using marshmallow, this is in fact a breaking change, as marshmallow 4.0 is required, not just optional, but for users not using marshmallow is not a breaking change.
Normally, if any part of the code is backwards incompatible, even if unused, should be a breaking release, I think this should be a breaking release too. The only reason we could try to sell this as a non-breaking release would be because it is an experimental package. But still, we are supposed to be committed to not break even those.
What we could do to keep this backwards compatible, is getting the installed marshmallow version and import a different module that is compatible with one or the other version conditionally based on that.
But I think in this case it makes sense to just bite the bullet and release it as a breaking change. For users not using marshmallow, it will be just changing the dependency and changing no code. For users using marshmallow, they are going through a marshmallow upgrade anyway, so fixing one more piece of code should not be too bad.
I'm not sure if keeping the deprecated ctor makes sense if we ship this in a breaking release anyway. It might ease the upgrading a bit for users if they don't need to make any other marshmallow-related updates, so we could keep it if we want to be extra nice, and only remove the deprecated ctor in v3, but I'm not sure it is worth the trouble.
|
I think I would be in favor of just breaking it then without a compatible c'tor :) |
Signed-off-by: Mathias L. Baumann <mathias.baumann@frequenz.com>
8cca58a to
081d31b
Compare
Marenz
changed the title
Signed-off-by: Mathias L. Baumann <mathias.baumann@frequenz.com>
|
🤖 Resolved the three stale Copilot threads: they all concerned the |
Signed-off-by: Mathias L. Baumann <mathias.baumann@frequenz.com>
|
If we go with this, we should release it as 2.0, unless you did some testing and at least none of our projects breaks when just bumping marshmallow to 4.x. |
Is this really true? I thought we ended up doing a change that really supported both 3.x and 4.x. |
|
🤖 Checked empirically: the branch only imports on marshmallow 4 — On breakage: fcr#591 fails CI, but only as a pip resolver conflict against released quantities 1.0.0, which pins marshmallow<4 — that's the metadata bug this PR fixes. No marshmallow-4 API breakage in fcr's or app-edge's own code. Since this is all under |
`frequenz-quantities` 1.0.2 changed its `marshmallow` optional dependency floor from `< 4` to `>= 4` (frequenz-floss/frequenz-quantities-python#107), correcting metadata that had been wrong since that library was ported to `marshmallow` 4. That cap was the only thing holding us on `marshmallow` 3, as our own requirement has allowed `< 5` since 870e880, so CI now resolves `marshmallow` 4. `marshmallow` 4 removed support for validators that signal failure by returning `False`, deprecated in 3.24.0: Custom validators must raise a `ValidationError` for invalid values (deprecated in 3.24.0). Returning `False` is no longer supported. The configuration test dataclasses used exactly that pattern, a `lambda s: s.startswith("test")` in the field metadata. The validator is still registered and still called, only its return value is ignored, so the invalid value is accepted and `test_load_config_dataclass` and `test_new_receiver_configurations[Validation Error]` fail. Replacing the lambdas with a function that raises restores the checks. The message stays `Invalid value.`, which is what `marshmallow` 3 emitted through its `validator_failed` error key, so the expected output in `test_manager.py` is unchanged and the surrounding test cases remain consistent in asserting on `marshmallow`'s stock messages. Nothing under `src/` needed changing: the validators there are `marshmallow.validate.Range` and `OneOf`, which already raise. A raising validator also works on `marshmallow` 3, so the supported range is not narrowed, but the same silent acceptance can hit users who wrote their own boolean validators, hence the release notes entry. Signed-off-by: Leandro Lucarella <luca-frequenz@llucax.com>
…ss#1456) CI started failing on `v1.x.x` yesterday with three unrelated-looking config test failures. They come from the `frequenz-quantities` 1.0.2 release, which raised its `marshmallow` floor to 4 (frequenz-floss/frequenz-quantities-python#107) and thereby dragged `marshmallow` 4 into our environment for the first time. Marshmallow 4 dropped support for validators that return `False` instead of raising, which is what the config test dataclasses were doing, so the validation silently stopped happening. The fix is test-only, but there is a release notes entry because the same silent acceptance hits anyone who wrote boolean validators for their own config dataclasses. Heads-up for whoever reviews: the docs job on this PR will stay red, for a second and unrelated problem from the same `frequenz-quantities` 1.0.2 release. That one is a docstring in `frequenz-quantities` using a relative mkdocstrings cross-reference, which misresolves whenever it is rendered under a path other than the one it was written in, as happens here with `BaseConfigSchema` inheriting `QuantitySchema.TYPE_MAPPING`. It cannot be fixed on this side, only worked around by dropping documentation, and frequenz-floss/frequenz-quantities-python#170 already fixes it upstream. frequenz-floss#1455 is blocked on the same thing.
The code has required marshmallow 4 since 4ddc29d; the metadata still advertises 3.x, which cannot import.
Correct the dependency floor and document the required consumer upgrade.