Skip to content

Exclude unwritten capacity from trajectory extraction - #921

Merged
david-zwicker merged 1 commit into
zwicker-group:masterfrom
antonsoo:fix/preallocated-trajectory-extraction
Oct 20, 2026
Merged

david-zwicker merged 1 commit into
zwicker-group:masterfrom
antonsoo:fix/preallocated-trajectory-extraction

Conversation

@antonsoo

@antonsoo antonsoo commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Extracting a trajectory from a partially filled preallocated file includes unwritten frames. Time-range searches also use the padded timestamps, which need not be sorted. For example:

import tempfile
from pathlib import Path
from pde import FileStorage, ScalarField, UnitGrid

with tempfile.TemporaryDirectory() as directory:
    storage = FileStorage(Path(directory) / "run.hdf5", max_length=8)
    field = ScalarField(UnitGrid([4]))
    storage.start_writing(field)
    for time in (1, 2, 3):
        field.data[:] = time
        storage.append(field, time)
    storage.end_writing()
    selected = storage.extract_time_range((1, 2))
    assert list(selected.times) == [1, 2]
    storage.close()

Limit time searches and component extraction to len(storage). The HDF datasets keep their existing lazy access and allocated capacity.

The regression covers partial, full and extensible files, compressed and uncompressed storage, time bounds, and scalar/vector component copies. Native stopped diffusion and wave trajectories also agree with independently copied callback snapshots; existing correct results and solver states stay unchanged.

File-storage tests, the storage/tracker/diffusion/wave checks, and project hooks pass. Whole-project Mypy reports the same diagnostics as unchanged upstream.

Rebased after #922 merged, retaining its complete storage-finalization regressions. The updated branch passes 166 storage, tracker, diffusion and wave tests (26 optional skips) and all configured hooks. All 576 extraction records and all 100 close/resume records remain exactly equal to the previously validated composition, including the independent callback/Fourier references and solver controls. Mypy still reports the same 169 baseline diagnostics.

- Limit time searches and component extraction to stored frames.
- Cover partial, full and extensible HDF files and scalar/vector components.
@antonsoo
antonsoo force-pushed the fix/preallocated-trajectory-extraction branch from 724109c to b4b4aad Compare October 19, 2026 18:56

@david-zwicker david-zwicker left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@david-zwicker
david-zwicker merged commit bb18877 into zwicker-group:master Oct 20, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants