Skip to content

stitch: migrate .csprj to native JSON and retain legacy reader - #561

Merged
janhubicka merged 44 commits into
mainfrom
agent/csprj-json-migration
Oct 21, 2026
Merged

janhubicka merged 44 commits into
mainfrom
agent/csprj-json-migration

Conversation

@janhubicka

@janhubicka janhubicka commented Oct 19, 2026 •

Copy link
Copy Markdown
Owner

Color-Screen 2.0 — native stitching project JSON

New .csprj files are plain UTF-8 JSON (org.colorscreen.stitch-project, schema 2). Each tile stores its filename, dimensions, shifts, position, measured angle/ratio, screen type, and only the schema-3 geometric mapping subtree. No redundant capture, MTF, color or render defaults and no embedded legacy .par text.

The old color_screen_stitch_project_version: 1 keyword-text reader remains supported; new saves always use JSON. The existing project FILE* API dispatches by signature, and the CLI stages complete JSON and atomically replaces the destination.

Safety and portability

  • Strict JSON field/version checks and transactional typed decode.
  • Bounds on grid dimensions, per-tile screen bitmaps (128 Mi cells), and aggregate bitmap cells (512 Mi).
  • UTF-8 source names and Windows Unicode project paths.
  • UTF-8 JSON reader independent of Qt; legacy keyword text retained.
  • Correct handling of Windows drive-letter absolute paths (including Z: and two-character drive-relative spellings).

Validation

  • Earlier Ubuntu x86-64 and ARM64 build/checking lanes succeeded with 130/130 tests.
  • New regression tests cover compact JSON round-trips, legacy text→JSON conversion, malformed nested state, filename portability and allocation safety.
  • Full unfiltered CI ran on 245be67: Windows GCC and Clang, macOS, sanitizers, and all Ubuntu build/checking jobs passed; Ubuntu distcheck failed solely because the checked-in generated src/libcolorscreen/Makefile.in omitted the new header from the source archive.
  • Fixed generated Makefile.in in commit cb838a2, matching the existing Makefile.am EXTRA_DIST entry. Full CI restarted on this head; review its distcheck result before merge.

Documentation

  • Update the 2.0 project and parameter wiki specifications, internal Qt/format tracking notes, README and NEWS.
  • The GitHub wiki publisher on main is a separate workflow, so these docs publish after merging.

Keep this PR unmerged until the final tests are reviewed.

Keep per-tile geometry independent of the Qt frontend and prepare a strict reusable serializer/deserializer for stitching projects.
…ile schema-3 parameters

Serialize project geometry and tile fields into typed JSON trees, reuse the schema-3 processing serializer, reject unknown or malformed fields and stage all decoded state before publishing.
…aders

Keep the existing mesh/map setup unchanged and reuse it when the new project loader populates native parameter state.
…aders

Keep the existing mesh/map setup unchanged and reuse it when the new project loader populates native parameter state.
Write complete schema-2 project JSON through the shared native codec. Preserve the version-1 text reader, reject unsafe grid sizes, and restore per-tile mapping/mesh state from parsed JSON.
Cover multi-tile typed roundtrips, deterministic encoding, malformed and unknown fields, transactional decode, and project FILE* save/load dispatch.
Create a genuine version-1 keyword project using the existing per-tile serializer, load it with the preserved reader, then verify new saves produce JSON. Reject malformed legacy dimensions before indexing fixed arrays.
…tion

Document exact schema-2 project keys, nested schema-3 parameters, location-relative image paths, bounds and existing version-1 text compatibility.
Replace obsolete ZIP project notes and the old schema-3 draft PR status with the implemented file-format architecture.
Replace obsolete ZIP project notes and the old schema-3 draft PR status with the implemented file-format architecture.
Replace obsolete ZIP project notes and the old schema-3 draft PR status with the implemented file-format architecture.
Replace obsolete ZIP project notes and the old schema-3 draft PR status with the implemented file-format architecture.
Replace obsolete ZIP project notes and the old schema-3 draft PR status with the implemented file-format architecture.
Include the new internal header in Automake EXTRA_DIST so source tarballs and distcheck builds compile.
…ites

Separate typed JSON document construction from FILE* output so callers can stage and atomically replace project files.
Allow callers to construct a complete native project document without touching an existing project file, keeping FILE* serialization available for tests and integrations.
Preflight the entire project codec and use shared UTF-8 atomic text replacement rather than truncating a destination before serialization succeeds.
Use fast-math-safe finiteness checks, validate positive image/screen dimensions and usable filenames, and qualify nested schema-3 errors with the offending tile index.
Expose the typed project codec consistently across DLL/static builds so standalone libcolorscreen tests can link on Windows.
Describe the new csprj writer and explicit preservation of the existing text reader alongside the cspar transition.
…r documents

Project tiles store a typed native screen type and schema-3 geometric mapping. Reuse strict .cspar validation by decoding geometry against default processing state, but reject rather than silently discard unrelated capture/color/MTF fields. Avoid repeated defaults in multi-tile csprj files.
…e state

Exercise final transforms and lens geometry round-trip, ensure no redundant capture or MTF data is serialized, and reject unknown nested domains or screen types transactionally.
Document the native schema's strict ownership boundary: project tiles do not contain irrelevant color/MTF/processing fields.
Specify the exact nested screen type and mapping fields, explicitly reject extra per-tile parameter domains, and update the internal migration comments.
Specify the exact nested screen type and mapping fields, explicitly reject extra per-tile parameter domains, and update the internal migration comments.
Specify the exact nested screen type and mapping fields, explicitly reject extra per-tile parameter domains, and update the internal migration comments.
Specify the exact nested screen type and mapping fields, explicitly reject extra per-tile parameter domains, and update the internal migration comments.
Specify the exact nested screen type and mapping fields, explicitly reject extra per-tile parameter domains, and update the internal migration comments.
A two-byte drive-relative token must not access name[2]; uppercase/lowercase Z are valid absolute drive roots. Keep project-relative paths independent of CWD.
Use native wide-character path opening for project files, consistent with .cspar APIs, while retaining CRT text-mode translation for historical CRLF project files.
…e paths

Check that the compact JSON codec preserves non-ASCII filenames and that drive-root paths, including Z, are not rebased into the project directory. Exercise short drive-relative input safely.
…format

Remove obsolete claims that current schema-3 is unmerged and that historical stitching projects were ZIP archives. Preserve the old keyword-text reader contract.
…format

Remove obsolete claims that current schema-3 is unmerged and that historical stitching projects were ZIP archives. Preserve the old keyword-text reader contract.
…format

Remove obsolete claims that current schema-3 is unmerged and that historical stitching projects were ZIP archives. Preserve the old keyword-text reader contract.
…format

Remove obsolete claims that current schema-3 is unmerged and that historical stitching projects were ZIP archives. Preserve the old keyword-text reader contract.
Bound eager known-screen bitmap allocations while allowing large source scans. Make common limits available to JSON and legacy readers.
…ng tiles

Apply a generous 128M-cell per-tile and 512M-cell aggregate safety ceiling to the JSON reader and writer, preventing integer overflow and memory exhaustion from malformed project dimensions.
The independent bitmap allocations created when loading legacy or JSON projects must stay inside a shared project-level resource ceiling.
Reject zero, negative, overlarge and cumulative screen-cell dimensions before allocating known-pixel maps; propagate a diagnostic for legacy files and account successful tile initialization.
Keep JSON and legacy project loading under the same cumulative bitmap resource ceiling.
… loading

Exercise writer rollback and strict staged JSON decode on giant screen sizes; verify original keyword projects enforce the same eager bitmap resource ceiling.
…al geometry

Document the enforceable 128 Mi-cell per-tile and 512 Mi-cell per-project safety limits, without conflating image pixel dimensions with known-screen masks.
Describe the separation of individual processing documents and stitching projects, identify both legacy text readers, and make the user-facing wiki references discoverable.
…der distribution

The repository tracks Automake-generated Makefile.in files. Although Makefile.am listed stitch-project-json.h in EXTRA_DIST, the unchanged generated Makefile.in omitted it from make dist, causing Ubuntu distcheck to fail compiling unittests.C in the release tarball. Mirror the EXTRA_DIST change so the source distribution contains the header.
@janhubicka
janhubicka marked this pull request as ready for review October 21, 2026 22:38
@janhubicka
janhubicka merged commit ed20a4a into main Oct 21, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant