Skip to content

Restructure the pipeline into measure, author and compile - #47

Open
johnathanchiu wants to merge 7 commits into
mainfrom
doc/nest-modules
Open

johnathanchiu wants to merge 7 commits into
mainfrom
doc/nest-modules

Conversation

@johnathanchiu

@johnathanchiu johnathanchiu commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

The module map listed ten modules numbered 0–9, then noted they "are not ten
sequential parts of the pipeline". This groups them into the three phases a run
passes through, and moves the packages to match.

Moves, renames and docs. No logic changes: 613 passed, 2 skipped, the same as
before.

The phases

  • measure — establish what the room contains: scene understanding, image
    selection, object merging, grouping.
  • author — produce an asset for every measured object: the layout agent,
    object reconstruction.
  • compile — link the authored assets into one deliverable scene: room
    definition, QC checks, integration, simulation export.
  • across every phase — agent tools, harness, viewing and telemetry, editing.

Phases depend forwards only: author reads what measure established,
compile reads what author produced, nothing reads backwards.

The package moves

pipeline/scene_init/ingest, layout, {artifacts,flow,paths}.py  ->  pipeline/measure/
pipeline/scene_init/reconstruct, seed.py                       ->  pipeline/author/
pipeline/realism_authoring/author                              ->  pipeline/author/realism
pipeline/realism_authoring/live                                ->  pipeline/author/live
pipeline/room_qc/publish                                       ->  pipeline/compile/publish
pipeline/room_qc                                               ->  pipeline/compile/quality_check
pipeline/simulate                                              ->  pipeline/compile/simulate
pipeline/author/reconstruct/mesh_qc                            ->  .../mesh_quality_check
src/litereality_agent                                          ->  src/lrauthor

scene_init spanned two phases — ingest and layout establish what the room
contains, reconstruct and seed produce assets — so splitting it is the point of
the move, and why stages.py now imports ingest from measure and reconstruct
and seed from author.

realism_authoring/author becomes author/realism: the package is the realism
pass over an authored room, and author/author would have said nothing. qc is
spelled out so the two layers read as a pair — mesh_quality_check gates one
object, quality_check gates the assembled room.

The package rename is the import path only. The distribution name
(litereality-agent) and the litereality console script are what users type
and are unchanged.

Docs

doc/ becomes docs/, and assets/doc/ follows it. Directory names are
lowercase so a link is guessable from a module name, and the sim-ready pages
move out of Sim-Ready-intergration/, which was misspelled. Every page is now
linked from modules.md; four were not before.

integration/metadata.md was a stub of notes and is now a reference for the
format properties.py actually writes: the sidecar's fields, that lengths are
metres and everything is Z-up, why colliders are convex, and why density is per
cubic metre of the bounding box rather than of the mesh. What is not covered —
how an object is fixed to a wall — says so.

The numbers are gone from the map. They implied a sequence the doc itself
denied, and a name is stabler to reference than a position in a list. Three
modules were never stages: Agent tools and Harness are used by every phase that
runs a model, which is what makes the harness swappable, and Editing happens
after a run finishes. One module split — the Layout agent was both merging
objects built into each other and making the layout simulation-ready; the first
becomes Object merging under measure, where merge_boxes.py already lives.

Two things a path-based config hid

ruff's extend-exclude named the vendored preprocessing tree by its old path,
so the move silently un-ignored it and produced twelve lint errors that looked
like new bugs. The Modal deploy apps pass the package name as a string to
add_local_python_source("litereality_agent"), which fails at deploy time
rather than import time. Both repointed.

Not in this PR

The run directory layout still uses the old names — <scan>/scene_init/obj_stage,
<scan>/realism_authoring/room. Renaming those changes where artifacts live, so
a scan already on disk stops resuming and re-runs its stages. That wants a
compatibility fallback and its own PR.

The map listed ten modules numbered 0-9, then said they "are not ten
sequential parts of the pipeline". The numbers said otherwise. They are
gone, and the modules are grouped into the three phases a run actually
passes through -- measure, author, compile -- plus the layers that every
phase uses.

Three of the ten were never stages. Agent tools and Harness are used by
every phase that runs a model, which is what makes the harness swappable.
Editing happens after a run finishes.

One module split: the Layout agent was merging objects built into each
other, which establishes what the room contains, and making the layout
simulation-ready, which changes it. The first becomes Object merging under
measure, where merge_boxes.py already lives.

Doc directories are lowercase so a link is guessable from a module name,
and the sim-ready pages move out of the misspelled Sim-Ready-intergration.

No source is moved. The tree spans the phases unevenly and the new map
records how, but scene_init and realism_authoring are named in 114 files
while two open pull requests touch more than seventy each, so the move is
a follow-up once the map is agreed.
metadata.md was a stub of notes. It is now a reference for the format
properties.py actually writes: the sidecar's fields, that lengths are metres
and everything is Z-up, why colliders are convex, and why density is per cubic
metre of the bounding box rather than of the mesh. What is not covered -- how
an object is fixed to a wall -- says so.

doc/ becomes docs/, and assets/doc/ follows it. Inbound links from README.md
and room_ops/README.md are repointed.
Package moves and import updates only; no logic changes.

  pipeline/scene_init/ingest        -> pipeline/measure/ingest
  pipeline/scene_init/layout        -> pipeline/measure/layout
  pipeline/scene_init/{artifacts,flow,paths}.py -> pipeline/measure/
  pipeline/scene_init/reconstruct   -> pipeline/author/reconstruct
  pipeline/scene_init/seed.py       -> pipeline/author/seed.py
  pipeline/realism_authoring/author -> pipeline/author/realism
  pipeline/realism_authoring/live   -> pipeline/author/live
  pipeline/room_qc/publish          -> pipeline/compile/publish
  pipeline/room_qc                  -> pipeline/compile/qc
  pipeline/simulate                 -> pipeline/compile/simulate

scene_init spanned two phases: ingest and layout establish what the room
contains, while reconstruct and seed produce assets. Splitting it is the point
of the move, and it is why stages.py now imports ingest from measure and
reconstruct and seed from author.

realism_authoring/author becomes author/realism -- the package is the realism
pass over an authored room, and author/author would have said nothing.

ruff's extend-exclude pointed at the vendored preprocessing tree by its old
path, which silently un-ignored it; repointed.

613 passed, 2 skipped, same as before the move.
src/litereality_agent -> src/lrauthor, and every import with it. The
distribution name (litereality-agent) and the `litereality` console script are
what users type and are unchanged; this is the import path only.

uv.lock and the ruff per-file-ignores follow the new path. 613 passed,
2 skipped.
pipeline/compile/qc -> pipeline/compile/quality_check
pipeline/author/reconstruct/mesh_qc -> .../mesh_quality_check
docs/qc -> docs/quality_check

The two layers read as a pair now: mesh_quality_check gates one object,
quality_check gates the assembled room.

Also repoints links that still said ../QC/ and pipeline/scene_init/.
@johnathanchiu johnathanchiu changed the title Group the modules into the phases a run passes through Restructure the pipeline into measure, author and compile Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant