Skip to content

Fix/frozen recording drift - #9

Merged
hongdy22 merged 7 commits into
mainfrom
fix/frozen-recording-drift
Sep 10, 2026
Merged

hongdy22 merged 7 commits into
mainfrom
fix/frozen-recording-drift

Conversation

@hongdy22

Copy link
Copy Markdown
Collaborator

No description provided.

deyao hong added 7 commits September 10, 2026 05:39
The rename before the release lengthened the fixed boundary by two characters.
httpbin echoes the Content-Length it received, and data/responses.json records
129 and 216 for the single- and two-field cases, which is what the shorter
boundary produces; the same pass rewrote the boundary inside the recorded bodies
but not the lengths beside them, so the 15 multipart cases could not be answered
by any submission.

Restoring both sides returns all 437 records to agreeing with their own
body_len. Stage 2 with State A as the submission: 2832/2832.
data/baseline.json records the consumer printing 1.0.20|26|2|32|dbcb, which is
the first two bytes of blake2b("RepoMorphBench"); the message the consumer
compiles today hashes to ff67, so the six checks that compare its output with
State A's could not pass. The baseline is not edited.
Six recorded responses carry public/index.html with integrity= rewritten to
audit=, which is four bytes shorter than the file State A serves and than the
body_len recorded beside it. Restoring the attribute makes all six byte
identical to repo/public/index.html in environment/original.tar.gz.

Stage 2 with State A as the submission: 6760/6760.
Every file in the sample tree is 14 bytes and every entry carries the same
pinned mtime, so sort=size and sort=date leave most of a listing tied, and a
tied run is ordered by whatever order the filesystem enumerated the directory
in: the same tree, built by tree.py, lists in three different orders on three
filesystems of one host. The capture froze one of them, so on any other
filesystem those cases fail whatever the submission answered.

Tied rows are put in name order on both sides as the recordings are read. The
sequence of sort keys is still compared exactly, and so is the dirs-first
grouping where the flag asks for one; name sorts are untouched, and the
recording is not edited.

Measured with State A as the submission. On a filesystem that enumerates as the
capture did: 12466/12466, before and after. With the sample tree on a tmpfs,
which enumerates differently: 269 failing before (listing 249, archives 20), 20
after. The 20 are the archive cases and are a different question -- the capture
recorded the archive of links/ with one member and this run builds it with
fifteen, so the two disagree about which entries a walk of the tree yields.
Reproduced, not addressed here.
Four assertions that would have caught the four above on a push: fw01's recorded
lengths against its own bodies, fw02's recorded static pages against the tree in
original.tar.gz, build01's probe against the string it hashes, and fw05's sorted
listings against a shuffled enumeration order.

tools/grade_state_a.py runs the stronger form -- State A submitted to its own
stage 2, where every check it fails is one no submission can pass. That needs
the stage images, so it stays out of CI and is run by hand.
tar::Builder::append_dir_all follows symlinks and stops at the first one it
cannot stat, so links/dangling ends the walk wherever the filesystem happened to
enumerate it. The capture recorded the archive of links/ with one member; the
same tree on a tmpfs yields fifteen. What the archive held was a property of the
host rather than of the tree, and the recorded bytes were unreachable on any
filesystem that enumerated differently.

The two archive sessions serve `archivable` instead -- the sample tree built
without links/dangling -- so the archive is a function of the tree again. The
symlink to a file and the symlink to a directory stay, since they are what the
tar and zip writers each answer their own way and neither cuts a walk short.
Every other session still serves the dangling one, so the listing and error
cases that ask about it are untouched. fw05's recording was refreshed against
the new tree.

Measured with State A as the submission, sample tree on a tmpfs: 20 failing
archive checks before, 4 after. Those 4 are tar.gz body lengths and are the next
commit. On a filesystem that enumerates as the capture did: 12466/12466.
A tar.gz body's length is the length those members happened to compress to in
the order the archiver walked them, which is the order the filesystem returned
them in. With the tree fixed the tar and zip cases are byte-identical across
filesystems; the gzipped ones hold the same members either way, and only their
length still says which order they were reached in.

test_body_length is routed over a new `sized_case` axis, which is every case
whose body has a length worth comparing -- the six tar.gz cases are not, and the
corpus goes from 12466 scored checks to 12460. What the stream contains is
graded as before by the archive facts, which read it back out: ok, kind, count,
names, members and digest, none of which the walk order moves.

Measured with State A as the submission, sample tree on a tmpfs: 4 failing
before, 0 of 12460 after. On a filesystem that enumerates as the capture did:
0 of 12460.
@hongdy22
hongdy22 merged commit ecbb692 into main Sep 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant