test(corpus): exploration query family (q055-q059) — naive first-touch profiling#1506
Open
aaj3f wants to merge 4 commits into
Open
test(corpus): exploration query family (q055-q059) — naive first-touch profiling#1506aaj3f wants to merge 4 commits into
aaj3f wants to merge 4 commits into
Conversation
aaj3f
force-pushed
the
perf/r2rml-loadtable-meta-cache
branch
from
July 15, 2026 17:34
6750286 to
6d9fe55
Compare
…q013 The manifest declares three queries expected_status.virtual=error (q013 subquery, q034 transitive path, q051 subquery), but the error_boundary self-test's expected set listed only q034/q051, so the assertion no longer matched the manifest it validates. Add q013 to the expected set and to the native-stays-ok loop. Isolated from the exploration-family change per review: this corrects a pre-existing manifest/test drift and stands on its own.
The BI-derived corpus had zero pure-wildcard coverage, yet an
unconstrained ?s ?p ?o is the first query every human or LLM agent runs
on an unknown dataset. Add the five naive first-touch profiling queries
and two closed Tag variants (Wildcard, Exploration):
q055 wildcard peek SELECT ?s ?p ?o LIMIT 5
q056 triple count SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }
q057 schema discovery SELECT DISTINCT ?p ... LIMIT 100
q058 class census SELECT ?type (COUNT(?s) ...) GROUP BY ?type
q059 predicate histogram SELECT ?p (COUNT(*) ...) GROUP BY ?p ORDER BY DESC(?n) ?p LIMIT 10
Native oracles blessed; honest per-member status from the actual run
(virtual-sf01 vs native-sf01, hot cache, 1 rep, HEAD 6750286):
q055 1ms / 2.44s 5 rows
q056 0ms / 168.29s 1 row (predicate-less COUNT — full
materialization of every triple)
q057 900ms / 34.93s 100 rows
q058 147ms / 5.40s 16 rows
q059 2.6s / 32.38s 10 rows
All five complete ok within timeout — none DNF (the earlier DNF
prediction for q056/q058/q059 was overturned by measurement).
expected_status stays ok/ok (omitted) for all five.
Gate decisions:
- q055, q057 -> rows_only. Both truncate an unordered set with LIMIT, so
two correct engines return different-but-valid subsets that cannot be
hash-compared. q057 is empirical: it was the sole parity failure in
compare, and the mapping declares ~120 distinct predicates (119
rr:predicate + rdf:type) > the LIMIT 100, so the 100-subset is
nondeterministic.
- q056, q058, q059 -> full hash; all three matched native exactly. q059
is deterministic despite LIMIT (ORDER BY DESC(?n) ?p tiebreak).
- q056 timeout_s 300 (measured 168.3s hot; 300s is CI-stability headroom
at the 93%-of-180s fragility, not an expectation).
… north-star Records the naive first-touch family's measured walls and mechanisms in its own section, deliberately OUT of the north-star 44/51 accounting (pending AJ's ruling on whether wildcard first-touch queries join the ≤3s bar). Headline: q056's predicate-less COUNT at 168s is the real first-touch cost — full object materialization of every triple across all 16 tables (69,046 files, 63x the fetch_bytes of the object-pruned q057 over the same file set), a PR-6-family un-fused-COUNT gap, not a hard failure. All five complete ok; the earlier DNF prediction was overturned by measurement. Notes q056's 300s gate as CI-stability headroom (168/180 = 93% fragility) and flags q029's same-ratio gate as a candidate for the same treatment when F17 lands.
… + q056 F22 register row - q055/q057: add the '# Determinism:' header line the q029/q045 family carries, mirroring the manifest self-test's rows_only rationale (unordered LIMIT truncation) into the query files so the convention is uniform. - 04-findings-register.md: add F22 for q056's un-fused predicate-less COUNT (168.3s materializes every triple to count rows it never inspects), pointing at the fused Σ(record_count × POM) fix class so it sits beside its q038 sibling. Exploration finding — no burndown row (out of the north-star count).
aaj3f
force-pushed
the
test/exploration-corpus-family
branch
from
July 15, 2026 17:42
2ae2f99 to
fd06533
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds the five naive first-touch queries every human or LLM agent runs on an unknown dataset — the BI-derived corpus had zero pure-wildcard coverage. New closed
TagvariantsWildcardandExploration; native oracles blessed; honest per-member status taken from the actual run.?s ?p ?o LIMIT 5COUNT(*){?s ?p ?o}DISTINCT ?p LIMIT 100?s a ?typecensus GROUP BYMeasured virtual-sf01 vs native-sf01, hot cache, 1 rep, on the #1503 head (
6750286df). All five completeok— none DNF (the predicted DNFs for q056/q058/q059 were overturned by measurement);expected_statusstays ok/ok for all five.Mechanisms
All five hit the same whole-dataset breadth (~69,046 files across all 16 TriplesMaps — the maximum-fan-out first-touch case). The wall splits by what columns get materialized:
COUNT(*)has no manifest COUNT shortcut (not a single resolved table), so it materializes every triple: full object-column fetch,fetch_bytes63× that of q057 over the same file set. It reads object values it only needs to count — same family as q038's un-fused COUNT. A fused predicate-lessCOUNT(*)(Σ over TriplesMaps of rows × POMs from manifest record counts) would take it to sub-second (PR-6 family).?pneeded → object columns pruned → per-file footer/decode overhead across 69,046 files dominates. This is PR-2a (the per-file decode-wall) at maximum fan-out.?s a ?typeneeds only subject-key + per-tablerr:classconstant → 16 scans (one per TriplesMap), column-pruned. The cheapest wildcard, and why the census DNF prediction was wrong.LIMIT 5doesn't prune the scan but stops after one fact table (~7,679 files, 63 MB).Gate decisions
rows_only. Both truncate an unordered set withLIMIT, so two correct engines return different-but-valid subsets that can't be hash-compared. q057 is empirical — it was the sole parity failure incompare, and the mapping declares ~120 distinct predicates (119rr:predicate+rdf:type) > theLIMIT 100, so the 100-subset is nondeterministic.ORDER BY DESC(?n) ?ptiebreak).timeout_s300 — measured 168.3 s hot; 300 s is CI-stability headroom (168/180 = 93% is fragile), not an expectation. The wall is recorded honestly in the ROADMAP row.Commits
test(corpus): fix error-boundary drift — declared_error must include q013— a pre-existing test/manifest drift: the manifest declared q013virtual=errorbut theerror_boundaryself-test's expected set listed only q034/q051. Isolated as its own commit.test(corpus): add the exploration query family (q055-q059)— the two tags, 5.rqfiles, manifest entries, and blessed native oracles.docs(roadmap): exploration family section— measured walls + mechanisms, kept OUT of the north-star 44/51 accounting pending AJ's ruling on whether wildcard first-touch queries join the ≤3 s bar.Stacked on #1503 (
perf/r2rml-loadtable-meta-cache). Additive-only — no existing blessed oracle or manifest entry changed. Corpus self-tests green (7 passed).