Skip to content

branch-4.1: [test](variant) Add compaction coverage for sparse path promotion - #70436

Open
eldenmoon wants to merge 1 commit into
apache:branch-4.1from
eldenmoon:branch-cir22588-41-tests
Open

eldenmoon wants to merge 1 commit into
apache:branch-4.1from
eldenmoon:branch-cir22588-41-tests

Conversation

@eldenmoon

Copy link
Copy Markdown
Member

What problem does this PR solve?

CIR-22588: after a full compaction, a VARIANT row's non-null sparse values (tag-labeling.text_info.open.tag_id/tag_name) became NULL in the materialized subcolumns and were removed from sparse, so the values disappeared from the output segment. The byte-ordered sparse writer (#70189) prevents the unordered input that made the compaction's raw-byte lookup miss the values.

What does this PR do?

Cherry-pick the test-only change from the master PR #70435 to branch-4.1. No production code change.

The suite writes the customer's six-row shape into a V2 table with variant_max_subcolumns_count=2, runs a full compaction, and asserts that row 1 still returns tag_id/tag_name/the parent object after compaction, with control rows 3/4 intact. It forces enable_variant_v2=true, since the issue is specific to the V2 sparse Map.

Verification

  • Passes on the fixed 4.1 V2 build; on the pre-fix build the after-compaction row 1 is NULL.
  • The same suite is added to the selectdb cloud hotfix branch and passed there.

@eldenmoon
eldenmoon requested a review from yiguolei as a code owner November 10, 2026 09:04
@hello-stephen

Copy link
Copy Markdown
Contributor

run cloudut

@eldenmoon

Copy link
Copy Markdown
Member Author

run external

@eldenmoon

Copy link
Copy Markdown
Member Author

run beut

@eldenmoon

Copy link
Copy Markdown
Member Author

run feut

CIR-22588: compaction promotes sparse paths to materialized subcolumns by a raw-byte lower_bound. Sparse keys written in component order were missed, filled with NULL, and removed from sparse, so the non-null values disappeared from the output file. The writer now publishes byte-ordered sparse keys; this suite verifies that after a full compaction the promoted tag_id/tag_name values are still present, using the customer's six-row shape and control rows.
@eldenmoon
eldenmoon force-pushed the branch-cir22588-41-tests branch from e5c34b5 to ff59663 Compare November 12, 2026 02:10
@eldenmoon

Copy link
Copy Markdown
Member Author

run buildall

@eldenmoon

Copy link
Copy Markdown
Member Author

run p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run vault_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run external

@eldenmoon

Copy link
Copy Markdown
Member Author

run nonConcurrent

@eldenmoon

Copy link
Copy Markdown
Member Author

run external

1 similar comment
@eldenmoon

Copy link
Copy Markdown
Member Author

run external

@eldenmoon

Copy link
Copy Markdown
Member Author

run nonConcurrent

@eldenmoon

Copy link
Copy Markdown
Member Author

run cloud_p0

@eldenmoon

Copy link
Copy Markdown
Member Author

run compile

@eldenmoon

Copy link
Copy Markdown
Member Author

run feut

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants