HBASE-30256 FuzzyRowFilter mishandles the fuzzy-mask encoding on the …

[liuxiaocs7]

…no-unsafe path

• see: HBASE-30256

[Copilot]

Pull request overview

This PR fixes and hardens FuzzyRowFilter mask handling across the “unsafe” and “no-unsafe” execution paths (including cross-platform / cross-version serialization), ensuring mask semantics are preserved during scanning, hinting, and (de)serialization.

Changes:

• Normalize and stabilize internal mask/key preprocessing in FuzzyRowFilter so masks are not mutated during scanning and wire format stays platform-independent ({0,1}).
• Ensure hinting uses the correct mask semantics on both paths by converting masks as needed without corrupting the satisfy logic.
• Add targeted tests for (a) serialization/equals stability after scanning and (b) end-to-end “no-unsafe” behavior, including interoperability with unsafe-encoded peers.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
hbase-client/src/main/java/org/apache/hadoop/hbase/filter/FuzzyRowFilter.java	Normalizes mask handling, prevents scan-time mutation, converts masks appropriately for satisfies vs hinting, and emits canonical {0,1} masks on serialization; updates equals/hashCode to align with serialized semantics.
hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilterWoUnsafe.java	New end-to-end tests that force the no-unsafe path and validate matching, hinting (forward/reverse), and interoperability with unsafe-encoded serialized masks.
hbase-server/src/test/java/org/apache/hadoop/hbase/filter/TestFuzzyRowFilter.java	Adds regression tests ensuring serialization and equals/hashCode remain correct even after filterCell has been invoked.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.