feat(plugins): backfill section bloom - #81
Open
317787106 wants to merge 7 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR adds an offline Toolkit command for rebuilding historical SectionBloom data:
The command reconstructs the
section-bloomdatabase from existing records intransactionRetStore. It supports an inclusive block range, automatically detects the available historical range, and limits the range to the latest solidified block.Processing is divided by 2,048-block bloom sections and can run concurrently. The command also provides terminal progress, periodic progress logs, execution statistics, parameter validation, and resumable/idempotent processing.
This PR also registers the command in the database toolkit, adds the required
chainbasedependency, and documents its usage and operational requirements.Why are these changes required?
If
isJsonRpcFilterEnabledwas disabled while historical blocks were processed, the corresponding SectionBloom data is unavailable after the option is enabled. As a result,eth_getLogscannot efficiently filter those historical blocks by contract address or topics.This command allows operators to rebuild the missing SectionBloom data directly from the existing transaction result database without replaying the blockchain.
This PR has been tested by:
DbBackfillBloomTestFollow up
None.
Extra details
The node and any other processes accessing the database must be stopped before running the command. Historical transaction storage must have been enabled when the target blocks were processed.
The command can be safely rerun after interruption, but multiple backfill processes must not operate on the same database concurrently.