add fs ephemeral and txoutset build logging - #1457
Merged
AloeareV merged 3 commits intoAug 20, 2026
Merged
Conversation
idky137
force-pushed
the
fix/release_0_8_0/slow_accumulator_build_and_fs_switchover
branch
from
August 18, 2026 15:41
344c8d3 to
f751366
Compare
AloeareV
approved these changes
Aug 20, 2026
This was referenced Aug 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finalised-state observability
Additive observability only — no control flow, status semantics, or routing behaviour changes.
Why
Two blind spots hit while syncing a mainnet node:
The accumulator rebuild is silent.
rebuild_tx_out_set_accumulatorlogs two lines up front and nothing again until it commits. In between it runs a fullspentcount pass and one full-chain block scan per shard — ~40 minutes on a mainnet DB — with no output. "Working" and "hung" look identical.chain_state: Readydoesn't mean the persistent DB is serving. During a long sync the router installs an ephemeral passthrough; it tracks the validator and reportsReadyexactly like a synced on-disk index. Nothing logged the install or release, and nothing exposed routing state. Containerised tests waiting onReadyhave been silently asserting against the passthrough.What
Rebuild progress — spent-entry count pass, per-shard start/spent-set-size/completion, and intra-shard height progress. Throttled to one line per 10s (reusing the existing
write_coreidiom, now a sharedPROGRESS_LOG_INTERVAL) so output stays bounded regardless of shard count — a memory-constrained container can reach 256 shards. Also covers the startupspentintegrity check and the incremental update path.A shard bisect logs at WARN, naming
accumulator_rebuild_memory_size— it means the budget under-provisioned and each split adds another full-chain pass.Routing mode — new
FinalisedStateMode(EphemeralConfigured/EphemeralRouted/Persistent), reported on every transition plus a one-shot "finalised state online" line:INFO router: finalised state switched to the ephemeral passthrough: reads are served from the
backing validator, NOT the persistent database ... mode=ephemeral(syncing)
INFO router: finalised state switched back to the persistent database mode=persistent
INFO router: finalised state online: reads are now served by the persistent database
The one-shot latch fires from both the ephemeral-release edge and the status poll — a restart against an already-current DB never installs a passthrough, so the release edge alone would never fire.
ephemeral_finalised_state = truenow warns at startup; that branch previously returned completely silently.Status line gains
fs_mode, so CI can gate onfs_mode: persistentinstead ofReady.Metrics (
prometheusfeature, off by default):zaino.db.finalised_ephemeral,zaino.db.accumulator_built_height,zaino.db.accumulator_rebuild_active.Notes for review
fs_heightwas dropped from the status line:db_height()is async,log_statusis a syncpub fn, and making it async is a breaking change. The height is available via the gauges.shard_ordinal/shard_totalfor logging), onepubaccessor chain mirroring the existingstatus()chain, oneAtomicBoollatch onRouter.zaino.chain.tip_heightwas emitted twice per sync iteration, once via a hard-coded string literal and once viaCHAIN_TIP_HEIGHT. Removed the literal. Happy to split this out.makers lintfails on this branch before this PR —--all-featuresenablestransparent_address_history_experimental, which doesn't compile (write_core.rs:404,:1541). Verified against a clean tree. Not addressed here.Verification
cargo fmt --check,cargo clippy --workspace --all-targetsand--features prometheusall clean (no new warnings);makers lint-boundary-conversionsclean; 586 tests pass. Two new tests cover the configured-ephemeral mode and the install→release transition.14 files, +584/−19.
Out of scope and worth separate issues: checkpointing the rebuild so restarts resume, relaxing NO_READAHEAD for the rebuild's sequential scans, and whether Router::status() should report Syncing while ephemeral-backed.