Repository navigation
perf: serve year-long financial series from monthly caches - #318
Merged
tekwani merged 1 commit intoOct 19, 2026
Merged
Conversation
The Project Financials tabs rebuilt a year of hourly data on every render: metrics/hashrate interval=1d re-scanned ~8,760 hourly buckets (and a year of pool samples with pool=true) per request, and a phase revenue-summary re-fetched the full-range by-meter series plus the manual consumption log every time. With the three windows the page fires at once, each tab cost several concurrent year-long scans on the ork. Cache the 1d day rows month by month like getMonthlyHashrate already caches its calendar months (ended, fully covered months only; partial edges and the running month recompute), cache the phase consumption hourly rows per UTC month with an explicit drop from saveEnergyConsumption so manual edits are never served stale, and let concurrent windows share one in-flight rollup per month in getDailySeries instead of each re-scanning the same cold months. Phase revenue-summary keeps its refresh semantics, so its coverage accounting is unchanged and only the fetch cost moves. Adjacent cold phase months fetch as one span rather than one query pair per month, a fetch that straddles a manual write no longer re-stores its pre-write rows over the invalidation (and a request arriving after the write reads fresh instead of riding the stale fetch), and a window ending exactly on a month boundary now answers the same from a warm cache as from a cold one - the zero-width edge month used to be dropped (getDailySeries) or rejected with ERR_INVALID_DATE_RANGE (the month rollup) once its neighbours were cached.
paragmore
force-pushed
the
feature/financials-month-caches
branch
from
October 19, 2026 12:34
9f4d156 to
b20e32b
Compare
Contributor
Author
|
Pushed a hardening pass after a deeper review and a differential QA run (develop vs this branch driven through ~200 randomized scenarios - timezones, data gaps, phases, concurrent window bursts, manual writes between reads - with every response deep-compared). Fixes folded in:
The differential run is clean apart from the boundary-window case above, where develop itself returns different answers for the same query depending on cache warmth and this branch consistently returns the cold (complete) one. Full unit and integration suites pass; new tests cover the boundary windows, the write-vs-fetch race, and the run batching. |
mukama
approved these changes
Oct 19, 2026
tekwani
approved these changes
Oct 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Project Financials phase tabs take 13-17s to render (Total Site ~6s). Each tab fires three revenue-summary windows plus a metrics/hashrate interval=1d call over 12 months, and almost all of that work was repeated on every request: the 1d hashrate series rebuilt ~8,760 hourly buckets (plus a year of pool samples with pool=true) per call, and a phase revenue-summary re-fetched the whole-range by-meter series and manual consumption log every time by design. The concurrent windows then each re-scanned the same cold months independently.
This reuses the pattern the 1M rollup already has. getHashrate interval=1d caches day rows month by month (ended, fully covered months only; edge slices and the running month always recompute; overwriteCache refreshes). Phase consumption caches its hourly rows per UTC month, and saveEnergyConsumption drops that cache on a confirmed write, so manual edits are never served stale; store-side backfills are bounded by the same 12h TTL the other monthly rollups accept. getDailySeries additionally single-flights per-month rollups so the overlapping windows a tab fires at once compute each cold month once. The phase revenue-summary keeps its always-refresh semantics, so coverage/missing-hours accounting is unchanged - only the fetch underneath got cheap.
One side effect worth calling out: phase consumption now fetches per month, so months at or after energyRollup.sinceTs read the energy-1h rollup even when the request started before it, where previously one pre-rollup start forced the whole span onto the legacy stat path. The two sources are meant to agree; the rollup is the higher-fidelity one.
Bench on the real handler stack against a simulated ork (span-proportional, serialized latency; responses byte-identical before/after): warm tab renders drop from ~12s/31s/8s to ~4s/7s/4s for total/phase1/phase1_5, and cold renders improve ~25% from the single-flight dedup. Full unit and integration suites pass; new tests cover the caches (scope keys, edge months, running month, tombstones, pool rows, invalidation, concurrent dedup, failure retry).