You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
perf(cache)!: share concurrent inner reads and serve an outage from the local tier - #223
On a local miss, concurrent single-key reads of one key wait for a single inner-tier read and share its result or failure. This covers GetAsync, GetCacheEntryAsync, the hash reads, and the read GetOrAddAsync makes before its lock.
Multi-key reads (GetAsync(CacheKey[]), GetCacheEntriesAsync) still read every missing key themselves: they go out as one MGET, and splitting that per key is a separate change.
Hash reads fetch the whole entry once, and each caller filters its own fields.
Cancellation:
A caller that cancels stops waiting.
The shared read runs on its own token, cancelled only once every waiting caller has cancelled.
GetOrAddAsync keeps a value the inner tier refused.
The generator still runs under the local lock, and lock settings mean what they did.
When the inner write fails, the generated value is kept locally for LocalMaxExpirationDisconnected.
So the callers waiting on the lock reuse it instead of each running the generator again, one after another.
An outage is served from the local tier by default.
CacheOptions.ConnectionMonitorEnabled and UseLocalOnlyWhenDisconnected now default to true.
New ClearLocalOnReconnect, also true by default: when a Redis Pub/Sub broadcast reconnects, the local tier is cleared, since invalidations published during the outage never arrived. Redis Streams replay them, so the local tier is kept.
A Redis Streams consumer checks after a reconnect whether entries past its last delivery were trimmed, and treats a consumer group or stream removed while in use as a loss too. Either way, every local entry on that topic expires, in each cache that uses it. The topic signals it through the new IEventSubject<T>.Invalidate().
The monitor setting is app-wide, so the broadcast providers' connection monitor turns on too.
A refresh is broadcast as CacheRefreshed.CacheEventPublisher.CacheRefreshedAsync sent the CacheRemoved type. The library's receivers treat both alike; only a subscriber filtering by type sees the change.
GetOrAddAsync miss path, before and after:
flowchart LR
A[callers miss locally] --> B[shared inner read]
B -->|hit| C[all return the value]
B -->|miss| D[local lock]
D --> E[generator runs once]
E --> F{inner write}
F -->|ok| G[kept locally, waiters hit]
F -->|refused| H[kept locally for the disconnected cap, waiters hit]
Loading
Breaking
A disconnected inner tier now serves the local copy instead of answering misses, and the connection monitor is on by default. For the previous behaviour, set UseLocalOnlyWhenDisconnected = false, or ConnectionMonitorEnabled = false.
IEventSubject<T> gains Invalidate(). A custom subject passed to the RedisPubSubTopic or RedisStreamsTopic constructor must implement it, calling OnEventsMissed() on each observer that implements the new public IMissedEventsObserver, as the library's change tokens do.
IMultilayerCacheOptions gains ClearLocalOnReconnect, which a custom implementation must add.
Tests
InFlightTests cover the coalescer:
one run per key
failure shared
a leaving caller doesn't cancel the others
cancelled once all callers leave
a fresh run after an abandoned one
InnerTierOutageTests cover:
shared cold reads (cache and hash, different fields)
one generator run when the inner write is refused (cache and hash)
local copy served while disconnected by default
clear on reconnect (default, on, off), only over Pub/Sub: kept over Streams and without a broadcast connection
Streams loss (real Redis): trimmed during an outage, replayed without loss, already-read entries trimmed, stream removed while in use, and the topic expiring its entries
the monitor default
Red-checked: each of the six changes, reverted on its own, fails its test.
Existing tests that assumed the old defaults now set them explicitly.
🔎 Maintainer heads-up: automated triage flagged this PR as potentially material, so it may need a signed CLA in addition to the DCO sign-off.
Strong signals
adds public API surface (PublicAPI.Unshipped.txt in src/UiPath.Caching)
This is advisory only — the bot does not decide. Please judge against the CLA criteria (material, product-critical, patent-sensitive, corporate contributor, broad commercial use). Note that thresholds can be gamed by splitting PRs, so use your judgement.
If a CLA is needed → add the cla-required label (a contributor comment with signing steps is posted automatically).
If it is not needed → replace needs-cla-review with cla-not-required so later pushes don't re-flag it.
This coalescer is only used by scalar reads. Both public batch overloads still send misses directly through _innerCache.GetCacheEntriesAsync (GetInnerAsync and GetCacheEntriesInnerAsync), so concurrent batch reads—or a batch read racing a scalar read—still issue multiple inner reads for the same key. That contradicts the stated coverage of GetAsync/GetCacheEntryAsync; route batch misses through the same per-key flights or explicitly narrow the advertised behavior.
The coalescing identity omits the caller-supplied CachePolicy. Both cache APIs accept a policy per read, and the shared operation uses the first caller's policy to populate L1 (MultilayerCache.cs:1248 and MultilayerHashCache.cs:574). Concurrent callers requesting different LocalExpiration values therefore silently apply whichever policy joined first; the existing caller-policy contract is explicitly covered in MultilayerCachePerNamePolicyWiringTests.cs:251-269. Include the effective policy in the flight identity, or coalesce only the inner fetch and apply each caller's local policy after awaiting it.
This check-and-reset is not atomic with the failure handler. If recovery observes all states connected, then another state fails and writes _down = 1 before this Exchange, the exchange clears that new outage marker and raises Recovered while a state is down; the later restore will not raise recovery, so outage entries can remain uncleared. Serialize failure/recovery transitions or revalidate without losing a concurrent failure marker.
Document conditional cache clearing with Redis broadcasting
docs/reference/settings.md:228
This setting is not inert for the in-memory provider when Redis broadcasting is enabled. InMemoryCacheProvider passes the Redis topic provider into MultilayerCacheBase; because that provider implements IConnectionState, the new app-wide monitor default subscribes to it and this option clears the in-memory cache after broadcast recovery. Document that conditional behavior rather than telling users the option has no effect.
Correct inaccurate comment about Redis reconnect cache clearing
This comment is inaccurate when the in-memory provider uses Redis broadcasting: the topic provider is monitored as an IConnectionState, so reconnecting it clears this provider's local cache by default. Describe that conditional effect instead of calling the option inert.
_down starts at zero and is not set until a failure event or an evaluation of the lazy aggregate. If a monitored source is already disconnected when this monitor is created and emits a restore before the first poll/IsConnected read, RaiseIfRecovered suppresses the recovery. A cold L2 read can populate L1 during that initial broadcast outage without evaluating this monitor, so the stale entry is then never cleared. Initialize the outage marker from the initial aggregate state and cover an initially-down, immediately-restored source.
Abandoned flights leak indefinitely when work ignores cancellation
src/UiPath.Caching/InFlight.cs:92
When the last waiter leaves, this cancels the work but leaves the flight in _flights. If an inner implementation ignores cancellation and hangs, every abandoned unique key retains its key, state, task, and cancellation source indefinitely unless that exact key is read again. Conditionally remove the key/flight pair as soon as the waiter count reaches zero (while still allowing late work completion to remove safely).
…he local tier
On a local miss, concurrent reads of one key now wait for a single
inner-tier read and share its result or failure: GetAsync,
GetCacheEntryAsync, the hash reads, and the read GetOrAddAsync makes
before its lock. Reads share only when their key, type and local
lifetimes match. A caller that cancels stops waiting; the shared read is
cancelled once every caller waiting on it has cancelled, and its local
copy is committed only while a caller still waits.
When the inner tier refuses a GetOrAddAsync write, the generated value is
kept locally for LocalMaxExpirationDisconnected, so the callers waiting on
the local lock reuse it instead of each running the generator again.
CacheOptions.ConnectionMonitorEnabled and UseLocalOnlyWhenDisconnected
default to true, so an outage serves and keeps values locally, and a hit
read while the tier is down is kept for the disconnected cap. The new
ClearLocalOnReconnect, true by default, clears the local tier once a
Redis Pub/Sub broadcast is connected again, since invalidations
published meanwhile never arrived. Redis Streams replay them, so their
local tier is kept. RedisConnector.IsConnected is false only when a
multiplexer exists and reports itself down, so a cold connector still
sends its first command.
A Redis Streams consumer now checks after a reconnect whether entries
past its last delivery were trimmed, and treats a group or stream
removed while in use as a loss too. Either way, every local entry on that
topic expires, in each cache that uses it, through the new
IEventSubject<T>.Invalidate().
Multi-key reads stored each hit locally as a key-value pair, so no later
read found it; they now keep the value. A refresh is broadcast as
CacheRefreshed; it was sent as CacheRemoved.
Closes#222.
BREAKING CHANGE: a disconnected inner tier now serves the local copy
instead of answering misses, and the connection monitor is on by
default. Set UseLocalOnlyWhenDisconnected or ConnectionMonitorEnabled to
false for the previous behavior. IEventSubject<T> gains Invalidate(); a
custom subject passed to a topic constructor must implement it, calling
the new public IMissedEventsObserver on its observers. IMultilayerCacheOptions
gains ClearLocalOnReconnect, which a custom implementation must add.
Signed-off-by: Cosmin Staicu <cosmin.staicu@uipath.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cla-not-requiredMaintainer reviewed: no CLA required for this contribution
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
github-actions
Bot
added
the
needs-cla-review
Closes #222.
What changes
Concurrent inner reads are shared.
GetAsync,GetCacheEntryAsync, the hash reads, and the readGetOrAddAsyncmakes before its lock.GetAsync(CacheKey[]),GetCacheEntriesAsync) still read every missing key themselves: they go out as one MGET, and splitting that per key is a separate change.GetOrAddAsynckeeps a value the inner tier refused.LocalMaxExpirationDisconnected.An outage is served from the local tier by default.
CacheOptions.ConnectionMonitorEnabledandUseLocalOnlyWhenDisconnectednow default totrue.ClearLocalOnReconnect, alsotrueby default: when a Redis Pub/Sub broadcast reconnects, the local tier is cleared, since invalidations published during the outage never arrived. Redis Streams replay them, so the local tier is kept.IEventSubject<T>.Invalidate().A refresh is broadcast as
CacheRefreshed.CacheEventPublisher.CacheRefreshedAsyncsent theCacheRemovedtype. The library's receivers treat both alike; only a subscriber filtering by type sees the change.GetOrAddAsyncmiss path, before and after:flowchart LR A[callers miss locally] --> B[shared inner read] B -->|hit| C[all return the value] B -->|miss| D[local lock] D --> E[generator runs once] E --> F{inner write} F -->|ok| G[kept locally, waiters hit] F -->|refused| H[kept locally for the disconnected cap, waiters hit]Breaking
A disconnected inner tier now serves the local copy instead of answering misses, and the connection monitor is on by default. For the previous behaviour, set
UseLocalOnlyWhenDisconnected = false, orConnectionMonitorEnabled = false.IEventSubject<T>gainsInvalidate(). A custom subject passed to theRedisPubSubTopicorRedisStreamsTopicconstructor must implement it, callingOnEventsMissed()on each observer that implements the new publicIMissedEventsObserver, as the library's change tokens do.IMultilayerCacheOptionsgainsClearLocalOnReconnect, which a custom implementation must add.Tests
InFlightTestscover the coalescer:InnerTierOutageTestscover: