Repository navigation
HybridCache Preserve Activity in Factory Callback #6543
Description
Description
When using HybridCache and there is a cache miss on GetOrCreateAsync, the factory method is called but the parent Activity is not preserved and the factory method call has an event that is emitted with a different Trace ID
Reproduction Steps
var result = await cache.GetOrCreateAsync(
"my-key",
async ct =>
{
return await dbContext.MyTable.ToListAsync(ct); // emits its own span with a different trace ID
},
cancellationToken: cancellationToken);Expected behavior
Trace ID of the factory method should be the same as the parent activity. This screenshot shows the trace without the use of HybridCache:
Actual behavior
Trace ID is different than the parent activity
Regression?
Unsure
Known Workarounds
Setting the Activity.Current to the outer Activity.Current within the factory method works, but isn't feasible or reasonable to do everywhere and should work with the default implementation of HybridCache:
var parentActivity = Activity.Current; // get parent activity
await cache.GetOrCreateAsync(
"my-key",
async ct =>
{
Activity.Current = parentActivity; // set current activity to parent
return await dbContext.MyTable.ToListAsync(ct);
});Configuration
.NET SDK:
Version: 9.0.200
Commit: 90e8b202f2
Workload version: 9.0.200-manifests.a8e064d4
MSBuild version: 17.13.8+cbc39bea8
Runtime Environment:
OS Name: Mac OS X
OS Version: 15.5
OS Platform: Darwin
RID: osx-arm64
Base Path: /usr/local/share/dotnet/sdk/9.0.200/
.NET workloads installed:
There are no installed workloads to display.
Configured to use loose manifests when installing new manifests.
Host:
Version: 10.0.0-preview.4.25258.110
Architecture: arm64
Commit: c22dcd0c7a
.NET SDKs installed:
8.0.402 [/usr/local/share/dotnet/sdk]
9.0.100-rc.2.24474.11 [/usr/local/share/dotnet/sdk]
9.0.100 [/usr/local/share/dotnet/sdk]
9.0.102 [/usr/local/share/dotnet/sdk]
9.0.200 [/usr/local/share/dotnet/sdk]
10.0.100-preview.3.25201.16 [/usr/local/share/dotnet/sdk]
10.0.100-preview.4.25258.110 [/usr/local/share/dotnet/sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.8 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0-rc.2.24474.3 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.0 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.2 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 10.0.0-preview.3.25172.1 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 10.0.0-preview.4.25258.110 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 8.0.8 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0-rc.2.24473.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.0 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.1 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.2 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 10.0.0-preview.3.25171.5 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Microsoft.NETCore.App 10.0.0-preview.4.25258.110 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
Packages:
> Microsoft.Extensions.Caching.Hybrid 9.6.0 9.6.0
> Microsoft.Extensions.Http.Resilience 9.5.0 9.5.0
> OpenTelemetry.Exporter.OpenTelemetryProtocol 1.12.0 1.12.0
> OpenTelemetry.Extensions.Hosting 1.12.0 1.12.0
> OpenTelemetry.Instrumentation.AspNetCore 1.12.0 1.12.0
> OpenTelemetry.Instrumentation.EntityFrameworkCore 1.12.0-beta.2 1.12.0-beta.2
> OpenTelemetry.Instrumentation.Http 1.12.0 1.12.0
> OpenTelemetry.Instrumentation.Runtime 1.12.0 1.12.0
> Serilog.AspNetCore 9.0.0 9.0.0
> Serilog.Sinks.OpenTelemetry 4.1.1 4.1.1
Other information
No response
Activity
KalleOlaviNiemitalo commentedon Jun 27, 2025
This is caused by the same ExecutionContext / AsyncLocal<T> propagation problem as #5568 and #5648.
jodydonetti commentedon Dec 8, 2025
As a (temporary?) fix you may look into using the FusionCache based implementation of HybridCache, see here.
It has full support for OTEL, see here.
This is a result with the hierarchy respected:
It also has a backplane to keep the cache coherent even in a multi-node environment, and that also works with the HybridCache impl.
It's a 100% compatible drop-in impl replacement, no code changes required except for 1 line during setup.
Hope this helps.
NB: FusionCache creator here, just to be clear.
dmitry-azaraev commentedon Jan 28, 2026
Unfortunately it is not always so easy: FusionCache completely ignores
IHybridCacheSerializerFactory-ies andIHybridCacheSerializer<T>as well. Additionally it exposes ownIFusionCacheSerializerwhich initially looks simple, but ends what implementation must implement serialization ofFusionCacheDistributedEntry<T>instead ofTserializers and may be something else, implementing own entry framing. So... this simply doesn't work out of box in environment, where types has multipath serialization and serializable types are strictly enforced (e.g. no default serializer allowed). It also known nothing aboutIBufferDistributedCachewhich also might affect compatibility with something. So, yes, it is offer HybridCache interface, but it is far away from "100% compatible drop-in impl replacement".PS: FusionCache still featureful and nice. But serialization is suck with sucky interface.
jodydonetti commentedon Jan 29, 2026
Hi @dmitry-azaraev
Correct, the FusionCache adapter is 100% compatible with the HybridCache abstraction (
public abstract class HybridCache), and not the default HybridCache impl (internal class DefaultHybridCache).The serializers abstractions are specific for the default implementation, not for the abstraction.
Now, I admit that the serializers are kind of "tangential", meaning they are not part of the public HybridCache abstraction but some may think about them as part of the overall "contract".
But as soon as you think in terms of DI component registrations, I think everything becomes clearer: for example would you expect the same logger (
ILogger<HybridCache>) to be used? probably not, I think.Overall the main point is that the DI registration is the FusionCache one, not the HybridCache one: thanks to this I can support extra things like fail-safe, factory timeouts, the backplane, auto-recovery, etc.
Basically the adapter is a way of saying "you can take a FusionCache registration (setup, config, options, etc) and use that as HybridCache".
Maybe the wording could be different? Any idea?
Yes, the entire entry (where entry = value + extra data/metadata) is serialized: 99.999% it works without issues, but I can see your friction in (I suppose) implementing a custom serializer.
FWIW I've recently started thinking about moving in the future (v3?) to a different approach, similar to the default HybridCache impl.
What I mean is like a combination of a header + value, where:
There are pros/cons/tradeoffs to consider and it's not as straightforward as it may look like, mainly because of edge cases (evolving schemas, backward/forward compatibility, etc), but I'm exploring this.
Also, I'm exploring adding direct (optional) support for
IHybridCacheSerializer, so users may just say "use them kthxbye".If you have opinios or xperiences to share they would be welcome.
Yeah well... multipath serialization is (imho, of course) not the best idea: it's way better to have different caches (like named caches) where you can more clearly split different setups and configs. Mixing things at the serialization level is... I don't know, more error prone (again, imho).
I'd like to add support for
IBufferDistributedCacheas soon as I have time, but:Thanks.
Ok 🤷♂️, if you'd like to expand I'm interested in knowing more.
Btw, thanks for sharing, appreciate it!
PS: even though some people may find the back and forth interesting, I'd like not to bother others in this repo. So if you would like to keep talking about this (FWIW: I would be interested), maybe we can create a new discussion over the FusionCache repo and just put a link here, what do you think?
dmitry-azaraev commentedon Jan 29, 2026
@jodydonetti this definitely offtopic, but I'm reply just here but will try be short. Sorry, if I'm miss something important. Also, please, sorry, my previous message has been unnecessarily cruel.
As for serializatoin interfaces and part of abstraction - they landed in abstractions assembly near HybridCache, they also present in official documentation, which teach how to use them. So, I'm assume they are part of this concept generally, however, technically, I agreed, what they part of default implementation.
Let's skip any discussion for now, I'm will come eventually into your repository directly, if it will be needed. As for now, for me is better write serialization adapter to integrate them as I want... after few thoughts, I'm understand what I'm disappointed in HybridCache, but it is impractically go away from it right now, but this would be possible after some time. I think it should be nice addition to have such hybrid serialization adapter in FusionCache arsenal to make migration more easier. [I come here by ExecutionContext, but I'm miss not only this.]
FC's serialization framing/header is definitely can be done in HybridCache-way, e.g. when header maintained by library itself, but I see advantage of current FusionCache-way too, because JSON/other payloads doesn't get mixed with different encodings, so they can be easier inspect, by single decoder (however, I'm guess it is not what everyone do).
Serialization interface is "suck" because it lacks of clear documentation (or I'm doesn't found it, I'm tried but probably tired and miss that), surely there is exist reference implementation(s), and generally it is easy, but still interface by itself nowhere say what it always would do
byte[] Serialize<T>(FusionCacheDistributedEntry<T?> obj). Alsobyte[]allocations are generally problematic for me, so I'm prefer avoid them, but to be fair: in real context for me where cache is used - their cost neglible (however in same software system in major part of it (physically divided) similar naive approach leads to significant performance issues, so it just like act red flag on me, but i can live with it). But still new version can do everything even better. :)Thank you!
SylvainRollinetSoftcom commentedon Mar 16, 2026
Just got the same issue today...
It seems it only happens if a cancellation token is passed. So one workaround is to omit it, if acceptable.