Adding multiple equities should resolve each requested ticker to the correct Symbol / SecurityIdentifier regardless of the order in which AddEquity() is called.
For example, given three valid equity map files representing three different securities:
life.1.csv
19980102,life,Q
20140203,life,Q
atyr.csv
20190701,life,Q
20240604,life,Q
20501231,atyr,Q
life.csv
20260129,life,Q
20501231,life,Q
the following securities should resolve to three different SIDs:
LIFE.1 -> old Life Technologies security
ATYR -> aTyr Pharma security, historically LIFE -> ATYR
LIFE -> current LIFE security
The result should not depend on the order of the calls:
self.AddEquity("LIFE.1", Resolution.Daily)
self.AddEquity("ATYR", Resolution.Daily)
self.AddEquity("LIFE", Resolution.Daily)versus:
self.AddEquity("LIFE", Resolution.Daily)
self.AddEquity("ATYR", Resolution.Daily)
self.AddEquity("LIFE.1", Resolution.Daily)Both call orders should resolve to the same three security identities.
Actual Behavior
The result currently depends on the order of AddEquity() calls.
When the securities are added in this order:
life1 = self.AddEquity("LIFE.1", Resolution.Daily)
atyr = self.AddEquity("ATYR", Resolution.Daily)
life = self.AddEquity("LIFE", Resolution.Daily)LEAN resolves LIFE.1 and LIFE to the same SID:
LIFE.1 internal ID: LIFE R735QTJ8XC9X
ATYR internal ID: LIFE X5SL0R29JU5H
LIFE internal ID: LIFE R735QTJ8XC9X
As a result:
even though life.1.csv and life.csv represent different equity securities with non-overlapping listing periods.
If the order is changed so that LIFE is added before LIFE.1:
life = self.AddEquity("LIFE", Resolution.Daily)
atyr = self.AddEquity("ATYR", Resolution.Daily)
life1 = self.AddEquity("LIFE.1", Resolution.Daily)the three securities resolve correctly to different SIDs.
This makes equity identity resolution dependent on the state of SymbolCache and on the order in which securities are registered.
Potential Solution
The behavior appears to be related to the fallback logic in SymbolCache.TryGetSymbol().
When an exact ticker is not present in the cache, SymbolCache performs a prefix search using:
var search = $"{ticker}.";and if exactly one matching cached key exists, it returns that Symbol.
This fallback is documented in the source as a backwards-compatibility mechanism for custom data symbols, for example:
However, the same logic also applies to equities.
Therefore, after:
has been added to the cache, a lookup for:
can find the single LIFE.* match and return the LIFE.1 equity before normal equity symbol generation / map-file resolution occurs.
One possible solution would be to restrict this dotted-symbol fallback to the security types for which it was intended, such as SecurityType.Base.
For example, conceptually:
var match = Symbols
.Where(kvp =>
kvp.Key.StartsWith(
search,
StringComparison.InvariantCultureIgnoreCase)
&& kvp.Value is not null
&& kvp.Value.SecurityType == SecurityType.Base)
.ToList();Another possible solution would be for AddSecurity / AddEquity to use an exact-cache lookup before creating an equity Symbol, instead of using the broader SymbolCache.TryGetSymbol() fallback behavior.
For equities, a cached permtick such as:
should not implicitly satisfy an exact request for:
unless the normal equity map-file / SID resolution determines that they represent the same security.
A regression test could also verify that the following two call orders produce identical SIDs:
and:
Reproducing the Problem
The issue can be reproduced using three equity map files.
Create:
Data/equity/usa/map_files/life.1.csv
with:
19980102,life,Q
20140203,life,Q
Create:
Data/equity/usa/map_files/atyr.csv
with:
20190701,life,Q
20240604,life,Q
20501231,atyr,Q
Create:
Data/equity/usa/map_files/life.csv
with:
20260129,life,Q
20501231,life,Q
Then run:
from AlgorithmImports import *
class LifeReuseAlgorithm(QCAlgorithm):
def Initialize(self):
self.SetStartDate(2024, 5, 20)
self.SetEndDate(2026, 3, 2)
self.SetCash(100000)
self.UniverseSettings.DataNormalizationMode = DataNormalizationMode.Raw
life1 = self.AddEquity("LIFE.1", Resolution.Daily)
atyr = self.AddEquity("ATYR", Resolution.Daily)
life = self.AddEquity("LIFE", Resolution.Daily)
self.Log("LIFE.1 internal ID: {}".format(life1.Symbol.ID))
self.Log("ATYR internal ID: {}".format(atyr.Symbol.ID))
self.Log("LIFE internal ID: {}".format(life.Symbol.ID))
if life1.Symbol.ID == life.Symbol.ID:
raise Exception(
"LIFE.1 and LIFE incorrectly resolved to the same SID: {}".format(
life.Symbol.ID
)
)Observed output:
LIFE.1 internal ID: LIFE R735QTJ8XC9X
ATYR internal ID: LIFE X5SL0R29JU5H
LIFE internal ID: LIFE R735QTJ8XC9X
Changing only the registration order to:
life = self.AddEquity("LIFE", Resolution.Daily)
atyr = self.AddEquity("ATYR", Resolution.Daily)
life1 = self.AddEquity("LIFE.1", Resolution.Daily)avoids the collision.
The expected behavior is that symbol identity should be independent of registration order.
A smaller unit test may also be possible directly against SymbolCache:
// Conceptual regression test
SymbolCache.Clear();
// Register an equity under a dotted permtick such as LIFE.1.
SymbolCache.Set("LIFE.1", oldLifeSymbol);
// An exact request for the separate equity ticker LIFE should not
// resolve to LIFE.1 through the custom-data dotted-symbol fallback.
Assert.IsFalse(SymbolCache.TryGetSymbol("LIFE", out var symbol));The exact assertion may need to be adapted to the intended SymbolCache API behavior, but the important condition is that an equity key such as LIFE.1 should not shadow the separate equity ticker LIFE.
System Information
LEAN ALGORITHMIC TRADING ENGINE v2.5.0.0
Mode: DEBUG
Python: 3.11.14
Execution: LEAN CLI / local backtesting environment
OS:
[Please add the host OS / Docker image information if available]
I can reproduce the problem consistently with the call order described above.
Checklist
Adding multiple equities should resolve each requested ticker to the correct
Symbol/SecurityIdentifierregardless of the order in whichAddEquity()is called.For example, given three valid equity map files representing three different securities:
the following securities should resolve to three different SIDs:
The result should not depend on the order of the calls:
versus:
Both call orders should resolve to the same three security identities.
Actual Behavior
The result currently depends on the order of
AddEquity()calls.When the securities are added in this order:
LEAN resolves
LIFE.1andLIFEto the same SID:As a result:
even though
life.1.csvandlife.csvrepresent different equity securities with non-overlapping listing periods.If the order is changed so that
LIFEis added beforeLIFE.1:the three securities resolve correctly to different SIDs.
This makes equity identity resolution dependent on the state of
SymbolCacheand on the order in which securities are registered.Potential Solution
The behavior appears to be related to the fallback logic in
SymbolCache.TryGetSymbol().When an exact ticker is not present in the cache,
SymbolCacheperforms a prefix search using:and if exactly one matching cached key exists, it returns that
Symbol.This fallback is documented in the source as a backwards-compatibility mechanism for custom data symbols, for example:
However, the same logic also applies to equities.
Therefore, after:
has been added to the cache, a lookup for:
can find the single
LIFE.*match and return theLIFE.1equity before normal equity symbol generation / map-file resolution occurs.One possible solution would be to restrict this dotted-symbol fallback to the security types for which it was intended, such as
SecurityType.Base.For example, conceptually:
Another possible solution would be for
AddSecurity/AddEquityto use an exact-cache lookup before creating an equitySymbol, instead of using the broaderSymbolCache.TryGetSymbol()fallback behavior.For equities, a cached permtick such as:
should not implicitly satisfy an exact request for:
unless the normal equity map-file / SID resolution determines that they represent the same security.
A regression test could also verify that the following two call orders produce identical SIDs:
and:
Reproducing the Problem
The issue can be reproduced using three equity map files.
Create:
with:
Create:
with:
Create:
with:
Then run:
Observed output:
Changing only the registration order to:
avoids the collision.
The expected behavior is that symbol identity should be independent of registration order.
A smaller unit test may also be possible directly against
SymbolCache:The exact assertion may need to be adapted to the intended
SymbolCacheAPI behavior, but the important condition is that an equity key such asLIFE.1should not shadow the separate equity tickerLIFE.System Information
OS:
I can reproduce the problem consistently with the call order described above.
Checklist
masterbranch