Skip to content

[UM futures metrics] Silent create_time labeling change on 2026-06-25 (end-of-period -> start-of-period), duplicate 00:00 label across consecutive daily files #491

Description

@adebrief

Summary

The USD-M futures metrics daily archives silently changed their create_time labeling convention at 2026-06-25 00:00 UTC, from end-of-period labels to start-of-period labels. The change shifts the meaning of every row by 5 minutes, applies to all six data columns, and is not mentioned in any changelog or README. The REST API documentation for the corresponding /futures/data/* endpoints still describes timestamp as "End time of the period", which no longer matches observed behavior.

What changed

before 2026-06-25 from 2026-06-25
daily zip label range 00:05:0000:00:00 next day 00:00:0023:55:00
stock columns (sum_open_interest, sum_open_interest_value, count_toptrader_long_short_ratio, sum_toptrader_long_short_ratio, count_long_short_ratio) at create_time=T snapshot taken at T snapshot taken at T+5min
sum_taker_long_short_vol_ratio at create_time=T flow over [T−5min, T) flow over [T, T+5min)

The new labeling matches the convention the COIN-M metrics archives have used all along (CM daily zips are labeled 00:00 … 23:55 at least back to 2025), so this looks like a UM→CM harmonization — possibly related to the CM-UM integration effective 2026-06-30 — but we found no announcement covering the metrics archives.

Evidence

1. Label-set flip at the day boundary (trivially reproducible):

import io, zipfile, urllib.request, pandas as pd
def labels(day):
    u = f"https://data.binance.vision/data/futures/um/daily/metrics/BTCUSDT/BTCUSDT-metrics-{day}.zip"
    z = zipfile.ZipFile(io.BytesIO(urllib.request.urlopen(u).read()))
    d = pd.read_csv(io.BytesIO(z.read(z.namelist()[0])))
    ct = pd.to_datetime(d.create_time)
    return ct.min(), ct.max(), len(d)

print(labels("2026-06-24"))   # ('2026-06-24 00:05:00', '2026-06-25 00:00:00', 288)  <- old
print(labels("2026-06-25"))   # ('2026-06-25 00:00:00', '2026-06-25 23:55:00', 288)  <- new

Because the 06-24 file is end-labeled and the 06-25 file is start-labeled, both files contain a row labeled 2026-06-25 00:00:00, with different values — the old file's row is the snapshot at 00:00, the new file's row is the snapshot at 00:05. We checked 180 symbols: in ≥127/129 symbols with both files present, the two same-labeled rows differ in every column by exactly the magnitude of one normal 5-minute step (they would be identical if the column's convention had not changed).

2. Taker column content, verified against public aggTrades. sum_taker_long_short_vol_ratio can be reconstructed from the public aggTrades archives (taker buy volume / taker sell volume over a 5-minute window). Scanning window offsets:

  • 2024-06, 2025-06, and every month through 2026-06-20: archive row T matches the window [T−5min, T) with r = 0.998–1.000; the [T, T+5min) window gives r ≈ 0.0–0.2.
  • From 2026-06-25 on: archive row T matches [T, T+5min) with r = 0.998–1.000, and [T−5min, T) drops to noise.
  • Day-level pinning: 2026-06-24 is fully old-convention, 2026-06-25 is new-convention from 00:00 (checked in 6-hour sub-windows). Verified on BTCUSDT, ETHUSDT, SOLUSDT, RVNUSDT.

3. Open-interest content, verified against an independent 5-second poll of /fapi/v1/openInterest (Tardis derivative_ticker, free first-of-month data):

  • 2026-06-01: archive sum_open_interest at create_time=T matches the instantaneous OI at T (median relative difference 2.2e-5) vs 3.3e-4 when aligned at T+5min.
  • 2026-07-01: reversed — 1.5e-4 at T+5min vs 4.1e-4 at T.

Impact

  • Any pipeline that assumed a fixed create_time convention now carries a silent 5-minute content shift starting 2026-06-25 — e.g. features sampled "as of row T" now embed data from T+5min that was not observable at T.
  • The duplicated 2026-06-25 00:00:00 label (one row in each of two consecutive daily files, different values) breaks naive de-duplication.
  • The REST documentation for /futures/data/* (Long/Short Ratio, Taker Buy/Sell Volume, etc.) still says timestamp is "End time of the period, in milliseconds" — inconsistent with current REST and archive behavior.

Questions

  1. Can you confirm the convention change and its exact effective time (we measure 2026-06-25 00:00 UTC for UM)?
  2. Will this be documented in the changelog / README of the metrics dataset, and will the REST endpoint docs be updated to match the current behavior?
  3. Which convention is authoritative going forward — is a revert possible? Downstream consumers need this to be stable to interpret create_time at all.
  4. Is the duplicated 2026-06-25 00:00:00 row intentional (transition artifact), and which of the two rows should be considered canonical?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions