Consolidation: an older same-day decision overwrites its newer amendment when it arrives second (eval case) #5938

Summary

Consolidation lets an older fact overwrite a newer observation when both are about the same rule, both were stated on the same day, and the older one arrives second. Prompt rule 10 ("RECENCY IS mentioned_at, NOT ARRIVAL ORDER") covers exactly this, and the prompt carries both timestamps. gemini-3.1-flash-lite still gets it wrong every time (6 of 6 replays of the exact prompt at temperature 0, and every variant below). gemini-3.8-flash gets it right with the identical prompt.

The existing recency-is-when-stated scenario (#2550) passes because it is the easy form: facts five days apart, plain wording. This is the hard form, and it shows up in real coding banks.

How it happens

A coding bank imports two past conversations at once:

1. 09:48, session-01 decides the overage rule: bill in 50-unit blocks, no free units.
2. 10:48, session-02 amends it: the first 25 overage units per month are free, applied before block rounding.

The two retains run in parallel, and session-02's facts land first (10:48:11.7 against 10:48:12.2). Consolidation creates the correct observation from session-02, then gets session-01's fact, which fact extraction worded as "The metering policy for overage_charge was updated to use prorated quotas and block-based billing … No waivers for small overages are permitted."

The model rewrites the observation to: "The metering policy for overage_charge was updated on 2026-10-21 to use prorated quotas and block-based billing, replacing the previous 25-unit grace allowance policy."

The bank now holds the superseded rule as current. Recall and reflect serve it, and a coding agent got told to remove the grace allowance it had just correctly implemented.

What we tried (exact prompt replayed, temperature 0)

No prompt wording moved the small model. It treats everything under "New facts" as the latest news.

Proposed eval scenario (evals/test_09_consolidation.py)

Scenario(
    id="recency-same-day-amendment-arrives-first",
    origin="an amendment and the rule it amends, stated an hour apart, retained newest first; "
           "the older rule is worded 'was updated'",
    waves=(
        _wave(
            Fact(
                "Amended overage policy: overage now includes a 25-unit monthly grace allowance applied before "
                "block rounding: billable = max(0, overage - 25), then rounded up into 50-unit blocks.",
                "2026-10-21T10:48:00Z",
                ("grace allowance", "25"),
            )
        ),
        _wave(
            Fact(
                "The metering policy for overage_charge was updated to use prorated quotas and block-based "
                "billing: overage bills in 50-unit blocks rounded up. No waivers for small overages are permitted.",
                "2026-10-21T09:48:00Z",
                ("overage_charge", "50-unit blocks"),
            )
        ),
    ),
    must=("The observations say the first 25 overage units each month are free (a grace allowance applied "
          "before block rounding) as the current rule.",),
    traps=(
        "Some observation says the 25-unit grace allowance was removed or replaced, or that no free overage "
        "units are allowed as the current rule.",
    ),
),

The Fact dates may need the scenario helper to accept a time of day. Same-day ordering is the point of the case, so if it only takes dates, extend it rather than spreading the facts across days.

Possible fix

A deterministic guard in the consolidator: when every source fact of an update is older (mentioned_at) than the observation it changes, keep the observation's current value and record the facts as its earlier state. The consolidator already has both timestamps, so this doesn't depend on the model following rule 10.