Skip to content

feat: introduce a temporal oracle for bitemporal memory replay - #873

Draft
Gardenia-zx wants to merge 23 commits into
apache:masterfrom
Gardenia-zx:temporal-oracle
Draft

Gardenia-zx wants to merge 23 commits into
apache:masterfrom
Gardenia-zx:temporal-oracle

Conversation

@Gardenia-zx

Copy link
Copy Markdown

What changes were proposed in this pull request?

OSPP 2026 project context

This PR is the Phase 1 milestone delivery of my Open Source Promotion Plan (OSPP) 2026 project for Apache GeaFlow, under the broader Dynamic Context Memory proposal described in #683.

The overall OSPP project explores how GeaFlow can support evolving contextual memory for AI applications. Phase 1 focuses on establishing a correctness-first Temporal Oracle: a deterministic reference implementation for representing, normalizing, replaying, querying, comparing, and recovering bitemporal memory state.

The Temporal Oracle is intended to serve as the correctness baseline for later integration, scalability, and performance work.

Phase 1 deliverables

Planned deliverable Implementation in this PR
Bitemporal memory model Temporal facts, versions, provenance, valid time, and transaction time
Deterministic reference replay Full replay oracle supporting ADD, CORRECT, and RETRACT
Incremental correctness validation Incremental integration checked against full replay
Temporal queries Queries using both valid-time and transaction-time points
State comparison Canonical snapshots and stable field-level diff reports
GeaFlow integration feasibility Keyed aggregation, serialization, recovery, and local pipeline tests
Graph representation Round-trip adapter between temporal snapshots and MemoryGraph
Reliability validation Regression, checkpoint recovery, JDK 8, and JDK 11 testing

Architecture overview

The main processing flow is:

MemoryEventEventNormalizerNormalizedMemoryEventFullReplayOracle / IncrementalTemporalIntegratorTemporalStateCanonicalSnapshot

The resulting canonical snapshot can then be:

  • compared through SnapshotComparator;
  • queried through the bitemporal query layer;
  • compared with simplified replay baselines;
  • converted to and restored from the existing MemoryGraph representation.

Main changes

  • Add temporal domain models for entities, facts, provenance, half-open time intervals, bitemporal fact versions, and version relations.
  • Support ADD, CORRECT, and RETRACT events, including interval splitting and tombstones.
  • Normalize event content, evidence ordering, timestamps, and payload hashes before replay.
  • Provide deterministic full replay with idempotent duplicate handling and rejection of conflicting event-ID reuse.
  • Track SUPERSEDES, DUPLICATE_OF, and CONFLICTS_WITH relations between fact versions.
  • Add bitemporal queries based on valid-time and transaction-time points.
  • Add incremental integration for late and out-of-order events.
  • Verify equivalence between incremental integration and full replay.
  • Add canonical snapshots, stable field-level comparison, and replay baselines.
  • Add a deterministic round-trip adapter between temporal snapshots and the existing MemoryGraph model.
  • Add GeaFlow keyed aggregation, serialization, checkpoint recovery, local pipeline, and UDGA feasibility tests.

Core semantics and invariants

  • Time intervals use half-open semantics: [start, end).
  • Events are replayed deterministically by transaction time and event ID.
  • Replaying the same event ID with the same normalized payload is idempotent.
  • Reusing an event ID with a different payload is rejected.
  • Full replay and incremental integration are expected to produce equivalent temporal state.
  • Late and out-of-order events are handled according to their temporal information rather than arrival order.
  • Canonical snapshots provide a stable representation for state comparison and recovery validation.
  • Graph snapshot import validates entity, evidence, source, version, and relation references.

Addressed high-priority pre-PR review findings

  • Restore conflict relations for residual active versions after partial corrections and retractions.
  • Preserve full Instant nanosecond precision during event normalization and payload hashing.
  • Reject orphan vertices during graph snapshot import instead of silently dropping them.

Suggested review order

The implementation can be reviewed in the following order:

  1. temporal/model: domain objects, interval rules, and invariants.
  2. temporal/semantics: event normalization, ledger behavior, and temporal state.
  3. temporal/oracle and temporal/query: replay rules and bitemporal queries.
  4. Canonical snapshots, snapshot comparison, and replay baselines.
  5. temporal/integration: incremental integration, aggregation, serialization, and recovery.
  6. temporal/adapter: snapshot-to-MemoryGraph round trip and graph validation.
  7. Pipeline, recovery, and UDGA feasibility tests.

Scope and boundaries

This is an additive Phase 1 correctness/reference implementation.

  • Production code is contained in the new org.apache.geaflow.ai.temporal package.
  • Existing production classes outside this package are unchanged.
  • The only existing project file modified is geaflow-ai/pom.xml.
  • This PR does not introduce an external storage schema, service endpoint, or production temporal UDGA.
  • More than half of the added lines are tests and test resources.

This PR intentionally contains the complete Phase 1 vertical slice so that replay equivalence, state recovery, and graph round-trip invariants can be reviewed together.

It does not represent completion of the entire OSPP project. Later work may build on this correctness foundation to address production-facing APIs, distributed temporal processing, retrieval integration, scalability, and performance evaluation.

Because this milestone is larger than a typical feature PR, it is intentionally opened as a Draft PR for early architecture and correctness review. In particular, feedback is welcome on:

  1. the temporal data model and invariants;
  2. replay, correction, retraction, and conflict semantics;
  3. the integration boundary with GeaFlow;
  4. whether this milestone should remain an end-to-end PR or be split into smaller sequential PRs.

If maintainers prefer smaller PRs, the implementation can be split into:

  1. temporal domain models;
  2. event normalization, ledger, and temporal state;
  3. full replay oracle and bitemporal query;
  4. canonical snapshots, comparison, and replay baselines;
  5. incremental integration, recovery, and UDGA feasibility;
  6. MemoryGraph adapter and graph validation.

How was this PR tested?

  • Tests have been added for the changes
  • Production environment verified

Validation completed:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant