Skip to content

Repository files navigation

FlowState

Deterministic Market Microstructure Research Framework

FlowState is a high-fidelity quantitative research platform designed for the empirical study of market making dynamics, adverse selection, and inventory risk in continuous, order-driven markets.

Python 3.11+ License Research Build

Research Domains: Market Microstructure • Optimal Execution • Inventory Management • High-Frequency Trading (HFT) • Order Book Dynamics


Core Research Finding

Extensive deterministic replay experiments conducted via FlowState demonstrated that rigorous calibration of the Avellaneda–Stoikov inventory risk parameter ($\gamma$) structurally reduces adverse selection and materially improves market making execution quality.

By analyzing tick-level limit order book data, we found that naive liquidity provision without dynamic inventory penalization exhibits a high probability of adverse execution. Incorporating robust inventory risk controls directly into the quoting mechanism systematically altered queue dynamics, transforming adverse selection from a primary source of structural loss into a quantifiable and manageable constraint. Through deterministic replay and rigorous statistical validation, this research confirms that responsive spread formulation effectively mitigates toxic order flow.


Core Thesis

Modern market making is fundamentally an inventory management problem, not a directional prediction challenge.

In empirical quantitative finance, profitability (PnL) alone is an insufficient evaluation metric for liquidity provision strategies. Strategies exhibiting positive historical PnL but negative expected markout are statistically fragile; they are structurally losing money but occasionally being rescued by stochastic market drift.

FlowState was engineered to separate genuine, repeatable structural edge from statistical luck. The framework mandates that every quotation decision is evaluated against its subsequent adverse selection profile, ensuring that strategies are mathematically robust across diverse volatility regimes before they are considered analytically valid.


Honest Backtesting Philosophy

The methodological core of FlowState relies upon a strict principle:

Capture Once. Replay Forever.

This is the only statistically meaningful methodology for comparing market making strategies in high-frequency domains.

When evaluating limit order strategies, the market's response is inherently tied to the state of the book. Attempting to simulate queue dynamics against a synthesized or aggregated data feed introduces massive look-ahead bias and systemic parameter overfitting.

To counteract this, FlowState captures live, top-of-book and Level 2 market data directly from the exchange and serializes it to immutable, append-only structures. By evaluating competing parameter sets against identical market realizations using deterministic replay, the platform guarantees that variance in performance is strictly attributable to strategy mechanics, isolating alpha from environmental noise.


System Architecture

FlowState enforces a strict separation of concerns, designed specifically for rigorous research rather than rigid production deployment. Strategies and signals can be seamlessly replaced without requiring modifications to the downstream simulation or statistical infrastructure.

graph TD
    A[Market Capture] -->|L2 Book / BBO| B[Replay Engine]
    B -->|Tick Events| C[Feature Engine]
    C -->|Microstructure Signals| D[Strategy Layer]
    D -->|Target Quotes| E[Queue Simulator]
    E -->|Latency Injection| F[Execution]
    F -->|Fills & Markouts| G[PnL Attribution]
    G -->|Performance Metrics| H[Statistical Validation]
    H -->|Significance Tests| I[Publication Assets]

    classDef core fill:#2d3436,stroke:#74b9ff,stroke-width:2px,color:#fff;
    classDef output fill:#0984e3,stroke:#74b9ff,stroke-width:2px,color:#fff;
    
    class A,B,C,D,E,F,G,H core;
    class I output;
Loading

Mathematical Foundation

The baseline quoting engine implements a robust formulation of the Avellaneda–Stoikov (2008) market making model, tailored for continuous, order-driven markets. Complete derivations from the underlying Hamilton-Jacobi-Bellman (HJB) equations are documented in docs/math.md.

Reservation Price

The reservation price $r(s, t)$ represents the indifference threshold where the market maker is equally satisfied holding an inventory of $q$ or $q \pm 1$:

$$ r(s, t) = s - q \cdot \gamma \cdot \sigma^2 \cdot (T - t) $$

Where:

  • $s$ is the current micro-price or mid-price
  • $q$ is the signed inventory position
  • $\gamma$ is the inventory risk-aversion coefficient
  • $\sigma^2$ is the realized variance of the asset
  • $(T - t)$ is the rolling risk horizon

Intuitively, as long inventory ($q > 0$) accumulates, the reservation price is driven strictly below the fair value $s$. This shifts both the bid and ask quotes downward, increasing the probability of a passive ask fill (reducing inventory) while simultaneously decreasing the probability of a passive bid fill.

Optimal Spread

The optimal half-spread $\delta^a$ and $\delta^b$ around the reservation price compensates the market maker for both inventory risk and order arrival elasticity:

$$ \delta^a + \delta^b = \gamma \sigma^2 (T - t) + \frac{2}{\gamma} \ln\left(1 + \frac{\gamma}{\kappa}\right) $$

The spread decomposes into two distinct components: a compensation for holding inventory variance over the time horizon, and a rent extracted based on the market's elasticity of order arrival ($\kappa$).


Framework Components

FlowState is not a monolithic trading bot; it is modular research infrastructure composed of dedicated analytical engines.

Replay Engine

Guarantees absolute determinism. It serializes live websocket streams into crash-safe, compressed JSON-lines formats and reconstructs identical market trajectories for subsequent backtests. It exists to enforce the "Capture Once, Replay Forever" methodology.

Queue Simulation

A high-fidelity limit order book simulator that maintains price-time priority. It accurately estimates the probability of passive execution under adverse market conditions, penalizing strategies that assume unrealistic fill rates.

Execution Research

Injects realistic latency and execution frictions into the simulation path. It evaluates strategy robustness against order routing delays and microstructure noise.

PnL Attribution

Deconstructs raw profitability into its constituent components: spread capture, inventory drift, and fee rebates. It verifies that positive PnL stems from structural edge rather than speculative drift.

Statistical Validation

Calculates markout distributions, Sharpe ratios, and significance metrics across extensive tick datasets. It is used to statistically reject fragile parameter configurations.

Experiment Framework

A suite of standardized, reproducible studies (e.g., latency degradation, parameter stability). It enforces scientific rigor and comparability across different research hypotheses.

Publication Engine

Automatically generates timestamped artifacts, including high-resolution plots, statistical tables, and Markdown reports. It bridges the gap between empirical backtesting and academic publication.


Experimental Framework

FlowState codifies research into standardized, reproducible studies. Each study isolates specific microstructure phenomena.

Tip

All experiments are self-contained and automatically output timestamped publication artifacts directly to the reports/ directory.

Study I: Queue Dynamics

  • Research Question: How does queue priority influence the probability of adverse execution in mean-reverting vs. trending regimes?
  • Method: Replay identical limit order placements with varying simulated queue positions and priority rules.
  • Outputs: Markout distribution plots, probability-of-fill matrices.

Study II: Latency

  • Research Question: At what latency threshold does a profitable market making strategy structurally decay into a losing strategy?
  • Method: Inject escalating fixed and log-normal delays ($0$ to $250$ ms) between decision generation and execution.
  • Outputs: PnL decay curves, adverse selection elasticity.

Study III: Parameter Stability

  • Research Question: Are the optimal model parameters globally stable or overfit to specific local market trajectories?
  • Method: Construct 2-D parameter heatmaps ($\gamma \times \text{inventory limit}$) across diverse volatility samples.
  • Outputs: Parameter sensitivity heatmaps, robustness scores.

Study IV: Cross Asset

  • Research Question: Do inventory management principles generalize universally across disjoint asset classes (e.g., BTC, ETH, DOGE)?
  • Method: Evaluate structurally identical models on distinct instruments without local parameter fitting.
  • Outputs: Cross-asset Sharpe comparisons, inventory variance analysis.

Study V: Historical Events

  • Research Question: How do market making models perform during macroscopic liquidity shocks and news events?
  • Method: Stress test models over specific historical timestamps corresponding to CPI, FOMC, and NFP releases.
  • Outputs: Drawdown trajectories, liquidity provision survival rates.

Study VI: PnL Attribution

  • Research Question: What proportion of gross profitability is attributable to genuine spread capture versus stochastic price drift?
  • Method: Deconstruct total return into markout-derived alpha and delta-derived beta components.
  • Outputs: Attribution waterfalls, risk-adjusted return metrics.

Representative Findings

Empirical studies conducted on the FlowState infrastructure have yielded several significant structural conclusions regarding liquidity provision in digital asset markets.

  • Inventory Penalization: Increasing the risk-aversion parameter ($\gamma$) structurally reduced adverse selection from an unconditional $54%$ down to $45%$. This mathematical constraint successfully forced the strategy out of toxic queues, flipping expected PnL from negative to positive.

  • Asset Rejection over Overfitting: When evaluated on high-volatility, low-liquidity pairs (e.g., DOGE), the statistical validation framework aggressively rejected the asset rather than allowing for parameter overfitting. Over a $2000\times$ sweep of $\gamma$, the model consistently identified the environment as structurally unprofitable for naive provision.

  • Latency Sensitivity: Execution quality decayed logarithmically with respect to latency injection. A degradation of $50$ ms shifted the strategy from the top quartile of queue priority to the bottom, rendering the base Avellaneda-Stoikov model unprofitable.

  • Queue Priority: The queue simulation demonstrated that fill probability is materially altered by time priority. Strategies assuming uniform fill probabilities across the spread consistently overstated expected returns by a factor of three.

  • Statistical Robustness: Parameter stability experiments successfully rejected fragile parameter sets that exhibited isolated "spikes" of profitability, favoring broader "plateaus" of stable, albeit slightly lower, theoretical PnL.


Research Pipeline

The analytical pipeline is designed for complete traceability from raw tick data to published academic outputs.

graph LR
    A[Capture] --> B[Replay]
    B --> C[Strategy]
    C --> D[Execution]
    D --> E[Diagnostics]
    E --> F[Statistics]
    F --> G[Reports]
    G --> H[Paper]

    style A fill:#2d3436,stroke:#74b9ff,stroke-width:1px,color:#fff;
    style B fill:#2d3436,stroke:#74b9ff,stroke-width:1px,color:#fff;
    style C fill:#2d3436,stroke:#74b9ff,stroke-width:1px,color:#fff;
    style D fill:#2d3436,stroke:#74b9ff,stroke-width:1px,color:#fff;
    style E fill:#0984e3,stroke:#74b9ff,stroke-width:2px,color:#fff;
    style F fill:#0984e3,stroke:#74b9ff,stroke-width:2px,color:#fff;
    style G fill:#00b894,stroke:#55efc4,stroke-width:2px,color:#fff;
    style H fill:#00b894,stroke:#55efc4,stroke-width:2px,color:#fff;
Loading

Repository Layout

The framework's structure deliberately emphasizes research capabilities, documentation, and reporting over minimal scripting.

FlowState/
├── research/                 # Experimental framework and statistical studies
│   ├── experiments/          # Standardized Study I-VI implementations
│   ├── config.py             # Research parameter dataclasses
│   ├── report_writer.py      # Artifact generation engine
│   └── synthetic_data.py     # Brownian-motion LOB generator
├── paper/                    # LaTeX sources for formal publication
├── docs/                     # Technical documentation
│   └── math.md               # Rigorous derivations of all quoting models
├── reports/                  # Generated experimental artifacts (CSV/PNG/MD)
├── src/                      # Core infrastructure (Replay, Execution, Strategy)
├── tests/                    # Invariant and property-based model tests
└── captures/                 # Serialized tick data for deterministic replay

Reproducibility

Scientific validity in quantitative finance requires absolute reproducibility. FlowState ensures this through several strict mechanisms:

  • Deterministic Replay: Every execution path is entirely deterministic given a serialized data capture and a random seed. Identical parameters produce identical fills, markouts, and PnL.

  • Seed Control: Stochastic elements within the queue simulator and synthetic data generators are bound to configurable, logged PRNG seeds.

  • Timestamped Reports: Every experiment automatically generates a globally unique, timestamped artifact directory containing exact parameter states alongside the results.

  • Automated Figures: All visualizations (heatmaps, distributions, decay curves) are generated programmatically without manual intervention.

  • IEEE Paper Export: Experimental artifacts are structured to seamlessly integrate into LaTeX paper templates.

  • Unit & Invariant Tests: The core mathematical models are subjected to rigorous property-based testing to verify invariant behaviors (e.g., bounded inventory logic).


Results

The following table outlines the representative cross-asset backtest findings utilizing a 3-hour deterministic capture sequence.

Note

PnL figures are intentionally secondary to Adverse Selection metrics. The primary objective is to validate the structural integrity of the quoting mechanism, not to maximize historical yield.

Asset Validation Status Optimal $\gamma$ PnL ($) Adverse Selection Sample Size (Fills)
AVAX ✅ Robust $0.10$ $+2.29$ $43.3%$ $71$
BTC ✅ Robust $0.03$ $+2.79$ $45.2%$ $127$
ETH ⚠️ Borderline $\approx -1.80$ $48.5%$ $104$
DOGE ❌ Rejected $-0.44$ $44.7%$ $274$

Interpretation

The empirical data strongly validates the theoretical foundation of the Avellaneda-Stoikov framework when properly calibrated. For BTC and AVAX, discovering the optimal risk aversion coefficient successfully constrained adverse selection to below $46%$, allowing the spread capture to outpace inventory drift.

ETH demonstrated borderline characteristics, suggesting that the standard model requires integration with micro-price signals to properly filter toxic flow on this specific instrument.

Crucially, the rejection of DOGE highlights the strength of the statistical validation framework. Despite extensive parameter sweeping, the model correctly identified the structural impossibility of profitable market making under the observed market conditions, prioritizing statistical confidence over curve-fitted PnL.


Known Limitations

Transparency regarding methodological limitations is paramount for institutional quantitative research.

  1. The Fill Model is a Model: True queue position depends on unobservable, order-by-order history that Top-of-Book and Level 2 feeds do not fully expose. The simulator is conservative but lacks exact calibration against venue-specific matching engines.
  2. Restricted Time Windows: Current empirical validations rely on continuous 3-hour macro-regimes. Extensive, multi-month walk-forward validation is required to confirm long-term parameter stability.
  3. Absence of Microstructure Alpha: The current baseline utilizes raw Avellaneda-Stoikov models. While Order Flow Imbalance (OFI) and micro-price signals are computed by the Feature Engine, they are not yet actively steering quote placement.
  4. Zero Market Impact: The simulation assumes zero market impact. This holds strictly true only at minimal quoting sizes and becomes invalid at institutional scale.
  5. Python Latency Profile: The framework operates with approximately $90$ µs tick-to-quote latency. While more than sufficient for high-fidelity research and strategy development, it is orders of magnitude too slow for competitive, production liquidity provision. FlowState is an analytical platform, not an execution facility.

Future Research

Ongoing development focuses on expanding the empirical capabilities of the FlowState framework.

  • Walk-Forward Validation: Implementing automated, rolling-window out-of-sample testing to evaluate regime-conditional parameter stability.
  • Queue Calibration: Enhancing the queue simulator with empirical calibration derived from full Level 3 (MBO) order-by-order data feeds.
  • Market Impact Modeling: Introducing linear and square-root price impact models to dynamically penalize large inventory unwinds during the execution phase.
  • Cross-Exchange Replay: Synchronizing deterministically aligned order book captures across multiple venues to research statistical arbitrage and latency arbitrage phenomena.
  • ML Fair Value Estimation: Replacing standard mid-price derivations with deep learning estimators (e.g., LSTM architectures) trained on high-dimensional order book features.
  • Multi-Agent Simulation: Transitioning from single-agent backtesting to a multi-agent environment to study the endogenous emergence of liquidity and flash crash dynamics.

Why FlowState Exists

FlowState was not built to maximize historical PnL.

It was built to provide a scientifically rigorous, entirely reproducible environment for studying market making under realistic market microstructure constraints. By isolating the mechanics of inventory risk, adverse selection, and queue dynamics from the stochastic noise of the broader market, FlowState empowers quantitative researchers to discover structurally defensible edge rather than chasing statistical illusions.

About

High-Frequency Market Making Research Framework

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages