Source-linked AI summary
Repurposing Deep Limit Order Book Forecasting for Scenario-Conditioned Market Impact Modeling
Eljas Linna, Kestutis Baltakys, Derrick Manoharan, Alexandros Iosifidis, Juho Kanniainen
TL;DR
The paper asks whether deep LOB forecasters can quantify responses to counterfactual messages, beyond passive prediction. It compares forecasts before and after mechanically valid scenario injection and finds strong scenario-ranking and directional agreement, with additional sequence-dependent information. The evidence supports repurposing pretrained forecasters as model-implied response estimators without retraining, while not establishing unrestricted causal effects.
Problem
It remains unclear whether deep LOB forecasters can support validated scenario-conditioned impact estimation for counterfactual order-book messages.
Method
The framework compares a trained forecaster’s predictions before and after mechanically valid scenario injection while holding the model and original history fixed.
Results
The forecaster recovered much of the direction and cross-scenario ordering observed in simulation and historical events, while estimated impacts captured variation beyond scenario identity and the pre-event forecast.
Takeaways & Limitations
Trained LOB forecasting models can serve as practical scenario-conditioned response estimators, with direction and relative ranking supported more strongly than exact magnitude.
Takeaways & Limitations
Validation does not establish an unrestricted causal effect, and evidence is limited to the studied architecture, horizon, assets, and market period.
Abstract
from arXiv · showhide
Deep Limit Order Book forecasting models capture nonlinear market dynamics, but their ability to quantify the effects of counterfactual order book messages has not been systematically validated. We introduce a model-agnostic framework that compares a trained forecaster's predictive distributions before and after injecting mechanically valid counterfactual messages, defining short-horizon model-implied market impact. A Transformer-based forecaster recovered scenario rankings with a Spearman correlation of 0.99 and 97.2% directional agreement with realized historical outcomes among non-neutral scenarios. Observation-level analysis further showed that estimated impacts captured incremental sequence-dependent variation beyond scenario identity and the pre-event forecast. These results provide evidence that pretrained Limit Order Book forecasters can be repurposed for scenario-conditioned response modeling without retraining.
1 Introduction
The paper addresses whether deep LOB forecasters can estimate sequence-dependent responses to counterfactual order-book messages, rather than only predict passively. It introduces a model-agnostic intervention framework that compares forecasts before and after mechanically valid message injection.
- Deep LOB forecasters learn nonlinear sequence relationships but are usually evaluated only as passive predictors, leaving their counterfactual impact response unvalidated.
- The framework compares a trained forecaster’s predictions before and after injecting a mechanically valid scenario into the same observed sequence.
- The paired forecast difference represents model-implied counterfactual impact conditional on the observed sequence, not an identified causal effect.
- The study combines full message-level conditioning with validation against simulation and corresponding natural events, avoiding an imposed response or decay function.
2 Background and Related Work
LOB price formation reflects interactions among aggressive orders, submissions, cancellations, available depth, and subsequent liquidity responses. Deep models represent these dynamics from sequences, but passive forecasting performance does not validate counterfactual impact estimates.
- Short-horizon price formation depends on aggressive orders, submissions, cancellations, order-flow imbalance, available depth, and subsequent liquidity replenishment.
- Deep LOB models provide data-driven sequence representations, including message-based models that retain explicit event attributes.
- Forecasting accuracy alone does not establish that a model’s response to a counterfactual order-book event matches subsequent market impact.
- Controlled perturbations measure fitted-model prediction responses, while simulators provide matched treated and untreated trajectories but encode their own market dynamics.
3 Methods
The method holds the model and observed history fixed while injecting valid scenarios, then compares original and scenario-conditioned predictive distributions. It summarizes signed directional shifts and distributional changes across eligible sequences.
- 3.1 Problem Formulation: The estimator holds model parameters and original history fixed, so paired forecasts isolate the fitted model’s response to an injected scenario conditional on the observed sequence.
- 3.2 Mechanically Valid Scenario Injection: A scenario appends an ordered sequence of artificial messages, whose actions, sides, price levels, quantities, and timing are converted through the same valid-data pipeline.
- 3.3 Impact Measures: Directional impact uses scores for downward, stationary, and upward movement; positive values indicate an upward forecast shift and negative values a downward shift.
- 3.3 Impact Measures: The primary scenario impact averages responses over eligible sequences, while Jensen–Shannon divergence measures the magnitude of the full predictive-distribution change.
- 3.3 Impact Measures: The two impact measures are complementary: directional impact captures signed outcome change, whereas divergence captures overall distributional response.
4 Experimental Setup
The experiments use LOBERT, a Transformer forecasting three-way mid-price movement over a 100-message horizon from 512-message sequences. Its non-trivial predictive performance supports, but does not by itself validate, the scenario analysis.
- 4.1 Deep Forecasting Model: LOBERT processes 512-message LOB sequences and predicts whether the average mid-price over the next 100 messages is higher, lower, or similar.
- 4.1 Deep Forecasting Model: F1 score was 47.7%, balanced accuracy was 47.2%, and negative log-likelihood loss was 1.012.
- 4.1 Deep Forecasting Model: Forecasting performance was non-trivial but does not establish the validity of the model’s impact estimates.
4.2 Empirical Data and Simulation Environment
The empirical study uses seven Nasdaq assets converted to LOBSTER format, with strictly separated training, validation, and held-out test periods. Simulation benchmarks generate paired treated and untreated trajectories, but represent only the simulator’s encoded dynamics.
- Empirical dataset: The empirical dataset covers 7 Nasdaq assets in LOBSTER format across separate training, validation, and test periods.The periods contain 795M, 99M, and 156M messages, respectively, and all preprocessing and model-selection choices exclude the test set.
- Simulation environment: Paired simulator trajectories identify scenario effects from identical initial states, without treating simulator outcomes as real-market ground truth.The simulator implements an event-driven continuous double-auction market with heterogeneous agents submitting, modifying, and cancelling orders.
4.3 Scenario Suite, Sequence Sampling and Eligibility
The scenario suite varies message type, side, depth, and intensity while including a low-impact placebo family. Scenarios are applied to eligible sequences only, and regime contrasts compare responses across asset–day-specific quantile bins.
- Scenario suite: 48 primary scenarios combine eight scenario families, three volume intensities, and symmetric bid- and ask-side variants.Injected-message volume is set to 25%, 50%, or 100% of a reference volume.
- Placebo scenarios: A deep level-10 placebo order establishes a minimum-impact threshold for excluding responses that cluster near zero.The placebo is designed to have minimal market impact.
- Eligibility: Sequences are excluded for a scenario when injection would create negative volume, cross the book, violate tick-size constraints, or otherwise be mechanically invalid.Each selected sequence is otherwise tested with every scenario.
- Regime analysis: Pre-event regimes are divided into asset–day-specific quartile bins for volatility, queue imbalance, normalized order-flow imbalance, and recent midpoint direction.Pooled means are observation-weighted, and positive or negative contrasts indicate more upward or downward responses in the high regime.
4.4 Evaluation Metrics and Neutrality Thresholds
Validation compares scenario-level estimates using rank and sign agreement, while sequence-level prediction uses error, R2, and clustered uncertainty. Placebo-based neutrality thresholds exclude negligible responses from directional comparisons.
- Evaluation metrics: Scenario-level agreement is measured with Spearman rank correlation and directional sign agreement against simulated or matched historical effects.Sequence-dependent historical validity additionally uses test-set mean absolute error, R2, impact coefficients, and asset–day clustered 95% confidence intervals.
- Neutrality thresholds: Neutrality thresholds are source-specific 95th percentiles of absolute placebo impacts, with rank and sign metrics restricted to mutually non-neutral scenarios.Separate thresholds are used for model, simulation, predicted historical, and realized historical impacts.
- Uncertainty: Sampling uncertainty in scenario-level mean impacts uses 95% percentile confidence intervals from an asset–day clustered bootstrap with B = 2,000 replicates.Thresholds and analysis choices are determined without using held-out test results.
4.5 Validation Protocol
The validation protocol benchmarks model-implied impacts against controlled simulations and matched natural events, then tests sequence-level incremental validity, microstructure consistency, sampling dependence, and initialization stability. Figure 1 presents model-implied versus simulator-defined average directional impacts across scenarios.
- Simulator benchmark: Controlled simulations compare treated and untreated trajectories from the same initial sequence, using K = 500 runs per scenario effect.The resulting simulator-defined impact is evaluated by Spearman correlation and directional sign accuracy.
- Historical validation: Matched historical validation compares predictions and realized directions after naturally occurring events, as well as artificial and natural prediction changes for corresponding scenarios.Agreement between artificial and natural changes indicates similar model responses to events with the same characteristics.
- Sequence-level validity: Sequence-level models test whether estimated impact predicts realized responses beyond scenario fixed effects and the pre-event directional forecast.Scenario effects absorb family, side, depth, and intensity differences; incremental validity requires M2 to outperform M1, while β > 0 supports positive association.
- Robustness and consistency: Consistency tests examine opposite bid–ask directions, smaller effects deeper in the book, and substantially weaker placebo responses than active scenarios.The protocol also prevents sequence overlap using endpoints separated by at least L = 512 messages and evaluates stability across 10 independently initialized models.
5 Results
The estimator reproduced scenario-response structure in simulations and historical events, while revealing small but significant sequence-dependent information beyond scenario identity and pre-event forecasts. Impact patterns were generally directionally coherent, state-dependent, robust across assets and models, and sensitive to scenario type, intensity, and depth.
- 5.1 Validation of the Estimator: ρ = 0.76 and 94.4% sign agreement recovered simulator-implied scenario rankings across 36 scenarios classified as non-neutral under both thresholds.Direction and ranking were recovered more reliably than magnitude, with complete best-level deletion as the principal outlier.
- 5.1 Validation of the Estimator: ρ = 0.99 and 97.2% sign agreement aligned model-implied directionalities with realized historical outcomes across 9.8M matched events and 36 non-neutral scenario groups.Complete best-level deletion remained non-monotonic, producing weaker directional responses but 2.0× larger Jensen–Shannon divergence than partial deletions.
- 5.1 Validation of the Estimator: Adding the artificial-impact estimate produced a small but significant improvement beyond scenario identity and the pre-event forecast, with MAE changing by −0.00231 and R2 increasing by 0.000254.The confirmatory regression estimated a positive artificial-impact coefficient of β = 1.796, supporting incremental sequence-dependent information.
- 5.2 Scenario-Conditioned Impact Patterns: Executions produced the largest directional responses, approximately 0.13 at 25% and 50% intensity versus approximately 0.10 at full consumption, while transient scenarios remained substantially weaker.Persistent additions, price-improving orders, and L2 deletions generally strengthened with intensity; complete L1 deletion reversed the directional point estimate.
- 5.3 Internal Consistency and Robustness: High–low contrasts across all included regime variables showed that the same scenario received different directional impacts under different pre-event market conditions.Across 90 matched comparisons, marginal 95% bootstrap confidence intervals were non-overlapping in 88 cases, and executions produced Jensen–Shannon divergences of 0.0023–0.0039 versus 0.0002–0.0005 for transient scenarios.
- 5.3 Internal Consistency and Robustness: Ask-side impacts closely mirrored sign-reversed bid-side impacts, with ρ = 0.995, normalized asymmetry of 0.051, and 100% opposite-sign agreement above the directional threshold.Absolute impact decreased with depth in 8 of 9 side-specific deletion and addition comparisons, except for complete L1 deletion; rankings were also stable across assets and model initializations.
6 Discussion
The framework shows that trained LOB forecasters can estimate sequence-dependent, scenario-conditioned market responses, with validation supporting direction, ranking, and incremental observation-level information more strongly than exact magnitudes or causal interpretation.
- Historical analyses found that post-event predictions aligned with realized outcomes and artificial interventions reproduced responses to corresponding natural events.
- Estimated artificial impact predicted incremental variation in realized price responses beyond scenario identity and the pre-event forecast.The association persisted after controlling for the parameterized scenario and pre-event forecast, and varied with observable pre-event conditions.
- The model recovered scenario direction and ordering across assets, although impact magnitudes varied.Executions were strongest, persistent additions shifted predictions oppositely, transient addition–deletion effects were small, and deeper-book interventions were generally weaker.
- Complete best-level deletion produced a qualitatively distinct predictive response rather than a proportional extension of partial liquidity removal.Its directional effect weakened or reversed while Jensen–Shannon divergence increased substantially.
- The evidence supports model-implied scenario-conditioned impact within the studied architecture, horizon, assets, and market period, but not an unrestricted causal effect.Simulator conclusions depend on simulation realism, historical events lack observable untreated counterfactuals, and absolute magnitudes may require calibration across assets, regimes, or models.
- Once trained and validated, the forecaster can evaluate many counterfactual interventions efficiently while conditioning on detailed LOB histories.This offers a practical alternative to repeated simulations for studying sequence-dependent execution costs and potentially disruptive order-book activity.
7 Conclusion
The paper introduces a framework that compares trained LOB forecaster predictions before and after mechanically valid scenario injection to estimate short-horizon market response. Results support direction and relative ranking more strongly than exact magnitude, while observation-level validation indicates information beyond scenario identity and the pre-event forecast.
- The framework estimates short-horizon market response by comparing trained LOB forecaster predictions before and after mechanically valid scenario injection.
- Direction and relative scenario ranking were supported more strongly than exact impact magnitude across controlled simulation and historical market events.The results also revealed nonlinear effects across event families, intensities, and book depths.
- Observation-level validation indicated that the estimates contain information beyond scenario identity and the pre-event forecast.Complete best-level depletion showed a qualitatively distinct response among the nonlinear effects.