PaperlayerView exact revision on arXiv

Source-linked AI summary

Event History Over Scale: Compact Transformers for Low-Latency Limit Order Book Forecasting

David Schaurecker, Lasse B. Strand, Kevin O'Sullivan, Robert Jakob

arXiv:2610.02917v1q-fin.CPcs.CE

TL;DR

Short-horizon limit-order-book forecasting must combine predictive quality with low latency and small model size for continuous market updates. This paper compares compact transformers over individual order events with Level-2 snapshot baselines, finding strong cross-market macro-F1 alongside sub-millisecond inference. The results support compact event-history models within the markets and evaluation settings studied.

  • Problem

    Short-horizon limit-order-book prediction must meet latency and memory demands at rapid, continuous market updates.

    Source: §1 · p. 1

  • Method

    The paper compares compact transformers processing recent Level-3 order-event histories with Level-2 baselines using sampled book states and features.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    One proposed model achieved the highest mean macro-F1 in eleven of twelve settings, and both models achieved sub-millisecond median inference.

    Source: §1 · p. 3, §5 · p. 7

  • Takeaways & Limitations

    For the studied markets and baselines, fine-grained event histories can support compact models that match or outperform substantially larger Level-2 models.

    Source: §5 · p. 7

    Test labels use the full test sequence’s price-change distribution; a live system would need to fix the threshold from past data, so absolute scores are not directly achievable online.

    Source: §2.1 · p. 3

Abstract

from arXiv · show

Short-horizon price-trend prediction from limit order books in equity and intraday electricity markets requires models that combine predictive quality with low single-sample latency and a small serialized model size to keep pace with rapid and continuous market updates. We introduce MBOFormer, a 7,203-parameter causal transformer, and MBOFusion, a 14,371-parameter extension with a slow temporal-context branch. Both models process market-by-order (level-3) histories of individual order submissions, cancellations, and executions. We compare them with level-2-based baselines that process sampled order-book snapshots and simple statistics instead of their underlying messages. Across three markets and four prediction horizons, our models achieve the highest mean macro-F1 in eleven of the twelve settings in a comparison against other benchmark models of varying sizes. Both our models achieve sub-millisecond median inference on a single Apple M2 CPU thread and have serialized state dictionaries below 70 kB. MBOFormer is 5.1x-8.1x faster than the million-parameter baselines at comparable predictive quality. Our results show that fine-grained market event history can reduce the need for model size under tight deployment constraints for short-term equity and electricity price forecasting tasks.

1 Introduction

Short-horizon forecasting must balance predictive quality with the latency and memory demands of continuous, individual market decisions. The paper tests whether compact models using granular order events can compete with larger models built on sampled book states.

1 Introduction

  • Because forecasts are made online at individual decision points, the study evaluates batch-one latency and memory alongside predictive quality.Repeated inference costs accumulate when models are queried continuously.Source: §1 · p. 1
  • Level-3 event histories preserve submissions, cancellations, and executions that aggregated Level-2 snapshots cannot recover, motivating tests of granular history for compact forecasting.The snapshots omit the identity, timing, type, and ordering of the events that produced them.Source: §1 · p. 2, Figure 1 · p. 2
  • MBOFormer uses 7,203 parameters, while MBOFusion adds a slow-context convolutional branch and uses 14,371 parameters; both are compared with Level-2 baselines across equities and electricity markets.The evaluation includes NASDAQ, NYSE, and EPEX continuous intraday electricity data.Source: §1 · p. 2
  • Eleven of twelve settings had one proposed model with the highest mean macro-F1; both models achieved sub-millisecond median inference, and MBOFormer was 5.1×–8.1× faster than million-parameter baselines at comparable quality.The comparison covers three markets and four forecast horizons.Source: §1 · p. 3

2 Method

The method defines an event-time forecasting task with smoothed price-change labels, then compares two compact architectures that process recent order events, with one adding slower market context.

2.1 Prediction task, labels, and event representation

  • Decision points follow every tenth book-changing message, and the model predicts mid-price direction 10, 20, 50, or 100 sampled steps ahead using fixed ten-step smoothing windows.The same decision points and targets are used for the Level-2 baselines and Level-3 models.Source: §2.1 · p. 3
  • The adaptive threshold labels changes as up, down, or flat, but uses the full labeled sequence, including the test sequence, so absolute scores are not directly achievable online.A live system would need to fix the threshold using past data.Source: §2.1 · p. 3
  • Each Level-3 model processes up to 128 recent events represented by seven event-derived features, without separate Level-2 inputs.Features include event type, side, quote-relative depth, size, elapsed time, immediate mid-price change, and a best-quote indicator.Source: §2.1 · pp. 3–4

2.2 Our models

  • MBOFormer uses two causally masked transformer blocks and pools the newest-event representation with the valid-event mean, yielding a 7,203-parameter model.Its fixed recent-event window means its memory footprint does not grow with the order-stream length.Source: §2.2 · p. 4, Figure 2 · p. 4
  • MBOFusion reuses the event encoder and adds a causal convolution over 16 recent context vectors, producing a 14,371-parameter model.The comparison with MBOFormer combines the effect of added temporal context and the branch’s additional parameters.Source: §2.2 · p. 4, Figure 2 · p. 4

3 Experimental Setup

The experiments use native L3 data from three markets and compare event-based models with L2 snapshot baselines. Chronological splits, event-step horizons, and macro-F1 evaluation define the testing setup.

3.1 Markets and data

  • The study uses EPEX SPOT electricity data from 13 delivery days, AAPL data from 7 days, and BAC data from 58 days.Test days postdate training days, and label windows do not cross split boundaries.Source: §3.1 · p. 5
  • At h = 10, flat windows make up about 70% of EPEX training data, versus 44% for BAC and 38% for AAPL.As horizons grow to h = 100, the flat share falls to roughly 51% on EPEX and 29–33% for the equities.Source: §3.1 · p. 5

3.2 Models and inputs

  • MBOFormer and MBOFusion process L3 events, while DeepLOB, BiNCTABL, MLPLOB, and TLOB use sampled L2 histories.TLOB variants span 7.2k to 1.20M parameters; the size-matched version uses a 16-step sampled window rather than 128 raw events.Source: §3.2 · p. 5, Table 1 · p. 6

3.3 Training and evaluation

  • Each model, market, and horizon combination is trained separately, with three random seeds and test macro-F1 reported.Macro-F1 averages the up, flat, and down classes equally.Source: §3.3 · p. 6

4 Results

The L3 models lead the benchmark comparison in most market-by-horizon settings, while remaining compact and fast in model-only CPU inference. Their relative advantage varies by market.

Predictive quality

  • One L3 model achieves the highest mean macro-F1 in eleven of twelve market-by-horizon settings.At h = 10, 7,203-parameter MBOFormer scores 0.651 on EPEX, 0.665 on AAPL, and 0.735 on BAC, exceeding both large baselines on all three markets.Source: §4 · p. 6

Inference efficiency

  • MBOFormer’s median latency is 5.1 and 8.1 times lower than the 1.20M-parameter TLOB and 3.14M-parameter MLPLOB, respectively.Both L3 models have sub-millisecond median latency and state dictionaries of 39 and 69 kB; compact baselines are faster but generally less predictive.Source: §4 · pp. 6–7, Table 2 · p. 7

Market variation

  • At h = 10, the best L3 result exceeds the best L2 result by 0.038 macro-F1 on EPEX, 0.009 on AAPL, and 0.002 on BAC.The experiments do not identify which market characteristic explains the differences.Source: §4 · p. 7

5 Conclusion

Across the tested markets and horizons, compact L3 models combine leading macro-F1 with sub-millisecond median latency and small state dictionaries. The study’s scope and measurements limit how broadly to interpret those results.

Main findings

  • The compact L3 models attain the highest mean macro-F1 in eleven of twelve settings, with sub-millisecond median latency and state dictionaries below 70 kB.The conclusion is scoped to the three markets and baselines considered.Source: §5 · p. 7

Limitations

  • Three-seed results do not establish statistical significance across test periods, and latency measures cover model-forward execution on one standard laptop CPU.The scores generally do not represent trading revenue, and energy was not measured directly.Source: §5 · p. 7

Future work

  • Future evaluation should include additional instruments and regimes, complete online pipelines on deployment hardware, and economic performance after transaction costs.Source: §5 · p. 7

A Baseline implementation details

The TLOB baseline was adapted to the shared evaluation pipeline and configured at multiple model sizes and input lengths.

TLOB adaptation

  • The authors adapted published TLOB code to use identical decision points, labels, and day splits across models.They also added dropout, replaced layer normalization with root-mean-square normalization, and used a more efficient attention implementation.Source: A Baseline implementation details · p. 11

TLOB configurations

  • TLOB configurations used 1.20M, 23.9k, or 7.2k parameters, with the largest processing 128 sampled L2 rows and smaller versions processing 16.Model width, depth, and input-window length were configurable.Source: A Baseline implementation details · p. 11

Validation

B Full predictive results

MBOFormer and MBOFusion attain the highest mean macro-F1 in eleven of twelve settings, despite having far fewer parameters than the strongest L2-based baselines.

B Full predictive results

  • In eleven of twelve settings, MBOFormer and MBOFusion attain the highest mean macro-F1 with 7,203 and 14,371 parameters, versus 1.20 and 3.14 million for the strongest L2-based baselines.Table 3 reports test macro-F1 for EPEX Spot CID, NASDAQ AAPL, and NYSE BAC.Source: B Full predictive results · p. 11, Table 3 · p. 11
  • Table 3 reports test macro-F1 as mean and standard deviation over three seeds for EPEX Spot CID, NASDAQ AAPL, and NYSE BAC.Source: Table 3 · p. 11
  • Within each market’s Table 3 results, the L2-based baselines precede the L3-based MBOFormer and MBOFusion models.Source: Table 3 · p. 11
Loading 2610.02917v1…