PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep Learning for Limit Order Books

Justin Sirignano

arXiv:1601.01987v7q-fin.TR

TL;DR

Modeling limit order books is challenging because they are complex and high-dimensional. The paper develops a spatial neural network that uses local book structure; on the considered dataset, it outperforms a standard neural network, especially in the tail.

  • Problem

    Modeling limit order books is challenging because they are complex and high-dimensional.

    Source: §1 · p. 2

  • Method

    The paper develops a spatial neural network for distributions on R^d that takes advantage of local limit order book structure.

    Source: §7 · p. 43

  • Results

    On the dataset considered, the spatial neural network has lower out-of-sample error, lower computational cost, and greater interpretability than the standard neural network; it performs especially well in the tail.

    Source: §7 · p. 43

  • Takeaways & Limitations

    Its outperformance of the standard neural network can be largely attributed to exploiting the limit order book’s local spatial structure.

    Source: §7 · p. 43

    Without framework (12), the number of output states becomes 10,201.

    Source: §6 · p. 31

Abstract

from arXiv · show

This paper develops a new neural network architecture for modeling spatial distributions (i.e., distributions on R^d) which is computationally efficient and specifically designed to take advantage of the spatial structure of limit order books. The new architecture yields a low-dimensional model of price movements deep into the limit order book, allowing more effective use of information from deep in the limit order book (i.e., many levels beyond the best bid and best ask). This "spatial neural network" models the joint distribution of the state of the limit order book at a future time conditional on the current state of the limit order book. The spatial neural network outperforms other models such as the naive empirical model, logistic regression (with nonlinear features), and a standard neural network architecture. Both neural networks strongly outperform the logistic regression model. Due to its more effective use of information deep in the limit order book, the spatial neural network especially outperforms the standard neural network in the tail of the distribution, which is important for risk management applications. The models are trained and tested on nearly 500 stocks. Techniques from deep learning such as dropout are employed to improve performance. Due to the significant computational challenges associated with the large amount of data, models are trained with a cluster of 50 GPUs.

1 Introduction

The paper presents and motivates a spatial neural network for modeling the future joint distribution of best bid and ask prices from limit order book data. It explains why neural networks suit the data’s structure and reports out-of-sample comparisons, with the spatial model particularly strong in distribution tails.

1 Introduction

  • By mimicking local limit-order-book behavior, the spatial neural network creates a low-dimensional model of price movements deep in the book and uses information beyond the best bid and ask.The authors report statistical evidence for this local behavior.Source: §1 · pp. 2–3

1.1 How does a Limit Order Book Work?

  • The target is the future joint distribution of best ask and best bid prices conditional on the current book state, rather than only an expected price change.Modeling both prices matters because the mid-price is artificial and cannot be traded at; the joint distribution supports risk-management uses such as value-at-risk.Source: §1 · pp. 2–3, §1.1 · p. 5

1 Introduction

  • Unlike the standard neural network’s truncated support, the spatial neural network can model distributions across all of R^d and is proved well-posed for several common hidden-unit choices.The spatial architecture is also described as offering better spatial generalization, lower computational expense, and greater interpretability than the standard architecture.Source: §1 · pp. 2–3
  • The spatial neural network outperforms the standard neural network, while both neural networks outperform logistic regression and the naive empirical model, especially in distribution tails.The tail performance is highlighted as important for risk management applications.Source: §1 · pp. 2–3
  • The models are evaluated on limit order book data for 489 stocks from January 1, 2014, to August 31, 2015.The paper describes roughly 50 terabytes of raw data and uses distributed storage and parallel computing to address processing and training demands.Source: §1 · p. 3

1.2 Related Literature

  • The proposed data-driven architecture improves on the standard neural-network architecture with lower computational cost and greater accuracy in the limit-order-book setting, and is more interpretable.The paper also compares performance with logistic regression using nonlinear features and a naive empirical model.Source: §1.2 · p. 6
  • Modeling the full joint distribution of best bid and ask prices supports risk-management calculations, while distinguishing this work from prior prediction of only price-move direction.The paper contrasts its full-distribution approach with Kercheval & Zhang (2015), which predicts whether the mid-price rises, falls, or stays unchanged; it also reports testing on many billions of samples.Source: §1.2 · p. 7

1.3 Advantages of Neural Networks

  • Neural networks suit limit order books because they handle high-dimensional data, capture nonlinear relationships, and scale favorably with large datasets.The paper describes limit order books as high-dimensional, nonlinear, and a source of large amounts of data.Source: §1.3 · p. 8

2 The Data

This section describes the NASDAQ Level III order-book data and the best-bid and best-ask prices used for prediction, including their nonlinear movements, spread sizes, and co-movement across stocks.

2 The Data

  • The data records NASDAQ Level III order-book events and states, including the first 100 nonzero levels: 50 ask-side and 50 bid-side.Submissions, cancellations, and transactions are recorded; event times have nanosecond decimal precision.Source: §2 · p. 9
  • The dataset covers 489 stocks from January 1, 2014, to August 31, 2015, with roughly 50 terabytes of raw data.Stocks are primarily drawn from the S&P 500 and NASDAQ-100.Source: §2 · pp. 9–10
  • The study predicts joint best-ask and best-bid prices at a fixed 1-second horizon or upon the next price move, whose timing varies.The next price move is the first time either best price changes.Source: §2 · p. 10

2.1 Nonlinearity of Limit Order Books

  • For Boeing, higher best-ask size and lower best-bid size increase the probability that the best ask price decreases.The plotted probability is from a neural network fitted to historical Boeing data for a 1-second horizon.Source: §2.1 · p. 11, Figure 2 · p. 12
  • Neural networks have consistently lower error than logistic regression across stocks, indicating nonlinearity beyond the order-book imbalance feature.Source: §2.1 · p. 11

2.2 Importance of modeling the joint distribution of best ask and best bid prices

  • For most stocks, best bid and ask prices do not move in lockstep more than half the time; for half, lockstep occurs only 17% of the time.For most stocks, one price changes without the other at least 25% of the time, and for half this occurs nearly 70% of the time, supporting joint modeling of both prices.Source: §2.2 · pp. 12–13
  • The authors compute spread quantiles across 489 stocks to characterize spread sizes, while Table 1 summarizes spreads and best-bid/best-ask co-movement.Source: §2.2 · p. 13, Table 1 · p. 13

3 Local spatial structure in limit order books

This section presents evidence of local spatial dependence in limit order books and develops a model of the best ask’s upper tail, motivating a local architecture that provides computational efficiency, output-space generalization, and support for distributions over the positive real line.

3.1 Local dependence on the limit order book state

  • For upward ask moves, the chance of exceeding level y depends most on ask size at y and decreases as that size grows; other levels have little influence.Source: §3.1 · pp. 13–14, Figure 3 · p. 14

3.2 Statistical evidence across many stocks

  • Across 489 stocks, coefficient-ratio analysis finds strong dependence on local size for most stocks, with a positive sign.Source: §3.2 · pp. 15–16, Table 2 · p. 16
  • Local dependence is strongest for stocks with larger best-ask price-change standard deviations and appears in the upper ask-price tail and lower bid-price tail.Source: §3.2 · pp. 16–17, Figure 5 · p. 17

3.3 A Local Model for the Upper Tail of the Best Ask

  • The proposed local model estimates per-level behavior f from that level’s size instead of directly fitting high-dimensional nonlinear global g, reducing estimation dimension and improving interpretability.Source: §3.3 · pp. 17–18
  • The architecture similar to (4) is computationally efficient, generalizes over the output space, and models distributions across the entire positive real line.It is used for the neural network proposed in Section 4.3.Source: §3.3 · pp. 18–19

4 Neural Network Architectures for Modeling Distributions on Rd

This section introduces the spatial neural network for modeling distributions on R^d, explaining how it uses local spatial structure efficiently and extends across dimensions. It contrasts the architecture with standard classification networks and a straightforward output-generalizing modification, and discusses why Gaussian mixtures are unsuitable for discrete limit-order-book prices.

4.3 A computationally efficient architecture for modeling spatial distributions

  • The spatial neural network models distributions on R^d while using local spatial structure efficiently and offering greater interpretability than the earlier architectures.The paper presents it as a new architecture for this task.Source: §4 · p. 19, §4.3 · p. 22

4.1 Standard Neural Network Architecture for Classification

  • Standard classification networks do not generalize across output events, so nearby outcomes are treated separately and sparse grid points may receive zero probability.The problem is especially acute as grid spacing shrinks, and the standard network also requires a finite truncated output region and costly probability calculations across spatial points.Source: §4.1 · pp. 20–21

4.2 Straightforward modification to allow generalization

  • A straightforward modification generalizes over space by using a continuous neural-network score for outcomes, but requires evaluation at every grid point and truncates R^d.Its computational cost is comparable to training a binary classifier on N × |Y| samples.Source: §4.2 · p. 22

4.3 A computationally efficient architecture for modeling spatial distributions

  • The proposed architecture is well-posed when its hidden units are bounded, but may not be well-posed when they are ReLUs.A network with at least one bounded hidden layer is also well-posed, even if other hidden layers use unbounded units such as ReLUs.Source: §4.3 · p. 23
  • Architecture (10) evaluates its network and gradient only up to observed y rather than over the full grid, and models all of R+ without a truncated grid.Source: §4.3 · p. 23

4.3.1 Extension to Rd

  • The extension to R^d uses dimension splitting so log-likelihood computational expense grows linearly with d, avoiding the curse of dimensionality.Source: §4.3.1 · p. 24

4.3.2 Advantages of the spatial neural network

  • Although it uses several networks whose number grows linearly with d, dimension splitting keeps computational cost linear in d, making whole-book modeling feasible.Source: §4.3.2 · pp. 25–26
  • By modeling local behavior from only the relevant subset of input values, the architecture can exploit local spatial structure, reduce input dimension, and improve interpretability.Source: §4.3.2 · pp. 25–26

4.4 Other approaches to modeling spatial distributions

  • Because best ask and bid prices lie on discrete levels rather than having a density, Gaussian mixtures can collapse to zero variances, creating numerical difficulties without improving on a discrete-space neural network.Source: §4.4 · p. 26

5 Model Training

The section describes deep-learning training, large-scale computation, data splits, and model hyperparameters. The spatial network outperforms the standard network despite having substantially fewer parameters.

5.1 Deep Learning

  • The models use four layers, and the same batch size, learning rate, momentum, ℓ2 penalty, dropout rate, and RMSProp optimizer are applied across models.Hyperparameter tests on a small stock subset found results robust to the tested choices.Source: §5.1 · p. 27, §5.4 · p. 29
  • Dropout, batch normalization, RMSProp, early stopping, and an ℓ2 penalty are among the techniques used to train the deep networks.Bounded hidden units are considered preferable because extreme bid and ask sizes can cause undesirably large gradient steps; both neural networks receive the same training methods.Source: §5.1 · pp. 27–28

5.2 Computational Approach

  • Training uses 50 GPUs because the dataset and neural networks’ parameter counts make computation expensive.The dataset includes roughly 50 terabytes of raw data, and model training takes over 3,000 GPU node hours.Source: §5.2 · p. 28

5.3 Division of Data into Training, Validation, and Test Sets

  • The test set covers June 1–August 31, 2015, while training and validation use data from January 1, 2014–May 31, 2015.The training sample is a random 95% of the earlier-period data, with the remaining 5% used for validation; each stock is modeled separately.Source: §5.3 · p. 29

5.4 Hyperparameters

  • The standard network uses 250 neurons per hidden layer, while the spatial network uses 50 to reduce computational time.The spatial network can perform well with relatively few neurons, whereas smaller standard networks did not perform as well.Source: §5.4 · p. 29
  • 20,000 parameters versus over 170,000: the spatial network outperforms the standard network despite its smaller model size.The paper attributes the spatial network’s lower complexity to a reduced learning problem with limit-order-book tail behavior embedded in the model.Source: §5.4 · p. 29

6 Out-of-sample Results

This section compares out-of-sample error and computational cost across two prediction cases and models. Neural networks outperform the naive empirical and logistic-regression models, while the spatial network performs especially well in the distribution tails; the section also examines how cost relates to error and ensemble use.

6 Out-of-sample Results

  • The spatial neural network strongly outperformed the standard neural network in the distribution tail, a gain the paper identifies as useful for risk management.Source: §6 · pp. 30, 32
  • Both neural networks strongly outperformed the naive empirical and logistic-regression baselines; the spatial network beat them on 100% of stocks, while the standard network beat them on nearly 100%.Source: §6 · p. 31
  • The spatial neural network beat the standard neural network on 94% of stocks in Case [1] and 97% in Case [2], with joint-distribution error reductions of 0.6% and 3.5%, respectively.The error is cross-entropy, equivalent to negative log-likelihood.Source: §6 · pp. 30–31
  • Neural networks averaged 10% lower joint-distribution error than logistic regression in Case [1] and 20% lower in Case [2].Source: §6 · p. 31
  • The spatial network's advantage is greater in Case [2] because it conditions on a best-ask or best-bid move; in Case [1], the best ask changes only 5-15% of the time.Source: §6 · p. 32
  • In Case [1]'s upper tail, the spatial network's error advantage increased with the standard deviation of best-ask changes, consistent with stronger dependence on deeper-book liquidity for higher-variation stocks.Source: §6 · p. 33, Figure 6 · p. 33
  • In Case [1]'s upper tail, the spatial network's accuracy advantage also increased with the standard deviation of best-ask changes.Source: §6 · p. 33, Figure 7 · p. 34

6.1 Case [1]: Fixed Time Horizon of 1 second

  • Case [1] evaluates the joint distribution of best-ask and best-bid prices one second ahead, given the current limit order book state.Source: §6.1 · p. 34
  • In both the fixed 1-second and next-price-move cases, neural networks consistently have lower error than the naive empirical model and logistic regression, while the spatial network beats the standard network.The reported comparisons cover marginal best-ask and joint best-ask/best-bid distributions across 489 stocks.Source: §6.1 · p. 34, Table 3 · p. 35, Table 4 · p. 35

6.2 Case [2]: Random Time Horizon at the Next Change of Bid or Ask Prices

  • For the next-price-move horizon, neural networks consistently have lower error than the naive empirical model and logistic regression, and the spatial network beats the standard network.The comparisons cover marginal and joint best-price distributions across 489 stocks.Source: §6.2 · p. 36, Table 7 · p. 36, Table 8 · p. 36

6.3 Model Performance in the Tail of the Distribution

  • The spatial network’s outperformance over the standard network increases substantially in the upper tail versus the full distribution in Case [1], matching the pattern in Case [2].This comparison concerns cross-entropy error for the best-ask distribution conditional on the best ask increasing.Source: §6.3 · p. 37, Table 11 · p. 38, Table 12 · p. 38

6.3.1 Another metric: Accuracy

  • Accuracy is an interpretable percentage-correct metric, but may be imperfect for financial applications, where modeling noise distribution matters as much as the most likely outcome; two models can share 99% accuracy while the second has much smaller cross-entropy.Source: §6.3.1 · pp. 38–39

6.3.2 Accuracy in the Tail of the Distribution

  • For tail top-k accuracy, the spatial network outperforms the standard network, and neural networks strongly outperform logistic regression and naive models.Top-k accuracy is the percentage of times the actual outcome is among the model’s k most likely outcomes; the comparison conditions on the best ask increasing.Source: §6.3.2 · p. 40, Table 13 · p. 41, Table 14 · p. 41

6.4 Error versus Computational Cost

  • For Amazon’s best-ask marginal distribution in Case [1], the spatial network reaches the standard network’s lowest error in about 90 seconds, versus about 1700 seconds.The comparison counts training time only, excluding data preprocessing.Source: §6.4 · p. 41
  • Reducing computational cost is associated with decreased out-of-sample error; Figure 10 plots this relationship for Amazon.The supplied passage gives no numerical values for the plotted results.Source: §6.4 · p. 42, Figure 10 · p. 43
  • Building an ensemble of neural networks for each stock can increase computational cost by an order of magnitude or more.Source: §6.4 · p. 42

7 Conclusion

The conclusion presents the spatial neural network as having lower out-of-sample error, lower computational cost, and greater interpretability than the standard neural network, with especially strong performance in the distribution tails. It also reports conditions under which F_N converges to 1 in the bounded-hidden-units analysis.

A List of Stocks

  • Nearly 500 U.S. stocks over 20 months form the evaluation dataset.Source: §7 · p. 43

7 Conclusion

  • The spatial architecture is designed for spatial distributions, with better generalization over space, lower computational expense, and use of local spatial structure.Source: §7 · p. 43
  • On the considered dataset, the spatial neural network has lower out-of-sample error, lower computational cost, and greater interpretability than the standard neural network.Source: §7 · p. 43
  • Its architecture mimics local limit-order-book behavior, yielding a low-dimensional model of movements deep into the book.Source: §7 · p. 43
  • The spatial neural network performs especially well in the distribution tail, which the paper identifies as important for risk management.Source: §7 · pp. 43–44

B Bounded Hidden Units

  • With bounded hidden units, F_N converges to 1; without a positive lower bound on q_k, convergence may fail because Y can escape to infinity, as with ReLU units.Source: B Bounded Hidden Units · p. 46

C ReLU Hidden Units

For ReLU hidden units, the large-y behavior of the network determines whether F_N tends to 1. The theoretical-model comparison is limited to the next best-ask move’s direction because the theoretical models cannot tractably compute the full distribution of interest.

C ReLU Hidden Units

  • For sufficiently large y, a ReLU network f_θ(x, y) takes one of three forms: 0, C_2 + K_2y, or C_3 − K_3y.The specific form depends on θ.Source: C ReLU Hidden Units · p. 46
  • F_N tends to 1 for large-y forms 0 and C_2 + K_2y, but approaches a limit strictly between 0 and 1 for C_3 − K_3y.The form taken by f_θ(x, y) for sufficiently large y depends on θ.Source: C ReLU Hidden Units · p. 46

D Comparison against a theoretical model

  • Cont & Larrard (2012) relates limit-order-book dynamics to order-book imbalance, a nonlinear feature used in trading algorithms and statistical models.The passage notes that data-driven models may provide better quantitative predictions in practice.Source: D Comparison against a theoretical model · pp. 47–48
  • The theoretical models cannot be directly compared with data-driven models on the full N × N distribution because they lack the required tractability.The paper instead compares models on the simpler task of predicting whether the next best-ask move is up or down.Source: D Comparison against a theoretical model · p. 48
  • The comparison uses cross-entropy error on the marginal next-move direction for the best ask across a subset of 109 stocks.Tables 16 and 17 present the comparison, but the supplied passages do not establish the values or a winning model.Source: D Comparison against a theoretical model · p. 48, Table 16 · p. 48, Table 17 · p. 48
Loading 1601.01987v7…