PaperlayerView exact revision on arXiv

Source-linked AI summary

Adaptive trading strategies across liquidity pools

Bastien Baldacci, Iuliia Manziuk

arXiv:2008.07807v1q-fin.TRq-fin.MF

TL;DR

The paper addresses how to trade an asset across multiple venues when spreads, imbalances, and execution outcomes are interdependent and market conditions may change. It formulates a flexible stochastic-control model with partial limit-order fills and market orders, then combines finite differences, deep reinforcement learning, and slice-wise Bayesian updates. The framework supports adaptive strategy recomputation, while computational complexity and data requirements constrain its practical scope.

  • Problem

    Optimal cross-venue trading requires modeling coupled spreads and imbalances, partial limit-order execution, and changing market parameters rather than relying on fixed estimates.

    Source: §1 · p. 2

  • Method

    The paper formulates a stochastic-control problem solved with HJBQVI methods, updates market parameters separately through Bayesian learning, and uses finite differences or deep reinforcement learning for computation.

    Source: §1 · pp. 2–3

  • Results

    The framework recomputes trading strategies for new market conditions on execution slices and captures market-state-dependent value functions with numerical approximations.

    Source: §1 · p. 3, §5.2.2 · p. 16

  • Takeaways & Limitations

    The approach provides a flexible basis for adaptive cross-venue trading and can be extended to more complex settings through deep reinforcement learning.

    Source: §1 · p. 3

    The Bayesian update requires sufficient executed-order observations across market zones, while finite-difference complexity grows exponentially with the number of venues.

    Source: §3.1.1 · p. 8, §5.2.1 · p. 15

Abstract

from arXiv · show

In this article, we provide a flexible framework for optimal trading in an asset listed on different venues. We take into account the dependencies between the imbalance and spread of the venues, and allow for partial execution of limit orders at different limits as well as market orders. We present a Bayesian update of the model parameters to take into account possibly changing market conditions and propose extensions to include short/long trading signals, market impact or hidden liquidity. To solve the stochastic control problem of the trader we apply the finite difference method and also develop a deep reinforcement learning algorithm allowing to consider more complex settings.

1 Introduction

The paper develops an adaptive stochastic-control framework for trading cross-listed assets across liquidity venues, modeling venue dependencies and updating parameters as market conditions change. It combines Bayesian parameter updates with numerical methods for recomputing strategies across execution slices.

1 Introduction

  • Cross-venue order splitting must account for dependencies between spreads and imbalances, limit-order execution probabilities, and partial fills across venues.The framework also allows market orders and different limit levels, linking order placement to observed venue conditions.Source: §1 · p. 2
  • Constant market parameters can make strategies non-robust when changing liquidity or participant activity removes execution opportunities.The paper motivates Bayesian updating using new observations of market dynamics, such as activity associated with a metaorder.Source: §1 · p. 2
  • The trader’s controls are volume splits between limit and market orders on each venue and the limits selected for those orders.The resulting optimization problem is expressed as an HJB quasi-variational inequality that can be solved by grid methods in a parsimonious model.Source: §1 · p. 2
  • Bayesian updates are decoupled from the control problem because continuous integration over posterior parameters would greatly increase state dimensionality and numerical difficulty.The decoupled procedure avoids Markov chain Monte Carlo and updates the strategy using posterior estimates.Source: §1 · p. 2
  • In practice, the trading period is divided into slices lasting several seconds to a few minutes, with market events recorded before updating parameters and recomputing the next strategy.Finite differences and deep reinforcement learning can resolve the resulting high-dimensional problems on each slice; the slice-wise procedure is time-inconsistent and presented as an initial learning framework.Source: §1 · pp. 2–3
  • The framework is intended to reproduce market stylized facts and help practitioners design strategies using real signals while adapting to differing market conditions.The paper also outlines extensions involving short or long signals, market impact, hidden liquidity, and an OTC market-maker application.Source: §1 · p. 3

2 Optimal trading on several liquidity pools

The paper formulates multi-venue optimal trading as a stochastic-control problem with coupled spread–imbalance regimes, partial limit-order execution, and market orders. Its HJBQVI has a unique viscosity solution, while the reduced formulation supports numerical computation and separate Bayesian strategy updates.

2.1 Framework

  • The multi-venue model generalizes optimal trading by jointly representing each venue’s spread and imbalance processes, including full cross-venue coupling.The state variables are modeled with continuous-time Markov chains whose transition structure describes market-regime changes.Source: §2 · p. 3, §2.1 · pp. 3–4
  • Spread and imbalance regimes are discretized into market states, with examples covering three spread levels and five imbalance levels.The framework permits parsimonious or fully coupled specifications across venues.Source: §2.1 · pp. 3–4
  • The trader optimizes cash and inventory while penalizing deviations from a precomputed trading curve such as an Almgren–Chriss or VWAP strategy.The formulation focuses on ask-side liquidation, while extending to both sides does not increase dimensionality.Source: §2.1 · pp. 4–5
  • Limit orders can use venue-specific limits and volumes, with execution probabilities depending on spreads, imbalances, order size, and selected limits.The model allows partial fills through categorical execution proportions and also includes fully executed market orders.Source: §2.1 · pp. 4–5

2.2 The Hamilton-Jacobi-Bellman quasi-variational inequality

  • The HJBQVI characterizes the trader’s value function, and the paper proves existence and uniqueness of its viscosity solution.An ansatz linear in cash and mark-to-market inventory reduces the problem to a system with 2N + 1 state variables.Source: §2.2 · p. 6
  • Conditional on market parameters, finite-difference schemes compute optimal volume splits and limits, while direct continuous Bayesian learning would make the control problem practically intractable.The proposed alternative updates parameters separately at the end of a trading slice and then revises the strategy.Source: §2.2 · p. 7

3 Adaptive trading strategies with Bayesian update

The paper develops conjugate Bayesian updates for venue, execution, and price parameters, then applies them slice by slice to recompute adaptive trading strategies. The approach balances explicit posterior formulas and practical updating against data requirements and time inconsistency.

3.1 Bayesian update of the model parameters

  • Conjugate Bayesian updates cover order-arrival intensities, executed proportions, venue characteristics, and mid-price parameters.The framework uses generic parametric priors chosen for conjugate updates, including Gamma, Dirichlet, and distributions for venue transition characteristics and price parameters.Source: §3 · p. 7, §3.1 · p. 7, §3.1.1 · p. 7

3.1.1 Update of the intensities

  • Execution-intensity estimates require sufficient observations in each spread-imbalance zone; a two-venue example contains 36 zones.The paper notes that sparse observations can make market-behavior estimates inaccurate and suggests prior choices that retain more influence when data are limited.Source: §3.1.1 · p. 8

3.1.2 Update of the executed proportion

  • Executed proportions update only when limit orders are partially or fully filled, while a venue-only parametrization converges faster by pooling observations.The more parsimonious specification lets market observations outweigh the prior more quickly because fewer parameters are updated.Source: §3.1.2 · p. 8

3.1.3 Update of the characteristics of the venues

  • Venue spread and imbalance parameters are updated from observed state transitions, helping distinguish venues after events and redirect orders toward less toxic liquidity platforms.The update separately infers transition probabilities and intensities under independent conjugate priors.Source: §3.1.3 · p. 9

3.1.4 Update of the mid-price

  • The mid-price update estimates drift and volatility from observed prices under Normal-Inverse-Gamma or simpler Normal priors, with prior weight controlled by ν.Volatility does not enter the HJBQVI explicitly but affects the trader’s optimal trading curve; larger ν gives market data more influence on drift estimation.Source: §3.1.4 · p. 10

3.2 Algorithm description

  • The practical algorithm estimates parameters, computes an optimal strategy for a short slice, observes executions and state changes, then updates parameters before the next slice.The procedure is explicitly decoupled from the optimization problem and is acknowledged to be time inconsistent.Source: §3.2 · p. 11
  • The slice-based procedure is time inconsistent because each updated strategy is recomputed after execution, but it is presented as a practical first step toward integrated Bayesian learning.The paper frames this practice as common for optimal control with online parameter estimation.Source: §3.2 · p. 11

4 Model extensions

The paper extends the trading framework with short- and long-horizon signals, market impact, and hidden liquidity. These additions broaden the model’s scope while increasing dimensionality in some cases and preserving tractability in others.

4.1.1 Short-term price signals

  • Short-term imbalance and spread signals can enter the price drift without increasing the state-process dimensionality.The modified HJBQVI replaces μq with μ + f_short(ψ, I).Source: §4.1.1 · p. 11

4.1.2 Mid/Long term and path-dependent price signals

  • Long-term or path-dependent signals can be represented through Markovian state variables, but the resulting 2N + 4 dimensional HJBQVI is suited to deep reinforcement learning rather than finite differences.The example uses moving-average and maximum-price information; additional path-dependent variables add dimensions.Source: §4.1.2 · p. 12

4.2 Extension 2: Market impact

  • Market-impact extensions model permanent, temporary, and transient effects, with exponentially decreasing kernels yielding a Markovian representation that adds 2N dimensions.The temporary component follows a square-root-law specification with exponents approximately equal to 1/2.Source: §4.2 · pp. 12–13

4.3 Extension 3: Hidden liquidity

  • Hidden liquidity can be incorporated into US-venue imbalance processes and trading signals, supporting arbitrage detection without increasing the dimensionality of Equation (2.3).The extension is motivated by the substantial hidden liquidity in US markets and cross-listed European-American trading.Source: §4.3 · p. 13

5 Numerical results

The numerical study specifies a two-venue liquidation problem, solves it with finite differences, and develops neural-network approximations for the strategy and value function. The neural approach benefits from encoding categorical market states and can be progressively adapted to changing settings.

5.1 Global parameters

  • The numerical example models a trader operating across two venues with discrete spread, imbalance, and execution regimes over ten one-minute slices.The initial inventory is q0 = 5 × 10^4.Source: §5.1 · pp. 13–14

5.2.1 Finite differences

  • Finite differences discretize inventories, time, and order volumes, with a terminal value of zero and a supremum evaluated over the discretized state space.The grids use Q = {0, ..., q0}, time steps within each slice, and order volumes from zero to q0.Source: §5.2.1 · pp. 14–15
  • Finite differences solve the two-venue problem with reasonable precision and calculation time, but their complexity grows exponentially as more venues are introduced.The most demanding computation is the supremum over a dimension involving venue count, inventory, order-volume, and market-state dimensions.Source: §5.2.1 · p. 15

5.2.2 Neural networks

  • The neural-network approach uses Actor-Critic approximations for both optimal controls and the corresponding value function, requiring 2#T networks across time steps.The value network is trained backward from the terminal condition using target values generated from the next-time-step approximation.Source: §5.2.2 · pp. 15–16
  • The target value changes considerably across spreads and imbalances, so the neural networks encode these categorical market states rather than treating them only as raw numerical inputs.The same categorical encoding is applied to both value-function and strategy-network approximations.Source: §5.2.2 · pp. 16–17
  • The strategy network takes inventory, spreads, and imbalances as inputs and outputs venue order volumes plus differentiable probabilities for limits −1, 0, and 1.Volumes are bounded by current inventory, while the trader can select the maximum-probability limit in practice.Source: §5.2.2 · p. 17
  • Normalization scales inventory, target values, and order volumes to improve training and makes pretrained models reusable across different problem settings.Separate network layers allow different learning rates for volume and limit outputs.Source: §5.2.2 · p. 18
  • The neural-network implementation can take about 1 minute on a simple two-CPU AWS instance and offers speed-up potential on more complex infrastructure.This contrasts with finite differences, which require recalculating the full grid when market parameters are updated.Source: §5.2.2 · pp. 18–19

5.3 Two identical venues

With identical venues, the value function and trading controls are computed using finite differences and neural networks. The strategies adapt limits and volumes to inventory, time, spreads, and imbalances, balancing execution probability against price improvement.

5.3.1 Value function

  • The value function’s maximum shifts toward zero as time advances, indicating a progressively lower optimal inventory to finish execution within the slice.Neural networks approximate the value function accurately.Source: §5.3.1 · p. 19

5.3.2 Strategy: limit orders and volumes with finite difference schemes

  • At the beginning of a slice, the trader waits at low inventory; as inventory rises, he moves from second-best to best and new-best limits to increase execution probability.This behavior expresses a trade-off between favorable execution prices and completing execution.Source: §5.3.2 · p. 20
  • The finite-difference implementation uses a homogeneous set of limit controls across market states, so a nominal limit p = −1 may represent p = 0 when volume is zero.This is attributed to the piecewise-monotone structure of the optimal limit strategy.Source: §5.3.2 · p. 20
  • Near the slice end, the trader posts earlier and accepts a less favorable limit to complete execution, although he does not fully liquidate inventory without a terminal penalty.The unexecuted inventory can be carried into later slices.Source: §5.3.2 · pp. 20–21
  • With unequal spreads, the trader collects spread from both venues at moderate inventories but stops posting in the higher-spread venue when execution urgency increases.For q > 30000, volume in the second venue decreases to zero while volume in the first venue continues increasing.Source: §5.3.2 · pp. 21–22
  • When one venue has a more favorable imbalance, the trader posts higher limits and more volume there, then concentrates execution there for large inventories.The favorable venue offers a higher execution probability, while large inventories reduce the value of collecting spread in the other venue.Source: §5.3.2 · p. 22

5.3.3 Strategy: limit orders and volumes with neural networks

  • Neural networks preserve the finite-difference limit strategy’s monotonicity while smoothing its controls across inventory and time.Early in the slice, small inventories favor the second-best limit; later, the best or new-best limit reduces execution risk.Source: §5.3.3 · pp. 22–23
  • Neural-network volume controls are smoothed versions of finite-difference controls and remain nearly symmetric across identical venues.Approximation may retain small nonzero orders in venues that finite differences would abandon, supporting exploration of venue parameters.Source: §5.3.3 · pp. 23–24

5.4 Two different venues

When venues differ in filling quality, the trader adjusts value, limits, and volumes across venues; neural networks broadly reproduce the finite-difference strategies while smoothing some controls.

5.4.1 Value function

  • The value function worsens from −45000 to −49000 at t = 0.5 and q = 50000 when the second venue has a worse filling ratio.The neural-network value function has a similar shape to the finite-difference solution.Source: §5.4.1 · p. 25

5.4.2 Strategy: limit orders and volumes with finite difference schemes

  • A higher spread in the second venue further reduces its posted volume because its lower filling probability makes collecting the larger spread less likely.The corresponding limit strategy changes across inventory ranges relative to identical venues.Source: §5.4.2 · p. 26
  • With a less favorable second venue, the trader posts earlier but less there, often choosing lower limits to raise execution probability at the cost of spread.The trader posts most volume in the first venue and can stop posting in the second near the slice end.Source: §5.4.2 · pp. 26–27
  • A more favorable imbalance in the second venue leads the trader to split liquidity between venues while choosing a more executable limit in the first venue.The split reflects the trade-off between the second venue’s imbalance and the first venue’s overall filling rate.Source: §5.4.2 · p. 27

5.4.3 Strategy: limit orders and volumes with neural networks

  • Neural networks reproduce the finite-difference limit and volume patterns, smooth the posted volumes, and retain exploratory nonzero orders in a less favorable venue.They place most volume in the better venue when its spread and filling ratio are more favorable, but increase second-venue volume when its imbalance is better.Source: §5.4.3 · pp. 28–29

5.5 Bayesian update

The Bayesian framework updates execution and market-dynamics parameters from successive execution slices, while the simulated strategy adapts order placement to venue-specific spreads, imbalances, and filling conditions.

5.5.1 Market simulation on a slice

  • The simulated trader changes venue volumes and limits according to the joint spread–imbalance configuration, balancing execution probability against spread collection.The execution trajectory follows the pre-computed Implementation Shortfall curve q⋆.Source: §5.5.1 · pp. 29–30

5.5.2 Update of the execution proportion

5.5.3 Update of the imbalance and the spread dynamics

  • Spread-transition parameters converge quickly, while imbalance parameters need a couple of slices for a good approximation and about a dozen slices for the right estimation.Finer slices can reduce the number of slices required for imbalance estimation.Source: §5.5.3 · p. 31, Figure 50 · p. 31, Figure 51 · p. 31

5.5.4 Update of the long term drift of the asset

  • The drift estimate reaches the true value µ_true = −0.5 after 20 slices from a prior µ = 0.1, despite an incorrect prior uncertainty ν = 0.02.The example uses continuously observed price increments.Source: §5.5.4 · p. 33, Figure 54 · p. 33

5.5.5 Update of the intensity of limit orders

  • Limit-order intensity is the hardest parameter to update because it depends on both venues and spans 32 states, so full convergence requires many observations.The paper notes that excessive precision is unnecessary for obtaining a strategy close to optimal.Source: §5.5.5 · p. 33, Figure 55 · p. 34, Figure 56 · p. 35

Appendix A Proof of Theorem 2.3

The appendix establishes the viscosity-solution properties of the trader’s HJBQVI through terminal conditions, test-function arguments, and dynamic-programming reasoning.

Appendix A Proof of Theorem 2.3

  • The proof begins by using the dynamic programming principle and a change of variables that removes the cash variable from the value function.Boundary and terminal arguments are handled through controlled exit times and Itô’s formula.Source: Appendix A · pp. 36–37
  • Test-function arguments establish the viscosity supersolution and subsolution inequalities for the HJBQVI, with the complementary part obtained by constant controls.The proof uses the terminal condition and local inequalities for smooth test functions.Source: Appendix A · pp. 36, 38
  • The appendix defines subjets and superjets and uses their limiting properties to formulate the viscosity-solution proof.The construction includes sequences converging to the contact point and associated derivative triples.Source: Appendix A · p. 38

The set

The appendix proves comparison for HJBQVI sub- and supersolutions and uses it to establish continuity and uniqueness of the viscosity solution.

The set

  • The comparison principle states that a lower-semicontinuous supersolution dominating an upper-semicontinuous subsolution at the terminal time dominates it throughout the domain.The result assumes polynomial growth.Source: The set · p. 39
  • The proof applies an exponential change of variables, penalization, and Ishii’s lemma to derive a contradiction from any positive interior gap.The penalized test function includes a quadratic state-distance term and temporal and spatial localization terms.Source: The set · pp. 39–41
  • Combining the comparison results with the terminal ordering yields equality of the semicontinuous envelopes, continuity, and uniqueness among continuous viscosity solutions.The maximum principle propagates equality across the full time domain.Source: The set · p. 41

Appendix B.1 Framework

The OTC market-making extension models quotes, transactions, inventory, cash, and portfolio value for multiple correlated assets. Its optimization is formulated through an HJB equation with inventory penalties and quote-dependent trading intensities.

OTC market-making framework

  • The framework extends cross-listed-stock trading to an OTC market maker quoting bid and ask prices across d correlated assets.The asset dynamics use drifts, volatilities, correlations, and a non-singular variance-covariance matrix.Source: Appendix B.1 · p. 42

Order-flow model

  • Trading volumes are modeled by marked point processes whose intensities depend on quote spreads and transaction sizes.The functions Λ_i,j model the probability of trading for each asset and side, while η_i,j models trade-size distributions.Source: Appendix B.1 · p. 42

Inventory dynamics

  • The market maker’s inventory changes through bid- and ask-side transactions across the assets.The inventory process is represented as a d-dimensional position vector updated by executed trade sizes.Source: Appendix B.1 · p. 42

Stochastic control

  • The optimization includes cash, marked-to-market portfolio value, and a running penalty against excessively large positions.The resulting value function is characterized by an HJB equation with terminal condition v(T, q) = 0.Source: Appendix B.1 · p. 42

Appendix B.2 Bayesian update for OTC market makers

The Bayesian OTC extension updates order-arrival, transaction-size, drift, and covariance parameters as observations accumulate. These updates let the market maker adapt quotes and risk factors to changing order flow and market dependence.

RFQ intensity updates

  • With sufficiently many observations, the Bayesian estimate of each RFQ intensity converges to the market’s real intensity, reducing the influence of prior parameters.The convergence follows from the law of large numbers.Source: Appendix B.2 · p. 43

Transaction-size updates

  • Bayesian updates can revise the transaction-size distribution’s shape parameter, scale parameter, or both according to the trader’s confidence in their estimates.The model uses a Gamma distribution for transaction sizes and allows selective parameter updating.Source: Appendix B.2 · p. 43

Factorial covariance model

  • A factorial representation reduces the multi-asset market-making problem from d dimensions to three factors associated mainly with the largest eigenvalues of the covariance matrix.These factors drive the market maker’s quotes.Source: Appendix B.2 · pp. 43–44

Adaptive covariance model

  • A sectorial tail event can raise within-sector correlations, alter covariance eigenvalues, and change quotes needed to avoid long positions in declining assets.The adaptive extension places a Normal-Inverse-Wishart prior on the drift and covariance parameters.Source: Appendix B.2 · p. 44
  • As more price information arrives, Bayesian estimates of drift and covariance converge and the market maker can recompute factors and adjust quotes.The extension is more parsimonious than the high-mid-frequency order-book setting, so its parameter convergence is expected to be faster.Source: Appendix B.2 · p. 44

RFQ intensity updates

  • Bayesian updates of RFQ arrival intensities let the market maker adjust quotes to observed request frequencies on each asset and side.Higher observed ask or bid request activity supports raising the corresponding ask or bid quote.Source: Appendix B.2 · pp. 43–44
Loading 2008.07807v1…