Source-linked AI summary
ViperQ: Order Flow Pattern Recognition via Auction Market Theory for Reinforcement Learning Trading
Asser Moustafa, Rares-Mihail Neagu, Jugal Kalita
TL;DR
Auction Market Theory had not been instantiated as a state representation for a peer-reviewed RL trading agent; ViperQ encodes its primitives in a 20-dimensional vector for PPO agents. On a held-out twelve-month test partition, it reports +163.6% ROI on TSLA and +116.5% ROI on NVDA.
Problem
Practitioner microstructure pattern theory had not been instantiated as a state representation for a peer-reviewed reinforcement learning trading agent.
Source: §1 · p. 1Method
Results
+163.6% ROI on TSLA and +116.5% ROI on NVDA on a held-out 12-month test partition, with zero leverage and regulatory-fee accounting.
Source: §1 · p. 2Takeaways & Limitations
Two reward configurations yield distinct profitable policies from one architecture: a TSLA scalper and an NVDA trend-following risk-averse agent.
Source: §1 · p. 2Evaluation covers two highly volatile US single-name equities over one 80/20 temporal split; generalization to other assets, markets, and intraday regimes is untested.
Source: §7 · p. 6
Abstract
from arXiv · showhide
Reinforcement learning trading systems published in the academic literature overwhelmingly rely on price-aggregate state representations (OHLCV bars) or limit-order-book depth features, leaving microstructure pattern theories from the practitioner literature, namely Auction Market Theory and Market Profile, without a peer-reviewed computational instantiation. We present ViperQ, a reinforcement learning system whose state representation is built explicitly from Auction Market Theory primitives: Volume Point of Control, Value Area position, Low Volume Node flags, Cumulative Volume Delta divergence, and tape-velocity signatures, assembled into a 20-dimensional Z-normalised vector. Two Proximal Policy Optimisation agents are trained with a prospect theory-grounded asymmetric reward function that penalises losing holds at a magnitude consistent with Kahneman and Tversky's loss-aversion coefficient. Evaluated on a held-out twelve-month partition of institutional tick data the agents have never seen, ViperQ achieves +163.6% ROI on TSLA (-27.5% max drawdown, 27,019 trades) and +116.5% ROI on NVDA (-47.8% max drawdown, 12,892 trades) under zero leverage. The results establish Auction Market Theory features as a tractable structured input modality for sequential decision-making on financial time series and motivate further work on microstructure-aware policy learning.
1. Introduction
ViperQ addresses the absence of peer-reviewed RL agents using Auction Market Theory (AMT) state features with an explicitly structured representation and asset-specific PPO policies. On a held-out 12-month partition, it reports positive ROI for both TSLA and NVDA.
1. Introduction
- ViperQ encodes AMT primitives—including VPOC, Value Area position, LVN flags, CVD divergence, tape velocity, and order-flow imbalance—in a 20-dimensional state vector.The vector is Z-normalised and consumed by a PPO agent at 1-second resolution on institutional tick data.Source: §1 · pp. 1–2
- The paper frames its task as recognizing balance versus imbalance regimes from microstructure observations, with sequential policy learning conditioned on that recognition.It presents the structured representation, rather than the trading application, as its broadest contribution to pattern recognition.Source: §1 · pp. 1–2
- The Sniper reward applies asymmetric pain penalties consistent with Kahneman and Tversky’s loss-aversion coefficient and adds synthetic in-training transaction-fee inflation.The fee inflation is intended to favor high-conviction setups over micro-scalping.Source: §1 · p. 2
- A single architecture with two reward configurations converges to a TSLA high-frequency scalper and an NVDA trend-following, risk-averse policy.The paper characterizes both policies as profitable local optima from the same state representation.Source: §1 · p. 2
- +163.6% ROI on TSLA and +116.5% ROI on NVDA are reported on a strictly held-out 12-month test partition.The evaluation uses zero leverage and regulatory-fee accounting.Source: §1 · p. 2
2. Related Work
Related work covers deep learning on limit-order books, RL trading and execution, microstructure features, Auction Market Theory, and pattern-recognition applications. The paper distinguishes ViperQ by explicitly using Auction Market Theory features as RL state inputs.
Deep learning on limit order books
- DeepLOB and its extensions learn representations from raw LOB snapshots, but the cited systems do not encode prior microstructure theory and typically act as classifiers rather than control agents.The passage identifies convolutional, transformer, and multi-asset attention approaches in this line of work.Source: §2 · p. 2
Reinforcement learning for trading and execution
- Prior RL work spans daily trading and execution through higher-frequency PPO agents, simulation environments, and market-making systems.Surveys catalog the field, while Pippas et al. identify microstructure-feature integration as a methodological gap.Source: §2 · p. 2
Microstructure-feature work
- Jaddu and Bilokon combine order-flow imbalance features with RL, whereas the reviewed systems do not use ViperQ’s AMT constructs.The passage identifies their work as the closest prior work to ViperQ in spirit.Source: §2 · p. 2
Auction Market Theory
- Market Profile and Auction Market Theory frame intraday price discovery as oscillation between mean-reverting balance and one-sided- liquidity-vacuum imbalance states.The passage reports that these constructs have been adopted in proprietary trading practice for over three decades, but were not found as explicit state features in a peer-reviewed RL system.Source: §2 · p. 2
Pattern recognition precedents
- Prior application-driven RL work includes chip-macro placement and financial trading, establishing precedent for domain-application contributions framed as pattern-recognition methodology.The cited examples are Liu et al. and Tsantekidis et al..Source: §2 · p. 2
3. Method
ViperQ converts order-flow and Auction Market Theory patterns into a structured state for PPO trading, with asymmetric rewards and asset-specific training. The method combines pattern extraction, policy learning, and risk-bounded execution.
3. Method
- Trading is modeled as a Markov Decision Process in a custom Gymnasium environment, with executions priced at the next bar’s open.This execution timing is used to eliminate look-ahead bias.Source: §3 · p. 2
3.1. Order flow patterns: matching engine and Auction Market Theory
- The state captures matching-engine liquidity consumption and balance-to-imbalance transitions, with the agent targeting transitions between value areas.The agent measures aggressive flow through CVD, tape velocity, and order-flow imbalance; it remains flat during balance and holds through breakouts.Source: §3.1 · p. 3, Figure 1 · p. 3, Figure 2 · p. 3
3.2. State representation
- The 20-dimensional Z-score-normalized state groups Auction Market Theory and microstructure features, runtime position variables, and lagged returns for short-horizon memory.VPOC is approximated by a one-hour rolling VWAP, and rolling-window normalization is used across volatility regimes.Source: §3.2 · p. 3, Table 1 · p. 3
3.3. Sniper reward function
- The Sniper reward aggregates six components, including realized PnL, asymmetric unrealized-PnL shaping, trend bonuses, time decay, stop losses, and drawdown penalties.The loss asymmetry is deliberately stronger than the Prospect Theory λ ≈ 2.25 baseline to compensate for 1-second reward density and suppress the disposition effect.Source: §3.3 · pp. 3–4
3.4. Asset-specific dual-PPO training
- Two PPO agents share the state representation but use asset-specific reward configurations, including an added held-winner bonus for NVDA; both enforce a 300-second minimum hold.The paper motivates PPO’s discrete-action suitability, on-policy clipped updates, sample efficiency, and comparability with prior microstructure-RL work.Source: §3.4 · p. 4
- The five-layer pipeline ingests institutional ticks, builds 1-second OHLCV and order-flow features, engineers the state, runs dual-PPO inference, and applies risk-bounded execution.The trading environment prices executions at the next bar’s open to eliminate look-ahead bias.Source: §3 · p. 2, §3.4 · p. 4, Figure 3 · p. 4
4. Risk-bounded execution
The execution layer converts discrete agent actions into cash-constrained position sizes and applies hard intraday risk controls.
4. Risk-bounded execution
- Position sizing adjusts dollar risk by conviction and volatility, then clamps shares to available cash so leverage is impossible.The sizing uses current equity, ATR, and available cash.Source: §4 · p. 4
- Three hard rules cap exposure: a −2% daily loss lockout, forced exit beyond $200 unrealized loss per trade, and flattening all positions at 15:55 ET.The daily loss limit, 2-ATR stop distance, and end-of-day flatten are described as standard intraday proprietary-trading conventions.Source: §4 · p. 4
- The −2% daily loss limit, 2-ATR stop distance, and end-of-day flatten follow standard intraday proprietary-trading conventions rather than hand-tuned parameters.The rules constrain trading to intraday positions and eliminate overnight gap exposure through end-of-day flattening.Source: §4 · p. 4
5. Experimental setup
The evaluation uses chronological held-out institutional tick data for TSLA and NVDA, with hyperparameters fixed through specified selection mechanisms and deterministic backtests.
5. Experimental setup
- An 80/20 chronological split reserves the final twelve months of Databento NVDA and TSLA data as a test partition excluded from training and hyperparameter selection.Nanosecond ticks are aggregated into 1-second OHLCV bars with Delta and trade count.Source: §5 · p. 4
- Hyperparameters are assigned through validation-slice selection, industry convention, theory-derived values, or Stable-Baselines3 defaults; a formal sensitivity sweep is deferred.Table 3 summarizes the constants and their selection mechanisms.Source: §5 · pp. 4–5, Table 3 · p. 5
- Deterministic backtests start with $10,000 per asset, execute at the next bar’s open, and charge real regulatory fees rather than the synthetic training fee.The evaluation fee is $0.0002 per share; the synthetic training fee is inactive.Source: §5 · p. 5
6. Results
On a held-out twelve-month partition, ViperQ reports substantial returns with zero leverage and regulatory fees. Its comparison with prior systems is qualified by inconsistent benchmarks and deeper drawdowns, while development observations describe how reward and holding constraints shape agent behavior.
6.1. Out-of-sample performance
- +163.6% ROI with −27.5% maximum drawdown across 27,019 TSLA trades, and +116.5% ROI with −47.8% maximum drawdown across 12,892 NVDA trades, on the held-out partition.Both results use zero leverage and full regulatory-fee accounting.Source: Table 4 · p. 5, §6.1 · p. 5
6.2. Comparison with published RL trading systems
- ViperQ’s +163.6% TSLA ROI exceeds the surveyed daily-horizon range, but deeper drawdowns and differing time resolutions preclude a claim of Pareto dominance.The comparison is landscape positioning because published systems differ in assets, horizons, data resolution, capital assumptions, and reporting conventions.Source: §6.2 · p. 5, Table 5 · p. 5
6.3. Behavioural divergence between the two agents
- The synthetic $0.50/share fee prevents sub-tick overtrading, while removing the 300-second hold constraint makes the TSLA policy collapse into rapid scalping.Without the drawdown shock, both agents converge to similar high-frequency behavior, eliminating the asset-specific divergence; formal ablation remains future work.Source: §6.3 · pp. 5–6
6.4. A note on Sharpe-style metrics
- Sharpe-style annualised metrics are omitted because the 1-second annualisation factor amplifies microstructure noise and yields values not comparable to daily-horizon Sharpe figures.The paper instead reports absolute ROI and maximum drawdown as more interpretable at this resolution.Source: §6.4 · p. 6
7. Limitations
The evaluation covers two highly volatile US single-name equities on one temporal split, and reward-component ablations and hyperparameter sensitivity remain untested. NVDA’s drawdown is materially deeper than typical daily portfolio-rebalancing systems, limiting the fit for capital preservation.
7. Limitations
- The evaluation covers only NVDA and TSLA over one 80/20 temporal split; generalisation to lower-volatility names, non-US markets, and other intraday regimes is untested.A formal one-factor-at-a-time reward ablation and hyperparameter sensitivity sweep are also reserved for future work.Source: §7 · p. 6
- −47.8% NVDA maximum drawdown exceeds the −5% to −15% range typical of daily-horizon portfolio-rebalancing RL systems, making ViperQ inappropriate for capital-preservation mandates.The paper characterises this as trading drawdown depth for absolute-return magnitude, not Pareto superiority.Source: §7 · p. 6
8. Conclusion
ViperQ instantiates Auction Market Theory primitives as an RL trading state and reports out-of-sample returns on TSLA and NVDA. The conclusion argues that this structured prior supports interpretable, profitable policies, while identifying broader testing and sensitivity analysis as future work.
8. Conclusion
- ViperQ is presented as the first peer-reviewed RL trading system whose state representation is built explicitly from Steidlmayer–Dalton Auction Market Theory primitives.The 20-dimensional vector encodes VPOC, Value Area position, Low Volume Node flags, CVD divergence, tape velocity, and order-flow imbalance.Source: §8 · p. 6
- +163.6% ROI on TSLA and +116.5% ROI on NVDA are reported over a held-out twelve-month partition with zero leverage and full regulatory-fee accounting.The conclusion attributes the policies to a prospect-theory-grounded asymmetric reward function and asset-specific behavioral constraints.Source: §8 · p. 6
- The conclusion presents the AMT state as a microstructure inductive prior supporting interpretable, profitable out-of-sample behavior from a small neural policy.It highlights sample efficiency, named interpretable dimensions, and use of the same state vector for distinct profitable policies without feature re-engineering.Source: §8 · p. 6
- Future work targets formal reward-component sensitivity analysis, a broader asset universe, and transfer of the AMT state representation to alternative policy classes.Source: §8 · p. 6