Source-linked AI summary
SAiFE-gym: Model-based Environments for Automated Market Making with Concentrated Liquidity
Georgios Chionas, Charalampos Kleitsikas, Stefanos Leonardos, Leandro Sánchez-Betancourt, Carmine Ventre
TL;DR
Research on AMMs with concentrated liquidity faces fragmented simulation environments, while stylized models often remain low dimensional. SAiFE_gym provides modular RL environments and evaluates agents across controlled variations in market parameters. The authors report experiments in different economic settings, supported by vectorized simulation across many trajectories.
Problem
AMM studies often use low-dimensional stylized models, and the research community lacks an open-source, unified, extensible simulation collection tailored to AMM microstructure.
Source: §1 · p. 1Method
SAiFE_gym assembles modular RL environments for AMMs and uses vectorized trajectories to model different economic settings and support scalable training.
Source: §1 · p. 1Results
The authors demonstrate SAiFE_gym’s versatility through experiments in different economic settings, with policies evaluated across many trajectories under controlled variations of the market model.
Source: Conclusion and Further Research · p. 7Takeaways & Limitations
The modular environments let researchers and practitioners create desired economic settings for studying trading in constant product markets with concentrated liquidity.
Source: Conclusion and Further Research · p. 7The RL environments assume the risky asset’s price formation is exogenous to the AMM.
Source: §3.2 · p. 3
Abstract
from arXiv · showhide
We present SAiFE_gym, a Python module that provides a collection of simulation environments for studying trading problems in Constant Product Markets (CPMs) with Concentrated Liquidity (CL). These markets give Liquidity Providers (LPs) granular control over how their capital is allocated and enable them to adjust their range of liquidity provision dynamically based on market conditions, which in turn, dictates how they earn fees. We decompose the microstructure of CPMs with CL in interactive components that allow researchers and practitioners to capture various economic settings. We employ a vectorized approach to optimize our environments, making them scalable for high dimensional Reinforcement Learning (RL) workflows that best describe sequential decision problems. We demonstrate the benefits of our environments by evaluating the performance of RL agents in CPMs with CL under uncertainty in market parameters.
1 Introduction
AMMs provide blockchain trading through bonding curves, but high-dimensional market-making problems and fragmented simulation tools leave a need for extensible RL environments. SAiFE_gym assembles AMM components and supports scalable experiments, including evaluation under uncertainty in market parameters.
1 Introduction
- AMMs use bonding curves to clear supply and demand, and have facilitated $4 trillion in trading volume across Ethereum and compatible networks.They were proposed as an alternative to limit order books suited to blockchain computational and storage constraints.Source: §1 · p. 1
- SAiFE_gym addresses the lack of an open-source, unified, extensible AMM simulation collection amid fragmentation and high-dimensionality limits in stylized models.Prior mathematical-finance models often focus on LPs or liquidity takers and remain low dimensional to accommodate analytical solutions.Source: §1 · p. 1
- The environments assemble interacting AMM components modularly, letting researchers and practitioners combine market dynamics to represent different economic settings.The described microstructure includes price evolution, order flow and toxicity, price impact, liquidity distribution, gas costs, and latency.Source: §1 · p. 1
- SAiFE_gym connects its environments to Stable Baselines 3 (SB3), whose rich suite of state-of-the-art RL algorithms enables plug-and-play training on previously unstudied problems and development of customizable agents.Source: §1 · p. 1
- The authors evaluate RL agents under uncertainty in market parameters, using domain randomization, an approach also applied in simulation-to-real work [20].The module is available on GitHub.Source: §1 · p. 2
2 Related Work
Research on concentrated-liquidity markets examines LP strategies and market mechanics, while RL offers a complementary approach to high-dimensional stochastic control. Model-based RL infrastructure remains limited in DeFi compared with limit-order-book trading.
2 Related Work
- The paper situates its AMM work alongside constant-function market research on arbitrage alignment and price oracles [5].Uniswap v2 is described as an early constant-product market, while Uniswap v3 enables concentrated liquidity.Source: §2 · p. 2
- Concentrated liquidity lets LPs specify active price ranges, addressing capital inefficiency in simple AMMs where substantial liquidity may go unused.The mechanism expands LPs’ action space beyond passive provision.Source: §2 · p. 2
- Prior LP studies examine dynamic provision, loss decomposition, and stochastic-control strategies, while some models abstract away blockchain gas fees and latency [15].The cited closed-form control model reallocates accumulated wealth across price intervals as the price evolves.Source: §2 · p. 2
- RL complements numerical methods for high-dimensional market-making problems, but model-based RL tools are more established for limit order books than for DeFi.Modular Gym-compatible environments exist for limit-order-book trading, while the authors characterize their own work as among the first modular infrastructure for testing RL agents in AMMs.Source: §2 · p. 2
3 Design of the Module
The module implements LP interaction with a CPM with CL as a discrete-time approximation to a continuous-time stochastic control problem. Each step comprises observation, action, market operation, and time advancement.
3 Design of the Module
- The RL environments discretize LP interaction over a continuous-time horizon into observation, action, market operation, and time-advancement phases.These phases are implemented in AMMEnvironment, inherited by the RL environments.Source: §3 · p. 2
3.1 Constant product markets and concentrated liquidity
- CL lets LPs choose price intervals for liquidity positions, changing tick-range depth while taker trades move the marginal rate and fees are shared pro rata.A position holds both assets when the marginal rate is within its interval and only one asset when it is outside.Source: §3.1 · p. 3
3.2 Mid-price processes
- The environments include exogenous external midprice models and an endogenous marginal exchange rate modeled as a pure jump process on a discrete grid.Implemented external-price choices include geometric Brownian motion, arithmetic Brownian motion, and arithmetic midprice with an Ornstein–Uhlenbeck alpha signal.Source: §3.2 · p. 3
3.3 Arrival processes
- Liquidity taker arrivals are modeled with intensities combining baseline noise, liquidity depth, and informed trading responses to the external-price gap.Exponential kernels encode how liquidity ahead of the current tick affects future arrivals, allowing LP actions to influence order flow.Source: §3.3 · pp. 3–4
3.4 Price Impact
- Price impact is modeled either as a one-tick move by trade direction or as a depth-dependent move based on smoothed neighboring liquidity and trade size.The authors describe the liquidity-kernel approach as, to their knowledge, the first considered in CPMs with CL.Source: §3.4 · p. 4
3.5 Blockchain Frictions
- Blockchain frictions include fixed gas costs for rebalancing and deterministic latency represented by the number of decisions during training and evaluation.Gas costs scale with rebalancing frequency.Source: §3.5 · p. 4
3.6 Agents
- LP agents can open active or single-sided positions, while liquidity takers control signed trading speed and incur depth-dependent price impact.When LPs act, they withdraw their holdings, collect accrued fees, and reinvest their wealth into the new position.Source: §3.6 · pp. 4–5
3.7 Reward Functions
- The reward can measure mark-to-market PnL net of gas or add terminal and running inventory penalties around a target holding.The parameter φ quantifies the agent’s urgency to liquidate the risky asset.Source: §3.7 · p. 5
4 Reinforcement Learning
Section 4 describes wrappers that connect SAiFE_gym’s AMM engine to RL agents, a vectorized simulation design, and experiments using stationary and domain-randomized market dynamics. The vectorized approach speeds up rollouts and reduces variation in reported mean rewards as parallel trajectory counts increase.
4.1 Wrappers
- The wrappers expose standard observation, action, reward, and termination transitions while providing agent-specific action, decision-frequency, and observation-space options.DecisionStrideEnv separates market and agent decision frequencies; other wrappers reparameterize actions or customize observations.Source: §4.1 · pp. 5–6
4.2 Vectorized approach
- ≈0.05 s versus ≈1.6 s for 1,000 trajectories, the vectorized approach achieves roughly 30× speedup over multiprocessing.The comparison uses identical per-trajectory workloads.Source: §4.2 · p. 6
- Increasing parallel trajectories from 10 to 10,000 reduces variance in reported mean episode reward, with little training-performance difference between 100 and 1,000 trajectories.Figure 4 tracks mean reward per episode across training updates for different parallelization levels.Source: §4.2 · p. 6, Figure 4 · p. 6
4.3 Reinforcement Learning under Domain Randomization
- The stationary PPO example uses fixed market parameters, while domain randomization samples volatility and noise-trading intensity during training and evaluates policies across in- and out-of-distribution parameters.The randomized training ranges are σ∼U(0.01, 0.05) and α1∼U(300, 600); evaluation uses nine parameter points.Source: §4.3 · pp. 6–7
Conclusion and Further Research
SAiFE_gym provides modular RL environments for trading in constant product markets with concentrated liquidity, supporting experiments across economic settings. The authors identify multi-agent environments, expanded LP actions, and additional liquidity-taker agents as directions for extending the work.
Conclusion
- SAiFE_gym’s vectorized environments make experiments computationally tractable and support policy evaluation across many trajectories under controlled market-model variations.The environments assemble AMM microstructure components so researchers and practitioners can configure economic settings.Source: Conclusion and Further Research · p. 7
Further Research
- Multi-agent LP competition, expanded LP action spaces including dynamic fees, and additional liquidity-taker agents are proposed future extensions.These extensions could support simulations where market dynamics emerge from interactions among heterogeneous LPs, arbitrageurs, and noise traders.Source: Conclusion and Further Research · p. 7
Conclusion
- Figure 6 compares a periodic-rebalancing baseline, nominal PPO, and domain-randomized PPO across in- and out-of-distribution market parameters.The dashed area marks in-distribution parameters.Source: Figure 6 · p. 8