Source-linked AI summary
Beyond Imitation: Constraint-Aware Trajectory Generation with Flow Matching For End-to-End Autonomous Driving
Lin Liu, Guanyi Yu, Ziying Song, Junqiao Li, Caiyan Jia, Feiyang Jia, Peiliang Wu, Yandan Luo
TL;DR
CATG addresses mode collapse and the difficulty of integrating safety and physical constraints into trajectory generation. It uses constrained flow matching with conditional controls for constraints and driving style, achieving an EPDMS score of 51.31 on NAVSIM V2. Sampling 100 steps remains computationally expensive, creating a stated efficiency limitation.
Problem
Imitation-learning planners can lack behavioral diversity, while generative planners may suffer mode collapse or lack direct hard-constraint integration for safe trajectories.
Source: §1 · p. 1Method
CATG uses flow matching without imitation learning, integrates constraints through guided probability flows and energy-based mechanisms, and conditions generation on reward signals for driving-style control.
Source: §1 · p. 1Results
Takeaways & Limitations
CATG provides a multimodal planner with explicit constraint enforcement and controllable aggressive or conservative trajectory generation.
Source: §1 · p. 1Generating trajectories with 100 sampling steps remains computationally expensive, while reducing steps may degrade trajectory quality.
Source: §5 · p. 4
Abstract
from arXiv · showhide
Planning is a critical component of end-to-end autonomous driving. However, prevailing imitation learning methods often suffer from mode collapse, failing to produce diverse trajectory hypotheses. Meanwhile, existing generative approaches struggle to incorporate crucial safety and physical constraints directly into the generative process, necessitating an additional optimization stage to refine their outputs. To address these limitations, we propose CATG, a novel planning framework that leverages Constrained Flow Matching. Concretely, CATG explicitly models the flow matching process, which inherently mitigates mode collapse and allows for flexible guidance from various conditioning signals. Our primary contribution is the novel imposition of explicit constraints directly within the flow matching process, ensuring that the generated trajectories adhere to vital safety and kinematic rules. Secondly, CATG parameterizes driving aggressiveness as a control signal during generation, enabling precise manipulation of trajectory style. Notably, on the NavSim v2 challenge, CATG achieved 2nd place with an EPDMS score of 51.31 and was honored with the Innovation Award.
1. Introduction
CATG addresses limited behavioral diversity and weak constraint integration in trajectory generation by combining flow matching with explicit controls and constraints. It achieves an EPDMS score of 51.31 on the ICCV NAVSIM V2 challenge while demonstrating planning accuracy and out-of-distribution generalization.
Motivation
- Imitation-learning methods often produce homogenized trajectories because expert demonstrations lack sufficient strategy diversity.This limits behavioral diversity in multimodal planning, especially where multiple trajectories may be appropriate.Source: §1 · p. 1
- Generative alternatives can remain vulnerable to mode collapse or produce unsafe, difficult-to-interpret trajectories because they lack explicit hard-constraint integration.The cited constraints include obstacle avoidance and traffic-rule compliance.Source: §1 · p. 1
Method
- CATG is a flow-matching trajectory generator that eliminates imitation learning while supporting diverse and flexible conditional controls.The framework is designed to model multimodal trajectory generation without relying on behavior cloning.Source: §1 · p. 1
- CATG integrates feasibility and safety constraints through prior-informed anchors that construct constraint-guided probability flows and energy-based guidance toward feasible regions.This mechanism directly injects constraints into the generative process rather than relying only on post-generation refinement.Source: §1 · p. 1
- Environmental reward signals condition generation, enabling controllable trade-offs between aggressive and conservative driving styles during inference.Driving style is treated as a controllable aspect of trajectory generation.Source: §1 · p. 1
Results
- CATG achieves an EPDMS score of 51.31 on the ICCV NAVSIM V2 End-to-End Driving Challenge and demonstrates robust generalization to out-of-distribution data.The result is reported when CATG is combined with an open-source scoring model.Source: §1 · p. 2
2. Preliminary
Flow matching constructs a probability path between a simple source distribution and an unknown target data distribution using a learned time-dependent vector field. CATG uses rectified flow to approximate this transport through linear interpolation and least-squares drift learning.
Flow Matching
- A time-dependent vector field constructs a flow whose probability path connects source distribution X0 ∼ π0 and target distribution X1 ∼ π1 through an ordinary differential equation.The flow is represented as a time-dependent diffeomorphic map.Source: §2 · p. 2
- Flow matching learns a neural-network vector field that transports samples from a simple source distribution, such as a standard normal, toward an unknown target data distribution.The formulation assumes access to target samples but not the target density function.Source: §2 · p. 2
Rectified Flow
- CATG uses rectified flow, directing the drift from X0 toward X1 along the direction X1 − X0 of a linear path.The drift is learned by solving a least-squares regression problem.Source: §2 · p. 2
- The intermediate state Xt is defined as the linear interpolation between the source sample X0 and target sample X1.This interpolation provides the state on which the drift direction is modeled.Source: §2 · p. 2
3. Method
CATG uses flow matching with flexible conditioning signals and integrates constraint-guided corrections directly into trajectory generation. It also conditions inference on reward signals to control driving aggressiveness.
3.1. Flexible conditioning signal
- CATG fuses latent flow features with perceptual features, trajectory rewards, anchors, target points, and driving commands through cross-attention.The model uses classifier-free conditioning and samples trajectories through a decoded velocity field refined by three correction strategies.Source: §3.1 · p. 2, Figure 1 · p. 3
- An 8,192-anchor vocabulary guides diverse generation, with training anchors selected by DTW similarity and inference anchors ranked by GTRS.The top-100 scored anchors serve as conditional inputs during inference.Source: §3.1 · p. 2
3.2. Constraint-Aware Trajectory Generation
- CATG constrains generation through velocity-field correction, DAC-compliant initialization, and energy-based constraint-aware training.These strategies target road compliance, while the method is described as adaptable to other constraint types.Source: §3.2 · pp. 3–4
- Starting from a DAC-compliant anchor enables more plausible trajectories, even when that anchor performs poorly on other evaluation metrics.The anchor is refined during generation rather than used unchanged as the final trajectory.Source: §3.2 · pp. 3–4, Figure 2 · p. 4
- CAT encodes the DAC constraint as an energy function during training, using a two-stage Flow Matching and Energy Matching procedure.The energy is computed from a Euclidean Signed Distance Field and penalizes deviations toward the road boundary.Source: §3.2 · p. 4
3.3. Reward as condition:
- An EP score controls driving style at inference, with EP set to 1 encouraging more aggressive behavior.The score is computed by evaluating NavTrain ground-truth trajectories in the NAVSIM simulator.Source: §3.3 · p. 4
4. Experiments
CATG is trained in two stages and evaluated in NAVSIM V2 using sampled candidate trajectories ranked by a pretrained scorer. The reported experiments present the proposed architecture’s NAVSIM V2 results.
4.1. Experiments Setup
- CATG uses two-stage training: 90 epochs for Flow Matching, perception, and segmentation, followed by 10 Energy Matching fine-tuning epochs.Both stages use batch size 64 and learning rate 2 × 10^-4 on the NavTrain split.Source: §4.1 · p. 4
- During inference, CATG generates 100 candidate trajectories with 100 sampling steps and selects the final output by ranking them with GTRS.The ranking pool also includes the trajectory vocabulary anchors.Source: §4.1 · p. 4
4.2. Experiments result
- The proposed CATG architecture’s NAVSIM V2 results are reported in Table 1.The supplied passages identify the table but do not provide its cell values.Source: §4.2 · p. 4, Table 1 · p. 4
5. Limitation
CATG’s 100-step trajectory sampling remains computationally expensive, and accelerating it may degrade generated trajectory quality.
5. Limitation
- 100-step trajectory sampling is computationally expensive, creating a future need to improve efficiency while preserving trajectory quality.The paper identifies faster sampling with maintained quality as future work.Source: §5 · p. 4
6. Conclusion
CATG is an end-to-end flow-matching planner that incorporates flexible conditional signals and explicit constraints during generation. Its reported experimental result is an EPDMS of 51.31.
6. Conclusion
- CATG combines flow matching, flexible conditional control, and three strategies for enforcing explicit constraints throughout trajectory generation.The conclusion reports an EPDMS of 51.31 in Table 1.Source: §6 · p. 4