PaperlayerView exact revision on arXiv

Source-linked AI summary

Flow-ERD: Agent-type Aware Flow Matching with Entropy-Regularized Distillation for Diverse Traffic Simulation

Seulbin Hwang, Kiyoung Om, Daejung Kim, Jinhan Lee

arXiv:2607.06957v1cs.ROcs.LG

TL;DR

Traffic simulators have largely prioritized realism, leaving diversity underexplored and difficult to assess. Flow-ERD jointly models realistic, type-consistent multimodal behavior and preserves rollout diversity, achieving state-of-the-art realism and dominating the realism–diversity Pareto frontier on WOSAC.

  • Problem

    Existing evaluation can miss plausible modes absent from the logged future, limiting assessment of realistic rollout diversity.

  • Method

    Flow-ERD combines agent-type aware flow matching for realistic multimodal actions with entropy-regularized distillation that preserves minority modes during closed-loop fine-tuning.

  • Results

    Flow-ERD achieves state-of-the-art realism on WOSAC and dominates the validation realism–diversity Pareto frontier among reproducible baselines.

  • Takeaways & Limitations

    The results support evaluating traffic-simulation diversity explicitly rather than inferring it from realism alone.

Abstract

from arXiv · show

Realistic and diverse traffic simulation is essential to autonomous driving development. Yet prevailing benchmarks predominantly reward realism, and recent methods have optimized accordingly, leaving diversity underexplored. We introduce \textbf{Flow-ERD}, a multi-agent simulator that pursues realism and diversity jointly. Its backbone, \textbf{Agent-Type Aware Flow Matching} (AFM), couples flow matching's multi-modal expressiveness with type-specific kinematic execution. It preserves fine-grained diversity while keeping motions consistent with each agent type. A second stage, \textbf{Entropy-Regularized Distillation} (ERD), fine-tunes the closed-loop rollout distribution with an entropy-regularized reverse-KL objective. This mitigates covariate shift while explicitly preventing collapse onto high-density modes. We evaluate Flow-ERD with a log-free diversity metric alongside standard realism scores. Flow-ERD ranks first on the WOSAC test benchmark and dominates the realism--diversity Pareto front among reproducible baselines. Our project page is available \href{https://seulbinhwang.github.io/flow-erd-project-page/}{here}.

I. INTRODUCTION · II. RELATED WORKS

Flow-ERD targets realistic and diverse closed-loop traffic simulation, addressing limitations in existing realism-focused evaluation and model design. It combines Agent-Type Aware Flow Matching with Entropy-Regularized Distillation and evaluates both realism and rollout diversity on WOSAC.

  • I. INTRODUCTION: Traffic simulation supports controlled autonomous-driving validation and planning-policy development, requiring surrounding agents to behave realistically and span plausible futures.The required behavior includes reacting to one another in closed loop across vehicles, cyclists, and pedestrians.
  • I. INTRODUCTION: WOSAC realism scores compare against a single logged future, so they cannot distinguish merely fitting that future from generating diverse plausible behaviors.Recent learning-based simulators have substantially improved realism under WOSAC.
  • II. RELATED WORKS: Discrete action vocabularies encourage realistic, type-compatible motion but coarsen fine-grained behavior and inherently bound attainable diversity.Continuous representations, especially diffusion models, remove token-vocabulary constraints.
  • II. RELATED WORKS: Open-loop training creates closed-loop covariate shift as a model’s actions feed subsequent predictions and accumulate error over time.Prior remedies include log-proximal rollout augmentation, realism-oriented reinforcement learning, and generative-adversarial rewards.
  • I. INTRODUCTION: AFM uses continuous flow matching for multi-modal expressiveness and agent-type-specific kinematic transitions for realizable, type-compatible motion.Continuous action sampling avoids token-codebook limits while kinematic execution constrains unconstrained motion.
  • I. INTRODUCTION: ERD fine-tunes the closed-loop rollout distribution with an entropy-regularized reverse-KL objective, mitigating covariate shift while preventing collapse toward high-density modes.The method explicitly preserves multi-modality during closed-loop fine-tuning.
  • I. INTRODUCTION: Flow-ERD evaluates WOSAC performance with the standard realism meta metric (RMM) and log-free Cross-Pair Diversity (CPD) to characterize realism and rollout spread jointly.This evaluation establishes a realism–diversity landscape rather than relying on realism alone.
  • I. INTRODUCTION: Flow-ERD ranks first overall in realism on the WOSAC test benchmark, while AFM achieves state-of-the-art kinematic score and both reach highest validation diversity among reproducible baselines.Together, the results dominate the realism–diversity Pareto front.

A. Learning-Based Multi-Agent Simulation … A. Multi-Agent Driving Simulation

The paper reviews learning-based traffic simulators, contrasting realism-oriented discrete or continuous approaches with diffusion models and identifying type compatibility and diversity as key challenges. It also formalizes closed-loop multi-agent simulation and motivates entropy-regularized rollout matching to reduce covariate shift while preserving multimodality.

  • A. Learning-Based Multi-Agent Simulation: Next-token prediction models achieve strong WOSAC performance by composing motion from discrete, data-supported primitives with type-specific patterns.Their vocabularies constrain pedestrians, vehicles, and cyclists to type-compatible motions and prevent implausible predictions from re-entering the input context.
  • A. Learning-Based Multi-Agent Simulation: Continuous mixture models unify discrete and continuous prediction under a common mixture-model view, but finite components can still limit multimodality.Anchor-based variants additionally tie coverage to predefined anchors, while anchor-free variants avoid that specific constraint.
  • A. Learning-Based Multi-Agent Simulation: Diffusion-based simulators model joint futures continuously, supporting multimodality and controllability without fixed mode counts or anchors.Because valid motion supports must be learned implicitly, naive continuous state or action modeling may produce type-incompatible motions.
  • B. Mitigating Covariate Shift in Traffic Simulation: Behavior cloning trains on logged histories but deploys on generated states, creating the standard covariate-shift problem in closed-loop simulation.Recent methods therefore fine-tune pretrained models using closed-loop rollouts to improve realism.
  • B. Mitigating Covariate Shift in Traffic Simulation: SMART-R1 and RLFTSim apply reinforcement fine-tuning to WOSAC Realism Meta Metric objectives, while DecompGAIL stabilizes GAIL by decomposing its discriminator.The discriminator uses separate ego–map and ego–neighbor terms for multi-agent settings.
  • B. Mitigating Covariate Shift in Traffic Simulation: Existing fine-tuning methods mainly pull rollouts toward recorded trajectories, realism statistics, or the logged-data manifold, improving stability but leaving scenario-conditioned diversity implicit.Flow-ERD instead matches the model’s own rollout distribution with entropy regularization to reduce covariate shift while preserving multimodality.
  • A. Multi-Agent Driving Simulation: A traffic scenario consists of a road map and N participants whose types belong to VEH, CYC, or PED, with each agent represented by a planar state and 2D box size.The joint scene state aggregates all participant states.
  • A. Multi-Agent Driving Simulation: Given historical scene context and an initial history H0, the simulator produces a state rollout over horizon T in closed loop, feeding each realized scene into the next prediction.The context includes recent scene states and the road map.

B. Holonomic and Bicycle-Style Motion · C. Closed-Loop Covariate Shift

The paper distinguishes holonomic motion for freely moving agents from bicycle-style non-holonomic motion for wheeled road users, then explains how teacher-forced behavior cloning creates closed-loop covariate shift. Reverse-KL alignment can reduce this mismatch but is mode-seeking and may concentrate rollouts on a few high-density modes.

  • B. Holonomic and Bicycle-Style Motion: Holonomic motion models freely moving participants, such as pedestrians, with independent longitudinal and lateral displacement in the agent frame.
  • B. Holonomic and Bicycle-Style Motion: Bicycle-style motion models wheeled road users without treating lateral displacement as an independent executed input.
  • B. Holonomic and Bicycle-Style Motion: The motion formulation represents one-step longitudinal displacement, lateral displacement, and heading change in the agent frame.It defines the planar rotation matrix, longitudinal and lateral heading directions, and sinc(z) = sin(z)/z with sinc(0) = 1.
  • B. Holonomic and Bicycle-Style Motion: Holonomic displacement executes both displacement channels, whereas bicycle-style motion suppresses lateral motion at a no-slip reference point.For bicycle-style motion, the offset r is the distance from the box center to that reference point.
  • B. Holonomic and Bicycle-Style Motion: Bicycle-style box-center motion combines forward progress with a signed swing caused by rotating the no-slip offset.With midpoint heading ψ̄ = ψ + aψ/2, forward progress is parallel to the longitudinal midpoint direction, while lateral box-center displacement comes from the swing term.
  • C. Closed-Loop Covariate Shift: Behavior cloning minimizes negative log-likelihood using conditioning contexts drawn from logged data, a procedure called teacher forcing.The logged dataset is denoted D = {(H0, τ)} and the model parameters are θ.
  • C. Closed-Loop Covariate Shift: At deployment, the simulator conditions on its own previous outputs, so generated states create off-data contexts and compounding error.The resulting closed-loop rollout distribution can differ from the logged-data distribution conditioned on the initial history.
  • C. Closed-Loop Covariate Shift: Reverse-KL alignment penalizes generated rollouts unlikely under the data distribution but is mode-seeking and can concentrate mass on a few high-density modes.

D. Training Flow-based Model · IV. METHOD

The method combines flow matching for trajectory generation with Agent-Type Aware Flow Matching (AFM) and Entropy-Regularized Distillation (ERD). AFM preserves realistic multimodal diversity, while ERD retains minority modes and avoids mode collapse.

  • D. Training Flow-based Model: Flow Matching defines a probability path from Gaussian source p0 = N(0, I) to data target p1 = pdata over x ∈ R^d.The learned velocity field v defines the corresponding flow map.
  • D. Training Flow-based Model: The method uses the affine optimal-transport path x_λ = (1 − λ)x0 + λx1 between noise x0 ∼ p0 and clean data x1.This path specifies the interpolation used for training.
  • D. Training Flow-based Model: The velocity target is v⋆ = x1 − x0, and the encoded context e is used to derive the flow-matching loss.The target directly reflects the displacement between noise and data samples.
  • D. Training Flow-based Model: After training, samples are generated by solving the learned ordinary differential equation using the parameterized neural velocity field vθ.The neural field learned by the flow-matching objective drives sampling at inference time.
  • IV. METHOD: The proposed method has two components: Agent-Type Aware Flow Matching (AFM) and Entropy-Regularized Distillation (ERD).AFM is the backbone, while ERD provides diversity-preserving fine-tuning.
  • IV. METHOD: AFM captures multimodal diversity in a continuous action space while maintaining realism through agent-type aware modeling.The modeling couples diversity with agent-specific behavior.
  • IV. METHOD: ERD fine-tunes AFM with an entropy-regularized objective that retains minority modes and avoids the mode collapse of standard reverse KL objectives.Its purpose is to preserve less-dense modes during distillation.

A. Flow-Matching Backbone … 3) Type-Specific State Transition:

Flow-ERD’s AFM backbone generates multimodal continuous kinematic actions and executes them through agent-type-specific transitions. This separates expressive action generation from type-compatible pose updates while preserving a shared action space.

  • 1) Motivation and Design Rationale:: AFM separates continuous action generation from execution to avoid invalid motions becoming future closed-loop context.Continuous generators remove token vocabularies’ codebook bottleneck, but type-aware execution is needed to prevent motions invalid for an agent type.
  • 2) Kinematic Action-Space Flow Modeling:: AFM samples multimodal continuous kinematic actions in a shared representation before executing them into poses.The simulator feeds back poses obtained through the specified action-to-pose transition.
  • 1) Motivation and Design Rationale:: AFM represents each agent’s motion as a metric kinematic action with local longitudinal, lateral, and heading increments.
  • 2) Kinematic Action-Space Flow Modeling:: Given history context H_t, AFM models the full H-step action sequence for flow-matching pretraining.The clean endpoint x_1 serves as the target action sequence.
  • 3) Type-Specific State Transition:: For each agent, AFM maps an action to the next pose using an agent-type-specific transition F_c^i applied to the planar state.
  • 3) Type-Specific State Transition:: The transition F_c^i is defined by a wrapped heading update and type-dependent displacement using the motion models from Section III-B.Angle wrapping maps headings to [−π, π].
  • 3) Type-Specific State Transition:: Vehicles and cyclists use a non-holonomic update with forward displacement, heading change, and no-slip offset, while the lateral channel is unused in pose updates.
  • 3) Type-Specific State Transition:: Keeping the lateral channel in the shared action vector while excluding it from vehicle and cyclist pose updates preserves shared actions with type-compatible execution.

4) Training: Transition-Consistent Action Targets:

Flow-ERD converts logged pose pairs into executable action targets by estimating type-specific no-slip offsets and matching target construction to sequential inference. Large-reexecution-error targets are filtered to improve training stability.

  • Transition-Consistent Action Targets: For each non-holonomic agent type, the no-slip offset is modeled as r_i = ρ_cℓ_i and ρ_c is estimated from logged trajectories.Straight intervals provide no information because a_ψ = 0; turning intervals are aggregated with a robust statistic.
  • Transition-Consistent Action Targets: With the no-slip ratio fixed, logged poses are converted into action sequences executed sequentially through each previously executed state.Each executed pose becomes the state for the next action, following the inference procedure.
  • Transition-Consistent Action Targets: Training targets use the same sequential re-execution transition as inference, producing executable flow-matching targets with aligned state sequences.This construction reduces the train-inference gap.
  • Transition-Consistent Action Targets: Targets with large re-execution error are filtered out to improve training stability.The filtering is applied after constructing the executable flow-matching target sequence.

5) Closed-Loop Inference: · 6) Architecture:

Flow-ERD performs closed-loop inference by repeatedly executing only an initial action prefix and feeding type-specific poses back into context. Its AFM architecture combines SMART-style scene encoding with chunked flow decoding, refinement, and type-specific execution.

  • 5) Closed-Loop Inference:: AFM generates an H-step action sequence, executes it with Fci, commits only the first B < H steps, and appends their poses to the next context.This receding-horizon procedure defines the closed-loop rollout distribution pCLθ(τ|H0).
  • 5) Closed-Loop Inference:: Using type-specific executed poses for feedback, AFM preserves continuous multimodality while reducing type-incompatible closed-loop states.Feedback therefore differs from unconstrained pose prediction in the direct open-loop distribution.
  • 5) Closed-Loop Inference:: ERD initializes generator and fake-score parameters from θ0, freezes the real score, and alternates fake-only score updates with joint generator–score optimization.The procedure repeats phase-wise updates until convergence and returns θ.
  • 6) Architecture:: Given Ht, a SMART-style scene encoder produces temporal, map–agent, and agent–agent context features et = Enc(Ht).Temporal history uses continuously executed motion features rather than discrete motion tokens.
  • 6) Architecture:: The flow decoder processes noisy H-step action sequences with flow timestep λ, embeds them into B-step chunks, and applies chunk-level self-attention with DiT-style conditioning.Scale–shift–gate conditioning modulates the chunk-level attention blocks.
  • 6) Architecture:: A step refiner applies within-chunk self-attention, while an MLP velocity head outputs the action-space field vθ(xλ, λ, et) used in Eq. (8).The decoder’s resulting action sequence is then executed by the type-specific transition in Eq. (11).

B. Entropy-Regularized Distillation

Entropy-Regularized Distillation fine-tunes AFM on closed-loop rollouts with an entropy-regularized reverse-KL objective to mitigate covariate shift while preserving diverse modes. It implements the resulting tempered-target matching through score-based Distribution-Matching Distillation on generated action sequences.

  • Objective: ERD fine-tunes AFM under closed-loop rollouts to mitigate covariate shift while explicitly preserving represented diversity.
  • Objective: Reverse-KL can concentrate probability on dominant modes because low-density data modes contribute little under the model-distribution expectation.
  • Objective: An entropy regularizer produces a tempered target with β = 1/(1 + γ), reducing density contrast and up-weighting minority modes as β decreases.β =1 (γ =0) recovers ordinary matching toward pdata.
  • Optimization: DMD realizes the intractable tempered reverse-KL through score gradients evaluated across flow-noise timesteps on generated H-step kinematic-action sequences.The real score is replaced by the tempered target score gpβdata = βgpdata.
  • Optimization: The method uses a frozen open-loop backbone as the data score and an on-policy closed-loop fake score from rollouts generated with B-step commitment.Executing the generated actions through type-specific deterministic transitions changes the induced closed-loop state rollout.

V. EXPERIMENTS · A. Evaluation Metrics

The experiments test whether AFM improves realism without sacrificing diversity, whether ERD preserves diversity while improving realism, and whether minority semantic modes remain represented. Evaluation combines WOSAC realism metrics with the log-independent Cross-Pair Diversity metric, interpreted at matched realism because spread alone may reflect error or drift.

  • V. EXPERIMENTS: The experiments ask whether AFM raises realism while maintaining diversity, whether ERD improves realism without erasing diversity, and whether minority modes remain represented.
  • A. Evaluation Metrics: On the validation split, AFM traces the upper-right realism–diversity Pareto frontier and reaches an RMM of 0.7840 at noise scale 1.05.AFM varies Gaussian noise scale, whereas UniMM, SMART, and TrajTok sweep top-k decoding during validation rollouts.
  • A. Evaluation Metrics: Realism is measured with WOSAC 2025’s realism meta-metric, RMM, which aggregates likelihoods over kinematic, interactive, and map-based statistics against the logged future.
  • A. Evaluation Metrics: The evaluation also reports RMM’s three components and minADE, the per-object minimum ADE over rollouts, although minADE does not contribute to RMM.
  • A. Evaluation Metrics: RMM can reward spread around one logged future while missing plausible modes absent from that log, so it cannot distinguish dominant-mode tracking from realistic diversity.
  • A. Evaluation Metrics: Cross-Pair Diversity, or CPD, is a log-independent metric that averages type-normalized pairwise distances across rollout pairs and scenarios.Agent types with no instances are omitted, and each type uses a training-set-fixed scale.
  • A. Evaluation Metrics: The validation evaluation includes multi-agent closed-loop rollouts over 1,048 scenes, with ego-maneuver intents labeled following WOMD.The analysis reports ego-trajectory diversity and average per-scene intent entropy of ego rollouts.
  • A. Evaluation Metrics: Higher CPD indicates more distinct futures from the same initial state, but CPD alone cannot separate genuine multimodality from prediction error or closed-loop drift.Accordingly, the paper compares CPD only at matched RMM and treats differences across substantially different realism levels cautiously.

B. Experimental Results … VI. CONCLUSION

Flow-ERD combines agent-type-aware flow matching with entropy-regularized distillation to improve traffic-simulation realism while preserving diverse, semantically meaningful futures. Experiments show strong WOSAC realism, superior diversity among baselines, and control over the realism–diversity trade-off.

  • 1) AFM breaks the backbone realism–diversity trade-off:: AFM achieves the highest overall RMM among continuous backbones and the best kinematic component across all baselines, including fine-tuned methods.This supports agent-type-aware kinematic modeling as a way to reproduce per-type motion faithfully.
  • 1) AFM breaks the backbone realism–diversity trade-off:: AFM attains the highest CPD across all other pretrained baselines, and competing models cannot reach its realism–diversity frontier under sampled settings.Against TrajTok, AFM retains substantially higher CPD even when their RMM matches.
  • 1) AFM breaks the backbone realism–diversity trade-off:: The all-holonomic backbone produces the largest raw CPD but the lowest kinematic score, while an all-non-holonomic transition restores realism but over-constrains pedestrians.The ablation attributes holonomic spread partly to type-incompatible lateral slip for vehicles and cyclists.
  • 2) ERD improves closed-loop realism without collapsing diversity:: Flow-ERD achieves the best RMM among all baselines and preserves higher CPD than every fine-tuned baseline, even without an entropy term.The result indicates ERD reshapes probability mass within the flow model’s broad support rather than providing only a generic fine-tuning gain.
  • 2) ERD improves closed-loop realism without collapsing diversity:: At β = 1, ERD gives the highest RMM, whereas lowering β trades a small amount of realism for higher CPD.The entropy temperature therefore provides a direct realism–diversity control.
  • 2) ERD improves closed-loop realism without collapsing diversity:: At β = 0.99, ERD recovers nearly all backbone diversity while remaining more realistic than every fine-tuning baseline; at β = 0.95, realism falls below the backbone.The authors report β = 1.0 and β = 0.99, omitting β = 0.95 because its realism cost is too steep.
  • 3) The preserved diversity reflects intent-level multimodality:: SMART concentrates probability on straight motion, whereas Flow-ERD reveals more same-intent diversity and, at β = 0.99, recovers rare U-turns as physically plausible maneuvers.The qualitative samples do not merely drift from the map or disperse through closed-loop error.
Loading 2607.06957v1…