PaperlayerView exact revision on arXiv

Source-linked AI summary

Hi-FLoop: Hierarchical State-Feedback Loops for Multi-Timescale World Modeling

Rx Fan, Zhan H

arXiv:2609.08796v1cs.CVcs.AI

TL;DR

HI-FLOOP addresses long-horizon closed-loop traffic generation under evolving generated states and multiple decision timescales. It uses persistent scene-level Worlds with hierarchical Goal, Preview, and Control states, feeding back only executed prefixes. On 955 public-validation scenarios, it reports an Overall score of 0.689987 and oracle-minADE@8 of 1.196636 m under agent-centric oracle evaluation.

  • Problem

    Long-horizon closed-loop traffic simulation must maintain coordinated, physically realistic futures while its generated states alter subsequent prediction contexts.

  • Method

    HI-FLOOP uses persistent scene-level joint Worlds with 8-second Goals, 2-second Preview interaction plans, 1-second Control, and executed-prefix state feedback.

  • Results

    oracle-minADE@8 of 1.196636 m was achieved over the 8-second horizon under agent-centric oracle evaluation.

  • Takeaways & Limitations

    The framework maintains a shared branch identity across agents while adapting Goals, interactions, and motion to newly generated facts within that branch.

  • Takeaways & Limitations

    The method does not explicitly represent or normalize the full 32N joint distribution and disallows stitching per-agent selections across Worlds.

Abstract

from arXiv · show

Multi-agent traffic simulation seeks diverse, coordinated, and physically realistic futures from maps and observed history. Long-horizon closed-loop generation must reconcile multiple decision time scales while its context evolves with generated states. Existing methods often unfold long futures from the initial scene and resolve intent, interaction, and motion monolithically, weakening cross-scale consistency and adaptation. We present HI-FLOOP, a branch-consistent multi-timescale state-feedback framework. Eight scene-level Worlds represent joint hypotheses, and all agents share the selected World identity throughout an 8-second rollout. Within the branch, an 8-second Goal anchors intent, a 2-second Preview coordinates interactions, and 1-second Control produces physical motion. Every 0.5-second commit feeds back only its executed prefix as new facts, while unexecuted hypotheses never enter factual memory. Joint Preview Interaction (JPI) induces a sparse directed future graph from Preview and uses conflict probabilities and signed arrival-time differences to gate interaction refinement. For generated-state recovery, a prefix-frozen A-to-B cascade lets frozen Model A generate 0-1 seconds, then transfers typed physical state, admissible context, and the branch index, but no latent state, to an independent Model B for re-encoding and 1-2-second recovery. On the full H-D public-validation split of 955 scenarios, one complete S1 run yields an Overall score of 0.689987 with the official evaluator. Under agent-centric oracle evaluation, HI-FLOOP achieves oracle-minADE@8 of 1.196636 m over the 8-second horizon and 0.526 m over the 6-second horizon.

1 Introduction

HI-FLOOP frames closed-loop traffic simulation as persistent multi-timescale joint world modeling, where long-term branches, interaction plans, and executable motion adapt through executed-state feedback. Its contributions establish shared scene-level branch identity, rolling interaction refinement, and causal boundaries for generated-state training.

  • Closed-loop errors can alter neighborhoods, map relations, conflicts, and right-of-way order, requiring evaluation of interaction consistency, safety, compliance, and distribution coverage.
  • HI-FLOOP models long-term intent, mid-term interaction plans, and short-term motion as generative states with distinct horizons, persistence, and adaptation scopes.
  • A scene-level World supplies shared long-term conditioning and preserves branch identity, while Goals, Preview interactions, and Control adapt to newly generated facts.
  • The framework contributes decoupled state-feedback generation, persistent joint branches, and alternating Preview structure–attribute interaction reasoning.

2 Related Work

Related work spans joint multimodal prediction, hierarchical and reactive closed-loop simulation, and training methods addressing generated-state distribution shift. HI-FLOOP distinguishes itself through persistent scene-level branch identity and explicit causal semantics across commits.

  • Multi-Agent Motion Prediction and Closed-Loop Traffic Generation: Prior predictors model joint dependencies, directed interactions, multimodal motion, or diffusion-based uncertainty, while traffic generators add reactive, autoregressive, controllable, and city-scale closed-loop mechanisms.
  • Hierarchical Closed-Loop Simulators: Hierarchical simulators separate intent from driving behavior, but the cited methods do not specify persistent scene-level multimodal branch identity across agents and commits.
  • Long-Horizon Traffic Generation: HI-FLOOP targets joint branch consistency across agents and rolling time rather than generic long-sequence autoregression.
  • Training and Post-Training: Closed-loop training research varies in learning signal, state source, and gradient scope; HI-FLOOP assigns prefix generation and generated-state recovery to disjoint Models A and B.
  • Training and Post-Training: Reinforcement learning and post-training add closed-loop signals for collision avoidance, compliance, behavioral quality, or diversity.

3 Method

HI-FLOOP executes a 16-commit rollout through a causal hierarchy of World, Goal, Preview, and Control states. Only executed physical prefixes enter factual history, while persistent branch state remains isolated from observations.

  • Overall Architecture: The architecture generates 80 future frames over 8 seconds in 16 commits, producing 1 second but executing only the first 0.5 seconds at each commit.
  • Overall Architecture: Eight Worlds propose joint branches, one selected World persists across commits, and the recurrent core combines an 8-second Goal, 2-second Preview refinement, and 1-second Control.
  • State Containers: Table 1 separates factual history, persistent branch state, and future interaction graphs through distinct read–write semantics.
  • State Containers: Persistent branch state stores World identity, query and anchor indices, Goal and interaction state, warm starts, and committed quality, but is not factual encoder input.
  • Causal Feedback: Only executed states X1:5 enter the next factual history; unexecuted states remain hypotheses, and feedback adapts continuous within-branch evolution without reselecting the World.
  • Agent and World Handling: Context agents participate in encoding and interaction computation, while official evaluation and generated rollouts use explicit masks and shared scene-level World execution.

3.2 Factual Scene Encoder with Causal Inputs Only

The factual scene encoder builds separate temporal, map, agent, and signal representations from observed inputs, recomputing dynamic relations after generated prefixes while excluding unobserved future elevation.

  • Factual Encoding: The encoder applies temporal self-attention, polyline encoding, dynamic Agent–Map Attention, and directed Agent–Agent relation attention to history and map inputs.
  • Factual Encoding: Keeping memory sources separate lets downstream Goal, Preview, and Control modules access history, map, agent, and signal facts without a single compressed scene vector.
  • Rolling Updates: After each commit, five generated states are appended, the latest 11 frames are retained, and dynamic relations are recomputed; static map representations may be cached.
  • Causal Inputs: Elevation is encoded only as an optional observed relative relation, with an availability bit, and missing endpoints zero both elevation slots.
  • Causal Inputs: Generated future states are marked as having unobserved elevation, preventing future ground-truth or map-projected elevation from re-entering factual encoding.

3.3 Goal Regions and World-Conditioned Coordination

HI-FLOOP generates type-specific 8-second Goal Regions and coordinates them through eight scene-level World hypotheses. Each World couples agent proposals sparsely while preserving a hard-selected Anchor and allowing bounded continuous adaptation without reselection.

  • Goal Regions: The model generates K = 32 type-specific Goal Regions per agent over the 8-second horizon, using separate prior banks for vehicles, pedestrians, and cyclists.Proposal features include map context, coarse ETA intervals, unary scores, and learned feasibility logits.
  • World-Conditioned Coordination: Each World adds bounded conditional residuals and sparse soft coordination over the factual agent graph to couple agents’ Goal distributions without enumerating 32^N combinations.The resulting coupling is tractable but is not an exactly normalized global distribution.
  • World-Conditioned Coordination: A joint World is a shared scene-level branch identity coupled through message passing, not an explicit or normalized representation of the full 32^N joint distribution.Per-agent argmax selections are not stitched across Worlds.
  • Goal Regions: After coordination, each World makes one hard selection per agent, and the Anchor class and topological basin remain fixed throughout the standard 8-second rollout.This prevents commit-wise reselection from causing mode switches.
  • Goal Regions: Continuous Goal updates read current facts, the persistent World condition, remaining horizon, and prior Preview information while preserving the discrete Anchor.After repeated failed commits, the state may be marked degraded or invalid, but the Anchor is not reselected and termination remains fixed at the initial t0 + 8 seconds.

3.4 Preview-Induced Sparse Future Interaction

Joint Preview Interaction refines a short-horizon Preview through two rounds of sparse future-graph reconstruction and interaction messaging. The rolling Preview is warm-started across commits, while the fixed Goal Anchor and factual history remain separate from this interaction operator.

  • JPI block: Each World generates four Preview nodes per agent at +0.5, +1.0, +1.5, and +2.0 seconds, and JPI alternates graph construction, message passing, and plan refinement.The two rounds produce intermediate and final Preview states without creating a fourth persistent state.
  • Commit update: After each 0.5-second execution, old Preview nodes shift into the next commit’s warm start and a new +2.0-second node is predicted.The warm start conditions planning but is never copied into physical state.
  • Sparse future graph: Directed candidate edges use envelope clearance, future-path clearance, closing speed, TTC, polygon overlap, and geometric ETA difference to identify potentially interacting agent pairs.Edges are eligible when current clearance is local or future clearance enters the warning range.
  • Sparse future graph: Each receiving agent retains its highest-ranked neighbors, while overlapping current or future envelopes bypass the neighbor cap.The graph is reconstructed after refinement, allowing the first round to affect second-round topology and interaction strength.
  • Interaction refinement: Continuous interaction attributes gate sparse messages, which update the World hidden state and refine Preview nodes; precedence follows signed ΔETA rather than an unconstrained priority head.The two JPI rounds do not alter the fixed Goal Anchor or write directly to factual history.
  • Interaction refinement: Ground-truth positive conflict edges are used only for interaction losses and never for inference-time message passing, preventing future-topology leakage.

3.5 Causal World Plan Scoring

HI-FLOOP scores eight lightweight World plans, selects one scene-level branch, and evaluates that branch causally across commits. Training combines initial plan quality with selected-branch execution quality but does not provide complete ranking supervision for every World.

  • Initial scoring: After eight Worlds complete lightweight Goal and Preview planning, the WorldPlanScorer predicts a six-dimensional quality vector and aggregate score for every scene–World pair.The components are Goal, Preview, Interaction, Map, Dynamics, and Closed-loop quality.
  • Branch selection: At inference, the model selects w* = arg maxw sb,w and retains that World identity for all 16 commits, evaluating subsequent causal scores only on the selected branch.The evaluator never stitches agent-wise Worlds.
  • Training supervision: Supervised training executes only one physical World per scene, while routing coverage gives different Worlds opportunities to receive full physical-execution labels.Unexecuted Worlds receive only lightweight plan-level supervision.
  • Training supervision: Because only the selected World receives a complete label, the accumulated-quality term calibrates long-horizon prediction but is not genuine full-ranking supervision over all eight Worlds.

3.6 Continuous Control and Differentiable Dynamics

The control head predicts bounded, temporally correlated continuous controls and differentiably integrates them into physical states. Vehicle and cyclist dynamics retain yaw channels, while pedestrians use holonomic acceleration with speed-dependent heading.

  • Continuous control: The control head reads the final Preview, Slow Goal, World state, factual memory, and current physical state to predict a continuous distribution over the next second.The latent variable represents a flattened 30-dimensional control sequence.
  • Continuous control: Stochastic export uses temporally correlated noise rather than independent white noise at every step, with L representing a rank-2 temporal covariance factor.
  • Bounded dynamics: At each 0.1-second step, query-frame acceleration is bounded to a disk and yaw acceleration to an interval, with pedestrians masking the yaw-control channel.Vehicles and cyclists retain all three control channels, while pedestrians use only 2D holonomic acceleration.
  • Bounded dynamics: Invertible control transformations and their Jacobians enter the control NLL so bounded physical controls are not modeled as unconstrained Gaussian variables.
  • Differentiable dynamics: After rotating controls into the scene frame, all agents are integrated synchronously in FP32 with explicit yaw and yaw-rate maintenance for vehicles and cyclists.Pedestrian heading follows motion direction at sufficient speed and otherwise retains its previous heading.

3.7 Learning Objectives and Streaming Generated-State Training

HI-FLOOP organizes learning objectives around the interfaces between Goal, World, Preview, interaction, Control, state, safety, map, kinematic, and closed-loop components. Streaming training feeds generated commits back into history while preserving numerical state evolution and truncating only gradient graphs at TBPTT boundaries.

  • Learning objectives: Learning objectives supervise Goal proposal, World assignment, continuity, Preview, interaction, Control, integrated state, kinematics, safety, map feasibility, and closed-loop error.Agent-level terms are averaged over valid reduction groups, and empty-support terms produce no gradient.
  • Numerical and physical supervision: Future interaction geometry, dynamics, probability operations, and loss denominators are evaluated in FP32, while EMA balancing avoids amplifying intrinsically small auxiliary terms.Kinematic terms supervise acceleration and regularize jerk, sideslip, and lateral acceleration; safety and map terms operate after dynamics integration.
  • Streaming generated-state training: Streaming training writes the first five generated states into the next commit’s history, exposing later rollouts to model-generated rather than exclusively logged states.The reported S1 evaluation uses the step-120k checkpoint learned from the first two commits starting from a real 11-frame history.

3.8 Prefix-Frozen Generated-State Recovery

The prefix-frozen recovery cascade assigns early-prefix generation and later-suffix recovery to separate models, preventing suffix optimization from changing the prefix model. A typed causal boundary transfers physical and admissible contextual state while discarding latent representations for B to re-encode.

  • Motivation and model ownership: A shared model’s later-commit gradients can alter parameters responsible for the early prefix, changing the recovering suffix’s input distribution and degrading prefix behavior.S2.1 addresses this by using identical-architecture Models A and B with disjoint parameters initialized from S1 step 120k.
  • Prefix and suffix responsibilities: Model A stays frozen and generates the 0–1-second prefix, while Model B alone learns 1–2-second recovery from the handoff at t = 1 second.A remains in evaluation mode and is excluded from optimization, scheduling, and AMP scaling; B does not execute or optimize the first two commits.
  • Typed handoff: The A-to-B boundary is an auditable typed causal interface rather than hidden-state distillation.The transferred information includes generated history, kinematic and lifecycle state, map and causally available context, the selected World index, and scene/provenance metadata.
  • Re-encoding: B discards A’s hidden states, caches, decoder memory, planning features, interaction summaries, warm starts, and computation graph before re-encoding the physical handoff.B regenerates Goal, World, Slow Goal, and interaction representations from the new factual encoder state.
  • Recovery optimization: S2.1 updates B only on generated-state continuation at Commits 3/4, disabling static Goal proposal, World Scorer, and World-diversity objectives.Recovery supervision begins at future steps 10 and 15, with a detached GT-WTA proxy selecting branches for C3/C4.

4 Experiments

Experiments evaluate a complete S1 checkpoint with shared-World scene metrics, agent-centric oracle diagnostics, and mechanism-level contract checks on the full 955-scenario H-D validation split. Results distinguish jointly realizable scene performance from marginal per-agent mode coverage and do not treat optimization curves or structural checks as performance evidence.

  • Protocol: The complete S1 checkpoint is evaluated with an eight-second closed-loop rollout on all 955 H-D public-validation scenarios.Scenario-joint displacement is primary because one World must explain all evaluated agents; per-agent World selection is complementary.
  • Data and training setup: The evaluation uses unified multi-dataset scene representations, fixed S1 training splits, and a five-source train-only S2.1 manifest.Approximately 3.2% of Waymo scenarios violating the 10 Hz sampling rule are removed under a predefined filter.
  • Configuration: The model uses d = 384, 32 Goal Regions, eight Worlds, four Preview nodes, two JPI rounds, and 114,106,461 trainable S1 parameters.Learnable modules use BF16, while future-interaction geometry, dynamics, key probability operations, and loss reductions use FP32.
  • Scenario-joint results: 2.377028/7.529927 m is the eight-second scenario-joint ADE-at-joint-minFDE@8/joint-minFDE@8, using one selected World for every evaluated agent in each scene.The full rollout completed successfully for all 955 scenarios without skipped or failed cases.
  • Agent-centric diagnostics: 1.196636 m is oracle-minADE@8 under independent per-agent ADE-selected Worlds, while 1.203089/3.879052 m is the eight-second ADE-at-minFDE@8/minFDE@8 pair.These agent-centric diagnostics measure marginal mode coverage and do not replace jointly realizable scenario-joint results.
  • Structural checks: Mechanism-level checks change one contract condition at a time to identify removed structural properties, but they are not empirical performance ablations.The checks isolate branch persistence, executed-state feedback, temporal decomposition, and explicit future-interaction conditioning.
  • Optimization diagnostic: Figure 2 reports the composite S1 training objective at 10k-step checkpoints across a 120k-step run as an optimization diagnostic only.Its logged markers are connected without smoothing and do not evidence held-out or closed-loop performance.

5 Conclusion

HI-FLOOP combines a persistent scene-level World with adaptive Goal, interaction, and Control states, feeding only executed prefixes back as factual context. On all 955 H-D validation scenarios, the complete S1 run reports shared-World and agent-centric displacement results, while A→B recovery separates prefix and suffix parameter ownership.

  • Framework: HI-FLOOP uses a fixed scene-level joint World while Goal, interaction relations, and local motion adapt to newly generated facts within that branch.Each commit appends only the executed 0.5-second prefix and re-encodes it as factual context.
  • Generated-state recovery: The A→B cascade isolates suffix gradients through a frozen prefix model, a disjoint-parameter recovery model, and re-encoding across a typed physical-state boundary.The cascade transfers physical and admissible contextual state rather than latent state.
  • Reported results: 2.377028/7.529927 m is the eight-second ADE-at-joint-minFDE@8/joint-minFDE@8 on all 955 H-D public-validation scenarios under one shared World per scene.Independent per-agent ADE selection yields oracle-minADE@8 of 1.196636 m.
Loading 2609.08796v1…