PaperlayerView exact revision on arXiv

Source-linked AI summary

ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow

Dongxiu Liu, Haoyi Niu, Peng Cheng, Yuan Gao, Xirui Kang, Sangli Teng, Koushil Sreenath, Xianyuan Zhan

arXiv:2607.27924v2cs.LGcs.CVcs.RO

TL;DR

Existing world models largely use discrete-time prediction despite the continuous nature of physical dynamics. ODEWorld learns a physical-time latent ODE flow and demonstrates arbitrary temporal resolution, backward prediction, long-horizon reconstruction, and planning-oriented representations across simulation and real-world benchmarks.

  • Problem

    Existing world models largely rely on discrete-time modeling despite the continuous space-time dynamics of the physical world.

  • Method

    ODEWorld models physical-time dynamics as a continuous latent velocity field governed by an ODE and predicts future states through latent-space temporal integration.

  • Results

    Across simulation and real-world benchmarks, ODEWorld provides planning-conducive dynamics abstraction with visual realism, long-horizon reconstruction, arbitrary temporal resolution, and backward prediction.

  • Takeaways & Limitations

    ODEWorld offers a compact continuous-time world model for long-horizon prediction, irregularly sampled video generation, and downstream policy learning.

  • Takeaways & Limitations

    ODEWorld assumes the system reaches a stationary state after the target condition is reached during extended-time modeling.

Abstract

from arXiv · show

In the physical world we inhabit, space and time are fundamentally continuous. However, existing machine learning paradigms for world modeling are largely confined to discrete-time prediction, thereby exhibiting significant inefficiency in capturing the dynamics of physical world. We introduce Physical-Time Flow (PT-Flow), a novel approach that learns a continuous latent velocity field operating in physical time. Crucially, the underlying dynamics of sequential data are parameterized by an ordinary differential equation (ODE) embedded in a well-structured representation space. Under this paradigm, the prediction of future can be recast as temporal integration via an ODE solver in the compressed latent space. Building upon PT-Flow, we construct ODEWorld, a continuous-time latent world model that is both efficient and versatile. By extracting time-variant features and enforcing ODE properties on both the dynamical representation space and the latent velocity field, ODEWorld effectively addresses the long-standing representation collapse issue in latent world model literature. This also enables high-quality image reconstruction even after long-horizon prediction. Moreover, its continuous nature allows for arbitrary temporal resolution and even backward prediction, which is impossible for most discrete-time models. Lastly, ODEWorld can provide rich planning-oriented information to facilitate downstream policy learning. Comprehensive experiments demonstrate that ODEWorld successfully reconciles planning-conducive dynamics abstraction with visual realism, excelling in both video generation and robotic control. Project page: https://odeworld.github.io/.

1 Introduction

ODEWorld addresses the mismatch between continuous physical dynamics and predominantly discrete-time AI modeling through PT-Flow, which learns ODE-governed latent dynamics in physical time. Its architecture targets representation collapse while supporting compact, high-fidelity, long-horizon prediction and flexible temporal resolution.

  • Physical-Time Flow: Physical-Time Flow models data dynamics as a continuous latent velocity field whose state transitions are governed by an ODE in physical time.PT-Flow represents underlying dynamics in a structured latent space and evolves latent states continuously along actual physical time.
  • Physical-Time Flow: PT-Flow combines dynamical representation decoupling with direct first-order supervision to isolate time-varying information and efficiently learn the latent velocity field.Dynamical representation decoupling removes background and time-invariant details into an ODE-compatible space, while direct supervision targets the latent velocity field.
  • ODEWorld: ODEWorld conditions its encoder and decoder on static context and directly supervises latent velocity while regularizing ODE properties, addressing representation collapse in JEPA-based frameworks.The approach is presented as avoiding trivial latent solutions associated with indirect supervision of coupled encoders and latent predictors.
  • ODEWorld: ODEWorld achieves high-fidelity image reconstruction with an extremely compact model across extended temporal horizons, while enabling arbitrary temporal resolution and backward prediction.These capabilities follow from direct latent-velocity supervision and the model’s continuous formulation.
  • Empirical evaluation: Evaluations on simulation and real-world benchmarks show that ODEWorld reconciles planning-conducive dynamics abstraction with visual realism through interpretable, smooth latent transformation paths.The continuous latent flow also supports temporally consistent, long-horizon prediction.

2 Related Work

Related work spans predictive world models that generate future observations, latent world models that improve efficiency through compact representations, and ODE-based approaches for continuous dynamical-system modeling.

  • Predictive world models: Predictive world models use video generation to simulate physical-world dynamics by forecasting future observations as full trajectories or subgoal images.The cited approaches include full trajectory videos and subgoal-image prediction.
  • Predictive world models: Latent world models address efficiency by performing predictions in a compact representation space.These approaches construct latent representations using objectives such as image reconstruction, contrastive learning, and explicit motion prediction.
  • ODE dynamics modeling: Ordinary differential equations are widely used to model continuous dynamical systems across physics, climate science, and neuroscience.One prominent methodology explicitly constructs physics-informed learnable ODE dynamics.

3 Physical-Time Flow

Physical-Time Flow models high-dimensional dynamics as a continuous latent velocity field, reframing future prediction as temporal integration in physical time. Its ODE-compatible representation learning enables arbitrary temporal resolution, backward prediction, and compact-latent ODE solving from discrete-time supervision.

  • Physical-Time Flow: PT-Flow represents physical-time dynamics as a continuous velocity field over a properly encoded latent state.The latent state z_t represents the physical state s_t at elapsed time t.
  • Physical-Time Flow: Future prediction becomes temporal integration of the learned latent velocity field, enabling arbitrary temporal resolution and backward prediction.The formulation also supports properties useful in control theory.
  • Dynamical representation decoupling: An initial-state-conditioned encoder and decoder extract essential dynamics-related representations instead of fitting static background and time-invariant visual details.The encoder is designed to produce clean representations of video dynamics that are compatible with ODE modeling.
  • Direct first-order supervision: The latent derivative is obtained through a Jacobian-vector product, using discrete state differences such as (s_t+1 − s_t)/∆t to approximate physical-state derivatives.This makes direct first-order supervision of the latent velocity practically feasible.
  • Direct first-order supervision: Joint optimization of dynamical reconstruction and velocity losses learns a structured latent space and velocity field for arbitrary-time prediction with an off-the-shelf ODE solver.Stop-gradient is used for optimization stability, although PT-Flow remains effective without it.

4 ODEWorld: Latent World Modeling via PT-Flow

ODEWorld instantiates PT-Flow as an efficient continuous-time architecture that combines high-fidelity video generation with latent planning for robotic policy learning. It uses frozen DINO-based observation representations, compact dynamics latents, stabilized velocity estimation, and ODE-based inference.

  • ODEWorld is a versatile, efficient continuous-time predictive architecture for high-fidelity video generation and robotic policy learning.
  • Architecture overview: Frozen DINOv2 encoding compresses raw images into a numerically stable feature space, while a dedicated decoder reconstructs images from those features.The encoder projects x_t into s_t, and the decoder reconstructs the image as x̂_t.
  • Architecture overview: A state-conditioned cross-attention dynamics encoder summarizes temporal changes in a compact latent representation z_t.A single token suffices in experiments, with z_t ∈ R^1×768 versus s_t ∈ R^16×16×768.
  • Additional enhancements on PT-Flow: Rescaled physical time τ and Savitzky–Golay filtering improve numerical stability and produce temporally smooth, physically accurate latent velocity targets.The rescaled time mostly lies in [0, 1] during training, and the implemented filter uses window size 2k + 1 = 5 with kernel w = 1/10[−2, −1, 0, 1, 2].
  • Training and inference pipeline: Training jointly minimizes dynamics reconstruction and velocity supervision losses, with λ_rec = λ_v = 1 generally yielding reasonably good results without tuning.This behavior demonstrates the learning robustness of PT-Flow.
  • Training and inference pipeline: At inference, an ODE solver integrates the latent velocity field to arbitrary time τ, enabling future-frame reconstruction and compact dynamics-centric subgoals for policy learning.The pipeline uses RK4 as an example solver and maps evolved latents through the dynamics and observation decoders.

5 Experiments

Experiments show that ODEWorld learns smoother, globally consistent latent dynamics while supporting bidirectional and arbitrary-resolution video generation. These properties translate into stronger video prediction efficiency and improved simulated and real-world robot policy performance.

  • Latent space analysis: ODEWorld produces significantly smoother latent trajectories across LIBERO tasks than raw DINO features and a discrete next-step predictive baseline.The comparison uses PCA visualizations of trajectories from multiple LIBERO tasks.
  • Velocity field and open-loop latent planning: Its learned velocity field yields globally consistent, goal-directed convergence and smooth latent plans that adhere well to ground-truth trajectories.Long-horizon rollouts use an RK4 ODE solver, and decoded predictions remain spatially coherent and semantically accurate.
  • Video generation flexibility: ODEWorld supports physically plausible backward prediction and arbitrary temporal resolution, including motion-speed control and recovery of intermediate frames from downsampled sequences.Backward generation reverses the integration sign, while continuous integration enables temporal interpolation beyond the training sampling rate.
  • Quantitative comparisons: 0.072s prediction latency at @64 frames accompanies higher PSNR and LPIPS than baselines across short and long horizons, while preserving long-horizon structural and perceptual fidelity.Table 1 evaluates video prediction quality and efficiency using pixel-level PSNR and perception-level LPIPS metrics.
  • Downstream policy learning: 83.6% average success rate is achieved by sequential-subgoal guidance on LIBERO-LONG, while real-world guidance raises success from 55% to 80% across four manipulation tasks.The real-world experiments use an X-VLA policy backbone on a bi-manual AgileX robot.

6 Conclusions · Appendix

ODEWorld introduces a continuous latent world model grounded in physical time through PT-Flow, which learns a latent ODE velocity field and predicts the future by ODE-based temporal integration. Experiments on simulation and real-world benchmark datasets demonstrate several desirable properties of this paradigm.

  • 6 Conclusions: ODEWorld is a continuous latent world model grounded in physical time.
  • 6 Conclusions: PT-Flow learns a latent ODE velocity field defined on physical time.
  • 6 Conclusions: ODEWorld recasts future prediction as temporal integration with the latent velocity via an ODE solver.
  • 6 Conclusions: PT-Flow operates in physical time rather than the noise space used by related flow-matching approaches.
  • 6 Conclusions: Comprehensive experiments evaluate ODEWorld on simulation and real-world benchmark datasets.
  • 6 Conclusions: The experiments show that ODEWorld enjoys a number of desirable properties.

A Experiment Details … B Ablation Studies

The experiments cover simulation and real-world manipulation, compare video-generation and policy-learning baselines, and include ablations of dynamical representation decoupling and first-order supervision. ODEWorld is evaluated on LIBERO, AgiBot-World, and four real-world tasks under specified implementation and computation settings.

  • A.1 Datasets: LIBERO provides 130 language-conditioned manipulation tasks and 6,500 human teleoperation demonstrations across five suites.The suites are LIBERO-Spatial, LIBERO-Object, LIBERO-Goal, LIBERO-90, and LIBERO-Long.
  • A.1 Datasets: AgiBot-World training uses approximately 30K manipulation trajectories averaging around 260 frames, covering diverse real-world robot-object interactions.The data come from the AgiBotWorldChallenge-2025 subset on the AgiBot World Colosseo platform.
  • A.2 Real-World Settings: Real-world evaluation uses a bi-manual AgileX robot with two 6-DoF arms, parallel-jaw grippers, and RGB cameras across four manipulation tasks.ODEWorld-generated latent subgoals guide an X-VLA policy under the original policy’s training and evaluation protocol.
  • A.3 Video Generation Baselines: Video-generation comparisons include V-JEPA 2 and LDP, both adapted to 256 × 256 visual inputs for LIBERO modeling.V-JEPA 2 uses first-tubelet-visible masking, while LDP adds spatial convolutional projection for SD-VAE latents.
  • A.4 Policy Learning Baselines: Policy-learning baselines comprise GLCBC, SuSIE, Seer, and VPP, implemented or fine-tuned on LIBERO following their respective original architectures.Their approaches use goal conditioning, future visual representations, inverse dynamics, or video-foundation-model fine-tuning.
  • A.5 Computation Resource: Models are trained on 8 A100 GPUs, while inference time in Table 1 is evaluated on 1 A100 GPU.The passage labels the computation-resource description as the experimental hardware setting.
  • B Ablation Studies: The ablation studies examine dynamical representation decoupling and first-order supervision.Figure 7 visualizes latent trajectories for the dynamical representation-decoupling ablation, and Table 4 reports the corresponding ablations.

B.1 Ablation on Dynamical Representation Decoupling

The ablation shows that dynamical representation decoupling is necessary for ODEWorld to focus latent representations on temporal changes rather than static context. Conditioning the encoder and decoder on the initial state supports smoother dynamics and more effective velocity-field supervision.

  • Ablation on Dynamical Representation Decoupling: Without decoupling static context from the state space, latent trajectories become signi…The variant retains static regions and reconstructs them faithfully, but its temporal and semantic consistency is worse despite stronger per-frame fidelity.
  • Ablation on Dynamical Representation Decoupling: Without decoupling, the latent structure cannot effectively scaffold direct first-order supervision for optimizing the velocity field.This failure reflects the inability to isolate essential dynamical changes from static content.
  • Ablation on Dynamical Representation Decoupling: An initial state-conditioned encoder and decoder make the latent representation focus exclusively on essential dynamical changes.The decoupling mechanism is crucial for learning a well-behaved, temporally smooth representation and velocity field.

B.2 Ablation on First-Order Supervision · B.3 Ablations on Vision Encoders · B.4 Ablation on Token Numbers

Ablations show that first-order supervision is preferable to consistency loss, while ODEWorld remains robust across vision encoders and performs strongly with a single latent token. These results support its compact, flexible latent dynamics design.

  • B.2 Ablation on First-Order Supervision: The consistency-loss variant replaces Ldyn-recon and Lv with supervision over K = 4 subsequent frames planned through the ODE latent space.This variant follows a consistency objective adopted in prior neural ODE work.
  • B.2 Ablation on First-Order Supervision: Consistency loss suffers from optimization difficulty, producing poor short-horizon prediction and visually implausible long-horizon rollouts.The resulting failures are reported in Tab. 4.
  • B.3 Ablations on Vision Encoders: Tab. 5 compares ODEWorld with different vision feature encoders by replacing DINOv2 with SigLIP 2 and LIV.The ablation tests compatibility while keeping the broader ODEWorld design under comparison.
  • B.4 Ablation on Token Numbers: Tab. 6 evaluates increasing the number of latent tokens from 1 to 4 while keeping other configurations unchanged.The comparison isolates token-count effects.
  • B.2 Ablation on First-Order Supervision: Its coupled prediction-reconstruction objective forces the latent space to be simultaneously reconstructable and ODE-plannable.Because ODEWorld need only fit some admissible latent-space ODE, the target is underdetermined and the entangled objective becomes overly restrictive.
  • B.3 Ablations on Vision Encoders: ODEWorld maintains robust video-generation quality across DINOv2, SigLIP 2, and LIV, with DINOv2 achieving the best overall short- and long-horizon performance.The gaps between encoders are small.
  • B.4 Ablation on Token Numbers: The 1- and 4-token models show comparable performance, indicating that one latent token is sufficient across the evaluated scenarios.This compact representation can efficiently capture underlying dynamics directly from pixels.

B.5 Ablation on Goal Descriptions · B.6 ODEWorld without Detach in Velocity Learning

ODEWorld extends goal conditioning from images to language through a lightweight goal image predictor, preserving video prediction quality. PT-Flow also remains effective without stop-gradient, indicating that its objective constrains meaningful temporal representations.

  • B.5 Ablation on Goal Descriptions: B.5 Ablation on Goal Descriptions: ODEWorld uses a lightweight goal image predictor to convert language instructions into visual goals for future dynamics prediction.The predictor bridges language and visual modalities by taking the current observation and instruction as inputs.
  • B.5 Ablation on Goal Descriptions: B.5 Ablation on Goal Descriptions: The predictor uses four cross-attention layers while retaining DINOv2 as ODEWorld’s visual backbone instead of adding a large-scale vision-language model.This design avoids additional multimodal alignment complexity in the dynamics model.
  • B.5 Ablation on Goal Descriptions: B.5 Ablation on Goal Descriptions: Given current observations and language instructions, the goal image predictor generates semantically consistent goal images for ODEWorld.Figure 8 presents qualitative results of the predicted goal images.
  • B.5 Ablation on Goal Descriptions: B.5 Ablation on Goal Descriptions: Using predicted goal images achieves nearly identical video prediction performance to ground-truth image goals on short- and long-horizon rollouts.The comparison is reported in Table 7 and supports language-specified goals without sacrificing prediction quality.
  • B.6 ODEWorld without Detach in Velocity Learning: B.6 ODEWorld without Detach in Velocity Learning: PT-Flow applies sg(·) to target latent velocity to stabilize optimization, following common representation-learning practice.The stop-gradient operation is used during velocity learning.
  • B.6 ODEWorld without Detach in Velocity Learning: B.6 ODEWorld without Detach in Velocity Learning: PT-Flow remains effective without stop-gradient, suggesting its objective independently constrains meaningful representations.The first-order velocity constraint directly regularizes temporal evolution, as shown in Table 8.

C Analysis of Representation Collapse · D Additional Results · D.1 Additional Video Generation Quality Comparisons

ODEWorld maintains high-rank dynamics representations across planning horizons, addressing representation collapse. Additional LIBERO visualizations show temporally coherent, physically plausible long-horizon rollouts compared with baseline degradation and inconsistency.

  • C Analysis of Representation Collapse: ODEWorld’s 768-dimensional dynamics latent z maintains an effective rank above 400 across planning horizons h ∈ {1, 4, 8, 16, 32}.The analysis randomly samples 5,000 mean-centered representations following RankMe.
  • C Analysis of Representation Collapse: The collapse analysis evaluates mean-centered representations using effective rank, following RankMe.Representations are sampled across five planning horizons to assess potential collapse.
  • C Analysis of Representation Collapse: Table 10 compares ODEWorld representations with V-JEPA 2 representations and DINOv2 CLS features under the same sampling protocol.The comparison uses centered RankMe effective rank.
  • C Analysis of Representation Collapse: Table 9 reports ODEWorld dynamics-latent effective ranks across different planning horizons.The table summarizes the centered RankMe analysis for ODEWorld’s dynamics latents.
  • D Additional Results: Additional results provide long-horizon video-generation comparisons in the LIBERO simulation environment.These visualizations appear in Fig. 9 and complement the main paper’s results.
  • D.1 Additional Video Generation Quality Comparisons: ODEWorld produces temporally coherent and physically plausible rollouts over extended horizons.The additional visualizations are presented as further evidence from the LIBERO environment.
  • D.1 Additional Video Generation Quality Comparisons: Baseline methods often exhibit visual degradation and temporal inconsistency during extended-horizon rollouts.The comparison is qualitative and is shown in the additional LIBERO visualizations.

D.2 Additional Bidirectional Video Generation Results

ODEWorld demonstrates additional bidirectional video generation on LIBERO and AgiBot World using closed-loop planning with continuous latent-state updates. Backward prediction is implemented by integrating the negated velocity and resetting the planned latent state at each step.

  • D.2 Additional Bidirectional Video Generation Results: Additional bidirectional video generation results are presented on both LIBERO and AgiBot World.The results are shown in Figure 10.
  • D.2 Additional Bidirectional Video Generation Results: Closed-loop planning updates the initial state after every integration step τ = 0.05.This procedure repeatedly refreshes the starting state during planning.
  • D.2 Additional Bidirectional Video Generation Results: For backward planning, ODEWorld integrates velocity −vθ(z0, 0; z0, c) at each step and resets the planned latent state to z0 for the next step.This enables backward prediction within the continuous-time planning procedure.
Loading 2607.27924v2…