PaperlayerView exact revision on arXiv

Source-linked AI summary

FluidWorld: Reaction-Diffusion Dynamics as a Predictive Substrate for World Models

Fabien Polly

arXiv:2603.21315v1cs.LG

TL;DR

The paper asks whether self-attention is necessary for predictive world modeling and introduces FluidWorld, which uses reaction-diffusion PDE integration as its predictive substrate. In a parameter-matched UCF-101 ablation, FluidWorld matches single-step loss while achieving stronger reconstruction fidelity and more stable multi-step rollouts than Transformer and ConvLSTM baselines.

  • Problem

    The paper examines whether self-attention is strictly necessary for predictive world modeling, given the dominant use of Transformer-based predictive engines.

  • Method

    FluidWorld replaces a separate neural-network predictor with reaction-diffusion PDE dynamics that generate future latent-state predictions.

  • Results

    2× better reconstruction fidelity and coherent multi-step rollouts 1–2 horizons longer than both baselines were achieved while all three architectures had comparable single-step prediction loss.

  • Takeaways & Limitations

    PDE-based dynamics are presented as a viable parameter-efficient alternative to attention and convolutional recurrence for world modeling at this scale.

  • Takeaways & Limitations

    The proof-of-concept is limited to this scale, and analogous regularization mechanisms were not individually ablated for the baselines.

Abstract

from arXiv · show

World models learn to predict future states of an environment, enabling planning and mental simulation. Current approaches default to Transformer-based predictors operating in learned latent spaces. This comes at a cost: O(N^2) computation and no explicit spatial inductive bias. This paper asks a foundational question: is self-attention necessary for predictive world modeling, or can alternative computational substrates achieve comparable or superior results? I introduce FluidWorld, a proof-of-concept world model whose predictive dynamics are governed by partial differential equations (PDEs) of reaction-diffusion type. Instead of using a separate neural network predictor, the PDE integration itself produces the future state prediction. In a strictly parameter-matched three-way ablation on unconditional UCF-101 video prediction (64x64, ~800K parameters, identical encoder, decoder, losses, and data), FluidWorld is compared against both a Transformer baseline (self-attention) and a ConvLSTM baseline (convolutional recurrence). While all three models converge to comparable single-step prediction loss, FluidWorld achieves 2x lower reconstruction error, produces representations with 10-15% higher spatial structure preservation and 18-25% more effective dimensionality, and critically maintains coherent multi-step rollouts where both baselines degrade rapidly. All experiments were conducted on a single consumer-grade PC (Intel Core i5, NVIDIA RTX 4070 Ti), without any large-scale compute. These results establish that PDE-based dynamics, which natively provide O(N) spatial complexity, adaptive computation, and global spatial coherence through diffusion, are a viable and parameter-efficient alternative to both attention and convolutional recurrence for world modeling.

1 Introduction

World models support planning by predicting future states, but Transformer predictors impose quadratic spatial cost and lack explicit spatial inductive bias. FluidWorld tests reaction-diffusion PDEs as an alternative predictive substrate through a parameter-controlled comparison.

  • World models predict future states or representations, enabling agents to simulate candidate actions before execution.
  • O(N^2) self-attention cost limits resolution, while Transformers must learn spatial propagation without an explicit spatial inductive bias.
  • The experiments focus on unconditional video prediction as an architectural proof-of-concept rather than state-of-the-art world-model performance.
  • FluidWorld replaces attention-based prediction with reaction-diffusion PDE integration, using diffusion for spatial propagation and learned reaction terms for nonlinear transformation.
  • The study compares PDE, Transformer, and ConvLSTM substrates at ∼800K parameters with identical encoders, decoders, losses, and data.
  • The analysis emphasizes O(N) efficiency, richer spatial representations, and longer coherent multi-step rollouts for PDE dynamics.

2 Related Work

Prior world models and representation-prediction systems rely on neural-network dynamics, especially Transformers, while PDE research has mainly targeted physical simulation or other processing tasks. FluidWorld instead treats the PDE itself as the learned world-model substrate.

  • Established world models use VAE-RNN, RSSM, Transformer, or related neural architectures as predictive engines.
  • JEPA systems predict representations rather than pixels but retain Vision Transformer predictors.
  • Prior PDE-inspired networks learn physical operators, simulate systems, or process images and graphs rather than model predictive world dynamics.
  • Convolutional recurrent models provide important video-prediction baselines with both spatial and temporal inductive biases.
  • FluidWorld replaces attention entirely with PDE dynamics that provide O(N) complexity by construction.

3 From Language to World Models: The Fluid Architecture Lineage

The Fluid architecture line extends reaction-diffusion computation across modalities, from language and perception to temporal world modeling. Across iterations, O(N) complexity and adaptive computation remain consistent design properties.

  • FluidLM applied reaction-diffusion dynamics to 1D language sequences, but remained behind standard Transformers on language tasks.
  • FluidVLA extended the substrate to 2D vision and robotics, achieving competitive perception and real-time control results.
  • FluidWorld extends the PDE substrate from perception to prediction through a BeliefField that uses reaction-diffusion dynamics for temporal evolution.
  • The same diffusion-plus-reaction equation has been applied to 1D sequences, 2D images, 3D volumes, and temporal prediction.

4 Method

FluidWorld encodes video frames into spatial features, writes them into a persistent BeliefField, evolves that state with reaction-diffusion PDEs, and decodes current and predicted frames. Its dynamics combine diffusion, nonlinear reaction, memory, adaptive integration, and biologically inspired regularization.

  • 4.1 Overview: FLUIDWORLD processes each frame by encoding spatial features, evolving a persistent BeliefField, and decoding reconstructed or predicted frames.
  • 4.1 Overview: The same PDE equation governs spatial encoding and temporal evolution, with conditioning differing between spatial structure and temporal dynamics.
  • 4.2 Reaction-Diffusion Dynamics: Laplacian diffusion propagates information across feature maps, while a position-wise MLP supplies learned nonlinear reaction dynamics.
  • 4.2 Reaction-Diffusion Dynamics: The PDE update combines diffusion and reaction with global and local memory terms during iterative integration.
  • 4.3 Adaptive Computation: Adaptive inference stops integration when a low-resolution probe stabilizes, using ∼3 steps for static scenes and up to 12 for dynamic scenes.
  • 4.4 Encoder: The encoder uses p = 4 patch embedding, three PDE layers, a spatial skip connection, and biologically inspired regularization.
  • 4.4 BeliefField: The BeliefField writes observations through a learned decay gate, evolves them with internal PDE dynamics, and reads predicted features by interpolation.
  • 4.5 Biologically-Inspired Mechanisms: Lateral inhibition, synaptic fatigue, and Hebbian diffusion counter channel collapse and promote structured, diverse representations.

5 Scaling Experiment: PDE vs Transformer vs ConvLSTM

The controlled ablation isolates predictive substrates by matching model parameters, data, losses, training, encoder front-end, and decoder across PDE, Transformer, and ConvLSTM models. FLUIDWORLD matches single-step prediction while achieving lower reconstruction error, richer representations, and more coherent multi-step rollouts, though iterative integration reduces training speed.

  • Controlled comparison: All three models use the same encoder front-end, decoder, losses, UCF-101 data, optimizer, learning-rate schedule, batch size, and 8,000 training steps.
  • Controlled comparison: The only architectural difference is the computational engine: PDE diffusion and reaction, Transformer self-attention, or ConvLSTM convolutional recurrence.
  • Prediction quality: 2× lower reconstruction error: FLUIDWORLD reaches 0.001 versus 0.002 MSE for the Transformer, while ConvLSTM also reaches 0.001.
  • Prediction quality: Comparable single-step prediction loss: all three models converge near 0.003–0.004 MSE.
  • Representation quality: FLUIDWORLD records the highest Spatial Std, 1.16 versus 1.12 for ConvLSTM and 1.05 for Transformer, and the highest Effective Rank, approximately 2.0×10^4.
  • Rollout stability: FLUIDWORLD maintains recognizable spatial structure through h=3, whereas both baselines degrade rapidly during multi-step autoregressive rollouts.
  • Computational efficiency: Training throughput is approximately 7.8 it/s for ConvLSTM, 5.2 it/s for Transformer, and 1 it/s for FLUIDWORLD.

6 Analysis

The analysis explains FLUIDWORLD’s advantage through linear-cost diffusion and spatially coherent error smoothing. Although all models perform similarly at one-step prediction, PDE dynamics preserve structure longer and can recover from accumulated rollout errors.

  • 6.1 Computational Complexity: PDE diffusion costs O(N), while Transformer attention costs O(N^2d), so the computational advantage grows with spatial token count.
  • 6.1 Computational Complexity: At higher resolution, PDE-based models become increasingly advantageous because diffusion scales linearly while attention scales quadratically.
  • 6.2 Rollout Stability: Comparable one-step prediction losses diverge in autoregressive rollout because teacher-forced metrics do not measure compounded errors from feeding predictions back into the model.
  • 6.2 Rollout Stability: FLUIDWORLD maintains spatial coherence 1–2 steps longer than both baselines because Laplacian diffusion enforces spatial continuity during integration.
  • 6.2 Rollout Stability: FLUIDWORLD preserves recognizable structure through h=3, while Transformer and ConvLSTM predictions collapse by h=2 into mean-color patches or texture noise.
  • 6.2 Rollout Stability: SSIM for FLUIDWORLD recovers at steps 6–9 after an earlier decline, unlike the monotonic decay shown by Transformer and ConvLSTM baselines.
  • 6.2 Rollout Stability: The three models exhibit distinct degradation signatures: contrast reduction for PDE, spatial mean-color patches for Transformer, and repetitive texture artifacts for ConvLSTM.
  • 6.3 Loss interaction: The fair comparison finds that edge and frequency losses can destructively interfere with Laplacian smoothing under equal training budgets.

7 Discussion

The discussion argues that PDE dynamics are most valuable for stable autoregressive prediction rather than single-step metrics, while emphasizing mechanistic evidence and important scope limits. FluidWorld shows self-organization, corruption recovery, and diffusion-mediated error dissipation, but remains a small, narrowly evaluated proof-of-concept.

  • Interpretation: Comparable single-step losses obscure the PDE substrate’s main advantage: diffusion smooths prediction errors during autoregressive rollouts.The paper therefore argues that rollout stability should receive greater emphasis than single-step loss when evaluating world-model architectures.
  • Representation dynamics: The PDE spontaneously generates spatial structure from a nearly uniform field, reducing the symmetry index from 1.0 to 0.2 within 10 integration steps.Spatial entropy rises from 1.7 to 2.7, and KMeans detects three emerging spatial clusters.
  • Representation dynamics: When 50% of the belief field is corrupted, the system recovers without an explicit repair mechanism, with recovery taking 0, 3, or 7 steps depending on corruption modality.Residual MSE remains approximately 0.034 up to 50% corruption across the tested modalities.
  • Mechanism: The PDE’s MSE spikes to 0.181 at step 4 and recovers to 0.050 at step 8, matching error accumulation followed by Laplacian dissipation.The authors present this oscillatory pattern as the mechanistic signature of a damped dynamical system relaxing toward an attractor.
  • Rollout dynamics: 66.8% of 500 rollouts exhibit measurable SSIM recovery, with statistical tests indicating that the non-monotonic trajectory is systematic rather than noise.The reported recovery effect has Cohen’s d = 0.739, with t = 16.5 and Wilcoxon p = 5.88 × 10−66.
  • Ablation: Ablating the Laplacian-supporting design with edge and frequency losses causes collapse at equal training budgets, linking smoothing to oscillatory self-correction.With doubled training, the alternative avoids collapse but produces monotonic recovery rather than the oscillatory burst-and-relax pattern.

8 Conclusion

FLUIDWORLD shows that reaction-diffusion PDE dynamics can replace attention in a parameter-matched world-model ablation, with stronger reconstruction fidelity and more coherent multi-step rollouts. The paper presents this as a proof-of-concept rather than a general replacement claim.

  • In a matched ∼800K-parameter comparison, PDE, Transformer, and ConvLSTM models achieve comparable single-step prediction loss, while the PDE achieves 2× better reconstruction fidelity.The comparison uses identical decoder, losses, and data.
  • The PDE maintains coherent multi-step rollouts 1–2 horizons longer than both baselines, which degrade rapidly beyond single-step prediction.The paper attributes this advantage to Laplacian diffusion acting as an implicit spatial regularizer.
  • 66.8% of rollouts exhibit measurable SSIM recovery after degradation, unlike the attention-based and recurrence-based baselines tested here.
  • Recovery remains partial at high corruption intensities of 0.7–0.9, while low-intensity corruption at 0.1–0.3 is recovered near-completely.The reported corruption types are Gaussian noise, channel zeroing, and spatial shuffling.
  • On Moving MNIST, a JEPA-style PDE rollout maintains cosine similarity ≥0.827 through step 19 without pixel-level supervision.

B.2 Decoder Architecture

The PixelDecoder uses a symmetric upsampling path that progressively reconstructs 64×64 output logits from latent features. Its design combines channel projection, residual blocks, bilinear upsampling, normalization, and convolution.

  • Decoder Architecture: The decoder maps (B, 128, 16, 16) latent features to (B, C, 64, 64) output logits through progressive upsampling.The path uses channel projection, residual blocks, and bilinear upsampling at 32×32 and 64×64 resolutions.
  • Decoder Architecture: The architecture retains residual processing at input, mid-resolution, and fine-detail stages.
  • Decoder Architecture: ∼231K decoder parameters support the reconstruction pathway, while bilinear upsampling avoids checkerboard artifacts.

C Additional Experimental Details

The supplementary experiments use short UCF-101 frame windows and compare autoregressive rollout behavior across the PDE, Transformer, and ConvLSTM models. The PDE degrades most slowly while preserving spatial structure longer than the baselines.

  • Additional Experimental Details: UCF-101 provides 13,320 videos across 101 action categories, resized to 64 × 64 with windows of T + 1 = 5 consecutive frames.No data augmentation is applied.
  • Autoregressive Rollouts: All three models produce recognizable content at h = 1, but degrade beyond one-step prediction at different rates.The rollout analysis focuses on how autoregressive errors affect coherence from the initial frame.
  • Autoregressive Rollouts: The PDE shows the slowest degradation, preserving edges and object boundaries up to h=3 through Laplacian spatial smoothing.
  • Autoregressive Rollouts: The Transformer converges toward uniform mean-color patches as global attention averages away spatial detail.
  • Convergence Dynamics: Figure 17 shows representation-quality gaps narrowing for Spatial Std and Effective Rank while the reconstruction-fidelity gap widens.The figure attributes the PDE advantage to structure rather than merely faster convergence.
  • Autoregressive Rollouts: The ConvLSTM dissolves into repetitive texture artifacts, degrading as fast as the Transformer despite its spatial inductive bias.Its fixed 3×3 receptive field limits global context propagation, while recurrent state accumulates spatially incoherent errors.

D Supplementary Figures from Experimental Notebooks

The supplementary figures from the experimental notebooks provide visual evidence for claims made in the main text.

  • Supplementary Figures from Experimental Notebooks: The figures are produced by experimental notebooks and serve as visual evidence for claims in the main text.

D.7 Feature Analysis (cf. §5)

The feature analysis shows that FluidWorld forms structured, stable, and resilient spatial representations through PDE integration. These representations recover from corruption, preserve information through occlusion when augmented with memory, and maintain diverse semantic features.

  • Spatial structure: Within 10 integration steps, near-uniform BeliefField states deterministically form distinct spatial clusters, with patterns varying by seed but remaining reproducible.Symmetry falls from 1.0 to 0.2, entropy rises from 1.7 to 2.7, and KMeans detects three clusters.
  • Spatial structure: The trained operating point D ≈0.25, ∆t = 0.1 lies in the super-critical regime, while dynamics freeze near ∆t ≈0.02.This phase behavior identifies the operating range in which spatial structure develops.
  • Stability: Energy converges to the same attractor E∗≈8,640 from uniform, random, and structured-gradient initializations within approximately 50 steps.With RMSNorm, energy remains bounded and high-frequency components decay to 10^-8 by step 200; without RMSNorm, energy diverges by ×7,467.
  • Stability: Numerical oscillatory instabilities appear when ∆t > 0.10, confirming a CFL-like stability bound for the discrete Laplacian.The timestep therefore constrains stable PDE integration.
  • Resilience: After 50% corruption, the BeliefField recovers its pre-corruption trajectory in 0, 3, or 7 steps for channel zeroing, Gaussian noise, or spatial masking.Residual MSE stays approximately 0.034 through 50% corruption and then increases monotonically without a cliff effect.
  • Representation and memory: Pixel-trained features encode object positions and boundaries without dead channels, achieving effective rank 19.5 and healthy representational diversity.With 20-frame occlusion, the Titans-augmented model reaches SSIM 0.6 versus 0.05 for PDE-only, while memory activity rises from 0.018 to 0.035.
Loading 2603.21315v1…