PaperlayerView exact revision on arXiv

Source-linked AI summary

Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning

Hansen Jin Lillemark, Alex Rojas, Zachary Novack, Runqian Wang, Yilun Du, Yian Ma, Taylor Berg-Kirkpatrick, Rose Yu

arXiv:2608.14706v1cs.CVcs.AIcs.LG

TL;DR

Autoregressive video denoising relies on rigid noise-conditioned schedules that cannot adapt to generation feedback. Equilibrium Forcing removes noise conditioning, decouples denoising-field learning from sampling, and improves quality and consistency through closed-loop, data-adaptive inference.

  • Problem

    Noise-conditioned video denoising uses open-loop schedules whose true noise level can diverge during autoregressive rollouts, limiting data adaptivity.

  • Method

    Equilibrium Forcing learns a single noise-unconditional denoising field and decouples its training objective from modular, feedback-driven sampling dynamics.

  • Results

    EQF improves video quality and consistency on challenging autoregressive benchmarks, including when finetuning pretrained 1.3B- and 5B-parameter Flow Matching models.

  • Takeaways & Limitations

    Noise conditioning and strict inference schedules are not necessary for stable modern video denoising, within the evaluated autoregressive generation settings.

Abstract

from arXiv · show

Standard autoregressive video generation algorithms based on Diffusion and Flow Matching rely on rigid training objectives and static sampling schedules, limiting inference procedures from adapting to the data. We introduce Equilibrium Forcing (EqF), a simplified framework for video denoising generative models without noise level conditioning. EqF pioneers modular training- and inference-time designs for noise-unconditional generation that decouple learning the denoising field from sampling. This flexibility allows for inference-time algorithms that operate in a closed loop by adapting to feedback from the sample, improving video quality and consistency on challenging autoregressive video generation benchmarks. Extensive analysis elucidates exactly how removing the noise level conditioning enables EqF's data-dependent inference properties to surpass the performance of standard noise level-conditional denoising video methods.

1 Introduction

Autoregressive video generation suffers from accumulating errors and rigid noise-conditioned inference schedules that prevent adaptation. Equilibrium Forcing addresses these limitations by separating denoising-field learning from sampling and enabling feedback-driven, data-adaptive inference.

  • Motivation: Autoregressive video models accumulate local errors over time, motivating samplers that use feedback from the generation state.Future frames are generated conditionally on previously generated segments under limited inference budgets.
  • Problem: Noise-conditioned denoising models follow rigid open-loop schedules that predetermine noise removal and preclude adaptive sampling.The schedules pass noise conditions directly to the model and were considered necessary for stabilizing autoregressive generation.
  • Method: EQF uses feedback to estimate denoising progress, reallocate the remaining sampling trajectory, and adaptively accelerate convergence toward higher-quality samples.At inference time, transport fields can be connected with fixed-point attractor fields, supporting gradient-based samplers and budget-adaptive inference.
  • Method: EQF simplifies noise-unconditional denoising by training with an equilibrium objective and shaping favorable sampling landscapes at inference time.This design avoids the multiple training-time hyperparameters previously suggested for noise-unconditional image generation.
  • Results: Despite using less information and a simpler objective than noise-conditional models, EQF improves generation quality and consistency on difficult autoregressive video datasets.The reported gains are attributed to closed-loop feedback, gradient-based samplers, and budget adaptivity.

2 Background

Flow Matching models video generation by learning a noise-level-dependent velocity field, while Diffusion Forcing assigns each frame its own noise level and supports context-conditional autoregressive sampling. Standard inference follows an externally defined schedule whose steps are determined by noise-level differences.

  • Flow Matching: Flow Matching transports samples from a simple prior q(ϵ) to the data distribution p(x) using a learned noise level-dependent velocity field.Within Diffusion Forcing, each video frame has its own noise level σ = [σ1, σ2, . . . , σT] ∈ [0, 1]T, and generation solves a probability flow ODE.
  • Flow Matching: The Flow Matching network is conditioned on the exact noise level to predict a conditional velocity from paired noisy data and corresponding noise.The noising process linearly interpolates Gaussian noise and clean data framewise.
  • Inference: Standard inference starts from pure noise at σ(0) = 1 and integrates to σ(N) = 0 along a predefined schedule.ODE solvers determine each step size from the difference between consecutive noise levels.
  • Autoregressive generation: Autoregressive video generation repeatedly samples p(x|c) with a sliding window, while Diffusion Forcing handles noisy and clean frames simultaneously to enable context-conditional sampling.The context c may consist of ground-truth context or previously generated frames.

3 Divergent Noise Level Conditions in Flow Matching

Noise-conditional flow-matching inference uses fixed, open-loop schedules that increasingly diverge from samples’ true noise levels during autoregressive generation. This divergence compounds rollout errors, motivating removal of noise conditioning to enable data-adaptive inference.

  • Scheduled Noise Levels Diverge During Flow Matching Inference: Fixed noise-conditioning schedules cannot respond when a sample’s true noise level diverges from the scheduled level during inference.The divergence is attributed to the open-loop nature of strict noise conditioning schedules.
  • Scheduled Noise Levels Diverge During Flow Matching Inference: A framewise predictor gϕ estimates each sample’s true noise level from an individually noised frame, with posterior concentration supporting accurate estimates in practice.The predictor maps xσ to an estimated noise level ˜σ ≈ gϕ(xσ).
  • Scheduled Noise Levels Diverge During Flow Matching Inference: Across 256 autoregressive rollouts, absolute error between scheduled and predicted noise levels increases over time, rather than remaining zero.The experiment starts with 25 ground-truth context frames and generates 275 frames; accurate conditioning would yield zero divergence throughout.
  • Noise Level Divergence Causes Compounding Error Over Autoregressive Video Rollouts: Removing noise conditioning altogether performs best over 256 autoregressive rollouts, while conditioning on estimated noise levels helps only slightly.The comparison is shown against standard flow matching with externally prescribed noise conditions.
  • Noise Level Divergence Causes Compounding Error Over Autoregressive Video Rollouts: Accumulated noise-level divergence pushes subsequent video-generation inputs further out of distribution, causing compounding errors across autoregressive rollouts.FVD is measured over 25-frame chunks of the same 275-frame rollouts against 25-frame ground-truth clips.

4 Equilibrium Forcing

Equilibrium Forcing (EqF) removes explicit noise-level conditioning, learning a single denoising field that supports sample-driven inference. Its noise-level estimation, attractor-style warping, and budget-adaptive solvers decouple denoising-field learning from fixed sampling schedules.

  • Framework: EqF learns a single field over data rather than separate velocity fields for each noise level, mitigating noise-conditioning errors.Samples follow the noise-autonomous ODE dx_σ/dσ = f_EQF(x_σ), with the field approximated by a neural network.
  • Theory: EqF’s Bayes-optimal denoiser averages fixed-noise targets over p(σ | x_σ), while high-dimensional posterior concentration makes this average approach the target at the identified noise level.This decomposition explains how EqF can learn the same denoising field as Flow Matching without explicit noise-level conditioning.
  • Adaptive inference: A lightweight framewise readout predicts the posterior-mode noise level from EqF activations, providing a proxy for sampling progress toward the data manifold.The estimator uses internal features z rather than the raw noisy input x_σ.
  • Adaptive inference: Choosing η(σ) → 0 as σ → 0 converts sampling into a fixed-point attractor search for η(σ) ⊙ f_EQF(x) = 0.This enables gradient-based optimization solvers within a closed-loop sampling framework, unlike transport-style sampling’s nonzero terminal constraint.
  • Budget-adaptive inference: Algorithm 1 uses EqF’s equilibrium field and noise-level estimates for budget-aware inference instead of relying solely on a schedule fixed before inference.The sampler dynamically rescales each step using the estimated noise level and remaining sampling budget.

5 Experiments

Experiments show that EqF’s noise-unconditional formulation enables data-adaptive sampling that improves long-horizon video generation over noise-conditional and noise-unconditional baselines. These gains extend from models trained from scratch to equilibrium-finetuned large pretrained models, while prediction analyses support the connection between FM and EQF fields.

  • Autoregressive Video Generation: EqF with NAG sampling achieves the best FVD and VBench scores on 300-frame autoregressive Minecraft rollouts across five seeds.Table 1 compares EQF with NAG against the evaluated baselines.
  • Equilibrium Finetuning: EQF-FT with closed-loop sampling and NAG improves upon FM on RealEstate10K, showing that equilibrium finetuning enables data-adaptive sampling in a large pretrained video model.The model starts from Wan 2.1 T2V 1.3B and is finetuned after removing external noise conditioning.
  • Closed-Loop Acceleration with NAG: Closed-loop acceleration experiments find that shaping the inference landscape into an attractor at the data manifold is necessary for NAG convergence because NAG is minimum-seeking.The study compares multiple η functions and identity mapping on the Minecraft setup.
  • Online Replanning Inference: At matched FVD thresholds in online replanning, Flow Matching requires 13% to 29% more compute than Adaptive EQF.EQF reuses previous predictions when new ground-truth context arrives without fully recomputing the future prediction.
  • Field Correspondence: Across datasets, FM and EQF predictions on identical held-out noisy inputs correlate above 0.98, while the residual gap can be as small as 2.4% of denoiser output magnitude.The FM–EQF loss gap also closes during RealEstate10K finetuning.

6 Related Work

EqF extends noise-unconditional generative modeling from images to video and separates denoising-field learning from data-adaptive sampling. Comparisons with NU-FM and EqM position adaptive inference and modularity as its key distinctions.

  • Noise-unconditional generative modeling: EqF is presented as the first work to study removing noise level conditioning for temporal data such as video.Prior noise-unconditional generative-modeling studies primarily focused on image generation.
  • Noise-Unconditional Flow Matching: NU-FM introduced a noise-unconditional Flow Matching training recipe but did not improve over Flow Matching on large-scale image datasets or explore data-adaptive sampling.Adapted to video through EqF’s framework, NU-FM provides a base inference setting without additional data-adaptive sampling.
  • Noise-Unconditional Flow Matching: EqF results with NU-FM support data-adaptive sampling as the key factor enabling noise-unconditional models to improve over traditional models.The passage attributes the improvement specifically to adaptive sampling rather than noise-unconditional training alone.
  • Equilibrium Matching: EqM removes explicit noise conditioning and uses a modulated training target whose field magnitude vanishes at clean data, enabling attractor-style image sampling.Unlike EqF’s separation of training and inference design, EqM requires training-time searches over the field shape c and magnitude λ.
  • Comparison on ImageNet: The paper also applies EqF’s insights to ImageNet, training an EQF model under the EqM paper’s settings and evaluating an inference-time c-function η warp across 3 seeds.This comparison uses the SiT architecture and an XL/2 model trained for 80 epochs.

7 Conclusion … F Details on Estimating Noise Levels from Data or EQF Activations

EQF uses noise-unconditional training to enable adaptive, closed-loop inference that improves video generation quality and consistency. Its modular separation of training and inference surpasses prior noise-unconditional frameworks and challenges the need for explicit noise conditioning.

  • 7 Conclusion: EQF’s simplified noise-unconditional objective enables adaptive, closed-loop inference for autoregressive video generation.The approach improves quality and consistency on challenging video generation tasks.
  • 7 Conclusion: Modularly decoupling noise-unconditional training from attractor-like inference allows EQF to surpass prior noise-unconditional frameworks.The decoupling separates learning from inference design.
  • 7 Conclusion: EQF’s results challenge the assumption that explicit noise conditioning is necessary for effective video denoising generation.The supplied passage frames this as a broader implication of the reported results.

G Details on Sampler Warp Schedules … A.2 EqM c-Function Inference Relationship to EQF

The analysis shows that EqM’s training-time modulation changes the induced noise-level weighting, whereas EQF can reproduce EqM-style behavior through inference-time warping under posterior concentration. This supports treating equilibrium and attractor-style behavior as modular inference decisions rather than training-objective requirements.

  • A.1 Effect of EqM Modulation on Target: When c(σ) = σ, EqM’s weighting becomes wEqM = 1, yielding an unweighted clean-data prediction objective.This example illustrates how the modulation term determines the effective weighting over noise levels.
  • A.1 Effect of EqM Modulation on Target: EQF trains directly on the unmodulated velocity target, producing a loss weighting that more heavily emphasizes low-noise regions, similar to Flow Matching.The comparison follows from rewriting both velocity objectives as induced clean-data prediction losses.
  • A.1 Effect of EqM Modulation on Target: EqM’s modulation c(σ) changes the induced clean-data prediction weighting by a factor c(σ)^2 relative to EQF.Both objectives share the same pointwise Bayes-optimal target under the stated concentration assumption, despite differing training weightings.
  • A.2 EqM c-Function Inference Relationship to EQF: Under noise-level posterior concentration, EqM’s target modulation c(σ) is equivalent to applying the same function as an inference-time warp to the EQF velocity field.The equivalence holds in the Bayes-optimal and posterior-concentration limit, with the warp matching c up to normalization.
  • A.2 EqM c-Function Inference Relationship to EQF: High-dimensional posterior concentration makes p(σ | xσ) approximately a Dirac delta, collapsing expectations over σ to evaluation at the estimated noise level bσ.The stated approximation is p(σ | xσ) ≈δ(σ −bσ), with E σ [g(σ) | xσ] ≈g(bσ).
  • A.2 EqM c-Function Inference Relationship to EQF: The proportionality between an EqM c-function and an EQF transport warp does not mean every c-function directly defines a valid probability-flow transport.A valid transport warp must integrate to 1, while EqM’s generally defined c-function satisfies c(σ) →0 as σ →0 and may require the additional hyperparameter λ.
  • A.2 EqM c-Function Inference Relationship to EQF: EqM’s training-time target modulation is theoretically unnecessary under noise-level concentration and introduces an undesirable training weighting over noise levels.The analysis therefore favors treating equilibrium landscapes and attractor-style sampling as modular inference-time choices rather than embedding them in training.

A.3 Proof of Proposition 4.1 … B.2 Noise Conditioning Divergence Directly Causes Increased Denoising Loss

The analysis shows that EQF’s unconditioned denoiser is lower-bounded by noise-conditioned Flow Matching but can attain the same optimum under pointwise error minimization, while noise-level divergence during autoregressive inference increases denoising loss. Empirically, trained FM and EQF fields remain closely aligned, whereas mismatched context and prediction-horizon conditions further worsen loss.

  • A.3 Proof of Proposition 4.1: EQF’s denoising loss is lower-bounded by the noise-conditioned Flow Matching loss, while pointwise error minimization can achieve the same optimum.The decomposition establishes the lower bound and the condition for equality.
  • A.3 Proof of Proposition 4.1: Under posterior concentration, EQF marginalizes Flow Matching’s Bayes-optimal denoisers and must implicitly estimate the noise-level posterior to minimize its objective.The EQF denoiser averages over p(σ | xσ), and the regression-target gap must be driven to zero under concentration.
  • A.4 FM-to-EQF Residual Gap: EQF’s denoising loss is only slightly larger than FM’s on identical held-out noisy inputs, consistent with the predicted lower bound.The comparatively small FM/EQF residual explains the loss gap, especially for Re10K and Droid.
  • A.4 FM-to-EQF Residual Gap: Across all three domains, trained FM and EQF velocity fields show small normalized discrepancy and high Pearson correlation despite EQF receiving no explicit noise-level condition.The comparison uses matched held-out samples with identical clean inputs, noise levels, and Gaussian noise.
  • B Additional Analysis on the Effect of Divergent Noise Levels in Flow Matching: The additional analysis suggests a causal relationship between noise-level divergence during Flow Matching inference and degraded autoregressive performance.The analysis focuses on how divergent noise conditioning affects inference-time behavior.
  • B.1 Flow Matching Predictions Frequently Terminate Scheduled Sampling at Nonzero Noise: Flow Matching samples frequently terminate at a significant, nonzero noise level instead of reaching the ideal endpoint of noise level 0.The experiment visualizes predicted framewise noise levels during 300-frame Minecraft autoregressive rollouts with fixed denoising steps.
  • B.2 Noise Conditioning Divergence Directly Causes Increased Denoising Loss: Diverging the scheduled noise level from the true level over the prediction horizon causes denoising error.The experiment gives context frames the correct noise level 0 and uses (σstable + δ) · 1 for the prediction horizon.
  • B.2 Noise Conditioning Divergence Directly Causes Increased Denoising Loss: Loss is strictly higher when noise-level conditions are mismatched for both context and prediction horizon than when only the prediction horizon is corrupted.The comparison measures v-loss using δ · 1 for context and (σstable + δ) · 1 for the prediction horizon.

B.3 Simulated Closed-Loop Flow Matching Sampling Improves Sample Quality … C.1 Adaptive Early Stopping Algorithm

The appendix shows that predicted-noise feedback improves Flow Matching inference, especially for NAG, while EQF achieves the lowest final noise and supports adaptive early stopping through refinement.

  • B.3 Simulated Closed-Loop Flow Matching Sampling Improves Sample Quality: Artificial noise-conditioning errors are evaluated over the active sampling horizon, with context corruption measured as an additional gain.Noise conditions are clamped to [0, 1], and errors are standardized with respect to δ ∈[−0.07, 0.07].
  • B.3 Simulated Closed-Loop Flow Matching Sampling Improves Sample Quality: Conditioning on the inferred noise level gϕ(x(i)) substantially improves Flow Matching autoregressive generation, particularly when combined with NAG under the c Function schedule.Table 7 compares different inference schedules and conditioning choices.
  • B.3 Simulated Closed-Loop Flow Matching Sampling Improves Sample Quality: Predicted noise levels close the Flow Matching sampling loop and substantially improve inference, especially for NAG, which otherwise diverges completely.Direct noise conditioning remains worse than EQF in the 300-frame Minecraft evaluation.
  • B.4 Closed Loop Sampling with EQF Produces the Lowest Final Noise Level: Closed-loop EQF samples terminate at lower noise levels than Flow Matching and NU-FM, approaching the ideal termination noise level of 0.Figure 15 compares FM, NU-FM, EQF, and EQF with NAG.
  • B.4 Closed Loop Sampling with EQF Produces the Lowest Final Noise Level: Adding NAG to closed-loop EQF accelerates the sampling schedule and produces an even lower final noise level.The comparison is based on termination noise across FM, NU-FM, EQF, and EQF with NAG.
  • C Additional Results: The additional results cover a data-adaptive early stopping algorithm, budget-adaptive result tables, and an inpainting task on Re10K.Minecraft results are also included for the early stopping algorithm.
  • C.1 Adaptive Early Stopping Algorithm: EQF estimates convergence by stopping when bσ < σthresh, then applying a short decaying-step refinement period before completing denoising.The refinement performs additional cleanup steps from the budget-adaptive algorithm.

C.2 Budget-Adaptive Full Results · C.3 Inpainting Results on RealEstate10K

Section C.2 details budget-adaptive experiments and an EQF early-stopping procedure, while Section C.3 introduces a 49-frame RealEstate10K inpainting task with context-conditioned denoising. The adaptive procedure uses noise-level readouts to stop sampling and then optionally apply cleanup refinement.

  • C.2 Budget-Adaptive Full Results: Full budget-adaptive results are reported for Minecraft, Re10K, and Droid in Table 9.These experiments are presented as the complete results for the budget-adaptive evaluations from Section 5.
  • C.2 Budget-Adaptive Full Results: EQF adaptive early stopping estimates noise levels during sampling and stops when the maximum readout falls below σ_thresh.The procedure uses f_eqf(x), h_omega(z), a sampling warp η(σ), and thresholds σthresh = 0.1, 0.2, 0.3, 0.5, 0.7.
  • C.2 Budget-Adaptive Full Results: During each adaptive iteration, EQF predicts a denoising field, reads out the noise level, and updates x using the corresponding sampling warp.The update is x = x - eta(sigma_hat) * v_hat.
  • C.2 Budget-Adaptive Full Results: After stopping, EQF performs C_s cleanup steps using progressively scaled readout-based updates.The cleanup update is x = x - sigma_hat / (C_s - j) * v_hat.
  • C.3 Inpainting Results on RealEstate10K: The RealEstate10K inpainting task provides frame 1 and the final 4 frames, requiring denoising of the intermediate frames.The input and output together comprise 49 total frames, with the given frames serving as context.
  • C.3 Inpainting Results on RealEstate10K: In this non-autoregressive setup, context frames have zero noise while the remaining frames are noisy and are denoised with end-of-sequence information available.The task uses the same context-conditioned style as the described model.

D Experiment and Dataset Details … E.1 Minecraft Equilibrium Finetuning

The paper evaluates EQF across Minecraft, RealEstate10K, Droid, and ImageNet using defined metrics, inference settings, compute analyses, and finetuning studies. Results show that equilibrium finetuning and adaptive or closed-loop inference improve Minecraft outcomes while retaining simple, efficient denoising models.

  • D Experiment and Dataset Details: The experiments cover Minecraft, RealEstate10K, Droid, and ImageNet, with comparable training budgets and hyperparameters documented across model variants.Minecraft is trained from scratch, while RealEstate10K and Droid use finetuned Wan checkpoints.
  • D.1 Evaluation Metrics: FVD measures autoregressive video quality with lower scores preferred, while VBench evaluates semantic quality, frame quality, temporal consistency, and motion.FID is additionally used for the ImageNet experiments.
  • D.2 Minecraft: Minecraft contains 200k 300-frame, 256 × 256 action-conditioned videos, and standard inference uses 25 context frames to generate 275 future frames with 250 sampling steps.Adaptive Minecraft settings vary sampling steps, emission stride, context, active horizon, and replanning thresholds across separate protocols.
  • D.3 RealEstate10K: RealEstate10K uses camera-pose-conditioned property-tour clips to finetune Wan 2.1 T2V 1.3B into EQF, with adaptive inference using 29 context and 20 active video frames.The standard rollout generates 152 frames from 37 context frames, while adaptive sampling ranges from 10 to 50 steps.
  • D.4 Droid Robotics Dataset: Droid uses language-specified robot manipulation videos and finetunes Wan 2.2 TI2V 5B into EQF using unified self-attention context conditioning and the model’s UMT5 encoder and VAE.Its adaptive inference uses a 49-frame window with 13 context frames, 36 active frames, and 10–50 sampling steps.
  • D.5 ImageNet: ImageNet experiments train XL/2 models for 80 epochs with SD VAE, v-prediction, 10% class-label dropout, a rectified-flow path, batch size 256, and learning rate 1e-4.The model directly predicts unmodulated velocity.
  • D.6 Denoising Model and Noise Level Predictor Compute Comparison: EQF and Flow Matching backbones have identical GFLOP requirements because EQF fixes the noise condition to zero; hω costs 0.30 GFLOPs and gϕ costs approximately 0.89% of backbone compute.The section also compares wall-clock sampling time on Minecraft.
  • E Details on Equilibrium Finetuning: Equilibrium finetuning is introduced as a method for converting pretrained noise-conditioned Flow Matching models into noise-unconditioned equilibrium models, including across multiple scales and datasets.The Minecraft proof of concept continues a 580k-step Flow Matching checkpoint for 100k steps, sets σ = 0, and applies the EQF objective.

E.2 Re10K Equilibrium Finetuning … F.1 Architecture Design

The paper finetunes noise-unconditional EQF models on Re10K and Droid, then details two framewise CNN approaches for estimating noise levels from data or EQF activations. Re10K training shows EQF catches up to Flow Matching while validation FVD improves under open-loop inference.

  • E.2 Re10K Equilibrium Finetuning: EQF-FT finetunes Wan 2.1 T2V 1.3B on camera-conditioned Re10K using p = 1 dropout of noise conditioning, while FM-FT is a matched Flow Matching baseline.Pose processing follows Song et al. (2025), with conditioning introduced into the AdaLN stream.
  • E.2 Re10K Equilibrium Finetuning: EQF-FT initially forgets noise conditioning, then catches up to FM-FT, after which both models exhibit similar training dynamics.Training runs for 100k steps, and validation FVD continues improving throughout.
  • E.2 Re10K Equilibrium Finetuning: Validation inference for Figure 17 is strictly open loop and therefore does not use adaptive inference.Figure 17 reports Re10K validation loss and FVD across equilibrium finetuning steps.
  • E.3 Droid Equilibrium Finetuning: Droid finetuning applies the same EQF procedure as Minecraft and Re10K to Wan 2.2 TI2V 5B, using p = 1 dropout and matched FM-FT settings.Models are trained for 60k steps, with denoising quality observed to continue improving after model loss behavior changes.
  • F Details on Estimating Noise Levels from Data or EQF Activations: The paper introduces auxiliary noise-level predictors gϕ and readout predictors hω to estimate noise levels directly from noisy frames or from EQF activations.These predictors support noise-level divergence analyses on Minecraft.
  • F.1 Architecture Design: The direct predictor gϕ is a 600k-parameter framewise CNN trained on Flow Matching-noised frames because autoregressive denoising windows contain contiguous frames at different noise levels.The noisy frame is formed as x_t = (1 − σ)x_t + σϵ, and the network predicts the noise-level logit.
  • F.1 Architecture Design: The direct predictor uses logit-space loss for more balanced supervision near σ ≈ 0 and σ ≈ 1, then applies a sigmoid to obtain a valid noise level.Gaussian-shell concentration makes noise level identifiable from noisy data in high dimensions.
  • F.1 Architecture Design: The readout predictor hω is a 260k-parameter framewise CNN trained on Flow Matching-noised frames by bootstrapping from activations of a pretrained EQF denoiser.Its prediction is converted to a valid noise level with a sigmoid.

F.2 Noise Level Predictor and Readout Training Dynamics on Minecraft … J Online Replanning Inference Details

The appendix explains how EQF estimates noise internally, trains readout predictors, and decouples inference through adaptive warp, gradient-based, autoregressive, and replanning procedures. Across these designs, attractor-style dynamics and feedback-based inference support stable, data-adaptive generation and stronger quality/compute tradeoffs.

  • F.2 Noise Level Predictor and Readout Training Dynamics on Minecraft: 10k steps suffice for the Minecraft readout predictor hω to converge, supporting internal noise estimation from EQF activations more readily than direct raw-data prediction.The readout predictor is bootstrapped from EQF activations, whereas gϕ is trained directly from noisy data.
  • F.3 Training Details of Noise Level Predictor on Re10K and Droid Datasets: The 1.3B Re10K and 5B Droid models use frozen denoising backbones and readout predictors trained for an additional 5k and 10k steps, respectively.The predictors contain 460k and 860k parameters, respectively, and share the same architecture.
  • G Details on Sampler Warp Schedules: EQF warps modulate the learned equilibrium velocity field during sampling without changing the training objective, separating denoising-field learning from sampling dynamics.The framework compares transport-style ODE reparameterizations with attractor-style dynamics.
  • G.1 Sampling Schedules and Warp Functions: The noise schedule ρ(s) maps solver time to decreasing noise, while η specifies the positive local denoising-speed multiplier applied along that trajectory.Solver time runs from pure noise at s = 0 to clean data at s = 1, opposite the direction of σ.
  • G.2 Transport and Attractor Warps: Attractor warps satisfy η(σ) →0 as σ →0, turning EQF sampling into fixed-point search and motivating gradient-based optimizers such as NAG.Exact σ = 0 requires infinite continuous time, so trajectories terminate at finite σend > 0 in practice.
  • G.3 Detailed Instantiations of Warp Functions: SD3 remains transport-style because ηSD3(0) ≠ 0, whereas the EqM c-function and linear log-SNR warps induce attractor-style dynamics by vanishing near the data manifold.Experiments find NAG less stable with SD3 than attractor-style warps; the EqM normalization comparison gives Aα = 1.7 versus Aα = 1.85.
  • G.4 Relation Between η Warp Function and Budget-Adaptive Inference.; H Details on Gradient Based Inference with Denoising Video Generative Models: Budget-adaptive inference uses the identity warp but calibrates update scaling from the latest noise estimate, approximating attractor behavior and reducing NAG overshooting.NAG is suitable when the warped EQF field vanishes on the data manifold; otherwise, it produces subpar samples.
  • I Autoregressive Inference Procedures for Standard Denoising Video Generative Models: Standard autoregressive inference uses a predetermined schedule of horizon, stride, and noise levels, while EQF replans closed-loop steps from progress estimates and can stop early.The rolling baseline uses frame groups sharing noise levels, emits clean groups, and appends fully noisy frames to maintain steady-state geometry.

J.1 Clean Buffer, Evaluation Buffer, and Compatibility Score · J.2 Planning on Minecraft

The replanning procedure separates committed predictions from ground-truth feedback, scores whether retained future frames remain compatible, and selectively reuses or regenerates the plan. On Minecraft, Budget-Adaptive EQF provides the strongest quality/compute tradeoff across replanning thresholds.

  • J.1 Clean Buffer, Evaluation Buffer, and Compatibility Score: The clean working buffer contains ground-truth context and a fully denoised future plan, conditioned on the ground-truth action sequence ydata.Only the context is ground truth; future-plan frames are clean predictions rather than noisy inputs.
  • J.1 Clean Buffer, Evaluation Buffer, and Compatibility Score: At each replanning step, the algorithm commits the first S predicted frames to the evaluation buffer, then replaces the working-buffer prefix with corresponding ground-truth observations.The retained future plan remains available for compatibility evaluation and possible reuse.
  • J.1 Clean Buffer, Evaluation Buffer, and Compatibility Score: Compatibility is estimated by temporarily noising only the retained future at M probe levels while keeping the newly observed prefix clean, then measuring denoising error.The compatibility probes do not modify the clean working buffer.
  • J.1 Clean Buffer, Evaluation Buffer, and Compatibility Score: Lower compatibility score S(ℓ) indicates greater consistency between the retained future prediction and updated observations; for EQF, the noise-level condition is ignored.This score uses MSES+1:H averaged over elements of the existing future frames.
  • J.1 Clean Buffer, Evaluation Buffer, and Compatibility Score: If S(ℓ) < τ, the existing plan is reused by shifting retained frames forward, appending S fresh noisy frames, and denoising only those appended frames.If S(ℓ) ≥ τ, the retained horizon is discarded and the complete H-frame future horizon is regenerated.
  • J.2 Planning on Minecraft: On Minecraft, compatibility scores use M = 5 evenly spaced probe noise levels, with τ swept across calibrated observed scores and boundary values included.The authors report that M = 5 provides stable estimates comparable to larger values of M.
  • J.2 Planning on Minecraft: Intermediate replanning thresholds smoothly trade prediction quality against computation, while always replanning maximizes quality and never replanning minimizes regeneration cost.The tradeoff is evaluated using average per-frame function evaluations (NFE) and overall FVD.
  • J.2 Planning on Minecraft: Across the threshold sweep, Budget-Adaptive EQF achieves the strongest quality/compute tradeoff, whereas the Flow Matching baseline exhibits the weakest tradeoff.Retained and newly appended frames can create mixed-context predictions, an out-of-distribution denoising setting for both methods.
Loading 2608.14706v1…