PaperlayerView exact revision on arXiv

Source-linked AI summary

Dino Forcing Flow Models: Do not denoise what you can predict

Arijit Ghosh, Lucas Degeorge, Paul Couairon, Alexei A Efros, Vicky Kalogeiton, David Picard

arXiv:2610.11751v1cs.CV

TL;DR

Co-denoising pretrained representations can speed flow matching, but adds a second denoising trajectory and representation-specific schedules. DINO Forcing predicts the representation and reuses it as conditioning, improving convergence and generation quality across latent and pixel settings.

  • Problem

    Flow-matching methods that co-denoise pretrained representations gain training speed, but add a second denoising trajectory and require representation-specific schedules.

    Source: §1 · p. 2

  • Method

    DINO Forcing predicts the pretrained representation from the image state and reuses that prediction as self-conditioning, avoiding a second ODE and representation-specific schedule.

    Source: §1 · p. 2, §6 · p. 10

  • Results

    Across latent and pixel space, DINO Forcing improves convergence and generation quality; in pixel space, FID improves over Latent Forcing from 7.20 to 5.53 at comparable model size and training budget.

    Source: §1 · p. 2, §6 · p. 10

  • Takeaways & Limitations

    When an auxiliary representation can be inferred from the evolving sample, prediction may be enough instead of generating it alongside the primary modality.

    Source: §6 · p. 10

    The method assumes its prediction branch produces semantically meaningful features, rather than merely codes useful for velocity prediction.

    Source: §5.3 · p. 9

Abstract

from arXiv · show

Co-denoising pretrained representations such as DINO can substantially improve the training speed and quality of flow matching models, but it introduces a second denoising trajectory and requires carefully designed schedules. We propose a simpler alternative: predict the pretrained representation directly, then condition the model on its own prediction. This removes the need for a second ODE and any representation-specific denoising schedules, while retaining the benefits of representation guidance. Our approach converges substantially faster and achieves better generation quality as measured by FID score. On ImageNet, it outperforms the state of the art in latent space at 2x fewer epochs than prior methods; in pixel space, it improves FID over comparable prior methods by more than 20%. These results support a simple principle: do not denoise what you can predict. Our code is openly available at https://github.com/arijit-hub/dino_forcing.

1 INTRODUCTION

DINO Forcing replaces co-denoising with direct prediction of pretrained representations for self-conditioning. It reports faster convergence and improved generation quality in latent and pixel space.

  • Jointly denoising images and DINO representations can speed training, but hides representation information in noise and adds independent noise despite deterministic image–representation coupling.Source: §1 · p. 2
  • DINO Forcing predicts the pretrained representation and uses that prediction as self-conditioning, keeping generation in image space rather than creating a second representation trajectory.The contribution describes the method as a schedule-free self-conditioning mechanism.Source: §1 · pp. 2–3
  • FID 2.47 after 100 epochs in latent space; in pixel space, DF improves over Latent Forcing from 7.20 to 5.53 FID at comparable model size and training budget.The latent result surpasses prior methods trained substantially longer.Source: §1 · p. 2

2 RELATED WORK

Prior work uses pretrained representations to accelerate image generation or preserve guidance through sampling, with trade-offs in sampling guidance, decoder design, or noise scheduling. DINO Forcing instead self-conditions on predictions aligned to an external feature space.

  • Feature alignment can accelerate training (REPA: Yu et al., 2025), but leaves sampling unguided; pre-aligned latent methods retain representations during sampling but require decoder and architectural changes (RAE: Zheng et al., 2026).Source: §2 · p. 3
  • Jointly denoising modalities preserves pretrained representations through inference and accelerates convergence, but requires a hand-crafted second noise schedule that may need modality-specific retuning.Source: §2 · p. 3
  • DINO Forcing self-conditions on its own predictions aligned to external DINO features, keeping representation guidance throughout generation without a handcrafted schedule.The passage distinguishes this approach from prior self-conditioning by its alignment with an external pretrained feature space.Source: §2 · p. 3

3 DINO-FORCING

DINO Forcing predicts the pretrained representation it needs and uses that prediction to condition image generation, rather than denoising a separate representation. Its two-pass training design carries into sampling, where predictions are rolled forward as conditioning.

3 DINO-FORCING

  • DINO Forcing predicts a pretrained representation and conditions image generation on that prediction instead of denoising a separate representation.At sampling time, the predictions are rolled forward across denoising steps as a model-generated guide.Source: §3 · p. 3

3.1 SELF-CONDITIONING ON PREDICTED REPRESENTATIONS

  • Training uses a no-gradient look-ahead pass, then a conditioned pass that predicts image velocity and aligns a refined representation with frozen DINO features.The two objectives are flow matching and representation alignment; additional conditions such as class or text are included in experiments.Source: §3.1 · pp. 3–4
  • Image and representation tokens are concatenated along the channel dimension, preserving the image-only model’s token count and compatibility with existing flow models.Source: §3.1 · p. 4

3.2 SAMPLING

  • Sampling starts from image noise and zero representation conditioning; each predicted representation conditions the next solver step as the image evolves.The conditioning signal grows progressively more coherent without modifying the standard denoising schedule.Source: §3.2 · pp. 4–5, Figure 4 · p. 5

4 PREDICTION VS CO-DENOISING

Co-denoising can accelerate convergence, but adds a representation ODE trajectory and handcrafted schedules. DINO Forcing instead predicts the deterministic representation needed to guide image generation, removing that second trajectory.

  • Co-denoising accelerates convergence over image-only baselines but requires an additional ODE trajectory for pretrained representations and handcrafted schedules.Source: §4 · p. 5
  • With independent image and representation noises and two time variables, co-denoising must fit across the square of noise-level pairs.Source: §4 · p. 5
  • Because the representation is deterministic given a clean image, it adds no sampling degrees of freedom; from noisy input, its best estimate is the conditional expectation given that input.A forward pass can predict it without a separate noise schedule.Source: §4 · pp. 5–6
  • Omitting the representation trajectory collapses the search time domain from [0, 1]^2 to [0, 1] and removes the corresponding handcrafted schedules.Source: §4 · p. 6

5 EXPERIMENTS

The experiments evaluate DINO Forcing across latent and pixel ImageNet generation, feature inversion, generation without class conditioning, and design ablations. Across these settings, the reported results show faster convergence and improved generation quality.

5 EXPERIMENTS

  • The ImageNet evaluation reports FID, Inception Score, precision, and recall from 50K samples; the latent and pixel implementations follow SiT and JiT, respectively.Pixel-space FID uses JiT reference statistics; IS, precision, and recall use the ADM suite.Source: §5 · p. 6, §5.1 · p. 6, §5.2 · p. 7

5.1 LATENT SPACE SETTING

  • 2.47 after 100 epochs versus REGLUE’s 2.50 after 200 epochs; DF reaches 2.61 at 80 epochs, on par with or better than longer-trained methods.The comparison is at a similar parameter count in latent space.Source: §5.1 · p. 6, Table 2 · p. 7
  • At 512 × 512 without guidance, DF reaches 3.11 FID versus 11.05 for REPA and 20.45 for SiT-XL/2; recall falls from 0.73 to 0.63 while precision stays 0.65.Inception Score rises from 116.2 to 274.5.Source: §5.1 · p. 7, Table 4 · p. 8

5.2 PIXEL SPACE SETTING

  • At 200 epochs, pixel-space DF reaches 5.53 FID with an L-sized model, versus 7.20 for Latent Forcing at comparable size and budget.Across model sizes, XL reaches an unguided FID of 5.45.Source: §5.2 · p. 8, Table 3 · p. 8
  • With guidance, pixel DF reaches 2.26 at L size, beating Latent Forcing (2.48) and PixelDiT-2 (2.35), and 2.13 at XL size versus JiT-XL’s 2.89.PixelGen-XL remains ahead at 1.83 after 160 epochs.Source: §5.2 · p. 8, Table 3 · p. 8

5.3 DINO INVERSION

  • DINOv2 inversion reconstructs images from features, with SiT reaching 16.86 dB / 0.48 SSIM and JiT 14.33 dB / 0.44 SSIM.The authors interpret these results as showing that the DINOv2 condition provides useful local information for reconstruction.Source: Table 5 · p. 8, Figure 6 · p. 8, §5.3 · p. 9

5.4 GENERATION WITHOUT CLASS CONDITION

  • With class conditioning nulled, DF reaches 9.14 FID in latent space versus 49.51 for SiT-L/2, and 15.19 in pixel space versus 53.97 for JiT-L/16 and 20.44 for Latent Forcing.This setting tests the predicted representation as the model’s only conditioning signal.Source: §5.4 · p. 9, Table 6 · p. 9

5.5 ABLATION

  • On SiT-L/2 at 80 epochs, DINO supervision reaches 3.04 FID, six times below baseline; the unsupervised branch reaches 12.96 from 18.80, while image supervision adds little.MAE features also improve substantially, by more than 10 FID points over baseline.Source: §5.5 · p. 9, Table 7 · p. 9

6 CONCLUSION

The conclusion presents DINO-Forcing as an alternative to co-denoising representations: it predicts representations from the current image state and reuses them as conditioning. The authors report improved convergence and generation quality in latent and pixel space, including against methods trained longer.

6 CONCLUSION

  • DINO-Forcing predicts the pretrained representation from the current image state and reuses it as conditioning, removing the second ODE and representation-specific denoising schedule.The conclusion reports improved convergence and generation quality in both latent and pixel space, including against methods trained longer.Source: §6 · p. 10

A IMPLEMENTATION DETAILS

The implementation-details table documents DINO-Forcing’s training and sampling configurations for ImageNet in latent and pixel space.

A IMPLEMENTATION DETAILS

  • Table 8 collects the ImageNet training and sampling configurations for DINO-Forcing in latent and pixel space.Source: Table 8 · p. 14

B EXPLANATION BEHIND OUR FUSING IMAGE AND FROZEN REPRESENTATIONS MECHANISM

DINO-Forcing fuses image and frozen-representation tokens by channel concatenation, keeping token count unchanged and avoiding spatial resampling. Other fusion strategies are left for future exploration.

B EXPLANATION BEHIND OUR FUSING IMAGE AND FROZEN REPRESENTATIONS MECHANISM

  • Image and representation patch tokens are spatially aligned and concatenated along the channel dimension, then projected to the transformer’s hidden dimension.The scheme keeps token count identical to an image-only model and requires no spatial resampling.Source: B EXPLANATION BEHIND OUR FUSING IMAGE… · p. 14
  • The authors use channel concatenation for comparison and leave exploration of other fusion strategies for future work.They describe their strategy as simpler than concurrent work because it requires no upsampling.Source: B EXPLANATION BEHIND OUR FUSING IMAGE… · pp. 14–15

C ABLATION

The ablations select the deepest representation-prediction branch and a low probability of training on Pass 1 alone.

C ABLATION

  • The ablations use SiT-L in the unguided setting, making their numbers directly comparable.Source: C ABLATION · p. 15
  • At 250K steps, Layer 23 reaches FID 3.32 versus roughly 5.84 for Layer 7, so the main experiments use the deepest block.The authors link the deeper branch's advantage to the abstract, high-level features that align with the DINOv2 target.Source: C ABLATION · p. 15
  • At 250K steps, p = 0.1 yields FID 3.32, compared with 3.53 for p = 0.2 and 6.81 for p = 0.5.Pass 1 is optimized alone with probability p; its look-ahead must orient sampling before real conditioning exists.Source: C ABLATION · p. 15

D EXTENDED RELATED WORK

Prior representation-alignment approaches either lack explicit guidance during sampling or add decoder and schedule complexity. The paper presents self-predicted pretrained features as a way to provide guidance throughout generation without a separate representation-denoising trajectory.

D EXTENDED RELATED WORK

  • Feature-alignment methods provide pretrained information only during training, while pre-aligned latent approaches require a new decoder and non-trivial architectural modifications.Source: D EXTENDED RELATED WORK · pp. 15–16
  • Co-denoising can accelerate convergence in latent and pixel spaces, but its representation noise schedule requires careful handcrafting and may need retuning across modalities.SFD and LatentForcing report best convergence when the representation branch is denoised to a low noise level before image denoising begins.Source: D EXTENDED RELATED WORK · p. 16
  • DINO Forcing conditions image generation on the model's own predicted representation, aligned with an external pretrained feature space, rather than denoising that representation separately.Source: D EXTENDED RELATED WORK · p. 16
  • This design makes pretrained information available throughout image generation without a separate decoder, architectural modifications, or handcrafted denoising schedules.Source: D EXTENDED RELATED WORK · p. 16

E ADDITIONAL QUALITATIVE RESULTS

The appendix captions identify uncurated ImageNet samples across model variants, resolutions, and guidance settings, along with visualizations of predicted-feature evolution during sampling.

E ADDITIONAL QUALITATIVE RESULTS

  • Figures 8 and 9 show uncurated DF-SiT-XL/2 samples on ImageNet at 256×256, without guidance and with cfg=4.0, respectively.Source: Figure 8 · p. 17, Figure 9 · p. 18
  • Figures 10 and 11 show uncurated DF-SiT-XL/2 samples on ImageNet at 512×512, without guidance and with cfg=4.0, respectively.Source: Figure 10 · p. 19, Figure 11 · p. 20
  • Figures 12 and 13 show uncurated DF-JiT-XL/16 samples on ImageNet at 256×256, without guidance and with cfg=4.0, respectively.Source: Figure 12 · p. 21, Figure 13 · p. 22
  • Figure 14 tracks images and PCA projections of predicted DINO features from noise to data for DF-SiT-XL/2 and DF-JiT sampling configurations.The PCA projection is fit once per sample on the final features and reused for all timesteps.Source: Figure 14 · p. 23
Loading 2610.11751v1…