Rectified Diffusion: Straightness Is Not Your Need in Rectified Flow

1 INTRODUCTION

• Diffusion models generate high-quality visual content but are slow because generation requires numerically solving an expensive generative ODE.Rectified flow seeks straighter ODE paths to support high-fidelity generation with fewer steps.Source: §1 · p. 1
• The authors argue rectification works by retraining on matched noise–sample pairs obtained from a pretrained diffusion model, making flow-matching and v-prediction unnecessary.They extend this design to diffusion variants including DDPM, EDM, and Sub-VP.Source: §1 · p. 2
• Rectified Diffusion keeps the pretrained model’s scheduler, prediction type, architecture, and training and inference code unchanged, changing the training pairs instead.The pairs are pre-collected noise and data generated by the pretrained diffusion model.Source: §1 · p. 2
• The essential target is a first-order approximate ODE path, not straightness; for DDPM, that path is inherently curved.Straightness is a special case associated with flow-matching, rather than a suitable general target for Rectified Diffusion.Source: §1 · pp. 2–3
• With only 8% of InstaFlow’s trained images, Rectified Diffusion reports significantly superior one-step performance while leaving the original Stable Diffusion configuration unchanged.The comparison follows InstaFlow’s training setting; Rectified Diffusion does not convert the model to v-prediction flow-matching.Source: §1 · pp. 2–3
• Rectified Diffusion with consistency distillation and its phased variant reports better performance with lower training cost than rectified-flow approaches, including InstaFlow and PeRFlow.Consistency distillation achieves better performance at 3% of the GPU days of InstaFlow’s further-distilled model; the phased method is evaluated on Stable Diffusion v1-5 and XL.Source: §1 · p. 3

2 RECTIFIED DIFFUSION: GENERALIZING THE DESIGN SPACE OF RECTIFIED

• Rectified flow is a subset of Rectified Diffusion, which accommodates general diffusion forms and prediction types.Prediction types can be converted through re-parameterization.Source: §2 · pp. 3–4

2.1 THE NATURE OF RECTIFICATION IS THE RETRAINING WITH PRE-COMPUTED NOISE-SAMPLE PAIR

• Rectification training replaces random noise–data pairing with deterministic pairs collected using a pretrained model, then uses those pairs in general diffusion training.Flow matching is a special case of general diffusion training, distinguished by its diffusion form and prediction type.Source: §2.1 · p. 4

2.2 UNDERSTANDING THE FIRST-ORDER ODE (⋆⋆⋆)

• A first-order ODE path corresponds to the predefined diffusion form with a deterministic noise–data pair, and perfect coupling yields that path in the ideal case without optimization error.Perfect first-order ODEs support identical generation results with arbitrary inference-step counts.Source: §2.2 · p. 5
• First-order ODE paths can be curved for DDPM and Sub-VP, while simple timestep-dependent scaling converts them to straight lines.Flow Matching and EDM have straight first-order paths, though EDM follows a different trajectory and magnitude from Flow Matching.Source: §2.2 · p. 5, Figure 3 · p. 6

2.3 RECTIFIED DIFFUSION (PHASED)

• Phased rectification divides the ODE time axis into segments and linearizes each separately, bringing the new path closer to the original than full linearization.The method uses a pretrained diffusion model to solve between intermediate steps and determine corresponding noise and intermediate samples.Source: §2.3 · pp. 5–6

2.4 RECTIFIED DIFFUSION FACILITATES THE CONSISTENCY DISTILLATION

• After rectification, consistency distillation is reported to train faster and perform better; it further improves low-step performance.Consistency models adjust prediction direction without changing the ODE path, whereas Rectified Diffusion changes the path toward a first-order form.Source: §2.4 · pp. 6–7

3.1 VALIDATION SETUP

• The three comparisons cover continued training for rectification, consistency distillation for low-step performance, and phased ODE segmentation on SD and SDXL.The phased variant enforces the first-order property within each sub-phase.Source: §3.1 · p. 8

3.2 COMPARISON

• Across these three comparative experiments, the authors report significantly superior performance, with lower total training cost than InstaFlow.Rectified Diffusion uses approximately 20 A800 GPU days, compared with InstaFlow's 75.2 A100 GPU days.Source: §3.1 · p. 8, §3.2 · p. 8
• Rectified Diffusion consistently outperforms rectified-flow methods across FID and CLIP scores, scenarios, and steps, while matching advanced distillation and GAN methods comparably.The comparisons cover COCO-2017 validation and the 30k COCO-2014 validation subset.Source: Table 1 · p. 8, Table 2 · p. 9, §3.2 · p. 10
• Human-preference evaluations using four scoring models consistently favor Rectified Diffusion over rectified-flow-based models.The evaluation uses 500 unique prompts from the Pick-a-pic validation set.Source: Figure 7 · p. 10, §3.2 · p. 10
• Consistency distillation uses 4.6 A800 GPU days, approximately 3% of the GPU days reported for InstaFlow distillation.The comparison is 4.6 A800 GPU days versus 110 A100 GPU days.Source: §3.2 · p. 10
• After 20,000 iterations, Rectified Diffusion achieves superior one-step performance to Rectified Flow using about 8% of its processed samples.The paper attributes this efficiency to avoiding conversion of the original DDPM epsilon-prediction model into a v-prediction flow-matching model.Source: §3.2 · p. 10

• Rectified Diffusion extends rectification to general diffusion forms, with first-order behavior—not straightness—as its essential training target.The conclusion also reports that consistency distillation and the phased variant further enhance training efficiency and model performance.Source: §4 · p. 11

• Diffusion research spans theory, design-space expansion, architecture, scaling for text-conditioned synthesis, and sampling acceleration through scheduler refinement, distillation, or GAN initialization.The passage cites work across these directions, including scheduler-level PF-ODE approximation and training-level acceleration.Source: I RELATED WORKS · p. 16
• Sampling acceleration includes both scheduler-level PF-ODE approximation and training-level distillation or diffusion-weight initialization for GAN training.These approaches are presented as distinct training- and scheduler-level directions.Source: I RELATED WORKS · p. 16
• Rectified flow uses flow matching to straighten diffusion ODE paths; later work scales it to one-step text-to-image generation and explores multi-phase paths and optimization.The passage also notes analysis suggesting one-time rectification can generally achieve pure straightness.Source: I RELATED WORKS · p. 16

II LIMITATIONS

• At low step counts, rectification methods lag state-of-the-art distillation and GAN methods, and additional distillation is needed to improve performance.Source: II LIMITATIONS · p. 16

III PROOF FOR FIRST-ORDER ODE

• The proof establishes that the two ODE transition forms are equivalent for arbitrary times if and only if epsilon prediction is constant along the trajectory.Source: III PROOF FOR FIRST-ORDER ODE · p. 17