PaperlayerView exact revision on arXiv

Source-linked AI summary

ReAL: Accelerating Flow Matching through Segment Advancement with Shared Lookahead

Xuanhua Yin, Chuanzhi Xu, Haoxian Zhou, Shunqi Mao, Weidong Cai

arXiv:2609.33202v1cs.CV

TL;DR

Repeated flow-matching evaluations make sampling costly, and local velocity agreement alone does not determine how far to advance without costly candidate-endpoint checks. ReAL uses one shared lookahead to select and correct a span while reusing its velocity. Across evaluated image, video, and editing systems, it reduces latency at reported quality levels, including 4.91× measured speedup on FLUX.1-dev with 97.0% of dense mean ImageReward.

  • Problem

    Fixed schedules cannot adapt to varying useful spans, while checking each candidate endpoint adds model evaluations; one local velocity mismatch must support decisions about longer advances.

    Source: §1 · p. 2

  • Method

    ReAL uses one shared lookahead to select a passing span, correct its endpoint, and provide the next starting velocity without separate candidate-endpoint queries.

    Source: §1 · p. 2

  • Results

    Across evaluated image, video, and editing systems, ReAL reduces latency at reported quality levels; FLUX.1-dev reaches 4.91× measured speedup with 97.0% of dense mean ImageReward.

    Source: §5 · p. 9

  • Takeaways & Limitations

    Local velocity disagreement can guide where computation is useful, but it is not a calibrated estimate of final error.

    Source: §5 · p. 9

    New deployments require model-specific calibration because later field changes and long segments can weaken local evidence and velocity carry.

    Source: §5 · p. 9

Abstract

from arXiv · show

Flow-matching models generate high-quality images and videos, but repeated neural network evaluations make sampling expensive. Skipping evaluations reduces this cost by extending an available velocity estimate over a longer span. However, local velocity agreement alone does not determine a suitable span, and checking each candidate endpoint adds costly model calls. We introduce ReAL, a training-free sampler that selects how far to advance using one shared lookahead. Our key insight is that the discrepancy between uncorrected and lookahead-corrected endpoint proposals can be computed directly from the observed velocity mismatch and the candidate span beyond the lookahead. This relation provides a span-dependent selection criterion without additional endpoint evaluations. The same lookahead selects the longest passing candidate span, corrects the accepted update, and supplies its velocity as the next starting estimate. After initialization, each regular iteration requires only one fresh evaluation. ReAL uses the pretrained velocity output and original noise schedule, with no additional training or access to internal features. Experiments cover four image-generation backbones, video generation, and image editing. ReAL achieves 4.91x measured speedup on FLUX.1-dev while retaining 97.0% of dense mean ImageReward. On HunyuanVideo, it achieves a 5.49x speedup while maintaining a VBench score close to that of dense sampling.

1 INTRODUCTION

Repeated velocity evaluations make flow-matching sampling costly, while fixed spans cannot adapt to variation and endpoint verification adds model calls. ReAL uses one shared lookahead to select and correct spans, and reports speedups on image and video generation.

1 INTRODUCTION

  • Fixed schedules cannot respond to span variation, while checking each candidate endpoint adds back some of the evaluation cost saved by skipping updates.A useful controller must judge different candidate lengths from shared local evidence.Source: §1 · p. 2
  • Skipping evaluations saves costly model calls but asks an existing velocity estimate to govern more of the trajectory before the field is observed again.The useful span can change across sampling and inputs.Source: §1 · p. 2
  • ReAL uses one nearby velocity observation to select the span, correct the endpoint, and supply the next starting estimate without separate candidate-endpoint evaluations.It uses the pretrained velocity output and original noise schedule, without task-specific training or intermediate features.Source: §1 · p. 2
  • 4.91× measured speedup on FLUX.1-dev retains 97.0% of dense mean ImageReward, while HunyuanVideo reaches 5.49× speedup with a 0.54-point VBench drop.The evaluation also covers four image-generation backbones and image editing.Source: §1 · p. 2

2 RELATED WORK

Related work includes fixed-model sampling accelerators, feature reuse and adaptive computation, and lookahead or verification methods across visual tasks. ReAL preserves the pretrained field and schedule while using complete model outputs and one next-index lookahead.

2.1 FLOW MODELS AND FAST SAMPLING

  • Fixed-model accelerators use implicit updates, higher-order integration, predictor-corrector formulas, and optimized schedules, while other methods train few-step generators.ReAL preserves the field and schedule while adapting evaluation placement.Source: §2.1 · p. 2

2.2 REUSE AND ADAPTIVE COMPUTATION

  • Reuse and adaptive-computation methods cache features, track change, forecast activations, or select computation using signals such as spectral change and latent stability.ReAL instead uses complete model outputs, no feature cache or learned predictor, and carries the fresh lookahead velocity across segment boundaries.Source: §2.2 · p. 3

2.3 LOOKAHEAD ACROSS VISUAL TASKS

  • Parallel samplers update multiple states, while diffusion verification methods check transitions or forecast trajectories; ReAL queries only the next-index lookahead.The shared output serves span selection, endpoint correction, and velocity reuse, and the controller is evaluated on image, video, and editing tasks.Source: §2.3 · p. 3

3 METHODOLOGY

ReAL uses one lookahead to select and correct a segment, then carries its velocity forward. A span-scaled discrepancy supports candidate backoff from shared evidence, with one fresh query per regular iteration after initialization.

3 METHODOLOGY

  • One lookahead supports segment selection, endpoint correction, and velocity reuse while retaining the pretrained velocity field and noise schedule.The dense reference applies one Euler update; ReAL may skip intermediate indices.Source: §3 · p. 3, §3.1 · p. 3, Figure 2 · p. 4

3.1 SOLVER STATE AND ONE-STEP LOOKAHEAD

  • At each index, ReAL uses a starting velocity to form a one-step lookahead state and makes one fresh query there; all candidates share both velocities.The starting velocity is normally carried from the previous iteration, or evaluated at the current state when no carry is available.Source: §3.1 · p. 3

3.2 A CORRECTED SEGMENT AND ITS VERIFICATION SIGNALS

  • The corrected endpoint uses the starting velocity for the first interval and the lookahead velocity for the remaining span, making the raw-to-corrected endpoint difference depend on observed velocity mismatch.The segment score scales this discrepancy by residual span relative to the first-interval reference, so the same nonzero mismatch scores higher for longer residual spans; actual schedule differences accommodate nonuniform schedules.Source: §3.2 · pp. 3–4

3.3 HIERARCHICAL BACKOFF, FALLBACK, AND CARRY

  • ReAL tests a descending set of candidate lengths up to the remaining-interval cap, accepting the longest passing candidate or falling back to the one-step lookahead state.The accepted endpoint uses the shared lookahead velocity; after either outcome, that velocity is carried forward, and each regular iteration after initialization uses one fresh query.Source: §3.3 · p. 4

4 EXPERIMENTS AND RESULTS

Across image, video, and editing tasks, ReAL is evaluated against dense sampling and acceleration baselines, with ablations and transfer studies examining its components and behavior across settings. The results include high measured speedups on FLUX.1-dev and HunyuanVideo, alongside quality comparisons and analyses of evaluation allocation.

4.2 TEXT-TO-IMAGE GENERATION

  • At the fastest of three FLUX tiers, ReAL achieves 4.91× speedup, IR 0.9611, and LPIPS 0.2551, versus direct step reduction at 4.38×, 0.8917, and 0.4683.Figure 3 says ReAL retains the requested three-object composition, cyan-haired character, and literal word “SEAL” in these matched examples.Source: §4.2 · p. 6, Figure 3 · p. 7

4.3 TEXT-TO-VIDEO GENERATION

  • At 5.49× speedup, ReAL leads HunyuanVideo’s high-acceleration settings with 82.73 VBench, 0.54 points below dense sampling; at medium acceleration, it scores 83.09 versus Spectrum’s 83.14 at 3.21×.At high acceleration, ReAL records 0.7696 SSIM and 0.2681 LPIPS, versus FlowCast’s 0.7266 and 0.3028.Source: §4.3 · p. 6, Table 2 · p. 6

4.4 IMAGE EDITING

  • In editing Tier 3, ReAL is fastest at 5.22×, with a score of 4.11 and 1.311-second latency, versus dense sampling’s 6.842 seconds.For Tiers 1 and 2, its overall-score point estimates are 4.24 at 3.86× and 4.21 at 4.58×, versus 4.20 for dense sampling; seed-level intervals are unavailable.Source: §4.4 · p. 7, Figure 5 · p. 8

4.5 ABLATION OF SHARED-LOOKAHEAD COMPONENTS

  • Under fallback-only carry, span verification raises IR from −0.0168 to 0.9621, while adding the local gate reaches 0.9864 at 4.20× speedup.The span test scales velocity mismatch by the residual schedule span relative to the first interval; dense verification requires 118.4 NFE and reaches 0.42× speedup.Source: §4.5 · p. 8, Table 3 · p. 8
  • Endpoint correction raises IR from 0.6492 to 0.9611 at the fastest tested setting as NFE changes from 8.76 to 9.51; carrying lookahead reduces NFE by 14.3%.At qseg = 1.65, carry reduces calls per iteration from 1.728 to 1.062, while IR changes from 0.8632 to 0.8719; the controls show distinct benefits from selection, correction, and carry.Source: §4.5 · p. 8

4.6 TRANSFER ACROSS BACKBONES AND SAMPLING SETTINGS

  • On SD3-Medium, ReAL reaches 0.594 IR at 12.43 NFE versus AdaptiveDiffusion’s 0.591 at 13.59; on SD3.5-Large, both round to 0.794 IR.The dense-head rule is stronger on Lumina at intermediate budgets, and these transfer sweeps choose settings for each workload.Source: §4.6 · pp. 8–9, Figure 6 · p. 9
  • With qseg = 2.6, increasing K from 12 to 24 keeps ReAL between 7.65 and 7.83 NFE and IR between 0.853 and 0.904, while the fixed rule’s IR falls to 0.283.At K = 24, ReAL’s paired IR advantage is 0.621 (95% bootstrap CI [0.481, 0.765]) using 1.83 more evaluations; on the 15-step schedule, it reaches 0.1979 IR at 6.245 NFE versus rule 3:6’s 0.0470 at 6 NFE.Source: §4.6 · p. 9
  • On SD3-Medium, early perturbations have larger final effects, while mean terminal drift divided by summed local discrepancy is 1.21 for ReAL versus 3.40 for uniform stride two.ReAL uses shorter early advances; on PartiPrompts, it uses 11.52 NFE for 98 prompts outside Basic and Simple Detail versus 10.42 for the other 52, without category labels as inputs.Source: §4.6 · p. 9

5 CONCLUSION

ReAL uses one shared lookahead to select and correct each segment while carrying its velocity forward, and reduces latency at reported quality levels across evaluated image, video, and editing systems. Its controller specifies exact evaluation accounting and a span-scaled proposal-correction score for choosing advances.

5 CONCLUSION

  • One shared lookahead selects the span, corrects the accepted endpoint, and supplies the next starting velocity, without changing the pretrained model or noise schedule.The controller avoids separate candidate-endpoint queries.Source: §5 · p. 9
  • 4.91× speedup on FLUX.1-dev retains 97.0% of dense mean ImageReward; HunyuanVideo reaches 5.49× speedup with a 0.54-point VBench-score drop versus dense sampling.Across evaluated image, video, and editing systems, ReAL reduces latency at the reported quality levels.Source: §5 · p. 9

A.1 EXACT ALWAYS-CARRY ALGORITHM

  • The exact guided-evaluation count is NReAL = M + F, where M counts regular carried iterations and F counts iterations starting without a carry.For T ≥ 3, initialization contributes one to F; tail intervals can require additional fresh evaluations, and failed verification attempts are included.Source: §A.1 · p. 15

A.2 PROPOSAL AND CORRECTION IDENTITY WITH SPAN SCALING

  • The segment score normalizes the RMS difference between raw and corrected endpoint proposals using the first schedule interval as its reference scale, preserving candidate-span dependence.A denominator proportional to the residual span would cancel that length dependence.Source: §A.2 · p. 15

A.3 SEPARATING CARRY ERROR FROM SEGMENT FREEZING ERROR

The error analysis separates accepted-segment error into carry mismatch and field variation over the remaining span. The verification residual also mixes carry error with local field variation, whose components can reinforce or cancel.

A.3 SEPARATING CARRY ERROR FROM SEGMENT FREEZING ERROR

  • Fallback aligns the next velocity with the accepted state but retains state error already accumulated; refreshing after every accepted segment removes the next carry mismatch at the cost of another evaluation.A separate carry-policy experiment measures this trade-off.Source: §A.3 · p. 17
  • Accepted-segment error decomposes into immediate carry error, its effect on the lookahead evaluation, and residual-span freezing error along the restarted dense trajectory.The decomposition holds for any stored velocity and carry-error magnitude.Source: §A.3 · p. 17
  • Under continuous differentiability along the lookahead displacement, the mean Jacobian relates the lookahead-velocity difference to the current carry error.With Jacobian norm bounded by L, the carry contribution is at most hi(1 + si,kL)∥δi∥2.Source: §A.3 · p. 17
  • The next verification residual combines stored-velocity error, fresh dense-step velocity variation, and displacement-induced lookahead error, which may reinforce or cancel.This motivates evaluating output fidelity alongside controller diagnostics.Source: §A.3 · p. 17

B VIDEO-GENERATION AND EDITING PROTOCOLS

The video and editing protocols report matched-run efficiency and quality measures, while the appendix supplies complete editing results, VBench dimension tables, and reproducibility materials. The reported high-acceleration VBench dynamic-degree score is below dense sampling but above direct step reduction.

B VIDEO-GENERATION AND EDITING PROTOCOLS

  • Video and editing runs use one NVIDIA H100 with external batch size one; speedups compare dense runs from the same task, hardware, precision, and timing session.Tables 2 and 5 report median event latency, with three untimed warm-up generations per method and operating point.Source: B VIDEO-GENERATION AND EDITING PROTOCOLS · p. 17

B.1 COMPLETE EDITING RESULTS

  • The editing results summary reports point estimates rather than seed-level intervals, with each acceleration tier’s configuration fixed across examples and seeds.Table 5 covers three quality scores, latency, speedup, NFE, and peak memory; its caption identifies 606 examples and five seeds.Source: §B.1 · p. 18, Table 5 · p. 20

B.2 VBENCH DIMENSION SCORES

  • At high acceleration, ReAL’s VBench dynamic-degree score is 66.94, versus 71.11 for dense sampling and 43.33 for direct step reduction.A high consistency score alone does not imply better motion; the reported total uses per-dimension normalization and combines quality and semantic scores on a different scale from the dimension tables.Source: §B.2 · p. 18, Table 8 · p. 21, Table 9 · p. 21

C ADDITIONAL EXPERIMENTAL DETAIL

  • The accompanying code includes a model-independent Algorithm 1 implementation, a CPU example, a FLUX.1-dev entry point, and tests of controller behavior and NFE counting.Full task reruns require external model and evaluator weights, benchmark inputs, environments, seeds, and configurations.Source: C ADDITIONAL EXPERIMENTAL DETAIL · p. 22

C.1 EVALUATION PROTOCOLS AND METRICS

The evaluation appendix specifies matched-prompt protocols, quality and compute metrics, and statistical comparisons, then describes operating-point studies across caps, schedules, and prompts.

C.1 EVALUATION PROTOCOLS AND METRICS

  • Evaluation uses matched prompts and seeds, task-quality metrics, and FLOPs, NFE, and end-to-end latency to characterize quality and compute cost.Runtime comparisons use each method’s own dense reference and timing session.Source: §C.1 · p. 22

C.2 STATISTICAL COMPARISONS

  • Paired comparisons preserve prompt-seed pairs and use 20,000 bootstrap samples; intervals reflect prompt variation under fixed seed assignments and are unadjusted for exploratory comparisons.A confidence interval that includes zero does not establish the sign of the population mean difference.Source: §C.2 · p. 23

C.3 MEASURED OPERATING POINTS AND PAIRED DIFFERENCES

  • Cap, schedule, and prompt studies report measured operating points, including prompt-group allocation differences and threshold sweeps.Figure 8 shows cap and schedule sweeps and reports greater allocation to 98 PartiPrompts outside the Basic or Simple Detail categories than to the other 52.Source: Figure 8 · p. 26

D THE EMPIRICAL FIXED-STRIDE ERROR RELATIONSHIP

The appendix fits an empirical relationship between fixed-stride error and LPIPS across tested configurations, then limits its scope to that fixed-stride family.

D THE EMPIRICAL FIXED-STRIDE ERROR RELATIONSHIP

  • The 22-configuration fit gives exp(α) = 1.1613 and β = 0.5063, with an RMS log residual of 0.0570.Four-fold leave-one-backbone-out evaluation refits both coefficients on the other three backbones.Source: D THE EMPIRICAL FIXED-STRIDE ERROR… · p. 27, Figure 9 · p. 27
  • Held-out backbone predictions have a largest individual relative error of about 20.0% for Lumina.The accompanying figure reports geometric-mean measured-to-predicted LPIPS ratios within each held-out backbone.Source: Figure 9 · p. 27
  • The fitted relation describes LPIPS within the tested fixed-stride family, while front-loaded and adaptive segments change error amount and placement.Calibration trajectories and configuration-level image comparisons have different statistical roles.Source: D THE EMPIRICAL FIXED-STRIDE ERROR… · p. 27

E ANALYSIS OF ADAPTIVE SEGMENT ALLOCATION

The analysis distinguishes local segment discrepancy from accumulated sampling drift and finds that advance placement matters, with ReAL selecting shorter early segments.

E ANALYSIS OF ADAPTIVE SEGMENT ALLOCATION

  • Early perturbations produce the largest final perceptual deviations across FLUX, SD3-Medium, and Lumina-Next-SFT, supporting shorter advances where discrepancies have greater later effects.The perturbation has norm 0.02∥Xi∥2 and points along Vi − Vi+1.Source: E ANALYSIS OF ADAPTIVE SEGMENT… · p. 28
  • The diagnostics compare dense-trajectory discrepancy for fixed strides with corrected-schedule discrepancy using carried or refreshed velocities.The free-running drift-to-local-discrepancy ratio compares replay endpoint drift with total local discrepancy on the dense path.Source: E ANALYSIS OF ADAPTIVE SEGMENT… · p. 28
  • On SD3-Medium, uniform stride two has a drift-to-discrepancy ratio of 3.40, compared with 1.04 for a dense prefix and 1.21 for ReAL.ReAL’s controller cost is 14.76 NFE; its diagnostic replay costs 15.18 NFE because some trajectories evaluate an unused terminal velocity.Source: E ANALYSIS OF ADAPTIVE SEGMENT… · p. 28

F CURRENT IMAGE-GENERATION COMPARISONS

The image-generation comparison appendix describes fixed FLUX.1-dev example settings and identifies additional accelerator comparisons, but the supplied passages report no comparative outcomes.

F CURRENT IMAGE-GENERATION COMPARISONS

  • FLUX.1-dev examples use a 50-step schedule, guidance 3.5, and fixed ReAL settings across prompts with shared initial noise and text embeddings.ReAL uses K = 12, qloc = 0.70, and qseg = 1.65, with carry after acceptance and fallback.Source: F CURRENT IMAGE-GENERATION COMPARISONS · p. 29
  • TaylorSeer uses first-order prediction and SpeCa uses its bf16 runtime; other main comparisons use diffusers fp16, while FlowCast uses serial draft verification.FlowCast is run within a 32 GB memory budget.Source: F CURRENT IMAGE-GENERATION COMPARISONS · p. 29
  • Additional comparisons use the same prompts and noise for ReAL and baselines including SpeCa, TaylorSeer, DPCache, Spectrum, JiT, and FlowCast.Runtime environments are specified above in the paper.Source: Figure 11 · p. 29

G CURRENT VIDEO-GENERATION COMPARISONS

The section presents matched-condition qualitative comparisons for HunyuanVideo and three additional image-generation backbones, with figures identifying the methods and evaluation settings shown.

G CURRENT VIDEO-GENERATION COMPARISONS

  • HunyuanVideo comparisons include adapted VDE, ZEUS, AdaptiveDiffusion, and JiT, plus FlowCast with serial draft verification.Each method uses one configuration across the three prompts.Source: G CURRENT VIDEO-GENERATION COMPARISONS · p. 30
  • Figure 12 compares ReAL with dense sampling, direct step reduction, twelve acceleration baselines, and references across three HunyuanVideo prompts.The matched-seed reproduction uses 65 frames at 544×960 and 24 fps; dense sampling uses 50 steps and direct reduction 11.Source: Figure 12 · p. 30, G CURRENT VIDEO-GENERATION COMPARISONS · p. 30

H CROSS-BACKBONE IMAGE-GENERATION EXAMPLES

Related papers

Loading 2609.33202v1…