PaperlayerView exact revision on arXiv

Source-linked AI summary

FlashRender: Few-Step Generative Rendering via Camera-Controlled Video MeanFlow

Byeongjun Park, Byung-Hoon Kim, Hyungjin Chung

arXiv:2609.03563v1cs.CV

TL;DR

Few-step generative rendering must address high sampling cost and sampling-step-dependent camera control that reflects discretization error. FlashRender uses RETA, MeanFlow, and on-policy flow map distillation to stabilize geometry and shorten denoising, matching multi-step quality and geometric consistency at 25× lower sampling cost while improving camera controllability. Its performance nevertheless degrades at extremely low sampling budgets, where MeanFlow becomes less effective.

  • Problem

    Existing generative rendering methods require many sampling steps, while coarse sampling causes step-dependent camera control and discretization errors that complicate few-step distillation.

  • Method

    FlashRender aligns source-video representations with frozen target-view geometry using RETA, then applies MeanFlow fine-tuning and on-policy flow map distillation.

  • Results

    FlashRender matches multi-step baselines in video quality and geometric consistency at 25× lower sampling cost while achieving superior camera controllability and robust out-of-distribution trajectory generalization.

  • Takeaways & Limitations

    RETA, MeanFlow, and on-policy flow map distillation play complementary roles in few-step generative rendering.

  • Takeaways & Limitations

    Performance degrades substantially as sampling is reduced further because MeanFlow becomes less effective under extremely low sampling budgets.

Abstract

from arXiv · show

We present FlashRender, a few-step generative rendering framework that retakes a source video along a target camera trajectory in seconds. We identify sampling-step-dependent camera control as a prominent manifestation of discretization error in existing multi-step generative rendering models and show that resolving this inconsistency substantially lowers denoising trajectory curvature, facilitating subsequent step distillation. To this end, we introduce Representation Transformation and Alignment (RETA), which aligns hidden source-video representations with target-video features from a frozen visual geometry model. This directly encodes the geometric transformation within the source-video stream, enabling sampling-step-consistent camera control. We then fine-tune the model with the MeanFlow objective on the lower-curvature denoising trajectory induced by RETA, allowing the model to more effectively address discretization error. Finally, we apply on-policy flow map distillation to correct self-rollout errors under fixed few-step sampling. Extensive experiments show that RETA, MeanFlow, and on-policy flow map distillation play complementary roles in few-step generative rendering. Together, they enable our approach to match multi-step baselines in video quality and geometric consistency at 25x lower sampling cost while achieving superior camera controllability, even under out-of-distribution target camera trajectories.

1 INTRODUCTION

FlashRender targets the high cost and discretization errors of few-step generative rendering, especially camera control that changes with sampling steps. It combines RETA, MeanFlow, and on-policy distillation to produce fast retakes with consistent geometry and controllability.

  • Existing generative rendering methods require many sampling steps, creating substantial inference costs that limit practical usability.
  • Sampling-step-dependent camera control changes realized camera motion, scene scale, and dynamic-object spatial grounding even for identical target trajectories.This inconsistency also changes source-to-target token correspondences across denoising timesteps, complicating few-step distillation.
  • RETA aligns intermediate source-video representations with frozen target-video geometry features, encoding target-view geometry directly in the source stream.This reduces the need to establish source-to-target correspondences separately at each sampling step.
  • Resolving camera-control inconsistency with RETA substantially lowers denoising trajectory curvature, creating a more favorable basis for step distillation.
  • 25× lower sampling costs let FlashRender match multi-step baselines in video quality and geometric consistency while achieving superior camera controllability.Experiments also report robust generalization to out-of-distribution target camera trajectories.

2 RELATED WORK

Camera-controlled video-to-video rendering extends camera control to retaking existing videos, but prior explicit and implicit approaches face geometric, warping, and few-step efficiency challenges. FlashRender extends the implicit paradigm with representation alignment tailored to source-to-target transformations.

  • Generative rendering methods re-render input videos along target camera trajectories and support dynamic novel views under occlusions and unseen regions.
  • Explicit approaches backproject and reproject video frames in 3D, but quality remains sensitive to estimated depth and warping errors.They may require large video diffusion models to correct resulting artifacts.
  • Representation Alignment transfers semantic and structural information from frozen visual encoders to noisy latent representations, with later work improving spatial structure and conditioning-token entanglement.
  • FlashRender extends representation alignment by internalizing source-to-target geometric transformations, providing consistent camera control across denoising timesteps.
  • Few-step generative modeling includes trajectory distillation, distribution matching, consistency models, and MeanFlow, with extensions to 3D scene generation and rendered-image enhancement.

3 PRELIMINARY

The preliminary section contrasts instantaneous-velocity flow matching with MeanFlow, which predicts average velocities between timesteps for accelerated sampling. It also describes on-policy distillation and the computational trade-offs of Jacobian-vector products and finite-difference approximations.

  • Flow matching: Rectified flows define a linear path between data and noise distributions and train a neural network to regress conditional velocities.
  • Flow matching: During inference, flow-matching samples are generated by solving an ODE from t = 1 to t = 0.
  • MeanFlow: MeanFlow predicts average velocity fields over intervals [r, t] to accelerate inference and reduce discretization error.
  • MeanFlow: MeanFlow trains a network to regress interval-average velocities while incorporating a time-derivative term, with stop-gradient used in the objective.
  • MeanFlow: MeanFlow enables few-step generation, such as 4-NFE sampling, by iteratively transporting the state with predicted average velocity.
  • On-policy flow map distillation: The flow-map transition r←t(zt) updates a state using the predicted velocity over the interval from r to t.
  • On-policy flow map distillation: On-policy flow map distillation trains the student on its own rollout samples to reduce training-inference mismatch.DMD instead uses score differences between target and student-induced distributions, while on-policy flow-map methods build on MeanFlow transitions.

4 METHODOLOGY

FlashRender uses RETA and relative camera-pose conditioning to align source and target video representations, then applies MeanFlow and on-policy distillation for efficient few-step rendering. Its three-stage pipeline targets consistent camera control, lower-curvature denoising, reduced discretization error, and improved visual fidelity under fixed sampling.

  • Stage 1: Multi-Step Generative Rendering: FlashRender fine-tunes Wan2.1-1.3B-CamCtrl, jointly processing source and target video latents with target-trajectory RoCE conditioning.Frame-wise source-to-target relative pose is additionally injected into source-video latents.
  • Stage 1: Multi-Step Generative Rendering: RETA aligns projected intermediate source-video representations with target-view features from a frozen VGGT encoder and camera-token attention.The objective uses cosine similarity to internalize the source-to-target geometric transformation in the source-video stream, with λ = 0.1.
  • Stage 2: MeanFlow Training: RETA consistently reduces denoising trajectory curvature throughout the 50-step sampling process, providing the trajectory used for MeanFlow fine-tuning.MeanFlow predicts average velocity over [r, t] to shortcut the denoising trajectory and mitigate discretization error.
  • Stage 2: MeanFlow Training: MeanFlow training replaces the Jacobian-vector-product term with a finite-difference approximation and uses interpolated timestep conditioning plus classifier-free guidance.The guidance scale is ω = 5, and the residual is weighted adaptively in the MeanFlow loss.
  • Stage 3: On-Policy Flow Map Distillation: On-policy flow map distillation trains the MeanFlow student on self-rollout samples generated with the fixed four-step inference schedule.An adversarial objective additionally aligns self-rollout and real-sample distributions, using λGAN = 0.03.

5 EXPERIMENTS

Experiments evaluate FlashRender against multi-step and few-step baselines on DAVIS and DyCheck, including qualitative comparisons and ablations. FlashRender preserves content, achieves strong visual and geometric performance, and improves camera controllability and out-of-distribution robustness.

  • EXPERIMENTAL SETUPS: The evaluation compares FlashRender with recent multi-step and few-step generative rendering baselines using established DAVIS protocols and metrics.The DAVIS evaluation uses 50 source videos and 10 target camera trajectories, yielding 500 cases.
  • MAIN RESULTS: FlashRender preserves input-video backgrounds and object details while accurately following target camera trajectories in qualitative multi-step comparisons.
  • MAIN RESULTS: FlashRender preserves content at distant target viewpoints and plausibly generates regions outside the input-video view in few-step rendering.
  • MAIN RESULTS: FlashRender achieves the best performance on nearly all few-step metrics, closely matches FlashRender-MS overall, and significantly outperforms it in camera controllability.NeoVerse has the best Imaging Quality score but poorer geometric consistency and camera accuracy; implicit baselines underperform their multi-step counterparts.
  • ABLATION STUDY: RETA, MeanFlow, and on-policy flow map distillation contribute complementary improvements: RETA stabilizes camera control, MeanFlow mitigates discretization error, and distillation corrects self-rollout errors.Ablating the second or third training stage degrades performance, while RETA consistently improves results.
  • GENERALIZATION TO OUT-OF-DISTRIBUTION CONDITIONS: On DyCheck, FlashRender outperforms multi-step and few-step baselines in visual quality and geometric consistency while more accurately following out-of-distribution camera trajectories.It has slightly lower fidelity than multi-step models, mainly because of scene-scale mismatch with the ground-truth video.

6 CONCLUSION

FlashRender addresses sampling-step-dependent camera control in few-step generative rendering with RETA, MeanFlow, and on-policy flow map distillation. Together, these components match multi-step methods in visual quality and geometric consistency at 4-NFE while improving camera controllability.

  • FlashRender generates video retakes from an input video and target camera trajectory in seconds.
  • RETA aligns source-video representations with target-view features to enforce sampling-step-consistent camera control and reduce denoising trajectory curvature.
  • MeanFlow learns trajectory shortcuts to mitigate discretization error, while on-policy flow map distillation corrects self-rollout errors under fixed few-step sampling.
  • With only 4-NFE, FlashRender matches multi-step methods in visual quality and geometric consistency while achieving superior camera controllability.

A EXPERIMENTAL DETAILS

The appendix details three training stages, MeanFlow and on-policy distillation procedures, camera-motion evaluation settings, and implementation resources. The methods use finite-step flow-map training and rollout-based optimization with specified schedules and camera increments.

  • Implementation: Training uses MultiCamVideo and SynCamVideo, with 81-frame 480 × 832 videos, eight RTX PRO 6000 GPUs, global batch size 8, and approximately two weeks for all stages.
  • Training stages: Stage 1 trains camera and attention components for 20k iterations at 10^-4 and applies RETA to source representations from the eighth DiT block.
  • Training stages: Stage 2 fine-tunes with MeanFlow for 20k iterations at 3 × 10^-5 while retaining RETA and sampling shifted timestep pairs with r ≤ t.The shift value is s = 5; r = t is sampled with probability 0.5 and r = 0 with probability 0.1.
  • Training stages: Stage 3 performs on-policy flow map distillation for 5k iterations at 10^-5, updating the student, fake-score model, and discriminator on specified schedules.One student update is performed for every four fake-score-model and discriminator updates.
  • Evaluation details: Evaluation estimates source camera trajectories with ViPE, applies sigma shift 5.0 and CFG scale 5, and defines per-frame target camera increments for camera-motion tests.The listed motion settings include zoom translation along the z-axis and vertical or horizontal translations.
  • MeanFlow training: MeanFlow training interpolates samples with noise, computes classifier-free-guidance targets, estimates velocity derivatives through perturbed states, and optimizes a finite-interval flow objective.
  • On-policy flow map distillation: On-policy distillation uses N = 4 sampling steps to backward-simulate a rollout, then re-noises the generated sample for further training.

B ANALYSIS ON MULTI-STEP GENERATIVE RENDERING

Existing generative rendering methods exhibit sampling-step-dependent camera control, which changes object localization, scene scale, and out-of-view regions. FlashRender reduces trajectory curvature, making few-step approximation more effective.

  • Previous methods produce inconsistent camera control when the sampling-step count changes, altering object localization, scene scale, and synthesized out-of-view regions.
  • Pairwise PSNR across sampling steps and denoising trajectory curvature under 50-step sampling quantify the stability and distillation difficulty of these methods.
  • FlashRender achieves substantially lower denoising trajectory curvature than previous methods.
  • Resolving inconsistent camera control makes the denoising trajectory easier to approximate with MeanFlow and facilitates subsequent on-policy flow map distillation.

C.1 MORE FEW-STEP GENERATIVE RENDERING RESULTS

Additional results show that FlashRender preserves input-video content while synthesizing realistic out-of-view regions across few-step rendering examples and target trajectories.

  • FlashRender faithfully preserves input-video content while synthesizing realistic out-of-view regions in additional few-step rendering results.
  • NeoVerse degenerates when scene reconstruction fails, while implicit baselines may miss input content or generate blurry objects.
  • Qualitative ablations compare model variants using the same input video, target camera trajectory, frame index, and 4-NFE generation setting.

C.2 ADDITIONAL ABLATION RESULTS

Ablations show that MeanFlow-based on-policy distillation better balances content preservation and out-of-view synthesis, while the training stages address distinct error sources.

  • MeanFlow-based on-policy flow map distillation better preserves input content while avoiding over-saturated out-of-view regions than DMD-based alternatives.
  • Without Stage 2, inaccurate flow transitions limit later correction of self-rollout errors and produce object duplication and degraded out-of-view regions.
  • Without Stage 3, the MeanFlow model retains residual artifacts under fixed few-step sampling.
  • The adversarial formulation improves overall performance, particularly input-video content preservation and visual quality, while retaining the DMD objective.

C.3 ANALYSIS ON RETA

RETA stabilizes camera control across sampling steps and lowers trajectory curvature, improving the basis for MeanFlow and subsequent distillation. Its benefits are visible in quantitative, qualitative, and feature-space analyses.

  • RETA and MeanFlow together achieve the best few-step results across different sampling steps.
  • RETA substantially improves camera controllability under few-step sampling, reflected by lower TransErr and RotErr.
  • Without RETA, target retakes vary across sampling steps, including dynamic-object mislocalization and partial bus disappearance; RETA stabilizes target-view geometry.
  • RETA makes on-policy flow map distillation more effective when both student and teacher use RETA.
  • Without RETA, source features largely preserve source-view structure, making geometric mapping dependent on the sampling step.
  • Spatial normalization reduces curvature, but the variant without spatial normalization consistently achieves lower curvature.
Loading 2609.03563v1…