PaperlayerView exact revision on arXiv

Source-linked AI summary

OneFixer: High-Quality and Consistent One-Step Autoregressive 3DGS Refinement for Driving Scenes

Boseong Jeon, Junhyeop Lee, Juhan Cha, Hayoung Kim

arXiv:2609.32175v1cs.CVcs.AI

TL;DR

One-step autoregressive 3DGS refinement must preserve visual quality and temporal consistency at low latency, while existing approaches add staged machinery and still fall short on one-step quality. OneFixer trains a deployment-matched shared rollout with flow matching and pixel-space supervision, and reports the strongest one-step perceptual metrics with consistency matching multi-stage DMD pipelines. In closed-loop simulation, its refined observations yield fewer collisions and off-road events for a driving policy.

  • Problem

    One-step autoregressive refinement must balance perceptual quality and temporal consistency at low latency, while existing approaches use multiple stages and still leave one-step quality short of deployment needs.

    Source: §1 · pp. 1–2

  • Method

    OneFixer trains a pretrained causal video generator in one stage, using its deployment-time predictions as both flow-matching history and the source of direct pixel-space supervision.

    Source: §1 · p. 2

  • Results

    OneFixer achieves the lowest one-step FVD, LPIPS, and DISTS on Waymo and proprietary scenes, with temporal consistency matching multi-stage DMD pipelines.

    Source: §1 · p. 3

  • Takeaways & Limitations

    In 60-scene closed-loop simulation, refined observations yield fewer collisions and off-road events for a driving policy.

    Source: §1 · p. 3

    Fine structures such as text and road markings are not reliably recovered, and strong perceptual scores do not guarantee their exactness.

    Source: §5 · p. 9

Abstract

from arXiv · show

Autoregressive video diffusion is a promising render-time fixer for 3D Gaussian Splatting (3DGS) in autonomous-driving simulation, but deployment demands high visual quality and temporal consistency at low latency. This is especially hard for one-step causal generation, where each imperfect prediction immediately becomes context for subsequent frames. Existing approaches stabilize rollouts through staged training with multiple modules and rollout-aware regularization, yet one-step quality still falls short of what deployment requires. We introduce OneFixer, a one-step autoregressive video-diffusion fixer trained in a single task-specific adaptation stage. Our key idea is a deployment-matched shared rollout: the model's own one-step predictions serve as the causal context for flow matching, exposing training to deployment-time errors, while the same rollout receives direct pixel-space perceptual supervision to preserve fine detail. Because the predictions optimized for current-frame quality are exactly those reused as future context, fidelity and autoregressive robustness are learned jointly, without bidirectional-to-causal conversion or teacher-student distillation. OneFixer further exploits cues that driving simulation readily provides, lane geometry and dynamic-agent states, to improve geometric fidelity. On Waymo and proprietary driving scenes with 900-frame rollouts, OneFixer achieves the lowest FVD, LPIPS, and DISTS among all baselines at one step, with temporal consistency matching or exceeding multi-stage DMD pipelines. Under identical backbone and conditioning, it matches a multi-stage DMD-with-Self-Forcing pipeline in under half the GPU-hours and keeps improving beyond its plateau. In closed-loop simulation with a driving policy, OneFixer reduces the collision rate by a third relative to raw 3DGS rendering. Project page: https://onefixer-web.vercel.app/

1 INTRODUCTION

One-step autoregressive refinement must balance frame quality, temporal consistency, and latency because imperfect predictions become context for later frames. OneFixer addresses this with a shared deployment-matched rollout, task-specific driving cues, and single-stage training.

1 INTRODUCTION

  • One-step generation is difficult because reduced denoising degrades predictions that then become causal context, compounding errors and causing history drift.Reducing OmniDreams from four steps to one lowers latency but worsens both FVD and FVMD, despite 3DGS anchoring.Source: §1 · pp. 1–2
  • Prior systems address rollout drift and one-step quality loss with separate multi-stage training, teachers, critics, or reward-based post-training.These recipes were transferred from generic generation without exploiting 3DGS refinement’s task structure.Source: §1 · p. 2
  • A shared rollout uses the model’s own one-step predictions both as flow-matching history and as inputs to direct pixel-space supervision.The single-stage procedure requires no bidirectional-to-causal conversion, teacher–student distillation, critic, or separate post-training.Source: §1 · p. 2
  • Joint optimization couples latent consistency at every rollout position with pixel-detail supervision that also trains preceding predictions as context.A sequential recipe does not provide this coupling.Source: §1 · p. 2
  • OneFixer exploits opacity-derived rendering reliability, lane geometry, and dynamic-agent states as cues available in driving simulation.Displaced lanes or hallucinated vehicles can also become incorrect observations for the policy under test.Source: §1 · p. 2
  • OneFixer achieves the lowest one-step FVD, LPIPS, and DISTS on Waymo and proprietary scenes, with temporal consistency matching multi-stage DMD pipelines.Evaluation includes 900-frame rollouts, 9.7× the training horizon.Source: §1 · p. 3

2 RELATED WORK

Prior render-time fixers offer frame-level quality or temporal modeling, but streaming one-step refinement remains challenging. OneFixer’s shared rollout pairs causal consistency training with pixel-space fidelity supervision.

2.1 DIFFUSION MODELS FOR RENDER-TIME 3DGS ENHANCEMENT

  • Driving fixers use causal autoregressive models for streaming, but existing approaches rely on staged adaptation or distillation and do not establish quality-preserving one-step deployment.Image fixers can miss transient interference, while bidirectional video models operate on fixed clips and cannot stream.Source: §2.1 · p. 3

2.2 AUTOREGRESSIVE VIDEO DIFFUSION AND EXPOSURE BIAS

  • Exposure bias arises because training uses clean histories while inference conditions on model predictions; existing remedies add teacher distillation and rollout stages, yet one-step quality still lags.Self-resampling uses a ground-truth-noised proxy and supplies no output-pixel-quality signal.Source: §2.2 · p. 3

2.3 DIRECT SUPERVISION FOR ONE-STEP FIDELITY

  • Prior pixel-space supervision for video either targets bidirectional models without reusable causal history or trains causal predictions through distribution matching rather than clean-reference pixel loss.OneFixer pairs pixel-space supervision with temporal objectives on the same deployment-matched rollout.Source: §2.3 · p. 4, Figure 2 · p. 4

3 METHOD

OneFixer starts from a pre-DMD causal OmniDreams checkpoint and trains self-rollout flow matching and frame-wise perceptual supervision on one deployment-matched rollout.

3 METHOD

  • OneFixer jointly trains self-rollout flow matching and frame-wise pixel supervision on a single deployment-matched rollout.Both objectives are evaluated on the same shared rollout.Source: §3 · p. 4, §3.3 · p. 5

3.1 ONE-STEP CAUSAL REFINEMENT FROM A 3DGS SOURCE

  • The opacity-derived mixed source preserves reliable rendered structure, making each one-step output a correction rather than noise-driven synthesis; replacing it with pure noise degrades every metric.The source retains rendered content where reconstruction is reliable and leaves generative freedom where it is not.Source: §3.1 · p. 4

3.2 SELF-ROLLOUT FLOW MATCHING

  • Training recursively uses the model’s own one-step predictions as causal history, while flow matching supervises every sequence position in latent space to improve temporal consistency.The figure contrasts this consistency gain with appearance variation under Diffusion Forcing.Source: §3.2 · p. 5, Figure 3 · p. 5

3.3 FRAME-WISE PIXEL SUPERVISION ON THE SHARED ROLLOUT

  • Pixel supervision compares decoded RGB frames at a sampled rollout position with clean references, and gradients through the preceding k predictions train them as useful context.The figure shows direct pixel-space supervision recovering fine details, particularly tree branches and foliage.Source: §3.3 · p. 5, Figure 3 · p. 5

4 RESULTS

OneFixer performs strongly across perceptual, geometric, and temporal metrics at one step, while maintaining long-rollout stability and improving closed-loop replay outcomes. Ablations favor jointly training flow matching and pixel supervision on the model’s own rollout, with geometry conditioning providing additional gains.

4.1 EXPERIMENTAL SETUP

  • For training and evaluation, models use separate Waymo and proprietary Seoul data; held-out rollouts span 198 Waymo frames and 900 Seoul frames.The rollouts are approximately 2.1× and 9.7× the 93-frame causal training horizon, respectively.Source: §4.1 · pp. 5–6

4.2 COMPARISON WITH 3DGS ENHANCEMENT METHODS

  • OneFixer achieves the strongest one-step balance across perceptual, geometric, and temporal metrics, while leading nearly every metric on 900-frame Seoul rollouts.It improves temporal consistency over image-based fixers while retaining frame-level fidelity, and improves perceptual quality over video fixers while matching or exceeding their temporal metrics.Source: §4.2 · p. 6
  • OneFixer is substantially faster than video-based fixers and approaches image-based enhancement latency using one flow step per causal block.The block’s chunked causal decoder produces several RGB frames at once, amortizing backbone cost.Source: §4.2 · p. 6
  • After roughly 345 GPU-hours, OneFixer falls below both same-backbone baselines’ final FVD and reaches the Self-Forcing FVMD plateau; it ends lower on both metrics.The same-backbone comparison holds backbone, conditioning, and causal initialization fixed; the DMD stage saturates early, with another 300–400 GPU-hours changing either metric only marginally.Source: §4.2 · p. 7, Figure 6 · p. 8
  • On 60 replay scenes, OneFixer reduces collisions versus raw 3DGS from 10.6% to 6.9% and halves the off-road rate.The replay scenes overlap with policy training data, so these results measure replay fidelity rather than policy generalization.Source: §4.2 · p. 7, Figure 5 · p. 7

4.3 ABLATION AND ANALYSIS

  • Replacing the opacity-mixed source with noise and channel-concatenated 3DGS latents degrades every metric substantially.This ablation identifies the structured source as important for practical one-step fine-tuning.Source: §4.3 · p. 8
  • Joint self-rollout flow matching and LPIPS gives the best balance; sequential training matches frame fidelity but loses on FVD and FVMD.Self-rollout flow matching alone mainly improves temporal consistency, LPIPS alone improves fidelity, and replacing model predictions with re-denoised ground truth degrades every metric, especially LPIPS (0.158 vs. 0.117).Source: §4.3 · p. 8
  • Removing lane and actor conditioning lowers Cuboid F1 and modestly worsens perceptual and temporal metrics, but the variant still beats one-step baselines on several metrics.The unconditioned variant remains ahead on FVD, LPIPS, and Cuboid F1 with competitive FVMD.Source: §4.3 · p. 8

5 CONCLUSION AND LIMITATIONS

OneFixer uses a shared rollout to train a single-model, single-stage pipeline for one-step 3DGS refinement. The conclusion reports strong perceptual quality and temporal consistency beyond the training horizon, while noting limits in fine-detail recovery and training memory.

5 CONCLUSION AND LIMITATIONS

  • The conclusion reports strong perceptual quality and temporal consistency beyond the training horizon at one-step latency.Source: §5 · pp. 8–9
  • Table 1 compares methods on Waymo and proprietary Seoul, while Table 2 covers ablations and closed-loop evaluation.Source: Table 1 · p. 9, Table 2 · p. 9
  • Fine structures such as text and road markings are not reliably recovered, and strong perceptual scores do not guarantee exactness.Source: §5 · p. 9
  • Peak training memory grows with parallel temporal context and the perceptual gradient window; scaling resolution or camera count trades context for memory.Source: §5 · p. 9

AI USE STATEMENT

The authors report using generative AI tools for language editing, clarity, and formatting, while retaining responsibility for technical content and claims.

AI USE STATEMENT

  • Generative AI tools assisted with language editing, clarity, and manuscript formatting; the authors say they reviewed and verified the technical content, results, and claims.Source: AI USE STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

The reproducibility statement points to supplementary code, a frozen evaluation protocol, and documented dataset and baseline configurations.

REPRODUCIBILITY STATEMENT

  • Supplementary material provides training, inference, and frozen evaluation code, including the Waymo recipe and metric implementations.Source: REPRODUCIBILITY STATEMENT · p. 10
  • The pretrained checkpoint is the publicly released pre-DMD causal OmniDreams model, and baseline configurations follow Appendix C.2.Source: REPRODUCIBILITY STATEMENT · p. 10
  • Waymo data generation is described in Appendix A.1; Seoul is proprietary, but its preprocessing and training configuration are documented and included in the code.Source: REPRODUCIBILITY STATEMENT · p. 10

A.1 3DGS DATA GENERATION - WAYMO

The appendix describes paired driving data generation for Waymo and internal scenes, followed by the self-rollout perceptual supervision and gradient truncation used in training.

A.1 3DGS DATA GENERATION - WAYMO

  • For Waymo, a dynamic Street Gaussians reconstruction supplies geometry, appearance, and actor motion for the subsequent rendering pipeline.Source: §A.1 · p. 14
  • Waymo degraded inputs are produced by shifting training views laterally by 1 m, training a cycle model, restoring actors, then rendering at logged poses to induce parallax, blur, and exposure mismatch.Source: §A.1 · p. 14
  • The Waymo corpus has 350 training scenes with 69,238 frame pairs and 80 validation scenes with 15,812 pairs after truncation to common sequence lengths.Source: §A.1 · p. 15, Table 3 · p. 15

A.2 3DGS DATA GENERATION - INTERNAL

  • Internal driving data pairs degraded MTGS renders with logged RGB, renderer opacity, and semantic maps, using synchronized multi-camera scenes from South Korea.Source: §A.2 · p. 15

A.3 ONEFIXER TRAINING DETAILS

  • Perceptual supervision compares decoded self-rollout frames with decoded ground truth using frozen VGG LPIPS; autoregressive gradients are retained only through the last four latent frames.Source: §A.3 · p. 16

B.1 ROBUSTNESS TO GEOMETRY-CONDITIONING ERRORS

The appendix tests whether geometry conditioning remains useful under controlled errors, examines its training-memory cost, and describes transfer evaluation beyond driving scenes.

B.1 ROBUSTNESS TO GEOMETRY-CONDITIONING ERRORS

  • The 45-setting sensitivity sweep uses fixed lane offsets up to 0.50 m and object translations up to approximately 0.354 m, but does not reproduce the full deployment error distribution.The tested magnitudes are grounded in reported decimeter-scale perception errors.Source: §B.1 · p. 16
  • Lane F1 stays above the unconditioned reference across the tested sweep, Cuboid F1 remains stable and above it, and I3D FVD shows no systematic degradation.At the largest lane displacement, lane lateral error exceeds the unconditioned reference, indicating a positional-accuracy trade-off despite retained lane-structure benefits.Source: §B.1 · pp. 16–17, Figure 7 · p. 17

B.2 TRAINING MEMORY

  • At 24 latent positions, the default perceptual window k = 4 uses 102 GiB; memory rises with both parallel context length and gradient-window size.At 24 positions, k = 8 reaches 124 GiB and k ≥ 12 exceeds device capacity; shortening context to 16 positions accommodates k up to 16. Shorter self-rollouts are used for multi-camera and high-resolution settings.Source: §B.2 · p. 17, Figure 8 · p. 17

B.3 GENERALIZATION TO NON-DRIVING SCENES.

  • Transfer beyond driving is evaluated by fine-tuning on 32 DL3DV scenes with degraded renders and no lane or actor conditioning, against official DL3DV-finetuned DiFix and ArtiFixer checkpoints.The supplied evaluation description identifies the setup but does not report its outcome.Source: §B.3 · p. 18

C.1 EVALUATION PROTOCOL AND METRICS

The appendix specifies evaluation metrics and aggregation protocols, then documents baseline configurations and their evaluation operating points. These details clarify what the reported comparisons measure and how the reference methods were run.

C.1 EVALUATION PROTOCOL AND METRICS

  • Clip-level metrics generally report means with 95% t-intervals, whereas I3D FVD and FVMD are distributional metrics without per-clip confidence intervals.Closed-loop replay covers 60 proprietary scenes with five policy seeds; collision reporting uses the union of front and lateral at-fault collisions.Source: §C.1 · pp. 18–19
  • I3D FVD compares distributions of spatiotemporal video features, while FVMD summarizes local motion consistency using trajectory statistics from sampled 16-frame windows.FVD values from smaller subsets or shorter clips are not comparable to Table 1 values; only within-figure comparisons are meaningful for those subsets.Source: §C.1 · pp. 18–19
  • LPIPS, PSNR, SSIM, and DISTS are computed against clean-reference frames and aggregated across clips; evaluation LPIPS uses AlexNet, unlike the frozen VGG-based training LPIPS.DISTS compares deep-feature structure and texture statistics and is less sensitive than PSNR to plausible high-frequency detail that does not align pixel-wise.Source: §C.1 · p. 18
  • Lane F1 and lateral error use detected, matched lanes, while Cuboid F1 measures overlap between projected detection masks rather than 3D-IoU box matching.The main lane lateral-error value combines near and far ranges, with the boundary at 30 m.Source: §C.1 · pp. 18–19

C.2 BASELINE CONFIGURATION

  • Baselines are evaluated at their reported operating points rather than a forced common denoising-step count; examples include one-step DiFix and eight-step GSFix3D and 3DGS Enhancer.The comparison also includes one-step Harmonizer variants, one-step ArtiFixer, and multi-stage Self-Forcing and One-Forcing pipelines.Source: §C.2 · pp. 20–21
  • The temporal Harmonizer baseline uses four prior enhanced frames; its teacher-forced recipe drifted in rollouts, so the evaluated variant instead trains with self-generated context.The self-generated-context variant retains the warping loss and is a modification of the official recipe, not a reproduction of its checkpoint.Source: §C.2 · p. 20, Figure 9 · p. 21

D APPEARANCE-CONDITIONED REFINEMENT

OneFixer is extended to refine degraded 3DGS renderings while generating requested appearances, with the aim of retaining scene structure. The supplied passages also describe its transfer evaluation on DL3DV.

D APPEARANCE-CONDITIONED REFINEMENT

  • Paired night, fog, and snow targets are generated from clean Waymo references, filtered for structural drift, and combined with clear-weather identity targets.The input remains the degraded, viewpoint-shifted 3DGS rendering with opacity and HD-map conditioning.Source: D APPEARANCE-CONDITIONED REFINEMENT · pp. 21–22
  • After DL3DV fine-tuning, OneFixer matches or exceeds DiFix’s frame-level quality while providing stronger temporal consistency, and produces higher-fidelity results than one-step ArtiFixer.DiFix and ArtiFixer are evaluated using their official DL3DV-finetuned checkpoints.Source: Figure 10 · p. 22
  • OneFixer trains on identity, night, fog, and snow targets with text conditioning and the shared-rollout objective, jointly correcting artifacts and synthesizing the requested appearance in one pass.This is refinement from an imperfect 3DGS input, not conventional stylization of an already clean video.Source: D APPEARANCE-CONDITIONED REFINEMENT · p. 22
  • General-purpose video editors can produce convincing styles but may alter scene content and geometry, whereas OneFixer combines appearance transformation with structural refinement.The paper identifies broader, more diverse training data as a direction for improving fine-grained appearance control.Source: Figure 11 · p. 22, D APPEARANCE-CONDITIONED REFINEMENT · p. 23

E SCALING TO MULTI-CAMERA AND HIGH-RESOLUTION REFINEMENT

OneFixer is extended to synchronized multi-camera and higher-resolution refinement by shortening the training self-rollout to manage memory. The reported qualitative results retain cross-view structure and recover finer details at higher resolution.

E SCALING TO MULTI-CAMERA AND HIGH-RESOLUTION REFINEMENT

  • Multi-camera training halves the latent sequence length per iteration and uses a longer optimization schedule, while jointly refining views with LiDAR alignment and coherent bird’s-eye-view composition.These qualitative cues indicate preservation of shared scene geometry across viewpoints rather than independent per-camera enhancement.Source: E SCALING TO MULTI-CAMERA AND… · p. 23, Figure 12 · p. 24
  • At up to 1920-pixel width, OneFixer retains its overall restoration behavior while recovering finer details such as distant road signs and small structures.The high-resolution training strategy shortens the self-rollout segment, reducing the temporal context processed in parallel.Source: E SCALING TO MULTI-CAMERA AND… · p. 23, Figure 13 · p. 24

Related papers

Loading 2609.32175v1…