PaperlayerView exact revision on arXiv

Source-linked AI summary

AutoWeather4D: Autonomous Driving Video Weather Conversion via G-Buffer Dual-Pass Editing

Tianyu Liu, Weitao Xiong, Kunming Luo, Manyuan Zhang, Peng Li, Yuan Liu, Ping Tan

arXiv:2603.26546v3cs.CV

TL;DR

Rare adverse-weather data are expensive to collect, while existing 3D-aware editing relies on costly optimization and entangles geometry with illumination. AutoWeather4D uses feed-forward G-buffer dual-pass editing to decouple these factors, achieving comparable photorealism and structural consistency to generative baselines with parametric physical control.

  • Problem

    Generative weather models require massive datasets for rare scenarios, while 3D-aware methods face costly optimization and static-scene geometry limitations.

    Source: §1 · pp. 1–2

  • Method

    AutoWeather4D uses feed-forward G-buffer geometry and decoupled light passes to support physically grounded weather and lighting edits in dynamic driving scenes.

    Source: §1 · p. 3

  • Results

    AutoWeather4D achieves comparable photorealism and structural consistency to generative baselines while providing parametric physical control without per-scene optimization.

    Source: §1 · p. 4

  • Takeaways & Limitations

    The framework serves as a practical complementary data source for autonomous-driving adverse-weather scenarios and potential perception-failure diagnostics.

    Source: §5 · p. 14

    The approach may be sensitive to imperfections in external feed-forward modules, with uncertain effects on editing quality and physical correctness.

    Source: §16.4 · p. 52

Abstract

from arXiv · show

Generative video models have significantly advanced the photorealistic synthesis of adverse weather for autonomous driving; however, they consistently demand massive datasets to learn rare weather scenarios. While 3D-aware editing methods alleviate these data constraints by augmenting existing video footage, they are fundamentally bottlenecked by costly per-scene optimization and suffer from inherent geometric and illumination entanglement. In this work, we introduce AutoWeather4D, a feed-forward 3D-aware weather editing framework designed to explicitly decouple geometry and illumination. At the core of our approach is a G-buffer Dual-pass Editing mechanism. The Geometry Pass leverages explicit structural foundations to enable surface-anchored physical interactions, while the Light Pass analytically resolves light transport, accumulating the contributions of local illuminants into the global illumination to enable dynamic 3D local relighting. Extensive experiments demonstrate that AutoWeather4D achieves comparable photorealism and structural consistency to generative baselines while enabling fine-grained parametric physical control, serving as a practical data engine for autonomous driving.

1 Introduction · 2 Related Works · 3 Method

AutoWeather4D is a feed-forward 3D-aware framework that converts driving videos across weather and illumination while avoiding massive rare-weather datasets and costly per-scene optimization. Its G-buffer Dual-pass Editing separates surface geometry from light transport, and VidRefiner preserves the resolved physical structure during photorealistic refinement.

1 Introduction

  • AutoWeather4D replaces per-scene optimization with feed-forward editing that decouples geometry and illumination for rapid, physically plausible weather and lighting control.Existing 3D-aware methods can require up to an hour per video clip, whereas AutoWeather4D uses explicit G-buffers and avoids auxiliary data.Source: §1 · pp. 2–4

3.1 Feed-Forward G-Buffer Extraction

  • The method addresses dynamic-scene limitations of NeRF and 3DGS pipelines by extracting dense frame-wise geometry and materials from videos with feed-forward models.Pi3 supplies coherent relative depth, while a zero-shot inverse renderer estimates albedo, normals, metallicity, and roughness; metric scale is aligned using LiDAR or camera-height priors, with sky masking preventing invalid material estimates.Source: §1 · pp. 2–3, §3.1 · pp. 5–6

3.2 G-Buffer Dual-pass Editing

  • The Geometry Pass anchors snow, rain, wetness, and ripples to explicit surface descriptors, while the Light Pass analytically combines local sources with global atmospheric illumination.This dual-pass design enables dynamic 3D relighting for effects such as headlights, streetlights, fog halos, and time-of-day changes.Source: §1 · pp. 3–4, §3.2 · pp. 6–7, Figure 2 · p. 6

2.2 Autonomous driving video simulator

  • Autonomous-driving world simulators reduce the cost of collecting complex traffic data, but AutoWeather4D uniquely operates directly on G-buffers for explicit geometric and illumination control.The framework therefore complements existing simulator paradigms with efficient scene modification rather than iterative neural optimization or latent-space manipulation.Source: §2 · p. 4

2.1 Climate simulation

  • Classical weather simulation provides physical priors for rain, snow, and fog, while neural and hybrid approaches offer video synthesis but generally address weather conversion or illumination control separately.Prior NeRF, mesh, and Gaussian-splatting methods commonly depend on static scenes or per-scene optimization; AutoWeather4D instead integrates physical rendering with feed-forward 4D reconstruction.Source: §2.1 · pp. 4–5

Geometry Pass: Surface-Anchored Interaction

  • The Geometry Pass implements multi-scale snow and physically grounded rain by modifying albedo, normals, and roughness while enforcing depth-aware spatial and temporal consistency.Snow combines surface buildup, ground density and wetness, and persistent falling particles; rain combines wind-and-gravity streaks with depth-tested drops, puddles, and procedural ripples.Source: Geometry Pass: Surface-Anchored… · p. 7

Light Pass: Decoupled Illumination Control

  • The Light Pass separately models nocturnal spotlights, fog scattering, and HDR-conditioned environment illumination to produce parametric local relighting and global ambient harmonization.It uses metric depth with Cook-Torrance BRDF evaluation for local sources, a single-scattering radiative-transfer model for fog, and blended HDR environment radiance for sparse geometry.Source: Light Pass: Decoupled Illumination… · pp. 7–8

3.3 VidRefiner

  • VidRefiner adds sensor-level texture refinement without changing the physically resolved scene by combining latent initialization with explicit spatiotemporal boundary conditioning.The rendered sequence supplies low-frequency layout, color, and lighting priors, while input-level boundaries constrain fine-grained synthesis to the edited geometry.Source: §3.3 · p. 8

4 Experiment

AutoWeather4D is evaluated on dynamic Waymo driving scenes across adverse weather and lighting conditions using fidelity, structural, identity, and human-preference measures. Results show comparable quantitative performance to data-driven baselines while providing explicit physical control, continuous relighting, and useful perception-data augmentation.

4.1 Experimental Setting

  • Experiments use 120 Waymo NOTR scenes, with the G-buffer Dual-pass Editing runtime reported separately.The method and most baselines run on V100 GPUs, while resource-intensive baselines use A100 GPUs.Source: §4.1 · p. 9

4.2 Baselines

  • Baselines span video-editing, foundation, inverse-rendering, and weather-specific models, evaluated on fog, midnight, rain, and snow with standardized prompts.Evaluation measures instruction adherence with CLIP, structural consistency with relative 2D LiDAR-box IoU, and identity stability with patch-level CLIP similarity.Source: §4.2 · pp. 9–10

4.3 User Study

  • A double-blind 2AFC study with 12 raters aggregates 1,440 responses comparing spatial fidelity and temporal coherence against four baselines.Sequences are randomized across weather conditions and standardized to 512 × 512 resolution and 30 frames.Source: §4.3 · p. 10

4.4 Results

  • AutoWeather4D matches massive data-driven baselines quantitatively while providing physically plausible, fine-grained control over localized lighting, fog, illumination transport, and foreground geometry.Qualitative comparisons attribute these properties to explicit illumination decomposition and parameterized light control; Table 3 summarizes conversions from 120 source videos into four adverse conditions across 27,360 frames.Source: §4.4 · p. 11, Table 3 · p. 12

4.5 Ablation Study

  • Continuous feed-forward 4D reconstruction removes the jagged aliasing caused by integer-quantized depth during local illumination by restoring a continuous floating-point geometric manifold.This continuous structural prior produces natural illumination gradients during dynamic relighting.Source: §4.5 · p. 12, Figure 5 · p. 13

4.6 Applications

  • AutoWeather4D preserves geometry sufficiently for perception augmentation and enables deterministic HDR-driven and parameterized control, although downstream diagnostic benchmarking remains future work.HRDA is fine-tuned on 6,480 augmented frames; Cosmos provides only a +0.04% mIoU increment on Dark Zurich, whereas AutoWeather4D yields consistent gains, and its controls scale fog density or toggle local lights.Source: §4.6 · pp. 12–14, Table 4 · p. 13

5 Conclusion

AutoWeather4D combines deterministic graphics, video diffusion, and explicit G-buffer priors to produce physically grounded, parametric adverse-weather simulations from real-world footage. Its remaining challenges involve extreme long-tail dynamics and preserving distant structures under severe perturbations.

Conclusion

  • AutoWeather4D aligns deterministic graphics with video diffusion, using explicit G-buffer priors to transform real footage into physically grounded, parametric adverse-weather scenarios.The framework provides a complementary data source for diagnosing perception failure modes under adverse conditions.Source: §5 · p. 14

Limitations

  • Extreme long-tail interactions, such as complex vehicle-splash fluid dynamics, remain challenging for the decoupled pipeline.Future work will integrate localized generative priors for these unstructured phenomena while retaining deterministic structural anchors.Source: §5 · pp. 14–15
  • Severe perturbations such as heavy-fog occlusion require careful calibration to preserve distant background structures.Future iterations could use semantic-aware attenuation masks to modulate diffusion intensity across critical autonomous-driving regions of interest.Source: §5 · p. 15

Supplementary Materials for AutoWeather4D · 6 Feed-Forward G-Buffer Extraction · 7 Snow Synthesis via G-Buffer Dual-Pass Editing

The supplementary materials detail metric-aware G-buffer extraction and snow synthesis through surface accumulation, material blending, wet-ground effects, and falling-particle simulation. They provide implementation parameters for reproducible physically controlled weather editing.

Supplementary Materials for AutoWeather4D

  • The supplementary document covers implementation details, extended quantitative evaluations, ablations, and qualitative visualizations supporting AutoWeather4D reproducibility.It includes sections on G-buffer extraction, weather synthesis, relighting, harmonization, prompting, VidRefiner configuration, evaluations, ablations, and visual comparisons.Source: Supplementary Materials for… · p. 21

6 Feed-Forward G-Buffer Extraction

  • Metric calibration aligns reconstructed depth with LiDAR by sampling 1,000 non-sky, non-occluded points and minimizing mean squared error to jointly solve scale s and bias b for physically plausible editing.For each frame, sky depth is set to the 99th percentile of the non-sky depth distribution across the entire sequence.Source: §6 · pp. 21–22

7.1 Metaball-based surface snow

  • Surface snow uses cascaded SPH Poly6 metaballs with support radius ρ = 0.1 m to smoothly merge multi-scale accumulation around surface points.The implementation uses k = 16 neighbors, L = 3 cascade levels, λ = 0.7 amplitude decay, and density jitter in [0.8,1.2].Source: §7.1 · pp. 22–23

7.2 Material blending

  • Snow material blending uses a sigmoid of snow height, with w = 0.8 and τbias = 0.03, then adjusts albedo, roughness, and metallicity toward diffuse snow.Coverage uses γ = 0.9; snow albedo is 1.0, roughness is 0.6, metallic is zero, and optional normal displacement adds geometric detail.Source: §7.2 · p. 23

7.3 Wet ground implementation

  • Wet ground darkens albedo using porosity and optical attenuation while reducing roughness to model water sheen selectively on non-snow-covered areas.The implementation sets p = 0.8, Awater ≈ 0.02, wetness intensity i = 0.5, and rwater = 0.1.Source: §7.3 · pp. 23–24

7.4 Particle-based falling snow

  • Falling snow updates particle positions using gravity and wind, simulating 6,000 particles within a view-frustum-aligned bounding box.The velocities are vgravity = [0, −2.0, 0]⊤ m/s and vwind = [0.3, 0, 0.1]⊤ m/s, producing downward drift and lateral movement.Source: §7.4 · p. 24

8 Rain Synthesis via G-Buffer Dual-Pass Editing

The rain synthesis pipeline uses G-buffer dual-pass editing to model geometry-anchored puddles, precipitation, ripples, and material responses with empirically tuned parameters. World-space puddle synthesis preserves geometric and temporal consistency, while surface-specific effects complete the wet-weather appearance.

8 Rain Synthesis via G-Buffer Dual-Pass Editing

  • The implementation parameters are empirically tuned on urban driving sequences for the rain rendering pipeline.These settings govern the detailed rain synthesis components described throughout the section.Source: §8 · p. 24

8.1 Geometry-Anchored Puddle Modeling via World-Space FBM

  • World-space FBM projected onto explicit 3D road geometry generates puddles with perspective foreshortening, physical occlusion, and temporal consistency during camera motion.Road pixels are transformed from metric-depth camera coordinates into temporally coherent world coordinates, and FBM is evaluated over lateral coordinates (X_w, Z_w).Source: §8.1 · pp. 24–25

8.2 Precipitation Dynamics

  • Precipitation dynamics simulate raindrops as tapered, motion-blurred capsules whose negative SDFs update G-buffer opacity and depth for translucent, properly occluded streaks.The pipeline uses 104 drops with 0.5–6.0 mm diameters, while streak length is computed from frame interval and drop velocity.Source: §8.2 · p. 25

8.3 Surface Perturbation Effects

  • Grid-based ripple waves perturb puddle-region normals through temporally modulated expanding rings, using strength factor 0.9 for surface variation.Ripples use 32-pixel cells, 31.0 rad/m frequency, smoothstep radial falloff, and intensity oscillating between 0.01 and 0.15.Source: §8.3 · p. 25

8.4 Material Response Parameters

  • Surface-specific G-buffer responses create wet-weather appearance through overcast sky tinting, reduced puddle roughness, ripple highlights, and alpha-blended raindrops with depth bias.Puddle roughness is set to 0.0, non-puddle surfaces receive wetness factor 0.2, and drops use 40% opacity with 10^-4 m depth bias.Source: §8.4 · pp. 25–26

9 Dense Semantic Annotations

AutoWeather4D constructs dense semantic annotations with a detect-segment-propagate pipeline that supplies structural priors for downstream weather synthesis. These annotations support illumination and precipitation rendering by anchoring street lights, roads, and vehicles in space and time.

Dense Semantic Annotations

  • Dense semantic tracks anchor emissive sources, accumulation regions, and dynamic occluders for headlamp estimation, precipitation rendering, and vehicle relighting.Street-light instances localize light sources, road segments constrain accumulation, and vehicle masks identify objects requiring headlight relighting.Source: §9 · p. 26
  • OWL-ViT detects street lights, roads, and cars on sampled keyframes, while SAM2 converts retained proposals into precise instance masks.The pipeline keeps the maximum-area road region per frame and filters low-confidence detections before mask extraction.Source: §9 · p. 26
  • Bidirectional SAM2 propagation tracks masks forward and backward through time, reducing flicker, preserving identities across occlusions, and producing dense per-frame annotations.The resulting street-light, road, and vehicle annotations form the basis for downstream depth analysis.Source: §9 · p. 26

3D Reprojection of Instance Masks and Light Source Estimation

  • Masked street-light pixels are reprojected with calibrated metric depth and camera intrinsics into a unified world coordinate system, forming a global point cloud for illuminant extraction.The reprojection pipeline explicitly combines per-frame semantic masks with reconstructed metric depth and estimated camera poses.Source: §9 · pp. 26–27
  • DSU groups spatially connected street-light points into distinct 3D instances, whose top 5% point centroids estimate bulb locations and whose directions point toward the road for plausible attenuation.The grouping uses a 0.5m distance threshold to accommodate reconstruction noise, and the resulting spotlight directions support Light Pass rendering.Source: §9 · p. 27

10 Nocturnal Local Relighting via G-Buffer Dual-Pass Editing

The nocturnal relighting pipeline combines a Cook-Torrance BRDF with physically attenuated spotlight illumination, then applies global tone mapping and adaptive exposure for realistic night appearance. It also uses sky-specific processing to preserve dark skies and smooth horizon transitions.

10.1 BRDF Modeling

  • The BRDF models outgoing radiance by integrating incident illumination over the surface hemisphere using Cook-Torrance diffuse and specular reflection.It uses surface position, normal, view and incident directions, metallic-roughness properties, and Fresnel, GGX, and Smith visibility components.Source: §10.1 · pp. 27–28

10.2 Light Source Implementation

  • Incident radiance sums RGB contributions from discrete lights with inverse-square distance falloff and combined angular-distance attenuation.Street lights and headlights use spotlight cones, with typical inner/outer angles of 15°/35° and 10°/25°, respectively; influence radii are typically 10-20 meters and 30-50 meters.Source: §10.2 · pp. 28–29

10.3 LUT Construction for Global Tone Mapping

  • A 256×3 LUT with strength parameter σ ∈[0, 1] darkens ambient regions, shifts color temperature, and preserves visibility in unlit areas for nocturnal appearance.Channel-specific scaling controls brightness and color balance, while adaptive exposure and highlight compression are applied before LUT processing to prevent over-darkening.Source: §10.3 · p. 29
  • Sky masks receive differential processing: sky pixels are darkened by αsky = 0.6, with 20-pixel dilation and Gaussian blur κ = 10 for natural horizon transitions.This preserves the dark night-sky appearance while preventing unrealistic brightening of sky regions.Source: §10.3 · p. 30

11 Volumetric Fog Synthesis via G-Buffer Dual-Pass Editing

The method synthesizes volumetric fog with a simplified radiative transfer model that combines attenuated surface radiance with in-scattered illumination. It then applies controllable fog-color blending based on transmittance-derived opacity.

11.1 Fog modeling

  • The fog model computes observed radiance as transmittance-weighted surface radiance plus accumulated in-scattered light.Surface radiance uses a Cook-Torrance BRDF, while transmittance is T(s) = exp(−σt · s) with σt = σa + σs.Source: §11.1 · p. 30
  • In-scattering sums attenuated contributions from all lights using the Henyey-Greenstein phase function with forward-scattering parameter g = 0.8.Directional, point, and spot lights use appropriate distance or angular attenuation, and density scaling α enables real-time adjustment of σa and σs.Source: §11.1 · p. 30

11.2 Fog blending

  • Final fog appearance blends observed radiance with fog color F using opacity f = 1 − T(s) and blend strength β = 0.5.The blend is L_final = (1 − β · f) · L_obs + β · f · F.Source: §11.2 · p. 30

12 Environment Harmonization · 13 Text Prompt Design and Templates

AutoWeather4D harmonizes ambient and directional illumination through adaptive linear-space blending, then uses structured prompts to guide physically plausible, semantically consistent weather and time-of-day conversion. The prompt system combines Qwen3-8B-Instruct scene descriptions [3] with scenario-specific conversion instructions emphasizing realistic lighting, atmospheric, and surface effects.

12 Environment Harmonization

  • Linear-space adaptive blending combines ambient and directional illumination while avoiding gamma-induced intensity distortion.Both maps are converted from sRGB to linear space before fusion and converted back afterward.Source: §12 · p. 31
  • The direct-light weight W_direct is derived from per-pixel RGB illuminance, clamped at 0.05, and scaled by a tunable strength factor.This suppresses weak or noisy direct-light signals while preserving meaningful illuminated regions.Source: §12 · p. 31
  • The fusion I_blended = (1 - W_direct) · I_ambient, linear + W_direct · I_direct, linear preserves ambient filling while emphasizing stronger local illumination.Bright direct-light regions such as streetlight highlights are dominated by directional lighting, whereas unlit regions retain the ambient background.Source: §12 · p. 31
  • This illumination strategy supports physically plausible light mixing and spatially adaptive transitions without over-saturation or unnatural scene changes.It respects the intensity hierarchy between global ambient illumination and localized direct lighting.Source: §12 · p. 31

13 Text Prompt Design and Templates

  • The final text prompt combines a base prompt with a conversion instruction, with the base prompt generated by Qwen3-8B-Instruct [3] from the original video frame.The description covers road layout, surrounding elements, objects, motion and traffic density, atmosphere, and visible traffic lights or signs, constrained to 2–4 sentences.Source: §13 · p. 31

13.1 Night Scene Conversion Instruction

  • Night-scene instructions specify midnight darkness, headlights, warm streetlights, photographic clarity, true-to-life colors, and realistic shadows.The design emphasizes time specificity and interactions among headlights, streetlights, darkness, and shadows.Source: §13.1 · p. 32

13.2 Fog Scene Conversion Instruction

  • Fog-scene instructions combine dense mist, headlights, diffused daylight, low visibility, atmospheric perspective, volumetric lighting, and subtle light glows through haze.These elements target realistic fog opacity, atmospheric scattering, and physically consistent illumination.Source: §13.2 · p. 32

13.3 Snowy Scene Conversion Instruction / 13.4 Rainy Scene Conversion Instruction

  • Snowy and rainy instructions encode scenario-specific surface and weather cues, including snow accumulation and flakes, wet-surface reflections, rain ripples, raindrops, and diffuse overcast light.Both maintain photographic clarity and realism while balancing cinematic presentation with physically grounded weather appearance.Source: §13.3 · p. 32, §13.4 · p. 33

14 VidRefiner Architecture and Configurations

VidRefiner postprocessing balances physical-edit preservation with artifact repair through controlled diffusion settings. Its collapsed searching-space scheme defines the denoising variables and generates refined videos from input video, prompts, and configurable editing strength and diffusion steps.

Configuration

  • Postprocessing uses α = 0.4, γ = 6, T = 20, WAN-FUN 2.2-5B’s default scheduler [56], and 1280×704 resolution for efficient refinement.These settings inject moderate noise, emphasize prompt adherence, and limit computational cost while preserving the underlying weather simulation.Source: §14 · p. 33

Collapsed Searching Space

  • The collapsed searching-space generation scheme controls injected noise through editing strength α, diffusion timesteps T, and a starting pivot timestep ts.It also defines latent representation z, timestep noise schedule σt, Gaussian noise ϵ, predicted noise ˆϵ, conditioning video cond, and refined output ˆv.Source: §14 · p. 33

Algorithm

  • Algorithm 1 implements postprocess(v, prompt, α, T) for generating a refined output video from the input video, prompt, editing strength, and diffusion-step count.The scheme is presented as pseudocode for collapsed searching-space generation.Source: §14 · p. 33

15 Extended Quantitative Evaluations

Extended evaluations assess structural, geometric, edge, temporal, and perceptual consistency using projected 2D IoU, depth and edge alignment, FVD, and DOVER. AutoWeather4D achieves the best CLIP score and vehicle geometric and perceptual alignment, while several metrics require interpretation because physically grounded edits alter scene structure.

AABB Bounding-Box Projection and IoU Formulation

  • Projected 2D AABB IoU strictly proxies structural preservation but can decline بسبب geometric degradation or reduced OvMono3D detector performance in severe adverse weather.Ground-truth and predicted 3D boxes are projected with the same formulation before IoU computation.Source: AABB Bounding-Box Projection and IoU… · p. 34
  • AutoWeather4D achieves the best CLIP score and strongest vehicle geometric and perceptual alignment among baseline approaches.These results indicate the strongest instruction adherence in the reported comparison.Source: AABB Bounding-Box Projection and IoU… · p. 34
  • Depth alignment is evaluated with si-RMSE across 120 sequences spanning fog, rain, snow, and night, with lower values indicating better alignment.The reported minor depth-alignment drop mainly results from explicitly generating falling snow and rain, which most baselines cannot geometrically represent.Source: AABB Bounding-Box Projection and IoU… · pp. 34–35
  • AutoWeather4D’s lower Edge F1 reflects physically grounded geometry and lighting changes rather than edge-preservation failure.Snow, puddles, wetness, and fog/night illumination alter silhouettes and edges, while original-video edge controls favor baselines that preserve the input distribution.Source: AABB Bounding-Box Projection and IoU… · pp. 35–36
  • FVD measures distributional similarity between real and generated videos, characterizing temporal coherence and visual realism, while DOVER serves as a perceptual overall-quality metric.DOVER combines its components as Q_pred = 0.428Q_pred,A + 0.572Q_pred,T; these perceptual metrics serve only as reference because visually appealing results may fail to adhere to editing instructions.Source: AABB Bounding-Box Projection and IoU… · p. 36

16 Comprehensive ablation studies

The ablations validate module necessity, module synergy, intra-module design choices, and the error tolerance of AutoWeather4D’s G-Buffer Dual-Pass Editing. Internal PSNR measures deterministic spatial divergence from the full pipeline rather than perceptual quality, while targeted studies identify VidRefiner strength 0.4 as the preferred setting and show compensation under severe extraction errors.

16 Comprehensive ablation studies

  • Internal PSNR compares each ablated variant with the deterministic full-pipeline output to quantify module-induced spatial divergence, not perceptual quality or physical realism.Lower internal PSNR indicates greater structural and photometric deviation, thereby measuring the module’s intervention magnitude and functional necessity.Source: §16 · p. 37

16.1 Single-module ablation studies

  • Single-module ablations compare each target component with and without that module to isolate its independent contribution, measure effectiveness, and avoid confounding from other modules; configurations are shown in Tab. 8.Source: §16.1 · pp. 37–38

16.2 Synergistic-modules ablation studies

  • VidRefiner is tested as a universally compatible optimizer with individual and multi-module subsets to assess whether it preserves module-specific effects and improves overall spatial quality.The synergy studies use PSNR against complete-framework outputs across combinations with functional modules ①–④, including 2-module and 3-module subsets.Source: §16.2 · p. 41, Table 9 · p. 41, Table 10 · p. 45

16.3 Intra-module Ablation Studies

  • Intra-module studies separately ablate rain and snow geometry components, vary active illuminants from 0, 4, and 8 to all sources, and evaluate their effects on controllable weather editing.Rain components comprise standing and falling water; snow components comprise accumulated snow, falling snowballs, and grid-based snow; Light Pass ablations assess cumulative lighting fidelity and photorealism.Source: §16.3 · p. 48, Figure 17 · p. 48, Figure 18 · p. 49
  • VidRefiner strength 0.4 balances visual quality and physical plausibility, whereas strength 0.6 reaches PSNR=10.36 but changes a black car to white; strength 0.4 achieves PSNR=10.19 without this semantic error.The study therefore adopts 0.4 as the optimal VidRefiner strength.Source: §16.3 · pp. 48–49

16.4 Case Study: Error Tolerance of G-Buffer Dual-Pass Editing

  • Under extreme low light, explicit sky-masking anchors macrostructure while VidRefiner absorbs high-frequency intrinsic noise into plausible wet-surface reflections, preventing cascading errors.The case involves high-ISO input, catastrophic sky-depth failure, and noisy normal/roughness maps; the decoupled compensations preserve macroscopic physical constraints.Source: §16.4 · pp. 52–53

17 Extensive Qualitative Results

Across sunny-to-snowy, nighttime, foggy, and rainy conversions, AutoWeather4D qualitatively avoids inherited shadows, synthesizes localized illumination, and preserves dynamic structures through decoupled G-buffer passes. It also enables geometry-anchored surface effects, but currently darkens self-illuminating objects such as traffic lights.

Mitigating Illumination Entanglement

  • AutoWeather4D avoids inherited directional shadows by separating material modification in the Geometry Pass from light-transport recalculation in the Light Pass.Baselines often retain sharp source shadows as darkened textures because they lack explicit intrinsic decomposition, whereas AutoWeather4D produces more physically plausible diffuse illumination.Source: §17 · pp. 53–54

The Missing Active Illumination Problem

  • AutoWeather4D injects volumetric headlight cones with plausible falloff and reflections while preserving unlit background structures as dark silhouettes.This contrasts with global color shifts and uniform darkening that fail to model local active illumination in nighttime driving scenes.Source: Figure 23 · p. 54, §17 · p. 55

Shadow Inheritance and Dynamic Reconstruction in Fog

  • In fog, AutoWeather4D erases spurious sunny-scene shadows and injects physically plausible headlights, while feed-forward G-buffer extraction preserves sharp dynamic vehicle geometry without motion ghosting.WeatherEdit’s optimization-based dynamic Gaussian splats can collapse on fast-moving objects, producing severe ghosting and blurred vehicle boundaries.Source: Figure 24 · p. 55, §17 · p. 56

Shadow Inheritance and Active Precipitation in Rain

  • In rainy conversions, AutoWeather4D recalculates diffuse ground illumination instead of inheriting sharp car shadows from sunny footage.WAN-Fun, Cosmos-Transfer, and WeatherEdit commonly retain source lighting through global shifts or static road darkening.Source: §17 · pp. 56–57

Dynamic Structure Preservation and Surface Interaction

  • Feed-forward G-buffer extraction preserves moving-object boundaries and enables localized surface interactions, avoiding the motion instability and limited geometric control of optimization-based 4DGS methods.WeatherEdit exhibits motion ghosting and geometric instability, while its image-space diffusion background editing limits geometry-aware water dynamics.Source: Figure 26 · p. 57, §17 · p. 58

Shadow Inheritance and Active Precipitation in Rain

  • AutoWeather4D uses world-space procedural modeling to create geometry-anchored puddles and dynamic ripples, whereas baselines often produce post-rain roads or spatially distorted water effects.The Geometry Pass supports explicit surface interaction through geometric grounding and surface-normal perturbation.Source: Figure 26 · p. 57, §17 · p. 58

Failure Case in Preserving Light-Emitting Objects

  • AutoWeather4D can unintentionally darken traffic lights because its G-buffer omits an emissive channel and bakes their brightness into albedo.Adding a dedicated emissive channel, potentially aided by traffic-signal detectors or VLMs, is proposed to preserve their brightness.Source: §17 · pp. 58–59, Figure 27 · p. 58

Related papers

Loading 2603.26546v3…