Source-linked AI summary
FluidRain: Incompressible Rain Flow as an Attention Bias for Loop-in-Loop Video Deraining
Pu Wang, Yongcong Wang, Wenhao Li, Xiang Chen, Guangwei Gao, Jinshan Pan, Siyuan Yao, Shujun Fu, Zhuoran Zheng
TL;DR
Video deraining lacks explicit guidance from rain motion when aggregating neighboring frames: alignment may be unreliable under heavy rain, while implicit aggregation leaves motion structure to data. FluidRain projects estimated rain flow onto a divergence-free subspace and reuses attention across frames and scales; with 0.80M parameters, it remains competitive with substantially larger models on four benchmarks.
Problem
Video deraining methods either rely on alignment that can be unreliable under heavy rain or aggregate implicitly without explicit guidance from rain’s directional and temporal structure.
Source: §1 · p. 2Method
FluidRain estimates and projects rain flow onto the divergence-free subspace, then uses it to guide attention reused across neighboring frames and resolutions.
Source: §1 · p. 2Results
With 0.80M parameters, FluidRain achieves the second-highest PSNR on NTURain and RainSynComplex while maintaining strong performance on RainSynLight and RainSynAll100.
Source: §4.2 · p. 7Takeaways & Limitations
RainSyn-Gust tests changing rain direction, and Fphys supports no-reference assessment on real rainy videos.
Source: §1 · p. 2The temporal-flow premise is limited to short inter-frame intervals, over which underlying rain flow varies slowly while individual streak instances are renewed.
Source: §3.1 · p. 4
Abstract
from arXiv · showhide
Existing video deraining methods typically exploit neighboring frames through either explicit alignment or implicit spatiotemporal aggregation. Explicit alignment relies on accurate motion estimation, which can become unreliable under dense rain, while implicit aggregation avoids alignment but lacks explicit guidance on the directional and temporally coherent structure of rain. This leaves a gap between reliable temporal aggregation and explicit modeling of rain motion. To address these limitations, we propose FluidRain, a lightweight video derainer that uses divergence-free rain flow to guide Loop-in-Loop attention across scales and neighboring frames. Motivated by fluid mechanics, we model rain motion as a divergence-free image-space flow and use it to organize multi-scale and temporal aggregation. Specifically, FluidRain first estimates a rain-flow field for each frame and projects it onto the divergence-free subspace. The resulting flow steers window attention along rain streaks, enabling neighboring frames to be aggregated without explicit alignment. Since rain-flow structure is preserved across scales and nearby frames, Loop-in-Loop reuses the same attention operator across both dimensions, resulting in a three-frame model with only 0.80M parameters. Experiments on four benchmarks show that FluidRain remains competitive with substantially larger restoration models. We further examine how temporal evidence scales with different input views. To evaluate whether the model remains reliable when rain motion changes across frames, we introduce RainSyn-Gust, which injects controlled changes in rain-streak direction into existing benchmarks. We also develop a physics-based no-reference metric that evaluates real-rain removal without requiring clean targets.
1 INTRODUCTION
Video deraining methods either align neighboring frames or aggregate them implicitly, while rain motion itself remains largely unused to guide aggregation. FluidRain uses rain-flow structure to guide attention and determine where a shared operator recurs across frames and scales.
1 INTRODUCTION
- Rain streak direction and length vary across sequences because they depend jointly on wind, scene depth, and camera motion.The streaks arise from motion blur of falling raindrops carried by wind and imaged by a moving camera.Source: §1 · p. 1
- Explicit alignment can become unreliable in heavily rain-corrupted regions, whereas implicit aggregation avoids alignment but leaves rain-motion structure to be learned from data.Prior physical knowledge has mainly entered through rendering, regularization, or synthesis rather than guiding cross-frame aggregation.Source: §1 · pp. 1–2
- The central question is whether rain’s physical regularities can determine how and where a shared operator should recur in video restoration.Rain direction, scale, and temporal motion are described as structured properties that may guide aggregation and computation reuse.Source: §1 · p. 2
- FluidRain models within-frame rain motion as divergence-free flow, using it to guide anisotropic attention along streaks and aggregate frames without explicit alignment.The same operator is reused across neighboring frames and resolutions, yielding a three-frame model with 0.80M parameters.Source: §1 · p. 2
- RainSyn-Gust tests robustness to changing rain direction, while Fphys enables no-reference assessment on real rainy videos.These are introduced alongside experiments reporting competitive restoration performance with substantially fewer parameters.Source: §1 · p. 2
2 RELATED WORK
Video deraining has shifted from physics-driven and hand-crafted temporal priors toward learned temporal modeling, including attention and state-space approaches. Related work also incorporates physical structure into rain models and feature interactions, while looped architectures reuse operators across depth or refinement.
Video deraining
- Video deraining progressed from temporal filtering, low-rank priors, and recurrent architectures to learned feature interactions and state-space models.Recent approaches include RainMamba’s locality-aware scanning and VDMamba’s spatial-temporal branches with adaptive multi-frame fusion.Source: §2 · pp. 2–3
- Temporal correspondence methods use optical flow, deformable alignment, or alignment-and-fusion, while other work explores semi-supervision, test-time adaptation, and video diffusion for real-world generalization.The field increasingly emphasizes temporal modeling, multi-frame fusion, and generalization beyond synthetic rain.Source: §2 · p. 3
Physical priors and looped architectures
- Physical rain knowledge has informed formation models, decomposition constraints, synthesis, and feature-interaction biases; recursive networks reuse shared operators across depth or iterative refinement.Existing looped architectures mainly organize recurrence along network depth or iterative refinement.Source: §2 · p. 3
3 METHOD
FluidRain uses a divergence-free image-space rain-flow field to guide attention and reuses rain-aware operators across scales and neighboring frames. Its method links physical properties of rain to the flow estimator, attention bias, Loop-in-Loop architecture, and training objective.
3.2 RAIN-FLOW ESTIMATION AND ATTENTION BIAS
- FluidRain estimates a rain-flow field for each frame, projects it onto the divergence-free subspace, and uses its direction and magnitude to bias attention logits rather than warp features.The projected flow guides attention along rain structure.Source: §3 · p. 3, §3.2 · p. 4
3.1 PHYSICAL BACKGROUND AND RAIN-FLOW PROPERTIES
- Under the stated assumptions, rain flow is divergence-free, preserves direction across resolutions as streak extent rescales, and varies slowly while streak instances renew across nearby frames.Neighboring frames therefore provide distinct observations of the same scene under similar flow.Source: §3.1 · pp. 3–4
3.2 RAIN-FLOW ESTIMATION AND ATTENTION BIAS
- The attention bias uses local flow to set along- and across-streak directions and the effective along-streak range, with separate terms controlling normal and tangent directions.A larger ℓ_s broadens the attention range along the rain direction; the bias adds two learnable parameters per head.Source: §3.2 · p. 4
3.3 LOOP-IN-LOOP ACROSS SCALE AND TIME
- The outer loop reuses the same block stack at three scales, relying on preserved rain-flow direction while allowing streak extent to remain local to each scale.The scales are s ∈ {1, 2, 4}.Source: §3.1 · p. 4, §3.3 · p. 5
- The inner loop reuses one attention operator to read the centre and neighboring frames without explicit alignment, then predicts temporal fusion weights using the centre feature and rain-flow change.The fused feature completes the inner-loop update before the outer scale loop continues.Source: §3.3 · p. 5
3.4 TRAINING
- Training uses an ℓ1 reconstruction loss on the restored centre frame and includes rain-flow supervision; λ_flow is 0.01 for NTURain and 0 for the three RainSyn benchmarks.For RainSynLight, RainSynComplex, and RainSynAll100, the flow head is learned solely through the restoration objective.Source: §3.4 · p. 6
4 EXPERIMENTS
Across four benchmarks, FluidRain remains competitive with substantially larger models, and ablations identify complementary views and projected rain flow as key parts of its design. Further evaluations test changing rain motion, no-reference assessment on real rain, and downstream perception.
4.1 SETTINGS
- The datasets cover synthetic and real rain, light and heavy rain, motion, and veiling effects; RainSyn-Gust adds controlled changes in rain direction.The protocol reports PSNR and SSIM on synthetic benchmarks and uses Fphys for real rainy videos without ground truth.Source: §4.1 · p. 6
4.2 COMPARISON ON FOUR BENCHMARKS
- With 0.80M parameters, FluidRain achieves the second-highest PSNR on NTURain and RainSynComplex while remaining strong on RainSynLight and RainSynAll100.Qualitative comparisons show rain removal while preserving scene structures and fine details.Source: §4.2 · p. 7, Figure 2 · p. 7
4.3 ANALYSIS OF THE LOOP-IN-LOOP DESIGN
- Both changing the spatial view and reading a neighbouring frame consistently improve restoration, while repeating the centre frame gives marginal, inconsistent gains.Using both immediate neighbours yields larger improvements; five frames offer no consistent advantage, and untying scale operators increases parameters without improving the shared three-frame configuration.Source: §4.3 · p. 7, Table 2 · p. 7
4.4 ANALYSIS OF THE RAIN-FLOW PATHWAY
- With 6.6K additional parameters, the full rain-flow pathway improves PSNR by 0.67–1.40 dB across four benchmarks.Removing the divergence-free projection or rain-flow bias consistently degrades performance; the differential term has only a minor effect.Source: §4.4 · p. 7, Table 3 · p. 8
- Attention preferentially follows projected rain-flow direction; removing the bias weakens that preference, while rotating the flow redirects attention.The intervention examines attention for a representative query.Source: §4.4 · p. 8, Figure 3 · p. 8
4.5 ROBUSTNESS TO CHANGING RAIN MOTION
- On RainSyn-Gust, FluidRain remains stable under sudden 30° and gradual 3°-per-frame rain-direction changes, while per-frame models suffer larger relative PSNR degradation.The benchmark rotates the rain layer while keeping the clean background fixed.Source: §4.5 · p. 8, Figure 4 · p. 8
4.6 Fphys: NO-REFERENCE EVALUATION ON REAL RAIN
- Fphys correlates with PSNR on synthetic NTURain (Spearman ρ = 0.881), and FluidRain improves precision and recall over its flow-less backbone on real NTURain.Fphys evaluates the removed residual by balancing rain-consistency precision and rain-removal recall; it is used because real rainy videos lack clean references.Source: §4.6 · p. 8, Figure 5 · p. 9
4.7 DOWNSTREAM PERCEPTION UNDER RAIN
- On KITTI Tracking and KITTI 2015, FluidRain recovers 25% of detection degradation and 52% of segmentation degradation relative to rainy inputs.Despite having 0.80M parameters, it remains competitive with substantially larger restoration models on both tasks.Source: §4.7 · p. 9, Figure 6 · p. 9
5 CONCLUSION
FluidRain uses divergence-free rain flow to guide attention, reusing one operator across resolutions and neighboring frames. The three-frame model has 0.80M parameters, and experiments cover standard benchmarks, RainSyn-Gust, and real rainy videos.
5 CONCLUSION
- FluidRain estimates a divergence-free rain-flow field to guide attention and reuses the same operator across resolutions and neighboring frames.This is the Loop-in-Loop design.Source: §5 · p. 9
- With 0.80M parameters, FluidRain performs alignment-free restoration from three frames.The conclusion also reports experiments on standard benchmarks, RainSyn-Gust, and real rainy videos.Source: §5 · p. 9
- The authors say code, configuration files, and data-processing scripts will be released with the paper.Reported tables and figures are generated from stored experimental results using the released evaluation scripts.Source: REPRODUCIBILITY STATEMENT · p. 9