Source-linked AI summary
TSRN-RTVD: Real-Time Video Deblurring System
Nikita Alutis, Danila Evsyukov, Egor Chistov, Mikhail Voronin, Evgeney Bogatyrev, Dmitriy Vatolin
TL;DR
Motion blur harms video quality and downstream perception, while computationally heavy deblurring can be impractical for real-time consumer-device use. TSRN-RTVD predicts camera-motion trajectories to guide recurrent video restoration, reaching 30.08 dB PSNR at 30 FPS on GoPro.
Problem
High-quality video deblurring methods can be computationally costly for real-time use on consumer devices, despite the need to restore frames as video arrives.
Source: §1 · pp. 1–2Method
Results
30.08 dB PSNR at 30 FPS on GoPro places TSRN-RTVD on the Pareto frontier within the real-time regime.
Source: §3 · p. 2Takeaways & Limitations
Abstract
from arXiv · showhide
As video capture moves to handheld and edge devices, motion blur from camera shake has become a pervasive degradation that lowers perceptual quality and harms downstream vision tasks. The strongest deblurring networks recover impressive detail, yet they remain computationally heavy and overwhelmingly complex, so their quality comes at a cost that consumer hardware cannot pay in real time. This gap between restoration quality and on-device speed is exactly what makes real-time deblurring difficult. We developed and implemented TSRN-RTVD, an efficient video deblurring system that explicitly reconstructs the underlying camera trajectory during exposure and uses the recovered motion to guide restoration. This approach turns the physical cause of blur into a signal that drives sharpening. Our system runs on a single consumer GPU and restores the video at 30 FPS while reaching 30.08 dB PSNR on the GoPro dataset. We demonstrate TSRN-RTVD on consumer devices with interactive side-by-side visualization of the blurry input and the deblurred output, live throughput, and an on-screen view of the recovered camera trajectory. Demo video is available at https://youtu.be/3alMwVrVALU.
1 Introduction and System Features
High-quality video deblurring is often too computationally costly for interactive, consumer-device use, while existing real-time methods leave a speed–quality gap. TSRN-RTVD targets that gap with explicit camera-motion guidance.
1 Introduction and System Features
- Motion blur degrades visual quality and downstream perception, while computationally intensive deblurring methods limit real-time use on consumer devices.Live preview, streaming, teleoperation, tracking, and detection are among the settings where restored frames must arrive with the video.Source: §1 · p. 1
- Existing real-time methods typically run at 40–100 FPS but restore less effectively than offline methods, leaving an underexplored speed–quality region.That region calls for near-offline restoration quality within a strict per-frame budget.Source: §1 · pp. 1–2
- TSRN-RTVD uses a compact trajectory signal to guide restoration, spending its compute budget on explicit motion rather than iterations or attention.It restores 720p video at 30 FPS on Apple Silicon M5 using a three-frame window with one-frame look-ahead.Source: §1 · p. 2
2 System Overview
TSRN-RTVD processes a three-frame window with one-frame look-ahead, predicts camera-motion trajectories, and uses them to align and fuse temporal features before reconstructing the center frame.
2 System Overview
- For each output, the pipeline uses a three-frame window, one-frame look-ahead, and predicted trajectories between neighboring frames to restore the center frame.The system receives a blurry video stream.Source: §2 · p. 2
- A lightweight network predicts a compact image-plane displacement signal approximating dominant camera-induced motion between neighboring frames.The trajectory is applied directly to the input video during inference.Source: §2 · p. 2
- Predicted trajectories shift neighboring-frame features and the recurrent state toward the current frame, approximating temporal alignment without dense optical flow, cost volumes, or deformable convolution.Source: §2 · p. 2
- Convolutional blocks fuse aligned temporal features, current-frame features, and motion embeddings; the decoder adds a predicted residual to the blurry input.The network learns how much neighboring and past information to use under different motion patterns, and cached encoded features are reused so each new frame is encoded once.Source: §2 · p. 2
3 Experimental Results
On GoPro, TSRN-RTVD reaches 30.08 dB PSNR at 30 FPS, placing it on the Pareto frontier within the real-time regime.
3 Experimental Results
- Evaluation uses the GoPro benchmark at 1280×720 on a single RTX 2080 Ti, reporting PSNR and throughput in frames per second.Source: §3 · p. 2
- A CUDA synchronization barrier around each forward pass measures latency across the full GPU workload rather than asynchronous kernel launches.The protocol follows MIMO-UNet [2].Source: §3 · p. 2
- 30.08 dB PSNR at 30 FPS places TSRN-RTVD on the GoPro speed–quality Pareto frontier inside the defined real-time regime.The comparison uses a 25 FPS threshold and reports quality close to slower offline systems.Source: §3 · p. 2
4 System Demonstration
The demonstration lets visitors test TSRN-RTVD on live handheld motion blur and inspect its output and throughput on consumer hardware.
4 System Demonstration
- Visitors connect a phone or webcam to a laptop with a low-end GPU and view live restored video with no perceptible delay.Source: §4 · p. 2
- Side-by-side blurry and deblurred views let visitors compare restoration directly on their own footage, including motion blur from camera shake or quick movement.Source: §4 · p. 2
- The interface reports live FPS so visitors can check that processing remains in the real-time regime on consumer hardware.Source: §4 · p. 2
5 Conclusion
TSRN-RTVD restores 720p video at 30 FPS with a one-frame look-ahead, using image-plane camera-motion shifts to align temporal features without dense optical flow.