PaperlayerView exact revision on arXiv

Source-linked AI summary

ClearGS: Reliability-Aware Gaussian Splatting from Handheld Videos

Xuanzhi Liu, Xinyi Wu, Hang Pan, Wensi Huang, Zhenyao Wu, Ruize Han, Song Wang

arXiv:2609.31509v1cs.CVcs.AI

TL;DR

Handheld-video 3DGS must contend with uneven viewpoint coverage and mixed frame quality, while degraded frames can still provide useful geometric evidence. ClearGS combines graded view weighting with no-reference, render-guided restoration and repair consolidation. It achieves state-of-the-art overall performance on GS2E and GSOTM, with sharper boundaries and more stable textures.

  • Problem

    Handheld videos combine uneven viewpoint coverage with mixed frame quality, so degraded observations may offer needed geometric coverage while carrying unreliable appearance.

    Source: Problem Definition · p. 3

  • Method

    ClearGS uses RVA for graded frame supervision and coverage reactivation, then applies no-reference RIVR with a pose-aligned render and consolidates accepted repairs.

    Source: Introduction · pp. 1–2

  • Results

    ClearGS achieves state-of-the-art overall performance on GS2E and GSOTM, with sharper boundaries and more stable textures under mixed-quality observations.

    Source: Introduction · p. 1, Conclusion and Discussion · p. 8

  • Takeaways & Limitations

    ClearGS preserves geometric coverage from useful observations while limiting unreliable appearance supervision and reusing accepted repairs.

    Source: Introduction · p. 1

Abstract

from arXiv · show

We present ClearGS for 3D Gaussian Splatting (3DGS) from handheld videos with uneven viewpoint coverage and mixed frame quality. Rather than selecting frames with binary decisions, ClearGS uses Reliability-aware View Allocation (RVA) to assign graded raw-supervision weights based on appearance reliability, degradation risk, and geometric utility, while weakly reactivating useful suppressed frames to maintain trajectory coverage. Since weighting cannot restore details lost to blur or distortion, ClearGS further introduces Render-Guided In-Video Restoration (RIVR). The current 3DGS render provides a pose-aligned structural candidate, a frozen no-reference restoration expert restores the corresponding raw video observation without any clean reference image, and no-reference perceptual scores select among the render, restored observation, and high-frequency fused candidate. ClearGS then applies Full-Trajectory Repair Consolidation to revisit accepted repairs and preserve details introduced early. On GS2E and GSOTM, ClearGS achieves state-of-the-art overall performance, with consistent CLIP-IQA and MUSIQ gains and LPIPS reductions in most degradation settings, without paired sharp supervision or matched clean references.

Introduction

ClearGS treats handheld-video reconstruction as mixed-quality supervision, preserving useful geometric evidence without letting unreliable appearance dominate. It combines graded view weighting with no-reference restoration and repair consolidation; experiments report state-of-the-art overall performance on GS2E and GSOTM.

Introduction

  • Handheld videos vary in frame quality and viewpoint coverage, making binary keep-or-drop selection too coarse when degraded views may still provide useful geometry.Using all frames equally can transfer blur, distortion, and inconsistent appearance into the scene.Source: Introduction · p. 1
  • RVA assigns graded raw-supervision weights using appearance reliability, degradation risk, and geometric utility, while weakly reactivating useful suppressed frames in under-covered regions.This preserves trajectory coverage without letting degraded appearance dominate the initial reconstruction.Source: Introduction · pp. 1–2
  • RIVR uses a pose-aligned scene render and a frozen no-reference restoration expert to select repair evidence without clean references or paired sharp targets.It compares the render, restored observation, and detail-fused candidate using no-reference perceptual scores.Source: Introduction · pp. 1–2, Figure 1 · p. 2
  • ClearGS achieves state-of-the-art overall performance on GS2E and GSOTM, with sharper boundaries and more stable textures under mixed-quality observations.The method combines RVA, RIVR, and Full-Trajectory Repair Consolidation.Source: Introduction · p. 1
  • Full-Trajectory Repair Consolidation revisits accepted repairs to preserve details introduced early in the trajectory.It reuses the latest accepted repair at each target pose without sampling new poses or generating new candidates.Source: Introduction · p. 1

Related Work

Prior reconstruction methods model blur or camera motion, while view-selection methods typically make hard choices. ClearGS instead grades each frame’s appearance influence while retaining useful geometric evidence.

Scene Reconstruction from Blurry Images and Handheld Videos

  • Blur-aware radiance-field and 3DGS methods model image formation or camera motion, whereas ClearGS addresses varying frame quality and coverage along a fixed handheld trajectory.ClearGS controls each frame’s appearance influence while retaining observations with useful geometric evidence.Source: Scene Reconstruction from Blurry… · p. 2

Frame and View Selection for Reconstruction

  • View-selection methods target acquisition efficiency or under-observed geometry, but binary choices cannot exploit a degraded frame’s geometry without accepting its unreliable appearance.RVA assigns graded weights and weakly reactivates useful suppressed frames to avoid coverage holes.Source: Frame and View Selection for… · p. 2

3DGS Recovery from Imperfect Observations

Existing repair methods may rely on cross-view cues that share handheld-video artifacts. RIVR instead uses the current render as a structural candidate while restoring and selecting evidence from the corresponding raw observation.

3DGS Recovery from Imperfect Observations

  • Existing restoration methods often depend on nearby RGB references, sparse-view features, or depth-guided correspondences that may be unreliable when neighboring frames share degradation.Corrupted appearance can propagate across views.Source: 3DGS Recovery from Imperfect… · p. 2
  • RIVR restores the corresponding raw observation with a frozen no-reference expert, then selects among the render, restored observation, and detail-fused candidate.The current scene render provides a pose-aligned structural candidate, and accepted repairs are reused through Full-Trajectory Repair Consolidation.Source: 3DGS Recovery from Imperfect… · p. 2, Figure 2 · p. 3

Problem Definition

Handheld posed videos contain heterogeneous image quality and uneven viewpoint coverage. The goal is to reconstruct a scene that renders observed and novel views while preserving geometric coverage without writing unreliable appearance into the scene.

Problem Definition

  • A degraded frame may be unsafe as appearance supervision yet remain the only observation of a weakly covered region.Video frames can contain motion blur, defocus, distortion, or other degradations, while viewpoint coverage varies along the trajectory.Source: Problem Definition · p. 3
  • The reconstruction goal is a 3DGS scene that faithfully renders observed and novel views using only the input video, without paired sharp targets or pose-matched high-quality references.The central challenge is retaining coverage from degraded observations without directly embedding their unreliable appearance.Source: Problem Definition · p. 3

Overview

ClearGS establishes a reliable initial scene using appearance and geometric evidence while preserving necessary trajectory coverage. It then adds selected repaired observations and consolidates them across the trajectory to produce the optimized scene.

Overview

  • ClearGS follows a reliability-first principle, allocating supervision from appearance and geometric evidence while preserving necessary trajectory coverage to initialize G0.Source: Overview · p. 3
  • RIVR forms render, restored-frame, and fused candidates at each target pose, then appends the selected candidate as a pseudo observation alongside original video supervision.Source: Overview · p. 3
  • Full-trajectory repair consolidation gathers accepted pseudo observations across the trajectory for a final joint distillation pass that produces G⋆.Source: Overview · p. 3

Reliability-aware View Allocation (RVA)

RVA assigns supervision weights using both appearance reliability and geometric utility, rather than simply retaining or discarding frames. It also weakly reactivates suppressed frames in uncovered trajectory regions to preserve coverage.

Reliability-aware View Allocation (RVA)

  • RVA evaluates each frame from appearance and geometric perspectives and assigns reliability-aware supervision weights instead of retaining or discarding frames by a single criterion.Source: Reliability-aware View Allocation (RVA) · p. 3

Appearance Evidence

  • A frozen Qwen3-VL evaluator estimates appearance reliability and degradation risk, with high-reliability, low-risk frames treated as appearance-reliable.Table 1 summarizes the score aggregation weights, thresholds, and supervision policy.Source: Appearance Evidence · pp. 3–4, Table 1 · p. 4

Geometric Evidence

  • Geometric utility combines coverage gain, pose diversity, and visibility redundancy using COLMAP points and camera poses.Coverage gain emphasizes additional points beyond reliable-frame coverage, with larger weights for points observed by fewer COLMAP images.Source: Geometric Evidence · p. 4

Trajectory Coverage Compensation

  • When a pose-aware trajectory bin has no active frames, RVA reactivates a suppressed frame and includes compensated frames in the supervision set used to initialize G0.The selected frame is favored by trajectory centrality, and compensation uses a fixed weak-supervision weight.Source: Trajectory Coverage Compensation · p. 4, Table 1 · p. 4

Render-Guided In-Video Restoration (RIVR)

RIVR addresses details missing from the initial scene by restoring the raw observation at a target pose without a matched clean reference. It selects among render, restored, and fused candidates using no-reference perceptual scores.

Render-Guided In-Video Restoration (RIVR)

  • RIVR uses the current 3DGS render and corresponding raw observation at the same target pose, without requiring a matched clean image or paired sharp target.A frozen motion-deblurring expert processes the raw observation; the render supplies pose-aligned structure but is not treated as a clean reference.Source: Render-Guided In-Video Restoration… · p. 5
  • RIVR selects among render, restored-observation, and fused candidates with MUSIQ, using CLIP-IQA when MUSIQ scores are close.Source: Render-Guided In-Video Restoration… · p. 5
  • The selected candidate is appended as a pseudo observation for later distillation without assuming a clean target or cross-view reference.Source: Render-Guided In-Video Restoration… · p. 5

Joint Distillation

ClearGS distills accepted RIVR outputs with active real views selected by RVA, appending repairs rather than replacing captured observations. Full-Trajectory Repair Consolidation revisits accepted repairs after processing all frames.

Joint Distillation

  • Accepted RIVR outputs are distilled with the active real views selected by RVA.Source: Joint Distillation · p. 5
  • Repaired views are appended rather than replacing real observations, retaining captured video evidence while adding selected restoration details.Source: Joint Distillation · p. 5
  • After all frames are processed, consolidation reuses accepted repairs from the whole trajectory, retaining only the latest repair for repeated target frames and introducing no new poses or candidates.Source: Joint Distillation · p. 5
  • Figure 3 compares observed-pose renderings from all-view 3DGS and ClearGS Select-view initialization; red boxes mark regions of better-preserved thin structures, boundaries, and local appearance.Source: Figure 3 · p. 5

Experiment

ClearGS is evaluated on mixed-quality handheld-video benchmarks, with reconstruction, repair, and coverage-allocation contributions assessed separately. It improves perceptual quality over baselines while testing how view weighting and repair-candidate selection contribute.

Experimental Setup

  • GS2E includes ten scenes at each of three synthetic blur levels, while GSOTM covers four camera-motion degradation settings; sharp targets are reserved for final evaluation.All methods receive the same degraded posed observations; sharp ground truth and matched high-quality references are unavailable during allocation, reconstruction, repair selection, and distillation.Source: Experimental Setup · p. 5

Effect of Reliability-aware View Allocation

  • RVA-selected initial scenes reduce LPIPS by 0.011, 0.013, and 0.015 and improve MUSIQ by 1.71, 2.49, and 4.11 from slight to severe blur.CLIP-IQA is equal under slight blur and shows a small gain under severe blur; the all-view renders show blur on thin structures and boundaries.Source: Effect of Reliability-aware View… · p. 5

Quantitative Comparison

  • On GS2E, ClearGS reduces LPIPS vs DiFiX3D by 0.016, 0.030, and 0.038 and raises CLIP-IQA by 0.030, 0.032, and 0.040; on GSOTM it leads CLIP-IQA and MUSIQ in all four settings.Its GSOTM LPIPS is best in three settings and within 0.001 of the best under rolling shutter.Source: Quantitative Comparison · pp. 5–6, Table 3 · p. 6

Qualitative Comparison

  • In the shown scenes, ClearGS produces cleaner object boundaries and more coherent local textures than the input and restoration baselines.The comparisons include the common pre-repair render, DiFiX3D, GeoQuery, SyncFix, ClearGS, and sharp ground truth.Source: Figure 4 · p. 6, Qualitative Comparison · p. 7, Figure 5 · p. 7

Coverage Allocation in RVA

  • Coverage top-up improves GS2E PSNR by 0.538 and 0.512 and SSIM by 0.017 and 0.019 under medium and severe blur, while its GSOTM effects are mixed but include gains under RS and PN.Under slight blur, it slightly lowers PSNR with nearly unchanged SSIM; on GS2E it lowers LPIPS under slight and medium blur and matches the no-top-up variant under severe blur.Source: Table 4 · p. 6, Coverage Allocation in RVA · p. 7

RIVR Candidate Selection

  • Removing the raw-observation candidate consistently worsens LPIPS, CLIP-IQA, and MUSIQ on both datasets, while removing the fused candidate generally lowers MUSIQ and CLIP-IQA.The w/o Obs. ablation changes Restormer’s input from the raw observation to the current render, so it removes video evidence rather than the restoration module.Source: Table 5 · p. 6, RIVR Candidate Selection · p. 7

Conclusion and Discussion

ClearGS combines reliability-aware supervision, no-reference in-video restoration, and trajectory-wide repair consolidation for handheld-video 3DGS. Experiments on GS2E and GSOTM show improved perceptual quality, sharper boundaries, and more stable textures.

Conclusion and Discussion

  • RVA separates visual reliability, degradation risk, and geometric utility to assign graded supervision while preserving coverage in weakly observed trajectory regions.The method is intended for handheld videos with mixed frame quality and uneven trajectory coverage.Source: Conclusion and Discussion · p. 7
  • RIVR uses the current render as a pose-aligned structural anchor and the corresponding raw frame as restoration evidence, without matched clean references.Full-Trajectory Repair Consolidation revisits accepted repairs across the trajectory to preserve details learned early.Source: Conclusion and Discussion · p. 8
  • Experiments on GS2E and GSOTM show improved perceptual quality, sharper boundaries, and more stable textures; future work will study degradation-specific restoration and geometric checks for repaired evidence.Source: Conclusion and Discussion · p. 8
Loading 2609.31509v1…