PaperlayerView exact revision on arXiv

Source-linked AI summary

Splannequin: Freezing Monocular Mannequin-Challenge Footage with Dual-Detection Splatting

Hao-Jen Chien, Yi-Chuan Huang, Chung-Ho Wu, Wei-Lun Chao, Yu-Lun Liu

arXiv:2512.05113v2cs.CV

TL;DR

Monocular Mannequin-Challenge footage can produce ghosting and blur because sparse observations leave some Gaussians weakly supervised. Splannequin detects hidden and defective primitives and temporally anchors them; on a 10-video benchmark, it reports improvements in CQA and COVER when applied to D-3DGS.

  • Problem

    Sparse monocular observations leave occluded or out-of-view Gaussians without reliable supervision, making high-fidelity frozen-scene reconstruction from Mannequin-Challenge footage challenging.

    Source: §1 · p. 2

  • Method

    Splannequin detects hidden and defective Gaussians and anchors them to nearby well-supervised states in time.

    Source: §1 · p. 2, §5 · p. 4

  • Results

    On 10 real-world Mannequin-Challenge videos, Splannequin improves CQA by up to 243.8% and technical quality (COVER) by 339.85% when applied to D-3DGS.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The method supports cleaner static views from single-camera captures without architectural changes or inference overhead.

    Source: §1 · p. 2

    The method assumes nearly static scenes and fails under rapid, non-rigid changes, fast-moving shadows, illumination shifts, or large motions.

    Source: §7 · p. 8

Abstract

from arXiv · show

Synthesizing high-fidelity frozen 3D scenes from monocular Mannequin-Challenge (MC) videos is a unique problem distinct from standard dynamic scene reconstruction. Instead of focusing on modeling motion, our goal is to create a frozen scene while strategically preserving subtle dynamics to enable user-controlled instant selection. To achieve this, we introduce a novel application of dynamic Gaussian splatting: the scene is modeled dynamically, which retains nearby temporal variation, and a static scene is rendered by fixing the model's time parameter. However, under this usage, monocular capture with sparse temporal supervision introduces artifacts like ghosting and blur for Gaussians that become unobserved or occluded at weakly supervised timestamps. We propose Splannequin, an architecture-agnostic regularization that detects two states of Gaussian primitives, hidden and defective, and applies temporal anchoring. Under predominantly forward camera motion, hidden states are anchored to their recent well-observed past states, while defective states are anchored to future states with stronger supervision. Our method integrates into existing dynamic Gaussian pipelines via simple loss terms, requires no architectural changes, and adds zero inference overhead. This results in markedly improved visual quality, enabling high-fidelity, user-selectable frozen-time renderings, validated by a 96% user preference. Project page: https://chien90190.github.io/splannequin/

1. Introduction

Monocular Mannequin-Challenge footage offers a low-cost route to frozen-scene synthesis, but sparse temporal observations and unintended motion leave existing dynamic Gaussian methods prone to blur and ghosting. Splannequin targets these artifacts with hidden/defective Gaussian detection and temporal anchoring, improving reported quality on a new benchmark.

1. Introduction

  • Multi-camera bullet-time production can cost over $750k per shot, while monocular Mannequin-Challenge reconstruction is vulnerable to minor motion and sparse observations.The single-camera approach is presented as a low-cost alternative.Source: §1 · p. 2
  • Sparse monocular observations leave occluded or out-of-view Gaussians without reliable data, producing ghosting and blur in frozen renders.The problem is particularly common under predominantly forward camera motion.Source: §1 · p. 2
  • Splannequin detects hidden and defective Gaussians, anchoring hidden ones to well-observed past states and defective ones to more strongly supervised future states.It integrates through loss terms without architectural changes and adds no inference overhead.Source: §1 · p. 2
  • On 10 real-world Mannequin-Challenge videos, Splannequin improves CQA by up to 243.8% and technical quality (COVER) by 339.85% when applied to D-3DGS.The paper reports strong gains over state-of-the-art baselines.Source: §1 · p. 2
  • Figure 2 contrasts blurred faces and double contours from SC-GS [23], 4DGaussians [90], and D-3DGS [97] with Splannequin’s crisp, temporally consistent reconstruction.The Splannequin frame reveals hair strands and facial detail without ghosting.Source: Figure 2 · p. 2

2. Related Work

Prior work spans dynamic scene reconstruction, static-view artifact removal, 2D stabilization, and micro-motion analysis. Splannequin instead regularizes poorly supervised 3D Gaussians to suppress subtle motion and artifacts in near-static frozen-scene reconstruction.

2. Related Work

  • Mannequin-Challenge clips support depth-learning research, while human renderers reproduce articulation and bullet-time pipelines retain micro-motions.The passage positions Splannequin against work on motion-aware rendering and reconstruction, including BTimer [43] and Shape of Motion [86].Source: §2 · p. 2
  • Static-view methods address sparse-view robustness and artifacts, but their rigid-scene assumptions make minor dynamics problematic.The cited approaches include artifact removal, depth and multi-view priors, and Gaussian pruning.Source: §2 · p. 3
  • Dynamic NeRF and Gaussian methods model time-varying scenes, but their objectives preserve observed motion rather than optimize near-static scenes for rigid reconstruction.The paper contrasts its goal with methods including D-NeRF, 4D Gaussian Splatting, and Shape of Motion [86].Source: §2 · p. 3
  • Video stabilizers warp 2D frames, whereas the paper frames Gaussian splatting as a route to 3D stabilization and novel-view synthesis.The cited stabilizers use mesh warps, homographies, optical flow, or depth-aware schemes.Source: §2 · p. 3
  • Micro-motion methods either amplify or model subtle motion; Splannequin instead uses a confidence-weighted classifier to improve hidden and defective Gaussians for freeze-time synthesis.The passage distinguishes these approaches as operating in 2D or capturing fine dynamics without suppressing them.Source: §2 · p. 3

3. Background

Dynamic Gaussian splatting represents a scene with canonical Gaussians whose means and covariances are deformed over time. It learns these representations from observed images, but direct timestamp supervision makes the framework prone to artifacts in frozen-scene reconstruction.

3. Background

  • A deformation network predicts time-dependent changes to each canonical Gaussian’s mean and covariance, defining its state at a given timestamp.The represented scene consists of canonical 3D Gaussians with static means and covariances.Source: §3 · p. 3
  • The model minimizes photometric reconstruction loss between rendered and observed images at the corresponding camera poses and timestamps.The loss sums image-wise reconstruction errors across the observations.Source: §3 · p. 3
  • In the freeze-time view, hidden Gaussians have already passed out of view and defective ones are not yet well observed; Splannequin anchors them to supervised states from other times.Figure 3 depicts hidden primitives using past states and defective primitives using future states under forward camera motion.Source: Figure 3 · p. 4
  • Because this framework relies on direct supervision at each timestamp, it is prone to artifacts when used for frozen-scene reconstruction.The issue is specific to the paper’s reconstruction setting.Source: §3 · p. 4

4. Problem Definition

The goal is a high-fidelity static rendering from any input camera view at a user-selected timestamp. Sparse monocular supervision leaves some Gaussian states unstable, so the paper distinguishes two types of ill-supervised primitives.

4. Problem Definition

  • The task is to synthesize a high-fidelity static rendering from any camera view at one user-selected timestamp, effectively freezing motion.The input is a monocular Mannequin-Challenge-style video.Source: §4 · p. 4
  • A Gaussian receives training signal only when visible and contributing to an observed image; otherwise its deformation at that time gets no gradient, leaving unstable state estimates.This sparse supervision arises from occlusion or leaving the camera frustum.Source: §4 · p. 4
  • The space-time diagram contrasts the diagonal observed trajectory with the horizontal fixed-time render, where ill-supervised primitives create artifacts.Figure 4 illustrates hidden Gaussians outside the frustum receiving no supervision, unlike visible Gaussians rasterized into the image.Source: §4 · p. 4, Figure 4 · p. 4

4.1. Identification of Ill-supervised Gaussians

  • A hidden Gaussian lies outside the camera frustum, while a defective one remains in-frustum but contributes negligibly and receives a near-zero gradient.Defective cases can include distant, highly transparent, or occluded primitives; a Gaussian is well-supervised only when neither condition applies.Source: §4.1 · p. 4

5. Approach

Splannequin stabilizes ill-supervised Gaussians with temporal consistency anchors, then renders a static scene at a user-selected timestamp. Hidden Gaussians use well-supervised past states, while defective ones use well-supervised future states.

5. Approach

  • Splannequin classifies ill-supervised Gaussians as hidden or defective and anchors them to nearby, well-supervised states with temporal-distance weighting.The pipeline applies dual-detection losses before rendering a freeze-time video.Source: §5 · p. 4, Figure 5 · p. 5

5.1. Temporally-Anchored Regularization

  • The regularizer targets Gaussian parameters and samples reference timestamps during training, checking whether each sampled state is well-supervised.The parameters include position, covariance, opacity, and spherical harmonic coefficients.Source: §5.1 · pp. 4–5

5.1.1. Anchoring Hidden Gaussians

  • Hidden Gaussians use a well-supervised past state as an anchor, with influence down-weighted exponentially as temporal distance increases.A sampled reference is valid when it precedes the hidden timestamp; the consistency discrepancy uses an L1 or L2 norm.Source: §5.1.1 · p. 5

5.1.2. Anchoring Defective Gaussians

  • Defective Gaussians use a well-supervised future state as an anchor when the camera has not yet reached them.The method applies the same consistency loss used for hidden Gaussians.Source: §5.1.2 · p. 5, §5.1.1 · p. 5

5.1.3. Total Objective Function

  • The final training objective adds weighted hidden- and defective-state regularization losses to the standard reconstruction loss.The regularization terms are summed over instances with valid anchors; λ_hidden and λ_defective weight their respective losses.Source: §5.1.3 · p. 5

5.2. Fixed-Time Rendering

  • After training, the model renders the training poses at a user-selected timestamp to produce a static freeze-time video.Preserved temporal variation lets users scrub through time before selecting the desired moment.Source: §5.2 · p. 5

6. Experiments

Experiments evaluate Splannequin on synthetic and real-world freeze-time videos, testing visual quality, temporal consistency, efficiency, and user preference. Across these settings, the results support improvements from the method and the complementary roles of its regularization components.

6.1. Implementation Details

  • Training runs for 30,000 iterations, with hidden and defective losses introduced at iteration 10,000 and evaluated every 10 iterations.The implementation extends the dynamic Gaussian framework and applies anchoring only when valid supervised reference states are available.Source: §6.1 · pp. 5–6

6.2. Experimental Setup

  • The real-world benchmark contains 10 videos, 2,869 frames, and 361 fixed-time renderings, with less than 10% consistent visibility on average.Because ground-truth frames contain motions the method aims to suppress, evaluation primarily uses no-reference perceptual metrics.Source: §6.2 · pp. 6–7

6.4. Real-World Evaluation

  • Splannequin consistently improves the real-world baselines, with especially large gains in technical artifact suppression and on the lowest-quality frames, while rendering at over 280 FPS.The reported efficiency follows from running deformation once for a target timestamp.Source: Table 2 · p. 7, §6.4 · p. 8

6.3. Validation on Simulated Benchmark

  • On 10 synthetic scenes with known static ground truth, adding Splannequin to 4DGaussians improves all reference metrics.The results indicate reconstructions closer to ideal static geometry while preserving motions.Source: §6.3 · p. 7, Table 1 · p. 7

6.5. Ablation Studies

  • Removing either hidden or defective regularization reduces performance; confidence weighting preserves fidelity, while its absence can over-smooth frames.The hidden loss anchors past, out-of-view primitives; the defective loss anchors future, not-yet-clear primitives.Source: §6.5 · p. 8, Figure 9 · p. 8
  • In the user study, the method was preferred in 96% of comparisons for better visual appeal and fewer artifacts; 80% of generated results were perceived as more “perfectly frozen” than the original captures.Source: §6.5 · p. 8, Figure 10 · p. 8

7. Conclusion

Splannequin uses architecture-agnostic temporal anchoring to stabilize Gaussians in monocular Mannequin-Challenge footage. Evaluations report improved perceptual quality and real-time rendering, while the method remains bounded by its nearly static-scene assumption.

7. Conclusion

  • Splannequin is a strategy for synthesizing high-fidelity frozen scenes from monocular Mannequin-Challenge footage.Its architecture-agnostic regularization stabilizes Gaussians through temporal anchoring.Source: §7 · p. 8
  • Evaluations report superior perceptual quality and real-time rendering, supporting user-selectable freeze-time reconstruction from consumer video captures.The method is described as eliminating artifacts.Source: §7 · p. 8
  • The method assumes nearly static scenes and fails under rapid non-rigid changes; fast-moving shadows, illumination shifts, and large motions can lack reliable temporal anchors.Motion thresholds and frame-position dependence remain for future work.Source: §7 · p. 8

Related papers

Loading 2512.05113v2…