Source-linked AI summary
Revisiting Ground-Truth Synthesis from High-Speed Video: Exact Validity Conditions and an Audited Consumer Capture Corpus
Abdullah Al Shafi, Sumaiya Rahim Suma
TL;DR
Synthetic blur labels rely on an often unstated assumption that the central frame matches the exposure center. The paper derives exact validity conditions and audits consumer-phone timing; it finds both a fixed parity offset and hidden zero-mean timing irregularities.
Problem
Synthetic deblurring data commonly pairs averaged high-frame-rate frames with the central frame, but the assumption that this frame centers the exposure is rarely stated.
Source: §1 · p. 1Method
The paper derives centroid-misalignment conditions under locally linear motion, audits smartphone frame-duration metadata, and provides a pre-synthesis checker.
Source: §3 · p. 3, §7 · p. 10Results
19 of 51 nominal-240-fps smartphone clips were captured near 176 fps; unbiased labels require odd windows and equal sample durations.
Source: §1 · p. 1, §7 · p. 10Takeaways & Limitations
Dataset builders should use odd windows, verify durations from container metadata, report offset spread, and retain timestamps to repair interpolation labels.
Source: §7 · p. 10The capture audit covers one device, one operating-system version, and two sessions, leaving session-level evidence effectively based on two samples.
Source: §6 · p. 10
Abstract
from arXiv · showhide
Motion-deblurring datasets are commonly synthesised by averaging $N$ consecutive high-frame-rate frames and labelling the result with the central frame. We show that this label is unbiased for every capture timing only when the window is odd and the frames' sample durations are equal. An even window shifts every label by a fixed fraction of the blur length, even under perfect timing. Unequal durations are subtler: their mean misalignment is zero, so dataset statistics cannot reveal them, yet when the offset cannot be read from the blur they convolve the supervision rather than adding noise to it. Auditing 51 smartphone clips recorded at a nominal 240 fps, we find 19 captured near 176 fps, a behaviour recorded only in the container's timing tables. In a controlled test the parity choice costs a fitted linear deblurring filter far more than these timing irregularities do, and interpolation labels, unlike blur labels, can be repaired with the true frame times. We provide a tool that checks both conditions without decoding, and will release the clips and their timing tables.
1 Introduction
The central-frame label is unbiased only under an odd window with equal sample durations. Even windows impose a fixed offset, while unequal durations can produce zero-mean temporal misalignment that blurs supervision.
1 Introduction
- An odd window with equal sample durations guarantees an unbiased label, and both conditions can be checked before synthesis.Source: §1 · p. 1
- An even window shifts every pair by 1/(2(N −1)) of the blur length, or 7.1% at N = 8, even with perfect timing.The offset is in the same direction in every pair.Source: §1 · p. 1, Figure 1 · p. 2
- In 19 of 51 nominal-240-fps smartphone clips, capture was near 176 fps; the zero-mean timing offsets convolve supervision rather than merely adding value noise.The container’s sample-duration table is the only way to reveal the held intervals.Source: §1 · p. 1, Figure 1 · p. 2
- The paper combines exact validity conditions, a corpus audit, consequences for deblurring and interpolation labels, and a tool for checking conditions before dataset construction.Source: §1 · p. 2
- The validity condition assumes locally linear motion, while the audit covers one device across two recording sessions; the authors do not claim benchmark numbers are invalidated.Source: §1 · p. 2
2 Related work
Prior synthetic blur pipelines average high-frame-rate frames, but their source frames are assumed to be uniformly spaced without being checked. Related work also spans parity choices, temporal-label assumptions, and audits of benchmark construction.
The averaging convention
- The established averaging recipe synthesizes blur from high-frame-rate frames, and later pipelines add processing or avoid synthesis, but synthetic pipelines do not check source-frame spacing.The convention is associated with Nah et al. (2017) and Su et al. (2017); later examples include Rim et al. (2022).Source: §2 · p. 2
Window parity in practice
- Published pipelines use both parities; for N = 30, Mahmud et al. (2026) give an offset of 1/58 = 0.017 of blur length, about 0.7 pixels on a 40-pixel streak.The authors present these cases to show that the condition applies in practice, not to question the studies’ results.Source: §2 · p. 2
Temporal labels and pipeline audits
- Interpolation-label studies use normalized time, distance-based indices, or acceleration models while assuming timestamps are correct; this paper instead questions their timing integrity.Kiefhaber et al. (2024) question interpolation-ground-truth content, while this work questions its construction.Source: §2 · p. 2
- Apple documents Auto FPS lowering 30- and 60-fps video to 24 fps in low light, but no documented high-frame-rate capture reduction.Source: §2 · p. 3
3 When is the central-frame label unbiased?
A central-frame label is unbiased exactly when the averaged frame times are symmetric around it; for every capture timing, that guarantee requires an odd window and equal sample durations. Even-window parity creates a systematic offset, while zero-mean timing irregularities can blur supervision, and a controlled test finds the parity cost larger for a fitted linear filter.
3 When is the central-frame label unbiased?
- Unbiasedness requires the designated frame to coincide with the mean sample time; equal durations guarantee this exactly when the window is odd.With unequal durations, either parity can be misaligned, except in balanced timing cases; guaranteeing unbiasedness requires both odd parity and equal durations.Source: §3 · pp. 3–4
- Under perfect timing, an even window shifts every label by 1/(2(N −1)) of the blur length, and dataset averaging preserves that offset.With random motion direction, the L2 optimum averages oppositely shifted labels into a faint double image.Source: §3 · p. 3
- Unequal durations can produce zero-mean temporal misalignment, yet the expected label is a clean frame convolved with reflected jitter density.The resulting L2 optimum is a blurred target, not the correct frame with noisy intensity.Source: §3 · pp. 3–4
- When an estimator cannot read the temporal offset from the blur, jitter of standard deviation σδ limits supervision sharpness regardless of sample size.A builder can estimate the misalignment scale from the measured hold probability; a hold may leave a gap that makes its offset readable.Source: §3 · pp. 3–4
3.1 What the misalignment costs a fitted estimator
- 0.42 dB for an eight-frame window versus 0.05 dB for real hold patterns in a seven-frame window at 16 to 32 pixels of blur.The comparison measures PSNR loss against the true-centroid frame for a fitted linear deblurring filter.Source: Table 1 · p. 4, §3.1 · p. 4
- With random motion direction, the fitted filter learns no net shift, while fixing direction makes the parity offset learnable; the tested filter recovers none of the hold-induced offset.The fixed-direction filter reproduces most, but not all, of the shift because its ridge penalty and L2 objective trade shift against sharpness.Source: §3.1 · p. 4
4 iPhone-HighFPS: an audited consumer capture corpus
The iPhone-HighFPS audit uses container timing tables to identify two capture populations, only one of which satisfies the validity condition. It measures how the affected clips’ timing irregularities are distributed and compare with the paper’s predictions.
4.1 Corpus and timing extraction
- The audit covers 51 camera-original QuickTime files from one iPhone across two sessions; exact sample durations were read from container timing tables without decoding.The timing analysis uses integer sample durations and distinguishes genuine variation from clock-rounding dither using cumulative-timestamp residuals.Source: §4.1 · p. 5
4.2 Two capture populations
- Affected clips had 20-tick intervals—exactly twice nominal—in 35.51% of intervals; across their duration, they held 356,656 frames versus 484,752 expected at 240 fps.Uniform clips had durations of 9, 10, and 11 ticks, with 10 ticks accounting for 99.46%.Source: §4.2 · p. 6
4.3 What the populations are not explained by
- The affected state begins 1.3–6.1 seconds into recording and then persists; auto-exposure and autofocus settling are offered only as a hypothesis.With two sessions, session-level comparisons are descriptive and carry no session-level statistical weight.Source: §4.3 · p. 7, Table 3 · p. 7
4.4 Placement of held intervals
- All 19 affected clips showed significant autocorrelation; holds clustered into short runs separated by clean stretches, with a recurring period of 22.6 frames.A cadence at that period carrying about eight holds reproduces both the 35.8% held fraction and the 176 fps rate, while placement remains irregular within a cluster.Source: §4.4 · p. 8
4.5 Predictions against measurement
- At N=7, 11, and 13, affected clips had σδ of 0.0255, 0.0228, and 0.0220; P3 was at worst 22% short at the 95th percentile for N=13.The uniform population’s median misalignment was 0.0000 at those window sizes, with standard deviation 0.0006–0.0009; the measured timing spread rose with N where P3 predicted a fall.Source: Figure 5 · p. 8, §4.5 · p. 8
5 Consequences for supervision
The consequences differ by label type: uneven sample timing can distort both deblurring and interpolation supervision, but true frame times can repair interpolation labels, not blur labels. The section also describes a pre-synthesis timing and parity check and the diagnostic scope of the audited corpus.
5 Consequences for supervision
- Uneven sample timing cannot be corrected by relabelling blur targets, while interpolation labels can use the true frame times instead of the designated midpoint.The proposed screening remedy is to synthesize blur labels only from windows whose durations pass the timing check.Source: §5 · p. 9, §5.2 · p. 9
5.1 Deblurring labels on this corpus
- At blur lengths of 10–12 pixels, transplanted hold patterns place blur labels a median 0.19 pixels from the true centroid, versus 0.75 pixels for an even window on the same footage.Affected clips are low-motion: median dense flow per 240 fps interval is 0.14 pixels, versus 1.18 in the uniform population; high-motion-window effects cannot be measured with this corpus.Source: §5.1 · p. 9
5.2 Interpolation labels are repairable
- At stride 8, the designated interpolation midpoint is near true normalised time 0.61 at the 75th percentile, although signed errors average to zero.For uniform footage, designated and true times agree to tick dither, with standard deviation at most 0.0012.Source: §5.2 · p. 9, Table 4 · p. 9
- Using true times gains 1.41 dB at stride 8 for a flow-based interpolator, which performs better in 85% of 277 triplets.The gain is reported for the time-conditioned warping of Jiang et al. (2018).Source: §5.2 · p. 9
5.3 Screening a corpus before synthesis
- Duration filtering retains 29% of affected 7-frame windows, 11% of 11-frame windows, and 3% of 13-frame windows; surviving windows are not a random sample of the footage.The screening tool reads sample-duration tables without decoding, checks duration-fit residuals and long durations, and checks parity against the intended window size.Source: §5.2 · p. 9, §5.3 · p. 9
- iPhone-HighFPS is a diagnostic instrument rather than a training corpus: its verified-uniform footage totals 26.8 minutes, too little to compete with established benchmarks.Subsampling uniform clips yields known sample times, while affected clips provide per-frame container timestamps for testing interpolation at arbitrary times.Source: §5.3 · p. 10
6 Limitations
The paper’s analytic condition assumes locally linear motion, and its empirical tests use a linear filter on synthetic scenes and one flow-based interpolator. The capture audit and several potential remedies also have limited scope.
6 Limitations
- The analysis assumes locally linear motion and tests the population-optimal predictor only with a linear filter on synthetic scenes; the interpolation gain comes from one flow-based interpolator.A deep network may read direction cues from the blur, potentially lowering hold-pattern costs and raising parity costs relative to the linear-filter test.Source: §6 · p. 10
- Capture measurements cover one device, one operating-system version, and two sessions, so the effective sample for session-level claims is two rather than 51 clips.The trigger is not observable from the files, and the paper does not quantify effects on benchmarks built from this footage or trace them through other datasets.Source: §6 · p. 10
- Three proposed remedies remain untested, and extracted image pairs lose the sample-duration table needed to check the timing condition.Rolling-shutter readout time is also absent from the metadata, placing intra-frame timing out of scope.Source: §6 · p. 10
7 Conclusion
Frame averaging is sound for every capture timing only with an odd window and equal sample durations, both checkable before synthesis. The paper recommends reporting offset spread and preserving timestamps; its released-footage safeguards are also specified.
7 Conclusion
- Frame averaging is sound for every capture timing only when the window is odd and sample durations are equal, and both conditions can be checked before synthesis.The conclusion recommends verifying durations, choosing an odd window, reporting the spread of label offsets rather than their mean, and preserving true timestamps with extracted pairs.Source: §7 · p. 10
Broader Impact Statement
- Released public-street clips carry only the video track, with faces and plates blurred, location metadata stripped, and a removal route under a permissive research licence.The analysis describes the commercial product’s measured behaviour without attributing intent or defect.Source: Broader Impact Statement · p. 10