PaperlayerView exact revision on arXiv

Source-linked AI summary

Metric Scale for Insta360 Structure-from-Motion from the Camera's Embedded IMU

Davide Cuttini, Diego Di Tommaso, Alessio Antoniacomi, Luca Simonetti

arXiv:2610.35554v2cs.CV

TL;DR

SfM from dual-fisheye video does not determine absolute scale, and the current pipeline relies on smartphone poses for metric scale. This paper estimates scale from the camera’s embedded IMU by aligning its measurements with SfM poses. Segment-wise alignment removes the joint formulation’s bias, and the evaluation reports agreement with independent references while identifying limits to attributing residual differences.

  • Problem

    SfM recovers scene structure and trajectory only up to a similarity transform, so the mapping pipeline needs an external measurement to supply metric scale.

    Source: §1 · p. 1

  • Method

    The method estimates camera–IMU calibration and applies a batch linear visual-inertial alignment to SfM poses, solving per contiguous run and averaging the resulting scales.

    Source: §3.1 · p. 3, §3.4 · p. 4

  • Results

    The inertial scale is 2.4 times more accurate than the smartphone reference on the LiDAR surveys.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The smartphone capture step can be removed from the procedure, with gravity magnitude, timing drift, and per-run scale dispersion available to reject failed alignments.

    Source: §7 · p. 16

    The production-survey scale difference cannot be assigned to either reference without a third metric reference; LiDAR drift over long paths is also uncharacterised.

    Source: §6.4 · p. 15, §7 · p. 16

Abstract

from arXiv · show

Structure-from-motion (SfM) from the dual-fisheye video of a consumer 360-degree camera recovers geometry only up to a similarity transform; in the pipeline considered here, metric scale is obtained from smartphone ARKit or ARCore poses captured alongside the camera. This paper evaluates a self-contained alternative: the IMU embedded in the Insta360 X5 and X6, recorded at 1 kHz. The sensor is characterised (noise density, camera-IMU synchronisation, accelerometer scale factor, bias stability), the camera-IMU rotation and frame timing are estimated from the data, and the scale is recovered by a linear visual-inertial alignment over the SfM poses of a whole survey. The standard formulation, with gravity and accelerometer bias shared across the survey, overestimates the scale by 3.4% on average; estimating each contiguous segment independently and averaging removes the bias. On nine surveys with a co-mounted LiDAR scanner the inertial scale deviates from the LiDAR by 0.81% in mean absolute error, against 1.94% for the smartphone scale; on printed fiducial markers both agree within the 1% uncertainty of the target. On the reference trajectories of thirty sequences of a public benchmark, brought to scales of 1/40, 1 and 40, the estimator recovers the metric scale with a mean error of -0.07% and a standard deviation of 1.28%. On fifty production surveys where the smartphone is the only reference, the two scales agree to a standard deviation of 2.3%, the inertial scale being 0.3% lower on average; the difference is uncorrelated with run length, grows with the extent of the smartphone anchor, and cannot be attributed to either reference. An ablation over window construction and a comparison with monocular metric-depth networks complete the study.

1 Introduction

SfM leaves absolute size undetermined, and the current pipeline transfers metric scale from smartphone poses. This paper evaluates whether the Insta360 camera’s embedded IMU can provide that scale instead.

Problem

  • Without a known-length baseline, SfM recovers shape and trajectory but not absolute size, so the mapping pipeline needs an external metric reference.The current pipeline registers smartphone ARKit or ARCore poses to fisheye frames and transfers their scale to the model.Source: §1 · p. 1

Motivation

  • The smartphone reference adds a second device and capture procedure and extrapolates from a few metres of poses to surveys spanning hundreds of metres.Across nine LiDAR surveys, smartphone-anchored models deviate from LiDAR by 1.9% mean absolute error; individual surveys reach 4.4%.Source: §1 · p. 1

Approach

  • The camera’s X5 and X6 gyroscope and accelerometer are sampled at 1000 Hz, enabling scale recovery by matching SfM trajectory acceleration with measured specific force while accounting for gravity.This is visual-inertial initialisation, applied here offline to sparse SfM poses over surveys lasting minutes rather than to an online tracker.Source: §1 · p. 2

Contributions

  • The paper characterises the embedded IMUs, estimates camera–IMU rotation and timing, and evaluates inertial scale against four independent references.It also finds that joint whole-survey alignment overestimates scale, while segment-wise estimation followed by averaging removes the bias.Source: §1 · p. 2

2 Related Work

Related work supplies the linear visual-inertial initialisation, calibration, and sensor-characterisation foundations. The paper adapts these ideas to offline SfM surveys and considers alternative metric references.

Visual-inertial initialisation

  • Visual-inertial systems use a closed-form linear relation among camera displacement, velocity, gravity, and preintegrated acceleration, including ORB-SLAM and VINS-Mono.Unlike those systems’ few-second online initialisation, this work applies the formulation in batch to SfM poses over a whole survey.Source: §2 · p. 2

Camera–IMU calibration

  • Camera–IMU calibration methods include target-based continuous-time estimation; this work instead estimates rotation by hand-eye alignment and fits an affine time model from the survey.The rotation solution uses Kabsch’s method.Source: §2 · p. 2

References and reconstruction

  • Related scale references include single-image metric-depth networks [2] [27], known-size fiducials, and LiDAR-map registration.Sensor characterisation commonly uses Welch’s method for noise density and Allan variance for bias instability; SfM reconstruction draws on COLMAP-family pipelines.Source: §2 · p. 2

3 Method

The method aligns SfM poses with 1 kHz IMU measurements to estimate metric scale. It calibrates camera–IMU rotation and timing from survey data, solves a linear alignment per contiguous run, and uses run-level checks to assess reliability.

3.1 Problem statement and notation

  • Each run’s scale, gravity, accelerometer bias, and frame velocities are unknowns; the inputs are SfM camera poses and 1 kHz gyroscope and accelerometer measurements.Camera–IMU rotation and the mapping from video frame indices to the IMU clock must be determined before alignment.Source: §3.1 · p. 3

3.2 Sensor data and timing model

  • A 0.1% error in nominal frame timing can shift video–IMU correspondence by 0.6 s over ten minutes and make the scale estimate fail, so the timing constants are estimated from data.The nominal interval is only the search starting point.Source: §3.2 · p. 3

3.3 Camera–IMU rotation and temporal calibration

  • Kabsch alignment of camera and gyroscope rotation vectors estimates camera–IMU rotation, while an affine timing model is fitted by minimizing median angular residual.A two-stage timing search is needed: a single unconstrained simplex can retain the nominal interval during brisk walking, producing scale errors of tens of percent; the first- versus last-quarter offset drift also diagnoses timing errors.Source: §3.3 · p. 3

3.4 Linear visual-inertial alignment

  • The method preintegrates IMU measurements between frames and solves the resulting position and velocity equations as sparse linear normal equations for scale, gravity, bias, and velocities.Bias dependence is retained to first order; velocities are chained only within contiguous runs, separated by gaps of no more than 1.2 s.Source: §3.4 · pp. 3–4

3.5 Segment-wise estimation

  • The joint solution overestimates scale by 3.39% on average because cross-run violations of shared gravity and bias assumptions can be absorbed into scale.Solving each run separately and averaging reduces mean absolute error from 3.43% to 0.85% on the same ten surveys; short runs bias low and long windows bias high, while typical 15–60 interval runs lie near the crossover.Source: §3.5 · p. 4

3.6 Quality criteria

  • Run-level gravity magnitude, scale dispersion, half-trajectory agreement, and residuals provide checks for timing, rotation, reconstruction, and internal scale drift.Runs outside 9.3–10.3 m s−2 gravity magnitude are discarded; surveys with excessive standard error, too few usable runs, or substantial half-trajectory disagreement are reported as failed.Source: §3.6 · pp. 4–5

4 Experimental Setup

The evaluation uses production SfM reconstructions and benchmark reference trajectories, with LiDAR, fiducials, and smartphone poses providing metric comparisons. The section describes the datasets and how the references are established.

4.1 Reconstruction pipeline and datasets

  • The estimator uses unchanged production reconstructions, with lens 1 selected before results were read; lens 0 is used as a robustness check.The pipeline uses COLMAP-format models and the public estimator implementation at default settings.Source: §4.1 · p. 5
  • Five survey sets cover nine co-mounted-LiDAR surveys, one fiducial survey, two production sets, and thirty public benchmark sequences.The benchmark estimator input is the supplied reference trajectory rather than a reconstruction; the production reconstructions use the pipeline’s native dual-fisheye frames.Source: §4.1 · p. 5, Table 1 · p. 5

4.2 Reference measurements

  • LiDAR scale is obtained by registering the reconstruction and scanner cloud, using trajectory initialization followed by scaled iterative closest point on dense clouds.The resulting factor converts reconstruction units to LiDAR metres.Source: §4.2 · pp. 5–6
  • Fiducial scale is measured by triangulating known-size ArUco markers from views within 50 degrees of the optical axis; smartphone scale comes from robustly registering ARKit or ARCore photo poses to the fisheye reconstruction.Views beyond 50 degrees are excluded because the reconstruction’s fisheye distortion model biases triangulated marker size by several percent.Source: §4.2 · p. 6

5 Results

Across the nine LiDAR surveys, the inertial scale was closer to LiDAR than the smartphone reference, and the fiducial comparison placed both scales within the target’s uncertainty. Results also show that scale estimates depend on how runs are combined, while monocular depth networks vary substantially across scenes.

5.1 Sensor characterisation

  • The X6 gyroscope showed white noise above 100 Hz, while the X5 was attenuated by a firmware low-pass filter; accelerometer scale factor was correct to 0.03%.Video–gyroscope correlation peaked at −2 ms, within ±5 ms timing resolution; the sensor characterization used a separate X5 unit to establish bias-drift magnitude.Source: §5.1 · pp. 6–7, Figure 1 · p. 6, Figure 2 · p. 7

5.2 Co-mounted LiDAR

  • 0.81% mean absolute error for the inertial scale versus 1.94% for the smartphone reference across nine LiDAR surveys; inertial scale was closer on seven.The inertial scale’s mean error was +0.07%, with a 2.11% worst case; the smartphone reference’s mean error was −0.41%, with a 4.36% worst case.Source: §5.2 · pp. 7–8, Table 2 · p. 8

5.3 Fiducial markers

  • In this survey, correcting for printed content about 1% smaller than nominal, the inertial estimate was 0.989 and ARKit scale 1.000, both within the marker-derived interval of 0.987–1.005.The marker reference itself carries a 1% uncertainty.Source: §5.3 · p. 9

5.4 Production set A: twelve surveys

  • The twelve set A surveys had a mean inertial-to-ARKit scale ratio of 0.991 and a 3.0% standard deviation, but these joint-estimator results are an upper bound on dispersion.Only three surveys were recomputed segment-wise; the estimator-related portion of dispersion is removed by that formulation, but smartphone-reference dispersion remains.Source: §5.4 · p. 10, Figure 6 · p. 10

5.5 Production set B: fifty surveys

  • Across fifty set B surveys, inertial and ARKit scales agreed to a 2.3% standard deviation, with the inertial scale 0.3% lower on average; neither reference was ground truth.The ratio correlated with smartphone anchor span (r = +0.52), but without a third reference the difference cannot be attributed to either scale.Source: §5.5 · p. 11

5.6 Ablation: window construction

  • Every windowed construction improved mean absolute error over the joint system by a factor of 2.5–4, while windowed errors ranged from 0.85% to 1.39%.Contiguous runs had the lowest error by a few hundredths of a percentage point and require no window length setting; longer sliding windows showed increasing bias.Source: §5.6 · p. 12, Table 3 · p. 12

5.7 Invariance to the input model

  • MoGe-2 and MoGe-3 depth estimates were scene-dependent: outdoor mean absolute biases were 11% and 12%, while indoor biases were 5% and 8%, respectively.The networks sometimes disagreed by up to 34% on the same scene, and the inertial scale stayed within 6% of ARKit across the ten surveys.Source: §5.7 · pp. 13–14, Figure 9 · p. 14

6 Discussion

The discussion separates the joint estimator’s bias from unresolved offsets between production references. It also revises quality checks around per-run estimates and identifies validation and method boundaries.

6.1 Bias of the joint formulation

  • 3.39% high on external ground truth; unbiased benchmark poses localize the shift to reconstructions, while bias drift is too small and varying effective gravity remains the candidate.The specific mechanism behind effective-gravity variation was not isolated.Source: §6.1 · p. 14
  • 15–60-interval contiguous runs lie near the crossover between low bias from short-window regressor noise and high bias from shared gravity in long windows.The per-run estimate is bounded by these opposite-sign effects.Source: §6.1 · p. 15

6.2 Offset on the production sample and the absence of ground truth

  • Where metres are available, inertial scale error stays below 1% mean absolute and 2.1% worst case; smartphone comparisons reach 3–4% worst case.The production-set difference averages 0.3% lower for the inertial scale, varies with smartphone-anchor extent, and cannot be assigned to either reference without a third metric measurement.Source: §6.2 · p. 15

6.3 Quality criteria

  • Quality checks moved to per-run quantities: gravity is checked run by run, halves compare mean per-run scales, and the joint bootstrap remains diagnostic.All fifty production surveys pass; time-offset drift, hand-eye residual, and per-run dispersion remain rejection criteria.Source: §6.3 · p. 15

6.4 Limitations

  • Validation remains bounded by uncharacterised long-path LiDAR drift, X5-only external-reference surveys, third-unit static characterisation, and no nonlinear refinement.A tape measurement of a known object is the stated way to validate either reference without assuming the other.Source: §6.4 · p. 15

7 Conclusion

The conclusion presents the embedded Insta360 IMU and existing SfM poses as a way to recover metric scale without smartphone data. Segment-wise alignment performs well against external references and benchmark trajectories, while the production-set offset remains unattributed.

7 Conclusion

  • 0.81% mean absolute error against LiDAR for inertial scale versus 1.94% for the smartphone reference; both references fall within the fiducial target’s 1% uncertainty.The segment-wise estimate reduces mean absolute error against external reference from 3.43% to 0.85%.Source: §7 · p. 15
  • On reference trajectories from thirty sequences of a public benchmark, the estimator was unbiased, with mean error −0.07% and dispersion 1.28% per sequence, whatever the input model’s scale.Source: §7 · pp. 15–16
  • Estimating timing from data and solving per contiguous segment are required; recovered gravity, timing-offset drift, and per-run dispersion help reject failed alignments.The paper identifies a third metric reference on production scenes and nonlinear refinement as open items.Source: §7 · pp. 15–16
  • On fifty production surveys, the references agree to 2.3% standard deviation; their 0.3% mean difference varies with smartphone-anchor extent and rig lens.Without ground truth on that set, the residual difference cannot be assigned to either estimate.Source: §7 · p. 16

Related papers

Loading 2610.35554v2…