PaperlayerView exact revision on arXiv

Source-linked AI summary

SCCM: Spherically Consistent Coarse Matching for ERP Dense Feature Correspondence

Gyeonggwan Lee, Eunsoo Im, Seunghwan Hong, Junghun Suh

arXiv:2609.36545v2cs.CV

TL;DR

Perspective-trained coarse matchers do not explicitly model ERP’s seam, latitude-dependent metric distortion, and nonuniform pixel area, despite dense panorama matches supporting pose, reconstruction, and SLAM. SCCM augments a chart-naïve coarse matcher with spherical positional attention for pairwise distortions and area-aware covisibility for per-pixel distortion. PCK@1° improves from 0.230 to 0.275 on Matterport3D when spherical priors are added under a fixed scaffold.

  • Problem

    Perspective-trained coarse matchers do not explicitly model ERP’s seam, latitude-dependent metric distortion, and nonuniform pixel area, despite dense panorama matches supporting pose, reconstruction, and SLAM.

    Source: §1 · pp. 1–2

  • Method

    SCCM augments a chart-naïve coarse matcher with spherical positional attention for pairwise distortions and area-aware covisibility for per-pixel distortion.

    Source: §1 · p. 3, §4.2 · p. 8

  • Results

    PCK@1° improves from 0.230 to 0.275 on Matterport3D when spherical priors are added under a fixed scaffold.

    Source: §1 · p. 3

  • Takeaways & Limitations

    SCCM transfers zero-shot to Stanford2D3D and leads on outdoor Holo360D when trained on that dataset.

    Source: §6 · p. 14

    Under synthetic pitch perturbations, SCCM is not tilt-equivariant; it scores 0.034 at 30° and remains best at every tested angle.

    Source: §6 · p. 14

Abstract

from arXiv · show

Dense feature matching between 360$^\circ$ panoramas underpins omnidirectional pose estimation, 3D reconstruction, and SLAM. Such panoramas are stored in the equirectangular projection (ERP), which unrolls the viewing sphere onto a flat chart and thereby introduces three distinct distortions -- a longitudinal seam (topology), latitude-dependent stretch (metric), and non-uniform pixel area (area) -- that the coarse stage of perspective-trained dense matchers does not model, so these matchers degrade systematically on ERP. We show that correcting the three distortions at the coarse-stage interfaces where they arise -- pairwise distortions in attention, per-pixel distortion in covisibility gating -- improves PCK@1$^\circ$ from 0.230 to 0.275 on Matterport3D under a fixed coarse scaffold, with the refiner architecture unchanged -- our central result. Concretely, SCCM (Spherically Consistent Coarse Matching) augments a chart-naive cross-attention/dual-softmax coarse matcher with two sphere-derived priors: Spherical Positional Attention (SPA) pairs a yaw-periodic RoPE (topology) with a tangent-plane bias (metric), and Area-Aware Covisibility (AAC) applies a pre-sigmoid log-area correction (area). The chart-naive scaffold serves as a controlled reference, separating the scaffold-replacement effect from the spherical-prior effect. Instantiated in the RoMa V1 framework with the same frozen encoder, refiner architecture, and loss, SCCM also outperforms the ERP-native EDM (0.163) and an ERP-retrained RoMa V1 (0.198) under a unified ERP dense matching protocol, while perspective-trained matchers largely fail on ERP. It further transfers zero-shot to Stanford2D3D and, when trained on outdoor Holo360D, leads there as well.

1 Introduction

ERP panoramas support dense correspondences for omnidirectional geometry, but their seam, latitude-dependent metric distortion, and nonuniform pixel area challenge perspective-trained coarse matchers. SCCM places sphere-derived corrections at the corresponding attention and covisibility interfaces; under a fixed scaffold, the priors raise PCK@1° from 0.230 to 0.275.

1 Introduction

  • Dense panorama correspondences provide wide-overlap constraints for relative pose, 3D reconstruction, and omnidirectional SLAM.The input panoramas are stored in equirectangular projection.Source: §1 · pp. 1–2
  • ERP’s seam, latitude-dependent stretch, and shrinking polar pixel area create distinct topology, pairwise-metric, and per-pixel-area challenges for coarse matching.The paper places pairwise corrections in attention and per-pixel correction in covisibility gating.Source: §1 · p. 2
  • The chart-naïve cross-/self-attention and dual-softmax scaffold serves as a controlled reference, while the encoder, refiner architecture, and loss remain unchanged.The comparison separates scaffold-replacement gains from spherical-prior gains.Source: Figure 1 · p. 2, §1 · p. 3
  • SCCM adds yaw-periodic RoPE and tangent-plane bias in SPA, plus pre-sigmoid log-area correction in AAC.These target topology, pairwise metric, and per-pixel area, respectively.Source: §1 · p. 3
  • PCK@1° rises from 0.230 to 0.275 when SCCM’s spherical priors are added under a fixed coarse scaffold.This controlled comparison isolates the priors from gains due to replacing the scaffold.Source: §1 · p. 3

2 Related Work

Prior approaches address panoramic geometry at different stages or for different tasks, while SCCM targets dense angular correspondence by placing spherical priors directly in coarse attention and covisibility gating.

2 Related Work

  • Perspective matchers and perspective-trained pointmap regressors do not explicitly model ERP geometry in their coarse stage; zero-shot MASt3R and VGGT fall below ERP-native EDM.The stated coarse-stage omissions include topology and latitude-dependent metric distortion.Source: §2 · p. 3
  • Sparse ERP matchers apply geometry at isolated keypoints, while dense matching exposes every pairwise matching score to chart distortion.CoMatch uses covisibility-guided token condensing and attention for perspective semi-dense matching; SCCM uses covisibility as an ERP area-corrected gate.Source: §2 · p. 3
  • EDM is an ERP-native dense matcher with spherical input embeddings and sphere-aware refinement, whereas SCCM injects geometry into coarse attention and covisibility logits.The related-work comparison distinguishes where the two methods inject spherical information.Source: §2 · p. 4
  • Spherical convolutions, tangent remapping, and panoramic transformers largely leave relative structure between two feature grids unaddressed.The paper contrasts these single-image geometry methods with SCCM’s priors for dense angular correspondence.Source: §2 · p. 4
  • Existing panoramic positional encodings handle seams or chart distortion in different ways, while SCCM places seam-periodic and latitude-aware priors directly in coarse attention and covisibility gating.The cited examples include Dense360, RoPE Rolling, SpheRoPE, and PanoFormer, which address other stated tasks.Source: §2 · p. 4

3 Preliminaries

The preliminaries map ERP coordinates to the viewing sphere and recall the planar mechanisms that SCCM later modifies: relative-position attention, additive position bias, and covisibility gating.

3 Preliminaries

  • An ERP pixel maps from normalized chart coordinates to longitude and latitude, then to a unit viewing ray on the sphere.The sphere’s area element is cos φ, and SCCM derives its priors from this map.Source: §3 · p. 4
  • RoPE rotates query and key vectors by position-dependent phases so their dot product carries relative-position information.SCCM later makes its longitudinal phase yaw-periodic across the ERP seam.Source: §3 · p. 4
  • CPB maps a continuous relative coordinate through a shared MLP to an additive scalar in the pre-softmax attention logits.SCCM later supplies a spherical log-map offset instead of a planar pixel offset.Source: §3 · p. 4
  • Covisibility gating uses a per-pixel matchability logit and sigmoid gate to down-weight pixels without valid matches before dual-softmax matching.AAC modifies this pre-sigmoid logit with a sphere-area correction.Source: §3 · p. 5

4 Method

SCCM augments a chart-naïve coarse-matching scaffold with spherical priors at the attention and covisibility interfaces. SPA corrects pairwise seam and metric distortions, while AAC corrects per-pixel area distortion before gating.

4 Method

  • The R1 chart-naïve scaffold combines cross-/self-attention with covisibility-gated dual-softmax and applies no spherical positional encoding, tangent-plane metric bias, or area correction.It serves both as a baseline and as the fixed scaffold for isolating spherical-prior effects.Source: §4 · p. 5
  • The SCCM study replaces RoMa V1’s Gaussian-process coarse stage with R1 and keeps the DINOv2-Large encoder, ConvRefiner, and loss unchanged across variants.The encoder is frozen, and the remaining modules are trained from random initialization.Source: §4 · p. 5

4.1 Spherical Positional Attention (SPA)

  • SPA combines yaw-periodic RoPE for seam consistency with tangent-plane bias for local pairwise metric correction in coarse attention.Integer longitudinal frequencies make the positional term periodic across a 2π seam wrap; TPB uses spherical log-map offsets in the CPB framework.Source: §4.1 · pp. 5–6, Figure 2 · p. 6
  • TPB projects each key’s spherical log-map offset into the query’s tangent basis and maps that offset to an additive pre-softmax attention bias.The tangent basis is aligned with ERP latitude and longitude directions; the offset depends on grid coordinates rather than image content or pose.Source: §4.1 · p. 7

4.2 Area-Aware Covisibility (AAC)

  • AAC adds a latitude-dependent log-area correction to the covisibility logit before the sigmoid, down-weighting pixel-overrepresented polar candidates.The gate filters spurious matches before dual-softmax and is passed to the refiner as the coarse certainty signal.Source: §4.2 · pp. 7–8, Figure 3 · p. 7
  • LAC uses the known ERP area element in an additive logit correction, with a positive learnable scalar initialized at 1 for each image side.The pole clamp is for numerical stability, and the learned correction strengths remain near the analytic value.Source: §4.2 · p. 8

5 Experiments

Under a unified ERP dense-matching evaluation, SCCM improves Matterport3D performance over the chart-naïve scaffold and tested ERP baselines, with ablations isolating the spherical-prior contribution. The evaluation also examines transfer, latitude and seam behavior, and downstream geometry.

5.1 Experimental Setup

  • Models share a 1 M-sample training protocol, frozen DINOv2-Large encoder, original RoMa V1 loss, and common data split and evaluator.Performance is measured using angular error on the sphere, with PCK@1° as the primary strict-precision metric.Source: §5.1 · pp. 8–9

5.2 Benchmark Comparison

  • 0.275 PCK@1° for SCCM versus 0.230 for the chart-naïve scaffold on Matterport3D, a +4.5 pp spherical-prior gain under the fixed scaffold.SCCM also exceeds ERP-native EDM (0.163) and ERP-retrained RoMa V1 (0.198) on Matterport3D.Source: §5.2 · pp. 10–11
  • SCCM exceeds every baseline on zero-shot Stanford2D3D and leads each metric on Holo360D after training with a shared budget.On Holo360D, its PCK@1° advantage is +2.6 pp over the chart-naïve scaffold and +3.5 pp over ERP-retrained RoMa V1.Source: §5.2 · p. 12
  • SCCM’s PCK@1° advantage grows toward the poles and its drop near the ERP seam is smaller than EDM’s.Fig. 4 analyzes Matterport3D’s full 15,682-pair test set on a common 448×896 grid.Source: Figure 4 · p. 10, §5.2 · p. 12

5.3 Ablation Study

  • On the fixed R1 scaffold, spherical modules add +4.5 pp PCK@1°; yaw-periodic RoPE contributes +3.7 pp, while TPB×LAC supplies the strict-precision step.TPB reduces the angular-error tail; neither TPB nor LAC improves PCK@1° in isolation.Source: §5.3 · pp. 12–13

5.4 Downstream Tasks

  • SCCM improves Pose AUC at every threshold over the chart-naïve scaffold and leads all seven reconstruction accuracy, completeness, and F-score metrics.Reconstruction triangulates matches under the ground-truth pose to isolate correspondence quality from pose-estimation errors.Source: §5.4 · p. 13

6 Conclusion

SCCM applies lightweight spherical corrections at coarse-stage attention and covisibility interfaces while preserving the encoder, refiner architecture, and loss. It outperforms tested ERP dense-matching baselines on Matterport3D, transfers zero-shot to Stanford2D3D, and leads on Holo360D when trained there.

6 Conclusion

  • SCCM outperforms tested perspective-trained, ERP-retrained, and ERP-native baselines on Matterport3D, transfers zero-shot to Stanford2D3D, and leads on Holo360D when trained there.The approach preserves the encoder, refiner architecture, and loss.Source: §6 · pp. 13–14
  • SCCM assumes gravity-aligned ERP and does not explicitly handle camera tilt; synthetic pitch perturbations sharply degrade all tested ERP matchers.SCCM PCK@1° falls from 0.273 to 0.142 at 10° pitch and 0.034 at 30°, though it remains best at every tested angle.Source: §6 · p. 14
  • SCCM has been trained only on Matterport3D and Holo360D, and leaves the refiner sphere-naïve.The frozen encoder also limits performance in texture-less, photometrically ambiguous, or low-overlap regions.Source: §6 · p. 14

A Implementation Details

Matterport3D models share a fixed retraining protocol, while Stanford2D3D evaluation uses a separate overlap-selected pair protocol. The latter retains 8,744 pairs from the stated overlap band.

A Implementation Details

  • All Matterport3D-retrained models use the same 1 M-sample protocol, official split, frozen encoder, loss, and input resolution.Inputs are resized to 448×896; the encoder is frozen DINOv2-Large and the original RoMa loss is unchanged.Source: A Implementation Details · p. 18, Table 1 · p. 18
  • The shared Matterport3D setup uses 54,015/4,625/15,682 train/validation/test pairs and a 448×896 input resolution.Training uses AdamW, a constant 1×10^-4 base learning rate with a 0.1 drop at step 112,500, and a budget of about 125K steps.Source: A Implementation Details · p. 18

B Stanford2D3D Test Protocol

  • Stanford2D3D preprocessing flips pose rotations to match the runtime ERP convention and converts 16-bit PNG depth values to meters using scale 1/512.The protocol uses 1,412 usable panoramas after dropping one truncated panorama.Source: B Stanford2D3D Test Protocol · p. 18
  • Stanford2D3D pair selection retains pairs with overlap scores in [0.30, 0.80], yielding 8,744 evaluation pairs.The protocol differs from EDM’s >0.50 criterion, which retains 3,460 pairs.Source: B Stanford2D3D Test Protocol · p. 19

C Downstream Evaluation

The downstream evaluation uses shared pose and reconstruction evaluators, and SCCM improves all five reported metrics over the chart-naïve baseline on Matterport3D and zero-shot Stanford2D3D. Qualitative reconstructions show closer alignment to ground truth on the presented examples.

C Downstream Evaluation

  • Pose AUC is evaluated on a common 11,574-pair subset, with a separate overlap-selected 9,268-pair subset reported for controlling pair difficulty.All methods use the same 8-point plus RANSAC pose estimator.Source: C Downstream Evaluation · p. 19, Table 2 · p. 20
  • On the three illustrated test pairs, SCCM’s triangulated clouds align more closely with ground truth than the EDM, ERP-retrained RoMa V1, and chart-naïve results.The panels show top-down X–Z projections; aggregate reconstruction metrics are reported separately.Source: Figure 1 · p. 20

D Performance Gains over the Chart-Naïve Baseline

  • SCCM gains 4.5 pp and 5.0 pp in PCK@1° over the chart-naïve baseline on Matterport3D and zero-shot Stanford2D3D, respectively, and improves all five metrics on both.The differences are summarized in Tab. 3.Source: D Performance Gains over the… · p. 20

E Sphere-Area-Weighted Metric

SCCM’s gains persist under sphere-area-weighted metrics, while the supplementary analyses characterize off-gravity behavior, design choices, and numerical sensitivity. The sections also document the controlled coarse-matcher scaffold and an absolute positional-embedding control.

E Sphere-Area-Weighted Metric

  • SCCM retains gains over the chart-naïve baseline on every sphere-area-weighted metric for Matterport3D and zero-shot Stanford2D3D.The comparison reweights pixels by cos φ; SCCM leads the listed baselines under both pixel-uniform and area-weighted aggregation.Source: E Sphere-Area-Weighted Metric · p. 20, Table 4 · p. 21

F Off-Gravity Behavior Beyond Upright ERP

G Rationale for Backbone and Baseline Training Choices

  • RoMa V1 is used for controlled ablations because its coarse stage is separable and its training recipe reproducible; RoMa V2’s coupled changes and unavailable comparable recipe would confound the comparison.EDM is evaluated from its released checkpoint because its training stack is not released.Source: G Rationale for Backbone and Baseline… · p. 22

H Yaw-Periodic RoPE: Frequency Choice

  • Integer longitudinal RoPE frequencies preserve 2π-periodicity at the seam, and the integer-frequency design outperforms the geometric-frequency control.At coarse-grid width 64, the maximum frequency is 16, half the 32-cycle Nyquist limit.Source: H Yaw-Periodic RoPE: Frequency Choice · p. 23

I Chart-naïve Coarse Matcher (R1 Backbone)

  • The chart-naïve R1 backbone replaces RoMa V1’s Gaussian-process coarse matcher with attention and dual-softmax, while retaining its frozen encoder and ConvRefiner architecture.R1 has no sphere-aware positional encoding, metric bias, or area correction; AAC adds a log-area term to the covisibility logit before the gate.Source: I Chart-naïve Coarse Matcher (R1… · pp. 23–24

J EDM Positional Embedding: Re-implementation

  • The EDM positional-embedding control adds a learned projection of each token’s unit ray to both token grids before attention, isolating an absolute input-side encoding rather than EDM’s full system.The control uses the R1 backbone under the same training protocol and evaluator.Source: J EDM Positional Embedding:… · p. 24

K AAC/LAC Numerical Sanity Checks

  • Changing the AAC numerical clamp across 10^-4 to 10^-2 leaves reported metrics unchanged, and fixing the learned LAC strength to 1 changes metrics by at most 10^-4.The clamp is active only in an extremely thin polar band, while learned LAC strengths remain near the analytic area value.Source: K AAC/LAC Numerical Sanity Checks · pp. 24–25, Table 6 · p. 25

L Computational Complexity Analysis

SCCM’s added computation is concentrated in the tangent-plane bias, whose fixed geometry allows caching. The reported deployed latency overhead is lower with caching than with recomputation.

L Computational Complexity Analysis

  • SCCM adds a parameter-free yaw-periodic RoPE, an approximately 1.1K-parameter TPB MLP, and one LAC scalar per covisibility side.Caching the content-independent TPB bias reduces deployed overhead to about 5%.Source: L Computational Complexity Analysis · p. 25

L.1 FLOPs

  • Recomputing TPB adds about 8.9 GFLOPs, approximately 1.2% of the 751 GFLOPs chart-naïve baseline; caching removes this per-forward cost.RoPE costs about 0.1 GFLOPs in total, and AAC’s pointwise term has negligible cost.Source: §L.1 · pp. 25–26

L.2 Inference Latency Protocol

  • On a single RTX 4090 at medium ERP resolution, TPB recomputation adds 20.9% latency, whereas caching the fixed geometry bias reduces SCCM’s overhead to 5.0%.Latency is averaged over 100 timed forwards after warm-up; the cached configuration is used for deployment.Source: §L.2 · p. 26, Table 9 · p. 26

M Module Contributions and Interaction

On the fixed ablation scaffold, yaw-periodic RoPE gives the largest PCK@1° gain, while TPB and LAC contribute different metric and area-aware effects. A factorial test shows that TPB and LAC interact on strict precision.

M Module Contributions and Interaction

N Analysis of Remaining Errors

SCCM’s remaining errors include a gravity-alignment failure under camera tilt and generic dense-matching ambiguities. Its gain over the chart-naïve scaffold is already present at coarse anchors and persists through refinement.

N Analysis of Remaining Errors

  • At 30° camera tilt, full-set PCK@1° drops from 0.273 to 0.034; texture-less, reflective/HDR, low-overlap, and depth-discontinuity regions remain generic failure cases.The tilt breaks the gravity-aligned prior; the listed visual ambiguities are described as shared by perspective and ERP matchers.Source: N Analysis of Remaining Errors · pp. 28–29

N.1 Certainty-gated vs. ungated reconstruction

  • Without the shared certainty gate τ = 0.5, a rare low-certainty tail affects SCCM’s mean accuracy, while median accuracy and majority-pair performance remain better than chart-naïve.The comparison concerns the main reconstruction protocol.Source: §N.1 · p. 29

O Refinement-Stage Diagnosis

  • SCCM’s advantage is +4.1 pp at the coarse stage and +4.5 pp at final dense output, with the unchanged refiner preserving rather than creating the gain.The chart-naïve model and SCCM receive similar coarse-to-final lifts: +10.8 pp and +11.2 pp, respectively.Source: O Refinement-Stage Diagnosis · pp. 29–30, Table 12 · p. 30

P Outdoor Evaluation on Holo360D

On Holo360D, SCCM leads the retrained matchers across reported metrics; its zero-shot Mapillary advantage is confined to strict precision. Additional ERP baselines and controls bound the scope and variability of these results.

P Outdoor Evaluation on Holo360D

  • On Holo360D, SCCM leads on all seven metrics, with a +2.6 pp margin over R1 at 1°.The test covers 8,000 pairs; all three matchers start from Matterport3D checkpoints and share a training budget. Holo360D supervision is sparser than Matterport3D, and paired views have median relative tilt of 12°.Source: P Outdoor Evaluation on Holo360D · p. 30, Table 13 · p. 31
  • On zero-shot Mapillary transfer, SCCM leads R1 by +1.9 pp at PCK@0.35° and +0.5 pp at PCK@1°, but not at 3°–5° or mean angular error.Mean angular error is 10.08° for SCCM versus 9.39° for R1.Source: P Outdoor Evaluation on Holo360D · pp. 30–31

Q Additional Zero-Shot Baselines on ERP

R Additional Controls: Seeds, Resolution, and Pose Solver

  • Across three training runs per endpoint, PCK@1° ranges from 0.272 to 0.283 for SCCM and 0.223 to 0.230 for R1; the +0.9 pp LAC step is a single-run estimate.Per-configuration sample standard deviation is 0.4–0.6 pp, so the LAC step should be read as indicative on the training side.Source: R Additional Controls: Seeds,… · pp. 31–32, Table 15 · p. 32
  • Across two EDM-resolution controls, SCCM scores PCK@1° of 0.273 and 0.279, while its gap over EDM (0.163) remains 11.0–11.6 pp.The controls vary whether only the input information or also the ground truth and scored pixels are matched to EDM’s grid.Source: R Additional Controls: Seeds,… · p. 32
  • Swapping the shared ray-space 8-point+RANSAC estimator for Robust 360-8PA changes AUC@5° by at most 0.01 pp, with rankings unchanged.The comparison uses identical matches and inliers across 15,112 pairs.Source: Table 16 · p. 32, R Additional Controls: Seeds,… · p. 32

Related papers

Loading 2609.36545v2…