PaperlayerView exact revision on arXiv

Source-linked AI summary

Gauge-Equivariant Attention for Rotation-Stable $360^\circ$ Scene Understanding

Tianjian Zhou, Yishan Li, Jie Jiang, Yifei Zhang

arXiv:2609.31755v1cs.CVcs.GR

TL;DR

Standard spherical transformers can lose more than half their segmentation accuracy under camera rotation, and the relative-position bias is a major contributor to this failure. The paper averages that bias over a finite gauge-rotation group and aligns teacher tokens during SSL; EquiSSL then supports rotation-stable segmentation, low-label gains, and transfer to other tasks.

  • Problem

    A published spherical transformer’s val mIoU falls from 67.53 to 31.74 after a 90° rotation, while gauge dependence in relative-position bias contributes to rotation collapse and SSL inflation.

    Source: §1 · p. 1

  • Method

    GE-RPE averages the attention bias over finite gauge rotations, and EquiSSL pairs it with teacher-token permutation π_R for rotated-view SSL alignment.

    Source: §1 · p. 2

  • Results

    EquiSSL matches the bias-free rotation drop, lifts low-label fine-tuning, and transfers to depth and zero-shot Structured3D segmentation.

    Source: §6 · p. 13

  • Takeaways & Limitations

    The paper presents bias-level gauge averaging as a checkpoint-compatible fix that leaves the surrounding icosphere-transformer architecture unchanged.

    Source: §1 · p. 2

    GE-RPE addresses gauge dependence at the bias level only, and global SO(3) rotation tests do not cover locally non-rigid distortions such as rolling shutter or per-pixel optical-flow warps.

    Source: §6 · p. 13

Abstract

from arXiv · show

Panoramic $360^\circ$ scene understanding increasingly relies on icosphere transformers, but a state-of-the-art spherical model loses more than half of its segmentation accuracy when the camera rotates by $90^\circ$, and controlled ablations identify gauge dependence in its relative-position bias as a major contributor. We propose gauge-equivariant relative position encoding (GE-RPE): a parameter-free Reynolds average of the bias over a finite cyclic subgroup $C_n\!\subset\!\mathrm{SO}(2)$ of gauge rotations. Plugged into a SphereUFormer backbone the change is invisible at deployment---zero added parameters and $1.5$--$4.4\%$ forward latency---and the matched three-seed GE-RPE model records a $1.3\%$ drop; the published SphereUFormer checkpoint records $53\%$ under the same stress protocol but a different training recipe. Once the gauge defect is removed and a teacher-token permutation $\pi_R$ aligns the SSL views to the rotated student frame, iBOT$+$MAE pretraining stops being a liability and becomes a clean low-label lever: the full framework EquiSSL (GE-RPE $+$ $\pi_R$ $+$ iBOT$+$MAE) tightens the drop to $0.8\%$ at $68.30\%$ val mIoU and lifts $1\%$-label fine-tuning by $+2.39$ mIoU on the $N{=}373$ test split (and by $+4.10$ on the smaller $N{=}40$ val split); the same fix carries over to monocular depth and to zero-shot Structured3D segmentation. The construction is provably $C_n$-invariant and $\mathcal{O}(n^{-2})$-close to the continuous $\mathrm{SO}(2)$ average, making the resulting model a usable $360^\circ$ visual-computing primitive across panoramic relighting, immersive video, and cross-dataset transfer. Code is available at https://github.com/Jaywalk18/equissl-release.

1 Introduction

Panoramic scene understanding must tolerate camera rotation, yet standard icosphere transformers can collapse under a 90° rotation and their relative-position bias is a shared contributor to inference and SSL failures. The paper introduces GE-RPE and EquiSSL to address the bias mismatch and align teacher tokens during pretraining.

1 Introduction

  • A 90° rotation reduces published SphereUFormer val mIoU from 67.53 to 31.74, a 53% relative drop, while gauge-dependent bias contributes to rotation collapse and SSL inflation.The published-checkpoint comparison uses a different training recipe from the matched GE-RPE results.Source: §1 · pp. 1–2
  • GE-RPE averages the bias lookup over evenly spaced gauge rotations, yielding exact C_n invariance and an O(n^-2) residual to continuous SO(2) averaging under a bounded-variation assumption.The SphereUFormer implementation adds zero parameters and 1.5–4.4% forward latency; its three-seed rotation drop is 1.3%.Source: §1 · p. 2
  • EquiSSL combines GE-RPE with teacher-token permutation π_R, which aligns SSL targets to the rotated student frame.The two changes address the bias lookup and cross-view token alignment, respectively.Source: §1 · p. 2
  • GE-RPE with iBOT+MAE tightens the rotation drop to 0.8% and lifts 1%-label fine-tuning by +2.39 mIoU on test (N=373) and +4.10 on val (N=40).The same framework transfers to depth and zero-shot Structured3D segmentation.Source: §1 · p. 2

2 Related Work

Prior spherical and equivariant methods address geometry at different levels, while this work targets the tangent-frame-dependent attention bias in icosphere transformers. Its bias-only retrofit leaves the surrounding architecture unchanged, with a correspondingly narrower guarantee.

Spherical perception and gauge equivariance

  • GE-RPE averages one scalar attention bias per query–key pair over finite C_n, leaving Q/K/V, pooling, and transport unchanged.Its guarantee is finite-group invariance of the bias, unlike feature-level constructions that use group representations or feature transforms.Source: §2 · p. 2

Position encoding in transformers

  • Swin-style bias tables assume globally consistent tangent coordinates that do not exist on S2, while RoPE requires a globally orientable axis.At each sphere vertex, choosing the axis would require a gauge choice.Source: §2 · pp. 2–3

Equivariant and spherical SSL

  • Prior SSL approaches leave the encoder gauge-dependent, whereas π_R restores teacher–student token correspondence without imposing encoder equivariance.The paper identifies the per-vertex attention bias as a blocker for rotation-consistent pretraining on icosphere transformers.Source: §2 · p. 3

3 Preliminaries

The preliminaries define the spherical mesh, local tangent frames, and relative-position bias used by the model. They then connect rotation-based self-supervision to a gauge mismatch in attention and state how gauge averaging controls the bias residual.

3 Preliminaries

  • An icosphere approximates S2 with vertices on the unit sphere, and a gauge is an oriented orthonormal basis of a vertex’s tangent plane.Because no globally smooth frame choice exists, another valid frame can differ by an SO(2) rotation.Source: §3 · p. 3

3.2 Problem statement: gauge dependence breaks equivariance

  • For a query–key pair, RPE applies a learnable bias to the key’s projected tangent-plane coordinates, so rotating the local frame can change the lookup despite unchanged spherical geometry.The controlled comparison changes only the RPE bias and does not assert exact equivariance for pooling, mesh permutation, or the full backbone.Source: §3 · p. 3, §3.2 · p. 4

3.1 Spherical self-supervised learning and its gauge obstruction

  • The SSL consistency objective compares encoder features across teacher and student views related by a random SO(3) rotation and its induced node permutation.The iBOT+MAE loss is a clean training signal only when the encoder is permutation-equivariant in expectation; otherwise the gradient injects a gauge-dependent prior.Source: §3.1 · p. 3

3.2 Problem statement: gauge dependence breaks equivariance

  • The SSL consistency bound combines a mesh-permutation term with a bias residual that is O(1) for standard RPE and O(n^-2) for GE-RPE.The bias residual vanishes exactly if and only if the bias is invariant under the relevant group.Source: §3.2 · p. 4

4 Method: Gauge-Equivariant Relative Position Encoding

GE-RPE averages relative-position bias lookups over finite gauge rotations, making the bias exactly invariant to C_n while approximating continuous SO(2) averaging. For rotation-consistent SSL, teacher-token permutation separately aligns teacher and student token indices.

4 Method: Gauge-Equivariant Relative Position Encoding

  • GE-RPE averages bias lookups across n rotations of the tangent-plane coordinates, yielding exact invariance to gauge rotations in C_n.It changes the lookup rather than removing the bias function’s directional information.Source: §4 · pp. 4–5
  • Figure 4 validates the mode-survival law and finds both the O(1/n^2) Fourier-tail rate and the pi/n worst-case Lipschitz radius tight in measurement.Panel (c) flags C2 as an even-mode parity outlier.Source: Figure 4 · p. 5, §4 · p. 5
  • Under a bounded-variation condition, GE-RPE’s residual from continuous SO(2) averaging decays as O(n^-2).The discrete average cancels modes unless n divides their order; the surviving aliased modes determine the approximation error.Source: §4 · pp. 5–6
  • For bilinear bias grids, C2 aliases the dominant m=2 mode and has a ~15–16% drop, while C4/C6 remove it and approach the 1.3% no-RPE reference band.The next surviving modes for C4 and C6 have coefficients 16×–36× smaller than C2’s m=2 leak.Source: §4 · p. 6

4.2 Rotation-consistent SSL via teacher-token permutation

  • Teacher-token permutation aligns teacher and student indices across rotated SSL views, while GE-RPE removes bias dependence on gauge at fixed token indices.The methods address distinct axes: cross-frame indexing and intra-layer bias lookup.Source: §4.2 · p. 6
  • Area weighting adds log dual area to attention logits, but it slightly hurts empirically and is omitted from the canonical configuration.The authors attribute this choice to the icosphere’s near-uniform sampling at rank 7.Source: §4.2 · p. 6

5 Experiments

The experiments evaluate GE-RPE and EquiSSL across panoramic segmentation, low-label transfer, depth estimation, ablations, zero-shot transfer, and cross-dataset relighting, with Stanford2D3D as the main benchmark. They assess rotation robustness, accuracy, rendering fidelity, and deployment costs under controlled settings, and specify shared reproducibility settings and the SSL head warm-up.

5.1 Experimental Setup

  • The experiments use SphereUFormer with a shared 7×7 GE-RPE bias grid, training on Stanford2D3D’s Area-5 split and evaluating rotation drops across 10 rotations and 3 repeats.The rotation-cap sweep covers 0° to 90°; GE-RPE denotes the random-init architecture, while EquiSSL denotes GE-RPE plus iBOT+MAE.Source: §5.1 · p. 6

5.3 Self-Supervised Pretraining as a Gauge Diagnostic

  • 68.30% val mIoU and 0.8% rotation drop: EquiSSL combines GE-RPE with iBOT+MAE and achieves the closest accuracy–robustness Pareto point.At 1% labels, it adds +2.39 mIoU on the N=373 test split and +4.10 mIoU on the N=40 val split over random-init GE-RPE C6. SSL leaves the 100%-label val result nearly unchanged but worsens Standard RPE’s rotation drop from 1.1% to 8.6%, while EquiSSL tightens GE-RPE’s drop from 1.3% to 0.8%.Source: Figure 5 · p. 7, §5.3 · p. 8, Table 3 · p. 9

5.2 Main Results: Semantic Segmentation

  • 67.85±0.38 val mIoU and 1.3±0.2% rotation drop: matched GE-RPE C6 improves mIoU over Standard RPE’s 66.35±0.48, with a similar rotation drop of 1.1±0.2%.The published SphereUFormer checkpoint [2] records a 53% drop under this rotation protocol, but uses a different training recipe; the matched architecture results isolate the effect of the gauge fix.Source: §5.2 · p. 7

5.4 Ablation Studies

  • 16.6%→1.7%→1.3%: rotation drop falls across C2, C4, and C6, while the C2 outlier reflects even-mode aliasing.The stress test reports 0.54% at C8 and 0.24% at C12, consistent with O(1/n^2) scaling up to n=12. At rank 7, removing area weighting improves val mIoU by +0.47 to +0.62 and test mIoU by +0.12; GE-RPE adds zero parameters and 1.5–4.4% forward latency.Source: §5.4 · pp. 9–11

5.5 Depth Estimation and Label Efficiency

  • +4.10 mIoU at 1% labels: EquiSSL’s lift over random-init GE-RPE C6 narrows to +2.00 at 10% labels and +0.45 at 100%.For monocular depth, the canonical configuration reaches 0.9216 val δ1, with RMSE 0.2184 and Abs Rel 0.0867, versus 0.8471 δ1 for the 14.6M backbone.Source: §5.5 · p. 11, Figure 10 · p. 11

5.6 Cross-Dataset Transfer and Fine-tune Protocols

  • +3.3 to +4.2 mIoU upright over Standard RPE: gauge-equivariant variants also reduce zero-shot Structured3D rotation drop from 3.0% to under 1.1%.On zero-shot Matterport3D, Standard RPE’s drop is 4.9% while EquiSSL stays within 0.7%; in the relighting probe, Standard RPE reaches about +13 pp overlay drift at 90° while EquiSSL stays at 1–3 pp across the tilt sweep.Source: Table 6 · p. 12, §5.6 · p. 12
  • At 90° tilt, EquiSSL records 21.83 dB PSNR versus Standard RPE’s 20.11 dB (+1.72 dB), with LPIPS 0.198 versus 0.273 (28% lower perceptual error).Table 8 averages 16 samples per tilt and method; at 0° the methods tie, with PSNR 22.68 versus 22.42 dB. The end-to-end PSNR slope ratio is −0.85 versus −2.31 dB.Source: §5.6 · p. 12
  • Fine-tuning holds the 642-vertex icosphere, equirectangular augmentation, and AdamW settings fixed, varying only warm-up, epochs, and loss across schedules.All schedules use AdamW with lr 10^-4, weight decay 0.05, cosine decay, batch 8, and one A100-80GB; the backbone has 35.1 M parameters.Source: §5.6 · p. 12, Table 9 · p. 13
  • In the chrome-sphere probe, SphereUFormer’s light-coverage drift reaches about +13 pp at 90°, while EquiSSL stays within 1–3 pp.The figure reports a 28.7% GT reference and bounds rendered radiance error by 2.44Δpp.Source: Figure 11 · p. 13
  • The two-stage iBOT+MAE schedule uses a 50-epoch frozen-head warm-up; removing it costs about 1.2 mIoU at 100% labels and further inflates Standard-RPE rotation drop.The warm-up separates encoder pretraining signal from randomly initialized head noise; depth uses the same warm-up because its shallower regression head converges within the budget.Source: §5.6 · p. 13

6 Conclusion

Controlled ablations identify gauge-dependent relative-position bias as a major contributor to rotation instability and SSL disruption. GE-RPE with teacher-token alignment makes EquiSSL composable with SSL, while the conclusion notes remaining scope limits and practical extensions.

6 Conclusion

  • A parameter-free Reynolds average targets gauge-dependent bias, and Theorem 3.1 bounds the SSL bias residual by the same O(n^-2) gauge-invariance quantity.The method is applied during inference and matched SSL pretraining.Source: §6 · p. 13
  • EquiSSL matches the bias-free no-RPE rotation drop, lifts low-label fine-tuning, and transfers zero-shot to a synthetic indoor corpus.The conclusion attributes this composition to removing the gauge defect once for both inference-time rotation and SSL.Source: §6 · p. 13
  • Gauge-stable encoders are presented as relevant to panoramic relighting, immersive video, and AR/VR; proposed extensions move the Reynolds average to features or intrinsic spherical SSL.The outlook connects unstable features to render-time or viewport flicker and proposes feature-level transport and sphere-native pretexts.Source: §6 · p. 13
  • GE-RPE fixes gauge dependence at the bias level only; equivariant feature tasks require composition with feature-level transport.Locally non-rigid distortions such as rolling-shutter and per-pixel optical-flow warps lie outside the global SO(3) rotation protocol.Source: §6 · p. 13
  • At 90°, boundary and small-region errors remain, while beam has 0.0% IoU and column, door, and sofa remain below 8% across measured variants.The canonical GE-RPE C6 model still has a 1.3% rotation drop, including mesh, resampling, and backbone effects beyond GE-RPE’s guarantee.Source: §6 · p. 14

A Full Proofs

The appendix fills in the lifting, network-level, and Fourier arguments behind the main bounds. It explains how token alignment, finite gauge averaging, mesh error, and bias smoothness enter the proofs.

Local-to-global lifting

  • The local-to-global lifting argument assumes GE-RPE in every layer and residual gauges matching Levi–Civita holonomies; it decomposes rotation into a cancellable Cn element plus a bounded Fourier residual.The proof reduces the rotation to SO(2), cancels the nearest subgroup element, and applies the Fourier bound.Source: A Full Proofs · p. 15

Teacher-token permutation removes the cross-frame mismatch

  • Teacher-token permutation exactly cancels the teacher–student node-indexing mismatch before loss evaluation, leaving the attention-bias residual bounded by GE-RPE.The bound accumulates layerwise residual through the Lipschitz loss, with a separate nearest-neighbour mesh-error term.Source: A Full Proofs · p. 15

Theorem 3.1

  • For a non-degenerate consistency loss and a faithful SO(2) action almost surely, zero SSL residual holds exactly when the bias is G-invariant.The forward direction pulls zero expected loss through the layerwise residual decomposition; the reverse follows directly from invariance.Source: A Full Proofs · pp. 15–16

Network-level Lipschitz residual

  • The network bound combines a gauge term L∥r∥π/n with mesh error, while query–key products are equivariant and only the bias requires bounding.The nearest Cn element cancels exactly; the remaining angular mismatch is at most π/n.Source: A Full Proofs · p. 16

Theorem 4.4

  • Bounded angular variation gives Fourier coefficients decaying as 1/m^2 and an O(n^-2) aliased-sum bound; bilinear grids satisfy this assumption, unlike merely Lipschitz biases.Sharper angular derivatives give faster rates, while a merely Lipschitz bias can yield a divergent aliased sum.Source: A Full Proofs · p. 16

Why even n? The bilinear-grid parity argument

  • Even n preserves the bilinear grid’s symmetric Fourier-mode pairs, while C2 leaves the dominant m=2 mode uncancelled; C4 and C6 enter the bias-free reference band.The appendix also motivates C6 by the hexagonal valence of nonpole icosphere vertices.Source: A Full Proofs · p. 16, Table 10 · p. 16

C Theoretical Validation: Approximation Error

The appendix checks theoretical approximation claims against trained bias grids, rotation-drop measurements, and Fourier spectra. The results distinguish head-wise bias residuals from end-to-end rotation behavior and show agreement with the predicted convergence trend at higher gauge counts.

Measured residual on trained bias grids

  • The median head-wise residual bound is never violated, with the measured-to-bound ratio below 0.5 throughout and tightening as n grows.The residual compares the finite Cn Reynolds average with a 360-point continuous reference on trained bias grids.Source: C Theoretical Validation:… · p. 16
  • Head-wise residuals decay closer to O(1/n), while end-to-end rotation drops recover O(n^-2) once the mesh floor dominates, reflecting distinct error reservoirs.The appendix identifies the difference as measurements taken at different points along the forward pass.Source: C Theoretical Validation:… · pp. 16–17

C8 and C12 stress-test provenance

  • 0.54%±0.10% at C8 and 0.24%±0.05% at C12 match the predicted 0.54% and 0.24% within seed standard deviation.These three-seed stress tests extend the observed n^-2 slope through n=12; the Haar-limit end-to-end drop was not measured.Source: C Theoretical Validation:… · p. 17, Table 13 · p. 17

Polar-Fourier spectrum of the learned bias

  • C2 retains every even Fourier mode, including dominant m=2, whereas C4 and C6 restrict survival and their learned spectra concentrate on those surviving modes.The C2-to-C4 collapse of the even-mode pile is linked to the aliasing spike and residual scaling after m=2 is removed.Source: C Theoretical Validation:… · p. 17

D Additional Theoretical Validation

Additional diagnostics separate finite gauge spacing from mesh-permutation error and examine the resulting composite mismatch. They also test how the chosen gauge count and bias parameterization affect the approximation behavior.

Discrete-gauge covering of SO(2)

  • At rank 7, the mesh-spacing term is about 0.2° while the C6 gauge half-gap is 30°, making gauge spacing the directly controlled lever in the tested regime.The covering bound is π/n, and the appendix notes that increasing n beyond 6 at rank 7 offers no further reduction once the gauge gap falls below the mesh floor.Source: D Additional Theoretical Validation · pp. 17–18, Figure 13 · p. 18

Mesh-floor decay across icosphere ranks

  • Rank-7 mean mesh-assignment error is about 0.2°, over two orders of magnitude below the C6 cutoff, but its contribution to end-to-end drop is not isolated.The measured mean and maximum errors follow the predicted O(2^-r) slope; robustness on finer meshes was not evaluated.Source: D Additional Theoretical Validation · pp. 18–19

Per-vertex distribution at the rank we ablate

  • At rank 3, nearest-neighbour residuals are unimodal with mean 3.06°, p95 4.74°, and maximum 5.93°; the max-to-mean ratio is 1.6×.The distribution remains well inside the C6 cutoff, with no hidden long-tail population in the reported sample.Source: D Additional Theoretical Validation · pp. 18–19

Putting both ingredients together: empirical |Δg|

  • The composite gauge mismatch stays within the predicted π/n envelopes; C6 and C4 cumulative masses closely match the uniform-quadrature predictions.Across the measured samples, the C6, C4, and C2 cumulative masses are 16.4%/25.3%/50.5%, versus predictions of 16.7%/25.0%/50.0%.Source: Figure 14 · p. 18, D Additional Theoretical Validation · p. 19

Bias-parameterisation: bilinear vs. MLP-continuous bias

  • Bilinear bias reaches the mesh floor by C4 with an O(n^-2) rate, whereas the tested MLP bias decays at O(1/n) and plateaus above it.The comparison is attributed to bounded angular variation for bilinear interpolation versus sharper angular structure in the MLP parameterization.Source: D Additional Theoretical Validation · p. 19, Table 14 · p. 19

E Implementation Details

The implementation precomputes gauge-rotated relative coordinates and reports training and rotation-evaluation settings. Depth estimation uses a two-stage fine-tuning schedule, while the rendering-equation analysis bounds lighting error for single-bounce reflection.

E Implementation Details

  • At rank 6 with window coefficient 2 and n = 6, precomputed buffers require approximately 60MB of float32 storage.Storage scales as O(n·N·W), where N is the number of vertices and W is the maximum neighborhood size.Source: E Implementation Details · p. 19
  • Segmentation trains for 350 epochs with AdamW, while rotation evaluation uses 10 random SO(3) rotations and three repeats per angle cap.The evaluation caps range from 0° to 90°; multi-run aggregates use seeds {42, 123, 456}.Source: E Implementation Details · p. 19
  • The canonical depth row reports test δ1 = 0.6787, RMSE = 0.3779, and Abs Rel = 0.1989.Depth training uses 50 epochs of frozen-encoder warm-up followed by 200 epochs of full fine-tuning with L1 loss.Source: E Implementation Details · pp. 19–20

F Rendering-Equation Grounding for the IBL Drift Metric

  • The IBL drift bound is |ΔL|/Lmax ≤ 2.44Δpp for the stated thresholded environment map and single-bounce mirror model.The bound also holds for Lambertian and glossy or microfacet reflection; multi-bounce indirect illumination is outside scope.Source: F Rendering-Equation Grounding for the… · p. 20

G Per-Class IoU Analysis

The per-class analysis finds that C6 no-area performs strongly on common structural classes, while rare-class measurements are noisy. Standard RPE’s aggregate penalty is broadly distributed, and the rotated per-class breakdown is not reported.

G Per-Class IoU Analysis

  • C6 no-area wins on 9 of 13 test classes against C4, including wall, table, ceiling, and column.C4’s clearest win is window, by 3.1 points over C6 no-area.Source: G Per-Class IoU Analysis · p. 20
  • Across upright evaluations, rare-class differences are within noise, while structural-class spreads reach 1.4–3.6 percentage points.Column, door, and beam spreads are below 0.4 points; the cited analysis attributes those entries to sample-count noise.Source: G Per-Class IoU Analysis · p. 20
  • Standard RPE’s per-class differences from No-RPE are distributed almost uniformly: deltas split in sign 7/6 and remain below 3 points except sofa, where No-RPE leads by 3.94.Source: G Per-Class IoU Analysis · p. 20
  • Per-class results are reported at 0°; rotated per-class IoUs at 90° remain a future-work extension.The rotation evaluation reports rotated outcomes at the aggregate level.Source: G Per-Class IoU Analysis · p. 20

H Rotation Evaluation Protocol Details

The rotation protocol evaluates models across angle caps and reports per-seed endpoints for reproducibility. EquiSSL remains nearly stable across the sweep, whereas the published SphereUFormer checkpoint drops sharply; undersampled C2 averaging shows a distinct intermediate-angle failure pattern.

H Rotation Evaluation Protocol Details

  • At rank 7, node-permutation error is below inter-vertex spacing, and the protocol detects SphereUFormer’s 53.0% drop while GE-RPE has below 2% mean drop for n ≥ 4.The permutation error decays as O(2^-r).Source: H Rotation Evaluation Protocol Details · pp. 20–21
  • The three-seed paired endpoint ratios reproduce the canonical GE-RPE C6 headline drop of 1.3% to rounding.Table 19 reports a 1.34% mean and 0.04% standard deviation across seeds; the larger displayed headline spread reflects rotation-sample variance.Source: H Rotation Evaluation Protocol Details · p. 21, Table 19 · p. 21
  • EquiSSL drops 0.8±0.2% from 0° to 90°, while the published SphereUFormer checkpoint drops 53.0% end to end.The per-angle table reports three-seed means and standard deviations for the paper’s models and a single-seed published-checkpoint reference.Source: H Rotation Evaluation Protocol Details · p. 21
  • C2 averaging reaches a 16.6% drop, with its worst behavior near 20°–45° and partial recovery by 90°.C4, C6, and EquiSSL stay within 1 mIoU of their upright scores across the sweep; C4 and C6 differ by less than per-seed noise.Source: H Rotation Evaluation Protocol Details · p. 21

I Label Efficiency Analysis

The label-efficiency analysis compares supervised and SSL-pretrained tracks across label fractions, with validation and test results reported separately. At 1% labels, SSL lifts test mIoU, while the larger validation lift is noisier; its gain is largely amortized at full supervision.

I Label Efficiency Analysis

  • The study compares 1%, 5%, 10%, and 100% labeled training data, corresponding to 10, 50, 101, and 1,013 samples.Low-label random-init rows are single runs; the 100% rows are three-seed means.Source: I Label Efficiency Analysis · p. 21, Table 21 · p. 21
  • At 1% labels, C6+SSL improves test mIoU from 12.40 to 14.79, a +2.39 lift on N=373; the validation lift is +4.10 on N=40.The paper identifies the larger test split as the more reliable estimate and notes proportionally higher noise on validation.Source: I Label Efficiency Analysis · p. 22
  • At 100% labels, C6+SSL is +0.15 test mIoU over C6, within seed standard deviation, indicating the SSL lift is largely amortized by full supervision.The reported values are 31.35 versus 31.20.Source: I Label Efficiency Analysis · p. 22
  • EquiSSL exceeds the Standard-RPE baseline by +4.61 mIoU at 1% labels, reported as roughly the lift of a 4× data injection.The comparison is 22.59 versus 17.98 mIoU; the proposed recipe pretrains on unlabeled 360° data before supervised fine-tuning.Source: I Label Efficiency Analysis · p. 22

J Self-Supervised Pretraining: Full Details and Extended Analysis

EquiSSL combines GE-RPE with rotation-aligned iBOT+MAE pretraining, achieving 68.30% segmentation mIoU with a 0.8% rotation drop. Ablations show the architectural fix is the load-bearing component, while training-time alignment and the combined pretext objectives refine performance.

Recipe

  • Each pretraining step rotates the student, masks 75% of its nodes, and re-permutes unrotated EMA-teacher patch tokens into the student frame before distillation.An L1 reconstruction loss on masked student nodes supplies the MAE signal.Source: J Self-Supervised Pretraining: Full… · p. 22

Pretraining failure and EquiSSL result

Three-way coupling

  • Removing GE-RPE raises the rotation drop to 8.6%, while removing rotation augmentation or teacher-token alignment yields 2.4% and 2.8%, respectively.The full-EquiSSL row is a canonical three-seed mean; leave-one-out variants are single-seed runs.Source: J Self-Supervised Pretraining: Full… · p. 23, Table 23 · p. 23

iBOT vs MAE individual contribution

Masking, EMA, and convergence

  • Mask ratios from 50% to 85% stay within ±0.55 mIoU, and constant EMA τ=0.999 reaches 68.20, within seed noise.The reported canonical setting uses 75% masking and a cosine EMA schedule from τ=0.996 to 0.9999.Source: J Self-Supervised Pretraining: Full… · p. 23
  • iBOT loss plateaus past epoch 66 and MAE past epoch 49; doubling training to 200 epochs adds approximately 0.25 pp validation mIoU.The canonical recipe is described as a wide, well-conditioned basin rather than a sharp optimum.Source: Figure 15 · p. 23, J Self-Supervised Pretraining: Full… · p. 23

Related papers

Loading 2609.31755v1…