PaperlayerView exact revision on arXiv

Source-linked AI summary

Remember, Revisit, Revise: Scaffold Memory for Long-Horizon 3D Geometry

Soumick Sarker, Mengmi Zhang

arXiv:2610.19674v1cs.CV

TL;DR

Long-horizon geometry models struggle to maintain consistent reconstructions: feed-forward windows accumulate alignment errors, while streaming models do not revise earlier estimates. SMORE equips frozen models with external memory for storage, retrieval, verification, and revision; across evaluated tasks, it improves long-sequence geometric accuracy, including up to 58% lower multi-view depth error.

  • Problem

    Long-horizon reconstruction remains challenging because feed-forward window alignment can accumulate errors and streaming models do not explicitly revise earlier estimates.

    Source: §1 · p. 2

  • Method

    SMORE adds external memory to frozen geometry models, associating spatial states with visual observations for retrieval, geometric verification, and revision of past predictions.

    Source: §1 · p. 3, §2 · p. 3

  • Results

    Across eight models and five datasets, SMORE consistently improves long-sequence pose estimation, reduces multi-view depth error by up to 58%, and improves other evaluated geometric tasks.

    Source: §1 · p. 3

  • Takeaways & Limitations

    SMORE complements feed-forward and streaming models with verified revisits to correct accumulated errors over long visual sequences without fine-tuning their geometry models.

    Source: §4 · p. 10

    Retrieval verification adds computation, and SMORE currently revises global window placement rather than local geometry.

    Source: §4 · p. 10

Abstract

from arXiv · show

Geometry foundation models estimate camera poses and 3D scene structure from video sequences, but maintaining consistent reconstructions over long horizons remains challenging. Feed-forward models process long sequences as independent, fixed-length frame windows and align their predictions, causing pose and reconstruction errors to accumulate across windows. Streaming models retain information from past frames to inform future predictions, but cannot revise earlier estimates when later observations reveal errors. To address this, we introduce Scaffold MemOry for Revision (SMORE), a framework equipping frozen geometry foundation models with a revisable external memory for long-horizon inference. Inspired by hippocampal associative memory in biological brains, which links spatial context with sensory experience, SMORE associates where an observer is with what it observes, enabling past observations to be stored, retrieved, verified against new evidence, and revised when inconsistencies are detected. We evaluate SMORE with eight competitive geometry models on five datasets spanning camera pose estimation, multi-view depth estimation, 3D point-cloud reconstruction, simultaneous localization and mapping (SLAM), and novel view synthesis. Across all evaluated models, SMORE consistently improves long-sequence pose estimation, reduces multi-view depth error by up to 58%, reduces Chamfer distance for 3D point-cloud reconstruction by up to 43%, and achieves average trajectory error as low as 2.9 cm in long-horizon SLAM. By retaining corrected poses and observations, SMORE further improves the fidelity and spatial consistency of large-scale 3D scene reconstruction. These results establish revisable external memory as a flexible interface for extending pretrained geometry models beyond fixed-context prediction toward iterative, error-correcting inference over long visual sequences.

1 INTRODUCTION AND RELATED WORK

Long-horizon 3D reconstruction remains difficult because errors accumulate across independently aligned windows, while streaming models cannot revise earlier estimates. SMORE adds revisable external memory to frozen geometry models, and improves results across multiple long-sequence tasks.

1 INTRODUCTION AND RELATED WORK

  • Feed-forward window alignment can accumulate errors across windows, while streaming models retain context but do not explicitly retrieve, verify, and revise earlier estimates.Long-horizon reconstruction still requires geometric consistency across observations or submaps.Source: §1 · p. 2
  • Existing retrospective correction methods include dedicated SLAM systems built around specific representations and optimization pipelines, motivating a shared memory for frozen geometry models.The paper asks whether existing foundation models can recall, verify, and revise past predictions while remaining frozen.Source: §1 · pp. 2–3
  • SMORE stores observations, retrieves related past observations, verifies them against current geometric evidence, and revises earlier poses and geometry when inconsistencies are detected.Corrected observations remain in memory as context for later inference.Source: §1 · p. 3
  • Across eight models, SMORE improves long-sequence pose estimation, reduces multi-view depth error by up to 58%, cuts point-cloud Chamfer distance by up to 43%, and reaches 2.9 cm average trajectory error in long-horizon SLAM.Evaluation spans five datasets and includes novel view synthesis.Source: §1 · p. 3
  • SMORE is an external, model-agnostic interface that keeps geometry foundation model weights frozen and requires neither additional training nor fine-tuning.The same memory framework can be applied across different geometry backbones and tasks.Source: §1 · p. 3

2 OUR PROPOSED SCAFFOLD MEMORY FOR REVISION (SMORE)

SMORE stores window-level observations and geometric predictions in an external memory, associates visual cues with spatial states, then verifies retrieved revisits before revising global window placements. Its geometry backbone remains frozen while the external memory is updated online.

2 OUR PROPOSED SCAFFOLD MEMORY FOR REVISION (SMORE)

  • SMORE processes overlapping windows, stores frames and geometric predictions in a content bank, and revises global placement while keeping the geometry foundation model frozen.By default, windows contain 32 frames with 16 shared between consecutive windows.Source: §2 · pp. 3–4

2.1 ASSOCIATING SPATIAL AND VISUAL MEMORY

  • The spatial scaffold links camera pose to a shared memory state, which is associated bidirectionally with visual descriptors so observations can cue related past spatial information.The projected state retains the 64 largest activations from a 1024-dimensional representation; sparse activation is intended to reduce interference and improve recall robustness.Source: §2.1 · p. 4

2.2 RETRIEVING AND VERIFYING REVISITS FOR WINDOW REVISION

  • Visual retrieval proposes candidate revisits, and geometric verification accepts them only when two historical placement proposals satisfy rotation and position consistency thresholds.If fewer than two valid retrieved pairs are available, no retrieval-based correction is applied.Source: §2.2 · p. 5
  • After a revisit reveals placement error, SMORE progressively distributes the correction from the fixed historical reference toward the current window, revising affected windows’ global placements.Stored camera poses and point maps remain in their source-window local coordinates and move together under the updated placement.Source: §2.2 · p. 5

3 EXPERIMENTS

Across long-horizon geometry tasks, SMORE improves reconstruction and pose estimates while retaining corrected observations for later use. The experiments also examine novel-view synthesis, design choices, and computational requirements.

3 EXPERIMENTS

  • The evaluation compares eight geometry models with and without SMORE across five downstream tasks, using online retrieval, verification, and revision without offline global optimization.Three runs per experiment are aggregated; offline pose-graph optimization results are reported in Appendix J.Source: §3 · p. 5

3.1 CAMERA POSE ESTIMATION

  • At N = 200 on CO3Dv2, VGGT improves from 90.0 to 93.9 AUC, VGGT-Ω from 91.6 to 94.7, and π3 from 89.9 to 93.3 with SMORE.Gains are more pronounced on longer CO3Dv2 sequences; some streaming baselines run out of memory without SMORE but complete with it.Source: §3.1 · p. 6, Figure 2 · p. 6

3.2 MULTI-VIEW DEPTH ESTIMATION AND 3D POINT-CLOUD RECONSTRUCTION

  • On Replica, SMORE reduces overall Chamfer error by approximately 58%; on ScanNet++, VGGT improves from 24.95 to 6.05 cm and VGGT-Ω from 20.80 to 4.95 cm.Table 1 reports accuracy, completeness, and overall error across all frames, with lower values better.Source: §3.2 · p. 7, Table 1 · p. 7

3.3 SIMULTANEOUS LOCALIZATION AND MAPPING

  • On 7-Scenes, SMORE reduces VGGT-Ω average ATE RMSE from 4.5 to 2.9 cm, below the best reported calibrated baseline at 3.1 cm and uncalibrated baseline at 3.8 cm.SMORE also reduces average pairwise positional error from 8.1 to 6.5 cm; the figure tracks per-frame error around verified revisits.Source: §3.3 · p. 8, Figure 3 · p. 8

3.4 NOVEL VIEW SYNTHESIS

  • On 7-Scenes, SMORE improves all three novel-view-synthesis metrics across six geometry models; on CO3Dv2, four models gain 0.34–0.64 dB PSNR alongside SSIM and LPIPS improvements.STream3R and Fast3R achieve comparable PSNR with and without SMORE on CO3Dv2, while improving SSIM and LPIPS.Source: §3.4 · p. 9, Table 2 · p. 9

3.5 DESIGN CHOICES AND COMPUTATIONAL REQUIREMENTS

  • Design studies find associative scaffolding outperforms a lookup table, DINOv2 performs best among tested descriptors, and progressive revision outperforms uniform correction placement.The default uses 32-frame windows with 16-frame overlap; larger windows and overlap generally improve performance but increase computational and memory costs.Source: §3.5 · pp. 9–10
  • As sequence length grows, SMORE keeps the geometry model’s active GPU working set largely bounded through fixed-size windows and selective retrieval, while adding modest CPU, PFLOP, and runtime overhead.Its external memory grows with stored visual history, primarily outside the model’s active GPU context; STream3R runs out of GPU memory at N = 100 while STream3R+SMORE processes the full sequence.Source: §3.5 · p. 10

4 CONCLUSION AND FUTURE WORK

SMORE extends frozen geometry models with revisable memory, while this section details its alignment and window-placement behavior. It also identifies retrieval verification’s computational cost and the current focus on revising global window placement.

4 CONCLUSION AND FUTURE WORK

  • SMORE corrects accumulated errors through verified revisits without fine-tuning, but retrieval verification adds computation and revision currently targets global window placement.Future work could extend revision to local geometric predictions when later observations provide stronger evidence.Source: §4 · p. 10

A APPENDIX ROADMAP

  • The appendix roadmap directs readers to memory and geometry details, evaluation and rendering protocols, ablations, resource costs, implementation settings, and failure analyses.Source: A APPENDIX ROADMAP · p. 14

B CAMERA-POSE ALIGNMENT

  • Consecutive windows are aligned using shared-frame camera poses; robust fitting reweights residuals and rejects correspondences with excessive rotational disagreement.The fitted scale is also checked against a pairwise camera-centre distance estimate because scale errors would affect later windows.Source: B CAMERA-POSE ALIGNMENT · pp. 14–15

C WINDOW OWNERSHIP AND GEOMETRIC INVARIANCE

  • Frames are stored once with local geometry, and revising a window’s global placement moves its cameras and points together without recomputing their local predictions.This placement-only change preserves within-window relative rotation and translation direction; jointly processing retrieved frames can separately produce new local poses and point maps.Source: C WINDOW OWNERSHIP AND GEOMETRIC… · pp. 15–16

D SCAFFOLD CONSTRUCTION, STORAGE, AND RETRIEVAL

SMORE links global pose states and visual observations through a shared memory representation, allowing visual cues to retrieve candidate revisits. Retrieval candidates are screened before geometric verification and revision.

D SCAFFOLD CONSTRUCTION, STORAGE, AND RETRIEVAL

  • Three periodic spatial modules encode position at different scales, while three orientation rings encode yaw, pitch, and roll in the scaffold state.Combining spatial scales reduces the ambiguity of any single periodic module, though the representation remains periodic rather than globally unique over an unbounded scene.Source: D SCAFFOLD CONSTRUCTION, STORAGE, AND… · pp. 16–17
  • A fixed projection and top-k sparsification map scaffold states into compact memory addresses, which link to content-bank entries holding RGB frames and local geometry.The address is assigned when a frame is first stored; placement corrections update its source window placement without recomputing the address.Source: D SCAFFOLD CONSTRUCTION, STORAGE, AND… · p. 17
  • The pose-to-address pathway uses a fixed random projection and a return mapping fitted before inference from sampled scaffold states and their addresses.The reverse mapping is fitted without RGB frames, geometry-model outputs, or evaluation labels.Source: D SCAFFOLD CONSTRUCTION, STORAGE, AND… · pp. 17–18
  • Frozen DINOv2 descriptors are associated online with memory addresses in both directions using recursive least squares, updating only the external memory.The geometry foundation model and descriptor encoder are not retrained for these updates.Source: D SCAFFOLD CONSTRUCTION, STORAGE, AND… · pp. 18–19
  • At retrieval, visual appearance predicts an address and the highest-scoring stored addresses supply candidates, which must pass temporal, source-window, and visual-compatibility eligibility checks.The recalled spatial state is not directly decoded as the current frame’s global pose; retrieved frames are subsequently checked using new geometric predictions.Source: D SCAFFOLD CONSTRUCTION, STORAGE, AND… · p. 19

E SCAFFOLD DYNAMICS AND CONTROLLED MEMORY DIAGNOSTICS

The scaffold represents pose through persistent activity patterns in periodic fields, then tests whether those states remain stable, track motion, and support associative recall. These controlled diagnostics characterize the memory representation, not end-to-end reconstruction.

E SCAFFOLD DYNAMICS AND CONTROLLED MEMORY DIAGNOSTICS

  • The scaffold uses a localized activity bump whose position around a ring or torus encodes a spatial or orientation coordinate.Pose is represented by the location of the activity pattern, rather than its overall magnitude.Source: E SCAFFOLD DYNAMICS AND CONTROLLED… · p. 19
  • The fitted content return map maintains low associative-recall error through the largest tested load, P = 160, while the Hopfield-style control accumulates more error.The diagnostic tests memory behavior, not end-to-end reconstruction accuracy or an unrestricted capacity limit.Source: Figure S1 · p. 20, E SCAFFOLD DYNAMICS AND CONTROLLED… · p. 21
  • A stable bump can move with camera motion, while observed camera poses anchor the represented state instead of relying only on accumulated motion.The orientation-ring diagnostic follows supplied angular velocity, and spatial phases return near their starting point after a closed path.Source: E SCAFFOLD DYNAMICS AND CONTROLLED… · p. 20, Figure S1 · p. 20
  • A scaffold return mapping reduces perturbations that make modules mutually inconsistent, but geometric verification is needed to distinguish a coherent displaced pose from the correct physical location.The return mapping encourages recalled states to match learned scaffold structure; it cannot establish physical correctness by itself.Source: E SCAFFOLD DYNAMICS AND CONTROLLED… · p. 20

F GEOMETRIC VERIFICATION OF RETRIEVED REVISITS

SMORE checks retrieved observations geometrically before using them to revise a current window’s placement. It requires agreement between placement proposals grounded in two separately placed historical source windows.

F GEOMETRIC VERIFICATION OF RETRIEVED REVISITS

  • A retrieved frame is paired with a nearby valid frame from the same source window, giving two camera poses a baseline for alignment.The default offset is three frames, making the second frame nearby but distinct.Source: F GEOMETRIC VERIFICATION OF RETRIEVED… · p. 21
  • The two retrieved pairs must come from different source windows so their placement proposals rely on separate historical references.The scaffold finds historical frames; the geometry model and stored source-window placements determine each proposal.Source: F GEOMETRIC VERIFICATION OF RETRIEVED… · pp. 21–22
  • The frozen geometry model jointly processes 32 current-window frames and four retrieved frames, placing all 36 predictions in one local coordinate system.This creates a direct geometric relationship to retrieved frames without using the existing chain of window placements to create it.Source: F GEOMETRIC VERIFICATION OF RETRIEVED… · p. 21
  • Each retrieved pair independently proposes a global placement, and the revisit passes only when the proposals agree in rotation and translation scaled to reconstruction size.If either check fails, the retrieved frames are rejected and no retrieval-based global placement correction is applied.Source: F GEOMETRIC VERIFICATION OF RETRIEVED… · p. 22
  • Agreement is a consistency check, not proof of correspondence; repeated structure, inaccurate historical placements, or weak two-camera geometry can still yield compatible but incorrect proposals.Jointly processing retrieved frames can also alter local predictions even when the revisit is rejected; the default limits this to predictions within 5° of the current-window-only result.Source: F GEOMETRIC VERIFICATION OF RETRIEVED… · p. 22

G TRAJECTORY REVISION AND GLOBAL REFINEMENT

SMORE revises global window placements online when verified revisits expose accumulated error, then performs a robust global refinement. The appendix also specifies the datasets and evaluation protocols used for pose and dense-geometry tasks.

G TRAJECTORY REVISION AND GLOBAL REFINEMENT

  • SMORE distributes a revisit-revealed placement correction across windows between the historical reference and current window, avoiding a discontinuous jump at the current window.The correction weight rises from 0 at the reference to 1 at the current window; only global placements change, moving cameras and associated geometry together.Source: G TRAJECTORY REVISION AND GLOBAL… · p. 23
  • The online correction is available to later windows, while final refinement jointly optimizes placements using retained sequential and accepted revisit connections.The final solve is pose-graph optimization over window placements, with Huber loss to reduce unusually large residuals; large-residual revisit edges are rejected and the graph is optimized again.Source: G TRAJECTORY REVISION AND GLOBAL… · pp. 23–24

H DATASETS AND EVALUATION PROTOCOLS

  • The evaluation protocols compare each baseline with +SMORE using the same geometry model and input frames, with reduced subsets matched within each pair when needed.Main-paper results generally use online retrieval, verification, and revision without offline global optimization.Source: H DATASETS AND EVALUATION PROTOCOLS · p. 24

H.1 CAMERA POSE ESTIMATION

  • Camera-pose tests vary sampled trajectory length on CO3Dv2 and RE10K, whose motion patterns differ: object-centric CO3Dv2 offers more viewpoint changes, while RE10K is mostly forward motion.The pose score evaluates relative rotation and translation direction, not translation magnitude, because monocular geometry is defined only up to scale.Source: §H.1 · pp. 24–25

H.2 MULTI-VIEW DEPTH ESTIMATION AND 3D POINT-CLOUD RECONSTRUCTION

  • Replica and ScanNet++ dense-geometry evaluations use 32-frame windows with 16-frame overlap; baseline chains windows, while +SMORE verifies revisits and revises global placements.Replica supplies synthetic indoor scenes with dense reference geometry; ScanNet++ tests the procedure on real indoor imagery and geometry.Source: §H.2 · p. 25
  • Dense reconstruction reports accuracy, completeness, and their mean as overall error in centimetres, with lower values indicating better reconstruction.Accuracy measures reconstructed-to-reference distance; completeness measures the reverse direction.Source: §H.2 · p. 26

H.3 CAMERA POSE RESULTS ACROSS SEQUENCE LENGTHS

SMORE’s long-sequence pose gains become more apparent when feed-forward models must connect multiple windows, while streaming models can exceed the tested memory budget. The additional sections describe the SLAM comparison and the novel-view synthesis evaluation and decoder.

H.3 CAMERA POSE RESULTS ACROSS SEQUENCE LENGTHS

  • At N = 200 on CO3Dv2, VGGT improves from 90.0 to 93.9 AUC@30, VGGT-Ω from 91.6 to 94.7, and π3 from 89.9 to 93.3.Feed-forward methods are identical at the shortest lengths that fit their initial context; RE10K improvements are generally smaller, consistent with fewer revisits.Source: §H.3 · p. 26

H.4 FULL 7-SCENES SLAM RESULTS

  • Table S3 reports per-scene and average 7-Scenes ATE RMSE after similarity alignment, separating calibrated methods from uncalibrated methods.Within each calibration block, the table marks the lowest value in each scene column.Source: Table S3 · p. 27, §H.4 · p. 27

I NOVEL VIEW SYNTHESIS PROTOCOL AND DECODER

  • Novel-view tests render held-out targets from long-sequence maps, comparing baseline and +SMORE on identical targets and source keyframes.Targets are unseen by the geometry model; the evaluation includes 156 held-out 7-Scenes targets and 888 CO3Dv2 targets per model, with CO3Dv2 decoders trained on 7-Scenes without adaptation.Source: §H.5 · p. 27

H.5 NOVEL VIEW SYNTHESIS

  • Qualitative renderings show sharper chess-piece boundaries on 7-Scenes and better-preserved computer-mouse contours on CO3Dv2 with SMORE.The figure compares ground truth, VGGT-Ω, and SMORE results, with magnified red-box crops.Source: §H.5 · p. 27, Figure S2 · p. 28

I.1 SOURCE VIEWS AND GEOMETRY

  • Baseline places source-frame geometry using chained window transformations, whereas SMORE uses placements after online revision; both use the same filtering and target-camera alignment procedure.Eight nearby input keyframes render each held-out target, which is excluded from the source views.Source: §I.1 · p. 28

I.2 GAUSSIAN DECODER AND TRAINING

  • A Gaussian decoder predicts per-pixel rendering parameters from source images and geometry, then rasterizes the Gaussians and refines the rendered image.Separate decoders are trained for chained and SMORE geometry using matched architecture, data, schedule, and seed; neither SMORE nor the geometry model is optimized by the decoder loss.Source: §I.2 · pp. 28–29

I.3 EVALUATION AND ADDITIONAL RESULTS

On 7-Scenes, revised geometry improves novel-view PSNR, with larger gains on revisited targets. A camera-aligned reference does not consistently outperform SMORE, and results on CO3Dv2 vary by model.

Results on revisited regions

  • 0.31–0.71 dB mean paired PSNR gains on 7-Scenes across six models, with all six 95% confidence intervals excluding zero.Gains on revisited targets are larger; for VGGT-Ω and STream3R they are 1.00 and 1.14 dB, versus 0.53 and 0.71 dB over all targets.Source: §I.3 · p. 29
  • Fast3R improves SSIM and LPIPS but decreases PSNR by 0.12 dB on CO3Dv2.STream3R accepts few revisits on object-centric trajectories and shows little PSNR change.Source: §I.3 · p. 29

Reference window placement

  • For π3 on 7-Scenes, the camera-aligned reference scores 16.43 dB, below the Baseline’s 17.24 dB.For the other five models, the reference exceeds +SMORE by 0.2–1.0 dB on both datasets; camera alignment alone does not ensure consistency between windows’ local geometry.Source: §I.3 · p. 29

Results on revisited regions

  • Figure S3 shows decoder training loss, paired PSNR gains on 7-Scenes, and per-trajectory Baseline versus +SMORE PSNR.Points above the diagonal in panel C indicate improvement.Source: Figure S3 · p. 30

J CONTROLLED ABLATIONS

The ablations support scaffold-based retrieval, DINOv2 cues, two-pair verification, and online trajectory updates, while window configuration affects accuracy. SMORE also keeps geometry inference on bounded windows, limiting GPU memory growth as sequences lengthen.

J.1 DOES THE SCAFFOLD IMPROVE RETRIEVAL?

  • Final 7-Scenes ATE falls from 0.034 to 0.029 m, and final CO3Dv2 AUC@30 rises from 90.9 to 93.3 with scaffold retrieval versus direct descriptor matching.The downstream revision procedure is unchanged, isolating the retrieval-index comparison.Source: §J.1 · p. 30, Table S4 · p. 31

J.2 WHICH VISUAL DESCRIPTOR SHOULD QUERY THE SCAFFOLD?

  • None of the tested alternative visual descriptors improves over the production DINOv2 configuration on the final reconstruction.The top-five retrieval budget, geometric verification, online revision, and final refinement remain unchanged.Source: §J.2 · pp. 31–32

J.3 HOW MUCH HISTORICAL EVIDENCE IS NEEDED?

  • One retrieved pair substantially degrades 7-Scenes ATE, while three pairs give similar online performance to two and keep 7-Scenes ATE at 0.029.Disabling the online placement update raises 7-Scenes ATE from 0.030 to 0.042 m and lowers CO3Dv2 AUC@30 from 93.4 to 91.3.Source: §J.3 · p. 32

J.4 SENSITIVITY TO SCAFFOLD AND WINDOW SIZE

  • Performance is relatively stable across tested scaffold dimensions, but increasing address dimension does not consistently improve results, and torus resolution 642 can reduce accuracy.Window size has a stronger effect: larger windows and moderate overlap improve performance, while more overlap or smaller windows increase junctions and chaining error.Source: §J.4 · p. 32

K COMPUTE AND MEMORY MEASUREMENTS

  • SMORE keeps geometry inference bounded to the current window and reintroduces historical observations only when retrieved for revisit verification.Its external memory stores long-horizon information outside the geometry model’s active GPU context.Source: K COMPUTE AND MEMORY MEASUREMENTS · p. 33

K.1 PEAK GPU MEMORY WITH INCREASING SEQUENCE LENGTH

  • VGGT peak GPU allocation rises from 9.1 GB at N = 10 to 23.9 GB at N = 200 in the baseline, while SMORE stays between 8.5 and 9.6 GB.VGGT-Ω rises from 5.7 to 19.5 GB in the baseline but remains near 7 GB with SMORE.Source: §K.1 · p. 33
  • At N = 200, StreamVGGT exceeds the 46 GB budget under native inference but uses 11.4 GB with SMORE; STream3R exceeds the budget from N = 50 onward but uses 25.4 GB with SMORE.The content bank grows with stored observations, but its history is not part of the model’s active GPU context.Source: §K.1 · p. 33

K.2 COMPUTE AND RUNTIME AT N = 100

At N = 100, SMORE’s external-memory operations add computation and modest host-RAM costs, while sequential bounded-input passes can reduce peak GPU allocation. Runtime remains strongly dependent on the geometry model.

K.2 COMPUTE AND RUNTIME AT N = 100

  • At N = 100, host RAM rises from 7.42 to 8.60 GB for Fast3R, while increases for most other models are small.For example, VGGT changes from 10.77 to 10.78 GB, and DUSt3R from 16.94 to 17.02 GB.Source: §K.2 · p. 34
  • VGGT increases from 2.20 to 2.44 PFLOP and from 0.33 to 0.54 min runtime, while peak GPU allocation falls from 15.1 to 9.6 GB.VGGT-Ω rises from 1.08 to 1.26 PFLOP while GPU allocation falls from 11.9 to 6.9 GB; π3 falls from 26.5 to 14.1 GB despite computation rising from 1.52 to 1.82 PFLOP.Source: §K.2 · p. 34, Table S5 · p. 34
  • SMORE adds CPU processing, geometry-model passes, and trajectory-revision operations; absolute runtime is primarily determined by geometry-model inference.At N = 100, SMORE runtime is 0.38 min for VGGT-Ω and 0.42 min for Fast3R, versus 17.01 min for DUSt3R and 18.49 min for MASt3R.Source: §K.2 · pp. 33–34

L IMPLEMENTATION SETTINGS AND REPRODUCIBILITY

The implementation keeps geometry models frozen and updates only memory associations online; task protocols can vary. Diagnostic analyses distinguish changes to current-window predictions from explicit placement updates, showing that joint processing can affect trajectories even without an accepted revisit.

L IMPLEMENTATION SETTINGS AND REPRODUCIBILITY

  • Geometry models and the DINOv2 retrieval encoder remain frozen; only appearance–address associations are updated online.Novel-view-synthesis decoders are the only separately trained task-specific components; neither the geometry model nor SMORE is trained for them.Source: L IMPLEMENTATION SETTINGS AND… · p. 35
  • Task-specific windowing differs: camera-pose and SLAM follow their evaluation protocols, while Replica and ScanNet++ use 32-frame windows with 16-frame overlap.Novel view synthesis uses the same windowing and online revision on 7-Scenes and CO3Dv2, with every eighth keyframe held out and no final global refinement.Source: L IMPLEMENTATION SETTINGS AND… · p. 35

M.1 WHY REJECTING A REVISIT DOES NOT RESTORE THE ORIGINAL PREDICTION

  • In the seed-1 chess diagnostic, a joint-pass window reaches 46.1° rotational disagreement and ATE rises from 0.025 to 0.459 m despite both revisit proposals being rejected.The joint pass changes the retained current-window prediction, so rejecting a revisit discards its placement correction but does not restore the ordinary-pass prediction.Source: §M.1 · p. 37

M.2 THREE WAYS TO RETAIN THE CURRENT-WINDOW PREDICTION

  • The joint policy retains the joint-pass prediction; the ordinary policy retains the ordinary-pass prediction, and the gate selects between them using rotational disagreement.Passing the gate does not accept a revisit; geometric verification is still required.Source: §M.2 · pp. 37–38, Table S9 · p. 38

M.3 DOES THE LOCAL PREDICTION MATTER WITHOUT A PLACEMENT UPDATE?

  • With online placement updates disabled, ordinary-policy trajectories match raw trajectories on all seven 7-Scenes sequences, while joint-policy mean ATE changes from 0.0451 to 0.0423 m.This separates local re-prediction from explicit trajectory revision; the seed-1 failure also shows joint-pass distortion can degrade results without an accepted revisit.Source: §M.3 · p. 38

M.4 HOW MUCH JOINT-PASS DISTORTION SHOULD BE RETAINED?

The diagnostics examine when to retain changes to local predictions from joint processing with historical frames, and how those changes interact with trajectory revision. They show that joint processing can help on default proposals but can also introduce large errors in selected failure cases.

M.4 HOW MUCH JOINT-PASS DISTORTION SHOULD BE RETAINED?

  • 93.4 AUC@30 with the default joint policy exceeds 92.4 when always retaining the ordinary prediction on CO3Dv2; scores rise from 92.5 at 0.5° to 93.4 at 5°.On 7-Scenes, all tested thresholds give the same ATE at reported precision because measured distortion is at most 0.311° and gates of at least 0.5° retain the joint prediction.Source: §M.4 · p. 38

M.5 STRESS TESTS ON FAILURE CASES

  • 0.092 m mean ATE with joint predictions falls to 0.029 m with 1° gating in the seed-1 failure case; the same pattern appears for CosPlace, BoQ, and DINOv2+SALAD.These are intentionally selected failure cases, not evidence that 1° is universally optimal; ordinary geometry stays within approximately 0.048–0.053 m across nine appearance cues.Source: §M.5 · p. 39

M.6 WHAT THESE DIAGNOSTICS ESTABLISH

  • Local-prediction selection and global-trajectory revision are separate decisions; revisit verification alone does not protect against an incorrect historical proposal distorting the local prediction.Retaining the ordinary prediction removes direct dependence on historical frames, while gating accepts only sufficiently small changes.Source: §M.6 · p. 39
  • The default 5° gate retains all measured default joint predictions and rejects the tens-of-degrees distortions in the selected failure cases, but tests rotational disagreement only.It does not guarantee small translation, scale, or dense-point error; the placement property in Appendix C assumes the local prediction has already been selected, unlike these diagnostics.Source: §M.6 · p. 39

N MEMORY GROWTH AND PRUNING

The retrieval diagnostic tests memory growth and pruning by measuring whether stored frames provide nearby historical revisits. Scaffold retrieval improves Recall@1 over direct DINOv2 retrieval at 3,100 stored frames, while pose-based pruning substantially reduces redundancy and improves Recall@5 on retained queries.

N MEMORY GROWTH AND PRUNING

  • At 3,100 stored frames, scaffold retrieval achieves 0.36 Recall@1 versus 0.25 for direct DINOv2 cosine retrieval, while Recall@5 is comparable at 0.42 versus 0.43.A correct revisit is from the same scene, within 10% of the scene’s spatial extent, and within 30° orientation of the query camera.Source: N MEMORY GROWTH AND PRUNING · pp. 39–40
  • Pose-based insertion reduces the bank from 3,100 to 1,119 frames while preserving a valid revisit for 157 of 159 eligible queries; retained-query scaffold Recall@5 rises from 0.42 to 0.54.The results do not establish an optimal pruning strategy or unbounded memory capacity.Source: N MEMORY GROWTH AND PRUNING · p. 40

Related papers

Loading 2610.19674v1…