BlazeVGGT: 13x Faster VGGT at 2,000 Frames via Camera Token without Training

I. INTRODUCTION

• Existing accelerators may require training or operate on tokens and attention rather than complete views, leaving near-duplicate frames or fragmenting intra-frame geometric evidence.BlazeVGGT instead selects complete views as a complementary frame-level strategy.Source: §I · pp. 1–2
• Shallow camera tokens preserve local view neighborhoods, while deeper tokens become more consistent with global view geometry, especially rotation; metric translation remains less calibrated.The evidence supports local topology early, rather than complete metric 6-DoF pose recovery.Source: §I · p. 2
• Local topology can suppress redundant nearby views in shallow layers, while deeper global geometry supports broader scene coverage without an external pose estimator or feature encoder.This principle follows the layerwise evolution of the pretrained camera tokens.Source: §I · p. 2
• At 2,000 frames, inference takes approximately 100s instead of more than 1,300s (about 13× speedup), while Chamfer Distance falls from 0.41 to 0.32.The method also maintains competitive point-cloud reconstruction and camera-pose estimation performance.Source: §I · p. 3
• BlazeVGGT uses camera-token relations and DPP-based selection to sparsify frame-level key–value context while retaining all queries and each selected view’s complete token block.It is training-free and introduces no learnable parameters, enabling plug-and-play integration with pretrained VGGT.Source: §I · pp. 2–3
• BlazeVGGT is compatible with FastVGGT and LiteVGGT and transfers to StreamVGGT and PAGE-4D inference settings.The reported experiments include sequences up to 2,000 frames.Source: §I · p. 3

A. 3D Reconstruction

• Classical reconstruction combines correspondence, geometric estimation, and optimization; learned matching and dense multi-view stereo extend these pipelines, which remain challenging in sparse wide-baseline and large-scale settings.Neural rendering improves novel-view synthesis but typically requires per-scene optimization.Source: A. 3D Reconstruction · p. 3

B. Feed-Forward Visual Geometry Models

• Feed-forward models replace explicit per-scene optimization with learned geometric priors, while VGGT jointly infers cameras, depth, point maps, and tracks across an image collection.Long-range approaches include chunking, incremental processing, and divide-and-conquer.Source: B. Feed-Forward Visual Geometry Models · p. 3

C. Efficient VGGT

• Efficient VGGT methods either adapt pretrained models or modify inference through token merging, descriptor compression, or sparse attention; BlazeVGGT instead selects complementary complete views while retaining all queries.Its view-level operation is described as complementary to token merging, sparse attention, and model adaptation.Source: C. Efficient VGGT · p. 3

A. Preliminary

• BlazeVGGT uses camera-token similarities to select diverse frames, retaining all frames as queries while using selected frames as the global-attention key–value context.The resulting features are passed to the camera and depth heads.Source: Figure 3 · p. 4
• VGGT independently encodes each image into patch tokens and augments each view with one learnable camera token and four register tokens.The model jointly predicts camera parameters, depth maps, point maps, and point tracks from RGB images.Source: A. Preliminary · pp. 3–4
• VGGT alternates frame-wise attention, which processes each view independently, with global attention, which jointly processes tokens across views.At a global-attention block, per-view token blocks are concatenated and their projected query, key, and value representations are used for standard global self-attention.Source: A. Preliminary · p. 4
• With P tokens per view, dense global attention has (S·P)^2 pairwise interactions and computation grows quadratically with the number of input views.Every query token attends to key–value tokens from all S views.Source: A. Preliminary · p. 4

B. Our Method

• BlazeVGGT builds a view-level similarity kernel from camera tokens and uses greedy DPP selection to choose mutually complementary views.The selection operates on evolving camera-token relations rather than decoded camera poses.Source: B. Our Method · pp. 4–5
• A thresholded greedy rule selects views until the largest residual falls below ε or the maximum view budget K is reached.The selected-set size adapts: more redundant representations retain fewer KV views, while more diverse representations preserve a larger context.Source: B. Our Method · p. 5
• The same selection rule suppresses nearby redundant observations in shallow blocks and favors complementary scene coverage as deeper camera-token relations become geometry-aware.No explicit stage boundary or separate shallow- and deep-layer selector is required.Source: B. Our Method · p. 5
• 13.2× and 14.0× speedups are reported for BlazeVGGT on ScanNet-50 at 2,000 and 1,000 views, respectively.The 2,000-view setting uses two-GPU sequence parallelism; the 1,000-view setting uses single-GPU inference.Source: Table I · p. 6
• The evaluation captions cover point-cloud reconstruction on ScanNet-50, 7 Scenes, and NRGBD, reporting Chamfer Distance or accuracy, completeness, normal consistency, and inference time.The 7 Scenes and NRGBD caption specifies stride-3 sampling.Source: Table I · p. 6, Table II · p. 6
• All input frames remain Queries, while only selected views supply the KV context, preserving dense predictions and all tokens within each selected frame.The method changes global-attention routing without learned parameters and is composable with token- or attention-level accelerators.Source: B. Our Method · p. 6
• With a bounded retained-view budget, the dominant global-attention cost scales approximately linearly rather than quadratically with input-view count.The full-attention baseline uses SP Queries and SP KV tokens; BlazeVGGT reduces KV length to K(ℓ)P.Source: B. Our Method · p. 6

A. Experimental Setup

• The main experiments compare point-cloud reconstruction on ScanNet-50, 7 Scenes, and NRGBD, and camera-pose estimation on ScanNet-50 across input lengths.ScanNet-50 ablations examine the selection threshold, Top-K safety constraint, and activated layer range.Source: A. Experimental Setup · p. 6
• For the 2,000-frame comparison, adapted implementations run on two NVIDIA A800 GPUs connected by NVLink; remaining experiments use one A800 except specified LiteVGGT and high-frame-count PAGE-4D runs on an H20.The 2,000-frame setup preserves each method’s algorithmic configuration.Source: A. Experimental Setup · p. 6
• Generalization studies test composition with existing VGGT accelerators and transfer to StreamVGGT and PAGE-4D.They cover ScanNet-50 and Tanks & Temples for accelerator composition, and 7 Scenes, Sintel, Bonn, and DyCheck for model transfer.Source: A. Experimental Setup · p. 7

Point Cloud Reconstruction

• On ScanNet-50, BlazeVGGT’s speedup over VGGT grows with sequence length, reaching 14.0× at 1,000 views and remaining above 13× under two-GPU inference at 2,000 views.Source: B. Comparison with State-of-the-Art · p. 7

Camera Pose Estimation

• At 1,000 views, BlazeVGGT achieves the lowest ATE, ARE, RPE-t, and RPE-r among compared methods.Across 1,000 and 2,000 views, it has the best overall pose accuracy among training-free methods.Source: B. Comparison with State-of-the-Art · p. 7, Table III · p. 8

Qualitative Comparison

• BlazeVGGT preserves crisp local details and complete scene structure under frame-level KV pruning, while avoiding structural failures seen in some alternatives.Fig. 5 compares selected local regions and full scenes across reconstruction methods.Source: B. Comparison with State-of-the-Art · p. 7
• In the second trajectory sequence, BlazeVGGT stays close to ground truth and improves on VGGT and LiteVGGT, while FastVGGT has slightly lower displayed ATE.Fig. 4 compares trajectories in the x–z plane across VGGT, FastVGGT, LiteVGGT, and BlazeVGGT.Source: B. Comparison with State-of-the-Art · p. 8, Figure 4 · p. 8

Selection Threshold

• ε = 0.20 is the balanced default: ε = 0.35 gives the lowest ablation CD (0.4011), but higher ATE, ARE, RPE-t, and RPE-r than ε = 0.20.Increasing ε beyond 0.20 does not monotonically improve CD, and pose errors increase overall.Source: C. Ablation Studies · p. 8

Top-K Constraint

• Enabling or removing Top-K yields nearly identical CD and pose metrics; the constraint instead caps unusually large selected KV sets and makes computation more predictable.The effective number of selected views remains determined by camera-token similarity structure.Source: C. Ablation Studies · p. 8

Layer Range

• Applying view selection throughout the stack gives the strongest reconstruction improvement and suppresses redundant KV computation at every stage.Selection only in deepest blocks stays close to the unsparsified reconstruction; the default applies it to all global-attention blocks.Source: C. Ablation Studies · pp. 8, 10

Point Cloud Reconstruction

• On ScanNet-50 and Tanks & Temples, adding BlazeVGGT to FastVGGT or LiteVGGT provides further speedups, with reconstruction quality generally improved or nearly unchanged.On Tanks & Temples, average F1 scores remain nearly unchanged across the two evaluation thresholds.Source: D. Compatibility with Existing VGGT… · p. 10

Camera Pose Estimation

• At 1,000 frames, BlazeVGGT improves FastVGGT’s absolute pose errors and all reported LiteVGGT pose metrics while reducing reconstruction-time cost.For shorter sequences, adding sparsification can modestly degrade some pose metrics.Source: D. Compatibility with Existing VGGT… · p. 10
• Combining acceleration strategies does not necessarily improve all metrics when the input context is already small.The reported complementarity is strongest for long sequences with substantial redundancy.Source: D. Compatibility with Existing VGGT… · p. 10

Online Reconstruction with StreamVGGT

• For streaming, the original full-sequence kernel scale is unavailable in single-pass inference, so experiments fix it to its default value and recommend a dual-memory design.A short-term FIFO retains recent observations; a determinantal criterion maintains long-term KV history, with kernel scale computed online from that memory and a candidate frame.Source: E. Generalization to VGGT-based Models · p. 10

Compatibility with VGGT Accelerators

• The supplied compatibility table evaluates camera-pose errors before and after integrating BlazeVGGT into VGGT, FastVGGT, and LiteVGGT on ScanNet-50.It reports ATE, ARE, RPE-t, and RPE-r, with lower values better.Source: Table VII · p. 11

Online Reconstruction with StreamVGGT

• At 200 frames, StreamVGGT with determinantal selection reduces per-frame latency from roughly 738 ms to 95 ms and peak memory from 31.9 GB to 10.2 GB, remaining close in accuracy while improving completeness and normal consistency.It also outperforms fixed-window and nearest-neighbor pruning in efficiency and memory usage; qualitative comparisons show coherent layouts and clearer object boundaries.Source: E. Generalization to VGGT-based Models · p. 11, Table IX · p. 11, Figure 6 · p. 11

Video Depth Estimation with PAGE-4D

• At 500 frames, PAGE-4D throughput increases from 0.81 FPS to 5.06 FPS, while accuracy trade-offs vary across Sintel, Bonn, and DyCheck.Around 100 frames, throughput nearly doubles while depth-estimation quality remains broadly comparable across the three benchmarks and two alignment protocols.Source: E. Generalization to VGGT-based Models · pp. 11–12, Table X · p. 11

• BlazeVGGT uses evolving camera-token relations to select complementary views for key–value sparsification while retaining all queries.The paper identifies frame-level redundancy as a key bottleneck in VGGT.Source: §V · p. 12
• Experiments show substantial long-sequence efficiency gains with competitive geometric accuracy, alongside compatibility with existing accelerators and VGGT-based models.Source: §V · p. 12
• Future work will explore joint view-token sparsification and scene-adaptive computation.Source: §V · p. 12