Source-linked AI summary
CoVeR: Coverage-Based Token Pruning for Multi-View 3D Reasoning in VLMs
Nhat-Tan Bui, Varshini Elangovan, Arun Reddy Anugu, Sreyas Mohan, Wei Ye, Dilin Wang, JQ Huang, Rakesh Ranjan, Aviral Chharia, Fernando De la Torre
TL;DR
Multi-view 3D reasoning creates redundant visual tokens, while existing learned and voxelization-based pruners struggle with spatial coverage or exact budgets. CoVeR selects tokens deterministically from their 3D coordinates, combining voxel initialization with distance-based expansion. Across three benchmarks and four VLMs, it achieves state-of-the-art results and retains 93.5% of full-token performance at 8% retention.
Problem
Multi-view 2D VLM reasoning produces costly redundant tokens, while learned importance and voxelization methods are limited by scene under-coverage, budget variability, or voxel saturation.
Method
CoVeR is a training-free, deterministic, geometry-only selector that combines adaptive voxel coverage initialization with spatial-distance expansion to meet an exact per-scene budget.
Results
CoVeR achieves state-of-the-art performance across ScanQA, SQA3D, and OpenEQA while generalizing across four VLMs; at 8% retention it preserves 93.5% of full-token performance.
Takeaways & Limitations
CoVeR provides a plug-and-play pruning module that preserves strong 3D reasoning performance under aggressive token reduction and exact per-scene control.
Takeaways & Limitations
CoVeR requires depth and camera information and is designed for indoor scenes, so performance may depend on estimated geometry quality.
Abstract
from arXiv · showhide
Representing a 3D scene as multi-view images allows 2D VLMs to reason in 3D by reusing priors from pre-training, sidestepping the scarcity of annotated 3D data. However, it produces thousands of redundant visual tokens whose cost grows with every view. Existing visual token pruners fall into two families, each limited in the 3D multi-view setting. Learned importance methods rank tokens by attention or encoder features; because redundancy here is fundamentally spatial, they keep near-duplicate tokens from a few prominent regions and leave most of the scene unrepresented. Voxelization methods improve spatial coverage but cannot enforce an exact token budget and saturate as multi-view observations overlap in 3D, capping retention well below the target. We show that spatial coverage is associated with 3D reasoning performance and introduce CoVeR, a deterministic, training-free selector that uses only token coordinates, with no learned signals. CoVeR selects tokens that collectively cover every region of the scene, and solves the limitations of both families: it enforces an exact per-scene budget, breaks the voxelization saturation plateau, and avoids the near-duplicate selections of learned importance. Extensive experiments show CoVeR outperforms prior SOTAs on all three 3D reasoning benchmarks and generalizes as a plug-and-play module tested across four VLMs. Notably, with only $\approx$8% of visual tokens, it preserves 93.5% of full-token performance, surpassing SOTA by 3.9 percentage points on average across benchmarks.
1. Introduction
Multi-view rendering lets 2D VLMs reason about 3D scenes despite scarce 3D-language data, but visual-token cost grows with every view. CoVeR addresses pruning limits by targeting spatial coverage while enforcing exact budgets.
- Motivation: 12 views produce 8,748 visual tokens in LLaVA-OneVision-7B, substantially increasing inference cost for multi-view 3D reasoning.The token count grows linearly with the number of views.
- Limitations of Prior Pruning: Learned importance methods rank attention or visual features, but multi-view redundancy is spatial and can leave most of the scene under-covered.The supplied comparison identifies learned importance and voxelization as the two limited pruning families addressed by CoVeR.
- Limitations of Prior Pruning: Voxelization improves coverage but cannot enforce exact per-scene budgets and saturates when overlapping views share voxels.About 31% of tokens overlap, leaving voxel-only methods with roughly 69% average retention and 46–57% in highly redundant scenes.
- CoVeR: CoVeR is a training-free, deterministic, geometry-only selector that optimizes scene coverage under a guaranteed exact per-scene budget.It is designed to avoid near-duplicate selections while preserving scene-wide representation.
2. Related Works
Prior 3D token pruners either reduce tokens through voxel aggregation or rank them using learned signals. CoVeR differs by using geometry alone, deterministic selection, and exact per-scene budgets.
- Voxelization-based Pruning: Voxelization-based methods back-project tokens into 3D and reduce tokens within voxels, but their output counts remain tied to scene geometry.VTC averages voxel features into synthetic tokens, while DTC increases voxel resolution and merges by feature similarity.
- Voxelization-based Pruning: Voxelization-based pruning reports dataset-average budgets rather than exact per-scene counts.Scene geometry and view overlap make token counts vary across scenes.
- Learned Importance-based Pruning: Learned importance methods anchor selection on attention or visual features, with methods such as SeGPruner combining attention-ranked initialization and semantic-spatial diversity.Geo3DPruner adds a VGGT encoder and retrains the backbone, while VisPruner uses text-visual attention and encoder features.
- CoVeR: CoVeR uses no attention, encoder features, or semantic similarity and provides deterministic selection with exact per-scene budgets.Its geometry-only design removes model-specific dependence while giving precise control over token count.
3. Methodology
CoVeR formulates token pruning as selecting exactly B world-coordinate tokens that cover the observed scene. It combines adaptive voxel initialization with farthest-point expansion to overcome voxel saturation and return exactly B tokens.
- Problem Formulation: Given posed RGB-D views, a frozen visual encoder produces M patch tokens, each back-projected to a world coordinate t_i.The selector uses token coordinates rather than learned importance signals.
- Problem Formulation: The selected index set C must satisfy |C| = B, and coverage is measured by the directed Hausdorff distance from each scene token to its nearest retained token.The objective corresponds to a discrete Euclidean k-center problem.
- Why Voxelization Is Insufficient: Voxelization maps tokens to occupied voxels and keeps one representative per voxel, but occupancy depends on scene geometry and cannot reliably equal B.At vs=0.2 m and B=1342, 56% of scenes fall below budget and 44% exceed it; retention also plateaus near 69% because overlapping observations share locations.
- Coverage Initialization: Coverage initialization adaptively searches voxel size and retains an original token per occupied voxel to form a coarse scene-wide cover.The search estimates voxel size per scene, and the representative is chosen near the voxel occupancy center.
- Budget Guarantee: The selector includes safeguards for undershooting or overshooting the initial occupancy, so its final output always has exactly B tokens.The exact-budget guarantee is independent of where the heuristic voxel search terminates, for any feasible B ≤ M.
- Coverage Expansion: Coverage expansion repeatedly adds the unselected token farthest from the current selection until exactly B tokens remain.Initializing FPS with voxel representatives directs additions toward regions underrepresented by the initial cover.
- Coverage Guarantee: In the common inactive-safeguard case, voxel initialization bounds worst-case coverage distance by 3v_s, while FPS makes d_H(X, C) nonincreasing during expansion.The bound applies when the initial occupied-voxel count does not exceed B.
4. Experiments and Results
Across three 3D reasoning benchmarks, CoVeR delivers the strongest aggregate performance while substantially reducing token-related computation and memory. Coverage analyses show that its geometry-only selections preserve informative regions, expand scene coverage, and transfer across models and view counts.
- Main Results: CoVeR retains 93.5% of full-token performance at 8% token retention, outperforming SeGPruner and VisPruner across the three datasets.It achieves the best aggregate performance at every token budget and also outperforms prior pruning methods on OpenEQA at aggressive budgets.
- Geometric Coverage Analysis: CoVeR is more spatially uniform than SeGPruner, with NNI 0.924 versus 0.458 and higher NND95 and NND100 coverage.The largest coverage gap occurs on NND100, the worst-case measure; all gaps are significant under paired statistical tests.
- Geometric Coverage Analysis: At 9% ScanQA retention, every SeGPruner token lies within 3.1% of the scene diagonal of a CoVeR token, while roughly 20% of CoVeR tokens cover additional regions.TR is 0.009 and TE is 0.020, indicating that CoVeR stays close to SeGPruner’s selections while expanding beyond them.
- Efficiency and Generalization: At a 9% budget, CoVeR uses 13.3× fewer TFLOPs, a 10.7× smaller KV cache, and achieves a 2.9× speedup for a 1.1-point performance drop.Against learned importance pruners, it also combines the highest performance with the lowest peak GPU memory by selecting from 3D coordinates alone.
- Efficiency and Generalization: CoVeR retains 93.5% of full performance versus 90.7% for Geo3DPruner and transfers without changing its selection rule or α across additional VLM backbones.It retains over 96% of ScanQA performance above 20% retention on Qwen2.5-VL-7B and Qwen3-VL-8B, and over 95% on SQA3D until retention falls below 20%.
- Design Ablations: Geometric distance is consistently strongest across budgets, preserving visually similar instances at different locations that semantic distance can suppress.At the tightest budget, it beats spatial+semantic FPS by 1.7/2.1 on ScanQA/SQA3D and semantic-only FPS by 2.9/4.3.
5. Conclusion
CoVeR is a training-free, deterministic, geometry-only framework that addresses token-pruning limitations through exact per-scene budgets and spatial coverage. It achieves state-of-the-art results across three benchmarks and generalizes across four VLMs without retraining, while depending on depth and camera estimates and targeting indoor scenes.
- 5. Conclusion: CoVeR uses spatial distance to extend beyond voxelization saturation while selecting tokens under an exact per-scene budget.It is training-free, deterministic, and geometry-only.
- 5. Conclusion: CoVeR achieves state-of-the-art performance on three benchmarks and generalizes across four VLMs from two model families without retraining.
- 5. Conclusion: CoVeR requires depth and camera estimates and is designed for indoor scenes, so performance may depend on estimated-geometry quality.The paper proposes combining coverage with reliable depth/pose estimation and extending selection to outdoor scenes as future work.
Supplementary Material
The supplementary material details the evaluated benchmarks, metrics, model and implementation settings, and coverage measures used to assess CoVeR's spatial selection behavior.
- Benchmarks: The evaluation covers ScanQA, SQA3D, and OpenEQA for spatial, situated, and embodied 3D reasoning.The benchmarks probe recognition, attributes, counting, localization, spatial relations, and related capabilities.
- Evaluation details: LLM-Match assigns scores from 1 to 5, with 1 indicating an irrelevant answer and 5 indicating a correct answer.The supplementary material follows the official OpenEQA prompt template and reports implementation versions for reproducibility.
- Models and settings: The compared VLM configurations include LLaVA-OneVision-7B, Qwen2.5-VL-7B, Qwen3-VL-8B, and Video-3D-LLM settings.In the 12-view setting, the listed models produce different visual-token counts per image and scene.
- Coverage analysis: Coverage is measured with NNI and NND, while TR tests whether CoVeR reaches regions selected by learned pruning and TE tests regions it omits.NND95 clips the worst 5%, whereas NND100 is the worst-case distance related to the directed Hausdorff distance.
A.7. Ablation Study
The ablation study separates voxel-based initialization from geometric expansion and evaluates their spatial-coverage behavior, including empirical validation of the coverage bound.
- Stage 1: Preserving encoder-native tokens: Stage 1 preserves encoder-native tokens by selecting representative tokens from occupied voxels under an exact budget.The ablation sets α = 1 and adjusts voxel search to return exactly B tokens, including a densest-token tie-break when necessary.
- Stage 1: Preserving encoder-native tokens: Merging differs from pruning because it averages voxel features into synthetic representations, whereas pruning retains real encoder features.Both approaches produce one feature per occupied voxel on the same partition.
- Stage 2: Geometric distance: Stage 2 isolates geometric expansion by targeting the full budget with spatial, semantic, and combined token distances.The spatial distance uses squared 3D-coordinate distance, while the semantic distance uses cosine distance between normalized features.
- Stage 2: Geometric distance: The spatial and semantic distance terms are normalized by the squared scene diagonal and cosine range because their scales differ.The formulation also uses scene coordinate extrema to define the spatial normalization.
- Coverage-bound validation: At 9% token retention, the measured directed Hausdorff distance is 2.3% of the scene diagonal, below the stage 1 bound of 9.6%.
B. Theoretical Analysis of CoVeR
CoVeR’s analysis establishes exact budget control, non-increasing coverage error during expansion, and a 2-approximation guarantee at the expansion budget. The guarantees avoid solving the NP-hard k-center objective, with a safeguard for oversized initial selections.
- Coverage monotonicity: Stage 2 cannot worsen coverage: dH(X, Cj+1) ≤ dH(X, Cj), and therefore dH(X, C) ≤ dH(X, Cinit).Each expansion adds a token, enlarging the selected set and reducing or preserving nearest-token distances.
- Exact budget: CoVeR returns exactly B tokens whenever B < M and |Cinit| ≤ B.Stage 2 adds B − |Cinit| new tokens, ensuring |C| = B.
- Approximation guarantee: CoVeR achieves dH(X, C) ≤ 2 · OPT_Bexpan(X) when Bexpan ≥ 1.The bound compares the final selection with the optimal Bexpan-point k-center solution.
- Interpretation: The guarantees are complementary and avoid solving the NP-hard k-center objective.Stage 1 supplies an absolute 3v_s coverage bound, while stage 2 provides the 2-approximation.
- Safeguard case: If |Cinit| > B, CoVeR keeps representatives from the B most populated voxels, so the budget remains exact but Lemma 1 need not hold.This safeguard case does not occur at any evaluated budget or split.
C. Results on Token Recovery and Expansion
Token Recovery shows that CoVeR preserves nearby coverage of regions selected by SeGPruner, while Token Expansion indicates additional coverage of regions SeGPruner misses. This supports a coverage advantage over learned importance selection.
- Token Recovery: At every budget, Token Recovery reaches one, with CoVeR retaining a nearby token for every region selected by SeGPruner.The reported maximum distance ranges from 1.1% to 2.4% of the scene diagonal across budgets.
- Token Expansion: At the tightest budget, CoVeR retains nearby tokens within 2.4% of the scene diagonal for regions selected by SeGPruner.Token Expansion additionally indicates coverage of regions SeGPruner leaves unrepresented.
- Comparison: CoVeR preserves informative regions selected by learned importance methods while also covering other scene regions.The comparison attributes the added coverage to prioritizing spatial coverage rather than only learned importance.
D. Additional Quantitative Results
Additional quantitative results show strong performance in OpenEQA category analysis and competitive absolute performance against broader 3D reasoning models after substantial token pruning.
- OpenEQA categories: At 17% retention, CoVeR’s OpenEQA spatial score exceeds the full model under GPT-4 LLM Match, 45.1 versus 43.6, and GPT-4o, 50.5 versus 49.1.These comparisons concern the spatial-understanding category.
- OpenEQA categories: At 8% retention, CoVeR achieves the best OpenEQA object-recognition score, 47.3 versus 44.6 for SeGPruner and 40.5 for VisPruner.Overall retention across both judges is 89.1–89.7%.
- Broader model comparison: CoVeR reaches 50.1 OpenEQA LLM-Match with 8% of tokens and loses only 0.7 points at 26% while outperforming prior models.On ScanQA, 23% retention yields 28.5 EM@1, exceeding the listed task-specific and open VLM systems.
E. Additional Qualitative Results
Qualitative visualizations across ScanQA, SQA3D, and OpenEQA show that CoVeR preserves diverse visual evidence for multiple forms of 3D reasoning despite substantial token pruning.
- Qualitative evaluation: Across token budgets and 3D reasoning tasks, CoVeR preserves visual evidence for object understanding, spatial reasoning, situated reasoning, and embodied question answering.The visualizations complement the quantitative findings by illustrating retained scene information after pruning.
- OpenEQA pruning: With 92% of visual tokens removed, CoVeR reaches 50.1 LLM-Match compared with the 56.2 full-token LLaVA-OV-7B baseline.The table reports a −6.1 change from the base model.
F. Broader Impact
CoVeR reduces visual-token use while maintaining baseline VLM performance and precise token-reduction control. The paper also notes potential deployment benefits alongside privacy and inherited-model risks.
- CoVeR significantly reduces inference computation and memory while maintaining baseline VLM performance.
- CoVeR provides precise token-reduction control in VLMs and may lower deployment energy costs in robotic and embodied AI applications.
- CoVeR remains competitive with task-specific 3D models, video-LMMs, and fine-tuned 3D-LMMs after removing up to 91% of visual tokens.
- Real-world visual reasoning can raise privacy concerns, particularly in healthcare or autonomous driving.
- As a general-purpose method, CoVeR may inherit societal risks, biases, and failure modes from its underlying VLMs and data.