CoV: Chain-of-View Prompting for Spatial Reasoning

1 Introduction

• Fixed, limited viewpoints make it difficult to acquire sufficient question-relevant context for complex EQA questions that require multi-step reasoning.The introduction illustrates this with a question about finding soda that is not directly visible.Source: §1 · p. 2
• CoV selects a question-relevant starting view, uses a bird’s-eye scene view for global understanding, then iterates camera actions and reasoning until enough information is gathered or the action limit is reached.This coarse-to-fine framework shifts from passive predetermined views to explicit question-aligned selection and fine-grained adjustment.Source: §1 · pp. 2–3
• 10.82% average improvement on OpenEQA, with a maximum gain of 13.62% on Qwen3-VL-Flash, is reported for CoV.The evaluation covers mainstream open-source and proprietary VLMs.Source: §1 · p. 2
• 2.51% average improvement as action steps increase, peaking at 3.73% on Gemini-2.5-Flash, supports CoV’s test-time scaling capability.The authors report gradual performance improvement with more exploration steps.Source: §1 · pp. 2–3
• Qualitative analyses report more coherent, interpretable reasoning chains, particularly in complex or cluttered environments.The authors frame the approach as improving scene understanding without additional model training or dataset-specific tuning.Source: §1 · p. 3

2 Related Work

• 3D scene-understanding methods combine object localization and description, while 3D VLMs join 2D multimodal perception with spatial context.Vote2Cap-DETR (Chen et al., 2023a), D3Net, and SpaCap3D integrate localization and description; LLaVA-3D (Zhu et al., 2024) and LL3DA (Chen et al., 2024) synthesize 2D perception with 3D context.Source: §2 · p. 3
• Training-free test-time adaptation includes in-context learning, prompt engineering, chain-of-thought prompting, scaling, adaptive compute, and calibration.The passage names Brown et al. (2020), Sahoo et al. (2024), Wei et al. (2022), Muennighoff et al. (2025a), Snell et al. (2024), and McKenna and Carse (2024).Source: §2 · p. 3
• 3D dense captioning moved from detect-then-describe pipelines toward end-to-end transformers, while EQA methods use multimodal fusion, navigation-conditioned reasoning, and LLM grounding.The passage cites Chen et al. (2021) and Wang et al. (2022) for early pipelines, and Chen et al. (2023a, 2022) and Huang et al. (2025a,b) for newer transformers.Source: §2 · p. 3
• 3D VLMs incorporate point clouds or multi-view images, with methods encoding global scene features, object-level features, or object-centric image patches.LL3DA (Chen et al., 2024), LEO (Yang et al., 2023), Chat-Scene (Zhang et al., 2024a), 3D-LLM (Hong et al., 2023), and Scene-LLM (Fu et al., 2024) illustrate these approaches.Source: §2 · p. 3

3.1 Problem Setting

• EQA generates a textual answer from observations of a 3D scene in response to a natural-language query.The paper models this objective as maximizing P(A | S, V, Q).Source: §3.1 · pp. 3–4

3.2 Method Overview

• CoV first filters input frames to question-relevant candidates, then adjusts perspectives within that subset before synthesizing an answer.The pipeline has coarse-grained view selection and fine-grained view adjustment stages.Source: §3.2 · p. 4

3.3 Coarse-Grained View Selection

• The View Selection Agent reduces redundant frames by retaining a small subset of views most relevant to the query.The selected subset V′ contains K views, where K ≪ T.Source: §3.3 · pp. 4–5

3.4 Fine-Grained View Adjustment

• The CoV agent uses discrete camera actions to generate new perspectives from the 3D scene, revealing information not visible in initially selected views.At each step, it converts an action instruction into an SE(3) transformation that updates the camera pose and yields a new viewpoint; reasoning stops when enough information is gathered or the action limit is reached.Source: §3.4 · p. 5

4.1 Benchmarks and Metrics

• Across OpenEQA, ScanQA, and SQA3D, the evaluation covers open-source and proprietary models and uses LLM-Match for OpenEQA and answer metrics for the other datasets.OpenEQA draws on more than 180 real-world environments; ScanQA focuses on object-grounded QA, while SQA3D assesses situated reasoning.Source: §4.1 · pp. 5–6

4.2 Implementation Details

• For each question, frames are uniformly sampled at 10:1; the baseline receives all images at once, while CoV filters views before its agent answers.Both agents use the same underlying VLM.Source: §4.2 · p. 6

4.3 Main Results

• ScanQA results include CIDEr 116 versus LEO’s 101.4 and EM@1 31.9%.The reported comparison covers ScanQA validation and SQA3D test results.Source: §4.3 · p. 6, Table 1 · p. 7

4.6 Qualitative Result

• In the qualitative example, CoV progressively explores object locations and produces an answer consistent with the scene’s spatial layout.The reported analyses also describe reasoning chains as coherent and interpretable, particularly in complex or cluttered environments.Source: Figure 4 · p. 6, §4.6 · pp. 7–8

4.3 Main Results

• On OpenEQA, CoV improves LLM-Match by an average of 11.56%, with a maximum gain of 13.62% on Qwen3-VL-Flash.The comparison is against the setting without CoV and is reported as training-free.Source: §4.3 · p. 6, Table 2 · p. 8

4.4 Test-Time Scaling

• Across VLMs, performance on OpenEQA improves as action steps increase; questions needing more steps trend higher in score, though most use one to three steps.The analysis examines the relationship between action-step count and performance.Source: §4.4 · p. 6, Figure 6 · p. 8

4.5 Ablation Studies

• Removing coarse view selection lowers average VLM performance by 4.59%, as redundant, low-information frames make question-relevant anchor identification harder.The ablation compares the approach with and without the coarse selection mechanism.Source: §4.5 · p. 7, Table 3 · p. 8

• CoV’s coarse-to-fine viewpoint adjustment helps its agents acquire question-relevant observations and address complex spatial reasoning through multi-step reasoning.Source: §5 · p. 8
• The agent’s performance improves as the number of action steps increases.Source: §5 · p. 8
• The framework emphasizes not only what the model sees, but also how it sees, remembers, and reasons.Source: §5 · p. 8

• In highly dynamic or cluttered environments, rapid context shifts may lead the coarse-to-fine paradigm to misinterpret scene elements.Source: Limitations · p. 9
• Overly long action trajectories may make excessive exploration introduce noise or hallucination.Source: Limitations · p. 9
• Suboptimal view selection can impair reasoning accuracy because CoV relies on selected views’ quality and relevance.Source: Limitations · p. 9
• The authors identify advanced view selection algorithms and adaptive reasoning mechanisms as directions for addressing these challenges.Source: Limitations · p. 9

• The baseline receives all view frames and the question, then generates an answer.Source: A Prompt Templates · p. 12
• The View Selection Agent receives all frames, the question, and frame indices, and outputs IDs of question-relevant views.Source: A Prompt Templates · p. 12
• The CoV agent receives coarsely selected views and the question, then answers after multi-step action execution.Source: A Prompt Templates · p. 12

• Across diverse indoor scenes, CoV identifies relevant details even when targets are partially occluded or distributed across views.Source: B Result Visualization · p. 12
• In the bathroom, CoV identifies soap and paper towel dispensers between two sinks and reasons about object material and spatial attributes.Source: Figure 10 · p. 13
• In the classroom, CoV localizes brown cushion sofas, chairs beside a table, and a tall bookshelf near the chalkboard.Source: Figure 11 · p. 13
• In the office, CoV identifies desk items and their positions, including a shelf to the right of the monitor and nearby chairs.Source: Figure 12 · p. 14
• In the kitchen, CoV answers questions about chair appearance, refrigerator proximity to glass doors, and the count of chairs facing a table.Source: Figure 13 · p. 14