SpaceR: Reinforcing MLLMs in Video Spatial Reasoning

• Video spatial reasoning [34] requires reconstructing 3D layouts from frames, yet MLLMs struggle because high-quality task-specific data are scarce and SFT is insufficient for deep reasoning.Unlike conventional video captioning [31], question answering, and temporal grounding [10], it requires inferring spatial structure rather than mainly recalling video content.Source: §1 · pp. 1–2
• 45.6% accuracy on VSI-Bench, exceeding GPT-4o [12] by 11.6% accuracy.SpaceR also reports gains on STI-Bench [17] and SPAR-Bench [39], while maintaining promising results on Video-MME [9], TempCompass, and LongVideoBench [32].Source: Figure 1 · p. 2, §1 · pp. 2–3
• Prior work found RLVR effective for reasoning in text [11] and multimodal tasks [7] [8] [23], motivating its use to improve MLLM video spatial reasoning.Video-R1 [8] extended GRPO with a temporal reward and reported promising video-understanding performance.Source: §1 · p. 2
• SpaceR-151k contains 151k samples: 91k spatial reasoning QA pairs curated based on ScanNet [5] and 60k general multimodal samples from Video-R1-260k [8].Its SR-91k portion covers six spatial reasoning tasks, including relative direction, object or room size, and appearance order.Source: §1 · p. 2
• SG-RLVR extends GRPO [28] with task-specific verifiable rewards and an optional cognitive map in <map> tags, whose inferred layout quality is evaluated by a map reward.The map imagination mechanism encourages models to infer spatial layouts during reasoning.Source: §1 · p. 2

2.1 Video Spatial Reasoning

• Unlike captioning, temporal grounding, and perception [31] [10] [14] [24] [19], video spatial reasoning reconstructs scene structure, and MLLMs still face shortcomings and scarce tailored training data.The task is important for world models [20] and embodied agents [6].Source: §2.1 · p. 3

2.2 Reinforcement Learning with Verifiable Reward

• RLVR methods including Visual-RFT [23] and LMM-R1 [25] have improved visual and mathematical reasoning, but video spatial reasoning remains underexplored.GRPO [28] helped demonstrate RLVR's potential for equipping MLLMs with reasoning capacity.Source: §2.2 · p. 3

Data Collection

• For SR-91k, ScanNet metadata and 24-FPS RGB clips support spatial QA, while each video receives a 10 × 10 object-distribution map.Object coordinates are derived from bounding-box centers projected onto a 2D map; the general-understanding subset uniformly samples 60,000 diverse Video-R1-260k QA instances.Source: §3 · p. 3

QA Generation

• ScanNet metadata generates multi-choice questions on relative distance, direction, and appearance order, plus numerical questions on size, absolute distance, and counting.Figure 2 illustrates these QA types and their corresponding video map.Source: §3 · p. 3, Figure 2 · p. 4
• Relative-distance answers use minimum Euclidean distances, directions use bounding-box centers, and appearance order uses objects’ first frame indices.Object or room size, absolute distance, and counting are also derived from point clouds, bounding boxes, and ScanNet metadata, respectively.Source: §3 · p. 4

Data Filtering

• Filtering limits QA pairs per video, balances correct multiple-choice positions, and adjusts numerical values; the resulting SR-91k retains 91k spatial QA pairs.These steps promote scene diversity and reduce answer-position bias and skewed or unrealistic numerical values.Source: §3 · p. 4

Data Statistics

• SpaceR-151k contains 151,310 samples: 91k spatial-reasoning QA pairs and 60k Video-R1-260k instances, spanning five verifiable QA types.The types are multi-choice, numerical, OCR, free-form, and regression.Source: §3 · p. 4, Figure 3 · p. 5

4.1 Verifiable Reward Function

• SG-RLVR uses task-specific verifiable rewards for multi-choice, numerical, OCR, free-form, and regression outputs.Format reward checks for <think> and <answer> tags; multi-choice reward uses exact match, and numerical reward evaluates relative accuracy across confidence thresholds. OCR uses WER, free-form uses average ROUGE-1, ROUGE-2, and ROUGE-L, and regression uses relative numerical distance.Source: §4.1 · p. 5

4.2 Map-Based Spatial Reasoning

• The map-imagination mechanism guides the model to generate an M × M scene map of object distributions, with map reward based on relative object-coordinate accuracy.The map reward averages relative accuracy across objects; a length-based reward encourages reasoning within [lmin, lmax] and applies only when the answer is correct.Source: §4.2 · p. 6
• Reward shaping gives additional map reward when the model answers correctly and understands the indoor scene, encouraging a more spatially aware reasoning policy.The augmented reward also includes format, task, and length rewards; map reward is omitted when the task answer is incorrect.Source: §4.2 · p. 6
• SG-RLVR computes response advantages from updated rewards within each group and adds KL divergence to keep the optimized policy from straying far from the reference model.The policy update follows GRPO’s clipped surrogate objective.Source: §4.2 · p. 6

5.1 Experimental Setups

• Evaluation covers VSI-Bench, STI-Bench, SPAR-Bench, Video-MME, TempCompass, and LongVideoBench; SpaceR is prompted to reason step by step on spatial tasks and answer directly on video-understanding tasks.The base model is prompted to answer directly, and evaluation uses 32 video frames.Source: §5.1 · pp. 6–7

5.2 Main Results

• SpaceR outperforms Qwen2.5-VL-7B-Instruct across all six benchmarks, surpasses GPT-4o on spatial reasoning, and reports accuracy gains of 11.2 on VSI-Bench.It also achieves higher accuracy than Qwen2.5-VL-7B-Instruct on Video-MME, TempCompass, and LongVideoBench.Source: Table 1 · p. 7, §5.2 · p. 7
• SG-RLVR improves performance across spatial reasoning and video-understanding benchmarks, whereas SFT improves some benchmarks but degrades others.The reported SFT gains are on VSI-Bench, Video-MME, and LongVideoBench; the comparison characterizes SG-RLVR as more generalizable.Source: §5.2 · p. 7
• Filtering SR-91k samples classified as all-correct or all-wrong leaves a balanced task-category distribution and yields gains across nearly all benchmarks after retraining.The authors exclude all-correct samples for low learning value and all-wrong samples as potential noise.Source: §5.2 · pp. 7–8, Figure 4 · p. 8

5.3 Ablation Study

• Removing map imagination lowers spatial-benchmark performance, removing general multimodal data lowers video-understanding performance, and removing SR-91k substantially lowers spatial performance.These ablations support the roles of map imagination, the 60,000 general multimodal samples, and SR-91k, respectively.Source: Table 2 · p. 8, §5.3 · p. 8

5.4 Qualitative Analysis

• On VSI-Bench examples, SpaceR maps the scene to choose the table nearest the stove and count at least three chairs, while Qwen2.5-VL-7B-Instruct answers stool and two.SpaceR’s chair count accounts for partially occluded objects.Source: §5.4 · pp. 8–9, Figure 5 · p. 9

• SpaceR-151k contains 91k spatial reasoning QA pairs and 60k samples for general video understanding.Source: §6 · p. 9
• SG-RLVR integrates task-specific reward functions and map imagination to guide spatial layout inference and foster structured reasoning.Source: §6 · p. 9
• Evaluations across three spatial reasoning benchmarks and video-understanding benchmarks validate SpaceR’s effectiveness and generalizability.Source: §6 · p. 9
• The current framework lacks adaptive control of reasoning depth, which may affect inference efficiency and is proposed for future work.Source: §6 · p. 9

QA Generation

• QA pairs are formatted by inserting corresponding objects into specified question templates.The templates are shown in Figure 6.Source: A More Details for Data Construction · p. 13, Figure 6 · p. 13

Data Filtering

• Filtering removes QA pairs involving noisy objects and numerical pairs whose objects are too small to identify.Examples of noisy objects include walls, floors, and ceilings.Source: A More Details for Data Construction · p. 13
• SR-91k excludes videos overlapping with VSI-Bench to ensure fair evaluation.The authors note that VSI-Bench is partially built on ScanNet.Source: A More Details for Data Construction · p. 13

Data Statistics

• The video-duration distribution for SR-91k is visualized in Figure 7.The passage provides no specific duration statistics.Source: A More Details for Data Construction · p. 13, Figure 7 · p. 13

B.1 Benchmarks Description

• Spatial evaluation covers VSI-Bench [34], STI-Bench [17], and SPAR-Bench [39] across diverse scenes and spatial reasoning settings.VSI-Bench has over 5,000 question-answer pairs across 288 indoor-scene videos; STI-Bench’s SR_sub subset has more than 2,000 pairs across six tasks. SPAR-Bench has over 7,000 pairs and single-view and multi-view settings.Source: §B.1 · p. 14
• General video evaluation uses Video-MME [9], TempCompass, and LongVideoBench [32] to assess video, temporal, and long-context understanding.Video-MME includes 900 videos and 2,700 questions; subtitles are excluded. TempCompass includes 410 videos and 7,540 questions, while LongVideoBench has 6,678 questions, uses its validation set, and excludes subtitles.Source: §B.1 · p. 14

B.2 Baselines Description

• The baselines include GPT-4o [12] and Gemini 1.5 Pro, Gemini 2.0 Flash, and Gemini 2.5 Pro.GPT-4o is described as a state-of-the-art MLLM; the Gemini models are noted for leading video-benchmark performance, with Gemini 2.0 Flash and Gemini 2.5 Pro showing enhanced complex reasoning abilities.Source: §B.2 · p. 14
• Additional baselines are VideoLLaMA3-7B [38], LLaVA-OneVision-7B [15], and MiniCPM-V-2.6 [35].VideoLLaMA3-7B uses Qwen2.5-7B and siglip-so400m-patch14-384; LLaVA-OneVision-7B combines Qwen2 and SigLIP; MiniCPM-V-2.6 is based on SigLIP-400M and Qwen2-7B.Source: §B.2 · p. 14
• Other baselines are InternVL2.5-78B [3], Kimi-VL-A3B-Instruct [30], and Qwen2.5-VL-7B-Insturct and Qwen2.5-VL-72B-Insturct [2].InternVL2.5-78B pairs InternViT-6B-448px-V2_5 with Qwen2.5-72B-Instruct; Kimi uses an MoE language model and MoonViT vision encoder; Qwen2.5-VL combines Qwen2.5 with a redesigned ViT.Source: §B.2 · pp. 14–15

B.3 Hardware Usage

• Training uses either 8 L20 80 GiB GPUs or 4 A800 80 GiB GPUs, while evaluation uses 4 L20 80 GiB GPUs.Source: §B.3 · p. 15

C More Empirical Results and Analyses

• The additional analyses include progressively increasing data scale and assess its variation across spatial-reasoning and video-understanding benchmarks.The section introduction identifies data scale as an analysis focus; Figure 8 specifies the benchmark domains but provides no results in the supplied passage.Source: C More Empirical Results and Analyses · p. 15, Figure 8 · p. 16

C.1 Impact of thinking on Model Performance

• In think mode, SpaceR SG-RLVR improves spatial reasoning but slightly reduces video-understanding performance.The comparison covers VSI-Bench, STI-Bench, SPAR-Bench, Video-MME, TempCompass, and LongVideoBench.Source: Table 3 · p. 15, §C.1 · p. 15
• Models not trained to reason, including Qwen2.5-VL-3B/7B-Instruct and SpaceR-Tiny SFT, drop on most benchmarks in think mode.They often produce shorter, less informative reasoning traces, reflected in lower average token counts.Source: Table 3 · p. 15, §C.1 · p. 15
• Because answer-focused SG-RLVR does not directly supervise reasoning traces, thinking can be inconsistent or add misleading noise to direct-answer video tasks.The authors identify supervising reasoning and adaptively deciding when and how to think as future directions.Source: §C.1 · p. 15

C.2 Impact of Model Size on Performance.

• Across 3B and 7B models, SpaceR-Tiny SG-RLVR and SpaceR SG-RLVR outperform base and SFT counterparts on multiple spatial benchmarks, especially in think mode.They also show promising generalizability on video-understanding benchmarks.Source: §C.2 · p. 16
• Qwen2.5-VL-3B/7B-Instruct [2] have limited spatial reasoning and struggle to reason coherently in think mode, while SFT yields modest VSI-Bench gains without substantial reasoning improvement or generalization.The models were fine-tuned using SpaceR-151k with both SFT and SG-RLVR.Source: §C.2 · p. 16

C.3 Impact of Data Scale on Model Performance.

• SpaceR shows notable VSI-Bench and SPAR-Bench (Single View) improvements with just 5k–15k training examples, indicating strong data efficiency.The authors attribute this efficiency to SG-RLVR.Source: §C.3 · p. 16
• Performance generally plateaus or fluctuates slightly as training data grows to 30k.Source: §C.3 · p. 16
• With the full 151k dataset, performance jumps substantially, especially on VSI-Bench, where accuracy exceeds 45%.Source: §C.3 · p. 16

D Documentation and Licensing

• SpaceR-151k is provided in JSONL with associated images and videos, and the SpaceR model uses the Qwen2.5-VL-7B-Instruct architecture.Source: D Documentation and Licensing · p. 16
• SpaceR-151k and SpaceR are released under CC BY-NC 4.0 for academic research; ScanNet [5] is released under the MIT license.Source: D Documentation and Licensing · p. 16