Uruqi: Learning Spatial Cognition from Visual Experience

1 INTRODUCTION

• Task accuracy can diverge from coherent spatial understanding: models may predict inaccurate locations or motion, produce inconsistent object locations across views, or depend on query order.In the examples, GPT-5.5’s object predictions remain consistent across views but change with M1/M2 query order.Source: §1 · p. 1
• Prior intermediate-supervision approaches use linguistic traces, explicit maps, structured representations, or programs, but leave open how to capture spatial information evolving across observations and generalize to unseen tasks.The paper notes that linguistic traces can be ambiguous for geometric transitions, while maps and programs rely on predefined representations and task-specific operations.Source: §1 · pp. 1–2
• Observer self-motion connects consecutive observations and determines how spatial information should change between viewpoints.The paper argues that motion estimation should be coupled with spatial-state updates, whereas separate task-specific questions leave that transition largely implicit.Source: §1 · p. 2
• URUQI generates motif-driven camera trajectories in diverse 3D scenes and compiles each into shared episodes linking self-motion tracking, persistent object mapping, and spatial operations.The episodes place motion, state updates, and operations within the same evolving visual experience rather than treating them as isolated questions.Source: §1 · pp. 2–3
• URUQI-Syn-8B improves episodic spatial reasoning and transfers to external spatial benchmarks; URUQI-SI-Mix further benefits from adding 100K SenseNova-SI-8M examples.The paper also reports improved self-motion estimation and maintenance of previously observed object locations during dense trajectory evaluation.Source: §1 · p. 3

Spatial Intelligence in Vision-Language Models

• Spatial instruction data, geometric priors, and specialized tools have advanced spatial reasoning, motivating the question of whether VLMs can learn spatial understanding natively from visual experience.URUQI uses privileged geometry to supervise how spatial information evolves during training.Source: §2 · p. 3

Intermediate Supervision for Spatial Reasoning

• Intermediate-supervision methods expose linguistic traces, cognitive maps, or visual and geometric states, but their targets typically depend on a particular reasoning format or representation.URUQI derives supervision from scene geometry to capture observer motion and spatial-information updates as shared components of reasoning.Source: §2 · p. 3

Spatial Reasoning from Egocentric Visual Experience

• Egocentric-video and persistent-memory research studies spatial evidence across viewpoints, while URUQI focuses on maintaining spatial understanding through observer motion without complete scene reconstruction.The related work describes self-motion as supporting continuous spatial updating without requiring complete scene reconstruction.Source: §2 · p. 3

3.1 PROBLEM FORMULATION

• The VLM observes only the RGB sequence, while world geometry and camera poses are privileged signals used to construct supervision for motion, spatial-state transitions, and query-dependent operations.The spatial state is an abstraction for supervision, not an assumption that the VLM maintains an explicit map or a particular internal representation.Source: §3.1 · p. 4

3.2 MOTIF-DRIVEN VISUAL EXPERIENCE GENERATION

• Spatial motifs specify reusable patterns of observer motion and visibility, which are instantiated as feasible trajectories in 3D scenes.The procedure produces 11,738 trajectories covering varied ways spatial information can emerge, disappear, and reconnect over time.Source: §3.2 · p. 5

3.3 COMPILING SPATIAL SUPERVISION INTO EPISODES

• For each trajectory, camera poses, object identities, and visibility annotations yield motion, object-mapping, and query-operation targets, including locations of objects no longer visible.Targets are aligned to when their supporting visual evidence becomes available; entities not yet observed are introduced only after their first supporting observation.Source: §3.3 · p. 5
• The episode preserves observation order and accumulates prior observations and QA exchanges, interleaving M1, M2, and M3 targets within the same context.Training uses autoregressive next-token prediction on answer tokens conditioned on the available images, current question, and preceding exchanges.Source: §3.3 · pp. 5–6

4.1 EXPERIMENTAL SETUP

• The benchmark contains 52,920 QA instances across 2,692 episodes from eight held-out scenes, and multi-turn evaluation presents observations and questions in temporal order.URUQI-Syn uses synthetic-only supervision; URUQI-Mix adds 100K SenseNova-SI-8M examples, while URUQI-SI-Mix starts from SenseNova-SI-1.5.Source: §4.1 · pp. 6–7

4.2 EVALUATING SPATIAL CAPABILITIES IN CONTINUOUS VISUAL EXPERIENCE (RQ1)

• Post-training InternVL3-8B solely on synthetic supervision raises its Overall score from 15.84% to 47.73%, with improvements in all 11 subclasses.Table 1 evaluates self-motion estimation (M1), object mapping (M2), and operations over spatial state (M3).Source: §4.2 · p. 8
• URUQI-SI-Mix-8B reaches 50.41% Overall, the highest among evaluated models, closely followed by GPT-6 Astra at 50.08%; GPT-6 Astra retains a higher M2 average (49.64% vs 37.41%).URUQI-SI-Mix-8B improves all 11 subclasses from a 17.02% Overall starting point.Source: §4.2 · p. 8

4.3 EVALUATING TRANSFER TO EXTERNAL SPATIAL BENCHMARKS (RQ2)

• URUQISyn-8B improves overall scores from 42.14% to 47.26% on VSI-Bench, from 27.80% to 33.10% on MMSI-Bench, and from 38.19% to 45.90% on MindCube-tiny.Adding 100K SenseNova-SI-8M examples yields 57.56%, 39.80%, and 56.00% on those benchmarks, respectively, for URUQI-Mix-8B.Source: Table 2 · p. 7, §4.3 · p. 8

4.4 DENSE EVALUATION OF EGOCENTRIC SPATIAL TRACKING (RQ3)

• Under No QA history, URUQISyn-8B reaches 49.56% overall localization accuracy, versus 1.00% for InternVL3-8B and 6.44% for Qwen3.8-27B.The evaluation covers 100 trajectories, 2,312 frames, and 231 registered objects; it also reports 35.13% absent-object accuracy for URUQISyn-8B.Source: §4.4 · p. 9
• Model history improves motion estimation but reduces URUQISyn-8B’s overall localization accuracy from 49.56% to 39.62%; GT motion history yields 38.30% absent-object accuracy.All three history conditions provide the full observed image history; they differ in whether prior questions and answers or ground-truth motion answers are included.Source: §4.4 · p. 9
• With No QA history, GT-pose propagation raises absent-object accuracy from 35.13% to 71.61%; with Model-history predictions fixed, predicted poses yield 34.02%, versus 72.86% with GT poses.The latter comparison reports 3.97% for Qwen3.8-27B using predicted poses.Source: §4.4 · p. 9, Table 4 · p. 9

5 CONCLUSION

• 50.41% on the URUQI benchmark is achieved by the 8B model, which the conclusion says matches GPT-6 Astra and ranks best among evaluated open-source models.It also leads open-source results on two of three external benchmarks.Source: §5 · pp. 9–10
• The conclusion presents continuous visual experience as a scalable supervision paradigm for spatially capable VLMs, jointly linking motion tracking, persistent mapping, and spatial operations.Source: §5 · pp. 9–10
• The models are for research; experiments use simulated or benchmark environments, and safety-critical deployment was not evaluated.Source: ETHICS STATEMENT · p. 10

A QUALITATIVE EXAMPLES OF EGOCENTRIC SPATIAL TRACKING

• With No QA history, the TV remains localized across changing viewpoints, and the picture has 0.08 and 0.05 m errors at two later observations when absent.Source: Figure 3 · p. 14, Figure 4 · p. 14, A QUALITATIVE EXAMPLES OF EGOCENTRIC… · p. 14
• In matched examples, history effects vary by step: the absent cabinet has lower error with Model history at t = 15 but higher error at t = 26.The painting’s error decreases after reappearance under both history conditions.Source: Figure 5 · p. 15, Figure 6 · p. 16, Figure 7 · p. 17

B.1 MOTIF INSTANTIATION

• Seven trajectory motifs impose distinct motion, visibility, and co-visibility constraints, from rotating in place to landmark chains.Source: §B.1 · p. 18

B.2 TRAJECTORY RENDERING AND VALIDATION

• Rendered trajectories retain RGB, pose, depth, and identity channels; visibility and occlusion evidence checks validate the required transitions.Source: §B.2 · p. 18

B.3 TRAJECTORY STATISTICS

• The source pool contains 11,738 trajectories across 50 BEHAVIOR scenes, with 212,209 rendered observations; each trajectory yields multiple QA episodes.Source: §B.3 · p. 18

C.1 COORDINATE CONVENTIONS AND MOTION TARGETS

• Motion targets encode displacement in the earlier observation’s forward-left frame and signed heading change; object locations express the same static object across observer frames.Source: §C.1 · p. 19

C.2 OBJECT IDENTITY AND PERSISTENT LOCALIZATION

• Instance identities link clearly observed objects across viewpoints, enabling location-query sequences through absence and, when available, reappearance.Source: §C.2 · p. 19

C.3 OPERATIONS OVER SPATIAL STATE

• M3 supervision applies spatial operations to information established along a trajectory, organized into six broad families.Source: §C.3 · p. 19, Table 5 · p. 20

D.1 EVALUATION PROTOCOL

• Objects register after two consecutive qualifying observations; at most three are registered chronologically and remain queryable when invisible or outside the registration range.Registration requires specified distance, mask-size, boundary, and coverage conditions.Source: §D.1 · p. 20
• After each first frame, evaluation queries motion from the preceding frame and current-frame locations of registered objects; 87 of 100 trajectories include absent-object queries.Source: §D.1 · p. 20

D.2 BASELINE PROMPTS

• Baseline prompts ask for numeric displacement and heading change or forward/backward and left/right object distances in the current camera frame.Source: §D.2 · pp. 20–21

D.3 METRICS AND GEOMETRIC PROPAGATION

• Localization accuracy counts every query, while mean position error averages valid estimates; missing or unparseable predictions count as incorrect.Source: §D.3 · p. 21
• Self-motion translation error is the Euclidean difference between predicted and ground-truth horizontal displacement; heading error uses wrapped angular difference.Source: §D.3 · p. 21
• Geometric propagation initializes a fixed world-plane point from a valid visible-object estimate and computes later observer-relative locations; subsequent VLM location estimates do not update it.The pose-source comparison varies whether poses are ground truth or integrated from predicted motion, holding object-location and auxiliary visibility predictions fixed.Source: §D.3 · p. 21

• Table 4’s supplement reports overall accuracy and mean position errors under the same evaluation conditions.Source: §D.4 · p. 21
• Table 7 compares direct prediction with pose-based propagation using GT poses, then varies the camera-pose source while holding model-history predictions and propagation fixed.GT poses in panel (b) are supplied only to the geometry module, not the VLM.Source: Table 7 · p. 22
• Acc@0.5m uses a 0.5 m horizontal-error threshold in the current camera frame, and all metrics weight queries equally.The table covers 4,207 All queries and 2,240 Absent queries.Source: Table 7 · p. 22

• The diagnostic gives the model images through the current frame, including the marked registration image, and asks visibility before requesting a pixel on visible objects.Valid positive responses require an in-bounds pixel; uncertain or invalid responses cannot initialize propagation, and instance masks score correspondence only.Source: §D.5 · p. 22
• Pixel Hit is the share of ground-truth-visible queries with a valid positive response and a pixel on the correct instance; missed detections and invalid responses receive no credit.Ground-truth visibility requires at least one target-instance pixel; 1,967 queries are visible and 2,240 are absent.Source: §D.5 · p. 22
• URUQISyn-8B achieves the highest Pixel Hit but has a higher false-visible rate than Qwen3.8-27B (40.54% versus 4.78%).The metrics are computed over their respective query sets, not only at initialization.Source: §D.5 · pp. 22–23, Table 8 · p. 22