Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence

Motivation

• Current VLM spatial reasoning remains challenging, and prior work mainly improves it through post-training or agentic use of external spatial tools.Post-training includes supervised fine-tuning and reinforcement learning, whereas tool-augmented agents gather intermediate evidence through depth estimation, reconstruction, or related spatial tools.Source: §1 · p. 2
• SMA targets an underexplored self-evolution route in which a frozen VLM uses verified experience to derive lessons that transfer to new visual contexts without changing weights or relying on expert spatial tools at inference.The goal is to reuse distilled lessons rather than memorize solved instances.Source: §1 · p. 2

Method

• SMA keeps the VLM frozen while converting verifier-grounded spatial experience into reusable transferable lessons.In a verifiable environment, the agent obtains a predicted answer and reward, then uses verifier-guided reflection to distill lessons.Source: §1 · p. 2
• Transferable lesson memory combines task-embedding semantic retrieval with visit-evidence calibration of each lesson’s Transfer Reliability Score.TRS starts uniformly and is calibrated from later retrieval outcomes to estimate future transfer reliability; calibrated TRS then ranks candidate lessons during read-only deployment.Source: §1 · p. 2

Results

• Across five spatial benchmarks and four base VLMs, SMA achieves the best accuracy among evaluated methods in most of 20 evaluations.These results demonstrate a practical parameter-update-free path for spatial self-evolution.Source: §1 · p. 2

2.1 Spatial Intelligence

• Spatial reasoning methods either adapt VLMs through spatial post-training or use expert spatial tools for intermediate evidence, whereas SMA uses verified experience and transferable lessons without weight updates or inference-time expert tools.SpatialVLM teaches object relationships and spatial grounding, RoboSpatial combines 2D and 3D spatial data, S-Agent calls specialized spatial tools, and SpaceTools studies tool-augmented reasoning with interactive reinforcement learning.Source: §2.1 · p. 3

2.2 Self-Evolving Agents and Memory

• Self-evolving memory methods mainly target text-centric agents and often rely on semantic similarity, while SMA introduces training-free spatial self-evolution with a Transfer Reliability Score for procedure transfer.The score estimates reliability under new visual contexts and tasks, addressing the limitation that surface relevance may not indicate reliable spatial transfer.Source: §2.2 · p. 3
• SMA writes procedural memories from verified spatial rollouts and reflection, then filters candidates semantically and ranks them by verifier-derived TRS during read-only deployment.Model parameters and the memory bank remain unchanged during deployment.Source: Figure 3 · p. 4

3.1 Problem Setup

• SMA builds an external library of transferable spatial procedures and estimates each procedure’s transfer reliability without changing the base model’s parameters.The environment and deployment splits are disjoint, and the memory bank stores task, rollout summary, transferable lesson, visit count, cumulative reward, and TRS while hiding prior predictions and verified answers from retrieval.Source: §3.1 · p. 4

3.2.1 Experience Acquisition

• During experience acquisition, retrieved memories guide the frozen VLM, which produces a raw output from which the task prediction is extracted.SMA uses One-Pass Memory Writing by default: memories are written during the first pass over X, while later passes reuse the fixed bank to update retrieved cards’ reliability state.Source: §3.2.1 · pp. 4–5

3.2.2 Procedure Memory Generation

• The reflection model converts each verifier-scored rollout into strict-JSON summary and procedure fields, using the verified target to diagnose failure or success without restating the answer.Each transferable lesson captures the pattern, trap to avoid, and check to apply in similar questions.Source: §3.2.2 · p. 5

3.2.3 Two-Stage Retrieval

• SMA retrieves guidance in two stages: semantic filtering removes cards below similarity threshold δ, then combined ranking merges normalized relevance with normalized TRS before selecting top-k cards.The selected guidance set is prepended to the current user prompt.Source: §3.2.3 · p. 5

3.2.4 Visit-Evidence Calibration

• Visit-evidence calibration initializes all TRS values uniformly and updates retrieved memories from subsequent rewards, shrinking low-visit estimates toward the common prior.This separates future transfer utility from the correctness of the rollout that created each memory.Source: §3.2.4 · p. 5

3.3 Read-Only Deployment

• In read-only deployment, SMA applies the same retrieval pipeline to deployment problems while keeping the memory bank, visit counts, cumulative rewards, and TRS values fixed.Retrieved task, summary, and transferable lesson fields guide the frozen VLM, with no new memory writes or state updates.Source: §3.3 · p. 5

4.1 Benchmarks, Models, Implementation, and Baselines

• The evaluation covers five complementary spatial benchmark slices, four frozen VLMs, held-out deployment splits, and comparisons against No memory, RAG, MemP, MemRL-R, and MemRL-GT.The same frozen model performs task solving and reflection, with deterministic decoding and accuracy reported across benchmarks.Source: §4.1.1 · p. 6, §4.1.2 · pp. 6–7, §4.1.3 · p. 7

4.2 Main Results

• SMA achieves the best macro average for every frozen base-model block: 68.8, 66.7, 69.8, and 63.5, with gains of 2.6, 2.9, 1.7, and 2.8 points over the strongest non-SMA baselines.On Qwen3.6-27B, SMA improves over No memory, RAG, MemP, and MemRL-GT by 6.5, 3.9, 3.0, and 1.7 points; gains also span RoboSpatial, Omni3D, and EmbSpatial.Source: §4.2 · p. 7

4.3 Ablations and Hyperparameter Sensitivity

• Removing the summary, transferable lesson, or semantic filter lowers RoboSpatial accuracy by 3.2, 3.5, or 5.8 points, while raw output and reward-only reflection lower it by 4.4 or 5.5 points.The retrieval sensitivity sweep peaks at η = 0.5 and k = 3.Source: §4.3 · p. 7

4.4 Comparison with Training-Based Self-Evolving

• Against SpatialEvo-7B, SMA raises macro accuracy from 47.1 to 63.5 (∆= +16.4) and scores higher on every reported benchmark in this evaluation.The comparison uses Qwen3.5-9B and the five benchmark slices.Source: §4.4 · p. 7

4.5 Memory Transfer Analysis

• Transferred memories consistently improve target no-memory baselines across model and benchmark transfer, with gains depending on source–target similarity.Model transfer is strongest on RoboSpatial and SAT, while benchmark transfer is positive on every selected probe and largest on RoboSpatial.Source: §4.5 · pp. 7–8

4.6.1 Memory TRS Analysis

• Deployment accuracy rises from 19.3% for mean TRS in [0.2, 0.3) to 97.3% in [0.9, 1.0], while successful-source memories yield 24.3 percentage points higher evaluation accuracy.The pooled TRS trend may also reflect benchmark difficulty and question composition; retrieval groups with all-success memories outperform mixed and all-failure groups.Source: §4.6.1 · pp. 8–9

4.6.2–4.6.4 Diagnostic Analyses

• SMA reduces macro retrieved-memory similarity from 0.792 to 0.698 while increasing macro accuracy from 66.8% to 69.8%, and one-pass writing uses one-tenth as many memories with 21% less redundancy.One-pass writing also achieves roughly twice the TRS-update coverage, while SMA improves all ten atomic abilities, led by Correspondence (+11.2 pp), Attribute (+8.0 pp), and Object motion (+7.6 pp).Source: §4.6.2 · pp. 9–10, §4.6.3 · p. 10, §4.6.4 · pp. 10–11

4.7 Qualitative Case Study

• In five qualitative cases, high-TRS memories provide concrete procedures for size checking, localization, depth comparison, motion simulation, and background anchoring that steer reasoning toward task-relevant geometry.These examples support the quantitative finding that high-TRS procedures transfer reliably.Source: §4.7 · p. 12

5 Conclusion

• SMA converts verified spatial experience into reusable transferable lessons for frozen VLM agents.Source: §5 · p. 12
• Rather than updating model weights or replaying instances, SMA uses verifier-guided reflection to distill lessons for later inference.Source: §5 · p. 12
• SMA calibrates each lesson’s Transfer Reliability Score from visit evidence and retrieves reliable lessons to guide subsequent inference.Source: §5 · p. 12
• Across five representative spatial benchmarks and four base VLMs, SMA achieved the best accuracy among evaluated methods in most of 20 evaluations.Source: §5 · p. 12
• The results position external transferable lesson memory as a practical parameter-update-free route for spatial self-evolution.Source: §5 · p. 12
• SMA complements training-based spatial post-training and tool-augmented spatial reasoning rather than replacing those approaches.Source: §5 · p. 12

A.1 Additional Results on SITE-image and ViewSpatial

• Table 7 reports extended results for SITE-image and ViewSpatial across all four base models.The evaluation includes all compared baselines.Source: Table 7 · p. 16, §A.1 · p. 16
• The table compares accuracy (%) on the two benchmark columns.Accuracy is the reported evaluation metric.Source: Table 7 · p. 16
• Table 7 includes an Avg. column computed as the macro average over the two benchmark columns.The average summarizes the two benchmark results.Source: Table 7 · p. 16
• All compared baselines are reported for every base model in the extended evaluation.This provides a complete baseline comparison across the four model settings.Source: Table 7 · p. 16, §A.1 · p. 16
• The table marks the best and second-best result per benchmark column.These markings distinguish the top two entries within each column.Source: Table 7 · p. 16
• The additional results cover both SITE-image and ViewSpatial rather than a single benchmark.The two benchmark columns are evaluated jointly in the reported macro average.Source: Table 7 · p. 16, §A.1 · p. 16

• The Omni3D ablation uses Qwen3.6-27B tuning runs and matches the main-paper RoboSpatial ablation in omitted components and reflection variants.The table follows the same reporting format as the main ablation table.Source: §A.2 · p. 16
• On Omni3D, Table 8 reports ablations and reflection-setting comparisons with accuracy and differences relative to SMA.The SMA reference is marked by a blue row and boldface.Source: Table 8 · p. 16
• Figure 7 presents the Omni3D hyperparameter sensitivity analysis.Source: Figure 7 · p. 17

B.1 Design of Visit-Evidence Calibration

• SMA initializes each memory with neutral TRS and calibrates it from later verified rewards, treating repeated transfer success as evidence of reliability.Each memory stores visit count n_i, cumulative reward c_i, and TRS v_i; new memories begin with n_i = 0, c_i = 0, and v_i = v_0.Source: §B.1 · p. 17
• The calibration is order-invariant and conservatively stable, so identical visit counts and cumulative rewards yield identical TRS regardless of success-failure order.This prevents single failed transfers from discarding useful lessons and single successful source episodes from permanently establishing trust.Source: §B.1 · pp. 17–18
• At retrieval, SMA first filters memories by semantic similarity, then combines normalized similarity with TRS to balance topical relevance against empirical reliability.Z-score normalization only places candidate-set similarity and TRS on a comparable scale; it does not replace prior-based low-visit shrinkage.Source: §B.1 · p. 18

B.2 SMA Pseudocode

• During experience passes, SMA shuffles environment problems, retrieves eligible top-k memories, queries the frozen VLM, evaluates rewards, and updates every selected memory’s evidence.The retrieval pool uses cosine similarity threshold δ, while η weights reliability in the ranking configuration.Source: §B.2 · p. 18

B.1 Design of Visit-Evidence Calibration

• The TRS update v_j = (λv_0 + c_j)/(λ + n_j) shrinks low-visit estimates toward the neutral prior while allowing accumulated evidence to dominate.With v_0 = 0.5 and λ = 2, the prior acts like two virtual visits; larger λ is more conservative, whereas smaller λ reacts faster.Source: §B.1 · pp. 17–18, §B.2 · p. 18

B.2 SMA Pseudocode

• On the first pass, SMA reflects on the frozen VLM’s output, question, and verified answer to create a lesson, then appends it with neutral calibration statistics.New memory cards contain the text, spatial lesson, visit count, cumulative reward, and TRS.Source: §B.2 · p. 18
• During deployment, SMA retrieves memories and generates predictions from the frozen VLM without writing new memories or updating visit evidence.Deployment returns predictions and the memory bank while preserving the learned calibration state.Source: §B.2 · p. 18

C.1 Benchmark Overview

• Seven benchmark slices cover complementary spatial inputs, answer spaces, and demands; RoboSpatial, ERQA, Omni3D, SAT, and EmbSpatial appear in the main table, while SITE-image and ViewSpatial are in the appendix.The suite therefore combines the five main-result benchmarks with two extended-evaluation slices.Source: §C.1 · p. 19

C.1.1 RoboSpatial

• RoboSpatial tests robot-oriented indoor spatial understanding through normalized point sets, configuration judgments, and compatibility decisions about object placement.Its official subcategories are context, configuration, and compatibility, covering localization, object relations, and fit without spatial conflict.Source: §C.1.1 · p. 19, §C.2.1 · p. 20

C.1.2 ERQA

• ERQA evaluates robot-relevant answers about state, actions, trajectories, and spatial facts grounded in visual observations.Its annotation categories include spatial, trajectory, action, state, pointing, multi-view, and task reasoning.Source: §C.1.2 · p. 19, §C.2.2 · p. 20

C.1.3 Omni3D

• Omni3D requires open-ended reasoning about 3D quantities and relations, including metric estimates, distance, occlusion, containment, capacity, and counterfactual placement.Each item uses a single real-world RGB image and may require a number, yes/no judgment, or short description.Source: §C.1.3 · p. 19

C.1.4 SAT

• SAT measures dynamic spatial aptitude with binary multiple-choice questions over ordered images involving goals, action consequences, perspectives, object movement, and ego movement.Its taxonomy additionally includes action sequence, with examples organized across six subcategories.Source: §C.1.4 · p. 19, §C.2.3 · pp. 20–21

C.1.5 EmbSpatial

• EmbSpatial emphasizes language-grounded embodied relations using left, right, above, under, close, and far labels over an embodied image QA pool.The released annotation distinguishes directional-relation and distance subcategories, illustrated in Tables 25–26.Source: §C.1.5 · p. 19, §C.2.4 · p. 21, Table 25 · p. 36

C.1.6 SITE-image; C.1.7 ViewSpatial

• SITE-image retains SITE’s image questions on 3D information, counting, movement, navigation, multi-view reasoning, localization, and spatial relationships, whereas ViewSpatial uses multiple views for camera-, person-, and object-relative localization.SITE-image excludes SITE’s video examples; ViewSpatial also tests orientation and scene simulation.Source: §C.1.6 · p. 19, §C.1.7 · p. 19

C.2 Benchmark Taxonomy and Atomic Capability Annotation

• The taxonomy maps heterogeneous benchmark categories to ten atomic capabilities, including correspondence, localization, relations, distance/depth, mental simulation, tracking, camera reasoning, and affordance.Benchmark categories may map to multiple capabilities when they require compound reasoning procedures.Source: §C.2 · p. 20

C.3 Dataset Split Construction

• Environment and deployment splits are disjoint 50/50 per category for non-SAT benchmarks, while SAT uses the official expanded test set for deployment and validation data matched by question type for environment writing.The split uses seed 42; odd-category examples alternate assignment, and the environment split writes memory while deployment evaluates it read-only.Source: §C.3 · p. 37, Table 27 · p. 38

C.4 Baseline Methods

• The no-memory baseline directly evaluates the frozen VLM on deployment prompts, whereas RAG retrieves prior task-output records by task-embedding similarity without reflected lessons or TRS.No memory performs no environment-stage writing or retrieval; RAG stores lightweight rollout records during environment runs.Source: §C.4.1 · p. 38, §C.4.2 · p. 38

C.4.3 MemP

• MemP reflects rollouts into compact transferable lessons and retrieves them by semantic similarity, isolating reflected procedure text without visit-evidence reliability calibration.MemP is a similarity-only procedural-memory baseline.Source: §C.4.3 · p. 38

C.4.4 MemRL-R / C.4.5 MemRL-GT

• MemRL-R reflects using model output and scalar verifier reward, while MemRL-GT uses ground-truth-guided feedback to separate reflection supervision from memory-value calibration.Both adapt the MemRL memory interface, but MemRL-R excludes verified ground-truth answers and MemRL-GT changes the reflection signal.Source: §C.4.4 · p. 38, §C.4.5 · p. 39

C.4.6 SMA

• SMA writes verified spatial experience as transferable procedural lessons, tracks visit count, cumulative reward, and TRS, and deploys read-only retrieval through semantic filtering followed by TRS-aware ranking.SMA is parameter-update-free and uses no parameter updates during deployment.Source: §C.4.6 · p. 39

C.5 Computing Infrastructure

• Experiments ran on a Linux server with four NVIDIA H200 GPUs, 192 logical CPU threads, 2.0 TiB system memory, and an Ubuntu 22.04.5 software environment.The reported stack includes CUDA 12.8, Python 3.12.13, PyTorch 2.11.0+cu128, Transformers 5.8.1, and vLLM 0.20.0.Source: §C.5 · p. 39

C.6 Hyperparameter Settings

• Selected SMA hyperparameters are reported in one compact table for each backbone: Qwen3.5-122B-A10B, Qwen3.6-35B-A3B, Qwen3.6-27B, and Qwen3.5-9B.The backbone-specific tables keep benchmark settings readable without shrinking the presentation.Source: §C.6 · p. 39, Table 28 · p. 39, Table 29 · p. 39

D.1 Credit Assignment in Memory Evolution

• SMA cannot precisely assign an improved or failed final answer to memory writing, reflection, retrieval, semantic filtering, or final memory use.This credit-assignment gap is especially important when spatial answers depend on coupled operations such as object identification, scale estimation, viewpoint transformation, and placement-rule application.Source: §D.1 · p. 40
• Future attribution mechanisms could localize memory-construction rewards, distinguish memory-operation effects across sessions, and propagate value through provenance-based memory dependencies.The cited directions include AttriMem (Li et al., 2026b), Memory-R2, and MemQ (Liao et al., 2026).Source: §D.1 · p. 40

D.2 Long-Term Memory Maintenance

• SMA writes, retrieves, and reliability-weights lessons but lacks a full long-term maintenance lifecycle as its memory bank grows.Older lessons may become redundant, conflicting, overly specific, or stale; TRS downweighting and semantic filtering do not explicitly manage storage or latency budgets.Source: §D.2 · pp. 40–41
• Explicit deletion, merging, compression, expiration, and trusted revision policies are natural extensions for maintaining SMA memories over time.Related systems frame lifecycle operations as first-class choices, including storage-budgeted consolidation in MemRefine (Kim et al., 2026), trustworthy consolidation in TRUSTMEM (Yang et al., 2026), and standardized operations in memorywire (Munirathinam, 2026).Source: §D.2 · p. 41

E.1 Successful Cases

• Eight wrong-to-right cases show SMA correcting baseline errors after retrieving relevant lessons, with cases spanning all seven benchmarks and requiring reusable procedures rather than copied answers.The six successful-transfer cases show both methods answering correctly after retrieving a relevant lesson, while the eight wrong-to-right cases show only SMA becoming correct; Figures 9–22 cover the seven benchmarks.Source: §E.1 · p. 41
• Each qualitative panel relates the question and image, retrieved lesson and final TRS, model output, and TRS trajectory to assess alignment between procedure, evidence, and decision.This layout makes it possible to inspect whether improvement is supported by the retrieved procedure and current-task visual evidence.Source: §E.1 · p. 41
• The qualitative cases cover 3D, camera-motion, temporal-depth, robot-scene, viewpoint, and embodied spatial reasoning across Omni3D, SAT, SITE-image, ERQA, ViewSpatial, RoboSpatial, and EmbSpatial.Figures 9–14 present successful-transfer cases, while Figures 15–22 present wrong-to-right cases.Source: §E.1 · p. 41

E.2 Failure Cases

• SMA failures separate underspecified benchmark evidence from base-model errors that persist despite relevant retrieved memories.Benchmark ambiguity includes missing metric or depth information, partial evidence, and hidden simulator geometry; model limitations involve misreading movement, object count, or connectivity despite TRS ≥0.6 memories.Source: §E.2 · pp. 41–42

CASE 4 · SAT

• CASE 4 failures show retrieved procedures can be weakly applied when depth cues are ambiguous or visual position substitutes for occlusion, contact, and perspective.EmbSpatial examples predict C/book instead of GT B/laptop and D/book instead of GT C/chair; the RoboSpatial example predicts Yes instead of GT No after substituting generic dimensions for unavailable measurements.Source: CASE 4 · SAT · pp. 56, 57, 59

CASE 5 · SAT

• CASE 5 failures arise when benchmark-defined poses or visual evidence do not uniquely support the retrieved procedure’s assumptions.The SAT example predicts B/right instead of GT left because destination heading is unspecified, while the SITE example’s systematic scan still yields three yellow stars through reasoning aligned with the retrieved lesson.Source: CASE 5 · SAT · pp. 60, 62

F Prompt Templates

• The appendix reports three prompt classes for each benchmark: solving the current task, reflecting on a rollout into a transferable lesson, and retrieving memories for a later task.These roles are summarized in Table 32.Source: F Prompt Templates · p. 63, Table 32 · p. 63

F.2 Benchmark-Specific Prompts

• During retrieval, each selected memory is rendered with runtime rank, similarity, task, transferable lesson, and summary values and prepended to the task; the block is omitted when no memory is retrieved.The memory item block is repeated once per retrieved memory.Source: §F.2 · p. 64

RoboSpatial: System prompt

• RoboSpatial system prompts cover vacant-region pointing, object-object relation verification, and placement compatibility or affordance reasoning from a single indoor RGB image.Outputs are normalized point lists or Yes/No answers, and compatibility requires reasoning about free volume, object extent, and obstacles rather than category membership alone.Source: RoboSpatial: System prompt · p. 64
• RoboSpatial reasoning requires single-image grounding, shape-specific validation, and cross-cutting checks for image-plane directions, unoccupied pointing coordinates, and both named objects in Yes/No questions.The protocol resolves visible instances while accounting for occlusion, support surfaces, clutter boundaries, and the exact relation phrase.Source: RoboSpatial: System prompt · p. 64
• RoboSpatial system prompts classify pointing, configuration, and compatibility tasks, requiring image-grounded checks such as anchors, relations, clearance, and obstacle contact.Current pixels override memory, and outputs must follow strict Yes/No or normalized-point formats.Source: RoboSpatial: System prompt · pp. 64–65

RoboSpatial: Reflection prompt

• RoboSpatial reflection converts rollout errors into abstract lessons for structurally similar scenes, explicitly avoiding answers, coordinates, layouts, and quoted outputs.Lessons pair one positive procedure with one trap, such as anchoring before spreading points or checking clearance before compatibility judgments.Source: RoboSpatial: Reflection prompt · p. 65

RoboSpatial: Memory retrieval prompt

• RoboSpatial retrieval matches memories by task shape, extracts at most one check or trap, and re-derives every answer from the current image rather than copying prior outputs.Applicable checks cover pointing anchors and free-space points, object binding, relational pixels, and obstacle clearance.Source: RoboSpatial: Memory retrieval prompt · pp. 65–66

ERQA: System prompt

• ERQA system prompts organize reasoning into five shapes: kinematics, trajectories, scene state or task outcomes, multi-view correspondence, and surface-bound pointing.The protocol aligns image placeholders, simulates motion, checks post-conditions, binds geometry across views, and tests markers against named surfaces.Source: ERQA: System prompt · pp. 66–67

ERQA: Reflection prompt

• ERQA reflection distills reasoning-level procedures from failures without revealing the correct letter, option text, image indices, or scene-specific details.Diagnoses include placeholder binding, trajectory overlays, state reading, geometric correspondence, rotation direction, and choice scanning.Source: ERQA: Reflection prompt · pp. 67–68

ERQA: Memory retrieval prompt

• ERQA retrieval treats memories as procedural checks matched to structural shape, ignores mismatches, and derives the final letter solely from current attachments.Checks include placeholder order, motion simulation, containment, cross-view geometry, and surface validation.Source: ERQA: Memory retrieval prompt · p. 68

Omni3D: System prompt

• Omni3D prompts cover metric estimation, rule-based counting, hypothetical visibility, relational fit, and concise identification, grounded in single-image object, depth, support, and occlusion analysis.They distinguish real-world size from image footprint and require systematic inclusion rules, line-of-sight tests, 3D extent comparisons, and correct-instance binding.Source: Omni3D: System prompt · pp. 68–69
• Omni3D’s system prompt assigns shape-specific procedures for metric, count, visibility, fit, and identification questions while making the current image authoritative.It also enforces answer-form matching and a single `Final Answer` line.Source: Omni3D: System prompt · p. 69

Omni3D: Reflection prompt

• Omni3D reflection converts rollout diagnoses into abstract lessons for structurally similar new items, explicitly excluding sample-specific answers, measurements, objects, and layouts.The output is strict JSON containing a task-shape summary and one transferable procedure with a trap and validation check.Source: Omni3D: Reflection prompt · p. 70

Omni3D: Memory retrieval prompt

• Omni3D retrieval matches memories by structural question shape, extracts at most one check or trap, and re-derives the answer from the current image.Retrieved notes guide procedures such as anchoring referents, honoring count clauses, simulating visibility, preserving decimals, and shortening string answers.Source: Omni3D: Memory retrieval prompt · p. 70

SAT: System prompt

• SAT’s system prompt distinguishes bearing, perspective, object displacement, and camera-motion shapes, requiring ordered image inspection, referent matching, and explicit frame comparison.The current images remain authoritative, and binary decisions must end with exactly one answer letter.Source: SAT: System prompt · p. 71

SAT: Reflection prompt

• SAT reflection distills reasoning gaps and successful habits into transferable lessons for new egocentric or temporal tasks without revealing the rollout’s letter or scene-specific details.Examples include binding marked referents, comparing before and after states, and distinguishing camera from object motion.Source: SAT: Reflection prompt · p. 72

SAT: Memory retrieval prompt

• SAT retrieval selects memories by question shape and image layout, uses one check or trap per memory, and independently re-derives the letter from current images.The procedures cover viewpoint and referent locking, simulated perspective changes, matched-object displacement, and global camera-motion comparison.Source: SAT: Memory retrieval prompt · pp. 72–73

EmbSpatial: System prompt

• EmbSpatial’s system prompt organizes four-option image reasoning around egocentric depth, horizontal and vertical relations, and lexical verification against distractor sentences.It emphasizes pixels, object identity, support and stacking, image-plane direction, and rejecting plausible but incorrect relation claims.Source: EmbSpatial: System prompt · p. 73
• EmbSpatial’s system prompt requires single-image evidence, shape-specific validation across all four options, and trust in the current image over conflicting memories.Near/far questions use depth and occlusion comparisons, while pairwise relations require anchoring both referents and evaluating every option before producing one final letter.Source: EmbSpatial: System prompt · p. 74

EmbSpatial: Reflection prompt

• EmbSpatial reflection distills each rollout into a transferable lesson about the structural question shape, with one reusable procedure and one trap rather than the sample’s answer.The prompt diagnoses gaps such as depth-size confusion, wrong object binding, skipped options, and image-plane/world-frame confusion, while prohibiting answer, scene, and identifier leakage.Source: EmbSpatial: Reflection prompt · p. 74

EmbSpatial: Memory retrieval prompt

• EmbSpatial memory retrieval matches memories by structural task shape, extracts at most one check or trap, and re-derives the answer from the new image.Retrieved notes provide procedures such as four-way depth comparison or testing each relation sentence, but irrelevant memories are ignored and prior letters are never reused.Source: EmbSpatial: Memory retrieval prompt · p. 75

SITE-image: System prompt

• SITE-image’s system prompt generalizes the protocol to one or more images or video frames by assembling a scene model, separating reference frames, and applying task-specific checks.It distinguishes image-plane, camera-motion, scene/world, and cross-view frames; checks duplicates and coverage for counting, anchors relations to references, validates 3D geometry, and tests motion and cross-image bindings.Source: SITE-image: System prompt · pp. 75–76

SITE-image: Reflection prompt

• SITE-image reflection produces strict-JSON memories that diagnose concrete spatial reasoning failures and encode a positive procedure, negative habit, and concrete validation check.The memory must teach how to reason on a new task with the same structural shape, while excluding answer letters, scene-specific counts, identities, indices, and other leakage.Source: SITE-image: Reflection prompt · pp. 76–77

SITE-image: Memory retrieval prompt

• SITE-image retrieval treats memories as procedural checklist items matched by structural shape, prioritizes current visuals over memory, and applies explicit checks for counting, multi-view binding, and 3D reasoning.Each relevant memory contributes only a check or trap; mismatched memories are dropped, and every option letter is re-derived from the current images or frames.Source: SITE-image: Memory retrieval prompt · p. 77

ViewSpatial: System prompt

• ViewSpatial frames multi-view reasoning around camera-referenced layout, object facing, and hypothetical egocentric relations after fusing several viewpoints.The prompt defines camera perspective as the reference axis and expresses relations in camera-frame sectors rather than the viewer’s offscreen bodily left/right.Source: ViewSpatial: System prompt · p. 78
• The system prompt distinguishes camera-referenced, embodied-agent, and multi-view simulated-egocentric questions, requiring frame-specific reasoning before selecting one answer letter.It instructs the model to inspect and align all views, avoid confusing image-plane and egocentric directions, read every option, and output exactly “Final Answer: <LETTER>”.Source: ViewSpatial: System prompt · pp. 78–79

ViewSpatial: Reflection prompt

• Reflection converts each rollout into a structural task-shape diagnosis and a transferable procedure with one success habit and one trap, while forbidding answer leakage.The lesson must generalize to different scenes and questions, using the ground-truth letter only to locate reasoning gaps and excluding letters, option text, and scene-specific details.Source: ViewSpatial: Reflection prompt · p. 79

ViewSpatial: Memory retrieval prompt

• Memory retrieval matches lessons to the current question’s structural shape, extracts at most one check or trap, and re-derives the answer from current images rather than copying prior outputs.The prescribed checks are to establish camera front, adopt an agent’s facing, or fuse views before simulating a stand-at-and-facing query; mismatched memories are ignored.Source: ViewSpatial: Memory retrieval prompt · pp. 79–80