PaperlayerView exact revision on arXiv

Source-linked AI summary

Anvil: Scaling Embodied Spatial Intelligence through Knowledge Recirculation

Weilin Ruan, Bohan Zhou, Jinghao Li, Yangkai Wei, Wenqian Li, Peiran Dong, Yingjie Xu, Leyi Wu, Haojian Huang, Binghui Xie, Han Yang, Kaiwen Zhou, Yinchuan Li, Ying-Cong Chen, James Cheng

arXiv:2610.21882v1cs.CVcs.AI

TL;DR

Embodied spatial intelligence requires connecting visual recognition to precise, action-relevant spatial judgments, yet supervision spans incompatible output forms. ANVIL combines scene-aligned data generation and evaluation with specialist-to-generalist distillation and feedback-guided verification. Its final model improves the seven-benchmark mean by 9.29 points over its base, with controlled studies supporting targeted coverage, selective transfer, and feedback-driven repair.

  • Problem

    Embodied spatial intelligence requires localizing entities, reasoning about relations, and identifying interactions, but learning these abilities needs supervision connecting visual observations to precise, action-relevant judgments.

    Source: §1 · p. 1

  • Method

    ANVIL links form-specific generation, cross-stage verification and repair, scene-aligned assessment, and specialist-to-generalist distillation with semantic-first-fault routing and stage normalization.

    Source: §1 · p. 2

  • Results

    The final model improves the seven-benchmark mean by 9.29 points over its base.

    Source: §5 · p. 10

  • Takeaways & Limitations

    Controlled studies support targeted coverage, selective transfer, and feedback-driven repair as complementary components for jointly designing reliable supervision and knowledge consolidation.

    Source: §5 · p. 10

    The framework currently organizes supervision through eleven Ground–Relation–Act forms, while open-world environments may require concepts and interaction requirements beyond this inventory.

    Source: G LIMITATIONS AND FUTURE WORK · p. 28

Abstract

from arXiv · show

Embodied spatial intelligence requires more than visual recognition: models must localize entities, reason about their relations, and express this knowledge in action-relevant forms. Yet joint training can conflate the knowledge needed at different stages because supervision is fragmented across incompatible geometric and semantic outputs. We introduce Anvil, a unified framework for data generation, training, and benchmarking for embodied spatial intelligence. (1) Anvil-Gen is a heterogeneous data flywheel that combines form-specific generation, cross-stage consistency verification, and failure-driven recollection across 11 Ground--Relation--Act forms. (2) Anvil-Ben provides scene-aligned supervision for learning diverse embodied spatial capabilities and scene-linked assessments that distinguish stage competence from error propagation. (3) Anvil-MOPD is a 9B student that consolidates heterogeneous spatial knowledge through distillation from specialist teachers. Semantic-first-fault routing targets teacher supervision to the student's earliest decisive failure, while stage normalization controls the loss scale across native outputs of different lengths. We construct 5.8M source-grounded records spanning 160K images, reserving 580K spatial records for held-out evaluation and using the remaining 5.22M records for training. On average across seven public benchmarks, Anvil-MOPD outperforms Qwen3.5-9B and RynnBrain-8B by 9.29 and 11.03 percentage points, respectively, while reaching 80.67% on VABench-Point. Controlled studies support targeted coverage, selective transfer, and feedback-guided repair as complementary framework components.

1 INTRODUCTION

Embodied spatial intelligence calls for linking visual observations to precise, action-relevant spatial judgments, but supervision is fragmented across different sources and output forms. ANVIL connects data generation, scene-aligned evaluation, and specialist-to-generalist learning through a shared Ground–Relation–Act interface.

1 INTRODUCTION

  • Scaling embodied spatial intelligence requires connecting visual observations to precise, action-relevant judgments across sources with different coverage and available geometry.Recognizing an object such as a coffee machine alone does not establish where a cup can be placed beside it.Source: §1 · p. 1
  • The paper asks how heterogeneous spatial expertise can become reliable supervision and be consolidated in a model that also supports subsequent data verification.Prior approaches report gains that vary across capabilities and transfer settings, while diagnostic benchmarks reveal differences aggregate scores can obscure.Source: §1 · pp. 1–2
  • ANVIL-GEN uses form-specific generation, cross-stage verification, and failure feedback to guide targeted data repair and collection.Its shared Ground–Relation–Act interface connects 11 native output forms through common entities and explicit dependencies.Source: §1 · p. 2
  • ANVIL-BEN aligns scene images and 11 annotation formats for training and held-out assessment across spatial understanding, reasoning, and state-conditioned plan verification.Native-output tasks assess individual capabilities, while linked assessments and controlled upstream substitutions measure error propagation across dependent stages.Source: §1 · p. 2
  • ANVIL-MOPD distills specialist knowledge into a generalist, routing teacher supervision to the student's earliest decisive error and normalizing loss across output lengths.The generalist can then assist ANVIL-GEN under evidence-backed verification.Source: §1 · p. 2

2 RELATED WORK

Related work spans spatial benchmarks and geometry-derived supervision, embodied foundation models, and selective knowledge distillation. ANVIL treats supervision reliability and knowledge transfer as coupled problems, linking scene-aligned annotations, specialist-to-generalist learning, and held-out evaluation.

Spatial intelligence

  • Spatial research covers relation recognition, geometric estimation, and interaction reasoning, while evaluation asks whether success on one task transfers to others.Training targets increasingly draw on geometry and robot experience, and generated annotations can be checked through program execution and trajectory consistency.Source: §2 · p. 2

Embodied foundation models

  • Embodied foundation models connect visual-linguistic knowledge to reasoning and robot control, while spatially focused models develop geometric and spatiotemporal understanding.Reasoning–action models increasingly connect these capabilities to action prediction; the challenge is generalizable capability rather than a collection of task-specific skills.Source: §2 · pp. 2–3

Knowledge distillation

  • Knowledge distillation research explores teacher and supervision selection across examples, tasks, trajectories, tokens, and intermediate reasoning steps.Selective supervision raises the question of retaining useful teacher knowledge without reinforcing its errors.Source: §2 · p. 3

ANVIL's approach

  • ANVIL couples supervision reliability with knowledge transfer by using scene-aligned annotations for specialist-to-generalist distillation and held-out evaluation.The resulting student can assist further data verification, linking data refinement and model learning for joint study of their effects on spatial competence.Source: §2 · p. 3

3 ANVIL: LEARNING THROUGH KNOWLEDGE RECIRCULATION

ANVIL links scene-aligned spatial records, evidence-backed verification, specialist distillation, and model-assisted data refinement. Its shared entities and task dependencies support alignment across heterogeneous targets while keeping evaluation targets outside training and feedback.

3 ANVIL: LEARNING THROUGH KNOWLEDGE RECIRCULATION

  • Shared scene references connect cross-form verification and localized repair to selective specialist transfer, while the resulting generalist assists subsequent verification.Evidence-backed checks retain acceptance authority, and ANVIL-EVAL targets remain outside the training and feedback loop.Source: §3 · p. 3, §3.3 · p. 5

3.1 SCENE-ALIGNED SUPERVISION

  • Eleven native contracts cover four Ground forms, four Relation forms, and three Act forms, with task graphs specifying active stages and their upstream dependencies.Records preserve entity identities and supported units; not every scene must support all eleven forms.Source: §3.1 · p. 4

3.2 GENERATION, VERIFICATION, AND FEEDBACK

  • Candidate records require native-field and geometry checks, declared cross-form constraints, and semantic support before admission.Mutually consistent outputs can still be wrong, so acceptance uses external evidence and explicit uncertainty; rejected candidates receive a failure category rather than a reference answer.Source: §3.2 · p. 4
  • ANVIL-TRAIN contains 5.22M records and ANVIL-EVAL holds out 580K spatial records from 5.8M source-grounded records.Scene groups remain in a single partition, evaluation targets stay outside training and feedback, and linked tasks vary upstream context while holding the downstream target fixed.Source: §3.2 · pp. 4–5

3.3 STAGE-ROUTED MULTI-TEACHER DISTILLATION

  • Stage-routed distillation selects a teacher and correction span from the earliest decisive fault in a student's linked response.Reference learning covers all active stages, while routed distillation concentrates specialist guidance on the selected failure; stage and span normalization prevent output length alone from setting supervision mass.Source: §3.3 · p. 5
  • Knowledge recirculation lets the generalist provide auxiliary signals for candidate verification and repair, while accepted records can support specialist updates before the next round.Admission remains governed by evidence-backed acceptance rather than student agreement alone, and deployment inference needs neither specialist teachers nor reference-aware routing.Source: §3.3 · p. 5
  • Table 1 reports spatial and embodied-perception evaluation results as percentages, with higher values better.The seven-benchmark average excludes the paper's evaluation benchmark.Source: Table 1 · p. 6, §3.3 · p. 6

4 EXPERIMENTS

The experiments assess the final model against public benchmarks and test how supervision coverage, teacher routing, repair, and task dependencies affect spatial learning and evaluation.

4.2 SPATIAL INTELLIGENCE ASSESSMENT AND COVERAGE

  • Across seven public benchmarks, ANVIL-MOPD achieves a 74.27% mean, exceeding QWEN3.5-9B by 9.29 points.The gains span spatial judgments and location prediction, with a regression on RefSpatial.Source: §4.2 · p. 7

4.3 MECHANISM ABLATIONS

  • Mixed training improves Where2Place over replay by 30.25 points across two seeds, while vacancy-only training favors placement and replay better preserves pointing and depth.The comparison uses matched updates and record exposures; mixed training does not improve every task, and unique images and realized tokens remain unmatched.Source: §4.3 · p. 7, Figure 3 · p. 7
  • With a matched maximum rollout budget, semantic-uniform routing improves strict Ground–Relation success over flat distillation by 2.13 points, with 99.83% general-VQA retention.Semantic references expose incorrect but well-formed geometry that structural checks miss; teacher supervision reaches 63.00% of sampled responses.Source: §4.3 · p. 7, Figure 3 · p. 7
  • With the same 885 rejected candidates and one call each, named feedback recovers 10.09× as many source-correct records as generic retry at comparable GPU cost.Recovery concentrates on depth ordering; grounding remains harder to repair.Source: §4.3 · p. 7, Figure 3 · p. 7

4.4 QUALITATIVE ANALYSIS AND PRODUCTION EFFICIENCY

  • Named feedback yields 39.38 correct records per GPU-hour versus 3.98 for generic retry, while source evidence also supports CPU-based target construction and structural checking.Constructing 2,463 linked records from 9,608 prepared RGB-D frames takes 325.25 seconds; rechecking 598,891 stored records takes 13.26 seconds, excluding source preparation and semantic review.Source: §4.4 · p. 8

4.5 TASK STRUCTURE AND EVIDENCE DEPENDENCE

  • On 282 images from 50 held-out physical scenes, linked supervision reaches 69.86% strict Ground–Relation success; closing stage tags raises the atomic mixture to 44.68% without changing linked performance.The remaining gap indicates that output syntax is only part of the composition problem; downstream stages also need correct entities and geometry.Source: §4.5 · p. 9, Figure 5 · p. 9
  • With questions and targets fixed, blank images reduce grounding accuracy from 57.35% to 7.66%, while shifting upstream boxes reduces Relation accuracy by 10.64 points.The interventions show dependence on both visual content and consumed geometry.Source: §4.5 · p. 9, Figure 5 · p. 9
  • On 1,012 balanced plan judgments, the Action specialist improves over the base by 6.72 points, reaching 89.53% accuracy on invalid plans but 55.53% on valid ones.Each valid plan is paired with an invalid permutation of the same actions, so action occurrence alone cannot distinguish the labels.Source: §4.5 · p. 9, Figure 5 · p. 9

5 CONCLUSION

ANVIL connects spatial data generation, evaluation, and multi-teacher learning through knowledge recirculation. Its final model improves on its base, and controlled studies support several complementary components.

5 CONCLUSION

  • The final model improves the seven-benchmark mean by 9.29 points over its base.Together, the findings support jointly designing reliable supervision and knowledge consolidation for embodied spatial intelligence.Source: §5 · p. 10
  • ANVIL connects spatial data generation, evaluation, and multi-teacher learning through knowledge recirculation.Shared scene references support cross-form verification, guide selective specialist transfer, and govern acceptance of returned supervision.Source: §5 · p. 10
  • Controlled studies support targeted coverage, selective transfer, and feedback-driven repair.Source: §5 · p. 10

A SPATIAL DATA FLYWHEEL AND NATIVE SUPERVISION

The section details the supervision interfaces and directs readers to appendices for routing, training, and evaluation protocols.

A SPATIAL DATA FLYWHEEL AND NATIVE SUPERVISION

A.1 NATIVE INTERFACES AND SCENE ALIGNMENT

The section connects scene-aligned native spatial interfaces to cross-stage verification and training, then describes how teacher supervision and student feedback are routed and normalized.

A.1 NATIVE INTERFACES AND SCENE ALIGNMENT

  • Eleven native forms retain shared entity references, while supported units and calibration metadata distinguish image-plane relations from metric measurements.Coordinates are normalized to a 0–1000 image scale; not every scene must support every form or a universal sequence.Source: §A.1 · pp. 16–17, Table 2 · p. 17
  • Generation templates specify geometric constraints, and review decisions bind support judgments and error categories to the exact saved candidate.Reviewers adjudicate disagreements; changes to an image, target, or task require the affected candidate to be reviewed again.Source: §A.1 · p. 18, Table 4 · p. 19
  • The 5.8M-record corpus spans 160K images, with 580K spatial records held out and 5.22M records used for training.The grounding evaluation panel is not a human audit of the held-out partition and does not establish corpus-wide annotation accuracy.Source: §A.1 · p. 19

B ROUTING, DISTILLATION, AND KNOWLEDGE RETURN

  • First-fault routing selects a downstream specialist only after preceding stages pass; uncertainty stops attribution rather than selecting a later stage.Malformed outputs use the nonempty suffix after the last valid boundary as the correction span.Source: B ROUTING, DISTILLATION, AND KNOWLEDGE… · p. 20

B.1 NORMALIZED TEACHER–TOKEN ALLOCATION

  • Teacher–token allocation jointly selects a specialist and correction span, with weights concentrated on sampled positions and optionally increased for structural tokens.Structural positions include fields, coordinates, relation labels, action keywords, and delimiters.Source: §B.1 · p. 20

B.2 WHY NORMALIZE THE CORRECTION SPAN?

  • Reference and eligible-span normalizations make aggregate loss coefficients independent of output serialization length.The transfer-loss bound limits loss magnitude, not gradient norms or downstream error.Source: §B.2 · p. 21

B.3 ROUND-LEVEL COUPLING

  • Student signals assist candidate generation or repair, but record admission remains governed by the verification rule and evaluation targets stay outside the process.Verification and distillation ensembles have distinct roles; accepted records may support specialist refresh between runs.Source: §B.3 · p. 21

C TRAINING AND EVALUATION PROTOCOLS

The protocols separate the final model and controlled-study checkpoints, define task-specific evaluation panels, and test coverage, routing, and repair under specified training conditions.

C TRAINING AND EVALUATION PROTOCOLS

  • The final ANVIL-MOPD main-table model is distinct from the earlier source-filtered checkpoint used in several controlled comparisons.Coverage and routing continue different checkpoints, while composition and data-quality comparisons start from the base model.Source: C TRAINING AND EVALUATION PROTOCOLS · p. 21
  • Evaluation keeps malformed or incomplete responses in the denominator and uses task-specific panels rather than substituting their scores for the held-out aggregate.Paired scene-bootstrap intervals use 5,000 resamples and condition on the two recorded routing seeds.Source: C TRAINING AND EVALUATION PROTOCOLS · pp. 21–22
  • Coverage arms match reported row exposures, while linked SFT and routing share specified LoRA settings, frozen vision encoding, and response-length limits.Coverage arms do not match unique images or realized tokens; routing uses frozen Ground and Relation teachers.Source: C TRAINING AND EVALUATION PROTOCOLS · p. 22
  • Named-feedback repair supplies the previous answer and failed category, whereas generic retry asks for correction without naming the stage.Neither repair arm receives reference boxes or target depth ranks, and both make one call per failed candidate.Source: C TRAINING AND EVALUATION PROTOCOLS · pp. 22–23
  • The image-exclusion subset removes 14 exact or near-image candidates from earlier corpora identically for every model and independently of correctness.On the full 300 cases, source-filtered and base models return 8.92 and 1.90 points per response on average, motivating first-point comparison.Source: C TRAINING AND EVALUATION PROTOCOLS · p. 23

D.1 COVERAGE AND RETENTION

  • Mixed supervision improves placement over replay in both runs, while vacancy alone maximizes placement but weakens several retained capabilities.Mixed training does not dominate all tasks; increasing the source-filtered model’s Where2Place response budget from 256 to 512 changes success from 35.16% to 32.96% on the same 100 cases.Source: §D.1 · p. 23

D.2 CORRECTION-STAGE SELECTION

The routing comparison tests whether semantic-uniform teacher–span allocation improves strict Ground–Relation success over flat allocation under shared stage normalization.

D.2 CORRECTION-STAGE SELECTION

  • 2.13 percentage points (95% CI: [0.39, 3.90]) is the paired gain for semantic-uniform over flat allocation on strict Ground–Relation success.Semantic-uniform exceeds flat in both seeds; both use uniform token weights, and stage normalization is shared across configurations.Source: §D.2 · p. 23, Table 9 · p. 24

D.3 REPAIR YIELD AND DOWNSTREAM LEARNING

Failure-specific feedback yields substantially more source-correct repairs, but the downstream learning gain is uncertain and varies with the rejected pool. The section also limits construction-cost evidence to structural rechecking, not semantic verification.

D.3 REPAIR YIELD AND DOWNSTREAM LEARNING

  • 59.22% strict success versus 56.38% for raw data differs by 2.84 points, but the paired interval [−1.41, 6.88] includes zero.Nested-pool performance is also nonmonotonic; the seed-17 results establish neither monotonic data scaling nor a reliable matched-size learning gain.Source: §D.3 · pp. 24–25
  • Named feedback recovers 111 source-correct records versus 11 for generic retry at similar GPU cost, a 10.09× per-call yield ratio.Allocated GPU time gives a 9.90× ratio; these production costs exclude subsequent student training.Source: §D.3 · pp. 24–25, Table 12 · p. 24
  • Named feedback recovers 65/78 depth-order failures but only 42/761 grounding failures, so yield depends on rejected-pool composition and information.The 885 rejected candidates form the repair pool; categories partition it by first failed check.Source: §D.3 · p. 24, Table 13 · p. 25
  • A retained grounding failure passes syntax and typed checks but fails source-based grounding, and both retry strategies return the same boxes.The selected case illustrates that localizing a failure can guide another attempt without guaranteeing correction; selected cases do not estimate failure frequency.Source: §D.3 · pp. 24–25, Figure 7 · p. 25

D.4 CONSTRUCTION AND RECHECKING COST

  • The CPU timing study covers structural rechecking only, excluding image loading, generation, and semantic review across the current eleven-format flywheel.The pass reads 598,892 lines, including one malformed line, and reports approximately 45.2K records/s; it covers eight historical storage types.Source: §D.4 · p. 26, Table 15 · p. 26

E TASK STRUCTURE AND EVIDENCE DEPENDENCE

The studies test linked versus atomic task structure, evidence dependence, and state-conditioned plan verification, then describe source-native specialist preparation. Their findings are bounded by task-specific setups and strict output requirements.

E TASK STRUCTURE AND EVIDENCE DEPENDENCE

  • Linked training reaches 69.86% success, while atomic-mixture success rises from 1.77% to 44.68%, leaving a 25.18-point linked-interface gap.The comparison uses 512 updates and 2,048 row exposures; recorded training tokens are not matched, and evaluation uses 282 images from 50 scenes.Source: E TASK STRUCTURE AND EVIDENCE DEPENDENCE · p. 26, Table 16 · p. 26
  • Visual and upstream interventions test dependence on supplied evidence, but their controls establish evidence dependence rather than an internal reasoning mechanism.All three specialist–joint intervals include zero, so point estimates do not establish a specialist advantage.Source: E TASK STRUCTURE AND EVIDENCE DEPENDENCE · p. 26, Table 18 · p. 27, Table 17 · p. 27
  • Action-order verification pairs valid plans with invalid permutations of identical actions, measuring state-conditioned validity rather than visual-only planning or standard ALFRED execution.State and source action rules are supplied, and class-wise scores expose invalid-answer bias.Source: E TASK STRUCTURE AND EVIDENCE DEPENDENCE · pp. 26–27, Table 19 · p. 27
  • A saved “left of the sink” prediction regresses after adaptation in both seeds, while the other figure panels measure structural validity, source correctness, and downstream learning.The selected regression case does not establish its frequency.Source: E TASK STRUCTURE AND EVIDENCE DEPENDENCE · p. 27

F SOURCE-NATIVE SPECIALIST PREPARATION

  • Source-native preparation scores remain task-specific: RoboBrain reaches 45/193 affordance IoU50, but all three preparation models fail the gripper-output contract.For Action, 653/3,327 baseline outputs are structurally valid and none matches the reference next-action label; strict scoring combines prediction with interface and label adherence.Source: F SOURCE-NATIVE SPECIALIST PREPARATION · p. 27, Table 20 · p. 28

G LIMITATIONS AND FUTURE WORK

The framework's scope is bounded by its predefined task inventory, specified spatial tasks and stage dependencies, and curated sources with explicit verification. The passages identify extending vocabulary, compositional competence, and interaction-based learning as future directions.

G LIMITATIONS AND FUTURE WORK

  • The eleven Ground–Relation–Act forms do not cover every open-world spatial concept or interaction requirement.Extending the vocabulary must retain the scene evidence supporting each target.Source: G LIMITATIONS AND FUTURE WORK · p. 28
  • Anvil studies transfer within specified spatial tasks and stage dependencies, leaving unfamiliar combinations of goals, environments, and embodiments for future work.Whether grounding, relational reasoning, and action knowledge compose into reusable expertise with less task-specific supervision remains an open question.Source: G LIMITATIONS AND FUTURE WORK · p. 28
  • The current framework draws on curated sources and explicit verification signals rather than learning through physical interaction outcomes.Connecting experience collection to specialist refinement and continued generalist learning remains future work.Source: G LIMITATIONS AND FUTURE WORK · p. 28

Related papers

Loading 2610.21882v1…