Source-linked AI summary
Geometry Conditioning in an Embodied SLM: Training Controls and Robustness Diagnostics in a 0.8B Hybrid Model
Hao Li, Haofei Sun, Lin He
TL;DR
The paper asks whether physical-state geometry reliably improves manipulation in a small hybrid language model and compares token and recurrent-gate conditioning routes. Using frozen-backbone adaptation, controlled geometry variants, and robustness tests, it finds no reliable benefit from correct training-time alignment and separates coordinate invariance from physical-layout generalization.
Problem
It is unclear whether geometry helps a visual manipulation policy or whether apparent gains depend on correctly aligned geometric signals.
Method
The study compares token and recurrent decay-gate conditioning, shuffled geometry, token clocks, and no explicit geometry across three tasks, seeds, and robustness tests.
Results
Correct gate geometry scores 28.9% versus 36.7% for shuffled training and 24.4% without explicit geometry, while clock conditioning scores 11.1%.
Takeaways & Limitations
Training-time geometric alignment provides no reliable advantage under this recipe, and coordinate invariance does not predict robustness to physical object displacement.
Takeaways & Limitations
The study covers one model, three tasks, and three training seeds in simulation, leaving stronger recipes and broader settings untested.
Abstract
from arXiv · showhide
We study how physical-state inputs affect a 0.8B hybrid language model adapted for manipulation with 6.2M trainable parameters. Six conditions are trained on three LIBERO-Spatial tasks and evaluated over three seeds and 540 held-out rollouts. Conditioning recurrent decay gates on geometric increments yields 28.9% success, compared with 36.7% when those increments are shuffled during training and 24.4% without explicit object/goal geometry. Both geometry policies receive correct inputs at evaluation. A token adapter using the same increments scores 27.8%; differences vary across seeds and remain inconclusive. Token-clock conditioning scores 11.1%, including one seed that fails to converge. In separate robustness tests, a state-only relative-coordinate policy retains 7/10 success under frame relabeling, whereas all four tested visual policies fall to at most 3/20 after a 5 cm object displacement. These results show no reliable advantage from training-time geometric alignment under this recipe and illustrate the gap between coordinate invariance and physical-layout generalization. Episode records, seed-level analyses, and figure-generation code accompany the paper.
1 Introduction
The paper tests whether physical geometry improves manipulation when injected into token features or recurrent decay gates of a small hybrid language model. Across controlled routes and robustness tests, it finds no reliable benefit from correct training-time alignment and distinguishes coordinate relabeling from physical-layout generalization.
- Motivation: Small hybrid language models can be adapted for robot manipulation with few trainable parameters, but the benefit of geometry conditioning remains unclear.The study asks whether geometry helps visual policies and whether apparent gains depend on alignment with demonstrations.
- Experimental comparison: The study compares token adapters, decay-gate adapters, shuffled geometry, token clocks, and a no-geometry condition on LIBERO-Spatial.The base model is Qwen3.5-0.8B with frozen base weights and separately trained conditioning routes.
- Main findings: Correct geometry does not outperform shuffled training in the recorded rollouts, while the token-versus-gate comparison remains unresolved.The clock condition performs worse and includes one failed training seed.
- Robustness: The study tests robustness through coordinate relabeling without moving the scene and through physical object displacement.These tests separate coordinate invariance from generalization to changed physical layouts.
- Implications: The paper positions its contribution as identifying limits of geometry conditioning under a small adaptation budget and supplying controls for interpreting apparent benefits.The work is situated among visual manipulation, small-language-model, and efficient-policy research.
2 Setup: one frozen SLM, two pathways
The setup freezes the hybrid language-model backbone and compares two geometry-conditioning pathways under closely matched adaptation budgets. Geometry enters either the input tokens or recurrent decay gates, with controls for richer state features, shuffled timing, and token clocks.
- Host and adaptation: The Qwen3.5-0.8B hybrid model uses gated-delta and full-attention layers, while LoRA, an input projection, and an action head remain trainable.Training uses behavior cloning with chunked actions, frozen dual-camera features, language conditioning, and matched parameter counts.
- Gate conditioning: Gate conditioning adds a scaled, tanh-squashed bias to per-head decay logits in each of 18 GDN layers using a six-dimensional physical feature vector.The features encode translation increments, rotation increments, log time increments, and event boundaries.
- Gate conditioning: The gate adapter is initialized as an exact no-op and adds 2,034 trainable parameters across the layers.A trainable layer scalar controls the bounded tanh-transformed bias, while the clock control uses the same form and parameter count.
- Pathway comparison: The token adapter embeds the same six-dimensional geometry through a 6→2 linear layer and GELU before the shared input projection, whereas c2 uses a separate 14-dimensional state representation.The matched-input c1 and f0 routes differ mainly in where the shared geometry is transformed and applied.
- Protocol: The protocol jointly trains three LIBERO-Spatial tasks across three seeds and evaluates 540 held-out rollouts with stage-level episode records.Each task contributes 35 demonstrations and ten held-out initial states per seed and condition.
3 Findings
Across held-out rollouts and robustness diagnostics, geometry conditioning showed no reliable training-time alignment advantage, while coordinate invariance did not transfer to physical layout changes.
- Training-time alignment: 26/90 correct-gate successes versus 33/90 shuffled-training successes and 22/90 no-geometry successes showed no clear benefit from correct temporal alignment.The f0–shuffled contrast was −7.8 percentage points with descriptive p = 0.28; both received correct geometry at evaluation.
- Training-time alignment: 25/90 token-adapter successes versus 26/90 gate-adapter successes left the matched-input pathway comparison inconclusive.Their difference was −1.1 percentage points, with per-seed differences ranging from −10.0 to +13.3 percentage points.
- Training-time alignment: 10.0 percentage points separated c2 from c0, but the sample was too small to resolve the potentially useful effect and cell-level signs weakened the evidence.The c2–c0 comparison was 16–7 with p = 0.093; shuffled training exceeded c0 by 12.2 percentage points, but the uncorrected result was not consistent at the cell level.
- Clock conditioning: 28.9% correct-geometry gate success versus 11.1% clock-gate success included one failed clock-condition seed and does not isolate semantic content alone.The gate inputs differed in semantics, scale, and temporal statistics; the failed seed had L1 loss 0.882 versus 0.262 and 0.258 for its siblings.
- Frame relabeling: 7/10 relative-policy successes persisted across frame relabelings, whereas absolute-coordinate policies scored 4/10, 0/10, and 0/10 at 0°, 45°, and 90°.Relative pose features remain unchanged under a common rigid transform without retraining; the absolute policies were sensitive to frame changes.
- Physical object displacement: At most 3/20 successes at 5 cm and 0/20 at 10 cm occurred for all four tested visual policies after physical object displacement.Frame invariance alone was therefore insufficient to predict performance after physical displacement, which changes the physical task.
4 Implications and limitations
The controls show that correct temporal geometry alignment does not outperform shuffled training, while frame invariance and physical-layout generalization remain distinct properties. The study is limited to one model, three simulated tasks, three seeds, and narrow robustness evaluations.
- Implications: Correct temporal alignment does not improve over shuffled geometry training, although gate conditioning improves over the no-geometry baseline in aggregate.A test-time gate knockout would separately assess whether either trained policy relies on geometry.
- Implications: Frame relabeling and physical displacement measure different robustness properties and should be reported separately.Coordinate invariance does not establish adaptation to changed physical layouts.
- Limitations: The study evaluates one model, three simulated tasks, and three training seeds, limiting how broadly the findings can be generalized.The layout test covers one seed and two tasks, while the frame test uses state-only checkpoints and training initial states.
- Limitations: The 24.4% baseline and near-floor performance on task 8 leave open whether a stronger training recipe would reveal geometry benefits.The matched-input token adapter also uses a two-dimensional bottleneck, and wider embeddings could alter the pathway comparison.
- Limitations: Closed-loop outcomes vary across machines, and the study measures adaptation size but not latency, memory, or energy.These scope boundaries affect interpretation of practical deployment properties.
A Reproducibility and analysis provenance
The paper accompanies its analyses with episode records, manifests, ledgers, executable recomputation scripts, and figure-generation code. However, the historical ledger documents an evolving research process rather than an immutable preregistration.
- Reproducibility: The supplementary archive contains 540 main-table, 90 frame-relabeling, and 240 layout-shift episode records, plus the split manifest and historical experiment ledger.The archive also includes scripts for recomputing success counts, intervals, paired tests, sensitivity analyses, and distance summaries.
- Reproducibility: The analysis script can regenerate descriptive statistics, sensitivity analyses, stage rates, and rollout figures.The optional figures argument regenerates two rollout figures with Matplotlib.
- Analysis provenance: The historical ledger records planned gates, protocol changes, and observations rather than serving as an immutable registration of the final manuscript.Later sensitivity analyses and revised interpretations were added after inspecting outcomes.
- Analysis provenance: Executable analysis with exact seed, task, and episode pairing supersedes earlier pooled ledger statistics.The current figures use the supplied staged records consistently without cross-run selection.
B Training-fit table (optimization diagnostic only)
Training-fit losses vary across conditions and seeds, but these optimization diagnostics do not predict closed-loop success and should not substitute for rollout outcomes.
- Optimization diagnostic: 0.214 ± 0.019 was the f0-shuffled training loss, compared with 0.217 ± 0.001 for f0 across three seeds.These are fit-loss summaries, not rollout success results.
- Optimization diagnostic: 0.295 ± 0.065 was the c2 training loss, while c1 recorded 0.256 ± 0.010 and c0 recorded 0.282 ± 0.087.The fit ordering is recipe-dependent.
- Optimization diagnostic: 0.882, 0.262, and 0.258 were the b1 seed losses, with seed 0 failing to converge.That seed’s closed-loop score was 0/30 and was reported both pooled and excluded.
- Interpretation: Fit ordering does not predict closed-loop success, supporting closed-loop-first evaluation.Training summaries and rollout outcomes are distinct measurements.
C Per-cell success table
Per-cell success is reported over ten held-out episodes for tasks 0, 4, and 8 across seeds. Task 8 is near the floor, and seed variation can be as large as the differences between conditions.
- Per-cell evaluation: 10 held-out episodes per seed-task cell were evaluated for tasks 0, 4, and 8 of LIBERO-Spatial.The table therefore provides cell-level rather than only pooled success outcomes.
- Task sensitivity: Task 8 is near the floor for every condition.This makes task-level performance highly uneven across the evaluated task set.
- Seed sensitivity: 13/12/6 was the c2 success count across seeds, with within-condition seed variation comparable to between-condition differences.The paired tests are therefore treated as descriptive.
D Research practice and broader impacts
The study reports simulation-only results and emphasizes limits on interpreting spatial robustness or physical deployment. Its analyses are supported by recorded simulator outcomes, while supplementary materials enable reanalysis but omit trained checkpoints and a complete training/evaluation environment.
- Reported success rates come from recorded simulator outcomes rather than language-model judgments.
- Simulation-only experiments do not establish safe physical deployment, especially after scene changes that could cause collisions or property damage.
- The supplement supports rollout reanalysis but excludes a complete training/evaluation environment, trained checkpoints, and a complete accounting of exploratory compute.