PaperlayerView exact revision on arXiv

Source-linked AI summary

TinyAstral: Transferring Self-Evolving Harnesses to Tiny Generalist Manipulation Agents

Kehong Gong, Xiaoxu Zheng, Guanli Hou, Seunghun Lee, Mengqi Guo, Runjia Tan, Zhengyu Wen, Qi Wang, Xiaoyu He, Mingyuan Zhang, Michael Bi Mi

arXiv:2610.41216v1cs.ROcs.CV

TL;DR

Robot agents rely on both their planner and the harness around it, raising the question of whether a large teacher’s harness improvements can transfer to a compact agent with lower inference compute. TinyAstral evolves a frozen teacher’s harness, then distills its reasoning and recovery behavior into a student deployed in that same harness. The final 4B student performs strongly across training and unseen benchmarks while using about one fifth of its teacher’s accelerator time in the reported deployment measurement.

  • Problem

    It is unclear whether improvements to a large teacher’s harness can transfer to a compact agent that runs with substantially less inference compute.

    Source: §1 · p. 2

  • Method

    TinyAstral evolves a frozen open-weight teacher’s harness through statistically gated edits, then distills teacher reasoning and recovery trajectories into a student deployed in the same frozen harness.

    Source: §1 · p. 2, §3.2 · p. 3

  • Results

    Across training and unseen benchmarks, the final 4B student stays close to its teacher, scores close to a 552B planner on unseen benchmarks, and uses about one fifth of its teacher’s accelerator time per episode in the deployment measurement.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Across four harness versions, distilled students track their corresponding teachers, supporting harness evolution followed by distillation as a route to capable robot agents with lower inference compute.

    Source: §5 · p. 9

    Harness edits are constrained to general behavior rather than task-name branching, and RoboTwin alone uses task memory; the other five benchmarks use none.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Robot agents built on vision-language models depend on both the planner and its harness: the tools, feedback, memory, and recovery logic around it. Can improvements to a large teacher's harness be carried into a compact agent with lower inference compute cost? We present TinyAstral, a two-stage approach. We first freeze an open-weight teacher and improve its harness through recursive self-improvement (RSI): frontier LLM agents read the teacher's trajectories and propose harness edits, and each edit is accepted only if it passes a paired statistical test. We then freeze the final harness, collect teacher trajectories with reasoning and failure recovery, and distill a small student that replaces the teacher in the same harness. On RoboTwin, harness evolution raises the teacher's success from 33.4% to 63.6%; the final 4B student reaches 62.8%. Across four harness versions, distilled 4B students score close to their corresponding teachers, and later versions yield stronger students, showing that harness evolution gains carry into compact models. The harness gains also transfer to other frozen planners without retraining. On identical episodes, the final 4B student outperforms open 4B agents, the open 27B harness-RSI model SenseNova-RoboRSI, and our own teacher model placed in three competitors' harnesses. It stays close to its teacher on all three training benchmarks, and on three benchmarks never used for RSI or training it scores close to a 552B planner running the same harness. On the measured deployment episodes, it uses about one fifth of its teacher's accelerator time.

1. Introduction

TinyAstral asks whether harness improvements to a large teacher can transfer to a compact agent with lower inference compute. It evolves a frozen teacher’s harness, then distills the teacher’s reasoning and recovery behavior into a student; results show strong performance across training and unseen benchmarks.

1. Introduction

  • The harness includes tools, feedback, task memory, and failure recovery that help ground and execute the planner’s choices.The paper frames harness behavior as contributing to performance alongside the model itself.Source: §1 · p. 2
  • TinyAstral links harness evolution and agent distillation to test whether an improved large teacher’s decision-making can transfer to a compact model.The motivation is to run the improved system with substantially less inference compute.Source: §1 · p. 2
  • The method freezes an open-weight teacher, evolves its harness through master-proposed edits gated by paired statistical tests, then distills its reasoning and recovery trajectories into a student.At deployment, the student replaces the teacher in the same frozen harness.Source: §1 · p. 2
  • The final 4B student reaches 62.8% on RoboTwin after harness evolution raises the frozen teacher’s success from 33.4% to 63.6%.Across four harness versions, 4B students score 36.2%, 49.8%, 60.0%, and 62.8%, each close to its corresponding teacher.Source: §1 · p. 2
  • On identical episodes, the student outperforms open competitors and the teacher placed in three competitors’ harnesses, and scores close to a 552B planner on unseen benchmarks.The reported comparisons include 93.6% vs. 78.9% on Guava and 78.3% vs. 57.3% and 53.0% on LIBERO-PRO.Source: §1 · p. 2

2. Related Work

Prior work improves robot agents through harness evolution or trains compact agents from stronger models’ trajectories. TinyAstral combines these lines by evolving a teacher’s harness before distilling its behavior into smaller students.

2. Related Work

  • TinyAstral evolves a locally hosted open-weight teacher’s harness under paired statistical selection and distills the evolved system into 0.8B, 2B, and 4B students.Related systems include harness evolution from traces and distillation of skill-evolved harness traces, but TinyAstral tests successive snapshots and transfer across planners.Source: §2 · p. 2
  • TinyAstral’s distinction is to evolve the harness of a frozen open-weight teacher, measure transfer across planners and successive snapshots, and distill into smaller students.The related-work discussion contrasts this combination with systems that evolve harnesses or distill tool-call traces without this full sequence.Source: §2 · pp. 2–3
  • Prior small-agent work includes end-to-end VLAs trained on robot datasets and compact agents trained from reasoning, recovery, or harness traces.TinyAstral first evolves the teacher harness, freezes it for data collection, and distills at several snapshots to test whether improvements carry into student performance.Source: §2 · pp. 2–3
  • Agent self-improvement research includes systems that rewrite scaffolds or code, while distillation transfers strong agents’ trajectories to smaller models.TinyAstral evolves tools, feedback, memory, and recovery under paired statistical selection, then supervises rationales with tool calls inside closed-loop visual interaction histories.Source: §2 · p. 3

3. Method

TinyAstral first improves a frozen teacher through statistically gated harness edits, then freezes that harness and distills the teacher’s reasoning and recovery behavior into a compact student. The student replaces the teacher in the same harness at deployment.

3.1. Planner, harness, and scoring

  • The method separates the planner’s decisions about what to do from the harness’s execution of actions through perception, geometry, motion planning, and closed-loop corrections.The planner receives observations and returns reasoning plus one tool call; the harness executes it and returns a structured result.Source: §3.1 · p. 3

3.2. Stage 1: Evolving the teacher harness

  • Only the harness changes during stage 1: frontier-agent masters inspect teacher trajectories, propose edits, and use pre-registered paired statistical tests to accept or reject candidates.Humans set the rules and budget before the loop, while the teacher weights remain fixed.Source: §3.2 · p. 3, Figure 2 · p. 4
  • Candidates are tested on the same task/seed pairs, with disjoint training, selection, and evaluation seeds; the gate checks targeted improvement and regressions elsewhere.General edits cannot branch on task name; during final integration, the prompt and existing tool descriptions remain unchanged, and new components must have explicit triggers and pass a code audit.Source: §3.2 · p. 4

3.3. Stage 2: Collecting supervision from the frozen system

  • After evolution, the final harness is frozen and the teacher’s reasoning, recovery branches, and strict-success-filtered interactions supervise the student.Branches inject failures such as slipped grasps or displaced targets, and are retained only when replay is faithful and the teacher still reaches strict success.Source: §3.3 · pp. 4–5

3.4. Student training and deployment

  • Offline real-robot datasets are cast as auxiliary harness-format tasks, with teacher reasoning paired to dataset-derived action targets.The final teacher dataset includes training seeds from RoboTwin, Guava, and LIBERO-90; the student is fine-tuned and deployed in the frozen harness.Source: §3.3 · p. 5, §3.4 · p. 5

4. Experiments

The experiments test whether harness improvements transfer to compact students, how the final student compares across benchmarks, and what contributes to its behavior and deployment cost. Students trained with later harness versions perform better while remaining close to their corresponding teachers.

4.1. Setup

  • The evaluation covers three training benchmarks and three benchmarks never used for RSI or training, with paired comparisons on identical episodes for systems run by the authors.Undetermined outcomes count as failures, and the zero-shot benchmarks use their official success measures.Source: §4.1 · p. 5, §4.2 · p. 6

4.2. Harness evolution and transfer to compact students

  • 33.4% to 63.6% teacher success and 36.2% to 62.8% student success on RoboTwin accompany harness evolution across H1 to H∗.Each student scored close to its corresponding teacher; Table 1 covers the same 500 episodes and separately trained students at each version.Source: §4.2 · p. 6, Table 1 · p. 6

4.3. Performance and generalization

  • TinyAstral-4B stays close to its teacher on RoboTwin, Guava, and LIBERO-PRO, and outperforms the tested open agents on the same evaluation episodes.The reported teacher/student pairs are 62.8% vs. 63.6% on RoboTwin, 93.6% vs. 94.7% on Guava, and 78.3% vs. 80.0% on LIBERO-PRO.Source: §4.3 · p. 6, Table 2 · p. 7
  • On three benchmarks never used for RSI or training, TinyAstral-4B scores within 5.0 pp of its teacher and within 2.4 pp of the 552B DeepSeek-V4.1-Flash using the same harness.Distillation improves performance over the untrained 4B model across all four reported zero-shot settings.Source: §4.3 · p. 6, Table 2 · p. 7

4.4. Ablations and recovery behavior

  • Removing teacher reasoning lowers success by 19.3 pp on LIBERO-PRO, 11.2 pp on SimplerEnv Google VM, and 10.0 pp on VLABench.On LIBERO-PRO, 49 of 300 episodes reached the call limit without reasoning, compared with 7 under the full recipe.Source: §4.4 · p. 6, Table 3 · p. 7
  • Real-robot data raises held-out next-call accuracy from 0.235 to 0.923 without a statistically detectable success change across seven simulated settings.The measured benefit is in offline decisions on real observations.Source: §4.4 · pp. 6–7
  • The student recovers by re-describing targets, correcting placements, and changing grasp points, and runs closed-loop on a real SO-ARM101 arm.Of 271 zero-shot episodes solved by the student but not the untrained 4B model, 171 (63%) ended with the untrained model exhausting its call budget.Source: §4.4 · p. 7, Figure 3 · p. 8

4.5. Inference compute cost

  • The deployed student uses 25.3 accelerator·s per episode, versus 123.8 for its teacher and 249.6 for DeepSeek-V4.1-Flash on 20 shared SimplerEnv episodes.Its median episode is also shorter: 25.3 s versus 31.0 s and 31.2 s, respectively.Source: §4.5 · pp. 7, 9, Table 4 · p. 7

5. Conclusion

TinyAstral evolves a frozen teacher's harness, then distills its reasoning and recovery behavior into a compact student. The reported results support this approach as a route to robot agents with lower inference compute cost.

5. Conclusion

  • Across four harness versions, later-version students perform better while staying close to their corresponding teachers.The harness gains also transfer to other frozen planners.Source: §5 · p. 9
  • The final 4B student outperforms every tested open competitor on identical episodes and stays close to its teacher on three training benchmarks.On three unseen benchmarks, it scores close to a 552B planner running the same harness.Source: §5 · p. 9
  • On the measured deployment episodes, the final student uses about one fifth of its teacher’s accelerator time.The paper identifies extending the approach to more RSI environments as future work.Source: §5 · p. 9

A. All runs

The appendix points readers to the training-benchmark runs and their definitions.

A. All runs

  • Table 5 lists all training-benchmark runs, with benchmark definitions in Sections 3.1 and 4.1.Source: A. All runs · p. 12

B. Competitor protocols

The appendix documents per-episode call limits and how competitor systems were run and scored. It also specifies configurations for comparisons using the same teacher in alternative systems.

B. Competitor protocols

  • Each planner in the authors’ harness uses the same per-episode tool-call limit for a given benchmark; Guava uses its official per-task round limits.The other limits were set before evaluation.Source: B. Competitor protocols · p. 12
  • Guava-4B runs in its official harness on 450 Guava episodes, while PyRUA-Lean-4B runs in its official harness on all 500 matched RoboTwin episodes.Both are scored with the authors’ unified strict metric.Source: B. Competitor protocols · p. 12
  • The same teacher is evaluated in Guava, MotorMind, and PyRUA-Lean harnesses with different decoding, stopping, and call-limit protocols.MotorMind uses uncapped calls; PyRUA-Lean uses benchmark-trained VLA executors.Source: B. Competitor protocols · p. 12

C. Published comparisons

The published-comparison table covers reported results on six benchmarks alongside the advantages each system uses; Figures 4 and 5 extend the qualitative examples in Figure 3.

C. Published comparisons

  • Table 7 lists published results on six benchmarks and the advantages each system uses over TinyAstral’s setting.The table reports results from each cited work under its own protocol and episodes.Source: C. Published comparisons · p. 12

E. RSI details

The RSI process screens and tests proposed harness changes against the current version, while tracking accepted snapshots and candidate capabilities. The supplied tables describe snapshot and published-comparison scopes, and a figure caption gives examples of recovery behavior.

E. RSI details

  • Candidates are discarded when the target-cluster task-averaged 95% upper bound on gain is below +2 pp; survivors are merged into H′.Source: E. RSI details · p. 12
  • The published-results table covers six benchmarks and reports each cited work’s results under its own protocol and episodes.It also lists advantages used by each system and gives TinyAstral-4B’s Tab. 2 value in each block header.Source: Table 7 · p. 13
  • Accepted snapshots report teacher success on 500 fixed-denominator RoboTwin-50 evaluation episodes.Table 8 identifies Hk as the kth accepted snapshot and H* as the frozen final version.Source: Table 8 · p. 13
  • The master proposed 296 candidate capabilities: 121 were adopted, 16 later superseded, 142 rejected, and 17 undecided.The final harness contains 101 capabilities, 87 verbatim and 14 in successor form; most adopted capabilities missing from it were on unmerged side branches.Source: E. RSI details · pp. 13–14
  • Recovery examples include switching to an object’s visual description, lifting a cup to reveal a hidden cube, and retrying a rejected completion after a short lift.The figure also shows restoring a knocked-off bowl and describing a rolling pin by shape when its name is not recognized.Source: Figure 4 · p. 14

F. Training details

Student training uses simulated decision samples and a fixed optimization recipe, with additional real-robot next-call samples filtered from candidate steps. The supplied examples show both recovery behavior and a student failure.

F. Training details

  • The training set contains 41,877 simulated samples across RoboTwin, Guava, and LIBERO main episodes and recovery branches.Each sample is one decision, and loss is applied only to the final assistant turn.Source: F. Training details · p. 14
  • Training uses a 10^-5 learning rate, cosine decay, 5% warm-up, global batch 32, and a maximum sequence length of 12,288 tokens.The 2B and 0.8B students use the same data and recipe.Source: F. Training details · p. 14
  • On one episode, the untrained 4B model repeats grasp(eggplant) 22 times, while the student re-describes the object and succeeds in 7 calls.A separate RoboTwin example shows a student falsely declaring success after its own checks pass, despite a 2.3 cm target miss.Source: Figure 5 · p. 15
  • Next-call relabeling retains 19,598 of 23,070 candidate steps, and 9,771 next-call samples are drawn from that pool.Retained steps include 4,509 with reasoning from a matching first prediction and 15,089 explained given the reference call.Source: F. Training details · p. 15

G. Harness interface details

The harness supplies recent visual observations, task and robot state, previous-call feedback, and a call counter to the planner. Its object-level tools return measured outcomes and mark unreliable checks as unknown.

G. Harness interface details

  • RoboTwin observations include three camera views, task and scene-object text, measured gripper states, prior-call results, and a call counter.Only the two most recent turns retain images; earlier turns keep text and tool calls without reasoning, and the first turn also includes a task note.Source: G. Harness interface details · p. 15
  • Guava and LIBERO-PRO use the same planner loop and tool semantics through benchmark adapters, retaining the three most recent single-view images.Source: G. Harness interface details · p. 16
  • The RoboTwin harness exposes object-level manipulation and verification tools whose returns report measurements rather than intended outcomes.Place re-measures after release and reports an offset when off target; unreliable checks return unknown (ok=null), not a pass.Source: G. Harness interface details · p. 16
Loading 2610.41216v1…