Teach and Grow: An Agent-Centered Architecture for General Robot Learning

Chang Nie; Zhe Liu; Hesheng Wang

Source-linked AI summary

Teach and Grow: An Agent-Centered Architecture for General Robot Learning

Chang Nie, Zhe Liu, Hesheng Wang

arXiv:2608.17209v1Aug 17, 2026cs.ROcs.AIcs.CVcs.LG

TL;DR

General robot policies incur a retraining tax when local physical gaps require global updates. TGL turns sparse demonstrations into reusable, feedback-driven skills, achieving state-of-the-art LIBERO task success while supporting persistent skill reuse and adaptation without task-specific policy retraining.

Problem
Monolithic robot policies require global data collection, updating, and regression testing to repair local gaps, motivating architectures that add validated competence locally.
Method
TGL uses an agent to induce closed-loop Skill Blocks from demonstrations, compose them with physical feedback, and preserve validated behavior in persistent memory.
Results
TGL attains state-of-the-art task success on standard LIBERO, with studies showing skill induction, persistent reuse, recomposition, and adaptation to execution departures.
Takeaways & Limitations
TGL treats deployment as continued learning in which reusable experience can expand what a robot reliably does across future tasks.
Takeaways & Limitations
Unfamiliar tasks currently require several rounds of agent reasoning, tool use, and observation, making time the main practical cost.

Abstract

from arXiv · show

End-to-end vision-language-action (VLA) and world-action models offer an elegant route to general-purpose robotics, but their reliability is bounded by validated physical coverage. When an unfamiliar object, sensor, embodiment, or contact falls outside that coverage and no validated fallback exists, correcting the failure requires new robot data, a policy update, and regression testing. This recurring burden is the retraining tax. Unlike text, embodied data must often be created by operating machines. We present Teach-and-Grow Learning (TGL), an agent-centered architecture for general robot learning. In its general form, a multimodal agent turns a few successful demonstrations into reusable Skill Blocks: closed-loop behaviors for meaningful subgoals. In a new scene, the agent grounds and composes these blocks, selects learned or geometric tools, observes the physical outcome, and revises the route when execution departs from intent. A Skill Library stores executable behavior, while structured Experience Memory carries forward success, failure, and repair. New tasks are acquired without task-specific policy retraining. Our LIBERO evaluation attains state-of-the-art performance; controlled studies expose skill induction, persistent reuse, and agent-directed adaptation. Finally, we propose the Teach-and-Grow scaling-law hypothesis: if X denotes effective reusable experience, future-task error and teaching demand should approach irreducible floors as power laws in X. The architecture therefore treats deployment as a period of continued learning, in which one task can make the next easier.

I. INTRODUCTION

Teach-and-Grow Learning (TGL) addresses the retraining tax by organizing robot learning around an agent that converts sparse teaching into reusable, validated Skill Blocks and adapts through physical feedback. Persistent Skill and experience state lets competence grow across tasks while separating adaptive planning from robot-native execution and motivating a scaling-law hypothesis based on reusable experience.

Motivation: Validated-coverage gaps require new robot data, policy updates, and regression testing, making repeated global updates a bottleneck as tasks, sensors, and embodiments expand.Embodiment-aligned data must capture observation, action, contact, and physical outcome; web-scale semantics alone do not provide reliable geometry, dynamics, or contact behavior.
Agent-centered architecture: An AI agent retrieves and composes blocks, gathers missing evidence, checks physical effects after actions, and revises the route, while robot-native executors retain responsibility for metric motion, contact, and control.This division prevents language-level plans from becoming motor authority and localizes improvement to the capability that needs it.
Teach-and-Grow Learning: TGL turns a few successful demonstrations into reusable Skill Blocks: closed-loop subgoal behaviors whose objects, geometry, and control targets are re-instantiated in the current scene.The blocks are executable behaviors rather than trajectory fragments to replay, and sparse teaching seeds task structure while limiting unsafe trial-based exploration.
Persistent growth: Persistent Skill Library and Experience Memory store validated behavior, successful outcomes, failures, and repairs so later tasks can reuse and extend acquired competence without task-specific policy retraining.The agent handles unfamiliar conditions and persistent growth, while fast policies can execute stable blocks directly or learn from verified trajectories.
Scaling hypothesis: The Teach-and-Grow scaling-law hypothesis relates effective reusable experience X to future-task error and teaching demand, predicting that both approach irreducible floors as X increases.X is accumulated interaction that survives validation and remains retrievable, groundable, and composable in new scenes; unlike model size or frozen offline data, it can grow throughout the robot’s life.

II. RELATED WORK … C. Agentic Tool Use, Memory, and Lifelong Robots

The related work frames generalist VLA scaling as promising but limited by physical coverage, motivating TGL’s architecture of reusable, agentically composed skills. TGL combines demonstrations, tools, memory, and feedback into a persistent learning cycle rather than relying on direct imitation or one-shot workflow generation.

A. Generalist VLA and World-Action Policies: VLA models map visual-language observations directly to robot actions, with RT-1 and RT-2 establishing behavior-cloning and web-to-robot transfer.Later model families and larger robot datasets broadened this route, while scaling data and model resources produced systematic gains.
A. Generalist VLA and World-Action Policies: Physical coverage remains the binding resource because web-scale semantic knowledge does not provide action-conditioned geometry, contact, embodiment dynamics, or recovery.Probing studies identify a gap between instruction understanding and reliable execution under distribution shift; online reinforcement learning and few-shot modules reduce adaptation burden.
A. Generalist VLA and World-Action Policies: TGL makes the generalist policy one physical capability within a larger learning system, acquiring explicit reusable competence before optionally distilling it into a fast end-to-end path.This differs from treating the generalist policy as the sole container of robot intelligence.
B. Demonstrations, Few-Shot Adaptation, and Reusable Skills: Demonstrations, play, and language provide successful structure that can support behavior transfer, recombination, continual learning, symbolic composition, and few-shot adaptation.This literature avoids unrestricted physical exploration while making skill structure increasingly explicit.
B. Demonstrations, Few-Shot Adaptation, and Reusable Skills: TGL uses demonstrations to infer semantic subgoals, reusable relations, ordering, expected effects, and the scope in which a strategy applies, rather than directly imitating actions.Its Skill Blocks can include multiple perception-action iterations and use learned policies, planners, servos, controllers, or composed tools; they are re-instantiated from current observations.
C. Agentic Tool Use, Memory, and Lifelong Robots: Language-conditioned planners and tool-using agents organize robot programs, constraints, and feedback, while operating layers extend this approach to skill graphs, memory, and cross-embodiment interfaces.Related systems also retain deployment experience or grow and repair skill libraries over time.
C. Agentic Tool Use, Memory, and Lifelong Robots: Memory, tools, and executable skills have distinct roles: memory recalls experience, tools expose capabilities, and skills connect intent, scene grounding, execution, and success criteria.TGL keeps these roles separate so experience guides choice while verified blocks change what the robot can actually do.
C. Agentic Tool Use, Memory, and Lifelong Robots: TGL joins these ingredients into a persistent learning cycle in which demonstrations reveal structure, agents create closed-loop Skill Blocks, feedback reshapes composition, and outcomes alter future capabilities.The agent remains active at semantic state transitions instead of generating a workflow once and stepping aside.

III. TEACH-AND-GROW LEARNING · A. Learning with Fixed Model Weights · B. Skill Blocks

TGL learns across tasks by expanding an executable Skill Library and structured Experience Memory while keeping pretrained model parameters fixed. Its Skill Blocks ground semantic subgoals, verify outcomes, support composition across executors, and enable route revision during execution.

A. Learning with Fixed Model Weights: TGL learns new tasks by expanding explicit skills and memory while pretrained model parameters remain unchanged.New behaviors enter the Skill Library, while task context, failures, and repairs enter Experience Memory.
A. Learning with Fixed Model Weights: The Skill Library stores executable behavior, whereas Experience Memory stores context, outcomes, diagnoses, and repairs without executing actions itself.Interaction history preserves evidence used to update both stores.
A. Learning with Fixed Model Weights: TGL treats demonstrations as evidence about task structure and learns executable capability itself rather than merely producing supervised action targets.This contrasts its learning object with behavior cloning and reinforcement learning’s policy or value-function updates.
B. Skill Blocks: A Skill Block specifies a goal, applicability conditions, reusable strategy, grounded variables, compatible executors, outcome verification, and bounded recovery choices.The agent selects an executor from the block’s compatible executor set at each decision step.
B. Skill Blocks: Skill Blocks represent composable semantic subgoals rather than low-level motor primitives, allowing later blocks to depend on achieved effects instead of backend-specific execution.A container-acquisition block grounds a grasp and verifies retention; the next block only requires that the object is held.
B. Skill Blocks: At each semantic decision step, the task is represented as an ordered working plan whose route length can change after block outcomes.The agent may keep, shorten, or replace the remaining route, so the plan is not fixed at episode start.

C. Structured Experience Memory … A. Few-Shot Teaching and Cross-Demonstration Abstraction

TGL separates executable behavior from structured lessons, turning a few demonstrations into reusable Skill Blocks whose scope is determined by shared effects and whose future value depends on retrievable, grounded experience. Its scaling-law hypothesis predicts that as effective experience grows, future-task error and teaching demand decline toward task-dependent floors.

C. Structured Experience Memory: Experience Memory records what happened, why it happened, and what worked next, while the Skill Library stores how to act.Records remain linked to observations and provenance but are retrieved as compact guidance that people can inspect or revise.
D. The Teach-and-Grow Scaling-Law Hypothesis: Effective experience X measures how much verified past experience can help a robot act in a new scene, not how many episodes it stores.The score uses evidence reliability, added coverage, retrievability, grounding validity, and compatibility with admitted blocks; future-task outcomes are excluded.
D. The Teach-and-Grow Scaling-Law Hypothesis: As X grows, related future-task error and teaching required should fall toward task-dependent floors, with exponents α and β describing experience-to-competence efficiency.Many episodes that cannot be retrieved or re-grounded scale poorly, whereas a few verified episodes converted into reusable blocks may scale well.
D. The Teach-and-Grow Scaling-Law Hypothesis: The scaling law is testable by holding foundation models and tools fixed, increasing X across sequential checkpoints, and measuring future-task error and marginal teaching.Dteach is teacher intervention time until a preregistered success criterion; observations at an exhausted teaching budget are recorded at that cap.
IV. GENERAL AGENTIC ROBOT ARCHITECTURE: The agent-centered architecture treats perception, planning, control, and learned policies as capabilities selected inside Skill Blocks rather than one fixed global pipeline.The agent interprets tasks, retrieves candidate blocks, and decides what remains, while selected blocks bind contracts to observations and executors produce local physical changes.
A. Few-Shot Teaching and Cross-Demonstration Abstraction: A few successful demonstrations are segmented around meaningful state changes, so the learner abstracts desired physical changes instead of copying the original motion.Examples include acquiring an object, opening a drawer, or establishing a relation; synchronized observations are interpreted with the instruction.
A. Few-Shot Teaching and Cross-Demonstration Abstraction: Segments are aligned by semantic effect, and the agent preserves shared roles, relations, ordering, and effects while recomputing scene-specific pose, grasp, path, and control.Variation across demonstrations determines scope: a pattern tied to one instrument remains narrow, while repeated strategies across interchangeable objects support broader reuse.

B. Skill Blocks in the Current Scene · C. Dynamic Agentic Composition and Tool Selection · D. Physical Feedback, Bounded Exploration, and Targeted Teaching

TGL makes Skill Blocks reusable by preserving their meaning while rebuilding physical realizations from the current scene. An agent dynamically composes blocks and tools, verifies physical outcomes, revises execution, and requests targeted teaching when needed.

B. Skill Blocks in the Current Scene: A Skill Block preserves its meaning while rebuilding its physical realization from the current scene, enabling reuse beyond the teaching demonstrations.A block is tested in new scenes; success identifies reuse, failure narrows its range, and reliable blocks enter the library.
B. Skill Blocks in the Current Scene: The same block can use different backends, including geometric grasping, a VLA, or a tactile policy, while achieving and verifying the same result.Adding a sensor or executor changes only the blocks that require it.
C. Dynamic Agentic Composition and Tool Selection: Given instructions, observations, and persistent state (Bn, Mn), the Agent retrieves blocks and forms a working plan that can change when physical state changes meaningfully.The retrieved sequence is a plan rather than a script, allowing direct learned-policy routing or alternative contact tools.
C. Dynamic Agentic Composition and Tool Selection: At semantic decision step t, each block runs its own physical loop and returns the next observation and outcome evidence instead of emitting high-frequency motor commands.The evidence records whether the intended effect passed, failed, or remained inconclusive.
C. Dynamic Agentic Composition and Tool Selection: TGL adds semantic control that can select a different sensor, geometric planner, retrieved behavior, or task-level route after verified effects, unlike a workflow fixed before the episode.Conventional VLA or WAM feedback normally changes the next action without changing these predetermined workflow components.
C. Dynamic Agentic Composition and Tool Selection: A Skill Block carries a goal, scope, strategy, grounding function, executor set, success test, and recovery, then binds current objects and geometry before acting and verifying.This reusable contract supports runtime executor choice and outcome verification.
D. Physical Feedback, Bounded Exploration, and Targeted Teaching: After execution, the agent compares observed and intended effects, then continues, reobserves, selects another executor or route, or requests focused teaching before the next block.A mismatch triggers plan correction before the next block runs.
D. Physical Feedback, Bounded Exploration, and Targeted Teaching: Targeted teaching requests demonstrations for missing behaviors, ambiguous subgoals, or faster acquisition than unguided trial, making later guidance increasingly selective.Teaching seeds productive exploration, while interaction broadens the teaching set.

E. Structured Growth and Fast-Policy Distillation … B. Persistent Memory and Non-Destructive Growth

TGL grows robot capability by storing reusable behaviors alongside structured experience, composing and adapting them through physical feedback, and optionally distilling verified behavior into faster policies. LIBERO studies show persistent reuse and capability gains without changing model weights, while separating acquisition from later compression.

A. Learning as Capability Growth: TGL stores reusable Skill Blocks and the experience explaining their success, failure, or recovery in searchable, versioned, human-editable stores.A block may be implemented by a learned policy, planner, controller, or conventional program.
E. Structured Growth and Fast-Policy Distillation: Verified trajectories can train a fast student, while the agentic path continues handling novelty, diagnosis, and capability expansion.The organizing loop is to teach, compose, act and check, then grow; stronger foundation models may consolidate specialist tools.
V. EXPERIMENTS AND RESULTS: TGL achieves state-of-the-art task success on the completed standard LIBERO evaluation, with controlled studies examining induction, persistence, and feedback-driven adaptation.The experiments evaluate both the complete system and the internal learning cycle.
V. EXPERIMENTS AND RESULTS: After two learned blocks are added, the same executor solves 4/6 evaluations instead of 0/6 with six blocks, without changing model weights, runtime, or evaluation budget.Other controlled traces show physical evidence rebuilding the route or prompting a fresh observation.
A. Learning as Capability Growth: Few-shot demonstrations provide task structure, while the robot recomputes scene-specific objects, geometry, contact, and control rather than replaying demonstrations.This narrows the search without limiting execution to the demonstrated settings.
B. Persistent Memory and Non-Destructive Growth: A related task can begin from named behaviors and contextual experience already stored, making growth persistent, legible, inspectable, and revisable.Retrieval, grounding, and execution are required for a saved block to remain useful.
B. Persistent Memory and Non-Destructive Growth: TGL carries forward verified subgoal structure, required evidence, and loop-closing outcomes across changes in object, pose, camera, or executor.The original trajectory is not simply replayed in the new scene.
B. Persistent Memory and Non-Destructive Growth: During acquisition, pretrained parameters θ stay fixed while the Skill Library gains executable behavior and Experience Memory gains context for later choice.Effective reusable experience X counts what remains retrievable, groundable, and composable, not merely episodes saved.

C. The Agent as Organizing Intelligence … VII. CONCLUSION

TGL places an Agent above fixed robot workflows to organize evidence, executors, outcomes, and route revisions while preserving fast low-level control. Across deployment, reusable experience supports lifetime scaling: unfamiliar tasks can be taught, verified, retained, and eventually transferred to fast policies, making future learning easier.

C. The Agent as Organizing Intelligence: The Agent selects missing evidence, executors, success checks, and route revisions at meaningful workflow boundaries.In LIBERO traces, these decisions can produce a better view or rebuild the task route.
C. The Agent as Organizing Intelligence: Most motor control remains inside fast, tested executors; the Agent intervenes when subgoals, evidence, executor contracts, or scene fit become consequential.This creates a readable boundary between routine control and sparse, consequential decisions.
D. The Teach-and-Grow Scaling-Law Hypothesis: Useful experience from one task changes how the next task begins, so lifetime scaling depends on effective reusable experience rather than redundant trajectories.A single bridge block can enable many new compositions, whereas a large archive of redundant trajectories may add almost nothing.
D. The Teach-and-Grow Scaling-Law Hypothesis: With foundation models and tools fixed, increasing X is hypothesized to lower future-task error and marginal teaching predictably.The present studies establish block creation, persistence, and reuse; Appendix G specifies a longitudinal measurement of the lifetime curve.
E. A Slow Teacher and a Fast Student: A two-speed system lets the Agent learn unfamiliar tasks while a distilled fast policy executes familiar behavior, with uncertainty returning control to the Agent.Verified agentic trajectories can train a VLA, WAM, diffusion policy, or block-specific controller.
E. A Slow Teacher and a Fast Student: Separating slow teaching from fast execution preserves failure explanations and repairs before stable behavior is compressed into routine policy execution.The lifetime boundary moves as today’s newly taught task becomes tomorrow’s familiar fast behavior.
F. Limitations: The current practical limitation is time: unfamiliar tasks may require several rounds of Agent reasoning, tool use, and observation.This gap is expected to narrow as multimodal agents accelerate and mature behaviors are distilled into fast policies.
VII. CONCLUSION: TGL turns sparse local teaching into reusable Skill Blocks whose physical outcomes revise plans, while Skill Library and Experience Memory preserve executable behavior and lessons.LIBERO studies show that blocks can be induced, saved, reloaded, and recomposed, with Agent adaptation when execution departs from intent; deployment remains a period of learning focused on retaining useful experience.

APPENDIX A WHY END-TO-END SCALING REMAINS COSTLY … B. From a Model in the Workflow to an Agent That Owns the Workflow

The paper argues that physical coverage, precision demands, and global retraining make end-to-end robot learning costly, especially for long-tail failures and changing modalities or embodiments. Teach-and-Grow instead updates explicit skills, memory, and workflow state, enabling persistent, compositional, agent-directed learning without changing foundation parameters.

A. The Missing Data Substrate: Robot learning lacks text’s preexisting data substrate because valuable evidence requires synchronized physical observations, actions, embodiment state, contact, calibration, and failures.The most consequential evidence often includes near collisions, unstable grasps, partial insertion, sensor disagreement, and recovery after interruption.
B. Coverage Grows by Interacting Factors: F physically relevant factors with n_j regimes create ∏_j n_j coverage cells, while pairwise screening still requires Σ_{j<k} n_jn_k joint cells.Adding an independent factor multiplies the joint space, making exhaustive environmental variation prohibitively expensive.
C. The Retraining Tax Is Global Even When Failure Is Local: A local policy failure commonly reopens global data collection, optimization, and regression, whereas Teach-and-Grow adds a separately addressable Skill Block tested and retained without rewriting unrelated behavior.The retraining tax therefore turns a small uncovered physical region into a new version of the whole policy.
D. The Modality and Embodiment Tax: New modalities and embodiments remain theoretically representable, but each end-to-end change requires aligned interaction, fusion, calibration, optimization, and broad regression.TGL introduces modalities where their evidence is consumed and embodiments through the blocks and executors that realize relevant behavior.
E. The Long Tail Is Where Explicit Knowledge Matters Most: Experience Memory preserves diagnosed long-tail lessons, while Skill Blocks preserve the corresponding responses for retrieval, grounding, and control failures.Recorded explanations such as transparent walls confusing depth or side approaches causing collisions make failures reusable for later repair.
A. The Learned Object and the Update Operator: TGL learns explicit task structure and applies its update operator to skills, memory, and history while foundation parameters θ remain fixed.Demonstrations establish hypotheses about semantic effects, ordering, and persistent relations; runtime geometry, grasps, paths, commands, and backends remain variable.
B. From a Model in the Workflow to an Agent That Owns the Workflow: Unlike systems that fill one workflow slot, TGL’s agent owns an evolving cycle of evidence acquisition, block or backend selection, outcome assessment, revision, memory, and persistent skill creation.Its cycle combines few-shot teaching, cross-demonstration abstraction, closed-loop composition, persistent success and failure updates, and possible distillation into a fast policy.

C. Teaching and Learning Promote One Another … B. Taxonomy

Teach-and-Grow couples few-shot teaching with agentic execution: demonstrations provide structure, while physical outcomes expose missing roles, recoveries, tools, transitions, and tests. Reusable Skill Blocks and structured memory let the system refine behavior, focus later teaching on uncovered frontiers, and organize blocks by semantic effect.

C. Teaching and Learning Promote One Another: Teaching and execution form a mutual refinement loop: demonstrations reveal productive structure, while robot outcomes expose underspecified roles, recoveries, embodiments, or outcome tests.The resulting evidence improves subsequent teaching and learning rather than merely following a teach-then-grow sequence.
C. Teaching and Learning Promote One Another: Later teaching can target rare failures, new tools, and missing transitions between familiar blocks, reducing teaching demand as reusable experience accumulates.Early teaching may introduce entire behaviors; later teaching concentrates on the uncovered frontier.
A. The Minimal Agent Route: Planning from robot-view images established task understanding, but depth, collision, contact, and high-rate control still required robot-native competence.This motivated moving beyond direct multimodal-agent planning toward tools and embodied execution.
B. Tool Use and Agentic Visual Feedback: Perception, grasping, planning, control tools, and fresh post-execution observations enabled route changes when the world disagreed with the plan, though learning could still require many trials and model calls.This made zero-shot agent control technically feasible without eliminating the cost of physical task learning.
B. Tool Use and Agentic Visual Feedback: Structured memory preserved successes as strategies and failures as conditions and repairs, updating explicit memory rather than model parameters.These trials therefore became useful beyond the current episode.
C. Why Few-Shot Teaching Is the Practical Default: Few-shot teaching supplies a successful starting structure, allowing autonomous behavior to concentrate on uncovered variation, correction, and recovery instead of unrestricted exploration.Unrestricted exploration is slow and can be unsafe on a physical robot, making few-shot teaching the default route.
C. Why Few-Shot Teaching Is the Practical Default: The Agentic system is positioned as a slow teacher for acquiring novelty and generating structured training data, while mature tasks should migrate to fast policies.Sequential model calls make current Agent operation costlier than feedforward policy inference; model and caching improvements are expected to reduce that cost.
A. Why the Term Skill Block: Skill Blocks are decision-level, semantically closed building elements whose granularity depends on independent meaning and reuse, and whose taxonomy includes acquisition, transport, placement, activation, observation, and recovery.The same semantic block may be realized by a learned policy, planner, servo, or human intervention; blocks can contain substantial internal feedback loops.

C. A Populated Block from the Experiment … F. Validation and Library Update

The paper induces reusable Skill Blocks by extracting semantic effects from multimodal demonstrations, separating reusable strategy from runtime variables, and validating each candidate before library inclusion. Blocks are scoped by supporting variation and updated when fresh cases reveal weaknesses.

C. A Populated Block from the Experiment: The populated release block preserves the held-object relation and release strategy while recomputing the opening frame, release band, path, and commands in the current scene.This converts a demonstrated release behavior into a scene-grounded block rather than retaining episode-specific coordinates.
A. Source Normalization: Teaching becomes observable event sequences that retain source type, combining robot observations and actions, human-visible state changes and relations, and written intent and order.Each modality contributes a different part of the induced block.
B. Semantic Decomposition: The Agent splits trajectories at meaningful state changes using task language, object relations, gripper state, and motion, then assigns each segment a semantic signature.The signature records the subgoal, manipulated entity, affordance, target relation, preceding effect, and ordering context.
APPENDIX E DETAILED AGENT-LED SKILL INDUCTION: Cross-embodiment reuse preserves object roles and effects while the receiving robot supplies its own executor, grounding, and local verification.The present study uses one embodiment, while the procedure is intended to support a natural cross-embodiment test.
C. Cross-Demonstration and Cross-Embodiment Alignment: Cross-demonstration alignment matches segments by semantic effect and compatible entities or relations, separating shared task structure from pose, path, timing, and embodiment-specific control.Variable-length demonstrations need not share raw segment counts if their semantic effects support the same task composition.
D. Invariant Synthesis: Reusable strategy consists of semantic and behavioral invariants, whereas pose, perception, grasp, path, timing, and robot configuration remain runtime instance variables.A segment dependent on original coordinates remains an episode rather than becoming a Skill Block.
E. Scope Selection: Scope expands only when variation supports it: repeated poses support pose invariance, multiple instances support object abstraction, and contradictory demonstrations yield alternatives or narrower conditions.A single unusual instrument remains object-specific until additional evidence justifies broader reuse.
F. Validation and Library Update: Before retrieval, candidate blocks are tested on separated cases for intended effects, preservation of old behavior, and compliance with claimed scope; weak blocks are narrowed or repaired, while validated blocks enter a new library version.Validation therefore links fresh-case testing to persistent library updates.

APPENDIX F AGENTIC EXECUTION, MEMORY, AND FAILURE LEARNING … C. Why Performance Scaling and Cost Scaling Diverge

The architecture assigns an agent responsibility for selecting, composing, monitoring, and revising reusable behaviors while preserving inspectable experience and multiple levels of retention. Its scaling-law discussion distinguishes persistent experience from prior robot-learning regimes and predicts that local Skill Block additions can keep cumulative cost near-linear rather than convex.

A. The Agent’s Decision Surface: At task time, the agent selects observations, relevant Skill Blocks, ordering, execution backends, continuation criteria, and recovery actions.It reasons at meaningful state transitions while delegating continuous behavior to each block’s executor.
B. Success and Failure as Memory: A successful trace can establish or validate a block, while a failure records structured explanations linking ambiguous conditions, unsuitable grasps, planner failures, or weak outcome tests to observations and outcomes.Explicit memory lets humans correct rules, forbid recoveries, or mark incompatible tool versions.
C. Four Levels of Retention: TGL evaluates retention at four levels: storage, retrieval, grounding, and behavioral success, rather than equating a saved block with preserved competence.Persistent storage avoids direct parametric overwriting but does not prevent behavioral degradation.
A. What a Scaling Law Means: A robot scaling law must specify what experience counts, what remains fixed, what performance it predicts, and where the relationship fails or saturates.Existing language-model laws connected parameters, tokens, and compute to predictable power-law loss trends, with Chinchilla emphasizing compute-optimal allocation.
B. What Existing Robot Scaling Laws Measure: Existing robot scaling regularities report power-law-like effects from data, model size, compute, demonstrations, environments, and object diversity.Large imitation-learning studies often find diversity more valuable than repeatedly sampling the same condition.
B. What Existing Robot Scaling Laws Measure: Prior laws mainly describe offline resource growth, training diversity, or test-time computation, not repeated deployment that converts new experience into reusable competence.They therefore do not directly predict how a later task’s acquisition burden changes after persistent learning.
C. Why Performance Scaling and Cost Scaling Diverge: Dense coverage over F interacting factors with r regimes scales as r^F, while near tolerance P ↓ c, demonstration demand follows log Ndemo ∝ 1/(P − c).Generalization can reduce demonstrations, but newly discovered interactions still require fresh robot operation.
C. Why Performance Scaling and Cost Scaling Diverge: In TGL, each semantic capability addition is an admitted Skill Block; bounded local validation and hierarchical retrieval yield cumulative cost O(K), whereas all-to-all testing is quadratic.Figure 5 conceptually predicts convex growth for broadly coupled updates and near-additive growth for local additions.

D. The New Scaling Resource: Effective Reusable Experience … C. The Agent Supplies the Missing Organizational Layer

Teach-and-Grow treats effective reusable experience as a scaling resource: with the evaluation setup fixed, future-task error and teaching demand are predicted to approach irreducible floors as power laws in X_n, while local capability updates can remain near-additive. Its agent-centered organizational layer routes heterogeneous evidence and specialized robot methods through semantic contracts, enabling local integration and persistent reuse.

D. The New Scaling Resource: Effective Reusable Experience: Each stored experience receives an X_n score from five preregistered factors: reliability, added coverage, retrievability, grounding validity, and compatibility with admitted blocks.The score is fixed before the scaling study, and future-task outcomes are hidden from it.
D. The New Scaling Resource: Effective Reusable Experience: With the foundation agent, tools, budget, task distribution, and scoring rule fixed, future-task error and teaching demand decay toward irreducible floors as power laws in X_n.The cost-side prediction is that explicit capability can grow close to additively when updates remain local, so deployment can make later tasks easier without full-policy retraining.
E. A Longitudinal Test: The longitudinal study freezes the agent, tools, inference budget, success criteria, task distribution, and scoring rule while tasks are learned sequentially and checkpoints receive X_n before later outcomes are observed.The evaluation mixes old tasks, new compositions, new instances of known affordances, and tasks requiring genuinely new behavior.
E. A Longitudinal Test: At each checkpoint, the primary variables are future-task error E_future(X_n) and teacher intervention time D_teach(X_n), with budget-exhausted acquisitions retained at the preregistered cap.Retention, reuse, harmful retrieval, latency, and cumulative acquisition cost explain departures from the main curve.
A. Sources of Teaching and Experience: Teaching and experience come from teleoperation, robot trajectories, simulation, procedural generation, sim-to-real pipelines, human demonstrations, egocentric video, CAD, and autonomous exploration.These sources contribute different evidence, including controlled variations, semantic structure, constraints, failures, repairs, and unexpected successful alternatives.
A. Sources of Teaching and Experience: Because sources justify different claims, TGL preserves source type and uncertainty so semantic structure can transfer while physical claims are validated by an appropriate process.For example, manuals can expose ordering constraints, videos can show contact sequences, and simulation can provide exact state and broad variation without replacing real-world calibration.
B. Existing Robot Methods Become Roles, Not Rivals / C. The Agent Supplies the Missing Organizational Layer: Existing learned and classical robot methods become interchangeable architectural roles, connected by semantic contracts that specify intended effects, applicability, evidence, backends, success observation, and recovery.This permits local module integration: a new VLA or tactile sensor can replace or strengthen relevant blocks while unrelated, validated blocks remain unchanged.
APPENDIX H A WHOLE-SYSTEM VIEW OF THE ROBOT LEARNING: TGL’s whole-system ecology routes human knowledge, simulation, learned policies, perception, planning, control, and safety mechanisms through semantic verification, while verified capability and experience persist separately.The architecture supports future distillation and fleet sharing.

D. Verified Data Generation and Distillation … APPENDIX J CURRENT SYSTEM

Teach-and-Grow alternates agentic acquisition with distilled student deployment, while preserving reusable skills, structured experience, and validation across robots and scales. The reported studies show verified demonstration decomposition, task-specific skill reuse, fixed-executor gains, and localized failure modes, alongside a replaceable current system stack.

D. Verified Data Generation and Distillation: Agentic records capture boundaries, selected blocks, bindings, outcomes, verifier decisions, failures, recoveries, and success to train block executors, selectors, or mature-task policies.The proposed cycle alternates acquisition with compression into a fast student for routine deployment; uncertainty, distribution shift, or failure returns control to the agentic route.
E. Personal, Fleet, and Ecosystem Growth: Robots can share semantic strategies, failure signatures, outcome tests, and distilled students, while each recipient re-grounds and revalidates them for its own embodiment.The architecture combines foundation-model priors, demonstrations, simulation, human knowledge, learned and classical competence, agent organization, persistent memory, and distillation.
A. Agent Configuration and Trace Provenance: Online traces use gpt-5.6-sol with seed 0, structured decisions, at most one evidence refresh, and two replans, without human in-episode changes.The two representative successful traces come from a frozen LIBERO-GOAL archive and study route rebuilding for bowl-on-plate and new-observation gathering for drawer opening.
B. Visual Demonstration Decomposition: Ordered role/type accuracy is 1.000, while boundary F1 is 0.100 exact, 0.633 within one frame, and 0.900 within two frames, with 20/20 effect confirmations.The study uses ten successful demonstrations from two related LIBERO-Object condiment-transfer tasks, yielding 40 predicted and 40 reference stages without production reward or hidden-state inputs.
C. Task-Specific Skill Learning: Three teacher trajectories produce acquisition and release blocks that succeed on 3/3 separated evaluation states, survive save-and-reload, and succeed again on 3/3.On farther states 6–8, execution stops at the missing semantic effect, enforcing the learned route’s scope.
D. Related-Task Fixed-Executor Pilot: Evaluation success is 0/6 with six blocks versus 4/6 with eight, while observed median attempts fall from 2.5 to 1.5 under a fixed executor and budget.The pilot freezes library contents and scopes before testing both libraries on the same disjoint states 15–17; the two-sided Fisher exact test gives p = 0.061.
E. Failure-Localization Cohort: A separate acquisition cohort records 0/8 task success, with failures split across pre-planning (2), path consistency or calibration (4), and gripper closure (2).These categories identify where the feedback loop should improve next.
APPENDIX J CURRENT SYSTEM: The current stack combines a multimodal coding agent, perception and state binding, Contact-GraspNet, MPLib, bounded controllers, fresh-effect observations, versioned skills, and structured failure memory.Its roles are replaceable, and future temporal, tactile, 3D, learned-policy, planning, reward, or verifier components may consolidate into one executor while persistent records preserve scope and composition.