PaperlayerView exact revision on arXiv

Source-linked AI summary

Intelligence Across Embodiments

Bo Ai, Henrik I. Christensen, Hao Su

arXiv:2609.27095v1cs.ROcs.AIcs.LG

TL;DR

Robotic embodiments differ across robots and over time, making it important for learning to accumulate across those differences. The paper proposes learned representations, embodiment diversity, and broad priors as ingredients for transfer, and calls for evaluations that characterize embodiment gaps. It identifies assumptions in engineered priors and the simulation-to-reality gap as limitations.

Abstract

from arXiv · show

Robotic embodiment encompasses the sensing, kinematics, dynamics, geometry, actuation, and control through which an agent physically interacts with the world. These properties vary across robots and change over time. We argue that general embodied intelligence requires learning that accumulates across these differences. Prevailing methods that engineer correspondences to bridge embodiment differences offer immediate practical gains, but their assumptions limit the scope of transfer in the long run. Instead, a more general approach should discover representations that support transfer to a larger range of embodiments as experience grows. We propose embodiment diversity as a promising axis of scaling, and identify broad learned priors as a complementary ingredient. We call for evaluations that better characterize embodiment gaps and transfer performance. More broadly, cross-embodiment learning connects the practical challenge of learning from heterogeneous robot experience with a broader scientific pursuit inspired by nature - physical intelligence that adapts and co-evolves with its embodiments to gain agency over its behavior and physical forms.

Experience Beyond One Embodiment

Robotic learning needs to accumulate experience across embodiments, because robots’ physical interfaces shape what they perceive, do, and contribute as data. Embodiments vary across populations and over time, so a general learning system must span both.

Experience Beyond One Embodiment

  • Extending machine learning to broadly capable robots depends on available experience, while robot-data scarcity remains a widely discussed obstacle.Learning from data complements specifying behavior through models and algorithms.Source: Experience Beyond One Embodiment · p. 1
  • Embodiment configures sensing, morphology, kinematics, dynamics, actuation, and low-level control, shaping what a robot can perceive and do.It also shapes how interactions are manifested as data.Source: Experience Beyond One Embodiment · p. 2
  • A general learning system must span embodiment differences both across robot populations and across generations.Variation arises from different environmental demands and also from manufacturing tolerances, payloads, calibration, sensors, controllers, wear, and redesign.Source: Experience Beyond One Embodiment · p. 2

From Engineered Alignment to Learned Representations

Engineered correspondences create shared interfaces for cross-embodiment transfer, but their task and environment assumptions can constrain what transfers. The paper argues for learning representations that can improve as experience grows.

From Engineered Alignment to Learned Representations

  • Engineered correspondences use shared hardware or model interfaces, including standardized grippers, image goals, contact, and end-effector actions.These interfaces connect embodiments with human knowledge to facilitate learning and transfer.Source: From Engineered Alignment to Learned… · p. 2
  • End-effector actions can limit tasks requiring whole-body coordination, while contact correspondences assume the target can reproduce relevant contacts within its constraints.When those assumptions fail, the prescribed interface may not express the interaction strategy needed for success.Source: From Engineered Alignment to Learned… · p. 2
  • Human-designed priors can support short-term progress, but their assumptions may constrain improvement from increasing experience and computation.The human form need not be best suited to every physical task and environment.Source: From Engineered Alignment to Learned… · p. 3
  • Cross-embodiment learning may discover representations ranging from embodiment-agnostic understanding to embodiment-specific predictions.The paper draws an analogy to fields where learned representations surpassed human-designed priors.Source: From Engineered Alignment to Learned… · p. 3
  • Representations may capture task semantics, spatial relations, interactions, or control, without following a fixed hierarchy.The goal is for accumulating experience to broaden transfer and reduce adaptation costs on new embodiments; latent actions could associate motor commands with similar physical effects.Source: From Engineered Alignment to Learned… · p. 3

What Enables Cumulative Transfer?

Broad learned priors and embodiment diversity are proposed as complementary ingredients for cumulative transfer. Early findings support transfer across embodiments, while scaling experience raises practical and architectural challenges.

What Enables Cumulative Transfer?

  • Broad pretraining may make later experience more useful, and learned priors may help robots interpret experience acquired through other embodiments.The passage cites gains from human demonstrations after broad robot pretraining and from human-video pretraining followed by robot-specific finetuning.Source: What Enables Cumulative Transfer? · p. 3
  • Embodiment diversity is proposed as a scaling axis alongside tasks and environments, with systematic variation needed to establish how far its gains extend.Experience across embodiments may reveal common interaction structure and how successful behavior changes with physical capabilities.Source: What Enables Cumulative Transfer? · pp. 3–4
  • Training across diverse end effectors supports adaptation to new hands and tools, and simulation training has enabled zero-shot transfer to unseen embodiments.In locomotion, increasing embodiment diversity produced broader generalization than increasing data on fixed embodiments.Source: What Enables Cumulative Transfer? · p. 4
  • Simulation can synthesize diverse embodiments, but generating feasible morphologies, compatible dynamics, and demonstrations poses challenges.The simulation-to-reality gap remains open, despite promising signs from visuomotor policies trained on simulated data.Source: What Enables Cumulative Transfer? · p. 4
  • Scaling to much larger embodiment populations raises questions about learning across distinct observation and action spaces and representing embodiment.Models may use explicit robot descriptions or infer properties from interaction history; visual identification may struggle when robots look alike but differ dynamically.Source: What Enables Cumulative Transfer? · p. 4

How to Evaluate Transfer?

The paper calls for evaluations that specify embodiment gaps and test-time information rather than relying on the label “unseen.” It distinguishes transfer settings and behavioral changes that require different capabilities.

How to Evaluate Transfer?

  • Transfer evaluations should specify how the target differs from training, its control interface, task assumptions, and test-time embodiment and interaction information.Reports should distinguish zero-shot generalization from few-shot adaptation.Source: How to Evaluate Transfer? · p. 4
  • Evaluation should distinguish held-out task–embodiment combinations from transfer to an entirely unseen embodiment with no collected data.Which calibration experiences on a new robot improve performance across other tasks remains a research question.Source: How to Evaluate Transfer? · p. 5
  • Motion-level transfer preserves task-space motion or contact patterns, whereas strategy-level transfer finds a different way to achieve the same goal.For example, a robot unable to shake a garment effectively may need slower, sequential unfolding.Source: How to Evaluate Transfer? · p. 5
  • Procedural simulation could benchmark transfer across mechanically plausible embodiments with systematic variation in sensing, morphology, dynamics, actuation, and control.Controlled comparisons can isolate accommodated differences, transfer failures, and how limits change as experience grows.Source: How to Evaluate Transfer? · p. 5

Towards Adaptive and Self-Evolving Robots

Cross-embodiment learning is framed as a route from practical experience reuse toward robots whose policies and physical forms adapt together. The section extends this vision to computational co-design and within-lifetime reconfiguration.

Towards Adaptive and Self-Evolving Robots

  • Efficient cross-embodiment transfer could reduce the cost of evaluating candidate robot designs and enable broader searches over morphology, sensing, and actuation.Designs could be assessed both for immediate performance and for how readily they support learning new capabilities.Source: Towards Adaptive and Self-Evolving… · p. 5
  • Cross-embodiment transfer could let policy learning and embodiment design improve together, with new designs exposing interactions and capabilities unavailable on earlier platforms.Better policies could make previously difficult designs viable, while better designs could make new behaviors easier to learn.Source: Towards Adaptive and Self-Evolving… · pp. 5–6
  • Within an agent’s lifetime, tools, sensors, or limbs could be selected or reconfigured to meet changing demands, with cross-embodiment control reusing capabilities across configurations.Examples include gaining reach, stability, or access without learning each configuration from scratch.Source: Towards Adaptive and Self-Evolving… · p. 6
  • The broader aim is physical intelligence that persists as embodiments change and gains agency over the physical means through which it pursues goals.The paper connects reusing experience across robots with adaptation and evolution in nature.Source: Towards Adaptive and Self-Evolving… · pp. 5–6
Loading 2609.27095v1…