PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Robot Behavior Priors from Non-Robotic Synthetic Data

Mateusz Odrowaz-Sypniewski, Michael Amir, Jasmine Bayrooti, Ajay Shankar, Amanda Prorok

arXiv:2610.31275v1cs.ROcs.LG

TL;DR

Robot learning commonly relies on costly physical demonstrations or engineered simulators, raising the question of whether unembodied data can provide useful control structure. The paper pretrains in-context models on synthetic mathematical control abstractions and finds non-trivial prediction on unseen robotic tasks, with performance varying by whether embodied baselines saw the task during training.

  • Problem

    Robot learning models rely heavily on video, simulation, and physical demonstrations, while physical data collection is expensive and simulation requires domain-specific engineering.

    Source: §I · p. 1

  • Method

    The authors pretrain Transformers for in-context action prediction on synthetic policy-induced rollouts from MDPs, Contextual Bandits, and Linear Dynamical Systems.

    Source: §I · p. 1, §IV · p. 2

  • Results

    Across eleven simulated and four real-world task sets, accuracy was ≈58% in simulations (chance ≈11%) and ≈17% in real-world scenarios (chance ≈1.6%), above chance.

    Source: §VI · pp. 5–6

  • Takeaways & Limitations

    Abstract synthetic data can provide a scalable complement to embodied-data pipelines, without requiring physical experiences or embodiment-specific simulations to acquire structural priors.

    Source: §I · p. 2, §VI · p. 6

    The models are substantially smaller than recent large models, so whether synthetic control-prior advantages persist at larger foundation-model scale remains unverified.

    Source: B. Limitations on Scale · p. 6

Abstract

from arXiv · show

Real robot data, rich across tasks and morphologies, is often treated as the bottleneck for training robot learning models, including embodied foundation models that aim for zero-shot physical transfer. In this paper, we investigate the degree to which such models can acquire necessary structure using data from systems without embodiment. To this end, we test if inexpensive, procedurally generated synthetic data can yield useful priors before a model observes any real-world behavior. We study this through robot action prediction, a natural pretraining objective and a direct test of whether structure learned from arbitrary synthetic tasks transfers to embodied settings. We introduce three data generation pipelines that model mathematical abstractions of robotic control, and use them as pretraining priors for a Transformer. We demonstrate that a model trained purely on this non-robotic data attains non-trivial zero-shot prediction performance on unseen physical robot tasks via an in-context learning mechanism, and that our specialized control priors notably improve upon generic causal priors. We also compare against a compute-matched baseline trained on real robot data, and find that it outperforms our model but only on the datasets it was trained on. These findings suggest that procedurally generated, mathematical abstractions can yield useful robotic priors, offering a potential complement to expensive embodied data collection.

I. INTRODUCTION

The introduction asks whether generic, unembodied data can provide useful structure for robot learning despite the cost of physical demonstrations and domain-specific simulator engineering. It proposes synthetic mathematical control priors and reports zero-shot action prediction on held-out robotic tasks, positioning the approach as a complement to in-task training.

I. INTRODUCTION

  • Physical demonstrations are expensive, while simulators shift the burden to domain-specific engineering, motivating investigation of generic, entirely unembodied training data.Action prediction may exploit control regularities shared across robot embodiments.Source: §I · p. 1
  • The approach uses policy-induced synthetic rollouts from MDPs, Contextual Bandits, and Linear Dynamical Systems as specialized priors for Transformer-based in-context prediction.It builds on Prior-Data Fitted Networks [8] and contrasts specialized control structure with generic causal priors such as TabPFN.Source: §I · p. 1
  • A model pretrained only on mathematical control abstractions predicts actions above chance on held-out robotic tasks and is consistent across unseen tasks, while embodied baselines excel on tasks seen during training.The synthetic model overtakes the compute-matched embodied baseline on the unseen task set.Source: §I · p. 1
  • The authors frame mathematical synthetic pretraining as a complement, not a replacement, for training or fine-tuning with in-task data when available.The model's zero-shot designation refers to pretraining without robot data; deployment remains in-context and few-shot.Source: §I · pp. 1–2

II. RELATED WORK

Related work spans behavioral cloning, simulation-based robot learning, cross-embodiment foundation models, and synthetic-data in-context prediction. The paper positions its approach as testing whether non-robotic control abstractions can transfer to physical embodiments.

II. RELATED WORK

  • Behavioral cloning learns state-to-action mappings from expert demonstrations, but these approaches depend on large, diverse robot datasets, with insufficient scale or diversity degrading performance [19], [7].Recent extensions scale data, use Transformers and diffusion policies, adopt pretrained vision-language models [1], and chunk actions.Source: §II · p. 2
  • Simulation-based methods reduce dependence on real robot data by generating policy-training data, while modeling robot physics, actuators, and sensors to support physical transfer, [22],,.This paper instead tests abstract, non-robotic data rather than simulating a target robot or environment.Source: §II · p. 2
  • Generalist policies such as π0 [4] and GR00T [5] combine data across embodiments, whereas this work asks whether cheap procedural data can yield useful priors before any real robot behavior.The paper distinguishes its premise and scale from large curated robot datasets.Source: §II · p. 2
  • Prior-Data Fitted Networks [8] learn in-context prediction from synthetic tasks, and prior reinforcement-learning work transfers within simulated decision-making,,.This paper tests transfer to physical embodiments and compares bandits, MDPs, and linear dynamics with a generic causal prior.Source: §II · p. 2

III. PROBLEM SETTING

The problem setting casts robotic action prediction as in-context classification: predict held-out actions from state-action examples in a context set. Continuous actions are discretized for prediction, with an additional metric assessing recovery of the continuous action space.

III. PROBLEM SETTING

  • The model takes context state-action pairs and query inputs, then predicts query-action distributions in one forward pass without gradient updates.Continuous actions are mapped to K = 64 K-means bins, with each label denoting its nearest centroid.Source: §III · p. 2
  • A task is a dataset drawn from a prior and split into context examples and held-out queries.The model predicts actions for the query inputs using the context set.Source: §III · p. 2
  • Relative Centroid Distance measures predicted-to-true action-centroid distance relative to uniform random guessing; 0.0 is perfect and 1.0 is random.This supplements Accuracy and NLL by evaluating recovery of the continuous action space from discrete labels.Source: §III · p. 2

IV. METHOD

The method pretrains models on synthetic in-context tasks while holding architecture, compute, and training procedure fixed within experiment groups. Its control priors combine generic tabular mappings with abstractions of goal-directed decisions, state-conditioned actions, and smooth dynamics.

IV. METHOD

  • Synthetic episodes sampled from a mixture of generators form in-context tasks, and controlled comparisons vary the generator set while fixing architecture, compute, and training procedure.Each task samples a generator uniformly and z-scores features for scale consistency.Source: §IV · p. 2

A. Tabular (Generic) Prior

  • The generic tabular prior samples structural causal models, Bayesian neural networks, or Gaussian processes to create static feature-to-label datasets without sequential or embodiment structure.For robotic datasets, it uses quantile binning with 64 classes to match the action tokens.Source: A. Tabular (Generic) Prior · pp. 2–3

B. Synthetic Control Priors

  • The control priors cover goal-directed sequential decisions (MDP), single-step state-action associations (Contextual Bandit), and temporally correlated smooth dynamics (LDS).They encode no kinematic chain, sensor, or task; randomly permuted feature dimensions require structure to be inferred from context.Source: B. Synthetic Control Priors · p. 3
  • MDP episodes use sampled tabular systems with smooth reward and transition functions, solved by truncated backward value iteration and rolled out under Boltzmann policies.States and actions are discrete, with next states snapped to the nearest sampled state.Source: B. Synthetic Control Priors · p. 3
  • The contextual-bandit generator samples independent state-action pairs and uses a smooth bilinear reward function to select actions from a Boltzmann distribution.This removes temporal dynamics while retaining state-conditioned action selection.Source: B. Synthetic Control Priors · p. 3
  • The LDS generator rolls out linear systems with near-one state decay rates, action-specific displacements, and small noise under sampled linear-softmax policies.The resulting high temporal autocorrelation supplies smooth state evolution absent by construction from tabular MDPs.Source: B. Synthetic Control Priors · p. 3

V. EVALUATIONS

The evaluation predicts held-out actions from context state-action tuples across simulated and real-robot datasets. It uses crossfitting over sampled windows and reports results across three random seeds.

V. EVALUATIONS

  • Models predict each held-out half of a window using the other half as context via crossfitting.Each dataset contributes 512 windows of up to 2,000 timesteps.Source: §V · p. 3
  • Results are reported as the mean and standard error across three random seeds.The selected checkpoint has the highest mean accuracy on held-out data from the training distribution.Source: §V · p. 3
  • Each sampled window contains up to 2,000 timesteps and is split in half for context and prediction.Source: §V · p. 3

A. Datasets

The evaluation spans simulated environments, manipulation tasks, and four real-world robot datasets. Across these settings, control priors improve on the tabular prior, with results varying by task domain.

A. Datasets

  • The evaluation covers six simulated VMAS environments, five RoboMimic manipulation tasks, and four real-world datasets spanning manipulator, wheeled, legged, and aerial robots.Continuous actions are discretized into 64 centroids; each task is a timestep window split into context and query.Source: A. Datasets · p. 3

B. Evaluation of Priors

  • The compared configurations use the tabular prior alone, control priors alone, or a mixture of all four priors.The control generators are MDP, Contextual Bandit, and LDS.Source: B. Evaluation of Priors · pp. 3–4
  • Control-prior variants reach 0.58 and 0.56 accuracy versus 0.49 for the tabular-only baseline on VMAS; control priors reach 0.16 on RoboMimic and 0.17 on real-robot data.The control generators generally match or exceed the Tabular + Control mixture, which is ahead only on NLL; the real-robot table reports dataset-level means and standard errors across three seeds.Source: Figure 2 · p. 4, B. Evaluation of Priors · p. 4, Table I · p. 4

C. Ablations

The ablations show that the control generators contribute unequally: LDS is the strongest individual prior, while combining generators improves selected RoboMimic metrics.

C. Ablations

  • LDS is the strongest individual control prior and matches the full mixture on the real-robot suite.MDP and Contextual Bandit each fall behind the tabular baseline on at least one metric.Source: C. Ablations · p. 4
  • Figure 4 compares synthetic control-prior pretraining with in-domain embodied training on VMAS, separating tasks seen and unseen by the baseline.Source: Figure 4 · p. 5
  • Smooth, highly autocorrelated continuous states suggest that local temporal continuity transfers more than long-horizon value structure or state-action association alone.Source: C. Ablations · p. 5
  • Combining all three generators gives the best NLL and accuracy on RoboMimic and remains generally competitive across other task sets.Source: C. Ablations · p. 5

D. Models Trained on Embodied Data

Embodied baselines excel on the datasets they saw during training, while synthetic control priors remain above chance across robotic datasets and overtake the baseline on unseen datasets. On VMAS, the embodied baseline leads on training environments, but the ranking reverses on held-out tasks.

D. Models Trained on Embodied Data

  • The embodied baselines were trained on subsets of the evaluation suite, allowing comparison where in-domain data is available and where it is not.The VMAS baseline used three environments; the robotic baseline used three RoboMimic tasks and two real-world datasets.Source: D. Models Trained on Embodied Data · p. 5
  • 0.27 vs. 0.61 on VMAS held-out tasks reverses the training-environment ranking, despite shared dynamics and state-space characteristics.On training environments, the embodied baseline scores 0.83 against 0.55 for synthetic pretraining.Source: D. Models Trained on Embodied Data · p. 5
  • On robotic datasets, the real-data baseline excels on training datasets but falls to chance elsewhere, while synthetic pretraining stays above chance and overtakes it on unseen datasets.The comparison is split by whether the baseline saw each dataset during training.Source: D. Models Trained on Embodied Data · p. 5, Figure 5 · p. 5

E. Context Scaling

Accuracy rises with context size across three domains, with VMAS nearing saturation around 800 samples while real-robot accuracy is still increasing at 1,000. Across eleven simulated and four real-world task sets, synthetic mathematical priors show non-trivial transfer, though absolute accuracy remains modest.

E. Context Scaling

  • Accuracy increases monotonically with context size in all three domains; VMAS largely saturates around 800 samples, while the real-robot curve is still rising at 1,000.The 1,000-sample context is used in the main experiments, making reported real-robot accuracies a lower bound for what the prior may support with more context; more context trades compute and latency for accuracy.Source: E. Context Scaling · p. 5

VI. CONCLUSIONS & ADDENDA

  • Models pretrained only on synthetic mathematical abstractions show non-trivial zero-shot transfer across eleven simulated and four real-world task sets, outperforming generic causal priors.The models had never observed a robot, simulated or real.Source: §VI · p. 5
  • Absolute accuracy is approximately 58% in simulations and 17% in real-world scenarios, against random-chance levels of approximately 11% and 1.6%, respectively.The authors report better generalization and consistency on tasks outside training that share similarities in dynamics, environment, and signals.Source: §VI · p. 6
  • Abstract synthetic data is positioned as a scalable complement to existing pipelines that may not require costly physical experiences or embodiment-specific simulations.The paper frames this as a potential implication for progress toward zero-shot physical AI.Source: §VI · p. 6

A. Open-Loop State Predictions

Open-loop multicopter rollouts remain reasonably close to ground truth after 300 timesteps, while the paper qualifies cross-embodiment baseline failures by model scale. Fine-tuning helps on small data subsets, but training from scratch surpasses it on some datasets after 1,000–5,000 in-domain samples.

A. Open-Loop State Predictions

  • After 300 open-loop timesteps (about 15 seconds), multicopter final-state mean error is below 0.48 m, with worst-case error around 1 m.The rollout uses predicted actions without new projected states, and zero-order Euler integration, described as a poor approximation for high-order ODEs.Source: A. Open-Loop State Predictions · p. 6

B. Limitations on Scale

  • Compute constraints required smaller TabPFN models, whose reduced capacity may explain weak cross-embodiment generalization despite roughly 7 ms/query inference.The authors call for retesting with larger models and expanded training budgets to determine whether synthetic-prior advantages persist at scale.Source: B. Limitations on Scale · p. 6

C. Counter-Intuitive Fine-Tuning

  • On small in-domain subsets, the pretrained model outperforms training from scratch, but the scratch model wins on some datasets after as few as 1,000–5,000 samples.Both models improve as subset size grows; the authors say the separate phases may not admit naive chaining because pretraining and fine-tuning use different objectives and exposure patterns.Source: C. Counter-Intuitive Fine-Tuning · p. 6

D. Datasets, Implementation, and Reproducibility

The evaluation combines VMAS scenarios, RoboMimic demonstrations, DROID timesteps, and multicopter downwash data. The implementation uses a specified PFN Transformer configuration and matched embodied baselines; code and datasets are slated for release upon acceptance.

D. Datasets, Implementation, and Reproducibility

  • The authors state that all datasets and packaged code used for the work will be released upon acceptance.Source: D. Datasets, Implementation, and… · p. 6
  • VMAS actions are natively discrete, while all other actions are tokenized into 64 K-means bins.Table III summarizes the evaluation datasets and their state and action dimensions.Source: Table III · p. 7
  • The evaluation suite includes six VMAS scenarios, five RoboMimic tasks, one million DROID timesteps, and multicopter downwash data.RoboMimic transport observations are truncated from 127 to 100 dimensions to fit the model input limit; downwash states stack two vehicles’ positions and velocities over two timesteps.Source: D. Datasets, Implementation, and… · p. 7
  • The implementation uses a 12-layer, four-head Transformer and trains on 2,048-sample datasets for 15 million robotic or 9 million VMAS datasets.Embodied baselines use the same architecture and budget, replacing synthetic generators with real data.Source: D. Datasets, Implementation, and… · p. 7

Related papers

Loading 2610.31275v1…