PaperlayerView exact revision on arXiv

Source-linked AI summary

The Gaussian Is Enough: Flow-Matching Priors Do Not Help When Fine-Tuning Large Behavior Models

Chen Xu, Rishi Shah, Hadas Kress-Gazit, Haruki Nishimura, Masha Itkina

arXiv:2609.27070v1cs.ROcs.AI

TL;DR

Whether non-Gaussian priors that help when training from scratch also benefit fine-tuning pretrained LBMs remains an open question. This paper compares priors across LBM fine-tuning settings and analyzes representations and outputs. The tested closer-to-target priors generally perform no better than the Gaussian, while encoder training is identified as the dominant factor.

  • Problem

    Whether the benefits of non-Gaussian priors established for training from scratch transfer to fine-tuning pretrained LBMs remains unknown.

    Source: §I · p. 2

  • Method

    The paper studies prior variants across LBM fine-tuning settings and analyzes prior-dependent embeddings, action predictions, and fine-tuning performance.

    Source: §I · p. 2, §VII · p. 7, §IX · p. 11

  • Results

    Across the evaluated settings, non-Gaussian priors that help from-scratch training generally yield fine-tuning performance statistically indistinguishable from or worse than a standard Gaussian.

    Source: §I · p. 2, §IX · p. 11

  • Takeaways & Limitations

    The reported encoder-learning-rate ablation identifies the observation encoder, rather than the prior, as the dominant factor in fine-tuning performance.

    Source: §IX · p. 11

    The evidence covers three LBMs and fine-tuning datasets of 20–400 demonstrations per task; behavior with substantially larger or more diverse data remains open.

    Source: B. Limitations · p. 10

Abstract

from arXiv · show

Modern robot imitation learning increasingly relies on generative policies based on diffusion or flow-matching models, which generate actions by transforming samples from a prior distribution. A key question is whether the choice of prior matters. Replacing the standard Gaussian with a closer-to-target, non-Gaussian prior has been shown to substantially improve performance when training from scratch. A natural next step is to ask whether these gains transfer to fine-tuning pretrained Large Behavior Models (LBMs) such as LBM 1.0, $\pi_{0.5}$, and GR00T~N1.5, where one might expect even larger gains. Surprisingly, we find that this is not the case, except possibly at very low fine-tuning data fractions. Across over 100K simulation rollouts spanning all three aforementioned LBMs on 40+ tasks in two simulation platforms, and 1250 hardware rollouts on five bimanual manipulation tasks, non-Gaussian priors that are demonstrably closer to the target yield statistically indistinguishable or worse fine-tuning performance than a standard Gaussian prior. Diagnostic analyses suggest why: fine-tuned imitation learning policies converge to similar action predictions across priors, despite their fine-tuned encoder embeddings diverging substantially from the pretrained embeddings and each other. A learning-rate ablation further confirms that encoder training is the dominant factor in fine-tuning performance, substantially outweighing the effect of prior choice. We conclude with concrete directions for future research on when and why learned priors might still matter in fine-tuning. Project page: https://cxu-tri.github.io/non_gaussian_FT/

I. INTRODUCTION

Although non-Gaussian priors closer to target actions help when training from scratch, whether those gains transfer to fine-tuning pretrained LBMs is uncertain. Across the studied settings, they generally do not improve fine-tuning, and diagnostics point to encoder training as more consequential than prior choice.

I. INTRODUCTION

  • The study asks whether closer-to-target non-Gaussian priors can transfer the from-scratch gains to pretrained LBMs, where the default Gaussian is a convention rather than a requirement.The motivating intuition is that a shorter transport distance might facilitate fine-tuning.Source: §I · p. 1
  • The paper presents a first large-scale study of non-Gaussian action priors for LBM fine-tuning.Its stated contribution is a negative finding at scale, consistency across settings, and diagnostic analyses.Source: §I · p. 2
  • After fine-tuning, non-Gaussian priors closer to target actions are statistically indistinguishable from or worse than the Gaussian, unlike their advantage from scratch.In the illustrated setting, Apre is 3× closer to target data, yet the Gaussian matches or exceeds it after fine-tuning; the full comparison covers seven priors.Source: §I · p. 2, Figure 2 · p. 2
  • The finding holds across seven prior constructions, three LBM architectures, most data fractions from 25% to 100%, and simulation and hardware settings, with a 5% data-fraction exception.The supplied passage identifies the exception but does not state its outcome.Source: §I · p. 2
  • Across priors, fine-tuned encoders diverge in representation while yielding comparable action predictions, and encoder training matters more than prior choice.The diagnostics report divergence from pretrained embeddings and each other alongside comparable policy-head outputs.Source: §I · p. 2

II. RELATED WORK

Prior studies use flow matching and diffusion for robot policies, while examining transfer and policy design without varying the action prior. This paper studies prior choice during fine-tuning and applies introspective metrics to that question.

A. Flow-Matching and Diffusion Policies for Robotics

  • Flow-matching policies transport actions from a prior to a target and permit arbitrary prior distributions, while diffusion and LBM work has typically used a Gaussian prior.Prior fine-tuning studies varied other design choices while holding the prior fixed.Source: A. Flow-Matching and Diffusion… · p. 2

B. Pretraining, Transfer Learning, and Introspection

  • Prior transfer research examines pretrained representations with probes, scaling laws, and likelihood metrics; this work applies such introspective tools to action-prior choice.The paper compares priors using downstream performance and metrics including LogME, per-observation ℓ2 distance, and cosine similarity.Source: B. Pretraining, Transfer Learning, and… · p. 2

C. Non-Gaussian Prior Distributions

Earlier non-Gaussian action-prior methods evaluate policies trained from scratch, leaving transfer to pretrained LBMs unexamined. The setup defines fine-tuning a pretrained policy and uses flow matching, which permits prior substitution without changing the procedure.

C. Non-Gaussian Prior Distributions

  • Existing non-Gaussian prior methods evaluate policies trained from scratch, and none of the cited work tests whether their benefits transfer to fine-tuning pretrained LBMs.Several approaches alter the policy architecture, while frozen-policy steering and inference-time warm-starting change sampling rather than training and are outside this study’s scope.Source: C. Non-Gaussian Prior Distributions · p. 3

III. PROBLEM SETUP

  • The setup fine-tunes a pretrained flow-matching or diffusion LBM on task demonstrations, with the action head transporting prior samples to demonstrated action chunks conditioned on encoder embeddings.The paper tests LBM 1.0, π0.5, and GR00T N1.5; “prior” denotes p0.Source: §III · p. 3
  • Flow matching requires paired prior and target samples, so replacing the Gaussian prior requires no change to training or inference, unlike DDPM-style diffusion.The cited passage notes that a non-Gaussian DDPM prior would require redesigning its noise schedule, objective, and sampling procedure.Source: §III · p. 3

IV. PRIOR VARIANTS

The study compares seven prior variants while preserving pretrained model weights so that prior choice alone varies. Three proposed priors derive from the pretrained LBM, alongside the Gaussian and three literature baselines.

IV. PRIOR VARIANTS

  • Seven prior variants include the default Gaussian, three proposed priors built from the pretrained LBM, and three literature baselines.The compared variants preserve pretrained encoder and action-head weights, so only the prior differs across experiments.Source: §IV · pp. 3–4
  • Apre uses a partially denoised action chunk from a frozen pretrained policy, shortening the transport distance to demonstrated actions.The denoising budget is split evenly between the pretrained policy and the fine-tuned action head.Source: §IV · p. 3
  • Apre+σAZ adds isotropic Gaussian jitter to pretrained actions to enrich a prior that could otherwise collapse around a few output modes.Source: §IV · p. 3
  • Epre+σEZ perturbs the pretrained encoder embedding with Gaussian noise before the pretrained policy produces an action prior.This varies the prior in embedding space rather than action space.Source: §IV · p. 3
  • The literature baselines are Cocos, a condition-dependent Gaussian; BRIDGER, a conditional CVAE prior; and Retrieval, a k-NN average of action chunks.Cocos learns its mean from the observation embedding, BRIDGER maps a Gaussian latent to an action chunk, and Retrieval uses a frozen embedding-indexed bank.Source: §IV · pp. 3–4

V. EXPERIMENTAL SETUP

The experimental setup evaluates whether non-Gaussian priors help fine-tuning, using controls intended to isolate prior choice across settings.

V. EXPERIMENTAL SETUP

  • The setup tests whether, and under what conditions, non-Gaussian priors benefit policy fine-tuning in simulation and on hardware.It also evaluates statistical significance and defines plotting conventions.Source: §V · p. 4
  • The setup also covers controls for confounds and significance testing, with simulation and robot hardware evaluations.The paper describes its plotting conventions as part of the evaluation setup.Source: §V · p. 4

A. Experimental Controls

  • Within each setting, task, data, architecture, optimizer, and fine-tuning steps are held constant across prior variants.Prior noise levels σA and σE are selected by grid search.Source: A. Experimental Controls · p. 4

B. Statistical Testing

The study uses formal multiple-comparison testing, randomized blind hardware trials, and three evaluation platforms, with plots indicating statistical separability.

B. Statistical Testing

  • Binary outcomes use STEP and continuous metrics use Welch’s t-test, with α=0.05 and Bonferroni correction for multiple pairwise comparisons.Methods sharing a CLD letter are not significantly different at the corrected level.Source: B. Statistical Testing · p. 4

C. Hardware A/B testing

  • Hardware trials randomize prior order within each fixed initial condition while keeping the operator blind to the prior under test.Interleaving trials distributes ambient drift approximately equally across priors.Source: C. Hardware A/B testing · p. 4

D. Benchmarks

  • Evaluation spans LBM Eval’s eight-task Drake simulation, RoboCasa’s 34-task MuJoCo suite, and a two-arm Franka FR3 hardware setup.Hardware rollouts receive binary success labels and task-specific milestone-based progress scores.Source: D. Benchmarks · p. 4

E. Plotting Conventions

  • Violin plots show posterior distributions for binary success and empirical distributions for continuous metrics; shared CLD letters indicate statistical indistinguishability.Arrows indicate whether higher or lower values are better.Source: E. Plotting Conventions · p. 4

VI. RESEARCH QUESTIONS AND RESULTS

The experiments ask whether closer priors help fine-tuning, whether any advantage depends on data fraction, and whether results transfer across architectures and settings.

VI. RESEARCH QUESTIONS AND RESULTS

  • The research questions test prior closeness, data-fraction effects—especially at low data—and consistency across model architectures, simulation, and hardware.These questions are introduced as the study’s experimental guide.Source: §VI · p. 4
  • Four settings address these questions: a controlled close prior, data-fraction sweeps, hardware evaluation, and tests on π0.5 and GR00T N1.5.Targeted ablations vary encoder learning rate, training objective, and noise scale.Source: §VI · p. 4
  • Table II indexes the settings, ablations, and analyses by their corresponding figures; daggered entries reuse existing rollouts.The dagger indicates no additional evaluations.Source: Table II · p. 5

Appendix controls & ablations

Across settings, non-Gaussian priors generally do not improve fine-tuning over the Gaussian baseline; the exception is a learned prior at 5% data, while diagnostics point to encoder training as more consequential.

A. Setting A: Fraction-Pretrained Prior

  • Setting A constructs a strong prior through two stages: Gaussian fine-tuning creates a task-aware checkpoint, then a second round varies only the prior.The checkpoint’s actions are measurably closer to the target than the Gaussian prior.Source: A. Setting A: Fraction-Pretrained Prior · p. 5
  • In Setting A, Apre reaches roughly 28% success versus 18% for Z from scratch, but becomes the worst-performing method under fine-tuning.No fine-tuned prior is statistically separable from the Gaussian baseline.Source: Figure 3 · p. 5, A. Setting A: Fraction-Pretrained Prior · p. 5

B. Setting Bsim: LBM Fine-Tuning in Simulation

C. Setting Breal: LBM Fine-Tuning on Robot Hardware

D. Setting C: Validation on Other LBM Architectures

  • On π0.5 and GR00T N1.5, Z and Apre+σAZ tie at the top, while Apre and Retrieval underperform on both atomic and composite tasks.Apre remains about 3× closer to the target than Z in ℓ2 on RoboCasa despite its lower performance in this setting.Source: Figure 6 · p. 7, D. Setting C: Validation on Other LBM… · p. 7

VII. ANALYSIS: WHY DOES THE PRIOR NOT MATTER UNDER FINE-TUNING?

  • Across the experiments, closer priors generally do not improve fine-tuning; the 5% data case is the limited exception, while hardware and other LBM tests show no general advantage.The analysis compares prior choice with encoder-focused diagnostics as a possible explanation.Source: B. Setting Bsim: LBM Fine-Tuning in… · p. 6, C. Setting Breal: LBM Fine-Tuning on… · p. 6, §VII · p. 7
  • The diagnostics compare predictive features, action predictions, and optimization trajectories across priors, then test encoder influence with a learning-rate ablation.Analyses focus on four priors and five data fractions across eight tasks in Setting Bsim.Source: §VII · p. 7
  • LogME feature-quality comparisons show all priors sharing CLD group “a” at every data fraction.Values are normalized against Z at each fraction because raw LogME drops sharply with more data due to a small-sample artifact.Source: Figure 7 · p. 8

A. Feature Quality: LogME Analysis

Across these diagnostics, priors yield broadly similar action predictions and feature quality despite differing encoder changes; reducing encoder training quality hurts fine-tuning more than changing the prior.

A. Feature Quality: LogME Analysis

  • LogME scores are inseparable across priors, indicating that fine-tuned encoders produce equally predictive features for action prediction.LogME is a training-free estimate of feature predictive power; higher scores indicate more predictive encoder features.Source: A. Feature Quality: LogME Analysis · p. 8

B. Action-Space Similarity After Fine-Tuning

  • Predicted actions largely overlap across priors by both ℓ2 distance and cosine similarity at every data fraction, despite small detectable mean gaps.The Gaussian is slightly worse on both metrics, but the open-loop gap does not translate into closed-loop success differences except against Epre+σEZ at 5%; large sample sizes can make tiny mean differences detectable.Source: Figure 8 · p. 8, B. Action-Space Similarity After… · p. 8

C. Representation Divergence Despite Similar Outputs

  • The authors speculate that changing both the prior and target with Apre forces greater encoder compensation; the broader transfer-learning implications of this divergence remain unexplored.They suggest different priors may lead optimization to distinct local minima with comparable downstream effectiveness.Source: C. Representation Divergence Despite… · p. 9
  • Fine-tuning changes action outputs similarly across priors, while encoder embeddings diverge substantially, with Apre restructuring them most and the Gaussian least.Action changes occupy a narrow band across priors, whereas embedding changes separate at every data fraction under both ℓ2 and cosine measures.Source: Figure 9 · p. 9, C. Representation Divergence Despite… · p. 9, Figure 10 · p. 10

D. Encoder Training Matters More Than Prior Choice

  • Reducing the encoder learning rate degrades performance more than prior choice, making encoder training quality the dominant fine-tuning factor.In simulation, success rates drop from 56–59% to 41–43%; on hardware, the standard encoder learning rate achieves 37% success versus 28% at one-tenth the rate.Source: D. Encoder Training Matters More Than… · pp. 9–10

VIII. DISCUSSION

Across the tested LBMs, non-Gaussian priors that help from-scratch training do not improve fine-tuning, while the encoder appears to matter more than prior choice. The discussion identifies limits to this evidence, open research questions, and the sampling procedures used for the tested priors.

A. Implications for Practitioners

  • The standard Gaussian is sufficient for fine-tuning LBMs with a flow-matching action head, while learned priors may still help when training from scratch.The encoder-LR ablation found a larger performance effect from reducing encoder learning rate than from prior choice.Source: A. Implications for Practitioners · p. 10, Figure 11 · p. 10

B. Limitations

  • Evidence covers three LBMs and 20–400 demonstrations per task; behavior with other model scales, pretraining distributions, or larger and more diverse fine-tuning datasets remains uncertain.The explanation that pretrained encoders dominate prior choice is plausible at other scales but unverified; alternative transfer metrics could reveal additional effects.Source: B. Limitations · p. 10

C. Future Work

  • At 5% data in Bsim, Epre+σEZ significantly improves over Z, but the transition where priors begin to matter is not characterized.Future work could map it across dataset size, task complexity, and model capacity, and test whether encoder divergence affects robustness or continual learning.Source: C. Future Work · p. 11

IX. CONCLUSION

  • Across 100K+ simulation and 1250 hardware rollouts, learned priors that help from-scratch training did not improve LBM fine-tuning, and encoder training was the dominant factor.Fine-tuned policies produced similar actions across priors despite divergent encoder representations.Source: §IX · p. 11

APPENDIX A PRIOR FORMULATIONS

  • The appendix specifies how seven priors construct a0 and allocates denoising budgets between prior construction and flow-matching transport.Priors that invoke the pretrained model use N/2 steps for each stage; others use all N steps for transport.Source: Appendix A · p. 13

APPENDIX B STATISTICAL TESTING DETAILS

The appendix describes sequential and continuous-outcome tests used for policy comparisons, followed by implementation and backbone hyperparameter details. For LBM 1.0, the controlled protocol holds settings constant across prior variants, with a separate setting checking whether the training objective confounds comparisons.

APPENDIX B STATISTICAL TESTING DETAILS

  • STEP provides anytime-valid error control for binary comparisons, while a Lai sequential test replaces it above about 600 samples per method.Welch’s t-test is used for continuous outcomes; CLD letters indicate which methods are not significantly different after correction.Source: Appendix B · p. 14

APPENDIX C IMPLEMENTATION DETAILS

  • Fine-tuning runs use one AWS p5.48xlarge node with up to 8 H100 80 GB GPUs.Source: Appendix C · p. 14

A. Fine-Tuning Hyperparameters: LBM 1.0 Backbone

  • For LBM 1.0, the controlled comparison holds hyperparameters identical across prior variants, and a flow-matching Gaussian run checks whether the objective choice confounds results.The Gaussian baseline otherwise retains LBM 1.0’s DDPM objective.Source: A. Fine-Tuning Hyperparameters: LBM… · p. 14

B. Fine-Tuning Hyperparameters: π0.5 Backbone

C. Fine-Tuning Hyperparameters: GR00T N1.5 Backbone

Under a flow-matching objective, no prior significantly outperforms the others. Diagnostics show comparable action predictions alongside prior-dependent changes in encoder representations.

C. Fine-Tuning Hyperparameters: GR00T N1.5 Backbone

  • GR00T N1.5 fine-tuning freezes the EAGLE backbone and updates only the action-head projector and DiT.The checkpoint uses an EAGLE vision-language backbone, Qwen3-1.7B LLM, and SigLIP-2 vision encoder.Source: C. Fine-Tuning Hyperparameters: GR00T… · p. 14

D. Tasks and Demonstration Counts

  • The experiments cover LBM Eval simulation tasks, five bimanual hardware tasks, and 34 RoboCasa tasks.The LBM Eval tasks include five Kitchen-K tasks and three unseen-variant tasks; the hardware set includes two adapted tasks and three new long-horizon tasks.Source: D. Tasks and Demonstration Counts · p. 14

A. Setting Bsim: Easier vs. Harder Task Breakdown

B. Raw LogME Scores

  • The sharp raw LogME-score drop from 5% to 25% data is a small-sample computation artifact, not necessarily a real change in feature quality.CLD groupings are identical to those computed from the same raw values in Fig. 7.Source: B. Raw LogME Scores · p. 14

C. Setting BFP: Flow-Matching-Pretrained Baseline

  • With every method pretrained and fine-tuned using flow matching, no prior significantly outperforms the others.This tests whether the main comparison was confounded by the Gaussian baseline using a different pretraining objective.Source: C. Setting BFP:… · p. 14

D. Setting BFP: Diagnostic Analyses

  • After fine-tuning, the four priors have indistinguishable LogME scores and comparably accurate action predictions, while encoder changes separate more than action changes.In the representation comparison, Z moves the encoder least and Apre the most.Source: D. Setting BFP: Diagnostic Analyses · p. 16

E. Setting BFP: Noise Ablation

  • Epre mixed-prior variants are robust across tested noise scales, whereas Apre variants degrade at higher noise.The selected main-experiment scales are σA=1 and σE=0.25.Source: E. Setting BFP: Noise Ablation · p. 16

F. Setting Breal: Hardware Prior-Selection Pilot

The hardware prior-selection pilot statistically ties three learned priors, and the broader per-task results leave the Gaussian statistically indistinguishable from other priors. Zero-shot analyses also show learned priors can be closer to the target or occasionally make task progress.

F. Setting Breal: Hardware Prior-Selection Pilot

  • All three pilot priors share a CLD group on both metrics; Epre+σEZ is carried forward based on its numerical lead.Partial progress was 69%, 74%, and 76%, and binary success was 31/75, 36/75, and 40/75, respectively.Source: F. Setting Breal: Hardware… · p. 17, Figure 20 · p. 18

G. Setting C: Zero-Shot Prior Quality

  • Across both RoboCasa task splits, zero-shot π0.5 actions from Apre and Retrieval remain about 3× closer to the target than Gaussian actions.This addresses whether their underperformance after fine-tuning simply reflects poor zero-shot prior quality.Source: G. Setting C: Zero-Shot Prior Quality · p. 17, Figure 21 · p. 19

H. Per-Task CLD Breakdowns

I. Zero-Shot Prior Rollouts

  • In zero-shot DumpVeg rollouts, the Gaussian is erratic, while Apre, Cocos, and Retrieval occasionally make legitimate task progress.The rollout comparison samples directly from each prior in place of the policy's predicted action.Source: §I · p. 18, Figure 24 · p. 21

Related papers

Loading 2609.27070v1…