PaperlayerView exact revision on arXiv

Source-linked AI summary

Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

Moo Jin Kim, Chelsea Finn, Percy Liang

arXiv:2502.19645v2cs.ROcs.AIcs.CVcs.LG

TL;DR

VLAs require effective fine-tuning for novel robots and tasks, but the best adaptation strategy is unclear. This paper evaluates decoding, action-representation, and learning-objective choices to form an OFT recipe, achieving strong LIBERO and real-world ALOHA results with faster action generation.

  • Problem

    Fine-tuning is necessary for satisfactory VLA deployment on novel robots and tasks, yet effective adaptation strategies remain unclear, particularly for high-frequency bimanual control.

  • Method

    The paper studies parallel decoding, action chunking, continuous actions, and L1 regression as an Optimized Fine-Tuning recipe for OpenVLA, with FiLM added for ALOHA language grounding.

  • Results

    97.1% average success rate across four LIBERO task suites and 43× faster throughput than base OpenVLA on ALOHA are reported for the proposed policies.

  • Takeaways & Limitations

    OFT enables OpenVLA to adapt to novel robots, bimanual configurations, and multiple input views while maintaining strong task performance and language following.

  • Takeaways & Limitations

    L1 regression may struggle with truly multimodal action distributions, while the source of inconsistent language grounding without FiLM remains unclear.

Abstract

from arXiv · show

Recent vision-language-action models (VLAs) build upon pretrained vision-language models and leverage diverse robot datasets to demonstrate strong task execution, language following ability, and semantic generalization. Despite these successes, VLAs struggle with novel robot setups and require fine-tuning to achieve good performance, yet how to most effectively fine-tune them is unclear given many possible strategies. In this work, we study key VLA adaptation design choices such as different action decoding schemes, action representations, and learning objectives for fine-tuning, using OpenVLA as our representative base model. Our empirical analysis informs an Optimized Fine-Tuning (OFT) recipe that integrates parallel decoding, action chunking, a continuous action representation, and a simple L1 regression-based learning objective to altogether improve inference efficiency, policy performance, and flexibility in the model's input-output specifications. We propose OpenVLA-OFT, an instantiation of this recipe, which sets a new state of the art on the LIBERO simulation benchmark, significantly boosting OpenVLA's average success rate across four task suites from 76.5% to 97.1% while increasing action generation throughput by 26$\times$. In real-world evaluations, our fine-tuning recipe enables OpenVLA to successfully execute dexterous, high-frequency control tasks on a bimanual ALOHA robot and outperform other VLAs ($π_0$ and RDT-1B) fine-tuned using their default recipes, as well as strong imitation learning policies trained from scratch (Diffusion Policy and ACT) by up to 15% (absolute) in average success rate. We release code for OFT and pretrained model checkpoints at https://openvla-oft.github.io/.

I. INTRODUCTION

VLAs perform strongly across robots and tasks, but adapting them to novel setups remains underexplored, especially for fast bimanual control. This work studies VLA fine-tuning choices and proposes OpenVLA-OFT, which improves benchmark performance and action-generation throughput.

  • Fine-tuning is crucial for deploying VLAs on novel robots and tasks, but the most effective adaptation strategy remains unclear.
  • Autoregressive VLA fine-tuning can be too slow for high-frequency control and unreliable on bimanual manipulation tasks.Prior autoregressive methods operate at 3–5 Hz, while high-frequency control may require 25–50+ Hz.
  • The study compares autoregressive versus parallel decoding, discrete versus continuous actions, and next-token prediction versus L1 regression or diffusion objectives.
  • Parallel decoding with action chunking enables simultaneous future-action prediction, improving efficiency while supporting flexible input-output specifications.OpenVLA’s original autoregressive scheme requires KD sequential decoder passes for K-step chunks, making chunking impractical for high-frequency robots.
  • 97.1% average success rate across four LIBERO task suites surpasses fine-tuned OpenVLA at 76.5% and π0 at 94.2%.OpenVLA-OFT also achieves a 26× action-generation speedup with 8-step action chunks.
  • 43× faster throughput than base OpenVLA is achieved by OpenVLA-OFT+ with 25-timestep action chunks on real-world bimanual ALOHA tasks.OpenVLA-OFT+ outperforms fine-tuned VLAs and imitation-learning policies by up to 15% absolute in average success rate.

IV. STUDYING KEY VLA FINE-TUNING DESIGN DECISIONS

The study implements controlled alternatives to OpenVLA’s original autoregressive, discrete-token, next-token formulation. The alternatives target faster decoding, action chunking, continuous action precision, multimodal inputs, and different learning objectives.

  • The experiments adapt OpenVLA with LoRA while preserving its original pretraining formulation.The relatively small fine-tuning datasets contain 500 demonstrations versus 1M pretraining demonstrations.
  • Action generation strategy: Parallel decoding replaces causal attention with bidirectional attention and predicts all output actions in one forward pass.The model receives empty action embeddings and reduces generation from D sequential passes to a single pass.
  • Action generation strategy: For chunk size K, parallel decoding predicts KD actions in one pass, increasing throughput approximately K-fold with minimal latency impact.Experiments report no performance degradation across diverse tasks despite the potentially lower expressiveness of parallel decoding.
  • Action representation: Continuous actions replace 256-bin discretization with direct values generated by an MLP action head.The study evaluates continuous actions because discretization can sacrifice fine-grained action details.
  • Learning objective: L1 regression trains the action head to minimize mean absolute prediction error against ground-truth actions.This preserves parallel-decoding efficiency while potentially improving action precision.
  • Learning objective: Conditional diffusion models denoise action samples during inference, requiring 50 diffusion steps in the implementation.The approach may provide more expressive action modeling but increases deployment latency through multiple forward passes.
  • Input-output flexibility: The flexible input pipeline combines multiple camera views, robot state, and language tokens in a shared latent sequence before action generation.

C. Augmenting OpenVLA-OFT with FiLM for Enhanced Language Grounding.

ALOHA’s multiple viewpoints can create spurious visual correlations that weaken language following. The OFT+ variant adds FiLM to modulate visual representations with language information, and applies it specifically in the ALOHA experiments.

  • Spurious correlations in wrist-camera and other visual inputs can cause policies to rely on visual cues rather than user instructions.
  • FiLM projects averaged task-description embeddings into scaling and shifting vectors that modulate visual features through an affine transformation.
  • FiLM modulation applies the same language-derived feature transformations across all visual patch embeddings rather than treating each patch as an independent feature.
  • FiLM is applied after self-attention and before the feedforward layer in each vision-transformer block, only for ALOHA experiments.
  • The evaluation studies how design decisions affect task success, inference throughput and latency, and input-output flexibility.

A. LIBERO Experimental Setup

The LIBERO study evaluates OpenVLA adaptation choices across four task suites and measures both task success and inference efficiency. It compares decoding, chunking, action representation, and learning-objective variants against baseline policies.

  • Experimental benchmark: LIBERO evaluates generalization across Spatial, Object, Goal, and Long task suites, each with 500 demonstrations across 10 tasks.The benchmark uses a Franka Emika Panda arm and includes images, robot state, task annotations, and delta end-effector pose actions.
  • Baselines: The primary comparison uses OpenVLA fine-tuned with its original recipe, alongside prior pretrained and from-scratch imitation-learning policies.The broader comparison includes Octo, DiT Policy, Seer, π0, Diffusion Policy, MDT, and other baselines.
  • Design variants: The study jointly evaluates parallel decoding and action chunking with discrete actions, continuous actions plus L1 regression, and continuous actions plus diffusion.The analysis motivates joint use of parallel decoding and action chunking for high-frequency control, particularly on bimanual robots.
  • Task performance: 14% absolute higher average success rates result from parallel decoding and action chunking versus autoregressive OpenVLA policies.Continuous action variants add another 5% absolute over the discrete variant, while L1 regression and diffusion achieve comparable performance.
  • Inference efficiency: 26× higher throughput results when action chunking with K = 8 is combined with parallel decoding, compared with baseline OpenVLA.Parallel decoding replaces seven sequential decoder passes with one; continuous L1 actions add negligible efficiency cost.

E. Optimized Fine-Tuning Recipe

The optimized fine-tuning recipe combines parallel decoding with action chunking, continuous actions, and L1 regression, then extends the recipe for real-world ALOHA evaluation. It targets efficient, high-frequency control while retaining simple optimization and flexible inputs.

  • Recipe: OFT combines parallel decoding with action chunking, continuous action representation, and an L1 regression objective.The recipe is designed to support high-frequency deployment while maintaining algorithmic simplicity.
  • Pretraining ablation: Removing OpenVLA’s pretrained representations causes a 5.2% absolute drop in average LIBERO success rate.The ablation indicates that pretrained representations remain beneficial despite distribution shift from the alternative fine-tuning formulation and additional inputs and outputs.
  • ALOHA evaluation: The real-world evaluation targets novel dexterous tasks on ALOHA, a high-frequency bimanual robot setup that differs substantially from OpenVLA’s single-arm pretraining data.The setup includes two ViperX 300 S arms, three camera viewpoints, and 14-dimensional joint-angle state inputs operating at 25 Hz.
  • OFT+: OFT+ augments OFT with feature-wise linear modulation for enhanced language grounding and is instantiated as OpenVLA-OFT+.The augmented recipe is used for the ALOHA experiments.
  • Task design: The four ALOHA tasks test deformable-object manipulation, long-horizon skills, tool use, and language-driven control.Examples include folding shorts, folding a shirt, scooping specified ingredients into a bowl, and placing items into a pot.
  • Evaluation comparison: OpenVLA-OFT+ is compared with ACT, Diffusion Policy, RDT-1B, and π0 using rubric-based partial-completion scores across the four tasks.The figure reports average percent completion for each task; OpenVLA-OFT+ achieves the highest average performance.
  • Training procedure: ALOHA fine-tuning runs independently per task for 50–150K gradient steps with action chunk size K = 25, executing each full chunk before re-querying.Training uses total batch size 32 across eight A100/H100-80GB GPUs.

B. Methods in Comparison

The ALOHA evaluation compares fine-tuned VLAs with imitation-learning policies trained from scratch across task execution and language following, including targeted qualitative and ablation analyses.

  • Evaluation setup: All methods are evaluated on four ALOHA tasks using a predetermined rubric that scores partial task completion.Figure 4 reports aggregate performance, while Figure 5 isolates language following on language-dependent tasks.
  • From-scratch baselines: Diffusion Policy matches or exceeds RDT-1B on clothes folding and scooping but struggles with the “put X into pot” task.The larger training dataset for that task is presented as evidence of limited scalability compared with VLA-based approaches.
  • Overall comparison: Fine-tuned VLAs generally outperform policies trained from scratch in task execution and language following.The comparison includes ACT, Diffusion Policy, RDT-1B, π0, and OpenVLA-OFT+.
  • Fine-tuned VLAs: RDT-1B follows language well but can continue acting after missed bowl placement, whereas π0 more often recovers from failed grasps through visual feedback.π0 has smoother, more reactive execution and better overall task completion, despite slightly weaker language following than RDT-1B.
  • Fine-tuned VLAs: OpenVLA-OFT+ achieves the highest performance across task execution and language following among the compared methods.Its rollouts include clothes folding, targeted scooping and pouring, and placing targeted objects into a pot.
  • FiLM ablation: 33% language-following success after removing FiLM equals random instruction choice on both language-dependent tasks.The ablation evaluates the last two tasks, where language grounding is required for successful execution.

D. ALOHA Inference Efficiency Comparisons

The ALOHA efficiency comparison measures action throughput and latency under standardized inputs and chunk sizes, showing that OpenVLA-OFT+ greatly accelerates OpenVLA while retaining competitive throughput.

  • Measured efficiency: 77.9 Hz throughput is achieved by OpenVLA-OFT+ on ALOHA, compared with 1.8 Hz for original OpenVLA.Measurements average 100 queries on an NVIDIA A100 GPU; OpenVLA processes three images, robot state, and a task command.
  • Measurement conditions: All methods use action chunk size K=25 except Diffusion Policy with K=24 and original OpenVLA with K=1.RDT-1B predicts 64 actions but executes only the first 25 for comparison.
  • Measured efficiency: OpenVLA-OFT+ reaches throughput near RDT-1B despite being 7× larger by generating actions in one forward pass.RDT-1B requires multiple denoising steps, whereas OpenVLA-OFT+ uses single-pass action generation.
  • Interpretation: OFT adapts OpenVLA to novel robots and tasks through parallel decoding, action chunking, continuous actions, L1 regression, and optional FiLM conditioning.The study reports effects on inference efficiency, task performance, input-output flexibility, and language following ability.
  • Limitations: OFT’s effectiveness for multimodal demonstrations remains unresolved because L1 regression may miss alternative valid action sequences.Diffusion may capture multimodality better but can overfit suboptimal modes in training data.

APPENDIX

The appendix describes OpenVLA’s architecture and the modifications used for OpenVLA-OFT, emphasizing parallel decoding and continuous action prediction instead of autoregressive discrete-token generation.

  • Base OpenVLA architecture: OpenVLA combines SigLIP and DINOv2 vision transformers, a Llama-2 7B language model, and a 3-layer MLP projector.The projector maps fused visual features into the language embedding space.
  • Base OpenVLA architecture: Original OpenVLA processes one third-person image and an instruction, then outputs a 7-dimensional robot action through the Llama-2 decoder.Visual features and language embeddings are concatenated along the sequence dimension before decoding.
  • OpenVLA-OFT modifications: OpenVLA-OFT adds multiple images, projected proprioceptive state, bidirectional attention, a continuous-action MLP head, and action chunks.These changes replace causal autoregressive generation with parallel action prediction.
  • Autoregressive decoding: Autoregressive training uses shifted ground-truth action tokens and causal attention, while test-time predictions are fed back token by token.This teacher-forcing setup limits each prediction to current and previous action-token context.
  • Parallel decoding: Parallel decoding uses empty action embeddings with distinct positional encodings and bidirectional attention to predict an action chunk non-causally.The model can use all intermediate features when predicting each chunk element.
  • Action representation: Continuous actions avoid the lossy discretization inherent in discrete action representations.Increasing the number of discrete bins improves precision but reduces token frequency and may hurt generalization.
  • Continuous-action objectives: The L1-regression action head is a 4-layer ReLU MLP mapping final Llama-2 hidden states directly to continuous actions.The appendix also specifies a 50-step DDIM sampler, squared-cosine beta schedule, and a 4-layer noise predictor for diffusion variants.

3) Input Processing Details:

The implementation processes visual, language, and robot-state inputs through projected embeddings, with FiLM optionally modulating visual features using language information. Training details differ across LIBERO, ALOHA, and baseline policies.

  • Input processing: OpenVLA converts each input image into 256 patch embeddings and projects both visual patches and low-dimensional robot states into the language-model embedding space.Visual patches use a 3-layer MLP with GELU activation, while robot states use a 2-layer MLP with GELU activation.
  • FiLM: The FiLM implementation multiplies features by (1 + γ) rather than γ so near-zero initialization preserves the pretrained visual activations.This design minimizes perturbation to the pretrained representation at the start of fine-tuning.
  • FiLM: Separate affine projectors generate γ and β for each transformer block, enabling block-specific modulation patterns.The modulation is spatially agnostic and applies across feature dimensions rather than independently to each patch embedding.
  • Training settings: LIBERO training uses normalized-action L1 convergence below 0.01, a learning-rate decay after 100K steps, and no FiLM.The best checkpoint is generally selected by evaluation every 50K steps, with the 150K checkpoint best except for LIBERO-Goal.
  • FiLM: FiLM modulates visual features in both SigLIP and DINOv2 transformers using language-conditioned scale and shift operations at each transformer block.The average task-description embedding generates the modulation used in OpenVLA’s fused vision backbone.
  • Training settings: ALOHA OpenVLA-OFT+ training retains the normalized L1 threshold and uses a 10× learning-rate reduction after 50K steps, while ACT, Diffusion Policy, RDT-1B, and π0 follow their specified training configurations.The ALOHA recipe includes FiLM, whereas the listed baselines use their respective implementation or default settings.

F. ALOHA Evaluation Details

The ALOHA evaluation details specify three bimanual manipulation tasks, their demonstration datasets, episode lengths, initial-state references, and evaluation setup. The tasks cover folding clothing, placing items into a pot, and scooping ingredients into a bowl.

  • Evaluation overview: The ALOHA experiments define detailed specifications for clothing-folding, pot-placement, and ingredient-scooping tasks.The specifications include datasets, episode lengths, initial states, and evaluation conditions.
  • Clothing folding: Bimanual folding tasks use 20 demonstrations for white shorts and 30 for long-horizon T-shirt folding, with 1000- and 1250-timestep episodes, respectively.The shorts task has 19 training and 1 validation demonstration; the T-shirt task has 29 training and 1 validation demonstration.
  • Clothing folding: The white-shorts task performs two synchronized folds, whereas the T-shirt task involves multiple synchronized bimanual folds over a longer horizon.Initial states are referenced in Figure 9 for shorts and Figure 10 for T-shirt folding.
  • Ingredient scooping: The ingredient-scooping task moves a bowl to the center and scoops raisins, almonds and green M&Ms, or pretzels into it using 45 demonstrations and 900-timestep episodes.The dataset has 42 training and 3 validation demonstrations, with initial states referenced in Figure 11.
  • Pot placement: The pot-placement task opens a pot, places a specified green pepper, red pepper, or yellow corn inside, and closes the pot.Its dataset contains 300 demonstrations, with 285 for training and 15 for validation, and episodes last 400 timesteps.
  • Pot placement: Pot-placement evaluation uses 24 trials split evenly between 12 in-distribution and 12 out-of-distribution evaluations.Food-item initial states vary across 45 cm horizontally and 20 cm vertically while the pot pose remains fixed; initial-state layouts are shown in Figures 12 and 13.

2) ALOHA Task Scoring Rubric:

The ALOHA tasks use cumulative staged scoring, while additional experiments test scaling across LIBERO suites, FiLM’s contribution, and the value of OpenVLA’s pretrained representation.

  • 2) ALOHA Task Scoring Rubric:: ALOHA success scores are cumulative across task stages, so later-stage points require successful completion of preceding stages.The four tasks use distinct stage allocations and penalties for visible failure modes.
  • 1) Single OpenVLA-OFT Policy for All LIBERO Task Suites Combined:: A single OpenVLA-OFT policy trained on all four LIBERO suites achieves comparable average task performance to suite-specific policies.This result supports scaling the method to larger fine-tuning datasets.
  • 2) Ablating FiLM in LIBERO:: The put X into pot dataset contains 300 demonstrations because earlier language-grounding difficulties prompted an investigation of whether more data would help, but more data alone was insufficient.Additional techniques were needed for more reliable language grounding, although the full dataset was retained for fine-tuning.
  • 2) Ablating FiLM in LIBERO:: FiLM slightly improves average LIBERO success, but its effect is minor compared with its reported impact in real-world ALOHA experiments.The ablation compares OpenVLA-OFT without FiLM against OpenVLA-OFT+ with FiLM.
  • 3) Ablating the OpenVLA Pretrained Representation:: Fine-tuning the underlying Prismatic VLM from scratch with OFT generally performs worse than full OpenVLA-OFT, indicating value from OpenVLA’s pretrained representation.The comparison uses the same LIBERO policy inputs while removing OpenVLA’s Open X-Embodiment robot pretraining.

4) Scaling Up OpenVLA-OFT to a Larger Real-World Dataset (BridgeData V2):

The paper extends OpenVLA-OFT beyond LIBERO and ALOHA to BridgeData V2, while documenting task scoring, model configurations, and the role of pretrained representations and action-recipe changes.

  • BridgeData V2 evaluation: OpenVLA-OFT is evaluated on BridgeData V2 WidowX tasks spanning visual, motion, physical, semantic, and language-grounding generalization.Performance is compared with the public OpenVLA checkpoint using the original OpenVLA scoring criteria.
  • BridgeData V2 evaluation: OpenVLA-OFT surpasses OpenVLA on average performance on the evaluated BridgeData V2 subset.The comparison uses the same evaluation criteria for both methods.
  • BridgeData V2 evaluation: BridgeData V2 still requires fine-tuning under OFT because the architecture, learning algorithm, action representation, and decoding scheme differ substantially from pretraining.The initial normalized action-regression L1 loss is roughly 0.5 at the beginning of training.
  • Model configurations: OpenVLA-OFT’s LIBERO configuration combines parallel decoding, action chunking, continuous actions with L1 regression, wrist-camera input, and robot state.The ALOHA OpenVLA-OFT+ configuration additionally uses two wrist-camera images and FiLM.
  • Baseline configurations: Baseline ALOHA policies use distinct training setups: modified ACT settings, DROID-based Diffusion Policy, default RDT-1B parameters, and default π0 fine-tuning.ACT is trained from scratch, while RDT-1B and π0 are fine-tuned.
  • ALOHA evaluation design: The ALOHA evaluations use varied initial states for clothing, bowl-and-spoon, and pot tasks, including unseen distractor bowls in the out-of-distribution pot condition.The figures document the spatial and rotational perturbations used across trials.
  • ALOHA scoring: The four ALOHA task rubrics report average cumulative staged scores, with task-specific penalties for shirt-folding and food-placement errors.Tables X–XIII provide detailed results for fold shorts, fold shirt, scoop X into bowl, and put X into pot.
  • Pretrained representation: Removing OpenVLA’s pretrained representation generally reduces LIBERO performance relative to full OpenVLA-OFT.This ablation evaluates fine-tuning the underlying Prismatic VLM directly with the OFT recipe.
Loading 2502.19645v2…