How (and How Not) to Use Data Augmentation in VLA Post-Training

1 Introduction

• VLA performance can degrade sharply under visual shifts such as new backgrounds, lighting, distractors, and camera poses.RL post-training has improved execution and semantic generalization, but does not consistently improve robustness to visual OOD shifts.Source: §1 · p. 1
• Prior augmentation-placement studies use small convolutional encoders and conventional actor heads, leaving transfer to pretrained-VLM and flow-matching VLAs unknown.In VLAs, pretrained visual features may be corrupted, and gradients must propagate through a flow-matching actor head.Source: §1 · p. 2
• The paper systematically studies augmentation placement, type, and strength, finding critic-only augmentation most effective and actor augmentation detrimental to VLA training dynamics.This reverses the conclusion for convolutional policies reported by Almuzairee et al. (2024).Source: §1 · p. 2

2 Prior Work on Data Augmentation for VLAs

• On-policy RL approaches for VLAs mainly improve in-distribution success, while Liu et al. (2025b) report little to no gain on visual OOD shifts.Other work reports execution and semantic-generalization improvements; preserving pretrained representations and adding regularizers or auxiliary losses are among the VLA robustness approaches described.Source: §2 · p. 2
• Visual-RL augmentation improves sample efficiency and generalization, with prior work studying value-path augmentation, policy-value invariance, distraction masking, and joint actor-critic augmentation.Hansen et al. (2021), Raileanu et al. (2021), Grooten et al. (2024), and Almuzairee et al. (2024) study these approaches, respectively.Source: §2 · p. 2
• Prior VLA robustness methods use auxiliary losses, regularization, world models, domain-randomized simulation, or augmented demonstrations; LIBERO-Plus success can fall from 95% to below 30% under visual changes.The reported robustness approaches include Grover et al. (2025), Zhang et al. (2025a), Xie et al. (2026), Peng et al. (2026), Gao et al. (2026), Shi et al. (2026), and Xue et al. (2025).Source: §2 · p. 2
• No prior work studies augmentation placement during VLA RL post-training; this study tests actor versus critic placement, augmentation type, and strength.The closest cited work studies auxiliary paired losses for VLAs or placement in small convolutional policies, rather than VLA RL post-training.Source: §2 · pp. 2–3

3 Experimental Framework & Augmentation Suite

• π0.5 and GR00T N1.5 start from LIBERO-Spatial SFT checkpoints and are post-trained with PPO for up to 300 epochs.The models pair a VLM with a flow-matching action head differently; a value head uses final VLM features, and the actor and critic share a backbone pass unless augmentation is applied to only one.Source: §3 · p. 3
• Flow-SDE makes each denoising step a Gaussian transition with a tractable density, allowing PPO to optimize the stochastic denoising chain as the policy.It adapts the ODE-to-SDE conversion of Flow-GRPO to flow-based VLAs while preserving model marginals at each timestep.Source: §3 · p. 3
• Augmentations are applied only during training forward passes; rollouts and evaluations use clean observations, so reported success rates use unaugmented inputs.Training augmentations can affect both base and wrist camera views.Source: §3 · p. 3
• Overlay blends observations with randomized ImageNet images, while shift pads each 224×224 view by 10 pixels and randomly crops it back.The overlay strength defaults to α = 0.5 and uses the same image across views with independent crops and flips; shift is applied independently per camera.Source: §3 · p. 4
• LIBERO-Spatial measures in-distribution success over 500 evaluation episodes, while LIBERO-Plus measures mean success across five visual OOD axes and 1,662 tasks.The OOD axes are background textures, lighting, object layout, camera viewpoints, and sensor noise.Source: §3 · p. 4
• Repeated evaluation of one checkpoint differed by 0.06 OOD-success points for π0.5 and 1.77 for GR00T N1.5, which the study treats as noise floors.Reported gaps are stated to be well above these evaluation-noise floors.Source: §3 · p. 4

4.1 Which modules should be augmented?

• On π0.5, critic-only overlay raises average OOD success from 73.9% to 81.7% (+7.8 points); augmenting the actor collapses training on both models.Critic-only overlay also raises π0.5 in-distribution success from 93.2% to 98.0%.Source: Table 1 · p. 5, §4.1 · p. 5

4.2 What type of augmentation works well?

• Overlay raises π0.5 average OOD success to 81.7% versus 78.9% for shift, while random shift reaches 76.0% on GR00T N1.5.The preferred augmentation type reverses across models; combining types is not additive and may over-perturb observations.Source: §4.2 · pp. 5–6, Table 2 · p. 6

4.3 How strong can we augment observations?

• For π0.5, overlay strength α = 0.5 gives the highest average OOD success, 81.7%, versus 77.0% at α = 0.10 and 76.6% at α = 0.90.All tested overlay strengths beat no augmentation, and the differences are small.Source: Table 3 · p. 6, §4.3 · p. 6

• With the VLM backbone frozen, critic-only augmentation cannot reshape actor-visible visual features and instead shapes actor updates through the advantage estimate.Source: §5 · p. 7
• The actor sees only clean pixels, while its update is shaped by a critic trained on augmented observations.Source: §5 · p. 7
• +7.8 OOD success points on π0.5 are attributed to the advantage-estimation pathway.Source: §5 · p. 7
• The authors call their numbers a lower bound and identify a trainable-backbone experiment as the next step.Unfreezing would let critic gradients reach visual features read by the actor.Source: §5 · p. 7

6 Conclusion

• +7.8 OOD success points on π0.5 and +10.0 points on GR00T N1.5 were achieved with each model’s best augmentation type, while in-distribution success did not decline.Source: §6 · pp. 7–8
• Augmenting actor observations collapses post-training on both VLA models, whereas critic-only augmentation remains stable.Source: §6 · p. 7
• Overlay helps most on sensor noise and won for π0.5, while shift helps most on viewpoints and layout and won for GR00T N1.5; combining augmentations was worse.Source: §6 · p. 7
• α ≈0.5 performed best, but every tested strength from 0.1 to 0.9 beat the unaugmented baseline.Source: §6 · p. 7
• The study covers simulation, one benchmark, and one RL algorithm, with one training seed per configuration; real hardware and off-policy settings remain untested.Source: §6 · p. 7
• Critic-only placement worked across two different VLA structures, suggesting the effect may follow their shared actor-critic structure rather than one architecture.Source: §6 · p. 8

A Hyperparameters and Experimental Setup

• The runs follow RLinf’s LIBERO recipes, with configurations held fixed except for augmentation placement, type, and strength.Where model settings differ, the difference was required by the model rather than chosen by the authors.Source: A Hyperparameters and Experimental Setup · p. 11, §A.1 · p. 11

A.1 Hyperparameters

• A single update epoch keeps gradient steps on-policy; both models freeze their vision-language backbone and train the action expert and value head.π0.5 uses a value-head learning rate twenty times the actor rate, while GR00T N1.5 uses one rate throughout.Source: §A.1 · p. 11
• The overlay blend α = 0.5 and equal clean/augmented critic weighting are inherited from SVEA, while the 10-pixel shift rescales DrQ’s 4-pixel pad.The authors sweep α because they had no principled reason to fix it; batch sizes are the largest that fit the hardware.Source: §A.1 · p. 11
• The appendix provides the post-training and augmentation settings, with shared augmentation hyperparameters and model-specific post-training settings tabulated.For π0.5 critic-only and actor-only runs, per-device batch size is halved to 64, with gradient accumulation preserving global batch size and optimization.Source: Table 5 · p. 11, Table 4 · p. 12, §A.1 · p. 12

A.2 Experimental Setup

• In-distribution evaluation covers 500 episodes across ten LIBERO-Spatial training tasks, using fixed initial states and the success_once criterion also used for OOD.An episode succeeds if the goal is reached at any point within the horizon.Source: §A.2 · p. 13
• Rollouts and evaluation always use clean observations, while training explores with stochastic flow_sde actions and evaluates with pure-ODE flow_ode integration.The noise prior remains random under ODE integration.Source: §A.2 · p. 13
• Tables 1–3 report epoch 300, the last saved checkpoint; the two collapsed configurations instead stopped early and are reported at epoch 100.The early-stopped results are marked in Table 1.Source: §A.2 · p. 13
• The reported OOD macro average equally averages success rates across five visual axes, rather than weighting axes by their task counts.The axes range from 258 to 385 tasks; the authors use macro averaging because axis sizes reflect benchmark assembly, not the importance of shifts.Source: §A.2 · p. 13

B Augmentations

• Augmentation follows preprocessing on normalized 224 × 224 images and is applied to the base and left-wrist slots, not the masked, zero-padded third slot.Both backbones order the inputs as base, left wrist, right wrist.Source: B Augmentations · p. 13
• The overlay bank samples ten images per ImageNet synset, with short-side resizing and center cropping; synsets with fewer than ten images contribute all available images.The bank is stored as a uint8 tensor; undecodable files are skipped.Source: B Augmentations · p. 13
• Each overlay image is randomly cropped, resized, and flipped before blending, while the observation itself is never cropped.A fresh crop rectangle is drawn each time, increasing variation beyond the stored bank size.Source: B Augmentations · p. 13
• Figure 4 shows five observations from one LIBERO-Spatial episode across overlay strengths; each column uses its own randomly transformed overlay, and the main experiments use α = 0.5.Rollouts and evaluations use clean views, while updates may receive augmented versions.Source: Figure 4 · p. 14
• Overlay strengths above α = 0.5 become difficult, although the critic may still infer the robot pose from slight pixel-value deviations.The strength sweep is shown in Figure 4.Source: B Augmentations · p. 14
• Critic-only augmentation keeps the actor on clean observations and fits the critic on clean and augmented views; uniform augmentation feeds one augmented view to both heads.Actor-only augmentation reverses the critic-only arrangement, and no augmentation provides the baseline; the two one-sided placements require two backbone passes.Source: B Augmentations · p. 14

C.1 Performance per OOD category.

• Critic-only augmentation pulls ahead only in the final quarter; the unaugmented baseline leads on both camera viewpoints and sensor noise through mid-training, by as much as 12 points at epoch 200.The plotted checkpoints cover the same 1,662 tasks used in the tables’ five-axis OOD score.Source: §C.1 · p. 14, C Additional results · p. 14
• The OOD gain comes from camera viewpoints and sensor noise, while background textures and lighting are already above 90% and barely separate between configurations.Figure 5 compares the unaugmented baseline with critic-only overlay augmentation at α = 0.5 by perturbation axis.Source: §C.1 · p. 14, Figure 5 · p. 15

C.2 Training analysis.

• Actor-augmented runs plateau at pretrained performance as higher policy loss, approximate KL, and clip fraction accompany action variability; critic-only runs rise with modestly higher value loss but no instability.The critic fits augmented and clean views, accounting for its slightly higher value loss.Source: §C.2 · p. 15, Figure 6 · p. 16

C.3 In-distribution vs OOD success.

• Every critic-only configuration maintains or improves both in-distribution and OOD performance relative to the unaugmented baseline.The final-checkpoint plot uses a dashed crosshair for the baseline; most critic-only configurations also improve OOD success.Source: §C.3 · p. 15, Figure 7 · p. 16