Source-linked AI summary
Frequency-Conditioned Flow Matching for Vision-Language-Action Models
Haochen Niu, Shengye Dong, Hao Liu, Peiwen Lin, Wang Chuang
TL;DR
Flow Matching VLA models typically generate temporally correlated robot actions without explicitly exploiting their highly uneven frequency structure. FreqFM makes frequency explicit throughout generation using DCT coordinates, and improves all three simulation benchmarks, including a 9.3-point LIBERO-Plus gain, without modifying the VLA backbone.
Problem
Flow Matching VLA models typically operate in temporal coordinates without explicitly modeling or systematically exploiting the heterogeneous frequency structure of temporally correlated robot actions.
Method
FreqFM uses DCT frequency coordinates to construct a spectrum-matched source, adaptively weight training across frequencies, and constrain per-frequency guidance by reference transport scales.
Results
FreqFM improves all three simulation benchmarks over paired temporal Flow Matching baselines, with up to a 9.3-point gain on LIBERO-Plus.
Takeaways & Limitations
Frequency can serve as an explicit conditioning dimension across the Flow Matching pipeline while preserving compatibility with existing VLA action experts.
Takeaways & Limitations
FreqFM relies on fixed training-set spectral statistics, which incurs an Extrapolation regression on VLA-Arena when task action statistics poorly match the training spectrum.
Abstract
from arXiv · showhide
Robot actions are temporally correlated trajectories whose frequency components encode motion at different scales with highly non-uniform energy distributions. Yet Flow Matching--based vision-language-action (VLA) models typically generate actions in temporal coordinates, without explicitly modeling or systematically leveraging this frequency heterogeneity. We introduce \emph{FreqFM}, a frequency-conditioned Flow Matching framework for VLA models. It raises action frequency from an implicit trajectory property to an explicit conditioning dimension that spans the entire generation pipeline. Concretely, in DCT frequency coordinates, FreqFM constructs a spectrum-matched source distribution, adaptively balances the objective across frequencies, and constrains per-frequency guidance residuals using the corresponding reference transport scales. FreqFM integrates into existing Flow Matching action experts without changing the VLA backbone. Across LIBERO, LIBERO-Plus, and VLA-Arena, FreqFM consistently improves performance, including a 9.3-point gain on LIBERO-Plus, and further demonstrates its effectiveness on six real-robot tasks.
Introduction
FreqFM makes action frequency an explicit conditioning dimension for Flow Matching VLA models, using DCT coordinates to adapt source statistics, training weights, and inference guidance. It integrates without changing the VLA backbone and improves the evaluated simulation benchmarks over paired temporal baselines.
- Introduction: Robot action energy is highly non-uniform across frequencies, with real-robot data spanning more than seven orders of magnitude and over 99.9% concentrated in the lowest three bins.
- Introduction: Frequency coordinates provide compact per-frequency parameterization for structure that would require dense cross-timestep coupling in temporal coordinates.
- Introduction: FreqFM applies the DCT along the action horizon to expose interpretable frequency components and conditions all three Flow Matching stages on them.The stages are source distribution construction, training-objective weighting, and inference guidance.
- Introduction: FreqFM estimates per-frequency power spectra to match source variance, normalize power in the objective, and constrain guidance residuals using reference transport scales.
- Introduction: FreqFM can be inserted into existing Flow Matching action experts without modifying the VLA backbone.
- Introduction: FreqFM improves all three simulation benchmarks over paired temporal Flow Matching baselines, with gains up to 9.3 points on LIBERO-Plus.The evaluation covers LIBERO, LIBERO-Plus, and VLA-Arena.
Related Work
Prior work uses frequency representations for tokenization, consistency, smoothness, or prescribed traversal in robot action generation. FreqFM instead treats frequency as a conditioning dimension across the Flow Matching pipeline and scales frequency-wise guidance by reference transport scales.
- Related Work: FAST and Frequency Autoregressive FreqPolicy use DCT-related representations to tokenize or progressively generate hierarchical frequency components.
- Related Work: Other methods use frequency for consistency, smoothness, derivative supervision, or frequency-band guidance rather than as a general conditioning dimension across Flow Matching.
- Related Work: FreqFM differs from prior frequency-based methods by conditioning the entire Flow Matching pipeline and budgeting frequency-wise CFG residuals by reference transport scale.
Frequency-Conditioned Flow Matching
FreqFM makes action frequency an explicit conditioning dimension for Flow Matching by operating in DCT coordinates and using frequency statistics across source construction, training, and guidance. Its spectrum-matched source preserves relative spectral structure during transport, while normalization and frequency-specific guidance budgets address non-uniform action scales.
- Framework overview: FreqFM conditions source construction, objective optimization, and inference guidance on DCT-domain frequency statistics without modifying the VLA backbone or action expert.The DCT exposes interpretable motion-scale coordinates, and the three-stage design applies frequency conditioning throughout the Flow Matching pipeline.
- Spectrum-Matched Source: Spectrum matching preserves the relative second-moment spectrum along the reference path, avoiding the frequency-dependent rescaling induced by a white source.The source uses per-frequency scales estimated from training actions, while the matched quantity is the second raw moment P_k,d rather than coefficient variance.
- PSD-Normalized Adaptive Objective: PSD normalization removes much of the power-induced loss scale before learned multi-task weighting, limiting early domination by high-energy components during finite-step optimization.The method uses a tempered normalization strength and likelihood-based adaptive scales; normalization changes practical joint optimization rather than the fully optimized stationary weighting alone.
- Spectral Transport-Budgeted Classifier-Free Guidance: FreqFM measures classifier-free guidance residuals in frequency-specific reference-transport units and clips them to per-frequency budget balls, rather than applying one raw-scale coefficient everywhere.The budget is uniform in normalized units, so admissible raw residuals scale with each frequency’s reference transport scale.
- Spectrum-Matched Source: FreqFM estimates per-frequency action power and uses it to shape source variance, reducing source–target covariance mismatch and defining reference transport scales.For independently sampled source and target, the spectrum-matched source and target share per-frequency second moments, while the linear-path velocity has a frequency-specific second moment.
Experiments
FreqFM is evaluated against matched temporal Flow Matching baselines and prior methods across simulation, real-robot, and ablation settings. It improves benchmark performance broadly, with the largest gains on LIBERO-Plus and precision-sensitive real-robot tasks, while exposing a distribution-shift limitation on VLA-Arena extrapolation.
- Simulation Results: +9.3 points is the largest reported LIBERO-Plus gain, while LIBERO gains are +0.7 and +1.2 points.On LIBERO-Plus, the largest gains occur on Camera, Robot, and Noise, where the temporal baseline is weakest.
- Simulation Results: +2.6 and +4.2 points improve VLA-Arena totals, although Extrapolation regresses on both backbones.The paper attributes this limitation to a fixed training-action spectral prior that can mismatch test-task action statistics.
- Real-Robot Evaluation: +23.3, +13.4, and +13.3 points occur on precision-sensitive real-robot tasks requiring fine positional alignment.The gains are reported for Cup Stacking, Folder Filing, and Stamp & Handover, compared with +6.7 and +3.3 on tasks with looser placement tolerances.
- Ablations and Analysis: The temporal component control improves by +0.3 and +2.5 points, whereas complete FreqFM improves by +1.2 and +9.3 points.Removing the matched source costs 4.2 points on LIBERO-Plus; transport budgeting reduces mean absolute jerk by 56%, from 0.442 to 0.193, at comparable LIBERO success.
Conclusion
FreqFM makes action frequency explicit throughout Flow Matching, improving performance without modifying the VLA backbone. Its fixed training-spectrum conditioning creates an extrapolation limitation that motivates observation- and instruction-conditioned statistics.
- Up to 9.3 points of improvement over paired temporal Flow Matching baselines on LIBERO, LIBERO-Plus, and VLA-Arena.
- FreqFM shapes the source distribution, adapts training, and budgets inference-time guidance using frequency-dependent statistics.
- Fixed training-set spectra can mismatch conditions whose action statistics differ, with the cost visible on VLA-Arena extrapolation tasks.
- Future work could condition frequency statistics on current observations and instructions or test transfer to other architectures.