SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models

Changti Wu; Bin Yu; Zhaolong Shen; Shijie Lian; Xiaopeng Lin; Cong Huang; Zhirui Zhang; Lei Zhang; Kai Chen

Source-linked AI summary

SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Models

Changti Wu, Bin Yu, Zhaolong Shen, Shijie Lian, Xiaopeng Lin, Cong Huang, Zhirui Zhang, Lei Zhang, Kai Chen

arXiv:2607.06442v1Jul 7, 2026cs.RO

TL;DR

Large VLA demonstration datasets contain redundancy and uneven coverage, while existing selection approaches miss reusable long-horizon structure. SIEVE selects demonstrations by primitive compositions and transitions, outperforming baselines and full-data training with 50% of demonstrations and training steps.

Problem
Large VLA demonstration datasets contain redundant, noisy, suboptimal, and unevenly distributed demonstrations, limiting the reliability of unfiltered imitation-learning data.
Method
SIEVE discovers reusable primitives, allocates budgets across composition patterns using reuse-aware structural exposure, and selects medoid trajectories within each pattern.
Results
Across datasets, benchmarks, and VLA models, SIEVE consistently outperforms competitive baselines and can outperform full-data training using 50% of demonstrations and 50% of training steps.
Takeaways & Limitations
The results highlight reusable structure as a basis for more efficient VLA imitation learning through structurally informed demonstration selection.

Abstract

from arXiv · show

Vision-Language-Action (VLA) models are typically trained by imitation learning on large-scale robot demonstration datasets, but more data does not necessarily yield better policies due to redundancy, noise, and uneven coverage. Existing data selection methods often assess demonstrations at either the trajectory or state-action level, missing the reusable structures that compose long-horizon behaviors. In this paper, we propose SIEVE, a structure-aware data selection method for VLA imitation learning. SIEVE views demonstrations as compositions of reusable primitives and transition interfaces. It first discovers visuo-motor primitives from segmented trajectories, then allocates selection budgets to composition patterns by maximizing reuse-aware structural exposure under diminishing returns. Finally, it selects medoid trajectories within each composition-pattern bucket to retain central, stable, and imitation-friendly demonstrations. Experiments across multiple datasets, benchmarks, and VLA models show that SIEVE consistently outperforms competitive data selection baselines. Notably, SIEVE can surpass full-data training while using only 50% of demonstrations and 50% of training steps, suggesting that reusable structure, captured through primitives and transitions, is an important signal for efficient VLA imitation learning.

Introduction

SIEVE addresses inefficient VLA imitation learning by selecting demonstrations according to reusable primitive-transition structures rather than only global trajectory utility. It combines structure-aware budget allocation with medoid selection to retain stable, predictable demonstrations, outperforming full-data training with 50% of demonstrations and 50% of training steps.

Motivation: Large VLA demonstration datasets contain redundancy and noise, so retaining a compact subset is important for policy learning.The introduction identifies data selection as an increasingly important problem because dataset growth does not automatically improve policies.
Limitations of Existing Selection: Trajectory-level utility signals can provide a global view but may collapse long-horizon behaviors and miss reusable internal structures.Existing methods estimate utility at trajectory or finer granularities, motivating a structure-aware alternative.
SIEVE Framework: SIEVE selects medoid trajectories within each composition-pattern bucket to favor central, stable, and predictable behavior-cloning supervision.This addresses inconsistent action targets that can arise from atypical or noisy realizations of the same behavior pattern.
SIEVE Framework: SIEVE represents trajectories as sequences of reusable primitives and adjacent transitions, then allocates budgets across composition patterns using reuse-aware structural exposure under diminishing returns.Primitive Discovery segments trajectories at physically grounded interaction boundaries and clusters segment representations before budget allocation.
Results: 50% of demonstrations and 50% of training steps suffice for SIEVE to outperform full-data training while consistently improving across datasets, benchmarks, and models.The reported result supports reusable structure as an important signal for efficient VLA imitation learning.

Related Work

Related work situates SIEVE within scalable VLA control and existing data-selection approaches that evaluate demonstrations at dataset, trajectory, or state-action levels. SIEVE instead organizes selection around primitive composition coverage and representative trajectories within composition-pattern buckets.

Vision-Language-Action Models: VLA models provide a scalable paradigm for language-conditioned robotic control by mapping visual observations and language instructions to robot actions.The passage identifies autoregressive action tokenization, diffusion, and flow-matching as examples of generative policy architectures.
Data Selection: Existing related studies evaluate data at the macroscopic dataset level, often combining optimization across data sources with VLM data selection.The passage distinguishes dataset-level evaluation from imitation-learning data selection.
Data Selection: Imitation-learning data-selection methods typically estimate sample utility at either the trajectory level or the state-action level.Trajectory-level methods use global signals such as representation similarity and reliability.
SIEVE: SIEVE discovers primitives, allocates selection budgets across composition patterns using structural exposure, and selects representative trajectories within each bucket.Its three stages are primitive discovery, structural exposure allocation, and learning-friendly trajectory selection.

SIEVE

SIEVE selects VLA imitation-learning demonstrations by modeling trajectories as compositions of reusable visuo-motor primitives and transition interfaces. It allocates budgets across composition patterns using reuse-aware structural exposure, then retains representative trajectories within each pattern.

Primitive discovery: SIEVE discovers reusable visuo-motor primitives from trajectory segments and represents each demonstration as a composition of those primitives.Primitives serve as proxies for reusable behavioral subprograms.
Primitive discovery: Trajectories are segmented at physically grounded grasp/release state flips, accepting transitions only when they persist for five consecutive frames.The persistence condition suppresses spurious boundaries from transient actuation jitter.
Composition patterns: Each trajectory becomes an ordered composition pattern containing primitives and transition interfaces between adjacent primitives.Transitions often coincide with critical state changes such as gripper grasp/release and provide cues for local behavior progression.
Budget allocation: SIEVE allocates the selection budget across composition-pattern buckets by maximizing structural exposure to primitives and transitions with reuse-based weights and diminishing returns.The objective is optimized greedily by repeatedly selecting the pattern with the largest marginal gain.
Trajectory selection: Within each pattern bucket, SIEVE selects representative, learning-friendly trajectories using representation-space centrality as a proxy for consistent state-action supervision.Central trajectories are less likely to contain ambiguous or inconsistent mappings for behavior cloning.

Experiments

Across three datasets and two VLA models, SIEVE consistently selects subsets that outperform full-data and competing baselines, including with half the data and training steps. Ablations show that transition and primitive exposure, along with learning-friendly within-bucket selection, are central to its performance.

Experimental setup: Experiments cover Bridge-V2, Fractal, and GR00T-X-Sim datasets, evaluated with Qwen3-VL-4B-GR00T and Qwen3-VL-4B-OFT across varied embodiments, environments, and benchmarks.Bridge-V2 contains approximately 53K WidowX demonstrations; GR00T-X-Sim contains 24K simulated humanoid demonstrations.
Main results: 56.3% average success rate: SIEVE surpasses Full-Training’s 51.8% using 50% of demonstrations and 25K training steps instead of the complete dataset and twice the training steps.Under the 50% budget, SIEVE reaches 56.3% with proportional training and 59.4% with 50K steps; under 70%, it reaches 62.3% with 35K steps and 62.5% with 50K steps.
Performance across datasets: 56.3% and 76.4% average success rates: SIEVE beats Full-Training’s 51.8% and 75.0% on Bridge-V2 and Fractal while using half the data and steps.On GR00T-X-Sim, Random selection is reported in the supplied passage, while the detailed comparison continues beyond the excerpt.
Performance across models: 56.3% and 56.5% average success rates: SIEVE outperforms Full-Training and Random on Qwen3-VL-4B-GR00T and Qwen3-VL-4B-OFT, respectively.For Qwen3-VL-4B-GR00T, Random and Full-Training score 39.6% and 51.8%; for Qwen3-VL-4B-OFT, they score 26.3% and 38.8%.
Ablations and analysis: Removing transition or primitive exposure lowers average success from 56.3% to 50.8% or 51.6%, while Most-Dissim within-bucket selection lowers it to 40.1%.The larger transition-exposure drop indicates transition interfaces provide important supervision; the Most-Dissim result indicates atypical trajectories are less suitable for imitation learning.

Conclusion

SIEVE is a structure-aware selection method for VLA imitation learning that exposes reusable primitive compositions and transition interfaces while favoring central, stable demonstrations. Across datasets, benchmarks, and VLA models, it improves over competitive baselines and can outperform full-data training with fewer demonstrations and training steps.

Method: SIEVE selects demonstrations by exposing reusable primitive compositions and transition interfaces for behavior cloning.It favors central and stable realizations within each composition pattern.
Results: Across multiple datasets, benchmarks, and VLA models, SIEVE consistently improves over competitive data-selection baselines.
Results: SIEVE can outperform full-data training while using fewer demonstrations and training steps.

Appendix · Training Hyperparameters

The appendix reports shared training hyperparameters for fair comparisons across methods and specifies the hardware used for all experiments.

Appendix: All experiments use the same training hyperparameters.These hyperparameters are summarized in Table 5.
Appendix: The shared hyperparameters ensure a fair comparison across different methods.
Training Hyperparameters: Table 5 summarizes the training hyperparameters used in all experiments.
Training Hyperparameters: All models are trained on 8 NVIDIA H100 GPUs.
Training Hyperparameters: Each NVIDIA H100 GPU provides 80GB of memory.
Training Hyperparameters: The hardware configuration applies to all models in the experiments.The reported setup uses 8 NVIDIA H100 (80GB) GPUs.

Implementation Details of Primitive Discovery via Clustering.

SIEVE determines the primitive-cluster count by evaluating 20 candidate K values on a randomly sampled 10% of each dataset and selecting the maximizer of (1 −J ) log R. This criterion peaks clearly across Bridge-V2, Fractal, and GR00T-X-Sim, providing a stable and practical heuristic.

Primitive-cluster selection: 10% of each dataset is randomly sampled, and 20 candidate K values are uniformly evaluated within a reasonable search range.Primitive clustering and the score (1 −J ) log R are computed for each candidate.
Primitive-cluster selection: The K maximizing (1 −J ) log R is selected for all subsequent experiments.The score is the clustering criterion defined for primitive discovery via clustering.
Primitive-cluster selection: The clustering score exhibits a clear peak on Bridge-V2, Fractal, and GR00T-X-Sim, supporting a stable and practical selection heuristic.Figure 4 reports the search results across all three datasets.

Details on Performance Across Different Datasets

SIEVE achieves the highest average success rate on both Fractal and GR00T-X-Sim, while remaining competitive across individual evaluation categories. Detailed tables report task-level success rates under reduced-data, reduced-step training for selection methods.

Cross-dataset performance: SIEVE achieves the highest average success rate on both Fractal and GR00T-X-Sim.The detailed results correspond to the main-paper results in Table 2.
Cross-dataset performance: SIEVE maintains competitive performance across individual evaluation categories.This pattern is reported consistently with the main results on both datasets.
Fractal: On Fractal, Table 6 reports success rates for each evaluation task.Full-Training uses the complete dataset with 100K training steps; selection methods use 50% of the training data and 50% of the training steps.
GR00T-X-Sim: On GR00T-X-Sim, Table 7 reports average success rates for each RoboCasa-GR1 task category.Full-Training uses the complete dataset with 100K training steps; selection methods use 50% of the training data and 50% of the training steps.