ICI-VLA: In-Context Imitation with Spatiotemporally Aligned Demonstrations for Vision-Language-Action Models

• Task-by-task adaptation typically requires additional demonstrations and parameter updates, while in-context learning offers conditioning on demonstrations without updating parameters.Repeated adaptation for new tasks is described as costly and limiting rapid adaptation in changing environments.Source: §1 · p. 1
• Robotic demonstrations couple visual observations, language, proprioception, and continuous actions, so poorly matched context can interfere with action prediction.Source: §1 · p. 1
• Prior robotic retrieval approaches leave robust general-purpose adaptation unresolved, including mismatches between long trajectories and short action chunks or between similar-looking tasks requiring different motions.VLA-0 demonstrates continuous action generation through a pure VLM's native text interface.Source: §1 · pp. 1–2
• ICI-VLA replaces task-specific parameter updates with fixed-policy inference conditioned on retrieved micro-demonstrations, using semantic filtering, DTW ranking, and Target Action Masking.The framework decomposes long trajectories into semantically labeled examples to align context with short action chunks.Source: Figure 1 · p. 1, §1 · p. 2
• 97.7% on LIBERO and 60.4% on RoboTwin 2.0, with the latter 19.3 percentage points above the strongest reported baseline average; physical tasks average 83.2%.Source: §1 · p. 2

2.1 Vision-Language-Action Models

• Generative-action, discrete-token, and customized architectures use different action interfaces, including continuous decoders, quantized action prediction, and specialized tokenizers or spatial modules.Source: §2.1 · p. 2
• VLA-0 uses the native text-generation interface without architectural modification, which this work adopts as a foundation for in-context action adaptation.Other paradigms may add engineering overhead or induce shortcut learning that could degrade native capabilities.Source: §2.1 · p. 2

2.2 In-Context Learning

• Robotic ICL must jointly represent perception, robot state, language, and continuous control; prior retrieval-conditioned VLAs do not jointly study decomposition, motion-aware supervision, and target-prefix corruption.ICI-VLA combines these elements for phase-aware retrieval and short-horizon action prediction.Source: §2.2 · p. 2

3 Methodology

• ICI-VLA separates offline library construction, retriever training, and VLA fine-tuning from inference, where both learned models remain fixed and retrieved context conditions action prediction.The framework has three stages: demonstration-library construction, spatiotemporal retriever training, and policy fine-tuning with Target Action Masking.Source: §3 · p. 2, §3.3 · p. 3, Figure 2 · p. 4

3.1 Preliminaries

• The text-action policy preserves the foundational VLM without action-specific parameters or heads, representing continuous actions as numerical text strings.The design aims to mitigate interference from external modules and retain the foundation model’s reasoning and generalization abilities.Source: §3.1 · p. 3

3.2 Problem Formulation

• At inference, the query combines global and active subtask instructions, camera observations, and proprioception; learned retriever and policy parameters receive no gradient updates.Training, library-construction, validation, and evaluation trajectories are non-overlapping, though benchmark task semantics, objects, layouts, or instruction templates may recur across partitions.Source: §3.2 · p. 3

3.3 The ICI-VLA Framework

• The library segments long trajectories into instructed subtasks and represents each example with instructions, camera observations, initial proprioception, and a textualized action chunk.The library contains approximately 139,659 subtask examples constructed from approximately 11,200 long-horizon trajectories.Source: Robot Demonstration Library Construction · p. 3
• The RD-Encoder combines semantic candidate filtering with offline DTW trajectory ranking to mine positives and phase-misaligned negatives, then retrieves from observable inputs at evaluation.Waypoints encode normalized 3D end-effector positions of active arms; the DTW-minimal candidate is positive, while same-subtask phase-misaligned candidates are negatives.Source: Spatiotemporal Retrieval Alignment · pp. 3–4
• Target Action Masking corrupts sampled target-action tokens during training and optimizes unmasked targets, reducing dependence on exact target-prefix continuation.At inference, masking is disabled and the policy generates actions autoregressively with fixed parameters; greater use of the current observation and retrieved context is hypothesized, not directly established.Source: VLA Fine-Tuning via Target Action… · pp. 4–5

4 Experimental Evaluation

• ICI-VLA is evaluated on LIBERO, RoboTwin 2.0, and a physical dual-arm Aloha setup.The evaluation also measures component ablations, context-size and retriever-mining-cycle sensitivity, and physical-system performance.Source: §4 · p. 5

4.1 Experimental Setup

• The evaluation uses approximately 11,200 long-horizon trajectories, including approximately 1,000 teleoperated physical Aloha trajectories.The physical trajectories cover grasping, placing, drawer manipulation, and sorting.Source: §4.1 · p. 5
• Success rate is the primary metric; simulated tasks are evaluated over 10 rollouts with varied initial states and random seeds.At evaluation, parameters remain fixed, the context is refreshed below a policy-confidence threshold of 0.65, and the three-shot setting and threshold are selected on validation data.Source: §4.1 · p. 5

4.2 Baselines

• External baseline entries follow their original protocols, so Table 1 provides benchmark-level rather than fully controlled comparisons.VLA-0 with naive ICL is the closest policy-backbone reference; REGENT, RICL, and MEM are discussed conceptually because their protocols are not directly matched.Source: §4.2 · p. 5

4.3 Main Results

• 97.7% average success on LIBERO exceeds OpenVLA-OFT’s highest reported baseline average of 96.4% in the benchmark-level comparison.ICI-VLA records the highest listed values on Spatial, Goal, and Long, and reaches 96.8% on LIBERO-Long versus the highest reported baseline value of 93.2%.Source: §4.3 · p. 5
• Figure 3 shows a long-horizon query rollout above references retrieved for different execution phases.Table 2 reports component ablations as average success rates; its caption identifies DTW and Semantic as retrieval components.Source: Figure 3 · p. 6, Table 2 · p. 6
• 60.4% average success on RoboTwin 2.0 is 19.3 points above the highest reported baseline; under a controlled comparison, VLA-0 with naive ICL reaches 10.7%.The controlled comparison uses the same policy backbone, trajectory partitions, optimization setting, and retrieved contexts.Source: §4.3 · p. 6

4.4 Ablation Studies

• Every ablated variant underperforms the complete configuration under the reported protocol.The comparisons quantify performance changes but do not establish explicit residual computation or a particular internal shortcut as the cause of every failure.Source: §4.4 · p. 6
• Without Target Action Masking, average success falls from 97.7% to 71.5% on LIBERO and from 60.4% to 10.7% on RoboTwin 2.0.The naive-ICL comparison uses the same policy backbone, trajectory partitions, optimization setting, and retrieved contexts as ICI-VLA.Source: §4.4 · p. 6
• Removing DTW ranking yields 92.5% on LIBERO and 31.4% on RoboTwin 2.0; removing semantic filtering yields 88.4% and 38.1%, respectively.Source: §4.4 · p. 6

4.5 Qualitative Analysis

• The visualization follows the compound instruction to put a black bowl in the bottom drawer and close it.Source: §4.5 · p. 6
• The RD-Encoder selects Reference 1 during drawer opening, then switches to Reference 2 during bowl placement.Source: §4.5 · p. 6
• The example illustrates locally relevant context changing as the long-horizon task progresses; retrieval accuracy is evaluated quantitatively in the next section.Source: §4.5 · p. 6

4.6 Direct Retriever Evaluation

• Recall@1/Recall@5 reaches 70.8/90.1 for the full five-cycle RD-Encoder, up from 27.8/52.4 for the base embedding.Figure 5 reports direct retriever evaluation on RoboTwin 2.0.Source: Figure 5 · p. 7, §4.6 · p. 7
• Recall@K is the percentage of queries whose offline DTW-defined positive appears among the top K results.The retriever ranks candidates from observable query inputs and never receives the query’s target action.Source: Figure 5 · p. 7, §4.6 · p. 7
• Recall@1/Recall@5 progresses from 41.6/67.9 after semantic filtering to 56.9/80.7 after one DTW-supervised mining cycle.The full five-cycle RD-Encoder reaches 70.8/90.1.Source: §4.6 · p. 7

4.7 Key Parameter Sensitivity

• 60.4% is the RoboTwin 2.0 validation peak at Ne = 3, with performance lower for both smaller and larger contexts.Figure 4 examines the number of retrieved demonstrations and retriever-mining cycles.Source: §4.7 · p. 7
• Performance rises from 34.6% without optimization to 60.4% after five mining cycles, with changes of at most 0.3 points through seven cycles.The reported configuration uses five cycles, the smallest tested setting attaining the maximum.Source: §4.7 · p. 7
• The reported three-example, five-cycle configuration was selected without tuning these parameters on the final test set.It is the smallest tested setting attaining the validation maximum.Source: §4.7 · p. 7

4.8 Real-World Evaluation

• 83.2% success across 1,000 trials (95% CI: 80.8–85.4) compares with 66.4% (63.4–69.3) for π0 and 63.0% (60.0–65.9) for VLA-0.The evaluation covered four physical tasks, with over 250 rollouts per task.Source: §4.8 · p. 7
• The physical evaluation covered Single-arm Grasp, Dual-arm Grasp, Drawer Placement, and Object Sorting under lighting variation, distractors, sensor noise, and contact dynamics.Evaluation objects, layouts, instructions, and trajectories were disjoint from policy training and the retrieval library.Source: §4.8 · p. 7
• The retrieval library contained approximately 1,000 other physical demonstrations, separate from the evaluation trajectories.The results are reported over 1,000 physical trials.Source: §4.8 · p. 7

5 Limitation and Conclusion

• ICI-VLA combines spatiotemporal alignment and Target Action Masking to use micro-demonstrations as context for robot control.All parameters remain fixed at inference.Source: §5 · p. 7
• ICI-VLA achieves 97.7% on LIBERO, 60.4% on RoboTwin 2.0, and 83.2% across four physical tasks.The paper presents these results as evidence for the potential of text-centric policies to adapt through well-aligned contextual experience across simulation and physical settings.Source: §5 · p. 7
• ICI-VLA remains dependent on demonstration and planner coverage and incurs offline training and retrieval costs.Future work includes improving retrieval efficiency, expanding demonstration coverage, and investigating cross-embodiment adaptation.Source: §5 · p. 7