IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation

Shijie Lian; Bin Yu; Xiaopeng Lin; Zhaolong Shen; Laurence Tianruo Yang; Yurun Jin; Haishan Liu; Changti Wu; Hang Yuan; Cong Huang; Kai Chen

Source-linked AI summary

IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulation

Shijie Lian, Bin Yu, Xiaopeng Lin, Zhaolong Shen, Laurence Tianruo Yang, Yurun Jin, Haishan Liu, Changti Wu, Hang Yuan, Cong Huang, Kai Chen

arXiv:2605.14712v2May 14, 2026cs.ROcs.AIcs.CLcs.CV

TL;DR

Aliased visual-language observations can leave frame-conditioned VLA policies uncertain about the locally selected continuation, undermining chunk consistency. IntentVLA conditions chunk generation on recent visual history, improving success and inter-chunk consistency across the evaluated benchmarks and real-world task.

Problem
Under aliased conditioning, the current frame and instruction may not reveal the continuation already selected within an episode, limiting frame-conditioned chunk policies.
Method
IntentVLA learns a compact short-horizon intent representation from recent visual observations and uses it to condition chunk generation.
Results
IntentVLA improves success and inter-chunk consistency across AliasBench, standard simulation benchmarks, and a real-world cleanup task; on AliasBench it reaches 45.8%.
Takeaways & Limitations
The results support recent-history conditioning for short-horizon intent disambiguation in locally aliased manipulation.
Takeaways & Limitations
IntentVLA targets locally aliased manipulation and does not address sparse events outside the recent window or large closed-loop deviations.

Abstract

from arXiv · show

Robot imitation data are often multimodal: similar visual-language observations may be followed by different action chunks because human demonstrators act with different short-horizon intents, task phases, or recent context. Existing frame-conditioned VLA policies infer each chunk from the current observation and instruction alone, so under partial observability they may resample different intents across adjacent replanning steps, leading to inter-chunk conflict and unstable execution. We introduce IntentVLA, a history-conditioned VLA framework that encodes recent visual observations into a compact short-horizon intent representation and uses it to condition chunk generation. We further introduce AliasBench, a 12-task ambiguity-aware benchmark on RoboTwin2 with matched training data and evaluation environments that isolate short-horizon observation aliasing. Across AliasBench, SimplerEnv, LIBERO, and RoboCasa, IntentVLA improves rollout stability and outperforms strong VLA baselines

1 Introduction

IntentVLA addresses a failure of frame-conditioned chunk policies under partial observability, where visually similar observations can require different short-horizon continuations. The paper introduces AliasBench and a history-conditioned framework that preserves local intent using recent visual observations, with validation across four manipulation benchmarks.

Problem: Similar bread-holding observations can require skillet placement or plate return under the same instruction, creating short-horizon observation aliasing.The ambiguity concerns different next action chunks despite visually similar states.
Problem: Frame-only chunk policies can break within-episode local commitment by resampling among multimodal demonstration behaviors under partial observability.Human demonstrations may contain multiple valid goals and feasible paths, but each episode remains locally committed.
AliasBench: AliasBench is a 12-task RoboTwin2 benchmark with matched simulation training data and evaluation environments designed to isolate short-horizon observation aliasing.It evaluates whether policies preserve local continuations across back-and-forth, crossing-path, bimanual, and multi-goal ambiguity.
IntentVLA: IntentVLA is a history-conditioned imitation learning framework that learns a compact short-horizon intent representation from recent visual observations to condition chunk generation.Its core objective is to preserve an episode’s local commitment rather than infer every chunk from the current frame alone.
Validation: IntentVLA is validated across AliasBench, SimplerEnv, LIBERO, and RoboCasa, including ambiguous-intent tasks that test short-horizon intent consistency.The evaluation extends beyond the purpose-built benchmark to three additional manipulation suites.

2 Related Work

Prior VLA work transfers vision-language semantic priors to robot action generation, improves action-generation efficiency, and extends reasoning with prediction or memory mechanisms. Intent-driven VLAs introduce intermediate representations such as latent intent, grounded action intent, gaze, and disentangled intent–execution features to connect semantic reasoning with low-level control.

Vision-Language-Action Models: RT-2 and OpenVLA transfer vision-language model semantic priors to robot action generation, while FAST targets more efficient action generation.The passage also situates related work around additional efficiency-oriented extensions, though the supplied text truncates their details.
Vision-Language-Action Models: Other VLA approaches add world-model, future-prediction, or memory mechanisms for longer-horizon reasoning.These mechanisms are attributed to Shen et al. (2025), Sun et al. (2026), Shi et al. (2026), and Chen et al. (2026a).
Intent-Driven VLAs: Intent-driven VLAs bridge semantic reasoning and low-level control through latent intent bottlenecks, kinematically grounded action intents, gaze, or separated intent and execution representations.DIAL uses visual foresight with a differentiable latent intent bottleneck; ACoT-VLA grounds reasoning in action intents; GazeVLA uses gaze; and MINT separates low-frequency intent from high-frequency execution residuals.

3 AliasBench: Ambiguity-Aware Benchmark Design

AliasBench is a 12-task RoboTwin2 benchmark with matched training data and held-out evaluation environments designed to isolate short-horizon intent ambiguity from current-frame observations. Its diagnostic confirms substantial current-frame aliasing while recent history provides the missing context.

Benchmark scope: AliasBench contains 12 manipulation tasks with matched simulation training data and held-out evaluation environments, targeting consistent decision-making under aliased observations.It addresses a gap in standard VLA benchmarks, which rarely isolate whether policies maintain consistent decisions when similar observations require different next actions.
Task families: The 12 tasks span back-and-forth phase, crossing-path source, bimanual handoff direction, and multi-goal cues as latent factors resolving visual aliases.These families represent manipulation patterns where the current frame is visually plausible for multiple continuations.
Observation-aliasing diagnostic: AliasBench uses nearest-neighbor retrieval over visual embeddings inside ambiguity windows to test whether visually nearby states correspond to different next intents.The observation-aliasing diagnostic is a check of alias measurability rather than a policy success metric.
Observation-aliasing diagnostic: 49.7% is the average different-intent neighbor ratio, while the largest median same-vs.-different intent gap is below 3 × 10−3 in cosine distance.These results indicate that current frames provide weak evidence about short-horizon intent and that recent history contains missing context.

4 Method

IntentVLA addresses aliased short-horizon decision-making by conditioning action-chunk generation on a deterministic representation of recent visual history alongside the current image-language context. It implements this with a frozen VGGT-1B history encoder, gated fusion, a compact evidence token, and a DiT-based conditional flow-matching head.

Short-horizon intent formulation: IntentVLA uses a finite recent visual-history window as compact evidence about episode context instead of the complete interaction history.The history window helps disambiguate the current frame without explicitly modeling the full sequence of observations and actions.
Short-horizon intent formulation: Aliased current frames and instructions can leave the already-selected continuation uncommitted, causing multimodal action chunks to conflict across replanning steps.The method therefore conditions chunk generation on recent visual history rather than treating multimodality itself as the problem.
Short-horizon intent formulation: IntentVLA learns a deterministic representation mt = fϕ(ot, ℓ, hK_t) that embeds history-conditioned intent evidence without explicit intent labels.The representation is formed from the current observation, language instruction, and recent visual history, rather than explicitly inferring or supervising a latent intent variable.
History and current-context encoding: A Qwen3-VL 4B backbone encodes the current image and instruction, while a frozen VGGT-1B encoder processes recent visual history.For multi-camera observations, the history branch uses only head-camera frames and retains one camera token plus four register tokens per frame.
Fusion and chunk generation: Gated cross-attention fuses history tokens into the current context, appends a compact history-evidence token, and conditions a DiT-based flow-matching head for action-chunk generation.Training matches the conditional velocity field to the ground-truth displacement τt − ϵ for interpolated noisy action chunks.

5 Experiment

Experiments evaluate IntentVLA under partial observability across AliasBench, established simulation benchmarks, and a real-world snack-cleanup task. The results show improved rollout stability and success, while ablations identify recent-window history fusion and compact intent evidence as key components.

AliasBench: AliasBench directly tests short-horizon observation aliasing against frame-only, raw-history, and memory-centric baselines.The benchmark uses matched 30K-step, 16-H100 training budgets; direct 8-frame and 16-frame history variants run out of memory, while the best feasible variant reaches 28.1% average success.
AliasBench: 45.8%: IntentVLA succeeds on AliasBench despite visually similar states corresponding to different short-horizon intents.Similarity-based memory consolidation can average visually close but intent-distinct entries, reducing discriminability for episode-specific continuation selection.
Transfer benchmarks: 72.9%: IntentVLA achieves the best overall average success rate on SimplerEnv, exceeding Qwen3-VL-GR00T by 7.6 points and 3D-Mix by 4.7 points.The overall result supports improved robustness from recent visual history, although Put Spoon on Towel is lower than the baseline.
Transfer benchmarks: LIBERO’s Spatial, Object, and Goal suites are near saturation, so the evaluation focuses on LIBERO-Long as the most relevant longer, multi-stage setting.Several strong recent VLAs exceed 98% on the saturated suites, leaving limited room to diagnose short-horizon history effects.
Transfer benchmarks: 57.0%: IntentVLA achieves the best 24-task average success rate on RoboCasa-GR1’s diverse articulated-object tabletop benchmark.The benchmark covers 24 manipulation tasks, including cabinet, drawer, microwave, and toaster scenarios.
Ablations: History fusion raises SimplerEnv’s average from 65.3% to 69.5%, while removing VGGT and temporal history recovers the frame-only baseline.Adding VGGT to only the current frame does not improve the average; its benefit comes from geometric and inter-frame evidence in a recent window.

6 Conclusion

IntentVLA introduces history-conditioned chunk generation to stabilize robot manipulation under partial observability, alongside AliasBench, a 12-task benchmark for short-horizon observation aliasing. Across simulation benchmarks and a real-world cleanup task, compact recent-history conditioning improves success and inter-chunk consistency.

IntentVLA is a history-conditioned VLA designed to stabilize chunk generation under partial observability.
AliasBench is a 12-task benchmark targeting short-horizon observation aliasing.
Compact recent-history conditioning improves success and inter-chunk consistency across AliasBench, standard simulation benchmarks, and a real-world cleanup task.

Limitations

IntentVLA is limited to short-horizon intent recovery from recent visual history for locally aliased manipulation. It does not claim general episodic or long-horizon memory, so tasks requiring sparse-event recall beyond recent context are outside its intended scope.

Memory and scope: IntentVLA uses recent visual history only as an auxiliary signal to disambiguate intent during chunk generation, not as general episodic or long-horizon memory.Its intended application is locally aliased manipulation, where recent frames reveal the active continuation but the current frame alone is ambiguous.
Memory and scope: Tasks requiring memory of sparse events outside recent context are beyond IntentVLA’s intended application scope.

Ethical Considerations

The work evaluates short-horizon intent disambiguation in controlled simulation benchmarks and a small lab robot setup using non-personal robot demonstrations. Its main ethical risk is misinterpretation of benchmark performance as evidence of open-world safety or reliability.

Scope: The study focuses on controlled simulation benchmarks and a small lab robot setup.It studies short-horizon intent disambiguation for VLA policies.
Data: The demonstrations contain robot observations and actions from expert teleoperation, without identifying human information.The work does not use personal, sensitive, or offensive language data.
Risk: The main ethical risk is misinterpreting high benchmark scores as evidence that a policy is safe or reliable for open-world use.The supplied passage identifies misinterpretation as the primary concern.

B Training and Data Details … C.2 Mode Switching Under Receding-Horizon Sampling

The paper uses a standardized StarVLA-based training setup across SimplerEnv, LIBERO, and RoboCasa, then analyzes how history conditioning preserves short-horizon intent under receding-horizon replanning. IntentVLA reduces action disagreement between adjacent chunks, including average, tail, and variability measures in AliasBench ambiguity windows.

B Training and Data Details: Standard benchmark experiments use the StarVLA pipeline on 16 NVIDIA H100 GPUs with AdamW, a 1 × 10−5 learning rate, cosine annealing, ZeRO-2, gradient clipping, and no accumulation.The maximum gradient norm is 1.0.
B Training and Data Details: SimplerEnv fine-tunes on Bridge-DataV2 for 30K steps using 16 GPUs and batch size 16 per device, evaluated with official scripts.Bridge-DataV2 is a subset of Open X-Embodiment.
B Training and Data Details: LIBERO trains one policy jointly across Spatial, Object, Goal, and Long suites, while RoboCasa-GR1 fine-tunes on Humanoid Robot Tabletop Manipulation data for 30K steps.RoboCasa-GR1 uses 16 GPUs with batch size 16 per device.
C.1 Inter-Chunk Consistency in Ambiguous Windows: ICC-L2 measures L2 disagreement between overlapping predictions from adjacent chunks inside annotated AliasBench ambiguity windows, with lower values indicating stronger consistency.It provides an observable action-space proxy for the unobserved mode-switching consequence.
C.1 Inter-Chunk Consistency in Ambiguous Windows: 17.6% relative reduction: mean ICC-L2 decreases from 0.219 to 0.181 across AliasBench’s 12 tasks, with lower tail inconsistency across ambiguity families.The reduction indicates that recent visual history makes adjacent replanned chunks more consistent in ambiguous regions.
C.2 Mode Switching Under Receding-Horizon Sampling: Under aliased observations, independent adjacent chunks can each be plausible yet correspond to different short-horizon intents, creating a temporal consistency problem during receding-horizon execution.The analysis distinguishes this failure from merely representing multiple valid behaviors.
C.2 Mode Switching Under Receding-Horizon Sampling: If adjacent intent distributions are uniform over M plausible intents, the diagnostic switch probability is Pswitch(t, r) = 1 − 1/M; concentrating history around one intent drives switching toward zero.IntentVLA does not explicitly infer a discrete intent label, and Pswitch is not an additional training objective.
C.2 Mode Switching Under Receding-Horizon Sampling: 21.7% relative reduction: task-averaged 90th-percentile ICC-L2 decreases from 0.298 to 0.233, while ambiguity-window standard deviation drops from 0.093 to 0.046.These results indicate reductions in both average overlap disagreement and unstable high-error windows.

D Full RoboCasa-GR1 Results

Table 7 presents the complete 24-task RoboCasa-GR1 results, corresponding to the average scores reported in Table 4.

Table 7 reports full results across 24 RoboCasa-GR1 tasks and corresponds to Table 4’s average scores.

E AliasBench Task Definitions · F Real-world robot setup.

AliasBench isolates short-horizon observation aliasing across 12 manipulation tasks, where recent episode context resolves ambiguities that the current frame alone cannot. The real-world setup uses two teleoperated Franka arms with wrist and head cameras to collect expert demonstrations.

E AliasBench Task Definitions: AliasBench contains 12 tasks in which the current frame is insufficient to determine the correct continuation, while recent episode context provides the missing information.The benchmark targets short-horizon observation aliasing rather than generic long-horizon memory demands.
E AliasBench Task Definitions: Each rollout ends at task success or a task-specific maximum environment-step budget.For newly sampled tasks, each maximum test budget is twice the corresponding training sampling horizon.
E AliasBench Task Definitions: The benchmark covers four manipulation families: back-and-forth, crossing-path, bimanual, and multi-goal ambiguity.These families are intended to capture common manipulation patterns rather than synthetic edge cases.
E AliasBench Task Definitions: Visually similar states can correspond to different intents, including moving a phone between stands, cooking and plating bread, or handing a roller in opposite directions.These examples show ambiguity arising across task phases, destinations, and transfer directions.
F Real-world robot setup.: The real-world snack-cleanup setup uses two Franka Research 3 arms equipped with Robotiq 85 grippers.Intel RealSense D435 cameras provide head and end-effector views, while Polymetis supports teleoperation and expert-demonstration collection.
F Real-world robot setup.: RoboCasa-GR1 reports Avg@50 success rates (%) across 24 tabletop tasks, with selected baseline results sourced from official repositories and experiments.The cited baselines are Isaac-GR00T N1.5, Isaac-GR00T N1.6, and Qwen3-VL-GR00T.