Source-linked AI summary
TTT in VLA Models: Exploring Training with Human Demonstrations
Qiangong Zhou, Jiafeng Liang, Yimin Dong, Yang He, Hongxiang Li
TL;DR
Long-horizon robot tasks may require models to retain and use information encountered earlier. This paper integrates TTT into two VLA models and trains on paired human demonstrations and robot trajectories; across the tested settings, it does not establish that models learned to use demonstration memory for target selection.
Problem
Long-horizon robot decisions may depend on information encountered much earlier, motivating models that retain and use historical context.
Source: §1 · p. 1Method
The paper integrates TTT into InternVLA-A1 and Pi0.5, processing paired human demonstrations before robot trajectories and carrying fast-weight memory across the phases.
Source: §1 · p. 1Results
Across the tested settings, evidence does not establish that the models learned to use demonstration memory for target selection, and real-robot tests showed fixed-position cup preferences.
Source: §5 · p. 14Takeaways & Limitations
The reported observations document implementation and training configurations for further inspection and replication, while leaving the causes of weak demonstration response unresolved.
Source: §1 · p. 1The observations primarily concern cup selection and limited historical dependencies; the required training scale for memory use cannot yet be determined.
Source: §6.4 · p. 16
Abstract
from arXiv · showhide
Long-horizon robot tasks require retaining and using past information. We integrate Test-Time Training (TTT), which stores context in fast weights, into InternVLA-A1 and Pi0.5 to study action generation conditioned on human demonstrations through training on paired demonstrations and robot trajectories. We implement memory transfer between the two phases and truncated backpropagation, and examine training objectives, gradient coverage, and contrastive regularization across video, short-video, and single-frame cue tasks. In our experiments, adding TTT and training on paired data at the current scale did not lead to consistent target selection based on demonstrations. Under the tested settings, shorter sequences, full trajectory gradient connectivity, and contrastive regularization did not clearly improve responsiveness to demonstrations. When demonstrations were reduced to single-frame cues, real-robot execution still showed a preference for cups at fixed positions. We document the implementation, training configurations, and observations to support further inspection and replication.
1 Introduction
The paper explores whether Test-Time Training (TTT) can let VLA models retain and use human-demonstration context for long-horizon robot action generation. It pairs demonstrations with robot trajectories and documents the implementation and observations, while noting that current training scale leaves memory capabilities insufficiently trained and evaluated.
1 Introduction
- TTT stores past information in fast weights, providing fixed-size memory during inference while supporting online updates and end-to-end learning of memory writes and reads [1].These properties motivate exploring TTT for decisions that may depend on much earlier information.Source: §1 · p. 1
- The approach pairs each human demonstration with its corresponding robot trajectory, transfers memory between phases, and integrates TTT into Pi0.5[2] and InternVLA-A1[3].The demonstration is processed first; TBPTT organizes state transfer and gradient propagation between the two phases.Source: §1 · p. 1
- Training infrastructure constraints left TTT’s memory capabilities insufficiently trained and evaluated at the scale studied.The paper explicitly leaves unresolved causes and incomplete validation open.Source: §1 · p. 1
2 Related Work
Related work traces TTT from test-time adaptation toward architectures that use parameter updates as memory, then reviews applications across language, vision, video, and robot policies. The paper draws on these approaches to integrate TTT into VLA models conditioned on human demonstrations.
2.1 From Test-Time Adaptation to Parametric Memory
- TTT progressed from adapting to test-time distribution shifts [5–7] to TTT layers that use inner-model weights as recurrent state for compressing context [1], alongside related parametric-memory methods [8–10].In the TTT layer used here, K and V support memory writing and Q reads it; outer-loop task training learns how information is written and used.Source: §2.1 · p. 2
2.2 TTT across Domains
- Across language, vision, and video, TTT methods vary in memory structure, inner objectives, and temporal processing, including masked reconstruction, feature learning, and online frame adaptation [1, 7–13].The reviewed video methods retain context across frames or segments, but their training procedures still require adaptation to robot tasks.Source: §2.2 · p. 2
2.3 TTT for Robot Policies
- Robot-policy work combines fast-weight memory with sequence training [4], explores alternative memory interfaces and objectives [15–17], and includes video-branch TTT trained with paired robot data [14].Building on these mechanisms, the paper integrates TTT into Pi0.5[2] and InternVLA-A1[3] to study how demonstrations enter memory and how robot supervision constrains their use.Source: §2.3 · p. 2
3 Architecture
This section describes how fast-weight TTT memory is integrated into VLA backbones and carried across human and robot trajectory segments, alongside truncated backpropagation whose reach depends on segmentation.
3.1 TTT Layer Structure
- TTT stores trajectory context in per-sample, per-head fast weights updated by inner learning, while outer task gradients train slow parameters and learned initial fast weights.The differentiable fast-weight updates let task losses train the memory-writing mechanism.Source: §3.1 · p. 4
3.2.1 InternVLA-A1
- InternVLA-A1[3] receives TTT after attention in every generation and action layer; state and learnable registers write memory, while action tokens read it during denoising.The register route restricts direct action-token access to registers, so register information reaches action representations through the TTT branch.Source: §3.2.1 · pp. 4–5
3.2.2 Pi0.5
- Pi0.5[2] places TTT in all 18 action-expert layers, using 16 register tokens plus state to write; each anchor writes once and 10 denoising steps read the same memory.Its constrained Fast MLP uses a 256 →1024 →256 architecture and clips update norms per trajectory and layer while preserving gradients.Source: §3.2.2 · p. 6
3.3.1 From RoboTTT to TBPTT
- Following RoboTTT[4], TBPTT divides chronological trajectory training into graph-retaining segments: fast-weight values carry across boundaries, but gradients do not.Later action losses can influence earlier writes within a segment, while retaining a full-trajectory graph would require more training GPU memory.Source: §3.3.1 · p. 6
3.3.2 Segment Backpropagation and Slow-Weight Updates
- Within a trajectory batch, fast weights update across anchors while slow weights remain fixed; slow-weight gradients accumulate across segments before one batch-level update.At each segment end, losses are backpropagated, the graph is released, and fast weights are detached without changing their values.Source: §3.3.2 · p. 7
3.3.3 Memory Read and Write Frequencies
- Demonstrations are processed before robot trajectories with fast weights transferred; human writes follow video time and robot writes action-chunk anchors at independently configured frequencies.The human phase writes through the inner reconstruction objective without robot action supervision; robot-phase action losses accompany memory updates along the trajectory.Source: §3.3.3 · pp. 7–8
3.3.4 Segments across the Human–Robot Boundary
- A transition segment joins final human and initial robot anchors so robot losses can supervise nearby human writes, but truncation still limits gradient reach across the full pair.Earlier human anchors may be processed in truncated prefix segments; memory values can persist even when gradient paths to later robot actions are cut.Source: §3.3.4 · p. 8
- TBPTT processes paired human demonstrations and robot trajectories by sampling and segmenting anchors, propagating fast-weight memory through human writes and robot training, truncating segment graphs, and updating slow weights.Human anchors write to memory; robot anchors receive task supervision, with gradients backpropagated per segment before memory is detached.Source: §3.3.4 · p. 9
4 Experiments
The experiments evaluate TTT-conditioned action generation from paired human demonstrations and robot trajectories, progressing from video demonstrations to short videos and single-frame cues. Across these settings, the models did not show consistent demonstration-dependent target selection, and the tested changes did not clearly resolve that behavior.
4.1.1 GT-State and AR-State Open-Loop Evaluation
- GT-state evaluation uses ground-truth states at each anchor, whereas AR-state evaluation feeds the previous predicted action into the next state; both carry fast weights along trajectories.AR-state remains open-loop because its images come from dataset trajectories, not from predicted actions in the environment.Source: §4.1.1 · pp. 9–10
4.1.2 Misalignment with Cross-Step Memory
- Cross-step memory can mismatch later inputs: GT-state carries memory alongside demonstration states, while AR-state propagates predicted states with demonstration images that may conflict.GT-state training does not directly train continuation from altered states and observations after executing erroneous actions.Source: §4.1.2 · pp. 10–11
Choice for subsequent experiments
- TTT abs had lower AR-state open-loop error but lower real-robot completion than TTT delta, so the improved AR metric did not translate into better task performance.The comparison led the authors to retain GT-state supervision and incremental action representations.Source: Choice for subsequent experiments · p. 11, Table 1 · p. 11
4.2.1 Training Setup
- Training pairs each human video identifying a target cup with the corresponding robot grasp trajectory, then carries demonstration-written fast-weight memory into robot action prediction.The video task used 654 complete paired samples; training was intended to teach target selection conditioned on demonstrations.Source: §4.2 · p. 11, §4.2.1 · p. 12
4.2.2 Experimental Results
- Both models showed fixed directional preferences in real-robot video-demonstration tests, tended to select similar targets with or without demonstration-memory writes, and scored below the 1/3 uniform-selection rate.Demonstration memory had only small effects on action error.Source: §4.2.2 · p. 12
4.3 Full-Trajectory Training on Short Sequences
- The experiments shortened sequences so a segment could cover an entire pair, then used a single-frame cue to remove the need to track target changes across video frames.Shorter sequences were intended to let robot decision losses reach earlier human writes; the cue dataset retained the final demonstration frame with the full robot trajectory.Source: §4.3 · p. 12, §4.4 · p. 13, §4.4.1 · p. 13
4.3.2 Experimental Results
- With full-trajectory gradient connectivity on short-video pairs, InternVLA-A1 reached 25% cup-selection success after 10,000 steps and 13% after 60,000 steps; contrastive regularization showed no clear improvement.One segment covered each human–robot pair, allowing robot decision losses to propagate through fast-weight updates to earlier demonstration writes.Source: §4.3 · p. 12, §4.3.2 · p. 13
4.4.2 Experimental Results
- In real-robot single-frame-cue tests, neither model showed the expected cue dependence; both preferred fixed cup positions and scored below 33%.These results do not establish that TTT cannot remember single-frame information; cue writing, retention, or action use may remain insufficient.Source: §4.4.2 · p. 14
5 Conclusion
The paper documents a TTT architecture and training procedure for pairing human demonstrations with robot trajectories in VLA models. In tested settings, the models did not show consistent demonstration-conditioned target selection, and the causes and effects of larger-scale training remain unresolved.
5 Conclusion
- The main contribution is a clear reproduction and description of TTT architecture and training for combining human demonstrations with robot trajectories in VLA models.Training infrastructure constraints limited pretraining and memory-task training scale, leaving memory capabilities insufficiently trained and evaluated.Source: §5 · p. 14
- Across tested settings, TTT models responded weakly to demonstrations and did not establish that they learned to use demonstration memory for target selection.The experiments progress from video demonstrations to short videos and single-frame cues; real-robot execution still showed fixed-position cup preferences.Source: §5 · p. 14
- The authors cannot yet identify whether memory writing, retention, or action use is the main issue, or determine whether scaling training would improve the observations.They document implementation, configurations, evaluation procedures, and unresolved observations for inspection and replication.Source: §5 · p. 14
6 Discussion
The discussion identifies open design and evaluation questions for TTT layers, their placement, and memory use. It also cautions that the findings reflect limited demonstration-conditioned training and do not establish the scale needed for effective memory use.
6.1 TTT Layer Design Choices
- The two-layer GELU fast-weight MLP is a starting configuration, not an established choice for robot tasks; matched ablations are needed to test its capacity, learning rate, and update count.Guidance from ViT3[11] does not establish this configuration as best because its relevant experiments and final architecture differ.Source: §6.1 · pp. 14–15
6.2 TTT Placement and Layer Coverage
- InternVLA-A1 uses separate TTT fast weights in generation and action branches, but their distinct roles remain unknown, while added layers raise computation and training-memory costs.Branch ablations and demonstration-condition controls, with fixed data and training procedures, are needed to test their use.Source: §6.2 · p. 15
6.3 Memory Evaluation
- Memory evaluation should distinguish information decodability from its contribution to decisions and compare correct-history memory against incorrect history or initial fast weights under controlled current inputs.Randomized object positions in later datasets make linear-probe interpretability analysis difficult.Source: §6.3 · p. 15
6.4 Training Scale and Limitations
- The demonstration-conditioned training data cover limited historical dependencies, so findings primarily apply to current tasks and settings; the scale needed for effective memory use remains unknown.The model must learn what to write, retain, and use, and larger-scale training may help, but its required scale cannot yet be determined.Source: §6.4 · pp. 15–16
A Action Representations and GT/AR Objectives
The appendix distinguishes action increments tied to the current ground-truth state from absolute targets that remain tied to demonstrated positions after state deviation. The action-representation comparison does not establish that AR-state training is better or worse than GT-state training.
A Action Representations and GT/AR Objectives
- TTT delta predicts increments relative to the current ground-truth state, whereas TTT abs predicts absolute action targets that remain at demonstrated positions after propagated-state deviation.Figure 5 depicts the target geometry, not learned corrective behavior during execution.Source: A Action Representations and GT/AR… · p. 18, Figure 5 · p. 18
- Absolute actions could reduce error propagation in AR-state open-loop evaluation, but this is an expectation rather than evidence of recovery behavior.Changing the action representation does not provide observations at deviated states or direct supervision for recovery.Source: A Action Representations and GT/AR… · p. 18
- TTT abs has lower AR-state MAE than TTT delta but a lower real-robot completion rate, so the comparison does not establish whether AR-state or GT-state training is better.Both action representations are trained with ground-truth images and states; only the action representation differs.Source: A Action Representations and GT/AR… · p. 18
B Contrastive Regularization: Design, Implementation, and Results
The section adds two regularizers to the original correct-pair trajectory-fitting objective: one compares action errors across demonstrations, and the other encourages distinct fast-weight representations.
B Contrastive Regularization: Design, Implementation, and Results
- Trajectory contrastive regularization compares action prediction errors across demonstrations, while memory separation regularization encourages distinct fast-weight representations; both retain the original correct-pair objective.Source: B Contrastive Regularization: Design,… · p. 19
B.1 Objectives and Sample Pairing
The objectives fit robot actions under correct demonstration–trajectory pairings while separately encouraging distinct demonstration memories and discriminating against incorrect pairings. Computation retains human-demonstration graphs and processes robot branches independently, with explicit gradient routing.
B.1 Objectives and Sample Pairing
- The batch defines correct pairs as Lii and cross-pairs as Lij for i ≠ j, with Lij measuring robot Ri’s action loss using memory from Hj.The fitting objective uses correct pairs; cross-pairings compare the same robot trajectory under alternative demonstration memories.Source: §B.1 · p. 19
- Incorrect pairings are penalized only when their loss is lower than the correct-pair loss for the same robot trajectory, using a zero-margin comparison.The comparison holds the robot trajectory fixed, and equal losses receive zero gradient.Source: §B.1 · p. 19
- The memory regularizer compares normalized changes in demonstration-written fast weights, but distinct memory representations alone do not ensure that actions use them.Features include only TTT branches written to by the demonstration.Source: §B.1 · p. 19
B.2 Computation and Gradient Flow
- For full cross-pairing, human graphs are retained while robot branches are processed one at a time from independent copies of terminal demonstration memory.Branch gradients accumulate at terminal human fast weights before propagating through the retained human graph.Source: §B.2 · p. 20, Figure 6 · p. 20
- Contrastive gradients return to human writes through demonstration memory, while memory separation backpropagates through both human branches; truncating robot gradients can sever paths to demonstration writes.The scheme reduces simultaneously retained graphs, not the total number of pair evaluations.Source: §B.2 · p. 20
Experimental setup
The experiment compares standard and contrastively regularized training on a short-video demonstration task, using InternVLA-A1 checkpoints after 10,000 steps. The regularized setup adds objectives and changes loss aggregation, while substantially increasing training time.
Experimental setup
- The short-video demonstration contains one cup-covering action, and the robot must select the demonstrated target cup.This shortens the gap between demonstration information and target selection compared with the original task.Source: Experimental setup · pp. 20–21
- Both groups start from original InternVLA-A1 backbone weights and are compared after 10,000 steps, without first-stage grasping training.TTT is in every generation- and action-expert layer; demonstrations write generation-expert memory at 1.5 Hz, and robot anchors predict 50-step action chunks.Source: Experimental setup · p. 21
- Regularized training retains action-fitting and generation losses, adds trajectory-contrastive and memory-separation terms, and aggregates losses differently from standard training.Its trajectory term is a zero-margin hinge; standard training aggregates valid-anchor action losses, while regularized training averages within trajectories and then across trajectories.Source: Experimental setup · p. 21
- Contrastive regularization runs at approximately 20% of standard training speed, taking approximately five times longer for the same number of steps.Source: Experimental setup · p. 21
Open-loop evaluation
Open-loop evaluation found no consistent demonstration benefit, and contrastive regularization did not improve the reported errors. The evaluation reports action MAE and compares validation predictions with demonstration-memory writes enabled or skipped.
Open-loop evaluation
- Contrastive regularization yields higher training- and validation-set errors in both open-loop evaluation modes than standard training.The table reports left-arm joint-action MAE in absolute coordinates, in radians, where lower is better.Source: Open-loop evaluation · p. 21, Table 5 · p. 21
With and without human demonstrations
- The validation comparison tests demonstration-memory use by contrasting predictions with demonstration writes enabled against predictions with those writes skipped.Source: With and without human demonstrations · p. 21
- Demonstration-related error changes are below 0.15% in magnitude for both checkpoints and have inconsistent signs, with no consistent benefit from contrastive regularization.The validation comparison defines benefit as MAE without a demonstration minus MAE with one, divided by MAE without a demonstration; positive values indicate reduced error with the demonstration.Source: With and without human demonstrations · pp. 21–22, Table 6 · p. 22
Real-robot results and behavior
In the tested three-cup task, the regularized 10,000-step checkpoint did not outperform standard training, and the overall setting showed no improvement in real-robot cup-selection success. The regularized checkpoint also showed repeated movements and hesitation during grasping, whose cause was not established.
Real-robot results and behavior
- Real-robot evaluation used human demonstration videos in a three-cup task, counting a trial as successful when the robot selected the demonstrated cup.Source: Real-robot results and behavior · p. 22
- The regularized 10,000-step checkpoint did not outperform standard training at the same step and showed repeated movements and hesitation during real-robot grasping.The available records do not establish the cause of this behavior.Source: Real-robot results and behavior · p. 22
- Under the tested setting, contrastive regularization increased training cost without improving open-loop error, demonstration benefit, or real-robot cup-selection success.This observation is limited to the task and configuration tested; it does not determine performance under other regularizer weights or training settings.Source: Real-robot results and behavior · p. 22, Table 7 · p. 22