Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models

Xingyu Ding; Yuzhong Zhao; Chunhai Zhao; Yinghuan Shi; Chaoyang Zhao; Yifan Zhang

Source-linked AI summary

Temporal Forcing: 4D Representation Alignment for Vision-Language-Action Models

Xingyu Ding, Yuzhong Zhao, Chunhai Zhao, Yinghuan Shi, Chaoyang Zhao, Yifan Zhang

arXiv:2608.30643v1Aug 31, 2026cs.RO

TL;DR

Existing VLA geometric alignment methods use current-frame 3D information, leaving long-horizon manipulation and visually similar states without temporal context. Temporal Forcing adds a history pathway and aligns its latent representations with pretrained 4D geometric features. It reaches 98.8% on LIBERO, improves its base model by 2.2 points, and raises physical full-task success from 20.0% to 43.3%.

Problem
Framewise 3D geometry captures the current state but not its evolution, limiting resolution of observation aliasing and task progress in long-horizon manipulation.
Method
Temporal Forcing summarizes bounded observation history into latent representations and aligns them with geometric features from a pretrained 4D foundation model.
Results
98.8% average success on LIBERO improves the base model by 2.2 points, while physical full-task success rises from 20.0% to 43.3%.
Takeaways & Limitations
4D representation alignment makes observation history useful to the model, with gains demonstrated on LIBERO, RoboTwin 2.0, and a physical multi-stage task.

Abstract

from arXiv · show

Recent vision-language-action (VLA) methods improve manipulation performance by aligning their representations with 3D scene geometry. However, these methods often struggle with long-horizon manipulation and observation aliasing between visually similar states due to a lack of temporal information: the 3D scene geometry captures only the current state, rather than how it has evolved over time. To resolve this, we present Temporal Forcing, a 4D representation alignment method for VLA models. Specifically, we first introduce a history pathway that enables a vanilla VLA model to summarize observation history into temporally aware latent representations. Then, the latent representations are aligned with the geometric features extracted by a pretrained 4D foundation model, which captures the evolving 3D world through temporally consistent geometric representations, enabling a deeper understanding of dynamic environments. Temporal Forcing reaches 98.8% on LIBERO, outperforming its base model by 2.2 points. On a physical hidden-placement task, it raises full-task success from 20.0% to 43.3%. Code will be publicly available.

1 Introduction

Framewise 3D alignment lacks the temporal information needed to resolve visually similar states and track progress in long-horizon manipulation. Temporal Forcing addresses this with history-aware latent representations aligned to pretrained 4D geometric features, improving benchmark and physical-task performance.

Motivation: 3D scene geometry captures only the current state, limiting long-horizon manipulation when earlier object-state transitions or task progress are no longer visible.These limitations produce observation aliasing between visually similar states and require tracking changes over time.
Method: Temporal Forcing adds a history pathway that summarizes bounded observation history into temporally aware latent representations.The method then aligns these representations with geometric features from a pretrained 4D foundation model.
Results: 62.8% mean success on twelve bimanual RoboTwin 2.0 tasks improves over 53.5% and exceeds the base model on nine tasks.The largest gains occur on handover tasks involving repeated object occlusion.
Results: 43.3% full-task success on the physical multi-stage task improves over 20.0%, while isolated manipulation stages remain comparable to the base model.The task requires resolving hidden-object references and tracking completed placement stages.
Results: 4D representation alignment makes observation history useful to the model, whereas history input alone is insufficient.The demonstrated gains span LIBERO, RoboTwin 2.0, and a physical multi-stage task.

2 Related Work

Related work develops geometric alignment, history-augmented VLAs, and temporal supervision, but these approaches differ in whether temporal information is supplied at input, stored in memory, predicted, or directly supervised in latent representations. Temporal Forcing extends training-time geometric alignment from framewise 3D features to temporally consistent 4D representations.

Training-time representation alignment: Existing VLA alignment methods supervise representations with geometric features from current observations, while some alternatives add explicit 3D inputs or rendered views.Explicit geometric inputs introduce additional deployment-time geometric inputs or preprocessing.
History-augmented VLAs: History-augmented VLAs extend what the model observes or stores through temporal modules, retrieval memory, few-shot memory, or aggregated 3D/4D history.These methods primarily modify input or memory rather than directly supervising temporal information in latent representations.
History-augmented VLAs: Temporal Forcing combines a lightweight History Pathway with 4D representation alignment so the History Latent Representation encodes scene evolution over time.Controlled experiments show that providing observation history alone is insufficient.
Temporal supervision for VLAs: World-model-based VLA methods introduce temporal supervision through future prediction or generation, whereas StreamVGGT produces temporally consistent geometric representations from causal observation streams.Temporal Forcing uses causal and dense geometric features only for training-time alignment.

3 Method

Temporal Forcing extends framewise 3D alignment with a history pathway and pretrained 4D geometric targets, aligning both temporal summaries and current-frame representations. Its training-only 4D supervision is removed at inference, leaving a fixed-token history pathway with gated injection into the base VLA.

Motivation: Temporal Forcing adds a history pathway because current-frame 3D geometry cannot represent object-state transitions or disambiguate visually similar long-horizon states.The base VLA receives only the current observation, whereas the history pathway supplies strictly past frames.
4D Representation Acquisition: A pretrained StreamVGGT processes a long causal context to provide temporally consistent Causal Geometric Features and current-frame Dense Geometric Features.Causal features supervise history representations, while dense features supervise current-frame image tokens; targets are extracted offline and stop-gradient.
History Pathway: The History Pathway compresses K past frames per camera into gist tokens, applies causal temporal aggregation, and summarizes them into 16 history tokens.Time-offset and camera embeddings are included, and the fixed output budget is independent of the history-window length.
Gated Injection and Inference: Zero-initialized gated cross-attention injects the 16 history tokens without extending the backbone sequence, preserving the base model mapping at initialization.At inference, the 4D foundation model, alignment losses, and projection heads are removed; only the history pathway and gated block remain.
4D Representation Alignment: Temporal alignment supervises history representations with state, change, and readout terms, while current-frame alignment matches image-token features to dense geometric targets.The change term isolates content that varies within the observation window by canceling components constant across it.

4 Experiments

Experiments evaluate Temporal Forcing across LIBERO, RoboTwin 2.0, controlled component studies, and a physical hidden-placement task. Results show gains are concentrated in settings where current observations omit earlier interactions, and controlled studies isolate the roles of temporal alignment and temporally consistent 4D targets.

Main Results: Temporal Forcing reaches an average success rate of 98.8% on LIBERO, compared with 96.6% for the base model.The model is trained jointly across the four suites; the largest suite gain is on Long, increasing from 93.8 to 97.2.
Main Results: On twelve bimanual RoboTwin 2.0 tasks, mean success rises from 53.5% to 62.8%, improving nine of twelve tasks.The largest gains occur on handover block, from 0 to 44, and handover mic, from 39 to 79, where objects are occluded during hand-off.
Ablation Study: History or 4D targets alone do not improve the recipe: the history-only variant drops 2.5 points below the base model, while current-frame 4D alignment drops Long by 8.0 points.The history pathway remains nearly unused without supervision, and current-frame models cannot recover the temporal context contained in the targets.
Ablation Study: In the controlled LIBERO study, adding temporal alignment raises the average from 83.7 to 93.6, while removing history at inference costs 5.4 points.The inference intervention keeps the current observation and model weights unchanged while setting the injection gate to zero.
Ablation Study: Temporally consistent 4D targets widen the history-enabled average gap over framewise 3D targets to 8.6 points, compared with 2.8 points without history.The study reports averages of 93.6 versus 85.0 with history and 90.5 versus 87.7 without it.
Real-Robot Experiments: On the physical hidden-placement task, full-task completion increases from 6/30 trials for the base model to 13/30 for Temporal Forcing, or 20.0% to 43.3%.The advantage emerges after the first block becomes hidden, when the model must retain task progress rather than rely only on visible manipulation skills.

5 Conclusion

Temporal Forcing extends framewise 3D geometric alignment with temporally aware history representations supervised by pretrained 4D features. It improves benchmark and physical multi-stage performance while remaining comparable on isolated stages.

Temporal Forcing equips a vanilla VLA model with a history pathway and aligns temporally aware latent representations with pretrained 4D geometric features.
98.8% LIBERO success is achieved, while full-task success on the physical multi-stage task rises from 20.0% to 43.3%.
History input alone is insufficient, whereas temporally consistent 4D targets make observation history useful to the model.
Isolated-stage performance remains comparable to the base model, distinguishing sequential-task gains from large single-stage differences.

A Implementation Details

Temporal Forcing constructs causal geometric targets from StreamVGGT and summarizes ordered visual history into fixed-length tokens. These components support temporal alignment while keeping the added pathway compatible with the VLA backbone.

StreamVGGT processes trajectories in causal order, extracting layer-21 features every four timesteps and spatially averaging each target frame’s feature grid.LIBERO uses causal segments up to 12 seconds; RoboTwin and real-robot targets use the available episode prefix.
Each past frame is encoded by frozen DINOv2, compressed into gist tokens, and aggregated by a causal temporal transformer using time-offset and camera embeddings.
Sixteen learned queries summarize the ordered sequence into a fixed set of history tokens independent of history-window length.
Zero-initialized gated cross-attention injects history tokens into the VLA backbone as an identity mapping at initialization.

B Training Protocols

The training protocols retain the base recipe while specifying benchmark-specific cameras, frozen components, and representation-probe procedures. Probes select StreamVGGT layer 21, and causal 4D targets provide stronger temporal contrast than single-frame 3D targets.

Table 4 specifies per-benchmark training protocols while retaining the base instruction template, image resolution, and action normalization.
Learning rates are 10−5 for the backbone, 10−4 for the action head and new modules, and 2.5 × 10−5 for remaining parameters, with gradient norm clipped to 1.0.
Real-robot fine-tuning freezes the backbone vision tower because demonstrations are small, and the base model uses the same setting.
Layer 21 is selected as the earliest layer jointly reaching depth R2 ≥0.99 and motion correlation ≥0.36.The final two layers raise motion correlation only modestly, to at most 0.376.
Causal 4D targets raise temporal contrast from 0.101 for 3D targets to 0.491, a delta of +0.390, on the first 60-anchor segment of LIBERO-Long.

C Additional Results

Additional evaluations show gains across RoboTwin tasks, LIBERO-Plus perturbations, sequential real-robot rollouts, and controlled history-use experiments. The strongest evidence concentrates on occluded or history-dependent decisions rather than isolated visible stages.

Temporal Forcing improves over the base model on 9 of 12 RoboTwin tasks.Largest gains occur on Handover Block, from 0 to 44, and Handover Mic, from 39 to 79; regressions occur where completed stages remain visible.
77.8 pooled LIBERO-Plus success improves over 75.0 without training on perturbations, with sensor-noise success increasing from 73.1 to 81.3.
Temporal Forcing and the base model remain close on every isolated real-robot stage, so the sequential-task difference is not explained by a large individual-stage gap.
Temporal alignment opens the injection gate to 13.4×10−3 versus 0.6 × 10−3 with action loss alone, while history contributes +5.4 success points at decision time.The +5.4-point contribution is concentrated on Goal and Long, each at +7.8.
13/30 full-task successes are achieved by Temporal Forcing versus 6/30 for StarVLA-OFT in qualitative sequential rollouts.Isolated-stage totals are 78/90 and 75/90, respectively.