PaperlayerView exact revision on arXiv

Source-linked AI summary

RLDX-1 Technical Report

Dongyoung Kim, Huiwon Jang, Myungkyu Koo, Suhyeok Jang, Taeyoung Kim, Beomjun Kim, Byungjun Yoon, Changsung Jang, Daewon Choi, Dongsu Han, Donguk Lee, Heeseung Kwon, Hojin Jeon, Jaehyun Kang, Jaekyoung Bae, Jihyuk Lee, Jimin Lee, John Won, Joonwoo Ahn, Junhyeong Park, Junyoung Sung, Kyungmin Lee, Minseong Han, Minsung Yoon, Sejune Joo, Seonil Son, Seungcheol Park, Seunggeun Cho, Seungjun Moon, Seungku Kim, Yonghoon Dong, Yongjin Cho, Youngchan Kim, Chang Hwan Kim, Dohyeon Kim, Heecheol Kim, Heewon Lee, Hensen Ahn, Hyungkyu Ryu, Hyunsoo Choi, Hyunsoo Shin, Jaeheon Jung, Jaewoo Kim, Jinwook Kim, Joochul Chang, Joonsoo Kim, Junghun Park, Jungwoo Park, Junho Cho, Junhyeok Park, Junwon Lee, Kangwook Lee, Kwanghoon Kim, Kyoungwhan Choe, Manoj Bhadu, Nayoung Oh, Sangjun Kim, Sangwoo Kim, Seunghoon Shim, Seunghyun Kim, Seungjun Lee, Seungyup Ka, Sungryol Yang, Wook Jung, Yashu Shukla, Yeonjae Lee, Yeonwoo Bae, Jinwoo Shin

arXiv:2605.03269v2cs.ROcs.AIcs.LG

TL;DR

VLAs focused on versatile scene and language understanding remain challenged by real-world dexterous tasks that require broader functional capabilities. RLDX-1 combines MSAT with system-level data, training, and inference choices, and consistently outperforms recent VLA baselines across simulation and real-world evaluations.

  • Problem

    Human-like dexterous manipulation in real-world environments remains a robotics goal, while existing efforts mainly emphasize versatile scene understanding and language-conditioned generalization.

    Source: §1 · p. 3

  • Method

    RLDX-1 combines MSAT’s modality-specific streams and joint self-attention with synthetic rare-scenario data, staged training, and inference optimization for real-time control.

    Source: §1 · pp. 3–4

  • Results

    Across simulation benchmarks and real-world manipulation tasks, RLDX-1 consistently outperforms baselines; on ALLEX Object-in-Box Selection it reaches 91.7%, versus baselines in the 30% range.

    Source: §1 · pp. 4–5

  • Takeaways & Limitations

    The paper positions RLDX-1 as a step toward real-world dexterous manipulation that incorporates motion awareness, long-term memory, and physical sensing.

    Source: §8 · p. 28

Abstract

from arXiv · show

While Vision-Language-Action models (VLAs) have shown remarkable progress toward human-like generalist robotic policies through the versatile intelligence (i.e. broad scene understanding and language-conditioned generalization) inherited from pre-trained Vision-Language Models, they still struggle with complex real-world tasks requiring broader functional capabilities (e.g. motion awareness, long-term memory, and physical sensing). To address this, we introduce RLDX-1, a general-purpose robotic policy for dexterous manipulation built on the Multi-Stream Action Transformer (MSAT), an architecture that unifies these capabilities by integrating heterogeneous modalities through modality-specific streams with cross-modal joint self-attention. RLDX-1 further combines this architecture with system-level design choices, including data synthesis for rare manipulation scenarios, learning procedures specialized for human-like manipulation, and inference optimizations for real-time deployment. Through empirical evaluation, we show that RLDX-1 consistently outperforms recent frontier VLAs (e.g. $π_{0.5}$ and GR00T N1.6) across both simulation benchmarks and real-world tasks that require broad functional capabilities beyond general versatility. In particular, RLDX-1 shows superiority in ALLEX humanoid tasks by achieving success rates of 86.8% while $π_{0.5}$ and GR00T N1.6 achieve around 40%, highlighting the ability of RLDX-1 to control a high-DoF humanoid robot under diverse functional demands. Together, these results position RLDX-1 as a promising step toward reliable VLAs for complex, contact-rich, and dynamic real-world dexterous manipulation.

1. Introduction

RLDX-1 targets dexterous real-world manipulation by combining versatile vision-language reasoning with motion awareness, long-term memory, and physical sensing. Its architecture, data, training, and inference choices are designed to support these capabilities in deployment.

1. Introduction

  • RLDX-1 adds motion-aware video processing, queued long-term observation features, and future physical-signal prediction, integrating modalities through dedicated MSAT streams and joint self-attention.On conveyor-belt catching, RLDX-1 exceeds 87.5% success while π0.5 remains below 29.2%.Source: §1 · pp. 3–4
  • 9.1% higher success on GR-1 Tabletop resulted from synthetic data added to real-data training, with generated videos and action annotations quality-filtered for video quality and motion consistency.The pipeline diversifies scenes and instructions, generates videos, optionally applies video-to-video transfer, and checks predicted-action rollouts against generated videos in simulation.Source: §1 · p. 4
  • The three-stage pipeline pre-trains on multi-embodiment data, mid-trains with embodiment-specific demonstrations and synthetic trajectories, then post-trains on task data with optional reinforcement learning.It produces a general pre-trained model and specialized ALLEX and FR3 variants.Source: §1 · p. 4
  • 43.7 ms per step for the all-modality model, a 1.63× speedup from 71.2 ms, follows static CUDA Graph capture and custom fused kernels that reduce launch overhead and memory traffic.The 71.2 ms baseline is measured under PyTorch Eager on an NVIDIA RTX 5090.Source: §1 · p. 4
  • 58.7% on GR-1 Tabletop, RLDX-1 outperformed GR00T N1.6 at 47.6% and consistently outperformed baselines across simulation benchmarks.On real-robot tasks, RLDX-1 also outperformed major baselines, with the gap especially pronounced on tasks requiring functional capabilities.Source: §1 · p. 4
  • 91.7% on ALLEX Object-in-Box Selection exceeded the 30% range achieved by both GR00T N1.6 and π0.5; RLDX-1 also beat π0.5 on unseen-object and unseen-task tests.On the OpenArm humanoid, RLDX-1 improved Unseen Object from 37.5% to 54.2% and Unseen Task from 45.8% to 54.2%.Source: §1 · p. 5

1.1. RLDX-1 Overview

  • RLDX-1 supports single-arm, dual-arm, and humanoid robots, mapping video, language, robot state, and physical signals to future action chunks through a unified framework.Its VLM produces cognition features augmented by memory, while the action model combines these features with physical signals and robot states.Source: §1.1 · p. 5, Figure 2 · p. 5

2. Neural Architecture

RLDX-1 combines a robot-adapted, temporally aware VLM with a multimodal action model that integrates cognition, proprioception, actions, and optional physical sensing. Its architecture uses modality-specific streams with joint attention to support cross-modal information exchange.

2.1. Vision-Language Model

  • Robot-trajectory VQA fine-tunes the VLM for spatial relations, subtasks, and low-level actions, while 64 cognition tokens extract action-relevant features from visual and language context.The action decoder uses intermediate-layer hidden states rather than the final language-specialized layer.Source: §2.1 · pp. 6–7
  • Multi-frame observations and a motion module make the VLM motion-aware, while temporal context is compressed into a pooled token to reduce computation.The vision encoder integrates space-time self-similarity features after its ninth of 27 layers; LLM compression follows the fourth layer.Source: §2.1 · p. 7
  • A causal-attention memory module combines current cognition features with a queue of the three most recent cached features, supporting long-term reasoning over sequential tasks.The queue stores cognition features at intervals of H + 1 timesteps, and the resulting memory features are supplied to the action model alongside the original cognition features.Source: §2.1 · p. 7

2.2. Action Model

  • MSAT processes cognition and action inputs in dedicated early streams, then jointly processes them; joint self-attention enables cross-modal exchange while retaining modality-specific parameters.When physical signals are available, MSAT adds a physics stream. Its design also uses action-stream RoPE, a flow-matching timestep token, RMSNorm, and SwiGLU.Source: §2.2 · pp. 8–9
  • For contact-rich tasks, an optional physics stream processes physical signals separately while exchanging information through joint attention and can be disabled when signals are unavailable.An auxiliary flow-matching objective predicts future physical signals during training to encourage the model to learn interaction dynamics and use physical feedback for action generation.Source: §2.2 · p. 8

3. Training Data

RLDX-1 training data combines diverse public and in-house real-robot datasets with synthetic data to cover specialized manipulation scenarios that are difficult to scale through direct collection. The data spans multiple embodiments and adds tactile and torque sensing alongside instruction, scene, and motion diversity.

3.1. Public Real-World Data

  • OXE aggregates over 1M trajectories across 20 embodiments, while DROID (Khazatsky et al., 2024) adds 92K FR3 trajectories across 564 scenes and 86 tasks.RLDX-1 follows a curated OXE mixture that includes BridgeV2, Fractal, and Kuka.Source: §3.1 · p. 9
  • Galaxea Open-World (Jiang et al., 2025a), Agibot World, and Fourier ActionNet (Fourier ActionNet Team, 2025) extend coverage to dual-arm and dexterous humanoid manipulation.RLDX-1 samples 275K Agibot World episodes, including all hand-manipulation episodes.Source: §3.1 · p. 9

3.2. In-house Real-World Data

  • In-house data uses an FR3 platform augmented with tactile and torque sensing, plus an ALLEX humanoid with torque feedback for high-DoF dexterous manipulation.ALLEX has 7-DoF arms, 15-DoF hands, a 2-DoF waist and neck, and stereo egocentric cameras; data collection uses multiple teleoperation devices and glove-based finger tracking.Source: §3.2 · pp. 9–10

3.3. Synthetic Data

  • Synthetic data augments public GR-1 and in-house ALLEX demonstrations with generated videos and inverse-dynamics-model action labels.The pipeline uses the source video’s initial frame and task instruction to generate a new video, then predicts its action sequence from current and future observations.Source: §3.3 · p. 10, Figure 4 · p. 10
  • Filtering combines VLM checks of instruction following and trajectory plausibility with simulator-based motion-consistency checks on IDM-predicted actions.A probe compares generated videos with simulator rollouts and retains samples whose alignment probability exceeds a threshold.Source: §3.3 · pp. 10–12, Figure 4 · p. 10
  • VLM-based factorized and skill-conditioned instruction generation diversifies behaviors while preserving scene feasibility, and I2I/V2V transformations vary scene appearance.I2I edits vary table, object, lighting, and background while preserving scene structure; V2V changes texture and color while preserving motion dynamics and action validity.Source: §3.3 · p. 11

4. Training Procedure

RLDX-1 is trained in three stages: multi-embodiment pre-training, embodiment-specific mid-training that expands functionality, and task-specific post-training to refine deployment performance.

4.1. Pre-Training

  • Pre-training uses flow matching on multi-frame video from gripper robots and dexterous humanoids, adds 150K synthetic GR-1 episodes, and uses shared-space projections to support unseen-embodiment adaptation.The corpus spans single-arm, dual-arm, and humanoid embodiments; embodiment-agnostic projections are applied to a small fraction of samples in each batch.Source: §4.1 · p. 12

4.2. Mid-Training

  • Mid-training combines ALLEX’s in-house teleoperation data with 72K synthetic episodes at a 5:5 sampling ratio, and FR3’s in-house data with 92K DROID episodes at 8:2.In-house data solely supervises new memory, torque, and (for FR3) tactile inputs, while larger-scale data preserves broad manipulation coverage.Source: §4.2 · p. 13, Figure 7 · p. 13
  • Mid-training adds motion awareness through STSS, long-term memory through queued cognition features, and physical sensing by predicting future sensor trajectories over each action chunk.ALLEX uses torque feedback; FR3 uses torque and tactile signals. Memory spans 120 past timesteps for ALLEX and 48 for FR3.Source: §4.2 · p. 13

4.3. Post-Training

  • Post-training adaptively collects base demonstrations across task variations, then targets observed failures with focused refinement data until the policy reaches the desired performance.The protocol decomposes tasks into ordered motion primitives, preserves consistency factors, and varies factors such as object poses and initial configurations.Source: §4.3 · p. 14
  • RECAP refines policies with offline VLM-critic training, advantage-conditioned supervision, and iterative rollouts; the text critic predicts integer values through the VLM’s native number-token interface.The critic takes the observation, task instruction, and discretized state as input, avoiding a newly initialized value-prediction head.Source: §4.3 · pp. 14–15

5. Inference Strategy

RLDX-1 targets closed-loop inference latency by optimizing both graph capture and kernel execution, reducing delays that can make actions stale in dynamic environments.

5.1. Graph Capture Optimization

  • Precomputing and reusing rotary embeddings and attention masks removes fixed-configuration computations from the forward pass, enabling a single CUDA Graph and one launch per inference step.This static graph conversion resolves fragmentation that otherwise splits the forward pass into multiple subgraphs.Source: §5.1 · p. 16

5.2. Kernel Optimization

  • Each inference iteration runs the full forward pass in one shot without autoregressive generation, interleaving compute-bound matmuls with memory-bound operators in a short-sequence prefill.End-to-end latency therefore depends on coordinating data movement and computation across operator blocks, not only on individual operator costs.Source: §5.2 · p. 16
  • Torch Compile’s fixed fusion patterns and opaque attention kernels block cross-operator fusion, leaving RMSNorm, RoPE, and attention intermediates repeatedly moved through global memory.The resulting decoupling of data movement from computation is a missed optimization opportunity in the short-prefill workload.Source: §5.2 · p. 16, Figure 10 · p. 16
  • Workload-aware custom kernels combine operator fusion with on-chip memory reuse and compute ordering, keeping selected intermediate tensors on-chip to reduce memory traffic.The authors profile the Torch Compile implementation with NVIDIA Nsight Compute, then replace inefficient operator groups with manually designed fused kernels.Source: §5.2 · pp. 16–17, Figure 10 · p. 16

6. Evaluation and Analysis

This section evaluates RLDX-1’s pretrained and mid-trained capabilities across simulation and robotic tasks, including tasks requiring motion awareness, memory, and physical sensing. It also analyzes the effects of architecture, data, post-training, reinforcement learning, and inference optimization on manipulation performance and latency.

6.1. Simulation Experiments

  • 97.8% on LIBERO, 86.7% on LIBERO-Plus, and 77.4% on SIMPLER Google-VA; RLDX-1 outperformed frontier models across simulated benchmarks.It also scored 58.7% on GR-1 Tabletop and 32.1% on RoboCasa365, including 19.0% and 5.6% on seen and unseen composite tasks, respectively.Source: Table 1 · p. 18, §6.1 · pp. 18–19

6.2. Real-World Experiments: OpenArm Humanoid

  • 50% on Basic Pick-and-Place versus π0.5 at 41.7% and GR00T N1.6 at 37.5%; RLDX-1 also reached 87.5% on Object Grounding versus GR00T N1.6 at 33.3%.RLDX-1 also led on directional placement and generalized to unseen objects and tasks.Source: Figure 14 · p. 20, §6.2 · p. 20

6.3. Real-World Experiments: ALLEX Humanoid

  • 100% on seen and 75% on unseen conveyor speeds, 91.7% Object-in-Box Selection, 97.2 progress on Card Slide-and-Pick, and 70.8 progress on Pot-to-Cup Pouring.RLDX-1 outperformed baselines across ALLEX tasks; at unseen conveyor speeds it adapted its action tempo, and it used cup weight changes to proceed after pouring.Source: Figure 16 · p. 22, §6.3 · p. 22

6.4. Real-World Experiments: Franka Research 3

  • 97.9% on Spin Tracking, 81.5% on Pong Game, 45.8% on Cup Swapping, 91.7% on Shell Game, 33.3% on Plug Insertion, and 61.1% on Egg Pick-and-Place.RLDX-1 outperformed baselines across FR3 tasks; the reported results associate its contact-rich manipulation gains with tactile and force/torque information.Source: Figure 18 · p. 24, §6.4 · p. 24

6.5. Ablation and Analysis

  • On RoboCasa Kitchen, Layer 18 features yielded 60.9% success, versus 51.1% with Layer 8 and 56.3% with Layer 28, making feature-layer selection the largest-impact VLM ablation.The ablation variants were trained from scratch without pre-training.Source: §6.5 · p. 24, Table 2 · p. 25
  • Removing robot-specific VQA training lowers success from 60.9% to 57.5%, while post-training attention concentrates more on the robot body and target objects.The analysis attributes the effect to robot-specific VQA supervision refining VLM features into more action-relevant representations; intermediate VLM representations balance semantic abstraction with fine-grained visual grounding.Source: §6.5 · p. 25
  • 50.1% success at full synthetic scale, up from 41.0% with real data alone, shows gains as synthetic GR-1 data increases in the tabletop benchmark.The controlled comparison fixed the real GR-1 dataset and used a 1:1 real-to-synthetic sampling ratio; synthetic data also improved spatial generalization on a specific task.Source: §6.5 · p. 25
  • 353 ± 22 frames and 4.1 ± 0.3 attempts let RECAP3 outperform BC (1056 ± 326 frames; 12.7 ± 3.0 attempts) and even surpass teleoperation on bulb twisting.RECAP improved speed and robustness over BC at every iteration; the smaller standard deviations for RECAP2 and RECAP3 indicate greater trial-to-trial consistency. The bulb-twisting task requires turning a socketed bulb until it lights.Source: §6.5 · p. 26, Figure 21 · p. 26
  • 1.61× and 1.63× cumulative speedups yield 41.6 ms and 43.7 ms per-step latency for the variant without physics and memory and the all-modality model, respectively.Static graph conversion reaches 1.45× and 1.46× speedups over eager execution, and operator fusion raises them further; Torch Compile reaches 1.18× and 1.19×. The gains are nearly identical across variants, supporting robustness to the added memory module and physics stream.Source: §6.5 · p. 26, Table 4 · p. 26

7. Related Work

Prior robotics work advances versatile language-conditioned policies, synthetic-data scaling, and individual manipulation capabilities, but leaves practical dexterous manipulation on humanoids and joint use of multiple capabilities open. RLDX-1 addresses these gaps with an end-to-end policy combining functional capabilities and a synthetic-data framework for ALLEX dexterous manipulation.

Foundation Models in Robotics

  • End-to-end VLAs unify perception, language-conditioned reasoning, and action generation, but RLDX-1 argues that versatility alone is insufficient for real-world manipulation.Prior VLA designs generate discretized action tokens or use dedicated flow-matching action decoders; more recent work adds future video or state prediction, while RLDX-1 combines versatile action generation with functional capabilities.Source: §7 · p. 27

Synthetic Data for Robot Learning

  • RLDX-1 targets synthetic dexterous-manipulation data for realistic ALLEX humanoid scenarios, extending video-based augmentation beyond prior work largely limited to tabletop pick-and-place.The proposed data includes scenarios such as pot pouring and bulb twisting, and remains beneficial when combined with torque-annotated real data despite containing no torque signals itself.Source: §7 · p. 27

Functionalities for Robot Manipulation

  • RLDX-1 unifies motion awareness, long-term memory, and physical sensing in one end-to-end policy, whereas prior approaches generally address these capabilities individually.Motion awareness work uses multi-frame context, key-frame selection, sequence summaries, or compressed observations; memory approaches process past observations or store them in queues; physical-sensing work uses one or more physical signals. Real-world manipulation can require these capabilities jointly.Source: §7 · p. 27

8. Conclusion … Model Training Pipeline

RLDX-1 combines functional capabilities with deployment-oriented data, training, and inference methods, and the conclusion reports stronger performance across humanoid and manipulation benchmarks.

8. Conclusion

  • Approximately 90% success on ALLEX humanoid tasks, versus frontier VLAs around 40%, accompanies consistent wins on every simulation and real humanoid benchmark requiring versatile intelligence.The reported results position RLDX-1 as a step toward general-purpose policies for human-like dexterous manipulation in complex real-world environments.Source: §8 · p. 28
  • RLDX-1 unifies motion awareness, long-term memory, and physical sensing, using synthetic dexterous data, three-stage training, and inference optimization for responsive control.Source: §8 · p. 28

Project Leads

  • Dongyoung Kim sub-led data and model training; Huiwon Jang sub-led data and architecture; Myungkyu Koo led architecture; Suhyeok Jang led synthetic data; Taeyoung Kim led training.Source: Project Leads · p. 37

Research Leads

  • Dongyoung Kim, Huiwon Jang, and Jinwoo Shin led the overall research direction and cross-area coordination.Source: Research Leads · p. 37

A.2. Core Contributors

  • Core contributors are grouped by area, listed alphabetically by first name, and repeated across groups when they held multiple roles.Source: §A.2 · p. 37

Model Architecture

  • Daewon Choi and Kyungmin Lee contributed to early MSAT design; Heeseung Kwon built the motion module; Jimin Lee integrated physical sensing; Seungcheol Park validated it.Source: Model Architecture · p. 37

Synthetic Data

  • Byungjun Yoon, Changsung Jang, John Won, Junyoung Sung, Minseong Han, and Sejune Joo contributed task/filtering, synthetic generation, evaluation, or validation work.Contributions included task and video-quality filtering; inverse-dynamics-based data generation and ALLEX motion-consistency filtering; scene, GR-1, and ALLEX data generation; and real-world validation experiments.Source: Synthetic Data · p. 37

Model Training Pipeline

  • The training-pipeline team optimized distributed VLM/VLA training, refactored code, and enabled baseline ALLEX demos and automated evaluation.Source: Model Training Pipeline · p. 37

Real-World Robot Evaluation & Demonstration … B.1. Image Preprocessing

The supplied sections document RLDX-1’s real-world evaluation, training and deployment infrastructure, data pipelines, and project contributions. They also describe image resizing that limits each frame to at most 64 vision tokens.

Real-World Robot Evaluation & Demonstration; Reinforcement Learning

  • Real-world experiments covered motion, memory, and physical sensing, while the ALLEX RECAP pipeline supported reinforcement-learning evaluation and task-specific data refinement.Experiments were designed for OpenArm, ALLEX, and FR3; ALLEX data strategies considered consistency and variance.Source: Real-World Robot Evaluation &… · p. 38, Reinforcement Learning · p. 38

Inference Optimization & Deployment; Training Infrastructure

  • Inference deployment combined asynchronous processing, static-graph conversion, CUDA Graphs, Triton and fused-attention kernels, alongside training and evaluation infrastructure for real and simulated environments.GraphSafe conversion addressed graph-break sources, and the infrastructure included teleoperation-based data collection and dataset management.Source: Inference Optimization & Deployment · p. 38, Training Infrastructure · p. 38

Robot Control System; Robot Hardware Research & Prototyping

Teleoperation System; Teleoperation Data Collection

  • Teleoperation systems supplied camera processing, video streaming, VR integration, and ALLEX operating support, while teams collected teleoperation data for ALLEX, OpenArm, and Franka.An automated training-inference-evaluation pipeline was also established for ALLEX simulations.Source: Teleoperation System · p. 39, Teleoperation Data Collection · p. 39

Human Data Pipeline; Live Demo Production

  • Human-data work established collection, processing, retargeting, and training pipelines, and live-demo teams collected task data and fine-tuned RLDX-1.The early processing and retargeting work laid a foundation for integrating human demonstrations.Source: Human Data Pipeline · p. 39, Live Demo Production · p. 39

Dexterity Benchmark for Industry Conversion; Project Coordination & Public Release; A.3. Contributors; Executive Sponsorship

B. Datasets Details; B.1. Image Preprocessing

  • Offline preprocessing applies smart_resize while preserving aspect ratio and rounding dimensions to multiples of 32, limiting each frame to at most 64 vision tokens.The target image area is capped at 256^2 pixels to reduce the vision-token budget during training.Source: B. Datasets Details · p. 41, §B.1 · p. 41

B.2. Public Real-World Real Data Details · B.3. Synthetic Data Details · C. Reinforcement Learning (RL) Details

These sections document RLDX-1’s OXE pre-training mixture, synthetic-video data generation and labeling, and RECAP reinforcement-learning post-training and inference procedures.

B.2. Public Real-World Real Data Details

B.3. Synthetic Data Details

  • Pre-trained video generative models (Ali et al., 2025) generate novel trajectories from initial images unseen in existing datasets; they are fine-tuned on robot demonstrations into dedicated models for GR-1 and ALLEX humanoids.Off-the-shelf video diffusion models are insufficient to capture egocentric robot viewpoints and manipulation dynamics.Source: §B.3 · p. 42
  • The GR-1 video model uses Cosmos-Predict2-14B, fine-tuned on ActionNet and public GR-1 videos with LoRA.Training used 3,027 ActionNet videos and 92 public GR-1 videos, with 93-frame clips at 432×768 resolution for 10K steps and batch size 4.Source: §B.3 · p. 42
  • The ALLEX video model uses Cosmos-Predict2.5-2B and mixes in-house ALLEX and OpenArm data with ActionNet episodes at a 2:1:1 ratio.It uses short, medium, and long recaptions and uniform or random 93-frame clip sampling; training runs for 20K steps at 432×768 resolution with batch size 8.Source: §B.3 · p. 42
  • An inverse dynamics model provides pseudo-action labels for generated videos, while video filtering checks instruction following and trajectory plausibility.The IDM is a 0.1B Diffusion Transformer with a SigLIP-2 encoder; the ALLEX model is trained from scratch on in-house teleoperation data. Quality assessment uses 16 frames for instruction following and 8 for trajectory plausibility; the attentive probe uses 16-frame clips at 256×256 resolution with stride 4.Source: §B.3 · p. 42

C. Reinforcement Learning (RL) Details

  • The gemma3-4b-it critic (Gemma Team, 2025) trains on target-task success demonstrations, while pre-trained RLDX-1 trains for up to 30K steps with its vision encoder frozen.The critic uses LoRA rank 128 for 1 epoch at learning rate 1 × 10^-4; policy training uses global batch size 128, a warmup-plus-cosine schedule, and action chunks of size 40.Source: C. Reinforcement Learning (RL) Details · p. 43
  • RECAP initializes a VLM critic and policy from demonstrations, then iteratively adds policy rollouts, trains the critic on successful trajectories, and refines the policy using re-annotated advantages.The algorithm repeats rollout collection and critic-policy refinement across iterations.Source: C. Reinforcement Learning (RL) Details · p. 43
  • At inference, RLDX-1 executes the first 20 actions of each predicted chunk, then re-queries the policy and replans using RTC.The report applies the same Real Time Chunking configuration as in the other experiments.Source: C. Reinforcement Learning (RL) Details · p. 43

D. Parameter-Efficient Fine-Tuning … E.2. Implementation Details

RLDX-1 can be adapted efficiently with LoRA, while test-time Best-of-N sampling uses a conservative chunk-level critic and a frozen post-RECAP1 backbone. The implementation details specify the critic architecture, optimization, and inference procedure.

D. Parameter-Efficient Fine-Tuning

  • At rank 64, action-model LoRA matches full fine-tuning’s 62.67% success rate while reducing trainable parameters from 2,376.0M to 1,150.5M.Joint LoRA on the top-4 backbone layers and action model achieves 55.33% success with 5.72% of total parameters and 35.93 GiB batch-32 VRAM versus 87.10 GiB; at batch size 1, it fits within 24 GiB. Table 6 averages over 24 tasks × 50 episodes and uses a single NVIDIA H200 GPU.Source: D. Parameter-Efficient Fine-Tuning · p. 44, Table 6 · p. 44
  • Freezing the backbone limits adaptation to 36.42% at best, 26.25 points below full fine-tuning, indicating that backbone updates are essential under downstream distribution shift.Source: D. Parameter-Efficient Fine-Tuning · p. 44

E.1. Test-time Sampling Technique

  • Best-of-N selects the highest-Q action chunk from sampled candidates, using a conservative IQL-style critic to address offline-to-post-RL distribution mismatch (Kostrikov et al. 2022).The critic conditions on VLA vision-language features and scores temporally extended chunks, using in-chunk and chunk-level discounts (Zhao et al., 2023; Li et al., 2025b); the approach builds on BoN sampling (Chen et al., 2023; Nakamoto et al., 2024) and DEAS (Kim et al., 2026a).Source: §E.1 · p. 45
  • At inference, RLDX-1 samples chunks with temperature 1.5–2.0 and executes the highest-Q candidate; VLA critic training uses τ=0.7, γ1=0.9, γ2=0.99, and chunk length H + 1 = 16.The critic uses AdamW at 1×10−4 with batch size 64 for 30K RoboCasa steps and 10K real-robot steps.Source: §E.1 · p. 45

E.2. Implementation Details

  • The critic uses pooled final-layer backbone tokens projected to a 64-dimensional embodiment-conditioned embedding, concatenated with proprioception; Q additionally evaluates action chunks.V and Q use BRONet with four pre-norm residual blocks of width 256; twin critics use min(Q1, Q2) for targets and BoN scoring, and support up to 36 embodiment slots.Source: §E.2 · p. 45
  • Critic training caches frozen post-RECAP1 backbone features and optimizes with AdamW at 3×10−5 for 10k steps, batch size 64, and weight decay 10−5.Training also uses 5% linear warmup and gradient clipping at 1.0; rewards are shifted by −1, and the categorical critic uses 101 atoms with Gaussian smoothing σ=0.75 bin-widths.Source: §E.2 · p. 45
  • The critic follows DEAS hyperparameters γ1=0.9, γ2=0.99, τ=0.7, and target-critic Polyak rate 0.005.The shifted reward support is [−100, 0], with scalar Q and V decoded as expectations over bin centers.Source: §E.2 · p. 45
  • BoN rescales flow-matching initial noise with temperature T∈[1.5, 2.0] and tiles observations across candidates so one backbone forward pass serves all N chunks.The stated sweet spot is T=1.5; noise is sampled from N(0, T^2I) rather than N(0, I).Source: §E.2 · p. 45

E.3. Experimental results · F. Kernel Optimization · G. Evaluation & Analysis

Best-of-N sampling improved Light Bulb Twisting performance for the less-converged RECAP1 policy but degraded it for RECAP2 and RECAP3; the kernel section describes fused-kernel comparisons. No substantive G. Evaluation & Analysis passage is supplied.

E.3. Experimental results

  • Offline RECAP3 analysis found Q-value separation between the best-of-N chunk and the mean became clear only at T > 2, while higher temperatures were unsafe on the physical robot.Action diversity grew steadily with temperature, and the Q improvement was distributed across episode frames; T = 1.5 was chosen as a practical compromise.Source: §E.3 · p. 46, Figure 22 · p. 46
  • Increasing RECAP1 sampling from N = 8 to 32 did not improve performance, changing attempts from 4.9 to 5.7.The report concludes that the benefit saturates quickly rather than resulting simply from a wider search.Source: §E.3 · p. 46, Figure 23 · p. 46
  • BoN acts primarily as exploration: it can recover modes for less-converged policies but introduce harmful stochasticity in well-converged policies, complementing rather than replacing further RL training.The behavior is described as consistent with observations in other domains (Yue et al., 2025).Source: §E.3 · p. 46, Figure 23 · p. 46

G. Evaluation & Analysis

  • No substantive passage for G. Evaluation & Analysis is included in the supplied excerpts.Source: §E.3 · p. 46

E.3. Experimental results

  • With BoN (N = 8, T = 1.5), RECAP1 reduced Light Bulb Twisting attempts from 8.5 ± 2.8 to 4.9 ± 1.3 (−3.6), while RECAP2 and RECAP3 increased attempts by +2.3 and +2.2.The figure caption summarizes improvement for RECAP1 and degradation for RECAP2 and RECAP3.Source: Figure 23 · p. 46, §E.3 · p. 46

F. Kernel Optimization

  • Table 7 compares original unfused operator sequences with corresponding fused kernels used in the system.The supplied passages do not provide the table’s individual kernel entries or performance values.Source: Table 7 · p. 47, F. Kernel Optimization · p. 47

G.1. Simulation Benchmark Details

RLDX-1 is fine-tuned and evaluated across varied simulation benchmarks, spanning single-arm manipulation, humanoid tabletop tasks, and household environments. The evaluation protocols and training settings are adapted to each benchmark.

G.1. Simulation Benchmark Details

  • LIBERO (Liu et al., 2023) covers 40 Franka tasks across four suites with 50 trials/task; LIBERO-Plus (Fei et al., 2025) evaluates 10,300 perturbation tasks.LIBERO-Plus tests robustness to object layout, camera viewpoint, robot initial state, language instruction, lighting, background texture, and sensor noise.Source: §G.1 · p. 48
  • SIMPLER Google-VM uses 200 trials/task and VA 100 trials/perturbation; SIMPLER WidowX averages 200 trials/task across four tasks (Li et al., 2024b).RLDX-1 is trained on Fractal for Google-VM/VA and BridgeV2 for WidowX; Google-VM/VA uses an ego-centric view, while WidowX uses a third-person camera.Source: §G.1 · p. 48
  • RoboCasa Kitchen (Nasiriany et al., 2024) covers 24 mobile-Panda kitchen tasks, using 300 generated demonstrations/task and 50 evaluation episodes/task.Evaluation uses two fixed external views and a wrist camera, each at 256×256 resolution.Source: §G.1 · p. 48
  • GR-1 Tabletop (Bjorck et al., 2025) evaluates 24 humanoid tasks using 1,000 generated demonstrations/task and average success over 50 episodes/task.Its task set includes 18 object-rearrangement tasks and six articulated-object manipulation tasks.Source: §G.1 · p. 48
  • RoboCasa365 (Nasiriany et al., 2026) trains on 300 human-pretraining tasks and evaluates 50 tasks across Atomic-Seen, Composite-Seen, and Composite-Unseen splits.The training dataset provides 100 demonstrations per task, and evaluation follows the official leaderboard protocol.Source: §G.1 · p. 48
  • Implementation fine-tunes pretrained RLDX-1 with the vision encoder and most LLM layers frozen, then tunes hyperparameters by benchmark and reports the best configuration.The default is 60K steps, global batch size 1024, and AdamW at 1 × 10^-4; exceptions include 20K steps for Google-VM/VA, 250K steps and batch size 196 for RoboCasa365, 0.5 state dropout for Google-VM/VA and GR-1, and 10 denoising steps for WidowX and RoboCasa Kitchen. GR-1 uses Beta(1.5, 1.0) timestep sampling (58.7% vs 58.0%), while other benchmarks use fixed denoising timesteps; action horizons use two chunks for SIMPLER and eight for LIBERO, LIBERO-Plus, and GR-1.Source: §G.1 · p. 49, Table 8 · p. 49

G.2. OpenArm Experiments Details · G.3. ALLEX Experimental Details · G.4. Franka Research 3 Experimental Details

The three experimental sections specify platform-specific task protocols, demonstration datasets, and sensing setups for OpenArm, ALLEX, and Franka Research 3. Their evaluations vary by task, including functional manipulation, humanoid coordination, and memory- or sensing-dependent interactions.

G.2. OpenArm Experiments Details

  • 348 demonstrations cover OpenArm’s four language-conditioned pick-and-place task families, with 24 evaluation trials per task and 48 for Directional PnP (Shelf).Tasks use a target and two distractors in a three-object tabletop scene; some configurations require bimanual coordination.Source: §G.2 · p. 50
  • OpenArm uses two 7-DoF arms, two 6-DoF hands, and neck-mounted stereo egocentric views; ExoArm-7 and Manus Pro gloves capture arm and finger demonstrations.The neck remains fixed during data collection.Source: §G.2 · p. 50

G.3. ALLEX Experimental Details

  • 86.8% is the ALLEX humanoid success rate reported in the paper context; these experimental details evaluate ALLEX tasks with task-specific trials, including speed, position, and hardware-safety conditions.Conveyor Pick-and-Place uses six trials at each of four speeds, Object-in-Box Selection uses eight trials at each of three positions, and Card Slide-and-Pick is evaluated at the lower training position for safety.Source: §G.3 · p. 51
  • ALLEX training uses 40–90 demonstrations per task; pouring uses 62 demonstrations with small plastic balls instead of liquid for safety.The other task datasets contain 90 Object-in-Box Selection demonstrations and 72 Card Slide-and-Pick demonstrations.Source: §G.3 · p. 51
  • ALLEX uses 48 manipulation joints and head-mounted stereo egocentric vision, with estimated joint torques from all 48 DoF as physical signals.No wrist or side cameras are used.Source: §G.3 · p. 51

G.4. Franka Research 3 Experimental Details

  • Franka Research 3 evaluation uses task-specific protocols, including 96 Spin Tracking interactions and 54 unseen Pong Game interactions, with success rate measured.Other protocols test Cup Swapping, Shell Game, Plug Insertion, and Egg Pick-and-Place across specified initial positions.Source: §G.4 · p. 52
  • Franka Research 3 pairs a 7-DoF arm and gripper with tactile and joint-torque sensing for the task experiments.The left-gripper tactile input is a 15-dimensional feature vector from five three-axis sensing units; joint torque input is a 7-dimensional vector in Newton-meters.Source: §G.4 · p. 52

G.5. Additional ALLEX Experiments

On ALLEX Pot-to-Cup Pouring, co-finetuning with synthetic data raised overall success from 66.7% to 83.3%, with fewer failures and more full successes across varied object positions.

G.5. Additional ALLEX Experiments

  • Evaluation covered 24 grasping trials across 3 cup and 4 coffeepot positions, with 2 trials per configuration and positions spaced approximately 5 cm and 9 cm apart, respectively.A trial scored 1.0 for grasping both objects, 0.5 for grasping only the cup, and 0.0 otherwise.Source: §G.5 · p. 53
  • Synthetic training data came from two-round, subtask-prompted multi-turn ALLEX video rollouts, alongside teleoperated demonstrations collected at positions sampled around a nominal location.The synthetic turns reused preceding final frames as conditioning; only video-quality filtering was applied, without task or scene augmentation.Source: §G.5 · p. 53
  • The model was fine-tuned for 30K steps, and co-finetuning mixed real and synthetic data equally while routing each dataset to its embodiment slot.Training used batch size 128, AdamW at learning rate 1 × 10−4 with 5% warmup, state dropout 0.4, and action horizon 40.Source: §G.5 · p. 53
  • Co-finetuning with synthetic data raised overall success from 66.7% to 83.3%, reduced failures from 7 to 2, and increased full successes from 9 to 14.The task comprised 24 trials, scored as full success (1.0), partial success (0.5), or failure (0.0).Source: §G.5 · p. 53, Table 9 · p. 53

G.6. Additional Simulation Results / Full Results

Larger training batches improve RLDX-1’s simulation benchmark performance, and the section provides full results across several simulation benchmarks. RoboCasa Kitchen per-task results span success rates from 22.0 to 92.2.

Effect of Batch Size

  • Larger training batch sizes improve RLDX-1 performance on simulation benchmarks.The comparison is presented in Table 10.Source: §G.6 · p. 54, Table 10 · p. 54

Full Results

  • The full results cover LIBERO, LIBERO-Plus, SIMPLER, RoboCasa Kitchen, GR-1 Tabletop, and RoboCasa365 benchmarks.The tables report results for RLDX-1; the captions identify the cited benchmark works as Liu et al. (2023), Fei et al. (2025), Li et al. (2024b), Nasiriany et al. (2024), Bjorck et al. (2025), and Nasiriany et al. (2026).Source: §G.6 · p. 54, Table 11 · p. 54, Table 12 · p. 54
  • RoboCasa Kitchen per-task success rates range from 22.0 for Close Blender Lid to 92.2 for Open Stand Mixer Head.Other listed rates are Close Fridge 88.0, Close Toaster Oven Door 60.0, Coffee Setup Mug 40.0, Navigate Kitchen 72.0, Open Cabinet 54.0, Open Drawer 74.0, PnP Counter to Cabinet 68.0, PnP Counter to Stove 74.0, PnP Drawer to Counter 76.0, PnP Sink to Counter 86.0, PnP Toaster to Counter 92.0, Slide Dishwasher Rack 76.0, Turn Off Stove 30.0, Turn On Electric Kettle 80.0, Turn On Microwave 54.0, Turn On Sink Faucet 72.5.Source: §G.6 · p. 55, Table 13 · p. 55

Related papers

Loading 2605.03269v2…