PaperlayerView exact revision on arXiv

Source-linked AI summary

GR-Dexter Technical Report

Ruoshi Wen, Guangzeng Chen, Zhongren Cui, Min Du, Yang Gou, Zhigang Han, Liqun Huang, Mingyu Lei, Yunfei Li, Zhuohang Li, Wenlei Liu, Yuxiao Liu, Xiao Ma, Hao Niu, Yutao Ouyang, Zeyu Ren, Haixin Shi, Wei Xu, Haoxiang Zhang, Jiajun Zhang, Xiao Zhang, Liwei Zheng, Weiheng Zhong, Yifei Zhou, Zhengming Zhu, Hang Li

arXiv:2512.24210v2cs.RO

TL;DR

VLA policies have enabled language-conditioned, long-horizon manipulation, but extending them to bimanual robots with dexterous hands remains challenging. GR-Dexter combines a compact hand, bimanual teleoperation, and co-training from multiple data sources; real-world evaluations report long-horizon task performance and generalization to unseen objects and instructions.

  • Problem

    Extending VLA policies from gripper-based systems to bimanual dexterous hands remains underexplored amid expanded control spaces, hand-object occlusions, and the need for diverse robot trajectories.

    Source: §1 · p. 1

  • Method

    GR-Dexter combines a compact 21-DoF hand and real-time bimanual teleoperation with a VLA policy co-trained on robot, vision-language, cross-embodiment, and human-trajectory data.

    Source: §1 · pp. 1–2

  • Results

    Real-world evaluations show long-horizon everyday task performance and generalization to unseen objects and previously unseen language instructions.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The hardware-model-data framework extends VLA-based generalist manipulation to high-DoF bimanual dexterous-hand robots in real-world evaluations.

    Source: §6 · p. 11

    Human-trajectory pretraining uses only a few hundred hours, and separate hand and arm control can hinder tight coordination in contact-rich behaviors.

    Source: §6 · p. 11

Abstract

from arXiv · show

Vision-language-action (VLA) models have enabled language-conditioned, long-horizon robot manipulation, but most existing systems are limited to grippers. Scaling VLA policies to bimanual robots with high degree-of-freedom (DoF) dexterous hands remains challenging due to the expanded action space, frequent hand-object occlusions, and the cost of collecting real-robot data. We present GR-Dexter, a holistic hardware-model-data framework for VLA-based generalist manipulation on a bimanual dexterous-hand robot. Our approach combines the design of a compact 21-DoF robotic hand, an intuitive bimanual teleoperation system for real-robot data collection, and a training recipe that leverages teleoperated robot trajectories together with large-scale vision-language and carefully curated cross-embodiment datasets. Across real-world evaluations spanning long-horizon everyday manipulation and generalizable pick-and-place, GR-Dexter achieves strong in-domain performance and improved robustness to unseen objects and unseen instructions. We hope GR-Dexter serves as a practical step toward generalist dexterous-hand robotic manipulation.

1 Introduction

GR-Dexter addresses the challenges of extending language-conditioned manipulation to bimanual robots with dexterous hands through a combined hand, teleoperation, and training approach. Real-world evaluations report long-horizon task performance and generalization to unseen objects and instructions.

1 Introduction

  • Extending VLA policies to dexterous hands remains challenging because high-DoF control expands the action space, hand-object occlusions complicate perception, and robot trajectories are costly to collect.The paper notes that VLA performance depends on the quality and diversity of dexterous bimanual robot trajectories.Source: §1 · p. 1
  • ByteDexter V2 is a compact, modular, linkage-driven anthropomorphic hand with 21 DoFs, an additional thumb DoF, and high-density fingertip tactile sensors.Its reported dimensions are 219 mm high and 108 mm wide.Source: §1 · p. 1
  • GR-Dexter is a VLA policy for a 56-DoF bimanual system, co-trained on teleoperated robot, vision-language, cross-embodiment, and human-trajectory data.A Meta Quest headset and Manus gloves track wrist poses and hand motions, which are retargeted to joint-position commands in real time.Source: §1 · p. 2
  • Anthropomorphic hand similarity allows GR-Dexter to use large-scale egocentric hand-object datasets, with on-robot trajectories used to fine-tune pretrained models for the target platform.The paper evaluates long-horizon manipulation and generalizable pick-and-place tasks.Source: §1 · p. 2
  • GR-Dexter performs long-horizon everyday manipulation and generalizes to unseen objects and instructions in real-world evaluations.The approach co-trains vision-language, cross-embodiment, and human-trajectory data to preserve in-domain grasping while improving out-of-distribution performance.Source: §1 · p. 2, Figure 2 · p. 3

2 ByteDexter Robotic Hand

ByteDexter V2 combines a compact, 21-DoF anthropomorphic hand with a bimanual robot platform and teleoperation system. The platform supports data collection and policy rollouts, while teleoperators demonstrate tasks from coarse manipulation to fine motor activity.

2 ByteDexter Robotic Hand

  • ByteDexter V2 is a linkage-driven 21-DoF hand, with four DoFs per finger and five in the thumb, and demonstrates all 33 Feix grasp types.The hand adds a thumb DoF while reducing overall size to 219 mm high and 108 mm wide.Source: §2 · p. 4

2.1 Hand Design

  • Finger PIP flexion is independently actuated, the thumb’s enlarged workspace supports opposition with all four fingers, DIP joints are coupled to PIP joints, and fingertip arrays measure normal contact forces.The five high-density tactile arrays resolve contact location and force across the fingertip, finger pad, and lateral surface.Source: §2.1 · p. 4

2.2 Bimanual System and Control

  • The 56-DoF platform pairs two Franka arms with ByteDexter V2 hands and uses four RGB-D views to capture hand-object interactions from multiple viewpoints.Teleoperation tracks wrist poses and hand movements with a Meta Quest setup and Manus gloves, retargeting motions to joint commands in real time.Source: §2.2 · p. 5
  • After minimal training, teleoperators complete tasks from coarse block-building to fine motor tasks such as knitting, supporting real-world data collection and policy evaluation.During policy rollout, action chunks are smoothed to support delicate grasping and transitions within and across chunks.Source: §2.2 · p. 5

3 The GR-Dexter Model

GR-Dexter combines a 4B-parameter VLA model for bimanual dexterous control with a training recipe that mixes robot, vision-language, cross-embodiment, and human data. A unified preprocessing and retargeting pipeline prepares heterogeneous trajectories for training.

3 The GR-Dexter Model

  • The 4B-parameter Mixture-of-Transformer policy generates k-length action chunks conditioned on language, visual observations, and robot state, with 88-dimensional actions for bimanual control.Actions include arm joints and poses, hand joints, and fingertip positions.Source: §3 · p. 6

3.1 Training Recipe

  • Training mixes web-scale vision-language data, cross-embodiment robot data, and human trajectories, with different losses applied to the VLM backbone and robot action model.Vision-language data use next-token prediction, while robot trajectories also train the action DiT with flow matching; unavailable or unreliable action dimensions are masked.Source: §3.1 · p. 6, Figure 6 · p. 6

3.2 Cross-Embodiment Motion Retargeting and Transferring

  • A unified pipeline standardizes visuals, filters trajectories, retargets robot data through fingertip alignment, and maps filtered human motions into a shared visual and kinematic representation.Robot trajectories are resampled by task category, while human trajectories are filtered for hand visibility and velocity.Source: §3.2 · p. 7

4 Experiments

Real-world experiments assess GR-Dexter on long-horizon bimanual manipulation and generalizable pick-and-place, including out-of-distribution layouts, objects, and instructions. Results show strong in-domain performance, improved generalization, and successful tool-use tasks.

4 Experiments

  • GR-Dexter shows strong long-horizon manipulation with 21-DoF hands and better generalization to unseen settings using a curated data mixture.Evaluations span dexterous tool use, long-horizon execution, and out-of-distribution scenarios.Source: §4 · p. 7

4.1 Long-Horizon Dexterous Manipulation

  • 0.89 vs 0.64: GR-Dexter vs plain VLA on unseen layouts; on seen layouts, their success rates are 0.97 and 0.96, respectively.The unseen-layout evaluation uses five layouts; seen-layout configurations appear in training.Source: §4.1 · p. 7, Figure 7 · p. 8
  • GR-Dexter also performs long-horizon tool use, including vacuuming with a four-finger grasp and thumb-operated controls and serving bread with tongs and a plate.The report describes reliable performance across both tasks.Source: §4.1 · p. 8

4.2 Generalizable Pick-and-Place

  • 0.93: GR-Dexter’s success rate on seen-object pick-and-place, compared with 0.87 for plain VLA and 0.85 without cross-embodiment data.The basic setting uses ten evaluation batches with five seen objects per batch.Source: §4.2 · p. 8
  • 0.85 and 0.83: final success rates for GR-Dexter on unseen objects and unseen instructions, respectively.These evaluations compare against plain VLA and GR-Dexter without cross-embodiment data; unseen-object tests use 23 objects.Source: §4.2 · pp. 8–9, Figure 8 · p. 9

5 Related Works

Related work spans dexterous-hand designs, VLA policies for dexterous manipulation, and datasets for bimanual control. GR-Dexter builds on these areas with an upgraded hand, mixed-data policy training, and a unified dataset pipeline.

5.1 Dexterous Robotic Hands

  • Linkage-driven hands offer durability, force transparency, and compact actuator integration; ByteDexter V2 adds a thumb DoF over V1 while reducing size and adding fingertip tactile sensing.The broader landscape includes commercial hands that commonly have six active DoFs and a smaller set with more than twelve.Source: §5.1 · pp. 9–10

5.2 VLA Models for Dexterous Hand Manipulation

  • Dexterous-hand VLA work faces higher control dimensionality and scarce teleoperated data; GR-Dexter extends VLA co-training to 21-DoF hands using robot, vision-language, cross-embodiment, and human data.Prior approaches include human-video pretraining and hierarchical planning with separate low-level execution.Source: §5.2 · p. 10

5.3 Bimanual Dexterous Manipulation Dataset

  • Existing datasets often focus on static single-hand grasps or lack language and whole-body trajectories; GR-Dexter combines public bimanual data, proprietary teleoperation, and human demonstrations through a standardized cleaning and retargeting pipeline.Teleoperated data offer precise robot states but can be limited in task diversity, while human recordings scale across scenes and tasks but have embodiment and state-estimation challenges.Source: §5.3 · p. 10

6 Limitations & Conclusions

GR-Dexter combines dexterous hardware, data collection, and cross-embodiment supervision, with strong in-domain performance and improved robustness to unseen objects and instructions. The authors identify limited human-trajectory data and separate hand–arm control as limitations.

6 Limitations & Conclusions

  • The system uses only a few hundred hours of human trajectories, and separate hand and arm control can hinder tight coordination in contact-rich behaviors.The authors identify broader cross-embodiment pretraining and embodiment-agnostic control abstractions as directions for future work.Source: §6 · p. 11
  • GR-Dexter achieves strong in-domain performance and improved robustness to unseen objects and instructions in real-world evaluations.Evaluations cover long-horizon everyday manipulation and generalizable pick-and-place.Source: §6 · p. 11
  • The 56-DoF bimanual policy is co-trained using teleoperated robot trajectories, vision-language data, cross-embodiment demonstrations, and human trajectories.The framework combines a compact anthropomorphic hand with an intuitive bimanual teleoperation pipeline.Source: §6 · p. 11

Contributions and Acknowledgements

The report credits separate teams for hardware, control and system, and model and data work. It also acknowledges support with data curation and system maintenance, and states that the work is for research purposes only.

Contributions and Acknowledgements

Grasping Capability

Figure 9 presents ByteDexter V2's grasping capability, including human-like grasping and a workspace supporting 33 grasp types.

Grasping Capability

Related papers

Loading 2512.24210v2…