PaperlayerView exact revision on arXiv

Source-linked AI summary

FastUMI-100K: Advancing Data-driven Robotic Manipulation with a Large-scale UMI-style Dataset

Kehui Liu, Zhongjie Jia, Yang Li, Zhaxizhuoma, Pengan Chen, Song Liu, Xin Liu, Pingrui Zhang, Haoming Song, Xinyi Ye, Nieqing Cao, Zhigang Wang, Jia Zeng, Dong Wang, Yan Ding, Bin Zhao, Xuelong Li

arXiv:2510.08022v1cs.ROcs.AI

TL;DR

Robotic manipulation learning needs larger, higher-quality demonstrations because existing teleoperation-centered datasets are difficult to scale and limited in smoothness, task complexity, and embodiment reuse. FastUMI-100K introduces over 100,000 multimodal trajectories collected with a modular, hardware-decoupled FastUMI system across 54 household tasks and hundreds of objects. Experiments report high policy success rates across imitation-learning and VLA settings, while supporting cross-embodiment transfer and long-horizon manipulation.

Abstract

from arXiv · show

Data-driven robotic manipulation learning depends on large-scale, high-quality expert demonstration datasets. However, existing datasets, which primarily rely on human teleoperated robot collection, are limited in terms of scalability, trajectory smoothness, and applicability across different robotic embodiments in real-world environments. In this paper, we present FastUMI-100K, a large-scale UMI-style multimodal demonstration dataset, designed to overcome these limitations and meet the growing complexity of real-world manipulation tasks. Collected by FastUMI, a novel robotic system featuring a modular, hardware-decoupled mechanical design and an integrated lightweight tracking system, FastUMI-100K offers a more scalable, flexible, and adaptable solution to fulfill the diverse requirements of real-world robot demonstration data. Specifically, FastUMI-100K contains over 100K+ demonstration trajectories collected across representative household environments, covering 54 tasks and hundreds of object types. Our dataset integrates multimodal streams, including end-effector states, multi-view wrist-mounted fisheye images and textual annotations. Each trajectory has a length ranging from 120 to 500 frames. Experimental results demonstrate that FastUMI-100K enables high policy success rates across various baseline algorithms, confirming its robustness, adaptability, and real-world applicability for solving complex, dynamic manipulation challenges. The source code and dataset will be released in this link https://github.com/MrKeee/FastUMI-100K.

I. INTRODUCTION

Existing robotic manipulation datasets remain difficult to scale and often underrepresent complex, fine-grained, dual-arm, and long-horizon tasks. FastUMI-100K addresses these gaps with over 100,000 multimodal demonstrations collected across diverse tasks and objects.

Motivation

  • Data-driven imitation learning is important for generalized manipulation policies, but collecting the required high-quality data is time-consuming and labor-intensive.Source: §I · p. 1
  • Existing robot datasets remain relatively small, while laborious teleoperation concentrates on simpler tasks and struggles with complex, fine-grained, dual-arm, and long-horizon manipulation.Source: §I · p. 1

Contribution

  • Over 100,000 demonstration trajectories were collected with FastUMI using both single-arm and dual-arm grippers, equivalent to 600 hours of interactive data.Collection used 10 data collectors, 3 technical support staff, and 5 standardized environments.Source: §I · p. 1

Dataset scope

  • FastUMI-100K integrates single-arm and dual-arm trajectories, wrist-mounted fisheye images, and fine-grained textual annotations across 54 tasks and hundreds of household objects.Source: §I · p. 1, Figure 1 · p. 2

II. RELATED WORK

Prior work spans handheld interfaces, teleoperation, and large cross-robot corpora, but these approaches differ in embodiment coupling, sensing, and reuse. FastUMI builds on UMI-style handheld collection to support hardware-decoupled transfer.

Handheld collection

Teleoperation

FastUMI

Large-scale datasets

III. DATA COLLECTION AND PROCESSING

FastUMI-100K uses standardized, modular hardware to collect synchronized multimodal data from single-arm and dual-arm configurations. Its pipeline combines tracking, fisheye imaging, interchangeable fingertip attachments, and organized collection operations.

Hardware Design

  • Dual-arm FastUMI integrates two T265 trackers and two GoPro fisheye cameras on the grippers to synchronously record multimodal data at 20 Hz.Source: A. Hardware Design · p. 3
  • The standardized plug-and-play fingertip attachment enables collected data to adapt directly to robotic grippers equipped with compatible accessories.Source: A. Hardware Design · p. 3

Data Collection

  • The collection team comprises 10 data collectors and 3 technical support staff organized into five two-person subgroups for collection, program switching, visualization, and quality monitoring.Source: B. Data Collection Pipeline · p. 3

Pre-Data Collection

2) During Data Collection:

During collection, FastUMI combines fixed initialization points, voice guidance, and real-time visualization to standardize operation and detect recording problems. A monitoring system also invalidates trajectories showing tracking anomalies.

During Data Collection

  • Fixed 3D-printed slots initialize handheld devices and T265 cameras at consistent scene positions, supporting accurate relative-arm positioning throughout trajectories.Source: 2) During Data Collection · p. 3
  • Real-time GoPro and T265 visualization lets operators monitor frame drops and trajectory drift during data collection.Source: 2) During Data Collection · p. 3

Quality Monitoring

  • Quality monitoring detects sudden trajectory jumps or global scale distortion from T265 data by checking adjacent-point linear and angular velocities, then marks affected trajectories invalid.Source: 3) Post-Data Collection · p. 3

C. Multi-Device Multi-Sensor Alignment

FastUMI synchronizes heterogeneous dual-arm sensor streams with a unified ROS clock and timestamp-based matching. The pipeline combines precise RGB alignment with pose matching to produce temporally consistent multimodal data.

C. Multi-Device Multi-Sensor Alignment

  • A unified ROS clock assigns consistent timestamps across four sensors operating at different sampling rates, including 60Hz GoPro cameras and 200Hz T265 sensors.The method extends FastUMI’s single-arm multisensor alignment technique to dual-arm collection.Source: C. Multi-Device Multi-Sensor Alignment · p. 3
  • RGB frames are synchronized by timestamps, downsampled uniformly to 20Hz, and paired with the temporally closest T265 pose data.The maximum RGB alignment error is half a GoPro sampling period, or 1/120 seconds.Source: C. Multi-Device Multi-Sensor Alignment · p. 4
  • The resulting procedure achieves sub-millisecond alignment of dual-arm multimodal streams for downstream learning.This synchronization provides a stable dataset across image and pose modalities.Source: C. Multi-Device Multi-Sensor Alignment · p. 4
  • FastUMI complements sensor alignment with a two-stage annotation workflow that divides long-horizon tasks into phases and describes motion-specific manipulation details.GPT-4o proposes subtasks from video, after which a custom GUI supports key-frame extraction and manual matching.Source: D. Data Annotation · p. 4, 1) Subtask-Level Annotation · p. 4

2) Motion-Level Annotation:

FastUMI-100K adds motion-level language annotations by converting frame-to-frame pose and gripper changes into directional, rotational, and opening/closing descriptions. This produces a detailed semantic connection between task phases and robot motions.

2) Motion-Level Annotation:

  • Motion annotations analyze each frame against the tenth subsequent frame to derive relative translation, rotation, and gripper-opening changes.These measurements are converted into basic motion units such as directional movement, rotation, and gripper opening or closing.Source: 2) Motion-Level Annotation · p. 4
  • The annotation system combines task-level subtasks with motion-level descriptions, creating a semantic chain from manipulation objectives to detailed movements.Cross-sample cross-validation yields 15,000 textual annotations intended to ensure semantic reliability.Source: 2) Motion-Level Annotation · p. 4
  • FastUMI-100K contains over 100,000 trajectories spanning 54 daily household task categories and hundreds of operational objects.The dataset uses single-arm and dual-arm FastUMI configurations and is designed to remain decoupled from specific robot embodiments.Source: §IV · p. 4
  • Its task design emphasizes diverse household manipulation beyond simple pick-and-place, treating the dataset as an atomic skill library for robotic manipulation.The task categories include basic, composite, hinged, and flexible-object manipulations.Source: A. Diversity of Task Types · p. 4
  • Figure 4 summarizes task statistics, including the distribution of six task types and selected dual-arm data comparisons.The figure also compares average linear and angular velocities for five dual-arm task types from FastUMI-100K and AgiBot.Source: Figure 4 · p. 5

B. Diversity of Objects

FastUMI-100K covers heterogeneous household objects and environments while retaining trajectories whose durations reflect task complexity. Its collection scheme produces compact, smooth sequences and shorter collection times than teleoperation.

B. Diversity of Objects

  • FastUMI-100K includes rigid, flexible, and articulated objects across living rooms, studies, bathrooms, and kitchens.Examples range from clothing, books, and cosmetics to appliances, tableware, plugs, and sockets.Source: B. Diversity of Objects · p. 5
  • The heterogeneous object set supports varied interactions intended to improve policy generalization across real-world scenarios.The passage links object diversity and interaction complexity to development of more generalizable policy models.Source: B. Diversity of Objects · p. 5
  • Trajectories span 6 to 25 seconds, corresponding to 120 to 500 frames after downsampling the original 60Hz videos to 20Hz.The range reflects differences in the information complexity and difficulty of individual trajectories.Source: C. Diversity of Task Horizon · p. 5
  • For the same manipulation task, FastUMI requires one-fifth the time of teleoperation collection, with clothes folding taking 10 seconds versus 50 seconds in AgiBot [11].The collection scheme avoids action delays and redundant command transmission, producing more temporally compact sequences.Source: C. Diversity of Task Horizon · p. 5

D. Flexibility and Human-like Motion Characteristics

FastUMI records direct, first-person gripper interaction with tactile feedback rather than relying on complex teleoperation setups. Compared with AgiBot [11], its trajectories are reported to more closely match human manipulation motion characteristics.

D. Flexibility and Human-like Motion Characteristics

  • FastUMI data is collected through direct gripper manipulation without VR devices or complex teleoperation setups.The operator interacts from a first-person perspective while receiving immediate tactile feedback from physical contact.Source: D. Flexibility and Human-like Motion… · pp. 5–6
  • Table I reports Diffusion Policy success rates across different tasks and platforms.The supplied table passage identifies the evaluation scope but does not provide individual success-rate values.Source: Table I · p. 6
  • Table II compares success rates for different ACT variants across representative tasks.The supplied table passage identifies the comparison scope but does not provide individual success-rate values.Source: Table II · p. 6
  • Compared with AgiBot [11], FastUMI shows motion characteristics closer to human manipulation in trajectory and angular variation per unit time.The comparison covers five dual-arm task types and examines average linear and angular velocities.Source: D. Flexibility and Human-like Motion… · p. 6

V. EXPERIMENTAL EVALUATION

FastUMI-100K is evaluated with single-task imitation, cross-platform transfer, and VLA fine-tuning experiments across diverse manipulation tasks. The results indicate reliable policy learning, embodiment-independent trajectory representation, and long-horizon evaluation on multiple robot platforms.

V. Experimental Evaluation

  • The experiments cover single-task learning, cross-embodiment transfer, and Vision-Language-Action model compatibility using Diffusion Policy, ACT, and π0.These evaluations assess policy training across single-task scenarios, robotic embodiments, and VLA models.Source: §V · p. 6

A. Single-task Imitation Learning

  • FastUMI-100K supports reliable single-task imitation learning across seven subtasks, including realistic and fine-grained manipulations, using Diffusion Policy.The dataset emphasizes realistic object manipulation, scene arrangements, and diversity beyond constrained pick-and-place settings.Source: A. Single-task Imitation Learning · p. 6
  • TCP pose representation captures trajectory shape and dynamics, while end-effector states provide embodiment-independent descriptions of dexterous, human-like motion.This comparison is drawn from the ACT training analysis in FastUMI [21].Source: A. Single-task Imitation Learning · p. 7

B. Cross-platform Deployment

FastUMI-100K is designed for deployment across different robotic embodiments and for joint training with conventional data in VLA models. Experiments show flexible cross-platform use, while long-horizon inference remains sensitive to accumulated errors and ambiguous observations.

B. Cross-platform Deployment

  • FastUMI-100K transfers Diffusion Policy models between Xarm6 and Flexiv Rizon4 without additional training, using coordinate mapping adjustments.The experiment evaluates transfer across distinct robotic brands and structures.Source: B. Cross-platform Deployment · p. 7
  • A three-dimensional bounding-box filter enables FastUMI-100K deployment across robotic platforms with different morphologies and workspace constraints.The filter removes demonstrations exceeding the manipulation range of smaller robot workspaces.Source: B. Cross-platform Deployment · p. 7

C. Fine-tuning on VLA Large Models

  • In long-horizon tasks, success declines after early stages because inference errors accumulate, and similar observations can trigger local failures between subtasks.The Heat Food example fails during the transition from opening the microwave door to grasping bread; the absence of a fixed third-person global view is identified as a contributing boundary.Source: C. Fine-tuning on VLA Large Models · p. 7

VI. CONCLUSION AND FUTURE WORK

FastUMI-100K addresses scalability and embodiment limitations in robotic manipulation datasets through a large, multimodal UMI-style collection. Its evaluations support single-task learning, cross-platform deployment, and VLA fine-tuning, while future work targets richer dual-arm and long-horizon capabilities.

VI. Conclusion and Future Work

  • FastUMI-100K contains over 100K demonstration trajectories spanning 54 household tasks and hundreds of objects, with multimodal data collected by a modular, hardware-agnostic FastUMI system.The dataset is intended to address insufficient scalability, trajectory smoothness, and limited adaptability across robotic embodiments.Source: §VI · p. 8
  • Experiments show high policy success rates for Diffusion Policy and ACT, cross-platform deployment across distinct robot architectures, and compatibility with π0 VLA fine-tuning.The reported cross-platform deployment requires only coordinate mapping adjustments.Source: §VI · p. 8
  • Future work will expand dual-arm collaborative manipulation, pursue faster and more human-like movements, and further fuse UMI-style and conventional data for complex long-horizon reasoning.These directions are stated as planned extensions of FastUMI-100K and VLA training.Source: §VI · p. 8

Related papers

Loading 2510.08022v1…