PaperlayerView exact revision on arXiv

Source-linked AI summary

EgoSteer: A Full-Stack System Towards Steerable Dexterous Manipulation from Egocentric Videos

Yifan Zhong, Zhang Chen, Tianrui Guan, Fanlian Zeng, Yuyao Ye, Tianjia He, Ka Nam Lui, Jiayi Li, Tingrui Zhang, Ruilin Yan, Xinhao Ji, Guangyu Zhao, Wenjie Lou, Jiayuan Zhang, Yuanpei Chen, Yaodong Yang

arXiv:2607.09701v1cs.RO

TL;DR

Steerable dexterous manipulation remains largely unrealized because existing systems lack large-scale, language-aligned, action-accurate demonstrations. EgoSteer addresses this with an egocentric-video data pipeline, unified robot stack, and world-model-enhanced VLA, achieving 75% average success across 40+ tasks and few-shot adaptation to long-horizon tasks.

  • Problem

    Steerable dexterous manipulation remains largely unrealized, as existing general-purpose systems often require task-specific fine-tuning or support free-form language mainly on grippers.

  • Method

    EgoSteer integrates EgoSmith’s curated egocentric-video corpus, a unified teleoperation and correction stack, and a world-model-enhanced VLA for robot post-training.

  • Results

    75% average success across 40+ tasks, with 80+% success on 22 tasks and generalization to compositional and unseen tasks.

  • Takeaways & Limitations

    EgoSteer demonstrates robust steerability, dexterity, failure recovery, and generalization, while few-shot adapting to long-horizon tasks across multiple embodiments.

  • Takeaways & Limitations

    Robotic degrees-of-freedom limitations restrict transfer of highly dexterous human knowledge and therefore constrain intricate operations.

Abstract

from arXiv · show

Steerability is a defining capability of generalist robot policies, yet remains largely absent in dexterous-hand systems for lack of large-scale, language-aligned, and action-accurate demonstration data. To address this bottleneck, we present a full-stack system that scales dexterous VLA pre-training from egocentric human videos and enables data-efficient real-robot post-training. It integrates EgoSmith, a data pipeline that curates in-the-wild egocentric videos into 9.6K hours of high-quality pre-training data with 9x higher throughput and better accuracy than prior SOTA; a unified robot stack for teleoperation and human-in-the-loop correction; and EgoSteer, a world-model-enhanced VLA trained on optimized infrastructure. Human-data pre-training equips EgoSteer with language-guided manipulation priors, which are grounded through robot post-training and improved by DAgger refinement. Empirically, EgoSteer robustly executes free-form instructions across 40+ diverse tasks, demonstrating failure recovery, dexterity, and generalization. The pre-trained model also few-shot adapts to complex long-horizon tasks, including box folding, on two embodiments with 75+% success. We open-source the system, data, and model at https://egosteer.github.io/.

1 Introduction

Steerable dexterous manipulation remains largely unrealized because scalable, language-aligned, action-accurate demonstration data and systems are difficult to obtain. EgoSteer addresses this gap with a full-stack pipeline, robot stack, and world-model-enhanced VLA that achieves broad real-robot instruction following.

  • Motivation: Dexterous-hand robots offer greater actuation capacity and fine-grained interaction potential, but steerable manipulation remains largely unrealized.Most existing systems require task-specific fine-tuning, while free-form language following is largely limited to grippers.
  • Data bottleneck: Egocentric human videos provide a scalable source of rich manipulation knowledge, but raw footage is noisy and lacks reliable language alignment.Collecting demonstrations directly on a specific dexterous embodiment is exceptionally difficult.
  • Results: 75% average success rate is achieved across over 40 tasks through egocentric pre-training, real-robot post-training, and DAgger refinement.EgoSteer follows free-form instructions while exhibiting fine-grained dexterity, failure recovery, and generalization.
  • System contributions: 9.6K hours of data across 12 datasets are curated by EgoSmith, achieving 9× throughput and better accuracy than prior SOTA.The pipeline integrates pre-filtering, 4D motion estimation, language labeling, and post-filtering to produce fully annotated training data.
  • System contributions: 187 hours of data across 193 dexterous tasks are collected by a unified robot stack integrating teleoperation, inference, and human-in-the-loop DAgger correction.The stack supports diverse real-robot post-training and efficient refinement.

2 Related Work

Related work advances generalist manipulation through foundation-model policies, egocentric human-video scaling, and human-in-the-loop post-training. However, existing methods remain limited by task complexity, data or computation demands, and real-time correction challenges in high-DoF spaces.

  • Generalist robot policies: Foundation-model manipulation policies pursue general-purpose robotics, but early methods remain largely confined to simple tasks and depend heavily on single-task fine-tuning.Recent works have improved generalization, yet rely heavily on massive real-robot multi-task datasets or intensive computation.
  • Scaling with egocentric human videos: 116K hours of egocentric videos provide a scalable source for dexterous-manipulation learning through large-scale pre-training or cross-embodiment co-training.Existing methods align human hands and robot action spaces through diverse approaches to transfer human priors into the robot domain.
  • Human-in-the-loop post-training: Human-in-the-loop post-training raises performance ceilings and resolves out-of-distribution failures efficiently, but struggles with real-time, high-frequency corrections in high-DoF joint spaces.Recent paradigms include online reinforcement learning with human copilots, compliant residual feedback, and hand-arm intervention frameworks for dexterous VLAs.

3 EgoSmith: Curating Egocentric Videos into Grounded Dexterous Priors

EgoSmith converts noisy, weakly annotated egocentric videos into grounded dexterous-manipulation training samples through automated filtering, 4D motion estimation, hierarchical language labeling, and quality control. Applied across 12 raw datasets, it produces 9.60K hours of fully annotated data spanning 2.09M episodes and 1.04B frames.

  • Pipeline overview: EgoSmith addresses camera jitter, occlusions, and missing annotations in monocular egocentric RGB videos with an automated four-stage curation pipeline.The stages are pre-filtering, 4D motion estimation, language labeling, and post-filtering.
  • Pipeline overview: Pre-filtering removes locomotion segments and hand misidentifications using optical-flow heuristics and occlusion checks, while 4D estimation reconstructs camera extrinsics, depth, and world-space hand trajectories.The motion-estimation scheme improves robustness and efficiency over HaWoR’s DROID-SLAM-based tracking by using DPVO for camera tracking.
  • Language labeling: 3.5% of clips passing heuristic rules are discarded for lacking active hand-object manipulation, and remaining clips receive coarse-to-fine five-level language instructions.The language annotations are designed to enable free-form instruction following.
  • Quality control: Post-filtering performs episode-, chunk-, and frame-level quality control by removing motion outliers, excessive head rotations, and spatial inconsistencies in wrist and finger coordinates.The process applies camera-translation and rotation thresholds, transforms wrist poses, projects finger keypoints, and discards coordinate outliers.
  • Corpus scale: 9.60K hours, 2.09M episodes, and 1.04B frames are curated from 12 raw datasets into a fully annotated egocentric pre-training corpus.Repetitive videos are subsampled in Egocentric-10K and Egocentric-100K, while EgoSmith is applied to VITRA subsets of Ego4D and EPIC-KITCHENS.

4 A Unified Robot Stack for Teleoperation and DAgger Post-Training

The Unified Robot Stack grounds egocentric manipulation priors on real embodiments by sharing low-level control across teleoperation, policy inference, and human-in-the-loop correction. Relative-motion handovers enable smooth intervention, yielding over 85% handover success and supporting a 187-hour dataset across 193 tabletop tasks.

  • Unified Robot Stack: The Unified Robot Stack addresses the embodiment gap by sharing low-level control and dynamics across teleoperation, policy inference, and human-in-the-loop correction.Real-robot teleoperation grounds visual, dynamics, and kinematic priors onto the target embodiment.
  • Teleoperation and Policy Inference: PsiBot SynGlove-Air gloves and Vive Trackers capture SE(3) wrist poses and hand joint angles for inverse-kinematics arm control and joint-mapped hand control.Policy inference uses wrist-frame hand keypoints and camera-frame wrist pose trajectories through the same arm and hand FK/IK and control nodes.
  • Human-in-the-Loop Correction: Relative motion mapping prevents sudden state jumps, allowing operators to take over by mimicking robot motion and return control after correcting failures.Intervention begins with a foot-pedal press and ends with another pedal press that resumes policy inference.
  • Human-in-the-Loop Correction: 85% handover success rate exceeding 85% enables efficient collection of corrective demonstrations, using only intervention segments for subsequent training.The handover design supports smooth transitions between policy control and human correction.
  • Robot Dataset: 187-hour robot dataset across 193 tabletop tasks spans seven categories, combining 56 common tasks with 137 long-tail tasks and manually verified Qwen3-VL-Flash language annotations.The dataset is designed to cover core manipulation primitives while facilitating human-to-robot transfer and grounding.

5 EgoSteer: A World-Model-Enhanced VLA for Steerable Dexterity

EgoSteer is a flow-based VLA that combines a Qwen3-VL 2B backbone, DiT action expert, real-time chunking, and a training-only world-model expert for steerable dexterity. Its world-model objective predicts future DINOv3 features to improve action accuracy without inference overhead, while optimized infrastructure supports efficient training.

  • Model architecture: EgoSteer pairs a Qwen3-VL 2B backbone with a DiT-based action expert that generates multimodal continuous action chunks through flow-matching.The action expert jointly attends to its own representations and the backbone.
  • Real-Time Chunking: Real-Time Chunking trains the expert to denoise subsequent actions after a randomly sampled clean action prefix, avoiding execution pauses during real-robot inference.The clean prefix is provided as ground truth, while only the suffix is denoised.
  • World-model enhancement: The world-model expert improves action accuracy with zero inference overhead by operating only during training.Figure 4 describes the expert as predicting future DINOv3 features to improve action accuracy without adding inference computation.
  • World-model enhancement: The training-only world-model expert predicts action-induced future DINOv3 features from ground-truth actions, relative camera motion, learnable query tokens, and backbone representations.Regression against future-frame DINOv3 features provides more direct and stable supervision than generative loss.
  • Optimized infrastructure: Efficient training combines HSDP, mixed precision, torch.compile kernel fusion, FlexAttention, and WebDataset sequential streaming to reduce communication and I/O bottlenecks.These infrastructure choices target batch scaling, GPU utilization, attention efficiency, and sequential data access.

6 Experiments

EgoSteer achieves robust, language-guided dexterous manipulation across 40 tasks, while DAgger refinement, scaled egocentric pre-training, and core training components substantially improve performance. Its 9.6K-hour pre-training also enables few-shot adaptation to challenging long-horizon tasks on novel embodiments.

  • Free-form instruction following: 75% overall success across 40 tasks demonstrates robust free-form instruction following, including failure recovery and fine-grained manipulation.EgoSteer exceeds 80% success on 22 tasks and handles cluttered, randomized layouts involving target objects, hand choices, and specific actions.
  • DAgger refinement: 62.5% average success after three DAgger iterations, up from 22.5%, shows targeted corrective demonstrations efficiently improve failure-prone dexterous tasks.The iterations total 8.3 hours of correction data and improve recovery and action adjustment at manipulation bottlenecks.
  • Pre-training scale: 9.6K hours of egocentric pre-training improves convergence and real-world execution while enabling failure recovery, stronger instruction-following, and more accurate actions.Models pretrained on 3K, 6K, and 9.6K hours are compared with a from-scratch model across 10 tasks.
  • Component ablations: Removing the world-model objective, training-time RTC, or data filtering causes substantial performance declines, confirming that each core component is necessary.Without the world-model objective, fine-grained manipulation accuracy decreases; without training-time RTC, action pauses disrupt contact-rich execution.
  • Few-shot adaptation: 75+% success in few-shot adaptation shows that EgoSteer-9.6K transfers dexterous priors to novel embodiments and complex long-horizon tasks despite limited demonstrations.The tasks are box folding on RealMan with 120 demonstrations and cake unboxing on AgiBot-G1 with 200 demonstrations; DP, IMLE, and the from-scratch variant completely fail.

7 Limitations & Conclusion … A.1.1 Pre-Filtering Heuristics

The paper concludes with a full-stack system for steerable dexterous manipulation and details EgoSmith’s pre-filtering heuristics for constructing high-quality egocentric-video data. The heuristics combine camera- and hand-quality gates to retain segments with stable motion and visible bimanual interactions.

  • 7 Limitations & Conclusion: EgoSteer integrates EgoSmith, a unified robot stack, and a world-model-enhanced VLA for steerable dexterous manipulation across 40+ diverse tasks.The system demonstrates dexterity, failure recovery, and generalization, while adapting to long-horizon tasks.
  • Appendix: The appendix documents implementation details and curated dataset statistics, including a 9.6K-hour corpus derived from 12 egocentric human video datasets.The listed appendix sections cover EgoSmith implementation, robot implementation, teleoperation, VLM co-training, and dataset statistics.
  • A.1.1 Pre-Filtering Heuristics: A.1.1 pre-filtering rapidly discards segments with locomotion, excessive head movement, hand absence, occlusion, or others’ hand interference.Two specialized gates handle these cases: a camera gate for motion-related issues and a hand gate for visibility anomalies.
  • A.1.1 Pre-Filtering Heuristics: A contiguous segment is pruned only when it contains at least three consecutive invalid frames, while isolated failures are retained.This preserves segments when individual invalid frames are unlikely to affect quality.
  • A.1.1 Pre-Filtering Heuristics: 128-point sparse optical flow tracks points 15 frames backward, and a RANSAC-fitted similarity transform passes when translation remains within 10% of the image’s longer dimension.This camera gate estimates ego-motion using pyramidal Lucas–Kanade tracking.
  • A.1.1 Pre-Filtering Heuristics: YOLO hand detections pass only with confidence ≥0.30 and bounding-box area within [2%, 50%] of the image.The upper bound excludes hands unusually close to the lens, while the lower bound filters most other people’s hands while retaining the operator’s.
  • A.1.1 Pre-Filtering Heuristics: Together, the camera and hand gates produce segments with stable camera motion and clearly visible bimanual interactions.The two gates jointly enforce motion stability and hand visibility for retained video segments.

A.1.2 4D Motion Estimation · A.1.3 Language Labeling Prompt · A.1.4 Post-Filtering Criteria

EgoSmith reconstructs metric world-space bimanual motion, generates structured multi-level language labels for valid hand-object interactions, and applies coarse-to-fine filtering to remove reconstruction anomalies and discontinuities.

  • A.1.2 4D Motion Estimation: DPVO camera trajectories are anchored to Any4D metric depth, producing metric world-space bimanual states, actions, camera parameters, MANO parameters, and scene depth.The pipeline calibrates the up-to-scale trajectory and transforms camera-space hand geometry into a unified world-space coordinate system.
  • A.1.2 4D Motion Estimation: Approximately 9× speedup over HaWoR is achieved by batching temporal windows and overlapping CPU frame processing with GPU inference.The benchmark uses an 8×A800 server processing 8 video segments of 2K frames each.
  • A.1.2 4D Motion Estimation: EgoSmith reduces RPE from 5.17 to 2.42 mm, ATE from 9.44 to 7.60 mm, WA-MPJPE from 38.7 to 25.9 mm, and W-MPJPE from 106.9 to 86.0 mm versus HaWoR.The improvements are attributed to DPVO trajectory estimation, Any4D metric scaling, cross-window scale alignment, and global re-anchoring.
  • A.1.3 Language Labeling Prompt: The labeling prompt asks Qwen3.5-VL-Plus to describe egocentric hand-object interactions at five granularities, from verb-object commands to dense physical sequences.Descriptions emphasize visible actions, manipulated objects, contact points, grasp types, spatial relationships, and task-relevant physical sequences.
  • A.1.3 Language Labeling Prompt: Videos are marked Invalid when they lack identifiable active manipulation or suffer severe occlusion, blur, or ambiguity, and Valid when identifiable hand-object actions are present.Valid examples include picking, placing, opening, closing, pouring, wiping, folding, pressing, turning, cutting, fastening, arranging, inserting, removing, scanning, and operating objects.
  • A.1.3 Language Labeling Prompt: The prompt enforces verb-first, precise physical instructions with definite object references, no transitional words, and no unsupported hidden intent or identities.It also requires spatial anchors, motion trajectories, and outcome states when describing the interaction.
  • A.1.4 Post-Filtering Criteria: Three post-filtering levels evaluate episode camera motion, chunk-level hand positions, and adjacent-frame motion to remove drift, reconstruction errors, and discontinuities.Episode checks use dataset-specific IQR criteria, chunk checks use canonical egocentric coordinates, and frame checks apply fixed physical speed thresholds.

A.2 Curated Dataset Statistics … B.1.4 Language Labeling Prompt

The system combines a 9.6K-hour, richly annotated egocentric corpus with a calibrated robot stack and structured language labeling to support steerable dexterous manipulation. The data spans diverse tasks and embodiments while preserving human–robot alignment and interaction-focused descriptions.

  • A.2 Curated Dataset Statistics: 9.6K hours of egocentric demonstrations form a standardized corpus with bimanual states, actions, camera geometry, metric depth, and multi-granularity language annotations.EgoSmith processes 12 raw datasets through rigorous quality control to produce modality-aligned samples.
  • A.2 Curated Dataset Statistics: 8969 distinct object nouns and 623 action verbs provide broad semantic coverage of manipulation tasks and objects.The dataset also analyzes frequent verb–object atomic tasks from L1 annotations.
  • B Details of the Robot Stack: 187 hours of real-robot data support implementation of the Robot Stack and grounding of human manipulation priors on physical embodiments.The robot stack includes collection protocols and statistics for the real-robot dataset.
  • B.1.1 Hardware Setup: Two embodiments are used: RealMan combines dual 7-DoF arms, dual 6-DoF dexterous hands, and head- and chest-mounted cameras, while AgiBot G1 provides a second platform.The RealMan setup uses Intel RealSense D455 cameras for dual egocentric viewpoints.
  • B.1 Implementation Details: 100 Hz wrist, arm, and joint-control loops plus 80 Hz glove, hand-solver, and control loops enable low-latency bimanual teleoperation; cameras capture at 30 Hz.Recorded data is resampled to 30 Hz for training.
  • B.1.2 Aligning Robot Data with Egocentric Human Data: Axially translating the robot wrist frame forward aligns wrist-to-fingertip scale with human anatomy, reducing the robot–human domain gap.The adjustment compensates for the robotic palm being slightly longer than a human palm.
  • B.1.3 Hand-Eye Verification and Re-Calibration: RGB-D point-cloud projection and robot-mesh rendering verify hand-eye calibration offline and trigger automated recalibration when spatial misalignment is detected.The pipeline addresses drift caused by maintenance, wear, or collisions.
  • B.1.4 Language Labeling Prompt: The labeling prompt integrates synchronized head and chest views into verb-starting, interaction-focused descriptions at gist, descriptive, and sequential levels.It warns that colored fingertip markers are part of the fingers, assumes empty-handed operation, and omits irrelevant clutter and trivial adjustments.

B.2 Teleoperation Data Collection · C Details of EgoSteer

The paper grounds egocentric-video manipulation priors through a diverse, high-quality real-robot teleoperation dataset designed for embodiment transfer and steerable dexterity. It also preserves vision-language knowledge through VLM co-training and details the EgoSteer implementation.

  • B.2 Teleoperation Data Collection: The collection targets embodiment grounding because human-video pre-training faces visual, dynamic, kinematic, and automated-label accuracy gaps.Robot kinematic constraints also prevent perfect replication of fine-grained human hand movements.
  • B.2 Teleoperation Data Collection: 193 semantically distinct dexterous tasks produce 187 hours and 55K trajectories of real-robot demonstrations under cluttered scenarios.Each task contributes approximately 300 randomized, diverse trajectories totaling around 1 hour.
  • B.2 Teleoperation Data Collection: 56 common tasks target everyday manipulations with high teleoperation success, while 137 long-tail tasks cover infrequent, physically challenging operations with lower collection success.Long-tail examples include contact-sensitive operations lacking tactile feedback and primarily expand semantic coverage.
  • B.2 Teleoperation Data Collection: Cluttered, unstructured scenes and strict trajectory requirements force language-action alignment toward general task semantics rather than visual memorization.The dataset emphasizes randomness, diversity, and quality throughout collection.
  • B.2 Teleoperation Data Collection: The 187-hour dataset exhibits diverse action and object vocabularies with balanced manipulation-category and task-duration distributions.Figure 8 analyzes word frequencies, seven-category duration breakdowns, and duration statistics for common and long-tail tasks.
  • B.2 Teleoperation Data Collection: Dual-view trajectories use three-level language annotations to align free-form instructions with natural, human-like physical actions in randomized clutter.The annotations describe manipulation from coarse to fine.
  • C Details of EgoSteer: EgoSteer co-trains VLM datasets with egocentric human videos and real-robot data to preserve vision-language knowledge and support generalization.The co-training mixture covers general vision-language knowledge, interaction understanding, and spatial-geometric priors.
  • C Details of EgoSteer: The implementation details of EgoSteer are presented after introducing the VLM datasets used for co-training with VLA data.The paper organizes these details into Sections C.1 and C.2.

C.1 VLM Co-Training Data

EgoSteer co-trains on a curated 10.4M-sample mixture spanning seven datasets and four data categories to preserve general vision-language reasoning while strengthening robotic task comprehension. The datasets are standardized to Qwen3-VL’s conversational format with normalized spatial coordinates and single-image samples for training stability.

  • Mixture composition: 10.4M samples across seven datasets form a VLM co-training mixture spanning open-world perception and embodied interaction grounding.The mixture is curated to preserve general vision-language reasoning while cultivating robust robotic task comprehension.
  • Mixture composition: Four categories organize the mixture: general VLM pre-training, spatial grounding, embodied QA, and embodied interaction grounding.FineVision supports open-world semantic concepts; RefSpatial and RoboPoint provide spatial referring and coordinate localization; RoboInter-VQA, Robo2VLM, and ShareRobot support embodied question answering and temporal reasoning.
  • Data standardization: Datasets are normalized to Qwen3-VL’s conversational input format, including 2D box and point coordinates mapped to its native [0, 1000] scale.For training stability, only single-image samples are utilized and samples exceeding the specified context-length limit are excluded.

C.2 Implementation Details

EgoSteer processes multimodal temporal histories with masking strategies that encourage visual and task-instruction use, while its action and world-model experts provide high-capacity, asynchronously deployed prediction modules. Training combines action flow matching, semantic feature regression, and autoregressive language modeling with weighted data sampling and augmentation.

  • Backbone Input Scheme: 6 frames at 1 FPS span a 5 s observation window, with proprioceptive states aligned to the corresponding timestamps.The backbone treats image history as a temporal video sequence for Qwen3-VL-2B.
  • Backbone Input Scheme: 75% proprioceptive-history masking and 50% chest-camera dropout reduce shortcut learning and over-reliance on chest-view observations.Masked proprioception forces attention toward the full multimodal context, while camera dropout is used during real-robot post-training.
  • Action Expert: 14 layers and approximately 300 M parameters define the DiT action expert, which predicts 32-step action chunks at 30 Hz.The expert uses hidden dimension 1024, intermediate size 2816, and 8 attention heads with dhead = 128.
  • Action Expert: 8 new action steps execute per inference cycle after a simulated delay of d = 4, retaining the first 12 steps for responsive asynchronous control.The retained prefix accounts for physical inference latency conditioning during real-world deployment.
  • World Model Expert: 4 layers and approximately 70 M parameters comprise the world-model expert, which regresses future DINOv3 features rather than pixels using an MSE objective.DINOv3 ViT-L/16 features use 384 × 384 inputs, 24 × 24 spatial resolution, and feature dimension 1024; feature regression filters lighting variation and background noise.
  • Data Processing and Optimization: The total objective combines action flow matching, world-model feature regression, and autoregressive VLM next-token prediction, while weighted sampling, ColorJitter, and percentile normalization support heterogeneous data.Dataset weights combine subjective quality scores with square-root frame counts, and action dimensions except wrist rotations are normalized to [−1, 1].

D Experimental Details

This section provides supplementary experimental details for EgoSteer, including training configurations, per-task success rates, scaling comparisons, ablations, and few-shot fine-tuning settings. It specifies the evaluation protocols and key pre-training warm-up schedule.

  • Main training configurations: The main pre-training schedule freezes the VLM backbone for the first 5,000 steps, warms up AE and WM for 2,000 steps, then warms up VLM for 2,000 steps after unfreezing.These configurations support the main experiments in Section 6.1.
  • DAgger refinement: Table 5 compares EgoSteer-DG with EgoSteer-FT on four highly dexterous, failure-prone tasks using 10 randomized trials per task.The comparison evaluates performance gains from DAgger refinement.
  • Pre-training scaling: The pre-training scaling study reports training configurations and per-task success rates, including comparisons between EgoSteer-9.6K, Being-H0.5, and π0.5 across 10 tasks.All methods in the baseline comparison are post-trained on the same real-robot dataset and evaluated with 10 randomized trials per task.
  • Ablation study: The ablation study compares EgoSteer-1K with ablated variants using 10 trials per task, with all variants trained on 1K hours of egocentric data and without DAgger refinement.Pretraining steps are selected at the lowest evaluation L1 loss for each pre-training run.
  • Few-shot fine-tuning: Few-shot fine-tuning hyperparameters are provided for EgoSteer-9.6K on two long-horizon dexterous tasks.These settings correspond to the experiments in Section 6.5.
Loading 2607.09701v1…