PaperlayerView exact revision on arXiv

Source-linked AI summary

Visible-Reachable Workspace for Perception-Aware Humanoid Design

Boxi Xia, Zijiang Yang, Ryan Shin, Bokuan Li, Eric Wun-Hao Lu, Jacob Lee, Jiaxun Liu, Boyuan Chen

arXiv:2609.08905v1cs.RO

TL;DR

For visually guided humanoid manipulation, reachability alone misses whether targets are visible from feasible reaching poses, and separated work regions may require concurrent views. The paper introduces VRW to evaluate visibility with reaching configurations; camera actuation improves coverage, and dual actuation reduces time and energy in a two-target benchmark.

  • Problem

    Reachability alone omits whether onboard cameras can see a target from feasible reaching poses, while tasks across separated regions can require concurrent views.

    Source: §I · p. 1

  • Method

    VRW retains targets visible from feasible reaching configurations, distinguishes manipulation joints from independently available sensing joints, and extends coverage to simultaneous views of a second region.

    Source: §I · p. 1, §III · p. 3, A. Camera configuration optimization · p. 6

  • Results

    Camera actuation raised visible-reachable coverage from 38% to 97%; in two-target tasks, Act2 reduced mean completion time by 17% and energy by 19% versus Fix2.

    Source: §I · p. 2, B. Two-Target Reach-and-Grasp Benchmark · p. 7

  • Takeaways & Limitations

    The results suggest VRW is a useful design signal because improving visible-reachable coverage can translate into more efficient whole-body execution.

    Source: §VI · p. 8

    The framework currently centers on geometric visibility; future work is proposed to incorporate sensing quality, uncertainty, and task-dependent perception requirements.

    Source: §VI · p. 8

Abstract

from arXiv · show

Workspace analysis measures where a robot can place its end effector. For visually guided manipulation, reachability alone is insufficient: a kinematically reachable target may not be visible in the specific pose required to reach it. The robot must then redirect its sensing or move its body to acquire a view, turning a perception limitation into additional motion. Existing humanoids largely inherit this limitation when copying human form factors. We introduce the visible-reachable workspace (VRW), a design-stage measure that conditions visibility on feasible reaching configurations and extends it to concurrent visibility of spatially separated work regions. We apply VRW by building a 31-DoF humanoid with independently actuated RGB-D cameras. On the same robot, camera articulation increases visible-reachable coverage from 38% to 97%. With actuated camera layouts, a second camera raises pairwise coverage from 0.45 to 0.95, while a third changes it only to 0.97. In a controlled two-target reach-and-grasp benchmark, our dual-actuated design reduces mean completion time by 17% and mechanical energy by 19% relative to the same robot with its cameras fixed. Hardware experiments demonstrate simultaneous observation and manipulation of front/back and left/right target pairs without torso reorientation. The results suggest that reachability becomes a more informative design quantity for perception-driven humanoid manipulation when it is evaluated together with the sensing configurations that make the reachable space observable. We will open-source all software and the humanoid hardware design. Our website is https://generalroboticslab.com/DukeHumanoidv2

I. INTRODUCTION

For perception-driven humanoids, geometric reachability can overstate usable workspace because a target may be hidden in the poses that reach it, prompting sensing or body motion. The paper introduces VRW and its concurrent-region extension, then uses them to evaluate camera layouts.

I. INTRODUCTION

  • A target can lie within the arm workspace yet outside every camera view at the configurations that realize the reach.The mismatch is associated with broad arm workspaces and vision concentrated in the head or chest of existing humanoids.Source: §I · p. 1
  • When sensing and manipulation geometry are poorly matched, a robot may redirect a camera, rotate its torso, or walk to a new viewpoint.The passage frames this as a design-time perception limitation that can become whole-body motion during execution.Source: §I · p. 1
  • For separated regions, shared gaze provides one viewing direction at a time, while broader passive sensing requires more sensors and computational power –.This can leave sequential sensing or frequent body reorientation when two regions matter simultaneously.Source: §I · p. 1
  • VRW retains reachable targets that are also observable from feasible reaching configurations, and its extension measures concurrent visibility of a second region.The definition distinguishes manipulation joints from independently available camera-aiming joints; a wrist camera remains coupled to the arm reach.Source: §I · pp. 1–2
  • On Duke Humanoid V2, camera actuation raised visible-reachable fraction from 38% with cameras fixed to 97%; evaluated humanoids ranged from 16% (Unitree G1) to 48–76% (actuated-neck platforms).The actuated-neck platforms named are PAL Talos, Booster T1, Fourier GR-3, and Apptronik Apollo.Source: §I · p. 2
  • Pairwise coverage rose from 0.45 with one independently actuated camera to 0.95 with two (0.97 with three); dual actuation reduced completion time by 17% and energy by 19% versus fixed cameras.The physical experiments reproduced dual-region behavior for front/back and left/right target pairs.Source: §I · p. 2

II. RELATED WORK

Prior work describes where robots can reach and incorporates visibility into planning or control, but the paper focuses on evaluating sensing layout against humanoid manipulation workspace before execution.

II. RELATED WORK

  • Classical workspace and capability maps represent reachable positions and sometimes feasible end-effector orientations,, but do not encode sensor visibility from feasible reaching configurations.The passage notes that this omission can make geometric reachability an optimistic measure of locally usable workspace.Source: §II · p. 2
  • Mightability maps, MoMaGen, and visibility-aware controllers integrate visibility with perspective-taking, configuration sampling, or whole-body motion; this work evaluates sensor arrangement before execution.Active-vision systems also redirect cameras or pan-tilt heads toward task-relevant regions, –.Source: §II · p. 2
  • Humanoid perception is commonly fixed to torso/head or a shared neck; other systems use omnidirectional or distributed sensing,, while our measure links onboard camera placement and articulation with arm workspace.It also considers whether separated regions can be viewed concurrently.Source: §II · p. 2

III. VISIBLE-REACHABLE WORKSPACE

The visible-reachable workspace (VRW) measures which reachable targets can also be observed from compatible reaching configurations. Its task-relative formulation separates manipulation from independently available sensing motion and extends to concurrent observation of separated regions.

III. VISIBLE-REACHABLE WORKSPACE

  • With the floating base fixed at nominal stance, VRW isolates the relationship between arm reach and onboard sensing, excluding locomotion from workspace analysis.The paper evaluates the ability to acquire a new viewpoint by walking separately in dynamic experiments.Source: §III · p. 2
  • The configuration is partitioned into manipulation joints qm, sensing joints qp, and other joints q0 held nominal; joints affecting an active end effector take manipulation priority.Feasible configurations satisfy joint-limit and self-collision constraints used by the IK solver.Source: §III · p. 2
  • A target counts as visible only from a feasible configuration that realizes the reach, with visibility requiring modeled camera view, range, and an unoccluded camera-to-target segment.A wrist camera can contribute visibility, but its motion is coupled to the arm; independent camera joints can be aimed after fixing the manipulation configuration.Source: §III · p. 3
  • The canonical-occlusion approximation evaluates self-occlusion with arms down, making maps consistent and tractable but omitting reach-conditioned occlusion.Unless otherwise noted, reported VRW coverage uses this approximation.Source: §III · pp. 3–4
  • Concurrent visibility measures whether the robot can manipulate at one region while observing a second, which need not itself be reachable.A single camera receives credit if both regions fit in one unoccluded frustum; η2 samples reachable targets by orientation reachability and the second region uniformly.Source: §III · p. 4
  • The scalar η2 depends on the evaluation distribution, so cross-platform values are descriptive rather than a universal ranking.Different embodiments induce different evaluation volumes and target-separation distributions.Source: §III · p. 4

IV. HUMANOID REALIZATION

The paper presents Duke Humanoid V2 as a platform for design studies using the embodiment-agnostic visible-reachable workspace metric. Its 31-DoF humanoid combines independently actuated camera gimbals with a whole-body control policy.

IV. HUMANOID REALIZATION

  • The authors describe VRW as embodiment-agnostic and use Duke Humanoid V2 for systematic design studies.The figure depicts the robot with a 27-DoF body and 4-DoF dual-camera head.Source: §IV · p. 4, Figure 3 · p. 4
  • Duke Humanoid V2 is a 31-DoF QDD humanoid with a waist, two 6-DoF legs, two 7-DoF arms, and two 2-DoF camera gimbals.Each parallel gripper adds another actuated DoF.Source: §IV · p. 4
  • Two independently actuated yaw-pitch RGB-D camera modules can simultaneously view spatially separated regions.Each wrist carries a parallel gripper; the cameras use Intel RealSense D436 sensors.Source: §IV · p. 5
  • The whole-body policy outputs residual joint-position actions for legs, arms, and cameras, tracked by joint-level PD control.The actor uses a 20-frame observation history; the critic receives privileged simulator quantities during training.Source: §IV · p. 5, Figure 4 · p. 5
  • Training uses 4096 parallel environments for 15k iterations, with randomized contact parameters, PD gains, inertia, friction, and periodic pushes.The command curriculum spans base linear and yaw velocities from −1 to 1 m/s and rad/s, respectively.Source: §IV · p. 5

V. RESULTS

This section frames VRW-guided camera design around the robot’s symmetric reachable workspace. It asks how VRW can guide camera configuration and whether higher VRW improves visual grasping.

V. RESULTS

  • The robot’s reachable workspace is symmetric, and VRW is used to guide camera-configuration design.The section poses two questions: how VRW can guide camera configuration and whether higher VRW improves visual grasping.Source: §V · p. 5
  • The first research question asks how VRW can guide the design of the camera configuration.The study considers camera configuration as a design decision on Duke Humanoid V2.Source: §V · p. 5
  • The second research question asks whether designing for higher VRW improves visual grasping performance.Source: §V · p. 5

A. Camera configuration optimization

The camera study compares layouts by single-target and pairwise visible-reachable coverage, and evaluates their performance across humanoids. It finds that camera articulation and independent views address different coverage needs.

A. Camera configuration optimization

  • Front- and back-mounted cameras were excluded because their coverage was limited to one side; top-mounted cameras could cover front, back, left, and right.Wrist cameras on the reaching arm were excluded because they offered no independent sensing actuation under the study’s configuration partition.Source: A. Camera configuration optimization · p. 6
  • Actuating cameras increases single-target VRW coverage more than increasing camera count; with actuated cameras, adding a second or third yields less than 4% coverage gains.The evaluated layouts include one, two, or three camera modules, either fixed or actuated.Source: A. Camera configuration optimization · p. 6
  • With one camera, pairwise coverage falls as target separation increases; two independently actuated cameras keep it nearly constant, while a third adds little and incurs hardware cost.The third camera adds 0.58 kg, two gimbal DoF, and $600; the authors adopt two independently actuated modules.Source: A. Camera configuration optimization · p. 6
  • The fixed-head G1 has much lower VRW coverage than the evaluated actuated-neck humanoids, but all evaluated platforms have lower pairwise coverage than the design.The external platforms’ cameras share neck joints and cannot independently aim at two separated regions.Source: A. Camera configuration optimization · p. 6
  • The figure arranges six scenarios by row and one target’s reach-and-grasp keyframes by column, from initial stage through retraction.The targets are 60 mm cubes; cyan and orange cones indicate the left and right camera fields of view.Source: Figure 6 · p. 6

B. Two-Target Reach-and-Grasp Benchmark

The benchmark compares fixed and actuated camera configurations on simulated two-target reach-and-grasp tasks, then tests the actuated design on hardware. Actuation improves completion time and energy relative to fixed cameras, while real-world demonstrations include separated and moving targets.

B. Two-Target Reach-and-Grasp Benchmark

VI. CONCLUSION AND FUTURE WORK

The conclusion presents VRW as a way to couple reachability with visibility, including simultaneous visibility of separated regions. Results on Duke Humanoid V2 link improved sensing geometry with lower two-target manipulation time and energy; future work would broaden VRW beyond geometric visibility.

VI. CONCLUSION AND FUTURE WORK

  • Camera actuation increased visible-reachable coverage from 38% to 97%, while a second independent view raised pairwise coverage from 0.45 to 0.95.Source: §VI · p. 8
  • In two-target manipulation, improved sensing geometry reduced completion time and mechanical energy, with reductions in search, approach, and manipulation time.Source: §VI · p. 8
  • Future work would extend VRW to sensing quality, uncertainty, and task-dependent perception requirements, and use it for automated co-design across tasks and embodiments.Source: §VI · p. 8

A. Reward terms

The reward-terms table lists terms and weights for the reported direct controller; nearby notation describes controller variables and constraints.

A. Reward terms

  • The reported direct controller's reward table lists reward terms r_i and weights w_i.Source: Table III · p. 9
  • The notation includes shared and joint-specific kernel widths, base linear and yaw velocities, projected gravity, and home and moving reference poses.Source: A. Reward terms · p. 9
  • Other listed quantities include foot height and planar velocity, swing duration, joint soft limits and torque limits, and foot-flat gravity alignment.Source: A. Reward terms · p. 9

B. Observation space

The observation table distinguishes the actor's stacked noisy, delayed frames from the critic's current clean observation, and specifies shared and critic-only blocks.

B. Observation space

  • The actor stacks a noisy, delayed 120-D observation frame over 20 steps.Source: Table IV · p. 9
  • The asymmetric critic receives the current clean 169-D observation; the table separates shared inputs from a critic-only block.Source: Table IV · p. 9
  • Listed inputs include IMU angular velocity and projected gravity, joint position and velocity, last action, and a twist command, with specified noise and delay.Source: B. Observation space · p. 9, Table IV · p. 9

C. Training hyperparameters

The section specifies the training algorithm, domain-randomization schedule, and command sampling used for the experiments.

C. Training hyperparameters

  • Training uses off-policy SAC with a distributional (C51) asymmetric critic and an RMA-CNN actor-history encoder.Source: Table V · p. 9

D. Domain randomization

  • Domain randomization is applied at episode reset or fixed time intervals; scale multiplies nominal values, while add is an additive offset.Source: Table VI · p. 9

E. Command sampling

  • Yaw uses heading mode with stiffness 0.5 and heading sampled over [−π, π]; 20% of environments command standing, with commands resampled every 2–8 s.Source: E. Command sampling · p. 9

Related papers

Loading 2609.08905v1…