PaperlayerView exact revision on arXiv

Source-linked AI summary

Robot-Aligned 3D Gaussian Splatting for Precise Reaching with Wrist-Camera VLA Policies

Zuoxu Wang, Xiao Liang

arXiv:2610.35626v1cs.RO

TL;DR

Precise reaching for VLA policies is difficult to train from costly, imperfect demonstrations, and building full digital twins for complex disassembly scenes is challenging. PrecisionReach-GS generates robot-aligned wrist-camera training data from 3DGS and adds an instruction-conditioned reach verifier. In motherboard disassembly, the reported experiments show reduced human collection effort and improved manipulation-ready reaching over limited teleoperation data.

  • Problem

    Precise VLA reaching requires aligned visual-action demonstrations, but real-robot collection is slow and noisy, while complex disassembly parts make full digital-twin modeling difficult.

    Source: §I · pp. 1–2

  • Method

    PrecisionReach-GS uses a robot-aligned 3DGS workspace to generate wrist-camera observations and smooth, feasible reaching trajectories, with an instruction-conditioned verifier for stopping.

    Source: §I · p. 2, C. Robot-Aligned 3DGS Data Generation · p. 4

  • Results

    Motherboard disassembly experiments report reduced active human collection effort and improved manipulation-ready reaching over limited teleoperation data.

    Source: §VI · p. 11

  • Takeaways & Limitations

    The demonstrated scope is precise pre-contact reaching with wrist-camera observations, rather than contact-rich manipulation or post-grasp goal selection.

    Source: §I · p. 3, §V · p. 11

    The system does not address contact-rich manipulation or post-grasp goal selection, and larger workspace or component changes may require reconstruction and new target demonstrations.

    Source: §V · p. 11

Abstract

from arXiv · show

Efficient robotic data collection remains a major bottleneck for vision-language-action (VLA) policy learning, especially for tasks requiring accurate end-effector positioning and camera-action alignment. This challenge is particularly severe in electronic-waste disassembly, where robots operate in cluttered assemblies of densely packed, visually similar components. Full digital-twin simulation is difficult because many parts are thin, reflective, or mechanically complex, making accurate object-level meshes and contact models difficult to obtain. In this work, we focus on pre-contact reaching and propose a 3D Gaussian Splatting (3DGS)-based data generation pipeline for wrist-camera VLA adaptation. Our method reconstructs the real workspace with 3DGS and renders diverse wrist-camera observations from robot-aligned poses. These observations are paired with smooth, structured reaching trajectories to provide robot-aligned visual-action supervision for VLA fine-tuning. We further augment the VLA policy with an instruction-conditioned reach verifier that estimates target proximity and provides reliable termination before physical interaction. Experiments in motherboard disassembly demonstrate precise VLA-based reaching using wrist-camera observations alone, without an external side-view camera. With $\pi$0.5, 1,500 generated trajectories achieve 80.0\% precise-reaching success across three targets, compared with 58.7\% using 150 teleoperated demonstrations, while requiring less than 10 minutes of active human capture. Project website: https://001-wang.github.io/PrecisionReach-GS/

I. INTRODUCTION

Precise reaching is a bottleneck in cluttered disassembly, while real-robot demonstrations and full digital twins are costly or difficult to build. PrecisionReach-GS uses robot-aligned 3DGS wrist-camera data and an instruction-conditioned reach verifier for VLA adaptation.

I. INTRODUCTION

  • Real-robot demonstrations are slow, costly, and potentially noisy or camera-action-misaligned, while full digital twins are difficult to model for thin, reflective, mechanically complex motherboard parts.Wrist-camera-only learning also needs diverse, well-aligned observations because the view moves with the end-effector.Source: §I · pp. 1–2
  • PrecisionReach-GS reconstructs the real workspace with 3DGS and generates robot-aligned wrist-camera observations paired with smooth reaching actions for VLA fine-tuning.The pipeline focuses on pre-contact reaching, where realistic observations and aligned action labels are its central requirements.Source: §I · p. 2, Figure 1 · p. 2
  • The method avoids object-level meshes, contact models, and full physics-based digital twins by using 3DGS as a photorealistic workspace representation for VLA adaptation.It samples feasible starting poses and generates synchronized wrist-camera observations, robot states, and smooth reaching actions with minimal human intervention.Source: §I · p. 2
  • An instruction-conditioned reach verifier assesses whether the robot has reached the target interaction region, supporting stopping under targets’ differing geometric precision requirements.The policy and verifier are used in a wrist-camera-only setting for precise target reaching and downstream manipulation skills.Source: §I · pp. 2–3

II. RELATED WORK

Prior robot-learning data pipelines use reconstructed scenes, simulation, or generated demonstrations, while many 3DGS systems target broader manipulation, evaluation, or physics-coupled simulation. PrecisionReach-GS instead uses robot-aligned 3DGS as a lightweight visual data generator for wrist-camera VLA adaptation in pre-contact reaching.

II. RELATED WORK

  • Robot data for VLA adaptation commonly comes from robot logs or human teleoperation, which can be embodiment- or task-specific, slow, labor-intensive, and noisy.Component-level reaching in disassembly is especially sensitive to small spatial errors.Source: §II · p. 3
  • Real-to-sim-to-real work spans object-mesh simulation, photorealistic policy distillation, single-demonstration data generation, and physics-aligned video generation [30]–[34].Examples include X-SIM [32], Real2Render2Real [33], and EmbodieDreamer [34].Source: §II · p. 3
  • 3DGS has been used for robotic reconstruction, policy learning, simulation, and evaluation, including systems that combine it with meshes or physics simulation –[41].The cited examples include RoboGSim [36], PolaRiS [37], GaussGym [38], SplatSim, Robo-GS, and GSWorld [39]–[41].Source: §II · p. 3
  • PrecisionReach-GS differs by using robot-frame-aligned 3DGS for feasible wrist-camera visual-action data generation without object meshes or contact simulation.It targets pre-contact, language-conditioned component reaching rather than full physical simulation or general policy evaluation.Source: §II · p. 3

III. METHOD

PrecisionReach-GS reconstructs the workspace in 3DGS and aligns it with the robot to generate wrist-camera observations for VLA adaptation. A depth-based plane check validates calibration for vertical reaching, while 3DGS preserves scene appearance and context without object-level meshes or contact models.

A. Overview

  • The pipeline captures the workspace and target key poses, calibrates the wrist camera, aligns a 3DGS reconstruction to the robot base, then uses the rendered data to fine-tune a VLA policy.At deployment, the policy runs closed loop with a reach verifier before downstream manipulation.Source: A. Overview · p. 3

B. Real-Scene Reconstruction

  • Hand–eye calibration relates the wrist camera to the end-effector and captures physical mounting offsets, including when a complete CAD-to-optical-frame chain is unavailable.The calibrated transform is used to express the camera pose in the robot base frame.Source: B. Real-Scene Reconstruction · p. 4
  • A depth-based plane check validates calibration because vertical errors along the surface-normal or approach direction can compromise precise pre-contact reaching.Held-out observations of a planar reference surface are transformed into the robot base frame and evaluated by point-to-plane residual.Source: B. Real-Scene Reconstruction · p. 4
  • The workspace is reconstructed as a photorealistic 3DGS scene from wrist-camera videos, retaining clutter, cables, clustered components, and reflective parts without object-level meshes or contact models.Short reaching videos toward target components provide key poses near desired interaction regions.Source: B. Real-Scene Reconstruction · p. 4
  • 3DGS preserves the real workspace’s appearance and surrounding context across novel robot-referenced views, whereas the mesh reconstructed from the same capture shows visible geometry and appearance artifacts.The rendering representation matters because artifacts alter wrist-camera policy inputs, while conventional image augmentation cannot synthesize geometrically consistent views along a new robot trajectory.Source: B. Real-Scene Reconstruction · p. 4, Figure 2 · p. 4

C. Robot-Aligned 3DGS Data Generation

Robot-aligned data generation renders wrist-camera views from feasible robot poses in the aligned 3DGS scene and pairs them with smooth trajectories and policy-specific actions. Kinematic filtering excludes visually plausible poses that are not suitable for physical execution.

C. Robot-Aligned 3DGS Data Generation

  • Each synthesized wrist-camera view is tied to an end-effector pose in the calibrated robot base frame, so observations correspond to physically meaningful robot states.The aligned 3DGS scene is shared across data-generation workers; diversity comes from sampled poses and view-consistent trajectories, not worker- or render-specific domain randomization.Source: C. Robot-Aligned 3DGS Data Generation · pp. 4–5
  • A weighted similarity transformation aligns the SfM reconstruction frame with the robot base frame using matched camera centers from SfM and robot kinematics.Task-relevant anchor frames near target components receive greater weight to improve local alignment accuracy around interaction regions.Source: C. Robot-Aligned 3DGS Data Generation · p. 4
  • Candidate start poses are accepted only when they lie within a bounded safe workspace, have sufficiently small IK residuals, satisfy joint limits, and are not near a kinematic singularity.The filtering prevents generated views and trajectories from representing poses that are visually plausible but physically infeasible.Source: C. Robot-Aligned 3DGS Data Generation · p. 4
  • For each accepted start pose, the method generates a smooth task-space path through a pre-approach pose to the target key pose, without using a sampling-based collision-aware planner such as RRT.Cubic smoothstep timing gives zero endpoint velocity to avoid abrupt segment transitions; rendered intermediate views and actions are converted to each VLA policy’s representation.Source: C. Robot-Aligned 3DGS Data Generation · pp. 4–5

D. Policy Learning

The generated reaching samples fine-tune the VLA policy by behavior cloning, pairing wrist-camera observations, target instructions, robot states, and actions. At deployment, actions are scaled and clipped, and action chunks shrink over the rollout to increase visual feedback near the target.

D. Policy Learning

  • Each behavior-cloning sample pairs a wrist-camera observation, target instruction, robot state, and reaching action to adapt the policy.Source: D. Policy Learning · p. 5
  • Predicted actions are scaled and clipped before execution, with joint-space targets constrained by the action scale, maximum joint update, and joint limits.Pose-space relative end-effector motions are also scaled and clipped before reaching the robot controller.Source: D. Policy Learning · p. 5
  • The coarse-to-fine action-chunking strategy gradually reduces the number of actions executed per query as rollout progresses.This supports fast motion early and more frequent visual feedback near the target.Source: D. Policy Learning · p. 5

E. Instruction-Conditioned Reach Verification

The reach verifier estimates whether the end-effector has reached the interaction region, accounting for target-specific geometric tolerances. It combines wrist-camera, instruction, and robot-state inputs, then uses persistent high-confidence predictions to determine when to stop.

E. Instruction-Conditioned Reach Verification

  • Target-dependent tolerances motivate a verifier that judges whether the end-effector has reached a sufficiently precise interaction configuration for the instructed target.Screws require tight vertical approach accuracy, while GPUs and RAM allow more variation along particular axes.Source: E. Instruction-Conditioned Reach… · p. 5
  • The separately trained verifier estimates target distance from wrist-camera observations, the target instruction, and robot state, using Cartesian-distance supervision and a Smooth-L1 loss.Source: E. Instruction-Conditioned Reach… · p. 5
  • Instruction-conditioned FiLM and spatial attention focus visual features on the target, while random state masking discourages reliance on workspace-specific shortcuts.The encoded state is zeroed with probability 0.5 during training.Source: E. Instruction-Conditioned Reach… · p. 5
  • At deployment, a predicted distance below 0.65 cm counts as high confidence, and termination requires three consecutive such predictions, with an optional geometric check.The persistence rule reduces premature stops and overshooting while allowing continued motion correction.Source: E. Instruction-Conditioned Reach… · p. 5

F. Downstream Grasp Execution

In real-robot tests, π0.5 reached all three targets precisely with 1,500 generated trajectories, while the evaluation tested a repositioned motherboard using wrist-camera-only input. The verifier hands off the reached pose to a fixed downstream controller.

F. Downstream Grasp Execution

  • After verifier confirmation, the current end-effector pose becomes the grasp pose and control passes to a fixed deterministic grasp-and-place controller.The controller is independent of the reaching framework and remains fixed across trials.Source: F. Downstream Grasp Execution · pp. 5–6, Figure 3 · p. 6

A. Experimental Setup and Evaluation Protocol

  • Evaluation used a Franka FR3 and three language-conditioned targets; PrecisionReach-GS used wrist-camera observations alone, while teleoperation used wrist and side views.Tests followed physical motherboard repositioning, with precise reach validated by the fixed grasp-and-lift routine.Source: A. Experimental Setup and Evaluation… · p. 6

B. Main Real-Robot Performance and Data Efficiency

  • At the largest generated-data scale, π0.5 achieved 21/25 precise screw successes, 19/25 precise GPU successes, and 20/25 precise RAM successes in real-robot trials.The GPU target is slightly more challenging because it is larger and taller than the other targets and requires greater wrist rotation; near the target, that rotation can occasionally cause a collision.Source: B. Main Real-Robot Performance and… · p. 7, Table II · p. 8
  • PrecisionReach-GS requires less than 10 minutes of active human capture for up to 1,500 trajectories, versus 90–200 minutes of continuous control for 50–150 teleoperation demonstrations.Generated trajectories also have lower RMS jerk, with reductions in median, 95th-percentile, and maximum jerk.Source: B. Main Real-Robot Performance and… · pp. 7–8

C. Robustness to the Initial Wrist-Camera View

The initial wrist-camera view affects reaching robustness: performance remains high with full or limited workspace context but drops when the workspace is initially out of view. The policy can nevertheless recover in some no-context trials as the workspace enters the camera view.

C. Robustness to the Initial Wrist-Camera View

  • The study groups starting views as full, limited, or no workspace context; in the no-context condition, the motherboard is outside the initial wrist-camera view.Peripheral cues such as the table edge may remain visible.Source: C. Robustness to the Initial… · p. 8, Table IV · p. 9
  • A representative no-context trajectory shows the workspace entering the wrist-camera view as the robot moves, followed by continued motion toward the target.Source: C. Robustness to the Initial… · pp. 8–9
  • With 1,500 trajectories, precise reach succeeded in 29/30 full-context trials and 27/30 limited-context trials, versus 8/30 when the workspace was initially absent.In the no-context condition, the policy reached the target neighborhood in 17/30 trials.Source: C. Robustness to the Initial… · p. 9, Table IV · p. 9

D. Controlled Comparisons and Ablations

Controlled comparisons examine the effects of observation modality and trajectory smoothness, while component ablations test the verifier and task-keyframe weighting. The reported ablations distinguish precise reaching from coarse arrival and identify infeasible labels as an unsuitable baseline for start-pose filtering.

D. Controlled Comparisons and Ablations

  • Controlled π0.5 comparisons test wrist-only teleoperation and linear-timed generated trajectories against the practical setup, finding contributions from the side view and smooth trajectory supervision.The generated wrist-only setting remains directly evaluable.Source: D. Controlled Comparisons and Ablations · p. 9
  • Removing the reach verifier leaves coarse success unchanged but reduces precise success, consistent with overshoot or premature termination near the target.Source: D. Controlled Comparisons and Ablations · p. 9
  • Removing task-keyframe weighting from robot-frame alignment degrades both coarse and precise metrics; disabling start-pose validity checks yields physically infeasible labels.These ablations use 900 generated episodes with the OpenPI implementation of π0.5.Source: D. Controlled Comparisons and Ablations · p. 9

E. Reach Verifier Ablation and Robustness

The reach verifier is evaluated for accuracy and robustness, while the reconstructed scene is assessed for visual fidelity and robot-frame alignment. Results show a trade-off between clean-input accuracy and robustness to state bias, alongside rendered views and geometric alignment suitable for robot-frame wrist-view rendering and feasible reaching.

E. Reach Verifier Ablation and Robustness

  • The verifier evaluation uses 142 held-out episodes containing 21,476 frames, and counts a stop as correct when true target distance is below 1 cm.Inference triggers a stop after three consecutive predicted distances below 0.65 cm.Source: E. Reach Verifier Ablation and… · pp. 9–10
  • Without workspace context, the recovery trajectory gradually brings the motherboard into view, allowing the policy to continue toward the target region.Figure 7 pairs robot poses in its top row with corresponding wrist-camera observations below.Source: Figure 7 · p. 10
  • Removing instruction conditioning reduces stop precision under biased state from 48.1±11.6% to 22.8±16.1%, with only a modest change under clean inputs.The full model’s small clean-condition differences from individual FiLM and spatial-attention ablations indicate neither mechanism alone dominates verifier performance.Source: E. Reach Verifier Ablation and… · p. 10
  • Under a 2 cm downward end-effector height bias, the full verifier retains 48.1 ± 11.6% episode-level stop precision, while removing state masking drops precision to 0%.Without masking, clean-input MAE improves from 0.64 to 0.44 cm and near-target MAE from 0.10 to 0.08 cm.Source: E. Reach Verifier Ablation and… · p. 10, Table VIII · p. 11
  • With clean inputs, the full verifier achieves 0.64 cm overall MAE, 0.10 cm near-target MAE, and 99.8% episode-level stop precision.The near-target MAE is 0.10 cm for the full model versus 0.12 cm for the three language-conditioning ablations.Source: E. Reach Verifier Ablation and… · p. 10, Table VIII · p. 11

F. Rendering Fidelity and Robot-Frame Alignment

  • Rendered wrist-camera views preserve overall scene appearance, component layout, and viewpoint-dependent content; mean correlation is 0.843, with local reconstruction and illumination differences remaining.The lower SSIM is consistent with sensitivity to small viewpoint and illumination changes and local reconstruction artifacts.Source: F. Rendering Fidelity and Robot-Frame… · p. 10, Figure 8 · p. 11, Table IX · p. 11
  • Robot-frame position RMSE ranges from 0.0094 to 0.0114 m, task-keyframe mean position errors stay below 0.0040 m, and mean orientation errors remain below 0.58°.The measurements support robot-frame wrist-view rendering and feasible reaching trajectory generation.Source: Table X · p. 11, F. Rendering Fidelity and Robot-Frame… · p. 11

V. DISCUSSION AND LIMITATIONS

The discussion bounds the system to precise reaching and moderate target-location changes, while the conclusion presents robot-aligned 3DGS as a route to wrist-camera-only VLA learning in cluttered workspaces.

V. DISCUSSION AND LIMITATIONS

  • Moderate motherboard repositioning is tolerated without rebuilding the dataset, but larger workspace or component changes may require a new reconstruction and target demonstrations.Performance may also degrade with calibration errors, out-of-distribution viewpoints, persistent occlusion, or severe reconstruction artifacts.Source: §V · p. 11
  • The current system targets precise reaching, not contact-rich manipulation or post-grasp goal selection.Future work could pair long-horizon approach with a contact-aware residual policy using visual, force, or tactile feedback.Source: §V · p. 11

VI. CONCLUSION

  • PrecisionReach-GS generates wrist-camera observations and smooth, feasible reaching trajectories from a robot-aligned 3DGS workspace without large-scale teleoperation, object-level meshes, contact models, or a complete physics-based digital twin.Motherboard disassembly experiments show reduced active human collection effort and improved manipulation-ready reaching over limited teleoperation data.Source: §VI · p. 11
  • The results support robot-aligned visual generation as a practical route to wrist-camera-only VLA learning in cluttered workspaces.Controlled comparisons and ablations identify the roles of observation modality, smooth trajectory timing, keyframe-weighted alignment, and stop verification.Source: §VI · pp. 11–12

Related papers

Loading 2610.35626v1…