PaperlayerView exact revision on arXiv

Source-linked AI summary

From Target Selection to Digging: A Learning-Based Framework for Continuous Autonomous Excavation

Shuai Zhao, Ji-an Pan, Quantao Yang, Zheng Wang, Chaoyi Chen, Qing Xu, Keqiang Li

arXiv:2609.29750v1cs.RO

TL;DR

Repeated excavation reshapes the pile, requiring target selection and coordinated motion to adapt across cycles. The paper combines terrain-aware selection, shared task-conditioned RL motion, and demonstration-trained digging on a scaled excavator; its studies report more stable targets, shorter local motion time, and higher payload against respective baselines, alongside consecutive five-scoop runs.

  • Problem

    Repeated excavation changes pile geometry, while target-conditioned motion and material acquisition impose different control objectives and supervision requirements.

    Source: §I · p. 1

  • Method

    The framework updates targets from LiDAR elevation maps, shares a task-conditioned RL policy for waypoint-guided approach and loaded transport, and uses an ACT policy trained on real demonstrations for digging and lifting.

    Source: §I · p. 1

  • Results

    Component studies report more stable targets, shorter local motion time, and higher completed-cycle payload against respective baselines; three five-scoop runs demonstrate consecutive autonomous operation without human intervention.

    Source: §VI · pp. 7–8, §VII · p. 8

  • Takeaways & Limitations

    Waypoint references connect updated geometric goals to a reusable local digging skill during coordinated operation on a physical hydraulic platform.

    Source: §VI · p. 7, §VII · p. 8

Abstract

from arXiv · show

Repeated excavation continuously reshapes pile geometry, requiring an autonomous excavator to adapt its digging targets and coordinate motion across successive excavation cycles. We present a learning-based framework for continuous autonomous excavation that integrates terrain-aware target selection with reinforcement- and imitation-learning controllers. The framework separates target-conditioned motion from local digging: a shared task-conditioned RL policy controls waypoint-guided approach and loaded transport, while an IL policy learns vision-based digging and lifting from expert demonstrations. Digging targets are selected from LiDAR elevation maps and converted into bucket-tip waypoints for motion control. The control architecture coordinates the learned policies and deterministic unloading through a shared motion interface. The complete system is deployed on a scaled hydraulic excavator with multimodal sensing and closed-loop actuator control. Offline replay and physical experiments demonstrate more consistent target selection, shorter local motion time, and increased payload compared with the respective baselines. The learned digging policy achieves a mean payload of 6.52 kg per completed cycle, compared with 2.68 kg for Fixed Dig. Three five-scoop runs further demonstrate consecutive autonomous excavation under continuously changing pile geometry.

I. INTRODUCTION

Repeated excavation changes pile geometry, so autonomous systems must select new digging locations and coordinate approach, digging, transport, and unloading. The paper presents an integrated learning-based framework that connects terrain-aware target selection, task-conditioned motion, and demonstration-trained digging.

I. INTRODUCTION

  • Terrain-aware target selection and learning-based excavation remain challenging to connect because target-conditioned motion and material acquisition have different control objectives and supervision requirements.Hydraulic dead zones and observation noise also complicate motion execution and target updates as the pile changes.Source: §I · p. 1
  • The framework updates LiDAR-map targets as the pile evolves, shares a task-conditioned RL policy across approach and loaded transport, and uses an ACT policy trained on real expert demonstrations for digging and lifting.The policies coordinate with deterministic unloading and repeated terrain observation to close the excavation cycle.Source: §I · p. 1
  • The scaled hydraulic platform integrates multimodal sensing, actuator feedback, onboard policy inference, and closed-loop actuator control.The introduction identifies these components as enabling deployment of the framework on physical hardware.Source: §I · p. 2
  • 6.52 kg per completed cycle versus 2.68 kg for Fixed Dig; three five-scoop runs also completed without human intervention.The framework combines target selection, motion, and digging components on a scaled hydraulic excavator.Source: §I · p. 2

A. Autonomous Excavation Systems and Target Selection

  • The Temporal Target Selector filters and scores elevation-map candidates, then uses cross-frame confirmation and hysteresis before passing a target to a separate waypoint planner.The related-work discussion distinguishes this focus on temporal consistency across repeated observations from prior target-selection approaches.Source: A. Autonomous Excavation Systems and… · p. 2

B. Motion Control for Hydraulic Machines

The framework assigns target-conditioned motion and local material acquisition to different learned components, linked by waypoint references and coordinated execution. Its target selector filters LiDAR elevation-map candidates and uses temporal confirmation to stabilize selections as the pile changes.

B. Motion Control for Hydraulic Machines

  • Hydraulic motion-control research includes trajectory optimization, predictive control, and learned policies; the RL Tracker instead shares one task-conditioned policy between empty-bucket approach and loaded transport.Its reward combines task-dependent bucket-orientation objectives with penalties for commands within direction-dependent dead zones.Source: B. Motion Control for Hydraulic Machines · p. 2

C. Learning Excavation Skills

  • Prior excavation-skill work ranges from expert-generated trajectories to learned action policies; this framework assigns changing digging locations to LiDAR selection and shared RL motion, while standard ACT handles local digging and lifting from RGB and proprioception.Bucket-tip waypoint references connect updated targets to the demonstration-trained skill.Source: C. Learning Excavation Skills · p. 2

III. AUTONOMOUS EXCAVATION FRAMEWORK

  • A waypoint planner links terrain targets to RL approach and loaded transport, while a mission scheduler and shared motion interface coordinate those policies with deterministic unloading.The design uses a pre-dig position and bucket pitch to bring the demonstration-trained digging skill to a suitable state; ACT then acts on image and machine state without an explicit terrain-goal input.Source: §III · p. 2

A. Temporal Target Selector

  • The Temporal Target Selector forms a pile mask from height, confidence, and point-support requirements, then filters candidates by relative height, slope, boundary clearance, and local observation support.It scores candidates using soil thickness, confidence, boundary clearance, relative-height preference, and roughness; cross-frame confirmation and hysteresis reduce observation-induced switches while allowing targets to change with the pile.Source: A. Temporal Target Selector · pp. 2–3

B. RL Tracker

The RL Tracker uses one policy for waypoint-guided approach and loaded transport, conditioning motion on task mode and target pitch. Its reward combines waypoint tracking and task-specific control objectives with penalties for undesirable commands and motion.

B. RL Tracker

  • One shared policy serves approach and transport by conditioning on task mode and target pitch, which changes from pile entry to material retention.The planner converts each selected target into a pre-dig goal and generates bucket-tip waypoints toward the goal or unloading location.Source: B. RL Tracker · p. 3
  • The policy observes proprioception, bucket-tip state, waypoint errors, path progress, task mode, pitch information, episode progress, and the previous action, then outputs normalized cylinder-speed and swing-rate references.It is trained with PPO on randomized waypoint trajectories in simulation.Source: B. RL Tracker · p. 3
  • Direction-specific dead-zone penalties target nonzero commands below actuator response thresholds, while separate reward terms address waypoint tracking, transport stopping, and swing near the destination.The dead-zone cost is proportional to 4z(1 − z) when 0 < |a| < D and is zero otherwise.Source: B. RL Tracker · p. 3

C. Demonstration-Based Digging and Lifting

Local digging and lifting are learned from real teleoperation demonstrations, with ACT predicting joystick-command chunks from visual and proprioceptive observations. The control architecture transfers command authority across approach, digging, transport, and unloading, then renews terrain observation.

C. Demonstration-Based Digging and Lifting

  • ACT learns coordinated penetration, curling, and lifting from real teleoperation demonstrations, predicting normalized joystick-command sequences from front RGB and proprioception.The proprioceptive inputs include cylinder, joint, and swing states.Source: C. Demonstration-Based Digging and… · p. 3
  • Only the first L < H commands in an ACT chunk execute before the policy receives the latest image and proprioception, unless the stage terminates earlier.This partial-chunk execution inserts observation feedback between action segments.Source: C. Demonstration-Based Digging and… · p. 3

D. Control Architecture

  • Approach completion triggers ACT; ACT termination initiates loaded transport; transport completion triggers Fixed Dump and renewed terrain observation.A mission scheduler selects the active command source according to the current stage.Source: D. Control Architecture · p. 4
  • A shared motion controller transfers command authority while preserving RL Tracker velocity-reference and ACT joystick semantics; both paths use common direction mapping, actuator limits, and a watchdog.All policies use the axis order [boom, stick, bucket, swing].Source: D. Control Architecture · p. 4

IV. EXPERIMENTAL SETUP

The experimental setup evaluates target selection, motion and transfer, digging payload, and repeated autonomous operation on an integrated scaled hydraulic excavator. It also specifies the measurement procedures and target-selector settings used in these experiments.

IV. EXPERIMENTAL SETUP

  • The evaluation covers target-selection stability, motion execution and transfer, digging payload, and consecutive autonomous operation.Component studies examine individual modules; integrated runs test coordination over repeated cycles.Source: §IV · p. 4

A. Hardware Platform and System Integration

  • The scaled hydraulic excavator integrates multimodal sensing, onboard policy inference, and closed-loop actuator control.Boom, stick, and bucket actuation are hydraulic, while swing is electric; LiDAR supplies elevation maps and a 5-cm-voxel OctoMap.Source: A. Hardware Platform and System… · p. 4
  • Payload measurements use a 500-kg scale with 0.05-kg display division, with the truck emptied and re-tared before each measurement.Component comparisons weigh individual scoops; continuous runs weigh five-scoop totals and also record per-scoop payloads.Source: A. Hardware Platform and System… · p. 4

B. Implementation and Training

  • The Temporal Target Selector scores 10-cm map cells and applies candidate thresholds plus temporal association, confirmation, replacement-margin, and dropout-retention settings.Candidate thresholds include soil height ≥0.15 m, confidence ≥0.70, and score ≥0.45; selection uses three-frame confirmation and at most ten-frame dropout retention.Source: 1) Target selection · p. 4

1) Target selection:

The motion policy uses proprioceptive and waypoint-related observations to guide approach and transport, while the learned digging policy acts from RGB and proprioception. Training and reward settings specify how these policies are deployed and optimized.

1) Target selection:

  • The RL Tracker receives a 38-D observation combining machine and bucket-tip state, waypoint errors, path progress, task mode, pitch information, and previous action.An adapter maps base-frame kinematics to training coordinates.Source: 2) Motion policy · p. 4

2) Motion policy:

  • Approach and transport use target pitches of 70° and 180°, measured relative to the upper structure’s forward axis in its sagittal plane.That plane rotates with swing.Source: 2) Motion policy · p. 4
  • The waypoint reward weights progress, distance, path deviation, and pitch by 5, 0.3, 0.5, and 0.2, respectively.Distance and pitch errors are normalized by 1.13 m and 180°; waypoint arrival and path completion receive bonuses of 1 and 10.Source: 2) Motion policy · p. 4
  • The motion reward also penalizes dead-zone behavior and adds a final-waypoint braking cost based on remaining distance and swing motion.The dead-zone cost has weight 0.03; the braking expression uses remaining distance in meters and normalized swing velocity and command.Source: 2) Motion policy · p. 4
  • ACT is trained on 104 accepted teleoperation episodes containing 13,007 synchronized RGB–proprioception–action frames, split into 83 training and 21 validation episodes.It predicts 20-action chunks and executes ten commands before querying updated observations; deployment weights remain frozen.Source: 3) Digging policy · p. 4

3) Digging policy:

The protocols test offline target stability, local empty-bucket motion, and simulation-to-real task completion. They define the compared methods, task conditions, and metrics used for those assessments.

1) Offline target stability:

  • Three soil resets provide six static sequences for comparing highest-point selection, Spatial Only, and the Temporal Target Selector.All methods share recorded starts, obstacles, and an approach offset; each sequence contains 1,062 maps.Source: 1) Offline target stability · pp. 4–5

C. Evaluation Protocols

  • The repeated excavation workflow assigns approach and loaded transport to the RL Tracker, digging and lifting to ACT, and unloading to Fixed Dump.The Temporal Target Selector chooses the next target before waypoint planning and the next approach.Source: Figure 3 · p. 5

1) Offline target stability:

  • Offline stability is measured by adjacent-valid-frame XY displacement, jumps above 0.1 m, and output fractions; missing outputs are not bridged.The static sequences test observation-induced variation, with frames nested within sequences and soil resets.Source: 1) Offline target stability · p. 5

2) Local empty-bucket motion:

  • The local-motion comparison tests five 10-cm upward motions from matched initial states and requires position error within 2 cm in 30 seconds.A separate ten-task, three-waypoint simulation-to-real test measures completion, tracking duration, and terminal target distance; its completion tolerance differs from the 2-cm local-motion criterion.Source: 2) Local empty-bucket motion · p. 5, 3) Simulation–real task completion · p. 5

3) Simulation–real task completion:

The protocols compare digging skills, target-selection methods, and repeated operation on the physical machine. They also define how payload, run performance, and controller handoffs are measured.

3) Simulation–real task completion:

  • The digging-skill comparison uses a shared sequence of five catalog targets, RL Tracker motion control with terminal braking, and unloading.Fixed Dig uses cylinder-position feedback for entry, curl/pull, and lift; ACT retains joystick semantics under the same final actuator limits.Source: 4) Digging-skill payload · p. 5

4) Digging-skill payload:

  • ACT and Fixed Dig are compared over four paired five-scoop blocks, yielding 20 weighed completed cycles per method.Payload is reported as block totals and mean completed-cycle payload, with paired blocks as the replication unit; each method follows its own stopping rule.Source: 4) Digging-skill payload · p. 5

5) Physical target selection:

  • Six paired comparisons test ACT with the Temporal Target Selector versus global highest-point selection on leveled soil and a central depression.Three completed five-scoop runs assess consecutive operation; target height is adjusted and pool-edge, X, and Z constraints are applied before planning, so executed targets can differ from raw targets.Source: 5) Physical target selection · p. 5, 6) Integrated operation and handoffs · pp. 5–6

6) Integrated operation and handoffs:

  • Integrated-operation reporting includes run and per-scoop payload, duration, interventions, return status, and controller handoff time.Handoff time spans the outgoing terminal-zero acknowledgment to the incoming first-nonzero acknowledgment on one Jetson monotonic clock.Source: 6) Integrated operation and handoffs · p. 6

V. RESULTS

The Temporal Target Selector sharply reduces target switching, while the RL Tracker completes local empty-bucket motions faster on average than the comparison controller. The section also reports offline target-selection and motion-evaluation protocols.

A. Offline Target Stability

  • 1 XY jump versus 300 for highest-point selection and 523 for Spatial Only across adjacent valid-output pairs.The Temporal Target Selector returned 1048/1062 targets (98.68%), while both alternatives returned a target in every frame.Source: A. Offline Target Stability · p. 6, Table I · p. 6
  • Temporal Target Selector outputs concentrate on both initial and depressed surfaces, and every emitted target passes the common offline planner.The offline comparison evaluates target selection over initial and depressed surfaces within three soil resets.Source: A. Offline Target Stability · p. 6, Figure 4 · p. 6, Table I · p. 6

B. Empty-Bucket Motion

  • The RL Tracker reduces mean tracking time from 1.93 to 1.04 s (46.1%) while both controllers meet the 2-cm tolerance in all five trials.Terminal kinematic errors remain similar, mean absolute pitch change is smaller, and tracking time is more variable for the RL Tracker (SD 0.54 versus 0.11 s).Source: B. Empty-Bucket Motion · p. 6, Table II · p. 6

C. Simulation–Real Task Completion

The learned digging skill outperforms Fixed Dig in completed-cycle payload, and autonomous operation completes paired simulation–real tasks and consecutive five-scoop runs. Physical target-selection trials also complete without interruption.

C. Simulation–Real Task Completion

  • All ten paired tasks complete in simulation and on the physical machine, with mean terminal distances of 24.12 and 24.21 cm, respectively.These distances are near the 25-cm stopping boundary; mean tracking durations are 2.686 s in simulation and 3.061 s on the machine.Source: C. Simulation–Real Task Completion · p. 7

D. Digging-Skill Payload

  • 6.52 kg versus 2.68 kg per cycle: ACT delivers more payload than Fixed Dig across 20 weighed completed cycles per method.All four paired blocks favor ACT, with a mean block difference of 19.19 kg.Source: D. Digging-Skill Payload · p. 7

E. Physical Target Selection

  • All twelve physical target-selection cycles complete without interruption; the Temporal Target Selector has a mean paired payload difference of 0.325 kg, with four positive pairs and two ties.The comparison is one cycle per method in each pair, across leveled-soil and central-depression conditions.Source: E. Physical Target Selection · p. 7, Table III · p. 7

F. Integrated Five-Scoop Operation

  • Three five-scoop runs complete without intervention or recorded anomaly, delivering 94.75 kg total and a mean run payload of 31.58±1.61 kg.The fifteen scoop payloads average 6.32±0.33 kg; index-wise means span 6.17–6.43 kg, demonstrating continued acquisition through the fifth scoop as the pile changes.Source: F. Integrated Five-Scoop Operation · p. 7, Table IV · p. 7
  • Autonomous scoops take 38.8–47.0 s including startup and finalization, compared with 32 s and 45 s in the human records.Professional demonstrations and developer teleoperation average 6.95 and 5.15 kg per scoop, respectively.Source: F. Integrated Five-Scoop Operation · p. 7
  • RL-to-ACT handoffs have a median interval of 199.74 ms, while ACT-to-RL handoffs have a median of 150.42 ms.The measurements cover thirty handoffs across the three runs and characterize command transitions at the acknowledgment boundary.Source: F. Integrated Five-Scoop Operation · p. 7

VI. DISCUSSION

The discussion distinguishes component-level benefits from integrated feasibility and explains how waypoint references connect changing digging locations to a reusable skill. The conclusion reports improved component outcomes and consecutive autonomous operation as pile geometry changes.

VI. DISCUSSION

  • Waypoint following establishes a state for the demonstration-trained digging skill, allowing geometric digging locations to change without retraining ACT or giving it an explicit terrain goal.Source: §VI · p. 7
  • The studies distinguish component benefits from integrated feasibility: target stability, motion execution and transfer, and payload are examined separately before combined operation.Source: §VI · p. 7
  • Fig. 7 plots per-scoop payload across three five-scoop runs: thin lines show S1–S3, the thick line averages each index, and dashed lines show professional-demonstration and developer-teleoperation means.The professional-demonstration mean uses 104 scoops and is also used for ACT; the developer-teleoperation mean uses 20 scoops.Source: Figure 7 · p. 8, §VI · p. 8
  • The evaluation spans target variation on static terrain, digging under a shared target sequence, and motion execution; continuous operation is limited to five-scoop sequences on one stationary scaled machine and one soil type.Target location and scoop order are coupled in the digging comparison.Source: §VI · p. 8

VII. CONCLUSION

  • Component studies show more stable targets, shorter local motion time, and higher completed-cycle payload than the respective baselines.Source: §VII · p. 8
  • Three five-scoop runs demonstrate consecutive autonomous operation without human intervention as pile geometry changes.Source: §VII · p. 8
Loading 2609.29750v1…