PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Adaptive Interaction Strategies for Human Companion Robot via Deep Reinforcement Learning

Cong-Thanh Vu, Yen-Chen Liu

arXiv:2609.25031v1cs.ROeess.SY

TL;DR

Fixed tracking positions can be impractical in dynamic or spatially constrained environments, yet adaptive human accompaniment remains underexplored. The paper combines interactive-space-based reinforcement learning with MPPI-CBF control, reporting improved success rate and tracking accuracy in real-world tests.

  • Problem

    Few studies address dynamically adjusting human-tracking positions, although fixed positions can be impractical in real-world environments.

    Source: §I · p. 2

  • Method

    The method uses an interactive space as input to an RL policy for tracking-position selection and an MPPI-CBF controller for motion execution.

    Source: §I · pp. 2–3

  • Results

    At least 24% improvement in success rate and 47% improvement in tracking accuracy were reported for the proposed method.

    Source: A. Comparison Results · p. 8

  • Takeaways & Limitations

    Real-world experiments validate adaptive switching among front, side, and rear tracking positions in indoor and outdoor environments.

    Source: §VII · p. 13

    The interaction-space parameters are manually defined.

    Source: §VII · p. 13

Abstract

from arXiv · show

In the field of Human-Robot Interaction (HRI), achieving flexibility in human-accompanying within real-world environments holds great potential for various applications but also poses significant challenges. Traditional methods typically restrict robots to fixed positions relative to humans, such as tracking from behind, in front, or side-by-side, which limits robot adaptability in dynamic workspaces. This study introduces a novel human-companioning strategy that uses Reinforcement Learning (DRL) to enable mobile robots to dynamically adjust their tracking positions according to varying conditions. An interaction space is defined to capture the relationship between the human and the robot while considering the environment, which serves as the basis for state spaces in DRL to assist the robot in adapting to environmental changes. A human-robot companion controller is developed by integrating Model Predictive Path Integral (MPPI) control with Control Barrier Functions (CBF), ensuring that the robot accurately follows the target's movement in both position and orientation while avoiding obstacles and enhancing social acceptance and safety. The proposed approach is evaluated in real-world scenarios, both indoors and outdoors, and compared with other studies. The results show that the proposed method improves the success rate and tracking accuracy by at least 24% and 47%, respectively, while enhancing human comfort. Experiments demonstrate the robot's ability to flexibly accompany a person walking at speeds of up to 1.7 m/s, dynamically adjusting its strategy without being confined to a fixed position. Additionally, the robot respects the human's intimate space to ensure safety, comfort, and effective obstacle avoidance.

I. INTRODUCTION

Flexible human accompaniment is needed because fixed tracking positions can be impractical in real environments. The paper proposes context-sensitive position selection with reinforcement learning and MPPI-CBF control.

I. INTRODUCTION

  • Following from behind can cause discomfort and complicate collaboration, while limited camera field of view can leave rear-following systems vulnerable to occlusion.People may repeatedly check the robot’s position, and rear placement can hinder placing or retrieving objects.Source: §I · p. 1
  • Fixed tracking positions can be impractical in constrained environments, motivating robots to adjust position as surroundings change.Side-by-side movement may be spatially impractical, while many existing approaches focus on front-tracking in static or simplified settings.Source: §I · p. 2
  • Classical control offers safety guarantees and accurate dynamic modeling but can lack adaptability, whereas learning-based methods adapt without an explicit model but face sim-to-real and low-level safety challenges.These trade-offs are especially relevant to human–robot interaction.Source: §I · p. 2
  • The approach defines an interactive zone around the human and combines it with an objective function to provide input to an RL model that selects a tracking position.The proposed strategy accounts for workspace context and social acceptance rather than relying on fixed positions.Source: §I · p. 2
  • An MPPI controller combined with Control Barrier Functions executes accompaniment while supporting safety, social compliance, and obstacle avoidance.The paper presents this controller as part of its adaptive human-accompanying framework.Source: §I · p. 3
  • The framework achieves up to 24% higher success rate and 47% better tracking accuracy than representative state-of-the-art methods.It was validated on an omnidirectional robot indoors and outdoors without prior map information.Source: §I · p. 3

A. Design of the Human-Companion Robot

The robot combines onboard sensing and omnidirectional motion for human accompaniment. Its formulation tracks relative human–robot position and orientation while allowing the desired tracking position to adapt to the environment.

A. Design of the Human-Companion Robot

  • The platform uses a 3D LiDAR for environmental perception and an electromagnetic tracker for human pose, with an IMU-equipped camera estimating robot motion.The tracker provides six-degree-of-freedom relative pose measurements and remains effective under visual occlusion.Source: A. Design of the Human-Companion Robot · p. 3
  • The RL policy operates at 30 Hz, while the tracking controller operates at 100 Hz.The IMU runs at 200 Hz, human position updates at 120 Hz, odometry at 50 Hz, and LiDAR measurements at 10 Hz.Source: A. Design of the Human-Companion Robot · p. 3

B. Problem Formulation

  • Robot and human states encode their global positions and orientations, while the human’s relative pose is represented in the robot’s local frame.The control objective includes relative position and orientation between the robot and human.Source: B. Problem Formulation · pp. 3–4
  • Because fixed desired tracking positions can be impractical amid limited workspace and obstacles, the method adjusts the robot’s position around the human within an interactive space while avoiding collisions.Source: B. Problem Formulation · p. 4

III. HUMAN-ROBOT INTERACTIVE SPACE

The section defines a human-centered interactive space, filters it for obstacles, and selects desired tracking positions by balancing proximity and navigation costs. Because the objective weights depend on the environment, the paper motivates learning adaptive position choices.

A. Collision-Free Interactive Space

B. Objective Function

  • The objective balances distance to the human, distance to the robot, and local obstacle cost when selecting a desired tracking point.The robot-to-point distance helps avoid unstable motion, while the costmap term supports obstacle avoidance.Source: B. Objective Function · p. 4
  • Because objective weights vary by environment, manually tuning them is impractical across diverse real-world settings, motivating RL-based position selection.The architecture passes policy-selected positions to the human-companion controller.Source: B. Objective Function · p. 4, Figure 3 · p. 5

IV. RL-BASED HUMAN-ACCOMPANYING STRATEGIES

The paper uses the interactive space as part of the RL state, extending beyond approaches that consider only human–robot relative position. The policy selects a collision-free point as the robot’s desired tracking position.

IV. RL-BASED HUMAN-ACCOMPANYING STRATEGIES

  • The RL state uses the proposed interactive space rather than only the human–robot relative position, providing the basis for adaptive companioning.The paper pairs this state representation with a reward function designed for human-accompanying.Source: §IV · p. 5

A. State Space and Action Space

  • The state includes the collision-free interactive space, obstacle cost map, robot velocity, human-relative position and velocity, and previous action.These observations inform the robot’s decision at each time step.Source: A. State Space and Action Space · p. 5
  • Each discrete action indexes a collision-free point in P, which becomes the robot’s desired tracking position.The policy’s selected point is used as the target for the human-companion controller.Source: A. State Space and Action Space · p. 5, §IV · p. 5

B. Reward Functions

The reward design combines human-distance, action-smoothness, and collision-avoidance terms. It balances responsive tracking against smooth motion while assigning collision avoidance overriding priority.

B. Reward Functions

  • The distance-to-human reward encourages closer tracking, rewarding progress toward the human and penalizing movement farther away.A positive tracking reward applies when the robot remains within permissible distance D.Source: B. Reward Functions · p. 5
  • The action-smoothness reward discourages abrupt position changes to stabilize the robot’s trajectory.It penalizes large changes between consecutive actions.Source: B. Reward Functions · p. 5
  • Tracking and smoothness rewards trade off responsiveness against stability: emphasizing tracking can produce jerky motion, while overemphasizing smoothness can make adaptation sluggish.The reward design addresses factors often overlooked in prior RL human-following work, including obstacle avoidance and the safe interaction zone.Source: B. Reward Functions · pp. 5–6
  • The collision penalty is −200 versus a maximum tracking reward of 50, prioritizing safety over tracking accuracy and smoothness near obstacles.The penalty applies when LiDAR measures obstacle distance below safety threshold R.Source: B. Reward Functions · pp. 5–6

C. Policy Training

The policy selects a tracking position in the interactive space, while MPPI executes that choice and compensates for physical and environmental variation. The accompanying controller adds a barrier-function safety mechanism to keep the robot from entering the human’s intimate zone.

C. Policy Training

  • Domain randomization covers walking speed, lateral deviation, sensor noise, relative position, obstacle placement, and interaction-space dimensions.The policy uses PPO, with observations normalized to align simulated and real sensor-data scaling.Source: C. Policy Training · p. 6
  • The proposed architecture is less sensitive to σh and σa than the RL baseline, whose performance degrades significantly.The high-level policy selects a target position while MPPI compensates for physical dynamics and environmental perturbations.Source: C. Policy Training · p. 6

V. HUMAN-COMPANION CONTROLLER

  • The MPPI-based companion controller integrates a control barrier function to prevent entry into the human’s intimate zone.The RL policy selects a high-level tracking position while MPPI handles low-level execution.Source: §V · p. 6, C. Policy Training · p. 6

A. MPPI-Based Human Accompanying

  • The MPPI controller minimizes tracking and collision-avoidance costs, with position and orientation weighted by Q = diag(cx, cy, cθ).The cost is formulated around the desired tracking position and collision-free movement.Source: A. MPPI-Based Human Accompanying · p. 7

B. CBF-Based Safe Human-Robot Interaction

A discrete-time control barrier function (DCBF) adds a safety cost to MPPI, defining a safe region around the human and penalizing trajectories that violate it.

B. CBF-Based Safe Human-Robot Interaction

  • The DCBF adds a cost term to MPPI so the robot maintains a designated interactive space around the human while satisfying safety constraints.The robot is considered safe when its state does not intersect the region surrounding the human.Source: B. CBF-Based Safe Human-Robot… · p. 7
  • The safety constraint uses an elliptical interactive space around the human, with the robot’s position transformed into the user’s coordinate system.The initial robot position is assumed to satisfy the safe-region condition.Source: B. CBF-Based Safe Human-Robot… · p. 7
  • MPPI trajectory samples can violate the barrier condition, so a local refinement step filters its output through a quadratic program.The DCBF cost is zero when the barrier condition is satisfied and otherwise applies a penalty weighted by csafe.Source: B. CBF-Based Safe Human-Robot… · pp. 7–8
  • On the Jetson Orin, MPPI runs at about 100 Hz, while the high-level RL policy runs at about 30 Hz.The controller uses Numba for GPU-accelerated parallel processing.Source: B. CBF-Based Safe Human-Robot… · p. 8

VI. VALIDATION RESULTS AND DISCUSSION

The proposed method is validated against other approaches and across multiple scenarios using the platform described earlier.

VI. VALIDATION RESULTS AND DISCUSSION

  • Validation compares the proposed method with other approaches and evaluates it across multiple scenarios on the previously described platform.Source: §VI · p. 8

A. Comparison Results

Across the reported comparison, the proposed method improves success rate and tracking accuracy while reducing discomfort time. It is evaluated under mapless conditions, and its remaining failures include cases where the robot becomes trapped among obstacles.

A. Comparison Results

  • At least 24% improvement in success rate and 47% improvement in tracking accuracy are reported, alongside reduced discomfort time.Table II presents quantitative comparison results as means and standard deviations.Source: Table II · p. 8, A. Comparison Results · p. 8
  • The comparison uses 50 trials with open and constrained routes, static obstacles, and 10 dynamic obstacles, at human speeds of 0.7–1.5 m/s.All methods operate maplessly at the velocity-command level, without future-target knowledge or additional high-level planning.Source: A. Comparison Results · p. 8
  • The high-level planner runs at about 30 Hz and low-level MPPI at 100 Hz; PID, DWA, MPC, and DRL run at about 200, 41, 35, and 27 Hz, respectively.PID has the lowest computational cost but fails to complete the task.Source: A. Comparison Results · pp. 8–9
  • The proposed method still has a 14% failure rate, including cases where the robot becomes temporarily trapped among obstacles.Prioritizing safety and social distance may lead it to stop or take a longer detour; a loss of tracking is counted when it remains over 2 m away for more than 5 seconds.Source: A. Comparison Results · p. 9

B. Interaction Space Evaluation Experiments

Across seven interaction-space configurations, tracking errors stayed stable without sim-to-real fine-tuning, while user preferences favored particular shapes and distances. Set 2 was selected for subsequent experiments.

B. Interaction Space Evaluation Experiments

  • Across seven configurations, |ex| ranged from approximately 0.019–0.022 m and |ey| from 0.023–0.029 m, despite no sim-to-real fine-tuning.The authors attribute this generalization to the DRL policy with domain randomization.Source: B. Interaction Space Evaluation… · p. 9
  • Users preferred elliptical over circular interaction spaces, consistent with the need for a larger frontal safety margin during human locomotion.Circular Set 3 scored lower than elliptical Set 2.Source: B. Interaction Space Evaluation… · p. 9
  • Set 2 received the highest user preference score and was selected for subsequent experiments.Personal Zone configurations (Sets 1–4) scored higher than Social Zone configurations (Sets 5–7).Source: B. Interaction Space Evaluation… · p. 9
  • Users described a distance trade-off: larger spaces raised concerns about losing track of the user, while smaller ones raised collision concerns.Set 1 was cited as an example of a smaller configuration.Source: B. Interaction Space Evaluation… · p. 9

1) Tracking Evaluation:

Indoor tests show context-dependent position changes and stable following through obstacles and occlusions. In hardware comparisons under occlusion, AmfiTrack had the highest reported success rate and faster operation than Camera and LiDAR.

1) Tracking Evaluation:

  • Indoor tests covered an unobstructed rectangular route, a static obstacle, an approaching pedestrian, and a progressively narrowing workspace, with freely chosen initial robot positions.The four scenarios were conducted without predefined tracking points.Source: 1) Tracking Evaluation · p. 9
  • Across indoor scenarios, the robot changed sides around static or moving obstacles and made context-dependent position choices rather than following a fixed pattern.In Scenarios 2 and 3 it shifted laterally; in Scenario 4 it resumed the right-side position.Source: 1) Tracking Evaluation · p. 9
  • Tracking points changed smoothly with workspace constraints, and tracking errors converged toward zero after positional or angular transitions.The x-axis position error was larger when following from behind in Scenario 4; angular error was larger in Scenario 1 during sudden human direction changes.Source: 1) Tracking Evaluation · pp. 9–10

2) Occlusion Experiments:

  • During doorway occlusion, the robot moved from side-by-side to behind the human, passed without collision, then resumed side-by-side when space allowed.In the multi-pedestrian case, it maintained safe distance and target identification, then accelerated to regain a side-following position.Source: 2) Occlusion Experiments · p. 10, Figure 11 · p. 11
  • AmfiTrack achieved a 92% Success Rate under occlusion, versus 55% for Camera and 39% for LiDAR.The comparison evaluates tracking hardware systems in cluttered and dynamic environments.Source: Table III · p. 11, 2) Occlusion Experiments · p. 11
  • AmfiTrack operated at approximately 120 Hz, compared with Camera at 30 Hz and LiDAR at 9 Hz, and provided omnidirectional sensing.The passage reports AmfiTrack's rate as 4× Camera's and more than 10× LiDAR's; LiDAR has 360° coverage and Camera approximately 110°.Source: 2) Occlusion Experiments · p. 11

D. Outdoor Experiments

Outdoor tests report low tracking errors across walking speeds and terrain, alongside measurable obstacle-distance relationships. Performance declined as pedestrian density increased, and the authors identify personalization, crowding, and uneven terrain as limitations.

D. Outdoor Experiments

  • Obstacle distances averaged 2.21 ± 1.09 m and 2.17 ± 1.07 m in Scenarios 1 and 3, versus 2.06 ± 0.98 m in Scenario 2.Obstacle distance and tracking error were significantly correlated across the three speed ranges (R2 = 0.66, 0.85, and 0.89; all p < 0.05).Source: Figure 13 · p. 12, D. Outdoor Experiments · p. 12
  • Across three walking speeds up to 1.7 m/s, mean |ex| and |ey| stayed below 4 cm and 3 cm, respectively, and |eθ| stayed below 0.06 rad.Three participants walked at average speeds of 0.75, 1.2, and 1.7 m/s.Source: D. Outdoor Experiments · pp. 11–12, Figure 12 · p. 12
  • Tracking on asphalt was more stable than on brick, particularly for ex and eθ, although the differences were not significant.Both surfaces maintained acceptable error levels during movement.Source: D. Outdoor Experiments · p. 12, Figure 15 · p. 12

E. Crowded Environment Experiments

  • Crowd success rate fell from 81% with 5 pedestrians to 26% with 20, while APT remained between 9.86 and 11.85 ms.Reported success rates were 65% with 10 pedestrians and 33% with 15; the authors link declining performance to obstacle avoidance and frequent trajectory adjustments.Source: E. Crowded Environment Experiments · pp. 12–13, Table IV · p. 13

F. Practical Considerations and Future Directions

  • Interaction-space dimensions were based on a survey of 10 participants; the authors identify personalization, larger user studies, dense crowds, and uneven terrain as future-work needs.The authors note a slight tracking-error increase on brick, indicating mecanum-wheel performance degradation on uneven terrain.Source: F. Practical Considerations and Future… · p. 13

VII. CONCLUSION

  • The conclusion describes a hierarchical controller pairing PPO-based tracking-position selection with MPPI-CBF motion execution, tested in indoor and outdoor real-world environments.It reports lower tracking errors and higher success rates than fixed-position baselines.Source: §VII · p. 13
Loading 2609.25031v1…