Source-linked AI summary
Deep Reinforcement Learning Assistance for Resilient Haptic Teleoperation over a 5G Network
Georgios Kokkinis, Aitor Hernandez Herranz, Qi Zhang
TL;DR
Haptic teleoperation over mobile networks faces delay that varies during operation, while the effects of live congestion and its interaction with assistance and QoS remain unclear. This paper evaluates an edge-hosted DRL assistant on a live Ericsson 5G SA testbed, blending its commands with an operator’s. Assistance reduces mean task completion time across network conditions, with the largest reduction under congestion.
Problem
It remains unclear whether assistance benefits operators over live networks and how it interacts with QoS, although delay can vary during a task and affect teleoperation.
Source: §I · p. 1Method
The authors evaluate an edge-hosted DRL assistant that blends its commands with the operator’s on a live Ericsson 5G SA testbed under different network conditions.
Source: §I · pp. 1–2Results
15.5–20.6% lower mean completion time and approximately 40% lower water-zone time with assistance across all conditions, with the largest completion-time improvement under congestion.
Source: §VIII · p. 11Takeaways & Limitations
The results indicate that communication, compute, and application functions should be co-designed as a coupled system rather than optimised in isolation.
Source: §VIII · p. 11All sessions involved one trained operator, so the results should not be generalised to a wider population.
Source: B. Limitations · p. 11
Abstract
from arXiv · showhide
Haptic teleoperation requires low-latency and reliable communication to preserve stable and responsive interaction between a human operator and a remote environment. However, network congestion can strongly degrade teleoperation performance, particularly when haptic traffic competes with other data flows for limited radio resources. This work investigates a deep reinforcement learning (DRL)-assisted teleoperation framework over an Ericsson 5G standalone (SA) testbed under different congestion and quality-of-service (QoS) configurations. In the evaluated task, a human operator uses a haptic device to steer a spaceship through a remote virtual game. The DRL agent runs inside the remote environment, hosted at the network edge, and assists the operator by blending its own actions with the operator's commands received over the 5G network. This placement gives the agent direct access to the current state of the environment, unaffected by network delay. Experimental results show that DRL assistance improves task performance under all network conditions, with the largest benefit under congestion, where it reduces mean task completion time by 20.6%. Under congestion, assisted operation even outperforms unassisted operation with QoS prioritisation, which restores latency close to the uncongested baseline.
I. INTRODUCTION
The introduction argues that teleoperation performance over mobile networks depends on how delay changes during a task, while prior work leaves assistance under live congestion and its interaction with QoS unclear. It presents an edge-hosted DRL assistant and evaluates it alongside network measurements on a live 5G SA testbed.
I. INTRODUCTION
- Prior studies often isolate delay or study network, signal-prediction, and endpoint-assistance approaches separately, leaving assistance over a live network and its interaction with QoS unclear.Under a shared mobile network, delay varies over time and can grow during sustained traffic, motivating joint measurement of network behaviour and task outcomes.Source: §I · p. 1
- The evaluated task is a virtual navigation game in which a haptic-device operator steers a spaceship while avoiding an invisible water-zone.The water-zone is perceived only through haptic feedback.Source: §I · p. 1
- The study evaluates an edge-hosted DRL assistant on a live Ericsson 5G SA testbed across uncongested, congested, and QoS-prioritised conditions.Sessions jointly record radio conditions, RAN Cell Trace Record data, transport-layer latency probes, and application-layer outcomes.Source: §I · p. 2
- The assistant is pretrained with haptic feedback in its observation space, blends its actions with operator commands, and reads the environment state directly despite network delays.This placement lets the agent act on current state while operator commands and state updates traverse the network.Source: §I · p. 2
- 15.5–20.6% lower mean task completion time and approximately 40% lower mean water-zone time with DRL assistance across network conditions, with the largest completion-time reduction under congestion.The evaluation covers an uncongested baseline, a congested cell, and a congested cell with QoS priority.Source: §I · p. 2
- 5.5 ms/s average latency growth during congested tasks links longer task duration to greater operator exposure, while QoS prioritisation reduces median RTT from 113.2 ms to 15.7 ms at the same cell load.Latency partially recovers between sessions; QoS removes the within-session accumulation and restores latency close to uncongested conditions.Source: §I · p. 2
II. RELATED WORK
Prior work addresses network conditions, haptic signals, and operator commands through separate approaches, including shared autonomy and model-mediated feedback. This paper combines command-level DRL assistance with QoS prioritisation and evaluates their relation to task performance and latency on a live 5G network.
II. RELATED WORK
- Shared-autonomy approaches blend operator commands with autonomous policies, but cited delayed-teleoperation studies introduced delay artificially rather than from a real network.Prior methods adapted blending to delay and confidence or addressed input latency and erroneous odometry.Source: §II · pp. 2–3
- Model-mediated teleoperation renders remote-environment feedback from a local model, whereas this system uses its operator-side replica for visuals and applies assistance remotely.Model mismatch can distort rendered feedback, and network-delivered corrections can cause abrupt force changes.Source: §II · p. 3
- Haptic signal predictors estimate delayed or lost samples at the receiver, complementing this paper’s assistance to commands sent to the environment.Prior methods reduce prediction error or improve predictor robustness –.Source: §II · p. 3
- Network-focused approaches shape resources or traffic, while this assistant changes the command applied to the environment and is evaluated together with network QoS prioritisation.Related work includes QoS measurements, resource allocation,, and congestion-prediction-based codec selection on emulated traces.Source: §II · p. 3
- Because session-level latency averages miss delay growth during operation, the study examines within-session latency changes and session-boundary behaviour alongside task performance.Prior queue-management and buffer-sizing studies link sustained load to growing network-buffer delay,.Source: §II · p. 3
- The paper combines DRL assistance with QoS prioritisation on a live 5G network and relates operator-experienced latency to task performance, unlike prior work focused on isolated mechanisms or injected delay.Earlier approaches studied command-level assistance under injected delay, haptic-signal prediction, or network mechanisms without task-level evaluation.Source: §II · p. 3
III. SYSTEM ARCHITECTURE AND DRL ASSISTANT
The system places the virtual environment and DRL assistant at the network edge, where the assistant acts on current state while operator commands and haptic feedback traverse the 5G link. The task uses haptic forces to guide a spaceship through a course with an invisible water zone.
III. SYSTEM ARCHITECTURE AND DRL ASSISTANT
- The edge application blends the assistant’s command, computed from the current environment state, with the operator’s command received over the 5G uplink; state and haptic feedback return over the downlink.The assistant acts on current state while the operator acts on state that has traversed the network.Source: §III · p. 3
- In congested and prioritised conditions, background user equipment and a synthetic video-like stream share the cell with the teleoperation traffic.The competing load includes operator commands on the uplink and state and haptic feedback on the downlink.Source: §III · p. 4
A. Teleoperation Task and Haptic Loop
- The course has a water-free channel whose centre and height vary along the route; outside it, drag slows the spaceship and a restoring force guides it toward the channel centre.The restoring force has a maximum magnitude of 2 N at the channel boundary and throughout the water zone; external forces are rendered as haptic feedback and used in the policy observation and reward.Source: A. Teleoperation Task and Haptic Loop · p. 4
B. Agent Placement and Operator-Side Rendering
The assistant runs alongside the remote environment, with direct access to current simulation state, while operator commands and state updates travel over the network. Its fixed-weight shared control combines the latest available operator command with an agent command based on haptic observations.
B. Agent Placement and Operator-Side Rendering
- The assistant accesses current simulation state inside the remote application, while operator commands and state updates traverse the 5G network.The operator-side replica renders visuals, but haptic feedback comes from the most recently received force, smoothed and held between updates.Source: B. Agent Placement and Operator-Side… · p. 4
C. Assistive Agent
- The policy uses spaceship velocity and external-force components as haptic observations, without visual input.The policy is pretrained offline and deployed with fixed parameters across network conditions.Source: 1) Observations and Shared Control · p. 4, C. Assistive Agent · p. 4
1) Observations and Shared Control:
- At each step, fixed-weight blending combines the latest available operator command with the agent command; β = 0.5 gives each equal weight.The operator command is subject to network delay, while the agent command comes from the current observation; β = 0 gives operator-only control and β = 1 agent-only control.Source: 1) Observations and Shared Control · pp. 4–5
2) Training Procedure:
The paper trains a haptic-only PPO policy for navigation through the water zone, then uses it to assist human control of the complete task. Among the tested observation inputs, the haptic-only policy learns fastest and achieves the highest final cumulative reward.
2) Training Procedure:
- Training omits human-command blending and gate obstacles but retains the water zone, focusing policy learning on water-zone navigation.A new course is generated for each training episode; the trained policy is later used to assist the operator on the complete task.Source: 2) Training Procedure · p. 5
- PPO [27] trains an actor–critic policy and value function, using a clipped probability-ratio objective to limit policy-update size.The paper selects PPO for continuous two-dimensional actions, stable learning with relatively little tuning, and compatibility with its Unity training environment.Source: 2) Training Procedure · p. 5
- The reward encourages forward progress while penalising elapsed time, water exposure, vertical force, and collisions; terminal rewards dominate per-step terms and prioritize finishing.The learned policy prioritises leaving the water over speed, and reward weights were selected empirically and held constant across training configurations.Source: 2) Training Procedure · p. 5
- Training uses D = 5, or 25 ms per decision, to reduce policy evaluations per simulated second; deployment uses D = 1, or 5 ms per decision.The paper states that the memoryless policy’s learned mapping is unchanged by the higher deployment decision rate.Source: 2) Training Procedure · p. 5
- 14.4 cumulative reward was reached by the haptic-only policy, versus approximately 12.4 for visual–haptic and 1.0 for visual-only policies.The haptic-only policy exceeded cumulative reward 10 within approximately 4 × 10^4 steps; the paper selects it for teleoperation experiments.Source: Figure 2 · p. 6, 2) Training Procedure · p. 6
IV. 5G TESTBED AND MEASUREMENT SETUP
The study evaluates the assistive system on Ericsson’s KINET 5G SA testbed using production radio scheduling and QoS mechanisms. The virtual environment and trained policy run in a container on an edge node co-located with the 5G core.
IV. 5G TESTBED AND MEASUREMENT SETUP
- The live testbed evaluation uses production radio scheduling and QoS mechanisms and describes the congestion conditions and cross-layer instrumentation.The study measures network and application behavior in the testbed setup.Source: §IV · p. 6
A. KINET 5G Network
- The KINET setup uses a single indoor Ericsson Radio Dot cell in the 3.5 GHz band with an 80 MHz TDD carrier, connected to a shared 5G core.The operator connects through a laptop and 5G USB modem; within each experimental block, RSRP varies by at most 6 dB and SINR remains above 33 dB.Source: A. KINET 5G Network · p. 6
B. Edge Deployment
- The virtual environment and trained policy run in a Docker container on an edge node co-located with the 5G core, managed by K3s and deployed with Helm.Experimental conditions can be selected in the deployment configuration without rebuilding the application image.Source: B. Edge Deployment · pp. 6–7
C. Traffic Conditions
The traffic conditions compare an uncongested baseline with congestion and QoS prioritisation under matched competing load. Concurrent measurement layers align radio, transport, and application data, while per-flow QoS assigns teleoperation traffic a higher scheduling priority.
C. Traffic Conditions
- Congested and prioritised conditions retain the same competing traffic, with mean downlink PRB utilisation of 92.7% and 92.8%, respectively; the uncongested baseline is 17.8%.Congestion uses traffic to seven competing UEs and a synthetic video-like stream to the operator UE.Source: C. Traffic Conditions · p. 7
D. QoS Prioritisation Mechanism
- QoS priority is requested per flow through CAMARA QoD and mapped to 5QI 8, a non-guaranteed-bitrate class with higher scheduling priority than competing traffic on default 5QI 9.Application flow addresses, protocol, and ports identify the teleoperation flows; the synthetic video stream uses a separate UDP socket.Source: D. QoS Prioritisation Mechanism · p. 7
E. Layered Instrumentation
- Measurements from UE, RAN, transport, and application layers share a common time base in an edge-hosted time-series database for joint analysis.TWAMP probes use five packet sizes at an aggregate rate of about 100 packets/s; RAN CTR streams at 1 s intervals, while application telemetry is exported every 6 s.Source: E. Layered Instrumentation · p. 7
V. EXPERIMENTAL METHODOLOGY
The methodology uses a single-operator repeated-measures block design across two control modes and three network conditions. Sessions run a shared-control simulation loop in which delayed operator commands are blended with agent actions at the remote side.
A. Experimental Design
- All six control-mode and network-condition combinations were run as continuous 10–12-minute blocks, with 104 retained sessions after exclusions.One practice run preceded each block; all blocks were recorded on the same day by one trained operator, and the prioritised condition reversed the control-mode block order.Source: A. Experimental Design · pp. 7–8, B. Session Protocol · p. 8
B. Session Protocol
- Every 5 ms, the remote side blends the latest available operator command with the agent command, while operator-side input, feedback, rendering, and the 1 kHz haptic servo run asynchronously.Sessions end when the course is completed; Human-only uses β = 0 and DRL-Assist uses β = 0.5.Source: B. Session Protocol · p. 8
C. Evaluation Metrics and Data Processing
Evaluation combines session-level task outcomes with separate network measurements. Task metrics capture completion and water-zone exposure, while network metrics distinguish transport delay from application-layer measurements.
1) Task-level metrics:
- Task performance is measured by completion time, cumulative water-zone time, and water exposure ratio, defined as water-zone time divided by completion time.Source: 1) Task-level metrics · p. 8
2) Network-level metrics:
- Transport delay is TWAMP round-trip time; application logs separately record ping round-trip time, synthetic-stream jitter, packet loss, and throughput.Radio and RAN measurements characterize cell load and radio conditions but are not outcome variables.Source: 2) Network-level metrics · p. 8
3) Statistical analysis:
DRL-Assist has lower mean completion times than Human-only in both uncongested and congested conditions, while congestion broadens network latency and task variability. QoS prioritisation brings network latency and task times close to the uncongested baseline.
1) Network-Level Effects:
- 113.2 ms versus 14.8 ms: median TWAMP RTT under Congested versus Uncongested conditions; congestion also raises the 95th percentile from 22.2 ms to 494.8 ms.The interquartile range increases from 4.8 ms to 242.6 ms, indicating less consistent delivered latency.Source: 1) Network-Level Effects · p. 9, Table VI · p. 9
- 15.7 ms median RTT and 21.8 ms 95th-percentile RTT under Prioritised conditions approach the Uncongested baseline despite matched mean downlink PRB utilisation of 92.7% and 92.8%.Fig. 4 shows the recovery across the latency distribution.Source: 1) Network-Level Effects · p. 9, Figure 4 · p. 9
2) Task-Level Effects:
- 40.31 s to 33.53 s under Uncongested conditions and 45.42 s to 36.07 s under Congested conditions: mean completion time is lower for DRL-Assist than Human-only.Water-zone time follows the same pattern, and under Prioritised conditions both modes are close to the Uncongested baseline.Source: 2) Task-Level Effects · p. 9
3) Statistical analysis:
- Congestion increases completion-time standard deviation from 1.29 s to 3.92 s for Human-only and from 1.34 s to 2.99 s for DRL-Assist.The analysis fits latency against elapsed session time and compares the final 5 s of one session with the first 5 s of the next; results are from a single experimental campaign.Source: 3) Statistical analysis · pp. 8–9, 2) Task-Level Effects · p. 9
B. Impact of DRL Assistance
DRL assistance improves task performance across network conditions, with its largest completion-time benefit under congestion. Congestion also produces rising latency during sessions, while assistance and QoS prioritisation address that accumulation from different sides.
B. Impact of DRL Assistance
- 20.6%: DRL assistance reduces mean completion time by 9.3 s under Congested conditions, the largest reduction across conditions; water-zone time falls about 40% and exposure ratio 27–29%.Completion-time reductions are 6.8 s (16.8%) under Uncongested and 6.1 s (15.5%) under Prioritised conditions.Source: B. Impact of DRL Assistance · p. 9, Figure 5 · p. 10
- 36.07 s under Congested DRL-Assist outperforms 39.48 s for Human-only under Prioritised conditions; prioritisation restores, but does not improve on, uncongested Human-only performance.Under Prioritised conditions, DRL-Assist completion time is 33.37 s.Source: B. Impact of DRL Assistance · pp. 9–10
- 1.9 s versus 2.7 s per 100 ms of RTT: completion time rises more slowly with latency under DRL-Assist, while water exposure remains about 0.18 lower across the observed range.Both task metrics worsen as RTT increases under both control modes.Source: B. Impact of DRL Assistance · p. 10, Figure 6 · p. 10
A. Latency Accumulation
- 5.5 ms/s: Congested TWAMP RTT rises during sessions, compared with negligible slopes of +0.007 ms/s Uncongested and +0.001 ms/s Prioritised.Across the observation window, Congested mean RTT approximately doubles from 106.8 ms to 218.1 ms, while the other conditions remain near 15 ms.Source: A. Latency Accumulation · pp. 10–11, Figure 7 · p. 11
- 50.9 ms: Congested RTT drops on average between sessions, consistent with a queue that fills during activity and partially drains at session boundaries.No corresponding pattern appears under Prioritised conditions, where the mean boundary drop is 0.0 ms.Source: A. Latency Accumulation · p. 11
- 80%: Controlling for session duration reduces the Congested latency difference between control modes by this amount, linking part of the observed network difference to time active.Longer sessions correlate with higher session-mean latency; the passage describes a possible feedback in which added latency may make tasks harder and extend them further.Source: A. Latency Accumulation · p. 11
- Assistance shortens tasks and reduces exposure to rising latency, whereas prioritisation prevents teleoperation traffic from entering the queue.The passage relates this distinction to DRL assistance having its largest benefit under Congested conditions.Source: A. Latency Accumulation · p. 11
- The proposed communication-demand benefit of compact state and force measurements depends on sensor and inference placement and was not evaluated in these experiments.The comparison is with transmitting images for policy inference from an edge server connected to the remote device.Source: A. Latency Accumulation · p. 11
B. Limitations
The evaluation is limited by its single-operator, non-randomised design, fixed assistance weight, and local rendering setup. The conclusion links assistance and prioritisation to different parts of congestion-related latency accumulation and argues for co-design across communication, compute, and application functions.
B. Limitations
- A single trained operator limits generalisation; non-randomised blocks leave order effects possible, and fixed β = 0.5 does not establish an optimal control allocation.The study also rendered the scene at the operator station rather than streaming video, and sampled application telemetry every 6 s.Source: B. Limitations · p. 11
VIII. CONCLUSION
- For the evaluated operator, DRL assistance reduces mean completion time by 15.5–20.6% and water-zone time by approximately 40% across conditions.At matched radio load, QoS prioritisation reduces median RTT from 113.2 ms to 15.7 ms and the 95th percentile from 494.8 ms to 21.8 ms.Source: §VIII · p. 11
- Under Congested conditions, RTT rises 5.5 ms/s during sessions and drops 50.9 ms at boundaries; prioritisation prevents queue entry while DRL assistance shortens task exposure.The conclusion describes this as coupling between task duration and experienced latency.Source: §VIII · p. 11
- The conclusion recommends co-designing communication, compute, and application functions rather than optimising them in isolation.It identifies edge placement and network APIs as components of this approach, with network context as a possible input for adapting human-agent blending.Source: §VIII · pp. 11–12