Source-linked AI summary
Residual Deep Reinforcement Learning-Based Computed Torque Control for a Cable-Driven Lower-Limb Rehabilitation Robot under Disturbances and Parametric Uncertainties
Mohammad-Hossein Fakouri, Ali Keymasi-Khalaji
TL;DR
Cable-driven rehabilitation robots need accurate, constraint-compatible control despite model mismatch and disturbances, while fully learning-based control can reduce transparency. This paper augments computed torque control with bounded residual DDPG compensation and evaluates the approach across uncertainty, disturbance, feasibility, and generalization tests. In the representative combined case, the residual controller reduced RMS Cartesian error by approximately 42%, while broader tests exposed trajectory-dependent joint-limit limitations.
Problem
Accurate rehabilitation-robot tracking is challenged by model uncertainty, disturbances, joint constraints, and pull-only actuation, while fully learning-based control can complicate transparency and constraint-aware operation.
Method
The framework retains computed torque control as the nominal nonlinear layer and adds a bounded DDPG residual torque, with constrained inverse kinematics and cable-feasibility diagnostics.
Results
In the representative combined dataset, RMS Cartesian error decreased from 0.0221419 m for CTC to 0.0128462 m with the residual actor, an approximately 42% reduction.
Takeaways & Limitations
The results support bounded residual learning as a useful compensation layer for the evaluated simulation model while preserving the nominal model-based control structure.
Takeaways & Limitations
The study does not establish formal stability, dynamic cable-force realization, actuator-saturation feasibility, clinical benefit, or hardware safety, and circular-path tracking can coexist with joint-limit violations.
Abstract
from arXiv · showhide
Accurate trajectory tracking in cable-driven lower-limb rehabilitation robots is challenging because model uncertainty, external disturbances, joint constraints, and pull-only cable actuation can degrade nominal control performance. Conventional model-based controllers provide an interpretable control structure but remain sensitive to model mismatch, whereas fully learning-based control can reduce transparency and complicate constraint-aware operation. This study proposes a residual deep reinforcement learning-enhanced computed torque control framework in which computed torque control generates the nominal command and a bounded Deep Deterministic Policy Gradient policy supplies only an additional compensating torque. The approach is evaluated in simulation under nominal, uncertain, disturbed, combined, and generalization conditions, together with trajectory-tracking, joint-limit, cable-demand, workspace-feasibility, and cable-Jacobian diagnostics. Across the evaluated conditions, the residual controller improves tracking and disturbance rejection relative to computed torque control while preserving the interpretable model-based command structure and satisfying the reported feasibility checks in the representative evaluation. Broader tests indicate that tracking improvements can persist beyond the representative case while also exposing trajectory-dependent constraint limitations. These results support bounded residual learning as a practical robustness-enhancement strategy for simulation-based rehabilitation robot control and motivate further constraint-aware and experimental validation.
1. Introduction
The study combines interpretable computed torque control with bounded residual DDPG compensation for a cable-driven rehabilitation robot. It evaluates tracking, feasibility, and robustness under uncertainty, disturbances, and trajectory variation.
- Cable-driven rehabilitation robots support programmable assistance but require accurate tracking, disturbance rejection, joint-limit compliance, and actuation-feasibility assessment.
- Computed torque control offers transparent inverse-dynamics compensation but remains sensitive to model mismatch and disturbance-related uncertainty.
- The study integrates a planar 3-DOF, three-cable model with constrained reference generation, cable-Jacobian mapping, nonnegative-tension checks, and joint-feasibility diagnostics.
- The proposed framework retains CTC as the nominal controller and adds a bounded residual DDPG torque rather than replacing the model-based command.
- Four matched simulation scenarios compare nominal, 10% parameter-uncertain, disturbed, and combined conditions using tracking, cable-demand, and joint-limit metrics.
- Additional tests examine disturbance seeds, time intervals, nearby initial configurations, and an alternate trajectory to characterize robustness and transfer limits.
2. Robot kinematics and cable geometry
This section defines the planar three-link kinematics, cable geometry, pull-only actuation mapping, reference path, and constrained inverse-kinematics procedure. It distinguishes tested trajectory feasibility from broader dynamic and hardware feasibility.
- Coordinate frame and generalized coordinates: The planar model uses hip-, knee-, and ankle-related generalized coordinates with cumulative orientations θ1 = q1, θ2 = q1 + q2, and θ3 = q1 + q2 + q3.The link lengths are b1 = 0.45 m, b2 = 0.35 m, and b3 = 0.21 m.
- Serial-link kinematics: Forward kinematics compute the end-effector coordinates from the three link lengths and cumulative joint angles, while the end-effector Jacobian maps joint velocity to Cartesian velocity.
- Cable lengths and actuation Jacobian: Figure 1 specifies fixed cable anchors, link-center attachment points, cable lengths, and pull directions for the three-cable simulation geometry.The wearable-style illustration is conceptual; reported results use the planar 3-link/3-cable model.
- Cable lengths and actuation Jacobian: The cable-actuation Jacobian uses a sign convention consistent with the implementation, and nonnegative cable tensions are required because cables can pull but cannot push.A negative tension component indicates that the requested generalized torque is infeasible for the selected routing at that posture without changing the allocation conditions.
- Reference trajectory and constrained inverse kinematics: The primary reference is a smooth closed 10-s rehabilitation-like path whose desired joint coordinates are obtained by bounded inverse kinematics at each sample.The implementation uses fmincon with sequential quadratic programming and a fixed initial guess, without an explicit intersample continuity penalty or previous-solution warm start.
- Reference trajectory and constrained inverse kinematics: The archived primary trajectory has approximately 0.268° maximum sample-to-sample joint change, but this smoothness is specific to the tested trajectory and initialization.
- Feasibility distinctions: Trajectory feasibility is narrower than full dynamic or hardware feasibility, which additionally requires cable elasticity, actuator dynamics, pretension, and physical force bounds.
3. Dynamic modeling and cable-force allocation
The paper models the robot in joint-torque coordinates while evaluating cable-force feasibility algebraically. It introduces structured, time-localized dynamic mismatch and disturbance conditions, alongside tension and cable-Jacobian diagnostics.
- Implementation scope and limitations: The cable calculation is a feasibility and demand assessment rather than a detailed actuator model with motor, pulley, elasticity, or tension-servo dynamics.Because a physical F_max is undefined, the analysis does not claim actuator-saturation feasibility, optimal allocation, or guaranteed pretension.
- Implemented joint-space dynamic model: The simulated equations use inertia, Coriolis/centrifugal, gravity, commanded generalized torque, and additive disturbance torque terms.The same joint-space equation structure supports the nominal controller and simulated plant, which may use different parameter values.
- Nominal controller model and uncertain plant: The CTC branch uses fixed nominal dynamics, while selected plant parameters increase simultaneously by 10% during 1 < t < 5 s.The perturbation changes dynamic coefficients without redefining the calibrated inverse- or forward-kinematic geometry.
- Disturbance model: The disturbance case gates three band-limited white-noise sources into joint-torque channels during 5 < t < 9 s.The maintained model specifies noise power 0.1, a 0.5 s correlation/sample parameter, and documented nominal seeds for the three joints.
- Cable-force allocation and pull-only feasibility: Cable-force allocation evaluates whether a requested generalized torque can be represented by cable tensions subject to lower and upper tension bounds.In the square system, negative tension cannot be repaired through null-space pretension; the implementation falls back to zero when its tension test fails.
- Rank and conditioning diagnostics: The cable mapping is assessed along the desired path using rank, minimum singular value, and condition number as trajectory-specific conditioning indicators.These diagnostics do not constitute global controllability or safety certificates.
4. Control design
The controller retains CTC as the nominal model-based command and adds a bounded residual DDPG torque to compensate for unresolved dynamics and disturbances. Training regularizes residual magnitude and variation, while deterministic evaluation uses a saved actor without online updates.
- Architecture: CTC supplies the nominal trajectory-tracking command, while DDPG contributes only a bounded residual torque rather than replacing the model-based controller.The combined command is τcmd = τctc + τRL, with the learned component restricted to joint-wise bounds.
- Control interpretation: The residual term is trained to reduce tracking effects associated with imperfect cancellation, command filtering, and external disturbance after nominal compensation.This interpretation is expressed through the tracking-error dynamics under model mismatch.
- Network design: The actor maps a 12-element unnormalized observation to three bounded residual torques, while the critic combines separate observation and action feature paths to estimate Q(o, a).The actor uses two 256-neuron ReLU layers and joint-wise output scaling; the critic produces a scalar action-value estimate.
- Training objective: The reward penalizes joint error, error rate, normalized residual magnitude, and changes in normalized residual torque, but not cable-tension margin or joint-limit proximity.Those constraint and interaction objectives are identified for a subsequent constraint-aware training stage.
- Training and evaluation: Training uses exploration, replay-buffer updates, gradient steps, and soft target-network updates, whereas evaluation uses the saved actor deterministically without parameter updates.The archived implementation uses the designated checkpoint and documents no separate state-based early termination or episode randomization.
5. Simulation model-consistency assessment and evaluation method
The evaluation combines cross-model consistency checks with paired controller comparisons under matched simulation settings. It reports tracking and feasibility-oriented diagnostics across nominal, uncertainty-only, disturbance-only, and combined scenarios, while separating broader sensitivity tests from the primary trajectory.
- Model consistency: Analytical, Simscape Multibody, and MSC ADAMS implementations are compared using matched nominal geometry and reference motion across Cartesian and joint-angle histories.The checks target sign, unit, joint-order, and implementation inconsistencies rather than experimental validation.
- Model consistency: RMS discrepancies range from approximately 0.98% to 2.67% of analytical motion range, while maximum absolute discrepancies range from approximately 2.47% to 6.15%.The third joint has the largest absolute joint-angle discrepancy, and the y coordinate has the largest range-normalized discrepancy.
- Primary comparison protocol: The paired CTC and residual-controller branches share trajectories, nominal model, gains, timing, initial state, uncertainty timing, and disturbance realization; only the saved actor output differs.This design isolates the controller addition, while the primary trajectory remains the archived training-environment reference.
- Evaluation scenarios: The four primary scenarios isolate nominal operation, model uncertainty, external disturbance, and their consecutive combined activation.The nominal case tests whether the residual policy changes performance when the model is already matched.
- Metrics and diagnostics: Tracking metrics include RMS, peak error, IAE, and ISE, while cable diagnostics record algebraic cable-demand extrema and zero-tension-boundary contact.Percentage improvement is computed relative to CTC for metrics where lower values are preferable.
- Robustness and generalization: Ten disturbance seeds, time-interval metrics, nearby initial conditions, and an alternate trajectory assess sensitivity and transfer beyond the representative run.Mean and standard deviation are reported descriptively without an inferential statistical test.
6. Results and discussion
The evaluation combines workspace and cable-feasibility diagnostics with representative, multi-scenario, interval-wise, residual-authority, and alternate-trajectory analyses. The bounded residual controller generally improves tracking, while results expose demand and trajectory-dependent constraint trade-offs.
- Workspace and cable feasibility: 33,921 of 91,125 joint-limit samples satisfy the static nonnegative-tension test, corresponding to 37.22%, and the desired path lies within both sampled sets.The test requires full cable-Jacobian rank, negligible reconstructed-torque residual, and nonnegative inferred tensions.
- Workspace and cable feasibility: Rank remains three along the desired trajectory, with minimum singular value 0.01481 and maximum two-norm condition number 71.25, indicating numerical sensitivity despite no exact singularity.Conditioning therefore provides information beyond a rank or determinant check.
- Representative combined case: RMS Cartesian error falls from 0.0221419 m to 0.0128462 m, a 41.9826% reduction, while peak error, IAE, and ISE decrease by 19.4%, 63.5%, and 66.8%.The larger IAE and ISE reductions indicate reduced sustained error over the full representative run.
- Four-case comparison: Residual DDPG reduces RMS error by 21.1% nominally, 39.4% under uncertainty, 27.7% under disturbance, and 41.6% in the combined case.The larger gains in uncertainty-containing cases support compensation of nominal-model mismatch, while nominal IAE changes by about 0.15%.
- Seed and interval robustness: Mean RMS reduction is 23.82% during the initial transient, 81.84% during deterministic uncertainty, 68.45% during disturbance, and 22.64% during recovery.The largest and most consistent gains occur while mismatch or disturbance is active.
- Alternate trajectory and residual authority: On the alternate circular path, RMS error decreases by 44.56%, but joint-limit channel-samples increase from 44 for CTC to 101 for the residual branch.Maximum inferred tension decreases from 230.42 N to 162.58 N, identifying trajectory-specific constraint coverage for subsequent training.
7. Conclusion
The study combines nominal computed torque control with bounded DDPG residual torque for a cable-driven rehabilitation mechanism under uncertainty and disturbance. In the representative combined simulation, residual control improved tracking and passed the reported feasibility checks, while broader tests identified important scope limits.
- Method: The controller retains computed torque control as the nominal nonlinear layer and adds a DDPG correction bounded by ±[5]ᵀ N m.The evaluation also combines constrained inverse kinematics, cable-Jacobian mapping, torque-to-tension calculations, and feasibility diagnostics.
- Representative results: 42% lower RMS Cartesian error reduced from 0.0221419 m to 0.0128462 m in the representative combined dataset.Peak error, IAE, and ISE also decreased by 19.4%, 63.5%, and 66.8%, respectively.
- Feasibility diagnostics: The residual-controller log had no joint-limit channel-sample violations, while algebraic cable-demand values remained above 3.41445 N.The cable-demand quantities are diagnostic torque-equivalent values, not outputs of a simulated cable-actuation loop.
- Robustness analysis: Ten-seed and interval analyses located the largest average reductions within the active uncertainty and disturbance intervals.Residual torque had smaller RMS magnitude than CTC for all three joints, although the first-joint residual briefly approached its action bound.
- Limitations: The evidence does not establish formal stability, dynamic cable-force realization, actuator-saturation feasibility, clinical benefit, or hardware safety.A circular-path test also showed that improved Cartesian tracking can coexist with trajectory-dependent joint-limit violations.
- Future work: Future work should add physical pretension and cable-force bounds, constrained quadratic programming, broader randomized testing, and higher-fidelity actuator and cable models.The proposed extensions also include cable-tension margins, joint-limit proximity, actuator effort, interaction force, and comfort-related quantities.
CRediT authorship contribution statement
The contribution statement assigns primary implementation and validation responsibilities to Mohammad-Hossein Fakouri and supervisory and review responsibilities to Ali Keymasi-Khalaji.
- Mohammad-Hossein Fakouri is credited with conceptualization, methodology, software, validation, formal analysis, investigation, drafting, and visualization.
- Ali Keymasi-Khalaji is credited with supervision, conceptualization, methodology, and writing review and editing.
- Both authors are credited with conceptualization and methodology.
Funding
The study reports no specific grant funding from public, commercial, or not-for-profit funding agencies.
- No specific grant was received from public funding agencies.
- No specific grant was received from commercial funding agencies.
- No specific grant was received from not-for-profit funding agencies.
Data availability
The supporting scripts, result files, trained-agent checkpoint, and post-processing routines are available from the corresponding author upon reasonable request, subject to stated constraints.
- MATLAB/Simulink scripts and saved result files are available from the corresponding author upon reasonable request.
- The evaluated trained-agent checkpoint and post-processing routines are also available upon reasonable request.
- File-size and software-licensing constraints apply, and exact regeneration of the original disturbance realization is not guaranteed.
Declaration of generative AI and AI-assisted technologies in the manuscript preparation process
The authors used ChatGPT for language editing, document organization, and manuscript-format checks, then reviewed and edited the content.
- ChatGPT supported language editing, document organization, and manuscript-format checks during preparation of the work.The authors reviewed and edited the content afterward and take responsibility for the published article.
- The tool was not used to generate simulations, figures, datasets, numerical results, or authorship decisions.