PaperlayerView exact revision on arXiv

Source-linked AI summary

Exo-Plore: Exploring Exoskeleton Control Space through Human-aligned Simulation

Geonho Leem, Jaedong Lee, Jehee Lee, Seungmoon Song, Jungdam Won

arXiv:2601.22550v1cs.ROcs.GRcs.LG

TL;DR

Demanding human-in-the-loop experiments make it difficult to optimize exoskeleton assistance for people with mobility impairments. Exo-plore uses neuromechanical simulation and Deep RL to fit gait adaptation trends and optimize assistance; it reports speed-dependent settings and pathology-severity relationships, with strong linear correlations in four of five pathological gait conditions.

  • Problem

    Personalizing exoskeleton assistance requires accounting for human adaptation, yet the hours-long human experiments used by HILO are difficult for people with mobility impairments to participate in.

    Source: §1 · p. 1

  • Method

    Exo-plore couples a Deep RL gait data generator, tuned to human adaptation trends, with a stochasticity-aware surrogate optimizer for hip-exoskeleton control parameters.

    Source: §1 · p. 2

  • Results

    Exo-plore identifies speed-dependent optimal assistance and strong linear relationships between pathology severity and optimal gain in four of five pathological gait conditions.

    Source: §1 · p. 2, §4.4 · p. 9

  • Takeaways & Limitations

    Sim-to-real matching supports Exo-plore as an alternative to experimental trials for controller optimization, including for pathological gaits.

    Source: §5 · p. 10

    Human-subject validation, including patient populations, is still required to establish prediction validity and clinical effectiveness; rewards, personalization, and muscle dynamics also remain limited.

    Source: §5 · p. 10

Abstract

from arXiv · show

Exoskeletons show great promise for enhancing mobility, but providing appropriate assistance remains challenging due to the complexity of human adaptation to external forces. Current state-of-the-art approaches for optimizing exoskeleton controllers require extensive human experiments in which participants must walk for hours, creating a paradox: those who could benefit most from exoskeleton assistance, such as individuals with mobility impairments, are rarely able to participate in such demanding procedures. We present Exo-plore, a simulation framework that combines neuromechanical simulation with deep reinforcement learning to optimize hip exoskeleton assistance without requiring real human experiments. Exo-plore can (1) generate realistic gait data that captures human adaptation to assistive forces, (2) produce reliable optimization results despite the stochastic nature of human gait, and (3) generalize to pathological gaits, showing strong linear relationships between pathology severity and optimal assistance.

1 INTRODUCTION

Exoskeleton assistance must account for users’ adaptation, but the demanding human experiments used to personalize controllers often exclude people with mobility impairments. Exo-plore combines neuromechanical simulation and Deep RL to fit human adaptation trends and optimize assistance across healthy and pathological gait.

1 INTRODUCTION

  • Human-in-the-loop optimization personalizes assistance through real human responses, but requires hours of walking that people with mobility impairments may rarely tolerate.Users also alter movement and muscle coordination in response to assistance, potentially eroding benefits predicted under fixed-gait assumptions.Source: §1 · p. 1
  • Neuromechanical simulation and Deep RL offer complementary approaches, but prior work lacked a unified framework to fit observed adaptations and predict unseen assistive conditions, especially pathological gait.Many human-like Deep RL implementations rely on motion-capture reference tracking to manage large observation and action spaces (Lee et al., 2019; Song et al., 2021; Park et al., 2025).Source: §1 · p. 1
  • Exo-plore fits simulated gait to human adaptation trends and identifies speed-dependent assistance settings and pathology-severity relationships, with strong linear correlations in four of five conditions.The optimal torque delay decreases as walking speed increases.Source: §1 · p. 2

2 RELATED WORK

Prior approaches to exoskeleton assistance and neuromechanical simulation face practical and predictive limits. Exo-plore is positioned as fitting experimental trends while predicting responses in unseen conditions.

Human-in-the-loop optimization (HILO) for exoskeletons

  • HILO optimizes controllers from human responses, but human experiments typically limit updates to fewer than 30 iterations and are often infeasible for mobility-impaired users.Studies have also optimized for user preference or perceived comfort.Source: §2 · p. 2

Neuromechanical simulation

  • Neuromechanical simulations couple musculoskeletal models, Hill-type muscle actuators, and control policies, but existing control strategies can generalize poorly to novel assistance or pathological conditions.Strategies include black-box motion reproduction through trajectory optimization or Deep RL, and hand-crafted controllers tuned for habitual gait.Source: §2 · p. 2

Deep RL-based Musculoskeletal Control

  • Deep RL musculoskeletal control faces unresolved questions about physiological fidelity and generalization, while sim-to-real validation for muscle activation, metabolism, and assistance adaptation remains limited.Exo-plore is described as fitting established experimental trends and predicting responses in unseen conditions.Source: §2 · p. 2

3 METHOD: THE EXO-PLORE FRAMEWORK

Exo-plore combines a human-aligned gait generator with a surrogate optimizer to find hip-exoskeleton control parameters that minimize simulated metabolic cost-of-transport. Its generator models assistance-related adaptation, while the optimizer samples and searches control parameters using simulation data.

3 METHOD: THE EXO-PLORE FRAMEWORK

  • Exo-plore optimizes gain κ and delay ∆t to minimize simulated metabolic cost-of-transport, using delayed-feedback hip torque scaled from relative hip motion.The gain scales torque magnitude and acts as a stiffness parameter; the delay introduces temporal lag.Source: §3 · p. 3

3.1 GAIT DATA GENERATOR

  • The gait generator couples a PoseNet, PD controller, and muscle coordination network with Deep RL and supervised learning to produce muscle-driven gait.The PD controller compensates for exoskeleton torque, and the muscle coordination network maps target joint torques to muscle activations.Source: §3.1 · p. 3
  • Its reward combines gait tracking, natural arm motion, metabolic energy, and human–exoskeleton interaction to model adaptation to assistance.The interaction reward is designed to account for physical human–exoskeleton interaction; muscle coordination also penalizes excessive activation and incoherent activations within anatomical muscle groups.Source: §3.1 · pp. 3–4

3.2 SIM-TO-REAL MATCHING IN GAIT DATA GENERATOR

  • The sim-to-real calibration selects metabolic-energy parameters by matching simulated preferred walking speed to human data and uses minimum cost-of-transport to identify that speed.The selected parameters are (α, β) = (1.5, 1.0).Source: §3.2 · pp. 4–5

3.3 EXOSKELETON OPTIMIZER

  • The optimizer uses Latin hypercube samples to train a neural surrogate, then applies gradient-based optimization to minimize predicted cost-of-transport for a target gait.Huber loss reduces the effect of simulation outliers, while a gradient penalty encourages a smooth surrogate landscape and stable gradients.Source: §3.3 · p. 6

4 RESULTS

The simulations reproduce several human gait and exoskeleton-assistance trends. Surrogate optimization identifies speed-dependent controls, while optimal gains correlate with pathology severity except in foot drop.

4.2 GAIT GENERATION – UNASSISTED

  • Simulated ankle, knee, and hip kinematics qualitatively matched human data except for hip internal rotation, and muscle activation patterns were similar without activation-pattern training.Activation timing and patterns are more informative than absolute values because human EMG is normalized to each muscle’s maximum voluntary contraction.Source: Figure 2 · p. 7, §4.2 · p. 7, Figure 3 · p. 7

4.3 GAIT GENERATION – ASSISTED

  • Under assistance at (κ, ∆t) = (8 Nm, 0.25 s), simulated hip kinematics and assistive moment and power showed trends reasonably consistent with human experiments.Gaps in absolute values may reflect differences in musculoskeletal conditions or policy overfitting to the fixed environment.Source: Figure 5 · p. 8, §4.3 · p. 8
  • The HEI reward aligned maximum metabolic reduction rates more closely with human results than the alternatives, though simulated reduction rates were slightly higher.The simulation referenced the unassisted condition, whereas human experiments referenced walking without an exoskeleton, partly explaining the discrepancy.Source: Figure 7 · p. 9, §4.3 · p. 9
  • At 4 km/h, assistive power increased 1.73-fold with rHEI versus 1.88-fold in human experiments; rHEI also achieved the highest correlation with human trends.The reported correlations were 0.83 for rHEI, 0.69 without rHEI, and 0.23 with the alternative assistance-maximizing reward.Source: §4.3 · p. 8, Figure 6 · p. 9

4.4 OPTIMIZING EXOSKELETON CONTROL PARAMETERS

  • A surrogate network smoothed stochastic local minima to support faster, more stable optimization, while optimized delay decreased monotonically as walking speed increased.Smaller gradient penalties allow more aggressive optimization but risk fitting artifacts; larger penalties reduce that risk but may limit sensitivity.Source: Figure 8 · p. 9, §4.4 · p. 9
  • Optimal gains showed strong linear correlations with pathology severity across all examined gait types except foot drop.Severity was quantified by deviation of muscle parameters from healthy reference values; foot drop showed irregular trends and unstable optimization convergence.Source: §4.4 · pp. 9–10, Figure 9 · p. 10

5 CONCLUSION AND LIMITATIONS

Exo-plore optimizes exoskeleton controllers through neuromechanical simulation and Deep RL, and extends the framework to pathological gaits. Human-subject validation remains necessary to establish prediction validity and clinical effectiveness.

5 CONCLUSION AND LIMITATIONS

  • Exo-plore optimizes exoskeleton controllers through neuromechanical simulation and Deep RL, with sim-to-real matching demonstrated as an alternative to experimental optimization trials.The framework is also extended to pathological gaits.Source: §5 · p. 10
  • Pathological-gait extension targets controller design for mobility-impaired individuals unsuitable for traditional human-in-the-loop optimization.Source: §5 · p. 10
  • Human-subject experiments, including patient populations, are still required to establish prediction validity and clinical effectiveness; simplified rewards, absent subject-specific motor-control personalization, and approximate muscle dynamics also warrant real-world evidence.Source: §5 · p. 10

A ARCHITECTURE OF HUMAN CONTROLLER

The human-controller architecture represents gait and assistance state, then uses pose-and-phase actions and a composite reward to train a gait generator. Its supporting analyses address activation plausibility, metabolic-cost parameters, and surrogate feasibility.

B.1 STATE REPRESENTATION •

  • The controller state includes body kinematics, gait features and phase, muscle-related terms, and recent left/right exoskeleton assist-torque history.Source: §B.1 · p. 15

B.2 ACTION REPRESENTATION

  • Residual pose actions combine pose displacement with a phase increment that advances or delays gait relative to the reference motion.The updated target pose uses the reference motion and a pose-displacement operation that handles 3D rotations.Source: §B.2 · p. 15

B.3 REWARD DESIGN

  • The composite reward combines gait, energy, upper-arm imitation, and human–exoskeleton-interaction terms; gait reward includes step-size and velocity targets, head stability, and sway.The head term penalizes abrupt changes in head angle and acceleration, while the sway term discourages abnormal body-orientation deviations.Source: §B.3 · p. 16

B.4 TRAINING CURVES

  • Training curves report each reward component across five runs, with shaded regions marking ±2σ.Source: Figure B.2 · p. 17

C INTRAMUSCULAR REGULARIZATION ENHANCES PHYSIOLOGICAL PLAUSIBILITY OF MUSCLE ACTIVATION

  • Without intramuscular regularization, line-muscle activations diverged despite the baseline activation regularizer; humans cannot selectively control individual segments of one muscle.Source: C INTRAMUSCULAR REGULARIZATION… · p. 17
  • Intramuscular regularization reduced activation discrepancies across line segments, while training without it produced imbalances absent from human data and physiologically implausible segment-specific control.The baseline activation regularizer alone did not prevent these imbalances.Source: Figure C.3 · p. 17, C INTRAMUSCULAR REGULARIZATION… · p. 17

D DETERMINING METABOLIC ENERGY EXPENDITURE PARAMETERS

  • The selected metabolic-energy exponents were α = 1.5 and β = 1.0: α brought preferred walking speed closest to the 1.25 m/s human reference, while β made cost scale linearly with muscle mass.Source: D DETERMINING METABOLIC ENERGY… · p. 18

E.1 GAUSSIAN PROCESSES CONSUME EXCESSIVE MEMORY

  • Gaussian-process datasets above 20k samples exceeded the RTX 3070’s 8 GB VRAM limit because GP memory scales as O(N^2).Source: §E.1 · p. 18

E.2 DATASET SIZE REQUIRED FOR PLAUSIBLE RESULTS EXCEEDS GP’S COMPUTATIONAL LIMIT

The optimizer’s neural-network surrogate supports larger training sets than the Gaussian-process memory limit, while Huber loss and Latin Hypercube Sampling yield smoother landscapes. Its objective regularizes parameter magnitude and changes across walking speeds.

E.2 DATASET SIZE REQUIRED FOR PLAUSIBLE RESULTS EXCEEDS GP’S COMPUTATIONAL LIMIT

  • 20k samples improved optimization-landscape smoothness and plausibility over 5k and 10k, while neural-network surrogates used large datasets efficiently and Gaussian-process memory became prohibitive.At 5k and 10k, landscapes were noisy and contained unrealistic local minima.Source: Figure E.5 · p. 18, §E.2 · p. 18

F HUBER LOSS AND LATIN HYPERCUBE SAMPLING ARE REQUIRED FOR SMOOTH SURROGATE NETWORK LANDSCAPE

  • Huber loss with Latin Hypercube Sampling produced the smoothest surrogate landscape; MSE was sensitive to noisy gait data, and uniform sampling caused aliasing even at N = 60k.The MSE sensitivity was especially apparent at higher exoskeleton gains, where assistance amplifies gait variability.Source: Figure F.6 · p. 19, F HUBER LOSS AND LATIN HYPERCUBE… · p. 19

G MOTIVATION OF THE GRADIENT PENALTY LOSS OF THE EXOSKELETON

  • A gradient penalty motivated by the Lipschitz condition aims to prevent abrupt predicted-CoT shifts and provide stable gradients for downstream optimization.The passage relates small gradient norms to local Lipschitz continuity and a uniform gradient bound to global continuity.Source: G MOTIVATION OF THE GRADIENT PENALTY… · p. 19

H OPTIMIZING EXOSKELETON WITH SURROGATE NETWORK

  • The surrogate optimizer minimizes predicted metabolic cost while regularizing extreme parameter magnitudes and promoting smooth solutions across gait parameters.These terms address fluctuations that could otherwise produce abrupt changes in control parameters.Source: H OPTIMIZING EXOSKELETON WITH… · pp. 19–20

I.1 SIMULATED CHARACTER

  • The simulated character’s muscle properties were matched to the Rajagopal et al. (2016) OpenSim model, with muscle-mass distributions based on Handsfield et al. (2014) and body dimensions matched to Lim et al. (2019a).Source: §I.1 · p. 20

I.2 EXOSKELETON

  • The simulation adopted Samsung Electronics’ GEMS exoskeleton; its controller combines exoskeleton torque with PD-controller torque and passes the net joint-motion torque requirement to the muscle controller.Source: §I.2 · p. 20

J SIMULATION PARAMETERS

  • Gait and exoskeleton parameters were uniformly sampled, while adaptive sampling increased sampling in muscle-parameter bins with shorter average human-controller episode lengths.Source: J SIMULATION PARAMETERS · p. 21

K GENERATED PATHOLOGICAL GAIT KINEMATICS

The framework generated gait patterns for pathological muscle conditions and compared them with unimpaired gait. Pathological cases used a fixed walking speed to compare severity consistently.

K GENERATED PATHOLOGICAL GAIT KINEMATICS

L OVERACTIVATION OF THE RECTUS FEMORIS IN THE SIMULATION

The simulation overuses rectus femoris, while simulated unassisted ground-reaction forces broadly resemble experimental profiles but have a sharper vertical peak. The appendix also contrasts HEI reward alternatives.

L OVERACTIVATION OF THE RECTUS FEMORIS IN THE SIMULATION

  • Deactivating rectus femoris leaves kinematics nearly unchanged but raises cost of transport by about 4%, and its inclusion reduces normalized cost of transport.The authors link the simulated overactivation to the muscle’s metabolic efficiency when used alone and suggest a synergy-control regularizer.Source: L OVERACTIVATION OF THE RECTUS FEMORIS… · p. 23, Figure L.8 · p. 23

M GROUND REACTION FORCE OF UNASSISTED GAIT

N ALTERNATIVE HEI REWARDS •

  • The alternatives include an assist-maximization reward and a no-HEI-reward condition that models energy minimization without additional exoskeleton considerations.Assist maximization encourages alignment with assistive power.Source: N ALTERNATIVE HEI REWARDS • · p. 24

O HEI MAKES HIGHER METABOLIC REDUCTION RATE THAN THE ALTERNATIVE MODELS

The proposed HEI reward yields higher measured metabolic reduction rates by shifting the gait parameters associated with maximum benefit, although the maximum normalized benefit remains similar across reward configurations.

O HEI MAKES HIGHER METABOLIC REDUCTION RATE THAN THE ALTERNATIVE MODELS

P.1 PATHOLOGY-SPECIFIC TRENDS IN OPTIMAL GAIN

Optimal assistance trends vary by pathology: some stable, energy-intensive gait patterns show increasing gains with severity, while unstable patterns show decreasing gains or no clear trend. The authors also report that HEI reward changes the calcaneus trend and describe their linearity and anthropometric checks.

P.1 PATHOLOGY-SPECIFIC TRENDS IN OPTIMAL GAIN

  • Crouch and calcaneus gaits show increasing optimal gains with severity, whereas equinus and waddling gaits show decreasing gains.The first group has stable but energetically demanding patterns; the latter prioritizes precarious balance over energy optimization.Source: §P.1 · p. 26

P.2 THE CAUSE OF UNCLEAR TREND OF FOOT DROP IS EXCESSIVE GAIT VARIABILITY OF FOOT DROP

  • Foot drop lacks a clear severity–gain trend; severe foot drop has a CoT coefficient of variation nearly four times higher than other pathologies.Frequent toe-ground collisions increase gait variability and obscure a systematic relationship.Source: §P.2 · p. 26, Figure P.11 · p. 26, §P.1 · p. 26

P.3 EFFECT OF HEI REWARD ON OPTIMAL GAIN TRENDS

  • Including the HEI reward changes calcaneus from a decreasing-gain pattern to the expected increasing-gain Group A trend, while linearity remains strong without it.The authors conclude the reward is needed to reveal the pathological gait’s assistance trend.Source: §P.3 · p. 27, Figure P.12 · p. 27

P.4 R2 VALUES FOR LINEAR FITS OF OPTIMAL GAIN ACROSS PATHOLOGY SEVERITIES

  • The analysis evaluates linearity between optimal exoskeleton gain and pathology severity using Pearson’s R² across random seeds.Table P.6 reports the R² values; the supplied passage does not state the values.Source: §P.4 · p. 27, Table P.6 · p. 27

Q ANTHROPOMETRIC CONSISTENCY BETWEEN REAL AND SIMULATED DATA

R LIMITATIONS OF HILL TYPE MUSCLE DYNAMICS.

The appendix describes limits of Hill-type muscle dynamics, defines simulation-to-experiment similarity measures, and catalogs model and optimizer components. The Hill model’s statelessness limits history-dependent muscle behavior, especially in fast running.

R LIMITATIONS OF HILL TYPE MUSCLE DYNAMICS.

  • Hill-type muscle force combines activation, force–length, force–velocity, and passive force–length relationships, but the model has identified limitations.Source: R LIMITATIONS OF HILL TYPE MUSCLE… · p. 28
  • Because the Hill model is stateless, it cannot reproduce force enhancement after stretch or force depression after active shortening.In fast running, this can lead simulations to underestimate available force and overestimate metabolic expenditure.Source: R LIMITATIONS OF HILL TYPE MUSCLE… · p. 28
  • The Hill model typically treats optimal fibre length and maximum shortening velocity as fixed constants, although experimental studies show these parameters depend on muscle activation.Source: R LIMITATIONS OF HILL TYPE MUSCLE… · p. 28

S QUANTITATIVE COMPARISON BETWEEN SIMULATED AND EXPERIMENTAL DATA

  • Similarity assessment uses NRMSE for normalized magnitude error, Pearson R for waveform shape, and normalized dynamic time warping for temporal alignment.Lower NRMSE and NDTW indicate closer agreement; R values nearer 1 indicate stronger temporal-pattern similarity.Source: S QUANTITATIVE COMPARISON BETWEEN… · p. 29

S.1 UNASSISTED GAIT / S.2 ASSISTED GAIT

T.1 HUMAN CONTROLLER / U.1 HUMAN CONTROLLER AND EXOSKELETON / U.2 RL REWARD

  • The human controller uses PoseNet, a PD controller, and a muscle coordination network to turn target poses and torques into muscle activations.The exoskeleton is parameterized by torque gain and controller time delay; the glossary also defines gait, energy, and human–exoskeleton-interaction reward terms.Source: §U.1 · p. 31, §U.2 · p. 31

U.4 MCN LOSS AND INTRAMUSCULAR REGULARIZATION / U.5 METABOLIC ENERGY

  • The muscle coordination network loss includes torque matching, activation regularization, and an intramuscular regularizer for coherent line-muscle activations.The appendix also defines the metabolic-energy model’s effort- and fatigue-like parameters.Source: §U.4 · p. 32, §U.5 · p. 32

U.6 EXOSKELETON OPTIMIZER / U.7 OTHER HYPERPARAMETERS

  • The optimizer samples control parameters, predicts cost of transport with a surrogate, and identifies optimal gain and delay; the appendix lists its training and hyperparameter notation.Source: §T.1 · p. 30, §U.6 · p. 32, §U.7 · p. 33
Loading 2601.22550v1…