Source-linked AI summary
Preference-Based Learning for Exoskeleton Gait Optimization
Maegan Tucker, Ellen Novoseller, Claudia Kann, Yanan Sui, Yisong Yue, Joel Burdick, Aaron D. Ames
TL;DR
The paper addresses the lack of a framework for optimizing exoskeleton gait comfort and the difficulty of reliably specifying numerical scores for walking preferences. It introduces COSPAR, which learns from user preferences and suggestions, and reports that it identifies preferred gaits in simulation and human experiments.
Problem
Exoskeleton gait methods can generate stable walking, but a framework for optimizing user comfort is lacking, and users may not reliably provide numerical scores or demonstrations.
Source: §I · p. 1Method
COSPAR combines pairwise preference queries with user-suggested improvements to learn preferred exoskeleton gaits.
Source: §I · p. 1Results
COSPAR efficiently converged to optima in simulation and, in human experiments, identified compact preferred gait subregions whose posterior rankings matched participants’ blind rankings.
Source: §IV · p. 4, §V · p. 5, §VI · p. 6Takeaways & Limitations
COSPAR models individual users’ gait preferences and identifies preferred walking trajectories on the Atalante exoskeleton.
Source: §VI · p. 6
Abstract
from arXiv · showhide
This paper presents a personalized gait optimization framework for lower-body exoskeletons. Rather than optimizing numerical objectives such as the mechanical cost of transport, our approach directly learns from user preferences, e.g., for comfort. Building upon work in preference-based interactive learning, we present the CoSpar algorithm. CoSpar prompts the user to give pairwise preferences between trials and suggest improvements; as exoskeleton walking is a non-intuitive behavior, users can provide preferences more easily and reliably than numerical feedback. We show that CoSpar performs competitively in simulation and demonstrate a prototype implementation of CoSpar on a lower-body exoskeleton to optimize human walking trajectory features. In the experiments, CoSpar consistently found user-preferred parameters of the exoskeleton's walking gait, which suggests that it is a promising starting point for adapting and personalizing exoskeletons (or other assistive devices) to individual users.
I. INTRODUCTION
The paper targets comfortable walking gaits for the Atalante exoskeleton, where existing methods can generate stable walking but lack a framework for optimizing comfort. COSPAR learns from user preferences and suggestions rather than relying on numerical scores that may not reflect individual comfort.
I. INTRODUCTION
- Atalante can generate dynamically stable walking, but no current framework optimizes its gait for user comfort.The paper identifies comfort as a critical objective for exoskeleton gait optimization.Source: §I · p. 1
- Unlike methods that optimize quantitative metrics such as metabolic expenditure, the proposed approach incorporates personalized feedback without assuming a numerical comfort objective.The paper argues this avoids optimizing a quantity that may not align with personalized comfort.Source: §I · p. 1
- For exoskeleton gait generation, people may struggle to reliably provide numerical scores or demonstrations, while relative preferences can measure comfort more reliably.Earlier studies found preferences more reliable than numerical scores in information retrieval and autonomous driving,.Source: §I · p. 1
- COSPAR combines preference queries with user-suggested improvements to learn preferred exoskeleton gaits and provide insight into users’ gait preferences.The approach builds on dueling bandits – and coactive learning,.Source: §I · p. 1
II. GAIT GENERATION FOR BIPEDAL ROBOTS
The section describes how stable walking gaits are generated and why selecting comfortable exoskeleton gaits remains difficult. It motivates using user preferences to choose among precomputed gaits while retaining physical and stability constraints.
Partial Hybrid Zero Dynamics Method
- The partial hybrid zero dynamics method represents system dynamics on an invariant reduced-dimensional surface by defining virtual constraints and driving them to zero with feedback linearization.A partial surface is used because desired hip velocity is constant while actual velocity jumps at impact.Source: §II · p. 2
- The method represents desired position-modulating trajectories with Bézier polynomials whose coefficients are α, while timing depends on the state through τ(q).The velocity-regulating output is driven toward desired velocity vd.Source: §II · p. 2
- Stable periodic walking requires virtual constraints that yield an impact-invariant orbit on the partial zero dynamics surface, subject to impact and physical constraints.The gait optimization can vary its cost function and add physical constraints such as step length and foot height.Source: §II · p. 2
Gait Generation Applied to Lower-Body Exoskeletons
- For exoskeletons, the relationship between gait-optimization parameters and user preferences is not well understood, and generating stable, human-preferred gaits remains largely unexplored.Gait generation is time-consuming because both constraint tuning and program execution take time.Source: §II · p. 2
Gait Library
- A gait library precomputes nominal gaits over parameter grids for immediate selection, but does not specify which gait best matches a user’s comfort or preference.The paper therefore considers learning from users’ preferences.Source: §II · p. 2
III. PREFERENCE-BASED LEARNING ALGORITHM
COSPAR extends Bayesian preference learning for gait selection by combining pairwise comparisons with user-suggested improvements. It selects trials using a learned utility model and updates that model from the feedback collected.
The COSPAR Algorithm
- COSPAR extends Self-Sparring with coactive feedback, letting users suggest improvements alongside preference comparisons,.The algorithm fits a Bayesian preference model to feedback and uses it to select new trials.Source: §III · p. 2
Modeling Utilities from Preference Data
- The model represents each finite action’s latent user utility with a Gaussian prior and estimates utilities from noisy pairwise preference data.A kernel specifies prior covariance; the posterior can be approximated by a multivariate Gaussian, with preference noise varied to weight comparisons differently.Source: §III · p. 3
The Learning Algorithm
- At each iteration, COSPAR samples utility functions from the posterior and selects the action maximizing each sampled function.The algorithm then updates the posterior using the accumulated feedback.Source: §III · p. 3
- Users compare the current n actions with one another and with up to b remembered prior actions, and may report no preference.The authors expect n = 1 with a larger b, limited by what users can accurately remember, to reduce the trials needed to reach a preferred gait.Source: §III · p. 3
- Users may suggest an improved action, which COSPAR records as preferred to the tested action and can weight by β relative to pairwise feedback.For example, a user can request a longer or shorter step length.Source: §III · p. 3
IV. SIMULATION RESULTS
In simulation, COSPAR efficiently converged toward optimal actions on both the compass-gait biped’s cost-of-transport objective and synthetic objectives. On synthetic 2D functions, coactive feedback improved results across parameter settings, while the setting receiving the fewest preferences performed worst.
IV. SIMULATION RESULTS
- COSPAR efficiently converged to the optimum on both the compass-gait biped’s cost-of-transport objective and synthetic objective functions.The compass-gait experiment used pairwise preferences between step lengths at a fixed forward hip velocity of 0.2 m/s.Source: §IV · p. 4
Optimizing the Compass-Gait Biped’s Cost of Transport
- With preference feedback alone, COSPAR’s posterior utility increasingly peaked at the lowest-cost step length as comparisons accumulated.This simulation used n = 2, b = 0, without coactive feedback.Source: §IV · p. 4, Figure 2 · p. 4
Optimizing Synthetic Two-Dimensional Functions
- On 2D synthetic objective functions, coactive feedback always helps across the three tested parameter settings; n = 2, b = 0, which receives the fewest preferences, performs worst.Mean +/- standard error is reported over 100 repetitions; maximal and minimal objective function values are normalized to 0 and 1, respectively.Source: Figure 4 · p. 4
- After 150 iterations with n = 1, b = 1 and coactive feedback, COSPAR learned a sharp peak around the synthetic objective’s optimum rather than a globally accurate utility landscape.The 2D tests used 100 random objectives and held objective evaluations constant at 150 trials per setting.Source: §IV · pp. 4–5, Figure 3 · p. 4
V. HUMAN SUBJECT EXPERIMENTS
Human experiments on Atalante used COSPAR to personalize gait parameters from user preferences and coactive feedback. Blind rankings supported the learned preferences in both step-length and two-feature experiments.
V. HUMAN SUBJECT EXPERIMENTS
- COSPAR was tested on Atalante in a three-subject step-length experiment and a two-dimensional experiment optimizing two gait-feature pairs.The approach operates independently of the chosen gait features.Source: §V · pp. 5–6
- Metabolic expenditure measurements were uninformative about user preferences because participants were not required to expend effort toward walking.The experiments optimized comfort using preference and coactive feedback.Source: §V · p. 5
- For all three subjects, blind rankings of three step-length gaits matched COSPAR’s preference posterior after 20 trials.After five trials, COSPAR had identified a relatively compact preferred step-length subregion.Source: §V · p. 5
Learning Preferences over Multiple Features
- In subject 1’s two-feature experiments, estimated preferences agreed with a three-sample blind ranking across step-length/duration and step-length/width spaces.Phase diagrams compared gaits at minimum, mean, and maximum posterior utility.Source: §V · p. 6, Figure 7 · p. 6
VI. CONCLUSIONS
The work develops and demonstrates COSPAR for optimizing exoskeleton gait comfort from human preferences. In simulation and an Atalante user study, it models preferences and identifies compact regions of preferred gaits; broader parameter spaces and gait generation remain future directions.
VI. CONCLUSIONS
- In an Atalante user study, COSPAR modeled users’ preferences and identified compact subregions of preferred gaits.The study was the first application of preference-based learning to optimizing dynamic crutchless walking.Source: §VI · p. 6
- COSPAR efficiently learned to select optimal actions in simulation and was demonstrated as an interactive framework for comfort-focused gait optimization using human preferences.The framework was developed and demonstrated for optimizing exoskeleton gaits with human preference feedback.Source: §VI · p. 6
- Optimizing larger sets of gait parameters will likely require integrating COSPAR with methods for learning over high-dimensional feature spaces.The authors also identify extending beyond precomputed gait libraries to new gaits or controller designs as a future direction.Source: §VI · p. 6