PaperlayerView exact revision on arXiv

Source-linked AI summary

Context-Continuous Preference Learning for Exoskeleton Personalization

Sunin Baek, Sungwoo Park, Daekyum Kim

arXiv:2609.28427v1cs.LGcs.RO

TL;DR

User-specific feedback for exoskeleton assistance across operating conditions requires physical exposure, and it is unclear when observations transfer between conditions. The paper introduces CCPL, which shares observations across nearby contexts while retaining context-specific utility estimates; simulations and retrospective human-data analyses support sharing when preferences vary smoothly, but identify negative transfer when continuity is weak.

  • Problem

    Collecting user-specific feedback requires users to experience assistance settings, while time, effort, fatigue, adaptation, and safety constrain data collection across operating contexts.

    Source: §I · p. 1

  • Method

    CCPL models latent utility jointly over assistance parameters and operating context, sharing observations through Gaussian-process covariance while retaining context-specific estimates.

    Source: §I · p. 2

  • Results

    In both human studies, CCPL improved reconstruction and held-out prediction over independent learning under limited feedback, while its advantage over pooled learning varied.

    Source: §IV · pp. 9–10

  • Takeaways & Limitations

    The findings support context continuity as a basis for sharing preference observations under limited feedback, with benefits depending on how preferences vary across conditions.

    Source: §V · p. 10

    The small cohort of healthy adults and limited range of evaluated contexts constrain generalizability; larger cohorts are needed to assess variation across individuals.

    Source: §IV · p. 10

Abstract

from arXiv · show

Personalizing exoskeleton assistance across operating conditions is constrained by the time and physical effort required to collect user feedback. We examined whether a user's preference landscape varies smoothly across operating conditions and when this continuity supports learning from limited feedback. We propose Context-Continuous Preference Learning (CCPL), a Gaussian-process preference model that shares observations across nearby contexts while retaining context-specific utility estimates. We evaluated CCPL through simulations and retrospective analyses of ankle and elbow exoskeleton preference data from nine healthy adults. In simulations, CCPL improved reconstruction and preference-based Bayesian optimization relative to independent learning when preferences varied smoothly, but showed negative transfer when continuity was weak. In both human studies, full-data reference landscapes estimated separately for each participant and context tended to be more similar between nearby operating conditions. With five exposures per context, CCPL increased mean reconstruction correlation with these references from 0.644 to 0.720 for ankle assistance and from 0.476 to 0.526 for elbow assistance relative to independent learning. The five-exposure budget was approximately 37% lower for ankle and 17% lower for elbow than the estimated independent-learning budgets needed to match these correlations. CCPL also improved held-out response prediction relative to independent learning, while benefits over pooled learning varied. These findings support context continuity as a basis for sharing preference observations under limited feedback, although benefits for online personalization in humans remain to be established.

I. INTRODUCTION

Personalizing assistance across operating conditions requires feedback from users, but collecting it can demand repeated physical exposure and become constrained by time, effort, fatigue, adaptation, and safety. The study asks whether preference landscapes vary smoothly across contexts and introduces CCPL to share nearby-context observations while preserving context-specific estimates.

I. INTRODUCTION

  • Collecting user-specific feedback requires experiencing assistance settings, and time, physical effort, fatigue, adaptation, and safety constrain available data.Preference judgments still require physical exposure even when they reduce reliance on physiological measurements.Source: §I · pp. 1–2
  • Because adding operating conditions can require more observations from the same user, reusing observations across contexts may address a scaling challenge while preserving personalized differences.Prior approaches include condition-specific optimization, interpolation, and learned relationships between assistance and gait conditions.Source: §I · p. 1
  • The paper hypothesizes that nearby conditions have more similar preference landscapes, while weaker continuity can make cross-context sharing harmful relative to learning each context separately.Context continuity denotes the extent of smooth variation at the scale of sampled conditions.Source: §I · p. 2
  • CCPL models preference over assistance parameters and operating context, sharing observations between nearby conditions while retaining context-specific estimates.The study compares it with separate per-context learning and context-blind pooled learning.Source: §I · p. 2

1) Problem Formulation and Preference Observations:

  • Ordinal assessments and pairwise responses constrain relative utility within observed contexts, but do not directly identify absolute utility offsets between contexts.Feedback budgets count exposures; ordinal-derived comparisons use existing assessments and require no additional exposure.Source: 1) Problem Formulation and Preference… · p. 2

2) Gaussian-Process Preference Model:

The preference model places a Gaussian-process prior on latent utility and fits it from weighted ordinal-derived and direct pairwise comparisons. It approximates the posterior with a Gaussian to make predictions at query inputs.

2) Gaussian-Process Preference Model:

  • Latent utility over normalized model inputs is assigned a zero-mean Gaussian-process prior with covariance matrix K.Repeated observations at the same model input refer to the same latent variable.Source: 2) Gaussian-Process Preference Model · p. 2
  • The covariance kernel uses coordinate-specific length scales, signal variance, and diagonal regularization to model dependence among latent utilities.The diagonal regularization is retained in the training covariance, while query covariances use the signal kernel.Source: 2) Gaussian-Process Preference Model · pp. 2–3
  • Direct pairwise responses and ordinal-derived comparisons use a probit preference likelihood with an indifference interval for ties.Directly reported ties favor utility differences within [−ϵ, ϵ].Source: 2) Gaussian-Process Preference Model · p. 3
  • Each ordinal anchor contributes total comparison weight 0.5, and the weighted likelihood combines these comparisons with unit-weight direct responses.The posterior mode is obtained by minimizing the negative log posterior, followed by a Laplace Gaussian approximation for prediction; primary hyperparameter settings are fixed across users and subsets.Source: 2) Gaussian-Process Preference Model · p. 3

3) Learning Strategies and Comparators:

The three strategies differ in whether they separate contexts, discard context, or share information through a context-aware Gaussian process. Simulations vary preference continuity by mixing smooth and independently generated utility components.

3) Learning Strategies and Comparators:

  • CCPL fits utility jointly over actions and contexts, linking distinct context-specific utilities through covariance while retaining context-specific predictions.Independent learning fits separate models per context; pooled learning shares one context-blind utility, so its predictions cannot vary by context.Source: 3) Learning Strategies and Comparators · p. 3
  • CCPL’s action and context covariance components use separate length scales, which govern how quickly correlation decreases across their respective inputs.Short context length scales weakly couple distinct contexts, whereas long scales encourage similar preferences.Source: 3) Learning Strategies and Comparators · p. 3
  • Figure 1 contrasts separate per-context utilities, a context-blind pooled utility, and CCPL’s context-aware utility with covariance-based sharing.Its dashed curve traces the maximizing action across contexts.Source: Figure 1 · p. 4

1) Simulation Setup:

  • Simulated utilities mix a smoothly varying component with independently generated context-specific components, with γ = 0 representing independent utilities and γ = 1 retaining only the smooth component.The same component landscapes are used across the five γ settings for each instance.Source: 1) Simulation Setup · p. 4
  • Simulated ordinal labels use two utility thresholds, while direct comparisons treat differences within 0.15 as ties and otherwise use noisy preference responses.Models are evaluated at budgets of 5, 10, 15, 20, 30, 40, and 50 exposures per context; ordinal partners are capped at 32 per anchor.Source: 1) Simulation Setup · p. 4

2) Evaluation Protocols:

The evaluation compares strategies retrospectively on shared recorded histories and uses separate protocols for utility reconstruction and preference-based optimization. The human studies collected ordinal and pairwise feedback across ankle or elbow assistance conditions.

2) Evaluation Protocols:

  • Reconstruction uses identical response-independent action sequences for all strategies and measures correlation between predicted and ground-truth utility over each context’s action grid.Constant predictions receive a correlation of zero.Source: 2) Evaluation Protocols · p. 4
  • Preference-based optimization follows each strategy’s adaptive trajectory and evaluates the recommended action by its normalized ground-truth utility.After the initial exposure, actions are selected by line-restricted posterior sampling once comparison data are available.Source: 2) Evaluation Protocols · p. 4

1) Participants, Ethics, and General Procedures:

  • Nine healthy adults participated: six in the ankle study, six in the elbow study, and three in both.Participants provided ordinal acceptability assessments and, from the second exposure in each context, comparisons with the preceding setting; analyses used final responses after consistency checks.Source: 1) Participants, Ethics, and General… · p. 5

2) Ankle Exoskeleton Study:

  • In the ankle study, participants evaluated 101 assistance profiles at each of three treadmill speeds: 1.00, 1.25, and 1.50 m/s.The assistance action grid contained 775 admissible profiles, varying peak plantarflexion torque and peak timing.Source: 2) Ankle Exoskeleton Study · p. 5

3) Elbow Exoskeleton Study:

The elbow study varied assistance profiles across load and prescribed fatigue-induction contexts, then used limited-feedback analyses to evaluate reconstruction, prediction, and source-observation reuse. Its analysis compared context-specific reference models and learning strategies under matched evaluation procedures.

3) Elbow Exoskeleton Study:

  • The elbow-assistance action grid contained 840 profiles spanning torque 0.5–5.0 N·m (0.5-N·m steps), onset 0–30% ROM, and rise width 5–60% ROM; testing used five load–dose contexts.The five contexts combined low and high loads at doses 0 and 1, plus intermediate load at dose 0.5. Dose represented prescribed exercise rather than measured fatigue.Source: 3) Elbow Exoskeleton Study · pp. 5–6
  • Each context included 26 ordinal assessments and 25 direct pairwise responses, with a shared five-profile sequence followed by context-specific profile sampling.Participants rested between contexts; the two zero-dose contexts came first, followed by dose 0.5 and then dose 1.Source: 3) Elbow Exoskeleton Study · p. 6

4) Analysis Setup:

  • A separate GP preference model fitted to all observations for each participant and context supplied the common full-data reconstruction reference.Action and context coordinates were normalized, with strategies sharing fixed action length scales and CCPL additionally using fixed context length scales.Source: 4) Analysis Setup · p. 6

5) Evaluation Protocols:

  • Evaluation measured context-distance patterns, reconstruction against full-data references, held-out response NLL, and target reconstruction when other contexts supplied source histories.Reconstruction used consecutive windows of 2–20 exposures per context; five exposures represented a limited-feedback condition, and independent learning’s matching budget was estimated without extrapolation.Source: 5) Evaluation Protocols · p. 6

D. Statistical Analysis

Simulation results favored CCPL over independent learning for reconstruction and preference-based optimization when context continuity was strong, while weak continuity produced negative transfer. The statistical analysis averaged across configurations and participants and reported paired differences with confidence intervals.

D. Statistical Analysis

  • Simulation estimates averaged contexts within instances, weighted six dimensionality configurations equally, and used bootstrap confidence intervals.Human-study analyses treated participants as statistical units and paired method differences within participants; results used percentile 95% confidence intervals without null-hypothesis tests or multiple-comparison adjustment.Source: D. Statistical Analysis · pp. 6–7

1) Limited-Feedback Reconstruction:

  • At γ = 1 and five exposures per context, reconstruction correlations were 0.611 for CCPL, 0.441 for independent learning, and 0.567 for pooled learning.CCPL had the highest mean reconstruction correlation at every evaluated budget; its advantage over independent learning declined from 0.170 at five exposures to 0.097 at 50.Source: 1) Limited-Feedback Reconstruction · p. 7, Figure 3 · p. 7

2) Preference-Based Bayesian Optimization:

  • At γ = 1 and five exposures per context, recommended-action utilities were 0.834 for CCPL, 0.621 for independent learning, and 0.792 for pooled learning.CCPL had the highest mean normalized ground-truth utility across evaluated budgets; its advantage over independent learning was smaller at 50 than at five exposures.Source: 2) Preference-Based Bayesian… · p. 7, Figure 3 · p. 7

3) Effects of Dimensionality and Context Continuity:

  • At γ = 1 and 10 exposures per context, CCPL outperformed independent learning in reconstruction and optimization across all six dimensionality configurations.Its mean advantage was larger with two context dimensions than one, although the two-dimensional setting also supplied more contexts for sharing; optimization advantage grew with action dimensionality, while reconstruction advantage did not increase monotonically.Source: 3) Effects of Dimensionality and… · p. 7, Figure 3 · p. 7
  • At γ = 0 and 0.25, CCPL showed negative transfer against independent learning in both reconstruction and optimization; at γ = 1, both differences were positive.Across all five γ settings, CCPL also had higher budget-averaged performance than pooled learning, with paired-difference confidence intervals above zero for both outcomes.Source: 3) Effects of Dimensionality and… · pp. 7–8, Figure 3 · p. 7

B. Human Results

Across ankle and elbow studies, reference landscapes were more similar at nearby contexts, and CCPL improved limited-feedback reconstruction and held-out prediction relative to independent learning. Reusing source histories helped at small target budgets, but source-count patterns and comparisons with pooled learning varied by study.

1) Context-Related Preference Structure:

  • Reference-landscape similarity decreased with normalized context distance in both studies, with mean participant-level slopes of −0.424 for ankle and −0.325 for elbow.All six participants in each study had negative slopes, although the relationship was nearly flat for one elbow participant.Source: 1) Context-Related Preference Structure · p. 8

2) Limited-Feedback Reconstruction:

  • With five exposures per context, CCPL’s mean reference correlations were 0.720 for ankle and 0.526 for elbow, exceeding independent learning by 0.076 and 0.050, respectively.All six participants in each study had higher correlation with CCPL than independent learning; estimated independent-learning budgets to match CCPL’s five-exposure performance were 7.94 ankle and 6.02 elbow exposures.Source: 2) Limited-Feedback Reconstruction · p. 8

3) Held-Out Response Prediction:

  • CCPL had lower mean held-out NLL than independent learning at every evaluated budget in both studies, but five-exposure comparisons with pooled learning differed by study.At five exposures, CCPL reduced ankle NLL by 0.050, while elbow NLL was higher with CCPL by 0.023.Source: 3) Held-Out Response Prediction · p. 8

4) Reuse of Source Observations:

  • With five target exposures and all other contexts as sources, CCPL mean reconstruction correlations were 0.780 for ankle and 0.601 for elbow.At 20 target exposures, the advantage over independent learning was 0.0287 for ankle and −0.0144 for elbow; additional sources improved ankle correlation throughout, while elbow gains reversed at larger budgets.Source: 4) Reuse of Source Observations · pp. 8–9, Figure 5 · p. 9

IV. DISCUSSION

The discussion finds that sharing preference observations can improve learning under limited feedback when preferences vary smoothly across contexts, while benefits depend on context continuity and feedback budget. The conclusion supports context continuity as a basis for sharing, but says prospective human studies are needed to establish online benefits.

IV. DISCUSSION

  • CCPL’s reconstruction advantage over independent learning was larger with fewer target observations and diminished, becoming negative in the elbow study, as target feedback increased.CCPL retained its held-out prediction advantage; the same exposure count represented a larger fraction of the reference history in the elbow study than in the ankle study.Source: §IV · p. 9
  • CCPL outperformed pooled learning in simulations at every tested continuity level, but retaining context did not prevent negative transfer when continuity was weak.In the human studies, reconstruction gains over pooled learning appeared under limited feedback, while advantages in source reuse and held-out prediction were less consistent.Source: §IV · p. 10
  • At γ = 0.5, CCPL’s budget-averaged reconstruction difference from independent learning was near zero, yet its budget-averaged recommended-action utility was higher.The discussion notes that selecting a high-utility action need not require reconstructing the entire landscape accurately.Source: §IV · p. 10
  • Preference feedback may change with assistance experience, so online personalization may need to account for user–robot co-adaptation and discount older observations.The ankle study’s higher-torque settings at unfavorable peak timings may have reduced later preference for higher-torque settings; the authors present this as a possibility.Source: §IV · p. 10
  • Future extensions could combine context sharing with transfer across users or adapt the observation model to physiological and task-performance objectives.Cross-user sharing would need to preserve individual differences, and other objectives would require assessing whether responses vary smoothly across contexts.Source: §IV · p. 10
  • The retrospective evaluation did not test online assistance selection in humans, and model-estimated full-data references may not recover users’ underlying preferences accurately.Generalizability is also limited by the small healthy-adult cohort and evaluated contexts; elbow dose effects could not be separated from testing-order effects.Source: §IV · p. 10

V. CONCLUSION

  • In simulations and retrospective ankle and elbow analyses, CCPL improved reconstruction and held-out prediction over independent learning under limited feedback, but gains depended on context continuity.Nearby operating conditions tended to have more similar full-data reference landscapes; simulations showed negative transfer when continuity was weak.Source: §IV · p. 9, §V · p. 10

A. Preference Modeling Details

The supplementary methods describe how ordinal comparisons are weighted and how synthetic preference landscapes and action-selection procedures are constructed. The simulations vary continuity and evaluate reconstruction and preference-based optimization over feedback budgets.

A. Preference Modeling Details

  • Each eligible unordered observation pair contributed twice, with each observation serving as anchor; both weighted terms were retained, and normalization did not remove dependence among comparisons from one ordinal assessment.Each anchor’s total contribution was normalized, but dependence among comparisons derived from the same ordinal assessment remained.Source: A. Preference Modeling Details · p. 14
  • Posterior-mode fits used trust-ncg with analytic gradients and Hessians, warm starts for source reuse, and fallback solvers or initializations when convergence failed.Fits allowed up to 300 iterations with a gradient tolerance of 10^-6; human-fit acceptance checks included finite posterior values and covariance and Hessian conditions.Source: A. Preference Modeling Details · p. 14
  • Held-out response probabilities integrated the preference likelihood over the posterior Gaussian for the latent utility difference, and responses were scored by negative log probability.The posterior mean and variance of that utility difference were used in the integration.Source: A. Preference Modeling Details · p. 14

1) Synthetic Landscapes and Action Selection

  • Synthetic utilities combined a context-varying plateau component with a sinusoidal residual, then normalized each utility slice over its action grid.The residual comprised eight sinusoidal components; context continuity mixed shared and independently resampled landscape parameters, while retaining instance-specific ordinal thresholds.Source: 1) Synthetic Landscapes and Action… · pp. 15–16
  • Neighboring-context correlation rose from 0.240 at γ = 0 to 0.944 at γ = 1, while mean absolute utility difference fell from 0.182 to 0.027.Limited-feedback reconstruction sampled actions under a minimum-distance constraint; PBO instead selected actions using the current context’s posterior and jointly sampled utilities.Source: 1) Synthetic Landscapes and Action… · p. 16

2) Performance Across Feedback Budgets, Dimensionality, and Context Continuity

Across simulated continuity settings, performance curves compare independent learning, pooled learning, and CCPL across feedback budgets and dimensionality configurations. At γ = 1 with 10 exposures per context, CCPL’s paired differences from independent learning were positive for both reconstruction and optimization.

2) Performance Across Feedback Budgets, Dimensionality, and Context Continuity

  • At γ = 1 and 10 exposures per context, CCPL’s reconstruction Pearson-correlation advantage ranged from 0.088 [95% CI 0.073, 0.104] to 0.202 [0.172, 0.233], and recommended-action utility advantage ranged from 0.072 [0.033, 0.117] to 0.314 [0.223, 0.407].Fig. S1 shows reconstruction in the left column and preference-based Bayesian optimization in the right across five γ settings and exposures from 5 to 50.Source: 2) Performance Across Feedback… · p. 16, Figure S1 · p. 17

C. Human Studies

The ankle and elbow studies each included six participants, with three participating in both. The studies used specified profile-selection rules and a shared normalized context length scale for CCPL.

C. Human Studies

  • Six participants completed each study, with three participating in both; Table S1 reports characteristics of the ankle and elbow cohorts.The study columns represent overlapping cohorts.Source: C. Human Studies · p. 18, Table S1 · p. 18
  • Ankle exposures 10–101 and elbow exposures 6–25 were selected using minimum distances from preceding profiles; the final elbow profile also had to be distant from the repeated reference.Ankle distances used peak torque and peak timing without normalization; elbow distances used normalized action-space distance.Source: C. Human Studies · p. 18
  • CCPL used a normalized context length scale of 0.50 for each context variable; action length scales differed by assistance variable.Action length scales were 1/3 and 4/15 for ankle torque and timing, and 0.25 for each elbow action variable.Source: C. Human Studies · p. 18

1) Reference Landscapes Across Participants and Contexts

The analyses constructed participant-specific, context-specific reference landscapes and assessed their similarity across context distances. They evaluated reconstruction and tested whether CCPL's performance persisted across context length scales.

1) Reference Landscapes Across Participants and Contexts

  • Reference landscapes were fitted separately for each participant and context using 101 ankle exposures or 26 elbow exposures; elbow plots project the three-dimensional action grid for display.Ankle references were evaluated over peak plantarflexion torque and peak timing; elbow projections display torque magnitude and onset position after maximizing over rise width.Source: Figure S2 · p. 19, Figure S3 · p. 20
  • Within-participant reference-landscape correlations were related to normalized context distance, with participant-specific slopes used for group inference.More negative slopes indicate a greater decline in similarity as context distance increases.Source: 1) Reference Landscapes Across… · p. 18, Figure S4 · p. 21

2) Reconstruction Evaluation and Variation

  • Reconstruction was measured by Pearson correlation with the corresponding full-data reference, with constant estimated landscapes assigned correlation zero.Constant references were treated as invalid; posterior means were normalized separately within each context.Source: 2) Reconstruction Evaluation and… · p. 22

3) Sensitivity to the Context Length Scale

  • The post hoc sensitivity analysis refitted CCPL at five context length scales using K = 5 windows; other feedback budgets and partial-source configurations were not reevaluated.The analysis used complete histories from all other contexts for source reuse and did not inform primary hyperparameter selection.Source: 3) Sensitivity to the Context Length… · p. 22
  • Across tested context length scales, CCPL's reconstruction advantage over independent learning stayed positive except for elbow at ℓc = 1.00; source-reuse gains stayed positive in both studies.For source reuse, 95% confidence intervals were above zero throughout; for reconstruction, they excluded zero except for elbow at ℓc = 1.00.Source: 3) Sensitivity to the Context Length… · p. 22, Figure S6 · p. 23

2) Reconstruction Evaluation and Variation

  • Fig. S5 compares CCPL–independent reconstruction-correlation differences across matched contiguous windows from K = 2 to 20 exposures per context.At K = 5, the figure includes 97 ankle and 22 elbow windows per context; overlapping windows were not treated as independent inferential units.Source: 2) Reconstruction Evaluation and… · p. 22, Figure S5 · p. 23
Loading 2609.28427v1…