PaperlayerView exact revision on arXiv

Source-linked AI summary

Targeted Active Learning for Preference-Based Treatment Effects on Multivariate Outcomes

Lola Giordani, Mathieu Even, Chloé Geoffroy, Jean-Christophe Corvol, Raphaël Porcher, Federico Pavone

arXiv:2610.03824v1stat.MLcs.LGstat.ME

TL;DR

Clinical decisions often involve multiple outcomes with patient-varying trade-offs, but preference rules must be elicited through costly comparisons. The paper selects comparisons for their expected information about the causal treatment effect and reports better overall performance than existing criteria on semi-synthetic Parkinson’s disease data.

  • Problem

    Clinical decisions balance multiple outcomes with patient-varying importance, while learning the preference rule from costly pairwise comparisons leaves uncertainty in the causal treatment effect.

    Source: §1 · p. 1, §3 · p. 5

  • Method

    CPTE-EIG selects clinician comparisons by expected information gain about the CPTE, using a Gaussian approximation from linearising the estimand around posterior mean utility.

    Source: §1.1 · p. 2, §4.1 · p. 7

  • Results

    On a semi-synthetic Parkinson’s disease cohort with 13 outcomes, CPTE-EIG was best-performing in CPTE estimation accuracy, policy regret, and robustness across scenarios.

    Source: §6 · p. 9, §5 · p. 9

  • Takeaways & Limitations

    The authors expect targeted elicitation to help wherever outcomes are compared rather than measured on a common scale.

    Source: §6 · p. 10

    The evaluation is semi-synthetic, uses utilities chosen by the authors rather than elicited from clinicians, and its treatment effects and policies have no clinical meaning.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Treatment efficacy is traditionally demonstrated on the basis of a single primary outcome. However, clinical decision-making usually requires consideration of multiple outcomes, balancing expected benefits against potential risks. The relative value assigned to these outcomes varies substantially from one patient to another. Given a preference rule over outcome profiles, treatment effects and optimal policies can be defined and estimated. Such a rule is rarely available in practice: it must itself be estimated from pairwise comparisons of outcome profiles, which are costly to collect from clinical experts. We propose an active learning framework that selects which comparisons to query. Standard criteria maximize the information gained on the preference rule itself. We instead target the quantities of interest, and select the query that most reduces uncertainty on the treatment effect and on the optimal policy induced by the learned rule. Under a Gaussian process model of the preference rule, we derive a closed-form approximation of this criterion. On semi-synthetic data built from a Parkinson's disease cohort with 13 clinical outcomes, our criterion achieves lower treatment effect estimation error and lower policy regret than existing criteria at equal query budget.

1 INTRODUCTION

Clinical decisions often trade off multiple outcomes whose importance varies across patients, while standard scalar outcomes fix those trade-offs in advance. The paper proposes active preference elicitation that targets a causal treatment estimand, rather than the preference model alone.

1 INTRODUCTION

  • Clinical decisions must balance benefits and risks across outcomes whose relative importance varies by patient, a trade-off fixed population-wide by standard scalar outcomes.Parkinson’s disease illustrates the problem: dopamine replacement can improve motor symptoms while causing or worsening other complications.Source: §1 · p. 1
  • Pairwise judgments are costly and noisy, so selecting comparisons that inform the causal estimand matters when only a limited set can be collected.The learned preference rule’s uncertainty propagates to the CPTE.Source: §1.1 · p. 2

1.1 CONTRIBUTIONS AND OUTLINE OF THE PAPER

  • The paper formulates CPTE as uncertain under limited preference data and introduces CPTE-EIG to select comparisons informative about the causal estimand.The framework is evaluated on a semi-synthetic Parkinson’s disease cohort with 13 outcomes, with lower CPTE estimation error and treatment-policy regret at equal query budgets.Source: §1.1 · p. 2

1.2 RELATED WORKS

  • Existing active preference-learning criteria target preference models or predictions, whereas CPTE-EIG targets a treatment effect defined by combining preferences with multivariate potential outcomes.Composite outcomes and hierarchical comparisons also require weights or rankings to be fixed in advance, typically yielding a population-level number.Source: §1.2 · p. 3

2 BACKGROUND

The paper separates cohort data, which inform treatment-related outcome distributions, from clinician comparisons, which inform preferences. It defines the conditional preference treatment effect using multivariate outcomes and a preference rule.

2 BACKGROUND

  • The framework uses cohort data for treatment-related outcomes and sequential clinician comparisons for preferences, with the two sources answering different questions.The cohort is fixed in advance; comparisons are collected sequentially and independently of it.Source: §2 · p. 3

2.1 NOTATIONS AND ASSUMPTIONS

  • Outcomes are bounded ordinal vectors that need not share a scale, and νx represents an independent treated–control pair while preserving within-arm outcome dependence.The causal assumptions make the conditional potential-outcome laws identifiable from observed data.Source: §2.1 · pp. 3–4

2.2 COMPARING OUTCOME PROFILES

  • Preferences are elicited using a fixed, clinically justified summary of patient features, and the model represents an average clinician rather than separate individual raters.The preference rule gives the probability that one outcome profile is preferred to another at the query covariate, under a no-ties assumption.Source: §2.2 · p. 4

2.3 THE CONDITIONAL PREFERENCE TREATMENT EFFECT

  • The CPTE is the probability that a treated outcome is preferred to a control outcome among patients with covariate x; treatment is preferred when it exceeds 1/2.The policy maximizing expected preference treats when Ψx > 1/2, and decision errors are weighted by how clear-cut the decision is.Source: §2.3 · p. 5

3 ACTIVE LEARNING PROBLEM

The active-learning problem is to choose a limited sequence of clinician comparisons that improves CPTE estimates and the policies they induce on a target population. CPTE-EIG selects comparisons by their expected information about the CPTE.

3 ACTIVE LEARNING PROBLEM

  • Given a comparison budget and estimated potential-outcome laws, the goal is to choose queries that make CPTE estimates and induced policies as accurate as possible on the target population.Candidate profiles come from the cohort, and each query receives one binary clinician answer before beliefs about the preference rule are updated.Source: §3 · p. 5
  • CPTE-EIG scores candidate comparisons by expected information gain about Ψx and averages that score over the target population.The next query maximizes this population-averaged score.Source: §3 · p. 5
  • Direct evaluation of the criterion may require repeated posterior sampling and integration for every candidate and target covariate, making it computationally difficult.The paper introduces a computationally efficient approach to address this difficulty.Source: §3 · p. 5

4 EFFICIENT CPTE-EIG COMPUTATION UNDER GAUSSIAN PROCESSES

The framework models clinician preferences with latent utility contrasts and treats the CPTE as uncertain under the utility posterior. It linearizes this estimand to obtain a tractable query score measuring information about the CPTE.

4 EFFICIENT CPTE-EIG COMPUTATION UNDER GAUSSIAN PROCESSES

  • Preferences are modeled through latent utility contrasts with a Gaussian-process posterior; applying the nonlinear CPTE functional makes Ψx random with a law that is not closed-form.The posterior is approximated using a Laplace approximation.Source: §4 · p. 6

4.1 LINEARISATION AND ERROR CONTROL

  • Linearizing Tx around posterior-mean utility yields an affine estimand and Gaussian joint law with query contrasts; a computable bound controls the neglected remainder.The bound depends on posterior variances of target-pair contrasts and can be computed at each round.Source: §4.1 · pp. 6–7

4.2 CLOSED-FORM CPTE-EIG

  • CPTE-EIG scores a candidate by mutual information between Ψx and its binary answer, computed as the gap between answer entropy and its Ψx-conditioned entropy.The gap measures how much the answer informs the estimand.Source: §4.2 · p. 7
  • The score requires no posterior sampling or model refit: its terms come from the current posterior at fixed cost per candidate, then are averaged over target patients.The target-patient expectation and integrals are approximated by Monte Carlo.Source: §4.2 · p. 7

5 EXPERIMENTS

In the semi-synthetic Parkinson’s disease experiments, CPTE-EIG generally achieved stronger CPTE accuracy and policy regret than preference-learning baselines, with performance depending on treatment-effect size.

Results

  • At 80 comparisons, mild-effect CPTE MSE is 2.4 × 10−3 and regret is 0.012; CPTE-EIG leads both metrics from 40 comparisons onward.With strong effects, its error is five times lower than EPIG’s and fourteen times lower than BALD’s at 80 comparisons; with tiny effects, the three criteria tie within overlapping intervals.Source: Figure 2 · p. 8, §5 · pp. 8–9

Setup

  • The study compares CPTE-EIG with BALD, EPIG, and random acquisition across four horizons; EPIG and CPTE-EIG use the same target pairs but target different quantities.All methods share a Gaussian-process utility model, 20 random warm-start comparisons, and candidate sets; experiments use 15 seeds.Source: §5 · p. 8
  • The semi-synthetic cohort contains 2,000 PPMI patients with known potential outcomes, alongside 1.37 million candidate profile comparisons for preference elicitation.A simulated clinician’s outcome weights vary with age and sex, and treatment effects range from tiny to strong; the agonist improves tremor while worsening cognition, making the preferred treatment patient- and horizon-dependent.Source: §5 · p. 8

6 DISCUSSION AND CONCLUSION

The paper introduces CPTE-EIG to prioritize clinician comparisons that inform the CPTE, and reports strong performance in semi-synthetic Parkinson’s disease experiments. Its evaluation and model leave real preference data and broader uncertainty as important boundaries.

6 DISCUSSION AND CONCLUSION

  • CPTE-EIG selects clinician comparisons by expected information gain about the CPTE; linearization gives a tractable Gaussian approximation with a computable error bound.On a semi-synthetic Parkinson’s disease cohort with 13 outcomes, it performed best in CPTE accuracy, policy regret, and robustness across scenarios.Source: §6 · p. 9
  • The framework conditions on estimated potential-outcome laws and a single average clinician; experiments use simulated preferences, leaving real preference data untested.Real preferences may include clinician disagreement and inconsistent answers.Source: §6 · p. 9
  • In the Parkinson’s disease experiments, targeting the estimand made a limited elicitation budget go further; the authors expect similar value where outcomes lack a common scale.This broader applicability is presented as an expectation, not an evaluated result.Source: §6 · pp. 9–10

AI USE STATEMENT

The authors disclose extensive LLM assistance and describe the study’s ethical constraints and reproducibility materials. The experiments evaluate methodology on semi-synthetic data, not clinical treatment decisions.

AI USE STATEMENT

  • LLMs assisted with theoretical analysis, coding, writing, proofreading, and draft feedback; the authors verified derivations, tested code, edited text, and accept responsibility for the final work.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The experiments use de-identified PPMI data, collect no new patient or clinician data, and simulate the clinician answering comparisons.Source: ETHICS STATEMENT · p. 10
  • Because the cohort is semi-synthetic and utilities were chosen rather than elicited, the estimated effects and policies have no clinical meaning and should not guide Parkinson’s treatment.Clinical use would require preferences elicited from real clinicians, ideally patients too, and validation on independent data.Source: ETHICS STATEMENT · p. 10

REPRODUCIBILITY STATEMENT

  • The paper states that assumptions, proofs, the query-selection algorithm, and experimental construction details are provided in the appendices.Source: REPRODUCIBILITY STATEMENT · p. 10

Appendices

The appendix establishes bounded contrast uncertainty and the regularity needed for the subsequent linearization argument.

A BOUNDED CONTRASTS AND INTEGRABILITY

B PROOF OF THEOREM 1

The proof approximates the nonlinear CPTE by linearizing it around the posterior mean, yielding a Gaussian approximation with an explicit remainder bound. The approximation can be loose early in active learning and tightens as informative comparisons accumulate.

B PROOF OF THEOREM 1

  • Because Tx is nonlinear in U, the proof linearizes it around posterior mean mt and establishes a bound on the resulting error.Source: B PROOF OF THEOREM 1 · p. 15

B.1 THE LINEARISATION

  • A first-order Taylor expansion along mt + ϵh yields the linearized estimand Ψx, an affine functional of the Gaussian process and thus Gaussian jointly with any candidate-query contrast.Source: §B.1 · pp. 16–17

B.2 THE REMAINDER

  • The linearization is accurate when posterior uncertainty in outcome-pair contrasts is small relative to λ2, but its remainder bound can be loose early and tightens as informative comparisons accumulate.Source: §B.2 · p. 18

C PROOF OF THEOREM 2

The proof derives the Gaussian quantities needed to compute expected information gain for candidate comparisons, then applies that criterion to target-pool averages. Its example shows that a query informative about individual pairs can be uninformative about their average.

C.1 MEAN AND COVARIANCE OF (Ψx, DcU)

C.2 TWO PROPERTIES

  • Gaussian conditioning and a probit–Gaussian integral identity provide the two properties used to compute query-answer probabilities.Source: §C.2 · p. 19

C.3 CLOSED-FORM ANSWER PROBABILITIES

  • The proof uses the joint Gaussian law of (Ψx, DcU) to derive the conditional query-answer probability ηt,x,c(ψ) and marginal probability pt,c.Source: §C.3 · p. 20

C.4 GAUSS–HERMITE QUADRATURE AND THE FINAL CLOSED FORM

  • Gauss–Hermite quadrature approximates the resulting expectation, yielding the closed-form approximation in Equation (12).Source: §C.4 · pp. 20–21

D TARGETING THE RIGHT INFORMATION

The contrast the target pool averages over is 1

E QUERY SELECTION ALGORITHM

  • The algorithm fits a Laplace-approximate GP posterior from observed comparisons and estimates the target covariates and outcome pairs used in scoring queries.Source: E QUERY SELECTION ALGORITHM · p. 22
  • For each candidate, the algorithm estimates target-specific quantities, averages CPTE-EIG over target draws, and selects the query with the largest score.Source: E QUERY SELECTION ALGORITHM · p. 22

F.1 DATA GENERATION

The experiments use a semi-synthetic PPMI cohort and a separate pool of real outcome-profile comparisons, with preference weights varying by patient characteristics. All acquisition methods share a pairwise Gaussian-process model and are evaluated under a common query protocol.

F.1 DATA GENERATION

  • A 2,000-patient semi-synthetic PPMI cohort supplies paired treatment and control outcomes, while real profiles provide 1,369,284 candidate comparison queries.The cohort supports oracle CPTE evaluation; 2,000 candidate comparisons are held out and never queried.Source: §F.1 · pp. 22–23
  • Effects are evaluated at 1-, 3-, 5-, and 7-year horizons; the mild and strong scenarios worsen cognition while improving tremor, with cognitive harm increasing by horizon and age.The target distribution is uniform over patient-horizon pairs, and the effect magnitudes are chosen from oracle CPTE properties rather than method performance.Source: §F.1 · p. 23

F.2 THE SIMULATED CLINICIAN

  • The simulated clinician uses positive, covariate-dependent weights across five outcome domains, and each patient's CPTE depends on weights at that patient's covariates.The weights for neuropsychiatric and motor symptoms vary with age, while motor-complication weights vary with sex.Source: §F.2 · p. 23

F.3 METHODS AND PROTOCOL

  • All methods use the same PairwiseGP with a Laplace approximation and mixed kernel, while lengthscales, kernel mixture, and noise are fitted during evaluation.The output scale is fixed to 1 because the noise scale and kernel amplitude are not separately identifiable.Source: §F.3 · pp. 23–24
  • After 20 shared random comparisons, methods select the best-scoring query from 2,000 sampled candidates per round, with budgets of 20–150 comparisons and 15 seeds per budget.CPTE-EIG targets information about average Ψx, whereas EPIG targets individual answers to treated-control queries; BALD targets utility information.Source: §F.3 · p. 24

F.4 METRICS

CPTE-EIG has lower policy regret than EPIG and BALD in the strong-effect case, while acquisition methods overlap when the effect is tiny. Preference-prediction scores do not track CPTE error, and Gauss–Hermite quadrature matches Monte Carlo accuracy at lower computational cost.

F.4 METRICS

  • Policy regret weights each wrong treatment decision by |2Ψx −1|, while held-out accuracy, Brier score, and negative log-likelihood assess predicted preference answers.The held-out scores use 2,000 test comparisons; reported means and bands use 15 seeds and 95% percentile-bootstrap confidence intervals.Source: §F.4 · p. 24

F.5 TINY AND STRONG TREATMENT EFFECTS

  • At 20 comparisons, CPTE MSE is 6.6 × 10−3 for tiny effects, versus 2.1 × 10−2 for mild effects and 5.0 × 10−2 for strong effects.For tiny effects, CPTE-EIG, EPIG, and BALD overlap at every horizon.Source: Figure S1 · p. 25, §F.5 · pp. 25–26
  • At 80 comparisons, CPTE-EIG’s regret is 0.003, against 0.024 for EPIG and 0.053 for BALD.Under a strong effect, the ordering holds for both metrics at every horizon and budget from 40 comparisons onward, with wider gaps than in the mild case.Source: Figure S2 · p. 25, §F.5 · p. 25

F.6 LEARNING w IS NOT LEARNING Ψx

  • CPTE-EIG and EPIG are within 0.01 accuracy of each other at every budget, yet methods differ by factors of 2 to 5 in CPTE MSE between 40 and 120 comparisons.BALD is at most 0.01 behind them in accuracy from 80 comparisons onward.Source: Figure S3 · p. 26, §F.6 · p. 26

F.7 GAUSS–HERMITE QUADRATURE VERSUS MONTE CARLO

  • Even 10-node Gauss–Hermite quadrature matches 8,000-sample Monte Carlo CPTE MSE at every budget and horizon, while Gauss–Hermite is several times cheaper per acquisition round.The comparison covers Gauss–Hermite rules with 10 and 40 nodes and Monte Carlo rules with 2,000 and 8,000 samples.Source: Figure S4 · p. 26, Figure S5 · p. 27, §F.7 · p. 27

F.6 LEARNING w IS NOT LEARNING Ψx

  • BALD and Random produce identical queries and fitted models across treatment-effect scenarios, but their CPTE errors change with the estimand.The passage attributes this to their not using target-patient outcomes; utility quality along the contrasts that determine Ψx matters.Source: §F.6 · p. 26
Loading 2610.03824v1…