Bayesian Propensity Score Predictive Inference for Extending Trial Findings to Target Populations

1.1 Background

• CODIACS randomized 150 post-ACS patients; intervention produced greater symptom reduction than usual care, with subgroup heterogeneity motivating extension to broader populations.The target population is post-ACS patients with depressive symptoms represented in the All of Us EHR cohort.Source: §1.1 · pp. 2–3

1.2 Generalizability and Transportability

• The study addresses separately sampled trial and All of Us cohorts, a transportability setting, while also developing methods for nested generalizability.Source: §1.2 · pp. 3–4

1.3 Related Work

• Existing approaches include outcome regression, propensity-score weighting, and hybrids; weighting can become unstable under covariate shift and does not explicitly model treatment-effect extrapolation.Trial-trained outcome models also face unreliable extrapolation when shifted covariates modify treatment effects.Source: §1.3 · pp. 4–5, §1.4 · p. 5

1.4 Our Contribution

• PSPI combines BART with separate spline terms for sampling-related variation, enabling distinct regularization and explicit linear-tail extrapolation beyond trial score boundaries.The inverse Gumbel transformation balances compression of small propensity scores against extrapolation distance.Source: §1.4 · pp. 5–7
• Across generalizability and transportability simulations, PSPI outperformed existing methods, particularly under substantial covariate shift.The inverse Gumbel transformation yielded lower bias and near-nominal coverage, with narrower credible intervals than alternative transformations.Source: §1.4 · p. 7

2.1 Notation and Setting

• Target estimands are potential-outcome means and the average treatment effect, averaged over the target population; the framework covers nested generalizability and non-nested transportability designs.The EHR cohort is treated as a simple random sample from the target population, while trial enrollment may be selective.Source: §2.1 · p. 8

2.2 Key Assumptions

• Identification assumes exchangeability over trial participation and positive participation probabilities, but these conditions do not ensure finite-sample overlap or rule out extrapolation-sensitive estimates.Exchangeability over participation requires measuring all covariates associated with potential outcomes and generally cannot be verified from observed data.Source: §2.2 · pp. 9–10

2.3.1 Outcome-Based Estimation and the Target Estimands

• The method imputes both potential outcomes for EHR individuals, using their empirical covariate distribution for MATE and a Bayesian bootstrap to account for target-distribution uncertainty in PATE.MATE is presented as a convenient approximation to PATE.Source: §2.3.1 · pp. 10–11

2.3.2 Outcome Model with the Sampling Propensity Score

• PSPI adds the estimated sampling propensity score to a joint, interaction-based outcome model, separating prognostic and treatment-effect functions so their regularization can differ.Separate regularization addresses the risk of disproportionately shrinking treatment effects and attenuating propensity-score contributions.Source: §2.3.2 · pp. 11–12

2.3.3 SplineBART

• SplineBART combines BART with a natural cubic spline of the transformed sampling propensity score in the treatment-effect component, allowing flexible fit within knots and linear extrapolation beyond them.The spline uses the propensity score as a one-dimensional summary of trial participation, while BART predictions do not naturally extend trends beyond the observed covariate range.Source: §2.3.3 · pp. 13–14

2.3.4 DSplineBART

• DSplineBART adds a separate propensity-score spline to the prognostic component as well as retaining one in the treatment-effect component, targeting more robust estimation of mean potential outcomes.All trial observations inform its prognostic spline, whose knots are placed across the full trial sample.Source: §2.3.4 · p. 15, DSplineBART · p. 15

2.3.5 Inverse Gumbel Transformation of Sampling Propensity Scores

• The inverse Gumbel transformation maps propensity scores to the real line, expands low scores more gradually than logit, and allocates more transformed range to trial scores to improve extrapolation and reduce variability.Low-score EHR observations may be sparsely represented or absent from the trial, requiring extrapolation toward the lower score range.Source: §2.3.5 · pp. 15–16

3.1 Simulation Design

• The simulations used 250 replicate datasets from a 100,000-person superpopulation, varying cohort sizes, outcome forms, covariate shift, and treatment allocation.Potential outcomes and trial participation depended on selected covariates, while other generated covariates served as noise variables.Source: §3.1 · pp. 17–18

3.2 Simulation Results

• In SS1, SplineBART and DSplineBART had lower bias and comparable RMSE to A-IPPW for treatment-effect estimation, while DSplineBART alone consistently estimated the treated mean with near-nominal coverage.Only IPPW and the two spline-based methods attained near-nominal coverage for the treatment effect; IPPW had wider intervals.Source: §3.2 · pp. 20–21, Table 1 · p. 21
• Under strong covariate shift, SplineBART and DSplineBART had much smaller bias and RMSE than other methods, with near-nominal coverage but wider intervals.This pattern held in nonlinear settings, where most other models showed severe undercoverage; both spline-based methods maintained near-nominal coverage for treatment effects and mean treated outcomes.Source: §3.2 · pp. 21–22

3.3 Choices of Transformations on Estimated Propensity Scores

• InvGumbel achieved the lowest bias and lower or comparable RMSE to Identity for treatment-effect estimation, with near-nominal coverage and narrower intervals than Logit and Cloglog.Identity compressed the score scale and was associated with undercoverage, whereas Logit and Cloglog led to excessive extrapolation and overly wide intervals; InvGumbel balanced these extremes.Source: §3.3 · pp. 23–25

4.1 Comparison of trial to EHR cohorts

• T1 includes patients with depression onset 2–6 months after ACS, while T2 includes onset at least 2 months after ACS; trial and EHR cohorts differed in distributions of treatment-effect modifiers and age.The transportability analyses used multiple imputation for missing covariates and pooled posterior draws across 20 imputed datasets.Source: §4.1 · pp. 25–27, §4.2 · p. 27

4.3 Main results

• SplineBART and DSplineBART estimated intervention-related BDI reductions of 4.3–5.1 points in T1 and 4.2–4.9 points in T2, versus 2.2 points in the trial-only analysis.DSplineBART produced the largest estimated reductions, consistent with A-IPPW; in T1, all PSPI models except FullBART had credible intervals excluding zero.Source: §4.3 · pp. 27–28, Figure 3 · p. 28
• Trial participants generally had higher sampling propensity scores than EHR patients, and both models’ treatment-effect splines showed positive relationships with the inverse-Gumbel-transformed score.Extrapolated spline trends predicted greater intervention-related BDI reductions at very low sampling propensity scores.Source: §4.3 · pp. 29–30, Figure 4 · p. 29

4.4 Sensitivity Analysis

• DSplineBART’s estimated treatment benefit was moderately sensitive to the bias-function prior: increasing σβ from 2 to 4 widened credible intervals, while smaller μβ yielded more negative MATE estimates.Point estimates approached zero at μβ = 8 in T1 and μβ = 4 in T2.Source: §4.4 · pp. 30–31, Figure 5 · p. 31

5 Discussion

• PSPI combines BART for high-dimensional covariates with separate natural cubic spline components for sampling-related variation, with linear extensions of fitted boundary trends beyond the observed score range.The inverse Gumbel transformation expands score values near zero more gradually and allocates more transformed range between trial and EHR cohorts.Source: §5 · pp. 31–32
• Across simulations, SplineBART and DSplineBART had lower bias, smaller RMSE, and closer-to-nominal coverage than alternatives for target average treatment effects, especially under substantial covariate shift.DSplineBART was the only method to consistently attain nominal coverage for potential outcomes across most scenarios.Source: §5 · p. 32
• In the application, SplineBART and DSplineBART estimated BDI reductions of 4.9 and 5.1 points in T1 and 4.7 and 4.9 points in T2, versus 2.2 points in the unadjusted trial-only estimate.Differences across methods arose primarily from the treated potential outcome, while estimates under control were similar.Source: §5 · pp. 32–33
• The authors cannot claim the models avoid misspecification; extrapolation assumes the propensity-score relationship with treatment effects remains similar beyond the trial range and near its boundaries.They recommend sensitivity analyses that allow different propensity-score functional forms for out-of-sample units.Source: §5 · p. 33
• PSPI and Bayesian Causal Forests share an interaction-based outcome-model form but address different problems: trial-to-target transport versus observational causal inference.Future work may adapt PSPI to ordinal or survival outcomes, multi-arm or continuous treatments, and settings requiring adjustment for both treatment confounding and population differences.Source: §5 · pp. 33–34

• The supplement contains derivations and model details, additional simulation materials, and data-application eligibility criteria and sensitivity analyses.Source: Supplementary Material · p. 34

A.1. Proof of Identifiability.

• Under A1–A5, target potential outcome means are identified by averaging trial conditional outcome means over the target population’s covariate distribution.The EHR cohort supplies that target covariate distribution; A4 equates conditional potential outcome means across trial participation status.Source: §A.1 · p. 40
• In the non-nested design, the EHR cohort consists of non-trial individuals, and A4 equates their conditional potential outcome means to those among trial participants.Positivity assumptions ensure the required trial participants and treatment arms are represented over the target covariate support.Source: §A.1 · pp. 40–41
• The target average treatment effect is identified as the difference between the target potential outcome means under intervention and control.Source: §A.1 · pp. 40–41

A.2. Prior Specifications for BART.

• The BART components use independent regularization priors for tree structures and leaf parameters, with default hyperparameters α = 0.95, β = 2, and k = 2.Splitting variables and split points are selected uniformly, while split probability declines with tree depth.Source: §A.2 · p. 41

A.3. Gibbs Samplers for SplineBART and DSplineBART.

• Both Gibbs samplers update BART components by backfitting, with tree parameters conditioned on other trees and current residuals.SplineBART updates a treatment-effect spline, while DSplineBART updates separate prognostic and treatment-effect splines; both generate target potential-outcome predictions.Source: §A.3 · p. 41

A.4. Properties of the Inverse Gumbel Transformation.

• The inverse Gumbel transformation behaves like the logit at high participation scores but expands the low-score range more gradually.Compared with logit, it allocates more transformed range to the right end, where most trial units lie, while keeping the out-of-sample range narrower for more stable extrapolation.Source: §A.4 · p. 43

B.1. Comparison Methods.

• The estimated sampling propensity score is fit with a probit BART model on all covariates among observed individuals and shared across methods that require it.The appendix defines trial-only, inverse probability of participation weighting, and augmented weighting estimators for the comparisons.Source: §B.1 · p. 44
• The comparison set includes BART, propensity-score-augmented BART, interaction BART variants, and FullBART, alongside weighting estimators.FullBART replaces the treatment-effect spline with a separate BART component; the listed interaction variants differ in whether the propensity score enters their prognostic and treatment-effect components.Source: §B.1 · p. 44

B.2. Additional Simulation Details and Results.

• The appendix also presents additional simulation designs and results.The supplied passage does not specify their outcomes.Source: §B.2 · p. 44

B.2.1. Transportability.

• SS5 evaluates a linear setting with an EHR cohort of N = 500 and randomized treatment probabilities of 0.3 or 0.7.The trial sample size is n = 150 in both listed SS5 settings.Source: §B.2.1 · pp. 44, 47
• Web Tables 1–3 report bias, RMSE, coverage, and average interval width for SS1–SS5 across the specified cohort sizes and treatment assignments.The scenarios include linear and nonlinear settings, with and without covariate shift.Source: §B.2.1 · pp. 44–47

B.2.2. Generalizability.

• The generalizability design samples an EHR cohort from a superpopulation, then draws the trial sample from within that cohort.It reuses the transportability scenarios and estimands, with the same potential-outcome and trial-participation mechanisms; no separate target population is formed.Source: §B.2.2 · p. 47

B.3. Comparison of Transformations of

• The transformation comparison evaluates Identity, Logit, Cloglog, and InvGumbel for DSplineBART using the EHR-cohort MATE.Transportability results are summarized for linear and nonlinear scenarios; comparable generalizability results are omitted.Source: §B.3 · pp. 51–52

C.1. Inclusion and Exclusion Criteria.

• The All of Us cohorts approximate trial eligibility using EHR criteria, substituting a depression diagnosis for unavailable Beck Depression Inventory screening results.Language and study-specific retention criteria that could not be reliably applied were not imposed; the two target cohorts also differ in the timing criterion for depression diagnosis after acute coronary syndrome.Source: §C.1 · p. 53

C.2. Additional Sensitivity Analysis.

• Web Figure 2 examines SplineBART’s MATE and PATEs and DSplineBART’s PATE across the same grid of bias-function prior means and variances, on the 6-month BDI-score-change scale.The sensitivity analyses cover target samples T1 and T2.Source: §C.2 · pp. 53–54