PaperlayerView exact revision on arXiv

Source-linked AI summary

Transfer Calibrated Prediction Powered Inference

Aditya T. Vadlamani, Jae Ho Chang, Srinivasan Parthasarathy, Subhadeep Paul

arXiv:2609.34156v1cs.LG

TL;DR

With scarce labeled target data, prediction-powered inference can lose efficiency when its predictor comes from a different source domain. TC-PPI adapts that predictor using target gold samples and cross-fitting; joint tuning yields a no-harm guarantee relative to classical inference and source PPI++ under the stated conditions.

  • Problem

    Prediction-powered inference’s efficiency can decline when a predictor trained on a distinct source domain has high residual variance on the target population.

    Source: §1 · pp. 1–2

  • Method

    TC-PPI learns a target-domain correction to a source predictor using gold data, with cross-fitting and joint power tuning.

    Source: §1 · p. 2, §2 · p. 3

  • Results

    Joint TC–Cross-PPI++ with optimal tuning is at least as effective as classical methods, and its joint class includes source PPI++ and TC–Cross-PPI++.

    Source: §1 · p. 2, §3 · p. 6

  • Takeaways & Limitations

    The joint tuning safeguard does not require a correctly specified transfer learner when foldwise estimates converge to a stable square-integrable function.

    Source: §3 · p. 6

    The framework assumes a pretrained source predictor and does not observe individual-level data from its source corpus.

    Source: §2 · p. 2

Abstract

from arXiv · show

Prediction-powered inference (PPI) and its power-tuned extension (PPI++) improve confidence intervals by combining a small gold-standard labeled sample with a large AI model's predictions. Its efficiency gain relies on low residual variance, which may not hold if the predictor is pre-trained on a different source domain. We propose Transfer Calibrated Prediction-Powered Inference (TC-PPI), adapting the source-domain predictor to the target domain using gold-standard samples through cross-fitting. This approach supports various adaptation methods, such as sparse linear calibration, LoRA, and fine-tuning. Our jointly tuned cross-fit estimator, Joint-TC-Cross-PPI++, maintains unbiasedness and is simultaneously at least as efficient as classical inference, PPI, and PPI++, thereby protecting against negative transfer. We provide high-dimensional MSE bounds for calibration and show empirical improvements over baseline methods across various real-world applications.

1 INTRODUCTION

Prediction-powered inference combines scarce labeled data with predictions on a larger unlabeled sample, but a source-domain predictor can lose efficiency when its residual variance is high on the target domain. TC-PPI adapts that predictor using target-domain gold data, and joint tuning with cross-fitting is designed to guard against ineffective transfer.

Motivation

  • A source-domain predictor may have high target residual variance, reducing the efficiency benefit of prediction-powered inference.The predictor is trained on data distinct from the target population.Source: §1 · pp. 1–2

Key Differentiators

  • TC-PPI uses target-domain data to calibrate a source predictor, with sparse linear calibration, LoRA, and fine-tuning among the supported approaches.The framework also covers mean, OLS, GLM, and Bradley–Terry coefficient estimation.Source: §1 · p. 2

Core Methodology

  • Joint TC–Cross-PPI++ combines joint power tuning with cross-fitting, and its optimal tuning is at least as effective as classical methods.Cross-fitting uses all gold observations for rectification.Source: §1 · p. 2

Key Differentiators

  • Cross-fitting addresses bias from using the same gold data for calibration and rectification, while avoiding the performance loss of naive sample splitting.The approach differs from transfer methods that require individual-level source-domain data.Source: §1 · p. 2

2 TRANSFER CALIBRATION AND PPI/PPI++ RECTIFICATION

TC-PPI adapts a source predictor to the target population by learning a correction from gold-standard data, then uses cross-fitting for rectification. Its theory covers unbiasedness and asymptotic behavior, while sparse linear calibration provides explicit high-dimensional rates.

Transfer Calibration and PPI/PPI++ Rectification

  • The transfer correction can be learned with sparse linear regression on residuals, embeddings, LoRA, adapters, or another low-complexity procedure.Validity results do not require the correction to be linear or sparse; those assumptions are used for explicit rates.Source: §2 · p. 3

Calibration and Rectification

  • Cross-fitting trains each fold’s correction on other folds and uses held-out observations for rectification, recovering all labels without the naive split’s sample waste.Each held-out observation is excluded from fitting its own rectification predictor.Source: §2 · p. 3

PPI under Domain Shift

  • Under the stated fold, moment, and calibration-consistency conditions, TC–Cross-PPI is unbiased and has asymptotic variance Vart(ε) + ρVart{ft(X)}.The unbiasedness proposition conditions on the source data; the asymptotic result assumes an = op(1).Source: §2 · pp. 3–4

Sparse linear calibration: high-dimensional lasso rates

  • For sparse linear calibration, the lasso specializes the cross-fitted theory to high-dimensional settings with p ≫ n, subject to its stated conditions.For fixed K, the condition is equivalent to s log p/n → 0.Source: §2 · p. 4

PPI under Domain Shift

  • When N ≫ n, consistent transfer calibration removes the source-to-target drift variance, making TC-Cross-PPI more efficient than vanilla PPI.The comparison uses all n gold observations for rectification.Source: §2 · p. 4

Transfer Calibrated Extension for PPI++

  • The PPI++ extension tunes the calibrated predictor’s influence, and its theorem permits foldwise predictors to converge to a stable function rather than the true target regression function.The oracle tuning parameter depends on unknown target-domain moments.Source: §2 · pp. 4–5

3 JOINT POWER TUNING: NEGATIVE TRANSFER PROTECTION

Joint power tuning retains both the source predictor and transfer correction as options, allowing the estimator to avoid relying on an ineffective correction. Its no-harm result compares the optimized joint class with source PPI++ and classical inference under stated conditions.

Joint Power Tuning: Negative Transfer Protection

  • For any fixed λ, the joint estimator is unbiased, and optimizing its vector weights can ignore, use, or combine the transfer correction with the source predictor.Some combinations cannot be represented by vanilla PPI++ or TC–Cross-PPI++ alone.Source: §3 · p. 5
  • The joint tuning class nests source PPI++, TC–Cross-PPI++, and classical inference, so optimizing over it cannot have greater asymptotic variance than those options.The no-harm guarantee does not require the transfer learner to be correctly specified, provided its foldwise estimates converge to a stable square-integrable function.Source: §3 · p. 6
  • When the transfer correction is consistent for the oracle correction, the joint estimator is guaranteed no less efficient than source PPI++ and can be more efficient than classical inference.The classical-inference efficiency comparison requires consistency for the oracle correction.Source: §3 · p. 6
  • A plug-in weight estimator avoids bootstrap-based standard errors, with its estimated weights adding O(n^-1 + rn + τ^2) inflation to the √n-scale variance.The source describes a small ridge regularization when needed.Source: §3 · p. 6

4 NONASYMPTOTIC RATES AND MINIMAX OPTIMALITY

The section bounds finite-sample MSE for cross-fitted estimators under sparse linear calibration and explains how power tuning changes the calibration cost. It then establishes minimax optimality for TC–Cross-PPI++ on the stated Gaussian sparse-transfer class.

4 NONASYMPTOTIC RATES AND MINIMAX OPTIMALITY

  • Theorem 4 gives a nonasymptotic MSE bound for TC-Cross PPI under fixed balanced folds and finite second moments.For the power-tuned estimator, the section considers λ0 = N/(n + N).Source: §4 · pp. 6–7
  • With a consistently learned target predictor, λ0 = N/(n + N) changes the oracle signal term from Vt/N to Vt/(n + N) and removes the O(√rn/N) term.The resulting calibration cost in bound 9 is proportional to rn.Source: §4 · p. 7

Appendix A.3, and let ψn,p,s(Vδ, σ2

  • On the stated Gaussian sparse-transfer class, the upper and lower bounds differ by a constant factor and, when s ≪ p, a logarithmic factor, making TC–Cross-PPI++ at λ0 = N/(n + N) minimax optimal.The lower bound is for an unknown sparse transfer direction.Source: Appendix A.3 · p. 7

5 EMPIRICAL EXPERIMENTS AND RESULTS

Across the evaluated applications, transfer-calibrated cross-fitted methods improve on classical inference in several shifted settings, with Joint TC–Cross-PPI++ best on Bikeshare. The appendix bounds the calibration-estimation cost, while the conclusion presents cross-fitting and joint power tuning as safeguards against bias and negative transfer.

5 EMPIRICAL EXPERIMENTS AND RESULTS

  • At n = 200, Census methods reach relative MSE of 0.58–0.61 and ESS gains of 63–73%; TC–Cross-PPI++ and Joint TC–Cross-PPI++ tie for best.Sparse calibration changes residual variance by only a few percent, so most of the gain comes from the source predictor.Source: §5 · p. 8
  • At n = 100, Bikeshare relative MSE ranges from 0.21 for PPI++ to 0.14 for cross-fitted methods, with ESS gains of 370–620%; Joint TC–Cross-PPI++ is best.Transfer calibration roughly halves residual variance when source errors vary with calendar and weather covariates.Source: §5 · p. 8
  • On AfriSpeech-200, every method improves on classical inference at every budget, while TC–Cross-PPI ESS gains range from 11.3% at n = 100 to 15.0% at n = 1000.Coverage stays within 0.91–0.98 across budgets and methods.Source: §5 · p. 8
  • For PovertyMap at n ≥200, cross-fitted relative MSE is 0.23–0.27 versus 0.31–0.34 for PPI/PPI++; for Camelyon17 it is 0.53–0.58 versus 0.76–1.04.Coverage across the two WILDS benchmarks stays between 0.88 and 0.99.Source: §5 · p. 8
  • On ProteinGym, cross-fitted methods reach MSE 0.61, or ESS gains up to 65%, as n grows; Joint TC–Cross-PPI++ performs best consistently.PPI++ has relative MSE of 0.70–0.74 across n = 200–500.Source: §5 · p. 9
  • For Chatbot Arena, PPI++ and Joint TC–Cross-PPI++ match or beat classical inference from n ≥500, reaching relative MSE 0.90 (ESS +11%) at n = 2000; AutoEval power-tuned methods help reliably.On AutoEval MT-Bench, PPI++ and Joint TC–Cross-PPI++ have relative MSE 0.910 and 0.870 at n = 100, and 0.932 and 0.933 at n = 500.Source: §5 · p. 9

6 CONCLUDING REMARKS

  • The conclusion presents cross-fitting and joint power tuning as ways to mitigate bias from using gold data for both calibration and rectification and to prevent negative transfer.The authors recommend adapting pretrained models to target data, with asymptotically negligible downside risk.Source: §6 · p. 9

A.1 NONASYMPTOTIC MSE UPPER BOUNDS

  • When the unlabeled sample is sufficiently large, transfer-calibration estimation contributes MSE of order s log p/n^2.This is the calibration-estimation contribution to the bound, not the full oracle PPI variance.Source: §A.1 · pp. 14–15

A.2 PROOF OF THEOREM 4

This section develops theoretical guarantees by bounding conditional MSE through a decomposition of the transfer estimator and its cross-fitting interactions, and by establishing a minimax lower bound via sparse regression. It also states integrability conditions for additional theoretical results.

A.2 PROOF OF THEOREM 4.

  • The TC-Cross-PPI error is decomposed into oracle and cross-fitting remainder terms, whose conditional covariance is controlled using fold-wise conditioning and independence.The proof uses that gold and unlabeled samples are independent and that held-out observations are independent of the fold-specific fitted function.Source: §A.2 · pp. 15–16

A.2 PROOF OF THEOREM 4

  • Conditional unbiasedness and covariance control yield the stated MSE upper bounds for TC-Cross-PPI and TC-Cross-PPI++.For TC-Cross-PPI++, the proof uses the fixed power parameter λ0 = N/(n+N) to make the calibration remainder orthogonal to the oracle PPI++ component.Source: §A.2 · pp. 17–18

A.3 MINIMAX LOWER BOUND.

  • The centered regression experiment learns the unknown transfer slope δ with effective sample size n − 1, linking sparse-regression difficulty to target-mean difficulty.Lemma 1 is introduced as a Bayes decomposition for the full labeled and unlabeled experiment.Source: §A.3 · p. 20
  • Under Theorem 5’s conditions, a finite prior on an exact Vδ shell gives a sparse-regression Bayes lower bound of minimax order.The proof constructs the prior for the centered Gaussian regression experiment.Source: §A.3 · p. 22
  • A nearest-parameter test and uniform prior, with average KL divergence bounded by a fixed fraction of log |V|, yield risk at least c4 r* and establish equation 37.Source: §A.3 · p. 23

B ADDITIONAL THEORETICAL DETAILS

  • Conditionally on the source-trained predictor, Y, ft(X), and ∆*(X) are square-integrable, and the conditional second moment of bft(X) is almost surely finite.Source: B ADDITIONAL THEORETICAL DETAILS · p. 23

B.1 SPLIT TC-PPI

This appendix develops split TC-PPI and its extensions to mean, low-dimensional parameter, and general M-estimation, including cross-fitting with full-sample scaling. It establishes inference properties and variance guarantees, describes uncertainty estimation, and gives conditions for valid inference, including when estimating joint tuning weights.

B.1 SPLIT TC-PPI

  • Split TC-PPI calibrates the source predictor on one target gold-data split and uses an independent split for PPI rectification.The estimator replaces the original source predictor with the calibrated predictor in the PPI mean estimator.Source: §B.1 · pp. 23–24
  • With N ≫ n2, split TC-PPI’s asymptotic variance simplifies to Vart(ε); calibration can reduce variance versus vanilla PPI when its correction error vanishes.Under sparse linear calibration, the stated variance reduction is (δ∗)⊤Σtδ∗ with high probability when the rectification sample sizes match; the correction rate is Op(s log p/n1).Source: §B.1 · pp. 24–25

B.2 VARIANCE INFLATION WITH ESTIMATED TUNING VECTOR

  • Estimating the joint tuning vector adds finite-sample cost relative to oracle tuning, and the section characterizes its excess risk and sparse-calibration limiting variance.The sparse linear calibration analysis assumes fixed K.Source: §B.2 · pp. 25–26

B.3 VARIANCE ESTIMATION AND CONFIDENCE INTERVALS

  • The proposed variance estimators avoid the bootstrap strategy used by Zrnic & Cand`es (2024), are consistent under the stated theorem conditions, and yield Wald intervals with asymptotic coverage 1 − α.The justification uses out-of-fold predictions and an asymptotic linear representation whose leading terms are empirical averages over independent gold and unlabeled samples.Source: §B.3 · p. 26, §B.3.1 · pp. 27–28

C.1 GENERALIZED LINEAR MODELS

  • For generalized linear models, the section defines a target-calibrated prediction-powered estimating equation and states conditional asymptotic linearity under regularity conditions.The listed conditions include fixed dimension, local smoothness, a nonsingular matrix, and finite (2 + η) score moments.Source: §C.1 · pp. 29–30

C.2 GENERAL CONVEX M ESTIMATION

  • For general low-dimensional convex M-estimation, TC-PPI calibrates on one target split and uses the calibrated predictor in an estimating equation on an independent split and unlabeled target data.The target parameter is defined by a population moment condition associated with a convex loss.Source: §C.2 · p. 30, §C.2.1 · p. 30

C.2.1 TARGET ESTIMAND AND RECTIFIED ESTIMATING EQUATION

  • The population rectified moment map equals zero at the target parameter for every measurable predictor for which the expectations exist.This follows because labeled and unlabeled covariates share the target covariate law.Source: §C.2.1 · p. 30

C.3 ASSUMPTIONS

  • The general M-estimation setup assumes local identification and smoothness around the target parameter, together with a nonsingular Jacobian.The parameter is the unique solution of the moment equation in a neighborhood of the target value.Source: §C.3 · p. 31
  • Assumption 5 is a high-level calibration condition that follows for sparse linear discrepancy with lasso under a stated rate and s log p/n1 → 0; smooth GLMs can often use Lipschitz continuity.The supplied passages identify these as ways to verify calibration consistency in score norm.Source: §C.3 · p. 31

C.4 RESULTS FOR SPLIT TC PPI M ESTIMATOR

  • Under Assumption 4, the split TC-PPI estimator is exactly conditionally unbiased for every fixed θ.Source: §C.4 · p. 31
  • Under Assumptions 2–5, split TC-PPI is consistent as n2 and N tend to infinity.The theorem states consistency for the split TC-PPI M-estimator.Source: §C.4 · p. 31

C.5 ASYMPTOTIC LINEARITY AND ASYMPTOTIC NORMALITY

  • Under Assumptions 2–5 and n2/N → ρ ∈ [0, ∞), the split estimator is asymptotically linear and normal.Source: §C.5 · p. 32

C.6 CROSS-FITTED EXTENSION FOR GENERAL M ESTIMATION

  • Cross-fitted TC-PPI extends to general M-estimation and replaces split-sample scale n2 with full-sample scale n.The result covers TC–Cross-PPI++ with λ ≠ 1; a consistent data-dependent bλ can replace bλ without changing the first-order limit under the stated stochastic equicontinuity condition.Source: §C.6 · p. 32

C.7 PROOFS OF RESULTS FROM THIS SECTION

The proofs establish asymptotic representations by replacing estimated predictors with oracle predictors under conditional mean cancellation and variance control. They then use central limit theorems for independent labeled and unlabeled averages, while the split estimator’s conditional variance decomposes into the sum of its component variances.

C.7 PROOFS OF RESULTS FROM THIS SECTION

  • The cross-fitted representation’s labeled and unlabeled oracle averages satisfy a multivariate central limit theorem, yielding the stated covariance.The covariance includes a labeled contribution and a separately scaled unlabeled contribution.Source: §C.7 · p. 33
  • Conditional mean cancellation and variance control make replacing the estimated predictor with the oracle predictor negligible at the proof’s scaled order.The conditional variance is o(1) after multiplication by n2 when n2/N = O(1), allowing Chebyshev’s inequality to establish the replacement.Source: §C.7 · p. 34
  • Foldwise nuisance replacement yields the full-sample asymptotic linear representation when the fold count is fixed and n/N = O(1).Conditional on each training complement, the held-out and unlabeled averages have the same population mean; the foldwise L2 condition controls their centered differences.Source: §C.7 · p. 34

D.1 SPLIT-TC-PPI PROOFS

  • For split TC-PPI, the calibration and rectification samples are conditionally independent, so the estimator’s conditional variance is the sum of their variances.As the discrepancy-estimation error vanishes and n2/N approaches ρ1, the variance converges to ρ1Vart{ft(X)} + Vart(ε).Source: §D.1 · pp. 34–35

D.2 LASSO RESULTS

This section presents deterministic and high-dimensional guarantees for sparse target-domain calibration under stated sparsity, compatibility, and score-control conditions, along with cross-fitting remainder and no-harm arguments for related TC-PPI estimators. It also gives proof steps for equations 54 and 55, using a resolvent identity and an exact square-completion identity for the excess population variance criterion.

D.2 LASSO RESULTS

  • Proposition 6 gives a deterministic lasso oracle bound when the calibration error vector is sparse and the stated events hold.The section also notes that score control can hold with λ ≍ (log p)/n1 under sub-Gaussian covariates and uniformly sub-exponential Xjε, without requiring marginal sub-Gaussian noise.Source: §D.2 · pp. 35–36

D.3 TC-CROSS-PPI PROOFS

  • If maximum foldwise prediction error is Op(rn) for rn → 0, the TC-Cross-PPI proof obtains a remainder of √n Rn = Op(√rn).The LASSO corollary states that fixed-fold calibration yields an = op(1) when s log p/n → 0, allowing Theorem 1 to apply.Source: §D.3 · p. 40

D.4 PROOFS FOR TC-CROSS-PPI++ AND JOINT TC-CROSS-PPI++

  • The joint parameter class contains classical inference and the source-PPI++ subfamily, so minimizing over it cannot have larger first-order variance than either alternative.This comparison does not require ∆† = ∆∗.Source: §D.4 · p. 43
  • The proof combines equations 82–83 with a resolvent identity to establish equation 54.Source: §D.4 · p. 46
  • For equation 55, the proof completes the square for the excess population variance criterion and takes λ = bλ using bounded eigenvalues of the fixed-dimensional Σ†.The preceding definition sets αn := n−1/2 + √rn + τ, with rn = o(1) and τ = o(1).Source: §D.4 · p. 46

E EXPERIMENTAL DETAILS

The experiments sample gold labels from each target population and use the remaining observations as an unlabeled pool. Applications span tabular, speech, vision, satellite, protein, and language-model settings, with calibration and evaluation details varying by task.

E EXPERIMENTAL DETAILS

  • Each repetition samples n labeled rows uniformly and uses the remaining N − n rows as the prediction-rich unlabeled pool.Table 2 records population size, gold-label range, repetitions, and cross-fitting folds, with LoRA counts shown where they differ.Source: E EXPERIMENTAL DETAILS · p. 46, Table 2 · p. 47

E.1 TABULAR

  • The tabular applications use 2019 California ACS data for three Census targets and a temporal-transfer task predicting 2012 Bikeshare rentals from a source trained on 2011 data.The Census targets are mean income, an income-regression coefficient, and private health-insurance coverage.Source: §E.1 · p. 47

E.2 SPEECH

  • The speech application evaluates word-error-rate transfer on Yoruba-accented AfriSpeech-200 audio, pooling its train, dev, and test splits into one target population.Its source is wav2vec2-base-960h trained on American-English speech and applied without fine-tuning.Source: §E.2 · pp. 47–48

E.3.1 CAMELYON17; E.3.3 MODEL SELECTION

  • Camelyon17 pools the two held-out hospitals as the target, and its calibration features include source probability, slide coordinates, and target-hospital slide indicators.Source checkpoint selection uses in-distribution validation rather than OOD labels; the reported OOD-test accuracy is 0.673 versus WILDS’s ERM baseline of 70.3%.Source: §E.3.1 · p. 48, §E.3.3 · p. 49

E.3.2 POVERTYMAP

  • PovertyMap pools ten out-of-distribution countries into one target population, doubling the target population and the number of countries available for the correction to key on.Calibration uses the source prediction, urban/rural indicator, two nightlight summaries, and country indicators; the target-derived binarized wealth column is excluded.Source: §E.3.2 · pp. 48–49

E.4 PROTEIN LANGUAGE MODELS

  • ProteinGym covers a beta-lactamase fitness-effect assay, protein-model ranking, and an annotator-quality sweep.The sparse-calibration features for the BLAT assay combine mutation identity with the ESM-2 score; LoRA uses a separately loaded ESM-2 8M encoder and VESPA as an auxiliary input.Source: §E.4 · p. 49

E.5 LLM ARENAS

  • The arena applications pair human preferences or ratings with precomputed LLM-judge scores, and their LoRA comparison uses distilgpt2 for both arenas.For the sparse-versus-LoRA comparison, the sparse arm receives distilgpt2’s frozen pooled hidden state over the same text, isolating the calibration mechanism.Source: §E.5 · pp. 49–50

E.6 HYPERPARAMETERS AND IMPLEMENTATION; F ADDITIONAL RESULTS

  • Sparse calibration uses cross-validated or fixed penalties depending on task, while all cross-fitted estimators use Wald intervals and the additional figures report ESS, interval width, and coverage.The two AutoEval replications use 90% nominal coverage; the other panels use 95%.Source: §E.6 · p. 50, F ADDITIONAL RESULTS · p. 51, Figure 4 · p. 53

F.1 RESULTS WITH ADDITIONAL DATASETS

Results vary across applications: cross-fitted transfer calibration improves on PPI++ on ProteinGym and Census OLS, while joint tuning is more conservative on Census OLS.

Census OLS

  • On Census OLS, TC–Cross-PPI++ improves on PPI++ relative MSE at n = 200 and n = 500, while Joint TC–Cross-PPI++ tracks PPI++ more closely.PPI++ has relative MSE 0.717 and 0.747; TC–Cross-PPI++ reaches 0.682 and 0.732, while Joint TC–Cross-PPI++ reaches 0.713 and 0.735, respectively.Source: §F.1 · p. 54

ProteinGym

  • On ProteinGym, cross-fitted methods reach relative MSE 0.65–0.77 versus PPI++ at 0.79–0.82, with ESS gains up to ∼53%.TC–Cross-PPI++ also reaches mean rank correlation 1.000 at n = 100, versus 0.917 for PPI++ and 0.250 for vanilla PPI.Source: §F.1 · p. 54

F.2 RESULTS LOOKING AT ANNOTATOR QUALITY

Across annotator-quality analyses, transfer-calibrated methods outperform PPI++ on reported error or efficiency measures, while residual-variance reductions depend on predictable source errors and available gold labels.

F.2 RESULTS LOOKING AT ANNOTATOR QUALITY

  • On ProteinGym, as annotator correlation increases from 0.396 to 0.626, PPI++ MSE falls from 2.89 × 10−4 to 2.18 × 10−4, while TC–Cross-PPI++ falls from 2.33 × 10−4 to 1.76 × 10−4.TC–Cross-PPI++ coverage is 0.976, 0.972, and 0.980 across the three annotators.Source: §F.2 · pp. 54–55
  • On AutoEval-ProteinGym, TC–Cross-PPI++ ESS gain rises from +36.8% to +70.2% across annotator-fitness correlations of −0.05 to 0.49, ahead of PPI++ at every annotator.Coverage remains 0.88–0.92 across methods and annotators, against a nominal 90%.Source: §F.2 · p. 55, Figure 6 · p. 55

F.3 VARIANCE RESULTS

  • Calibration roughly halves residual variance on Capital Bikeshare, but changes little on Census Income when source errors are not predictable from calibration features.With very few labels, correction can instead increase residual variance, as on PovertyMap at n = 25; joint power tuning can down-weight it while retaining the source predictor.Source: §F.3 · p. 55

F.4 RESULTS WITH LORA

LoRA calibration results are application-dependent: it is strongest in the SPG1 comparison, while sparse calibration is slightly better on BLAT and both are near classical inference on MT-Bench Arena.

F.4 RESULTS WITH LORA

  • On SPG1 at n = 500, LoRA calibration reaches relative MSE 0.223 versus 0.423 for sparse calibration, an ESS gain of 348%.LoRA is noisier at small n but eventually overtakes sparse calibration in this comparison.Source: §F.4 · p. 56, Table 5 · p. 56, Figure 7 · p. 57
  • On BLAT at n = 1000, sparse TC–Cross-PPI++ has relative MSE 0.854 versus 0.904 for LoRA calibration.Source: §F.4 · p. 56, Table 5 · p. 56, Figure 7 · p. 57
  • On AutoEval MT-Bench Arena, neither calibration style meaningfully improves on classical inference; sparse stays within 1–4%, while LoRA moves from 1.300 at n = 100 to 1.010 at n = 750.The authors interpret this pattern as evidence that LoRA can need more gold labels to stabilize, not that either approach dominates.Source: §F.4 · p. 56, Table 5 · p. 56

F.5 RESULTS WITH OTHER PPI VARIANTS

Comparisons with methods trained or stratified differently show that transfer-calibrated approaches can outperform from-scratch Cross-PPI and provide a distinct alternative to StratPPI.

F.5 RESULTS WITH OTHER PPI VARIANTS

  • On Capital Bikeshare, Cross-PPI relative MSE is 0.75–0.91, versus 0.16–0.21 for PPI and 0.07–0.14 for TC–Cross-PPI.Cross-PPI coverage is 0.91–0.98.Source: §F.5 · p. 57, Figure 8 · p. 58
  • On Census Income, Cross-PPI has relative MSE 1.85 at n = 100 and reaches only 0.87 at n = 2000, while PPI, PPI++, and transfer-calibrated methods stay between 0.57 and 0.67.The comparison trains Cross-PPI from scratch with the same model class as the source.Source: §F.5 · p. 57, Figure 8 · p. 58
  • StratPPI median-MSE ratios are 0.82–1.01 on AutoEval and 0.89–1.30 on Chatbot Arena at n ≥500, versus PPI++ at 0.70–0.96 and 0.72–0.99, respectively.StratPPI coverage is 0.83–0.93 on AutoEval and 0.92–0.97 on Chatbot Arena; its strata use the judge’s own output.Source: §F.5 · p. 58, Figure 9 · p. 59

Related papers

Loading 2609.34156v1…