PaperlayerView exact revision on arXiv

Source-linked AI summary

Shared Canonical Component Removal Beyond Expert Views

Jose Roberto Tello Ayala, Akl C. Fahed, Roy Perlis, Finale Doshi-Velez

arXiv:2610.13819v1cs.LG

TL;DR

Raw-input models may use predictive signal beyond an expert view, leaving unclear what that signal is. SCCR selectively removes strongly shared raw–expert directions and audits the remaining signal; the authors report retaining black-box performance while reducing redundancy, with high-stakes use requiring validation and oversight.

  • Problem

    It is unclear what predictive signal raw-input models use beyond expert views, limiting assessment of whether those views are sufficient.

    Source: §1 · p. 1

  • Method

    SCCR selectively removes strongly shared linear directions between learned raw and expert embeddings, then uses the residual raw signal with the expert view for prediction.

    Source: §1 · p. 2

  • Results

    The authors report that the dual-branch model can retain black-box performance while reducing redundancy with the expert view.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    The audit can help determine whether an expert view is sufficient and surface complementary signal or potentially spurious shortcuts for review.

    Source: IMPACT STATEMENT · p. 13

    SCCR targets shared linear structure and does not guarantee that all unwanted information has been removed; high-stakes deployment requires domain oversight and task-specific validation.

    Source: IMPACT STATEMENT · p. 13

Abstract

from arXiv · show

In clinical practice, experts often begin by converting raw inputs into a set of expert features before making a decision. While black-box models trained on raw input can outperform models trained on expert views, it remains unclear what predictive signal the raw model exploits beyond what the expert view captures. We introduce an audit framework to quantify this gap using Shared Canonical Component Removal (SCCR). Unlike standard residualization or concept erasure, SCCR selectively removes only those directions strongly shared between learned raw and expert embeddings. Evaluations on synthetic data, coronary artery disease (CAD) detection, antidepressant treatment dropout prediction, and skin lesion classification using SkinCon annotations show that SCCR identifies the ways in which expert views are insufficient and recovers residual predictive signal while minimizing redundancy with the expert view.

1 INTRODUCTION

Expert views encode domain-informed features, but raw-input models may use predictive cues those views omit. SCCR audits the residual signal after selectively removing strongly shared raw–expert directions.

1 INTRODUCTION

  • Raw models may exploit cues omitted from expert views, including angiogram grayscale intensity or broader EHR signals relevant to antidepressant dropout.The introduction frames whether the raw model uses predictive signal beyond what the expert view captures as a key audit question.Source: §1 · p. 1
  • SCCR removes raw–expert linear directions whose canonical correlation meets threshold τ, then combines remaining raw signal with the expert view for prediction.“Beyond the expert view” means signal remaining after this selective linear removal, not independence from the expert view.Source: §1 · p. 2
  • The framework tests expert-view sufficiency and measures complementary predictive signal across synthetic data and clinical tasks involving CAD, antidepressant dropout, and skin lesions.The expert view serves as an audit anchor, including in the angiogram-and-vessel-morphology example.Source: §1 · p. 2
  • The authors report that the dual-branch model can retain black-box performance while reducing redundancy with the expert view.The introduction also identifies frozen calibration for deterministic inference as a contribution.Source: §1 · p. 2

2 RELATED WORK

SCCR is framed as an audit method, distinct from approaches that build concept bottlenecks, erase concepts, or learn shared representations across peer views. It uses the expert view asymmetrically to examine residual predictive signal in raw representations.

Concept-based interpretation and bottleneck models

  • Unlike concept bottleneck methods, the framework does not complete a concept set or optimize an interpretable bottleneck; it measures raw predictive signal not captured by domain expertise.The expert view is used as an audit anchor for assessing its sufficiency.Source: §2 · p. 2

Concept erasure and information removal

  • Unlike concept erasure, SCCR does not seek complete erasure or unlearnability; thresholding suppresses strongly shared directions while preserving lower-correlation directions.This distinction matters when an expert view contains only a noisy proxy for a complementary predictive factor.Source: §2 · pp. 2–3

Multi-view learning and shared/private representations

  • Unlike multi-view learning that treats views as peers, this framework uses the expert view asymmetrically to test sufficiency and identify remaining raw predictive signal.The related work describes CCA and other methods that model shared, private, or distinctive view structure.Source: §2 · p. 3

3 METHOD

The method pairs raw and expert views, selectively removes strongly shared canonical directions from the raw embedding, and predicts using both the residual raw and expert branches. Training estimates the operator from minibatches, while calibration freezes it for deterministic inference.

3.1 Setting

  • The paired-view setting treats the expert view as relevant but not necessarily complete, and asks whether raw input adds predictive signal.The architecture encodes the raw input and expert view in separate branches, then predicts from the residual raw representation and centered expert embedding.Source: §3.1 · p. 3

3.2 Shared canonical component removal

  • SCCR removes raw-embedding directions with canonical correlations at or above τ, retaining lower-correlation directions as candidate complementary evidence.Unlike standard orthogonal projection, this thresholding does not remove every raw component with any linear overlap with the expert embedding.Source: §3.2 · p. 3

RAW INPUT

  • The operator uses OAS-regularized branch auto-covariances for whitening, then thresholds the resulting regularized canonical correlations.The raw and expert ridge matrices improve conditioning for Cholesky whitening; the cross-covariance is left unregularized.Source: EXPERT VIEW · p. 4

EXPERT VIEW

  • Separate transformed raw-residual and expert states feed prediction alongside an explicit interaction term, allowing the model to assess utility from both branches together.The interaction term is the elementwise product of the two branch states.Source: EXPERT VIEW · p. 4

3.3 End-to-end Training

  • During end-to-end training, minibatch embeddings estimate the SCCR operator, and gradients pass through covariance estimation, regularization, whitening, and SVD.The hard threshold is the only non-differentiable point; for fixed selected directions and distinct singular values, the map is smooth.Source: §3.3 · pp. 4–5

3.4 Calibration for Frozen Inference

  • A calibration pass over training embeddings freezes the operator so test-time residuals do not depend on test-batch composition.The resulting residual map is deterministic and consistent with the training distribution.Source: Figure 1 · p. 4, §3.4 · p. 5
  • Calibration uses full-training-set statistics, so τ should be chosen relative to calibration-scale correlations rather than minibatch spectra.The differing effective sample sizes produce different ridge levels during training and calibration.Source: §3.4 · p. 5

4 ANALYSIS

The analyses compare SCCR with other residualization methods, establish conditions for stable calibration, and test residual predictive value across synthetic and clinical tasks. Results vary by task: residual gains appear in skin-lesion and dropout prediction, but not clearly in the angiogram audits.

4.1 Pitfalls of Existing Residualization Techniques and Benefit of SCCR

  • For γ < τ < ρ, SCCR removes S and A while preserving both weakly shared factors F1 and F2, unlike methods that retain A or discard low-variance F1.D-CCA retains R²_A = .857; JIVE with q = 1 and AJIVE with initial ranks (4, 4) retain A completely.Source: §4.1 · p. 5

4.2 Theoretical Properties

  • Under the stated assumptions, small calibration perturbations preserve the number of removed canonical directions, and the calibrated operator converges as covariance blocks converge.The stability result assumes a strict threshold gap; once frozen, the residual map is deterministic at inference.Source: §4.2 · pp. 5–6

4.3 SCCR on Synthetic Data

  • In synthetic image experiments, quadratic recovery of A remains .818 ± .057, supporting reduced linear recovery.The task label depends on A and the interaction between F1 and F2; SCCR is trained on the label without supervision on the underlying factors.Source: §4.3 · pp. 6–7

4.4 Skin-lesion classification

  • On SkinCon, SCCR’s residual reduces diagnosis log loss by .073 and .051 across two seeds, exceeding the gains from end-to-end OLS.Restoring true Crust annotations reduces the gain more than shuffled annotations by 0.020 and 0.009, with the second-seed interval including zero; residual gains are negative on DDI.Source: §4.4 · p. 7, Table 2 · p. 8

4.5 Antidepressant Treatment Dropout from EHR Data

  • Across three held-out hospitals, SCCR at τ = .40 improves AUROC over the psychiatric view by .048, .031, and .023 while remaining close to raw-only prediction.The omitted psychotherapy code remains recoverable from the residual, but adding it to the expert predictor with the residual fixed leaves the predictive gain largely unchanged.Source: §4.5 · p. 8, Table 2 · p. 8

4.6 Coronary angiograms

  • At τ = .5, zeroing the mask reduces AUROC by .261; at τ = .7, zeroing the raw image reduces it by .387, but audits at τ = .1 and .3 show no clear residual gain.The branch controls indicate a shift from mask-based to image-based prediction, while single-frame inputs leave temporal mechanisms and clinical relevance unresolved.Source: §4.6 · pp. 8–9

5 DISCUSSION AND CONCLUSION

SCCR selectively removes strongly shared linear directions while retaining weakly shared directions that may help maintain prediction quality. Its explicit linear removal rule supports targeted audits, but it does not guarantee removal of arbitrary nonlinear dependence.

5 DISCUSSION AND CONCLUSION

  • SCCR removes strongly shared linear directions while retaining weakly shared directions that may be needed to maintain prediction quality.The paper describes it as distinct from baselines that fail to do both.Source: §5 · p. 9
  • The threshold τ can be selected as the minimum value at which the dual-branch model matches prediction quality from training on raw data.The paper states that SCCR requires one parameter, τ.Source: §5 · p. 9
  • SCCR does not guarantee removal of arbitrary nonlinear dependence, so future work should extend removal while preserving weakly shared predictive information.Linear probes support specific audit questions, while nonlinear probes reveal information that remains recoverable.Source: §5 · p. 9

IMPACT STATEMENT

The impact statement presents SCCR as an audit aid for high-stakes settings; the Gaussian example compares residualization operators against a specified suppression-and-retention objective.

IMPACT STATEMENT

  • In high-stakes settings, SCCR can help assess whether an expert view is sufficient and surface complementary signal, but it does not guarantee removal of all unwanted information.The statement calls for domain oversight, task-specific validation, and review of the residual branch before conclusions about fairness, safety, or causality.Source: IMPACT STATEMENT · p. 13

A.1 Construction and notation

  • The expert embedding contains exact S, a strongly correlated proxy Q for A, and weakly correlated proxies Pi for Fi.The blocks have equal total variance, tr(Srr) = tr(See) = 3.03.Source: §A.1 · p. 13

A.2 Optimal linear recovery

  • Small variance alone does not reduce optimal population recovery from a retained representation, although it changes the cost of reconstruction errors.The population outputs are fixed linear functions of Gaussian sources, so nonlinear squared-error prediction cannot improve these values; this does not extend to learned nonlinear embeddings or predicting F1F2.Source: §A.2 · p. 14

A.3 Feature-space projection

  • Because the expert support spans all four feature directions, feature-space projection removes the entire raw representation in this construction.The construction assumes the raw and expert feature coordinates have already been identified.Source: §A.3 · p. 14

A.4 Full OLS residualization

  • OLS makes its residual uncorrelated with the expert view but does not erase A behind a noisy proxy, and it only partially attenuates each Fi.Supplying the expert view alongside the residual allows reconstruction of the original raw embedding, so residual-only recovery values do not establish failure of combined prediction.Source: §A.4 · p. 15

A.5 Exact LEACE

  • With full-rank Sre, exact unregularized LEACE maps the raw embedding to zero, erasing even Fi because each has nonzero covariance with its proxy.These are not the regularized LEACE-style adaptation values used with learned embeddings.Source: §A.5 · p. 15

A.6 Decomposition-based CCA

  • D-CCA makes its distinctive scores uncorrelated with each other, not necessarily with the original expert score; at c = 1 its distinctive output is zero.Its substantial recovery of A is consistent with its objective, rather than a failure to satisfy the D-CCA constraint.Source: §A.6 · p. 16

A.7 JIVE with prescribed ranks

The Gaussian comparison shows that residualization outcomes depend on each method's objective and specified settings. Under the stated formulations, SCCR achieves the chosen suppression-and-retention pattern, while JIVE and AJIVE outputs depend on rank and selection choices.

A.7 JIVE with prescribed ranks

  • With JIVE joint rank q = 2, the reconstruction optimum discards low-variance F1 while retaining A; the outcome depends on the stated ranks, feature scales, and retained output.At q = 1, the raw individual output instead retains all three evaluated factors.Source: §A.7 · pp. 17–18

A.8 AJIVE with initial ranks (4, 4)

  • Under the stated boundary convention, AJIVE selects only S as joint, while A and both Fi remain outside the joint space; this result does not imply AJIVE always retains A.Different initial ranks or nonzero discarded variation can change AJIVE’s joint-rank decision.Source: §A.8 · p. 19

A.9 SCCR and matched top-2 CCA removal

  • For γ < τ < ρ, SCCR removes S and A while preserving both Fi, although the retained coordinates remain correlated with P1 and P2.On these fixed embeddings this is equivalent to removing the top two raw canonical directions.Source: §A.9 · p. 20

A.10 Summary and scope of the comparison

  • In this specified Gaussian comparison, feature projection and exact LEACE erase weak factors, OLS and D-CCA retain them alongside A, while SCCR removes the two strongly shared directions and preserves Fi.These are formulation-specific comparisons, not claims that all other ranks, preprocessing choices, or outputs fail.Source: §A.10 · pp. 20–21
  • The analytic example does not establish that learned encoders recover these factors, that linear suppression implies nonlinear independence, or that retaining Fi is clinically useful.Those questions require separate learning and recovery measurements.Source: §A.10 · p. 21

B.1 Stability of the Frozen SCCR Operator

This section formalizes SCCR’s threshold-selective removal and examines the stability of its frozen inference operator, as well as minibatch fitting, post-training calibration, and the scope of image-experiment comparisons. Under stated conditions, small calibration covariance errors preserve the selected rank and the operator converges as the calibration covariances converge.

B.1 Stability of the Frozen SCCR Operator

  • The proof uses symmetric square roots without changing the unwhitened SCCR operator, because Cholesky whitening differs by an orthogonal rotation.The selected left singular projector rotates by the same orthogonal matrix, leaving the unwhitened operator unchanged.Source: §B.1 · p. 22
  • Under population conditioning, a locally Lipschitz OAS ridge map, and a strict threshold gap, sufficiently small calibration error preserves the selected rank and bounds the empirical SCCR operator’s deviation from its population counterpart.As calibration covariance blocks converge to their population values, the theorem’s bound implies that the empirical operator converges to the population operator.Source: §B.1 · pp. 22, 23, 29
  • The small-error condition keeps the OAS ridge maps within their local Lipschitz neighborhoods, ensures empirical conditioning, and prevents singular values from crossing threshold τ.Empirical conditioning is derived from population conditioning rather than imposed as a separate assumption.Source: §B.1 · p. 23
  • Weyl’s singular-value inequality ensures that sufficiently small perturbations preserve the threshold-selected rank, while the selected and unselected singular clusters remain separated.The proof then uses singular-subspace perturbation to control the selected projector.Source: §B.1 · pp. 27–28
  • Under the stated small-error condition, the frozen inference operator converges to its population counterpart as calibration covariance blocks converge.Source: §B.1 · p. 30
  • SCCR removes the selected canonical components exactly, leaving residual cross-covariance singular values equal to the sub-threshold values in its regularized CCA metric.The result uses the convention that the maximum over an empty set is zero.Source: §B.1 · pp. 30–31

1 Training-time operator on a minibatch

  • The minibatch algorithm uses OAS-selected branchwise ridge, a thin SVD, and a projector-based operator to remove shared canonical components.It computes branchwise ridge levels from current embeddings and clips them to the specified bound.Source: C ALGORITHM · p. 33, §1 · p. 33

21 Post-training calibration for deterministic inference

  • Post-training calibration recomputes global training moments and freezes the operator for deterministic inference.Source: §21 · p. 33

D EXPERIMENTAL DETAILS: SYNTHETIC

  • The image experiment reports seeds 907–911 and separates recovery of generating variables from task-label prediction across embedding and prediction comparisons.The tables include learning controls, probes, native prediction heads, rank sensitivities, and unsuccessful decomposition fits.Source: D EXPERIMENTAL DETAILS: SYNTHETIC · p. 34

D.1 Data generation and audit objective

The synthetic audit uses paired image views with shared and view-specific generating factors to test whether the raw representation retains task-relevant signal beyond the expert view. Its objective is to suppress linear recovery of A while retaining F1, F2, and their interaction.

D.1 Data generation and audit objective

  • The raw and expert rendering coefficients have unit variance and pre-pixel-noise correlations of (1, .95, .30, .30).The coefficients are not scaled by those correlations; earlier unequal variances instead isolate variance effects in the population decomposition.Source: §D.1 · p. 34
  • The label depends on A and F1F2, while the audit seeks to reduce residual recovery of A and retain F1, F2, and their interaction.Recovery of the shared nuisance S is measured as an additional diagnostic; none of the generating variables is an extra training target.Source: §D.1 · p. 34
  • Independent pixel noise affects both views, while disjoint tiles make variables separately observable without forcing encoders to use separate coordinates.The chosen pixel scale yields a .02 standard deviation in a coefficient estimate using the known template.Source: §D.1 · p. 34
  • Inputs remain unclipped 64 × 64 grayscale images repeated across three channels, with ImageNet channel normalization and no resizing or augmentation.The images use channel means (.485, .456, .406) and standard deviations (.229, .224, .225).Source: §D.1 · p. 34

D.2 Training, calibration, validation, and test data

The experiments train dual-view models, calibrate operators on paired embeddings, and compare residualization methods using held-out test pairs. On common frozen embeddings, SCCR and a matched top-q removal control produce identical saved test scores, while several comparison methods have explicit convergence or inference constraints.

D.2 Training, calibration, validation, and test data

  • Separate calibration, validation, and test pairs are used, but seeds 907–908 were added after test inspection and reuse the same cohort as the initially locked seeds.The five-seed tables are not a prospectively selected replication on a new cohort and do not establish success for every initialization.Source: §D.2 · p. 35

D.3 Neural architecture and optimization

  • Image models use fine-tuned ImageNet-pretrained EfficientNetV2-S features, while dual models have separate raw and expert branches and a classifier that can use their interaction.Neural arms train for 100 epochs without early stopping, selecting the first strict minimum of the validation objective.Source: §D.3 · p. 35, §D.2 · p. 35

D.4 End-to-end residualization models

  • End-to-end SCCR, OLS, and regularized LEACE-style models replace the centered-raw baseline’s operation while retaining the same encoders, head, and task loss.SCCR removes selected shared canonical directions; OLS residualization requires both views, whereas the saved SCCR residual map at inference uses only the raw embedding.Source: §D.4 · p. 36

D.5 Shared/private neural models

  • Shared/private controls use distinct objectives and encoder counts, so equal epoch budgets do not imply matched capacity or computation.VCCA-private uses shared and private codes with reconstruction and KL terms, while DSN-MMD predicts from the raw shared code and reconstructs both views.Source: §D.5 · p. 37

D.6 Operators fitted to common frozen embeddings

  • Common-embedding operator comparisons use the same frozen raw and expert embeddings, with calibration-fitted maps fixed for probe fitting, validation, and test.This design isolates operator comparisons from changes in encoder learning.Source: §D.6 · p. 38
  • SCCR and matched top-q removal yield identical saved test scores when using the same ranks and canonical spectrum, making this an equivalence control rather than an advantage for thresholding.Common-embedding SCCR uses OAS with τ = .40; selected ranks for seeds 907–911 are 2, 2, 2, 3, 3.Source: §D.6 · p. 38
  • All 30 positive-q JIVE fits missed the convergence criterion and were excluded from scoring, while AJIVE’s primary rank heuristic selected joint rank zero in every included seed.The reported AJIVE tables use paired individual outputs; paired extensions, including D-CCA, require both views at inference, unlike fitted SCCR residuals.Source: §D.6 · pp. 38–39

D.7 Probe fitting and evaluation

The audit fits validation-selected probes to standardized retained codes and evaluates both factor recovery and label prediction. Probe scores are finite-sample measures of accessibility under the chosen probe families, not evidence of information absence or causal relevance.

D.7 Probe fitting and evaluation

  • Linear probes use standardized coordinates, while quadratic probes add every pairwise product rirj for i ≤ j, with transformations fitted on probe-training data.Separate ridge regressions predict S, A, F1, F2, and F1F2; logistic regressions predict y, with penalties selected by validation.Source: §D.7 · p. 39
  • AUROC, accuracy, and log loss evaluate label prediction, while native-head metrics assess the trained classifier without a new readout.VCCA-private has no native task head, and DSN’s native head uses its shared rather than private raw code.Source: §D.7 · p. 40
  • VCCA-private and DSN expert-side fields use the same-seed dual baseline’s expert reference, not measurements of their own expert encoders.Those models’ expert-side codes were not exported for this audit.Source: §D.7 · p. 40
  • Low probe scores indicate poor accessibility under that probe family, not absence of information, nonlinear independence, or causal relevance.Aggregate standard deviations across the five seeds measure initialization variation conditional on shared generating streams and test observations, not confidence intervals over independent datasets.Source: §D.7 · p. 40

D.8 What the image results establish

The image results report reduced linear A recovery in residuals while preserving F1 and F2 recovery, but interpret task AUROC alongside expert accessibility and the label’s dependence on A.

D.8 What the image results establish

  • End-to-end OLS changed what the expert encoder learned, while control models showed that good native prediction need not yield a useful private audit representation.Mean own-expert linear A recovery was .144 ± .308 for OLS versus .809 ± .128 for SCCR; DSN’s shared code predicted the interaction, but its private code had little interaction recovery.Source: §D.8 · p. 40
  • The learned image embeddings did not reproduce every Gaussian premise, and the unequal-variance follow-up did not establish suppression and retention after image encoding.In a separate pilot, SCCR’s learned expert exposed almost no A and linear raw-to-residual A recovery remained .877 → .876; these experiments were excluded from five-seed aggregates.Source: §D.8 · p. 40
  • Linear A recovery fell from .890 in raw codes to .051 in residuals on average, while F1 and F2 recovery remained .708 and .644, respectively.Native-head AUROC was .9928 ± .0025; the audit interprets results through suppression, retention, expert accessibility, and combined prediction rather than residual AUROC alone.Source: §D.8 · p. 40, Table 5 · p. 41

D.9 Complete results for learned representations

  • D.9 separates trained native heads from frozen post-hoc readouts, with no native VCCA-private head and no own-expert codes exported for some models.For some combined post-hoc readouts, the comparison uses the model’s retained output with the same-seed dual baseline expert rather than its own expert code.Source: §D.9 · pp. 41, 43, 44

D.10 Common-embedding results

  • Common-embedding results use a previously inspected shared assessment cohort, collapse identical controls, and report positive-rank JIVE variants as ineligible.The five-seed estimates use 4,096 assessment pairs and describe initialization variation conditional on shared data.Source: §D.10 · p. 45, Table 14 · p. 48

D.11 Exploratory JIVE iteration-budget follow-up

  • The JIVE iteration-budget follow-up remained exploratory: only seeds 908 and 911 passed the unchanged rule at the 50,000-step cap.Its factor R2 and frozen task AUROC are based on two converged seeds, not five-seed estimates; relative matrix change and sampled stability do not establish a global optimum.Source: Table 15 · p. 48, §D.11 · p. 48

E EXPERIMENTAL DETAILS: SKIN-LESION CLASSIFICATION

  • The skin-lesion experiments separately test residual prediction, whether Crust explains part of a fixed-residual gain, and whether retraining changes its linear accessibility.The reported studies include both seeds, the Fitzpatrick17k test set, and the external DDI cohort, while keeping end-to-end models separate from common-embedding operators.Source: E EXPERIMENTAL DETAILS: SKIN-LESION… · p. 49

E.1 Data, exclusions, and diagnosis label

This section describes the SkinCon cohort, expert-view inputs, matched training setup, and methods for assessing the raw representation beyond expert annotations. It also outlines diagnostics and source-fitted probes for residual diagnosis signal and Crust accessibility, distinguishing fixed-residual tests from retraining interventions and their limits.

E.1 Data, exclusions, and diagnosis label

  • After SkinCon-flagged and specified label/duplicate exclusions, the study retained 3,225 Fitzpatrick17k images and 636 DDI images.Four DDI actinic-keratosis cases were assigned to the positive class by study convention, increasing positives from 167 to 171. Group-based 60/20/20 splits used seed 909; DDI was external, and unavailable patient IDs mean splits are image-level.Source: §E.1 · p. 49

E.2 Expert views and image preprocessing

  • The Crust-only intervention kept 48 expert-input slots, comparing deliberate Crust omission, true Crust restoration, and shuffled restoration while Erosion and Scale remained zero.The shuffled condition permuted true Crust labels within partitions without diagnosis stratification; concept recovery was evaluated against the true annotations.Source: §E.2 · p. 50

E.3 Matched diagnosis networks and training

  • Matched diagnosis networks shared an architecture and training recipe, with SCCR fitting its operator on calibration embeddings and freezing that map for validation and evaluation.The 1,935 source training images were split into 965 calibration and 970 readout images; the readout subset fit and selected post-hoc predictors, while both subsets trained the image encoder.Source: §E.1 · p. 49, §E.3 · p. 51

E.4 Baseline objectives and adaptations

  • Common-embedding SCCR removed one direction in seed 909 and two in seed 910; both selected LEACE-style fits removed all 32 directions.Matched top-q reused SCCR’s fitted spectrum and removal ranks, with identical saved matrices; this comparison does not show an advantage for thresholding.Source: §E.4 · p. 52
  • JIVE’s individual outputs were zero because its rank rule selected individual ranks (0, 0), while AJIVE selected joint rank zero and individual ranks (1, 1).The rank-conditioned JIVE result does not show that JIVE always discards useful signal; the paired individual extension for both methods requires both embeddings and can inherit expert information.Source: §E.4 · p. 53

Decomposition-based CCA.

  • D-CCA used all 32 canonical pairs in both seeds and requires both views at inference.Each view had 32 supported directions; this empirical-covariance adaptation used no denoising or significance-based common-rank selection.Source: Decomposition-based CCA · p. 53
  • VCCA improved RGB reconstruction but not annotation BCE relative to the reference, while DSN learned diagnosis without improving its reconstruction criteria.These are learning diagnostics, not bootstrap comparisons or common loss functions across models; the VCCA values use posterior-mean reconstructions.Source: Decomposition-based CCA · pp. 54–55, Table 18 · p. 55

E.5 Audit predictors and their selection

  • Readouts are fitted on 970 audit images using three-fold stratified group cross-validation, with concept probes selected by mean held-out AUROC and diagnosis predictors by weighted log loss.Predictors are fitted after encoders and decomposition maps are frozen; validation images select checkpoints, not final readout settings. Raw, residual, and learned expert codes are probed separately, with full, shared, and private codes additionally examined for VCCA-private and DSN-MMD.Source: §E.5 · pp. 55–56

E.6 Three audit questions

  • Positive G means the residual improves fitted diagnosis prediction over the 45 observed annotations, but G is not a direct estimate of conditional mutual information or proof that annotations lack equivalent nonlinear information.G is reported separately from residual-only AUROC and native neural-head performance, and depends on the fitted predictors and their selection.Source: §E.6 · p. 56
  • With the SCCR residual held fixed, adding correct versus randomly reassigned Crust annotations tests predictive overlap with omitted Crust, not causal mediation or Crust removal from the residual.The encoder, expert input, and SCCR operator remain unchanged in this test.Source: §E.6 · p. 56
  • Retraining with Crust restored tests changes in Crust accessibility across separately learned codes and maps, but cannot isolate projection effects because the encoders, operator, and head also change.The dual baseline controls for encoder learning, while its zero raw-to-residual gap is structural; reduced diagnosis gain against stronger annotation inputs need not indicate greater residual removal.Source: §E.6 · p. 57

E.7 Uncertainty, external evaluation, and reporting limits

The completed comparisons report positive in-distribution residual-prediction gains in one evaluation, but negative SCCR gains on DDI. The interpretation is limited by exploratory comparisons, uncertainty not captured by the bootstrap, and the scope of the tested outcomes.

E.7 Uncertainty, external evaluation, and reporting limits

  • The bootstrap omits training, model-selection, and unknown patient-dependence uncertainty; two seeds on one cohort are not independent clinical replications.Intervals are pointwise 95th percentiles from 1,000 ordinary image-bootstrap samples, and the comparison, Crust-only intervention, and threshold follow-up are exploratory.Source: §E.7 · p. 57
  • These experiments do not establish nonlinear independence, causal relevance, complete concept removal, or reliable external clinical benefit.DDI tests transfer of the complete source-trained pipeline under different images, diagnosis composition, and annotation relationships; the observed loss of benefit does not isolate a single cause of distribution shift.Source: §E.7 · p. 57

E.8 Interpretation of the completed comparisons

  • On Fitzpatrick17k, nonlinear residual diagnosis gains for end-to-end SCCR were .073 and .051, while paired advantages over end-to-end OLS were .105 and .050; all intervals were above zero.SCCR’s native diagnosis AUROCs (.845 and .805) were below the dual baseline and DSN-MMD; on common dual-baseline embeddings, gains were approximately .000 and .018 with intervals including zero.Source: §E.8 · p. 57
  • With the omission residual fixed, true Crust attenuated nonlinear gain more than shuffled Crust by .020 and .009, but only seed 909’s interval excluded zero.After Crust-only retraining, linear true-minus-shuffled contrasts were .016 and .045 with both intervals above zero; the nonlinear contrasts and both linear true-minus-omitted contrasts included zero, while true-restored residuals still predicted Crust nonlinearly with AUROCs .581 and .578.Source: §E.8 · pp. 57–58
  • The Crust results support a modest linear response to correct rather than shuffled restoration, not complete removal or consistent improvement over omission.The primary JIVE zero output followed from the supplied individual rank zero and is not a general failure of JIVE.Source: §E.8 · p. 58
  • On DDI, SCCR’s primary nonlinear gains were −.187 and −.229, and correct restoration did not establish a consistent external improvement.VCCA-private’s low full-code diagnosis and Crust accessibility limit conclusions about its private decomposition; DSN learns diagnosis mainly through its shared code, and its source seed 910 linear private gain was .021 with interval [.012, .032].Source: §E.8 · p. 58

E.9 Complete primary Crust and diagnosis results

This appendix reports Crust and diagnosis audits, exploratory threshold sensitivity, and the tabular dropout-task pipeline.

E.9 Complete primary Crust and diagnosis results

  • Dual-baseline raw and residual diagnosis results coincide, regularized LEACE-style fits have literal zero residuals, and DDI gains do not establish useful Crust-explained residual signal.Source: §E.9 · pp. 59, 60, 63

E.10 Exploratory threshold sensitivity

  • At τ = .20, omitted-model nonlinear diagnosis gains were −.002 and .033 on F17k, versus .073 and .051 at .40; lower τ gave no uniform improvement.The linear true-minus-shuffled restoration contrast remained positive in both seeds, but nonlinear contrasts included zero; this sensitivity analysis supplies no threshold-selection rule.Source: §E.10 · p. 67

F.1 Dataset and prediction task

  • The dropout task covers 155,552 patients; the raw branch uses a fixed panel of 256 codes, while the expert branch uses five psychiatric codes and permitted demographics.The omission audit removes CPT 90806 from the expert input while retaining it in the raw input.Source: §F.1 · p. 69

F.2 Train, validation, and test splits

  • Site A supplies training and validation data, while Sites B–D are held out for testing; operator calibration and audit-probe fitting use separate Site A training partitions.Source: §F.2 · p. 69

F.3 Count preprocessing and TF-IDF

  • The fixed panel selects the 256 allowed codes with largest Site A training sums, and shared training-derived inverse-document-frequency weights transform both views.The expert mask is applied before the weighting transformation.Source: §F.3 · p. 69

F.4 Dual-head tabular-transformer architecture

  • Independent tabular-transformer encoders project each branch to 64 dimensions; raw-only and expert-only baselines use single encoders, while SCCR and comparison models use two branches.Source: §F.4 · pp. 69–70

F.5 Shared canonical component removal

  • For centered 64-dimensional embeddings, SCCR uses branchwise OAS regularization and removes canonical directions with singular values at least τ; calibration freezes the operator for test inference.Source: §F.5 · p. 70

F.6 Prediction head

  • The dropout head combines residual and expert main effects with their elementwise interaction to produce a predicted dropout probability.Source: §F.6 · p. 70

F.7 Training protocol

This section specifies training, held-out evaluation, threshold sensitivity, and the fixed code panel for dropout prediction.

F.7 Training protocol

  • Models minimize weighted binary cross-entropy with AdamW and early stopping; completed runs use seeds 1996–2000 and unfinished runs are excluded.Source: §F.7 · p. 70

F.8 Evaluation

  • Native AUROC and AUPRC are evaluated on held-out sites; separate probes assess omitted CPT 90806 and retained psychiatric-code recovery, while fixed-residual restoration tests use patient-bootstrap intervals.The restoration audit compares true and within-partition shuffled code values; its intervals do not measure training-seed variability.Source: §F.8 · pp. 70–71

F.9 SCCR threshold sweep

  • The τ sweep describes sensitivity rather than an optimal threshold for a new hospital.The main audit uses τ = .40, with additional reported settings including .05 and .70.Source: §F.9 · p. 71

F.10 Fixed code panel

  • The fixed panel contains 256 codes ranked by panel order, including 123 CPT4, 86 ICD10, and 47 RxNorm codes; CPT 90806 is omitted from the expert view in the controlled audit.Source: §F.10 · p. 71, Table 37 · p. 71

G EXPERIMENTAL DETAILS: CAD DETECTION IN CORONARY ANGIOGRAPHY EXPERIMENT

The CAD appendix details angiogram-and-mask experiments and follow-up residual audits, including their evaluation boundaries and mask-generation pipeline.

G EXPERIMENTAL DETAILS: CAD DETECTION IN CORONARY ANGIOGRAPHY EXPERIMENT

Train, validation, and test splits.

Shared canonical component removal.

Prediction head.

  • The prediction head combines residualized raw and centered mask representations with an interaction term, then outputs a disease probability.Source: Prediction head · p. 74

Evaluation.

  • Follow-up comparisons include native heads and common-embedding readouts; they are retrospective, single-seed analyses on an inspected test set, without external-site or training-seed validation.Several baseline settings were ineligible or did not converge, and VCCA-private and DSN were not run in this follow-up.Source: §G.1 · p. 75

G.1 Follow-up prediction and residual audits

  • At τ = .10, own-expert linear diagnosis AUROC was .794 [.777, .813], versus pre-removal raw AUROC .548 [.522, .571]; neither raw nor residual showed a clear log-loss gain beyond the expert.The true-versus-shuffled appearance-panel contrast was −.0004 [−.0011, .0004] for linear probes and .0009 [−.0015, .0033] for nonlinear probes, so branch sensitivity and reduced linear recovery did not establish clinically useful complementary evidence.Source: §G.1 · p. 76

G.2 Vessel mask generation

  • A U-Net trained on ARCADE segmentation data generates the vessel masks, which the classifier uses for each video's longest frame.Polygon annotations are rasterized at 256 × 256, and the mask generator is trained with binary cross-entropy plus Dice loss.Source: §G.2 · p. 77

Related papers

Loading 2610.13819v1…