ModelDiff: A Framework for Comparing Learning Algorithms

1 Introduction

• Design choices can change model biases even when accuracy barely changes, as prior work finds for shape bias and transfer-learning behavior.Hermann et al. [HCK20] report shape-bias variation among models differing by less than 1% in accuracy; Liu et al. [LXL+22] report differing implicit biases at equal loss.Source: §1 · p. 2
• ModelDiff compares learning algorithms by finding subpopulations where models differ, designing candidate feature transformations, and validating them through counterfactual analysis.Its output is a set of distinguishing subpopulations, while an interpreter uses their patterns to design transformations.Source: §1 · p. 2, Figure 1 · p. 3
• 15% versus 1%: adding a spider web raises “spider” confidence more for augmentation-trained than non-augmentation models.This case study uses LIVING17 [STM21].Source: §1 · p. 2
• +12% versus -4%: adding a yellow patch changes “landbird” confidence for from-scratch versus pre-trained models, while a human face produces +4% versus -1%.The comparison uses ImageNet-pre-trained and from-scratch classifiers fine-tuned or trained on WATERBIRDS [SKH+20].Source: §1 · p. 3
• 7% versus 2%: a window-like pattern raises “truck” confidence more under a smaller than a larger learning rate.The comparison uses CIFAR-10 classifiers with different learning rates and batch sizes.Source: §1 · p. 3
• Across all three case studies, the framework surfaces fine-grained behavioral differences and helps analyze the role of learning-algorithm design choices.The framework is intended to compare any pair of algorithms without a prior hypothesis about their differences.Source: §1 · pp. 2–3

2.1 Formalizing algorithm comparisons via distinguishing transformations

• Model classes isolate differences induced by learning algorithms rather than training non-determinism, focusing comparison on features used for prediction.The model class induced by an algorithm is the distribution over models obtained from randomly sampled datasets.Source: §2.1 · p. 4
• Distinguishing transformations change one model class’s predictions consistently while leaving the other’s comparatively unchanged.Their effects are defined through expected loss changes for the two models and can test hypotheses such as differing texture sensitivity.Source: §2.1 · p. 4
• Informative distinguishing transformations must capture naturally occurring, semantically meaningful features rather than imperceptible or pathological changes.Adversarial and out-of-distribution transformations can satisfy the formal definition without improving qualitative understanding.Source: §2.1 · p. 4

2.2 Datamodel representations for comparison

• Datamodels approximate a model-output function by fitting a regularized linear predictor from sampled training subsets to outputs such as correct-class margin.The resulting vectors form a datamodel matrix over test and training examples.Source: §2.2 · p. 5
• A datamodel represents each test example by training-example weights, with each coordinate aligned to the same training example across algorithms.This consistent basis makes representations comparable even for models without explicit representations, such as decision trees.Source: §2.2 · p. 5
• Datamodel trends have a quantitative counterfactual interpretation because they can predict the effect of adding or removing training examples on model outputs.This predictive property supports comparing how algorithms use their fixed training set.Source: §2.2 · p. 5

3 Comparing learning algorithms with MODELDIFF

• MODELDIFF outputs distinguishing subpopulations: groups of examples sharing features used by one learning algorithm but not the other.The method examines how each algorithm uses training data and supports designing informative distinguishing transformations from these groups.Source: §3 · pp. 6–7
• Normalized datamodels are computed for both algorithms, residualized to remove shared training-data influence, and analyzed with PCA.The residual datamodel for one algorithm captures training directions that remain after projecting away components also influencing the other.Source: §3 · p. 6
• PCA components represent weighted combinations of training examples that one algorithm uses more distinctly than the other.These distinguishing training directions are selected as high-variance directions in the residual datamodel space.Source: §3 · p. 6
• A training direction distinguishes algorithms when it explains more datamodel variance for one algorithm than the other, producing a larger average output change under corresponding training-set reweighting.Proposition 1 connects explained-variance differences to differences in model behavior.Source: §3 · p. 6
• The final workflow inspects each distinguishing subpopulation, infers a shared feature, applies it as an input transformation, and measures its average effect on both model classes.Figure 2 summarizes the sequence from datamodels and residuals through PCA, inspection, and transformation testing.Source: §3 · p. 7, Figure 2 · p. 7

4 Case studies: Applying MODELDIFF

• Across three case studies, MODELDIFF identifies subpopulations, researchers inspect them for candidate transformations, and those transformations are tested on models trained with each algorithm.The studies cover data augmentation, pre-training, and optimizer choices.Source: §4 · p. 7

4.1 Case study: Data augmentation

• The analysis surfaces distinguishing subpopulations involving spiders on spider webs and salamanders with yellow polka dots.Source: Figure 3 · p. 8
• For LIVING17, standard augmentation models achieve 89% average test accuracy versus 81% without augmentation.The compared ResNet-18 models use identical settings except for standard augmentation.Source: §4.1 · p. 8

Appendix B.1 provides further experimental details for this case studies and the others.

• Data augmentation increased sensitivity to spider-web and polka-dot features, with stronger effects than models trained without augmentation.Spider-web overlays increased spider confidence by 11% versus 1%, while polka-dot overlays increased salamander confidence by 14% versus 2%; greater transformation intensity widened the gap.Source: Appendix B.1 · p. 9

4.2 Case study: Pre-training

• ImageNet pre-training suppressed reliance on yellow color but introduced or amplified reliance on human faces for landbird predictions.Yellow patches increased landbird confidence for models trained from scratch but decreased it for pretrained models; face patches increased confidence by up to 4% only for pretrained models.Source: §4.2 · pp. 10–12

4.3 Case study: Optimizer hyperparameters

• Lower SGD noise increased reliance on black-and-white texture and rectangular shape when predicting dogs and trucks.Six-pixel black-and-white patches increased dog probability by 14% versus 9%, while eight-pixel rectangular patches increased truck probability by 7% versus 2% for low- versus high-noise models.Source: §4.3 · pp. 12–13

5 Discussion

• MODELDIFF can support aggregate algorithm comparison by aggregating datamodel cosine similarities, although its computational cost scales with datamodel estimation.Similar datamodels and distinguishing subpopulations were recoverable with 10× fewer samples, and sample-efficiency improvements would directly speed up the method.Source: §5 · p. 14

6 Related work

• The framework differs from fixed-model comparison by targeting learning-algorithm-induced differences, fine-grained features, and model-agnostic comparisons.It aims to separate algorithmic design choices from random-seed variability, while remaining applicable to models without explicit representations.Source: §6 · p. 15

7 Conclusion

• The paper introduces a general framework that compares learning algorithms through how their resulting models rely on training data.Its formal target is distinguishing transformations, and three case studies show how data augmentation, pre-training, and optimization hyperparameters shape model behavior.Source: §7 · p. 16

A Algorithm analysis

• The formal analysis equates the explained-variance gap between datamodels with the difference in model-output sensitivity along a training direction.This result assumes datamodels perfectly approximate the corresponding model outputs.Source: A Algorithm analysis · p. 23

B Experimental Setup

• Experiments compare data augmentation, ImageNet pre-training, and SGD hyperparameters on LIVING17, WATERBIRDS, and CIFAR-10.The setup varies the specified learning algorithms while documenting architectures, optimization settings, and accuracies.Source: §B.1 · p. 24, §B.2 · pp. 24–25

B.3 Datamodels

• Datamodels encode training-example importance by fitting sparse linear surrogates that predict a model’s correct-class margin from sampled training subsets.Estimation samples random subsets, trains models on them, evaluates target examples, and applies ℓ1 regularization.Source: §B.3 · pp. 25–26

B.4 Feature transformations

• Feature transformations add patches or perturbations, and their counterfactual effects are estimated by averaging class-confidence changes across test examples and 500 trained models per algorithm.The transformations target yellow color, human faces, spider webs, polka dots, black-white texture, and rectangular shapes.Source: §B.4 · pp. 26–27

D Additional analysis of distinguishing subpopulations

• Additional analysis identifies distinguishing subpopulations through class-specific inspection and relative influence of training examples.Relative influence compares aggregate datamodel weights for the two algorithms.Source: D Additional analysis of… · p. 29

D.1 Case study: Standard data augmentation

• On LIVING17, standard data augmentation can amplify co-occurrence bias toward spider webs and texture bias toward polka dots.Source: §D.1 · p. 30
• Relative-influence analysis finds web-like and yellow-black training images strongly influence the corresponding test subpopulations only for augmented models.Class-specific inspection contrasts these surfaced subpopulations with random same-class images.Source: Figure 9 · p. 30, Figure 12 · p. 33, Figure 13 · p. 33

D.2 Case study: ImageNet pre-training

• ImageNet pre-training reduces dependence on yellow color but introduces dependence on human-face features in WATERBIRDS models.The associated feature transformations and relative-influence analysis support this contrast.Source: §D.2 · p. 31, Figure 10 · p. 31

D.3 Case study: SGD hyperparameters

• Low SGD noise amplifies black-white texture and rectangular-shape features in CIFAR-10 models.Relative-influence analysis and class-specific inspection identify these features for dogs and trucks.Source: §D.3 · p. 32, Figure 11 · p. 32, Figure 14 · p. 33

E.1 Case study: Standard data augmentation

• Increasing transformation intensity and the minimum crop size of RandomResizedCrop increases sensitivity to spider-web and polka-dot transformations.The trend is reported consistently across the evaluated augmentation settings.Source: §E.1 · p. 34, Figure 15 · p. 34

E.2 Case study: ImageNet pre-training

• ImageNet-pretrained ResNet18 and ResNet50 models respond similarly to the yellow-color and human-face transformations.Both transformations alter predictions in comparable ways across the two pretrained architectures.Source: §E.2 · p. 35, Figure 16 · p. 35

E.3 Case study: SGD hyperparameters

• Higher SGD noise makes CIFAR-10 models more sensitive on average to black-white texture and rectangular-shape transformations across classes.The transformation effects are class-dependent, with larger differences for semantically similar classes.Source: Figure 17 · p. 36, §E.3 · p. 36

F.1 Aggregate metric for algorithm comparison

• Datamodel cosine similarity measures whether models trained by two algorithms depend on the same training examples for predictions.The metric compares datamodel vectors over test examples.Source: §F.1 · p. 37
• On WATERBIRDS, ImageNet-pretrained ResNet50 models are more similar to pretrained ResNet18 models than to synthetic-pretrained or scratch-trained models.The comparison is based on the distribution of datamodel cosine similarities.Source: §F.1 · p. 37, Figure 18 · p. 37
• On CIFAR-10, high-SGD-noise ResNet models are more similar to other high-noise models than to low-noise models.This pattern also holds when comparing high-noise ResNet9 models with smaller-width high-noise ResNet9 models.Source: §F.1 · p. 37, Figure 18 · p. 37

• The top 5–6 residual principal components correspond to training directions with high explained variance for one algorithm but not the other.Explained variance measures the importance of a direction, or weighted combination of training examples, to model predictions.Source: §F.2 · p. 38
• Across all three case studies, top-100 PCs of θ(1\2) explain more variance under A1 than A2, while top-100 PCs of θ(2\1) explain less under A1 than A2.Source: Figure 19 · p. 38, §F.2 · p. 38

• The sample-size experiment uses WATERBIRDS datamodels estimated from 5,000, 10,000, 25,000, or 50,000 independently sampled training subsets.Source: §F.3 · p. 39
• Average cosine similarity between datamodels exceeds 0.9 when the estimation sample size drops tenfold from 50,000 to 5,000.Source: §F.3 · p. 39, Figure 20 · p. 40
• Top-10 principal components estimated with 5,000 or 50,000 samples have similar explained variance when evaluated across sample sizes from 5,000 to 50,000.This indicates that the principal-component directions preserve their importance across the tested sample sizes.Source: §F.3 · p. 39, Figure 21 · p. 40

• The analysis targets prediction-level differences, for which the paper notes that no existing algorithm-comparison methods provide such analysis.Source: §F.4 · p. 41
• The experiment compares directions computed from all test examples with directions computed only from examples whose models share the same prediction mode.Source: §F.4 · p. 41
• Across every case study, distinguishing training directions retain high cosine similarity after removing test examples with differing average model predictions.Source: §F.4 · p. 41, Table 2 · p. 41

F.5 Leveraging CLIP to analyze distinguishing subpopulations

• CLIP specializes the framework’s verification step for vision classifiers trained on ImageNet-like data using shared vision-language embeddings.Source: §F.5 · p. 42
• CLIP extracts distinguishing captions by contrasting the top-k images aligned with a training direction against images sampled from the same distribution.The pipeline embeds images and captions, computes image–caption cosine similarities, and selects captions with the largest contrast.Source: §F.5 · p. 42
• For WATERBIRDS pre-training comparisons, captions such as yellow, lemon, and sulphur match the manually inferred “yellow color” feature.Source: §F.5 · p. 42, Figure 22 · p. 43
• For the same WATERBIRDS comparison, captions such as florist, faces, and counselors match the manually inferred “human face” feature.Source: §F.5 · p. 42, Figure 23 · p. 43

F.6 Subpopulations surfaced by principal components of residual datamodels

• Residual-datamodel principal components define top-k and bottom-k subpopulations through the most positive and negative projections, respectively.Source: §F.6 · p. 44
• The supplementary figures show principal-component subpopulations for LIVING17, CIFAR-10, and WATERBIRDS across data augmentation, SGD noise, and ImageNet pre-training comparisons.Source: Figure 24 · p. 44, Figure 25 · p. 45, Figure 26 · p. 46