Evaluation of Similarity-based Explanations

1 INTRODUCTION

• Similarity-based explanations present similar training instances and their predictions as evidence for a test prediction, making them accessible to users with little machine-learning expertise.They are motivated as analogous to human decisions based on prior experiences.Source: §1 · p. 1
• The study addresses which relevance metrics are appropriate for similarity-based explanation, a question that had received little exploration despite its importance.It evaluates metrics through comprehensive experiments and summarizes results in Table 1.Source: §1 · p. 2, Table 1 · p. 2

1.1 PRELIMINARIES

• The benchmark uses classification models trained with cross-entropy loss, with test predictions assigned by the highest-probability class.The trained parameter minimizes the training loss, and each instance is represented as an input-output pair.Source: §1.1 · p. 2

1.2 RELEVANCE METRICS

• The evaluation compares similarity metrics based on identity, last-hidden-layer, or all-hidden-layer representations with gradient-based metrics including IF, RIF, FK, GD, and GC.Identity-map metrics measure input similarity without model information, serving as naive baselines.Source: §1.2 · p. 3

2 RELATED WORK

• Earlier metric studies targeted classification accuracy, so their findings are not directly applicable to evaluating relevance metrics for similarity-based explanations.The paper distinguishes its goal from choosing metrics that improve predictive accuracy.Source: §2 · p. 3
• Explanation research discusses faithfulness, plausibility, robustness, and readability, but no standard test has been established for evaluating these requirements.Designing such tests remains an open problem.Source: §2 · p. 3
• The paper addresses this gap with minimal requirement tests for faithfulness and plausibility rather than directly evaluating those broad requirements.It presents this as the first empirical study evaluating existing relevance metrics for similarity-based explanation.Source: §2 · p. 4

3 EVALUATION CRITERIA FOR SIMILARITY-BASED EXPLANATION

• Three tests assess whether relevance metrics meet minimal requirements for similarity-based explanations: model randomization, identical class, and identical subclass.The first targets faithfulness, while the latter two target plausibility.Source: §3 · p. 4

3.1 MODEL RANDOMIZATION TEST

• A valid metric should produce different rankings for a well-trained model and a randomly initialized model, indicating dependence on the model’s inference process.A rank correlation near zero supports model dependence, whereas correlation near one suggests model independence.Source: §3.1 · p. 4

3.2 IDENTICAL CLASS TEST

• The identical class test requires the most relevant training instance to share the test instance’s class, avoiding explanations that pair a prediction with a different-class example.Such mismatches can make users question valid predictions and ignore the model.Source: §3.2 · pp. 4–5, §3.4 · p. 5

3.3 IDENTICAL SUBCLASS TEST

• The identical subclass test requires the retrieved instance to share the test instance’s subclass when the prediction is correct, using randomly reassigned classes to avoid dependence on a specific taxonomy.The experimental datasets place original classes as mutually exclusive subclasses within two newly formed classes.Source: §3.3 · p. 5, §3.4 · p. 6

3.4 DISCUSSIONS ON VALIDITY OF CRITERIA

• Same-subclass retrieval is intended to make explanations plausible across users, but it assumes models represent subclasses; preliminary experiments confirmed this for the studied models.The requirement matters because users may infer different taxonomies and otherwise regard explanations as logically broken.Source: §3.4 · pp. 5–6

4 EVALUATION RESULTS

• The evaluation repeated each test ten times after training models, scoring relevance for training instances and computing correlations or test success rates.The experiments used image, text, and table datasets with logistic regression and deep neural networks.Source: §4 · p. 6

4.1 RESULT OF MODEL RANDOMIZATION TEST

• Gradient-based metrics ℓlast 2, coslast, and gradient-based metrics showed correlations close to zero in the model randomization test across datasets and models.Correlations close to zero indicate model dependence in this test.Source: §4 · p. 6, Figure 3 · p. 7, §4.1 · p. 7

4.2 RESULTS OF IDENTICAL CLASS AND IDENTICAL SUBCLASS TESTS

• Dot metrics, IF, FK, and GD performed poorly on both the identical class and identical subclass tests.Source: §4 · p. 6, §4.2 · p. 7, Figure 4 · p. 7

4.1 RESULT OF MODEL RANDOMIZATION TEST

• Some metrics failed the model randomization test because they produced correlations above 0.7 on MNIST and Vehicle, allowing model-irrelevant instances to be raised.Source: §4.1 · p. 7

4.2 RESULTS OF IDENTICAL CLASS AND IDENTICAL SUBCLASS TESTS

• GC performed well on both identical class and identical subclass tests across all datasets and models, making it the recommended practical choice.Source: §4.2 · p. 7, Figure 4 · p. 7

Failure of Dot Metrics and Gradient-based Metrics

• IF, FK, and GD often selected scarce, extreme-norm instances regardless of class, whereas GC ignored large norms through normalization.Figure 5 and Figure 6 link these selections to large norms and show GC selecting same-class instances instead.Source: §5 · p. 8, Figure 6 · p. 9
• In experiments, although instances with extremely large norms were scarce, IF, FK, and GD selected only such instances as relevant, whereas GC was not attracted to large norms.Source: §5 · p. 8

Success of GC

• For logistic regression, GC decomposes into cosine similarity between prediction residuals and cosine similarity between inputs.The residual-cosine term is identified as essential for GC.Source: §5 · p. 8
• The residual cosine is always nonnegative for same-class pairs but can be negative for different-class pairs, increasing the chance that GC favors same-class instances.Source: §5 · p. 8
• Input-gradient cosine alone achieved about chance-level success on the identical subclass test, suggesting that the entire gradient is needed for GC to work effectively.Source: §5 · pp. 8–9

• Gradient cosine similarity performs best across the three tests and is recommended for similarity-based explanation in practice.The tests assess model dependence and explanation plausibility.Source: §6 · p. 9
• The study evaluates metrics with a model randomization test plus newly designed identical-class and identical-subclass tests.Source: §6 · p. 9
• Some relevance metrics fail the requirements, making them inappropriate for similarity-based explanation.Source: §6 · p. 9
• The proposed criteria cover only limited aspects of faithfulness and plausibility, motivating further evaluation criteria and studies of other explanation methods.The authors specifically identify counter-examples as another explanation method for future evaluation.Source: §6 · p. 9

A GRADIENT-BASED METRICS

• Gradient-based metrics measure relevance using gradients of each instance’s loss with respect to model parameters.For an instance z, the relevant quantities are the model parameter θ, loss ℓ(z; θ), and gradient ∇θℓ(z; θ).Source: A GRADIENT-BASED METRICS · p. 13
• Influence Function measures relevance by the increase in test loss caused by removing a training instance from the training set.A larger value means the instance is more essential for predicting the test instance.Source: A GRADIENT-BASED METRICS · p. 13
• Relative IF uses cosine similarity between Hessian-adjusted loss gradients, while Fisher Kernel uses the Fisher information matrix.Relative IF uses φ(z) = H^-1/2∇θℓ(z; bθ), and Fisher Kernel incorporates the Fisher information matrix I.Source: A GRADIENT-BASED METRICS · p. 13
• Grad-Dot measures relevance by the dot product of loss gradients, representing loss decrease after a small model update; Grad-Cos is its cosine-based modification.The cosine version is also discussed in relation to Charpiat et al. (2019) and Perronnin et al. (2010).Source: A GRADIENT-BASED METRICS · p. 13
• The section relates these gradient-based metrics to prior proposals including the representer point value, Influence Function, Relative IF, and Fisher Kernel.Source: A GRADIENT-BASED METRICS · p. 13

B.1 DATASETS AND MODELS

• The study evaluates six datasets spanning image, text, and tabular classification: MNIST, CIFAR10, TREC, AGNews, Vehicle, and Segment.The datasets cover handwritten digits, objects, questions, news, vehicle types, and image regions.Source: §B.1 · pp. 13–14
• The benchmark models include logistic regression, CNNs, MobileNetV2, Bi-LSTMs, MLPs, and dataset-specific neural architectures.MNIST uses logistic regression and CNN; CIFAR10 additionally uses MobileNetV2; TREC and AGNews use logistic regression and two-layer Bi-LSTM; Vehicle and Segment use logistic regression, MLP, or the same Vehicle models.Source: §B.1 · p. 14
• Training uses the Adam optimizer with learning rate 0.001, while sampled training-set sizes vary by dataset.Examples include 5,500 MNIST instances, 5,000 CIFAR10 instances after pre-training, 2,726 TREC instances, and 423 Vehicle instances.Source: §B.1 · p. 14
• Without CIFAR10 pre-training, classification performance dropped significantly.Source: §B.1 · p. 14
• Influence Function and Fisher Kernel were not evaluated on MobileNetV2 because their matrix inverses required too much computation time.The limitation remained even with the conjugate-gradient approximation proposed by Koh & Liang (2017).Source: §B.1 · p. 14

B.2 COMPUTING INFRASTRUCTURE

• Model training used an NVIDIA GTX 1080 GPU with an Intel Xeon Silver 4112 CPU and 64GB RAM, while testing and relevance computation used an Xeon E5-2680 v2 CPU with 256GB RAM.Source: §B.2 · p. 14

• Linear logistic regression models p(y | x; θ) using the y-th softmax entry of Wx, with θ = W and W containing the class weight vectors.C is the number of classes and d is the input dimensionality.Source: C DERIVATION OF GC FOR LINEAR LOGISTIC… · pp. 14–15
• The loss gradient with respect to each class weight is expressed using the residual rz = σ(Wx) − ey.The residual compares the softmax prediction with the one-hot target vector.Source: C DERIVATION OF GC FOR LINEAR LOGISTIC… · p. 15

D REPAIRING GRADIENT-BASED METRICS

• Replacing dot products with ℓ2 or cosine similarity reduces the influence of training instances with extremely large norms, and the repaired metrics achieve higher success rates on several criteria.The repaired variants are ℓIF 2, ℓFK 2, ℓGD 2, cosIF, cosFK, and cosGD.Source: D REPAIRING GRADIENT-BASED METRICS · p. 15
• The repaired metrics are named ℓIF 2, ℓFK 2, and ℓGD 2 for the ℓ2 metric, and cosIF, cosFK, and cosGD for the cosine metric, respectively.Source: D REPAIRING GRADIENT-BASED METRICS · p. 15

E DO THE MODELS CAPTURE SUBCLASSES?

• The models capture latent subclasses imperfectly: MNIST and TREC show relatively clear clusters, whereas CIFAR10 and AGNews have vague cluster boundaries.UMAP visualizations indicate that different subclasses are not completely mixed.Source: E DO THE MODELS CAPTURE SUBCLASSES? · p. 15

F.1 FULL RESULTS

• Complete results are reported for the model randomization, identical class, and identical subclass tests in Tables 2–4, consistent with observations in Section 4.Table 2 summarizes Spearman correlations, while Tables 3 and 4 summarize average success rates.Source: §F.1 · p. 15, Table 2 · p. 16, Table 3 · p. 18
• The repaired metrics are marked with a ♦ in the complete-results tables, and cosIF and cosGD correspond to RIF and GC, respectively.This naming equivalence is noted in the appendix.Source: §F.1 · p. 15, Table 2 · p. 16, Table 3 · p. 18
• UMAP visualizations show latent representations for each dataset and model, supporting inspection of subclass structure alongside the full test results.The supplied figure caption identifies Figure 9 as a visualization of hall for each dataset and model.Source: Figure 9 · p. 17

F.2 ADDITIONAL RESULTS

• The additional criteria extend identical class and subclass testing to the top 10 relevant training instances, requiring each selected instance to match the test instance’s class or subclass.The top-10 identical class test results appear in Table 3, while the top-10 identical subclass test results appear in Table 4.Source: §F.2 · p. 20

G EXAMPLES OF EACH EXPLANATION METHOD

• Dot-product gradient metrics such as IF, FK, and GD tend to select non-typical instances because they favor training instances with large norms.The examples cover CIFAR10 with CNN and AGNews with LSTM, including both correct and incorrect predictions.Source: G EXAMPLES OF EACH EXPLANATION METHOD · p. 20
• The appendix provides example selections for correct and incorrect predictions on CIFAR10 with CNN in Figures 10–11 and AGNews with LSTM in Tables 7–8.The figure and table captions identify the datasets, models, and prediction conditions.Source: G EXAMPLES OF EACH EXPLANATION METHOD · p. 20, Figure 10 · p. 23, Figure 11 · p. 24
• The reported examples compare several relevance metrics, including repaired and unrepaired gradient-based variants, across the selected-instance outputs.The supplied result passages list values for dot-product variants, IF, FK, GD, GC, and repaired metrics.Source: G EXAMPLES OF EACH EXPLANATION METHOD · p. 20