PaperlayerView exact revision on arXiv

Source-linked AI summary

Domino: Discovering Systematic Errors with Cross-Modal Embeddings

Sabri Eyuboglu, Maya Varma, Khaled Saab, Jean-Benoit Delbrouck, Christopher Lee-Messer, Jared Dunnmon, James Zou, Christopher Ré

arXiv:2203.14960v3cs.LGcs.AI

TL;DR

Machine learning models can fail systematically on important, often hidden data slices, while existing slice-discovery evaluations provide limited quantitative evidence. Domino addresses this gap with a scalable evaluation framework, cross-modal embeddings, and an error-aware mixture model, identifying 36% of ground-truth coherent slices and generating slice descriptions.

  • Problem

    Important model errors often occur on hidden, semantically meaningful slices, but existing slice-discovery evaluations are qualitative, synthetic, or small-scale.

  • Method

    Domino combines cross-modal representations with an error-aware mixture model, alongside a framework for quantitatively evaluating slice-discovery methods across 1,235 settings.

  • Results

    Domino identifies 36% of ground-truth coherent slices and is the first slice-discovery method to generate natural-language descriptions for identified slices.

  • Takeaways & Limitations

    Cross-modal embeddings and Domino's slicing approach outperform existing slice-discovery methods and enable semantically meaningful slice descriptions.

  • Takeaways & Limitations

    Domino may miss important failure modes, and its web-sourced image-text embeddings may reflect societal biases.

Abstract

from arXiv · show

Machine learning models that achieve high overall accuracy often make systematic errors on important subsets (or slices) of data. Identifying underperforming slices is particularly challenging when working with high-dimensional inputs (e.g. images, audio), where important slices are often unlabeled. In order to address this issue, recent studies have proposed automated slice discovery methods (SDMs), which leverage learned model representations to mine input data for slices on which a model performs poorly. To be useful to a practitioner, these methods must identify slices that are both underperforming and coherent (i.e. united by a human-understandable concept). However, no quantitative evaluation framework currently exists for rigorously assessing SDMs with respect to these criteria. Additionally, prior qualitative evaluations have shown that SDMs often identify slices that are incoherent. In this work, we address these challenges by first designing a principled evaluation framework that enables a quantitative comparison of SDMs across 1,235 slice discovery settings in three input domains (natural images, medical images, and time-series data). Then, motivated by the recent development of powerful cross-modal representation learning approaches, we present Domino, an SDM that leverages cross-modal embeddings and a novel error-aware mixture model to discover and describe coherent slices. We find that Domino accurately identifies 36% of the 1,235 slices in our framework - a 12 percentage point improvement over prior methods. Further, Domino is the first SDM that can provide natural language descriptions of identified slices, correctly generating the exact name of the slice in 35% of settings.

1 INTRODUCTION

Systematic model errors often concentrate in hidden, important slices, yet existing slice discovery methods lack comprehensive quantitative evaluation and may produce incoherent groups. Domino addresses these gaps with large-scale evaluation and cross-modal embeddings that identify and describe coherent error slices.

  • Models can make dangerous errors on important slices despite strong overall performance, including chest X-rays without chest drains.
  • Hidden slices in high-dimensional inputs are difficult to extract or annotate, motivating slice discovery from unstructured data.
  • Existing automated slice discovery methods use learned representations to find semantically meaningful subgroups where models perform poorly.
  • No quantitative framework comprehensively evaluates slice discovery methods for both underperformance and coherence across realistic settings.
  • Domino combines cross-modal embeddings with an error-aware approach to identify coherent slices and generate natural-language descriptions.

2 RELATED WORK

Related work establishes the prevalence of underperforming slices, the difficulty of discovering them in unstructured data, and the complementary role of cross-modal representations. Existing robustness benchmarks provide useful slice labels but do not support comprehensive SDM evaluation.

  • Prior studies report underperforming slices across medical imaging, radiograph classification, melanoma detection, language processing, and object detection.
  • Earlier slice discovery work focused largely on structured inputs or metadata, while images and similar unstructured data require more complex methods.
  • Robustness benchmark datasets are insufficient for systematic SDM evaluation because they annotate few slices or lack models known to underperform on them.
  • Cross-modal models embed inputs and text in a shared space, producing semantically meaningful representations used in retrieval and zero-shot classification.

3 SLICE DISCOVERY PRELIMINARIES

The slice discovery problem assumes coherent, potentially overlapping subgroups on which a trained classifier performs worse than elsewhere. An SDM receives a trained classifier and labeled data, then outputs slicing functions intended to recover those subgroups.

  • Each data example has membership indicators for k coherent, potentially overlapping slices defined by concepts familiar to domain experts.
  • In the chest X-ray example, the slices are normal images with chest drains and collapsed-lung images without chest drains.
  • Inputs, labels, and slice memberships are jointly distributed, with training, validation, and test data drawn independently and identically.
  • A classifier has degraded slice performance when its expected loss on that slice is at least ϵ worse than outside the slice.
  • An SDM takes a trained classifier and labeled dataset as inputs and outputs slicing functions that partition the data into subgroups.
  • A discovery succeeds when each ground-truth slice is predicted by some slicing function with precision above threshold β.

4 SLICE DISCOVERY EVALUATION FRAMEWORK

The evaluation framework estimates how often SDMs recover coherent slices with degraded model performance by generating many realistic settings from labeled datasets and models. It spans multiple slice types, model classes, domains, and precision-based metrics.

  • The framework evaluates SDMs on whether they recover coherent slices where the model underperforms, addressing the lack of labeled slices in public datasets.
  • Evaluation fits an SDM on validation data, applies its slicing functions to test data, and compares predictions with ground-truth slices using metrics L.
  • Settings are generated from base datasets with hierarchical labels or rich metadata, enabling coherent target and slice labels and models with degraded slice performance.
  • The framework covers rare, correlation, and noisy-label slices, representing different causes of degraded model performance.
  • Both trained and synthetic models are used, with synthetic models helping isolate recovery of intended slices from other coherent underperforming slices.
  • 1,235 settings span natural images, medical images, and EEG time-series data.
  • Precision-at-k measures the proportion of the top k discovered elements belonging to the ground-truth slice, using k = 10.

5 DOMINO

Domino is a three-step slice discovery method that embeds inputs and text jointly, identifies coherent underperforming regions with an error-aware mixture model, and describes them in natural language.

  • Embed: Domino embeds inputs and text in a shared cross-modal representation space.The input and text embedding functions are learned jointly or obtained from pretrained cross-modal models.
  • Slice: The slicing stage fits an error-aware mixture model to identify coherent, underperforming regions in the embedding space.The model jointly represents embeddings, labels, and predictions, encouraging slices homogeneous in error type.
  • Slice: The mixture model assumes embeddings, labels, and predictions are conditionally independent given slice membership.Each slice has distributions over embeddings, true labels, and model predictions.
  • Slice: A hyperparameter γ controls the trade-off between slice coherence and underperformance in the mixture-model objective.The objective is maximized using expectation-maximization.
  • Describe: Domino ranks independently sourced text phrases against distilled slice prototypes to generate natural-language descriptions.Descriptions are selected by maximizing the dot product between distilled slice prototypes and text embeddings.

6 EXPERIMENTS

Experiments evaluate Domino and its components across 1,235 slice discovery settings, finding consistent benefits from cross-modal embeddings and strong performance from the error-aware mixture model, while descriptions often identify slice names.

  • Evaluation: 1,235 slice discovery settings are used to compare Domino with existing methods and validate its core design choices.The evaluation covers three datasets and two input modalities.
  • Cross-modal embeddings: 0.570 mean precision-at-10 is achieved by CLIP embeddings on synthetic natural-image models.This is a 9 percentage point increase over BiT embeddings and a 23 percentage point increase over random activations.
  • Cross-modal embeddings: Nearly 15 percentage points separate CLIP or BiT embeddings from trained classifier activations on trained natural-image models.CLIP and BiT do not differ on these models, while both outperform classifier activations.
  • Cross-modal embeddings: 0.765 mean precision-at-10 is achieved by ConVIRT embeddings on synthetic medical-image models, a 7 point improvement over BiT's 0.695.On trained models, classifier activations are worst on rare and noisy-label slices but competitive on correlation slices.
  • Cross-modal embeddings: 0.697 mean precision-at-10 is achieved by cross-modal EEG embeddings on synthetic medical time-series models, a 17 point gain over unimodal embeddings at 0.532.Cross-modal embeddings also outperform unimodal embeddings with trained models.
  • Error-aware mixture model: 0.639 mean precision-at-10 is achieved by the error-aware mixture model on noisy and rare natural-image slices, a 105% improvement over George.ConfusionSDM nevertheless outperforms the mixture model on correlation slices.
  • Natural-language descriptions: 34.7%, 41.0%, and 39.0% of rare, correlation, and noisy-label natural-image slices respectively receive the correct slice name or synonym at rank one.The corresponding top-ten rates are 57.4%, 55.4%, and 48.7%.

7 CONCLUSION

The paper introduces a quantitative framework for evaluating slice discovery across underperformance and coherence, and proposes Domino, which combines cross-modal representations with an error-aware mixture model.

  • Domino combines cross-modal representations with an error-aware mixture model for slice discovery.The method identifies coherent slices and generates semantically meaningful descriptions.
  • The programmable framework measures slice discovery performance across underperformance and coherence.It addresses the lack of large-scale quantitative evaluation for existing approaches.
  • Domino's embedding and slicing steps outperform those of existing slice discovery methods.
  • Cross-modal embeddings enable semantically meaningful slice descriptions, while Domino requires only black-box model access.

8 REPRODUCIBILITY STATEMENT

The authors provide an open-source evaluation framework and a Python package that lets users run Domino on their own models and datasets.

  • The evaluation framework is available as an open-source implementation.
  • Users can run Domino on their own models and datasets.
  • The Domino Python package can be installed with pip install domino.

9 ETHICS STATEMENT

The ethics statement cautions that Domino may miss important model failures and that its web-sourced image-text embeddings may reflect societal biases.

  • Domino may fail to identify some important underperforming slices.Practitioners should continue standard evaluations on accurately labeled, representative test sets.
  • Practitioners should use standard evaluations alongside Domino for model auditing.
  • Image-text embeddings trained on web data may reflect societal biases in slice identification and description.The statement raises concern that underrepresented groups or concepts may be less likely to be identified as underperforming slices.

A.1 EXAMPLES OF SLICE DESCRIPTIONS

The paper illustrates Domino's generated slice descriptions across natural images, medical images, and medical time-series data, using representative inputs or report fragments alongside ranked descriptions.

  • Figure 5 shows natural-image slice descriptions, associated top images, and description scores across rare, correlation, and noisy-label settings.The examples include settings where the exact slice name appears among the top five descriptions.
  • Figure 6 presents five descriptions for medical-image slices across rare, correlation, and noisy-label settings.Colored bars indicate accurate descriptions, gray bars indicate incorrect descriptions, and bar lengths represent dot-product scores.
  • Figure 7 shows EEG examples where Domino describes seizure-detection errors on young patients with reports mentioning young age.The three settings span all three slice categories, and report fragments are shown because full reports cannot be provided.

A.2 EXTENDED RELATED WORK: SURVEY OF SLICES IN THE WILD

Prior studies document systematic slice failures across medical imaging, facial analysis, speech recognition, object recognition, and text systems. These failures commonly arise from rare subgroups, noisy labels, or spurious correlations with clinically or contextually irrelevant attributes.

  • Survey scope: Systematic slice failures have been reported across medical imaging, facial analysis, speech recognition, object recognition, and named entity disambiguation.The surveyed examples include pneumothorax, hip-fracture, skin-lesion, melanoma, COVID-19, pneumonia, gender, and rare-entity settings.
  • Correlation slices: Spurious correlations with devices, markings, patient attributes, hospitals, or other contextual variables can degrade performance when those correlations do not hold.Examples include chest drains, mascara, skin markings, hospital identity, department, and patient-specific attributes.
  • Correlation slices: External-hospital evaluations found lower pneumonia performance and showed that CNNs could classify hospital system and department, indicating hospital-specific confounding features.The reported setting involved pneumonia screening CNNs evaluated across three external hospitals.
  • Noisy label slices: Noisy labeling functions can systematically miss coherent subsets, causing models trained on weak labels to underperform on those subsets.The surveyed example concerns aortic valve malformation classification under weak supervision.
  • Rare slices: Models underperform on rare slices when those subgroups occur infrequently in training data or are underrepresented in the dataset.Examples include African American speakers, non-Western and low-income-community household objects, rare entities, cervical fractures, and darker- or female-faced subjects.

A.3 EXTENDED DESCRIPTION OF EVALUATION FRAMEWORK

The evaluation framework organizes slice discovery around controlled rare, correlation, and noisy-label conditions and compares automated methods across broad settings. Results show strong variation by slice type, model construction, and dataset correlations, while the method assumes suitable paired input-text data when pretrained cross-modal embeddings are inadequate.

  • Evaluation framework: 1,235 trained slice discovery settings span three domains, four slice categories, and five slice parameters α.The framework evaluates SDMs across a broad collection of tasks, applications, and base datasets.
  • Slice categories: Rare slices occur infrequently in training data, so they may contribute little to model loss and remain poorly learned.The framework constructs rare-slice settings from datasets with hierarchical label schemas.
  • Slice categories: Correlation slices arise when a model relies on a variable C correlated with target Y, degrading performance when S = 1[C ≠ Y].The framework generates these settings by subsampling datasets with metadata annotations to achieve controlled correlations.
  • Slice categories: Noisy label slices contain subsets with higher label error rates because of ambiguous classes, failed labeling heuristics, or limited annotator expertise.The framework treats these uneven label errors as a distinct source of degraded model performance.
  • Evaluation results: Performance is generally lower on rare slices, while synthetic-model settings often outperform trained-model settings.Rare slices are difficult to identify because their subclasses occur at very low frequency; trained models may also contain unlabeled underperforming coherent slices.
  • Evaluation results: The naive Confusion SDM performs strongly on correlation slices and can outperform the error-aware mixture model when correlations are strong.The result suggests that inspecting the confusion matrix may suffice in some strongly correlated settings.
  • Evaluation results: The error-aware mixture model performs significantly better on rare slices than prior SDMs, especially for trained models.The paper attributes this to jointly modeling input embeddings, class labels, and model predictions.
Loading 2203.14960v3…