Source-linked AI summary
Model Cards for Model Reporting
Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru
TL;DR
High-impact ML models lack standardized documentation of performance and intended use. This paper proposes model cards and demonstrates them with two example models, showing how reporting can expose suitability, biases, and evaluation details.
Problem
High-impact ML models lack standardized documentation communicating their performance characteristics before deployment.
Method
The paper proposes model cards that document model context, intended uses, evaluation procedures, disaggregated performance, ethical considerations, and caveats.
Results
Two example cards demonstrate reporting for a smiling classifier and toxicity detector, including model details, intended uses, disaggregated evaluation metrics, and caveats.
Takeaways & Limitations
Model cards can increase transparency about how well models work and indicate contexts in which they may not be well suited.
Takeaways & Limitations
Model cards depend on their creators’ integrity and may not prevent misleading representations of model results.
Abstract
from arXiv · showhide
Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that released models be accompanied by documentation detailing their performance characteristics. In this paper, we propose a framework that we call model cards, to encourage such transparent model reporting. Model cards are short documents accompanying trained machine learning models that provide benchmarked evaluation in a variety of conditions, such as across different cultural, demographic, or phenotypic groups (e.g., race, geographic location, sex, Fitzpatrick skin type) and intersectional groups (e.g., age and race, or sex and Fitzpatrick skin type) that are relevant to the intended application domains. Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information. While we focus primarily on human-centered machine learning models in the application fields of computer vision and natural language processing, this framework can be used to document any trained machine learning model. To solidify the concept, we provide cards for two supervised models: One trained to detect smiling faces in images, and one trained to detect toxic comments in text. We propose model cards as a step towards the responsible democratization of machine learning and related AI technology, increasing transparency into how well AI technology works. We hope this work encourages those releasing trained machine learning models to accompany model releases with similar detailed evaluation numbers and other relevant documentation.
1 INTRODUCTION
The introduction identifies the absence of standardized documentation for model performance, especially in high-impact applications, and proposes model cards as concise records for transparent, context-specific reporting. Model cards should describe intended uses and model characteristics while reporting performance across relevant demographic, cultural, phenotypic, conditional, and intersectional groups.
- Motivation: No standardized procedures communicate trained ML and AI models’ performance characteristics, posing particular risks in health care, employment, education, and law enforcement.Systematic biases in deployed commercial models were often exposed only after negatively affected users reported their experiences.
- Contribution: Model cards are proposed as short, one- to two-page records accompanying released machine learning models.They complement dataset documentation paradigms such as “Datasheets for Datasets” and resemble the proposed medical tripod statement.
- Contribution: Model cards report model type, intended use cases, attributes for which performance may vary, and measures of model performance.The framework focuses on trained model characteristics rather than only the datasets used for training and testing.
- Contribution: Performance evaluations should be quantitatively broken down across cultural, demographic, phenotypic, domain-relevant, and intersectional groups and conditions.Model cards should also explain the motivation for selected metrics, group definitions, and other relevant factors.
2 BACKGROUND
The background motivates standardized, population-based model reporting because machine learning lacks common stress tests and reporting formats, while subgroup and intersectional analyses reveal errors that aggregate or isolated categories can miss.
- Benchmarking and reporting: Machine learning systems lack standardized stress tests and reporting formats despite their broad reach and impact.Mature industries such as electronic hardware benchmark systems under different conditions and document performance in datasheets.
- Population-based evaluation: Population-based reporting is needed because outcomes can differ across demographic groups, as shown by failures to account for women in crash testing and clinical trials.The passage describes serious head injuries among women in side impacts and overdosing in women after exclusively male clinical trials.
- Intersectional analysis: Model error and success analyses should examine intersectional groups, not only unitary categories such as men, women, or nonbinary groups.Intersectional analysis considers two or more characteristics simultaneously because isolated characteristics may not capture their interaction.
- Intersectional analysis: The General Motors case illustrates how analyzing race and gender separately can obscure discrimination against Black women.Hiring patterns involving White women in secretarial roles and Black men in factory roles concealed the treatment of Black women.
- Sensitive characteristics: Race and gender are socially sensitive characteristics, and labeling these categories as dataset ground truth is not always clear or appropriate.The passage notes unresolved questions about how such categories should be labeled and whether datasets should include them.
3 MOTIVATION
The rapid growth of machine learning has exposed serious errors without giving prospective users a way to understand systematic impacts before deployment. Model Cards address this gap by standardizing transparent, inclusive, and ethically informed reporting that supports comparison and oversight across stakeholders.
- Motivation: Rapidly increasing machine learning use has brought more reported errors and failures, while prospective users lack insight into systematic impacts before deployment.The consequences of such errors can be serious.
- Motivation: Model Cards standardize reporting so stakeholders can compare candidate models using traditional evaluation metrics alongside ethical, inclusive, and fair considerations.They also help policymakers and regulators identify questions to ask and benchmarks for judging model suitability.
- Stakeholder use cases: Model reporting serves practitioners, developers, and software teams by clarifying intended-use performance, enabling model comparisons, and informing product design decisions.Practitioners can also track model performance over time.
- Stakeholder use cases: Reporting helps policymakers, adopting organizations, ML-knowledgeable users, and affected individuals understand how systems may succeed or fail and evaluate deployment options.The supplied passage also identifies fine-tuning, model combination, and additional rules or constraints as options for curating models.
- Fairness and informed selection: Sliced group evaluations can expose errors that disproportionately affect some groups, support fairness analysis, and help potential users compare models across evaluation datasets.Standardized reporting can also incentivize model releasers to provide details that improve informed model selection.
4 MODEL CARD SECTIONS
Model cards should disclose how a model was built, its development assumptions, expected behavior across relevant population groups, and performance for those groups. The proposed sections are informative rather than exhaustive and may be tailored to the model, context, and stakeholders.
- Proposed model card sections: Model cards disclose model construction, development assumptions, behavior across cultural, demographic, or phenotypic groups, and performance across those groups.They are proposed to inform stakeholders about trained machine learning models.
- Proposed model card sections: The proposed sections are not exhaustive and may be tailored to the model, context, and stakeholders.Additional details may include interpretability approaches and stakeholder-relevant explanations.
Model Card · 4.1 Model Details · 4.2 Intended Use
The model card framework organizes transparent reporting around model details and intended use. These sections document development facts, disclose appropriate usage contexts, and acknowledge limits on proprietary or sensitive information.
- Model Card: Model cards should identify the developing person or organization, model date, version, type, training algorithms, parameters, and fairness-related information.They provide basic facts that help stakeholders understand what the model represents.
- Model Card: Model documentation can include explainability and privacy approaches used during model training and serving.These disclosures address psychological and contextual factors affecting what counts as a good explanation.
- 4.1 Model Details: Model dates indicate when development occurred, helping stakeholders infer which techniques and data sources were available.This information situates the model historically and technically.
- 4.1 Model Details: Model versions help stakeholders identify the latest release, associate known bugs with correct versions, and support model comparisons.Version descriptions should explain differences from previous versions.
- 4.1 Model Details: Disclosure should not require compromising private information or revealing proprietary training techniques.Organizations should share basic decisions and facts while recognizing that disclosure detail may differ across corporations and academic research groups.
- 4.2 Intended Use: The intended-use section should explain what the model should and should not be used for, why it was created, and its original users, use cases, and contexts.It also frames the statistical analysis presented elsewhere in the model card.
- 4.2 Intended Use: Intended-use disclosures may restrict application to text examples longer than 100 tokens or to black-and-white images only.The examples direct users toward a full-color-image classifier when color images are involved.
4.3 Factors
The Factors section asks model cards to summarize performance across groups, instruments, and deployment environments, with factor selection tied to intended use and evaluation rationale. It emphasizes intersectional and potentially vulnerable groups while requiring privacy-aware decisions about what can be responsibly inferred and reported.
- Factors: Model cards should evaluate performance across relevant groups, instruments, and deployment environments.These factors capture variation associated with evaluation-data categories, input-capture technologies, and deployment conditions.
- Groups: Intersectional group analysis should reflect combined cultural, demographic, and phenotypic characteristics relevant to the model’s intended deployment.Group selection should consider context and whether some groups may face greater risk of unjust or prejudicial treatment.
- Groups: For computer vision, age, gender, and Fitzpatrick skin type may matter, but privacy, policy, and legal expertise should guide responsible inference and access.Differential privacy is offered as one example of how group information could be stored and accessed.
- Instrumentation and environments: Camera hardware, software, and settings can change model performance, while low lighting and humidity can reduce face-detection accuracy.Relevant instrumentation includes lenses, stabilization, high dynamic range, portrait-mode blurring, aperture, shutter speed, and ISO.
- Evaluation factors: Model cards should report foreseeable relevant factors and explain which evaluation factors were selected, including differences caused by unavailable annotations.Fitzpatrick skin type may be relevant for face detection even when suitable evaluation data are not yet available.
4.4 Metrics
Model cards should report metrics selected according to a model’s structure and intended use, with context explaining which measures were prioritized. Classification systems should disclose relevant error rates, while score-based and intersectional analyses can compare distributions or scores across groups and report confidence intervals.
- Metric selection: Metrics should be selected based on the model’s structure and intended use, with the card explaining which performance measures were reported and why.The appropriate metrics differ between classification systems with class-label outputs and systems whose primary output is a score.
- Classification metrics: Classification model cards can report false positive, false negative, false discovery, and false omission rates derived from confusion matrices.The relative importance of these metrics depends on the system, product, and context.
- Classification metrics: Model cards should list all classification metric values and explain which were prioritized during development, since surveillors and surveilled people may value different error rates.In surveillance, surveillors may prioritize low false negative rates, whereas surveilled people may prioritize low false positive rates.
- Score-based metrics: For score-based systems, cards can compare group distributions using central-tendency and dispersion measures, extending to cross entropy, perplexity, KL divergence, and pinned AUC.Relevant summaries include mode, median, mean, range, quartiles, absolute deviation, variance, and standard deviation.
- Intersectional analysis: Intersectional analysis can compare scores across demographic groups for systems such as translation and speech recognition, and disaggregated metrics make confidence intervals especially important.Examples include comparing BLEU scores for translation systems and word-error rates for speech recognition systems; confusion-matrix confidence intervals can treat matrices as probabilistic performance models.
4.5 Evaluation Data · 4.6 Training Data
The framework calls for evaluation datasets that are transparent, representative, publicly usable, and challenging, while training-data documentation should provide comparable detail when feasible or basic group distributions when constraints prevent it.
- 4.5 Evaluation Data: Evaluation datasets should be publicly available for third-party use or supplied alongside model-card analyses to enable further benchmarking.Documentation should identify the datasets used and explain their motivation and preprocessing.
- 4.5 Evaluation Data: Evaluation-data documentation should provide visibility into each dataset’s source and composition.The framework recommends referencing documents that expose this information.
- 4.5 Evaluation Data: Evaluation datasets should represent typical use cases, anticipated test scenarios, and challenging cases to support statistically accurate and verifiable model cards.Evaluation may include both a set matching the expected use case and a more challenging set.
- 4.5 Evaluation Data: When representative datasets are difficult to find, synthetically generated datasets may provide coverage for otherwise unevaluated use cases.The paper points to Section 5.2 for an example of synthetic data in model evaluation.
- 4.6 Training Data: Training-data documentation should ideally contain as much information as evaluation-data documentation.The paper recognizes that proprietary data or non-disclosure agreements may make equivalent detail infeasible.
- 4.6 Training Data: When detailed training-data disclosure is infeasible, model cards should provide basic group distributions and other details that inform stakeholders about potential biases.The stated constraints include proprietary data and non-disclosure agreements.
4.7 Quantitative Analyses · 4.8 Ethical Considerations · 4.9 Caveats and Recommendations
The paper recommends disaggregated quantitative evaluation with metric variation and confidence intervals where possible, alongside explicit ethical analysis and documentation of caveats, uncovered groups, further testing, and model-use recommendations.
- 4.7 Quantitative Analyses: Quantitative analyses should disaggregate results by chosen population factors and report evaluation metrics for each subgroup.Parity across disaggregated subgroups corresponds to a common definition of fairness.
- 4.7 Quantitative Analyses: Analyses should provide confidence intervals when possible and demonstrate metric variation, such as with error bars.These practices clarify uncertainty and variation in subgroup performance.
- 4.8 Ethical Considerations: The ethical considerations section should surface development challenges and solutions for stakeholders, while informing responsible practices and future work.Ethical analysis may not produce precise solutions, but the process of ethical contemplation remains useful.
- 4.8 Ethical Considerations: Ethical review should examine sensitive data, implications for human life or flourishing, and risk-mitigation strategies.Relevant concerns include protected classes and potential effects on health or safety decisions.
- 4.8 Ethical Considerations: Where possible, documentation should include additional development safeguards, such as external-board review or testing with a specific community.These activities provide further evidence of ethical consideration during model development.
- 4.9 Caveats and Recommendations: Caveats should identify concerns not addressed elsewhere, including whether results indicate a need for further testing.This section should also record limitations in the evaluation and implications for future assessment.
- 4.9 Caveats and Recommendations: Caveats and recommendations should note relevant groups missing from the evaluation dataset and provide additional recommendations for model use.The section should also describe the ideal characteristics of an evaluation dataset for the model.
5 EXAMPLES
The examples apply model cards to image- and text-based classifiers, revealing subgroup-specific performance issues and the need to document model changes across releases. They show how evaluation results can identify unsuitable contexts and motivate targeted fine-tuning or updated reporting.
- Examples: The paper presents worked model-card examples for an image-based classification system and a text-based scoring system.The examples cover a smiling classifier and Perspective API’s TOXICITY classifier.
- Smiling classifier: The smiling classifier is evaluated on CelebA across age and gender categories, exposing different error patterns among demographic groups.The analysis examines performance across both age and gender categories.
- Smiling classifier: The false discovery rate is much higher for older men, while men overall have a higher false negative rate.Older men are often incorrectly classified as smiling, whereas many smiling men are classified as not smiling.
- Smiling classifier: These findings indicate that the smiling model may be unsuitable for diverse audiences and better suited to applications prioritizing smile detection over absence detection.The paper suggests that additional fine-tuning with images of older men may improve performance balance.
- TOXICITY classifier: The TOXICITY example compares versions 1 and 5 using an intersectional Identity Phrase Templates test set, showing why model cards should be updated with each release.TOXICITY v. 1 has low performance for several terms, especially “lesbian”, “gay”, and “homosexual”.
6 DISCUSSION & FUTURE WORK
The paper presents model cards as flexible documentation for model context and disaggregated performance, illustrated through smile and toxicity classifiers. It argues that model cards can support transparency across institutions and high-impact applications, while their usefulness depends on careful, ethical creation and evaluation.
- Framework: Model cards document a model’s context and performance across unitary and intersectional population groups.They are intended to accompany models only after review finds that foreseeable benefits outweigh foreseeable risks.
- Future work: The smile-detector example highlights missing race and skin-type evaluation and calls for broader gender, Fitzpatrick skin-type, camera, and environmental annotations.Its binary gender classes are acknowledged as inadequate for evaluating a spectrum of genders.
- Demonstrations: The paper demonstrates model cards with a smile classifier evaluated on CelebA and a toxicity detector evaluated on Identity Phrase Templates.The examples report confusion-matrix metrics for smiling and Pinned AUC for toxicity, alongside intended use, data, ethics, and caveats.
- Applicability: The framework is intended to apply across institutions, contexts, stakeholders, and critical social-institution applications such as benefits, employment, criminal risk, and DNA analysis.Its scope is designed to accommodate varied machine-learning model types and potential use cases.
- Limitations: Model cards are one flexible approach to transparency, but their usefulness and accuracy depend on the integrity of their creators.The framework is designed to vary in scope and specificity for different models and use cases.