PaperlayerView exact revision on arXiv

Source-linked AI summary

Interactive Model Cards: A Human-Centered Approach to Model Documentation

Anamaria Crisan, Margaret Drouhard, Jesse Vig, Nazneen Rajani

arXiv:2205.02894v1cs.HCcs.AIcs.CL

TL;DR

Documentation for deep-learning NLP models often targets ML- or NLP-trained readers, although analysts without that training increasingly use such models. The authors developed interactive model cards through studies with experts and non-expert analysts; their findings emphasize careful information and visual design to support transparency and interpretability.

  • Problem

    Model documentation is tailored mainly to ML- or NLP-trained readers, despite increasing use of deep-learning NLP models by analysts without that training.

    Source: §1 · p. 1

  • Method

    The authors used expert and non-expert input to develop and evaluate an interactive model-card prototype and examine documentation design.

    Source: §1 · p. 2

  • Results

    Industry feedback validated many design choices, and the findings emphasize information and visual design for model-card transparency and interpretability.

    Source: §4.4 · p. 14

  • Takeaways & Limitations

    Careful design choices matter because model cards may otherwise remain inaccessible to many people who use AI/ML systems.

    Source: §5 · p. 14

    The study does not fully establish how particular design decisions affect model-card understandability and actionability.

    Source: §5 · p. 14

Abstract

from arXiv · show

Deep learning models for natural language processing (NLP) are increasingly adopted and deployed by analysts without formal training in NLP or machine learning (ML). However, the documentation intended to convey the model's details and appropriate use is tailored primarily to individuals with ML or NLP expertise. To address this gap, we conduct a design inquiry into interactive model cards, which augment traditionally static model cards with affordances for exploring model documentation and interacting with the models themselves. Our investigation consists of an initial conceptual study with experts in ML, NLP, and AI Ethics, followed by a separate evaluative study with non-expert analysts who use ML models in their work. Using a semi-structured interview format coupled with a think-aloud protocol, we collected feedback from a total of 30 participants who engaged with different versions of standard and interactive model cards. Through a thematic analysis of the collected data, we identified several conceptual dimensions that summarize the strengths and limitations of standard and interactive model cards, including: stakeholders; design; guidance; understandability & interpretability; sensemaking & skepticism; and trust & safety. Our findings demonstrate the importance of carefully considered design and interactivity for orienting and supporting non-expert analysts using deep learning models, along with a need for consideration of broader sociotechnical contexts and organizational dynamics. We have also identified design elements, such as language, visual cues, and warnings, among others, that support interactivity and make non-interactive content accessible. We summarize our findings as design guidelines and discuss their implications for a human-centered approach towards AI/ML documentation.

1 INTRODUCTION

Model documentation is often expert-oriented and may omit details such as unintended uses or disaggregated performance, leaving analysts to conduct their own analyses. The paper proposes interactive model cards and develops them through expert and non-expert studies to support access, interrogation, and risk awareness.

1 INTRODUCTION

  • Existing model cards depend on developers’ clear, accurate reporting, while important details may be omitted and users may need to implement their own analyses.The paper notes that this leaves non-experts underserved and may exacerbate potential deployment harms [15].Source: §1 · p. 2
  • Earlier interactive model-card explorations do not establish how much or what kind of interactivity benefits users.Examples include customizable model inputs and tools for interactively creating data slices [73] [23].Source: §1 · p. 2
  • The proposed interactive model card aims to lower barriers to model information, support deeper model interrogation, and surface risks and limitations without extra developer work..Source: §1 · p. 2
  • The authors used expert-informed design guidelines and a subsequent study with non-expert analysts to develop and evaluate an interactive model-card prototype.The work examines standard model cards, Robustness Gym reports, and interactive model cards through a human-centered lens [44] [23].Source: §1 · p. 2
  • The paper contributes design guidelines, conceptual dimensions for evaluating documentation across worker backgrounds, and a functional prototype for further use.The authors also make research artifacts available online.Source: §1 · p. 2

2 BACKGROUND AND RELATED WORK

Related work frames model documentation as part of broader efforts to support transparency, trust, and accountability. The paper connects interactive exploration and productive skepticism with practical needs around safety, organizational use, and record-keeping.

2 BACKGROUND AND RELATED WORK

  • The authors argue that stimulating productive skepticism alongside interaction and sensemaking could mitigate over-trust, while opportunities for productive skepticism can help stakeholders calibrate appropriate trust in model behavior for a particular context and use case.Source: §2 · p. 3
  • The authors describe model cards as foundational documentation for AI/ML model performance and characteristics [44]; robustness reports summarize performance on data slices [23], and both are core elements of the IMC design.Source: §2 · p. 3
  • Participants in both studies believed interactive model cards could support record-keeping and organizational alignment during deployment..Source: §2 · p. 3
  • The paper’s exploration of subpopulations and high- and low-confidence examples responds in part to practitioners’ needs for model and data exploration.Prior work describes data- and model-centric exploration and examination of cases where models perform well or poorly [39].Source: §2 · p. 3
  • The authors report that guidance, shareability, and traceability begin to address practitioner needs related to explainability, auditing, and organizational dynamics.Those needs include understanding stakeholder goals, supporting traceability and auditing, and addressing concerns within organizational structures [8] [32] [31] [57].Source: §2 · p. 4

3 STUDY 1 : CONCEPT STUDY WITH ML, AI, AND NLP EXPERTS

Before engaging non-expert analysts, the authors worked with researchers and practitioners in ML/AI development, AI Ethics, and NLP to co-create design guidelines for interactive model cards.

3 STUDY 1 : CONCEPT STUDY WITH ML, AI, AND NLP EXPERTS

  • Experts in ML/AI development, AI Ethics, and NLP contributed their expertise to co-create interactive model-card design guidelines..Source: §3 · p. 4

3.1 Design Probes

The design probes focused on disaggregated model performance and dataset understanding, areas the authors considered undersupported by standard model cards. The interactive concept added user-data exploration and visual comparison to standard-card and robustness-report elements.

3.1 Design Probes

  • The probes focused on disaggregated model performance and dataset understanding because the authors considered these tasks undersupported by standard model cards.They scoped out other possibilities, including comparing multiple models or datasets and training on analyst data.Source: §3.1 · p. 4
  • The reference probe used a DistilBERT model fine-tuned on SST-2 for sentiment analysis, and the authors found it unclear how much information about its base models to include.They therefore used text from both the DistilBERT and SST-2 fine-tuned model cards.Source: §3.1 · p. 4
  • The standard-card probe followed the reference card’s layout and retained its ability to test a user-provided sentence.The robustness-report probe explored built-in and potentially sensitive subpopulations, including race and gender.Source: §3.1 · p. 4
  • The interactive concept adds ways to upload analyst data, generate examples or define subpopulations, and compare sentence embeddings from model and analyst data.The additions are organized into data upload, example generation, and data comparison panels.Source: §3.1 · p. 5
  • The authors implemented the probes as Google Slides paper prototypes, simulating interactive behavior with links, transitions, and animations.Prototype text was drawn, with small exceptions, from the existing DistilBERT and SST-2 fine-tuned model cards.Source: §3.1 · p. 5

3.2 Study Procedures

The first study used expert-facing documentation probes to elicit needs for model cards, then synthesized interview data into themes that informed the interactive prototype. Participants came from ML/AI development and design, ethics, and NLP.

3.2.1 Study Design.

  • Experts explored a standard model card, a Robustness Gym report, and an interactive-card concept in think-aloud interviews, reflecting on their own and non-expert analysts’ needs.Interviews used a grounded approach and lasted 60 minutes.Source: §3.2.1 · p. 5

3.2.2 Recruitment.

  • The study set a target of 10 participants and reached out to 23, recruiting until reaching the target while aiming to include at least one participant from ML/AI development and design, Ethics, and NLP.Source: §3.2.2 · p. 5

3.2.3 Data Collection and Analysis.

  • Researchers conducted iterative thematic analysis, coding transcripts and synthesizing themes that informed the functional prototype.Only six participants consented to recording; the study recorded approximately 11 hours and 15 minutes of video.Source: §3.2.3 · p. 5, §3.2.2 · p. 5
  • Table 2 organizes expert perspectives into themes and includes supporting quotes alongside the model-card context in which they arose.Contexts include standard model cards, interactive model cards, Robustness Gym reports, or general comments.Source: Table 2 · p. 6

3.3 Results

The concept study identified interconnected perspectives on stakeholders, design, guidance, trust and safety, and sensemaking. Participants described barriers in existing documentation and opportunities for interactive cards, while emphasizing the need for careful design and contextual interpretation.

3.3 Results

  • Participants’ perspectives formed five interconnected conceptual dimensions spanning stakeholders, design, guidance, trust and safety, and sensemaking and skepticism.The themes summarize perspectives on the standard model card, Robustness Gym report, and interactive model card probes.Source: §3.3 · p. 6

3.3.1 Stakeholders.

  • All participants doubted non-expert analysts would understand model cards or know how to act on their information, and several found standard cards better suited to engineers.Participants also noted that even ML/NLP students struggled with terminology and needed a terminology “cheat sheet.”Source: §3.3.1 · pp. 6–7

3.3.2 Design Considerations.

  • Participants criticized technical language and dense text, while many said the IMC’s visual layout helped them notice information the standard card and report obscured.Some nevertheless found the IMC cluttered; participants also requested a concise model-purpose summary and clearer prioritization of performance information.Source: §3.3.2 · p. 7

3.3.3 Guidance.

  • Participants sought actionable guidance through definitions, visual explanations, prompts, and examples, with a useful default example seen as important.Examples were also viewed as a way to help analysts build a mental model of how the model works and what it lacks.Source: §3.3.3 · p. 7

3.3.4 Trust and Safety.

  • Participants questioned vague ethical statements and called for explicit disclosure of whether bias assessments were conducted, with warnings when analyses were absent.They supported disaggregating performance for sensitive groups but cautioned that group definitions vary by context and subgroup sizes should be prominent.Source: §3.3.4 · p. 7

3.3.5 Sensemaking and Skepticism.

  • Participants saw interaction as a way to elicit questions, test semantically meaningful cases, and inspect model limitations while forming mental models of the data.One participant revised an initially favorable subgroup interpretation after noticing its small sample size.Source: §3.3.4 · pp. 7–8, §3.3.5 · p. 8

3.4 Summarizing Design Guidance for an Interactive Model Card

The authors distill their findings into four design guidelines for interactive model cards. The guidelines emphasize deliberate information design, conceptual interaction, actionable guidance, and defaults that encourage productive skepticism.

3.4 Summarizing Design Guidance for an Interactive Model Card

  • DG1 calls for deliberate design of information hierarchies, representations, and interactions so content is effectively understood by a wide audience.The authors argue information design should be testable to check whether it is interpreted accurately.Source: §3.4 · p. 8
  • DG2 uses interaction to build conceptual understanding, balancing contextual exploration with objective model and performance details and making interactions discoverable with helpful defaults.The guideline extends interaction beyond previewing model outputs.Source: §3.4 · p. 8
  • DG3 recommends actionable guidance, such as warnings, prompts, summaries, and visual cues, to help non-experts interpret information and know how to act on it.The authors note that model cards have primarily been tailored toward developers and ML experts.Source: §3.4 · p. 8
  • DG4 recommends enticing, discoverable defaults that promote productive skepticism about what a model does and how it can be used.The authors propose sensitive populations as a starting point for default information.Source: §3.4 · p. 8

4 STUDY 2 : EVALUATION WITH NON-EXPERT ANALYSTS

The authors implemented an interactive model-card prototype from the first study’s guidelines and evaluated it with 20 non-expert analysts. The evaluation used interviews comparing the prototype with a reference card, with recruitment and analysis procedures described.

4.1 Interactive Model Card Functional Prototype

  • The prototype combined reference-card information with interactive tools for probing model performance and underlying data, guided by the first study’s design principles.It included performance visualizations, custom subpopulations, analyst-provided sentences or datasets, and real-time updates to the visualizations.Source: §4.1 · p. 9
  • The prototype paired layered information and reduced jargon with definitions, visualization guidance, instructions, and warnings about potentially unreliable information.It alerted analysts when a subgroup fell below a specified size threshold.Source: §4.1 · pp. 9–10

4.2.1 Study Design.

  • Each 60-minute interview asked participants about their model use and documentation, then had them explore a reference model card and the IMC before discussing usability, functionality, safety, and ethics.The IMC was demonstrated after participants explored the reference card, and they were asked to compare the two.Source: §4.2.1 · p. 10

4.2.2 Recruitment.

  • Of 176 questionnaire respondents, 117 were excluded; the remaining 59 candidates were grouped by code-tool use into Type I (n=35) and Type II (n=24).Type I analysts used programming tools not at all or less than once a quarter; Type II analysts reported daily or weekly use.Source: §4.2.2 · p. 10

4.2.3 Data Collection and Analysis.

  • Researchers recorded approximately 23 hours and 30 minutes of video, including the interview and an additional on-camera debrief between the study moderator and notetaker; themes surfaced in the first study seeded this analysis.Source: §4.2.3 · p. 10

4.3 Results

Non-expert analysts favored the IMC overall, particularly for making model information easier to understand, while identifying design trade-offs and limits around ethics and safety. Their responses also showed how model cards could fit into organizational practices.

4.3 Results

  • Overall, participants favored the IMC over the SMC, and nearly all found the IMC easier to read and understand.The study found no large differences in response patterns between the two analyst groups.Source: §4.3 · pp. 10–11, §4.3.1 · p. 11

4.3.1 Evaluating our Design Choices.

  • Guiding elements, visual cues, warnings, and prompts helped participants interpret information, while interactivity let them explore data; some nevertheless reported overload or wanted design improvements.Participants also valued layout, language, adding examples, and visual and interactive elements.Source: §4.3.1 · pp. 11–12

4.3.2 Model Understandability - Getting the Basics Right.

  • Nearly half could not correctly explain the model using the SMC alone, while the IMC’s side panel helped participants distinguish the fine-tuned model from its base model.Participants found it hard to articulate that the SMC described sentiment analysis; the side panel highlighted potentially relevant words.Source: §4.3.2 · pp. 12–13

4.3.3 The Role of Model Cards in Sociotechnical Systems.

  • Participants saw model cards as shareable resources for colleagues and stakeholders, and IMC interactivity supported lightweight experimentation that was cumbersome with the SMC.Some also saw potential for integrating model cards into technical infrastructure and reducing documentation burden if IMCs were easy to produce.Source: §4.3.3 · p. 13

4.3.4 Ethics and Safety are Challenging Topics.

  • Participants struggled to contextualize safety and ethical risks, and some worried that actionable guidance could create a false sense of security.One participant emphasized that a model card should not substitute for further analysis of ethical and safety risks.Source: §4.3.4 · p. 14

4.4 Summary

The evaluation of the functional IMC prototype with non-expert analysts validated many design choices while identifying avenues for improvement. Their feedback also illuminated how model cards might be used in organizational sociotechnical systems.

4.4 Summary

  • The functional IMC prototype was assessed with twenty non-expert analysts drawn from across industry.Their feedback validated many design choices and offered avenues for further improvement.Source: §4.4 · p. 14
  • Analysts’ feedback provided a more detailed view of how model cards might be used within organizational sociotechnical systems.Source: §4.4 · p. 14
  • The findings point to the importance of information and visual design for model-card transparency and interpretability.Source: §4.4 · p. 14

5 DISCUSSION

The discussion argues that carefully designed interaction can help data workers understand and interrogate model behavior, while model cards remain a bridge to deeper analysis rather than a substitute. It also emphasizes organizational knowledge-sharing and the contextual, difficult-to-action nature of ethics and safety considerations.

5 DISCUSSION

  • Interaction helped participants understand model behavior and its implications for their work, with benefits relying on careful information architecture.The architecture included data visualizations, layout, language, visual cues, warnings, and prompts.Source: §5 · p. 14
  • Interactive model cards are a first step toward exploring a model’s trustworthiness and guiding deeper interrogation, not a substitute for further analysis.Participants differed on the appropriate balance between interactivity and an analysis interface.Source: §5 · p. 14
  • Interactivity can support productive skepticism by letting people contextualize and interrogate model behavior, but these interactions require clear, actionable guidance.The paper connects curiosity-driven interaction with potentially mitigating anchoring bias.Source: §5 · p. 15
  • Model cards should support collaborative knowledge-sharing and negotiation across organizational roles, not only individual use.Collaborative development and integrated programmatic tooling could reduce burdens of generation, reuse, and refinement.Source: §5 · p. 15
  • Ethics and safety content can spark ethical thinking, but participants found these issues contextual and difficult to make actionable through documentation alone.The findings include cynicism toward ethics statements and indicate that ethical considerations are personal and contextual.Source: §5 · p. 15
  • The findings may depend on design choices and sentiment-analysis tasks, while the authors expect the broader guidelines and conceptual dimensions to generalize.Specific design elements are expected to be refined and adapted in future work.Source: §5 · p. 15
  • Creating and learning to use interactive model cards is more burdensome than using text-based cards, and reducing or redistributing documentation work remains future work.The authors describe steps toward adaptable, extensible generators.Source: §5 · p. 15

6 CONCLUSION

The authors co-developed an interactive model card for a large language model with ML/AI experts and non-experts. They frame the work as a human-centered examination of model cards and invite others to build on the findings toward interrogable, contestable, responsibly documented models.

6 CONCLUSION

  • The authors co-developed an interactive model card for a large language model with ML/AI experts and non-experts.Source: §6 · p. 15
  • The work presents a human-centered examination of model cards across the broader range of stakeholders using and deploying ML/AI models.Source: §6 · p. 15
  • The authors invite researchers, practitioners, and developers to build on the findings toward interrogable, contestable, responsibly documented ML/AI models.Source: §6 · p. 15
Loading 2205.02894v1…