Source-linked AI summary
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models
Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, Ann Yuan
TL;DR
NLP model behavior is difficult to analyze across individual predictions, datasets, and controlled input changes because existing tools often cover only part of this workflow. LIT integrates local explanations, aggregate analysis, and counterfactual generation in an extensible browser-based interface, and its case studies demonstrate interactive analyses including sentiment counterfactuals and coreference bias. The tool supports diverse NLP tasks and model frameworks while trading large-scale performance and deep model integration for portability.
Problem
Understanding why NLP models make predictions, when they fail, and how controlled input changes affect them requires combining local, aggregate, and counterfactual analyses.
Method
LIT provides a browser-based toolkit integrating local explanations, aggregate analysis, counterfactual generation, and side-by-side visualization through extensible, framework-agnostic APIs.
Results
LIT supports interactive workflows across classification, seq2seq, language modeling, and structured prediction, including counterfactual sentiment analysis and gender-bias measurement in coreference.
Takeaways & Limitations
LIT enables users to connect individual explanations with dataset-level patterns and counterfactual tests across a broad range of NLP models and workflows.
Takeaways & Limitations
LIT handles about 10,000 examples at once, scales less well than offline tools, and lacks the deep model integration of framework-specific tools.
Abstract
from arXiv · showhide
We present the Language Interpretability Tool (LIT), an open-source platform for visualization and understanding of NLP models. We focus on core questions about model behavior: Why did my model make this prediction? When does it perform poorly? What happens under a controlled change in the input? LIT integrates local explanations, aggregate analysis, and counterfactual generation into a streamlined, browser-based interface to enable rapid exploration and error analysis. We include case studies for a diverse set of workflows, including exploring counterfactuals for sentiment analysis, measuring gender bias in coreference systems, and exploring local behavior in text generation. LIT supports a wide range of models--including classification, seq2seq, and structured prediction--and is highly extensible through a declarative, framework-agnostic API. LIT is under active development, with code and full documentation available at https://github.com/pair-code/lit.
1 Introduction
NLP models achieve strong task performance, but their behavior under shifts, adversarial inputs, social biases, and shallow heuristics remains difficult to understand. LIT addresses this by integrating explanations, aggregate analysis, and counterfactual exploration in an extensible, framework-agnostic interface.
- Researchers need to identify poor-performing examples, explain individual predictions, and test whether predictions remain consistent under controlled input changes.
- Existing workflows often require switching among tools that separately provide aggregate metrics, single-datapoint explanations, or limited model support.Adapting research code or moving between tools can take days of work.
- LIT integrates local explanations, aggregate metrics and visualizations, flexible dataset slicing, counterfactual generation, and side-by-side comparisons in a browser-based UI.Users can add new datapoints and immediately visualize their effects on model behavior.
- LIT is flexible and extensible, supporting classification, seq2seq, language modeling, and structured prediction workflows.
- LIT is framework agnostic and easy to use, supporting Python-run models and requiring only a small amount of code to add models and data.Its components are modular and designed for experimentation and extension.
2 User Interface and Functionality
LIT organizes interactive NLP model analysis through compatible modules in a multi-pane browser interface. Its workflows span dataset exploration, datapoint selection, local explanation, counterfactual generation, comparison, and metric computation.
- LIT combines a browser UI of task-specific modules with interactive visualizations, metrics, and dynamically generated datapoints.Modules respond to user input and support richer analysis than a basic prediction demo.
- Explore the dataset: Users can explore datasets through tables and PCA or UMAP embedding projections, including zooming, panning, and cluster inspection.
- Find interesting datapoints: Users can find datapoints by selecting confusion-matrix groups, embedding outliers, scalar ranges, or saved slices.
- Explain local behavior: Local behavior can be examined with salience maps, local gradients, LIME, and attention-head visualizations.
- Generate new datapoints: New datapoints can be created through manual edits or counterfactual generators such as backtranslation, nearest-neighbor retrieval, word substitutions, and HotFlip.Datapoint provenance is tracked for comparison.
- Compare and compute metrics: LIT supports side-by-side model or datapoint comparisons and computes metrics for datasets, selections, and manual or automatically generated slices.
3 Case Studies
LIT’s case studies show how interactive exploration connects dataset-level metrics, local explanations, counterfactuals, and training-data search across sentiment, coreference, and text-generation workflows.
- Sentiment analysis: In sentiment analysis, BERT correctly classified all 56 SST development examples containing “not,” but counterfactuals exposed sensitivity to wording changes.Removing “not” produced a strongly positive prediction, while replacing “ultimate” with “worst” produced a mildly positive score.
- Sentiment analysis: Salience maps highlighted “not” and “ultimate” as important to the sentiment model’s prediction.
- Gender bias in coreference: The coreference case study compares pronoun sensitivity and error patterns by paging through Winogender examples and slicing results by pronoun type and true referent.
- Gender bias in coreference: On male-dominated Winogender occupations, occupation-reference accuracy was 83% for male pronouns versus 37.5% for female pronouns.The passage defines male-dominated occupations as those with fewer than 25% female representation by BLS.
- Debugging text generation: For text generation, LIT selects a middling-ROUGE CNN-DM example, inspects an erroneous summary, and retrieves similar training examples to investigate the error.The language-modeling module showed token “by” predicted with 28.7% probability; nearest-neighbor examples contained repeated “captain,” “former,” and “replaced by” patterns.
4 System design and components
LIT combines a browser-based, modular interface with framework-agnostic semantic specifications that let models, datasets, interpreters, metrics, and generators interoperate across NLP tasks.
- Architecture: A TypeScript browser UI communicates with a Python backend through NumPy arrays, JSON, and a declarative spec system that minimizes cross-dependencies.
- Frontend: The frontend uses shared service objects to coordinate otherwise independent modules that provide controls and visualizations.
- Components: LIT components include models, datasets, interpreters, generators, and metrics, each exposing focused interfaces for predictions, data, explanations, new inputs, or aggregate scores.
- Components: Components communicate through minimalist APIs and standard Python and NumPy types, supporting TensorFlow and PyTorch while remaining portable to notebooks or standalone scripts.
- Running with your own model: Users can launch LIT by passing model and dataset dictionaries to lit.Server, which can host multiple models and datasets for side-by-side comparison.
- Extensibility: the spec() system: Semantic types describe model and dataset fields so components can infer operations, decode labels, compute metrics, and automatically display compatible modules.
- Extensibility: the spec() system: New semantic types and custom components can be added, such as SpanLabels for named-entity recognition outputs.
5 Related Work
LIT builds on interactive model-analysis and interpretability tools while distinguishing itself by integrating multiple analysis modes, supporting broad workflows, and remaining extensible.
- Prior tools: Existing tools span general-purpose model analysis, fairness applications, counterfactual analysis, and NLP tools specialized to particular model classes.
- LIT’s position: LIT integrates local explanations, aggregate analysis, and counterfactual generation in one tool, similarly to Errudite but with broader workflow and extension support.
- Limitations: LIT’s framework-agnostic design improves simplicity and portability but lacks the deep model integration available in AllenNLP Interpret or Captum.
- Limitations: LIT is intended for evaluation rather than training-time monitoring and handles about 10,000 examples at once, limiting scalability relative to offline tools.
6 Conclusion and Roadmap
LIT integrates interactive visualization and exploration of NLP model behavior across individual datapoints and datasets, while supporting diverse workflows, model types, and extensibility. The project remains open-source and actively developing with planned additions for counterfactuals, metrics, visualizations, and UI customization.
- LIT integrates a UI and components for exploring NLP model behavior at both single-datapoint and whole-dataset levels.
- It supports workflows including individual prediction explanation, disaggregated analysis, and counterfactual bias probing.
- LIT supports diverse model types and techniques out of the box through simple, framework-agnostic APIs.
- Planned additions include counterfactual-generation plug-ins, metrics and visualizations for sequence and structured outputs, and greater UI customization.
- LIT is open-source under an Apache 2.0 license and welcomes community contributions.
A Appendices
The appendices illustrate LIT’s counterfactual-generation workflow, architecture, model and dataset integration, and visual analysis interfaces. These examples show how users can stage and edit generated data, connect diverse model implementations, and inspect predictions with embeddings, metrics, confusion matrices, and salience maps.
- Figure A.1: The counterfactual generator stages generated datapoints for review, allows manual label edits, and can replace “great” with “terrible” across a dataset.
- Figure A.2: LIT’s architecture separates a backend for models, datasets, metrics, generators, interpretations, and caching from a modular TypeScript frontend.
- Figure A.2: Backend components can be passed to lit.Server, while frontend modules are added through layout.ts, which lists modules and their UI positions.
- Figure A.3: A demo can run two NLI models on MultiNLI using implementations wrapped in Python classes, provided they define spec fields and a predict() function returning NumPy arrays.
- Figures A.4–A.5: The UI combines embedding clusters, genre-faceted accuracy metrics, confusion matrices, and prediction salience maps for model inspection and comparison.