Source-linked AI summary
Virgil: Navigating Explainability for Transformer-based Language Models
Martino Ciaperoni, Sezer Kutluk, Benedetta Muscato, Marta Marchiori Manerba, Fosca Giannotti
TL;DR
Explainability tools for transformer-based language models are expanding but difficult to navigate, especially as these models enter high-stakes domains. Virgil addresses this gap with a curated, interactive system for discovering, exploring, executing, and comparing explainers, and is intended to develop into a broader community resource.
Problem
The expanding landscape of explainers is difficult to navigate, while surveys rarely guide users in selecting suitable tools for real-world settings.
Method
Virgil combines a curated knowledge base, query-based retrieval, and an exploration engine for interacting with and comparing explainers.
Results
Virgil provides an interactive system for discovering, exploring, and comparing explainers for transformer-based language models.
Takeaways & Limitations
Virgil is positioned as a resource for practitioners and researchers, including non-experts, and future work will develop it into a central community resource.
Abstract
from arXiv · showhide
Explainability for transformer-based language models is becoming crucial as these systems are deployed in high-stakes applications. As a result, the ecosystem of explainability tools is rapidly evolving, becoming richer, but also more fragmented and harder to navigate. To address this challenge, we present Virgil, an interactive system that lets practitioners and researchers, including non-experts, navigate explainability tools for transformer language models. Supported by a curated knowledge base, the system enables users to discover and compare explainability tools within a unified interface.
1 Introduction
Virgil addresses the difficulty of navigating a rapidly expanding landscape of explainers by helping users identify, inspect, run, and compare tools through an interactive interface.
- Transformer-based language models are deployed in high-stakes domains, but their decision processes remain difficult to interpret.
- The explainability landscape spans approaches from intuitive input attributions to mechanistic interpretability and is challenging to navigate.
- Surveys provide overviews and taxonomies but rarely guide users in selecting suitable explainers for real-world settings.
- Virgil uses structured filters or natural-language queries over a curated knowledge base to help users identify explainers suited to their needs.
- The interface displays matching explainers centrally and detailed information about selected explainers on the right.
2 Virgil Architecture
Virgil combines a curated explainer knowledge base, retrieval engine, and exploration engine in a modular architecture for finding and comparing explainers.
- Virgil is a Python web application built with Streamlit and organized around three modular components.The components are a knowledge base, retrieval engine, and exploration engine.
- The knowledge base stores structured descriptions of explainers represented as explainer cards.
- The retrieval engine identifies explainers matching user requirements and supports ranking by expertise requirements.
- Similarity scores are aggregated through weighted ranking with weights of 0.5, 0.4, and 0.1 assigned to three fields, respectively.
- The exploration engine presents explainer characteristics, can execute explainers with custom inputs and models when available, and supports side-by-side comparison.
3 Using Virgil
Virgil supports no-code online use and local execution, enabling practitioners and researchers to retrieve, explore, and optionally execute explainers for transformer language models.
- Virgil can be used online without coding or run locally with hardware acceleration support.
- Users query the knowledge base, select relevant explainers, explore them, and optionally execute them.
- The system targets practitioners with limited experience, researchers conducting preliminary analyses, and users comparing different explainers.
- Virgil can support educational, training, and collaborative activities involving multiple stakeholders.
- Token-level attribution scores can be computed in Virgil using Input×Gradient and Integrated Gradients.The example input is "The movie was good".
4 Conclusion
The paper presents Virgil as an interactive system for discovering, exploring, and comparing explainers, with future work aimed at expanding it into a community resource.
- Virgil is an interactive system for discovering, exploring, and comparing explainers for transformer-based language models.
- Future work will develop Virgil into a central community resource that organizes explainers for transformer-based language models.
- The authors state that future work will contribute to democratizing explainability and facilitate translating research advances into practice.