Source-linked AI summary
ConvoKit: A Toolkit for the Analysis of Conversations
Jonathan P. Chang, Caleb Chiam, Liye Fu, Andrew Z. Wang, Justine Zhang, Cristian Danescu-Niculescu-Mizil
TL;DR
Conversational analysis lacks a public toolkit specifically designed for its data and social interactions. ConvoKit provides a unified framework, analysis tools, and ready-to-use datasets; the paper presents its user-oriented abstractions as a way to promote reproducibility and adoption.
Problem
Public NLP toolkits are not specifically targeted at conversational analysis, while conversational datasets and methods use fragmented, incompatible formats.
Source: §1 · p. 1, §2 · pp. 1–2Method
Results
The paper presents ConvoKit as an open-source toolkit with intuitive abstractions for representing and manipulating conversational data.
Source: §5 · p. 4Takeaways & Limitations
The authors aim for these abstractions to make conversational analysis accessible to a broad audience and promote reproducibility and adoption.
Source: §1 · p. 1, §5 · p. 4ConvoKit is currently centered on text analysis; spoken dialogs are only indirectly supported as metadata, with first-class support identified as future work.
Source: §5 · p. 4
Abstract
from arXiv · showhide
This paper describes the design and functionality of ConvoKit, an open-source toolkit for analyzing conversations and the social interactions embedded within. ConvoKit provides an unified framework for representing and manipulating conversational data, as well as a large and diverse collection of conversational datasets. By providing an intuitive interface for exploring and interacting with conversational data, this toolkit lowers the technical barriers for the broad adoption of computational methods for conversational analysis.
1 Introduction
ConvoKit addresses the lack of public NLP toolkits specifically designed for conversational analysis by offering a unified framework and compatible datasets. Its design reflects the need to represent interaction structure, integrate speaker information, accommodate varied contexts, and remain accessible beyond NLP specialists.
1 Introduction
- ConvoKit provides an open-source framework for computationally analyzing conversations and their social interactions, alongside compatible conversational datasets.The paper situates this contribution in a gap: existing public NLP toolkits are not specifically targeted at conversational data.Source: §1 · p. 1
- Conversations require representations that capture reply and tree structure, rather than treating utterances as disconnected items.This design principle motivates native support for relationships among utterances.Source: §1 · p. 1
- Because speakers participate in multiple conversations and interpretation depends on their histories, analysis must integrate speaker information and behaviors across conversations.Source: §1 · p. 1
- A toolkit must abstract over interaction formats ranging from dyadic face-to-face exchanges to institutional, online, and large threaded discussions.Source: §1 · p. 1
- The framework is intended for a broad audience, including social scientists and people without extensive NLP expertise.Source: §1 · p. 1
- The paper presents the framework architecture, analysis methods, datasets, and future-development directions.Source: §1 · p. 1
2 Framework Architecture
ConvoKit responds to fragmented formats and incompatible methods with shared abstractions for conversational data and its manipulation. Its Corpus hierarchy represents conversational relationships and metadata, while Transformers provide a common way to analyze and modify data at multiple levels.
2 Framework Architecture
- Conversational datasets use different formats and task-specific schemas, while method code is often ad hoc and incompatible, creating barriers to reproducibility and broader adoption.Source: §2 · pp. 1–2
- Transformers can target utterances, conversations, speakers, or combinations, and return the Corpus with modifications, commonly as added or changed metadata.A shared fit() and transform() API provides a general syntax for preparing a Transformer and applying its computation.Source: §2 · pp. 2–3
- ConvoKit centers on two abstractions: a Corpus collects conversations, and a Transformer performs a computation or action on a Corpus.Source: §2 · p. 2
- The Corpus hierarchy links conversations, utterances, and speakers, with reply relationships and speaker attribution represented in the data model.The hierarchy enables navigation such as traversing conversation trees and following a speaker’s utterance history.Source: §2 · p. 2, Figure 1 · p. 2
- Each hierarchy object supports custom metadata, accommodating information from speaker ages to subutterance-level DAMSL speech-act tags.Source: §2 · p. 2
- Transformers can be chained so later computations use earlier outputs; a summarize() helper can provide tabular or graphical views of computed results.The PolitenessStrategies example plots average occurrences of politeness features.Source: §2 · p. 3
3 Transformers
ConvoKit groups its Transformers into preprocessing, feature extraction, and analysis, with each stage supporting conversational work at different levels. The paper illustrates how these components can be combined for a practical comparison of movie dialogue.
3 Transformers
- Transformers support preprocessing, feature extraction, and analysis of conversational data.Source: §3 · p. 3
- Preprocessing prepares Corpus objects for analysis through steps such as cleaning text, parsing dependencies, merging consecutive utterances by one speaker, or combining a speaker’s multiple user accounts.Source: §3 · p. 3
- Figure 2 demonstrates chaining Transformers to compare language in movie dialogs with characters of different genders against dialogs containing only one gender.Source: Figure 2 · p. 3, §3 · p. 4
- Feature extraction converts utterance text or conversational structure into numerical features, including politeness counts or conversation-level hypergraph measures.HyperConvo extracts measures such as generalized reciprocity and indegree and outdegree distributions.Source: §3 · p. 3
- Analysis combines utterance, conversation, and speaker features or metadata in statistical or machine-learning models, including methods for comparing language subsets and modeling conversation trajectories.FightingWords implements a principled language comparison method; Forecaster models a conversation’s future trajectory.Source: §3 · pp. 3–4
4 Datasets
ConvoKit distributes conversational datasets as ready-to-use Corpus objects and supports extending the collection by converting other datasets. The examples span varied settings, and their diversity is presented as evidence of the representation’s expressiveness.
4 Datasets
- ConvoKit includes diverse datasets already formatted as Corpus objects and ready for use.Source: §4 · p. 4
- The collection spans court transcripts, Wikipedia talk pages, Reddit discussions, and movie dialogs.Source: §4 · p. 4
- Guidelines and code for converting other datasets into ConvoKit format extend its reach beyond the data already offered, while the collection’s diversity demonstrates the representation’s expressiveness.Source: §4 · p. 4
5 Conclusions and Future Work
ConvoKit aims to make conversation analysis accessible through intuitive abstractions for representing and manipulating data, promoting reproducibility and adoption. Its current emphasis is text analysis, with spoken-dialog support and continued expansion planned.
- ConvoKit uses intuitive abstractions for representing and manipulating conversational data to promote reproducibility and adoption.Source: §5 · p. 4
- The toolkit is heavily centered on text analysis; spoken dialogs are only indirectly supported as metadata, and first-class support is a future extension.Source: §5 · p. 4
- The project welcomes contributions ranging from bug fixes and documentation to expanded corpus metadata, new datasets, and analysis methods.Source: §5 · p. 4