Source-linked AI summary
Characterizing and curating conversation threads: Expansion, focus, volume, re-entry
Lars Backstrom, Jon Kleinberg, Lillian Lee, Cristian Danescu-Niculescu-Mizil
TL;DR
Users must allocate attention across evolving discussion threads, and prior research had largely not systematized their algorithmic curation. The paper formalizes length and re-entry prediction, analyzes thread structure, and finds that different features are most useful for predicting each outcome.
Problem
Thread curation requires helping users allocate attention among discussions, but the problem has largely not been systematized in prior research.
Source: §1 · p. 1Method
The authors analyze thread structure, including arrival patterns, social links, and timing, and use these properties in learning-based length and re-entry prediction.
Source: §1 · pp. 1–2Results
Arrival-pattern structure is most useful for re-entry prediction, while temporal properties together with arrival patterns give the strongest length prediction.
Source: §1 · p. 2Takeaways & Limitations
Length prediction can inform whether to surface a thread, while re-entry prediction can inform whether to keep a contributing user notified.
Source: §1 · p. 1The generative model is difficult to analyze rigorously, and proving bimodality and its transition with α remains open.
Source: §7 · p. 9
Abstract
from arXiv · showhide
Discussion threads form a central part of the experience on many Web sites, including social networking sites such as Facebook and Google Plus and knowledge creation sites such as Wikipedia. To help users manage the challenge of allocating their attention among the discussions that are relevant to them, there has been a growing need for the algorithmic curation of on-line conversations --- the development of automated methods to select a subset of discussions to present to a user. Here we consider two key sub-problems inherent in conversational curation: length prediction --- predicting the number of comments a discussion thread will receive --- and the novel task of re-entry prediction --- predicting whether a user who has participated in a thread will later contribute another comment to it. The first of these sub-problems arises in estimating how interesting a thread is, in the sense of generating a lot of conversation; the second can help determine whether users should be kept notified of the progress of a thread to which they have already contributed. We develop and evaluate a range of approaches for these tasks, based on an analysis of the network structure and arrival pattern among the participants, as well as a novel dichotomy in the structure of long threads. We find that for both tasks, learning-based approaches using these sources of information yield improvements for all the performance metrics we used.
1. INTRODUCTION
The paper frames thread curation as helping users allocate attention, then formalizes length and re-entry prediction as two component tasks. It analyzes thread structure and uses learning-based methods, finding different features most useful for each task.
1. INTRODUCTION
- The study evaluates length and re-entry prediction on Facebook and Wikipedia threads, addressing curation problems that prior research had largely not systematized.Earlier length studies primarily concerned blog and news threads, which the authors describe as essentially expansionary; re-entry had not, to their knowledge, been studied.Source: §1 · pp. 1–2
- Length prediction estimates eventual thread length from its initial portion; re-entry prediction estimates whether an identified commenter will contribute again.Length serves as a proxy for the interest a thread may generate, while re-entry relates to whether a participant should remain notified.Source: §1 · p. 1
- Long threads show a two-type pattern: some have few distinct commenters, while others have many commenters who generally do not return.The authors present this as evidence that participants’ interest in returning varies across active discussions.Source: §1 · p. 2
- The authors analyze arrival patterns, social links among early commenters, and how quickly commenters arrive to characterize threads’ future trajectories.Arrival patterns capture how new entrants and returning participants are interleaved.Source: §1 · p. 2
- Arrival-pattern structure is most useful for re-entry prediction, while temporal properties together with arrival patterns give the strongest length prediction.These are the authors’ high-level findings about the features that best predict each outcome.Source: §1 · p. 2
- A probabilistic model built from assumptions about turn-taking and new entrants can produce focused and expansionary threads across different random realizations.The model connects discussion threads with nonlinear urn processes.Source: §1 · p. 2
2. DATA AND BASIC DEFINITIONS
The study uses Facebook and Wikipedia discussions as contrasting settings for evaluating conversation-thread properties. It defines thread terminology and draws on two Facebook user populations and a corpus of Wikipedia editor discussions.
2. DATA AND BASIC DEFINITIONS
- The study draws on Facebook and Wikipedia discussions, including socially oriented conversations and task-oriented editor discussions.The Wikipedia conversations are treated as comment threads despite lacking formal interface structure.Source: §2 · p. 2
- A full thread consists of the initiating post and its comments, and its length or volume counts all items, including the post.Comment-thread length excludes the post.Source: §2 · p. 2
- The Facebook data includes 100,000 randomly selected US users and 100,000 heavily engaged users, with threads from their posts over the same 80-day period.The heavily engaged sample posted 200–300 status updates during that period; the data was anonymized and analyzed in aggregate.Source: §2 · p. 2
- The Wikipedia corpus contains 118,447 editor discussion threads of at least one comment, from talk-pages hosted by 6,555 highly active editors.Posts average 2.12 comments; automated-bot posts without responses were excluded.Source: §2 · p. 2
3. FOCUSED VS. EXPANSIONARY THREADS
The section tests whether long threads divide into focused discussions with few repeat participants and expansionary discussions with many mostly one-time commenters. It reports evidence for this pattern and explains its relevance to re-entry prediction.
3. FOCUSED VS. EXPANSIONARY THREADS
- Focused threads involve a small group in extensive conversation, whereas expansionary threads grow through many users who each generally comment once.The proposed dichotomy concerns the structure of long threads, not simply two extremes in a broad distribution.Source: §3 · p. 3
- The analysis macro-averages across users and parametrizes distinct-participant counts by thread-prefix length to avoid domination by high-volume users and account for thread length.For each user, it averages distinct participants across qualifying prefixes, then measures the fraction of users whose average floors to d.Source: §3 · p. 3
- Facebook shows peaks at d=4 and near k, while grouped Wikipedia values also support two local maxima; for k≥15, extreme density intervals exceed middle ones.The Facebook pattern appears in both uniform and high-activity populations; the Wikipedia effect is harder to discern because its data volume is smaller.Source: §3 · p. 3
- The distinction motivates re-entry prediction, which estimates whether a particular participant will comment again rather than whether a thread overall is focused or expansionary.A thread may be focused through back-and-forth among other participants even when a given user’s probability of returning is low.Source: §3 · p. 3
- Re-entry prediction covers whether a user will write another comment, not whether they will continue reading comments without intending to contribute.The authors identify predicting comment consumption as an open question.Source: §3 · p. 3
4. EARLY PARTICIPANTS: SOCIAL, SEQUEN-TIAL, AND TEMPORAL STRUCTURES
Early social ties, arrival sequences, and timing patterns reveal information about thread length and participant re-entry. Their relationships vary by setting, while arrival patterns and timing provide useful foundations for prediction.
4.1 Links Among the Initial Participants
- Linked first commenters correspond to longer Facebook comment threads but fewer distinct commenters, while links shorten like threads where repeated interaction is absent.The authors suggest that Wikipedia’s task-oriented discussions may be less discursive, but note that the available data are sparse.Source: §4.1 · p. 4
- More links among the first distinct commenters correspond to longer Facebook comment threads but shorter like threads; Wikipedia threads show no corresponding increase, though data are sparse.For Facebook, expected comment-thread length increases almost monotonically with the number of edges among the first k distinct commenters, and the first participant’s re-entry also increases strongly with those edges.Source: Figure 2 · p. 4, §4.1 · p. 4
4.2 Arrival Patterns
- Arrival patterns encode each commenter’s identity and can represent prefixes of a thread, distinguishing back-and-forth participation from mostly one-time arrivals.The post author receives code 0; other commenters are coded by the order in which they first appear.Source: §4.2 · p. 4
- Length-two arrival patterns have mostly disjoint 95% confidence intervals for mean thread length, and patterns with many distinct IDs tend to have lower re-entry probabilities.For Facebook, more appearances of ID code 1 correlate with a greater likelihood that this commenter re-enters; the two most frequent patterns are (1,0) and (1,2), reflecting turn-taking and successive new arrivals.Source: §4.2 · p. 5, §4.3 · p. 5, Figure 4 · p. 5
4.3 Timing effects
- A longer wait for the first comment corresponds to a shorter thread, while first-comment timing eventually becomes approximately independent of the first commenter’s re-entry probability.The analysis uses temporal structure alongside arrival patterns in the subsequent prediction methods.Source: §4.3 · p. 5
5. PREDICTING THREAD LENGTH
The section predicts eventual thread length from an early thread prefix, using participant, arrival-pattern, timing, textual, and other features. The combined feature set outperforms two baselines across all five reported metrics, while timing and recent comments stand out in feature analyses.
5. PREDICTING THREAD LENGTH
- A longer wait for the first comment corresponds to a shorter expected thread, while first-comment arrival time eventually becomes independent of that commenter’s re-entry probability.Figure 5 presents the two relationships on separate axes.Source: Figure 5 · p. 6
- The prediction tasks use early thread states: after five Facebook comments, predict whether the thread reaches eight comments; after two Wikipedia comments, predict whether it gets a third.The test sets contain 1,996,624 Facebook posts and 44,732 Wikipedia posts; the Facebook Uniform results are similar but have larger error bars.Source: §5 · p. 6
5.1 Features (used here and in §6)
- The features span participant links, arrival patterns, timing, original-post text, and miscellaneous properties; indexed features are also built for the original post when applicable.Text features were selected by text regression, while the feature table marks some features as Facebook-only.Source: §5.1 · p. 6, Table 2 · p. 6
5.2 Performance Results
- The combined feature set performs best on all five metrics against the positive-percentage-bias and text-regression baselines.Table 3 reports the main thread-length results; the supplied text does not give its metric values.Source: §5.2 · p. 6, Table 3 · p. 7
- In Facebook feature selection, timing for the first five comments is highly indicative of reaching eight comments, and most key features come from the fifth comment.The later the single comment used for prediction, the more informative its features; link-based features have little effect, which the authors attribute to their low recall.Source: §5.2 · p. 7, Figure 6 · p. 7
- For Facebook, removing TIME:time[5] drops AUC from 0.729 to 0.588, and the other features do not make up for losing temporal information.The relative ordering of the other features remains more or less unchanged, including when predicting only within TIME:time[5] ∈[15m, 20m).Source: §5.2 · p. 7
- For Wikipedia, the second comment’s length is the leading feature, followed by its arrival time and the second and first commenters’ ID codes; the dataset is too small to assess other features clearly.The text-regression features sometimes perform worse than the positive-percentage-bias baseline.Source: §5.2 · p. 7
6. PREDICTING THREAD RE-ENTRY
The section introduces predicting whether an identified commenter will contribute again, using the initial thread and commenter identity. In Facebook cross-validation, the full feature set reaches AUC 0.855 with a nine-comment prefix and 0.808 with a five-comment prefix.
6. PREDICTING THREAD RE-ENTRY
- Re-entry prediction asks whether a known commenter will contribute another comment later, helping determine whether to keep that user notified about the thread.The section presents an overview of results and omits detailed feature analysis for space.Source: §6 · p. 7
- Facebook full-feature cross-validation reaches AUC 0.855 for the 9-comment task and 0.808 for the 5-comment task.The task asks whether the first commenter reappears after occurring in the initial arrival-pattern prefix.Source: §6 · p. 7, Table 5 · p. 8
- Commenter identity—especially identities among the most recent commenters—is most important, while the gap between the two latest comments also matters.The authors relate a longer gap to a slower-moving conversation and a greater likelihood that it will end.Source: §6 · p. 7
7. MODELING THREAD RE-ENTRY
The section examines generative models of participant arrival patterns to understand the observed bimodality in distinct commenters. A weight-updating model produces bimodality in simulation, while a broad class of fixed new-participant probability models is proved to produce only unimodality.
7. MODELING THREAD RE-ENTRY
- The section develops models to test whether arrival-pattern dynamics can produce the observed bimodality in distinct commenters.The authors first consider class F, then turn to alternatives after its unimodality result.Source: §7 · p. 8
- The first model family F assigns each comment a fixed probability of introducing someone new and otherwise selects a re-entrant through a rule θ.The rule can select uniformly, favor frequent commenters, or favor recent commenters; each model is specified by thread length, θ, and the probability sequence.Source: §7 · p. 8
- The class F cannot reproduce the empirically observed focused-versus-expansionary bimodality: every model in F has a unimodal distribution of distinct participants.The theorem applies to arbitrary thread length, re-entrant selection rule, and sequence of new-participant probabilities within the class.Source: §7 · p. 8
- A mixture of two post types can build bimodality into a model, but the authors note that this is not the whole explanation.For the same shared news link, Facebook threads showed both drive-by comments and extended small-group discussions.Source: §7 · p. 8
- The authors motivate the dynamic model as producing expansion/focus bimodality without assuming post type causes it, using internal symmetry-breaking inspired by nonlinear urn processes.The probability of a new participant depends on the identities of earlier participants.Source: §7 · p. 8
- The proposed process chooses between existing and new participants using weights; re-entry boosts a participant’s weight, whereas a new arrival reduces existing participants’ weights.A new arrival can suppress re-entry and promote further arrivals; a re-entrant’s increased weight promotes repeated back-and-forth interaction.Source: §7 · pp. 8–9
- In simulation, the proposed weight-update process develops stronger bimodality as α increases, with thread length 40 and β = 1.The illustrated settings use α = 1.5, 2, and 4; larger α amplifies both focused and expansionary effects.Source: §7 · p. 9, Figure 7 · p. 9
- A rigorous proof of bimodality for the proposed model, and characterization of its transition from unimodality as α increases, remain open questions.The model is described as challenging to analyze rigorously.Source: §7 · p. 9
8. RELATED WORK
The related-work section situates conversational curation among research on comment interest, volume prediction, information diffusion, and discussion structure. It identifies re-entry prediction as a novel focus within the broader curation problem.
8. RELATED WORK
- The authors state that prior work had not considered algorithmic conversation curation as an overall problem, and identify re-entry prediction as novel to their knowledge.They distinguish selecting interesting posts, deciding which known discussions merit continued updates, and understanding discussion structure.Source: §8 · p. 9
- Related approaches address comment-thread interestingness, predicting comment volume, information diffusion, post quality, and discussion structure.The cited work on discussion structure includes tree-based characterizations, sentiment or agreement, and sociological turn-taking.Source: §8 · p. 9
- For Facebook, distinctive original-post text is associated with more comments, whereas the same effect is absent for task-oriented Wikipedia posts.Figure 8 considers only posts of at least eight words that generated responses.Source: Figure 8 · p. 9
9. FUTURE WORK: DISTINCTIVENESS
The section explores whether distinctive post text and first commenters relate to thread length. These features are presented as preliminary explorations, without an identified way to use them to improve prediction.
Future Directions
- The authors had not identified a way to apply text or early-commenter distinctiveness to improve prediction performance.Source: §9 · p. 9
Distinctiveness
- Unusual wording might provoke discussion or be ignored as hard to understand; an unfamiliar first commenter may signal greater reach, while a familiar one may ease growth or be passed over.Source: §9 · pp. 9–10
Distinctiveness of Text
- For posts of at least 8 words that received comments, text distinctiveness positively tracked macro-averaged post length in both Facebook populations, but showed no apparent Wikipedia effect.The post subsets were restricted to posts with at least one Facebook comment or two Wikipedia comments.Source: §9 · p. 9
- Post distinctiveness is the average token log(1/p(w)) from a unigram model trained on 3.5 million other Facebook posts; lower values indicate more likely text.Source: §9 · p. 9
Distinctiveness of First Commenter
- Facebook threads were longer when the first commenter rarely responded first to that poster; Wikipedia showed a significant decreasing trend among users with at least 10 posts.The Wikipedia trend was observed after binning distinctiveness values; the figure depicts only Facebook.Source: §9 · p. 10, Figure 9 · p. 10
10. CONCLUSIONS
The paper identifies length and re-entry prediction as key problems in conversational curation, while emphasizing that long threads differ in their participant structure. It points to early-trajectory prediction and system integration as future directions.
10. CONCLUSIONS
- Length prediction is complicated by long threads that may focus on a few participants or expand to many.Source: §10 · p. 10
- Re-entry prediction, not previously formulated to the authors’ knowledge, addresses notification timing for users who have participated in a discussion.Source: §10 · p. 10
- Further work includes identifying early features that predict discussion trajectories and integrating these techniques into systems delivering discussion-oriented content.Source: §10 · p. 10