Source-linked AI summary
Toxicity Ahead: Forecasting Conversational Derailment on GitHub
Mia Mohammad Imran, Robert Zita, Rahat Rizvi Rahman, Preetha Chatterjee, Kostadin Damevski
TL;DR
Toxicity harms OSS communities, while manual proactive moderation is difficult to scale and existing detection is typically post-hoc. The paper forecasts GitHub conversation derailment using Least-to-Most-generated conversation-dynamics summaries, achieving strong results on its curated data and external GitHub threads.
Problem
Existing toxicity detection methods are post-hoc, while manual proactive moderation is difficult to sustain across OSS communication channels and automated systems must account for subtle, context-dependent behaviors.
Source: §1 · p. 1Method
The framework uses Least-to-Most prompting to generate Summaries of Conversation Dynamics from pre-toxicity comments, then estimates the likelihood of derailment.
Source: §1 · p. 2Results
At threshold 0.3, LtM achieved F1-scores of 0.901 (Qwen) and 0.852 (Llama), outperforming CRAFT (best F1-score 0.580); external Qwen validation reached 0.797.
Source: §5.4 · p. 8, §6.2 · p. 10Takeaways & Limitations
The explainable SCD-based approach supports transparent moderation decisions and threshold-based interventions for early warning in OSS discussions.
Source: §10 · p. 11The findings are limited to GitHub issue threads and a relatively small curated dataset, and possible overlap with LLM pretraining data may slightly inflate performance estimates.
Source: §9 · p. 11
Abstract
from arXiv · showhide
Toxic interactions in Open Source Software (OSS) communities reduce contributor engagement and threaten project sustainability. Preventing such toxicity before it emerges requires a clear understanding of how harmful conversations unfold. However, most proactive moderation strategies are manual, requiring significant time and effort from community maintainers. To support more scalable approaches, we curate a dataset of 159 derailed toxic threads and 207 non-toxic threads from GitHub discussions. Our analysis reveals that toxicity can be forecast by tension triggers, sentiment shifts, and specific conversational patterns. We present a novel Large Language Model (LLM)-based framework for predicting conversational derailment on GitHub using a two-step prompting pipeline. First, we generate \textit{Summaries of Conversation Dynamics} (SCDs) via Least-to-Most (LtM) prompting; then we use these summaries to estimate the \textit{likelihood of derailment}. Evaluated on Qwen and Llama models, our LtM strategy achieves F1-scores of 0.901 and 0.852, respectively, at a decision threshold of 0.3, outperforming established NLP baselines on conversation derailment. External validation on a dataset of 308 GitHub issue threads (65 toxic, 243 non-toxic) yields an F1-score up to 0.797. Our findings demonstrate the effectiveness of structured LLM prompting for early detection of conversational derailment in OSS, enabling proactive and explainable moderation.
1 Introduction
Toxicity harms OSS communities, while manual proactive moderation is difficult to scale and automated detection must account for GitHub’s subtle, context-specific interactions. The paper characterizes early derailment signals and introduces an LLM framework to forecast derailment before toxic comments appear.
1 Introduction
- Toxic interactions harm online communities, and existing detection methods are post-hoc, identifying toxic content only after it appears.Manual proactive moderation is difficult to sustain across OSS channels, while automated systems must handle subtle behaviors and domain-specific language.Source: §1 · p. 1
- The study investigates how GitHub conversations derail into toxicity and develops an automated LLM-based approach to predict derailment for proactive moderation.Its research questions address derailment patterns, LLM prediction, and generalization to independent GitHub data.Source: §1 · pp. 1–2
- Derailment often precedes toxicity by a median of 3 comments, and 64% of toxic comments occurred within 24 hours.Early warning signals include reasoning terms, WH-questions, second-person pronouns, and tones such as Frustration or Impatience.Source: §1 · p. 2
- The framework uses Least-to-Most prompting to create Summaries of Conversation Dynamics, then estimates derailment likelihood from comments preceding the first toxic remark.The summaries abstract technical details to foreground interaction patterns, emotional tone, and rhetorical shifts.Source: §1 · p. 2
- F1-scores of 0.901 (Qwen) and 0.852 (Llama) show the framework outperforming CRAFT and Hua et al.’s few-shot SCD approach while remaining interpretable.Ablations identify sentiment evolution and tension triggers as especially critical for prediction accuracy.Source: §1 · p. 2
- External validation on Raman et al.’s dataset achieved F1-scores of 0.797 (Qwen) and 0.776 (Llama), despite differences in collection methods, time periods, and class distributions.The authors interpret this as evidence that the method captures broader patterns of GitHub conversational dynamics.Source: §1 · p. 2
2 Example of Conversational Derailment on GitHub
The section illustrates how a GitHub discussion can shift from productive conversation toward toxicity, with a derailment point preceding the first toxic comment.
2 Example of Conversational Derailment on GitHub
- Derailment points are preceding comments where a conversation has moved away from productive discussion toward negativity.The example attributes the shift to a contributor’s mocking comment before the external participant responds with frustration.Source: §2 · pp. 2–3
- A toxic conversation can include an initiating comment, a derailment point, a first toxic comment, and zero or more subsequent comments.The described example moves from a mocking comment to frustration and then an insulting remark, followed by another toxic comment.Source: §2 · pp. 2–3
- Table 1 presents definitions and examples of uncivil tone-bearing discussion features.Source: Table 1 · p. 3
3 Dataset
The authors assemble toxic, non-toxic, and independently annotated GitHub threads, then identify derailment points and exclude sudden-toxicity cases from the forecasting dataset.
3.1 Toxic Conversations Dataset
- The toxic-thread data starts from GitHub conversations locked for reasons including heated discussion, spam, or off-topic content, with threads toxic from the initial post excluded.GPT-4o assessed uncivil comments using preceding conversation context; two authors reviewed flagged comments, with initial Cohen’s Kappa of 0.78. The authors also draw on an existing incivility dataset and distinguish incivility from toxicity.Source: §3.1 · p. 3
3.2 Non-Toxic Conversations Dataset
- The non-toxic comparison threads were sampled immediately before and after toxic threads within the same repositories, then screened for non-toxic content.The final set contains 207 non-toxic threads; 23 threads identified as toxic during screening were added to the toxic collection.Source: §3.2 · p. 3
3.3 Derailed Toxic Dataset
- Annotators used uncivil features as a guide but also considered negative politeness, and excluded sudden-toxicity threads without an identifiable derailment point.They retained 159 toxic threads after excluding 34 sudden-toxicity cases and five cases with unresolved disagreement; 88 retained threads had multiple derailment points and 71 had one.Source: §3.3 · pp. 3–4
3 Dataset
- The final Derailed Toxic Dataset contains 159 toxic threads with 382 derailment points, alongside a separate set of 207 non-toxic threads.The dataset description emphasizes annotation reliability and examples representative of OSS discourse.Source: §3.2 · p. 3, §3 · p. 3, §3.3 · p. 4
3.4 Raman et al.’s Toxicity Dataset
- The independent Raman et al. dataset contributes 308 issue threads with comment-level toxic locations and no overlap with the study’s dataset.The evaluation set has 65 toxic and 243 non-toxic threads after excluding threads without comment-level annotations and those beginning with toxicity.Source: §3.4 · p. 4
4 RQ1: What are the characteristics and patterns of conversational derailment in GitHub discussions?
GitHub derailment is often a short step before toxicity, but it can also precede it by days. Linguistic cues, incivility, and technical or communication problems characterize how discussions move off track.
4.1 Timing and Distance to Derailment Points
- The median gap between the first derailment point and first toxic comment is 3 comments; 64% of toxic comments occur within 24 hours, though over 25% follow after more than 7 days.The median first derailment point is comment 4 and the median first toxic comment is comment 8.Source: §4.1 · p. 4
4.2 Linguistic Features
- Derailment-point comments show more second-person targeting, negation, and questioning than regular comments, with reasoning terms common in both derailment and toxic comments.Compared with regular comments, derailment points have +16.8% second-person targeting, +14.9% negation, and +13.2% questioning; reasoning occurs in 70.4% of both derailment and toxic comments.Source: §4.2 · p. 5
4.3 Incivility TBDFs in Derailment Points
- Bitter Frustration (42.82%) and Impatience (22.65%) are more common at derailment points than in toxic comments, while Insulting and Vulgarity are less common.Bitter Frustration and Impatience occur in toxic comments at 24.65% and 9.15%, respectively; Insulting and Vulgarity occur at 5.80% and 2.49% at derailment points.Source: §4.3 · p. 5, Table 3 · p. 5
4.4 Derailment Triggers
- The most frequent trigger is failed use of a tool, code, or error messages (23.27%), followed by technical disagreement (20.12%) and communication breakdown (16.98%).Two annotators achieved Cohen’s Kappa of 0.84; communication breakdown included misunderstandings, misinterpretations, typos, and language barriers.Source: §4.4 · pp. 5–6
5 RQ2: Can Large Language Models effectively predict conversational derailment on GitHub?
The paper adapts conversation-dynamics summaries for GitHub and uses a two-step LLM prompting process to forecast derailment from interaction patterns rather than technical content.
5.1 Baseline Models
- The evaluation compares the SCD-based approach with CRAFT, a neural-network derailment predictor, and Hua et al.’s few-shot SCD prompting strategy [26].CRAFT is described as an established baseline for conversational derailment prediction [9].Source: §5.1 · p. 6
5.2 LLM Prompt Design
- The two-step prompt first summarizes interaction dynamics, emotional tone, and discourse strategies, then predicts derailment probability from the summary alone.The stages separate reasoning from classification and are intended to build explainability into the pipeline.Source: §5.2 · p. 6
5.2.1 GitHub-Specific SCD Prompt.
- The GitHub-specific SCD prompt adapts Hua et al.’s examples [26] to GitHub interactions and excludes technical details in favor of participant responses, misunderstandings, and tone shifts.Least-to-Most prompting decomposes summarization into steps, incorporating intentions, conversational features, sentiment and tone, and tension triggers before synthesizing the trajectory summary.Source: §5.2.1 · pp. 6–7
5.2.2 Derailment Prediction Prompt.
- A follow-up prompt asks the model to assign a 0–1 derailment probability based on the generated conversation summary.The prediction is based on the summary’s interaction patterns and tone changes, not technical aspects or specific arguments.Source: §5.2.2 · p. 7
5.3 Experiment Setup
The experiments evaluate two openly available LLMs on conversation context preceding toxicity. Least-to-Most SCD prompting yields the strongest reported F1 results at the 0.3 threshold.
5.3 Experiment Setup
- Experiments use the Llama 3.3 70B and Qwen 2.5 32B instruct models, with temperature set to 0 and a 32k context window.The models are freely available; the authors cite reproducibility and deployment cost as reasons for avoiding proprietary models.Source: §5.3.1 · p. 8
5.3.1 Large Language Models.
- For toxic conversations, the model receives comments up to but excluding the first toxic comment, so it predicts from pre-toxicity context.Conversations exceeding the context window are truncated from the beginning.Source: §5.3.1 · p. 8
5.4 Results and Discussion
- At θ=0.3, LtM achieves F1-scores of 0.901 (Qwen) and 0.852 (Llama), outperforming CRAFT and exceeding Hua et al.’s SCD strategy at most thresholds.At thresholds 0.3–0.7, the approach combines high precision with good recall.Source: Table 4 · p. 8, §5.4 · p. 8
- The authors envision higher thresholds for moderator review and lower thresholds for automated reminders promoting civil discourse.Their results emphasize the practical balance between precision and recall at thresholds of 0.3–0.7.Source: §5.4 · pp. 8–9
5.5 Error Analysis
Ablations identify sentiment and tone features and tension triggers as the strongest contributors to predictive performance, while error analysis finds missed subtle toxicity and overestimated tension among the main failure patterns.
5.5 Error Analysis
- Among 30 reviewed errors, false positives often overestimated civil tension, while false negatives missed subtle toxicity or delayed escalation.The false negatives included missed frustration, sarcasm, and nuanced tones.Source: §5.5 · p. 9
5.6 Ablation Study
- The ablation study tested feature contributions using Qwen at θ=0.3, the best-performing configuration in the experiments.The LtM prompt features include sentiment and tone, individual intentions, conversation features, and tension triggers.Source: §5.6 · p. 9
5.6.1 Prompt Modifications Per Component.
- Each ablation removed one semantic component while preserving other prompt steps, then compared precision, recall, F1, and BH-corrected McNemar significance.The tested removals were II, CF, STF, and TT.Source: §5.6.1 · p. 9, §5.6.2 · p. 9
5.6.2 Results.
- Removing STF reduced F1 by -7.8% and recall by -10.2%; the prediction change was significant (p=0.0032).The study characterizes STF as critical for recall-oriented classification performance.Source: §5.6.2 · p. 9
- Removing TT reduced F1 by -6.5% and recall by -8.8%, with a significant prediction shift (p=0.0190).The authors attribute part of the predictive performance to tension-trigger features.Source: §5.6.2 · p. 9
- Removing II reduced F1 by -3.6% and removing CF by −3.7%, but neither prediction change was statistically significant.II removal raised precision by +1.8% while lowering recall by -8.1%; CF removal changed precision by -0.4% and recall by -6.6%.Source: §5.6.2 · p. 9
6 RQ3: To what extent does the proposed LLM-based derailment prediction approach generalize to independent GitHub datasets?
The study evaluates LtM derailment prediction on an independent GitHub dataset with different annotation procedures. LtM achieves the highest reported F1-scores for both evaluated models on that benchmark.
6.1 Dataset and Experimental Setup
- The evaluation uses Raman et al.'s independent dataset, filtering to 308 threads with comment-level annotations and excluding six threads toxic from the first comment.The final set contains 65 toxic and 243 non-toxic threads; the authors note that sudden toxic conversations may remain unverified.Source: §6.1 · p. 10, §6 · p. 10
6 RQ3: To what extent does the proposed LLM-based derailment prediction approach generalize to independent GitHub datasets?
- On this dataset, the study evaluated two prompt-based techniques using Llama-3.3:70B and Qwen-2.5:32B-Instruct at thresholds 0.1, 0.3, 0.5, and 0.7, reporting precision, recall, and F1-score.Llama exhibited more stable performance across thresholds, while lower thresholds increased recall at the expense of precision.Source: §6.1 · p. 10, §6.2 · p. 10
6.2 Results and Discussion
- LtM achieved the highest F1-score on the independent benchmark: 0.797 for Qwen at threshold 0.3 and 0.776 for Llama at threshold 0.5.Qwen outperformed Llama on this benchmark, and the results reaffirmed LtM's performance over Hua et al.'s few-shot SCD prompt.Source: §6.2 · p. 10
7 Recommendations
The recommendations focus on integrating early-warning summaries into moderation workflows while calibrating thresholds and addressing generalization, efficiency, and transparency.
Recommendations for GitHub Maintainers
- Maintainers can integrate LLM-based early-warning systems into moderation pipelines to flag discussions for timely intervention without exhaustive manual review.The paper also suggests adding a dedicated derailment tag to GitHub's existing tagging options.Source: §7 · p. 10
- Thresholds should reflect moderation goals: higher thresholds suit passive alerts, while intermediate thresholds can trigger automated reminders.The suggested ranges are > 0.7 for passive alerts and 0.3–0.7 for reminders.Source: §7 · p. 10
- Because 64% of toxic exchanges occur within 24 hours of derailment, more frequent runs are suggested for new or rapidly evolving threads.Hourly runs are suggested for those threads, while running after each new comment may suffice for slower discussions.Source: §7 · p. 10
Recommendations for Researchers
- Researchers should test whether tonal shifts, tension triggers, and sentiment trajectories generalize across diverse OSS communities, languages, and moderation norms.The paper also calls for reducing prompt sensitivity and improving reproducibility.Source: §7 · p. 10
- Standardized benchmarks across platforms such as GitHub and BugZilla should support consistent model comparisons.The proposed benchmarks would include annotated derailment points, incivility categories, and moderation outcomes.Source: §7 · p. 10
- Efficiency and scalability remain challenges; incrementally updating summaries could enable near-real-time assessment with reduced computational cost.The paper also recommends evaluating latency, resource trade-offs, and feedback dynamics with human moderators.Source: §7 · pp. 10–11
8 Related Work
Prior research covers toxicity in software engineering and conversational derailment prediction, but existing toxicity tools are post-hoc and derailment approaches often lack adaptation to technical forums. This work extends those lines with structured LLM prompting tailored to GitHub.
Toxicity analysis in SE artifacts
- Software engineering research has examined incivility, emotional tone, toxicity, and links between toxic interactions and contributor engagement.Prior work has linked toxic interactions to developer stress and increased dropout rates.Source: §8 · p. 11
Conversation derailment
- Existing software-engineering toxicity tools address toxicity post-hoc, while prior derailment prediction has centered largely on Wikipedia and Reddit.The paper builds on CRAFT's early detection and Hua et al.'s SCD approach, extending SCDs with GitHub-specific dynamics.Source: §8 · p. 11
- Prior approaches often lack domain adaptation for technical forums, where toxicity can emerge through subtle signals such as entitlement or miscommunication.The paper's contribution is structured, LLM-driven prompting tailored to GitHub, intended to provide performance and explainability in derailment forecasting.Source: §8 · p. 11
9 Threats to Validity
The study identifies construct, internal, and external validity threats involving how derailment is captured, how annotations and LLM outputs vary, and how broadly findings may generalize.
Construct validity
- Construct validity is limited because some toxic behavior arises abruptly without prior conversational signals, while human judgments and LLM-generated summaries may vary or miss subtle tone shifts.The authors used detailed annotation guidelines, high inter-annotator agreement, and error analyses to mitigate these risks.Source: §9 · p. 11
Internal validity
- Internal validity may be affected by stochastic LLM generation, prompt sensitivity, and possible human annotation error or bias.The authors addressed these risks with model-in-the-loop annotation, cross-checking, and systematic prompt design.Source: §9 · p. 11
External validity
- External validity is bounded by a relatively small dataset of GitHub issue threads, possible overlap with LLM pretraining data, and untested differences across platforms and communities.The authors call for validation on unseen platforms or domains.Source: §9 · p. 11
10 Conclusion
The paper presents an LLM-based approach for forecasting GitHub conversational derailment and reports strong benchmark and external-validation results. It frames the approach as a basis for proactive moderation while identifying future directions for improving and extending it.
10 Conclusion
- F1-scores of 0.901 with Qwen and 0.852 with Llama outperform established baselines, while independent-dataset external validation yields F1-scores up to 0.797.The study analyzes 159 derailed toxic conversations and 207 non-toxic conversations.Source: §10 · p. 11
- The authors conclude early derailment prediction is feasible for moderating technical discussions, with explainable summaries supporting transparent decisions and threshold-based interventions.They note limitations in platform specificity and sudden toxicity without warning signs.Source: §10 · p. 11
- Future work includes extending evaluation to other platforms, incorporating behavioral signals, and testing live deployment and intervention outcomes.The authors also suggest generating human-written GitHub conversation summaries and instruction-tuning LLMs to improve summary generation.Source: §10 · pp. 11–12