PaperlayerView exact revision on arXiv

Source-linked AI summary

Taming the Online Software Engineering Tongue: Detecting and Reducing Offensive Language in Software Communities

Jithin Cheriyan, Bastin Tony Roy Savarimuthu, Stephen Cranefield

arXiv:2610.19961v1cs.SE

TL;DR

To address offensive language in software engineering comments, the paper builds OLRS-SE to detect and classify offenses, explain them, and offer paraphrases. Across six tasks, the best models split between BERT and SVM, and feedback from 30 participants was generally positive.

  • Problem

    The paper investigates offensive language in software engineering communities because hostility is associated with mental-health issues and members leaving Stack Overflow.

    Source: §1 · p. 2

  • Method

    OLRS-SE combines machine-learning detection and classification with explanations of offensive content and paraphrasing suggestions.

    Source: §5 · p. 29

  • Results

    BERT-based models led three of six tasks, including RO detection at F1-score 0.92, while politeness-feature SVMs led the other three; 30 participants' feedback was generally positive.

    Source: §5 · pp. 29, 31

  • Takeaways & Limitations

    The paper emphasizes the design, integration, and evaluation of a model-agnostic OLRS-SE architecture rather than any single language model.

    Source: §5 · p. 31

    The system considers English comments only and cannot detect offenses in other languages.

    Source: §5.1 · p. 33

Abstract

from arXiv · show

Online Software Engineering (SE) communities such as Stack Overflow have become unwelcoming and hostile spaces, as evidenced by the presence of rude and offensive comments that are detrimental to the emotional wellbeing of participants. The main objective of this work is to investigate the prevalence and nature of offensive language in online SE communities, and develop approaches to classify them with the ultimate goal of reducing offensive language by proposing an Offensive Language Reduction System (OLRSSE) that has the ability to detect, classify, and explain to the poster why a comment is offensive and also propose non-offensive alternative suggestions. Using data from 130,000 user comments on discussions held in four prominent SE platforms: Stack Overflow, GitHub, Gitter and Slack, we developed six sets of twelve (72 in total) different machine learning models (both traditional and deep learning). The F1-Score of the best performing models in the system ranges from 0.56 to 0.94, with an average F1-Score of 0.85, predominantly using BERT as the primary model for various classification tasks. Also, two language-based models were developed for paraphrasing offensive to non-offensive comments. The overall utility of the OLRS-SE was evaluated using user studies with 30 participants, which showed that the developed system is easy to use and also effective in reducing offensive language. Investigating the capabilities of different large language models for offensive language reduction is a promising avenue for future research.

1. Introduction

Although interactions on software engineering platforms are described as generally professional and inclusive, offensive language is increasingly observed. The paper investigates its prevalence and nature and proposes OLRS-SE to detect and classify offenses, explain them to writers, and suggest non-offensive alternatives.

1. Introduction

  • Offensive and abusive language is increasingly observed despite norms of professional, positive, and inclusive interaction on software engineering platforms.Stack Overflow's Code of Conduct calls for inclusive and respectful participation.Source: §1 · p. 2
  • Hostility on Stack Overflow is associated with mental-health issues and may lead members to leave the platform.Newcomers also face neglected, deleted, or closed posts and downvotes without proper reasoning.Source: §1 · p. 2
  • The paper investigates offensive language's prevalence and classes and proposes OLRS-SE to detect and classify offenses, explain them, and suggest non-offensive alternatives.Its research questions address prevalence, system development, machine-learning classification, and language-model-based explanation and rephrasing.Source: §1 · pp. 2–3

2. Methodology

The methodology examines deleted Stack Overflow comments, develops a three-level taxonomy of offense, and implements OLRS-SE as a four-phase detection, classification, explanation, and paraphrasing system.

2.1.1. Data collection

  • The study uses Stack Overflow's 2020 deleted-comment dataset and distinguishes Rude/Offensive (RO) from milder Unfriendly/Unwelcoming (UN) comments.The dataset contained 108,473 RO comments (34.87%) and 28,030 UN comments (9.01%); both were removed through moderation for violating the site's Code of Conduct.Source: §2.1.1 · pp. 3–4

2.1.2. Taxonomy generation

  • The taxonomy extends prior work to three levels and 14 attributes, using top-down and bottom-up coding of 500 RO and 500 UN comments.One author identified attributes and another validated them; the cited prior taxonomy had one level and eight attributes.Source: §2.1.2 · p. 4
  • Level 1 distinguishes RO from UN; Level 2 labels offense targets, and Level 3 labels racial or swearing content.Level-2 and Level-3 labels may overlap; comments can carry multiple labels.Source: §2.1.2 · pp. 5, 7
  • The coded comments showed more RO than UN, more individually targeted than other Level-2 comments, and more swearing than racial comments.Counts exclude non-English comments, emoji-only comments, and duplicates; the second author checked a random sample of 200 comments.Source: §2.1.2 · p. 7

2.2. OLRS-SE framework (RQ2)

  • OLRS-SE combines rule-based and machine-learning approaches in four phases: offense detection, fine-grained classification, explanation, and paraphrasing.The explanation highlights offensive content and names the offense type; paraphrases replace offensive terms while preserving the comment's tone.Source: §2.2 · pp. 7–8

2.2.1. OLRS-SE pipeline

  • The pipeline screens RO comments with both a Level-1 classifier and Regex/Perspective API checks, then routes non-RO outcomes to the UN classifier.It uses separate model sets for RO targets and UN targets; no Level-2 UN models were developed for classes with scarce data. Explanation and paraphrasing follow classification, and users can choose an original or detoxified comment.Source: §2.2.1 · pp. 9–12

2.3. Offence detection and classification (RQ2.1)

The paper compares traditional and BERT-based classifiers for six offense-detection tasks, using task-specific preprocessing and feature combinations. Its model sets combine TF-IDF, sentiment, and politeness features with Random Forest, SVM, and fine-tuned BERT variants.

2.3.1. Text preprocessing

  • Traditional-model preprocessing removes usernames and non-alphabetic characters, whereas deep-learning preprocessing removes usernames only.The paper notes that stopword removal has been shown to have little effect on deep-learning model performance.Source: §2.3.1 · pp. 12–13

2.3.2. ML model building

  • Traditional models use TF-IDF, sentiment, and politeness features with Random Forest and SVM, while deep-learning models fine-tune BERTBase-Uncased and BERTBase-Cased.Five RF and five SVM feature-combination models were built per task, alongside two BERT variants.Source: §2.3.2 · pp. 13–14
  • The study developed 60 traditional and 12 BERT models per six-task set structure, totaling 72 models, and evaluated traditional models with 10-fold cross-validation.BERT models were trained for three epochs with a maximum token length of 250 and a low learning rate to avoid overfitting.Source: §2.3.2 · p. 14

2.4. Offence explanation and rephrasing (RQ2.2)

The paraphrasing module explains offensive terms and generates less-offensive alternatives, aiming to preserve disagreement and software-engineering relevance. It uses a modified Generate-Prune-Select process for SE-related comments and a BART-based counter-narrative model for non-SE comments or when suitable paraphrases are unavailable.

2.4. Offence explanation and rephrasing (RQ2.2)

  • For SE-related comments, the modified Generate-Prune-Select process masks offensive terms, generates alternatives, and recommends the top three that meet the J-score threshold.The threshold is 0.818; the J score evaluates style, context, and grammatical accuracy.Source: §2.4 · pp. 15–17
  • LIME identifies up to five prominent terms, and terms scoring above 0.5 are selected for masking, alongside offensive terms identified with regex patterns.LIME uses 500 perturbed inputs; its scores are scaled to 0–1 before applying the threshold.Source: §2.4 · pp. 15–16
  • RoBERTa generates candidate replacements for masked terms, with the five highest-ranked non-offensive token suggestions retained.Source: §2.4 · p. 16
  • For non-SE comments, or when no paraphrase meets the J-score threshold, a BART-based seq2seq model generates counter-narratives using 500 offensive source comments paired with 500 non-offensive suggestions.The model aims to reduce offensiveness while preserving the input comment’s negative tone, but the parallel corpus is generic and does not preserve the input’s surrounding context.Source: §2.4 · p. 17
  • The J score is applied only to suggestions from the modified GPS process, not to the seq2seq model’s outputs.The authors state that scoring seq2seq suggestions is pointless because input and output style and context vary significantly.Source: §2.4 · p. 17

3. Results

The results compare six classification tasks, showing strong F1-scores for most best-performing models but a weaker result for L2.UN. Additional analyses compare model families and feature groups, test rephrasing, and validate performance with human evaluators and comments from other platforms.

3. Results

  • BERT models performed best on RO tasks, while SVM models led on UN tasks across the six classification tasks.BERTBase-Uncased led L1.RO and L3.RO, BERTBase-Cased led L2.RO.1, and SVMs led L1.UN, L2.RO.2, and L2.UN.Source: §3 · p. 18
  • F1-scores were 0.88 or above for the best models at every level except L2.UN, whose top result was 0.56.The authors attribute the lower L2.UN result to nuances in identifying mildly offensive comments.Source: §3 · p. 19
  • SVM, multi-attribute, and deep-learning models significantly outscored their comparison groups; politeness features improved traditional models, while TF-IDF and sentiment did not significantly alter F1-scores.The group comparisons reported p < 0.05 for SVM over RF, multi-attribute over single-attribute, deep learning over traditional ML, and traditional models with politeness features over those without.Source: §3 · pp. 19–21, Figure 5 · p. 20
  • The paraphraser's average J-score was 0.8; modified GPS handled 91% of comments and seq2seq handled the remaining 9%.The evaluation sampled 200 highly offensive, SE-related comments.Source: §3 · p. 21

3.1. Additional validation

  • Across GitHub, Gitter, and Slack, offence-detection accuracy ranged from 57% to 100%, while F1-scores and MCC were comparatively lower than on Stack Overflow.The authors identify fine-tuning on a new platform or training on combined platform data as possible approaches.Source: §3 · p. 22, Table 3 · p. 23
  • Human validation averaged 81% accuracy and κ = 0.76 agreement; agreement on paraphrase quality was κ = 0.71.Classification agreement across tasks ranged from 0.58 to 0.91.Source: §3.1 · pp. 22–24

4. User evaluation

The evaluation presents OLRS-SE as a web tool for detecting, classifying, and rephrasing offensive comments, then assesses its usability and perceived utility with computing-related participants. Participants rated the tool positively while also suggesting improvements and future integration.

4. User evaluation

  • The web tool checks comments up to 250 characters, classifies offenses by type and target, and offers rephrasing suggestions for highly offensive comments.Users can select a suggested rephrasing or write their own alternative.Source: §4 · pp. 24–26

4.1. Methodology for user evaluation

  • Participants completed tasks testing detection, classification, and rephrasing, then answered a questionnaire covering usability, usefulness, and specific functionality.The questionnaire included selected questions from the System Usability Scale and Technology Acceptance Model.Source: §4.1 · p. 26

4.2. Results of user evaluation

  • Participants valued sentiment-preserving edits and offense categories, requested platform integration, and noted that paraphrase suggestions could be slow.The authors suggest faster servers as a possible way to address suggestion speed.Source: §4.2 · p. 27
  • Participants proposed continuous online learning from users’ rephrasing feedback, potentially integrated into real-life systems such as Stack Overflow.The paper presents this as a suggestion for future work.Source: §4.2 · pp. 27–28
  • Among 30 participants, 87.3% found the tool easy to use, 97% were satisfied with its performance, and 98% found it helpful.Figure 7 summarizes the grouped responses for ease of use, performance, and helpfulness.Source: §4.2 · p. 27, Figure 7 · p. 28

5. Discussion

The discussion presents a three-level offence taxonomy and an integrated detection-to-paraphrasing system, then weighs classification and user-evaluation findings against limitations in scope and coverage.

5. Discussion

  • The three-level taxonomy classifies offence severity, targets, and fine-grained classes, and the discussion reports offensive language across SO, GitHub, Gitter, and Slack.The authors argue that systems should help reduce such comments at the point of posting.Source: §5 · pp. 28–29
  • Across six classification tasks, deep-learning models were statistically better overall, while BERT led three best-model tasks and politeness-feature SVMs led the other three.BERT achieved an F1-score of 0.92 for Level-1 RO detection; the best Level-2 UN model, an SVM, scored 0.56.Source: §5 · pp. 29–30

5.1. Threats to validity

  • Cross-platform analysis was a pilot, and the system’s scalability beyond the four platforms and direct use outside SE remain unevaluated.The models were trained on SE-related datasets; other domains would require relevant fine-tuning and testing.Source: §5 · p. 30, §5.1 · p. 34

5. Discussion

  • The paraphrasing module uses a modified GPS pipeline for SE-related comments and counter-narrative generation for non-SE comments, with an average J score of 0.8.When suggestions did not meet the J-score threshold, the seq2seq model supplied an output for every offensive comment.Source: §5 · pp. 30–31
  • In a 30-participant evaluation, responses were positive about usability and usefulness, but only 46% strongly agreed that the tool was faster than human judgments.The authors attribute slowness to numerous ML models, some large neural networks, and running on a shared cloud platform without GPUs.Source: §5 · p. 31
  • The contribution is the integrated OLRS-SE design rather than a particular language model; the system was not compared with newer encoders or large language models.The authors describe the architecture as model-agnostic and say newer models can be integrated with minimal modification.Source: §5 · pp. 31–32

5.1. Threats to validity

  • The evaluation covers English text, while comments over 250 characters may be missed and omitted commenting context could affect offence judgments.About 6% of RO and UN comments exceeded the 250-character limit; mild UN offence is also subtle and nuanced.Source: §5.1 · pp. 33–34
  • The classifiers distinguish high RO and low UN offence rather than a continuum, and the five-term limit does not rule out comments containing more offensive terms.The authors also note that the Regex check counts only the first match, while other classification levels consider all comment words.Source: §5 · p. 31, §5.1 · pp. 34–35

6. Related work

The related work frames this study as addressing limited evidence on offensive language in SE communities and extending prior work across detection, classification, explanation, and rephrasing. Its stated distinctions include expanded classification, the first attempt at offence explanation in this domain, and rephrasing approaches that do not require annotated datasets.

6. Related work

  • The study addresses limited investigation of offensive language in SE communities and reports its presence across all four communities considered.Existing toxicity detectors did not perform adequately on SE-specific discussions, while BERT-based models had been proposed as candidates.Source: §6 · p. 36
  • It extends prior SE classification work with additional abuse-type and target attributes, beyond existing taxonomies and detection of specific abuse categories.Earlier work included an SO offence taxonomy, rule-based swearing and racial-comment detection, and ML detection of sexist and misogynistic GitHub discussions.Source: §6 · p. 36
  • The authors describe this as the first work to attempt offence explanation in the SE domain.The system explains why a comment is offensive and proposes non-offensive alternatives.Source: §6 · pp. 36–37
  • Earlier rephrasing work was outside the SE context, and counter-narrative approaches require annotated datasets, unlike the language-model paraphrases used here.The passage cites prior rephrasing work and counter-narrative generation.Source: §6 · p. 37
  • Table 5 compares 11 OLRS-SE functionalities; the authors say their work substantially extends prior work, many of whose systems cover only two.The compared functionalities span detection, classification, explanation, and rephrasing.Source: §6 · p. 37

7. Conclusion

The conclusion describes OLRS-SE as a prototype built from offence detection and classification models, with explanation and paraphrasing functions. The authors see potential for reducing offensive language in SE communities.

7. Conclusion

  • The work developed 72 machine-learning models across six tasks relevant to offence detection and classification, using deleted Stack Overflow comments.The system uses the best models from the six sets.Source: §7 · p. 37
  • OLRS-SE detects offences, classifies them into fine-grained classes, highlights offensive words, and paraphrases offensive posts into non-offensive ones.Source: §7 · p. 37
  • The authors conclude that OLRS-SE has potential to reduce offensive language at source in online SE communities.Source: §7 · p. 37

Appendix A. Additional explanation of the 11 steps in the paraphraser module

The paraphraser module is organized into 11 steps and four main components, which the appendix details in subsequent subsections.

Appendix A. Additional explanation of the 11 steps in the paraphraser module

  • The paraphraser module is implemented as 11 steps.Source: Appendix A · p. 38
  • Its four main components are an SE relation checker, a LIME-based explainer, a modified GPS pipeline, and sequence-to-sequence model creation.Source: Appendix A · p. 38
  • The following subsections detail each of the four components.Source: Appendix A · p. 38

Appendix A.1. SE relatedness check

The paraphraser uses different approaches for SE-related and non-SE-related comments. Its SE-relatedness check relies on an SE-term corpus, but achieved only 29% accuracy in the sampled comments.

Appendix A.1. SE relatedness check

  • For SE-related comments, the approach tones down offensiveness while preserving negative sentiment and SE relevance, including SE-related words.For non-SE-related offensive comments, a separate seq2seq model generates paraphrases using counter-narrative generation.Source: Appendix A.1 · p. 38
  • Only 23% of the sampled offensive comments from Stack Overflow and Gitter were judged SE-related.The authors randomly selected 100 comments from each platform and assessed them manually.Source: Appendix A.1 · p. 38
  • The SE-relatedness check used the 5,000 most commonly occurring SEthesaurus terms, and this corpus achieved the best accuracy, 29%, on the 200-comment dataset.The reported 6% accuracy difference (29%−23%) was attributed to false-positive terms such as question and answer, which can also occur in non-SE-related domains.Source: Appendix A.1 · pp. 38–39

Appendix A.2. LIME-based explainer module

The explainer identifies words contributing to an SE-related comment’s offensiveness, then highlights them for the user and masks them for paraphrasing. It combines LIME explanations with a Regex check to reduce missed offensive terms.

Appendix A.2. LIME-based explainer module

  • LIME identifies the five most prominent words contributing to offensiveness, using 500 perturbed comment instances.The authors limited the list to five because comments with more than five offensive terms were rare in their Stack Overflow experience.Source: Appendix A.2 · p. 39
  • LIME assigns each word a proportional probability contribution to the offensive label, and the scores are scaled to 0–1.A 0.5 threshold selects words for masking; other potential impolite words remain unmasked.Source: Appendix A.2 · pp. 39–40
  • A Regex check against identified Stack Overflow patterns additionally masks impolite terms that LIME may miss on perturbed comments.The authors note that LIME’s identification can vary with model behavior on perturbed instances.Source: Appendix A.2 · p. 40
  • The identified words can be shown as an explanation, while the masked comment proceeds to the paraphraser for replacement suggestions.The paraphraser replaces masked offensive words to reduce offence.Source: Appendix A.2 · p. 40

Appendix A.2.1. Modified GPS pipeline

The modified GPS pipeline generates paraphrases that retain non-offensive content while removing offensive language. It prunes and selects suggestions using style, context, and grammatical accuracy, with a seq2seq fallback when none meet the required score.

Appendix A.2.1. Modified GPS pipeline

  • The modified GPS pipeline retains the original comment’s non-offensive content rather than generating counter-hate-speech responses.It keeps generated sentences close to the original while excluding offensive language.Source: Appendix A.2.1 · pp. 40–41
  • Fine-tuned RoBERTa generates contextual token suggestions for masked words, with the pipeline limited to the top five suggestions per mask.RoBERTa is fine-tuned unsupervised on user-network comments to preserve their negative tone.Source: Appendix A.2.1 · p. 41
  • The J score evaluates style, context, and grammatical accuracy, so pruning removes candidates that fail those criteria as well as ungrammatical ones.The original GPS pipeline used grammatical accuracy alone for pruning.Source: Appendix A.2.1 · p. 41
  • A J score threshold of 0.8 out of 1 selects final suggestions instead of relying on cosine similarity alone.If no suggestion meets the threshold, the pipeline uses a seq2seq model to produce counter-narratives.Source: Appendix A.2.1 · pp. 41–42
  • The pipeline has generator, pruning, and selection stages; its illustrated example recommends the top three suggestions.In the example, generated alternatives are censored, infilled, scored, and then ranked for selection.Source: Appendix A.2.1 · pp. 41–42

Appendix A.3. Rephrasing non-SE-related comments

For offensive comments unrelated to SE discussions, the authors use a seq2seq counter-narrative approach trained on paired offensive and less or non-offensive comments. The suggestions aim to reduce offensiveness while preserving the original comment’s negative tone.

Appendix A.3. Rephrasing non-SE-related comments

  • The seq2seq model generates counter-narratives for offensive comments that are not related to SE discussions.This applies a counter-narrative approach to non-SE-related comments.Source: Appendix A.3 · p. 42
  • The training set pairs 500 offensive, non-SE-related source comments with 500 less or non-offensive target suggestions.The authors describe creating parallel corpora as cumbersome because each source needs a corresponding target sentence.Source: Appendix A.3 · pp. 42–43
  • The suggested rephrasing can preserve disappointment while reducing offensiveness, as in changing a complaint about a closed question into a statement of offense.The example changes “my question got closed” into “I am offended because my question got closed.”Source: Appendix A.3 · p. 44
  • In a separate SE-related example, BART supplies suggestions when no GPS candidate meets the J-score threshold.BART is fine-tuned on parallel corpora with offensive source statements and less or non-offensive target statements.Source: Appendix A.3 · p. 45

Related papers

Loading 2610.19961v1…