Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspective

1. Introduction

• Online harassment and cyberbullying are widespread, and exposure to abusive content can cause psychological trauma, radicalization, self-harm, or suicide.The cited U.S. surveys report that over 40% of adults experienced online harassment and 36.5% of high school students reported lifetime cyberbullying.Source: §1 · p. 1
• The sheer volume of posts challenges moderation, while human moderators report burnout, depression, and PTSD; automatic detection can assist and partly ease manual inspection.The paper frames ethical dilemmas as consequential beyond academia, noting potential real-world human and economic harms.Source: §1 · pp. 2–3
• Abusive language broadly includes language that could offend, demean, or marginalize, spanning profanities through threats and severe insults.Related research studies this problem under labels including hate speech, toxicity, cyberbullying, and microaggression.Source: §1 · p. 2
• The survey brings technical and social issues together under an ethical and human-rights framework, covering textual abuse detection and challenges such as fairness, privacy, transparency, and free speech.It includes work on abuse from hate speech to microaggressions, while focusing on individual textual utterances rather than multimedia or broader conversational context.Source: §1 · p. 2
• Ethical considerations should span task formulation, data collection and annotation, model training and evaluation, and deployment.The authors also draw on social-science literature and compile recommendations for stages across this pipeline.Source: §1 · p. 2

2.1 Task Formulation

• Abusive-language detection is commonly treated as supervised classification, with labels spanning abuse types, implicitness, targets, legality, and stereotypes.Definitions can differ across datasets, and limited context, collection complexity, and privacy concerns leave much research using coarse-grained labels.Source: §2.1 · pp. 3, 5
• Researchers distinguish abuse expression from its target; implicit abuse is often underrepresented, while individual, group, and entity targets can require different handling.Implicit abuse can rely on sarcasm, humor, stereotypes, or ambiguity, and may require background knowledge to annotate; target distinctions also matter for annotation and classification.Source: §2.1 · pp. 3, 5, 6

2.2 Language Resources

• Abusive-language lexicons and annotated corpora are important resources, but creating them is laborious and costly because abuse is sparse and definitions are ambiguous.Lexicons can become outdated, miss spelling variations, and fail to capture abusive texts without individually abusive terms.Source: §2.2 · p. 6, §2.2.1 · p. 6

2.2.1 Lexicons

• Lexicons often help locate rare abusive examples, but keyword-based collection can introduce topic bias into trained classifiers.Existing sampling strategies use lexicons or query terms; a review of multilingual hate-speech lexicons also highlighted the need for a unified taxonomy for harmful-content searches.Source: §2.2.1 · pp. 6–7

2.2.2 Annotated Datasets

• Annotated datasets cover limited ranges of harmful behavior, and vague or overlapping category definitions can produce low agreement among annotators.Guidelines, multiple annotators, and expert annotators are among the practices used to support reliable annotation.Source: §2.2.2 · p. 7

2.3 Algorithms

• Detection systems use varied classification setups and algorithms, from dictionary and feature methods to deep learning, transformers, multi-task learning, and data augmentation.Reported challenges include weak cross-dataset generalization, high variance on small datasets, and limited annotated data; some studies report improvements from multi-task and augmentation approaches.Source: §2.3 · pp. 8–9

2.4 Applications

• Applications include platform moderation, toxicity feedback, monitoring, and chatbot safety, with human review retained in some workflows.Automated moderation during COVID-19 was described as erroneous, while research on using detection to preempt unwanted behavior had not studied its practical and technical implications.Source: §2.4 · pp. 9–10
• Legal abuse classification must account for changing definitions across jurisdictions, so platforms need solutions that are transparent and adaptable to local requirements.The paper describes translating legal rules into target-detection and abusive-act-detection tasks, while noting that each jurisdiction’s definition can evolve.Source: §2.4 · pp. 10–11

3. Current Ethics-Related Challenges

• The eight principles are privacy, accountability, safety and security, transparency and explainability, fairness and non-discrimination, human control, professional responsibility, and promotion of human values.The survey draws these principles from an analysis of 36 prominent AI-principles documents spanning multiple sectors and regions.Source: §3 · p. 11
• The analyzed AI-principles documents vary in cultural contexts and objectives, yet the survey reports convergence around these eight themes.More recent documents tend to include all eight principles.Source: §3 · p. 11
• The survey reviews technological and sociological challenges in automatic online abuse detection through eight common ethical and human-rights principles; Table 3 summarizes ethical challenges in addressing online abuse, discussed in what follows.Source: §3 · p. 11

3.1 Promotion of Human Values

• Abuse detection implicates freedom of speech alongside equality and dignity, since abusive content can harm victims and dehumanizing language can reduce prosocial behavior.The passage also notes that marginalized groups may internalize messages of inferiority, and that extreme consequences can include atrocities and exploitation.Source: §3.1 · p. 13
• Protecting equality and dignity is presented as necessary for everyone’s free speech, including minority voices threatened or silenced by offensive behavior.The authors argue that safe environments are a social responsibility and that online spaces should be considered alongside offline spaces.Source: §3.1 · p. 13
• Corporate moderation can reproduce structural inequalities, and disparities in language-technology access can amplify them.The passage notes that research often focuses on a small number of highly resourced languages and describes a 2019 report that Facebook’s hate-speech detection covered 40 languages.Source: §3.1 · pp. 13–14
• Online abuse reflects offline marginalization and othering, so technological, corporate-policy, and legal interventions may be more effective when combined with cultural change.The authors connect online communication conditions, including anonymity and limited social cues, with a higher likelihood of heated conversations and offensive behavior.Source: §3.1 · p. 14

3.2.1 Semantic Bias

• Embedding models can encode social stereotypes and toxic language, and upstream bias mitigation transfers to fine-tuned hate-speech classifiers but is less effective than downstream mitigation.Upstream mitigation is described as more efficient and accessible; its effectiveness may vary across languages.Source: §3.2.1 · p. 15

3.2.2 Selection Bias

• Dataset sampling can skew identity-related concepts and harm generalizability; across datasets, textual-source differences may matter more than sampling methods.Over-represented benign topics can also impair generalization, while removing them can improve it.Source: §3.2.2 · pp. 15–16

3.2.3 Label Bias

• Subjective abuse definitions and majority-vote labeling can amplify majority views and suppress minority voices; annotator knowledge and dialect awareness also shape labels.Re-annotation that accounts for speaker identity and dialect may be more effective than automatic debiasing in some cases.Source: §3.2.3 · p. 16

3.2.4 Bias in System Output

• Fairness can be measured through error disparities across identity groups, including counterfactual comparisons that swap target terms in otherwise similar utterances.The section also notes that improving fairness may reduce overall test-set accuracy.Source: §3.2 · p. 14, §3.2.4 · p. 17
• Fairness research aims to identify and mitigate biases that may harm sub-populations, while recognizing that statistical models cannot be completely bias-free.Researchers can quantify output unfairness across demographics, identify bias origins, and minimize harmful outcomes.Source: §3.2.4 · p. 18

3.3.1 Transparency

• Model cards, FactSheets, data statements, and dataset datasheets document model or dataset details, purposes, performance, scope, and provenance.Model cards can specify applications for which a model is not recommended.Source: §3.3.1 · p. 18

3.3 Transparency and Explainability

• Transparency supports trust and oversight, but its practical meaning depends on deployment circumstances and documentation can conflict with privacy.Interpretability concerns explaining opaque model dynamics, whereas explainability usually means post-hoc explanations designed for lay users.Source: §3.3.1 · p. 18, §3.3 · p. 18

3.3.2 Interpretability and Explainability

• LIME and attention mechanisms can highlight words or text spans, but highlighted features do not necessarily explain why a model classified text as abusive.Research has also used partial occlusion and classifier scores; score alignment with human judgments of abuse severity is not fully clear, though recalibration can improve alignment.Source: §3.3.2 · p. 19
• Richer toxicity labels and abuse-target annotations provide more detailed outputs, while language models still struggle to reproduce human explanations of implied meaning.Source: §3.3.2 · p. 19

• Benchmark datasets raise privacy concerns because public posts are often scraped without explicit consent, and researchers may infer personal attributes such as gender or location.Distributing post IDs rather than full texts can support the right to be forgotten when users delete posts or profiles.Source: §3.4 · p. 20
• Large-scale deployed filters would require access to platform users’ personal data, unlike research studies that typically use small convenience samples of public data.Federated learning and edge computing keep training data on users’ devices, while encrypted model parameters are exchanged with the cloud.Source: §3.4 · p. 20
• Doxxing—publishing private information to facilitate harassment—links privacy to safety, yet automated detection of this behavior has received little research attention.Source: §3.4 · p. 20

• False positives can threaten freedom of speech and reputations, while false negatives leave hurtful behavior unaddressed; deployment risk depends on the application.HateCheck offers functional tests across expressions of hate and contrastive non-hateful utterances, and systems need maintenance as user language evolves.Source: §3.5 · p. 21
• Safety measures seek to minimize expected harms through failure prevention, identification and monitoring, and repair; security protects systems from unauthorized compromise.Source: §3.5 · p. 21
• Abuse detectors can be manipulated through typos, word changes, or innocuous additions, and such attacks have reduced classifier confidence or flipped predictions.Studies report that character-based models can be more resistant, but remain brittle; subword information alone does not prevent attacks.Source: §3.5 · pp. 21–22

3.6 Accountability

• Accountability concerns who answers for automated decisions and impacts, and includes verifiability, impact assessment, audits, appeals, liability, and legal responsibility.It is closely connected to safety and security, transparency and explainability, and human control.Source: §3.6 · p. 22
• Developers and deploying organizations are generally considered responsible for AI systems’ outcomes, while legal accountability for the AI systems themselves may be unnecessary and troublesome.Guidelines may distinguish developer liability from deployer liability; transparency and professional responsibility are commonly emphasized during development.Source: §3.6 · p. 22
• Audits can assess technical and institutional practices; Raji et al. (2020) propose an internal lifecycle framework with five stages: Scoping, Mapping, Artifact Collection, Testing, and Reflection.External audits often occur after deployment, whereas the SMACTR framework supports development and deployment end to end.Source: §3.6 · p. 22
• Platforms have little accountability for moderation decisions, while users and civil society groups describe practices as inconsistent, confusing, and lacking transparency and adequate appeals.Documented cases also link moderation to unnecessary restrictions on posts with certain words and imagery.Source: §3.6 · pp. 22–23
• The Santa Clara Principles call for public transparency, meaningful appeal opportunities, and justifications for content removals; proposed alternatives include procedural accountability and social media councils.Social media councils and e-courts could discuss terms of use, adjudication, and fundamental ethical questions.Source: §3.6 · p. 23

3.7 Human Control of Technology

• Human control includes users’ ability to appeal automated decisions, request human review, or opt out of automation.The paper links this principle to ambiguous language and the need to protect freedom of speech.Source: §3.7 · p. 23
• Many studies report accuracies in the 80% range, which the paper says means one in five automated decisions will be incorrect; even correct decisions may be disputed.The authors argue this makes human review necessary for uncertain or contested decisions.Source: §3.7 · p. 23
• Detection research often leaves unspecified what to do with flagged content and how to handle false positives and false negatives, while those decisions vary by platform and community.The paper notes that platforms combine algorithmic detection with human moderators and user appeals.Source: §3.7 · p. 23
• Human review can occur before publication when a system flags low-confidence or potentially problematic content, keeping harmful posts from public view but risking publication delays.The delay can frustrate users and disrupt conversation.Source: §3.7 · pp. 23–24
• Post-moderation allows users to request human review after an automated decision, so outcomes need understandable explanations of the factors and logic behind them.Feedback from human moderators could be used to improve classifier decisions in either review arrangement.Source: §3.7 · p. 24

3.8 Professional Responsibility

• Professional responsibility includes system accuracy, responsible design, attention to long-term effects, engagement with affected stakeholders, and scientific integrity.The paper connects these obligations to the Harvard AI Ethics Framework and professional codes including ACM, IEEE, and AoIR.Source: §3.8 · p. 24
• Blodgett et al. (2020) urge researchers to ask which groups are affected, whether systems reinforce social hierarchies, and whose definition of offence guides detection.Their questions also include whether affected communities were consulted and whether systems could be used to silence dissidents or marginalized groups.Source: §3.8 · p. 24
• As AI development outpaces legal and regulatory frameworks, practitioners have increased responsibility to influence their field’s values and ethics.The paper recommends declining harmful projects, taking stakeholder feedback seriously, and publishing limitations, failures, and negative results.Source: §3.8 · pp. 24–25

4. Ways to Move Forward

• Ethics-aware responses span the development pipeline and include public education, intelligible systems, interdisciplinary grounding, and affected-community participation.The authors stress that online abuse is social and that design should draw on social sciences, communication studies, and psychology.Source: §4 · p. 25

4.1 Reimagining the Task of Confronting Abusive Language Online

• Binary abusive/not-abusive classification can limit real-world applicability because abusive language is nuanced and varies in definition.The survey describes multi-dimensional and hierarchical frameworks as ways to represent targets, expression, intention, and severity more finely.Source: §4.1 · p. 26, §4.1.1 · p. 26

4.1.1 Moving Away from Coarse-Grained Definitions of Abuse

• Abuse ranges from explicit, severe content to implicit or unintentional messages, so context-sensitive responses are needed beyond simple removal.Explicit abuse can be surfaced for human moderators, while implicit abuse is difficult to detect and may not violate platform rules or law.Source: §4.1.1 · p. 26

4.1.2 Moving Towards Flexible, Rights-Respecting Moderation

• Flexible moderation can preserve choice through temporary quarantine, behavior-shaping nudges, and non-abusive rephrasings rather than relying only on bans.Quarantined messages can let recipients decide whether to view or post them; nudges can discourage antisocial interactions without eliminating them, while style transfer requires care to avoid bias or manipulation.Source: §4.1.2 · p. 27

4.1.3 Extending the Task Beyond Detection

• Beyond detection, counter-narratives, virality-aware human review, and public education can mitigate abuse, support victims, and address harmful beliefs.Community reporting can validate victims and encourage bystander support; personalized education tools may raise privacy concerns, and these approaches are linked to free speech, user control, and fairness.Source: §4.1.3 · pp. 28–29

4.2 Advancing Interpretability and Explainability

• Interpretability can help identify bias and vulnerabilities, while explanations must suit different stakeholders; plain-language reasons remain unreliable, making prediction reliability scores a more achievable explanation.Interpretability can link unfair outputs to data imbalances and help identify spurious correlations that threaten robustness and generalization.Source: §4.2 · pp. 29–30

4.3 Grounding Research in Work from Other Disciplines

• Because online abuse is a social problem shaped by identity, communication, and context, the authors call for collaboration across AI and social-science disciplines.They note that geographical, temporal, and political contexts influence abuse dynamics but are rarely considered in AI tools.Source: §4.3 · pp. 30–31

4.4 Engaging Affected Communities

• Since marginalized communities are often targeted while powerful majorities control technology decisions, affected communities should shape system design and research priorities.Engagement should build trust, include disadvantaged groups’ voices in setting priorities, and provide fair, transparent compensation.Source: §4.4 · p. 31

5. Conclusion

• The survey examines abusive-language detection across sub-fields through an ethics and human-rights lens.It considers how NLP systems can protect people while also accounting for broader social impacts.Source: §5 · p. 31
• Fairness, non-discrimination, and human-value concerns are often rooted in task formulation, data collection, and annotation.The survey urges attention to who formulates the task, whether affected communities are consulted, data representativeness, and annotator biases.Source: §5 · p. 32
• After classification, binary labels raise questions about explanation and appeal, while accountability, human control, and professional responsibility also require attention.The survey encourages interdisciplinary collaboration and considers counter-narratives, automated rewording, and educational applications as ways to help reduce abuse; these approaches will not solve its underlying social problems.Source: §5 · p. 32