Source-linked AI summary
Deep Learning for Detecting Cyberbullying Across Multiple Social Media Platforms
Sweta Agrawal, Amit Awekar
TL;DR
Cyberbullying detection is limited by single-platform and single-topic studies and reliance on handcrafted features. This paper evaluates deep learning and transfer learning across Formspring, Twitter, and Wikipedia datasets. The authors report that these models overcome the identified bottlenecks and beat state-of-the-art results across all three datasets.
Problem
Existing cyberbullying detection studies often target one platform or topic and rely on handcrafted features, despite variation across social media settings.
Method
The paper evaluates CNN, LSTM, BLSTM, and attention-based BLSTM models with multiple text representations across three real-world social media datasets.
Results
Deep neural networks combined with transfer learning beat state-of-the-art results on all three datasets.
Takeaways & Limitations
Deep learning models can detect cyberbullying across multiple topics and social media platforms, while learned knowledge can be transferred between datasets.
Abstract
from arXiv · showhide
Harassment by cyberbullies is a significant phenomenon on the social media. Existing works for cyberbullying detection have at least one of the following three bottlenecks. First, they target only one particular social media platform (SMP). Second, they address just one topic of cyberbullying. Third, they rely on carefully handcrafted features of the data. We show that deep learning based models can overcome all three bottlenecks. Knowledge learned by these models on one dataset can be transferred to other datasets. We performed extensive experiments using three real-world datasets: Formspring (12k posts), Twitter (16k posts), and Wikipedia(100k posts). Our experiments provide several useful insights about cyberbullying detection. To the best of our knowledge, this is the first work that systematically analyzes cyberbullying detection on various topics across multiple SMPs using deep learning based models and transfer learning.
1 Introduction
Cyberbullying detection is difficult because its meaning, topics, vocabulary, and communication styles vary across social media platforms. This work systematically evaluates deep learning and transfer learning across three platforms and multiple cyberbullying topics.
- Motivation: Cyberbullying detection is challenging because the definition of bullying is subjective and varies across social media platforms.Frequent swearing may indicate bullying generally but not necessarily on teen-oriented platforms such as Formspring.
- Research gaps: Prior methods commonly focus on one platform, one cyberbullying topic, or handcrafted features that are not robust to writing-style variation.These limitations correspond to bottlenecks B1, B2, and B3.
- Approach: The experiments compare traditional machine-learning and deep-neural-network models using character n-grams, word unigrams, GloVe embeddings, and SSWE embeddings.Deep models include CNN, LSTM, BLSTM, and BLSTM with attention.
- Contributions: The work applies transfer learning and systematically analyzes cyberbullying across multiple platforms and topics.It is presented as the first study to combine this cross-platform, cross-topic analysis with transfer learning.
- Experimental scope: The study uses manually annotated datasets from Formspring, Twitter, and Wikipedia, covering Q&A, microblogging, and collaborative knowledge-repository settings.The datasets address different topics, including racism, sexism, personal attacks, and general cyberbullying, while cyberbullying labels remain minorities.
- Findings: Swear words are neither necessary nor sufficient for detecting cyberbullying.For Formspring, 78% of swearing posts are non-bullying, while 82% of Twitter bullying posts contain no swear words.
3 Related Work
Prior cyberbullying detection studies commonly use single-platform datasets, handcrafted features, and traditional machine learning, limiting robustness across platforms and bullying styles.
- Existing studies typically train on a single social media platform and evaluate performance within that platform.
- Traditional approaches engineer features such as swear-word usage and apply classifiers including logistic regression, SVM, random forest, or naive Bayes.
- These handcrafted features can yield low precision because bullying styles vary across social media platforms and topics.
4 Deep Neural Network (DNN) Based Models
The paper evaluates four neural architectures within a shared model design, varying word-embedding initialization to capture platform- and topic-specific cyberbullying language.
- The evaluated DNN architectures are CNN, LSTM, BLSTM, and BLSTM with attention, ordered by increasing architectural complexity and information use.
- The models share the same overall architecture and differ only in their neural architecture layer.
- Word embeddings are initialized randomly or with GloVe or SSWE, then learned during training as task-specific representations.
- SSWE incorporates text sentiment into word-embedding generation, whereas GloVe considers syntactic context.
- Two dropout layers use rates of 0.25 and 0.5 before and after the neural architecture layer, respectively.
- Training uses backpropagation with Adam optimization and categorical cross-entropy loss while tuning task-specific word embeddings.
5 Experiments
Experiments compare traditional and deep neural models across cyberbullying datasets, examining imbalance, embeddings, model architectures, task-specific semantics, and transfer learning. Deep models with transfer learning outperform traditional models and prior results, while learned embeddings capture platform- and topic-specific bullying language.
- Model comparison: Traditional machine-learning models performed significantly worse than DNN models across the evaluated cyberbullying datasets.The traditional models included logistic regression, SVM, random forest, and naive Bayes with character n-gram and word unigram representations.
- Class imbalance: Tripling bullying posts in training substantially improved all three evaluation measures, especially for the smallest Formspring dataset.Oversampled datasets F+, T+, and W+ outperformed their non-oversampled counterparts; Formspring had only 825 bullying training instances.
- Embeddings and architecture: Initial embedding choice had little effect after oversampling, while architecture differences narrowed as dataset size increased without oversampling.The compared initializations were random, GloVe, and SSWE embeddings.
- Embeddings and architecture: Among the evaluated DNNs, LSTM was usually weaker, while CNN, BLSTM, and BLSTM with attention had no significant performance gap.The comparison used precision, recall, and F1 measures with SSWE initialization.
- Task Specific Word Embeddings: Task-specific embeddings formed clusters for bullying topics and reflected platform-specific semantic differences in words such as “slave.”The study used t-SNE to visualize learned embeddings; Twitter showed racism- and sexism-related associations that Wikipedia did not.
- Transfer Learning: Feature-level transfer learning substantially improved recall and precision over complete transfer, whereas transferring network weights added no significant improvement.Feature-level transfer reused learned word embeddings, while model-level transfer also reused network weights.
- Transfer Learning: BLSTM with attention and feature-level transfer achieved F1 scores of 0.94 on both Wikipedia and Twitter, and 0.95 on Formspring.The Formspring result also reported 98% accuracy; prior reported F1 scores were 0.68 for Wikipedia and 0.93 for Twitter.
- Conclusion and Future Work: The study used four DNN architectures across three datasets and concluded that transfer-coupled DNNs surpassed prior results for all three datasets.The datasets represented Formspring, Twitter, and Wikipedia, covering multiple social-media platforms and cyberbullying topics.