Source-linked AI summary
Can Artificial Intelligence Support Healthcare and Mental Health Through Early Cyberbullying Detection ? The Impact of Emotion-Aware AI on Proactive Online Safety
Hamed Jelodar, Amir Firouzi, Yen-Wu Lo, Maryam Tanha, Sajjad Dadkhah
TL;DR
Cyberbullying threatens mental health, while existing systems often provide static classification without early warning or intervention support. CareGuard combines emotion-aware filtering, semantic labeling, transformer classification, and LLM post-analysis; experiments identify RoBERTa-base as the most balanced evaluated model, while highlighting category-level errors and dataset limitations.
Problem
Cyberbullying poses significant mental-health risks, and existing methods often lack early-warning or intervention mechanisms while struggling with implicit language and cross-platform generalization.
Method
CareGuard is an end-to-end framework integrating zero-shot semantic labeling, emotion-aware cosine-similarity filtering, transformer-based classification, and LLM-based post-analysis.
Results
RoBERTa-base provided the most balanced overall performance, achieving an accuracy of 0.91 and a Macro F1-score of 0.90.
Takeaways & Limitations
CareGuard extends cyberbullying classification toward early warning, interpretable outputs, and support for proactive mental-health and online-safety interventions.
Takeaways & Limitations
The framework was trained and evaluated on a single publicly available dataset, so performance may not represent linguistic diversity across platforms.
Abstract
from arXiv · showhide
Healthcare systems, mental health, and public well-being are increasingly affected by cyberbullying and harmful online interactions. This paper presents CareGuard, an early-warning framework designed to support healthcare-driven mental health protection and proactive online safety through the detection of cyberbullying-related content using advanced natural language processing techniques. CareGuard integrates zero-shot semantic labeling with fine-tuned transformer-based models, including BERT, DistilBERT, and RoBERTa, to enable robust and context-aware classification across sensitive cyberbullying categories. To improve efficiency and reduce unnecessary computation in healthcare-oriented monitoring settings, the framework incorporates an emotion-aware filtering mechanism alongside cosine similarity-based semantic screening, allowing the system to focus on semantically relevant and emotionally salient content. Experimental results on benchmark datasets demonstrate that CareGuard effectively balances detection accuracy and computational efficiency, highlighting its potential for scalable deployment in healthcare systems, mental health monitoring, and online safety applications.
1. Introduction
Cyberbullying poses serious mental-health risks and remains difficult to detect promptly in dynamic online environments. CareGuard addresses this need with an AI-based early-warning framework combining semantic, emotion-aware, transformer, and post-modeling techniques.
- Cyberbullying can cause anxiety, depression, and social isolation, while its anonymous and pervasive nature makes it harder to detect and control than traditional bullying.
- Increasing cyberbullying severity, online communication volume, and linguistic diversity make timely detection and intervention challenging.
- CareGuard introduces an AI-based system for early cyberbullying detection and proactive monitoring of harmful online content.
- The framework integrates zero-shot semantic labeling with fine-tuned transformer models for cyberbullying detection.
- Emotion-aware filtering using emotion annotation and cosine similarity reduces irrelevant processing and improves computational efficiency.
- CareGuard extends conventional classification toward early warning and mental-health-oriented intervention support.
2. Related Works
Cyberbullying detection has progressed from handcrafted-feature methods to deep learning and transformer models, but challenges remain in contextual understanding, generalization, and proactive response. CareGuard is positioned as an early-warning framework addressing the gap between classification and intervention support.
- Cyberbullying exposure is linked to depression and suicidal ideation, and the World Health Organization recognizes bullying as a major public-health concern.
- Traditional machine-learning approaches are simple and interpretable but rely on handcrafted features and capture complex, sarcastic, and contextual language poorly.
- Prior studies include ensemble, BERT, BiLSTM, CNN, and MLP approaches for cyberbullying detection across social-media data.
- Existing methods struggle with implicit or context-dependent cyberbullying, cross-domain generalization, and mechanisms for early warning or intervention.
- Deep learning and transformer approaches improve contextual representation and classification performance but often depend on large labeled datasets and platform-specific evaluation.
- CareGuard integrates detection with an early-warning mechanism intended to support proactive prevention and intervention in real-world applications.
3. Proposed Model
CareGuard is a four-phase, end-to-end early-warning pipeline that combines preprocessing, semantic and emotion-aware filtering, transformer refinement, and LLM-based post-analysis. Its design prepares cyberbullying text, focuses processing on relevant content, and produces interpretable outputs for timely intervention.
- 3. Proposed Model: CareGuard combines text preprocessing, emotion-aware semantic filtering, transformer-based classification, and LLM-based post-analysis in one pipeline.
- 3. Proposed Model: The framework has four phases: data collection and preprocessing, semantic embedding and emotion-aware filtering, transformer refinement, and post-modeling LLM analysis.
- 3.2. Phase A: Data Collection and Text Pre-processing: Phase A uses an existing Kaggle dataset of N cyberbullying-related tweets as input for subsequent semantic and classification analysis.
- 3.2. Phase A: Data Collection and Text Pre-processing: Preprocessing applies stemming, stop-word removal, and HTML cleaning to standardize text and reduce irrelevant variation.
- 3.2. Phase A: Data Collection and Text Pre-processing: Stemming reduces each word to its root form before further preprocessing operations.
- 3.2. Phase A: Data Collection and Text Pre-processing: The processed dataset is represented for downstream analysis after the preprocessing operations.
3.3. Phase B: Semantic Embedding and Emotion-aware Filtering
Phase B filters preprocessed tweets using emotion-aware semantic analysis, retaining content that is semantically relevant and emotionally salient for deeper cyberbullying analysis.
- Emotion-aware semantic filtering extracts negative sentiment and emotional cues from each preprocessed tweet.
- The filtering operation uses an emotion score and negativity threshold to determine each tweet’s data-flow path.
- The phase reduces noise and computation by removing tweets unlikely to provide meaningful cyberbullying evidence.
- Tweets judged semantically and emotionally relevant are prioritized for deeper analysis in the next stage.
3.4. Phase C: Transformer-based Cyberbullying Refinement
Phase C performs cyberbullying classification with a hybrid transformer refinement strategy centered on RoBERTa and enhanced by sequential modeling.
- The main classifier combines fine-tuned RoBERTa with a Bidirectional Gated Recurrent Unit layer.This combines transformer contextual representations with recurrent sequential modeling.
- RoBERTa serves as the primary classifier, while BERT and DistilBERT supplement analysis of potentially harmful tweets.
- The multi-model design is intended to improve robustness and provide flexibility under different computational constraints.
- Model performance is evaluated using accuracy, precision, recall, and F1-score, with an aggregated score defined across n models.
- The Bi-GRU captures sequential dependencies to help identify implicit, context-dependent, and semantically connected bullying patterns.
3.5. Phase D: Post-modeling LLM-based Analysis
Phase D applies LLaMA-based post-modeling analysis to flagged cyberbullying tweets, extending classification with mental-health assessment, entity extraction, and concise explanations.
- The post-modeling layer analyzes flagged tweets with an LLM-based multi-task strategy to improve interpretability and support practical intervention.
- A LLaMA-based model performs mental health analysis, bullying-entity extraction, and semantic point extraction with summarization.
- Mental health analysis examines aggression, humiliation, and emotional distress as indicators of harmful intent and potential victim impact.
- Entity extraction identifies targets of abuse to clarify the social and contextual structure of the bullying event.
- Semantic point extraction produces a concise explanation of why a tweet was flagged for human analysts and platform moderators.
- Prompt engineering uses Chain-of-Thought reasoning and zero-shot/few-shot learning to address tone, context, interaction cues, and unseen patterns.
3.6. Why the Proposed Model is Different
CareGuard differs from conventional cyberbullying classifiers by combining emotion-aware filtering, multi-transformer refinement, and LLM-based post-analysis within an early-warning framework.
- Emotion-aware filtering precedes classification to reduce noise and improve efficiency.
- Multiple transformer models are integrated in a refinement stage rather than relying on a single classifier.
- LLM-based post-analysis adds interpretability, mental health-oriented assessment, and semantic explanation beyond conventional detection.
- The framework is positioned as an early-warning and decision-support system rather than only a classifier.
4. Experiment and Settings
The experiments compare fine-tuned BERT-base, RoBERTa-base, and DistilBERT models using performance metrics and error visualizations. RoBERTa-base provides the most balanced results, while sensitive categories and DistilBERT remain important sources of error.
- Experimental setup: The study evaluates BERT-base, RoBERTa-base, and DistilBERT with BiGRU layers on a publicly available cyberbullying dataset.Training uses a learning rate of 2 × 10−5, batch size 32, and validation-loss early stopping.
- Error analysis: Confusion-flow visualizations isolate misclassification pathways, while row-normalized matrices show category-specific error patterns across models.Religion is frequently confused with non-bullying content, and sensitive categories can be misclassified as not_cyberbullying when cues are weak or ambiguous.
- Model comparison: 0.91 accuracy and 0.90 Macro F1-score are achieved by both RoBERTa-base and BERT-base.RoBERTa-base also has a slightly higher weighted F1-score.
- Class-level performance: RoBERTa-base achieves the strongest class-level performance on ethnicity/race and gender categories.BERT-base marginally outperforms RoBERTa-base on religion, but performs more weakly on other sensitive categories.
- Model comparison: DistilBERT underperforms the other models, showing a trade-off between computational efficiency and detection reliability.Its higher error rate makes it less appropriate for safety-critical cyberbullying detection despite potential suitability for resource-constrained settings.
- Model selection: RoBERTa-base is selected as the primary model because it combines strong global performance with more balanced sensitive-category results.The selection follows its overall robustness relative to BERT-base and DistilBERT.
5. Importance of the Application of CareGuard in Real Life
CareGuard is presented as a practical monitoring tool for settings where harmful online interactions affect safety and well-being. Proposed applications span education, social platforms, government, workplaces, and healthcare organizations.
- Education and platforms: Schools and universities can use CareGuard to monitor online interactions and enhance student safety.The proposed use focuses on detecting harmful content in educational settings.
- Education and platforms: Social media platforms can integrate CareGuard to detect harmful content and enforce community guidelines.The paper names Facebook and Instagram as example platforms.
- Public and workplace uses: Non-profits and government agencies can use CareGuard to develop policies and awareness programs against online harassment.Employers may also apply such systems to digital workplace harassment.
- Healthcare applications: Hospitals and medical organizations can use cyberbullying detection tools to protect patients, staff, and institutional reputation.The paper connects these tools with proactive monitoring of online interactions and respectful digital environments.
6. Discussion
The discussion interprets CareGuard as an early-warning extension of transformer-based cyberbullying classification. Results favor RoBERTa-base overall but also expose sensitivity to class imbalance, implicit language, deployment context, and validation requirements.
- Interpretation of experimental results: RoBERTa-base provides the most balanced overall performance, with 0.91 accuracy, 0.90 Macro F1-score, 0.9129 weighted F1-score, and 0.88 Macro Recall.BERT-base performs comparably overall, whereas DistilBERT declines across major metrics.
- Interpretation of experimental results: DistilBERT records 0.8777 accuracy and 0.8526 Macro F1-score, below both RoBERTa-base and BERT-base.The comparison illustrates a measurable performance cost associated with model distillation.
- Class-level interpretation: RoBERTa-base achieves F1-scores of 0.89 for ethnicity/race, 0.90 for gender, 0.95 for not cyberbullying, and 0.85 for religion.Religion is comparatively difficult to distinguish from non-bullying or ambiguous language.
- Class-level interpretation: Sensitive-category errors may be driven by class imbalance and semantic ambiguity, so high overall accuracy can coexist with meaningful category-specific failures.This matters especially for safety-oriented applications.
- Comparison with previous studies: CareGuard extends classification by combining emotion-aware filtering, transformer classification, and LLM-based post-modeling analysis.The added stages are intended to provide contextual information for interpreting detected harmful interactions.
- Early warning and mental health support: The framework adopts an early-warning perspective by analyzing emotional characteristics, targeted entities, and semantic information after classification.These analyses can produce a more interpretable warning signal for subsequent human review.
- Early warning and mental health support: CareGuard is intended as a decision-support and early-warning framework rather than a clinical diagnostic system.Healthcare or mental-health deployment requires additional validation, human oversight, privacy safeguards, and domain-specific evaluation.
- Limitations: The framework is trained and evaluated on a single public dataset, limiting evidence about linguistic diversity and evolving cyberbullying across platforms.Vocabulary, slang, communication styles, and community norms may create domain-shift challenges during deployment.
7. Conclusion
CareGuard is an early-warning system for harmful online interactions related to cyberbullying and mental health concerns. Its integrated approach supports detection, interpretation, early warning, and decision-making for online safety applications.
- CareGuard integrates emotion-aware filtering, transformer-based classification, and LLM-based post-analysis.The combination is intended to improve detection accuracy and interpretability.
- RoBERTa-based models achieve the most balanced performance, particularly for sensitive categories.
- The framework supports early warning and decision-making for real-world online safety applications.
- Future work will address generalization, fairness, and multimodal analysis across diverse environments.