BenCoDiC: A Large-Scale Contextual Cyberbullying Corpus for Bengali Political News Discourse FULL VERSION

A. Cyberbullying Corpora in High-resource Languages

• High-resource-language research includes comment-level datasets and session-level corpora with richer video, image, user, temporal, and social-graph context.Examples include media-session, Instagram session-based, and Chinese session-based resources.Source: §I · p. 1, A. Cyberbullying Corpora in… · p. 2

I. INTRODUCTION

• Existing Bengali corpora often lack conversational links and contextual metadata, provide limited annotation documentation, and are small or inaccessible.These gaps constrain context-aware detection, reproducibility, and downstream use.Source: §I · p. 1
• BenCoDiC preserves caption–comment–reply relationships and organizes content and metadata for context-aware detection and multidimensional analysis.Its two-tier human annotation with expert verification achieved Fleiss’ κ = 0.86 and Krippendorff’s α = 0.86; the corpus contains 50,250 publicly available labeled samples.Source: §I · p. 2, A. Cyberbullying Corpora in… · p. 2

B. Bengali Cyberbullying and Related Text Resources

• Existing Bengali resources generally lack conversational hierarchy and contextual metadata, while annotation documentation, reliability reporting, scale, and public access vary.Table I groups comparison features around the three research gaps; a checkmark denotes explicit reporting, not quality or strength.Source: B. Bengali Cyberbullying and Related… · pp. 2–3

III. BENCODIC CORPUS

• The corpus-development process includes source selection, keyword retrieval, data modeling, extraction, cleaning, anonymization, annotation, and reliability evaluation.The corpus is finalized after annotation reliability is confirmed.Source: §III · p. 3, Figure 1 · p. 4

A. Source Selection

• The study targets politically focused discussion on eight Bengali news channels during Bangladesh’s July–August 2024 unrest.Channels were selected using subscriber count, uploaded videos, total views, and channel age.Source: A. Source Selection · pp. 3–4

B. Keyword Search

• The 100 randomly selected videos span six political categories and were drawn from July 24–November 10, 2024, using manual party and leader keyword searches.Selection was approximately balanced across eight channels and political categories.Source: B. Keyword Search · p. 4

C. Data Modeling

• The CyberbullyingAnalysis fact table links nine dimensions, while the Context dimension connects comments to captions and replies to both parent comments and captions.Dimensions cover users, videos, textual content, context, time, engagement, political parties, target identities, and language; the schema supports detection and behavioral, network, and link-prediction analyses.Source: C. Data Modeling · pp. 4–5, Figure 2 · p. 4

D. Data Extraction

• Two-stage extraction retrieved captions, comments, replies, and metadata, yielding 50,352 textual instances from 100 videos before cleaning.Auxiliary political-party, target-identity, and language attributes were treated as metadata and did not undergo the binary-label inter-annotator agreement procedure.Source: D. Data Extraction · pp. 5–6

E. Data Cleaning

• Cleaning removed 102 noisy samples, reducing the dataset from 50,352 to 50,250 instances; single-word and emoji-only samples were retained.Numeric-only samples and unsupported non-Bengali or non-Latin scripts were excluded.Source: E. Data Cleaning · p. 6

F. Data Anonymization

• Channel-related identifiers, including user IDs, handlers, and titles, were anonymized while consistent mappings preserved the same anonymized identity across comments and replies.Source: F. Data Anonymization · p. 6

1) Annotation with AI Chatbots:

• None of the five tested free-tier chatbot configurations met the 90% reliability criterion; Grok 3 had the highest mean accuracy at 79.75%.The 95% confidence-interval upper bounds remained below the threshold; these results are limited to the 500-sample evaluation and tested free-tier versions.Source: Table III · p. 6, 1) Annotation with AI Chatbots · p. 6
• The chatbot trials found bias patterns and weak hierarchical labeling, particularly for replies, alongside Grok 3’s tendency to over-predict bully labels.The other chatbots tended toward not-bully labels.Source: 1) Annotation with AI Chatbots · p. 6

2) Human Annotation:

• The human process used three independent Level-1 annotators, majority voting, and review by three domain experts, with further guideline-based training if expert agreement fell below 0.81.The Level-1 annotators were 51 final-year computer science undergraduates; Level-2 reviewers were three faculty members with NLP research experience.Source: 2) Human Annotation · p. 7
• BenCoDiC labels target-directed content as cyberbullying when it meets at least one of seven contextual criteria, without requiring inferred intent, repetition, or power imbalance.The criteria are applied as non-exclusive checks, and the retained annotation is binary.Source: 2) Human Annotation · p. 7

H. Annotation Assessment

• Fleiss’ κ and Krippendorff’s α rose from 0.81 at Level-1 to 0.86 at Level-2; unanimous decisions increased from 86.14% to 89.87%.The share of 2–1 majority decisions fell from 13.86% to 10.13%.Source: Table IV · p. 8, H. Annotation Assessment · p. 8
• 3,280 samples (6.53%) changed labels during expert verification: 1,282 changed from not bully to bully and 1,998 from bully to not bully.Source: H. Annotation Assessment · p. 8
• Disagreement was not quantified by content type, language category, or other sample characteristics, despite qualitative ambiguity involving context-dependent expressions, replies, Romanized Bengali, code-mixing, and emojis.The reported ambiguity observations were not presented as subgroup disagreement rates.Source: H. Annotation Assessment · pp. 8–9

I. Final Corpus

• The final corpus contains 50,250 samples with twenty-four columns and is publicly available; Table V presents examples of its cyberbullying classes.Source: §I · p. 9, Table V · p. 9

J. Ethical Considerations

• The data were collected from publicly accessible YouTube content for research, and channel identifiers were anonymized without attempting to identify or contact users.Political-party, language, and target-identity attributes describe textual-instance context, not verified user characteristics.Source: J. Ethical Considerations · p. 9
• Annotators received guidance and training for potentially distressing content, could pause or withdraw, and based decisions on observable text and conversation rather than assumed intentions or personal characteristics.Source: J. Ethical Considerations · p. 9
• The corpus reflects selected Bengali news-channel political discussions from a specific period and is not representative of all Bengali speakers, political communities, or online interactions.The authors also caution against using it for harassment, deanonymization, surveillance, political profiling, or punitive decisions about individuals.Source: J. Ethical Considerations · p. 9

• 50,250 samples comprise 566,435 words, with approximately 60% labeled Bully and 40% Not Bully; Bully samples average 12.52 words versus 9.41 for Not Bully.The class word-count distributions largely overlap, but median lengths are 8 words for Bully and 5 for Not Bully samples.Source: K. Dataset Statistics · p. 9, Table VI · p. 9, Figure 4 · p. 10
• Comments dominate the corpus (26,806 Bully and 17,562 Not Bully), followed by replies (3,274 and 2,508); captions comprise just 3 Bully and 97 Not Bully samples.Comments account for approximately 88.3% of all samples.Source: K. Dataset Statistics · p. 10, Figure 5 · p. 10
• Spearman’s correlations were −0.1314 for like count and −0.0628 for reply count, with both associations statistically significant (p < 0.001).Not Bully comments received more likes on average (3.35 versus 1.62) and replies (0.18 versus 0.10), while both classes had zero medians for both measures.Source: K. Dataset Statistics · p. 10, Table VIII · p. 11
• August has the most samples (15,432), while November has the highest Bully proportion (72.01%); after November, monthly counts fall below 120.July and March are partial months, and the Bully proportion shows no consistent upward or downward trend.Source: K. Dataset Statistics · p. 10, Table IX · p. 11, Figure 6 · p. 11
• The eight channels have 12 or 13 videos each, but sample counts vary; Jamuna TV contributes the most samples and Bully instances, while DBC NEWS contributes the fewest in both classes.Both Bully and Not Bully samples occur across all eight channels.Source: K. Dataset Statistics · p. 10, Table X · p. 11
• Bully proportions range from 43.94% for Jamaat to 77.70% for JaPa, while both classes occur in every political-party category.AL, BNP, JaPa, Jamaat, and NCP each have 16 videos; Other has 20.Source: K. Dataset Statistics · pp. 10–11, Table XI · p. 11
• Organizations and people groups account for 50.64% of Bully samples; males are the largest target category (11,900), while only 12.50% of general-people instances are Bully.The target–language cross-tabulation also finds more Not Bully than Bully samples for general people across all five language categories.Source: K. Dataset Statistics · pp. 11–12, Figure 7 · p. 12
• Bengali has the most samples (24,462 Bully and 14,459 Not Bully), followed by Romanized Bengali (4,144 and 1,739); English and emoji-only samples show the opposite class pattern.Code-mixed, English, and emoji-only samples are considerably less frequent; the five language categories are substantially imbalanced.Source: K. Dataset Statistics · p. 12, Figure 8 · p. 12

• The section evaluates context-aware cyberbullying classification by comparing context-free and flat-context settings across six transformer-based models.Source: §IV · p. 12
• The evaluation uses video-disjoint partitions, repeated evaluations with multiple split and run seeds, macro-averaged and weighted metrics, and paired statistical analysis.Source: §IV · p. 12
• Additional analyses examine a hierarchical-context formulation, controlled ablations of individual contextual elements, and representative prediction errors.Source: §IV · p. 12

A. Experimental Protocol

• All content associated with a video is assigned to one partition, preventing validation or test context from originating in a training video.The 100 videos are divided 60:20:20 into train, validation, and test sets, with three split seeds.Source: A. Experimental Protocol · pp. 12–13
• Across the three selected splits, sample proportions remain near 60:20:20 and Bully proportions range from 59.14% to 61.03%.Content-type distributions are also comparable across corresponding partitions.Source: A. Experimental Protocol · p. 13

B. Context Formulation

• The paired formulations keep prediction targets, labels, and data partitions unchanged, isolating the input representation as the setting that varies.Source: B. Context Formulation · p. 13
• Context-free inputs classify each caption, comment, or reply using only its own text; flat-context inputs combine available conversational context with the target text.For replies, the flat input includes the video caption, parent comment, and target reply, in conversational order.Source: B. Context Formulation · p. 13

C. Models and Training Configuration

• Six encoder-based models span Bengali-specific, Bengali-English, and multilingual pretraining, enabling context comparisons across differing linguistic coverage.The models are BanglaBERT, Bangla-Bert-Base, BanglishBERT, IndicBERT v2, mBERT, and XLM-R-base.Source: C. Models and Training Configuration · p. 13
• All models use learning rate 2 × 10−5, weight decay 0.01, batch size 16, and three epochs; checkpoints are selected by validation macro-F1.Inputs are tokenized with model-specific tokenizers and a maximum sequence length of 192 tokens.Source: C. Models and Training Configuration · pp. 13–14
• Each model-setting combination is evaluated across three video-disjoint split seeds and three run seeds, yielding nine matched evaluations per model.Context-free and flat-context settings use the same split and run seed combinations.Source: C. Models and Training Configuration · p. 14

D. Evaluation Metrics and Statistical Protocol

• Macro-F1 is primary among the reported metrics, and contextual gain is measured as the paired flat-context minus context-free macro-F1 difference.A 95% paired confidence interval is estimated from the nine matched differences; an interval wholly above zero indicates evidence of positive mean improvement.Source: D. Evaluation Metrics and Statistical… · p. 14

E. Comparison of Context-Free and Flat-Context Results

• 0.8277 ± 0.0115 (context-free) to 0.8353 ± 0.0085 (flat-context) macro F1 for BanglaBERT; all five mean metrics improve across six models, and BanglaBERT leads overall.Table XIV reports means ± standard deviations across nine evaluations.Source: E. Comparison of Context-Free and… · p. 14, Table XIV · p. 15
• 0.0048 to 0.0084 macro F1 mean gains; all six models' 95% paired confidence intervals lie above zero under the stated protocol.The largest mean gain is reported for mBERT; the results are considered statistically significant under the paired protocol.Source: E. Comparison of Context-Free and… · p. 14, Table XV · p. 15

F. Hierarchical Context and Context-Element Ablation Analysis

• Hierarchical context encodes caption, parent comment, and target separately with shared BanglaBERT, concatenates available representations, then classifies.Unavailable context components are replaced with zero vectors.Source: F. Hierarchical Context and… · p. 15
• 0.8320 ± 0.0114 macro F1 for hierarchical context, above context-free BanglaBERT (0.8277 ± 0.0115) but below flat-context BanglaBERT (0.8353 ± 0.0085).The same ordering is reported for weighted F1-score and accuracy.Source: F. Hierarchical Context and… · p. 15
• Reply-only ablations hold encoder, fusion, classifier, training, splits, and seeds fixed, varying the availability of caption and parent-comment context.Replies were selected because they are the only content type with both contextual sources available.Source: F. Hierarchical Context and… · p. 15
• 0.7789 ± 0.0260 (A0), 0.7817 ± 0.0204 (A1), 0.7855 ± 0.0223 (A2), and 0.7882 ± 0.0186 (A3) reply-level macro F1; A3 is highest.A3 gains approximately 0.93 percentage points over A0; parent-comment A2 gains 0.66 points, more than caption-only A1 at 0.28 points.Source: F. Hierarchical Context and… · pp. 15–16, Table XVI · p. 16

G. Qualitative Error Analysis

• The error analysis compared context-free and flat-context predictions on the same instances in one evaluation (split seed 0, run seed 12).It examined corrected errors, introduced errors, and errors that persisted under both settings.Source: G. Qualitative Error Analysis · p. 16
• Context corrected errors for context-dependent expressions, including a reply whose parent comment enabled the correct bully prediction.It also corrected a false positive when the associated caption was considered.Source: G. Qualitative Error Analysis · p. 16
• Context sometimes introduced misleading cues, flipping previously correct predictions in either direction.The examples include a false positive after adding a caption and a false negative under flat context.Source: G. Qualitative Error Analysis · p. 16
• False negatives can persist despite context; implicit insults, Romanized Bengali expressions, and symbolic forms remain challenging.Source: G. Qualitative Error Analysis · p. 16
• Benign or supportive expressions in politically charged discussions can remain false positives under both settings.Source: G. Qualitative Error Analysis · p. 16

V. POTENTIAL APPLICATION TASKS

• Cyberbullying detection can classify instances as bully or not bully using textual content and conversational context.Source: §V · p. 16
• Persistent anonymized user identifiers and labeled histories support identifying repeat perpetrators and examining their activity patterns.Source: §V · p. 16
• Combining user, video, context, time, engagement, political-party, language, and target-identity dimensions supports behavioral, targeting, and community-interaction analyses.Source: §V · p. 16
• Perpetrator networks can link anonymized users by behavioral or contextual similarities for potential association prediction, while metadata may support multitask and few-shot learning.Source: §V · p. 16

VI. CONCLUSION

• Conversational context yielded modest but consistent improvements over isolated-text classification.Source: §VI · p. 17
• BenCoDiC contains 50,250 instances in a nine-dimensional star schema, and its expert-verified two-tier labels achieved Fleiss’ κ = 0.86 and Krippendorff’s α = 0.86.Source: §VI · pp. 16–17
• Future work will extend coverage beyond political news, improve underrepresented language categories, align with FAIR and Linked Data principles, and investigate ontology- and graph-based methods.Source: §VI · p. 17
• The corpus is bounded to selected Bengali YouTube political-news channels and a specific period, and auxiliary political-party, target-identity, and language metadata lack the labels’ reliability assessment.Source: LIMITATIONS · p. 17