PaperlayerView exact revision on arXiv

Source-linked AI summary

BanglaRhet: Benchmarking Classical and Transformer Models for Rhetorical and Persuasion Detection in Bangla Political Speech

Rohit Kumar Sen, Anik Chowdhury

arXiv:2610.09464v1cs.CL

TL;DR

Fine-grained rhetorical and persuasion detection in Bangla political speech remains underexplored. This study benchmarks transformers against classical TF-IDF baselines using BanglaRhet, with BanglaBERT achieving 65.40% macro-F1 for rhetorical detection and 66.46% for persuasion detection.

  • Problem

    Bangla political NLP has mainly addressed sentiment and opinion, while fine-grained rhetorical and persuasion techniques lack an equivalent Bangla resource and remain underexplored in benchmark evaluation.

    Source: §I · p. 1

  • Method

    The study benchmarks Bangla-specific and multilingual transformers against classical TF-IDF baselines on fine-grained detection tasks using BanglaRhet’s 30,289 manually annotated political speech segments.

    Source: §I · p. 1

  • Results

    65.40% macro-F1 for rhetorical detection and 66.46% for persuasion detection; BanglaBERT led both tasks, gaining 19.2 and 13.8 points, respectively, over the best tuned classical baseline.

    Source: A. Benchmark Results · p. 4

  • Takeaways & Limitations

    The results establish initial benchmark baselines and indicate that rhetorical and persuasion-aware political discourse analysis in Bangla remains challenging.

    Source: §VII · p. 6

    The benchmark remains imbalanced, making minority or overlapping categories harder to classify despite controlled downsampling and class-weighted loss.

    Source: A. Limitations · p. 6

Abstract

from arXiv · show

Political discourse often uses rhetorical and persuasive language to frame narratives, influence public opinion, and mobilize audiences. While Bangla natural language processing has made progress in sentiment analysis and opinion mining, systematic benchmarking of transformer models for fine-grained rhetorical and persuasion technique detection in Bangla political speech remains largely underexplored. This paper presents a benchmark study of transformer-based models for detecting rhetorical form and persuasive intent in Bangla political discourse. Using BanglaRhet, a manually annotated corpus of 30,289 Bangla political speech segments collected from publicly available political news sources, we formulate two supervised single-label classification tasks: rhetorical technique detection (contrast, repetition, exaggeration, metaphor, rhetorical questions) and persuasion technique detection (blame assignment, call to action, unity call, moral, emotional, and logical appeals). We evaluate four transformer-based models, BanglaBERT, BanglaBERT-Base, SahajBERT, and XLM-RoBERTa-Base, against classical TF-IDF baselines. BanglaBERT achieves the highest performance, with 65.40% macro-F1 for rhetorical technique detection and 66.46% for persuasion technique detection, outperforming the best tuned classical baseline by 19.2 and 13.8 macro-F1 points, respectively. Class-level analysis indicates that errors are mainly associated with semantic overlap among labels, figurative language, and class imbalance. The results provide initial benchmark baselines for Bangla rhetorical and persuasion-aware political discourse analysis and highlight the need for context-aware and multi-label modeling.

I. Introduction

The introduction frames fine-grained detection of rhetorical form and persuasive intent in Bangla political discourse as an underexplored challenge. It presents BanglaRhet and a benchmark comparing transformer models with classical TF-IDF baselines.

I. Introduction

  • Rhetorical techniques describe how a statement is expressed, while persuasion techniques describe its communicative purpose; figurative language, emotional wording, implicit intent, and overlapping categories make both difficult to detect.The paper argues that models need stylistic and pragmatic cues beyond surface-level meaning.Source: §I · p. 1
  • BanglaRhet provides 30,289 manually annotated Bangla political speech segments for a benchmark evaluating Bangla-specific and multilingual transformer models, –[11].The evaluated models include BanglaBERT, SahajBERT, Bangla-BERT-Base, and XLM-RoBERTa-Base.Source: §I · p. 1
  • The work contributes a fine-grained benchmark, compares transformer representations with classical TF-IDF baselines, and analyzes challenges from overlapping label semantics, figurative usage, and class imbalance.The stated contributions include evaluation on the manually annotated corpus and class-level analysis of these challenges.Source: §I · pp. 1–2

II. Background and Related Work · A. Rhetorical and Persuasive Language in Political Discourse · B. Political Text Analysis and Persuasion Detection

The background separates rhetorical form from persuasive function and surveys political-text and fine-grained persuasion research, identifying the lack of a Bangla approach that models both at the technique level.

A. Rhetorical and Persuasive Language in Political Discourse

  • Rhetorical categories describe how political language is expressed, whereas persuasion categories describe a statement’s communicative intent or strategic function,.Source: A. Rhetorical and Persuasive Language… · p. 2
  • The study’s rhetorical taxonomy includes contrast, repetition, metaphor, exaggeration, and rhetorical questions, while its persuasion taxonomy includes blame, moral, logical, and emotional appeals, and calls to action.Source: A. Rhetorical and Persuasive Language… · p. 2

B. Political Text Analysis and Persuasion Detection

  • Political text analysis includes sentiment, ideology, stance, argument mining, and persuasion detection, but sentiment polarity cannot distinguish rhetorical techniques with similar sentiment.Source: B. Political Text Analysis and… · p. 2
  • PoliBERT classifies political social-media messages using coarse functional categories, rather than detecting technique-level rhetorical form.Source: B. Political Text Analysis and… · p. 2
  • SemEval-2020 Task 11 showed that fine-grained propaganda techniques are learnable, but no equivalent Bangla taxonomy jointly models stylistic form and communicative function.Source: B. Political Text Analysis and… · p. 2
  • CORPS uses audience applause as a weak, effect-level proxy for persuasiveness, while traditional methods predict a single scalar score,.Source: B. Political Text Analysis and… · p. 2
  • Fine-grained persuasion labels have been used in advertising, online campaigns, and social media, but that work remains limited to English, non-political domains [16].Source: B. Political Text Analysis and… · p. 2

C. Bangla Political NLP and Existing Resources · III. Benchmark Dataset and Task Formulation · A. Data Collection and Segment Extraction

BanglaRhet addresses the lack of Bangla technique-level rhetorical and persuasion analysis with a taxonomy and manually annotated benchmark of political speech segments. The paper describes how it collected and screened those segments from political news and speech-related articles.

C. Bangla Political NLP and Existing Resources

  • BanglaRhet introduces a Bangla-specific taxonomy of nine rhetorical and seven persuasion techniques, a manually annotated benchmark of 30,289 political speech segments, and an evaluation of Bangla-specific versus multilingual pretraining.The authors describe it as, to their knowledge, the first benchmark of its kind for Bangla.Source: C. Bangla Political NLP and Existing… · p. 2
  • Prior Bangla political NLP mainly addresses sentiment or stance, while technique-level rhetorical and persuasion analysis has no Bangla counterpart.Motamot annotates political news for binary sentiment; other work examines audience reaction, ideology or stance, or informal online discussion rather than formal speech.Source: C. Bangla Political NLP and Existing… · p. 2

A. Data Collection and Segment Extraction

  • The corpus draws on 1,890 Motamot links and links from Prothom Alo and Manab Zamin covering 2014–2025; 15,032 articles yielded usable segments.Inaccessible, duplicate, malformed, or non-political pages were removed.Source: A. Data Collection and Segment… · p. 2
  • Segments were included when they contained a direct quotation or reported statement attributed to a political figure or institution, by name or role, as characterized by the outlet.When the speaker’s identity was indeterminable, the segment was retained and marked unknown_speaker.Source: A. Data Collection and Segment… · p. 2
  • Journalistic analysis, background context, and non-political actors’ commentary were excluded, except when retained to construct the None class.Extraction was collaborative, with periodic crosschecks and discussion to resolve ambiguous cases.Source: A. Data Collection and Segment… · p. 2
  • Segments exceeding the 128-token input limit were condensed, with both authors manually verifying preservation of rhetorical and persuasive content.Source: A. Data Collection and Segment… · p. 2

B. Task Formulation, Labels, and Annotation · C. Benchmark Statistics and Imbalance

BanglaRhet frames rhetorical and persuasion detection as separate single-label tasks and documents the annotation process, corpus composition, and substantial class imbalance. The benchmark excludes the exceptionally rare Parallelism class from evaluation because its test count is too small for stable macro-F1.

B. Task Formulation, Labels, and Annotation

  • 30,289 segments were assigned one dominant label in each of two independent tasks: nine rhetorical labels and seven persuasion labels, including None.Multi-label modeling is left for future work; annotators chose the technique most central to each segment’s overall effect.Source: B. Task Formulation, Labels, and… · pp. 2–3
  • Two authors with prior political-discourse study annotated the corpus after jointly piloting examples until disagreements stabilized, especially for conceptually related labels.None was used when no technique was discernible or, less commonly, when borderline language did not clearly meet a single label’s definition.Source: B. Task Formulation, Labels, and… · p. 3
  • Cohen’s κ was 0.67 for rhetorical labels and 0.71 for persuasion labels on a random 500-segment subset before consensus resolution.The authors characterize this as moderate-to-substantial agreement.Source: B. Task Formulation, Labels, and… · p. 3

C. Benchmark Statistics and Imbalance

  • BanglaRhet contains 30,289 segments from 15,032 sources, averaging 48.34 words per segment, with nine rhetorical and seven persuasion labels.Figure 1 presents both label distributions.Source: C. Benchmark Statistics and Imbalance · p. 3, Figure 1 · p. 3
  • Contrast, Emotional Language, and Repetition dominate rhetorical labels, while Blame Assignment, Call to Action, and Unity Call dominate persuasion labels.Parallelism is rare, and the observed imbalance motivates macro-F1 and the class-balancing strategy described in Section IV.Source: C. Benchmark Statistics and Imbalance · p. 3, Figure 1 · p. 3
  • 137 Parallelism instances were over 15 times rarer than the next-smallest class, Metaphor (2,128), yielding about 14 test instances, so Parallelism was excluded.The authors judged that test count too small for stable macro-F1 and opted against oversampling because of near-duplicate memorization risk or merging it with structurally distinct Repetition.Source: C. Benchmark Statistics and Imbalance · p. 3

IV. Experimental Setup · A. Task Setup

The experiments establish benchmarks for rhetorical and persuasion detection in Bangla political speech by evaluating existing transformer models under a fixed setup. The benchmark uses stratified data splits and model-specific tokenization with a 128-token limit.

IV. Experimental Setup

  • The study benchmarks whether Bangla-specific and multilingual transformers learn fine-grained rhetorical and persuasive patterns, rather than proposing a new architecture.The evaluation uses a fixed experimental setting.Source: §IV · p. 3

A. Task Setup

  • The benchmark uses 80%–10%–10% training, validation, and test splits stratified by rhetorical labels, with persuasion labels encoded separately over the same splits.Segments are tokenized with each model’s tokenizer and padded or truncated to 128 tokens.Source: A. Task Setup · p. 3

B. Class Balancing Strategy · C. Baseline Models

The benchmark addresses joint label imbalance through controlled downsampling and class-weighted training, then compares Bangla-specific and multilingual transformers with tuned TF-IDF baselines. The reported ablation and baseline comparisons support the balancing step and transformer models, respectively.

B. Class Balancing Strategy

  • Blame Assignment, Call to Action, and Unity Call comprise 67.2% of segments, and Contrast co-occurs with Blame Assignment 2,923 times versus at most 824 with any other persuasion label.Source: B. Class Balancing Strategy · p. 3
  • Because uniform per-class downsampling cannot correct pairwise skew, the authors manually inspected the co-occurrence table and reduced the ten most overrepresented pairs (Table II).Source: B. Class Balancing Strategy · p. 3, Table II · p. 4
  • The iterative manual reductions are acknowledged as a limitation, but ablation results show training without them yields lower macro-F1 across all four models.Source: B. Class Balancing Strategy · p. 3
  • After Parallelism was removed and controlled downsampling applied, the experimental benchmark contained 22,565 samples from the full 30,289-segment corpus.The processed benchmark was used for the train–validation–test split, and both tasks used class-weighted cross-entropy during training.Source: B. Class Balancing Strategy · pp. 3–4

C. Baseline Models

  • Every transformer model outperformed both tuned TF-IDF classical baselines, which used Logistic Regression and linear SVM.Hyperparameters were selected by validation-set grid search.Source: C. Baseline Models · p. 4
  • The evaluation fine-tuned Bangla-BERT-Base, SahajBERT [9], BanglaBERT, and multilingual XLM-RoBERTa-Base [11] with a standard sequence-classification head.The comparison tests Bangla-specific pretrained models against a multilingual cross-lingual baseline.Source: C. Baseline Models · p. 4

D. Training Configuration and Evaluation … B. Effect of Source-Level Splitting

BanglaBERT leads the reported test-set results on both tasks, while source-level splitting causes only modest macro-F1 drops. Controlled downsampling improves macro-F1 across models, and the training setup prioritizes reproducible, imbalance-aware baselines.

D. Training Configuration and Evaluation

  • Models trained for ten epochs with AdamW, learning rate 2 × 10−5, batch size 16, and maximum sequence length 128, using fixed seed 42 and validation macro-F1 checkpoint selection.Class-weighted cross-entropy used weights inversely proportional to class frequency; extensive hyperparameter tuning was not performed.Source: D. Training Configuration and Evaluation · p. 4
  • Held-out test evaluation reports accuracy and macro-F1, emphasizing macro-F1 because of class imbalance; the dataset, guidelines, preprocessing scripts, and code are slated for public release upon acceptance.The stated goal is reproducible benchmark baselines rather than maximizing architecture-specific performance.Source: D. Training Configuration and Evaluation · p. 4

A. Benchmark Results

  • 65.40% macro-F1 for rhetorical detection and 66.46% for persuasion detection put BanglaBERT 19.2 and 13.8 points above the best tuned classical baseline, respectively.Every transformer model outperformed both classical baselines on both tasks.Source: A. Benchmark Results · p. 4
  • Controlled downsampling improved macro-F1 across all models on both tasks, with BanglaBERT’s largest single gain on rhetorical detection.The ablation retained the same label set, seed, class-weighted loss, and training configuration; Parallelism was excluded in both settings.Source: A. Benchmark Results · p. 4

B. Effect of Source-Level Splitting

  • The source-grouped split used GroupShuffleSplit to keep each article in one split; originally, 61.23% of test and 59.35% of validation segments shared a source with training.The split addresses possible leakage when segments from the same article occur in both training and test sets.Source: B. Effect of Source-Level Splitting · p. 4
  • Under source-level splitting, BanglaBERT’s macro-F1 drops only 1.02 points for rhetorical detection and 1.08 points for persuasion detection.The paper interprets these modest drops as evidence that results are not meaningfully inflated by segment-level splitting.Source: B. Effect of Source-Level Splitting · p. 4

C. Confusion Matrices and Per-Class Analysis

BanglaBERT’s confusion matrices show that fine-grained rhetorical and persuasion categories are often difficult to distinguish when their surface cues or meanings overlap. Categories with clearer lexical or functional cues perform more strongly than several overlapping categories.

C. Confusion Matrices and Per-Class Analysis

  • 46 Emotional Language instances were predicted as Repetition, with additional confusion between Repetition and Contrast and between Exaggeration and Emotional Language.In the example, unity-related lexical repetition led the model to predict Repetition although annotators judged the dominant effect to be emotional appeal to sacrifice.Source: C. Confusion Matrices and Per-Class… · p. 5, Figure 2 · p. 5
  • Unity Call and None were classified more reliably, while Blame Assignment, Moral Appeal, and Emotional Appeal showed more confusion.Moral and Emotional Appeal were often predicted as Blame Assignment when explicit rival-party naming or accusations overrode their underlying framing.Source: C. Confusion Matrices and Per-Class… · p. 5, Figure 2 · p. 5
  • BanglaBERT had stronger per-class F1 for None, Rhetorical Question, and Unity Call than for Emotional Language, Blame Assignment, Moral Appeal, and Logical Appeal.The analysis attributes performance differences not only to class frequency but also to conceptual overlap among fine-grained political-language categories.Source: C. Confusion Matrices and Per-Class… · p. 5, Table VI · p. 5

D. Discussion

BanglaBERT leads both tasks, while downsampling improves macro-F1 across models and confusion patterns reveal substantial semantic overlap among labels. Comparisons with related benchmarks place the task in the context of fine-grained classification.

D. Discussion

  • BanglaBERT leads both tasks, suggesting Bangla-specific pretraining helps fine-grained classification while bag-of-words is insufficient; XLM-RoBERTa-Base is competitive but lower.The discussion suggests multilingual representations do not fully capture Bangla-specific rhetorical and political phrasing.Source: D. Discussion · p. 5
  • Controlled downsampling improves macro-F1 across all models on both tasks, with BanglaBERT’s largest gain at +3.26 points on rhetorical detection.Effects are more mixed for other models; persuasion detection sometimes benefits more, possibly because Call to Action, Unity Call, and None have clearer lexical or functional cues.Source: D. Discussion · p. 5
  • BanglaBERT reaches 88.10% accuracy on Motamot’s coarser binary sentiment task, while SemEval-2020’s best system scored 62.1% micro-F1 and BERT systems reported 0.32–0.45 macro-F1.The authors describe BanglaRhet’s 65.40% and 66.46% macro-F1 results as broadly in line with, and in some respects ahead of, the closest related benchmark.Source: D. Discussion · p. 5
  • Confusion matrices link rhetorical errors to overlap among Emotional Language, Repetition, and Exaggeration, and persuasion errors to overlap among Blame Assignment, Moral Appeal, and Emotional Appeal.The discussion attributes these patterns to segments combining expressive intensity, repetition, and emotion, or combining blame, emotion, and value-based argumentation.Source: D. Discussion · p. 5

VI. Limitations and Ethical Considerations · A. Limitations

The benchmark’s limitations include subjective fine-grained annotation, persistent class imbalance, and data and task formulations that constrain generalizability and representation of political strategies.

A. Limitations

  • Cohen’s kappa was 0.67 for Emotional Language vs. Exaggeration and 0.71 for Emotional Appeal vs. Moral Appeal, indicating moderate to substantial agreement despite unavoidable annotation ambiguity.The categories overlap semantically, making fine-grained political discourse annotation inherently subjective.Source: A. Limitations · p. 5
  • Despite controlled downsampling and class-weighted loss, class imbalance persists, and minority or overlapping categories are harder to classify.The passage relates these challenges to moderate macro-F1 and per-class scores.Source: A. Limitations · p. 6
  • Downsampling relied on manual inspection of the co-occurrence table rather than an automated parameterized rule, so future work should formalize the step.The authors release the exact resulting counts and data for reproducibility.Source: A. Limitations · p. 6
  • The proportion of segments requiring manual condensation was not logged during annotation and cannot be reconstructed retrospectively.Source: A. Limitations · p. 6
  • The data cover articles from 2014–2025, mainly from Bangladeshi political news sources, so performance may vary across time periods, speakers, or Bangla-speaking regions.Source: A. Limitations · p. 6
  • Both tasks use single-label classification with the dominant label per segment, although political statements may contain multiple strategies, motivating future multi-label modeling.Source: A. Limitations · p. 6

B. Ethical Considerations · VII. Conclusion

The paper benchmarks Bangla rhetorical and persuasion detection, finding BanglaBERT strongest while framing the corpus as research and education material rather than a tool for political profiling or manipulation.

B. Ethical Considerations

  • The benchmark uses publicly accessible news articles and speeches, analyzes linguistic patterns rather than political truth or validity, and is intended only for research and education.The authors state it does not endorse or rank political individuals or groups and should not be used for political profiling or voter manipulation; extraction and annotation were manual.Source: B. Ethical Considerations · p. 6

VII. Conclusion

  • 65.40% macro-F1 for rhetorical detection and 66.46% macro-F1 for persuasion detection were BanglaBERT’s highest scores on BanglaRhet’s 30,289 manually annotated speech segments.Source: §VII · p. 6
  • All evaluated transformer models outperformed tuned classical TF-IDF baselines by a wide margin on both detection tasks.Source: §VII · p. 6
  • Controlled downsampling improved macro-F1 across all evaluated models, most notably for BanglaBERT on rhetorical detection.Source: §VII · p. 6
  • Under a fully source-disjoint split, macro-F1 dropped by only about one point, indicating results were not substantially inflated by source-level leakage.Source: §VII · p. 6
  • Errors clustered among semantically related categories, especially overlapping emotional, moral, exaggerative, and blame-oriented language.Source: §VII · p. 6
  • The results establish initial benchmark baselines while indicating that Bangla rhetorical and persuasion-aware political discourse analysis remains challenging.Source: §VII · p. 6
  • Future work will address broader Bangla-speaking political contexts, minority-class modeling, source-level evaluation, and context-aware or multi-label approaches.Source: §VII · p. 6
Loading 2610.09464v1…