PaperlayerView exact revision on arXiv

Source-linked AI summary

REFLECT: Role-Aware Detection of Sentiment, Emotion, Politeness, and Formality

Sangpil Youm, Annabel Stocks Natalias, Hayden Heninger, Leah Jones, Xiaoyang Jin, Xiaoya Zhang, Bonnie Dorr

arXiv:2610.13797v1cs.CL

TL;DR

SEPF detection often relies on contextual language representations while leaving participant-role structure implicit. REFLECT maps these dimensions to theory-grounded semantic roles in transformer attention, and improves benchmark detection of emotion, politeness, and formality over strong BERT-based baselines.

  • Problem

    SEPF tasks are typically treated as separate surface-level classifications, without explicitly modeling participant roles despite their relevance to internal states and interpersonal cues.

    Source: Introduction · p. 1

  • Method

    REFLECT incorporates theory-grounded role configurations into transformer self-attention and fuses role-aware representations with contextual embeddings through a learned gate.

    Source: Introduction · p. 2, Fusion and Prediction · p. 5

  • Results

    Across four benchmark datasets, REFLECT outperforms strong BERT-based baselines for emotion, politeness, and formality, with statistically significant gains in those three tasks.

    Source: Introduction · p. 2, Conclusion · p. 7

  • Takeaways & Limitations

    REFLECT also improves emotion prediction on real-world mental-health data and provides traceable, role-grounded explanations.

    Source: Conclusion · p. 7

    The real-world evaluation uses outgoing messages without recipients’ replies and self-reported emotion labels; interpretability analysis is qualitative and illustrative.

    Source: Role-Aware Emotion Detection on EMA Data · p. 7, Conclusion · p. 7

Abstract

from arXiv · show

Detecting communicative meaning in dialogue requires modeling not only lexical content but also the participant-role structures through which sentiment, emotion, politeness, and formality (SEPF) are expressed. Existing approaches rely on contextual language representations while leaving semantic roles implicit. We introduce a role-aware transformer architecture that maps the four SEPF dimensions to theory-grounded Semantic Role Labeling (SRL) configurations, incorporating these configurations directly into the transformer's self-attention mechanism. Role-aware attention models non-directional internal states (sentiment and emotion), while directional role-aware attention captures initiator-to-recipient relations for external cues (politeness and formality). The resulting role-aware representations are fused with contextual embeddings through a learned gate that adaptively weights how much role information each input contributes, rather than appending semantic roles as an auxiliary feature vector. Across four benchmark datasets, the framework yields statistically reliable gains over strong BERT-based baselines for emotion, politeness, and formality, with gains concentrated in theoretically predicted cases such as low-frequency, complex emotions and highly polite expressions; sentiment improvements are marginal and non-significant, suggesting that sentiment is conveyed more through lexical choice than participant-role structure. Additional evaluation on real-world data demonstrates that the framework improves both predictive performance and model traceability. These results demonstrate that linguistically grounded modifications to transformer self-attention improve the detection of communicative meaning, generalize beyond benchmark datasets to real-world mental-health data, and yield traceable, role-grounded explanations that extend beyond predictive performance to interpretability.

Introduction

REFLECT addresses the gap between surface-level SEPF classification and the participant-role structures that theories identify as central to communication. It maps these structures into transformer self-attention and reports gains across emotion, politeness, and formality benchmarks.

Introduction

  • Prior computational approaches typically treat SEPF as separate surface-level tasks, leaving the theory-grounded participant-role structures largely unmodeled.Internal states concern one participant, whereas external cues arise from relationships between participants.Source: Introduction · p. 1
  • REFLECT represents utterances through who is speaking, what is discussed, and who is addressed, incorporating corresponding role configurations into transformer self-attention.The framework fuses role-aware and contextual representations through a learned gate.Source: Introduction · p. 2
  • Unlike SemBERT (Zhang et al. 2020), REFLECT injects roles into self-attention, which selects relevant roles for prediction rather than concatenating role embeddings with contextual representations.Automatic SRL assignment means REFLECT requires no manual role annotation.Source: Introduction · p. 2
  • The paper proposes theory-grounded role mappings and complementary attention mechanisms to make predictions traceable to semantic-role configurations.Its evaluation spans benchmark datasets and real-world mental-health dialogue.Source: Introduction · p. 2
  • Across four benchmark datasets, REFLECT outperforms strong BERT-based baselines for emotion, politeness, and formality, with largest gains in low-frequency emotions and highly polite expressions.The framework incorporates interpretable semantic-role representations without sacrificing predictive performance.Source: Introduction · p. 2

Related Work

Related work commonly approaches sentiment, emotion, politeness, and formality as separate detection tasks. REFLECT instead frames them as expressions of shared interactional structure involving participants, events, and evaluative context.

Related Work

  • REFLECT treats the four tasks as expressions of shared interactional structure, distinguishing internal states from external cues between participants.This framing motivates theory-grounded role mappings.Source: Related Work · p. 2

Sentiment Detection

  • Sentiment detection spans lexicon-based methods (Mitra and Mohanty 2021), engineered-feature models (Rahat, Kahir, and Masum 2019; Bayhaqy et al. 2018), contextual BERT models, and prompted decoder-only LLMs.REFLECT maps sentiment through semantic-role configurations linking events to evaluative context.Source: Related Work · p. 2

Emotion Detection

  • Emotion detection has progressed from supervised feature-based classifiers to embedding models, BERT-family systems, fine-grained datasets, and prompt-based LLM approaches.REFLECT models emotional meaning through roles connecting events, experiencers, and modifiers.Source: Related Work · p. 2

Politeness Detection

  • Politeness detection has moved from feature-based classifiers to neural, transformer, and graph-based models, while REFLECT represents speakers, recipients, and mitigation through directional roles.PoliteLex-based RoBERTa highlights lexical and phrasal predictive cues.Source: Related Work · p. 2

Formality Detection

  • Formality research includes linguistic-feature classifiers, neural models, and transformer approaches; REFLECT represents participant relationships and predicts a continuous score from 0 to +1.Prior approaches include a hybrid DeBERTa and BiLSTM model with the strongest reported performance.Source: Related Work · pp. 2–3

Theory-Grounded Role Mappings

REFLECT operationalizes four communication constructs as semantic-role configurations. Sentiment and emotion are internal states, while politeness and formality are external cues arising through participant relationships.

Theory-Grounded Role Mappings

  • The four mappings divide into internal states centered on an individual and external cues arising from relationships among participants.This distinction motivates complementary attention mechanisms.Source: Theory-Grounded Role Mappings · p. 3
  • Sentiment is mapped to Predicate (PRED) plus Evaluative Modifier (ARGM), linking an event to its evaluative orientation, intensity, or contextual stance.The predicate anchors the evaluated event; modifier structures encode evaluation.Source: Theory-Grounded Role Mappings · p. 3
  • Emotion is mapped to Predicate (PRED) plus Affected Entity (ARG1) plus Modifier (ARGM), representing an affective event, its affected entity, and contextual intensity or manner.The mapping treats emotional meaning as relationally structured.Source: Theory-Grounded Role Mappings · p. 3
  • Politeness is mapped directionally as Initiator (ARG0) to Recipient (ARG2), with Mitigation (ARGM) encoding hedging, softening, or mitigation.The configuration models interactions between distinct communicants.Source: Theory-Grounded Role Mappings · p. 3
  • Formality is mapped as Initiator (ARG0) to Affected Entity (ARG1), whose presence, omission, or positioning signals degrees of social distance and communicative formality.The mapping treats formality as an interactional phenomenon involving participant structure.Source: Theory-Grounded Role Mappings · p. 3

REFLECT Framework

REFLECT integrates theory-mapped semantic roles directly into transformer attention, using different mechanisms for internal states and external cues. It combines role-aware representations with BERT representations through concatenation or input-specific gated fusion.

REFLECT Framework

Benchmark Datasets

  • For internal-state detection, GoEmotions supplies 27 emotion categories plus neutral; sentiment uses four consolidated labels, while politeness and formality use separate binary-classification datasets.Emotion categories are mapped onto a two-dimensional valence-intensity space; politeness uses Holistic Politeness and formality uses GYAFC.Source: Benchmark Datasets · p. 4

Architectural Overview of SRL-Informed Detection

  • The architecture pools selected role representations for internal states and uses frame-level role and position information for directional external-cue attention.Both branches combine their role-aware representation with BERT through feature concatenation or gated fusion before task-specific prediction.Source: Architectural Overview of SRL-Informed… · p. 4

Role-Aware and Directional Attention Mechanisms

  • Internal-state role pooling aggregates tokens assigned to each task role, masks unrealized roles, and attends from [CLS] over pooled role vectors.For sentiment the role set is PRED and ARGM; for emotion it is PRED, ARG1, and ARGM. Frame-level role-aware features are aggregated across valid frames.Source: Role-Aware and Directional Attention… · pp. 4–5

Fusion and Prediction

  • Gated fusion aligns role-aware information to BERT’s hidden space, then uses an input-specific gate to control its contribution as a residual update.The gate modulates hidden dimensions independently, allowing role information to refine rather than overwrite contextual representations; concatenation instead treats role information identically across inputs.Source: Fusion and Prediction · p. 5

Experiments and Results

Across four communication tasks, SRL-informed BERT configurations outperform baselines, with statistically significant gains for emotion and formality but not sentiment. The strongest configurations and gains vary by task, with politeness improvements concentrated in higher-politeness ranges.

  • Models are compared on official train/dev/test splits, with development-set checkpoint selection and held-out test reporting across six configurations per task.The six configurations combine two BERT baselines and four SRL-informed variants; each configuration uses a single training run.Source: Experiments and Results · p. 5
  • SRL-informed architectures outperform baselines across four tasks; Table 1 reports significant gains for emotion and formality, but not sentiment.Source: Table 1 · p. 6
  • Sentiment’s best SRL-informed configuration reaches macro-F1 0.6695, but its improvement over BERT (MLP) is marginal and not statistically significant.McNemar’s test: p = .1527.Source: Experiments and Results · p. 6
  • Gated fusion with a linear head yields the highest emotion macro-F1, with significant paired correctness gains and larger improvements in low-frequency, complex emotions.For rare categories, grief rises from 0.000 to 0.286, relief from 0.167 to 0.222, and pride from 0.316 to 0.476.Source: Experiments and Results · p. 6
  • Directional SRL-informed attention achieves the highest overall politeness correlation, with gains concentrated in the Slightly Polite and Polite ranges.Slightly Polite: r = 0.299 vs. 0.246 (+0.053); Polite: r = 0.435 vs. 0.326 (+0.109).Source: Experiments and Results · p. 6
  • Gated fusion with a linear head leads formality detection, reaching F1 0.9480 and recall 0.9345, with a significant difference from BERT (linear).McNemar’s test gives p = 3.0 × 10−5.Source: Experiments and Results · p. 6

Mental-Health Data

The paper evaluates REFLECT on naturalistic adolescent EMA data and compares SRL-grounded explanations with ungrounded prompts. The evaluation reports emotion-prediction gains and role-linked rationales, while remaining limited to emotion labels from outgoing messages.

Mental-Health Data

  • Naturalistic EMA pairs 1,924 messages from 74 adolescents aged 13–15 with daily self-reported emotion ratings.Participants were monitored over one or two 21-day bursts; the messages were captured during real-world mental-health monitoring.Source: Mental-Health Data · pp. 6–7

Role-Aware Emotion Detection on EMA Data

  • The EMA evaluation is emotion-only because it has outgoing messages without recipients’ replies and labels only for self-reported emotion ratings.The data also support a correlation between PRED+ARG1+ARGM presence and higher baseline likelihood of target emotional expressions across all 14 categories (χ2 = 47.87, p = 6.87 × 10−6).Source: Role-Aware Emotion Detection on EMA Data · p. 7
  • SRL-informed BERT with feature concatenation achieves Pearson correlation 0.5754 versus the strongest baseline’s 0.5687, with significant correlations for 9 of 14 emotions.The baseline performs better on four emotions, while neither model reaches significance for happy.Source: Table 2 · p. 6, Role-Aware Emotion Detection on EMA Data · p. 7

Interpretability of Role-Aware Detection

  • Prompts with and without SRL grounding isolate its contribution; grounded explanations reference prediction-linked role components, a structural prerequisite rather than full interpretability without systematic human evaluation.The politeness comparison contrasts lexical and grammatical cues with explicit references to ARG0 and ARG2.Source: Interpretability of Role-Aware Detection · p. 7, Table 3 · p. 7

Conclusion

REFLECT integrates semantic-role-conditioned mechanisms into transformer representations, improving detection across benchmark tasks while making role configurations behind predictions interpretable. Its real-world mental-health evaluation improves emotion prediction, but the interpretability analysis remains qualitative and illustrative.

Conclusion

  • REFLECT improves all four detection tasks over baseline transformers, with statistically significant gains for emotion, politeness, and formality.Sentiment gains are not characterized in these passages.Source: Conclusion · p. 7
  • The architecture combines role-aware and directional attention, feature concatenation, and gated fusion to model predicate-argument dependencies across SEPF tasks.Analyzing emphasized role configurations makes the semantic-role basis of predictions interpretable.Source: Conclusion · p. 7
  • On real-world mental-health data, REFLECT improves emotion prediction over baseline transformers, while its interpretability analysis is qualitative and illustrative.Systematic large-scale evaluation of structural correspondence and perturbation-based explanation faithfulness remains future work.Source: Conclusion · p. 7

Related papers

Loading 2610.13797v1…