Source-linked AI summary
Leveraging Turn-taking Dynamics for Intent Recognition in Multi-party Conversations
Galo Castillo-López, Alexis Lombard, Gaël de Chalendar, Nasredine Semmar
TL;DR
Multi-party intent recognition often overlooks interaction dynamics such as turn-taking, despite the complexity of multiple potential next speakers. The paper introduces a self-supervised multitask method that predicts turn-transition entropy alongside intent, and reports improvements over prior approaches across multiple backbones and corpora. The entropy target can also be learned as a single-task objective, indicating that it carries useful information beyond multitask regularization.
Problem
Most multi-party intent recognition approaches fail to leverage inherent interaction dynamics such as turn-taking, conversation disentanglement, and related cues.
Method
The method predicts self-supervised turn-transition entropy from speaker transitions through an auxiliary regression objective while optimizing intent classification.
Results
The method improves intent detection over previous approaches across three model backbones and one private plus two public multi-party dialogue corpora, while the entropy target is also learnable as a single task.
Takeaways & Limitations
Turn-transition entropy is a useful interaction signal rather than only a multitask regularizer for multi-party intent recognition.
Takeaways & Limitations
The approach assumes speaker identities are available for each utterance, and automatic diarization errors may reduce the quality of the entropy signal.
Abstract
from arXiv · showhide
We propose a multi-task learning approach for multi-party dialogue intent recognition that leverages an auxiliary task that models turn-taking dynamics. Specifically, we introduce turn-transition entropy, a self-supervised target computed from the sequence of speaker transitions, which quantifies the predictability of interaction patterns. Experiments on multiple pre-trained models demonstrate that incorporating this auxiliary task improves intent recognition performance, outperforming existing approaches which ignore multi-party interaction dynamics. We find that our proposed continuous target can be learned as a single-task objective, suggesting that it is an actual signal carrying useful information.
1 Introduction
The paper addresses overlooked interaction dynamics in multi-party intent recognition by introducing turn-transition entropy as a self-supervised auxiliary signal. The measure captures the predictability of speaker transitions without requiring additional annotations or a fixed participant count.
- Multi-party intent recognition has been overlooked relative to dyadic dialogue, and most existing approaches do not exploit turn-taking or other interaction dynamics.
- The proposed approach adds an auxiliary task that captures turn-taking dynamics for multi-party dialogue intent recognition.
- Turn-transition entropy is a self-supervised continuous target computed from speaker-transition pairs using Shannon entropy.
- Low entropy indicates organized turn-taking, whereas high entropy indicates more unpredictable speaker transitions.Figure 1 contrasts structured and varied transition patterns while holding the number of speakers and turn shifts constant.
- The paper evaluates the practical use of turn-transition entropy as an auxiliary task for fine-tuning multi-party intent recognition models.
2 Related Work
Prior work models speaker-related dynamics in dialogue understanding, but the paper applies a self-supervised multitask strategy specifically to multi-party intent recognition. Its entropy-based signal avoids additional costly annotations and a fixed number of speakers.
- Multitask dialogue methods have used auxiliary tasks including named entity recognition, dialogue act classification, and slot filling.
- Speaker-turn embeddings signal speaker switching or continuation, but reducing identities to a binary pattern loses speaker-specific turn-taking information.
- Qamar et al. address speaker-specific dynamics with a graph model that learns representations from speaker–utterance connections for multi-party dialogue act classification.
- This paper instead uses turn-transition entropy as a self-supervised regression signal within multitask learning for multi-party intent recognition.The strategy does not require additional costly annotations or a fixed number of speakers.
3 Experimental Procedure
The study fine-tunes pretrained encoders for multi-party intent classification using speaker-aware context and a turn-transition entropy auxiliary regression task. It evaluates this approach across public and internal corpora, multiple baselines, and classification and regression metrics.
- The task assigns each utterance its intent label while leveraging interactions among dialogue participants.
- The input combines token embeddings, segment embeddings distinguishing context from the target utterance, and speaker embeddings identifying the target speaker’s contextual turns.
- Turn-Transition Entropy Auxiliary Task: Turn-transition entropy is computed from observed speaker-transition pairs as Shannon entropy, yielding a continuous value where low values indicate organized transitions and high values indicate diverse shifts.
- Intent Classification: The multitask model predicts intent and entropy jointly, attaching an intermediate-layer regression head and combining classification and regression losses with a weighted geometric mean.
- Baselines: Comparisons include pretrained encoder baselines, zero-shot DeepSeek-R1-Distill-Llama, Label Space Reduction, speaker-aware graph modeling, and speaker-turn embeddings.
- Datasets and Evaluation: Experiments use MPGT, MIntRec2.0, and MEETInt, evaluating macro-F1 and accuracy over five runs and entropy prediction with MAE.
4 Results
Across the evaluated pretrained models and datasets, adding turn-transition entropy consistently improves macro-F1 over the authors’ baseline, with normalized entropy usually producing the largest gains. The approach also outperforms the compared baseline methods on MEETInt, while LLM performance is lower than the fine-tuned encoder systems.
- Across pretrained models, the entropy auxiliary task improves macro-F1 over the authors’ baseline on both MPGT and MIntRec2.0.
- The normalized entropy variant provides the largest baseline improvement in all cases except BERT on MPGT, while the unnormalized variant is second best.
- Macro-F1 gains are consistent across models, although improvements are smaller on MIntRec2.0 than on MPGT.
- LLM performance is much lower than that of the authors’ baseline and multitask methods, while Label Space Reduction has the poorest performance among the compared models and corpora.
- The proposed multitask approach outperforms all baseline methods on MEETInt, with improvements over He et al. (2021c) and Qamar et al. (2023) of about 4 and
5 Turn-Transition Entropy Learning as a Single Task
The paper tests whether turn-transition entropy is learnable independently of intent classification. Real entropy targets produce lower MAE than shuffled targets, with learning curves showing faster and sustained improvement.
- The authors replace the classification head with a regression head to predict turn-transition entropy as a standalone task.They compare real entropy values with randomly shuffled targets to test whether the model learns meaningful dialogue patterns.
- Lower MAE is consistently achieved with real entropy targets than with randomly shuffled targets across datasets and backbone models.Table 3 evaluates mean absolute error averaged over five runs.
- After approximately six epochs, models trained on true entropy converge and consistently outperform models trained on shuffled targets on MPGT.Figure 4 tracks MAE over 25 training epochs.
- These results support turn-transition entropy as a meaningful, learnable dialogue-context signal rather than an arbitrary continuous target.The authors argue that its multitask effectiveness reflects captured contextual information, not only regularization.
6 Multitask Learning on Random Entropy Targets
The paper replaces real auxiliary entropy targets with shuffled values to test whether the auxiliary task itself contributes useful information. Shuffled targets consistently reduce intent-classification performance.
- The experiment compares multitask learning with real entropy targets against a control condition using randomly shuffled entropy values.The comparison isolates the effect of the auxiliary target while retaining the multitask setup.
- Negative F1random−F1real differences occur across all settings, indicating that true entropy targets outperform shuffled targets.The table defines negative differences as classification-performance degradation under random regression targets.
- For both BERT and RoBERTa, shuffled-target multitask models achieve lower F1 scores than models trained with the real entropy signal.
7 Target Variables Relationship Analysis
The paper examines whether turn-transition entropy is associated with intent labels using a Kruskal–Wallis test and epsilon-squared effect sizes. Entropy distributions differ across intents in both datasets, with a stronger association on MPGT.
- The Kruskal–Wallis test compares the continuous, non-Gaussian entropy variable across multiple intent classes.The test’s null hypothesis is that the distributions are identical across groups.
- The analysis supplements statistical significance with epsilon-squared, an effect-size measure ranging from 0 to 1.Values closer to 1 indicate stronger associations between entropy and intent labels.
- p < 0.01 rejects identical entropy distributions across intents for both datasets.
- ε2 = 0.112 for MPGT versus ε2 = 0.021 for MIntRec2.0, indicating a stronger entropy–intent association on MPGT.
8 Conclusions
The paper adds turn-taking dynamics to multi-party intent detection through a self-supervised entropy-based auxiliary task. Across three backbones and three multi-party corpora, the method improves intent detection, and entropy is also learnable as a standalone signal.
- The method models entropy over speaker turn transitions as additional dialogue context for multi-party intent detection.
- The approach improves intent-detection systems over previous-work methods across three model backbones and one private plus two public corpora.
- Turn-transition entropy can be modeled as a single task, indicating that it provides dialogue information beyond acting as a multitask regularizer.
- The authors suggest that the findings may inform future classification tasks involving multi-party conversations.
Limitations
The approach does not constrain the number of speakers, but it assumes speaker identities are available for every utterance. Automatic diarization errors may reduce the quality of the turn-transition entropy signal.
- The method assumes speaker identities are available for each utterance, despite imposing no constraint on the number of speakers.
- Speaker identities are readily available when diarization is controlled or participants use dedicated audio channels, such as in video conferencing.
- Automatic diarization errors may affect the quality of the turn-transition entropy signal.
Ethical Considerations
The authors report that the corpora were curated to prevent malicious actions and identify no potential risk from publicly releasing the work and its findings. Training used private infrastructure and produced an estimated 13.82 kgCO2eq in emissions.
- The publicly available and internal corpora were curated to prevent malicious actions.
- The authors report no potential risk from releasing the work and its findings publicly.
- 13.82 kgCO2eq were estimated from approximately 80 hours of training across experiments, analyses, seeds, and hyperparameter tuning.
A Utterance Properties Correlation Analysis
The analysis examines whether turn-transition entropy is associated with utterance properties and whether normalization reduces relationships with simple properties such as turn index and speaker count. It reports that normalized entropy better represents turn-taking dynamics without capturing this spurious information, while Figure 6 shows entropy distributions by intent class.
- Correlation analysis: The study computes Spearman correlations between normalized or unnormalized entropy and utterance properties to investigate potential relationships.The Spearman coefficient measures the strength and direction of monotonic association.
- Correlation analysis: Moderate to high correlations appear between entropy and turn index or speaker count on both corpora.
- Correlation analysis: Normalization weakens the relationships between entropy and turn index or speaker count.
- Interpretation: Normalized entropy is argued to better represent turn-taking dynamics without capturing spurious information from simple utterance properties.
- Experimental setup: The experiments use large uncased BERT, RoBERTa, and DeBERTa models, with baseline and multitask implementations described alongside their training configurations.The multitask setup adds an auxiliary-loss weight λ and a regression-head layer, optimized with Optuna.
- Class-wise analysis: Figure 6 shows turn-transition entropy distributions per class for MPGT, illustrating expected differences among intents.