PaperlayerView exact revision on arXiv

Source-linked AI summary

HiBRIDGE: A Hierarchical Bayesian Neural Network Framework for Interpretable Dialogue Management in Group-Robot Interaction

Massimiliano Nigro, Hatice Gunes, Micol Spitale, Fethiye Irmak Dogan

arXiv:2609.08678v1cs.RO

TL;DR

Group-robot dialogue management must choose whom to address and what to say despite multiple plausible behaviours, while existing approaches offer limited uncertainty representation and interpretable intermediate structure. HiBRIDGE combines Bayesian neural networks with hierarchical behaviour selection. Across offline datasets, Bayesian models outperform deterministic counterparts and several baselines; hierarchical explanations are preferred and rated more helpful, and both Bayesian variants are positively perceived in an in-person study.

  • Problem

    Group-robot dialogue management must choose whom to address and what to say amid plausible behaviours, while current approaches have limited uncertainty representation and semantically meaningful intermediate decision stages.

    Source: §1 · pp. 1–2

  • Method

    HiBRIDGE combines Bayesian neural networks with hierarchical behaviour prediction, comparing flat and two hierarchical architectures and using decision-tree surrogates to support explanations.

    Source: §3.1 · p. 4, §4.3 · p. 7

  • Results

    Across three offline datasets, Bayesian formulations improve predictive performance over deterministic counterparts and outperform several baselines; hierarchical explanations are preferred and rated more helpful, while both Bayesian variants are positively perceived in-person.

    Source: §1 · p. 2, §8 · p. 18, §9 · p. 19

  • Takeaways & Limitations

    The findings support a modular design in which a learned model selects conversational behaviour and an LLM generates utterances, retaining explicit behavioural control alongside generative flexibility.

    Source: §8 · p. 18

    Neither explanation type improved participants’ accuracy in predicting robot behaviour, and short exposure may have limited their opportunity to learn the models’ behavioural patterns.

    Source: §8 · p. 18

Abstract

from arXiv · show

In multi-party human-robot interaction, a robot must continuously decide whom to address and what to say to participate effectively in the conversation. In real-world interactions, this is challenging because several behaviours may be plausible at the same time: a robot might continue a topic with one participant, involve another through a question, or address the whole group, with the appropriate choice depending on both whom it addresses and the interaction context. Current approaches remain limited in representing uncertainty when several behaviours are plausible and in structuring decisions into semantically meaningful intermediate steps that make robot decisions easier to interpret. Addressing these, we present HiBRIDGE, a hierarchical Bayesian neural network framework for group-robot dialogue management. Its Bayesian formulation enables uncertainty-aware prediction and robust learning from limited interaction data, while the hierarchical approach formulates behaviour selection as a structured, multi-stage decision process. We further use decision-tree surrogates to investigate whether this structure can support more interpretable explanations. Across three offline group-HRI datasets, our findings show that Bayesian formulations outperform their deterministic counterparts and several state-of-the-art baselines. Next, through an online study (N=20), we show that explanations derived from the hierarchical model are rated as more helpful for understanding robot behaviour and are preferred over those derived from the flat model. Finally, through our in-person study (N=12), we demonstrate the feasibility of HiBRIDGE for autonomous real-time group interaction, with both hierarchical and flat Bayesian variants positively perceived. Overall, HiBRIDGE combines strong predictive performance with a structured decision process that supports more interpretable explanations of robot behaviour.

1. Introduction

Group-robot dialogue management requires choosing whom to address and what to say amid multiple plausible, interdependent, context-sensitive behaviours. HiBRIDGE combines Bayesian prediction with hierarchical decisions and evaluates predictive performance, explanation preferences, and real-time interaction.

Motivation

  • Group dialogue management must select both an addressee and conversational content, although several plausible behaviours may fit the same moment.Appropriate choices depend on the interaction context and on the interdependence between whom the robot addresses and what it says.Source: §1 · p. 1

Research gap

  • Prior approaches leave uncertainty and interpretability challenges: deterministic models can be overconfident amid plausible behaviours, while supervised models lack semantically meaningful intermediate decision stages.Small HRI datasets also make generalisation difficult.Source: §1 · p. 2

Approach

  • HiBRIDGE uses Bayesian neural networks and hierarchical classification to structure behaviour selection, then fits decision-tree surrogates to generate explanations from model predictions.The hierarchy examines both addressee-first and dialogue-act-first orderings.Source: §1 · p. 2

Results

  • Across three group-HRI datasets, HiBRIDGE models outperform state-of-the-art baselines, and Bayesian formulations consistently improve predictive performance over deterministic counterparts.Online, hierarchical explanations are preferred and rated more helpful; in-person, hierarchical and flat Bayesian variants are positively perceived in real-time interaction.Source: §1 · p. 2

2. Background and Related Works

Prior group-robot dialogue systems use LLMs or supervised models, while dialogue-act taxonomies and hierarchical classification offer ways to structure behaviour. HiBRIDGE combines Bayesian neural networks with hierarchical classification to expose intermediate decisions.

2.1. Modeling Group-Robot Interaction

  • LLM-based selection offers flexible behaviour but makes control and decision rationales harder to interpret; supervised prediction offers more control but faces small-data and interpretation challenges.Prior sequential prediction models add dependencies between addressee and dialogue-act decisions without explicitly structuring intermediate semantic levels.Source: §2.1 · p. 3
  • HiBRIDGE retains supervised control of dialogue-act and addressee choices while using an LLM only to generate contextually appropriate utterances.Its Bayesian formulation supports learning from limited HRI data, and its hierarchy exposes semantically meaningful decision stages.Source: §2.1 · p. 3

2.2. Dialogue Acts Taxonomies

  • The paper adapts MIDAS for social-robot interaction, focusing on its semantic layer and combining relevant semantic elements with DAMSL-inspired features.The resulting dialogue-act categories formulate what the robot says as high-level categories suited to the task.Source: §2.2 · p. 3

2.3. Bayesian Neural Network and Hierarchical Classification Approaches

  • Bayesian neural networks place prior distributions over network weights to estimate uncertainty, a property described as useful for learning from small HRI datasets.The cited HRI applications include spatial-relation learning and safe hand-avoidance behaviour, with reported generalisation to unseen scenarios.Source: §2.3 · pp. 3–4
  • Hierarchical classification predicts labels at multiple abstraction levels; the paper applies it with Bayesian neural networks to expose robot decisions in a limited, severely imbalanced-data setting.The authors state that, to their knowledge, this is the first application of hierarchical classification in HRI.Source: §2.3 · p. 4

3. Problem Formulation

The task combines decisions about whether to speak, whom to address, and which dialogue act to perform. HiBRIDGE represents behaviour selection as paths through alternative hierarchies and uses a streamlined taxonomy of initiative and responsive acts.

3.1. Task Definition

  • The task defines two alternative hierarchies: addressee-first and dialogue-act-first, each representing a complete behaviour as a unique root-to-leaf path.The two orderings reverse whether addressee or dialogue-act selection comes first.Source: §3.1 · p. 4
  • The addressee branch distinguishes addressing the whole group from addressing an individual.The streamlined hierarchy uses the addressee set A = {All, Individual}.Source: §3.1 · p. 4
  • The dialogue-act hierarchy first distinguishes initiative from responsive behaviour, then predicts the corresponding dialogue-act category.The task uses a streamlined hierarchy to mitigate class imbalance and reduce the number of unique classes in limited HRI datasets.Source: §3.1 · pp. 4–5
  • When the robot speaks, it predicts a complete conversational behaviour through the task-specific hierarchy; the full decision includes the speak-or-silence step.The complete task is represented as conversational state leading to speak status, a hierarchy path, and then dialogue-act and addressee outcomes.Source: §3.1 · pp. 4–5

3.2. Dialog Act Taxonomy

  • The taxonomy adapts MIDAS for social, multi-party HRI, preserving its initiative-responsive distinction while drawing targeted changes from DAMSL and the three study datasets.It focuses on semantic categories for the robot’s what-to-say decision.Source: §3.2 · p. 5
  • Initiative acts begin conversation threads and include Questions and Statements; the taxonomy broadens Statements and expands Questions to include offers and small talk.Commands were removed as unrepresentative of the interaction settings examined.Source: §3.2 · p. 5
  • Responsive acts continue existing threads through Acknowledgements, Follow-ups, and Answers; the taxonomy groups opinions and comments as Acknowledgements and adds Follow-ups.Answers are categorised as positive, negative, or other.Source: §3.2 · p. 5
  • The authors report that these taxonomy changes preserve MIDAS’s strengths while better capturing semantic and interactional patterns in their data.Source: §3.2 · p. 5

4. HiBRIDGE Framework

HiBRIDGE maps conversational state to robot behaviour, first deciding whether to speak and then predicting a dialogue act and addressee. It offers flat and hierarchical deterministic or Bayesian variants, with decision-tree surrogates used to generate explanations.

4. HiBRIDGE Framework

  • HiBRIDGE maps conversational state to robot behaviour, gating behaviour selection on whether the robot should speak; its formulation is agnostic to hierarchy semantics and depth.Source: §4 · pp. 5–6, Figure 3 · p. 6

4.1. Feature Representation

  • Features combine a six-turn window of conversational events with cumulative directed speaking-time statistics for each group member.Inputs assume a maximum group size of four; categorical features are one-hot encoded and numerical features are standardized.Source: §4.1 · p. 6

4.2. Speak-or-Silence Prediction

  • A separate binary classifier decides whether the robot should speak; only a positive decision leads to dialogue-act and addressee selection.Source: §4.2 · p. 6

4.3. Conversational Behaviour Prediction

  • After the Speak decision, HiBRIDGE predicts a joint behaviour comprising a dialogue act and addressee, using one of three architectures in deterministic or Bayesian form.Source: §4.3 · p. 7

4.3.1. Flat and Hierarchical Architectures

  • Flat models predict valid dialogue-act–addressee combinations jointly; hierarchical models expose intermediate decisions, ordered addressee-first or dialogue-act-first.The dialogue-act hierarchy distinguishes initiative from responsive behaviour before predicting an act category, while addressee prediction distinguishes an individual from the group.Source: §4.3.1 · pp. 7–8

4.3.2. Deterministic and Bayesian Formulations

  • Bayesian variants place zero-centred Gaussian priors on network weights and use SVI to learn an approximate posterior by maximizing the ELBO.For hierarchical Bayesian models, the hierarchy-based penalty is also added to the optimization objective.Source: §4.3.2 · p. 8

4.4. Explanation Generation for Behaviour Selection

  • Decision-tree surrogates approximate model decisions: flat models use one tree, while hierarchical models fit per-head trees and combine traversed paths into natural-language explanations.For the interpretability experiments, GPT-5.4 converts the extracted paths into explanations.Source: §4.4 · pp. 8–9

5. Offline Dataset Experiments and Results

Offline tests compare HiBRIDGE variants with existing methods across three varied group-HRI datasets. Proposed models generally outperform baselines, while Bayesian gains vary by dataset and metric.

5.1. Offline Datasets

  • The three datasets cover mall entertainment, hospital information, and negotiation; analyses use textual data across datasets.MuMMER has 33 conversations, Addlesee 30, and Spitale 28; visual features were excluded because camera motion made them unstable, and preliminary audio features did not improve on text-based conversational features.Source: §5.1 · p. 10

5.2. Baselines for Offline Datasets

  • Baselines comprised two LLMs and Gillet et al.’s GNN, adapted to the paper’s dialogue-act taxonomy and group-addressee labels.The GNN predicts addressees and dialogue acts separately; whole-group annotations were duplicated per participant during training, with an all-participants rule at inference.Source: §5.2 · pp. 10–11

5.3. Implementation Details

  • Training used Gaussian-noise augmentation and stratified ten-fold splits, with hyperparameters selected by validation accuracy on MuMMER.The split yields 72% training, 18% validation, and 10% test data; tuning used 75 Optuna trials per model.Source: §5.3 · p. 11

5.4. Evaluation Procedure on Offline Datasets

  • Six flat or hierarchical, Bayesian or deterministic variants were evaluated using Hit@1, Hit@3, and Hit@5, with corrected pairwise tests and factorial analysis.The hierarchical variants order addressee and dialogue-act decisions differently; the factorial analysis separates modelling type from architecture.Source: §5.4 · pp. 11–12

5.5.1. Proposed Models versus Baselines

  • Best-model Hit@1 reached 0.6396 on MuMMER, 0.678 on Spitale, and 0.8019 on Addlesee.The paper describes Addlesee as comparatively easier, with a passive, question-responsive robot role; MuMMER and Spitale involve more varied or proactive behavior.Source: §5.5.1 · p. 12

5.5.2. E

  • Factorial tests found no significant hierarchical-modelling effect on any metric, while Bayesian modelling had significant effects on every MuMMER and Spitale metric (all p < .05), including Spitale Hit@5.At Spitale Hit@5, post-hoc comparisons were not significant for flat (U = 61.5, p = .202) and hierarchical addressee-first (U = 72.0, p = .052) architectures. Addlesee Hit@5 also lacked an overall Bayesian-modelling effect (F = 2.50, p = .119).Source: §5.5.2 · pp. 12–13

5.5.1. Proposed Models versus Baselines

  • Proposed models generally outperformed all three baselines across datasets and metrics; the exception was DACT→ADDR Hit@1 on Spitale versus Gillet, where the mean was still higher.The exception was not statistically significant; the other comparisons were significant at p < 0.05.Source: §5.5.1 · p. 12, Table 1 · p. 13

6. Online Interpretability Study and Results

In an online study with 20 participants, hierarchical and flat explanations were compared through ratings, choices, and behavior-prediction tasks. Participants preferred hierarchical explanations in direct comparisons, but ratings and prediction accuracy did not significantly differ.

6.1.1. Hypotheses

  • The hypotheses predicted higher understandability, coherence, and usefulness for hierarchical explanations, along with preference and improved behavior inference.They also predicted that explanations would improve inference compared with no explanation.Source: §6.1.1 · p. 13

6.2. Online Study Participants and Procedure

  • Twenty Prolific participants completed the online questionnaire; eligibility required English as a first language, age over 18, and a 99% minimum approval rating.The sample was balanced at 10 men and 10 women, and the questionnaire took about 30 minutes.Source: §6.2 · p. 14

6.3. Online Study Tasks

  • Across 14 clips, Task 1 rated explanations, Task 2 tested masked behavior inference, and Task 3 elicited predictions before side-by-side explanation choice.Task 1 also used the Explanation Satisfaction Scale.Source: §6.3 · p. 14

6.4. Online Study Metrics

  • Analyses compared ESS and questionnaire ratings, explanation preferences, and prediction accuracy, using paired t-tests or Wilcoxon tests according to distributional assumptions.Rating-item comparisons also used Benjamini–Hochberg correction.Source: §6.4 · pp. 14–15

6.5. Online Study Results

  • Hierarchical explanations averaged 3.59 versus 3.34 on ESS and 4.28 versus 4.18 on custom ratings, with neither difference significant (p > .05).The authors note that a larger sample might be needed to detect a relatively small effect.Source: §6.5 · p. 15
  • Across 80 choices, hierarchical explanations were selected 45 times versus 23 for flat explanations; paired preference scores were 2.60 versus 1.50 (W = 21.00, p = .014).“Both” judgments counted toward each explanation’s per-participant score.Source: Figure 5 · p. 15, §6.5 · p. 15
  • Prediction accuracy did not significantly differ among no-explanation, hierarchical-explanation, and flat-explanation conditions.The authors suggest that several plausible behaviors may have made the prediction task challenging.Source: §6.5 · p. 15

7. In-person Study and Results

The in-person study integrated HiBRIDGE into a real-time robot system and assessed both hierarchical and flat Bayesian models in group interactions. Participants rated both models favourably, while the hierarchical model’s ratings were consistently higher but not significantly different.

7.1. Robotics Deployment

  • The system used HARMONI to connect speech and speaker processing with HiBRIDGE behaviour selection, then used an LLM to generate utterances that were synthesised into speech.A Speak-or-Silence classifier gated whether the robot spoke; the conversational behaviour model then selected the addressee and dialogue act.Source: Figure 6 · p. 16, §7.1 · p. 16

7.2. In-Person Study Participants and Procedure

  • Twelve participants in six pairs interacted with both models in randomized, counterbalanced order during shopping-mall-inspired sessions.Each interaction lasted about three minutes; participants also completed a six-item questionnaire and semi-structured interviews.Source: §7.2 · p. 17

7.3. In-person Study Results

  • Most questionnaire items scored above 3 out of 5, and the hierarchical model consistently outperformed the flat model by moderate margins, with no significant differences across the six dimensions.Both models were perceived as satisfactory, and participants considered the robot able to interact effectively regardless of which model drove its behaviour.Source: Figure 7 · p. 17, §7.3 · pp. 17–18
  • Appropriateness and usefulness ratings favoured the hierarchical model, while conversation balancing and inclusivity received the lowest scores overall.For appropriateness, means were 4.00 (SD = 0.60) for hierarchical and 3.42 (SD = 1.16) for flat; for usefulness, they were 4.00 (SD = 0.60) and 3.33 (SD = 1.07), respectively.Source: §7.3 · p. 17

8. Summary and Discussion

The discussion presents Bayesian modelling as most useful for prediction in sparse, ambiguous HRI settings, while hierarchy supports explanation through intermediate decisions. The studies also leave open whether explanation benefits extend to predictive understanding or longer interactions.

8. Summary and Discussion

  • Bayesian advantages were strongest on Hit@1 and Hit@3 and in MuMMER and Spitale, but weaker on Hit@5 and in the more constrained Addlesee setting.The experiments did not directly evaluate uncertainty calibration.Source: §8 · p. 18
  • Participants preferred hierarchical explanations, which expose intermediate decisions such as whether to act responsively or initiatively and whom to address.The discussion suggests that interpretability can be considered in a model’s internal structure, rather than only as a post-hoc process.Source: §8 · p. 18
  • Explanation benefits were not uniform: explanation ratings and robot-behaviour prediction accuracy did not significantly improve, and the in-person models also showed no significant differences.The authors identify short exposure as a possible constraint and call for studies of longer, repeated interactions.Source: §8 · p. 18

9. Conclusions

HiBRIDGE combines Bayesian prediction with hierarchical behaviour selection for multi-party human–robot interaction. Across the reported studies, Bayesian models improved prediction, hierarchical explanations were preferred, and both Bayesian variants were positively perceived in real-time group interaction.

9. Conclusions

  • Across three offline datasets, Bayesian formulations improved predictive performance over deterministic counterparts and outperformed several baselines.The conclusion characterizes probabilistic modelling as strengthening behaviour prediction under sparse interaction data.Source: §9 · p. 19
  • Hierarchical modelling exposed intermediate decision stages that supported explanations participants preferred and found more helpful for understanding robot behaviour.The conclusion notes that hierarchy did not yield a significant predictive advantage.Source: §9 · p. 19
  • Both hierarchical and flat Bayesian variants were positively perceived when integrated into an autonomous real-time group interaction system.The paper presents the two formulations as complementary: Bayesian modelling strengthens prediction, while hierarchy supports transparency in behaviour selection.Source: §9 · p. 19

Declaration of Generative AI and AI-assisted Technologies in the Manuscript Preparation Process

The authors used Claude and ChatGPT for rephrasing and grammar correction during manuscript preparation, and state that no new content was generated by these tools.

Declaration of Generative AI and AI-assisted Technologies in the Manuscript Preparation Process

  • Claude and ChatGPT assisted with rephrasing and grammar correction; the authors state that no new content was generated and that they reviewed and edited the manuscript.The authors take full responsibility for the published content.Source: Declaration of Generative AI and… · p. 19

Appendix A. Feature Representation

The appendices document the conversational features and the prompt-to-explanation pipeline, including hierarchical and flat decision-path examples.

Appendix A. Feature Representation

  • The conversational representation combines per-turn and interaction-level features into a 158-dimensional vector, with categorical values listed for one-hot encoding.Appendix A points to separate tables for the feature inventory and category values.Source: Appendix A · p. 23

Appendix B. Prompts for generating explanations

  • The explanation prompts address the robot’s addressee and utterance-type decisions during a three-party interaction.The robot must decide whom to address and what kind of utterance to produce.Source: Appendix B · p. 23, Appendix B.1 · p. 23

Appendix B.1. System Prompt

  • The system prompt uses a fixed template and target word count, grounding plain-language explanations in decision paths and limiting them to two relevant features.The default target is 150 words, divided equally between two required questions.Source: Appendix B.1 · p. 23

Appendix B.2. User Prompt

  • The user prompt supplies the decision-tree path and recent-speaker sequence; hierarchical paths may be multiple, while flat models provide one path.Source: Appendix B.2 · p. 24

Appendix C. Explanation Generation Examples

  • The explanation-generation examples show decision-tree path inputs alongside the resulting natural-language explanations in a two-part format.Source: Appendix C · p. 24

Appendix C.1. Example 1: Hierarchical Decision Path

Generated Explanation (Output Example)

Appendix C.2. Example 2: Flat Decision Path

  • The flat example provides a single decision path predicting FOLLOW UP/ALL and a recent-speaker list as inputs to explanation generation.Source: Appendix C.2 · p. 25

Appendix D. Hyperparameters optimised during experiments

Appendix D reports optimized model configurations, while Appendix E describes the online interpretability-study questionnaire.

Appendix D. Hyperparameters optimised during experiments

  • Hyperparameters were selected over 75 Optuna trials using average validation accuracy from 10-fold cross-validation on MuMMER.Source: Appendix D · pp. 25–26
  • The reported configurations cover deterministic models, Bayesian flat and hierarchical models, and the Gillet et al. baseline.Tables D.4–D.6 report the respective configurations, including separately tuned addressee and utterance-type baseline sub-models.Source: Appendix D · pp. 26–27, Table D.4 · p. 26

Appendix E. Custom-Designed Online Study Questionnaire

  • The online study used a five-point Likert scale to rate explanation clarity, understanding, perceived behaviour, and coherence with the robot’s decisions.Source: Appendix E · p. 27

Appendix F. Online Study Task Interface

The appendices describe the online study interface and the post-part questionnaires used in the in-person study.

Appendix F. Online Study Task Interface

  • The online interpretability study was hosted on Qualtrics, with example interface screenshots for each of its three tasks.Source: Appendix F · p. 27

Appendix G. Custom-Designed In-Person Study Questionnaire

  • After each in-person study part, participants rated usefulness, behavioural appropriateness, conversation management, engagement, inclusion, and participation balance on five-point scales.Source: Appendix G · pp. 27–28
Loading 2609.08678v1…