BiCFlow-MER: Orchestrating Discriminative and Generative Multimodal Emotion Recognition via Conditional Transport

Introduction

• Audio-text MER combines practical availability with evidence challenges: affect can be entangled with speaker style and lexical content, and tone can conflict with words.The paper notes applications including customer-experience analytics, human-computer interaction, intelligent assistants, and mental-health applications.Source: Introduction · p. 1
• BiCFlow-MER separates emotion-oriented evidence from speaker style and lexical content, using audio-text conflict to construct an affective condition.CT-BiRF transports a condition-dependent start toward an explicit emotion-space endpoint; A-GCH and BGD score it through forward prototype geometry and backward compatibility.Source: Introduction · pp. 1–2
• The proposed paradigm replaces terminal fusion-to-label mapping with conditional transport and forward geometric and backward compatibility scoring.Experiments on IEMOCAP, MELD, and zero-shot CASE, with controlled variants, evaluate standard recognition, conflict transfer, and component roles.Source: Introduction · p. 2

Discriminative audio-text MER

• Utterance-level discriminative MER aligns or attends across speech and language before mapping fused representations to emotion labels (Tsai et al. 2019; Ghosh et al. 2022; Wang et al. 2025).Variants improve representation learning but retain terminal classification, leaving evidence formation and class compatibility implicit.Source: Related Work · p. 2

Affective evidence factorization

• Factorization methods separate emotion from speaker style, lexical content, or other nuisance factors, but stop before post-fusion evidence organization and verification (Liu and Wang 2026; Kingma and Welling 2013; Cheng et al. 2020).BiCFlow-MER uses emotion-oriented factors to construct its transport condition.Source: Related Work · p. 2

Generative modeling and conditional transport

• Generative affective systems offer emotional descriptions or rationale-based analysis (Wu et al. 2025; Li et al. 2026; Rha et al. 2026), while rectified-flow and related methods model transport or distributional supervision (Liu, Gong, and Liu 2022; Peebles and Xie 2023; Wang et al. 2026; Wu et al. 2026).The cited approaches do not construct and verify an audio-text emotion endpoint; BiCFlow-MER generates a condition-dependent endpoint.Source: Related Work · p. 2

Prototype geometry and verification

• Prototype scoring measures class proximity but not compatibility with the condition-dependent start region (Schroff, Kalenichenko, and Philbin 2015); BiCFlow-MER combines adaptive prototype clouds with backward compatibility scoring.The backward score evaluates whether a candidate class mean can invert toward that start region.Source: Related Work · p. 2

Overview and Problem Formulation

• The method maps paired audio and transcript inputs to an affective condition, transports a start through a 64-dimensional emotion flow space, and evaluates the endpoint in both directions.Forward scoring tests endpoint membership in a class cloud; backward scoring tests whether a class hypothesis reconstructs the condition-dependent start region.Source: Overview and Problem Formulation · p. 2

Affective Evidence Factorization

• CGD-IB separates emotion, style, and content factors, reconstructs modality representations, and constrains emotion factors to preserve label-relevant semantics.Emotion factors feed downstream condition construction, while style and content factors remain within CGD-IB for reconstruction and regularization.Source: Affective Evidence Factorization · pp. 2–3

Conflict-Aware Condition Construction

• Multi-scale fusion combines global, local, and short-term audio-text evidence, while ECA uses emotion-factor conflict to set modality weights and construct the affective condition.The multi-scale bottleneck uses learned-query attention, Gumbel Top-K selection, and one-dimensional convolutions; ECA scores factor pairs using learned energy and cosine disagreement.Source: Conflict-Aware Condition Construction · p. 3

CT-BiRF

• CT-BiRF transports a stochastic start sampled around condition-dependent center m(c) toward an emotion-space target, using forward and backward fields along the same conditional path.Training adds off-path neighborhood perturbations while retaining the nominal rectified displacement as supervised velocity; low-confidence inference samples receive three-step integration before rescoring.Source: CT-BiRF · p. 4

Geometric Evidence and Backward Compatibility

• A-GCH scores the transported endpoint against adaptive class prototype clouds, while BGD checks whether each class mean maps back toward the condition-dependent start region.The final score combines both channels; BGD reuses the learned backward field and adds no separate trainable parameters or class-wise objective.Source: Geometric Evidence and Backward… · p. 5

Training Objective and Curriculum

• Training proceeds from variational emotion-factor learning to joint fusion and flow objectives, then adds conflict-pair and backward-landmark supervision for terminal inversion refinement.BGD adds no separate training loss.Source: Training Objective and Curriculum · p. 5

Experimental Setup

• Evaluation uses five-fold speaker-independent IEMOCAP with audio-text utterances and official MELD splits; each utterance is classified without visual features, speaker metadata, or conversational context.IEMOCAP reports UA, WA, and W-F1; MELD reports WA and W-F1, with W-F1 as its main metric.Source: Experimental Setup · p. 5
• BiCFlow-MER ranks first on all four IEMOCAP and MELD metrics, exceeding second-best values by 0.40/0.11 points on IEMOCAP WA/UA and 0.55/1.43 points on MELD WA/W-F1.Baseline values come from the corresponding literature.Source: Experimental Setup · p. 6

Cross-Dataset Evaluation on CASE

• On zero-shot CASE, BiCFlow-MER reaches 61.90% ACC and 59.82% W-F1, exceeding FAS by 2.52 and 4.74 points, respectively.CASE uses 378 utterances with no samples used for training, validation, calibration, or hyperparameter selection.Source: Cross-Dataset Evaluation on CASE · p. 6, Table 4 · p. 6

Analysis and Ablation

• The staged curriculum reaches high validation UA earlier and remains more stable after Stage II.b than the abrupt-activation baseline.Stage II.a introduces ECA, conditional transport, and geometric scoring; Stage II.b adds N3 conflict pairs and backward landmark supervision.Source: Analysis and Ablation · p. 6, Figure 3 · p. 7
• Removing ECA causes the largest losses on CASE—7.90 ACC and 6.26 W-F1 points—and on IEMOCAP UA—2.67 points; other component removals also reduce CASE performance.Removing factorization, CT-BiRF, and A-GCH reduces CASE ACC/W-F1 by 5.03/5.52, 4.89/3.38, and 4.78/3.85 points; removing BGD reduces them by 2.25/0.96 points.Source: Analysis and Ablation · pp. 6–7
• Audio-text input reaches 78.18±1.44% WA and 78.90±2.80% UA, improving on speech alone by 4.35 and 3.60 points, respectively.The text-only setting reaches 65.75±1.70% WA and 67.56±2.94% UA, while speech alone reaches 73.83±2.05% WA and 75.30±3.31% UA.Source: Table 6 · p. 6, Analysis and Ablation · p. 7

• Emotion-oriented factorization separates affective evidence from speaker-style and lexical-content factors, while ECA incorporates audio-text disagreement into condition construction.Source: Conclusion · p. 7
• Conditional transport evaluates candidate emotions through forward endpoint geometry and backward compatibility with the condition-dependent start region.Source: Conclusion · p. 7
• IEMOCAP and MELD results establish standard recognition performance, while zero-shot CASE results and controlled variants assess transfer under acoustic-semantic conflict.Source: Conclusion · p. 7
• The formulation presents multimodal emotion evidence as explicitly constructed, transported, and verified within a generative-decision paradigm.Source: Conclusion · p. 7