Source-linked AI summary
Learning to Prefer Reliably: Error-Augmented Emotion Preference Optimization with Calibrated Fusion
Zilong Huang, Junyi Peng, Junjie Li, Kai Li, Wenze Ren, Kong Aik Lee, Man-Wai Mak, Tatsuya Kawahara
TL;DR
Emotion preference learning is limited by sparse negative supervision and the model-specific biases of single MLLM judges. EAPO augments preference data with controlled emotional errors, adapts multiple independent judges, and fuses calibrated margins. Experiments report improved preference prediction and robustness, including 80.31% WAF on Original Val for the selected fusion.
Problem
Emotion preference learning lacks broad coverage of emotionally incorrect descriptions, while single MLLM judges can introduce model-specific judgment biases.
Method
EAPO retains natural negatives, adds four controlled error types, independently adapts multiple MLLM judges, and aggregates normalized signed margins through calibrated soft fusion.
Results
80.31% WAF on Original Val was achieved by the selected three-judge fusion, alongside 85.35% in 4-Error Avg and 76.38% in Swap Cons.
Takeaways & Limitations
EAPO improves agreement with human preferences while maintaining robustness to controlled semantic errors and candidate-order consistency.
Takeaways & Limitations
Generated error-aware pairs and intermediate descriptions may vary in quality, and fusion calibration stability under model replacement or domain shift remains unclear.
Abstract
from arXiv · showhide
Emotion preference learning uses pairwise comparisons between candidate descriptions to align multimodal large language models (MLLMs) with human judgments of open-ended emotion descriptions and to train reward models that capture human emotional preferences. However, conventional pairwise supervision is often sparse, typically providing only a single negative description for each positive description, and therefore offers limited coverage of the diverse ways in which an emotion description can be incorrect. In particular, models may be insufficiently exposed to semantically fluent but emotionally inconsistent descriptions. Beyond this data-level limitation, relying on a single MLLM judge introduces a distinct model-level concern: its judgments can be affected by model-specific biases when interpreting fine-grained or ambiguous multimodal emotional cues. To address these limitations, we propose Error-Augmented Preference Optimization (EAPO), a framework for improving the reliability of MLLM-based emotion preference judgment at both the data and model levels. First, we construct an error-augmented dataset by generating multiple controlled and emotion-aware negative descriptions from each preferred description. We then adapt multiple independent MLLM judges to this richer supervision and aggregate their preference margins using margin-calibrated soft fusion, which maps heterogeneous margins to a common scale before aggregation. Experiments on the MER2026-EmoPrefer Challenge dataset and our error-augmented dataset demonstrate that EAPO improves emotion preference prediction and enhances the robustness of MLLM judges when evaluating fluent descriptions that conflict with the video's multimodal emotional evidence. Our code is available at https://github.com/slash1028/EAPO-EmoPrefer.
1 Introduction
Emotion preference learning addresses the difficulty of evaluating open-ended multimodal emotion descriptions, but conventional supervision and single-judge evaluation provide limited coverage and diversity. EAPO responds with controlled error augmentation, independently adapted judges, and calibrated multi-judge fusion.
- Motivation: Open-ended emotion descriptions are difficult to evaluate because a single reference cannot cover every defensible interpretation or expose all emotional error modes.Preference learning instead compares candidate descriptions according to how well they reflect the observed emotional state.
- Limitations: Conventional supervision typically pairs each preferred description with one naturally occurring rejected description, leaving diverse error modes under-specified.The identified modes include Emotion Flip, Intensity Mismatch, Evidence Contradiction, and Modality Omission.
- Limitations: Single-MLLM judging makes preference decisions dependent on model-specific biases when interpreting fine-grained or ambiguous multimodal emotional cues.The framework is motivated by improving both error-relevant data coverage and the diversity of judgment sources.
- EAPO: EAPO retains naturally occurring negatives and adds four controlled error categories to provide richer supervision for emotion preference judgment.The judges are adapted through supervised fine-tuning followed by direct preference optimization, with candidate-order swapping used during training.
- EAPO: EAPO independently adapts multiple MLLM judges and combines their signed preference margins after mapping them to a common scale.Margin-calibrated soft fusion preserves graded preference strength instead of using hard voting or directly averaging raw margins.
- Results: Experiments on MER2026-EmoPrefer and controlled error-augmented subsets demonstrate EAPO’s effectiveness, with the submission ranking sixth on the MER2026 leaderboard.The paper reports that EAPO outperformed the official baseline models on the reported evaluation metrics.
2 Related Work
Multimodal emotion recognition has expanded from fixed labels toward open-vocabulary descriptions, creating a need for relative preference evaluation. Existing work uses human preference pairs and MLLM-based judges, while EAPO combines error-augmented supervision with independent judgment and calibrated fusion.
- Emotion Recognition: Traditional multimodal emotion recognition commonly predicts categorical or dimensional emotions using facial, vocal, linguistic, and conversational signals.These approaches rely on representation learning and cross-modal fusion for closed-set recognition.
- Open-Vocabulary Emotion Understanding: Open-vocabulary descriptions express subtler emotions and richer multimodal evidence but are difficult to evaluate because valid interpretations can use different wording.Comparing predictions with only one reference may penalize accurate descriptions expressed differently.
- Emotion Preference Learning: Emotion preference learning compares candidate descriptions by their support from multimodal content rather than dependence on a unique textual reference.Human-annotated preference pairs support training emotion-aware judges and evaluating agreement with human preference.
- EAPO Framework: EAPO independently trains MLLM judges and combines their preference margins through normalized calibrated soft fusion to obtain the final preference.The framework overview also includes error-augmented pairs and candidate-order swapping during SFT and DPO.
- MLLM Preference Judges: Existing approaches use MLLMs as preference judges through direct comparison, intermediate descriptions, external-LLM reasoning, or model-based crowdsourcing.Supervised fine-tuning adapts models to pairwise decisions, while DPO increases the relative likelihood of preferred responses over rejected ones.
3 Method
EAPO constructs controlled emotion-aware negatives, adapts multiple MLLM judges, and fuses their calibrated preference margins to improve emotion preference judgment reliability.
- Each comparison uses multimodal evidence, two candidate descriptions, and a binary human-preference label that the judge must predict.
- Error-Augmented Negative Construction: Controlled negatives augment each preferred description with four targeted error types while retaining the naturally occurring rejected description.The four synthetic categories are Emotion Flip, Intensity Mismatch, Evidence Contradiction, and Modality Omission.
- Error-Augmented Negative Construction: Emotion Flip changes the high-level affective interpretation while preserving references to the original multimodal evidence.An edit planner proposes a structured local edit; deterministic checks and semantic verification assess locality, non-target preservation, fluency, and the intended error type.
- Judge Adaptation: Judges are independently adapted with LoRA-based supervised fine-tuning followed by direct preference optimization on original and generated preference pairs.Candidate-order swapping assigns the corresponding label after reversal to reduce position-dependent predictions.
- Margin-Calibrated Multi-Judge Fusion: Margin-calibrated fusion combines three independent judges after normalizing their raw preference margins to comparable scales.The signed margin uses answer-label negative log-likelihoods; its sign selects a candidate, while normalization preserves preference direction and equal-weight fusion uses the fused sign for prediction.
4 Experiments
The experiments evaluate data augmentation, judge adaptation, calibrated fusion, and official test performance across original and controlled-error preference settings. EAPO and calibrated fusion generally improve robustness and aggregate preference prediction, though gains vary by backbone and setting.
- Experimental Setup: Error-Aug Train-Set contains 2,908 generated pairs spanning four controlled error types, while the official Stage 1 and Stage 2 test sets contain 379 and 515 samples.The generated pairs exclude Original Rejected pairs and support controlled augmentation and diagnosis.
- Experimental Setup: WAF evaluates preference prediction, while 4-Error Avg and Swap Cons measure controlled-error performance and candidate-order consistency.4-Error Avg macroaverages WAF across four generated-error subsets.
- Overall Results: Error-Aug SFT+DPO achieves the strongest performance across all three validation metrics for Qwen3-Omni, while adaptation gains vary across backbones.MiniCPM improves across all three metrics with error-augmented training, whereas Qwen2.5 reaches its best Original Val.
- Fusion Results: Calibrated fusion reaches 80.31% Original Val, 85.35% 4-Error Avg, and 76.38% Swap Cons by combining complementary judge margins.It averages scale-normalized continuous margins rather than discarding margin magnitude through hard voting.
- Overall Results: Training on original preference pairs can improve Original Val without improving controlled-error robustness, whereas error-augmented training strengthens discrimination of semantic errors.For Qwen3-Omni, Normal SFT improves Original Val but reduces 4-Error Avg; Error-Aug SFT improves all three metrics over Normal SFT.
- Official Test Results: All single judges outperform the strongest official baseline on Stage 1, while Stage 2 remains more challenging.Qwen3-Omni Error-Aug SFT+DPO obtains the highest single-judge results on both stages and a 79.40% Macro WAF.
- Official Test Results: Calibrated Fusion reaches 80.23% Macro WAF, compared with 79.40% for the best single judge and 79.31% for Raw Fusion.The official Macro WAF is the arithmetic mean of Stage 1 and Stage 2 WAF.
- Limitations and Future Work: The construction of error-aware pairs and S2 intermediate descriptions relies primarily on MLLM generation, whose quality may vary across samples.Fusion calibration has been evaluated only with the current judge families and data distribution, leaving stability under model replacement or domain shift unclear.
5 Conclusions
EAPO combines error-augmented preference training with calibrated multi-judge fusion for robust multimodal emotion preference learning. Experiments show improved human-preference agreement, error-type robustness, candidate-order consistency, and overall performance.
- EAPO augments human preference pairs with four controlled error types and incorporates them into SFT-to-DPO adaptation.The controlled errors are used to expose judges to plausible semantic mistakes beyond original preference pairs.
- Training only on original preference pairs may improve in-distribution agreement while weakening robustness to plausible semantic errors.
- Error-augmented training improves agreement with human preferences while maintaining strong error-type robustness and candidate-order consistency.
- Calibrated continuous preference-margin fusion aggregates independently trained MLLM judges while preserving complementary information about preference strength.The combined approach achieves the strongest overall performance among the evaluated configurations.