DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning

• Hard-label rewards conflict with subjective, distributed emotions, while implicit affective cues exacerbate visual hallucinations and can undermine generalization.Source: §1 · pp. 1–2
• DSPO constructs a context-grounded emotional distribution prior by combining an annotated emotion’s lexical prior with image-specific context.Source: §1 · pp. 2–3
• Its Distribution-Aligned Emotional Diversity Reward uses each candidate’s leave-one-out contribution to bring the predicted emotion set closer to the prior without unconstrained dispersion.Source: §1 · pp. 2–3
• Counterfactual Visual Intervention Gating evaluates causal probability drops under visual masking to verify evidence and reduce hallucinations.Source: §1 · pp. 2–3
• +10.8% average cross-domain accuracy over EMO-R3; DSPO also achieves state-of-the-art performance on multiple emotion reasoning benchmarks.Source: §1 · p. 3

2.1 PRELIMINARY

• The model identifies visual evidence, generates multiple emotion candidates, and selects a label within a GRPO-style framework that samples groups of rollouts.Cause-grounded Emotional Thinking asks the model to locate key visual evidence with a bounding box before producing emotion responses.Source: §2.1 · pp. 3–4

2.2 CONTEXT-GROUNDED EMOTIONAL DISTRIBUTION PRIOR

• The prior combines a Warriner lexical emotion distribution with image-caption context mapped to VAD parameters, then fuses the two sets of parameters.The caption is generated by Mimo-v2 and processed by a sentence-level VAD regression model.Source: §2.2 · p. 4

2.3 COUNTERFACTUAL VISUAL INTERVENTION GATING

• CVIG masks the region cited as visual evidence, measures each candidate’s relative probability drop with an area penalty, and uses normalized weights to down-weight unsupported interpretations.The method compares predictions from original and counterfactual image features; the area penalty discourages selecting the entire image.Source: §2.3 · pp. 4–5

2.4 DISTRIBUTION-ALIGNED EMOTIONAL DIVERSITY REWARD

• DEDR maps candidate emotions to VAD vectors and uses leave-one-out marginal contributions to reward candidates whose inclusion moves the set toward the constructed prior.It compares the candidate-set centroid’s Mahalanobis distance to the prior with the distance after removing each candidate.Source: §2.4 · pp. 5–6

2.5 OVERALL REWARD AND TRAINING

• Training combines accuracy and format rewards with the counterfactual and diversity rewards in a weighted total reward.The format reward checks the prescribed reasoning stages and the formatting of the bounding box, emotion list, and chosen option.Source: §2.5 · p. 6

3.1 MAIN RESULTS

• At G = 8, DSPO leads in- and out-of-domain accuracy, with +46.7%/+8.7% improvements over SFT/EMO-R3 and +13.6%/+12.3% out-of-domain improvements over GRPO/EMO-R3.The latter gains are specifically reported for out-of-domain evaluation.Source: §3.1 · pp. 6–7
• DSPO achieves the best performance on KL, JS, and En-MAE among the four distribution-based metrics.The authors say these results indicate its sampling distributions are closer to the true human distribution; an entropy-based GRPO reward improves Hmodel but declines on the three relative metrics.Source: Table 2 · p. 7, §3.1 · p. 7

3.2 ABLATION STUDIES

• CVIG alone mainly improves in-domain accuracy but degrades on WebEmo, DEDR alone improves out-of-domain performance, and combining them further improves overall performance.The authors attribute DEDR's out-of-domain gains to learning continuous emotion distributions rather than fitting source-training hard labels.Source: Table 3 · p. 7, §3.2 · p. 7
• Performance drops with K = 1 and begins to decline as K increases to 7; the authors select K = 5 to balance subjective coverage and candidate reliability.A moderate candidate set covers multiple plausible regions of the target VAD distribution and provides more informative marginal-contribution estimates; excessive candidates add redundancy and noise.Source: Table 4 · p. 7, §3.2 · p. 7

3.4 CASE STUDY

• In the EmoSet case study, DSPO identifies awe where EMO-R3 predicts disgust, and CVIG assigns the lowest weight to the hallucinated sadness interpretation.Removing sadness reduces the overall distance to the estimated human-aligned diversity, whereas removing awe increases it.Source: Figure 4 · p. 9, §3.4 · p. 9

3.3 EFFICIENCY ANALYSIS

• On Emotion6, DSPO improves out-of-domain accuracy by 12.8% over EMO-R3 while consuming 27% more training time, with no additional inference-time cost.The efficiency analysis uses an 8-rollout setting on EmoSet; the gating and diversity modules are removed during inference.Source: §3.3 · pp. 8–9, Figure 5 · p. 9

4 CONCLUSION

• DSPO targets two limitations of conventional RL for emotion reasoning: sparse discrete rewards and exacerbated visual hallucinations.The framework is designed for robust emotion reasoning.Source: §4 · p. 9
• A context-grounded emotional distribution prior and Distribution-Aligned Emotional Diversity Reward assess each candidate emotion's marginal contribution toward the prior.Counterfactual Visual Intervention Gating estimates candidate-wise causal drops through counterfactual masking.Source: §4 · p. 9
• The authors report that DSPO consistently improves overall accuracy and delivers particularly strong cross-domain generalization.Source: §4 · p. 9

A.1 MULTIMODAL EMOTION REASONING

• Early multimodal emotion reasoning treated the task as classification, while later work pursued open-ended and interpretable reasoning because fitting ground-truth labels alone lacks emotional interpretability.The passage describes modality-fusion methods and later systems including AffectGPT (Lian et al., 2025a), OV-MER (Lian et al., 2025b), and EmotionLLaMA (Cheng et al., 2024).Source: §A.1 · p. 14
• Despite open-ended approaches and reinforcement-learning methods, prior work still treats multimodal emotion reasoning as label-level prediction, overlooking continuous emotional subjectivity.The passage names R1-Omni (Zhao et al., 2025), OmniOPSD (Cheng et al., 2026), and EMO-R3 (Fang et al., 2026) among the RL approaches.Source: §A.1 · p. 14
• DSPO uses group-level soft distributional matching over generated rollouts to encourage valid emotional diversity and continuous emotional reasoning.The passage notes that semantically similar emotions can coexist and that discrete labels prevent models from learning continuous emotional reasoning.Source: §A.1 · p. 14

A.2 HALLUCINATION MITIGATION IN MLLMS

• Multimodal hallucinations are inconsistencies between generated text and provided images; proposed sources include language-prior over-reliance, visual-perception errors, and inadequate cross-modal reasoning.The passage surveys dataset-based fine-tuning, post-hoc tools or self-reflection, and decoding interventions on anomalous attention tokens.Source: §A.2 · p. 14
• Emotion reasoning is especially vulnerable because cues such as micro-expressions, lighting, and atmosphere are implicit rather than explicit physical entities, and models may fabricate visual facts.The authors infer this vulnerability from the abstract, implicitly nested nature of emotional cues (Ye et al., 2024).Source: §A.2 · p. 14
• CVIG masks specific visual regions, compares emotion probabilities on the counterfactual image, and rewards causal visual evidence while penalizing hallucinated fabrications.Source: §A.2 · pp. 14–15

Objective

• DSPO aims to maximize conditional mutual information I(O; Z|V), equivalently minimizing residual uncertainty H(Z|O, V) about human emotion given the image and MLLM response.The analysis treats H(Z|V) as a constant determined by human priors.Source: §A.3 · p. 15

ELBO

• Because Ptrue(Z|O, V) is difficult to observe directly, the analysis introduces an evidence lower bound (ELBO) to approximate it.The proof then focuses on maximizing the reconstruction term because the KL-divergence term is a non-negative constant unrelated to πθ.Source: §A.3 · p. 15

Reconstruction term

• The reconstruction term links factually accurate MLLM descriptions to CVIR and fitting outputs to the true human distribution to DEDR.The analysis identifies these two abilities as the basis for treating DSPO as a variational lower bound of genuine human emotion.Source: §A.3 · p. 15

Datasets and metrics

• Emotion accuracy is measured by hard-matching the final boxed prediction to the ground-truth label across EmoSet, Emotion6, and WebEmo.The evaluation uses each dataset in turn as a training source and the other two as external datasets.Source: §A.4 · p. 15

Base models and implementation

• The study compares two backbones against zero-shot, SFT, GRPO, DAPO, and EMO-R3 baselines under two in-domain and out-of-domain dataset arrangements.The backbones are LLaVA-1.5-7B and Qwen2.5-VL-3B-Instruct.Source: §A.4 · p. 15

Distribution-based evaluation

• Distribution evaluation uses 50 independent samples per image to approximate predictions for comparison with Emotion6 multi-annotator labels.The sampled category frequencies are smoothed with α to avoid divergence collapse from zero probabilities.Source: §A.4 · p. 16

Distribution-based metrics

• Distribution similarity is assessed with KL and JS divergence, while entropy metrics measure predicted uncertainty and its proximity to ground-truth entropy.The study reports model entropy and entropy mean absolute error (En-MAE).Source: §A.4 · p. 16

Visual grounding

• The template first identifies an emotion-triggering visual element and provides its bounding-box coordinates.The listed elements include an action, facial expression, object, or environment.Source: §A.5 · p. 16

Psychological reflection

• It next describes how a human observer would emotionally resonate with the visual trigger, expressing valence and arousal.Source: §A.5 · p. 16

Emotion candidates and selection

• The template synthesizes the visual evidence and psychological reflection into K distinct emotions, each linked to the visual trigger, then selects the most appropriate option.The candidate list and final choice use separate output formats.Source: §A.5 · p. 17