PaperlayerView exact revision on arXiv

Source-linked AI summary

VISTA: Value-Informed Event Appraisal for Multimodal Emotion Conflict

Jiale Dai, Liuxian Ma, Xiaoke Niu, Wenjing Zhang, Huiying Zhao, Zhaoxiang Liu, Shiguo Lian, Guojie Song

arXiv:2609.37324v1cs.AI

TL;DR

Conflicting emotional cues can be individually valid, and their interpretation depends on what an event means to the person. VISTA uses a learned seven-field appraisal to condition modality arbitration and prediction while retaining joint evidence. On CA-MER, its benefit concentrates on conflict, and the paper argues for scene-specific appraisal as an intermediate representation for interpreting conflicting evidence.

  • Problem

    When emotional cues disagree, the challenge is to interpret each cue in relation to the person’s concerns and the event rather than treating signal reliability alone as sufficient.

    Source: §1 · pp. 1–2

  • Method

    VISTA uses a learned seven-field appraisal of concerns, event relations, and expression conditions to condition modality arbitration and affect prediction.

    Source: §1 · p. 2

  • Results

    64.5% conflict accuracy on CA-MER versus 61.5% for Generic-CoT-SFT; the reported improvement concentrates on conflicting evidence.

    Source: §4.2 · p. 7, §4.5 · p. 9

  • Takeaways & Limitations

    The analyses and experiments support scene-specific appraisal as an intermediate representation that helps interpret conflicting emotional evidence.

    Source: §5 · p. 10

    Appraisal is a contextual event hypothesis, not a durable personal attribute; consequential uses require context-specific validation and human review.

    Source: Ethics statement · p. 10

Abstract

from arXiv · show

Conflicting emotional cues can be individually valid: a subdued voice may reflect a blocked goal while a smile satisfies a social obligation. Their interpretation depends on what the event means to the person. We introduce VISTA (Value-Informed Semantic Trust Arbitration), a learned seven-field appraisal interface that conditions modality arbitration on concerns, event relations, and expression conditions while retaining a joint-evidence residual. A log-odds decomposition separates emotion expectation from cue diagnosticity, motivating an interface that lets appraisal change how evidence is interpreted. With a shared Qwen2.5-Omni-7B backbone and matched training examples and steps, VISTA reaches 64.5% conflict accuracy on CA-MER, improving on modality gating by 2.5 percentage points on conflict and 0.2 on consistency. Shuffling appraisal across scenes or removing its decision connection reduces this benefit. A common frozen-backbone probe reaches 0.600 macro CCC for appraisal readout, compared with 0.505 for emotion-only fine-tuning. Evaluations across five benchmarks connect recognition under increasing conflict with appraisal readout and downstream decision use. Together, the analyses and experiments support scene-specific appraisal as an intermediate representation that helps interpret conflicting emotional evidence.

1 Introduction

Conflicting emotional cues can each be valid, but their interpretation depends on the person’s relation to the event and its social setting. VISTA introduces a seven-field appraisal interface to connect scene meaning to cue interpretation and modality arbitration.

1 Introduction

  • A subdued voice can reflect a blocked goal while a smile and congratulations perform public courtesy, making the cues conflict for the same person.The challenge is to interpret the disagreement in relation to the person and event, not simply select a more reliable stream.Source: §1 · pp. 1–2
  • Cross-modal affective disagreement appears in 49.0% of audited CH-SIMS clips, motivating a distinction between broad disagreement and strong conflict.The audit found different sentiment polarities across modalities in 1,117 of 2,281 clips under a fixed three-class mapping.Source: §1 · pp. 1–2
  • A person’s concerns make an event a setback or success, while public courtesy can explain an accompanying smile.The relevant reference is local to the event rather than an inferred fixed profile of broader priorities.Source: §1 · p. 2
  • VISTA’s seven-field appraisal represents concerns, event relations, and expression conditions to condition both modality arbitration and affect prediction.This makes the scene’s evaluative meaning available when conflicting evidence is interpreted.Source: §1 · p. 2
  • The paper’s contributions connect appraisal-dependent cue interpretation to a learned interface and matched controls that test where its recognition benefit arises.The analyses also distinguish conflict resolution, appraisal readout, and broader recognition scope across five benchmarks.Source: §1 · p. 2

2 Background and Related Work

Prior work addresses multimodal disagreement, appraisal, and concept-based representations, but VISTA focuses on using concern-relative appraisal to interpret sensory cues during recognition. It retains a direct evidence pathway rather than making appraisal the sole representation.

2 Background and Related Work

  • Conflict-aware work evaluates which modality agrees with an affective reference, while CHASE uses conflict to steer attention; VISTA instead supplies concern-relative interpretation to arbitration and prediction.The paper’s controlled gate comparison tests this decision role on a common backbone.Source: §2 · p. 2
  • The Figure 1 comparison holds cues fixed while changing the supplied goal, illustrating conditional interpretations rather than model predictions.The shared public setting supplies an expression condition.Source: Figure 1 · p. 3
  • Appraisal research includes emotion prediction, support retrieval, appraisal reasoning, and emotion–cause extraction, whereas VISTA uses appraisal to interpret conflicting sensory evidence at recognition time.Its correspondence and connection tests examine the interface’s contribution.Source: §2 · pp. 2–3
  • Unlike concept bottlenecks that organize prediction around concepts, VISTA retains a direct evidence pathway and uses appraisal as an auxiliary decision interface.The paper distinguishes this computational route from whether an explanation is faithful.Source: §2 · p. 3

3 Value-Informed Semantic Trust Arbitration

VISTA represents event meaning relative to a person’s concerns and uses that appraisal to alter how emotional evidence is interpreted. Its design connects a cue–appraisal interaction to modality weighting while retaining joint evidence and treating appraisal as a potentially decision-relevant representation.

3.1 Scene meaning and the diagnosticity of emotional cues

  • Appraisal can change cue diagnosticity, not only emotion expectation: a nonzero interaction rules out an additive log-odds representation and motivates cue–appraisal interactions.The design criterion is a decision pathway in which appraisal can change how a cue is interpreted, beyond shifting an emotion prior.Source: §3.1 · p. 4

3.2 The appraisal-conditioned decision pathway

  • VISTA defines a seven-field event appraisal and uses it to condition modality scores while retaining a separate joint-evidence residual.Appraisal changes prediction through modality weights; the residual preserves joint evidence. Figure 2 depicts the weighting, prediction, and fixed-evidence readout roles.Source: Figure 2 · p. 4, §3.1 · p. 4, §3.2 · p. 5

3.3 Learning and interventions

  • In the isolated label-visible condition, copying is 21.0% versus 3.0% under label-blind supervision, while grounding is 67.0% versus 73.0%.The contrast supports scene grounding as a criterion for appraisal supervision; training also combines task, unimodal, appraisal, conflict, and arbitration losses with partial-label masks.Source: §3.3 · p. 5
  • Shuffling tests scene correspondence, while the connection ablation retains appraisal supervision but removes appraisal from arbitration.These controls distinguish scene-specific correspondence from decision access to appraisal.Source: §3.3 · p. 5

3.4 Decision-relevant representation refinement

  • Appraisal can recover decision distinctions omitted by compressed evidence features, despite being computed from the same recognition input.The risk analysis says positive Bayes-risk reduction requires a decision distinction; if appraisal is already determined by the compressed representation, this gain is zero.Source: §3.4 · pp. 5–6
  • A learned classifier benefits from the refined representation only when its excess-risk change is smaller than the representational gain.The decomposition separates an opportunity created by retained distinctions from the readout’s ability to realize it.Source: §3.4 · p. 6

4 Experiments

The experiments test where recognition gains occur and whether correspondence and connection controls support the proposed interface. They use conflict-focused evaluations alongside appraisal probes and other recognition tasks.

4 Experiments

  • The evaluation plan first tests whether recognition gains concentrate on conflicting evidence, then examines correspondence and decision connection, with interventions and complementary tasks extending the account.The design predicts the greatest help when a cue admits competing emotional interpretations.Source: §4 · p. 6

4.1 Evaluation design

  • CA-MER conflict accuracy averages video-aligned and audio-aligned subsets, while the study also evaluates EmoMM, CH-SIMS v2, MELD, and THERADIA tasks.THERADIA is assessed separately for frozen-backbone appraisal probes and downstream emotion-intensity regression.Source: §4.1 · p. 6
  • Figure 3 organizes comparisons as subgroup accuracy, conflict-versus-consistency gains, the audio–video accuracy gap, and external methods under a common output interface.Its points show aggregate scores, and gains and gaps are in percentage points.Source: Figure 3 · p. 7
  • The trained models share Qwen2.5-Omni-7B, 8,000 examples, and an optimization schedule; Generic-CoT-SFT and VISTA both average 360 target tokens.The gate and VISTA have 12.19M and 12.39M trainable parameters, respectively.Source: §4.1 · p. 7

Conflict specificity.

VISTA’s advantage is concentrated on conflicting evidence, and tests associate that benefit with scene-matched appraisal connected to prediction. Across additional conflict strengths and evaluation modes, the results extend to selective responses, appraisal readout, and broader recognition.

4.2 The improvement concentrates on conflicting evidence

  • 64.5% conflict accuracy and 67.7% overall accuracy; versus Modality-Gate-SFT, VISTA gains 2.5 points on conflict and 0.2 on consistency (S = 2.3).The comparison locates the larger gain on conflicting inputs.Source: §4.2 · p. 7, Table 3 · p. 8

4.3 Scene correspondence and decision access strengthen recognition

  • 64.5% conflict accuracy falls to 62.5% without appraisal’s decision connection; cross-sample shuffling yields 62.0%, versus 63.7% with same-emotion donors.Removing expression regulation gives 63.6%; removing appraisal, conflict, or reliability supervision gives 63.2%, 63.5%, or 63.0%, respectively.Source: §4.3 · p. 8, Figure 4 · p. 9

4.4 Relevant changes elicit directed, selective responses

  • 75.0% direction agreement accompanies 92.0% irrelevant-rewrite stability, while 36.0% of relevant interventions change the predicted class.Direction agreement tracks target-probability movement; class-change rate counts movements that cross a class boundary.Source: §4.4 · p. 8, Table 2 · p. 8, Figure 4 · p. 9

4.5 The pattern extends across conflict strength and evaluation modes

  • On EmoMM, conflict and conflict-plus-missingness accuracy reach 52.0% and 43.5%, gains of 4.0 and 4.9 points over Generic-CoT-SFT.The consistency-to-conflict drop is 4.3 points, versus 6.6–7.5 for trained controls.Source: §4.5 · p. 9, Figure 5 · p. 10
  • On CH-SIMS v2, the gain over Emotion-SFT grows from 0.2 points in Q1 to 4.5 in Q4; Q4 accuracy is 81.5% and MAE falls from 0.365 to 0.315.This progression is across training-defined conflict-intensity groups.Source: §4.5 · p. 9, Figure 5 · p. 10
  • THERADIA frozen-backbone probes reach macro CCC 0.600 for VISTA versus 0.505 for Emotion-SFT; downstream ten-emotion CCC is 0.450 with generated appraisal and 0.480 with human appraisal.Downstream CCC is 0.390 without appraisal and 0.415 with disconnected auxiliary supervision.Source: §4.5 · p. 9, Figure 5 · p. 10
  • On MELD, weighted-F1 improves by 1.46 points over Emotion-SFT, with higher recall in all seven classes; VISTA uses 72 GPU-hours per seed and 15.5 s median latency.Equal-length Generic-CoT uses 40 GPU-hours and 13.5 s latency; gating uses 64 GPU-hours and 6.0 s.Source: Table 3 · p. 8, §4.5 · p. 9
  • Frozen appraisal prompting reaches 56.0% conflict accuracy versus 56.4% for generic reasoning, but training reverses the ordering: 64.5% versus 61.5%.The reported contrast supports the learned interface rather than prompting alone.Source: §4.5 · p. 9, Figure 5 · p. 10

5 Discussion and Conclusion

VISTA links event meaning to how emotional evidence is interpreted, while its appraisal fields distinguish concerns, event relations, and expressive conditions. The accompanying analyses stress that these are context-bound interpretations, not fixed personal attributes or uniquely determined emotion labels.

5 Discussion and Conclusion

  • VISTA connects event meaning to emotional-evidence use; its analysis separates emotion expectation from cue diagnosticity and relates scene-specific appraisal to conflict-sensitive recognition.The paper describes correspondence, connection, and intervention tests as connecting the recognition benefit to the scene-specific interface.Source: §5 · p. 10

A.1 Field types and missing supervision

  • Field values, confidence, evidence, and masks are stored separately; missing fields remain null rather than becoming numerical zero or valid negative judgments.The mask excludes missing fields from supervision, and each norm bit has its own validity mask.Source: §A.1 · p. 14, Table 4 · p. 14

A.2 The unit of appraisal and its evidence

  • Each appraisal is indexed to a person, event, and response time; its seven fields ask distinct questions about concern, congruence, expectation, agency, control, norms, and regulation.The fields are not interchangeable positive–negative scores.Source: §A.2 · p. 14
  • Appraisal fields form linked interpretations, not independent corroborating observations: congruence depends on concern, agency on the appraised event, and regulation on display conditions.A coherent goal edit can require a congruence edit.Source: §A.2 · p. 15

A.3 Evidence anchors and inferential distinctions

  • Agency and control distinguish who produced an outcome from whether the person can alter its consequences; norms and regulation distinguish standards from the display serving them.Separating these related questions avoids treating one contextual fact as several independent reasons for an emotion label.Source: §A.3 · p. 16

A.4 Underspecified scenes and unresolved appraisals

  • When evidence leaves an appraisal unresolved, that field remains open; other fields may still be informative, such as low control after a final decision.A short clip may omit concerns, expectations, or available responses.Source: §A.4 · p. 16

B.1 The same outcome can support different concerns

  • For the same non-selection, wanting the role makes the outcome obstructive, while entering to support a colleague can make it support the person’s stated purpose.The concern changes coherent cue explanations without determining the final emotion.Source: §B.1 · p. 17

B.2 The same setback can be expressed under different social demands

  • Audience and setting alter plausible explanations for a smile and subdued voice; a public courtesy obligation differs from a private conversation, without fixing the emotion.A separate paired scene contrasts disappointment with blame through agency and norms, while noting that a fair procedure can still yield an unwanted outcome.Source: §B.2 · p. 17, §B.3 · p. 18

C.1 Emotion expectation and cue diagnosticity

This section explains how appraisal shapes emotion expectations, cue diagnosticity, and weighted readout, including how uncertain or predicted appraisal may affect decisions. It also defines evaluation approaches and metrics for appraisal semantics, grounding, conflict recognition, and benchmark comparisons, distinguishing scene correspondence, decision use, event changes, and conflict prevalence from error burden.

C.1 Emotion expectation and cue diagnosticity

  • A fixed cue can change diagnosticity across appraisals, and appraisal–cue interactions cannot be reduced to context-only prior shifts or additive log-odds scores.In the constructed example, a positive display’s likelihood ratio falls from 4 to 4/3 under courtesy while prior satisfaction remains 0.4, changing the posterior from 8/11 to 8/17; these are analytical calculations, not VISTA outputs. The text says aggregate interventions do not provide the four-condition cue comparison needed to establish this interaction empirically.Source: §C.1 · pp. 18–20, Figure 7 · p. 19

C.2 Uncertain and imperfect appraisal

  • Uncertain appraisal should be marginalized at the probability level using evidence-conditioned state weights; bounded errors preserve a decision when its margin exceeds 2(δ + η).The two error terms distinguish misallocating probability among scene interpretations from misreading cues within an interpretation; with shared cue kernels, η = 0, and appraisal uncertainty matters most near a decision boundary.Source: §C.2 · pp. 20–21

C.3 What a predicted representation can contribute

  • A predicted appraisal adds no observation beyond the input, but can improve Bayes decisions when it preserves decision-relevant distinctions; trained readout error can erase or reverse that opportunity.The Bayes-risk gain vanishes when the coarse label remains optimal across appraisal states, and separately trained readouts can benefit or deteriorate depending on how they use the representation.Source: §C.3 · pp. 21–22

D A guide to appraisal and scene interventions

  • The intervention families distinguish scene correspondence, appraisal’s connection to the decision, semantic organization, and whether responses track relevant meaning rather than wording.They include appraisal shuffling, removing appraisal from arbitration while retaining auxiliary supervision, comparison with a generic semantic bottleneck, and relevant-change versus irrelevant-paraphrase tests.Source: D A guide to appraisal and scene… · p. 22, §D.1 · p. 23

D.1 Correspondence, connection, and content

  • Shuffling tests whether appraisal belongs to the current scene, while removing its forward decision connection tests whether supervision alone suffices; these are distinct from stop-gradient.Within-emotion shuffling preserves donor emotion identity but changes its relation to the scene; cross-sample shuffling removes that constraint, and both replacements keep the checkpoint fixed.Source: §D.1 · p. 23

D.2 Scene edits and representation replacement

  • Scene edits can alter both evidence and appraisal, whereas direct appraisal replacement holds the evidence representation fixed; coherent event changes may require coordinated field changes.For example, changing a goal while holding the outcome fixed may alter goal congruence, and making a setting public may change both norm and expression-regulation interpretations.Source: §D.2 · p. 23

D.3 Relevant response and label change are separate judgments

  • A relevant event change can alter appraisal or cue interpretation without changing the strongest affective label, so pair evaluation separates those judgments.Irrelevant paraphrases test stability when wording changes but concern, outcome, and expression conditions remain preserved.Source: §D.3 · p. 23

D.4 How appraisal changes the weighted readout

  • Appraisal replacement isolates its effect on the weighted readout by holding the checkpoint, evidence representations, and modality-availability mask fixed.The branch logits and joint-evidence residual remain fixed while the appraisal-conditioned modality weights change.Source: §D.4 · pp. 23–24
  • Appraisal changes class preferences by reallocating weight among branches with different class-pair margins, while the shared residual affects the reference margin and the change does not guarantee higher accuracy.With evidence representations and available modalities fixed, the weighted margin change depends on the weight change and branch margins; if all branches have the same margin, redistribution leaves the preference unchanged.Source: Figure 8 · p. 24, §D.4 · pp. 24–25

D.5 Semantic organization and evaluative references

  • 64.2% conflict accuracy with randomized field names is close to the complete system’s 64.5%, while a generic semantic bottleneck gives 63.2%.The comparison associates the benefit with organized appraisal content rather than literal field names; goal and norm provide evaluative reference, and expression regulation relates that reference to the observed display.Source: §D.5 · p. 25

D.6 Grounding the interface in observed context

  • The field audit distinguishes sampling consistency from adequate grounding, and partial-field masking preserves supported relations without treating every appraisal field as observed.Grounding asks whether utterances, visible events, or conversation history support a predicted field; the audit evaluates 400 candidates and records coverage, sampling consistency, and adequate evidence separately.Source: §D.6 · p. 25

E Dataset roles and scoring conventions

  • CA-MER comparisons use a shared normalized output interface, with conflict average, consistent accuracy, overall accuracy, and invalid-output rate reported separately.The nine-class parser requires a unique string-valued final_label; missing, multiple, refused, or unknown outputs count as errors in the main accuracy denominator.Source: E Dataset roles and scoring conventions · p. 26

E.1 Conflict definitions and error burden

  • 1,117/2,281 CH-SIMS clips (48.97%, or 49.0%) show cross-modal disagreement under the audit rule, independently of VISTA predictions.The audit counts each clip once, including 455 clips with both positive and negative modalities and 662 additional clips with neutral versus non-neutral disagreement. Conflict prevalence, conditional error rates, and error-category shares are distinct quantities and require separate denominators.Source: §E.1 · pp. 27–28

E.2 Public benchmark metrics and the normalized interface

  • 67.7% is VISTA’s overall accuracy from the displayed CA-MER subgroup means, while public set-matching scores remain distinct from normalized-interface accuracy.The public protocol’s Acc is precision-like for multiple predicted labels; the present comparisons use a shared normalized interface for MoSEAR, CHASE, and VISTA.Source: §E.2 · pp. 29–30

E.3 Complementary evaluation settings

The evaluation spans conflict recognition, appraisal readout, and ordinary emotion recognition under different transfer and adaptation settings. Directional analysis tests whether conflict gains are shared across both conflict subsets.

E.3 Complementary evaluation settings

  • EmoMM evaluates common-stage models without adaptation across conflict and missing-modality conditions, with CHASE reported separately because its router used EmoMM supervision.The evaluation uses 800 source examples; the equally weighted four-condition macro average is not full-sample accuracy because conditions overlap.Source: §E.3 · p. 30
  • THERADIA separately tests frozen-backbone appraisal readout on four available human dimensions and downstream regression of ten emotion intensities.The four labels do not cover VISTA’s full seven-field schema, and novelty is not equated with Expectation.Source: §E.3 · p. 30
  • MELD evaluates adapted models on seven-class conversational emotion recognition, while Base is evaluated zero-shot.Weighted-F1, macro-F1, and accuracy are reported.Source: §E.3 · p. 30
  • The evaluation protocol’s CH-SIMS split counts sum to 4,403, while the public corpus description lists 4,402 labeled clips; the counts remain separate.Conflict quartiles are defined from the training split and applied unchanged to test inputs.Source: §E.3 · p. 30

F Conflict specificity and directional improvement

F.1 Improvement in both conflict directions

VISTA’s average conflict improvement is shared across the two annotation-defined conflict directions. Across complementary evaluations, reported gains concentrate in higher-conflict settings, while the appraisal interventions reduce conflict-specificity gains.

F.1 Improvement in both conflict directions

  • Table 13 reports positive gains and positive directional specificity against all three trained controls, so average conflict improvement is shared by both conflict directions.The directional contrasts are calculated from displayed subgroup means in Table 14.Source: Table 13 · p. 31, §F.1 · p. 31
  • Cross-sample shuffling and cutting appraisal’s decision connection each reduce specificity from 3.3 pp to 1.5 pp; same-emotion shuffling yields 2.7 pp.Cross-sample shuffling lowers conflict and consistent accuracy by 2.5 and 0.7 pp; same-emotion shuffling lowers them by 0.8 and 0.2 pp.Source: §F.1 · pp. 31–32

G.1 CA-MER: all subsets and output validity

  • The complete CA-MER results table reports all five core models’ visual-aligned, audio-aligned, and consistent accuracies, plus output validity.Conflict Avg equally weights the two conflict directions; this passage gives the table scope but not its cell values.Source: Table 14 · p. 32, §G.1 · p. 32

G.2 EmoMM: conflict, missingness, and their intersection

  • On EmoMM, VISTA gains 4.0 pp under conflict and 4.9 pp under conflict plus missingness over Generic-CoT-SFT, but CHASE scores higher under both missingness conditions.CHASE had prior EmoMM supervision; the comparison separates conflict transfer from recovering absent evidence.Source: §G.2 · p. 32, Table 16 · p. 33

G.3 CH-SIMS v2.0: increasing conflict intensity

  • On CH-SIMS v2.0, VISTA’s gains over Emotion-SFT rise from 0.2 pp in Q1 to 4.5 pp in Q4, and Q4 MAE falls from 0.365 to 0.315.All three control comparisons have their largest gain in Q4; the groups are severity-ordered, not equally spaced conflict increments.Source: §G.3 · pp. 32–33, Table 17 · p. 33

G.4 The complete appraisal and training ablations

Ablations favor scene-specific appraisal content connected to the decision over field names alone or a generic bottleneck. Further tests examine prompts, paired interventions, modality diagnostics, appraisal readout, and downstream emotion prediction.

G.4 The complete appraisal and training ablations

  • Cross-sample shuffling reduces conflict accuracy by 2.5 pp, compared with 0.8 pp for same-emotion shuffling; randomizing field names changes it by 0.3 pp and a generic bottleneck by 1.3 pp.Removing appraisal, conflict, and reliability supervision reduces conflict accuracy by 1.3, 1.0, and 1.5 pp, respectively; changing the external explanation alone leaves accuracy unchanged.Source: §G.4 · p. 33, Table 19 · p. 34

G.5 Frozen-backbone prompts and explanation quality

  • On frozen-backbone prompts, Appraisal raises Conflict Avg from 54.5% to 56.0%, while generic conflict CoT reaches 56.4%.Appraisal prompting also has the longest outputs and largest invalid-output rate; its second-pass grounded-appraisal rate is 60.0% versus 52.0% for Direct, after the label is locked.Source: Table 21 · p. 34, §G.5 · p. 34, Figure 10 · p. 35

H.1 Two paired tests with different success criteria

  • On 150 valid intervention pairs, VISTA’s target-probability and log-odds differences are 0.120 and 0.484; irrelevant-rewrite stability is 92.0%.The valid-intervention label-change rate is 36.0%; the final configuration uses no counterfactual training loss (λcf = 0).Source: §H.1 · pp. 35–36, Table 22 · p. 36

H.2 Intervention dimensions and diagnostic weighting

  • VISTA’s target-modality change under the masking diagnostic is +0.040, compared with the controls’ reported +0.015–+0.020 range.These diagnostic weights summarize output changes across fresh masked-input evaluations and are distinct from native model weights.Source: §H.2 · p. 36

I.1 A shared linear probe on frozen representations

  • VISTA raises macro CCC by 0.095 over Emotion-SFT and improves all four available human appraisal dimensions in the shared frozen-backbone probe.The probe measures four dimensions, not full seven-field correctness.Source: Table 25 · p. 37, §I.1 · p. 37

I.2 Using appraisal to predict ten emotion intensities

  • For ten-emotion intensity regression, connecting model-generated appraisal yields 0.450 macro CCC, versus 0.415 with auxiliary supervision alone; human appraisal reaches 0.480.The generated-appraisal condition improves by 0.035 over auxiliary-only training and 0.060 over no appraisal.Source: Table 26 · p. 37, §I.2 · pp. 37–38, Figure 11 · p. 38

J.1 Teacher generation and field masks

  • The teacher generates three appraisal samples per input, and each field stores its value, confidence, evidence, and validity mask separately.Missing values remain null; partial masks support field-level supervision, including individual social-relevance components.Source: §J.1 · p. 38

J.2 From candidate pool to retained training set

The appraisal training set is curated from a larger candidate pool with partial-field masking, then audited for coverage and grounding. Additional diagnostics distinguish scene grounding from label association and account for resource costs.

J.2 From candidate pool to retained training set

  • The retained training set contains 8,000 of 10,256 candidates: 2,872 automatically accepted, 1,231 accepted after adjudication, and 3,897 retained with partial-field masking.The 2,256 remaining candidates are rejected; the retained examples come from CH-SIMS v2.0, THERADIA, and MELD.Source: Table 27 · p. 39, §J.2 · p. 39

J.3 Seven-field audit on 400 candidates

  • In the 400-candidate audit, agency and goal congruence have grounded proportions of 84.0% and 81.0%, while expectation, social norm, and expression regulation have coverage of 43.0%, 34.0%, and 43.0%.Coverage and grounding use distinct denominators; removing expression regulation reduces conflict accuracy by 0.9 pp, while partial masking retains examples without forcing unsupported fields.Source: §J.3 · pp. 39–40, Table 28 · p. 39

J.4 An isolated label-visibility diagnostic

  • Making the teacher label-aware raises field consistency from 0.53 to 0.65 and appraisal–label MI from 0.120 to 0.240 bits, while copying rises and grounding falls.Copying changes from 3.0% to 21.0%, and grounding from 73.0% to 67.0%; higher label association and scene grounding are distinct properties.Source: Table 29 · p. 41, §J.4 · p. 41

J.5 Second-pass explanation audit

  • In the second-pass explanation audit, grounded-appraisal rates are 52.0% for Direct and 60.0% for Appraisal prompt, with the first prediction locked before explanations are elicited.The second pass does not update the recognition decision.Source: §J.5 · p. 41

K.1 Training and inference resources

  • VISTA uses 72 GPU-h per seed versus 40 for Generic-CoT-SFT and has 15.5 s P50 latency versus 13.5 s.The three-seed common-training total is 216 GPU-h and excludes teacher generation, task adaptation, ablations, and extra diagnostic calls.Source: Table 30 · p. 41, §K.1 · p. 42

K.2 Errors corrected, introduced, and shared

The error audit distinguishes interpretation mistakes from missing context, while the following evaluations report ordinary-recognition results and specify the shared training and evaluation protocols.

K.2 Errors corrected, introduced, and shared

  • Among Emotion-SFT-only errors, conflict-source mistakes account for 24.0% and expression-regulation/sarcasm for 22.0%; shared errors include insufficient context at 20.0%.Within VISTA-only errors, goal/concern mistakes account for 11.0% and congruence/Expectation mistakes for 12.0%.Source: §K.2 · p. 42, Table 31 · p. 43

L Ordinary conversational recognition and class recall

M.1 What is held fixed across the core comparison

  • All four trained core methods share the common-stage conditions, while VISTA combines structured appraisal, conflict recognition, and semantic arbitration.The common-stage step count is 375; downstream task adaptation is a separate optimization stage.Source: §M.1 · pp. 43–44, Table 34 · p. 44

M.2 Training objective and missing supervision

  • The final objective combines emotion, unimodal, appraisal, conflict, and arbitration losses; missing targets contribute neither a loss nor denominator weight.The sample-level appraisal loss averages valid fields, and confidence is not automatically used as a loss weight.Source: §M.2 · p. 44

M.3 Checkpoint use across evaluation tasks

  • Evaluation distinguishes direct checkpoint use, task adaptation, and frozen-representation probing rather than treating every task as the same protocol.EmoMM uses the common-stage checkpoint without additional adaptation; CH-SIMS and MELD use adapted task predictors, while THERADIA compares frozen representations at equal probe capacity.Source: §M.3 · p. 45, Table 35 · p. 45

M.4 The complete cross-task result overview

The overview keeps conflict accuracy, appraisal readout, and ordinary-recognition metrics distinct; the accompanying reporting notes specify how scores and comparisons should be interpreted.

M.4 The complete cross-task result overview

  • The six headline measurements retain their task-specific metrics rather than being averaged into a universal score.They answer different parts of the scientific question: conflict accuracy, appraisal CCC, and ordinary weighted-F1.Source: §M.4 · p. 45, Table 36 · p. 45

N.1 Quantities, denominators, and precision

  • Appraisal and performance quantities retain different scales: classification rates are percentages, while CCC, MAE, and RMSE use native scales.Reported values and differences should be read using the metric’s stated units.Source: §N.1 · p. 45
  • Recognition and readout metrics are computed for seeds 42, 43, and 44 separately, averaged equally, and rounded once for display.F1 and CCC are computed within each seed before averaging; aggregate scores alone do not determine uncertainty intervals or significance tests.Source: §N.1 · pp. 45–46
  • Denominators vary by analysis, including 10,256 selection candidates, 8,000 retained examples, and 400 candidates for coverage.The six intervention types have 25 pairs each, and direction-compliance failures remain in the fixed evaluation denominator.Source: §N.1 · p. 46
  • No EmoMM-specific adaptation describes the optimization protocol, but does not establish source disjointness from shared CH-SIMS v2 training material.Source identity can differ from local clip IDs, so disjointness requires canonical-source comparison as well as exact clip or media matching.Source: §N.1 · p. 46

N.2 A complete map of the final experimental evidence

The evidence map locates the experiment families and frames their scope, while the independent public-label audit and responsible-use discussion clarify interpretation boundaries.

N.2 A complete map of the final experimental evidence

  • Table 37 maps the experiment families and their numerical results, including output-validity, prompt, external-method, class-recall, and resource analyses.It serves as a guide to the complete experimental evidence.Source: Table 37 · p. 46, §N.2 · p. 46
  • The public CH-SIMS audit independently finds 1,117 conflicts among 2,281 released clips using a fixed sentiment mapping.The audit is independent of VISTA predictions and training selection.Source: §N.2 · p. 46
  • The empirical comparisons support conflict gains, appraisal readout and downstream utility, but do not separately estimate the analytical Bayes-risk terms or fixed-evidence logit contrasts.Prompting, missingness, class recall, and latency characterize operating conditions.Source: §N.2 · p. 46
  • Appraisal is an event-specific hypothesis, not a durable personal attribute or moral judgment; consequential applications require context-specific validation and human review.Unsupported fields should remain uncertain or masked, and conversational and audiovisual data require privacy and dataset-access compliance.Source: §N.2 · p. 46

Related papers

Loading 2609.37324v1…