PaperlayerView exact revision on arXiv

Source-linked AI summary

LongEmotion: Measuring Emotional Intelligence of Large Language Models in Long-Context Interaction

Weichu Liu, Jing Xiong, Yuxuan Hu, Zixuan Li, Minghuan Tan, Ningning Mao, Hui Shen, Wendong Xu, Chaofan Tao, Min Yang, Chengming Li, Lingpeng Kong, Ngai Wong

arXiv:2509.07403v2cs.CL

TL;DR

Existing EI benchmarks often miss continuous emotional processing in long contexts and do not cover multiple EI capabilities together. LongEmotion evaluates long-context EI across six tasks, and its RAG and CoEM frameworks improve performance on the vast majority of tasks.

  • Problem

    Existing EI benchmarks often overlook continuous emotional processing in long-context inference, and current research commonly measures a single capability rather than multiple dimensions of EI.

    Source: §1 · p. 1

  • Method

    LongEmotion evaluates emotion recognition, knowledge application, and empathetic generation across six tasks, and introduces RAG and CoEM frameworks for those tasks.

    Source: §1 · p. 2

  • Results

    The RAG and CoEM frameworks improve performance on the vast majority of LongEmotion tasks; DeepSeek-V3 and GPT models also show stable gains with vanilla RAG.

    Source: §5.2 · p. 7, §6 · p. 9

  • Takeaways & Limitations

    LongEmotion provides a benchmark and experimental analysis of models’ emotional intelligence in long-context scenarios.

    Source: §1 · p. 2, §6 · pp. 8–9

    The benchmark and CoEM use text-only English data in psychological and emotional domains, and the authors do not establish whether translated data would preserve the same quality.

    Source: §7 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) have made significant progress in Emotional Intelligence (EI) and long-context modeling. However, existing benchmarks often overlook the fact that emotional information processing unfolds as a continuous long-context process. To address the absence of multidimensional EI evaluation in long-context inference and explore model performance under more challenging conditions, we present LongEmotion, a benchmark that encompasses a diverse suite of tasks targeting the assessment of models' capabilities in Emotion Recognition, Knowledge Application, and Empathetic Generation, with an average context length of 15,341 tokens. To enhance performance under realistic constraints, we introduce the Collaborative Emotional Modeling (CoEM) framework, which integrates Retrieval-Augmented Generation (RAG) and multi-agent collaboration to improve models' EI in long-context scenarios. We conduct a detailed analysis of various models in long-context settings, investigating how reasoning mode activation, RAG-based retrieval strategies, and context-length adaptability influence their EI performance. Our project page is: https://longemotion.github.io/

1 Introduction

LongEmotion addresses the gap between continuous emotional processing and existing EI evaluations that often focus on short contexts or single capabilities. It introduces a six-task benchmark and CoEM, then analyzes models’ EI in long-context settings.

1 Introduction

  • Existing EI benchmarks often overlook continuous emotional processing in long contexts and commonly measure only one capability, despite EI’s multiple dimensions.The paper identifies accurate emotion recognition, appropriate knowledge application, and empathetic expression as abilities relevant to long-context inference.Source: §1 · pp. 1–2
  • LongEmotion comprises six tasks spanning emotion recognition, knowledge application, and empathetic generation in long-context inference.Emotion Classification and Detection target recognition; Emotion QA and Summary target knowledge application; Emotion Conversation and Expression target empathetic generation.Source: §1 · p. 2
  • CoEM combines retrieval-augmented generation with multi-agent collaboration to retrieve and enrich contextually relevant information.The paper also adapts Self-RAG and Search-o1 by using conversational context as their retrieval corpus.Source: §1 · p. 2
  • The authors conduct experiments across settings and a case study to analyze LLMs’ emotional intelligence in long-context scenarios.The introduction identifies reasoning-mode activation, RAG-based methods, and emerging approaches as areas for analysis.Source: §1 · p. 2

2 Related Work

Prior work evaluates emotional intelligence and long-context modeling, but the cited benchmarks leave long-context emotional intelligence comparatively unaddressed.

2 Related Work

  • Emobench, EQ-Bench, and EmotionQueen assess different EI capabilities, but focus on short or synthetic interactions rather than extended conversational or narrative contexts.The passage characterizes these benchmarks as lacking the long contextual depth needed to assess EI in such settings.Source: §2 · p. 2
  • LongBench, LooGLE, and XL2Bench evaluate long inputs across tasks or domains, while RULER and InfiniteBench examine long-context reasoning and unbounded contexts.The cited works report persistent challenges involving extended inputs, distant dependencies, and context lengths beyond trained windows.Source: §2 · pp. 2–3
  • The related-work review distinguishes these long-context evaluations from EI assessment, noting that most target retrieval or general comprehension rather than emotional intelligence.The passage cites Liu et al. (2025) as a survey of long-context modeling and evaluation paradigms.Source: §2 · p. 3

3 LongEmotion: Construction and Task

LongEmotion organizes long-context EI evaluation into six tasks across recognition, knowledge application, and generation, and constructs examples through dataset adaptation, expansion, and annotation.

3.1 Task Design

  • Emotion Classification locates a target entity’s emotional category amid long-context noise, while Emotion Detection identifies the single segment with a distinctive emotion among N+1 segments.Classification is evaluated against ground-truth accuracy; Detection scores whether the predicted segment index matches the ground truth.Source: §3.1 · p. 3
  • Emotion QA tests answers grounded in psychological literature, while Emotion Summary assesses five aspects of psychological pathology reports against reference answers for factual consistency, completeness, and clarity.The summary criteria are validated in CPsyExam (Zhao et al., 2024).Source: §3.1 · p. 3
  • Emotion Conversation evaluates counseling dialogue at checkpoints across four stages with 12 therapy-informed metrics, while Emotion Expression elicits a five-phase emotional self-narrative assessed across six dimensions.GPT-4o serves as evaluator for both tasks; Emotion Conversation uses metrics informed by five therapeutic frameworks.Source: §3.1 · pp. 3–4
  • Six tasks cover emotion recognition, knowledge application, and empathetic generation, with long-text input for five tasks and long-text output for Emotion Expression.Emotion Classification and Detection target recognition; Emotion QA and Summary target knowledge application; Emotion Conversation and Expression target generation.Source: §3.2 · p. 4, Table 1 · p. 4, Figure 2 · p. 4

3.2 Data Construction

  • Emotion Classification and Expression adapt existing datasets, while Emotion Detection builds contrast sets by grouping texts by emotion and inserting mismatched segments.Classification embeds Emobench and FinEntity excerpts in BookCorpus passages; Expression uses situations from EmotionBench.Source: §3.2 · pp. 4–5
  • Emotion Conversation is built from 100 expanded dialogues and evaluated by psychology experts and GPT-4o, whose scores correlate at 0.934 (p = 0.066).The dialogues are organized into reception and inquiry, diagnostic, consultation, and consolidation and ending stages.Source: §3.2 · p. 5, Figure 3 · p. 5
  • Emotion Summary construction expands CPsyCounR experiences and reflections, labels five dimensions, and selects 150 samples; Emotion QA uses expert-written questions and filtering to retain 120 question-answer pairs.DeepSeek-V3 performs structured decomposition and content augmentation for Emotion Summary; QA reference answers are refined for clarity and consistency with F1 evaluation.Source: §3.2 · p. 5, Figure 4 · p. 5

4 Collaborative Emotional Modeling

CoEM is a hybrid retrieval-generation framework for long-context EI tasks, combining chunk selection and enrichment with multi-agent collaboration before task-specific generation.

4 Collaborative Emotional Modeling

  • CoEM combines retrieval-augmented generation with modular multi-agent collaboration in a five-stage architecture for long-context EI tasks.The stages are chunking, initial ranking, multi-agent enrichment, re-ranking, and emotional ensemble generation.Source: §4 · p. 5, Figure 5 · p. 6
  • CoEM segments context into token-length-constrained chunks, with each segment treated as a chunk for Emotion Detection and chunk sizes tailored to task characteristics.Parameter settings are provided in Appendix C.Source: §4 · p. 5
  • CoEM-Rank first scores chunks by dense semantic similarity using cosine similarity, then re-ranks enriched chunks for semantic relevance.The authors describe the first ranking as preserving factual relevance and the second as ensuring affective coherence.Source: §4 · p. 6
  • CoEM-Sage enriches selected chunks with external knowledge or latent emotional signals, with task-specific roles for recognition, knowledge application, and empathetic generation.The passage says these signals are incorporated into original chunks without task-specific leakage.Source: §4 · p. 6
  • CoEM-Core generates task-specific outputs from selected enriched chunks, the context, and the prompt; unlike CoEM, the RAG setting uses chunking, one ranking, and generation.The output may be classification, summarization, or dialogue generation.Source: §4 · p. 6

5 Experiment

Across LongEmotion tasks, DeepSeek-V3 and GPT models generally show strong EI and stable gains with vanilla RAG, while CoEM mitigates some models’ difficulty integrating retrieved chunks. The experiments also examine reasoning mode, CoEM-Sage choice, context length, and retrieval settings.

5.1 Experiment Setup

  • The experiments use GPT-4o-mini, GPT-4o, DeepSeek-V3, Llama3.1-8B-Instruct, and Qwen3-8B, with GPT-4o evaluating tasks that use automatic evaluation.The setup also includes broader Base-setting comparisons with GPT-5, Qwen3-14B, and Qwen3-32B; RAG and CoEM are applied only in Emotion Conversation’s fourth stage.Source: §5.1 · p. 6, §5.2 · p. 7

5.2 Results on LongEmotion

  • DeepSeek-V3 and GPT models generally show strong EI and stable gains with vanilla RAG, while CoEM mitigates Qwen3-8B and Llama-3.1-8B-Instruct’s retrieved-chunk integration difficulty.The authors attribute CoEM’s mitigation to improved contextual alignment and emotional reasoning through multi-agent collaboration.Source: §5.2 · p. 7, Table 2 · p. 7
  • Self-RAG’s selective enrichment further improves Emotion Summary performance, whereas the observed Search-o1 drop indicates Qwen3-8B struggles with autonomous search-based reasoning in emotional tasks.Search-o1 uses five search turns; Table 3 reports the ablation results.Source: §5.2 · p. 7, Table 3 · p. 8
  • Thinking gives Qwen3-8B the most significant improvement across the tested tasks, while Qwen3-14B’s improvement is not substantial.The ablation covers Emotion Classification, Emotion Detection, and Emotion Expression under the Base setting.Source: §5.2 · pp. 7–8, Table 4 · p. 8
  • On MC-4, DeepSeek-V3 also drives higher performance than GPT-4o when used as CoEM-Sage, matching their Base-setting ordering.Figure 6 presents the CoEM-Sage comparison.Source: §5.2 · p. 8, Figure 6 · p. 8
  • On Emotion Classification across context lengths, DeepSeek-V3 and Qwen3-8B show high stability and strong overall performance, while GPT-based models are less robust and sometimes fall below Llama-3.1-8B-Instruct.Figure 7 shows model accuracy by context length.Source: §5.2 · p. 8, Figure 7 · p. 8
  • GPT-4o-mini performs best on Emotion QA with 128-token chunks and eight retrieved segments; larger settings introduce noise and reduce overall performance.Figure 8 reports the chunk-size and retrieved-count ablation.Source: §5.2 · p. 8, Figure 8 · p. 8
  • The case study finds GPT-5 more theoretically strong but more mechanical and prone to hallucination, GPT-4o-mini more human-like but less theoretically grounded, and GPT-4o a balanced trade-off.The case study also visualizes CoEM and analyzes LongEmotion’s role in advancing EI.Source: §5.2 · p. 8

6 Conclusion

LongEmotion is presented as a six-task benchmark spanning three dimensions of emotional intelligence. The paper also reports improvements from its RAG and CoEM frameworks on most tasks.

6 Conclusion

  • LongEmotion comprises six tasks that challenge models across emotion recognition, knowledge application, and empathetic generation.The benchmark is intended to measure models’ Emotional Intelligence in long-context scenarios.Source: §6 · pp. 8–9
  • The paper builds RAG and Collaborative Emotional Modeling frameworks for each task, reporting improvements on the vast majority of tasks.CoEM is the paper’s named collaborative emotional modeling framework.Source: §6 · p. 9
  • The authors conduct exhaustive experiments and a detailed case study to analyze models’ EI in long-context scenarios.Source: §6 · p. 9

7 Limitations

The authors limit LongEmotion and CoEM to text and psychological or emotional content, and LongEmotion’s data cover English only.

7 Limitations

  • LongEmotion datasets use only text and are restricted to psychological and emotional domains.Source: §7 · p. 10
  • CoEM focuses on textual inputs and does not extend to modalities such as vision or audio.Source: §7 · p. 10
  • LongEmotion includes only English texts, and the authors say it remains uncertain whether translation would preserve the same quality.Source: §7 · p. 10

8 Ethical Considerations

The paper describes dataset privacy and safety practices and outlines the academic backgrounds and roles of its annotation team.

8 Ethical Considerations

  • The adopted datasets are formally published and comply with privacy and ethical protection standards; the authors report that augmentation and manual inspection introduced no ethical risks.Source: §8 · p. 10
  • LongEmotion’s datasets are intended solely for academic research and are used according to their source datasets’ license terms.Source: §8 · p. 10
  • The authors state that evaluated models are officially released and evaluation prompts are disclosed and designed with safety in mind.Source: §8 · p. 10

A Qualifications of Annotators

  • Psychology researchers develop the theoretical basis and conduct annotation, while computer science researchers adjust and organize task data.Student annotators and researchers receive compensation commensurate with local standards and task complexity.Source: A Qualifications of Annotators · p. 13

B.1 Comparison of GPT series models

GPT-5 outperforms GPT-4o and GPT-4o-mini overall, but its stronger use of psychological theory does not consistently produce natural or empathetic responses. The models show different trade-offs across reasoning, conversation, expression, and summary tasks.

B.1 Comparison of GPT series models

  • GPT-5’s overall capabilities surpass GPT-4o and GPT-4o-mini, with significantly stronger reasoning on Emotion Classification and Emotion Detection.For these two tasks, models were prompted to output only the final label.Source: §B.1 · p. 13
  • GPT-5’s understanding can reduce ground-truth alignment in Emotion QA, while GPT-4o and GPT-4o-mini respond more literally.The passage attributes GPT-5’s lower F1 score to modifying content according to its own understanding.Source: §B.1 · p. 13, Figure 9 · p. 13
  • GPT-5 scores higher on psychology theory-driven Emotion Conversation metrics but offers advice rather than genuinely demonstrating empathy.The comparison is illustrated by the model outputs in Figure 10.Source: §B.1 · p. 13, Figure 10 · p. 13
  • GPT-4o-mini’s Emotion Expression sounds more human, GPT-4o’s more rigid, and GPT-5’s more comprehensive and balanced.The comparison of model outputs appears in Figure 11.Source: §B.1 · p. 13, Figure 11 · p. 13
  • In Emotion Summary, GPT-5 structures its analysis based on psychological theories but often adds non-existent facts; Figure 12 highlights “slapping” in red although the source data never mentions such an action.Source: §B.1 · p. 13
  • Across tasks, GPT-4o-mini has richer emotional features but weaker theory application; GPT-5 is more theory-aware but rigid, while GPT-4o is more balanced.The paper notes that GPT-5’s mechanical output might lead to a less empathetic user experience in practice.Source: §B.1 · pp. 13–14

B.2 Case Analysis of RAG and CoEM

The case analysis contrasts Base prompts with RAG’s retrieved chunks and CoEM’s chunks plus summaries, showing task-specific retrieval and enrichment strategies. The appendix also reports benchmark-wide model limitations, synthetic-data comparisons, and implementation settings.

B.2 Case Analysis of RAG and CoEM

  • In Emotion Classification, RAG retrieves a more accurate emotional segment from noisy context, and CoEM further analyzes it for the greatest performance improvement.Figure 13 illustrates the comparison.Source: §B.2 · p. 14, Figure 13 · p. 14
  • Emotion Detection ranks multiple emotional segments by relevance; CoEM enriches their emotional features and reranks the packs, skipping initial ranking.The paper says this relevance-based approach boosts emotion distinction; Table 8 reports task-specific chunk sizes and retrieved counts.Source: §B.2 · p. 14, Figure 14 · p. 14, C Details of RAG and CoEM · p. 16
  • For Emotion QA, RAG retrieves relevant source content, while CoEM-Sage extracts query-relevant portions from each retrieved chunk.Figure 15 illustrates the procedure; performance is evaluated by F1 similarity with the ground truth.Source: §B.2 · p. 14, Figure 15 · p. 14
  • In Emotion Conversation, RAG ranks chunks against the previous three turns, and CoEM summarizes those turns with selected chunks before a second relevance ranking.The additional ranking is intended to further ensure relevance-assessment accuracy.Source: §B.2 · pp. 14–15, Figure 16 · p. 15
  • For Emotion Summary, RAG ranks chunks against target characteristics, and CoEM adds CoEM-Sage analysis of those chunks.Figure 17 illustrates the process.Source: §B.2 · p. 15, Figure 17 · p. 15
  • In Emotion Expression, RAG ranks chunks by each stage’s query, while CoEM-Sage analyzes emotional cues to help CoEM-Core understand and express its emotions.Figure 18 illustrates the process.Source: §B.2 · p. 15, Figure 18 · p. 15

B.3 Advantages of LongEmotion in Enhancing Emotional Intelligence

  • Two-stage CPsyCoun generation uses counseling notes and detailed skills in prompts, and its results are compared with straightforward one-stage generation.The comparison appears in Table 6; Two-Stage denotes the paper’s generation method.Source: §B.3 · p. 15, Table 6 · p. 16
  • LongEmotion experiments across Base, RAG, and CoEM reveal distinct model limitations, including GPT-5’s overly mechanical responses despite stronger theoretical capabilities.The analysis is accompanied by detailed case studies based on model outputs.Source: §B.3 · p. 15

D LLM as Judge Metrics Design

The appendix describes LLM-judged metrics for Emotion Summary, Conversation, and Expression, alongside sample-data formats and collections of evaluation and generation prompts.

D LLM as Judge Metrics Design

E Unified Format of Data

F Comprehensive Prompt Collections

  • The prompt collection spans evaluation, multi-agent enrichment, and emotional ensemble generation across tasks; the Emotion Expression ensemble prompt consolidates an originally staged prompt.GPT-4o evaluates tasks with automatic metrics, using prompts shown in Figures 26–31.Source: F Comprehensive Prompt Collections · p. 17, Figure 43 · p. 33
Loading 2509.07403v2…