PaperlayerView exact revision on arXiv

Source-linked AI summary

Emotions Where Art Thou: Understanding and Characterizing the Emotional Latent Space of Large Language Models

Benjamin Reichman, Adar Avsian, Larry Heck

arXiv:2510.22042v2cs.CLcs.AI

TL;DR

The internal representation of emotion in LLM hidden states remains underexplored, despite the importance of affective processing. The paper analyzes sentence-level hidden-state geometry and tests alignment, decodability, and steering. It reports a transferable, interpretable emotional subspace and controllable emotion steering, with scope conditioned on pretraining coverage.

  • Problem

    How emotion is internally represented in LLM hidden states remains relatively underexplored, despite emotion's role in language and its relevance to interpretability and safe deployment.

    Source: §1 · pp. 1–2

  • Method

    The paper extracts emotion-relevant directions with centered SVD on sentence-level hidden states and evaluates their alignment, decodability, and steerability.

    Source: §1 · p. 2, §3 · p. 3, §4.4 · p. 9

  • Results

    The authors report a low-dimensional, interpretable emotional manifold that is distributed across layers, aligns across real and synthetic domains, and supports cross-domain probe generalization.

    Source: §5 · pp. 9–10

  • Takeaways & Limitations

    The learned intervention module steers internal affective state while preserving semantic content, with stronger control for basic emotions than nuanced categories.

    Source: §4.4 · p. 9, §5 · p. 10

    The universality claim depends on language and style being reasonably represented in pretraining; out-of-distribution settings show somewhat higher distortion and stress.

    Source: §5 · p. 10

Abstract

from arXiv · show

This work investigates how large language models (LLMs) internally represent emotion by analyzing the geometry of their hidden-state space. The paper identifies a low-dimensional emotional manifold and shows that emotional representations are directionally encoded, distributed across layers, and aligned with interpretable dimensions. These structures are stable across depth and generalize to eight real-world emotion datasets spanning five languages. Cross-domain alignment yields low error and strong linear probe performance, indicating a universal emotional subspace. Within this space, internal emotion perception can be steered while preserving semantics using a learned intervention module, with especially strong control for basic emotions across languages. These findings reveal a consistent and manipulable affective geometry in LLMs and offer insight into how they internalize and process emotion.

1 INTRODUCTION

Because emotion shapes persuasion, social signaling, and narrative context, understanding LLMs’ affective processing matters for interpretability and safe deployment. This paper addresses the limited understanding of internal emotion representations by analyzing hidden states and examining their structure and controllability.

1 INTRODUCTION

  • Emotion shapes persuasion, social signaling, and narrative context, making LLM affective processing relevant to interpretability and safe deployment.Source: §1 · p. 1
  • Prior sentiment classification demonstrates emotion identification but does not establish how emotional meaning is represented internally.Source: §1 · p. 1
  • Behavioral evaluations of emotional scenarios and affective tone assess model outputs rather than internal mechanisms.Source: §1 · p. 1
  • Emotion mapping and generation studies generally treat emotion as a label or generation condition, not a latent internal representation.Source: §1 · p. 1
  • The internal geometry of emotion in LLM hidden states remains underexplored compared with output behavior and classification accuracy.The paper investigates emotion representations across layers, datasets, and languages.Source: §1 · p. 2
  • The paper extracts an interpretable, directionally encoded emotional subspace and tests its cross-dataset, cross-language generalization and steerability.Its learned steering module is reported to preserve semantic content, with especially strong control over basic emotions.Source: §1 · p. 2

2 RELATED WORKS

Prior work treats emotion through discrete categories or continuous dimensions, while studies of LLMs have often examined outputs, supervised representations, or selected emotion dimensions. This paper investigates whether emotion structure emerges in decoder-only LLM hidden-state geometry without an imposed psychological model.

Models of Emotions

  • Discrete emotion theories posit distinct categories, while continuous models represent emotions as points in a low-dimensional space.The six basic emotions are one discrete taxonomy; valence-arousal-dominance is a common continuous formulation.Source: §2 · p. 2
  • The paper tests whether classic emotion dimensions emerge naturally in hidden-state geometry without explicit emotional supervision.Source: §2 · p. 2

Neuroscientific models of emotion

  • A constructionist-style interpretation is supported by results describing emotion as distributed across neurons and layers rather than localized to a small subset of units.The account links high separability to overlapping, multi-purpose components.Source: §2 · p. 2

Emotions in Latent Space

  • Output-level VAD mapping studies suggest emotion-relevant dimensions are accessible but do not analyze their internal structure or geometry.Source: §2 · p. 2
  • Earlier learned emotion spaces used classification objectives, external supervision, or labeled data, unlike the emergent space studied here.Wang & Zong (2021) report valence clustering and transferability across datasets.Source: §2 · p. 2
  • Prior probing found linear valence in contextual states but less separable arousal and dominance, using encoder-only models and fixed affective lexicons.This paper instead studies decoder-only LLMs and seeks structure directly from hidden-state geometry.Source: §2 · p. 2
  • Cross-language emotion classification and reported language-specific tuning motivate testing transfer through geometric alignment and projection-based analysis.Source: §2 · p. 3

3 METHODS

The methods extract emotion-relevant directions from sentence-level hidden states, interpret and align those directions across datasets, and evaluate geometric preservation and emotion decodability. Additional analyses examine variation across layers, compression, and neuron-level selectivity.

Centered-SVD

  • Centered SVD on mean-pooled sentence-level hidden states yields orthogonal directions used for alignment, probing, and causal manipulation analyses.The leading components are expected to capture primary affective axes when emotional content is the dominant structured difference across inputs.Source: §3 · p. 3
  • Emotion centroid ordering and standardized component polarity support cross-layer interpretation, while t-SNE provides a coarse nonlinear visualization of cluster structure.Source: §3 · p. 3

Space Alignment

  • Centroid cosine similarity and least-squares regression assess directional agreement and map paired real-dataset activations to the synthetic emotional space.Alignment error, transformation Frobenius norm, and spectral flatness characterize reconstruction and scaling.Source: §3 · p. 3

Geometry Preservation Metrics

  • Stress metrics quantify distance-matrix embedding discrepancy, while average, ℓ2, and σ-distortion distinguish global rescaling from uneven or inconsistent deformation.Classical low-stress thresholds for low-dimensional maps do not directly apply to high-dimensional hidden-state spaces.Source: §3 · p. 4

Layer-Level Distortion Analysis

  • Layer-level distortion fractions identify localized high-distortion regions, and full-space versus 50D comparisons test whether compression preserves coherent structure.Source: §3 · p. 4

Linear Probing and ML-AURA

  • Linear probes test emotion recovery after projection, and ML-AURA measures neuron selectivity using one-vs-all emotion scoring.ML-AURA summarizes each neuron's maximum token activation and uses AUROC above 0.9 to designate emotion-selective units.Source: §3 · pp. 4–5

4 EXPERIMENTS

The experiments test whether emotion forms a consistent geometry across datasets and whether that geometry can be steered. They report distributed, stable, interpretable emotional structure and steering that shifts target-emotion predictions while preserving semantic similarity.

4.1 DATASETS

  • Eight categorical-emotion datasets span five languages and varied domains; synthetic rewrites provide clean affect directions, while downstream evaluations use human-written data.Datasets limited to polarity or star ratings were excluded, and the languages were selected based on available high-quality emotion datasets and LLaMA 3.1 support.Source: §4.1 · p. 5

4.2 UNIVERSALITY ANALYSIS

  • 0.83–0.93 centroid cosine similarity between real and synthetic emotion directions indicates stable category encoding; instruction-tuned variants show higher similarity and lower regression error.MSE values are broadly comparable across languages, while spectral flatness and Frobenius norms do not consistently distinguish base from instruction-tuned variants.Source: §4.2 · p. 5, Table 1 · p. 6
  • High aggregate alignment coexists with local stress and distortion warping, yet projected activations remain above-chance decodable and substantial layer subsets retain coherent, transferable geometry.The results concern directional organization rather than strict isometry.Source: §4.2 · pp. 6–7

4.3 MODEL PSYCHOLOGY

  • Across six Ekman emotions, 75% of neurons per layer exceed the selectivity threshold on average, with sadness at 98%, surprise at 97%, and fear at 48%.Selectivity measures discrimination rather than activation magnitude; selectivity is distributed across depth without a monotonic trend, and MLP layers are slightly higher than attention layers (79% vs. 76.5%).Source: §4.3 · p. 7, Figure 1 · p. 7
  • 0.87, 0.83, and 0.80 Spearman correlations across layers for PC1–PC3 show stable emotion rankings; Go-Emotions also retains high correlations of 0.92, 0.74, and 0.73.The corresponding Kendall’s Tau values are 0.82, 0.77, and 0.74 for the first three components in the main analysis.Source: §4.3 · pp. 7–8
  • PC1 resembles valence, PC2 dominance or control, PC3 approach–avoidance, and PC4 arousal or urgency, though these unsupervised alignments are not exact.The components may blend multiple emotional signals.Source: §4.3 · p. 8
  • In t-SNE space, emotion classes form distinguishable but partly overlapping clusters, with happy and excitement often co-localized and fear and joy more distant.The clustering is consistent with a distributed yet semantically coherent emotional geometry.Source: §4.3 · p. 8, Figure 3 · p. 8

4.4 STEERING EMOTION REPRESENTATIONS

  • For most emotions, post-steering accuracy exceeds 80%, many cases reach 90–100%, and nearly all settings reach at least 50%, with low semantic-similarity losses.The module projects hidden states into the emotional latent space, computes an MLP shift, and adds it residually; its objective combines token-level control and semantic preservation.Source: §4.4 · pp. 8–9, Table 2 · p. 9

5 CONCLUSION

The paper concludes that LLMs encode emotion in a stable, interpretable latent manifold that transfers across datasets and languages, and that this structure can be steered. Its fidelity and controllability have boundaries, particularly when pretraining coverage is sparse and for nuanced emotions.

5 CONCLUSION

  • Emotional representations form low-dimensional, directionally consistent subspaces across layers and domains, with interpretable axes and cross-dataset probe generalization.The conclusion describes these structures as distributed and redundant, supporting high linear separability despite no explicit supervision.Source: §5 · pp. 9–10
  • Steering reliably shifts internal affect toward target emotions across languages, with strongest control for basic emotions and less consistent control for nuanced categories, especially in Hindi.The intervention preserves semantic content; envy and excitement remain challenging.Source: §5 · p. 10
  • Universality depends on language and style being reasonably represented in pretraining; sparse coverage raises distortion and stress, though usable detection and steering remain.Hindi and nineteenth-century German drama are given as out-of-distribution examples.Source: §5 · p. 10
  • The steering method is intended for transparent, user-directed contexts, not covert manipulation or emotionally charged rewriting without consent.It acts on intermediate hidden states, uses a semantic-alignment regularizer, and does not amplify harmful content beyond what the base model permits.Source: §5 · p. 10

B STYLISTIC VARIATIONS AMONG SELECTED DATASETS

The selected corpora span diverse linguistic styles and complexities, while the paper argues that their aligned emotional manifold reflects emotion rather than surface style. The dataset profiles illustrate substantial variation, and Bhaav is excluded from the style-metric table because of limited language tooling.

B STYLISTIC VARIATIONS AMONG SELECTED DATASETS

  • The study selects eight categorical-emotion datasets across languages, modalities, and writing styles, excluding polarity-only and star-rating datasets for insufficient granularity.The listed final dataset is French journalistic and encyclopedic text for children.Source: B STYLISTIC VARIATIONS AMONG SELECTED… · p. 14, §7 · p. 14
  • Table 4 omits Bhaav because language-specific tooling is limited; the synthetic dataset combines high syntactic complexity with comparatively constrained vocabulary.It has the longest sentences, highest syllables per word, and highest Flesch–Kincaid grade level, but the second-lowest TTR.Source: §7 · p. 14, Table 4 · p. 14

7. MultiEmotions-It (Sprugnoli et al., 2020): Italian YouTube and Facebook comments

  • Go-Emotions combines short sentences with high TTR, while CARER has long sentences and high TTR, and EmoTextToKids has long, comparatively regular sentences.CARER’s within-passage sentence-length variability is low; EmoTextToKids has moderately high TTR and middle-ranked lexical complexity.Source: §7 · pp. 14–15
  • Spanish and English EmoEvents tweets balance middle-range sentence length with high word length, lexical variety, and sentence-length variability.EmoEvents-English also has one of the highest sentence counts per passage.Source: §7 · p. 15
  • MultiEmotions-It has the fewest sentences per passage despite high lexical complexity, while German Drama pairs short sentences with high word length and TTR.MultiEmotions-It also has relatively high sentence-length variability; German Drama has high sentence counts and low sentence-length variability.Source: §7 · p. 15
  • GSM8K has the highest sentence count and lowest TTR, whereas SemEval headlines have the shortest sentences and passages but very high TTR.GSM8K also shows high sentence-length variability; SemEval shows nearly none.Source: §7 · pp. 14–15
  • Across styles ranging from mathematical instruction to dramatic dialogue and encyclopedic writing, the aligned manifold is argued to capture emotional content rather than textual style.The selected datasets vary substantially in style and complexity.Source: §7 · p. 15

C COSINE SIMILARITIES BETWEEN DATASETS

Appendix comparisons report high average cosine similarity between synthetic and human-written emotion activations across several models. A neutral, task-oriented GSM8K reference is poorly aligned, supporting the authors’ claim that alignment metrics depend on consistent emotional structure.

C COSINE SIMILARITIES BETWEEN DATASETS

D DISTORTION SCORE CONTEXTUALIZATION

  • GSM8K, used as an affect-neutral reference despite the impossibility of a truly emotionless baseline, has the largest distortions and among the highest stress values.The authors conclude that alignment metrics hold only for datasets with consistent emotional structure, and that the observed structure is not a surface-feature artifact.Source: D DISTORTION SCORE CONTEXTUALIZATION · p. 17

E UNIVERSALITY EXPANDED

The expanded analysis finds broadly coherent emotional geometry across datasets and languages, especially in stable base models, while distortion varies by model and dataset. Many layers retain cross-domain structure despite local failures.

E UNIVERSALITY EXPANDED

  • LLaMA-3.1-8B-Base has the lowest stress values across datasets; instruction-tuned LLaMA and Ministral are also relatively low, while both OLMo variants are substantially higher.Projecting into the 50D synthetic subspace increases stress across models and does not mitigate distortion for OLMo.Source: E UNIVERSALITY EXPANDED · p. 17
  • Base models generally have distortions nearer ideal, while instruction-tuned models show more outliers; OLMo-v2-Instruct has elevated distortion across nearly all layers and datasets.Ministral has five outlier datasets out of nine tested, yet retains broadly consistent emotional structure across multiple datasets and languages.Source: E UNIVERSALITY EXPANDED · p. 17
  • Even with localized distortion, large fractions of layers retain universal emotional spaces; OLMo-v2-Instruct shows majority distortion in three of nine datasets.For LLaMA-3.1-8B-Base, Go-Emotions is the only dataset with substantial distortion, at 43% of layers.Source: E UNIVERSALITY EXPANDED · pp. 17–18
  • Despite variation in stress and distortion, probe accuracy supports a shared, dataset-invariant emotional subspace across datasets and languages, especially in more stable base models.The table reports per-dataset distortion metrics and probe accuracy across three models; lower distortion indicates greater geometric consistency.Source: E UNIVERSALITY EXPANDED · p. 18, Table 5 · p. 18

F ML-AURA ACROSS MODELS

Across the evaluated models, most neurons at every layer separate the emotion of interest with AUROC above 0.9. Instruction-tuning improves this separability for Llama-3.1-8B.

F ML-AURA ACROSS MODELS

  • More than a majority of neurons at all layers in every evaluated model achieve AUROC > 0.9 for one-versus-all emotion classification.Ministral-8B is only slightly better than Llama at the same training stage; OLMo-Base performs worse than OLMo-Instruct, and both trail Ministral-8B and Llama.Source: F ML-AURA ACROSS MODELS · p. 19, Figure 10 · p. 19, Figure 12 · p. 19
  • Instruction-tuning boosts Llama-3.1-8B’s average share of emotion-separating neurons per layer by 4.8%.The comparison concerns the number of neurons per layer able to separate emotions.Source: F ML-AURA ACROSS MODELS · p. 19

G EMOTIONAL CONSISTENCY ACROSS PRINCIPAL AXES

Emotion rankings remain stable across layers and broadly consistent across models, although the models’ emotional manifolds occupy distinct internal coordinate systems.

G EMOTIONAL CONSISTENCY ACROSS PRINCIPAL AXES

  • Emotion orderings show consistently high Spearman and Kendall correlations across layers for both the synthetic dataset and Go-Emotions labels.The reported stability is especially pronounced along the first three principal components.Source: G EMOTIONAL CONSISTENCY ACROSS… · p. 20, Table 6 · p. 20
  • Across models, emotion rankings are highly correlated, but aligning their latent spaces requires high-complexity transformations.The shared topology coexists with model-specific parameterizations, likely shaped by architectural and pretraining differences.Source: G EMOTIONAL CONSISTENCY ACROSS… · p. 20, Table 7 · p. 21

H EMOTIONAL STEERING PER DATASET RESULTS

Learned steering controls internal emotional representations across datasets and languages while retaining low semantic-similarity loss. Performance varies by language and model.

H EMOTIONAL STEERING PER DATASET RESULTS

  • Post-steering accuracy typically exceeds 85% for most emotions across models, with particularly strong results in French, German, and Italian.Steerability remains high for many Hindi emotions, while lexical sparsity and data imbalance remain limiting factors for some emotions in under-resourced settings.Source: H EMOTIONAL STEERING PER DATASET RESULTS · p. 21, Table 8 · p. 22
  • Semantic similarity loss is low across datasets, models, and emotions, indicating that steering preserves much of the original representational structure.LLaMA-3.1-8B is most steerable for a plurality of emotion-dataset pairs, whereas Olmov2 has the greatest average delta across them.Source: H EMOTIONAL STEERING PER DATASET RESULTS · p. 21, Table 8 · p. 22

I ABLATIONS FOR EMOTIONAL STEERING

Ablations identify steering dimensions, semantic-loss margins, target-token cross-entropy weight, and selective layer intervention as important design choices. Tests use three fixed emotion-dataset pairs representing high, moderate, and poor main-result performance.

I ABLATIONS FOR EMOTIONAL STEERING

  • Ablations test seven design choices on sad/EmoTextToKids, anger/CARER, and fear/Bhaav, selected to represent high, moderate, and poor main-result performance.The tested choices include steering-subspace rank, GELU, synonyms, target-token cross-entropy weight, semantic-similarity-loss components, margin-loss structure, and target layers.Source: I ABLATIONS FOR EMOTIONAL STEERING · p. 22
  • At R = 20, steering is near-perfect or perfect across all tested settings; extremely low and some intermediate ranks fail or behave inconsistently.Performance generally improves with rank before gains saturate or regress beyond R = 20, particularly for fear.Source: I ABLATIONS FOR EMOTIONAL STEERING · p. 22, Table 9 · p. 23
  • At m2 = 20, all emotion-dataset pairs steer successfully, while low m2 values consistently fail; m1 shows relative stability with some fear instability.The selected configuration is m1 = 0.75 and m2 = 20.Source: I ABLATIONS FOR EMOTIONAL STEERING · p. 22, Table 10 · p. 23, Table 11 · p. 24
  • The best overall results occur at a target-emotion cross-entropy weight of 25, while lower weights perform poorly, particularly for fear.Higher weights generally improve steering.Source: I ABLATIONS FOR EMOTIONAL STEERING · pp. 22–23, Table 12 · p. 24
  • Removing GELU or the emotion-margin loss severely degrades steering, while steering across all layers performs worse than targeting layers aligned with the emotion direction.Removing synonyms causes failure on fear; removing the full semantic-similarity loss also causes collapse, although either its delta-norm or cosine component can be removed with limited degradation.Source: I ABLATIONS FOR EMOTIONAL STEERING · p. 23, Table 13 · p. 24

Related papers

Loading 2510.22042v2…