PaperlayerView exact revision on arXiv

Source-linked AI summary

Nine Emotion Centroids: A Label-Free Valence Axis That Transfers Across Four Modalities

Yousef Radwan

arXiv:2608.18090v1cs.CLcs.AI

TL;DR

Valence axes are typically learned from thousands of polarity-labeled examples, leaving their label-free and cross-modal generality unresolved. This paper derives a valence direction from nine emotion centroids and finds that a text-trained classifier transfers across independently trained text, vision, audio, and EEG encoders, while ablating the direction sharply reduces sentiment accuracy.

  • Problem

    Prior valence-axis methods require thousands of polarity labels, and it was unclear whether a label-light direction transfers across independently trained modalities.

  • Method

    The method embeds roughly 50 emotion-evocative paragraphs for each of nine categories, averages their representations, and takes the top principal component as the valence axis.

  • Results

    A text-trained 2-parameter classifier exceeded AUC 0.70 in all 12 cross-modal transfers, reaching 0.961 on EmoSet, while valence-axis ablation reduced sentiment accuracy by 5.5–37.2 percentage points.

  • Takeaways & Limitations

    The results support a task-relevant one-dimensional valence signal shared across independently trained modalities rather than a generic shared subspace.

  • Takeaways & Limitations

    On EEG, the unsupervised nine-class PC1 is at chance (AUC 0.512), so the reported text-to-EEG transfer uses a supervised binary-valence axis.

Abstract

from arXiv · show

Inside a modern language model sits a single internal direction that tracks how positive or negative a sentence feels. We show how to find this valence axis (V-axis) from just 9 emotion category names plus 50 short narrative paragraphs per emotion -- about 1,500 fewer labels than the usual supervised approach -- and that the same direction appears in vision, audio, and human-brain encoders never jointly trained. The recipe: embed nine emotion-anchored story sets in a frozen encoder, take the top principal direction of the nine averaged embeddings. Projecting new inputs onto it captures 93% of supervised performance on SST-2 (Llama-3-8B-Instruct, AUC 0.772 vs. 0.828), correlates with human valence ratings on 11,811 EmoSet images at r=0.636, reaches AUC 0.906 on ESC-50 audio (p<2.2e-15), and AUC 0.720+/-0.055 on EEG from 123 subjects (p<3.65e-8). The direction is mechanistically active: ablating it collapses sentiment accuracy by 5.5-37.2 pp across three LLMs vs. at most 0.88 pp for matched random directions (z>12). A 2-parameter classifier trained on text labels transfers to images (AUC 0.961), audio (0.764), and brain recordings (0.828) without target-modality labels; a generic 16-D subspace stays at chance (0.525). The recipe is bounded to continuous attributes -- seven tests on categorical concepts return near-chance -- and steering is family-specific (Llama/Mistral yes, Qwen/Gemma no).

1 Introduction

The paper introduces a label-light V-axis for continuous emotional valence, constructed as the top principal direction of nine emotion centroids. This direction predicts sentiment across text, vision, audio, and EEG, has causal effects in language models, and supports cross-modal transfer.

  • Method: 18 supervision events construct the V-axis from nine emotion categories rather than thousands of polarity labels.The recipe uses nine category names and nine writing prompts, with approximately 50 unlabelled sentences per emotion; its axis is PC1 of K=9 category centroids.
  • Four-modality probe: 0.772 AUC on SST-2 reaches 93% of the supervised probe’s 0.828 AUC using Llama-3-8B-Instruct.The projection uses the residual state at block 20; Qwen3-8B reaches AUC 0.787 after selecting block 31.
  • Causal evidence: 5.5–37.2 pp sentiment-accuracy drops from V-axis ablation exceed the matched random-direction cost of at most 0.88 pp.Across three LLMs, the signal is at least 12 standard deviations above the random null in every model, reaching 196σ in Qwen3-8B.
  • Universal cross-modal classifier: 0.961 AUC is achieved on EmoSet when a 2-parameter classifier trained on text transfers to another modality without target-modality labels.All 12/12 off-diagonal cells in the four-modality transfer matrix exceed AUC 0.70.
  • Limitations: Seven independent categorical-concept tests return at-or-near chance, bounding the recipe to continuous concepts rather than categorical ones.The paper presents the recipe as an empirical regularity, not an analytical theorem.

2 The Recipe

The recipe builds a one-dimensional valence direction from nine emotion-anchored story sets by averaging frozen-encoder representations and taking their top principal direction. The same construction adapts across modalities, requires only 18 supervision events, and fails when emotion words replace paragraph-length text.

  • Emotion centroids: Nine emotion centroids are formed from approximately 50 paragraph-length, emotion-anchored narratives per category without curating sentiment polarity, syntax, or domain.The nine categories are anger, disgust, fear, sadness, amusement, joy, inspiration, tenderness, and neutral; the total supervision budget is 9 emotion words plus 9 writing prompts.
  • Encode, pool, and extract: The V-axis is the top right-singular vector after centering the nine emotion centroids, and new inputs receive scalar projection scores onto it.Centroids come from a fixed encoder layer and final-token residual-stream activations.
  • Recipe summary: The procedure uses 450 forward passes, no concept labels beyond nine emotion names, and produces a 1-D direction.The count is 9 × 50 = 450 passes total.
  • Per-modality changes: Across vision, audio, and EEG, the recipe replaces narratives with modality-native objects and uses each encoder’s standard penultimate residual layer, except EEG’s unsupervised PC1 construction.It uses approximately 50 EmoSet images or ESC-50 clips per emotion, while EEG uses video clips and a different axis-building procedure.
  • What does not work: AUC 0.50 on SST-2 results when the nine emotion words alone replace paragraph-length pools, while non-trivial AUC requires roughly Nc ≥20.The one-word construction is random on Llama-3, demonstrating that paragraph-length per-emotion text is necessary.

3 Four Modalities

The label-free nine-emotion recipe recovers valence-tracking directions across four independently trained encoders, with strong results in text, vision, and audio. Cross-modal tests find reliable but small statistical alignment, while the EEG result specifically concerns axis alignment and uses a supervised axis construction with label-free bridge classification.

  • Four Modalities: Across four independently trained foundation encoders, the recipe recovers a 1-D direction tracking valence at supervised-classifier level.The four axes occupy different encoder spaces and are not literally the same vector.
  • Text: 0.772 AUC on SST-2 is achieved by Llama-3-8B-Instruct using 9 prompts and 450 sentences, versus 0.828 for a supervised head trained on 6,920 labels.The label-cost ratio is ≈1,546×; Qwen3-8B peaks at 0.787 at layer 31 and is at chance at half depth.
  • Vision: +0.636 Pearson r links the visual V-axis to held-out EmoSet human valence ratings, while 1,000 random directions reach |r| ≤0.112.The correlation is significant at two-sided p < 10^-50.
  • Audio: 0.906 mean AUC is reached by the CLAP-audio V-axis across 50 ESC-50 one-vs-rest splits, with paired-permutation p = 2.2×10^-15.The paper attributes this strength to CLAP’s contrastive natural-language objective.
  • EEG: 0.828 text→EEG AUC comes from a 1-D logistic head trained on SST-2 labels and applied to CBraMod activations, with no FACED labels during head fitting.The EEG bound concerns axis alignment rather than the cross-modal classifier; the EEG V-axis itself is constructed with a binary-valence LDA.

4 Causal Evidence

Directional ablation provides causal evidence that the V-axis carries sentiment information rather than merely correlating with it: removing it harms SST-2 accuracy far more than matched-norm random directions across three LLMs. Steering effects are family-specific, appearing in Llama and Mistral but not Qwen or Gemma.

  • Directional ablation: The intervention projects the V-axis out of the residual stream at every layer and token position during inference, then evaluates SST-2 accuracy with a fixed logistic-regression readout.The V-axis is taken from each model’s sentiment-optimal layer, while controls use three isotropically sampled directions of equal norm.
  • Causal identification: The ablation effect is sign-insensitive because projecting out +v̂1 or −v̂1 removes the same subspace, and the V-axis is fixed without sign-flip or neighborhood searches.Random directions are sampled before evaluation, making the reported z-scores unconditional.
  • Directional ablation: 5.5–37.2 pp V-axis drops versus ≤0.88 pp for matched-norm random directions, with z ≥12σ across all three models.The strongest result is Qwen3-8B: −37.2 pp at block 23 and 196σ above the random-direction null.
  • Steering versus ablation: Steering correlates with downstream sentiment polarity in Llama and Mistral, reaching ρ=0.45 for Mistral-7B-Instruct, 0.44 for Mistral-7B-base, and 0.37 for Llama-3-8B-Instruct.This steering effect differs from ablation and is family-specific.
  • Steering versus ablation: Steering is at chance for Qwen and Gemma, with all |ρ| < 0.05 and no significant effects reported.The contrast supports a family-specific steering pattern rather than a uniformly transferable intervention.

5 A Universal Cross-Modal Classifier

A two-parameter logistic head trained on SST-2 text labels transfers sentiment across images, audio, and EEG without target-modality labels, achieving AUC ≥0.70 in all 12 cross-modal directions. Transfer depends on the sentiment-specific V-axis rather than generic shared dimensions, while EEG requires a supervised valence axis despite label-free EEG head fitting.

  • Cross-modal transfer: AUC ≥0.70 in 12/12 off-diagonal cells shows SST-2-trained sentiment transfer across all four modalities without target-modality labels.The two-parameter head calibrates each modality’s one-dimensional V-axis score using only source-modality labels.
  • Cross-modal transfer: 0.961 image AUC, 0.764 audio AUC, and 0.828 EEG AUC result from the SST-2 text-trained head without target-modality labels.The same head transfers symmetrically across source choices; only the source modality used to fit the logistic regression changes.
  • Sentiment-specific bridge: 0.18–0.44 AUC separates the one-dimensional V-axis from generic 16-dimensional substrates in text-to-image sentiment transfer.The comparison includes a generic K=16 shared substrate, a K=16 random subspace, and raw 768-D CLIP features.
  • Sentiment-specific bridge: 0.771 raw CLIP-text AUC remains below 0.961 V-axis AUC, while substrate-only transfer stays at chance, ruling out generic agreement or high-dimensional features as sufficient.The V-axis succeeds because it is a task-relevant direction built for the continuous attribute of valence.
  • EEG limitation: 0.512 self-AUC for the unsupervised EEG PC1 rises to 0.867 with supervised binary-valence LDA, clarifying that text→EEG 0.828 is label-free only at the head-fitting stage.Joy and fear both project positive on the unsupervised EEG axis, producing an arousal-like direction.

6 What This Is Not

The 9-emotion recipe is bounded to continuous attributes rather than categorical concepts, and its steering capacity depends on model family. It yields usable causal steering directions for Llama and Mistral, but not for Qwen and Gemma.

  • Scope: Seven categorical-concept tests return at-or-near chance, bounding the recipe to continuous attributes.Causal-inner-product directions are approximately orthogonal to the V-axis (mean | cos |=0.038), visual retrieval reaches −0.84 pp R@10 versus supervised baseline, and five concept axes jointly reach AUC 0.52 versus per-axis-supervised 0.78.
  • Steerability: Llama and Mistral yield usable causal V-axes for steering, with ρ ∈[0.32, 0.45] across 4 models and all p < 10−9.Steering is family-specific: Qwen and Gemma produce probeable but not steering-usable V-axes, with all |ρ| < 0.05.
  • Steerability: Mistral-7B-base ρ=0.44 matches Mistral-7B-Inst ρ=0.45, so training stage does not predict steerability.The model family, rather than base-versus-instruct training stage, determines steering capacity.

7 Reasoning-Distillation Depth Shift

The V-axis shifts with model reasoning style: standard LLMs encode it most strongly at mid-depth, whereas reasoning-distilled models suppress it there and recover it near the final layer.

  • Mid-depth (ℓ≈L/2) is optimal for Llama, Mistral, and Qwen3 instruction-tuned and base variants.
  • Reasoning-distilled models suppress the V-axis at mid-depth and recover it near the last layer.This pattern appears in DeepSeek-R1-Distill and Qwen3-thinking models.
  • AUC 0.74 →0.78: DeepSeek-R1-Distill-7B and Qwen3-32B-thinking recover sharply at ℓ=1.00 after remaining near chance across ℓ∈[0.50, 0.70].

8 Related Work and Discussion

The paper positions its V-axis recipe as a low-label, continuous-attribute alternative to supervised concept directions and reports task-relevant convergence across independently pretrained encoders. It also frames the findings as empirical and limited in scope, with family-specific steering and several proposed tests.

  • Related work: 18 total labels—9 emotion names and 9 writing prompts—are at least 50× fewer than AxBench’s approximately 100–1,000 labelled examples per concept.The comparison also distinguishes the recipe by concept type, with AxBench targeting categorical concepts.
  • Related work: 3/3 text↔image↔audio V-axis pairs pass matched-norm CKA null tests at p ≤0.032, extending cross-encoder convergence to a task-relevant 1-D direction without shared pretraining.The generic K=16 substrate baseline is ruled out in the reported 4×4 matrix.
  • Limitations and future work: The recipe is an empirical regularity rather than an analytical theorem, bounded to continuous attributes and four tested encoders.The EEG axis remains supervised, and “causal” refers to inference-time projection rather than counterfactual circuit-level intervention.
  • Limitations and future work: Steering is family-specific, motivating tests of whether video-emotion metadata can make the EEG axis label-free and whether reasoning distillation restores steerability.These are presented as falsifiable next steps.

NeurIPS Paper Checklist … C Nc sweep

The paper’s checklist reports that its empirical claims, limitations, reproducibility details, and statistical reporting are addressed, while documenting the prompt corpus, model-specific layer selection, and prompt-count dependence of the V-axis. It also specifies ethical, licensing, broader-impact, and release considerations.

  • NeurIPS Paper Checklist: The checklist confirms that the abstract’s recipe, four-modality AUCs, causal ablations, and cross-modal transfer claims are established, with scope limited to continuous attributes and seven categorical failures.The paper frames the result as an empirical contribution rather than a theorem.
  • NeurIPS Paper Checklist: The paper acknowledges five limitations: empirical rather than theoretical evidence, continuous-attribute scope, supervised EEG axis, four tested encoders, and inference-time rather than counterfactual causality.Steering is treated as a separate, family-specific scope statement.
  • NeurIPS Paper Checklist: Reproducibility is supported by an end-to-end Algorithm 1, Nc≈50, layer-selection and modality protocols, prompt seeds and counts, depth-search procedures, and the Nc sweep.The checklist also states that experimental settings specify splits, ablation nulls, and two-parameter logistic-regression heads.
  • NeurIPS Paper Checklist: The checklist states that code release is planned, public datasets and asset licenses are identified, no new human-subject data were collected, and broader impacts discuss both label-efficient applications and family-specific steering risks.The recipe operates on frozen public encoders; FACED has consent and IRB documentation, while Llama and Mistral are described as steerable.
  • NeurIPS Paper Checklist: EEG reports AUC = 0.720 ± 0.055 with p=3.65×10−8, ESC-50 reports p=2.2×10−15, and causal ablations report z-scores 12–196 against matched-norm random directions.The vision permutation null is |r| ≤0.112 across 1,000 random directions.
  • A Prompt corpus and per-emotion examples: The 9-emotion corpus contains ∼50 paragraph-length continuations per emotion, totaling 450 prompts, and is released with the code.Per-emotion counts range from 47 to 52 across anger, disgust, fear, sadness, amusement, joy, inspiration, tenderness, and neutral.
  • B Per-LLM layer search: Standard models peak at ℓ≈0.50–0.67 with ∆AUC ≤0.02 across that band, whereas reasoning-distilled models peak at ℓ≥0.86 and lose > 0.20 AUC at mid-depth.The paper uses per-model peak blocks for Llama-3, Mistral-7B, Qwen3, and DeepSeek-R1-Distill.
  • C Nc sweep: Llama-3-8B SST-2 AUC rises from 0.500 at Nc=1 to 0.772 at Nc=50, becoming non-trivial near Nc≈20 and plateauing by Nc≈50.The sweep uses Nc ∈ {1, 5, 10, 20, 30, 50}.

D Steering panel (8-model) · E Depth-shift on categorical concepts

V-axis steering transfers across Llama and Mistral models but not Qwen, Gemma, or DeepSeek, while categorical-concept centroids remain at chance across tested depths. The categorical result indicates the depth shift is V-axis-specific rather than a generic late-layer effect of distillation.

  • D Steering panel (8-model): ρ=0.45 for Mistral-7B-Inst, the strongest steering association among the eight tested models.The association was measured between steering coefficient α and downstream sentiment polarity across 500 SST-2 prompts per coefficient.
  • D Steering panel (8-model): ρ=0.37 for Llama-3-8B-Inst and ρ=0.32 for Llama-3-8B-base.Both Llama variants showed positive associations between V-axis steering and downstream sentiment polarity.
  • D Steering panel (8-model): ρ=0.44 for Mistral-7B-base, with p=8×10^-14.The Llama–Mistral group passed p < 10^-9.
  • D Steering panel (8-model): ρ≈0 for Qwen3-8B-Inst and DeepSeek-R1-Distill-Qwen-1.5B, while Gemma-4-e2b reached ρ=0.004.Qwen, Gemma, and DeepSeek did not show the positive steering associations observed for Llama and Mistral.
  • E Depth-shift on categorical concepts: AUC stayed near chance from 0.498 to 0.511 across ℓ∈{0.25, 0.50, 0.70, 0.86, 1.00} × L.The categorical-concept recipe used nine CIFAR-100 superclass centroids mapped to text descriptions on DeepSeek-R1-Distill-7B.
  • E Depth-shift on categorical concepts: No late-layer recovery appeared, with performance flat-near-chance everywhere.The passage characterizes the depth shift as V-axis-specific, not a generic late-layer effect of distillation.

F AxBench Concept-500 NULL table

On AxBench’s 500 categorical concepts, the 9-centroid recipe fails to recover supervised concept directions beyond a matched-norm random-direction null. The recovery distributions are statistically indistinguishable.

  • Categorical concepts: 0.49 ± 0.04 recovery AUC across 500 concepts matched the random-direction null of 0.49 ± 0.05 (KS p=0.41).The recipe does not recover categorical-concept directions in Llama-3-8B-Inst at ℓ=16.

G ESC-50 per-class V-axis breakdown · H EEG salvage details · I Figure: four-modality bar chart

The V-axis performs strongly across ESC-50 classes but fails as an unsupervised EEG valence axis, where supervised LDA substantially improves discrimination. Across four modalities, the V-axis closely matches supervised heads while null directions remain at chance.

  • G ESC-50 per-class V-axis breakdown: 0.906 mean AUC on ESC-50, with class-wise AUC spanning 0.61 for silence to 0.99 for laughing.The paired permutation test gives p=2.2×10−15 across 50 binary one-vs-rest splits.
  • G ESC-50 per-class V-axis breakdown: Silence, wind, brushing teeth, washing machine, and snoring are the five hardest ESC-50 categories.The five easiest are laughing, crying baby, dog barking, chirping birds, and helicopter.
  • H EEG salvage details: 0.512 self-AUC for the unsupervised 9-class PC1 V-axis on EEG.Joy and fear both project positively, suggesting that PC1 captures an arousal-like direction rather than binary valence.
  • H EEG salvage details: 0.18 cosine between supervised binary-valence LDA and PC1 in 200-D CBraMod feature space.The two directions are near-orthogonal, and supervised LDA reaches 0.867 self-AUC on the reported subject result.
  • I Figure: four-modality bar chart: 1,500× label ratio advantage for the V-axis relative to supervised heads across four modalities.The figure reports V-axis performance matching supervised heads within 7 pp AUC, with null directions at chance.
  • I Figure: four-modality bar chart: Within 7 pp AUC across four modalities, the V-axis matches supervised heads while null directions sit at chance.Vision is evaluated by Pearson r against crowdworker valence on EmoSet; other modalities use held-out AUC.
Loading 2608.18090v1…