PaperlayerView exact revision on arXiv

Source-linked AI summary

Emotion Is Not External Knowledge: Internal Attention Signals for Affective State Reasoning

Wenqiu Tang, Zhen Wan, Yihui Wang, Takahiro Komamizu, Ichiro Ide

arXiv:2610.21347v1cs.CL

TL;DR

Affective cues in dialogue may be sparse and persist across topic shifts, while existing attention interventions rely on external control signals. SIAB localizes emotion-sensitive heads and uses their own attention to select tokens to boost. The paper reports gains across models, datasets, and prompting settings, and finds that internal and external signals can be complementary.

  • Problem

    Affective cues may be sparse and persist across topic shifts, and prior attention interventions rely on externally selected heads or target spans.

    Source: §1 · p. 1

  • Method

    SIAB uses activation patching to localize emotion-sensitive heads, then boosts the content tokens those heads already attend to at inference time.

    Source: §1 · p. 2

  • Results

    SIAB shows consistent gains across models, datasets, and prompting settings, with analyses finding sparse, emotion-enriched heads and layer-proximity-dependent complementarity with PASTA.

    Source: §1 · p. 2

  • Takeaways & Limitations

    A fixed causally localized head set transferred across two contrasting affective-evidence demands, while internal and external signals showed complementary gains.

    Source: §6 · p. 9

    Evaluation covered English ECC and GoEmotions only; transfer of the recovered head set across languages remains untested.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Large Language Models (LLMs) increasingly operate in multi-turn interactions that require maintaining latent conversational states, not only explicit facts. We study affective state as an implicit state inferred from sparse contextual cues that remain relevant after topic shifts, and ask whether LLMs expose internal attention signals for routing such evidence at inference time. Prior attention interventions rely on labeled in-domain head profiles and externally specified target spans, limiting transfer across affective-evidence distributions. We propose Self-targeted Internal Attention Boosting (SIAB), which localizes sparse affective heads via causal localization (in detail activation patching) on synthetic minimal pairs and boosts, at the answer position, the content tokens these heads already attend to; both edited heads and target tokens are derived from the model's own behavior, enabling the same head set to be reused across tasks.We evaluate on ECC and GoEmotions across three LLMs and multiple prompting settings. Across settings, SIAB improves prompt-matched baselines by up to +10.3 macro-F1 on ECC and +5.5 on GoEmotions. Combined with PASTA-style external supervision, it achieves up to +13.84 accuracy points and +12.51 macro-F1 points, while mechanistic analyses trace the gains to sparse, emotion-enriched heads and show that layer proximity affects additivity.

1 Introduction

Affective state must be inferred from sparse dialogue cues that can remain relevant after topic shifts, making prediction a problem of routing existing evidence. SIAB uses internal attention signals to select heads and target tokens, and the paper studies its performance and mechanisms across contrasting affective-evidence settings.

1 Introduction

  • The paper frames affective prediction as routing sparse contextual cues that may remain relevant after topic shifts, rather than supplying missing information.Emotion recognition is used as a controlled proxy for studying this routing problem at inference time.Source: §1 · p. 1
  • The central questions are whether internal attention signals can route affective evidence without external spans and whether internal and external signals are redundant or complementary.Source: §1 · pp. 1–2
  • SIAB derives both the edited heads and boosted content tokens from the model’s own behavior, using activation patching to localize heads and their attention to select targets.At inference time, it adds a scalar bias to pre-softmax attention logits for content tokens those heads already attend to.Source: §1 · p. 2
  • The evaluation uses ECC, where cues are dispersed across earlier turns, and GoEmotions, where dense cues must be distinguished among fine-grained categories, reusing the same head set across both.Source: §1 · p. 2
  • Mechanistic analyses find that gains come from a sparse set of heads that attend more to affective words, while SIAB–PASTA complementarity depends on layer-band proximity.Source: §1 · p. 2
  • The contributions are SIAB without parameter updates or annotated spans, gains across models, datasets, and prompting settings, and characterization of its mechanisms and boundary conditions.Source: §1 · p. 2

2 Related Work

Prior attention and steering methods use varying degrees of external control, while mechanistic studies of emotion recognition have not addressed head-level recognition. This paper motivates deriving both intervention signals internally to study affective-evidence routing.

2 Related Work

  • Prior attention-level interventions use heads selected by development-set steering effects and user-annotated spans, heuristically extracted targets, or head-output interventions; residual-stream steering is coarser-grained.Source: §2 · p. 2
  • Mechanistic interpretability tools such as activation patching and direct logit attribution support locating internal components and linking intermediate contributions to final-token logits.The paper uses this framing to motivate localizing heads that internally route affective contrast instead of specifying emotion spans externally.Source: §2 · p. 2
  • Existing emotion-recognition mechanistic work studies sentiment directions, multi-head units, or residual-stream circuits, while accuracy-focused work uses retrieval or training strategies without examining internal attention.Source: §2 · p. 2

3 Method

SIAB derives both the attention heads to edit and the target tokens from internal model behavior. It identifies affect-sensitive heads through activation patching, refines them on dialogue content, then boosts tokens those heads already attend to at answer positions.

3 Method

  • SIAB uses the model’s internal behavior to select both emotion-sensitive heads and target tokens, then boosts those tokens at inference time.The intervention adds a pre-softmax attention-logit bias without parameter updates or externally specified spans.Source: §3 · p. 2, Figure 2 · p. 3

3.1 Head Selection via Activation Patching

  • Activation patching tests each head by restoring its clean-run final-token output in a corrupted prompt and measuring recovery toward the clean emotion prediction.The clean and corrupted prompts differ only in the affective cue, and heads above a recovery threshold form the candidate set.Source: §3.1 · p. 3

3.2 Label-free Refinement on Dialogues

  • Dialogue refinement filters candidate heads by their attention and value mass on content words, using 200 DailyDialog conversations without gold emotion labels.The final set consists of the top-K heads by the combined attention/value score.Source: §3.2 · pp. 3–4

3.3 Attention Logit Boosting

  • At inference, SIAB biases the pre-softmax logits of each selected head’s top-r attended content tokens at the final query positions.The head set is reused across both datasets, and the bias magnitude is calibrated per model and held fixed across them.Source: §3.3 · p. 4

4 Experimental Evaluation

SIAB improves corresponding prompting baselines across 18 model–prompt–dataset settings, while PASTA’s in-domain strength transfers poorly; composing both signals is usually additive.

4.1 Datasets

  • ECC tests affective cues dispersed across long multi-turn dialogues, whereas GoEmotions tests dense local cues among 27 emotion categories; neutral-only examples are excluded.The evaluation retains 3,821 test examples after excluding samples labeled solely as Neutral.Source: §4.1 · p. 4

4.2 Evaluation Metrics

  • Macro-F1 and accuracy are reported; ECC uses 10-label argmax, GoEmotions uses multilabel evaluation with tuned temperature and threshold, and paired bootstrap uses 10,000 resamples.For GoEmotions, Top-1 accuracy counts an argmax label as correct when it belongs to the gold label set.Source: §4.2 · p. 4

4.3 Models

  • The evaluation uses Gemma3-4B, Qwen3-4B, and LLaMA3-8B, three open, small-to-mid-sized instruction-tuned models from different families.Source: §4.3 · p. 4

4.4 Baselines and Prompting Settings

  • Baselines include zero- and few-shot prompting, dataset-profiled and shared-head PASTA variants, PASTA+SIAB, and same-layer random heads matched in size and tuned on dev data.The PASTA variants test in-domain profiles, cross-dataset transfer, shared profiles, and composition with SIAB.Source: §4.4 · pp. 4–5

4.5 Results

  • 18 model–prompt–dataset settings show SIAB beating matched prompting baselines, with gains up to +10.27 macro-F1 on ECC and +5.47 on GoEmotions.The same head set was evaluated across models, datasets, and prompting settings.Source: §4.5 · pp. 5–6
  • PASTA profiles were competitive with or stronger than ZS+SIAB in-domain but transferred unstably, whereas SIAB had positive gains in all 18 settings.Applying PASTA-ECC heads to GoEmotions lowered LLaMA3 by −7.19 macro-F1; SIAB was weaker at the in-domain peak but more stable across tasks.Source: §4.5 · p. 6
  • The two signals composed additively in five of six configurations; PASTA+SIAB recovered 79–116% of their naive summed ZS gains in five configurations and yielded the best macro-F1 in four.LLaMA3 on ECC was the exception, recovering 54%; the authors attribute this to layer-band disjointness.Source: §4.5 · p. 6
  • SIAB outperformed the mean of 20 layer-matched random head subsets across models, datasets, and metrics, indicating that same-layer heads are not interchangeable.The random subsets matched SIAB's layer set, head-set size, and tuning protocol.Source: §4.5 · p. 6

5 Mechanistic Analysis

Mechanistic analyses find that SIAB’s signal is concentrated in a small, selected set of emotion-enriched heads, while rescue and combination gains depend on sample properties and head-set proximity.

5.1 Sparsity of the Affective Head Set

  • Across models and datasets, F1 rose quickly with a few boosted heads, saturated near small K*, then fell as more heads were added.The same K* appeared on ECC and GoEmotions for each model, suggesting sparsity was more model-dependent than dataset-specific.Source: §5.1 · p. 6, Figure 3 · p. 7

5.2 Selected Heads Attend More to Emotion Words

  • Selected heads had the highest emotion-word ratios in all three models and beat random heads within early, middle, and late layer regions.This suggests the enrichment belongs to the selected heads themselves, not merely their layer positions.Source: §5.2 · pp. 6–7, Table 2 · p. 7

5.3 What Makes a Sample Rescuable?

  • On Gemma3, HARD had a 16.8 Top-1-versus-Top-2 margin and a −20.2 gold-label gap; rescue fell sharply when the gold label trailed by more than 10 logit points.HARD mostly occupied this deeper-gap regime, where SIAB moved the gold label but rarely enough to change the prediction.Source: Table 4 · p. 7, §5.3 · p. 8
  • Selected-head attention favored gold-label over wrong-label emotion tokens; POS showed stronger directional bias than HARD (0.767 vs. 0.656; wrong-aligned 0.140 vs. 0.195).On HARD, the intervention amplified a noisier signal, which can partially explain its resistance to rescue.Source: Table 5 · p. 8, §5.3 · p. 8
  • POS shifted label_gap strongly upward and NEG strongly downward, while HARD moved upward less.POS means ZS incorrect and SIAB correct; NEG means ZS correct and SIAB incorrect; HARD means both predictions are incorrect.Source: §5 · p. 6, Table 3 · p. 7, §5.3 · p. 8
  • Doubling β rescued 4.5% of HARD samples on Gemma3, 0.1% on LLaMA3, and 0.4% on Qwen3; sink-token suppression did better, with a 5.5% gain on LLaMA3.The effect of increasing β saturated beyond ×2; suppression gains remained modest overall.Source: Table 6 · p. 8, §5.3 · p. 8

5.4 Layer-band Proximity in SIAB–PASTA Additivity

  • 108.0% F1 additivity followed moving LLaMA3 PASTA heads near SIAB; forcing Gemma3 and Qwen3 heads farther away reduced additivity to 79.4% and 70.0%, from 95.8% and 118.5%.The manipulated results tracked layer gap across all three models.Source: Table 7 · p. 9, §5.4 · p. 9

6 Conclusion

The conclusion presents SIAB as a way to route affective evidence already present in dialogue, without externally marked spans or task-specific training. It reports transfer of a fixed head set across two contrasting attentional demands and points to sample-adaptive control as future work.

6 Conclusion

  • A fixed causally localized head set transferred across two contrasting attentional demands, without externally marked spans or task-specific training.The conclusion contrasts dispersed cues across turns with dense local cues among fine-grained categories.Source: §6 · p. 9
  • PASTA-style external profiles were competitive in-domain but transferred poorly, while combined gains suggested internal and external signals captured complementary aspects of affective evidence.Source: §6 · p. 9
  • Future work should test sample-adaptive intervention strength and target tokens, and whether the interface extends to uncertainty, trust, or intent.Source: §6 · p. 9

Limitations

The paper bounds its evidence to small English-language emotion benchmarks and leaves transfer to larger models, other languages, and non-emotion tasks unresolved. It constructs synthetic minimal pairs across emotion categories and cue positions, with strong results on labels unseen during localization.

Limitations

  • The evaluation covers 4–8B models; whether the recovered signal generalizes to 70B+ models remains open because exhaustive activation patching is computationally prohibitive.The stated model families are Gemma3, Qwen3, and LLaMA3.Source: Limitations · p. 9
  • Both evaluated benchmarks are English-only, leaving cross-language transfer of the recovered head set and possible language-specific circuits untested.Source: Limitations · p. 9
  • The two benchmarks reuse one head set and span complementary emotion-recognition demands, but generalization to non-emotion classification remains future work.Source: Limitations · p. 9
  • Some HARD samples remain unrescued even after stronger interventions; whether sample-adaptive intervention can rescue them is left open.Scaling β and suppressing attention sink were insufficient.Source: Limitations · p. 9

A Pair Construction for Causal Patching

B Implementation Details for Refinement and Intervention

Implementation details specify the hyperparameter inventory, model-specific head sets and intervention strengths, and the token filter used to refine candidate heads.

B Implementation Details for Refinement and Intervention

C Hyperparameters and Calibration

Calibration uses development data and freezes most SIAB settings across evaluation configurations; GoEmotions decoding parameters are tuned separately under the baseline and then held fixed for SIAB.

C Hyperparameters and Calibration

  • The GoEmotions multi-label decoding parameters k and t are listed per model and shot configuration.Source: C Hyperparameters and Calibration · p. 12
  • ECC calibration uses 200 dialogues from its official development split, while GoEmotions uses 200 development samples and evaluates on its test split excluding Neutral-only samples.Source: C Hyperparameters and Calibration · p. 12
  • GoEmotions decoding parameters T and t are tuned on development data under the baseline and frozen for +SIAB, so ΔF1 reflects intervention rather than recalibration.Source: Table 10 · p. 12
  • The structural hyperparameters and each model’s β are calibrated once on ECC development data and held fixed across datasets and shot settings.No hyperparameter is tuned on test data.Source: C Hyperparameters and Calibration · p. 12

D Results on Neutral-only GoEmotions Samples

SIAB improves Neutral recall on the excluded Neutral-only GoEmotions subset. The accompanying attention visualization contrasts selected and random heads, while prompts differ between zero-shot and few-shot settings.

E Qualitative Attention Visualization

D Results on Neutral-only GoEmotions Samples

  • Neutral recall rose by +1.08 to +1.93 percentage points on 1,606 Neutral-only test samples across Gemma3, Qwen3, and LLaMA3.The reported model pairs are Gemma3 9.10 → 10.54, Qwen3 10.05 → 11.13, and LLaMA3 8.93 → 10.86.Source: D Results on Neutral-only GoEmotions… · p. 13

F Prompt

  • Zero-shot uses multi-turn ECC chats but single-message GoEmotions inputs; few-shot prepends fixed class demonstrations in an utterance/cues/reasoning/emotion format.Source: F Prompt · p. 13

G Bootstrap

Paired bootstrap intervals assess uncertainty in SIAB and PASTA comparisons. All prompt-matched SIAB settings and all PASTA+SIAB comparisons had positive intervals, while stand-alone PASTA gains were less consistent.

G Bootstrap

  • Each contrast used 10,000 paired resamples with identical example indices, fixed outputs and settings, and pointwise 95% percentile confidence intervals.For GoEmotions, temperature and threshold were tuned on the development set for each baseline and frozen for its SIAB counterpart.Source: G Bootstrap · pp. 13–14
  • All 18 prompt-matched SIAB settings had positive differences on both metrics, with all 36 metric-level confidence intervals excluding zero across GoEmotions and ECC.The comparisons are prompt-matched; Tables 11 and 12 report the GoEmotions and ECC intervals, respectively.Source: Table 11 · p. 14, Table 12 · p. 14, G Bootstrap · p. 14
  • All 12 PASTA+SIAB-versus-ZS metric-level intervals were strictly positive, while several stand-alone PASTA intervals included zero.The smallest positive lower bounds occurred under few-shot prompting, but remained above zero.Source: G Bootstrap · p. 14
  • Transferring ECC-profiled PASTA heads to GoEmotions produced the largest degradations, particularly for LLaMA3; its in-domain PASTA-GOE configuration also decreased Top-1 accuracy.The paired-bootstrap results nevertheless supported SIAB consistency across models, datasets, prompting regimes, and metrics.Source: G Bootstrap · pp. 14–15
Loading 2610.21347v1…