Source-linked AI summary
Cross-lingual Functional Vectors for Emotion Detection in Large Language Models
Jieying Xue, Phuong Minh Nguyen, Minh Le Nguyen, Shogo Okada
TL;DR
Function vectors have been effective in structured in-context settings, but their performance on semantically complex tasks and transfer across languages remain underexplored. This paper studies cross-lingual FVs on multilingual multi-label emotion recognition, evaluating source-language vectors for target-language steering in clean and perturbed zero-shot settings. The experiments find effective cross-lingual transfer, stable attention-head ranges across languages, and partial replication of few-shot steering without processing additional demonstrations during inference.
Problem
Whether function vectors remain effective for semantically complex multilingual tasks and encode transferable task-level representations rather than language-specific behavior remains underexplored.
Method
The study evaluates source-language function vectors on multilingual multi-label emotion recognition under clean and perturbed zero-shot prompting, and examines few-shot prompting, layer intervention, and attention-head selection.
Results
Function vectors effectively steer task behavior across languages, improve multilingual few-shot performance, partially reproduce few-shot steering without additional demonstrations, and show stable effective attention-head ranges across languages.
Takeaways & Limitations
Function vectors provide a lightweight, transferable, and scalable mechanism for multilingual task adaptation when demonstrations during inference are limited or unavailable.
Takeaways & Limitations
The study does not examine label-dependency modeling and leaves generalization to other multilingual reasoning or generation tasks unclear, with possible over-steering risks for long outputs.
Abstract
from arXiv · showhide
Function vectors (FVs) have recently emerged as a promising mechanism for steering the behavior of large language models (LLMs) by injecting task-specific latent direction representations derived from in-context demonstrations. While prior studies have shown that FVs can recover task behavior in structured in-context learning settings, their effectiveness on semantically complex tasks and their ability to generalize across languages remain underexplored. We investigate the cross-lingual transferability of FVs using multilingual multi-label emotion recognition as a challenging semantic classification benchmark. Specifically, we examine whether FVs extracted from a source language can steer task behavior in another language under both standard clean and perturbed zero-shot settings without providing demonstrations during inference. Across diverse cross-lingual settings, applying FVs substantially improves performance, suggesting that FVs capture language-agnostic, task-relevant signals rather than purely language-specific lexical patterns, and highlighting their potential as a lightweight and transferable mechanism for multilingual task adaptation. We observe that each LLM exhibits a relatively stable optimal range of attention heads for constructing effective FVs, and the pattern remains consistent across languages. In addition, FVs can partially replicate the task-steering effects of standard few-shot in-context learning while avoiding the computational overhead of processing multiple demonstrations, making them effective for large-scale practical applications. Our code is available at https://github.com/yingjie7/cross_lingual_fvs.
1 Introduction
This study extends function-vector evaluation to semantically complex multilingual emotion recognition and tests whether vectors transfer task behavior across languages. It finds effective zero-shot steering, complementary gains with few-shot prompting, and stable attention-head selection across languages.
- Motivation and scope: Function vectors are evaluated on multilingual multi-label emotion recognition, extending prior work beyond constrained tasks to contextual semantic reasoning.The study investigates whether FVs remain effective when tasks require richer contextual understanding and reasoning.
- Intervention strategy: Distributing FV interventions across multiple transformer layers substantially outperforms single-layer intervention on complex semantic tasks.This provides a practical strategy for applying FVs to semantically demanding settings.
- Cross-lingual transfer: Cross-lingual FVs transfer task behavior across languages under both clean and controlled perturbed zero-shot prompting.The perturbed setting introduces distractor labels to test whether FVs recover intended behavior despite misleading context.
- Attention-head selection: Each LLM exhibits a stable range of attention heads for constructing effective FVs, with the pattern remaining consistent across languages.The finding supports stable head-selection behavior in cross-lingual FV construction.
- Few-shot prompting: FVs consistently improve multilingual few-shot performance while partially reproducing few-shot task steering without additional inference demonstrations.They complement rather than completely replace standard few-shot in-context learning.
2 Related Work
Prior work frames in-context learning as involving latent task representations and studies function vectors as internal mechanisms for steering model behavior. However, their effectiveness for multilingual semantic classification requiring richer contextual reasoning remains underexplored.
- Function vectors and mechanistic interpretability: In-context learning research suggests task behavior is partly encoded in internal activation patterns and latent task representations derived from demonstrations.Information-flow studies also examine how task-relevant information propagates through models.
- Research gap: Function-vector evidence has largely focused on constrained tasks, leaving multilingual semantic classification with richer contextual reasoning insufficiently explored.The open question concerns both contextual reasoning and cross-lingual semantic understanding.
- Multilingual representations: Multilingual LLMs are reported to learn partially shared latent representations across languages while preserving language-specific characteristics.This motivates studying how task-relevant representations transfer across languages.
3 Method
The method evaluates whether function vectors (FVs) transfer task representations across languages and complement few-shot in-context learning. It extracts task-relevant activations from selected attention heads and injects the resulting vector into the model’s residual stream.
- Experimental Settings: Cross-lingual FV transfer is evaluated in zero-shot and few-shot settings, including clean and perturbed prompts without target-language demonstrations.The study compares zero-shot prompting with source-language FV injection and few-shot prompting with and without FV injection.
- Problem Formalization: The formalization distinguishes demonstration language g1 from query language g2, with FV extraction using g1 and zero-shot inference using only g2.When demonstrations and queries share a language, g1 = g2; cross-lingual transfer uses different languages.
- FV Extraction: For emotion detection, the method aggregates mean activations from selected heads into a multi-label emotion-detection FV.The FV is computed using clean prompts reconstructed separately for each language.
- FV Extraction: FV heads are selected by ranking attention heads according to average indirect effect after activation patching between clean and corrupted prompts.Corrupted prompts are created by randomly shuffling labels, and the top-k heads are retained.
- FV Injection: Extracted FVs are injected directly into the LLM residual stream at a layer chosen using the corresponding attention-head causal-effect scores.The residual-stream update adds the FV projection alongside the MLP and attention projections.
- Experimental Settings: Clean prompts provide task instructions and candidate emotion labels, whereas perturbed prompts add distractor labels to test task steering under misleading context.The perturbation includes auxiliary labels such as colors and nations.
4 Experimental Setup
The experiments evaluate multilingual multi-label emotion classification across five typologically diverse languages using a standardized macroaveraged F1 protocol and two open-source LLMs. Results are averaged over five random seeds, with distinct decoding settings for zero-shot and few-shot evaluation.
- Dataset: The dataset covers sentence-level emotion annotations in 28 languages, with experiments selecting English, German, Chinese, Spanish, and Russian.Each instance may have zero or more labels among joy, sadness, fear, anger, surprise, and disgust.
- Dataset: The task is multilingual multi-label emotion classification over six emotion categories.
- Evaluation: Performance is measured using the macroaveraged F1 score across emotion classes.The metric averages class-level F1 scores, with C denoting the number of classes and F1c the F1 score for class c.
- Experimental Environments: Experiments use Qwen3-8B and Llama-3.1-8B-Instruct, with reported results averaged as mean±std over five random seeds.
- Experimental Environments: Zero-shot evaluation uses the first generated token with greedy decoding, whereas few-shot evaluation permits up to 10 new tokens.
5 Results
Experiments show that function vectors improve multilingual emotion recognition in zero-shot and few-shot settings, transfer across languages, and benefit from stable, distributed intervention configurations.
- 5.1 Effectiveness of FVs – Zero-shot Setting: FV injection consistently improves performance over standard clean zero-shot baselines across evaluated languages and both LLMs.Because clean prompts already specify the task, the gains indicate additional task-relevant representations beyond prompt instructions.
- 5.1 Effectiveness of FVs – Zero-shot Setting: In perturbed zero-shot prompts, FV injection substantially improves performance when weakened task cues otherwise produce low Macro-F1 scores and near-random predictions.Cross-lingual FVs guide predictions toward correct emotion labels instead of irrelevant label spaces.
- 5.2 Effectiveness of FVs – Few-shot Setting: FV-enhanced few-shot prompting consistently outperforms standard few-shot prompting across monolingual and cross-lingual settings.Some cross-lingual settings, including Spanish-to-Russian transfer, outperform corresponding monolingual few-shot baselines.
- 5.3.1 Cross-lingual Alignment of FVs: FV similarities across source languages range from 0.94 to 0.96 for Qwen3-8B and 0.85 to 0.93 for Llama-3.1-8B.The high cosine similarities support shared functional directions across languages rather than language-specific vectors.
- 5.3.1 Cross-lingual Alignment of FVs: FV injection remains effective from 5-shot to 9-shot construction, with small performance variation across shot counts.The results indicate robustness to the number of demonstrations used to construct FVs.
- 5.3.2 Intervention Configuration: Each LLM has a model-specific but cross-lingually stable optimal FV configuration, while distributing interventions across multiple layers substantially outperforms single-layer intervention.The analysis also finds consistent gains across a broad range of top-k head selections in FV-enhanced few-shot prompting.
- 5 Results: Confusion-matrix comparisons show improved prediction behavior across emotion categories and languages, producing more balanced and reliable recognition performance.This complements the aggregate Macro-F1 findings with category-level prediction analysis.
6 Conclusion
The paper finds that function vectors transfer task behavior across languages for multilingual multi-label emotion recognition, including zero-shot steering without inference-time demonstrations. They also partially reproduce few-shot task-steering effects while offering a more efficient adaptation mechanism.
- 6 Conclusion: Function vectors exhibit strong cross-lingual generalization, enabling zero-shot task steering across languages without in-context demonstrations during inference.The result is demonstrated on multilingual multi-label emotion recognition.
- 6 Conclusion: Cross-lingual FVs partially replicate standard few-shot ICL task-steering effects while avoiding the computational overhead of processing multiple demonstrations.The paper characterizes FVs as a lightweight, transferable, and scalable mechanism for multilingual task adaptation.
Limitations
The study limits its scope to task-level cross-lingual emotion classification and does not model dependencies among co-occurring labels. It also leaves transfer to other multilingual reasoning or generation tasks unresolved, with potential over-steering risks for long outputs.
- The study focuses on task-level cross-lingual transfer rather than modeling correlations among co-occurring emotion labels.
- The evaluation centers on multilingual emotion recognition as a classification or multi-label classification task.
- Whether the observed transferability generalizes to multilingual reasoning or generation tasks remains unclear.The paper specifically mentions machine translation as an example of an untested generation task.
- The authors note that over-steering throughout long generation sequences could potentially degrade generation quality.
C Cross-lingual Zero-shot Emotion Classification with Llama
The paper evaluates Llama-3.1-8B-Instruct on cross-lingual emotion classification under perturbed and standard clean zero-shot prompts, using function vectors extracted from random five-shot demonstrations.
- Perturbed zero-shot setting: Table 11 evaluates perturbed zero-shot cross-lingual emotion classification with Macro-F1 scores for Llama-3.1-8B-Instruct.Function vectors are extracted from random 5-shot demonstrations using the top-k heads with |A| = 5.
- Comparison: Both evaluations report Macro-F1 scores (%) for the same cross-lingual emotion classification task.The shared setup enables comparison between perturbed and standard clean zero-shot prompts.
- Clean zero-shot setting: Table 12 evaluates standard clean zero-shot cross-lingual emotion classification with Macro-F1 scores for Llama-3.1-8B-Instruct.Function vectors are extracted from random 5-shot demonstrations using the top-k heads with |A| = 5.
E Additional Cross-lingual Few-shot Results
Additional experiments examine how selecting different numbers of attention heads affects function-vector-enhanced prompting in Llama-3.1-8B-Instruct and Qwen3-8B.
- Head selection: Figure 5 examines the effect of top-k attention heads used to construct function vectors under monolingual and cross-lingual settings in Llama-3.1-8B-Instruct.The figure focuses on how the number of selected heads relates to function-vector construction across language settings.
- Few-shot comparison: Figure 6 plots classification performance against the number of selected FV heads for FV-enhanced few-shot prompting on Qwen3-8B.The dashed horizontal line marks the corresponding standard few-shot prompting baseline without FV injection.
- Language settings: Figure 6 distinguishes Fs Lang, the language used for FV extraction and few-shot demonstrations, from Test Lang, used in the evaluation setup.The caption identifies these language roles while describing the head-selection comparison.
F Mechanism Analysis
The mechanism analysis compares function-vector construction across multiple source languages using Llama-3.1-8B-Instruct.
- Cross-language similarity: Figure 7 reports pairwise similarity scores for function-vector construction across English, German, Chinese, Spanish, and Russian source languages.The language abbreviations are vkg for g ∈ {eng, deu, chn, esp, rus}.
- Experimental setup: The similarity analysis uses Llama-3.1-8B-Instruct with 5-shot demonstrations and five selected heads.The caption specifies the experimental configuration as head = 5.
- Cross-language similarity: The figure enables comparison of function-vector construction similarity among the five listed source languages.The comparison is organized pairwise across source-language constructions.