Source-linked AI summary
Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning
Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun
TL;DR
The paper addresses the under-explored mechanism by which LLMs learn from demonstrations during ICL. Using an information-flow analysis, it finds that label words act as anchors, supporting accuracy improvement, faster inference, and error diagnosis.
Problem
The underlying mechanism through which LLMs learn from provided context during ICL remains under-explored.
Method
The paper analyzes information flow and tests a label-word-as-anchor hypothesis with saliency metrics, attention manipulation, and prediction-correlation analysis across GPT2-XL and GPT-J.
Results
Label words gather semantic information in shallow layers and serve as references for final predictions in deep layers; anchor re-weighting improves average accuracy by 16.7%.
Takeaways & Limitations
The anchor perspective supports demonstration re-weighting, compressed ICL inference, and an error-analysis framework for GPT2-XL.
Takeaways & Limitations
The study covers classification tasks and conventional ICL, examines models up to 6 billion parameters, and does not evaluate generative tasks, CoT, or larger models.
Abstract
from arXiv · showhide
In-context learning (ICL) emerges as a promising capability of large language models (LLMs) by providing them with demonstration examples to perform diverse tasks. However, the underlying mechanism of how LLMs learn from the provided context remains under-explored. In this paper, we investigate the working mechanism of ICL through an information flow lens. Our findings reveal that label words in the demonstration examples function as anchors: (1) semantic information aggregates into label word representations during the shallow computation layers' processing; (2) the consolidated information in label words serves as a reference for LLMs' final predictions. Based on these insights, we introduce an anchor re-weighting method to improve ICL performance, a demonstration compression technique to expedite inference, and an analysis framework for diagnosing ICL errors in GPT2-XL. The promising applications of our findings again validate the uncovered ICL working mechanism and pave the way for future studies.
1 Introduction
This paper investigates ICL through an information-flow perspective and proposes that demonstration label words act as anchors. It validates this mechanism experimentally and derives methods for improving accuracy, accelerating inference, and diagnosing errors.
- Motivation: ICL’s underlying working mechanism remains an open question despite enabling LLMs to perform diverse tasks from few-shot demonstrations.The paper motivates studying how models learn from their supplied context without parameter updates.
- Method: Saliency visualization and metrics are used to characterize information flow among demonstration text, label words, and the target position.The approach analyzes attention interaction patterns and distinguishes shallow aggregation from deep prediction-related flows.
- Information Flow with Labels as Anchors: Label words aggregate demonstration information in shallow layers and provide information for final prediction in deep layers.The hypothesis describes shallow-layer semantic representation formation followed by deep-layer extraction for prediction.
- Experiments: Experiments with GPT2-XL and GPT-J across text-classification benchmarks validate the proposed information-flow hypothesis.Blocking shallow-layer aggregation paths significantly impairs performance, indicating that label words collect useful information during forward propagation.
- Applications: Anchor re-weighting improves average accuracy by 16.7% over standard ICL baselines, while demonstration compression achieves a 1.8 × inference speedup with minimal performance trade-off.The paper also presents an error-analysis framework connecting label confusion with distances between anchor key vectors.
2 Label Words are Anchors
The paper proposes that label words act as anchors in ICL: shallow layers aggregate demonstration information into label-word representations, while deeper layers extract that information for prediction. Saliency metrics and intervention experiments support this information-flow account across GPT models and classification tasks.
- 2.1 Hypothesis Motivated by Saliency Scores: The saliency matrix I_l measures the significance of information flow between input words, revealing increasing label-word dominance as layers deepen.It is computed by averaging attention-head saliency matrices derived from attention and loss gradients.
- 2.1 Hypothesis Motivated by Saliency Scores: The metrics Swp, Spq, and Sww separately quantify text-to-label, label-to-target, and other word-to-word information flow.Swp measures aggregation onto label words, while Spq measures extraction from label words toward the target position.
- 2.2 Shallow Layers: Information Aggregation: Across layers, Swp is initially substantial and decays, whereas Spq becomes dominant; label-word interactions generally exceed other word interactions.This pattern is reported on SST-2 and AGNews, with comparable findings described across the analyzed settings.
- 2.1 Hypothesis Motivated by Saliency Scores: Label words gather information from demonstrations in shallow layers and provide semantic representations for later computations and final prediction.The hypothesis distinguishes shallow-layer aggregation from deep-layer extraction.
- 2.2 Shallow Layers: Information Aggregation: Isolating label words in the first five layers substantially affects behavior, unlike isolation in the last five layers or of random non-label words.The intervention supports a specific role for shallow-layer aggregation through label words.
- 2.3 Deep Layers: Information Extraction: In deep layers, target-position attention over label words strongly correlates with final predictions, while cumulative contribution rises mainly in middle and deep layers.The reported AUCROC for deep layers approaches 0.8, and shallow layers show negligible cumulative contributions.
3 Applications of Our Anchor-Based Understanding
The anchor-based view motivates three applications: re-weighting label anchors for accuracy, compressing demonstrations into hidden states for faster inference, and using anchor distances to analyze ICL errors.
- 3.1 Anchor Re-weighting: Anchor re-weighting adjusts label-word contributions with a learnable vector, achieving a 16.7% average accuracy boost over standard ICL baselines.The method uses an auxiliary training set and is evaluated with GPT2-XL; its parameters are trained by gradient descent.
- 3.1 Anchor Re-weighting: The method improves performance across tasks while avoiding the extra inference cost of adding auxiliary examples as demonstrations.The authors report particularly strong gains on SST-2 and EmoC and attribute the shorter context to nearly unchanged inference speed.
- 3.2 Context Compression: Hiddenanchor compresses demonstrations into formatting and label-word hidden states, preserving aggregated information while reducing the input context.Label-word states can be cached because their aggregation depends only on preceding tokens, but formatting states are also needed to determine the output space.
- 3.2 Context Compression: Hiddenanchor performs best among the compression methods on all metrics for both GPT2-XL and GPT-J.The comparison includes Textanchor, Hiddenrandom, and Hiddenrandom-top, evaluated using accuracy, label loyalty, and word loyalty.
- 3.2 Context Compression: Hiddenanchor yields speed-up ratios from 1.1× to 2.9×, with larger acceleration reported for GPT-J than GPT2-XL.For GPT-J, anchor-state compression produces a 1.5 accuracy-point drop relative to the uncompressed setting.
- 3.3 Error Analysis: Anchor distances support an error-analysis framework because confusing categories tend to have similar label-anchor representations.Confusionpred identifies Description-Entity as the most confusing case and performs reasonably on Entity-Abbreviation and Description-Abbreviation.
4 Related Work
Prior ICL analysis studies either how demonstration inputs influence performance or how ICL works through conceptual mechanisms. This paper instead examines information flow to improve ICL’s effectiveness, efficiency, and interpretability.
- Input and Demonstration Factors: One research stream studies input factors such as demonstration order, formatting, and example selection, along with construction and calibration strategies.These approaches target performance changes caused by perturbing or designing the demonstrations.
- Mechanistic Perspectives: A second stream explains ICL through conceptual analogies, including gradient descent and Bayesian inference.The paper positions its information-flow perspective as distinct from these existing lenses.
- This Paper’s Perspective: The paper investigates information flow in language models as a new perspective for understanding ICL and improving its effectiveness, efficiency, and interpretability.This framing connects mechanism analysis with practical ICL enhancements.
5 Conclusion
The paper frames label words as anchors that aggregate and distribute task-relevant information in ICL, then develops applications for accuracy, efficiency, and error diagnosis. Its scope is limited to classification tasks, conventional ICL, and models up to 6 billion parameters.
- Label words serve as anchors that aggregate and distribute task-relevant information in ICL.
- The paper proposes anchor re-weighting, demonstration compression, and an analysis framework for diagnosing ICL errors.
- The study covers classification tasks and conventional ICL, leaving generative tasks and paradigms such as chain-of-thought prompting unexplored.
- The experiments mainly investigate models up to 6 billion parameters because of hardware constraints.
A Experimental Settings
The experiments use GPT2-XL and GPT-J with four classification datasets spanning sentiment, question type, topic, and emotion classification.
- The study uses GPT2-XL with 1.5B parameters and GPT-J with 6B parameters.
- The datasets are SST-2 for sentiment analysis, TREC for question type classification, AGNews for topic classification, and EmoContext for emotion classification.
B Results of Swp, Spq, and Sww on TREC and EmoC
Information flow involving label words dominates these classification settings: text-to-label flow is stronger in shallow layers, while label-to-target flow dominates deeper layers.
- In shallow layers, Swp, the text-to-label information flow, is prominent while Spq, the label-to-target flow, is less significant.
- In deeper layers, Spq dominates the information flow.
- Swp and Spq generally exceed Sww, indicating that interactions involving label words predominate.
- Word loyalty is used alongside label loyalty because label loyalty alone may miss changes in non-label-word probabilities and label-word probabilities relative to the vocabulary.
D Isolating Different Numbers of Layers
Layer-isolation experiments show that shallow-layer information aggregation through label words is important, while isolating deep layers has little effect; results remain similar with two demonstrations per class.
- D Isolating Different Numbers of Layers: Isolating shallow layers causes a significant impact, whereas isolating deep layers has negligible impact even as the number of isolated layers increases.
- D Isolating Different Numbers of Layers: Figure 8 compares label and word loyalty when label or non-label words are isolated in the first several versus last layers.
- D Isolating Different Numbers of Layers: These results further illustrate the important role of information aggregation via label words in shallow layers.
- D Isolating Different Numbers of Layers: The analysis uses attention distributions on label words at the target position as a classifier prediction and quantifies its correlation with model outputs using AUC-ROC.
- D Isolating Different Numbers of Layers: With two demonstrations per class, the results for SST-2, TREC, AGNews, and EmoC are highly similar to results using one demonstration per class.
F.2 Results for In-Context Learning with semantically-unrelated labels
Experiments with semantically unrelated labels reproduce the paper’s information-flow findings: shallow-layer label isolation has the strongest effect, while deeper-layer anchors support classification.
- Using LLaMA-33B on SST-2, the experiment replaces labels with “A”/“B” and uses eight shots per class, achieving 83.0% accuracy.
- Isolating label words in the first five layers has a stronger effect than isolating them in the last five layers or isolating non-label words.
- The model leverages information from anchors in deeper layers to perform classification.
G Implementation of Anchor Re-weighting
Anchor re-weighting modifies attention computations by adjusting attention involving selected label-word positions; the section also contrasts shallow-layer isolation with other layer interventions.
- Anchor re-weighting modifies each attention head’s attention matrix before subsequent computations.
- The modification applies a learned weighting to attention from the target query position to selected demonstration label-word positions, while leaving other entries unchanged.
- Isolating label words within the first five layers has a more pronounced effect than isolating them within the last five layers.
H Training Settings of Anchor Re-weighting
The experiments specify optimization, reproducibility, compression, and confusion-analysis settings, including GPT2-XL-only re-weighting evaluation and M = 10 for predicted confusion calculations.
- Anchor re-weighting optimization uses Adam with learning rate 0.01, β1 = 0.9, β2 = 0.999, batch size 1, and 10 epochs.
- Because of computational-resource limitations, the anchor re-weighting evaluation is restricted to GPT2-XL and excludes GPT-J.
- Acceleration ratios correlate with the ratio of total demonstration length Ldemo to predicted-text length Lx.
- Datasets with longer demonstrations tend to have higher acceleration ratios; AGNews has the longest Ldemo and the highest acceleration ratio among the analyzed datasets.
- For predicted confusion, query vectors are centered, their largest variation directions are selected, and key vectors are projected onto those directions.
- The analysis uses M = 10 because predicted confusion matrices show similar outcomes across tested M values.
- Confusionij is computed with AUC-ROC and approaches 1 when confusion between categories is less pronounced.
- Five random seeds—42, 43, 44, 45, and 46—are fixed to support reproducibility across devices.