PaperlayerView exact revision on arXiv

Source-linked AI summary

From Relation-Specific Heads to Shared Interventions: Contextual Entrainment in Language Models

Jinghan Zheng, Jiajun Shi

arXiv:2610.18674v1cs.CLcs.AI

TL;DR

The paper asks whether contextual-entrainment interventions are relation-specific or recover components shared across relations. It searches and tests head interventions using factual-distractor prompts across lexical-semantic relations, finding a reusable global component whose effects are shaped by semantic structure. The authors suggest heterogeneous attack data as a starting point for mitigation, while noting important limits of the controlled benchmark and case study.

  • Problem

    It remains unclear whether relation-conditioned searches identify isolated contextual-entrainment mechanisms or overlapping shared components, a distinction relevant to interpreting and mitigating context sensitivity.

    Source: §1 · pp. 1–2

  • Method

    The paper searches for attention-head interventions using factual-distractor relation-completion prompts and tests their transfer across subtypes, semantic families, and mixed-relation conditions.

    Source: §1 · p. 2

  • Results

    Relation-conditioned searches repeatedly recover a strong relation-general component, while semantic-family and subtype structure modulate its effects rather than replacing it with separate circuits.

    Source: §6 · p. 8

  • Takeaways & Limitations

    For mitigation, the authors suggest starting with heterogeneous attack data to identify reusable interventions and using relation-specific searches for finer specialization.

    Source: §7 · p. 9, §6 · p. 9

    The primary case study uses LLaMA-3.1-8B and a controlled single-token relation-completion benchmark; conclusions also condition on valid attacked cases and do not characterize context uses that help answer correctly.

    Source: §8 · p. 9

Abstract

from arXiv · show

Large language models rely on prompt context to perform in-context learning, follow formats, and adapt to local evidence. This ability, however, also raises a robustness concern: models may reuse irrelevant tokens from earlier context when answering a later query. Prior work identifies this behavior as contextual entrainment and finds attention heads whose ablation reduces it, arguing that these heads are largely task- or relation-specific. But are the mechanisms behind contextual entrainment truly isolated by relation, or do relation-conditioned searches recover reusable components shared across relations? Using factual-distractor relation-completion prompts across 19 lexical-semantic subtypes, we identify attention heads whose ablation reduces entrainment and test how the resulting head sets transfer recovery ability across exact relations, semantic families, and mixed relation distributions. Same-family transfer exceeds different-family transfer, family residuals weaken once globally shared heads are removed, and subtype-specific searches repeatedly rediscover many global heads. Clean-completion and context-use probes further suggest that this shared intervention can reduce harmful copying while preserving ordinary relation completion and simple in-context abilities. Our findings provide a more precise perspective on contextual entrainment and its mitigation: harmful context following is not best addressed by searching for isolated relation-specific circuits, but by using heterogeneous attack prompts to reveal reusable intervention targets whose effects are shaped by semantic relation structure.

1 Introduction

The paper asks whether contextual-entrainment head sets are isolated by relation or contain reusable shared components. It tests this question with factual-distractor relation-completion prompts and examines how the resulting interventions transfer.

1 Introduction

  • Irrelevant context can shape factual predictions in heterogeneous-input settings, and the paper studies a token-level case where a true fact for another subject misleads a parallel query.The earlier answer token becomes misleading evidence for the later query.Source: §1 · p. 1
  • Earlier work identified heads whose ablation reduces contextual entrainment and argued that the recovered sets are largely task- or relation-specific, but their overlap remained unexplained.The paper positions its question as a head-level complement to Cheng et al. (2025).Source: §1 · p. 2
  • The hypothesis predicts strongest transfer within the same subtype, weaker positive transfer within a semantic family, and weakest transfer across unrelated families.Purely relation-specific heads would transfer little beyond their subtype, while fully general heads would show no semantic-relatedness effect.Source: §1 · p. 2
  • The benchmark pairs clean relation-completion prompts with attacked prompts prepending a true but irrelevant parallel fact, then tests transfer across subtype, family, and mixed-relation conditions.This setup isolates prompt-local token reuse without adding an explicit false statement about the queried subject.Source: §1 · p. 2
  • Relation-conditioned searches repeatedly recover shared components, while semantic-family and subtype structure shape how those components are used.The paper frames this as a mixed-circuit account rather than isolated relation-specific modules.Source: §1 · pp. 2–3
  • Mixed-relation search reduces factual-distractor entrainment while better preserving clean external relation completion than the evaluated subtype-specific intervention.This comparison is reported for the primary-model experiments.Source: §1 · p. 3

2 Related Work

Prior work links contextual entrainment to earlier-token reuse and identifies heads whose ablation reduces it. Related mechanistic research motivates testing whether such effects reflect shared, distributed structure rather than isolated circuits.

Distraction and contextual entrainment

  • Niu et al. (2025) formalize contextual entrainment as increased likelihood of generating earlier prompt tokens, including when random or semantically irrelevant, and identify heads whose ablation reduces it.They emphasize relation specificity while also observing some cross-relation functional overlap.Source: §2 · p. 3

Attention heads as context mechanisms

  • Induction heads reuse continuations from earlier contexts at later matching positions, while knowledge-conflict work finds head-level interventions can generalize across relations and prompt formats (Jin et al., 2024).These findings make a reusable context-following component plausible across differing surface relations.Source: §2 · p. 3

Circuit discovery by differentiable masking

  • Differentiable masking methods, including the Gumbel-Sigmoid head-search framework used by Niu et al. (2025), enable causal searches for whether relation-specific findings recover shared structure.Yu et al. (2025) use Gumbel-Sigmoid masking for gradient-based joint computation-graph pruning.Source: §2 · p. 3

Shared and distributed mechanisms

3 Methodology

The methodology defines a factual-distractor relation-completion task, searches for head interventions, and evaluates their recovery and transfer across a 19-subtype lexical-semantic benchmark.

3.1 Task Definition

  • Attacked prompts prepend a structurally parallel factual statement about another subject, placing its answer token in context to compete with the query’s gold answer.The factual distractor is true for another subject, isolating prompt-local token reuse from explicit memory–context conflict; this makes the attack weaker but cleaner.Source: §3.1 · pp. 3–4

3.2 Gumbel-Sigmoid Head Search

  • A differentiable head search learns binary masks with a Gumbel-Sigmoid straight-through relaxation, then ranks heads for top-K interventions.The search maximizes recovery of the gold-over-distractor margin while encouraging sparsity; K = 80 was chosen by a held-out sweep balancing attack recovery and clean-prompt harm.Source: §3.2 · p. 4

3.5 Transfer and Circuit Conditions

  • Family-residual sets retain the top-K heads from a family ranking after excluding global-shared heads, without retraining.The methodology also reports normalized recovery centered within each target to compare source circuits against the average source on that target.Source: §3.2 · p. 4, §3.3 · p. 5

3.3 Evaluation Metrics

  • Recovery is measured using the gold-versus-distractor logit margin and attack-induced margin loss, with valid attacked cases requiring D ≥ 0.5 and a clean gold preference.Normalized recovery R = 1 indicates full recovery of the attack-induced loss; values above 1 are interpreted as intervention effects, not bounded percentages.Source: §3.3 · p. 4

3.4 Lexical-Semantic Benchmark

  • The benchmark contains 19 relation subtypes grouped into Association, Attribute, Part–Whole, and Taxonomy.The paper identifies taxonomy, part–whole, and attribute as standard lexical-semantic relations and Association as covering non-taxonomic factual and functional mappings.Source: §3.4 · p. 5

3.5 Transfer and Circuit Conditions

  • Transfer tests compare source and target circuits within the same subtype, within the same family, and across different families, alongside global, family, subtype, and family-residual sets.Frequency-global heads are defined by how often they are rediscovered across subtype-specific searches.Source: §3.5 · p. 5

4 Results

Relation-conditioned searches reveal a reusable global intervention component alongside family- and subtype-level variation. Transfer is strongest within the exact subtype, weaker within its semantic family, and weakest across families.

4.1 Transfer Is Graded by Semantic Relatedness

  • Target-centered recovery is −0.054 across different families, +0.069 within the same family, and +0.512 within the same subtype.The hierarchy places transfer between isolated subtype circuits and fully universal heads; it remains significant under clustered bootstrap, cell-level tests, and family-label permutation controls.Source: §4.1 · pp. 5–6, Figure 2 · p. 6

4.2 Family Effects Are Built on Global Shared Heads

  • Average recovery is 1.464 for global-shared circuits, 1.404 for subtype-specific circuits, and 1.306 for family-shared circuits.After removing global heads from family rankings, average family recovery falls from 1.306 to 0.959 but remains positive, showing that much of the family effect overlaps the global core.Source: Table 2 · p. 6, §4.2 · p. 6

4.3 Relation-Specific Searches Rediscover the Global Core

  • Global-shared circuits overlap subtype-specific circuits by 32.84 heads on average, and 75 of 80 global heads appear in at least one subtype-specific top-K circuit.At K = 80, frequency-defined global heads reach recovery 1.350 versus 1.464 for global-search heads, while rarely rediscovered heads reach 0.082.Source: Table 3 · p. 6, §4.3 · p. 7

4.4 The Mixed-Circuit Decomposition Predicts Intervention Effects

  • The global component has the largest fitted effect (βG = 2.936), while family and subtype residuals contribute positively (βF = 1.009, βS = 1.159).The additive model explains R2 = 0.572; adding interactions yields R2 = 0.578, with predictions correlated at r = 0.995. A matched 80-head G40/F20/S20 mixture has the strongest recovery.Source: §4.4 · p. 7

4.5 Global Circuits Preserve Clean Relation Completion

  • Global and family-shared circuits preserve external clean relation completion more closely than subtype-specific circuits, which cause larger collateral degradation.The authors suggest heterogeneous search favors heads that reduce entrainment across relation environments while better preserving clean completion.Source: Figure 3 · p. 7, §4.5 · p. 7

5 Robustness and Reproducibility

Additional checks support the stability and broader applicability of the main findings.

5 Robustness and Reproducibility

  • The graded transfer hierarchy survives threshold, template, and seed checks, and experiments replicate on Mistral-7B and Qwen2.5-7B.Random-head controls are near zero or negative, and global ablation does not notably damage simple few-shot spelling correction, translation, or in-context mapping.Source: §5 · p. 8

6 Discussion

Relation-conditioned head sets are not functionally isolated: they repeatedly contain a reusable global component, with family and subtype structure shaping its effects. Mixed-relation search also better preserves clean relation completion while reducing harmful context following.

From task-specific heads to mixed circuits

  • Relation-conditioned searches repeatedly recover a global component that transfers across semantic families, while family and subtype structure modulate its use.Different tasks need not select identical heads for their selected heads to share functional components.Source: §6 · p. 8

Why mixed search can reduce side effects

  • Mixed-relation search favors heads that consistently reduce distractor entrainment across relations while better preserving ordinary relation completion.Single-relation search can entangle distractor suppression with machinery for normal completion.Source: §6 · pp. 8–9

Implication for RAG and factual context use

  • In controlled relation-completion experiments, mixed-relation search identifies reusable interventions that reduce factual-distractor copying while largely preserving clean completion.The findings motivate future evaluation in RAG and long-context tasks, where contexts can mix useful and irrelevant true facts.Source: §6 · p. 9

7 Conclusion

Across lexical-semantic relations, contextual entrainment is neither fully relation-specific nor fully relation-general. The findings support shared interventions with relation-sensitive structure, and suggest beginning mitigation searches with heterogeneous attack data.

7 Conclusion

  • Context following is neither fully relation-specific nor fully relation-general: mixed circuits combine a reusable global component with family- and subtype-level variation.Apparent task specificity reflects overlapping shared and relation-sensitive components.Source: §7 · p. 9
  • The shared global component supports broad context following, while semantic-family and subtype structure shape its use in particular relations.Source: §7 · p. 9
  • For mitigation, the findings suggest starting with heterogeneous attack data to identify broad reusable interventions and using relation-specific searches for fine-grained specialization.Source: §7 · p. 9

8 Limitations

The study's conclusions are bounded by its primary model, controlled relation-completion prompts, single-token answers, and filtering to valid attacked cases. Supplemental benchmark and transfer analyses describe the operational groupings and test robustness across attack thresholds and inference procedures.

8 Limitations

  • The primary case study is LLaMA-3.1-8B, and controlled templates with single-token answers leave open whether mixed-circuit structure extends to natural long-context or retrieval settings.The authors also report that larger language models were not tested because of resource limitations.Source: §8 · p. 9
  • Conclusions are conditional on valid attacked cases and do not characterize context following when context helps the model answer correctly.Ablation identifies causally useful components but does not fully specify the internal algorithm for resolving gold-answer and distractor competition.Source: §8 · p. 9

A Benchmark Details

  • The benchmark uses 19 relation subtypes with single-token gold and distractor answers, and some family labels overlap at material relations.The authors retain material subtypes in both Part–Whole and Attribute families, describing this grouping as a conservative family-transfer test.Source: A Benchmark Details · p. 12

B Additional Transfer Statistics

  • Supplemental transfer analyses test nearby valid-attack thresholds and use target-centered recovery, clustered bootstrap inference, and random controls.The reported same-family advantage remains stable across tested attack-drop thresholds.Source: B Additional Transfer Statistics · p. 14

C Robustness and Cross-Model Replication

Robustness checks find positive but weaker cross-template recovery and a graded transfer pattern across seeds. Cross-model replications support the qualitative conclusion that a shared global core persists while subtype-specific residual structure remains comparatively weak.

Template robustness

  • Cross-template recovery is positive but weaker than same-template recovery across all four semantic families.Recovery drops from 1.671 to 1.488 for Association, 1.855 to 1.610 for Attribute, 0.801 to 0.690 for Part–Whole, and 1.721 to 0.968 for Taxonomy.Source: C Robustness and Cross-Model Replication · p. 15

Seed stability

  • Across three seeds, same-family transfer remains positive, same-subtype transfer is larger, and different-family transfer remains negative.The reported values are 0.071, 0.276, 0.259 for same-family; 0.510, 0.726, 0.631 for same-subtype; and −0.097, −0.167, −0.148 for different-family.Source: C Robustness and Cross-Model Replication · p. 15

Cross-model replication

D External Clean and Context-Use Probes

The global mixed-relation intervention preserves clean relation completion and simple few-shot spelling and translation, while harder synthetic mapping shows small accuracy drops at top-3 and top-10.

D External Clean and Context-Use Probes

E Activation Replacement Robustness

Activation replacement supports a reusable global intervention component, while additive and fixed-budget tests indicate that global heads dominate and residual components contribute smaller effects. The appendix also describes the illustrative intervention, search procedure, and evaluation resources.

E Activation Replacement Robustness

  • Global-shared heads recover 60.4% of attack-induced margin loss (raw margin increase 1.40) under replacement, versus 52.4% for frequency-defined global heads and 6.5% for rare heads.Clean activations replace selected heads’ attacked answer-position outputs, testing whether those heads carry attack-sensitive contextual signal.Source: E Activation Replacement Robustness · p. 16, Figure 5 · p. 17

F Additive Decomposition Details

G Illustrative Gumbel-Sigmoid Intervention

  • Figure 6 illustrates a ranking-based intervention: the distractor shifts probability from crew toward council, and zero-ablating the top-80 ranked heads reverses the shift.The example is illustrative, not a separate evaluation result.Source: Figure 6 · p. 18, G Illustrative Gumbel-Sigmoid… · p. 18

H Gumbel-Sigmoid Search Details

  • Gumbel-Sigmoid search learns relaxed binary head masks and ranks heads for top-K=80 interventions, with K chosen by a held-out sweep to balance recovery and clean-prompt harm.Source: H Gumbel-Sigmoid Search Details · p. 18

I Artifact use.

  • The evaluation uses publicly available model checkpoints and relation-completion resources under their access conditions; the constructed benchmark uses synthetic templates and factual lexical-semantic pairs.Source: I Artifact use · p. 18

Related papers

Loading 2610.18674v1…