Source-linked AI summary
A Circuit for Plural Reference: How LLMs Represent and Retrieve Singular and Plural Entities
Anh Danh, Rick Nouwen, Massimo Poesio
TL;DR
LLM plural-reference behavior is important for understanding contextual reasoning, yet its internal mechanism is not well characterized. The paper combines mechanistic interpretability, causal interventions, and attention analysis to identify a circuit for representing coreference, forming plural entities, and selecting pronouns. It finds that plural reference is favored for ontologically similar entities linked by and, while the study remains limited to basic two-element conjoined noun phrases and an incomplete circuit account.
Problem
The paper investigates how LLMs represent and retrieve entities for plural reference, an area less understood than singular coreference resolution.
Method
The authors combine mechanistic interpretability, causal intervention techniques, and attention-pattern analysis to trace information flow across attention heads and layers.
Results
The study identifies attention-head groups that encode coreference, form plural entities, transfer that information, and support antecedent selection and pronoun prediction.
Takeaways & Limitations
LLMs favor plural reference when candidate entities are ontologically related and connected by a preferred conjunction.
Takeaways & Limitations
The study focuses on two-element conjoined noun phrases and does not establish a fully unique coreference circuit or thoroughly explain MLP functionality.
Abstract
from arXiv · showhide
Coreference resolution is an important task in contextual reasoning. In this paper, we investigate the mechanism for representing and retrieving singular and plural entities for plural reference. We use a combination of mechanistic interpretability and attention pattern analysis to study the process in which LLMs predict a pronoun to refer back to previously mentioned entities. Using a range of causal intervention techniques, we find a set of attention heads that are responsible for (1) representing coreference information in the input, (2) identifying entities that form a plural reference, (3) transferring the information to the component that is responsible for selecting the antecedents and predicting the pronoun. We also find that LLMs align with humans in preference for plural pronoun. Specifically, entities in a plural construction are more likely to be referred to as a plural entity if they are ontologically similar and are linked by the conjunction "and".
1 Introduction
The paper studies how LLMs represent and retrieve singular and plural antecedents during pronoun prediction. It identifies a circuit that encodes coreference, forms plural entities, and transfers that information to pronoun selection, with plurality influenced by ontological similarity and conjunction choice.
- Understanding LLM mechanisms can improve their controllability, safety, and explainability.
- Plural reference requires tracking multiple entities and grouping those that should be co-referred to by a plural pronoun.
- The circuit contains Pronoun Interpretation Heads for coreference representations, Plurality Formation Heads for grouping entities, and Pronoun Selection Heads for choosing the pronoun.
- LLMs prefer plural entities whose members have similar ontological properties, such as John and Mary rather than John and his bike.
- A plural entity is more available with and, as in John and Mary, and less preferred with with, as in John with Mary.
2 Background and Related Work
The background frames plural reference as an underexplored aspect of coreference resolution and motivates mechanistic analysis of how LLMs represent candidate antecedents. It introduces plural-reference phenomena, mechanistic interpretability, and the self-attention architecture examined in the paper.
- 2.1 Coreference Resolution: Coreference resolution is crucial for language interpretation, but most evaluation work focuses behaviorally on singular reference.
- 2.1 Coreference Resolution: Much less is known about how modern language models track and retrieve plural entities throughout discourse.
- 2.2 Plural Reference: Plural reference links a plural noun phrase to one or more entities, often using a conjunction such as and.
- 2.2 Plural Reference: Plural reference can remain possible without and, while entities that form a unified object may instead receive a singular pronoun.
- 2.3 Mechanistic Interpretability: Mechanistic interpretability reverse-engineers model computations by attributing predictions to sparse components such as layers, heads, or neurons.
- 2.4 Self-Attention Mechanism: Self-attention assigns token-pair weights using query and key vectors, while value information is projected into the residual stream.
3 Dataset and Evaluation
The study evaluates how LLMs choose singular or plural pronouns in controlled sentence prefixes containing plural constructions. It tests how gender, ontological similarity, and linking conjunction affect plural-reference preference.
- Task design: The pronoun-prediction task presents sentence prefixes with entities joined in plural constructions and asks the model to complete the reference.Dpl favors plural pronouns, whereas Dsg preserves the construction but makes plural reference dispreferred.
- Datasets: Dpl uses animate entities such as Mary and her sister, while Dsg replaces the second entity with an inanimate object such as her bike.The datasets therefore contrast plural-reference-compatible and plural-reference-dispreferred constructions while retaining the sentence structure.
- Dataset formalization: Each Dpl prefix contains a subject, two antecedents, and a conjunction, with the antecedents forming the plural entity referred to by them.The plural construction is formalized as (e1, c, e2).
- Dataset construction: The experiments generated 300 prefixes per dataset, using single-token entities, equal-length prefixes, and randomized entity gender.
- Evaluation variables: The evaluation varies antecedent gender, ontological similarity, and the linking form and versus with.Ontological similarity ranges from two proper names to an entity paired with an inanimate object.
- Evaluation results: Less ontological similarity reduces plural-reference preference, and with reduces it relative to and.These effects replicate patterns reported in psycholinguistic research.
4 Experimental Setup
The experiments use activation and path patching to isolate how model components affect plural-pronoun probability. Interventions compare clean and corrupted prompts across components, positions, and models.
- Intervention methods: Activation patching measures a component’s indirect output effect, whereas path patching restores downstream activations to isolate its direct effect.A decrease in Ppl after path patching is attributed to the intervened component’s direct effect.
- Intervention procedure: The path-patching procedure intervenes on high-level residual streams or lower-level head value vectors while blocking indirect propagation from corrupted activations.The procedure records an original run, caches a corrupted run, then recomputes the intervened prediction.
- Corrupted prompts: Corrupted prompts alter either the second entity or the linking conjunction because ontological similarity and conjunction affect plural-pronoun preference.The corrupted forms are represented as (e1, c, e′2) and (e1, c′, e2).
- Intervening locations: Interventions are performed at multiple positions, but only the last token and corrupted-token positions show considerable effects.The subsequent findings therefore focus on these two positions.
- Metric: The evaluation metric M is the percentage probability difference between original and intervened plural-pronoun probabilities, with negative values indicating suppressed plurality signals.
- Models: The study patches Qwen3-0.6B, Qwen3-1.7B, and GPT2-medium, while presenting the main-text circuit for Qwen3-1.7B.Comparable results for the other models are reported in the appendix.
5 The Plural Reference Circuit
The circuit separates plural-reference resolution into heads that encode coreference, form plural entities, and select antecedents for pronoun prediction. Causal interventions and attention patterns identify how these components interact across layers.
- Pronoun Selection Heads: L23H12 and L23H13 act as Pronoun Selection heads, attending to candidate antecedents and directly affecting pronoun prediction.L23H12 attends to both entities in plural contexts but primarily to e1 for singular contexts; L23H13 also strongly attends between the conjunction and e1.
- Information Transfer: K/V vectors, rather than the Q vector, strongly affect plural-pronoun prediction at the Pronoun Selection head.Path interventions trace relevant Value information to K/V heads at layers 15, 17, 19, and 21, with L17H9 showing the strongest effect.
- Plurality Formation Heads: The L17H9 Plurality Formation head selectively attends to entities and conjunctions that satisfy constraints for forming a plural entity.It attends across the qualified plural construction, while its pattern changes for singular constructions and for conjunctions using with rather than and.
- Pronoun Interpretation Heads: The Pronoun Interpretation head encodes coreference by attending from pronouns to earlier antecedents and from them to the whole plural construction.For plural reference, them attends especially strongly to the second entity and the conjunction, supporting a coreference-map representation.
- Generalization: The identified circuit generalizes across sentence templates, within the Qwen3 family, and to GPT2-medium.The reported generalization concerns the same attention patterns across the circuit’s heads.
6 Discussion
The model builds representations of candidate entities, forms plural groups when they match its preferences, and transfers that information to pronoun-selection components. Coreference information is encoded separately and used during prediction.
- In middle layers, the model builds representations of an entity or group before producing a reference.
- Plurality Formation heads attend across candidate-entity phrases when the entities are ontologically related and linked by a preferred conjunction.
- Plurality Formation heads write the selected entity or group into the residual stream for downstream processing.
- Pronoun Selection heads receive this information, attend to known entities, and use it to choose the pronoun.
- A separate attention head encodes coreference information for all mentioned entities.
7 Conclusion
The paper investigates how LLMs represent and use coreference information for pronoun prediction, especially with plural references. It identifies constraints for singular versus plural pronouns and extracts a mechanistic circuit for selecting and propagating antecedent information.
- The study examines how LLMs represent and use coreference information when predicting pronouns for plural references.
- The paper identifies constraints that guide singular-versus-plural pronoun choices in plural constructions.
- Mechanistic interpretability and attention-pattern analysis are used to extract a circuit that identifies antecedents and propagates their information across components.
Limitations
The study is limited to simple two-element conjoined noun phrases and does not fully characterize the complete coreference circuit or MLP functionality. More complex reference patterns remain for future analysis.
- The study covers only conjoined noun phrases with two elements, excluding more complex plural references such as split antecedents.
- The model’s behavior in cases where plural antecedents form a new unified object remains unexamined.
- The analysis emphasizes component functions rather than extracting a complete coreference circuit, leaving uncertainty about circuit uniqueness.
- The functionality of MLPs is not thoroughly investigated, although late-layer MLP intervention strongly affects Ppl.
8 Ethical Considerations
The study uses synthetic datasets generated by LLMs.
- The study uses only synthetic datasets generated by LLMs.
9 Statement of AI Usage
The project used AI for coding assistance, dataset generation, and spelling checks, while the authors retained responsibility for the ideas, experimental setup, interpretation, and writing. The study examined models from the Qwen3 and GPT2 families across configurations described in Table 1.
- AI assisted with coding, dataset generation, and spelling checks.
- The authors conducted the project’s ideation, experimental setup, interpretation, and writing.
- The study examined LLMs from the Qwen3 and GPT2 families across multiple model configurations.The configurations include model size, layers, attention heads, key-value heads, and hidden-activation size.
B Statistical Analysis
The statistical analysis models plural-versus-singular pronoun probability using controlled prompt variables and examines how attention differences relate to pronoun choice. The reported correlation results connect stronger attention to the second antecedent with higher plural-pronoun probability.
- Statistical analysis: The analysis used linear mixed-effects regression to predict the probability difference between plural and singular pronouns.The models used verbs as random effects and were fitted with Restricted Maximum Likelihood.
- Statistical analysis: The independent variables included gender, ontological similarity, and linking conjunction.Ontological similarity contrasted name-plus-name, name-plus-relation, and name-plus-object combinations; conjunction contrasted “and” with “with.”
- Attention analysis: The attention analysis defined αdiff as the difference between attention to the two antecedents and Pdiff as the plural-minus-singular pronoun probability difference.A negative αdiff means the second antecedent receives more attention than the first.
- Attention analysis: Pearson correlations were positive for L23H12 (r = .78, p < .001) and L23H13 (r = .77, p < .001).The results indicate that stronger attention to e2 relative to e1 accompanied higher probability for the plural pronoun.
- Figures and materials: Figure 8 plots attention-weight differences between e1 and e2 against differences between Psg and Ppl for Dsg and Dpl.Blue dots represent Dsg and brown dots represent Dpl.
- Figures and materials: Figures 9–11 visualize attention patterns of major circuit heads across Qwen3 and GPT2 model variants.The visualizations include Pronoun Selection, Plurality Formation, and Pronoun Interpretation heads, alongside intervention results where specified.