Source-linked AI summary
Distinct dynamics of conceptual and referential disruptions in human reading and large language model processing
Rui He, Nihal Altay, Wolfram Hinzen
TL;DR
Conceptual and referential meaning are processed differently, but their downstream dynamics have rarely been compared within a common framework or tested in language models. The study disrupted each dimension in narratives and measured human reading and language-model responses, finding that conceptual disruptions were stronger and more localized, whereas referential effects were weaker and more distributed.
Problem
Conceptual and referential processing have rarely been compared within a common perturbational framework, including in language models’ handling of downstream context.
Method
The study introduced conceptual or referential distortions into short narratives and compared their propagation in human self-paced reading and language-model surprisal and representations.
Results
Conceptual disruptions produced stronger, rapidly declining effects, whereas referential disruptions produced weaker, more gradual effects; output representations instead showed larger initial displacement for referential disruption.
Takeaways & Limitations
The findings support distinguishable processing dynamics in which conceptual information is integrated locally while referential information is maintained more distributively across discourse.
Takeaways & Limitations
Conceptual and referential distortions differed in lexical, grammatical, frequency, salience, and manipulation properties, so trajectory differences may not reflect meaning type alone.
Abstract
from arXiv · showhide
Linguistic meaning is grounded in conceptual content, from which reference to particular entities emerges as words enter discourse. To examine the processing dynamics associated with these two dimensions of meaning, we selectively disrupted conceptual or referential information in short narratives and traced the resulting effects in human self-paced reading and in the predictive and representational processing of large language models. In human reading, conceptual disruptions produced a strong but localized processing cost, emerging immediately after the distorted word, reaching an early maximum, and then declining rapidly. Referential disruptions produced weaker effects, which decreased more gradually across subsequent words, and were more strongly modulated by sentence boundaries. In the language model, both disruptions emerged immediately at the manipulated word. Contextual model surprisal showed a pattern closely paralleling human reading: conceptual disruption produced a larger, more locally concentrated effect that decayed rapidly, whereas referential disruption produced a smaller and more gradual downstream effect. Output-layer representations showed a different pattern: referential disruption produced a larger initial displacement, while both distortions were subsequently characterized by power-law decay. Together, these results provide convergent evidence for distinguishable processing dynamics of two types of meaning: conceptual information imposes a more locally concentrated integration cost, whereas referential information engages a more distributed process of maintaining discourse-level identity.
1. Introduction
The introduction distinguishes conceptual content from referential identity as separate dimensions of linguistic interpretation that must be integrated incrementally across discourse. It motivates testing whether their effects propagate differently through subsequent context in humans and language models.
- Conceptual and referential meaning: Conceptual content contributes lexical meaning, but context determines what an expression refers to on a particular occasion.This distinction is grounded in semantic theory, including Frege’s separation of an expression’s presented meaning from its referent.
- Conceptual and referential meaning: Because interpretation unfolds incrementally, perturbations may influence not only the manipulated expression but also how subsequent material is interpreted.The introduction therefore treats the temporal distribution of processing difficulty as theoretically informative, alongside its magnitude.
- Language-model motivation: The study asks whether conceptual and referential perturbations leave distinguishable downstream signatures in language-model context processing, as they do in human processing.This question is linked to evidence that language-model hidden states can predict neural responses during natural-language processing.
- Language-model motivation: Contextual surprisal measures how a perturbation reshapes upcoming lexical expectations, while contextual representations capture the states supporting those predictions.These provide complementary language-model measures of how prior context is carried forward.
- Study hypotheses: The authors hypothesize that conceptual distortion produces a more concentrated downstream effect, whereas referential distortion persists across a broader span and is more sensitive to sentence boundaries.They test these predictions with word-by-word self-paced reading and language-model analyses.
2. Methods
The study combined an online word-by-word self-paced reading experiment using controlled fable variants with matched analyses of predictive and representational processing in a pretrained language model. Human and model measures were analyzed around distortion positions and across downstream effect trajectories.
- Human experiment: Ninety-nine native English speakers aged 20–50 years completed the online self-paced reading experiment.Participants were recruited through Prolific; mean age was 36.47 years, with 60 females and 39 males.
- Materials and procedure: Participants read 21 modern-English Aesop fables word by word, with each story presented in Original, conceptual-disruption (CD), or referential-disruption (RD) form.Stories were approximately 60–120 words long, with a median length of 95 words; each participant encountered seven stories in each condition.
- Materials and procedure: Distortions were manually inserted while preserving narrative structure, leaving opening sentences intact and avoiding sentence-final CD confounds with added adjuncts.These choices allowed readers to establish discourse context and reduced confounding with sentence-final wrap-up effects.
- Behavioral analyses: Local reading-time effects were estimated from K-3 to K+3 and at K_END using mixed-effects models comparing CD and RD separately with matched Original observations.Residualized log RT was modeled with condition as the predictor, controlling for word position, lexical surprisal, and word length; p values were FDR-adjusted across positions.
- Trajectory analyses: Downstream distortion trajectories were anchored at the final K token and extended from K+1 to sentence end or a maximum of K+10.The study also directly compared CD and RD trajectories using distortion type, categorical word distance, their interaction, and sentence-boundary interactions in a mixed-effects model.
- LLM analyses: The same K-centered framework was applied to surprisal and representational distance from the pretrained 4-billion-parameter Qwen3 base model.The model ran in evaluation mode without fine-tuning, processed each complete story in one forward pass, and summed surprisals across subword tokens when needed.
3. Results
Conceptual and referential distortions produced distinct temporal profiles in human reading and LLM processing. Conceptual effects were more locally concentrated, whereas referential effects were weaker but more distributed across discourse processing.
- Stimulus comparability: CD and RD increased whole-story perplexity relative to Original, did not differ significantly in disruption magnitude, and had comparable mean word lengths.RD showed a small lexical-frequency difference, likely reflecting pronoun or determiner replacement.
- Effects in LLM surprisal: At K, CD produced a larger contextual-surprisal peak, followed by rapid decline, whereas RD produced a smaller, more distributed response including K+1 and sentence-final positions.The distance-by-distortion-type interaction across K to K+10 was significant, F (10, 122) = 4.73, p < 0.001.
- Effects in reading time: Human reading-time effects emerged from K+1 onward, with significant propagation differences between CD and RD across K+1 to K+10.The distance-by-distortion-type interaction was significant, χ2 (9) = 66.58, p < 0.001.
- Effects in LLM representations: Output-layer representation distance peaked at K for both distortions, with a substantially larger initial displacement for RD and power-law attenuation thereafter.The distance-by-distortion-type interaction across K to K+10 was significant, F(10, 122) = 7.70, p < 0.001.
- Generalization: The principal model-based findings generalized to Llama-3.2-3B, including immediate effects, concentrated CD surprisal responses, persistent RD responses, and power-law representational decay.Sentence-boundary effects also followed the reported model-based pattern.
4. Discussion
The discussion identifies distinct but interacting dynamics for conceptual and referential disruptions across human reading and LLM measures. It also notes design and measurement limitations that constrain interpretation and generalizability.
- Processing dynamics: Conceptual distortion produced stronger, rapidly declining effects, whereas referential distortion produced weaker, more gradual effects that were more sensitive to sentence boundaries.This distinction appeared in both human reading times and LLM surprisal.
- Representational dynamics: Referential distortion produced a larger initial output-layer displacement, but conceptual and referential distortions showed similar downstream power-law decay.Sentence boundaries reduced both representational distances from the Original condition and did not selectively modulate referential distortion.
- Theoretical interpretation: The results support a dynamic distinction between conceptual content and reference while emphasizing that the dimensions interact during comprehension rather than forming identical processes.Conceptual content helps identify discourse entities, whereas reference links expressions to particular entities and maintains those links across discourse.
- Limitations: The study’s main limitation is that conceptual and referential distortions were not process-pure and differed in lexical, grammatical, frequency, salience, and manipulation-rate properties.Some trajectory differences may therefore reflect formal properties rather than conceptual–referential status alone.
- Limitations: Generalizability and measurement-based interpretation were limited by 21 manually distorted fables, self-paced reading’s temporal constraints, and descriptive curve fitting across approximately 11 distance points.The study also notes that single-distortion materials are needed to establish the independence of each effect.
Appendix A. Example excerpts for conceptual and referential distortions · Original:
The appendix provides an original narrative excerpt in which an ant is swept away by a river current and nearly drowns before a dove notices its struggle.
- Original:: The original excerpt depicts an ant being swept away by a rushing current while trying to quench its thirst, with a dove observing from a tree.The passage introduces the ant’s peril and the dove’s noticing of its struggle.
Conceptual distortion:
The conceptual-distortion section presents a narrative in which an ant is swept away while trying to drink at a riverbank. A dove observes the ant’s struggle from a tree overhanging the water.
- Conceptual distortion:: An ant goes to a riverbank to quench its thirst but is swept away by rushing mountain water and nearly drowns.The passage introduces the ant’s attempted action and ensuing danger.
- Conceptual distortion:: A dove perched on a tree hanging over the water notices the ant’s struggle.The dove is introduced as an observer of the ant’s predicament.
Referential distortion:
The referential-distortion narrative introduces an ant at a riverbank whose struggle in the current is observed by a dove. The passage establishes the discourse context through these entities and events.
- Referential distortion:: An ant goes to a riverbank to quench its thirst.The narrative begins with the ant’s goal and location.
- Referential distortion:: The ant is swept away by the rushing current and nearly drowns.This event creates the ant’s central predicament.
- Referential distortion:: A dove perched on a tree over the water notices the ant’s struggle.The dove is introduced as an observing entity linked to the ant’s predicament.
Appendix B. Curve fitting and model comparison
Distance-specific distortion effects were fit with 11 candidate functions using covariance-aware generalized least squares. Model selection used AICc, with bootstrap procedures quantifying uncertainty in derived trajectory characteristics.
- Candidate functions: Distance-specific effects across supported word distances were modeled using 11 candidate functional forms, including constant, polynomial, broken-stick, exponential, and power-law functions.The candidate functions were defined over word distance x = 1, . . . , 10.
- Curve fitting: Parameters were estimated with generalized least squares, incorporating uncertainty in distance-specific estimates and covariance among estimates from the same model.Covariance matrices were symmetrized and regularized when needed for numerical positive definiteness.
- Curve fitting: Nonlinear models used bounded covariance-weighted least squares with multiple starting values, retaining the solution with the smallest χ2.This procedure reduced sensitivity to initialization and local optima.
- Model comparison: AICc was the primary selection criterion because distance-point counts were small relative to some models’ free parameters; the lowest-AICc model was selected.Models with n ≤k + 1 were considered unestimable under AICc.
- Uncertainty estimation: Uncertainty in derived reading-time trajectory characteristics was estimated from 1,000 parametric-bootstrap iterations using the selected functional form and covariance-weighted refitting.Bootstrap confidence intervals were obtained from the resulting distributions.
Appendix C. Results with Llama 3.2
Appendix C presents Llama 3.2 results on local effects and effect decay for surprisal and representational distance. The appendix includes separate figures for surprisal and representational-distance dynamics.
- Local effects: Figure S1 reports local effects of surprisal and representational distance with Llama 3.2 3B.The figure addresses both measures in the 3B model.
- Effect decay: Figure S2 presents the effect decay of Llama 3.2 surprisal.The figure focuses on how surprisal effects decay.
- Effect decay: Figure S3 presents the effect decay of Llama 3.2 representational distance.The figure focuses on how representational-distance effects decay.