Source-linked AI summary
When a sentence does not introduce a discourse entity, Transformer-based models still sometimes refer to it
Sebastian Schuster, Tal Linzen
TL;DR
Tracking discourse entities is essential for understanding longer narratives and conversations, yet it remains unclear whether language models can perform this function like humans. The paper adapts psycholinguistic language-model assessment into targeted English evaluations of sentential operators and indefinite noun phrases, finding that GPT-2 and GPT-3 show some sensitivity but lack systematic entity tracking, especially with multiple noun phrases.
Problem
Language models lack explicit discourse-entity representations, and it remains largely open whether they can match human entity tracking.
Method
The paper adapts the psycholinguistic assessment of language models paradigm and introduces targeted evaluation suites for interactions between sentential operators and indefinite noun phrases.
Results
GPT-2 and GPT-3 show some sensitivity to operator–indefinite-NP interactions, but models lack systematicity and are challenged by contexts containing multiple noun phrases.
Takeaways & Limitations
The findings provide direct evidence of shortcomings in language models’ discourse-entity tracking and a considerable gap between humans and GPT-3 for basic discourse phenomena.
Takeaways & Limitations
High accuracy on this evaluation suite is necessary but not necessarily sufficient evidence of human-like discourse entity processing.
Abstract
from arXiv · showhide
Understanding longer narratives or participating in conversations requires tracking of discourse entities that have been mentioned. Indefinite noun phrases (NPs), such as 'a dog', frequently introduce discourse entities but this behavior is modulated by sentential operators such as negation. For example, 'a dog' in 'Arthur doesn't own a dog' does not introduce a discourse entity due to the presence of negation. In this work, we adapt the psycholinguistic assessment of language models paradigm to higher-level linguistic phenomena and introduce an English evaluation suite that targets the knowledge of the interactions between sentential operators and indefinite NPs. We use this evaluation suite for a fine-grained investigation of the entity tracking abilities of the Transformer-based models GPT-2 and GPT-3. We find that while the models are to a certain extent sensitive to the interactions we investigate, they are all challenged by the presence of multiple NPs and their behavior is not systematic, which suggests that even models at the scale of GPT-3 do not fully acquire basic entity tracking abilities.
1 Introduction
Tracking discourse entities is necessary for understanding narratives and conversations, but it remains unclear whether language models can match human behavior. This work introduces a targeted evaluation of how sentential operators interact with indefinite noun phrases and finds that GPT-2 and GPT-3 show some sensitivity but lack systematicity, especially with multiple noun phrases.
- Motivation: GPT-2 and GPT-3 lack explicit discourse-entity representations, leaving open whether they can match human entity tracking.Humans are described as maintaining and updating representations of entities and their properties, whereas these models do not have explicit representations.
- Motivation: Indefinite noun phrases generally introduce discourse entities, but operators such as negation can block that introduction and make later reference pragmatically odd.For example, a dog in “Arthur doesn’t own a dog” is treated as non-introducing, so a subsequent elaboration about it should have low probability relative to a matched control.
- Contribution: The targeted evaluation suite addresses a granularity gap left by broad datasets such as LAMBADA, which cannot identify which contexts support or impair entity tracking.The suite focuses on interactions between sentential operators, embedding verbs, and indefinite noun phrases.
- Contribution: The study adapts psycholinguistic language-model assessment to discourse entity tracking and evaluates GPT-2 and GPT-3 with expert-created evaluation suites.The adaptation also examines methodological challenges involved in applying the paradigm to discourse phenomena.
- Contribution: All evaluated models show some sensitivity to preceding context but lack systematicity and struggle when contexts contain multiple noun phrases.These findings motivate a fine-grained assessment rather than relying only on broad narrative-completion benchmarks.
2 Related Work
Prior language-model evaluations have focused mainly on syntax, while discourse studies have examined reference biases, noun-phrase distinctions, and representational evidence for coreference. These approaches provide partial evidence of entity tracking but do not establish consistent tracking across entities.
- Existing evaluations: Most systematic evaluations of autoregressive and masked language models target syntactic abilities such as agreement, binding, and filler-gap dependencies.Higher-level evaluations have received comparatively less attention in this research overview.
- Discourse and reference: Prior work tested referential biases, definite-versus-indefinite noun-phrase judgments, and whether model representations contain coreference information.These studies span GPT-2, Transformer-XL, contextual language models, LSTMs, Transformer models, and BERT attention patterns.
- Discourse and reference: The reviewed probing and attention studies found some coreference information in evaluated representations, but not consistently for all entities.This provides partial evidence for entity-tracking abilities without demonstrating systematic behavior.
3 Background
Indefinite noun phrases typically introduce discourse entities, but negation, embedding verbs, and modals can prevent this, affecting whether later reference is felicitous. These generalizations have exceptions involving specific interpretations, which the paper largely sets aside.
- Indefinite noun phrases and operators: In affirmative statements, an indefinite noun phrase generally introduces a new discourse entity, whereas negation prevents introduction and makes later pronominal reference pragmatically odd.The contrast is illustrated by affirmative versus negated statements about Arthur owning a dog.
- Embedding verbs: The implicative manage to permits discourse introduction under its indefinite noun phrase, while negative implicative fail to blocks it.“Sue managed to write a book” supports later reference, whereas the corresponding fail-to sentence does not.
- Embedding verbs: Factive know allows an embedded indefinite noun phrase to introduce an entity, whereas non-factive doubt does not.The contrast is shown by whether a later pronoun referring to the baked cake is felicitous.
- Modal operators: The modal want blocks discourse-entity introduction under its indefinite noun phrase.A pet rat obtained by Mary supports later reference, but one Mary wants to get does not.
- Exceptions and scope: Specific interpretations can allow an indefinite noun phrase to scope over negation and support later reference, but listeners generally do not infer them without additional context.The paper therefore largely ignores these exceptions in the remainder of its analysis.
4 Experiments
The experiments test whether GPT-2 and GPT-3 track discourse-entity introduction through referential continuations, using minimal contrasts and relative probabilities. Models show partial sensitivity, but multiple noun phrases expose unreliable and nonsystematic entity tracking.
- Experiment 1: For affirmative-negation, affirmative-modal, and managed-failed contrasts, models exhibited the expected relative-probability pattern on almost all items, whereas know-doubt performance was approximately at chance.Human preferences were strongest for affirmative-negation and managed-failed, while affirmative-modal and know-doubt results were less clear-cut.
- Experiment 1: Human results complicate interpretation of affirmative-modal and know-doubt contrasts, and the five models agreed on fewer than 33% of items in item-level comparisons.Participants often selected referential continuations even in non-entity-introducing contexts for these contrasts, so model similarity to humans cannot be inferred from aggregate results alone.
- Experiment 1: GPT-2 and GPT-3 generally favored referential continuations after contexts that introduced discourse entities, but their behavior could also reflect spurious operator-continuation correlations.Experiment 1 therefore motivates a second experiment designed to distinguish genuine operator–NP integration from learned continuation biases.
- Experiment 2: With multiple indefinite NPs, humans selected coreferential continuations for introduced entities and non-coreferential continuations for non-introduced entities across contrasts, validating the materials.The second experiment used contexts crossing noun and operator order to test entity tracking under greater structural complexity.
- Experiment 2: Models were above chance in many coreferential conditions but remained below ceiling, performed worse on non-coreferential continuations, and showed strong variation by model size.GPT-3 performed consistently best, while the smallest GPT-2 performed consistently worst; non-coreferential cases require both identifying the introduced entity and rejecting unrelated references.
- Experiment 2: Under the stricter requirement that all four noun/operator orders produce the correct result, performance was much lower and remained sensitive to minor contextual changes, although larger models were more systematic.The smallest GPT-2 models often favored one noun regardless of context, whereas GPT-3 exhibited this behavior for 7% of items.
5 Likely Continuations
Likely-generation analysis shows that GPT-2 XL and GPT-3 are somewhat sensitive to which indefinite NPs introduce discourse entities, but they still often refer back to non-entity-introducing NPs.
- Materials and method: The study sampled continuations beginning with “the” from Experiment 2 contexts and annotated whether they referred back to discourse-entity or non-discourse-entity NPs.GPT-2 XL used top-40 sampling, while GPT-3 used temperature sampling at 0.7.
- Materials and method: The examples distinguish a continuation referring to an introduced entity, such as “the letter,” from one referring to a non-introduced NP, such as “the bag.”The first example follows “she wrote them a letter,” whereas the second refers back to “a bag” under failed negated action.
- Materials and method: Constraining generations to begin with “the” increased noun-phrase references while imposing few constraints on the overall continuation.
- Results and discussion: GPT-2 XL and GPT-3 generated more references to discourse-entity-introducing NPs than to non-discourse-entity-introducing NPs, while still producing over 20% non-entity references.This extends the Experiment 2 findings to likely model generations.
6 General Discussion
The paper concludes that Transformer models show partial sensitivity to interactions between sentential operators and indefinite NPs but lack systematic human-like entity tracking. It argues that targeted discourse evaluations remain useful and discusses architectural, semantic-supervision, and grounding-based paths forward.
- Summary: Transformer-based models were somewhat sensitive to sentential-operator and indefinite-NP interactions, but Experiment 2 showed nonsystematic behavior, including in high-probability generations.The authors interpret this as evidence that models do not combine these linguistic elements as humans do.
- Learnability of meaning: The results provide direct evidence of shortcomings in language models’ discourse-entity tracking and contribute to debate over whether language models can mimic human language understanding.
- NLG evaluation: Targeted discourse evaluation suites can complement NLG benchmarks and human studies because human-like surface judgments may coexist with a gap in basic discourse phenomena.The paper highlights a considerable gap between humans and GPT-3 for these phenomena.
- Comparison to probing results: Probe-based decoding of entity states may overestimate tracking ability because classifiers can exploit spurious correlations between training and test splits.
- Potential solutions: The authors propose explicit entity memories, supervision with meaning representations, and more grounded learning scenarios as possible routes toward more systematic tracking.They suggest that simply training on more data is unlikely to yield human-like discourse-entity processing.
- Potential solutions: The evaluation suite is intended to help assess progress in future models and improve discourse-entity processing in natural-language-understanding systems.
Ethics Statement
The work highlights risks of overstating model abilities from evaluation-suite performance and reports safeguards for human-subject participation. It also limits the study to English-trained models.
- Researchers should use the dataset for out-of-domain evaluation because models fine-tuned on similar examples may perform substantially better.
- The study evaluated only models trained on English data, so entity-tracking abilities in models trained on other languages may differ.
- The human-subject experiment recruited US crowdworkers, paid about $14 per hour, received NYU IRB approval, and posed no participation risks.
A Human experiment details
The human experiment used practice, critical, and filler trials to validate the stimuli with native English-speaking participants in the United States.
- Participants completed two practice trials, four critical trials, and two randomly interspersed filler trials.Figure 4 illustrated an example trial.
- Participation was limited to people living in the United States whose native language is English.
B Model experiment details
The model experiments used separate scoring tools for GPT-2 and GPT-3 and compared raw probabilities without temperature transformation.
- GPT-2 experiments used the LM-Scorer library, whereas GPT-3 experiments used the official OpenAI API.
- All evaluations compared raw, untransformed probabilities with the temperature parameter set to 0.
- The GPT-2 scoring setup required approximately 8 hours of CPU time on a 3.7GHz CPU with 32GB of RAM.