Source-linked AI summary
Solving Hard Coreference Problems
Haoruo Peng, Daniel Khashabi, Dan Roth
TL;DR
Hard pronoun coreference can require background knowledge that gender and plurality do not provide. The paper encodes predicate-centered knowledge as Predicate Schemas and compiles it into inference constraints. Its general system improves performance on hard cases while retaining state-of-the-art performance on standard coreference datasets.
Problem
Hard pronouns may require background knowledge because gender and plurality information alone cannot resolve some cases.
Source: §1 · p. 1Method
Results
76.76% precision on hard Winograd cases exceeds the general system's 53.26% and the special-purpose classifier's 73.05%, while the system retains the state-of-the-art level on standard coreference.
Source: §1 · p. 1, §5.3 · p. 8Takeaways & Limitations
The paper treats hard pronoun resolution as a standard coreference problem and reports that its added Predicate Schemas do not harm predictions for regular mentions.
Source: §1 · p. 2, §5.3 · p. 8Type 1 and Type 2 schemas cannot resolve instances whose necessary context goes beyond their triple representation; additional cases require more informative, structured schemas.
Source: §2 · p. 4
Abstract
from arXiv · showhide
Coreference resolution is a key problem in natural language understanding that still escapes reliable solutions. One fundamental difficulty has been that of resolving instances involving pronouns since they often require deep language understanding and use of background knowledge. In this paper, we propose an algorithmic solution that involves a new representation for the knowledge required to address hard coreference problems, along with a constrained optimization framework that uses this knowledge in coreference decision making. Our representation, Predicate Schemas, is instantiated with knowledge acquired in an unsupervised way, and is compiled automatically into constraints that impact the coreference decision. We present a general coreference resolution system that significantly improves state-of-the-art performance on hard, Winograd-style, pronoun resolution cases, while still performing at the state-of-the-art level on standard coreference resolution datasets.
1 Introduction
Hard pronoun coreference often requires background knowledge that gender and plurality cannot provide. The paper represents predicate-centered knowledge as Predicate Schemas, compiles it into inference constraints, and reports gains on hard cases while preserving state-of-the-art performance on standard coreference.
1 Introduction
- The method addresses pronouns that cannot be resolved from gender or plurality alone and may require background knowledge.For example, resolving “it” in the bird-and-limb sentence requires knowing that a limb is more likely than a bird to bend.Source: §1 · p. 1
- 76.76% precision on hard Winograd cases exceeds the general system's 53.26% and the special-purpose classifier's 73.05%.The paper presents this as the result of a general-purpose coreference approach.Source: §1 · p. 1
- Predicate Schemas capture predicate-centered knowledge and are compiled automatically into constraints used in coreference decisions.The paper studies schemas for one predicate with its subject and object, and for two predicates with a semantically shared argument.Source: §1 · p. 2
- Predicate Schema knowledge is compiled into inference-time constraints rather than injected as features, and constraints can support global decisions across interacting pronouns.The paper notes that knowledge features can lose impact during standard training due to sparsity.Source: §1 · p. 2
- The system is presented as improving hard coreference while maintaining state-of-the-art performance on standard coreference problems.The paper frames hard pronoun resolution as a standard coreference problem and incorporates the constraints into a general coreference system.Source: §1 · p. 2
2 Predicate Schema
Predicate Schemas represent predicate-centered knowledge used to resolve challenging coreference cases. The paper describes two schema types and notes that some cases require richer representations than either type provides.
2 Predicate Schema
- The paper uses m for a mention, pred_m for its predicate, and a for another argument of that predicate.A mention may be an entity or a pronoun; predicate-argument order matters when applying scores.Source: §2 · pp. 2–3
- Type 1 represents one predicate with its subject and object, capturing preferences for which mention fills each role.The examples contrast how plausible different predicate-mention-argument combinations are.Source: §2 · p. 3
- Type 2 captures role preferences across two predicates that share a semantic argument, and may also use mention-argument information and the discourse connective.The paper says changing the connective in its examples can completely change the coreference resolution.Source: §2 · pp. 3–4
- Type 3 cases exceed the triple representation and cannot be resolved with Type 1 or Type 2 schemas.The paper says more informative, structured schemas will be needed for additional hard coreference instances.Source: §2 · p. 4
3 Constrained ILP Inference
The system uses Predicate Schema scores within an ILP coreference model, either as features or as constraints on mention-pair decisions. Schema evidence and pronoun-pair rules constrain inference, with thresholds controlling when schema-based constraints are added.
3 Constrained ILP Inference
- The ILP uses binary mention-pair coreference variables and permits at most one antecedent for each mention.A variable equals 1 when two mentions corefer and 0 otherwise; antecedents precede the mention they link to.Source: §3 · p. 4
- Predicate Schema scores form a vector for mention pairs and can augment pairwise features or be cast as ILP constraints.Larger score values indicate that the mention pair is more likely to be coreferent.Source: §3 · p. 4
- Schema-score comparisons generate ILP inequalities between coreference indicators; higher thresholds yield fewer, more reliable constraints.Thresholds are tuned on development data, and contradictory constraints from different score dimensions are ignored.Source: §3 · pp. 4–5
- The system automatically instantiates relevant schemas for a mention pair in context, then generates and evaluates corresponding constraints.The authors describe this as the first automatic constraint-generation and tuning method for coreference with ILP inference.Source: §3 · p. 5
- Additional constraints require semantically different pronouns, and certain non-possessive pronouns sharing a predicate, to have different antecedents.These pronoun-pair constraints apply within a fixed distance; the distance is three sentences.Source: §3 · p. 5
4 Knowledge Acquisition
Knowledge acquisition combines score vectors from four resources: Gigaword, Wikipedia, web queries, and context polarity. These sources supply occurrence, co-occurrence, query, and polarity information for the Predicate Schemas.
4 Knowledge Acquisition
- The overall mention-pair score vector concatenates scores from Gigaword, Wikipedia, web queries, and polarity.Each resource generates its own score vector.Source: §4 · p. 5
4.1 Gigaword Co-occurence
- From 4,111,240 Gigaword documents, the system extracts predicate-argument triples and uses log occurrence counts for Type 1 schemas.For Type 2 schemas, it counts neighboring triple co-occurrences within three sentences, using argument-matching heuristics and discourse connectives; it retains subject/object roles rather than exact strings to reduce sparsity.Source: §4.1 · pp. 5–6
4.2 Wikipedia Disambiguated Co-occurence
- Wikipedia text processed with a disambiguator supplies Type 1 schema scores from word co-occurrences in four syntactic positions.The system extracts entities, verbs, and nouns and keeps probability and count as separate score dimensions.Source: §4.2 · p. 6
4.3 Web Search Query Count
- Google queries provide Type 1 schema scores by averaging counts across query forms and noun-number and verb-tense variations.The query forms combine the antecedent, predicate, and argument in different orders; one form is restricted to adjective arguments with a to-be predicate.Source: §4.3 · p. 6, §4.4 · p. 6
4.4 Polarity of Context
- Context-polarity scores for Type 1 schemas adjust predicate polarity for object roles, polarity-reversing connectives, and negative comparative adverbs.The resulting mention-pair polarity score vector follows the mapping in Table 4.Source: §4.4 · pp. 6–7, Table 4 · p. 6
5 Experiments
The experiments evaluate Predicate Schemas on hard pronoun-resolution data and standard coreference datasets, finding gains on hard cases while preserving state-of-the-art-level performance on standard data. The analysis examines schema coverage and how Type 1 and Type 2 knowledge perform across instance categories.
5.1 Experimental Setup
- WinoCoref extends the Winograd annotations to include all pronouns and linked entities, and the experiments also evaluate on ACE and OntoNotes.The setup compares feature-based, constraint-based, and combined Predicate Schemas systems against Illinois baselines and Rahman and Ng (2012) on Winograd.Source: §5.1 · p. 7, Table 6 · p. 7
5.2 Results for Hard Coreference Problems
- KnowComb improves by over 20% over a state-of-the-art general coreference system on Winograd, outperforms Rahman and Ng (2012) by 3.3%, and improves by 15% on WinoCoref.The results also favor constraints over features when knowledge quality is high.Source: Table 7 · p. 8, §5.2 · p. 8
5.1 Experimental Setup
- AntePre evaluates WinoCoref by counting correct binary decisions over antecedent–pronoun pairs, while full ACE and OntoNotes use MUC, BCUB, Entity-based CEAF, and their average.Winograd uses precision.Source: §5.1 · p. 8
5.3 Results for Standard Coreference Problems
- On ACE and OntoNotes, KnowComb matches the baseline system’s performance, with no significant improvement because hard coreference cases are rare in standard datasets.The authors report that Predicate Schemas do not harm predictions for regular mentions.Source: Table 8 · p. 8, §5.3 · p. 8
5.4 Detailed Analysis
- Predicate Schemas cover 73% of Winograd instances, with Cat1 and Cat2 denoting instances requiring Type 1 and Type 2 knowledge, respectively.Other instances are assigned to Cat3.Source: §5.4 · p. 8, Table 9 · p. 8
- In the WinoCoref ablation, Type 1 and Type 2 knowledge have higher precision on their respective categories than on the full dataset; Category 2 is harder to solve than Category 1.The performance drop from either category to the full dataset motivates more complicated schemas and refined knowledge acquisition.Source: §5.4 · p. 9, Table 10 · p. 9
6 Related Work
Related work traces anaphora resolution from manually designed rules to machine-learning approaches focused on standard coreference. It also describes work on acquiring world knowledge and using external resources for coreference, alongside the paper’s goal of a coherent view.
Anaphora Resolution
- Early anaphora systems used manually designed rules, while later machine-learning work shifted attention toward standard coreference resolution.Examples of early systems include Hobbs (1978) and Lappin and Leass (1994); later work includes Connolly et al. (1994), Soon et al. (2001b), and Ng and Cardie (2002a).Source: §6 · p. 9
Winograd Schema
- Winograd-style sentence pairs use small contextual differences that can change a pronoun’s reference, posing a challenge for computer programs.The references are described as readily solvable by humans.Source: §6 · p. 9
World Knowledge Acquisition
- Prior coreference work used web-query information or alignment to external knowledge bases, while the paper aims to bring such approaches together through Predicate Schemas.The text notes no consensus on representing, gathering, and using high-quality world knowledge.Source: §6 · p. 9