PaperlayerView exact revision on arXiv

Source-linked AI summary

Anaphora and Coreference Resolution: A Review

Rhea Sukthanker, Soujanya Poria, Erik Cambria, Ramkumar Thirunavukarasu

arXiv:1805.11824v1cs.CL

TL;DR

Entity resolution research must clarify the related but distinct tasks of anaphora and coreference resolution, while addressing diverse reference types, evaluation practices, and unresolved cases. This survey synthesizes their definitions, methods, datasets, metrics, and applications, distinguishing their boundaries and reviewing rule-based, statistical, and neural approaches. Its principal conclusion is that multiple datasets remain unused by state-of-the-art methods, motivating more exhaustive evaluation and clearer standards.

  • Problem

    Anaphora and coreference resolution encompass closely related but distinct problems with diverse references, inconsistent evaluation, and unresolved scope and coverage issues.

  • Method

    The paper conducts a comprehensive survey of reference types, constraints, rule-based, statistical, and deep learning methods, datasets, evaluation metrics, toolkits, and applications.

  • Results

    The survey finds that multiple available datasets have not been used to evaluate state-of-the-art methods.

  • Takeaways & Limitations

    The review clarifies AR–CR boundaries and provides standards-oriented coverage of metrics, datasets, and open-source toolkits for more focused research progress.

  • Takeaways & Limitations

    Cataphora are excluded because they are rarely used in spoken language, while split anaphora and world knowledge remain insufficiently handled by prominent methods.

Abstract

from arXiv · show

Entity resolution aims at resolving repeated references to an entity in a document and forms a core component of natural language processing (NLP) research. This field possesses immense potential to improve the performance of other NLP fields like machine translation, sentiment analysis, paraphrase detection, summarization, etc. The area of entity resolution in NLP has seen proliferation of research in two separate sub-areas namely: anaphora resolution and coreference resolution. Through this review article, we aim at clarifying the scope of these two tasks in entity resolution. We also carry out a detailed analysis of the datasets, evaluation metrics and research methods that have been adopted to tackle this NLP problem. This survey is motivated with the aim of providing the reader with a clear understanding of what constitutes this NLP problem and the issues that require attention.

1 Introduction

This survey clarifies anaphora resolution (AR) and coreference resolution (CR) as closely related but distinct entity-resolution tasks in NLP. It synthesizes their methods, evaluation metrics, datasets, applications, and unresolved issues.

  • AR and CR resolve repeated references to entities and support NLP tasks including machine translation, sentiment analysis, paraphrase detection, and summarization.
  • The survey builds on earlier AR and CR reviews by providing a coherent, holistic overview of both fields.
  • Research has shifted from hand-crafted, rule-based systems toward deep learning approaches that learn feature representations.
  • Different evaluation metrics make performance comparisons difficult across proposed methodologies.
  • The survey analyzes evaluation metrics, standard datasets, open-source toolkits, applications, and debates including commonsense knowledge.

2 Types of References in Natural Language

Natural-language references take many semantic and syntactic forms, making anaphora resolution challenging and creating coverage gaps for existing algorithms. The survey reviews these reference types, including uncommon, implicit, and multiple-antecedent cases.

  • AR must handle diverse reference forms, and many algorithms target only specific types, creating a coverage issue.
  • One anaphora uses “one” to refer to an antecedent, while comparative, presuppositional, contextual, and pronominal forms introduce additional ambiguity.
  • 2.5 Discontinuous Sets (Split anaphora): Split anaphora uses one pronoun to refer jointly to multiple antecedents, a phenomenon that prominent AR and CR algorithms often fail to consider.
  • References may include ambiguous, definite, adjectival, and other forms whose antecedent relations require semantic or discourse interpretation.
  • Cataphora points forward to an entity introduced later and is excluded from this review because it is rarely used in spoken language.
  • Bridging anaphora refers implicitly to an earlier entity through an inferred relationship rather than an explicitly stated antecedent.

3 Non-Anaphoric pronominal references

AR systems must identify and eliminate pronouns that do not refer to antecedents. The main non-anaphoric categories discussed are clefts, pleonastic “it,” and extraposition.

  • Empty referents are pronouns or expressions that potentially do not refer back to any antecedent.
  • Cleft sentences can use “it” as a focus marker rather than an antecedent-referring expression.
  • Pleonastic “it” can appear in weather expressions without referring to a specific entity.
  • In extraposition, “it” is semantically empty and represents a displaced clause or proposition.

4 Constraints for Anaphoric resolution

AR and CR methods use syntactic, semantic, discourse, and world-knowledge constraints to filter and rank antecedent candidates. These constraints improve resolution but vary in scope, and commonsense knowledge cannot generally be fully incorporated into a system.

  • Most AR and CR approaches apply syntactic and semantic constraints during feature extraction, mention filtering, or direct rule-based resolution.
  • Gender, number, and person agreement prune incompatible antecedents before final resolution.
  • Verb-selection constraints use whether entities are animate or inanimate to disambiguate possible antecedents.
  • Subject position, recency, repeated mention, and parallelism assign greater preference to some antecedents than others.
  • 4.9 World Knowledge: World knowledge is a critical indicator because syntax alone can assign different antecedents when verb choice changes.
  • 4.9 World Knowledge: Commonsense constraints have wide scope and generally cannot be completely incorporated into resolution systems.

5 Evaluation metrics in CR

CR evaluation uses multiple metrics that differ in how they compare predicted and gold reference clusters, links, or entities. The surveyed metrics include mention-weighted scoring, link-modification scoring, and optimal cluster mapping.

  • CR evaluation metrics have been proposed to assess predicted reference clusters against gold annotations using different comparison strategies.The section introduces standard metrics for evaluating coreference resolution.
  • B-cubed metric: B-cubed computes precision and recall for individual mentions and combines them through a weighted sum, with greedy matching for chain pairings.Entity weights are commonly set to 1/N, where N is the number of entities in the document.
  • MUC metric: MUC represents each reference cluster as linked references and measures the link modifications needed to match system and truth sets.Its precision and recall use partitions describing how result and truth clusters intersect.
  • CEAF metric: CEAF first creates an optimal one-to-one mapping between result and truth clusters, then calculates precision and recall using self-similarity.The mapping function assigns each result cluster to a true cluster while preventing multiple result clusters from sharing one mapped cluster.

5.4 ACE-Value

ACE evaluates coreference using true positives, false positives, and false negatives under optimal matching, while CoNLL aggregates three established metrics into one score.

  • ACE-Value: ACE calculates precision and recall from true positives, false positives, and false negatives among predicted coreferent entities.Unlike CEAF, ACE does not normalize its precision and recall values.
  • CoNLL score: CoNLL score averages the B-cubed, MUC, and CEAF scores for coreference evaluation on the OntoNotes corpus.The score was used by the CoNLL-2012 shared task.
  • BLANC metric: BLANC adapts the Rand index as a link-based metric that separately evaluates coreference and non-coreference links.Rc and Rn denote recall for coreference and non-coreference links, respectively.
  • BLANC metric: BLANC is not widely adopted because mention identification effects strongly affect its metric values.

5.7 LEA metric

LEA is designed to evaluate coreference resolution by considering both entity importance and resolution quality. The broader metric discussion notes that earlier metrics have distinct shortcomings affecting interpretability and system comparison.

  • LEA metric: LEA weighs entities by importance and evaluates how well each entity is resolved using link similarity.Importance depends on entity size, while the resolution score is based on link similarity.
  • LEA metric: LEA uses result sets and key or gold sets to compare resolved entities against reference annotations.
  • Metric limitations: MUC can ignore singleton entities, favor systems producing fewer entities, and sometimes assign higher F-measure to worse systems.The passage also describes separate concerns for B-cubed, BLANC, ACE, and CEAF.

6 Comparison between Anaphora and Coreference Resolution

Anaphora resolution and coreference resolution overlap but differ in scope: AR usually links textual references to earlier expressions, whereas CR targets the same extra-linguistic entity across broader reference phenomena. The review argues that CR datasets and models should specify which reference types they cover.

  • Conceptual distinction: AR resolves entities within text with the same sense and usually requires little world knowledge, while CR links expressions to the same extra-linguistic entity despite possible sense differences.
  • Conceptual distinction: The claim that AR is a subset of CR fails for bound variables, as in “Every speaker had to present his paper,” where “his” is anaphoric but not coreferential.
  • Conceptual distinction: CR spans AR, cataphora resolution, and split antecedent resolution, so a coreferential reference need not have an antecedent earlier in the discourse.The example “If he is unhappy with your work, the CEO will fire you” illustrates a coreferential but non-anaphoric pronoun.
  • Recommendations: The review recommends that datasets explicitly identify annotated and omitted coreference types and that models report exhaustive error analyses by reference type.These practices are intended to clarify the task and focus research on unresolved phenomena such as co-referent event resolution.

7 Coreference and Anaphora Resolution Datasets

AR and CR datasets vary substantially by domain, genre, language, annotation scheme, and reference types, making dataset understanding essential for evaluating research methods. The survey covers major general-purpose, multilingual, task-specific, and biomedical corpora.

  • Dataset characteristics: CR datasets differ in domain, annotation schemes, and the types of references they label, so these factors must be understood before selecting research methodologies.The survey emphasizes that datasets support either model training or rule development.
  • Major corpora: MUC, ACE, and OntoNotes are major CR corpora, while ACE primarily targets pronominal AR and uses the ACE-score metric.MUC-6 and MUC-7 were among the first sizable CR corpora and contain annotated Wall Street Journal articles.
  • Multilingual datasets: SemEval 2010 and CoNLL-2012 provide multilingual CR datasets, including Catalan, Dutch, German, Italian, Spanish, Chinese, and Arabic.CoNLL-2012 targeted multiple languages, while SemEval 2010 Task-1 preceded the CoNLL-2012 shared task.
  • Cross-domain and task-specific corpora: GNOME supports cross-domain AR and CR evaluation with museum, pharmaceutical, and tutorial dialogue texts, whereas ParCor targets parallel pronoun CR for machine translation.GNOME focuses on utterance labelling; ParCor contains German and English TEDx and Bookshop texts.
  • Web and knowledge-linked corpora: GUM and WikiCoref extend corpus coverage to richly annotated web texts and Wikipedia, respectively, with WikiCoref linking annotated entities to FreeBase.GUM includes conversational, instructional, and news texts; WikiCoref contains 30 annotated Wikipedia articles.
  • Biomedical datasets: Biomedical CR resources divide mainly into abstract and full-text annotation, with examples including MedCo, the BioNLP-2011 Protein Coreference task, and CRAFT.CRAFT contains 97 full-length open-access biomedical journal articles, while abstract resources restrict annotation primarily to biomedical entity types.

8 Reference Resolution Algorithms

Reference-resolution research progresses from knowledge-rich, hand-crafted rules and sieves toward neural models that learn mention and entity representations. The surveyed methods span syntax-, discourse-, pair-, cluster-, and end-to-end approaches, while raising coverage and domain-adaptation concerns.

  • Rule-based entity resolution: Knowledge-poor algorithms reduce reliance on external knowledge while retaining rule-based coreference resolution.CogNIAC is described as an early high-precision resolver using limited resources.
  • Rule-based entity resolution: Successive hierarchical sieves apply increasingly permissive rules, forming the architecture underlying Stanford CoreNLP’s deterministic coreference system.The sieve approach was motivated by the H and K model and developed through successive rule-based systems.
  • Rule-based entity resolution: Xrenner used dependency syntax, semantic and syntactic constraints, and antecedent filtering to target reference types omitted from CoNLL 2012 annotations.It addressed cataphora, compound modifiers, and i-within-i references, and outperformed Stanford CoreNLP and Berkeley systems on GUM and WSJ.
  • Rule-based entity resolution: Rule-based algorithms vary by linguistic basis: Hobbs is syntax-based, centering theory is discourse-based, and Lappin–Leass combines syntax, discourse, morphology, semantics, and knowledge resources.Lappin–Leass also ranks antecedents using weighted salience categories.
  • Statistical and neural entity resolution: Mention-pair and cluster-based models address coreference decisions by combining pairwise evidence with structured partitioning or entity-level information.Approaches include integer linear programming, recurrent cluster representations, and cluster ranking over mention-pair representations.
  • Deep learning models for CR: End-to-end neural coreference jointly models mention detection and resolution using minimal features, span extraction, embeddings, and a simple log-likelihood loss.The surveyed state-of-the-art system avoids an external mention extractor and heuristic loss while outperforming earlier models.

9 An analysis of entity resolution research progress on different datasets

Research progress is traced across the MUC, ACE, and CoNLL/OntoNotes datasets, showing advances in coreference systems alongside persistent evaluation challenges. The comparison emphasizes changing model families and difficulties caused by inconsistent datasets and metrics.

  • MUC: MUC was the first publicly available annotated corpus for coreference resolution, but its datasets were small despite widespread training and evaluation use.
  • ACE: ACE comparisons were difficult because test sets were unavailable to non-participants, prompting researchers to define their own train-test splits.
  • CoNLL Shared Task: The CoNLL progression moved from multi-sieve rule-based systems through mention-pair and cluster-feature models to entity-centric modeling with higher-order cluster information.
  • Evaluation challenges: Evaluation comparison was hindered by nonstandardized datasets, metrics, releases, and guidelines, especially for the early MUC and ACE corpora.

10 Open source tools

The review surveys open-source and off-the-shelf tools for applying entity resolution in NLP. It identifies tools spanning dedicated anaphora resolution, combined methodologies, and Stanford coreference models.

  • Applications: Open-source tools support applying anaphora and coreference resolution to tasks including question answering, summarization, sentiment analysis, and information retrieval.
  • Named tools: GuiTAR is described as a domain-dependent open-source anaphora resolution tool developed primarily for segmentation and summarization.
  • Tool comparison: The review includes an off-the-shelf entity resolution systems comparison alongside examples such as Reconcile and the Stanford coreference toolkit.

11 Reference Resolution in sentiment analysis

Reference resolution supports sentiment analysis by linking reviews and pronouns to the entities or aspects that carry opinions, extending analysis beyond sentence-level references. The survey reviews applications, methods, and reported improvements while noting that imperfect resolution can harm downstream sentiment analysis.

  • Applications: Global entity resolution helps infer the reviewed entity when product reviews omit it because the topic is understood from context.
  • Applications: Fine-grained aspect resolution links pronominal references to product aspects, aiding extraction of aspect-specific opinions.
  • Methods: Reference resolution contributes to aspect-opinion extraction through ontology-based and lightweight NLP procedures, followed by sentiment rating.
  • Applications: Coreference information can support sentiment analysis by identifying entities such as product names and later referring expressions.One evaluated approach reported an average 10% improvement over baseline coreference-resolution modules across datasets.
  • Evaluation: A modified CogNIAC-based coreference configuration achieved approximately 50% improvement for extracting opinion targets and opinion words.
  • Challenges: Entity resolution can lose or misassign opinion information when object and attribute coreference is unresolved, motivating features such as sentiment consistency.
  • Challenges: Integrating entity resolution into sentiment analysis remains challenging because imperfect resolution may be detrimental when incorporated incorrectly.

12 Reference Resolution: Issues and Controversies

The survey identifies evaluation metrics, dataset coverage, and commonsense knowledge as major controversies in entity resolution. It describes progress toward richer metrics while emphasizing unresolved reference types and the difficulty of modeling linguistic and world knowledge.

  • Major issues: Entity-resolution research debates evaluation metrics, dataset scope, and the induction of commonsense knowledge.
  • Evaluation metrics: Current coreference evaluation averages MUC, B3, and CEAF despite criticisms that these component metrics are faulty.
  • Evaluation metrics: LEA and modified existing metrics were proposed to address shortcomings in earlier coreference evaluation measures.
  • Dataset scope: The survey notes that standard datasets may not cover the diverse reference types found in natural language.
  • Commonsense knowledge: Some references remain difficult for machines because resolving them requires external world knowledge, including cases matching the Winograd Schema Challenge.
  • Commonsense knowledge: Modeling complex linguistic constraints in coreference systems remains an uphill battle.

13 Conclusion

The survey maps entity resolution by describing reference types, task boundaries, evaluation metrics, datasets, algorithmic trends, and its relationship with sentiment analysis. It concludes by identifying outstanding issues that can guide future research.

  • Scope and task boundaries: The survey details reference types and constraints to establish the broad scope of entity resolution.
  • Scope and task boundaries: It clarifies the boundaries between coreference resolution and anaphora resolution to support more focused research.
  • Evaluation and datasets: The paper compares commonly used evaluation metrics and catalogs available datasets, noting that state-of-the-art methods have not been evaluated on multiple datasets.
  • Research methods: The survey analyzes the shift from rule-based algorithms toward deep learning methods.
  • Sentiment analysis: It establishes background on the interdependency between anaphora resolution and sentiment analysis, where research at their intersection is scarce.
  • Future research: The paper states outstanding issues requiring attention as a foundation for future entity-resolution research.
Loading 1805.11824v1…