Source-linked AI summary
Learning Global Features for Coreference Resolution
Sam Wiseman, Alexander M. Rush, Stuart M. Shieber
TL;DR
Coreference systems need global cluster information, but informative cluster-level features are difficult to design despite strong local baselines. The paper learns latent cluster representations from mentions with recurrent neural networks and integrates them into an end-to-end mention-ranking system. The resulting system improves over state of the art by over 0.8 CoNLL points and does so without additional search.
Problem
Global information could benefit coreference prediction, but informative cluster-level features are difficult to craft, while local mention-ranking systems can already achieve state-of-the-art performance.
Method
The model sequentially embeds cluster mentions with LSTMs and incorporates the learned global representations into an end-to-end mention-ranking system.
Results
Over 0.8 CoNLL points are gained over the current state of the art, with statistically significant improvement on all three CoNLL metrics.
Takeaways & Limitations
Learned cluster representations are especially useful for resolving difficult pronominal mentions and support simple greedy inference in the end-to-end system.
Takeaways & Limitations
Training on oracle clusters creates a mismatch between training and test that can hurt performance.
Abstract
from arXiv · showhide
There is compelling evidence that coreference prediction would benefit from modeling global information about entity-clusters. Yet, state-of-the-art performance can be achieved with systems treating each mention prediction independently, which we attribute to the inherent difficulty of crafting informative cluster-level features. We instead propose to use recurrent neural networks (RNNs) to learn latent, global representations of entity clusters directly from their mentions. We show that such representations are especially useful for the prediction of pronominal mentions, and can be incorporated into an end-to-end coreference system that outperforms the state of the art without requiring any additional search.
1 Introduction
The paper argues that global context is needed for further coreference improvements, but manually designing informative cluster-level features is difficult. It therefore learns cluster representations with an RNN and integrates them into an end-to-end mention-ranking system that improves over state of the art.
- State-of-the-art coreference performance can be achieved with a completely local mention-ranking system despite the promise of structured non-local models.
- Informative cluster-level features are difficult to devise, limiting the effectiveness of global coreference modeling.
- The model learns global representations from the mentions in each entity cluster and incorporates them into a mention-ranking system.
- The entire recurrent and mention-ranking model is trained end-to-end as a local classifier with fixed context.
- Over 0.8 CoNLL points are gained over the current state of the art, with statistically significant improvement on all three CoNLL metrics.
2 Background and Notation
Coreference resolution partitions document mentions into clusters, while mention-ranking systems simplify inference by predicting one antecedent independently for each mention. The paper seeks to retain this efficiency while allowing predictions to use global information about previously induced clusters.
- Coreference resolution partitions intra-document mentions into clusters whose members refer to the same underlying entity.
- A valid clustering assigns every mention to exactly one cluster and can be represented by a vector z over cluster labels.
- Mention-ranking systems predict one antecedent or a dummy antecedent ϵ for each mention instead of predicting an entire clustering.
- Mention ranking is attractive because it performs efficient left-to-right inference using local antecedent scores.
- The proposed objective preserves mention-ranking simplicity while allowing a global function to examine the clustering induced by earlier antecedent predictions.
3 The Role of Global Features
Pronouns are a substantial source of mention-ranking errors, and their resolution can depend on non-local cluster consistency. Existing global features have mixed results because fixed-length cluster features are either coarse or sparse, motivating learned representations.
- Pronominal mentions account for almost 59% of mention-ranking precision errors and almost 24% of recall errors on the CoNLL 2012 English development set.
- Local pronoun linking can create incoherent clusters when nearby pronouns differ in properties such as number and coreference structure.
- Over 70% of pleonastic “it” and over 74% of pleonastic “you” instances follow previous instances of the same pronoun in the CoNLL training data.
- Non-local feature results are mixed across prior systems, with some studies reporting improvements and others finding no benefit.
- Cluster-level features are difficult to define because clusters vary in size or shape, making representations either too coarse or too sparse.
- Concatenating mention-level features can be expressive but often produces sparse features because clusters may have any size.
4 Learning Global Features
The paper learns global cluster representations by running recurrent networks over ordered mention sequences, replacing manually designed cluster features. The learned states capture entity-type and subtler mention relationships.
- Learned structure: The learned representations roughly separate clusters by predominant mention type and number while capturing geographic and pronoun-string relationships.These patterns are illustrated with a T-SNE visualization of gold clusters on the CoNLL development set.
- Recurrent architecture: The system uses LSTMs in all experiments as its recurrent architecture.The paper identifies LSTMs as a common NLP variety of recurrent neural network.
- Integration: The resulting RNN cluster representations provide the basis for the global term in a mention-ranking coreference system.Clusters are treated as sequences ordered by their mentions’ document appearance.
- Mention encoding: A nonlinear mention embedding maps local indicator features, including head word and gender, into vectors consumed by the cluster RNN.The embedding is parameterized by W c and bc.
- Cluster representations: RNN hidden states represent partial entity clusters after consuming their mentions sequentially.The model maintains one parameter-sharing RNN per cluster and produces a hidden state at each mention step.
5 Coreference with Global Features
The model extends mention ranking with recurrent, cluster-level representations that contribute global scores while preserving simple end-to-end training and greedy inference. Each cluster’s hidden state is updated from its mentions and used for antecedent and non-anaphoric decisions.
- Global representations: Cluster RNN states provide learned global representations that augment local mention-ranking scores for coreference decisions.The system maintains one shared-parameter RNN per cluster and uses cluster states in global scoring.
- Global scoring: Non-anaphoric predictions receive a separate score based on the current hidden states of all clusters.This term can incorporate information about potential antecedents and singleton-cluster patterns.
- Global scoring: For an antecedent, the global score measures how well the current mention matches the antecedent cluster’s existing representation.The matching score is expressed as a dot-product between the mention representation and the antecedent cluster state.
- Training: The recurrent network and mention-ranking subsystem are trained end-to-end as a local classifier with fixed context, avoiding complicated training-time inference.Using oracle clusters during training permits precomputation of hidden states, but creates a train–test mismatch that can hurt performance.
- Global scoring: Dot-product global matching performed best among the nonlinear functions tested.This comparison is reported as an experimental design choice for the antecedent-cluster matching term.
- Search: The model uses greedy test-time search with O(n^2) time, matching the local mention-ranking objective’s complexity rather than solving the full NP-hard clustering problem.The greedy algorithm updates the selected cluster’s RNN state after each decision and uses that representation in subsequent scoring.
6 Experiments
Experiments evaluate learned global cluster representations on CoNLL 2012 English, comparing RNN-based models with baselines and oracle-history settings. The RNN improves overall performance, especially for pronominal mentions, while greedy inference retains a significant portion of the oracle-history gain.
- Experimental setup: The experiments use the CoNLL 2012 English shared-task split from OntoNotes, comprising 3,493 documents across various domains and formats.Mention extraction and features φa and φp come from the Berkeley Coreference System.
- Main results: The system achieves a statistically significant improvement of over 0.8 CoNLL points over the previous state of the art and the highest F1 scores on all three CoNLL metrics.The comparison uses the CoNLL 2012 English test set; gains over Wiseman et al. (2015) are significant for all metrics.
- Methods: The model combines learned cluster representations with mention ranking and trains the recurrent and mention-ranking components end-to-end as a local classifier with fixed context.This training procedure avoids the complicated inference required by approaches that search over latent structures during training.
- Results by mention type: The RNN improves overall performance most dramatically on non-anaphoric pronouns, while also significantly reducing errors for certain nominal and proper mentions following a same-head mention.Wrong-link errors decrease in these categories, but false-new errors increase; performance remains significantly better than the averaged-history baseline.
- Global representations: The RNN outperforms the averaged-history baseline, indicating that modeling the sequence of mentions in a cluster is advantageous.Greedy inference degrades precision and recall relative to oracle history but recovers a significant portion of the possible improvement.
- Qualitative analysis: Qualitative examples show the greedy RNN correcting baseline errors for ambiguous “his” and pleonastic “It’s” through cluster-level information.Figure 3 visualizes cluster-association scores, while Figure 4 visualizes gradients of the NA score with respect to preceding-cluster mentions.
7 Related Work
Related work distinguishes local mention-based and structured coreference systems, while positioning recurrent modeling of cluster-state trajectories as a novel use of RNNs for coreference.
- Coreference approaches: Prior systems include mention-pair, mention-ranking, cluster-ranking, and hybrid approaches that combine mention-level predictions with clustering.These approaches differ in whether they classify mention pairs, select antecedents, rank clusters, or combine multiple system outputs.
- RNN-based representations: Applying RNNs to model the trajectory of a cluster’s state is presented as apparently novel, with some similarity to LSTM representations of parser-stack states.The comparison is to Dyer et al. (2015).
8 Conclusion
The paper presents an end-to-end approach that incorporates global information through learned cluster representations rather than manually defined global features. It also supports simple greedy inference, while identifying recall and more sophisticated global training as future directions.
- Conclusion: The approach incorporates global information without manually defining global features and permits simple greedy inference.The conclusion characterizes the approach as simple and state of the art.
- Future work: Future work will examine improving recall and developing more sophisticated approaches to global training.