Source-linked AI summary
Coreference Resolution as Query-based Span Prediction
Wei Wu, Fei Wang, Arianna Yuan, Fei Wu, Jiwei Li
TL;DR
Coreference resolution must handle missed mention proposals and capture lexical, semantic, and syntactic context more deeply. CorefQA addresses these issues by turning resolution into query-based span prediction and leveraging question-answering data augmentation, achieving state-of-the-art results on GAP and CoNLL-2012. Its retrieval flexibility remains bounded because clusters with no successfully proposed mention cannot supply a query.
Problem
Existing coreference systems may permanently omit mentions missed during proposal and model mention interactions without deeply connecting them to surrounding context.
Method
CorefQA generates a context-based query for each candidate mention and uses a question-answering span predictor to extract coreferent spans.
Results
83.1 (+3.5) F1 on CoNLL-2012 and 87.5 (+2.5) F1 on GAP are reported as new state-of-the-art scores.
Takeaways & Limitations
The formulation supports retrieving some missed mentions, deeper context examination, and augmentation with existing question-answering datasets.
Takeaways & Limitations
If every mention in a coreference cluster is missed during proposal, none can be used to construct a query, so the cluster cannot be retrieved.
Abstract
from arXiv · showhide
In this paper, we present an accurate and extensible approach for the coreference resolution task. We formulate the problem as a span prediction task, like in machine reading comprehension (MRC): A query is generated for each candidate mention using its surrounding context, and a span prediction module is employed to extract the text spans of the coreferences within the document using the generated query. This formulation comes with the following key advantages: (1) The span prediction strategy provides the flexibility of retrieving mentions left out at the mention proposal stage; (2) In the MRC framework, encoding the mention and its context explicitly in a query makes it possible to have a deep and thorough examination of cues embedded in the context of coreferent mentions; and (3) A plethora of existing MRC datasets can be used for data augmentation to improve the model's generalization capability. Experiments demonstrate significant performance boost over previous models, with 87.5 (+2.5) F1 score on the GAP benchmark and 83.1 (+3.5) F1 score on the CoNLL-2012 benchmark.
1 Introduction
CorefQA reframes coreference resolution as query-based span prediction to address missed mentions and shallow context modeling. The formulation also enables question-answering data augmentation, and experiments report state-of-the-art benchmark results.
- Motivation: Existing systems can permanently lose mentions omitted during mention proposal, while their pairwise scoring captures contextual cues only superficially.Coreference datasets also provide weak supervision for singleton mentions because they are not explicitly labeled.
- Approach: CorefQA generates a context-based query for each candidate mention and uses span prediction to extract its coreferent mentions.This formulation is presented as analogous to question answering.
- Advantages: Span prediction can retrieve mentions left out during proposal, reducing the negative effect of undetected mentions.The benefit applies when at least one mention in a coreference cluster remains available as a query.
- Advantages: The question-answering formulation permits pre-training or augmentation with existing question-answering datasets, improving generalization and transferability.Coreference annotation is described as expensive, cumbersome, and often requiring linguistic expertise.
- Results: 83.1 (+3.5) F1 on CoNLL-2012 and 87.5 (+2.5) F1 on GAP establish new state-of-the-art scores.The experiments compare the proposed framework with previous models on two widely used datasets.
2 Related Work
The paper situates CorefQA within mention-ranking coreference systems and the broader trend of casting NLP tasks as reading comprehension. Its distinct contribution is using existing question-answering datasets for coreference data augmentation.
- Coreference Resolution: CorefQA is essentially a mention-ranking model, but identifies coreference using question answering rather than conventional antecedent scoring.The related-work discussion distinguishes entity-level and mention-ranking approaches.
- Question Answering: Machine reading comprehension offers a general, extensible task form that abstracts away task-specific modeling constraints.Prior work converted multiple NLP tasks into reading-comprehension formats.
- Question Answering: Unlike a concurrent coreference QA approach assuming gold mentions at inference, CorefQA jointly trains mention proposal and coreference resolution without that assumption.The comparison concerns inference-time mention availability and model training.
- Data Augmentation: Data augmentation increases the diversity of training data and has been explored across question answering, text classification, and dialogue understanding.The paper positions its use of QA data for coreference as distinct from prior coreference augmentation work.
- Data Augmentation: The authors identify their use of existing question-answering datasets for coreference augmentation as, to their knowledge, the first such application.Earlier coreference augmentation studies addressed other goals, including gender-bias debiasing and joint ellipsis modeling.
3 Model
CorefQA represents documents and spans, proposes candidate mentions, and uses speaker-aware contextual representations. Long documents are handled with independently encoded sliding-window segments.
- Notations: The model represents a document as tokens X and enumerates all possible text spans, with each span indexed by its start and end positions.The number of possible spans is defined as N = n ∗(n + 1)/2.
- Notations: Candidate spans are treated as potential mentions, linked to antecedents, and clustered through the links between spans.Non-entity or non-coreferent candidates receive a dummy antecedent token ϵ.
- Input Representation: SpanBERT supplies token representations, while speaker information is directly concatenated into the input rather than encoded only as pairwise binary features.The direct strategy is motivated by the importance of speaker information for coreference resolution.
- Input Representation: A sliding-window approach segments long documents into T-sized chunks, encodes them independently, and selects token representations with maximum context.Segments are created after every T/2 tokens.
3.3 Mention Proposal
The mention proposal module scores candidate spans using separate start, end, and span-validity signals, then prunes them for efficiency and relies on pretraining to improve proposal quality.
- Mention Proposal: Candidate spans up to maximum length L are scored as potential mentions and greedily pruned during training and evaluation.The score combines whether a span starts, ends, and forms a valid span.
- Mention Proposal: Three separately parameterized feed-forward networks score span starts, span ends, and the combination of both endpoints.The overall mention score averages these three components.
- Mention Proposal: The model retains at most λn spans with the highest mention scores, where n is the document length.This pruning controls the number of proposed spans passed onward.
- Mention Proposal: The mention proposal model is pretrained with three binary classifiers for span starts, span ends, and endpoint combination.This pretraining is described as crucial because otherwise most proposed mentions entering linking are invalid.
3.4 Mention Linking as Span Prediction
Mention linking is formulated as question answering: a candidate mention supplies a context-based query, and span prediction scores possible coreferent spans, including a bidirectional relation.
- Mention Linking as Span Prediction: For each proposed mention ei, the linking network scores every text span ej for whether ei and ej are coreferent.The document is the context, while the sentence containing ei becomes a query with ei marked by special tokens.
- Mention Linking as Span Prediction: The question-answering backbone uses the document as context, a marked sentence as query, and coreferent mentions as answers.Queries are unanswerable when the candidate is not an entity mention or has no coreferent mention.
- Mention Linking as Span Prediction: The span score sa(j|i) uses BERT representations of span j's first and last tokens, with the candidate mention query concatenated to the context.A feed-forward network maps these endpoint representations to the mention score.
- Mention Linking as Span Prediction: The final linking score combines directional scores sa(j|i) and sa(i|j) to model the bidirectional relation between two mentions.The paper motivates this symmetry because coreference should hold in both directions.
- Mention Linking as Span Prediction: A hyperparameter λ controls the tradeoff between mention proposal and mention linking.
3.5 Antecedent Pruning
Because scoring all span pairs is computationally expensive, the model prunes candidates for each query before performing the full linking computation.
- Antecedent Pruning: Scoring all mention pairs has O(n^4) complexity, so a further pruning procedure is required.Even for an extracted mention ei, backward span prediction requires running question answering models on all queries q(ej).
- Antecedent Pruning: For each query q(ei), the model collects C span candidates to limit subsequent antecedent scoring.
3.6 Training
Training optimizes antecedent selection over pruned candidates, while the mention proposal and linking modules are trained jointly end to end.
- Training: For each proposed mention ei, training maximizes the marginal log-likelihood of all correct antecedents implied by the gold clustering.A dummy token ϵ is appended so the model can select no coreferent candidate when appropriate.
- Training: The mention proposal and mention linking modules are jointly trained end to end with shared SpanBERT parameters.
3.7 Inference
CorefQA decodes mention clusters from an undirected graph whose nodes are candidate mentions and whose edge weights come from overall coreference scores. Its question-answering formulation can recover some mentions missed during proposal, but not clusters with no proposed query mention.
- Graph-based decoding: CorefQA constructs an undirected graph in which nodes are candidate mentions and edge weights are overall coreference scores.Edges are pruned by retaining each node’s largest-weight edge, and nodes linked most closely to the dummy token are abandoned before decoding clusters.
- Mention recovery: The question-answering formulation can retrieve mentions omitted by the mention proposal stage during mention linking.A missed mention can be recovered when another mention from its cluster remains available to construct a query.
- Mention recovery: Clusters remain irreversibly missed when every mention in the cluster is absent from the proposal stage.The framework still depends on at least one proposed mention per cluster for query construction.
4 Experiments
Experiments evaluate CorefQA against end-to-end neural baselines on CoNLL-2012 and GAP, reporting state-of-the-art performance on both benchmarks. The CoNLL-2012 evaluation averages MUC, B3, and CEAFφ4 F1 scores.
- Baselines: Baselines include end-to-end systems using GloVe, ELMo, BERT-large, cluster representations, and SpanBERT-large.The comparison excludes ensemble models for fairness.
- Datasets and metrics: The English CoNLL-2012 data contains 2,802 training, 343 development, and 348 test documents across seven genres.Its main evaluation is the average of MUC, B3, and CEAFφ4 on the test set.
- CoNLL-2012: 83.1 F1 on CoNLL-2012 is a 3.5-point improvement over the previous state-of-the-art system.With SpanBERT-base, CorefQA reaches 79.9 F1 and exceeds the previous SpanBERT-large state of the art by 0.3.
- GAP: GAP is gender-balanced and contains 8,908 labeled pairs of ambiguous pronouns and antecedent names sampled from Wikipedia.Evaluation uses an off-the-shelf resolver trained on CoNLL-2012 under established protocols.
5 Ablation Study and Analysis
Ablations and analyses show contributions from SpanBERT, mention-proposal pre-training, QA pre-training, query-based span prediction, and speaker-name inputs. Qualitative examples illustrate gains in long-distance and conversational coreference.
- Ablation results: Replacing SpanBERT with vanilla BERT causes a 3.5 F1 degradation.This supports the contribution of span-level pre-training to coreference performance.
- Ablation results: 7.2 F1 degradation occurs when mention-proposal pre-training on golden mentions is skipped.The authors attribute this to randomly selected mentions producing mostly unanswerable queries early in training.
- Ablation results: QA pre-training contributes 0.7 F1 from Quoref and 0.3 F1 from SQuAD.These results demonstrate the usefulness of existing QA datasets for data augmentation.
- Ablation results: 8.1 F1 degradation occurs when query-based span prediction is replaced with mention-pair scoring.The ablation isolates the performance difference between the two paradigms while keeping the remaining components unchanged.
- Speaker analysis: Speaker-name input performs significantly better than speaker-as-feature modeling on documents with more speakers.The proposed strategy directly concatenates speaker names with utterances, whereas the baseline uses binary same-speaker features.
- Mention recall: CorefQA consistently exceeds the baseline in mention recall across values of λ and is less sensitive to small λ.Missed mentions can still be retrieved during mention linking, reducing dependence on the number of retained spans.
- Qualitative analysis: Qualitative cases show successful long-distance antecedent identification and conversational resolution using local context and speaker names.One example identifies Freddie Mac over a longer distance; another resolves “I” to Thelma Gutierrez using speaker information.
6 Conclusion
CorefQA formulates anaphora identification as query-based span prediction and demonstrates that this formalization can recover omitted mentions while enabling question-answering data augmentation.
- CorefQA casts anaphora identification as query-based span prediction in question answering.
- The proposed formalization can retrieve mentions left out at the mention proposal stage.
- The question-answering formulation makes data augmentation using existing question-answering datasets possible.
- A new speaker modeling strategy can boost performance in dialogue settings.
- Empirical results on two widely used coreference datasets demonstrate the model's effectiveness.