Source-linked AI summary
Distant Supervision for Relation Extraction beyond the Sentence Boundary
Chris Quirk, Hoifung Poon
TL;DR
Existing distant-supervision methods largely restrict relation extraction to single sentences, leaving cross-sentence relations under-explored. DISCREX uses a document-level graph with multiple-path features to model entity relations within and across sentences. In biomedical experiments, it doubled extraction yield at similar accuracy and expanded the knowledge base by two orders of magnitude.
Problem
Distant supervision has been limited to single-sentence relations, leaving cross-sentence extraction under-explored despite the need to integrate scattered biomedical knowledge.
Method
DISCREX uses a document-level graph combining syntactic, adjacency, discourse, and coreference relations, with features extracted from multiple paths.
Results
Cross-sentence extraction doubled the yield at similar accuracy, while DISCREX expanded GDKD coverage by two orders of magnitude without annotated examples.
Takeaways & Limitations
Cross-sentence relations are prevalent in biomedical literature, and distant supervision can extract them at high precision from a small knowledge base and unlabeled text.
Takeaways & Limitations
Future work includes improved integration with coreference and discourse parsing, active learning, crowdsourcing, and evaluation in other domains.
Abstract
from arXiv · showhide
The growing demand for structured knowledge has led to great interest in relation extraction, especially in cases with limited supervision. However, existing distance supervision approaches only extract relations expressed in single sentences. In general, cross-sentence relation extraction is under-explored, even in the supervised-learning setting. In this paper, we propose the first approach for applying distant supervision to cross- sentence relation extraction. At the core of our approach is a graph representation that can incorporate both standard dependencies and discourse relations, thus providing a unifying way to model relations within and across sentences. We extract features from multiple paths in this graph, increasing accuracy and robustness when confronted with linguistic variation and analysis error. Experiments on an important extraction task for precision medicine show that our approach can learn an accurate cross-sentence extractor, using only a small existing knowledge base and unlabeled text from biomedical research articles. Compared to the existing distant supervision paradigm, our approach extracted twice as many relations at similar precision, thus demonstrating the prevalence of cross-sentence relations and the promise of our approach.
1 Introduction
Distant supervision has largely been limited to single sentences, leaving cross-sentence relations under-extracted despite their importance in specialized domains. DISCREX addresses this gap with a document graph and multiple-path features, and experiments show substantially greater extraction yield.
- Motivation: Traditional extraction requires annotated examples, making it difficult to scale knowledge integration across millions of biomedical articles.The paper motivates indirect supervision, including distant supervision, for precision medicine.
- Motivation: Cross-sentence relations can require evidence from multiple sentences and cannot be inferred from either sentence alone.The example connects Dasatinib, resistance, and Notch-related biology across sentence boundaries.
- Approach: DISCREX is presented as the first distant-supervision approach for relation extraction beyond the sentence boundary.Its document-level graph combines intra-sentential dependencies with adjacent-sentence and discourse relations.
- Approach: Multiple graph paths improve robustness to linguistic variation and parser errors by exposing alternative structural patterns.The graph provides a unified feature space for classifying entity-pair relations.
- Results: Excluding non-minimal candidates substantially improves classification accuracy in the cross-sentence setting.Minimal-span candidates address overlapping co-occurrences that can otherwise create incorrect examples.
2 Related Work
Prior relation-extraction systems generally focus on single sentences, while cross-sentence extraction remains little studied. DISCREX differs through distant supervision, a general document graph, and large-scale biomedical evaluation.
- Single-sentence limitation: Earlier distant-supervision systems aggregate evidence across sentential instances but cannot extract relations whose entities occur in different sentences.Their instances still require both entities to co-occur within a sentence.
- Research gap: Cross-sentence relation extraction has received little attention, even in supervised learning.The paper identifies this as a broader gap beyond distant supervision.
- Scope: DISCREX combines a more general document graph with distant supervision and evaluates large-scale extraction on about one million full-text articles.The comparison is made against supervised, single-sentence, and smaller-scale prior settings.
- Graph features: Prior work typically uses the shortest path within a sentence, whereas DISCREX allows multiple paths that interleave dependency and adjacency edges.The contrast concerns both sentence scope and path selection.
3 Distant Supervision for Cross-Sentence Relation Extraction
DISCREX extends distant supervision to cross-sentence entity pairs using a document graph containing syntactic, adjacency, discourse, and coreference links. It selects minimal-span candidates and extracts features over multiple graph paths to improve robustness.
- Distant supervision: DISCREX learns a relation classifier from an existing knowledge base and unlabeled text, allowing candidate spans to comprise multiple sentences.Known co-occurring pairs supply positive examples, while unknown pairs are sampled as negatives.
- Minimal-Span Candidates: Minimal-span candidates retain overlapping co-occurrences only when no shorter span contains the same entity pair.Distance is measured by the number of consecutive sentences between entity mentions.
- Document Graph: The document graph represents words as nodes and combines dependency, word-adjacency, sentence-adjacency, discourse, and coreference edges.Adjacent-sentence roots receive NEXTSENT-style connections, while anaphors connect to antecedents.
- Coreference Baseline: Coreference augmentation added 2% more positive candidates, but only 1% were cross-sentence, producing little additional candidate coverage.This alternative still performs extraction within single sentences.
- Features: DISCREX defines features over paths whose nodes and edges may interleave dependency, adjacency, and discourse relations.Features include whole-path indicators and sliding path n-grams of lengths 1–5.
- Multiple Paths: Multiple paths provide alternative routes between entities, helping accommodate noisy language and parser errors.The system varies the number of shortest paths and benefits from penalizing, but retaining, adjacency edges.
4 Experiments
Experiments show that DISCREX can extract cross-sentence drug-gene interactions at similar precision to single-sentence extraction while substantially increasing recall. Multiple graph paths and penalized adjacency edges improve accuracy, whereas discourse relations slightly reduce performance.
- Candidate Selection: K = 3 doubled the candidate count while retaining a limited sentence window; minimal-span candidates reduced unlikely matches.There were 87,773 GDKD-matching instances with minimal spans, compared with 225,520 without the restriction.
- Automatic Evaluation: Cross-sentence extraction achieved similar accuracy to single-sentence extraction despite substantially higher recall.The comparison used five-fold cross-validation with accuracy as the reported metric.
- Path Features: Adding more than the shortest path substantially improved accuracy for both single-sentence and cross-sentence extraction.The improvement was consistent across the two extraction settings.
- Discourse Features: Adding discourse relations consistently caused a small performance drop, especially when few paths were used.Manual inspection attributed many errors to biomedical coreference mistakes, while the effect of discourse parsing remained uncertain.
- Edge Weights and Paths: Penalizing adjacent-word edges produced large accuracy gains, while increasing the number of paths yielded a consistent and substantial increase in accuracy.The results support treating dependency edges as more meaningful while retaining adjacency edges as a back-off for parser errors.
- PubMed-Scale and Manual Evaluation: Cross-sentence extraction increased absolute recall by 89-102% and produced about 37,000 correct unique interactions at threshold 0.5.At threshold 0.9, it produced about 20,000 correct unique interactions; both settings expanded the knowledge base by two orders of magnitude.
5 Conclusion
DISCREX applies distant supervision to cross-sentence relation extraction using a document-level graph that combines intra- and inter-sentential relations. It doubles unique-interaction yield at similar accuracy and expands GDKD coverage by two orders of magnitude without annotated examples.
- DISCREX applies distant supervision to cross-sentence relation extraction through a document-level graph combining dependencies, adjacency, and discourse relations.
- Cross-sentence extraction doubled the yield of unique interactions while maintaining the same accuracy.
- Two orders of magnitude: DISCREX improved GDKD coverage without requiring annotated examples.
- Future work includes improved coreference and discourse parsing, additional feature exploration, and evaluation in other domains.