Source-linked AI summary
Document-Level $N$-ary Relation Extraction with Multiscale Representation Learning
Robin Jia, Cliff Wong, Hoifung Poon
TL;DR
Document-level n-ary extraction is needed because higher-order biomedical relations can span distant parts of long documents, while prior cross-sentence methods use short text windows. The paper proposes an entity-centric multiscale architecture that combines representations across text spans and subrelations, and it substantially outperforms prior methods. Its recall remains bounded when entities are absent from the document or occur only in figures, supplements, indirect cell-line references, or unresolved links.
Problem
Higher-order biomedical relations often span distant document locations, whereas prior cross-sentence extraction is confined to short spans, limiting coverage.
Method
The paper uses an entity-centric multiscale architecture that scores document-level candidates and combines representations across text spans and subrelations.
Results
The method substantially outperforms prior n-ary relation extraction approaches, with an 8.4 AUC-point improvement over DPL in the best CKB test setting.
Takeaways & Limitations
Combining document-level candidate scoring with multiscale representations increases potential recall while preserving precision across text spans and the subrelation hierarchy.
Takeaways & Limitations
Maximum recall remains below 100% because over 20% of relations lack all three entities in the document, often because entities occur in figures or supplements.
Abstract
from arXiv · showhide
Most information extraction methods focus on binary relations expressed within single sentences. In high-value domains, however, $n$-ary relations are of great demand (e.g., drug-gene-mutation interactions in precision oncology). Such relations often involve entity mentions that are far apart in the document, yet existing work on cross-sentence relation extraction is generally confined to small text spans (e.g., three consecutive sentences), which severely limits recall. In this paper, we propose a novel multiscale neural architecture for document-level $n$-ary relation extraction. Our system combines representations learned over various text spans throughout the document and across the subrelation hierarchy. Widening the system's purview to the entire document maximizes potential recall. Moreover, by integrating weak signals across the document, multiscale modeling increases precision, even in the presence of noisy labels from distant supervision. Experiments on biomedical machine reading show that our approach substantially outperforms previous $n$-ary relation extraction methods.
1 Introduction
Document-level n-ary extraction is motivated by relations whose entity mentions span distant parts of biomedical articles, beyond the short windows used by prior cross-sentence methods. The proposed multiscale architecture expands coverage across the document while combining text-span and subrelation signals.
- Biomedical literature contains drug-gene-mutation relations expressed across multiple paragraphs, making document-level extraction necessary.
- Prior n-ary extraction methods generally restrict cross-sentence processing to short spans such as three consecutive sentences.Documents may contain hundreds of sentences and tens of thousands of words.
- The proposed architecture expands candidate extraction to the entire document rather than restricting it to entities co-occurring in short spans.This maximizes potential recall while addressing document-level extraction difficulties such as low precision.
- Multiscale learning combines representations from text spans of varying scales and from subrelations, including cases where all entities never co-occur in one discourse unit.Subrelations can be expressed in separate parts of the document and support prediction of the higher-order relation.
- The approach combines weak signals across text spans and subrelations and substantially outperforms prior n-ary relation extraction methods.On a large benchmark dataset, ablations identify multiscale modeling as the key to these gains.
2 Document-Level N-ary Relation Extraction
Document-level n-ary extraction addresses relations involving more than two entities across full documents, where mention-level candidate enumeration becomes impractical. The paper instead scores entity-level candidates and represents their subrelations.
- Document-level n-ary extraction targets relations with n > 2 in documents that may contain hundreds of sentences.The task supports applications such as identifying drug relevance for patients with mutations in particular genes.
- Entity-centric formulation avoids the large number of irrelevant mention tuples created when entities are mentioned repeatedly in a document.The mention-centric formulation does not scale well to document-level extraction.
- A candidate is an (n + 1)-tuple of entities and a text span, with each entity mentioned at least once in that span.The model predicts whether the entity tuple expresses the target relation.
3 Our Approach: Multiscale Representation Learning
The approach builds document-level n-ary relation predictions by combining mention representations across discourse units, entity subsets, and aggregation scales. It supports distant supervision and document-wide evidence while addressing missing co-occurrences and noisy individual signals.
- Multiscale representation learning: The framework constructs mention-level representations within discourse units, aggregates them into entity-level representations across the document, and predicts relations from their concatenation.It supports sentence- and paragraph-level discourse units and represents both full relations and subrelations.
- Aggregation: Logsumexp aggregation accumulates weak signals from ambiguous mention tuples and substantially improves extraction accuracy compared with max pooling.The method is motivated by mention tuples that are individually uncertain but collectively express a document-level relation.
- Subrelations: Subrelation representations capture evidence for entity subsets when higher-order relations are not fully expressed within one discourse unit.The model constructs representations for every entity subset of size at least two, including the complete relation.
- Document model: The document model varies by discourse units, subrelations, and aggregation operators, and can also ensemble predictions from restricted models operating on individual units.The paper explores sentences and paragraphs, max and logsumexp aggregation, and ensemble operators including max and noisy-or.
- Training: Entity-centric distant supervision is used because entity-level supervision is less noisy than mention-level supervision and avoids requiring sophisticated denoising strategies.The model can also be trained with standard supervised or indirectly supervised methods.
4 Experiments
Experiments evaluate MULTISCALE on biomedical drug-gene-mutation extraction using CKB and compare it with restricted variants and DPL. MULTISCALE improves precision, recall potential, and cross-paragraph extraction, while results also expose limits from missing entities and curation differences.
- Evaluation setup: CKB CORE provides a manually curated document-level test set with minimal overlap with distant-supervision knowledge bases.Training documents were removed from CKB entries before development and test splitting.
- Evaluation setup: The evaluation reports AUC, maximum recall, and thresholded precision, recall, and F1.AUC uses average precision, while maximum recall measures the fraction of true facts for which a candidate was generated.
- Main results: 8.4 AUC points: MULTISCALE improves over DPL in the best setting using noisy-or and the gene-mutation filter.Across scenarios, the full model substantially outperforms DPL; expanding from sentences to paragraphs provides the largest restricted-variant gain.
- Main results: About 20 absolute points higher maximum recall: MULTISCALE and DOCLEVEL exceed PARALEVEL and DPL in candidate coverage.The authors caution that thresholded actual-recall differences are less pronounced because thresholds maximize F1.
- Aggregation analysis: The gene-mutation filter raises recall while maintaining respectable precision, especially for document-level models that otherwise sacrifice recall for F1.Changing the ensemble operator from max to noisy-or also produces a substantial gain.
- Main results: MULTISCALE generally improves precision while capturing more cross-paragraph relations, and all three ensemble variants contribute to performance.Removing SENTLEVEL, PARALEVEL, or DOCLEVEL is used to assess the ensemble components.
- Aggregation analysis: Replacing logsumexp with max reduces MULTISCALE AUC by 3.8 absolute points, supporting aggregation of weak signals across mentions.The performance drop occurs broadly across the evaluated systems.
- Cross-sentence and cross-paragraph extractions: MULTISCALE extracts relations whose entities never co-occur in a sentence or paragraph, including cases linked through patient cohorts, drug effects, and bridging anaphora.Manual inspection of twenty correct cross-paragraph extractions identified these linguistic phenomena.
5 Related Work
Prior work spans n-ary relation extraction, document-level relation extraction, multiscale modeling, and distant supervision, but this paper combines whole-document extraction with subrelation hierarchy modeling.
- N-ary relation extraction: N-ary relation extraction often reduces higher-order relations to binary relations between a reified relation and its arguments, known as slot filling.
- Document-level relation extraction: Document-level extraction work has largely remained sentence-level or relied on restrictive assumptions such as explicit coreference or a single event per document.
- Document-level relation extraction: Entity-level representations aggregate mention-level information, but prior approaches focus on binary relations or shorter documents such as abstracts.
- Multiscale modeling: Multiscale modeling has mainly supported generative learning, whereas this paper applies it to discriminative relation extraction across sentence, paragraph, and document scales.
- Multiscale modeling: The approach models the entire subrelation hierarchy and extracts from whole documents rather than single sentences, extending related clique-based methods.
- Distant supervision: Distant supervision generates large but potentially noisy labeled datasets, with noise increasing as extraction expands beyond single sentences.
6 Conclusion
The paper concludes that its multiscale, entity-centric document-level approach improves recall while preserving precision for n-ary relation extraction in precision oncology. It also identifies multimodal integration and higher-order relations as future directions.
- The proposed approach scores document-level candidates to increase maximum recall while combining subrelation and multiscale span representations to preserve precision.
- The method substantially outperforms prior cross-sentence n-ary relation extraction approaches in precision oncology.
- The document-level view could support multimodal learning by integrating information from tables and figures.
- The framework could extend beyond drug-gene-mutation relations to higher-order relations containing fields such as effect, cancer type, and evidence.
- The paper identifies validation in assisted curation, where systems propose candidate facts for human verification, as a future direction.
A.1 Preprocessing
The preprocessing pipeline obtains full-text documents from PMC, tokenizes them, and extracts drug, gene, and mutation entities using rules and dictionaries.
- Full-text documents are obtained from PMC and tokenized using NLTK.
- Mutation mentions are extracted with a regular expression rule for missense mutations.
- Gene mentions are identified through lookup in the HUGO Gene Nomenclature Committee dataset.
- Drug mentions are identified using a curated drug list and its synonyms.
A.2 Gene-mutation Rule-based System
The rule-based system builds mutation-gene associations from external resources and document evidence, then applies increasingly flexible textual rules for remaining mutations.
- The system builds a global mutation-to-gene map from COSMIC, COSMIC Cell Lines Project, CIViC, and OncoKB datasets.
- It augments the map by selecting the gene most frequently co-occurring with each mutation in PMC full-text articles under three high-precision rules.
- The first rule matches a gene and mutation in the same token, such as EGFR-T790M.
- The second rule matches a gene token immediately followed by a mutation token, such as EGFR T790M.
- The third rule matches a gene, a single-character token, and then a mutation, such as EGFR - T790M.
- For each mutation, the system uses the first rule yielding a match and selects the gene occurring in the most matches.
- Within a document, a mutation is associated with the mapped gene that appears closest to any mutation mention.
- Remaining mutations are handled with two recall-friendly regular-expression rules applied within the document.
4. Mutation is in same sentence as “GENE mut”
The method selects a gene for each mutation using ordered document-level matching rules, then falls back to the document’s most frequent gene. Its sequence model combines word vectors with discourse-unit index embeddings and processes them using a bidirectional LSTM.
- The method first selects the earliest gene satisfying one of two ordered matching rules for each mutation.
- If no gene matches either rule, it selects the most frequent gene in the document for that mutation.
- The BiLSTM receives concatenated 200-dimensional biomedical word vectors and 100-dimensional discourse-unit index embeddings.The word vectors are initialized from biomedical word2vec vectors and updated during training; the index embeddings are sinusoidal.
B Corrections
A source-code error introduced minor incorrectness in several reported figures and tables, but the authors state that the article’s conclusions remain unaffected. Corrected versions are provided, including analyses of precision-recall behavior, recall scope, ensemble variants, and aggregation choices.
- A source-code error caused minor incorrectness in Figures 3–4 and Tables 3–5, without affecting the article’s conclusions.The authors provide corrected figures and tables below.
- MULTISCALE generally achieves better precision than PARALEVEL and higher maximum recall like DOCLEVEL on CKB.
- Cross-sentence and cross-paragraph extractions contribute importantly to MULTISCALE’s recall.
- Table 3 compares the multiscale system with restricted variants and DPL.
- Table 4 evaluates removing SENTLEVEL, PARALEVEL, or DOCLEVEL from the MULTISCALE ensemble using maximum recall, precision, and recall.
- Table 5 evaluates replacing logsumexp with max under the noisy-or and gene-mutation filter, reporting precision and recall.