Reasoning with Latent Structure Refinement for Document-Level Relation Extraction

• Document-level extraction infers entity relations by synthesizing relevant information across multiple sentences.Source: §1 · p. 1
• The model uses iterative refinement to incrementally capture complex interactions and perform multi-hop reasoning.Source: §1 · p. 2
• Experiments report significant improvements over existing approaches on DocRED and new state-of-the-art results on two biomedical datasets.Source: §1 · p. 2
• LSR induces a document-level graph end-to-end without relying on co-references or rules, then dynamically refines it for information aggregation.Source: §1 · p. 2

• LSR has three components: a node constructor, dynamic reasoner, and classifier.Source: §2 · p. 2
• The node constructor encodes each sentence and extracts mention and shortest-dependency-path token representations as graph nodes.Source: §2 · p. 2
• The dynamic reasoner induces a document-level structure from the extracted nodes and updates their representations.Source: §2 · p. 2

• A context encoder produces contextualized representations for each word in every document sentence.Source: §2.1.1 · p. 2
• The document-level graph therefore contains mention nodes, entity nodes, and meta dependency path nodes.Source: §2.1.2 · p. 3
• Mention nodes represent entity mentions, while entity nodes are formed by averaging the representations of their mentions.Source: §2.1.2 · p. 3, Figure 2 · p. 3
• Meta dependency path nodes represent tokens on shortest dependency paths between mentions within a sentence.Source: §2.1 · p. 2, §2.1.2 · p. 3

Dynamic Reasoner

• The dynamic reasoner alternates structure induction with multi-hop reasoning and stacks N blocks for iterative graph refinement.Each block induces a structure, updates node representations, and produces a latent structure for inference.Source: §2.2 · p. 3, Figure 3 · p. 3, §2.2.3 · p. 4

Structure Induction

• The model treats document-level graph structure as a latent variable and induces it end-to-end rather than using co-reference links or heuristics.A variant of Kirchhoff’s Matrix-Tree Theorem generates task-specific dependency structures for non-local entity interactions.Source: §2.2.1 · p. 3
• Pairwise node representations are transformed into unnormalized attention scores, while root scores identify candidate roots for the latent structure.The scores use feed-forward networks, a bilinear transformation, and a linear root-scoring transformation.Source: §2.2.1 · pp. 3–4
• The induced edge marginals form a weighted adjacency matrix A that feeds the multi-hop reasoning module.The model derives marginal dependency-edge probabilities using the Matrix-Tree formulation before updating node representations.Source: §2.2.1 · p. 4

Multi-hop Reasoning

• Dense graph convolutional networks update each node from neighboring information and support deeper reasoning over large document-level graphs.Dense connections help capture richer local and non-local information while using the previous layer’s node representations as input.Source: §2.2.2 · p. 4

Iterative Refinement

• Repeated refinement addresses the shallow structures produced by one-pass latent-structure induction by incorporating increasingly non-local information.Early iterations mainly propagate between neighboring nodes, whereas later refinements can generate more informative structures.Source: §2.2.3 · pp. 4–5

Classifier

• The classifier applies a bilinear function to the final representations of each entity pair to compute probabilities for every relation type.The relation probability uses trainable weights and bias, a sigmoid function, and relation-specific indexing.Source: §2.3 · p. 5

3.1 Data

• The evaluation uses F1, Ign F1, and separate intra- and inter-sentence F1 scores across DocRED, CDR, and GDA.Source: §3.1 · p. 5, §3.2 · p. 5

3.3 Main Results

• 54.18 F1 is achieved by LSR with GloVe on the DocRED test set, the best result among models with GloVe.Source: §3.3 · p. 6
• 3.1 F1 points separate LSR from the best sequence-based model, BiLSTM, on DocRED.Source: §3.3 · p. 6
• 3.0 and 2.4 higher F1 are obtained by LSR than EoG on the development and test sets, respectively.Source: §3.3 · p. 6
• LSR consistently outperforms graph-based models using static graphs or attention mechanisms and also outperforms GAT and AGGCN.Source: §3.3 · p. 6
• 59.05 F1 is achieved by LSR with BERT on DocRED, establishing a new state-of-the-art result.Source: §3.3 · p. 6

3.4 Intra- and inter-sentence performance

• About 45% of DocRED development-set entity pairs require information aggregation over multiple sentences.Source: §3.4 · p. 6
• LSR outperforms all other models in both intra- and inter-sentence settings.Source: §3.4 · p. 6
• The differences between LSR and other models are larger for inter-sentence than intra-sentence entity pairs.Source: §3.4 · p. 6
• The majority of LSR’s superiority comes from inter-sentence relational facts, indicating that its latent structure synthesizes information across multiple sentences.Source: §3.4 · pp. 6–7
• LSR with GloVe performs better than two BERT-based models in the inter-sentence setting.Source: §3.4 · p. 7

CDR

• Full LSR performs worse than state-of-the-art models on CDR because spaCy-extracted MDP nodes contain less informative context than those from DocRED.Removing MDP nodes and building a fully connected graph consistently improves performance over sequence-based and graph-based models.Source: §3.5 · p. 7

GDA

• The simplified LSR model achieves the new state-of-the-art result on the distantly supervised GDA dataset.The comparison distinguishes the simplified LSR from the full EoG-based model and reports stronger overall performance for simplified LSR.Source: §3.5 · p. 7

Structure comparison

• LSR outperforms QAGCN, EoG, and AGGCN in overall F1 across settings with one to four refinement blocks.The comparison uses the same GCN model for multi-hop reasoning across the evaluated structures.Source: §3.6.1 · pp. 7–8

Refinement analysis

• LSR achieves its best performance at the second refinement, improving overall F1 by 0.72% over the first induction.Additional iterations can reduce F1 because of over-fitting.Source: §3.6.2 · p. 8

Ablation results

• Removing structure induction lowers F1 by 3.26 points, indicating that latent structure is central to LSR’s performance.The study reports performance deterioration when individual components are removed.Source: §3.7 · p. 8

Case study

• LSR predicts the Japan–World War II relation “participant of” by propagating information through intra- and inter-sentence structures.The case study examines the reasoning process over two refinement steps using attention to World War II mentions.Source: §3.8 · pp. 8–9
• LSR attends strongly to New Ireland and consequently fails to predict that New Ireland and World War II have no relation.The example identifies a NIL relation that the model misses.Source: §3.8 · p. 9

Document-level relation extraction

• Document-level relation extraction extends earlier sentence-level relation extraction to datasets requiring document understanding and multi-hop reasoning.The task involves relations among entities across multiple sentences and domains.Source: §4 · p. 9
• Prior approaches use sequence encoders such as CNNs, LSTMs, BiLSTMs, and attention-based models for document-level relation extraction.These methods encode sentences or document sequences without the structural graph constructions emphasized by graph-based approaches.Source: §4 · p. 9

Structure-based relational reasoning

• Structure-based methods model non-local dependencies with dependency graphs, co-reference links, heuristics, GCNs, or related graph reasoning architectures.These approaches have been applied in both question answering and relation extraction.Source: §4 · p. 9

Conclusion

• LSR dynamically learns a document-level structure and makes predictions in an end-to-end fashion instead of relying on syntactic trees, co-references, or heuristics.The model is presented as a latent structure refinement approach for document-level relation extraction.Source: §5 · p. 9
• Future work could extend structure induction to construct nodes without relying on an external parser.This is identified as one possible direction for broadening the model’s structure induction scope.Source: §5 · p. 9