Source-linked AI summary
PiPMRE: A Pipeline Based on Language Model for Medical Relation Extraction
Jiaxin Duan, Fengyu Lu, Junfei Liu
TL;DR
Medical relation extraction must jointly identify entities and relations, but prior approaches struggle with tagging-schema complexity and multiple relations. PiPMRE reformulates the task with a language-model generator and filter that produce and validate triplets. It significantly outperforms previous methods across two public datasets and full-data and few-shot settings.
Problem
Existing MRE methods rely on difficult tagging schemas or struggle to extract multiple relations from medical text.
Method
PiPMRE uses a Seq2Seq relation generator and a relation filter, with template-based text reformulation and no tagging schema.
Results
PiPMRE significantly outperforms previous methods on two public MRE datasets in full-data and few-shot settings.
Takeaways & Limitations
PiPMRE provides a pipeline framework that handles multiple relation triplets without complicated tagging schemas.
Abstract
from arXiv · showhide
Medical relation extraction (MRE) is commonly known for extracting entities and their relations jointly from a medical text, which has attracted considerable attention in recent years. Previous studies treat MRE as a sequence tagging task, which results in either a challenging design of the tagging schema or a failed extraction of multiple relations, due to intricate relationships among medical entities. In this work, we review the task from the linguistic perspective and propose a novel pipeline framework, PiPMRE, developed on language models to enhance MRE performance. Specifically, PiPMRE consists of a relation generator and a relation filter. Given a text, the generator first yields multiple relational triplets, and then the filter scores each triplet and retains only those that pass the borderline as the final results. Implementing PiPMRE requires no tagging schema; instead, we use a simple template to reformulate the input text, ensuring that entities and relations are generated in a contextual order. Extensive experimental results on two public datasets demonstrate the advancement of PiPMRE. It surpasses the previous state-of-the-art by an average of 5.6 recall points and 4.4 accuracy points. PiPMRE's superiority is also demonstrated in few-shot settings.
Introduction
MRE extracts entity-relation triplets from medical text, but existing tagging and Seq2Seq approaches struggle with schema complexity and multiple relations. PiPMRE addresses these issues with a language-model pipeline that generates and filters relation triplets, outperforming prior methods in full-data and few-shot settings.
- Motivation: MRE extracts paired entities and their relations from unstructured medical text as relation triplets.The task supports applications including medical dialogue, knowledge-graph completion, and question answering.
- Limitations of Prior Methods: Sequential tagging requires complicated schemas, while Seq2Seq generation is limited by variable target length in multi-relation extraction.Tag-to-triplet mapping is also complicated, especially for numerous medical terminologies and intricate relations.
- PiPMRE: PiPMRE uses a relation generator followed by a relation filter to produce and retain valid relational triplets.The generator uses corrupted text and beam search, while the filter scores candidates against a borderline.
- PiPMRE: PiPMRE reformulates medical text with a simple template so entities and relations are generated in contextual order without a tagging schema.The approach is designed to handle an unknown number of relations in fixed-length text while using semantic information.
- PiPMRE: PiPMRE has tailored learning paradigms for its generator and filter.The generator receives incremental cross-domain pre-training and preference optimization, while the filter uses contrastive learning and pair-wise margin loss.
- Results: PiPMRE significantly outperforms previous methods on two public MRE datasets in both full-data and few-shot settings.The study also includes ablation experiments examining factors affecting performance.
Task Formulation
PiPMRE formulates MRE as Seq2Seq text-infilling over reformatted medical instances. A template textualizes triplets, corruption masks their slots, and the model predicts the missing structured text in contextual order.
- Task Definition: An MRE dataset contains texts, relation triplets, and medical entities, with the task extracting all triplets from each text.Each relation connects a subject entity to an object entity through a relation type.
- Task Reformulation: PiPMRE avoids enumerating all possible triples and avoids linearizing each triplet into a variable-length word sequence.Instead, it textualizes and corrupts each instance to create a text-infilling problem for a pretrained language model.
- Triplet Textualization: Triplet textualization expresses each relation as “The {To} {Obj} is the {Rel} of the {Ts} {Sub}.”The template includes subject and object entities, their types, and their relation.
- Instance Reformat: Instance reformatting appends textualized triplets to the original text in a coherent language context.This format is intended to be understandable to both humans and language models.
- Instance Corruption: Instance corruption replaces entity, entity-type, and relation spans with distinct sentinel tokens while preserving the remaining content.The resulting corrupted instance contains slots that the model must restore.
- Task Formulation: Seq2Seq text-infilling predicts a structured text containing five missing slots in their contextual order.The slots are ordered as object type, object, relation, subject type, and subject.
Relation Generator
The relation generator adapts a T5-style text-infilling model to medical relation extraction by injecting medical knowledge and learning to reconstruct reformulated triplet text. Prompt tuning and preference-based objectives are used to preserve knowledge and favor valid multi-relation outputs.
- Relation Generator: The generator starts from T5, uses incremental cross-domain pre-training to inject medical knowledge, and fine-tunes toward the reformulated extraction task.T5 is pre-trained with Seq2Seq text-infilling.
- Incremental pre-training: Medical knowledge is converted into a pre-training corpus by generating sentences from direct and two-hop neighbors in CMeKG.The corpus contains 269,930 sentences and covers diseases, drugs, technologies, and common relationships.
- Incremental pre-training: Entity tokens are masked with probability 0.8 and other tokens with probability 0.2, after which the generator restores the original text.The masking procedure follows the text-infilling setup.
- Fine-tuning: During fine-tuning, prompt tuning inserts learnable continuous soft tokens while freezing pre-trained parameters to help preserve pre-training knowledge.The corrupted instance is mapped to a target text containing the missing slots joined by sentinel tokens.
- Preference optimization: An additional objective maximizes probabilities of preferred target texts and minimizes probabilities of fault target texts through direct preference optimization.The preferred texts are established by gold relation triplets, while fault texts are built from fake triplets.
Relation Filter
The relation filter evaluates generator outputs with a BERT-based template-infilling model. It converts the probability of “is” into a confidence score and learns to distinguish valid triplets from fabricated alternatives using contrastive training.
- Relation Filter: The BERT-based filter estimates whether a generated output contains correct triplet elements for a corrupted instance.It embeds the instance and generated slots into a language template.
- Template infilling: The filter predicts whether the object is or is not the relation of the subject, rather than making an absolute binary choice directly.Entity-type slots are omitted because preliminary experiments found them negligible.
- Confidence scoring: The confidence score f(x,y*) is defined as the sharpened probability of “is” for the candidate subject, relation, and object slots.A temperature τ < 1 sharpens the output distribution.
- Filter training: Training contrasts valid generator outputs containing gold-triplet elements with fabricated outputs created by replacing one or more slots with random or peer tokens.The filter uses cloze prompt tuning with learnable soft tokens and a pair-wise margin loss.
- Filtering: The filter compares candidate confidence scores with a borderline parameter ζ, retaining candidates above the threshold.The candidate set is dynamically derived from positive training outputs and has the same size as the corresponding negative set.
Pipeline Inference
At inference, PiPMRE generates constrained candidate relation texts and then filters them by confidence. The pipeline uses beam search with Trie constraints to produce valid structured outputs before threshold-based selection.
- Pipeline Inference: The generator first produces a cluster of candidate outputs from a templated instance, after which the filter scores and compares each candidate with a borderline ζ.The pipeline retains candidates judged correct by the filter.
- Modulated Decoding: Modulated decoding addresses invalid generated slots caused by autoregressive sampling from the generator vocabulary.The mechanism constrains the search space during decoding.
- Modulated Decoding: Beam search samples K candidate target texts while a Trie restricts each next token to children allowed by the pre-given entity and relation sets.The constraint follows the structural requirements of the partially generated triplet.
- Modulated Decoding: Type-aware Trie constraints require the next token to begin an entity of the appropriate type, improving output validity and search efficiency.The example constrains disease positions according to the relation structure.
- Scoring and Filtering: The filter scores each generator output and establishes a triplet when its score exceeds ζ; otherwise, the candidate is abandoned.The datasets used to evaluate the method are CMeIE and CHIP.
Comparison Methods
PiPMRE is compared with sequential tagging and Seq2Seq relation-extraction methods, including approaches designed for overlapping medical relations. The comparison includes methods using different tagging schemas and special-token triplet linearization.
- Sequential tagging methods: Sequential tagging baselines use schemas such as BIEO-R-SO, binary sequences, handshaking labels, and binary trees to capture relation structures.The methods include NovelTagging, CASREL, TP-Linker, and BiTT.
- Comparison Methods: The comparison spans methods from sequential tagging and Seq2Seq families alongside PiPMRE.The cited methods use different foundation-model architectures and extraction formulations.
- Seq2Seq methods: Seq2Seq baselines generate linearized relation triplets, using special tokens to mark subject, relation, object, and object-end positions.GenIE is described as an example of this formulation.
Implementation Details
PiPMRE uses T5-large for generation and BERT-base for filtering, with prompt tuning and beam search configured for evaluation. Performance is reported using micro precision, recall, and F1, while CMeIE results distinguish reproduced from previously reported values.
- Models and Training: PiPMRE uses T5-large for the generator and BERT-base for the filter.The implementation also uses soft-token prompt tuning of length 20 for both components.
- Models and Training: Prompt tuning uses 20 soft tokens for both the generator and filter.Beam search uses width K = 16.
- Evaluation: Table 3 reports evaluation results on the CMeIE test set, distinguishing prior reported results from reproduced results.Results marked with † come from original or previous papers.
- Evaluation: Evaluation reports micro precision, recall, and F1 scores as the main metrics.The experiments also compare the foundation model and its scale across methods.
Main Results
PiPMRE improves over strong baselines on full test sets and multiple-relation cases, while H@1 reveals an advantage that is less apparent at H@3. The evaluation covers single- and multiple-relation settings on CHIP and CMeIE.
- Full-data Results: PiPMRE surpasses GenPT by 1.1 F1 points on CHIP and 0.7 F1 points on CMeIE.GenPT was the strongest comparison method before PiPMRE in the reported full-data results.
- Single-relation MRE: Table 4 reports evaluation results on CHIP and CMeIE test instances with a single relation.The metric H@M samples the top-M candidate triplets according to estimated likelihood.
- Multi-relation MRE: Table 5 reports F1 scores for CHIP and CMeIE test instances containing multiple relations.The evaluation separates entity-pair overlap, single-entity overlap, and no overlap scenarios in the accompanying analysis.
- Single-relation MRE: H@3 evaluates methods closely, but PiPMRE shows obvious advantages when the metric is converted to H@1.The authors attribute this to generator fine-tuning producing more promising triplets at higher probability.
- Multi-relation MRE: PiPMRE surpasses BiTT by a large margin in most multi-relation scenarios.The scenarios are EPO, SEO, and NEO; GenIE and GenPT often generate incomplete or invalid sequences.
Further Analysis
Ablations identify generator preference optimization and incremental pre-training as important components, while contrastive learning is important for filter scoring. Few-shot experiments show prompt tuning benefits and a changing comparison with S3AAL as data increases.
- Ablation Study: Removing DPO from generator fine-tuning hurts PiPMRE most, followed by removing incremental pre-training.The ablation evaluates five PiPMRE variants on the CMeIE test set.
- Ablation Study: Removing contrastive learning greatly harms PiPMRE because generator outputs receive inadequate scoring.The alternative filter is supervised with cross-entropy loss after assigning labels to positive and negative texts.
- Few-shot Learning: Few-shot experiments train PiPMRE and S3AAL on 1%–50% of the training samples and evaluate micro F1.PiPMRE-FPFT removes prompt tuning from both component-learning processes.
- Few-shot Learning: Original PiPMRE outperforms variants without prompt tuning across all tested training-sample sizes.The comparison is based on micro F1 scores in Figure 3.
- Few-shot Learning: PiPMRE performs close to S3AAL below 10% training data, while their gap increases as the training-set size grows.The paper attributes S3AAL’s changing advantage to its specialization for few-shot scenarios.
Conclusion
PiPMRE is a two-stage MRE pipeline that generates formatted relational-triplet text and filters the generated results. It avoids complicated tagging schemas while addressing texts containing multiple relation triplets across full-data, few-shot, and single- or multiple-relation settings.
- Conclusion: PiPMRE combines a Seq2Seq PLM that generates formatted triplet text with a filter that validates generation results.The pipeline is presented as a two-stage approach for medical relation extraction.
- Conclusion: PiPMRE abandons complicated tagging-schema design while handling medical texts with multiple relation triplets.The conclusion reports robustness across full-data and few-shot settings and across single- and multiple-relation settings.