Dynamic Prefix-Tuning for Generative Template-based Event Extraction

1 Introduction

• The paper focuses on extracting typed event triggers and role-filled arguments without gold entity annotations.The authors describe this setting as more practical for real-world applications.Source: §1 · p. 1
• Generation-based methods are described as more data-efficient and flexible, but their manually designed prompts can be suboptimal and their type information ignores associations among event types.The prompt and event-type information are identified as two challenges to competitive results.Source: §1 · p. 1
• The method combines type-specific tunable prefixes with context information to learn a context-specific prefix for each context, using BART for conditional generation.It extracts event records one event type at a time.Source: §1 · pp. 1–2
• GTEE-DYNPREF achieves competitive results with state-of-the-art classification-based model OneIE on ACE 2005, the best performances on ERE, and effective adaptation to new event types.The method is evaluated on both ACE 2005 and ERE.Source: §1 · p. 2

2.1 Classification-based Event Extraction

• Classification-based event extraction includes sequence-labeling approaches, joint inference, and machine-reading-comprehension formulations.These approaches model triggers and arguments through classification, event dependencies, or constructed questions.Source: §2.1 · p. 2

2.2 Generation-based Event Extraction

• The paper presents conditional generation as a more flexible and portable alternative that reduces the annotation burden.The related work describes translation, constrained decoding, and template-based conditional generation approaches.Source: §2.1 · p. 2, §2.2 · p. 2
• GTEE-DYNPREF replaces manually designed discrete templates and static type instructions with continuous, tunable dynamic prefixes.The static instructions do not account for connections between events in the same context.Source: §2.2 · p. 2

2.3 Prompt Tuning

• Prompt-tuning work includes discrete prefix- or cloze-style prompts and continuous methods that operate directly in the model’s embedding space.The passage also notes approaches initialized with discrete prompts and hybrid prompt tuning.Source: §2.3 · pp. 2–3

3.1 Problem Statement

• Each event-type subtask takes a context and prompt as input and generates an answered prompt containing extracted event records.The formulation divides extraction for a context into subtasks according to the event ontology’s event types.Source: §3.1 · p. 3

3.2 Basic Architecture

• The base architecture uses an encoder-decoder language model to generate an output sequence conditioned on its input sequence.The model computes token probabilities given the preceding output tokens and encoder input; [SEP] separates concatenated input sequences.Source: §3.2 · p. 3

3.3 Prompt Design

• The prompt combines a natural-language event-type instruction with a type-specific template containing trigger and argument placeholders.The trigger portion is shared across event types, while the argument portion follows the event type’s pre-defined ontology template.Source: §3.3 · p. 3
• Ground-truth sequences fill event records into templates, using a trigger-only output when no record of that type is present.Arguments with the same role are ordered by spans and joined with “and”; multiple records are ordered by trigger spans and joined with <OUT_SEP>.Source: §3.3 · p. 4

3.4 Training, Inference and Parsing

• For the base model, the trainable parameters are the encoder-decoder language model, and training minimizes negative log-likelihood across subtasks.Inference uses beam search with BEAM = 6; parsing matches templates and maps slots to ontology roles, retaining a trigger when argument parsing fails.Source: §3.4 · p. 4

3.5 Irrelevant Event Types

• The model tends to generate event records for irrelevant event types, which can hurt precision and F1 when contexts contain no event records.The paper reports that 80.28% of ACE 2005 sentences and 71.02% of ERE sentences contain no event records.Source: §3.5 · p. 4
• A separately trained irrelevance classifier uses a context’s BERT [CLS] representation and an MLP to classify whether it contains any event record.The classifier is intended to alleviate the irrelevant-record problem.Source: §3.5 · p. 4

4.1 Type-Specific STATIC PREFIX

• STAPREF consists of paired continuous activation sequences for the encoder and decoder, virtually prepended to their input sequences.These trainable vectors provide context for both encoder and decoder.Source: §4.1 · p. 4
• Static prefixes give subtasks of the same event type identical output-type instructions and preserve information about only that type, ignoring associations between event types.Source: §4.1 · p. 5

4.2 Context-Specific DYNAMIC PREFIX

• DYNPREF uses context information to integrate prefix vectors from all ontology event types into a context-specific prefix for each position.A BERT [CLS] representation encodes the context, and multi-head attention performs the integration.Source: §4.2 · p. 5

4.3 Training

• Training proceeds in three stages: train the language model, learn type-specific information with other event types masked, then remove masking and train prefixes to capture related-type associations.The language-model parameters remain fixed during the second and third stages.Source: §4.3 · p. 6

5.1 Datasets

• ACE 2005 has 599 documents, 33 event types, and 22 argument roles; ERE has 458 documents, 38 event types, and 21 roles.The preprocessed ACE05-E+ and ERE-EN datasets include pronoun roles and multi-token event triggers, unlike ACE05-E.Source: §5.1 · p. 6, Table 1 · p. 6

5.2 Evaluation Metrics

• Evaluation reports precision, recall, and F1 for trigger classification (Trg-C) and argument classification (Arg-C).Trg-C requires a correct trigger offset and event type; Arg-C also requires a correct argument role.Source: §5.2 · p. 6

5.3 Baseline Methods

• Baselines span classification-based systems, including ONEIE, and generation-based methods, including TANL, BART-GEN, TEXT2EVENT, and DEGREE-E2E.ONEIE is described as the state-of-the-art classification-based method.Source: §5.3 · p. 6

5.4 Implementation Details

• The implementation uses BART-large as the encoder-decoder model and BERT-large for the irrelevance classifier and context encoder.Models are optimized with AdamW; only 4% of negative samples are retained in train and development splits, while all test samples are kept.Source: §5.4 · p. 7

6.2 Transfer Learning Setting

• GTEE-DYNPREF achieves the highest Trg-C and Arg-C F1 on held-out event types, and source-to-target transfer yields higher F1 than ONEIE and TEXT2EVENT.The transfer setting pretrains on the ten most frequent event types and fine-tunes on the remaining 23; the comparison is against training on target types alone and against two baselines.Source: §6.2 · pp. 7–8, Table 6 · p. 8

6.1 Supervised Learning Setting

• 7.5% and 4.6% F1 gains for Trg-C and Arg-C, respectively, establish new state-of-the-art results on ERE-EN; GTEE-DYNPREF is competitive with ONEIE on ACE05-E and ACE05-E+.On ACE05-E, it has the highest Trg-C and Arg-C F1 scores among generation-based baselines; on ACE05-E+, its Arg-C F1 is competitive with ONEIE.Source: §6.1 · p. 7, Table 4 · p. 8

6.3 Ablation Study

• Adding context-specific information to the static prefix raises Trg-C F1 by 0.8%, 0.6%, and 0.5%, and Arg-C F1 by 1.5%, 0.5%, and 0.8% across the three datasets.The ablation sequence builds the final GTEE-DYNPREF model from GTEE-BASE by adding proposed modules.Source: §6.3 · p. 8, Table 5 · p. 8

6.4 Irrelevance Classifier

• A trained irrelevance classifier improves Trg-C and Arg-C F1 by more than ten percentage points, while gold classifiers show further possible gains.The authors leave further optimization of the irrelevance classifier for future work.Source: §6.4 · pp. 8–9, Table 7 · p. 8

6.5 Intrinsic Evaluation

• On ACE05-E+, both Trg-C and Arg-C F1 increase with prefix length up to 80, then fluctuate slightly; the model uses length 80.Increasing embedding dimension improves F1 through D′ = 512, but continued improvement stops around 512; the model uses D′ = 512. Figure 5 presents the intrinsic evaluation results.Source: §6.5 · p. 9, Figure 5 · p. 9

7 Conclusion

• GTEE-DYNPREF combines tunable type-specific prefixes with context-specific prefixes to steer pretrained language models for template-based event extraction.The paper reports competitive results with the state of the art on ACE 2005 and effective portability to new event types.Source: §7 · p. 9

8 Ethical Consideration

• The authors report no significant ethical concerns and identify user-provided sentences or documents as expected inputs for event-record identification.They describe adapting the method to new event types with examples and predefined templates.Source: §8 · p. 9

• Argument templates are used for both ACE and ERE.Source: A Argument Template · p. 12
• Table 8 presents the argument templates for ACE, sourced from the RAMS AIDA ontology.Source: A Argument Template · p. 12
• Table 9 presents the argument templates for ERE, sourced from the KAIROS ontology.Source: A Argument Template · p. 12

• The listed top-10 frequent event types come from the src split of ACE05-E+.Source: B Transfer Learning Details · p. 12
• The list includes Transfer-Ownership, Phone-Write, Elect, End-Position, Transport, Injure, Attack, Transfer-Money, Meet, and Die.Source: B Transfer Learning Details · p. 12
• Table 8 contains all argument templates for ACE05-E and ACE05-E+, while Table 9 contains all templates for ERE-EN.Source: Table 8 · p. 13, Table 9 · p. 13