Source-linked AI summary
ASER: A Large-scale Eventuality Knowledge Graph
Hongming Zhang, Xin Liu, Haojie Pan, Yangqiu Song, Cane Wing-Ki Leung
TL;DR
Existing knowledge graphs largely emphasize entities while omitting activities, states, events, and their relations, despite their role in describing how things act. ASER builds an eventuality-centric graph by extracting dependency-based eventualities and bootstrapping relations from text. The resulting resource contains 194 million eventualities and 64 million edges, with evaluations supporting its quality and effectiveness.
Problem
Existing large-scale knowledge graphs mainly focus on entities and concepts rather than activities, states, events, and their relations.
Method
ASER extracts semantically complete eventualities using dependency-based patterns, seeds relations with unambiguous PDTB connectives, and expands them through neural bootstrapping.
Results
194 million unique eventualities and 64 million edges form ASER, whose human and extrinsic evaluations support its quality and effectiveness.
Takeaways & Limitations
ASER provides a large-scale eventuality knowledge graph with inference and applications for language understanding tasks.
Takeaways & Limitations
The eventuality representation assumes English syntax is relatively fixed and consistent and that semantic meaning is determined by contained words.
Abstract
from arXiv · showhide
Understanding human's language requires complex world knowledge. However, existing large-scale knowledge graphs mainly focus on knowledge about entities while ignoring knowledge about activities, states, or events, which are used to describe how entities or things act in the real world. To fill this gap, we develop ASER (activities, states, events, and their relations), a large-scale eventuality knowledge graph extracted from more than 11-billion-token unstructured textual data. ASER contains 15 relation types belonging to five categories, 194-million unique eventualities, and 64-million unique edges among them. Both intrinsic and extrinsic evaluations demonstrate the quality and effectiveness of ASER.
1 INTRODUCTION
ASER addresses the limited representation of activities, states, events, and their relations in predominantly entity-focused knowledge graphs. It constructs a large-scale eventuality graph from textual data and evaluates its quality and usefulness for language understanding.
- Motivation: Existing large-scale knowledge graphs mainly represent entities and concepts, while eventuality-centered resources remain limited in scale and coverage.Eventualities describe activities, states, and events expressed by verb-related clauses.
- ASER: ASER treats activities, states, and events as eventualities and organizes their relations in an eventuality-centric knowledge graph.Activities and events are occurrences, whereas states are typically described by stative verbs; events are inherently countable occurrences.
- Scale: 194 million unique eventualities and 64 million edges make ASER substantially larger than earlier eventuality-related resources.The comparison distinguishes the broader ASER full resource from ASER core, which filters singleton eventualities for better accuracy.
- Construction: ASER extracts eventualities with dependency-based patterns, then uses unambiguous PDTB connectives for seed relations and neural bootstrapping to expand relation extraction.The approach targets large-scale extraction from unstructured text while retaining semantically complete eventuality structures.
- Inference: ASER supports conditional-probability inference over one-hop and multi-hop eventuality and relation retrieval.The paper presents inference as a way to retrieve related eventualities and relations from the graph.
- Evaluation: Intrinsic and extrinsic evaluations, including human verification, ConceptNet correlation, and the Winograd Schema Challenge, support ASER’s quality and effectiveness.The authors conclude that ASER is a promising large-scale knowledge graph with potential for downstream tasks.
2 OVERVIEW OF ASER
ASER represents activities, states, and events as eventualities in a hybrid graph, linking word-based hyperedges through typed relations. It uses dependency-based extraction and organizes 14 discourse relations plus co-occurrence into five categories.
- ASER representation: ASER represents each eventuality as a hyperedge linking words, while relations connect pairs of eventualities in a traditional graph.This hybrid structure combines a word-based hypergraph with an eventuality-level relation graph.
- Eventuality extraction: Dependency-parsing patterns extract eventualities from unstructured corpora, prioritizing high precision while sacrificing some recall.The method relies on assumptions that English syntax is relatively consistent and that eventuality meaning is determined by included words.
- Relation types: The relation inventory contains 14 discourse relation types from PDTB plus co-occurrence, grouped into five categories.The categories are Temporal, Contingency, Comparison, Expansion, and Co-Occurrence.
- Storage: ASER stores eventuality information and inter-eventuality relations separately, including words, internal dependencies, frequencies, and relation endpoints.The two components are maintained in separate SQL database tables.
3 KNOWLEDGE EXTRACTION
ASER is extracted from diverse textual resources by parsing sentences, matching dependency patterns, and forming eventuality pairs. Relations are seeded with relatively unambiguous PDTB connectives and expanded through neural bootstrapping.
- System overview: The extraction pipeline parses diverse corpora, extracts eventualities with dependency patterns, pairs them within sentences, and creates training instances.The corpora include reviews, news, forums, social media, movie subtitles, and e-books.
- Eventuality extraction: Fourteen dependency-based patterns identify semantically complete eventualities, retaining optional edges and rejecting candidates containing negative edges.The extraction algorithm has time complexity O(|S| · |D| · |v|).
- Seed relation selection: Seed relations come from PDTB connectives whose annotations indicate one relation type in more than 90% of cases.The method uses these connectives to label training instances when dependency parses connect two eventualities through the connective.
- Relation bootstrapping: Neural bootstrapping trains a classifier on seed instances, labels high-confidence predictions, and repeatedly adds them to the labeled set.The classifier uses bidirectional LSTMs, a feed-forward network, softmax relation scores, cross-entropy loss, and Adam optimization.
4 INFERENCE OVER ASER
ASER supports probabilistic retrieval of eventualities and relations over one-hop and multi-hop paths. The paper demonstrates these inferences with examples connecting eventualities through direct and intermediate relations.
- Inference types: ASER provides eventuality retrieval and relation retrieval for both one-hop and multi-hop paths, with complexity O(A^k).A is the average number of adjacent eventualities and k is the number of hops.
- Eventuality retrieval: Eventuality retrieval finds target eventualities and probabilities for paths whose relation sequence follows a specified ordered list.The one-hop probability is normalized from relation weights, and the two-hop probability sums products over intermediate eventualities.
- Relation retrieval: Relation retrieval finds relation sequences and probabilities connecting a given head eventuality to a given tail eventuality.The paper defines one-hop and two-hop relation probabilities for paths between the two eventualities.
- Examples: Inference examples recover plausible links such as hunger leading to lunch, calling preceding departure, and shared tiredness connecting sleeping with resting on a bench.The final example uses a two-hop inference to identify a common cause.
5 INTRINSIC EVALUATION
ASER’s intrinsic evaluation measures the quantity and quality of extracted eventualities and relations. The results show broad coverage, high relation quality, and meaningful correspondence with ConceptNet commonsense relations.
- Eventuality extraction: Simpler extraction patterns such as “s-v-o” occur more frequently than more complex patterns such as “s-v-be-a”.
- Eventuality extraction: ASER eventualities follow Zipf’s law, with a few highly frequent items and most appearing only rarely.General-purpose eventualities such as “You think” occur more often than domain-specific examples, and many describe everyday life.
- Eventuality extraction: Human annotation evaluates whether each extracted eventuality precisely represents the meaning of its source sentence.For each eventuality pattern, 200 extracted examples are sampled with their original sentences for annotation.
- Relation extraction: 86.5% and 84.3% overall accuracy are obtained for ASER’s core and full versions, respectively.The core version is more accurate, while the full version covers more relations with acceptable accuracy.
- Comparison with ConceptNet: ConceptNet’s Causes relations strongly overlap with ASER’s Precedence relations, while MotivatedByGoal pairs commonly align with Reason or Condition.These patterns indicate that ASER covers commonsense knowledge represented in ConceptNet.
6 EXTRINSIC EVALUATIONS
ASER is evaluated on Winograd Schema Challenge resolution through direct inference and language-model fine-tuning. The experiments find that ASER alone has limited coverage, but adding ASER knowledge can improve strong pretrained models.
- Evaluation setup: ASER is applied to all 273 selected Winograd Schema Challenge questions using string-match inference and pretrained-language-model fine-tuning.The selected version contains 273 questions to remain consistent with baseline methods.
- String match and inference: String-match inference scores each candidate by the number of ASER edges supporting its eventuality pair and predicts the higher-scoring candidate.Only one-hop ASER relations are used, and tied scores produce no prediction.
- Fine-tuning language models: ASER knowledge is converted into pronoun-coreference examples and used to fine-tune BERT-large, optionally together with WSCR.The generated examples replace a shared word in an ASER edge with a pronoun referring to that word.
- Result analysis: Pure knowledge-based methods, including ASER inference, provide limited help because of low coverage and weak application methods.
- Result analysis: Pretrained language models perform much better, and fine-tuning on a similar dataset such as WSCR is helpful.
- Result analysis: Adding related ASER knowledge helps state-of-the-art models without changing their architectures or hyper-parameters.The authors report this as evidence for ASER’s value in the WSC setting.
7 CONCLUSIONS
ASER is introduced as a large-scale eventuality knowledge graph built from text using dependency-based extraction and relation discovery. Human and extrinsic evaluations indicate that it is promising for downstream tasks.
- ASER extracts eventualities from texts using dependency graphs.
- Seed relations from unambiguous PDTB connectives are expanded through neural bootstrapping.
- Human and extrinsic evaluations show ASER is promising for many downstream tasks.