Source-linked AI summary
Financial Event Extraction Using Wikipedia-Based Weak Supervision
Liat Ein-Dor, Ariel Gera, Orith Toledo-Ronen, Alon Halfon, Benjamin Sznajder, Lena Dankin, Yonatan Bilu, Yoav Katz, Noam Slonim
TL;DR
Company-related event extraction has largely relied on rules, while event taxonomies can be difficult to enumerate and manual annotation limits dataset size. This paper generates weak sentence labels from Wikipedia and trains a type-independent classifier; its S&P-wiki model exceeds 90% precision and recall on Extended-wiki test data, including companies largely unseen in training.
Problem
Rule-based event extraction is common, but hand-crafted rules are time-consuming and comprehensive event taxonomies are difficult to specify, while manual annotation limits dataset size.
Source: §1 · p. 1Method
The paper trains a sentence-level classifier using weak labels drawn from dated sentences in event-prone sections of company Wikipedia articles.
Source: §3.1 · p. 2Results
The S&P-wiki model exceeds 90% precision and recall over Extended-wiki test data, although less than 15% of its companies appear in S&P-wiki.
Source: §4.1 · p. 4Takeaways & Limitations
The results demonstrate the usefulness of Wikipedia-based weak labels for detecting important events related to specific companies.
Source: §5 · p. 5The framework retrieves articles containing the company name in the title; detecting events in articles not focused on that company remains future work.
Source: §5 · p. 5
Abstract
from arXiv · showhide
Extraction of financial and economic events from text has previously been done mostly using rule-based methods, with more recent works employing machine learning techniques. This work is in line with this latter approach, leveraging relevant Wikipedia sections to extract weak labels for sentences describing economic events. Whereas previous weakly supervised approaches required a knowledge-base of such events, or corresponding financial figures, our approach requires no such additional data, and can be employed to extract economic events related to companies which are not even mentioned in the training data.
1 Introduction
The paper targets company-related news events that may merit analysts’ attention, without requiring a fixed taxonomy of event types. It proposes sentence-level supervised classification trained with weak labels generated from Wikipedia, avoiding the effort and limits of manual annotation.
1 Introduction
- Prior finance-event work often targets events tied to a particular company and studies their association with that company’s stock measures.Stock-movement prediction is recognized as a formidable challenge, though business-event extraction may still serve other use cases.Source: §1 · p. 1
- Analysts may benefit from automatically surfacing relevant company news because they must review large news bodies under severe time constraints.The paper focuses on highlighting meaningful company-related events likely to deserve analyst attention.Source: §1 · p. 1
- Ad-hoc event taxonomies differ across studies, with events included in one ontology or analysis omitted from another.For example, CoProE includes patent issuance and delayed report filings, while Du et al. do not; their treatment of analyst-related events also differs.Source: §1 · p. 1
- A comprehensive, fine-grained event taxonomy is difficult to enumerate, while event-type classification may be secondary to analysts’ need to detect events of interest.The paper therefore frames detection as type-independent binary classification of relevant events.Source: §1 · p. 1
- Hand-crafted rules are time-consuming to create and maintain, and are less suited to a setting that assumes no predefined event types.The paper instead uses a statistical sentence classifier to identify events in articles discussing a given company.Source: §1 · pp. 1–2
- Wikipedia content supplies automatically generated weak labels, reducing dependence on costly human annotation while allowing larger, noisier datasets.The paper reports experiments demonstrating the approach’s potential merit.Source: §1 · p. 2
2 Related Work
Earlier event-extraction research used hand-crafted rules or supervised, type-based classifiers. This paper instead uses weakly labeled data for type-independent binary classification, without requiring a specific-event knowledge base.
2 Related Work
- Arendarenko and Kakkonen (2012) used hand-crafted rules for 41 company-event types, while Du et al. (2016) used about 600 patterns covering 15 business-event types.Source: §2 · p. 2
- Jacobs et al. (2018) used manual annotations for a multi-class classifier over 10 event types plus a no-event class and seven companies.They report that a linear SVM performed best for most event types.Source: §2 · p. 2
- Unlike the type-based annotated approach of Jacobs et al. (2018), this paper frames the task as type-independent binary classification using weak labels.Source: §2 · p. 2
- R¨onnqvist and Sarlin (2017) required a knowledge base of dated bank-distress events, whereas this approach aims to cover varied relevant entities, event types, and occurrences.The prior method matched bank mentions to known events near article publication dates.Source: §2 · p. 2
3 Data
The paper uses both automatically weak-labeled Wikipedia data and manually labeled SentiFM data, then compiles 2019 news sentences for evaluation. Wikipedia labels rely on event-prone sections and date-patterns; SentiFM is adapted to a binary task.
3 Data
- The data comprises one automatically generated weak-label dataset and one manually annotated dataset.Source: §3 · p. 2
3.1 Weakly labelled datasets - Wikipedia
- Wikipedia positives are sentences in event-prone company-article sections that begin with a date-pattern; other sentences outside event sections and without that pattern are treated as negatives.The section-title lexicon includes history, leadership, acquisitions, finance, lawsuits, and related terms; the sets are balanced in size and for company-name or “the company” sentence patterns.Source: §3.1 · pp. 2–3
- S&P-wiki uses S&P-500 company articles, while Extended-wiki covers 3.8K companies traded on five major exchanges; positives through 2018 train the models and 2019 positives form the test sets.Source: §3.1 · p. 3
3.2 Manually labelled dataset - SentiFM
- SentiFM’s 10 event types are converted to a binary dataset by labeling no-event sentences negative and all event types positive, while retaining the original train/test split.The paper notes that this differs from its type-independent task and that an event of interest may be absent from SentiFM’s taxonomy.Source: §3.2 · p. 3
3.3 2019 News Sentences - News-2019
- News-2019 contains 2019 Seeking Alpha articles whose titles include one of 10 selected S&P companies, with sentences restricted to 10–50 tokens.The companies were selected for having the most 2019 events mentioned on their Wikipedia pages.Source: §3.3 · p. 3
4 Experiments
The experiments evaluate BERT sentence classifiers trained on three datasets, including tests on news and on companies largely absent from training. Extended-wiki outperforms SentiFM-binary in the news evaluation, while the models capture substantially different event populations.
4 Experiments
- The models use single-sentence BERT classifiers fine-tuned on SentiFM-binary, S&P-wiki, and Extended-wiki data for three epochs.The setup uses BERTBASE, a maximum sequence length of 256, batch size 16, dropout 0.1, and learning rate 5e-5.Source: §4 · pp. 3–4
4.1 Initial model evaluation
- On Extended-wiki test data, S&P-wiki exceeds 90% precision and recall despite less than 15% company overlap, suggesting detection for companies unseen in training.All three models perform highly on test sets matching their training data.Source: §4.1 · p. 4
4.2 Identifying Wikipedia events in the news
- For 26 Wikipedia events identified in News-2019, Wikipedia-based models had lower recall than on Wikipedia, while SentiFM-binary had the lowest news recall.The authors suggest source writing-style differences and a mismatch between SentiFM and Wikipedia event types may contribute.Source: §4.2 · p. 4
4.3 Identifying general events in the news
- Extended-wiki outperforms SentiFM-binary in average precision over top-20 predicted news events, compared with randomly selected sentences.Table 5 reports average precision for the top-20 predicted events.Source: §4.3 · pp. 4–5, Table 5 · p. 5
- Extended-wiki captures a richer vocabulary: 20% of tokens are covered by its 36 most frequent tokens versus 19 for SentiFM-binary, and their top-200 predictions overlap by under 10%.The authors suggest the models are complementary and may benefit from combination.Source: §4.3 · p. 5, Figure 1 · p. 5
5 Discussion
The paper uses Wikipedia information to create weakly labeled data for detecting important company-related events in news, and reports that the resulting classifier is useful. Extending coverage to articles not focused on the company would require adapting the weak labels and classifiers.
5 Discussion
- The authors report that a classifier trained with Wikipedia-derived weak labels is useful for detecting important events related to a specific company in news.They identify additional weak-label sources, such as relevant knowledge bases, as a possible way to improve results.Source: §5 · p. 5
- Extending coverage to articles that do not name the company in their title is a future direction, requiring weak labels and classifiers adapted to sentences not necessarily relevant to that company.Source: §5 · p. 5