Source-linked AI summary
Impact of News on the Commodity Market: Dataset and Results
Ankur Sinha, Tanmay Khandait
TL;DR
Research on news-derived financial information has largely focused on headline sentiment and stock prices, while commodity news has received less attention. This paper introduces a framework and human-annotated gold-news dataset for extracting multiple information dimensions, then reports a significant relationship between price-related news and gold prices.
Problem
News-based financial analysis has largely focused on headline sentiment and stock prices, while commodity news has received less attention and lacked publicly available datasets for machine-learning models.
Source: Introduction · pp. 1–2Method
The paper introduces a framework that classifies gold headlines across nine dimensions using a human-annotated dataset of 11,412 headlines and evaluates text representations and classifiers.
Source: Introduction · p. 2, System Design · p. 3, Classifier Models · p. 7Results
Price-related news on gold significantly impacts gold prices, and the reported impact is observable even 24 hours later.
Source: Introduction · p. 3, Conclusion · p. 12Takeaways & Limitations
The released dataset can support analysis of hidden meanings in gold headlines that may interest investors and policy-makers.
Source: Conclusion · p. 11The causality analysis is framed as testing whether directionality score S is related to price P.
Source: Model · p. 11
Abstract
from arXiv · showhide
Over the last few years, machine learning based methods have been applied to extract information from news flow in the financial domain. However, this information has mostly been in the form of the financial sentiments contained in the news headlines, primarily for the stock prices. In our current work, we propose that various other dimensions of information can be extracted from news headlines, which will be of interest to investors, policy-makers and other practitioners. We propose a framework that extracts information such as past movements and expected directionality in prices, asset comparison and other general information that the news is referring to. We apply this framework to the commodity "Gold" and train the machine learning models using a dataset of 11,412 human-annotated news headlines (released with this study), collected from the period 2000-2019. We experiment to validate the causal effect of news flow on gold prices and observe that the information produced from our framework significantly impacts the future gold price.
Introduction
The paper addresses limited study of news effects on commodities and the lack of public datasets for learning from commodity news. It introduces a human-annotated gold-headline dataset spanning nine information dimensions and reports a significant relationship between news discussions and gold prices, observable even 24 hours later.
Introduction
- News effects on commodity prices have received less attention than stock effects, despite commodities’ high volatility and potential for news analysis to explain uncertainty.Gold is the commodity examined in this study.Source: Introduction · pp. 1–2
- The authors report that no publicly available datasets supported machine-learning models for extracting useful information from commodity news.They identify dataset scarcity and reliance on commercial software as barriers to research on news and commodities.Source: Introduction · p. 2
- The study introduces 11,412 human-annotated gold headlines labeled across nine dimensions, including price relevance, asset comparison, past or future movement, and directionality.Source: Introduction · p. 2
- The framework targets information beyond sentiment, including past price movements, expected directionality, asset comparisons, and general information in headlines.Source: Introduction · p. 2
- News-derived information has a significant causal relationship with gold prices, with an impact observable even 24 hours later.The introduction identifies the causal analysis as a key finding.Source: Introduction · p. 3
System Design
The system maps gold-news headlines to category predictions through a pipeline built around a human-annotated dataset. Its design combines distinct information categories with multiple text representations and classifiers.
System Design
- Incoming headlines are cleaned, vectorized, and passed to classifiers that predict whether each headline belongs to one or more categories.The system design identifies creating an annotated dataset as necessary for building the classifier.Source: System Design · p. 3, Categories for Annotation · p. 6
Gold News Dataset
- The released dataset contains 11,412 human-annotated news headlines about gold, collected from around the world during 2000 to 2019 by scraping financial news provider and aggregator sites.Examples include Reuters, The Hindu, The Economic Times, Bloomberg, Kitco, and MetailsDaily.Source: Gold News Dataset · p. 3
Categories for Annotation
- Headlines distinguish past and future price information from past and future general gold information, including future non-price events.The latter category can capture events useful to investors and policy-makers even when they do not explicitly discuss gold prices.Source: Categories for Annotation · pp. 3, 5
- Price-related headlines also mark upward, stable, or downward movement, while a separate class flags comparisons between gold and another asset.A headline can belong to multiple categories.Source: Categories for Annotation · pp. 3–6
Process of Annotation
- Three expert annotators labeled headlines independently using headlines only, and disagreements were resolved by consensus; agreement exceeded 0.85 for every category.The consensus series was used in the experiments.Source: Process of Annotation · pp. 6–7
Classifier Models
- The TF-IDF/SVM baseline is compared with GloVe-based recurrent classifiers and a finance-adapted BERT approach.The recurrent models include simple and bidirectional RNN, LSTM, and GRU variants.Source: Classifier Models · pp. 7–8
Results
The finance-adapted BERT model performs best overall, while GRU leads the recurrent models; class imbalance remains a challenge for F1 scores.
- The baseline also worked well; bidirectional models slightly outperformed unidirectional versions, and LSTM and GRU outperformed simple RNN.Source: Results · p. 9
- BERT adapted to the financial domain performs best overall; among recurrent models, GRU leads LSTM and trains faster.Table 3 reports test-set precision, recall, and F1 values.Source: Results · pp. 9–10, Table 3 · p. 10
- Categories with severe class imbalance had relatively low F1 because no imbalance-handling technique was used, while BERT handled imbalanced categories better.Source: Results · p. 10
Causality Analysis
The authors relate US gold news to gold prices using a daily news-direction score and regression. They report a significant relationship in both examined years.
Dataset for Evaluation
- For US gold news and prices from 2017–2019, headlines were classified as up, down, or constant, then aggregated into a directionality score.The score is (up-count − down-count) divided by the total counts across up, constant, and down categories.Source: Dataset for Evaluation · pp. 10–11
Model
- Daily directionality scores, computed from news released between 1700 hours on consecutive days, were related to gold prices at 1700 using linear regression.The model tests whether directionality score S and price P are related.Source: Model · p. 11
- p = 0.0318 (April 2017–March 2018) and p = 0.00218 (April 2018–March 2019); β was significant in both periods, so the authors reject the null hypothesis.They conclude that directionality scores and gold prices have a causal relationship.Source: Model · p. 11
Conclusion
The paper releases a human-annotated dataset of gold headlines across nine dimensions and reports that price-related gold news significantly impacts gold prices.
- The authors release a human-annotated dataset of 11,412 gold headlines classified across nine dimensions.Source: Conclusion · p. 11
- The dataset can support analysis of hidden meanings in headlines that may interest investors and policy-makers.Source: Conclusion · pp. 11–12
- Price-related news about gold significantly impacts gold prices, according to the paper’s causality analysis.Source: Conclusion · p. 12