Source-linked AI summary
Sentiment trading with large language models
Kemal Kirtac, Guido Germano
TL;DR
The paper examines whether large language models can use financial-news sentiment to predict stock returns. It compares LLM-derived scores with dictionary methods and reports a 355% return for the long-short OPT strategy, compared with 0.91% for the Loughran-McDonald model.
Problem
Financial text analysis often relies on dictionary representations that omit word order and context, while evidence on LLM-based representations for individual-stock return modeling remains a research focus.
Source: §1 · pp. 1–2Method
The paper uses LLMs to represent financial news as sentiment scores and compares their stock-return modeling with FinBERT and Loughran-McDonald dictionary scores.
Source: §1 · pp. 1–2Results
355% return for the long-short OPT strategy, compared with 0.91% for the Loughran-McDonald dictionary model.
Source: §4 · p. 8Takeaways & Limitations
The authors conclude that advanced LLMs surpass traditional sentiment methods in predicting and explaining stock returns.
Source: §4 · p. 7GPT-4 was unavailable, and limited computer resources led the authors to use smaller OPT versions.
Source: §1 · p. 2
Abstract
from arXiv · showhide
We investigate the efficacy of large language models (LLMs) in sentiment analysis of U.S. financial news and their potential in predicting stock market returns. We analyze a dataset comprising 965,375 news articles that span from January 1, 2010, to June 30, 2023; we focus on the performance of various LLMs, including BERT, OPT, FINBERT, and the traditional Loughran-McDonald dictionary model, which has been a dominant methodology in the finance literature. The study documents a significant association between LLM scores and subsequent daily stock returns. Specifically, OPT, which is a GPT-3 based LLM, shows the highest accuracy in sentiment prediction with an accuracy of 74.4%, slightly ahead of BERT (72.5%) and FINBERT (72.2%). In contrast, the Loughran-McDonald dictionary model demonstrates considerably lower effectiveness with only 50.1% accuracy. Regression analyses highlight a robust positive impact of OPT model scores on next-day stock returns, with coefficients of 0.274 and 0.254 in different model specifications. BERT and FINBERT also exhibit predictive relevance, though to a lesser extent. Notably, we do not observe a significant relationship between the Loughran-McDonald dictionary model scores and stock returns, challenging the efficacy of this traditional method in the current financial context. In portfolio performance, the long-short OPT strategy excels with a Sharpe ratio of 3.05, compared to 2.11 for BERT and 2.07 for FINBERT long-short strategies. Strategies based on the Loughran-McDonald dictionary yield the lowest Sharpe ratio of 1.23. Our findings emphasize the superior performance of advanced LLMs, especially OPT, in financial market prediction and portfolio management, marking a significant shift in the landscape of financial analysis tools with implications to financial regulation and policy analysis.
1. Introduction
Financial text mining remains at an early stage, often drawing on a single source and dictionary scores; this paper uses LLM-based news representations to model stock returns, including at the individual-stock level.
1. Introduction
- Financial research commonly turns text into numerical representations before modeling economic or financial outcomes, often using dictionaries such as Loughran-McDonald.The introduction characterizes this broader integration of text mining into financial models as still in its early stages.Source: §1 · pp. 1–2
- Financial text is unstructured and linguistically complex, while sophisticated analysis can demand technical expertise, computing resources, or funding.The introduction describes textual analysis in finance and economics as only beginning to explore the potential of available text data.Source: §1 · p. 1
- The paper converts financial news into LLM-derived representations, models stock returns, and evaluates sentiment-guided portfolios with transaction costs and news-timed execution.The analysis also relates news sentiment categories to aggregated 3-day excess returns and tests next-day return prediction.Source: §1 · pp. 1–3
- Earlier sentiment indicators examined macro- or market-level asset-price links, while this study targets sentiment dynamics and predictive scores at the individual-stock level.The cited examples include Baker and Wurgler (2006), Lemmon and Ni (2014), and Shapiro et al. (2022).Source: §1 · p. 2
- Prior dictionary methods rely on term frequency, overlook word order and context, and can require inefficiently many parameters; LDA does not resolve the underlying information loss.The paper also notes that reduced representations can be specific to one text source and domain.Source: §1 · p. 2
- The comparison uses BERT, OPT, financial-domain FinBERT, and the Loughran-McDonald dictionary; GPT-4 was unavailable, and limited computing resources constrained OPT size.The selected BERT and OPT models have 345 million and 2.7 billion parameters, respectively.Source: §1 · p. 2
2. Data and methods
This section describes the U.S. news-and-returns dataset and the methods used to turn news-derived language-model scores into return predictions and simulated trading strategies.
2.1. Data
- The study combines Refinitiv news about U.S. companies with CRSP daily returns for companies appearing in at least one article.Source: §2.1 · p. 3
- Articles were limited to individual stocks with available three-day returns, and near-duplicates from the prior 20 days with cosine similarity of 0.8 or more were excluded.Source: §2.1 · p. 3
- After filtering, the sample contains 965,375 articles from January 1, 2010, to June 30, 2023, covering 2,732,845 news items about 6,214 companies before filtering.Source: §2.1 · p. 3
2.2. Methods
- BERT and OPT are fine-tuned using probing; aggregated three-day excess returns assign sentiment labels, with positive returns coded 1 and non-positive returns coded 0.The three-day measure covers the publication day and the following two days.Source: §2.2 · pp. 3–4
- Models are evaluated on a held-out 20% test set for return-direction prediction, while panel regressions relate model scores to next-day returns.Regressions include firm and date fixed effects, use standard errors double-clustered by firm and date, and use a linear specification.Source: §2.2 · p. 4
- Market value-weighted long, short, and long-short portfolios use daily top- and bottom-20% sentiment stocks, with news-timed trades and 10-basis-point transaction costs.Trade entry and exit timing varies according to whether news arrives before 6 am, between 6 am and 4 pm, or after 4 pm.Source: §2.2 · pp. 4–5
3. Results
Across three evaluations, language-model sentiment scores outperformed the Loughran-McDonald dictionary in news sentiment accuracy, return prediction, and portfolio performance. OPT led the reported model comparisons, including in long-short portfolio returns and risk-adjusted performance.
3.1. Sentiment analysis accuracy in U.S. financial news
- OPT ranked most accurate, followed by BERT and FinBERT, while the Loughran-McDonald dictionary had significantly lower accuracy.The comparison evaluated sentiment prediction on 20% of a dataset of 965,375 U.S. financial news articles spanning January 2010 to June 2023.Source: §3.1 · p. 5, Table 3 · p. 5
3.2. Predicting returns with LLM scores
- OPT scores correlated strongly with next-day stock returns; FinBERT also showed robust predictive power, BERT a more modest significant relationship, and combined LLM scores increased predictive strength.The regression specifications include firm and time fixed effects, with standard errors clustered by date and firm.Source: §3.2 · p. 5, Table 4 · p. 6
3.3. Performance of sentiment-based portfolios
- 355% gain for long-short OPT, versus 235% for BERT and 165% for FinBERT; OPT also showed the strongest risk-adjusted performance.The reported strategy returns include 10 bps trading costs for August 2021 to July 2023; the Loughran-McDonald dictionary strategy returned 0.91%.Source: §3.3 · pp. 6–7, Table 5 · p. 6, Figure 1 · p. 7
4. Conclusion
The authors report stronger stock-market performance for advanced language-model strategies than for the Loughran-McDonald dictionary model. They also highlight implications for investment decisions, financial regulation, and further research on finance-specific LLMs.
4. Conclusion
- The OPT self-financing strategy had a 3.05 Sharpe ratio versus 1.23 for the dictionary strategy, while dictionary scores showed little to no significant positive correlation with subsequent returns.The OPT strategy buys stocks with positive scores and sells stocks with negative scores after news announcements.Source: §4 · p. 7
- The authors connect their findings to policy discussions on AI in finance, including market behavior, information dissemination, and price formation.They say the results add insights to regulatory-policy dialogue and optimal practices for incorporating LLMs into financial markets.Source: §4 · p. 7
- The findings offer asset managers and institutional investors empirical evidence on LLM forecasting and could improve performance while reducing reliance on traditional analysis.The authors describe the evidence as relevant to investment-strategy choices.Source: §4 · pp. 7–8
- 355% return for the long-short OPT strategy contrasted with 0.91% for the Loughran-McDonald dictionary model in the August 2021–July 2023 analysis.The authors say these results could help asset managers and institutional investors make more informed choices about integrating LLMs into investment strategies.Source: §4 · pp. 7–8
- The study encourages further research into LLMs in financial markets and models designed for finance's distinctive needs.The authors frame the work as contributing to debate about AI's role in finance and examining both LLM possibilities and boundaries.Source: §4 · pp. 7–8
CRediT authorship contribution statement
The statement credits Kemal Kirtac and Guido Germano with distinct research and review responsibilities.
CRediT authorship contribution statement
- Kemal Kirtac is credited with the original draft, conceptualization, methodology, data curation, software, investigation, and formal analysis; Guido Germano with review and editing, validation, and supervision.Source: CRediT authorship contribution statement · p. 8
Data availability
The authors state that they do not have permission to share the data.
Data availability
- The authors state that they do not have permission to share the data.Source: Data availability · p. 8