Source-linked AI summary
Large Language Models in Finance: A Survey
Yinheng Li, Shaofei Wang, Han Ding, Hang Chen
TL;DR
Financial applications need practical evidence about which LLM solutions fit their tasks and constraints. This survey synthesizes existing approaches and performance findings, proposes an adoption framework, and discusses limitations affecting responsible use.
Problem
Financial professionals need guidance on whether LLMs are suitable for finance tasks and how to weigh their costs, risks, and limitations.
Method
The survey reviews pretrained, fine-tuned, and custom finance LLMs and organizes adoption guidance around data availability, compute resources, and performance needs.
Results
Experimental results show significant performance gains over general-purpose LLMs across financial tasks including sentiment analysis, question answering, and summarization.
Takeaways & Limitations
The proposed framework guides practitioners from low-cost experimentation toward customization when performance requirements justify greater investment.
Takeaways & Limitations
LLMs can produce disinformation and bias, while their generated content remains highly limited in explainability despite possible reasoning steps.
Abstract
from arXiv · showhide
Recent advances in large language models (LLMs) have opened new possibilities for artificial intelligence applications in finance. In this paper, we provide a practical survey focused on two key aspects of utilizing LLMs for financial tasks: existing solutions and guidance for adoption. First, we review current approaches employing LLMs in finance, including leveraging pretrained models via zero-shot or few-shot learning, fine-tuning on domain-specific data, and training custom LLMs from scratch. We summarize key models and evaluate their performance improvements on financial natural language processing tasks. Second, we propose a decision framework to guide financial professionals in selecting the appropriate LLM solution based on their use case constraints around data, compute, and performance needs. The framework provides a pathway from lightweight experimentation to heavy investment in customized LLMs. Lastly, we discuss limitations and challenges around leveraging LLMs in financial applications. Overall, this survey aims to synthesize the state-of-the-art and provide a roadmap for responsibly applying LLMs to advance financial AI.
1 Introduction
The survey reviews LLM solutions for finance and proposes practical guidance for deciding whether and how to adopt them. It synthesizes current developments for professionals, researchers, and developers.
- LLMs may support financial tasks including trading, risk modeling, and customer service through language understanding and generation.
- The survey covers existing finance applications and models, including fine-tuning pretrained LLMs and training domain-specific models from scratch.
- It provides adoption guidance addressing task suitability, cost-benefit tradeoffs, risks, and limitations.
- The paper synthesizes state-of-the-art literature and developments for financial professionals, researchers, and developers exploring AI and finance.
2 Basics of Language Models
Language models estimate probabilities for word sequences, while their architectures evolved from n-gram and recurrent models to transformers. This progression was enabled by advances in computing, data, and neural network design.
- A language model is trained on extensive text corpora to predict the probability distribution of word sequences.
- Conditional word probabilities capture the likelihood of each word given its preceding words.
- N-gram models represent word sequences as Markov processes, conditioning predictions on a fixed number of preceding words.
- RNN-based models such as LSTM and GRU were developed to capture long-term dependencies in sequential data.
- Transformers use self-attention to model parallel relationships between words and enable efficient training on large-scale datasets.
- Language-model evolution has been driven mainly by computational power, large-scale datasets, and novel neural network architectures.
3 Overview of AI Applications in Finance
Finance already applies AI across trading, risk modeling, text mining, and advisory services, while LLMs expand flexibility and the range of tasks that can be addressed. Their benefits include limited-data operation, multi-task adaptation, and tool-assisted planning, alongside important risks.
- Key financial AI applications include trading and portfolio management, risk modeling, financial text mining, and advisory or customer services.
- Traditional financial AI spans statistical, tree-based, recurrent, and deep learning approaches for forecasting, fraud detection, credit scoring, and default prediction.
- Financial text mining extracts information from unstructured data to support decision-making in trading and risk modeling.
- AI chatbots and advisory systems are emerging in finance as cost-effective alternatives or complements to human customer service and investment advice.
- LLMs can perform zero-shot tasks using pretrained knowledge, which is useful when labeled financial data are limited or documents are restricted.
- Prompt changes can adapt one LLM to sentiment analysis, summarization, and keyword extraction without additional training.
- Tool Augmented Generation lets LLMs decompose complex instructions into plans that use financial data and external tools.
- LLM finance applications remain subject to limitations and associated risks.
4 LLM Solutions for Finance
LLM solutions for finance range from API access and open-source zero- or few-shot use to fine-tuning and training domain-specific models from scratch. The survey compares these approaches and reports stronger finance-task performance from specialized models, with results varying by task type and model scale.
- Access and solution types: Finance LLM solutions include API-based services, self-hosted open-source models, fine-tuning, and training domain-specific models from scratch.Open-source models provide weight access, output customization, and greater privacy, while APIs provide hosted language-model capabilities.
- Fine-tuning techniques: Fine-tuning approaches comprise standard fine-tuning and instructional fine-tuning, with LoRA and quantization reducing computational requirements.Standard fine-tuning uses raw datasets, whereas instructional fine-tuning uses explicit instructions and demonstrations to shape model behavior.
- Fine-tuned finance LLMs: 136K task-specific instruction samples were used to fine-tune PIXIU, while FinGPT used approximately 50k samples with LoRA and evaluation limited to finance classification tasks.Instruct-FinGPT fine-tunes LLaMA on 10k instruction samples derived from two financial sentiment-analysis datasets and also evaluates only finance classification.
- Fine-tuned model evaluation: Fine-tuned finance LLMs outperform LLaMA and other open-source models across reported finance tasks, especially classification, and outperform BloombergGPT in most reported tasks.Against ChatGPT and GPT-4, they perform better on most finance classification tasks but show similar or worse performance on finance generative tasks.
- Models trained from scratch: 62.51 versus 54.35: BloombergGPT outperforms BLOOM176B on its reported average score, while Fin-T5 scores 81.78 versus T5’s 79.56.On Bloomberg’s internal benchmark, BloombergGPT scores 62.47 versus BLOOM176B’s 33.39.
- Models trained from scratch: 64.83 versus 45.43: BloombergGPT outperforms BLOOM-176B on finance-related generative tasks, while Fin-T5 scores 68.69 versus T5’s 66.06.The reported tasks include question answering, named entity recognition, and summarization.
- General capabilities: BloombergGPT remains comparable or superior to similar-sized public models on general generative tasks, although it is slightly inferior to larger models such as GPT-3 or PaLM.The survey reports that enhanced finance-task capabilities do not come at the expense of general abilities.
5.1 Determining the Need for a LLM
The survey recommends determining whether an LLM is necessary before selecting an LLM solution. LLMs are most suitable when tasks require pretrained knowledge, reasoning, handling high-variance requests, or coordinating multiple models and tools, while conventional models may suffice for clearly defined data-rich tasks.
- When LLMs may help: LLMs can use pretrained knowledge when annotated data are scarce and common-sense knowledge is required.This makes them a possible choice for tasks lacking sufficient training data or labeled data.
- When LLMs may help: LLMs suit tasks requiring reasoning, unclear expected answers, or handling out-of-distribution data.The survey identifies high-variance, complex financial-advisory conversations as a customer-service setting where LLM virtual agents may assist.
- When LLMs may help: LLMs can orchestrate different models and tools when robust automation requires collaboration across a solution pipeline.Their role is to integrate and use multiple models and tools together.
- Cost and alternatives: Because LLM use carries significant API or fine-tuning costs, conventional models should be considered first for clearly defined, well-annotated tasks.The survey specifically identifies regression, classification, and ranking tasks with ample annotated data as cases where LLM adoption may not initially be necessary or justified.
5.2 A general decision guidance for applying LLMs on finance tasks
The framework organizes LLM adoption into four levels whose data and computational requirements, and therefore costs, increase progressively. It recommends starting with lightweight options and advancing only when performance remains unsatisfactory.
- The four-level framework increases data, computational, and training costs as practitioners move from lower to higher levels.The framework is intended to align choices with resource constraints.
- Confidential data requires self-hosting an open-source LLM, while nonconfidential tasks can use third-party APIs for lightweight experimentation.Examples include LLaMA, OpenLLAMA, Alpaca, Vicuna, GPT3.5/GPT4, and BARD.
- Few-shot learning adds example question-answer pairs to the prompt when Level 1 performance is inadequate.Selecting the number and relevance of examples requires experimentation.
- External tools or plugins can support complicated tasks when in-context learning is insufficient, but development and context-length costs are higher.Tool retrieval or filtering may be needed when descriptions exceed the model’s input limit.
- Fine-tuning requires annotated data, computational resources, and language-model expertise, whereas training from scratch may require millions of dollars and trillions of tokens.Training a domain-specific model can take a professional team several months or years.
5.3 Evaluation
The survey distinguishes accuracy metrics from task-performance measures when evaluating LLMs in finance. It recommends combining task-specific testing with broader assessments of quality, fairness, robustness, and bias.
- Financial LLM evaluation separates accuracy metrics from performance measures tied directly to the downstream task.Accuracy includes regression and classification metrics, while performance can include profit or Sharpe Ratio.
- Regression accuracy can be measured with MAPE, RMSE, and R2, while classification accuracy can use Recall, Precision, and F1 score.
- Task evaluations may use historical data, backtest simulations, or online experiments, with accuracy aligned to performance to support meaningful decisions and guard against overfitting.
- Comprehensive LLM evaluation systems can assess accuracy, fairness, robustness, bias, and other dimensions across varied scenarios.These systems can guide model selection or evaluation of fine-tuned models.
5.4 Limitations
LLMs in finance face risks from disinformation, bias, and limited explainability. The survey therefore emphasizes retrieval, output controls, robust evaluation, and safeguards for responsible deployment.
- Disinformation and racial, gender, and religious biases are major challenges for LLM use in finance.Financial decisions require accurate information, and financial services require fairness.
- Retrieval-augmented generation can help mitigate hallucination, while content censoring and output restrictions can reduce bias.Output restrictions may limit answers to a predefined list.
- LLMs remain black boxes with highly limited explainability despite potentially providing reasoning steps when prompted correctly.
- Responsible financial use requires continuous research, robust evaluation frameworks, and appropriate safeguards.
6 Conclusion
The survey synthesizes LLM solutions for finance and proposes adoption guidance based on practical constraints. It reports gains over general-purpose models across several financial language tasks and offers a roadmap from experimentation to customization.
- The conclusion organizes the survey around two pillars: LLM solutions for finance and guidance for adopting them.
- Reviewed solution strategies include using pretrained models, fine-tuning on domain data, and training custom LLMs.
- Experimental results demonstrate significant performance gains over general-purpose LLMs for sentiment analysis, question answering, and summarization.
- The adoption framework selects strategies according to data availability, compute resources, and performance needs, progressing from low-cost experimentation to customization.
- The survey provides a practical roadmap for applying LLMs to financial AI.