Source-linked AI summary
Template-Based Financial Report Generation in Agentic and Decomposed Information Retrieval
Yong-En Tian, Yu-Chien Tang, Kuang-Da Wang, An-Zi Yen, Wen-Chih Peng
TL;DR
Template-based financial reporting requires precise coverage of structured topics that broad summarization often misses. This paper compares a collaborative full-template agentic workflow with prompt-chained template decomposition, finding that DecomposedIR outperforms AgenticIR across financial and weather-domain evaluations. The results inform selection of agentic workflows for real-world report generation.
Problem
Method
The paper compares AgenticIR, which uses collaborative agents with the full template, with DecomposedIR, which answers template-derived subqueries through prompt chaining.
Source: §1 · p. 2, §3.2 · p. 3Results
DecomposedIR outperforms AgenticIR across both evaluation scenarios, including 27% higher average performance across financial-report characteristics and 45% higher ROUGE-1 with self-reflection on SumIPCC.
Source: §4.1 · p. 4, §4.2 · p. 4, §5 · p. 4Takeaways & Limitations
Decomposed prompt chaining offers broader and more detailed template-based reports than the compared multi-agent approach across the evaluated domains.
Source: §1 · p. 2, §5 · p. 4Readability evaluation is limited because syllable counts and related metrics may not adequately measure readability in financial reports.
Source: §4.1 · p. 4
Abstract
from arXiv · showhide
Tailoring structured financial reports from companies' earnings releases is crucial for understanding financial performance and has been widely adopted in real-world analytics. However, existing summarization methods often generate broad, high-level summaries, which may lack the precision and detail required for financial reports that typically focus on specific, structured sections. While Large Language Models (LLMs) hold promise, generating reports adhering to predefined multi-section templates remains challenging. This paper investigates two LLM-based approaches popular in industry for generating templated financial reports: an agentic information retrieval (IR) framework and a decomposed IR approach, namely AgenticIR and DecomposedIR. The AgenticIR utilizes collaborative agents prompted with the full template. In contrast, the DecomposedIR approach applies a prompt chaining workflow to break down the template and reframe each section as a query answered by the LLM using the earnings release. To quantitatively assess the generated reports, we evaluated both methods in two scenarios: one using a financial dataset without direct human references, and another with a weather-domain dataset featuring expert-written reports. Experimental results show that while AgenticIR may excel in orchestrating tasks and generating concise reports through agent collaboration, DecomposedIR statistically significantly outperforms AgenticIR approach in providing broader and more detailed coverage in both scenarios, offering reflection on the utilization of the agentic framework in real-world applications.
1 Introduction
Template-based financial reporting addresses the need for precise, structured analysis of lengthy earnings-call documents. The paper compares AgenticIR’s full-template agentic workflow with DecomposedIR’s prompt-chained subqueries and finds decomposed prompting produces more detailed reports across evaluations.
- Financial reports support inter-company performance evaluation and applications including stock movement, financial risk, and sentiment prediction.Source: §1 · p. 1
- Existing summarization methods often produce broad results that lack the topic-specific depth required by financial analysts.Source: §1 · p. 2
- AgenticIR uses the full report template for autonomous end-to-end retrieval and generation, whereas DecomposedIR converts template sections into stepwise subqueries.Source: §1 · p. 2
- DecomposedIR consistently produces more detailed reports than AgenticIR across four financial-report characteristics and shows similar findings on a weather-domain dataset.Source: §1 · p. 2
2 Related work
Related work spans autonomous multi-agent systems and decomposed prompting with retrieval augmentation. The paper positions its comparison as distinct because it evaluates these strategies for template-based report generation.
- Agent frameworks have been proposed to enhance LLMs’ collaborative and problem-solving capabilities; CAMEL [19] enables autonomous cooperation between communicative agents through role-playing, while AutoGen provides a multi-agent conversation framework.Source: §2 · p. 2
- Goldsack et al. [7] studied multi-agent earnings-call summarization, whereas this work compares AgenticIR with decomposed-prompting retrieval for template-based reports.Source: §2 · p. 2
3 Method
The method reframes financial summarization as template-based question answering over earnings releases and evaluates agentic and decomposed workflows on shared retrieved evidence. AgenticIR coordinates multiple agents, while DecomposedIR processes template-derived subqueries before aggregating section answers.
3 Method
- The task is reframed as template-based question answering to extract concise information aligned with analysts’ specific topics.Source: §3 · p. 2
3.1 Data Collection and Preprocessing
- The dataset contains 74 earnings documents from five semiconductor companies collected from 2021–2024, with chunked transcripts and JSON financial statements.Source: §3.1 · p. 2
3.2 AgenticIR
- AgenticIR uses six AutoGen agents and two core functions, task_decompose and retrieve_generate, to collaboratively generate structured reports.Source: §3.2 · p. 2
- AgenticIR iteratively decomposes tasks, retrieves top-n transcript chunks, combines them with financial statements, and has a financial manager answer each query.Source: §3.2 · p. 3
3.3 DecomposedIR
- DecomposedIR turns template sections into subqueries, retrieves top-n chunks using 768-dimensional embeddings and cosine similarity, then aggregates generated answers by report section.Source: §3.3 · p. 3
4 Experiments
The experiments compare AgenticIR and DecomposedIR on financial and climate-report datasets using reference-free and reference-based evaluation. DecomposedIR consistently achieves stronger report-quality results, while self-reflection improves alignment but increases complexity and can misalign with section-specific requirements.
4 Experiments
- The financial experiments use GPT-4o-mini, retrieve three chunks with fin-mpnet-base, and evaluate four characteristics with DecompEval and G-Eval.The broader-domain SumIPCC experiments use stella_en_1.5B_v5 for retrieval.Source: §4 · p. 3, §4.1 · pp. 3–4
4.1 Dataset: Finance
- Financial reports are evaluated without expert-written references using DecompEval and G-Eval, whereas SumIPCC reports are evaluated against ground truth with ROUGE and BERTScore.This reflects different evaluation conditions across the two datasets.Source: §4.1 · p. 3, §4.2 · p. 4
- DecomposedIR statistically significantly outperforms AgenticIR by 27% on average across all financial-report characteristics, with p<0.05.The comparison covers Financial Takeaways, Financial Context, Reasoning Correctness, and Management Expectation.Source: §4.1 · p. 4
- Self-reflection increases DecomposedIR’s financial DecompEval and G-Eval scores by 9.8% on average, while slightly improving AgenticIR.The mechanism analyzes generated content, checks alignment with the four characteristics, and adjusts answer-generation strategies.Source: §4.1 · p. 4
- Higher financial-report quality is associated with greater linguistic complexity, and self-reflection produces longer documents.Readability is assessed with FKGL, CLI, ARI, and average sentences per report, where higher scores indicate greater complexity.Source: §4.1 · p. 4, Table 2 · p. 4
4.2 Dataset: SumIPCC
- DecomposedIR outperforms AgenticIR by 33% in ROUGE-1 and 6.3% in BERTScore on the SumIPCC dataset.SumIPCC provides ground-truth reports, enabling ROUGE and BERTScore evaluation.Source: §4.2 · p. 4, Table 3 · p. 4
- Self-reflection yields a 45% ROUGE-1 and 8.3% BERTScore advantage for DecomposedIR over AgenticIR on SumIPCC.Because SumIPCC reports contain sections with distinct focal points and styles, generalized self-reflection guidance may misalign with section requirements and lower performance.Source: §4.2 · p. 4
5 Conclusion
The paper compares AgenticIR and DecomposedIR for template-based financial report generation and reports that decomposed prompt chaining performs better across two evaluation scenarios. It also identifies self-reflection as improving quality while reducing readability.
5 Conclusion
- Decomposed prompt chaining outperforms the multi-agent approach in both evaluation scenarios, while self-reflection improves quality but reduces readability.The comparison informs framework selection for template-based financial report generation.Source: §5 · p. 4