Source-linked AI summary
Can LLM Agents Be CFOs? Benchmarking Long-Horizon Resource Allocation in an Uncertain Enterprise Environment
Yi Han, Yan Wang, Lingfei Qian, Haohang Li, Yupeng Cao, Yueru He, Xueqing Peng, Nanhan Shen, Yitao Xu, Yankai Chen, Dongji Feng, Jimin Huang, Xue Liu, Jian-Yun Nie, Sophia Ananiadou
TL;DR
LLM agents’ ability to allocate scarce resources over long horizons under uncertainty remains unclear. The paper introduces EnterpriseArena, a 132-month CFO simulator with costly observation, delayed actions, and changing economic conditions, and evaluates 23 LLMs across four frameworks. Only 15.4% of trials survive the full horizon, larger models do not reliably outperform smaller ones, and failures span observation, timing, and capital sizing.
Problem
LLM agents remain insufficiently evaluated on committing scarce, non-recoverable resources under partial observability, delayed consequences, hard budgets, and changing dynamics.
Method
The paper introduces EnterpriseArena, a 132-month FinTech lending CFO simulator requiring liquidity management, costly book closing and signals, and equity or debt fundraising under changing macroeconomic regimes.
Results
Only 15.4% of trials survive the full horizon, model scale does not reliably predict success, and the strongest configuration reaches only 7% of the human expert baseline.
Takeaways & Limitations
Long-horizon resource allocation under uncertainty is a distinct and challenging capability for LLM agents, with recurring failures in observation, fundraising timing, and capital sizing.
Takeaways & Limitations
The environment is a simulation that cannot capture extreme events such as prolonged funding droughts or market freezes and models AI agents rather than real organizational hierarchies.
Abstract
from arXiv · showhide
Large language model (LLM) agents are increasingly tested on complex tasks, but their ability to allocate scarce resources over long horizons remains unclear. Unlike reactive tasks with immediate feedback, this setting requires agents to make binding commitments under partial observability, delayed consequences, hard resource budgets, and shifting dynamics. We introduce EnterpriseArena, a 132-month CFO simulator that evaluates long-horizon resource allocation under uncertainty in a FinTech lending firm. Agents must manage liquidity, close books, gather costly signals, and request equity or debt financing across changing macroeconomic regimes. The simulator is built from transformed firm-level financial data, anonymized business documents, decade-scale macroeconomic and industry signals, and expert-validated operating rules. Experiments across 23 LLMs and four agent frameworks show that current agents remain far from robust: only 15.4% of trials survive the full horizon, larger models do not reliably outperform smaller ones, and failures cascade across observation, action timing, and capital sizing. These findings establish long-horizon resource allocation under uncertainty as a distinct capability gap for LLM agents.
1 Introduction
Long-horizon resource allocation under uncertainty requires LLM agents to make binding commitments with scarce resources, partial observability, delayed consequences, and changing dynamics. EnterpriseArena evaluates this capability in a 132-month CFO simulation and finds current agents remain far from robust.
- LLM agents’ ability to allocate scarce resources under long-horizon uncertainty remains underexplored relative to benchmarks for navigation, software engineering, tool use, and financial applications.
- Long-horizon resource allocation requires committing scarce, non-recoverable resources whose value is not immediately verifiable while sustaining a coherent strategy as conditions evolve.
- Four structural properties define the setting: hard resource budgets, long horizons, latent consequences, and non-stationary dynamics.
- EnterpriseArena instantiates the challenge through CFO decisions about book closing, fundraising, or holding, costly state observation, shifting macroeconomic regimes, and delayed consequences.
- Only 15.4% of trials survive the full 132-month horizon, while model scale does not reliably predict success across 23 backbones and four agent frameworks.
- The strongest configuration reaches $34.7M, only about 7% of the human expert baseline of $476.7M, with failures involving observation allocation, fundraising timing, and capital sizing.
2 Related Work
Existing financial and interactive-agent benchmarks test important capabilities but generally omit the combination of scarce capacity, delayed and partial consequences, and evolving states required for enterprise resource allocation.
- Financial agent benchmarks cover trading, investment recommendation, financial analysis, and tool-augmented reasoning, but primarily test market-facing decisions or static workflows.
- Interactive agent benchmarks capture sequential interaction, tool use, and long-horizon behavior, yet commonly provide explicit feedback, use pre-recorded trajectories, or allow errors to remain local and recoverable.
3 EnterpriseArena
EnterpriseArena models a CFO making sequential decisions in a stochastic FinTech enterprise where survival depends on liquidity and growth depends on terminal valuation. The environment constrains observation and action capacity while combining internal operations with external economic dynamics.
- The agent acts as a simulated CFO over monthly timesteps, must keep cash non-negative to survive, and then seeks to maximize terminal enterprise valuation.
- The benchmark induces organizational trade-offs by combining stochastic dynamics, budget-constrained tools, and mutually competing actions for visibility and capital acquisition.
- Internal dynamics update financial position, users, contracts, and organizational records through multiple operational indicators, while external dynamics follow macroeconomic and industry trajectories across expansion, neutral, and recession phases.
- Agents access partial enterprise views through staff operations, with at most 20 tool calls per timestep, and accurate financial state requires a book-closing reconciliation.
- Each timestep permits exactly one action—book_closing, fund_raising_request, or pass—so delayed and irreversible consequences require proactive commitment under uncertainty.
- Fundraising requests specify equity or debt and an amount, with stochastic approval, partial capital realization, settlement delays, and contract costs shaped by market and enterprise conditions.
- The FinTech lending setting provides continuous cash flows and recurrent fundraising needs, while anonymization, accounting-guided rules, and expert validation support realistic but controlled simulation dynamics.
- Terminal valuation combines final revenue, remaining cash, and a penalty for total tool usage, measuring enterprise growth with minimal organizational resources.
4 Experiments and Results
Experiments evaluate agents over a 132-month, multi-cycle enterprise simulation using backbone-model and framework comparisons, then diagnose environmental and allocation failures. Results show weak and inconsistent survival, strong framework effects, delayed-feedback sensitivity, and failures in timing and resource sizing.
- Experiment Settings: The 132-month simulation spans expansion, neutral, and recession phases, testing adaptation across multiple economic cycles.The enterprise starts with $15M cash, 5,000 borrowers, $10K average loans, zero debt, and 10.5M equity shares.
- Experiment Settings: The evaluation varies 23 ReAct backbone LLMs and compares Claude Code, Codex, and OpenClaw with native frontier backbones.The framework comparisons use Claude Opus 4.7, GPT-5.5, and DeepSeek-V4, respectively.
- Results and Analysis: 13% of ReAct trials survive the full horizon, while 12 of 23 models never survive a single run.All models pass the first downturn, but only 50% survive the second and fewer than 18% reach the third.
- Results and Analysis: Model scale does not reliably predict performance: Llama-3.1-8B achieves 40% survival and $30.6M, while Llama-3.3-70B never survives.The best agent score is $34.7M from Codex, only 7% of the $476.7M human baseline.
- Results and Analysis: Framework choice changes outcomes substantially: GPT-5.5 rises from 0% survival under ReAct to 60% with Codex, whereas DeepSeek-v4-pro falls from 60% under ReAct to 20% under OpenClaw.The corresponding tool-use rates are 0.88 actions per month for Codex GPT-5.5 and 0.27 tools per month for OpenClaw DeepSeek-v4-pro.
- Results and Analysis: Failed trials fundraise too late and allocate fewer resources, averaging 1.8 versus 5.0 tool calls per month and requesting $12.6M versus $19.5M.Survivors begin fundraising during the uptrend, whereas failed trials wait until cash declines and settlement delays prevent timely capital receipt.
5 Conclusion
ENTERPRISEARENA evaluates long-horizon resource allocation under uncertainty and finds that current LLM agents remain unreliable across extended sequences of binding decisions. Performance is limited by low survival, weak scaling relationships, and recurring strategic failures.
- Only 15.4% of trials survive the full horizon.The benchmark spans 132 months and tests whether agents can sustain coherent strategies across binding decisions.
- Model scale does not reliably predict success.
- Even the strongest configuration reaches just 7% of the human expert baseline.
- Failures follow a hierarchy of mis-allocated observation, mis-timed fundraising, and under-sized capital requests.Together, these failures suggest that agents struggle to sustain coherent strategies across sequences of binding decisions.
Limitations and Ethical Concerns
The study’s environment is a simulation and does not represent the full complexity of real enterprises. Its scope excludes extreme market events and multi-stakeholder organizational hierarchies.
- The simulation cannot capture extreme events such as prolonged funding droughts or market freezes.
- The environment models AI agents rather than the multi-stakeholder hierarchy found in real organizations.
Potential risks
The benchmark may support more systematic study of long-horizon enterprise decision-making, but its results carry risks if interpreted beyond the controlled simulator. Misuse could encourage overfitting or exploit known weaknesses.
- The controlled environment may support more reliable planning strategies and benchmarking for real-world enterprise use cases.
- Strong ENTERPRISEARENA performance could be misinterpreted as evidence that LLM agents are ready for real-world financial or operational deployment.The passage explicitly notes a persistent gap between simulated and real-world complexity.
- The benchmark may incentivize overfitting to simulator dynamics or evaluation metrics, producing systems that fail under distributional shifts.
- Insights into agent weaknesses could be exploited to induce poor long-term decisions or resource exhaustion.
- The findings should be interpreted as diagnostic signals of current limitations rather than indicators of deployment readiness.
Ethical considerations
The experiments use publicly available models and datasets without personal, sensitive, or confidential company information.
- All experiments use publicly available models and datasets and exclude personal, sensitive, and confidential company information.
A Comparison of prior benchmarks
Prior benchmarks test components of agentic behavior, but generally do not combine scarce capacity, long horizons, latent consequences, and non-stationary dynamics. ENTERPRISEARENA is designed to instantiate these four properties simultaneously in an evolving enterprise environment.
- Existing benchmarks often evaluate explicit goals with relatively explicit feedback and local or recoverable errors, even when tasks are sequential.
- The comparison framework identifies four required properties: resource allocation, long horizon, latent consequences, and non-stationary dynamics.
- Financial benchmarks commonly test reversible trading, static recommendations, or fixed workflows rather than binding enterprise actions shaped by prior commitments.
- ENTERPRISEARENA combines hard action and observation budgets, 132 monthly decision steps, delayed and partially observed consequences, and regime-dependent dynamics.
- Fundraising outcomes are stochastic, with success probability factorized into macroeconomic and company-specific components.
- Successful fundraising may be partially filled and delayed by 1–6 months, while debt costs are determined at settlement.
C Details of the curated dataset
The curated dataset combines economic, industry, enterprise, and document sources to construct the ENTERPRISEARENA environment. Its coverage spans the full 132-month simulation at monthly frequency.
- The dataset includes 16 data and document types spanning economic, industry, and enterprise layers.
- Economic inputs include macroeconomic indicators, debt-market signals, and equity-market signals collected monthly across the full 132-month period.
D Evaluation
The evaluation covers 23 LLMs across four agentic frameworks, with models spanning proprietary and open-source systems and a wide range of scales. It also examines representative failure trajectories and human-expert validation.
- The study evaluates 23 LLMs across four agentic frameworks, covering proprietary and open-source systems from small models to very large MoE architectures.
- The evaluation includes small, medium, and large open-source models alongside closed-source frontier systems to analyze model-family and scale effects.
- Failure cases: GPT-5.4 explores initially but then stops acting, reasoning from stale data until cash runs out despite a 99.1% pass rate.
- Failure cases: Qwen3.5-397B repeatedly uses forecasting and market tools but records 0.0% book closing and delays fundraising until cash is nearly exhausted.
- Agent setup: The agent prompt requires monthly tool use and exactly one action—fundraising, book closing, or passing—while memory persists across resets through a notepad.
- Human evaluation: Two finance experts verified accounting consistency across intermediate outcomes and full trajectories, including reconciliation across the balance sheet, income statement, and cash flow statement.
J Limitations
The study’s evaluation is constrained by simulation scope, a single-agent structure, limited empirical breadth, and the computational cost of long-horizon execution.
- The environment remains a simulation and may not capture black swans, human irrationality, multi-year funding droughts, or total market freezes.
- The current work uses a single-agent decision-making structure, whereas real enterprises involve stakeholders with potentially conflicting objectives and governance needs.
- The empirical evaluation is initial and limited in scope, relying primarily on ReAct across 23 backbones and three additional frameworks.
- Running 132 monthly steps with repeated reasoning and tool calls incurs computational cost, limiting straightforward scaling to higher-frequency operations or very large organizations.