PaperlayerView exact revision on arXiv

Source-linked AI summary

The Price of Thought: Does Test-Time Reasoning Pay in LLM Trading?

Jiayi Chen, Guiling Wang

arXiv:2609.30705v1cs.AI

TL;DR

Whether additional inference-time reasoning improves investment decisions after trading costs remains an open economic question. This study varies reasoning while holding the financial workflow fixed across three LLM families and input conditions. It finds no reliable improvement in net portfolio returns and observes unstable effects across repeated generations.

  • Problem

    It is unclear whether buying more inference-time reasoning improves portfolio outcomes after costs when the model and trading workflow are otherwise held fixed.

    Source: §1 · p. 1

  • Method

    The study varies reasoning within DeepSeek, GPT, and Gemini while fixing date-available information, prompts, outputs, and portfolio construction, then evaluates annual returns and repeated generations.

    Source: §1 · p. 1, §3.2 · p. 4

  • Results

    None of nine annual low-versus-baseline tests establishes a material return benefit, and DeepSeek’s replicated masked-news audit finds a 9.016 bps/day reduction relative to none.

    Source: §1 · p. 2, §9 · p. 9

  • Takeaways & Limitations

    Within the tested setting, added reasoning changes decisions but does not reliably pay for itself, so baseline reasoning is the conservative default absent reproducible task-specific value.

    Source: §9 · p. 9

    The simulator omits nonlinear market impact, short-sale borrow availability, capacity, latency, and live-order uncertainty, so prospective shadow trading is needed before claims about live deployment.

    Source: §8 · p. 9

Abstract

from arXiv · show

While inference-time reasoning in large language models (LLMs) promises better decision making, its higher computational cost may not yield better economic outcomes. Yet reasoning controls are rarely evaluated as economic interventions, where changes in model outputs must translate into better portfolios after trading costs. We conduct a controlled study of representative LLMs from the DeepSeek, GPT, and Gemini families. We vary reasoning effort while holding information available at each formation date, prompts, output formats, and portfolio construction fixed. Our evaluation covers a full year of U.S. equities under three input conditions: numerical, identifiable news, and masked news. It includes more than 800,000 asset predictions and repeated model generations. Across all three model families, additional reasoning does not produce a reliable improvement in net portfolio returns. For DeepSeek, where we examine the full progression from no reasoning to maximum reasoning, performance is nonmonotonic. Repeated generations also produce unstable treatment effects and portfolio selections, even when overall scores remain similar. These findings show that additional reasoning can change financial decisions without reliably improving their economic value, motivating validation for each task before deployment.

1 Introduction

The introduction asks whether buying more inference-time reasoning improves portfolios after costs, since changes in scores may not survive selection, trading costs, or repeated generations. The study holds the financial workflow fixed to test this economic value and finds no reliable return improvement across its annual comparisons.

1 Introduction

  • Extra computation may change scores without changing holdings, or alter a selection near a boundary; transaction costs can erase gross gains, and stochastic calls may not reproduce effects.The paper defines a stock score as the model’s relative expected-return assessment of a candidate.Source: §1 · p. 1
  • Prior studies show LLM systems can trade, but changing models, prompts, tools, and procedures together leaves the value of added reasoning inside a fixed trader unresolved.Test-time-compute research focused on tasks with immediate correctness signals, unlike trading’s noisy, delayed outcomes.Source: §1 · p. 1
  • The experiment varies reasoning within each model family while fixing date-available information, prompts, required outputs, candidate stocks, and portfolio rules.It follows three model families over 241 formation dates in 2024 and compares numerical, identifiable-news, and masked-news inputs.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • None of nine annual low-versus-baseline tests establishes a material return benefit; DeepSeek’s masked-news audit instead finds a 9.016 bps/day reduction relative to none.The 95% confidence interval is −16.248 to −1.784 bps/day, and DeepSeek’s four-level response is nonmonotonic.Source: §1 · p. 2
  • Added reasoning has token, monetary, and latency costs, while its economic benefit remains unresolved within the tested models, inputs, year, and portfolio mechanism.The authors caution that this does not establish that reasoning never helps or that estimated effects equal zero.Source: §1 · p. 2
  • The study connects structured scores to portfolios and after-cost returns, then repeats generations to detect favorable effects that fail to reproduce.It also reports return effects, portfolio stability, invalid outputs, token use, and inference cost.Source: §1 · p. 2

2 Related Work

Related work spans financial language models and trading systems, test-time reasoning methods, and evaluation under financial noise. These strands motivate testing reasoning as a controlled economic intervention rather than assuming benchmark accuracy or richer agent designs establish portfolio value.

2.1 Financial Language Models and Return Prediction

  • Finance NLP moved from finance-adapted encoders to generative models and expanded benchmarks, but benchmark accuracy alone does not establish that added reasoning improves portfolios.The cited resources include FinQA [8], PIXIU and FinBen [36] [37], and FinTradeBench [1].Source: §2.1 · pp. 2–3

2.2 LLM Trading Agents and Financial Benchmarks

  • Prior financial reasoning studies [27],, and motivate controlled comparisons; this study adds three model families, three information conditions, explicit costs, and repeated generations.Richer trading systems vary interacting components such as memory, tools, and agent roles [35] [43] [45].Source: §2.2 · p. 3

2.3 Test-Time Compute and Adaptive Reasoning

  • Test-time methods allocate, sample, search, or verify reasoning paths, but typically optimize accuracy against a verifier rather than realized portfolio return.The related approaches include chain-of-thought [31], self-consistency [30], Tree of Thoughts [41], and budget allocation methods [21].Source: §2.3 · p. 3

2.4 Evaluation Under Financial Noise

  • Financial evaluation must address nonstationarity, data snooping, and multiple testing; this study uses paired daily returns, HAC inference, corrections, and intention-to-treat handling.The paper frames its estimate narrowly as reasoning’s marginal economic value inside a frozen decision pipeline, not a comparison of complete agents or a claim that reasoning has no financial value.Source: §2.4 · pp. 3–4

3 Experimental Design

The study defines a paired net-return comparison of low versus baseline reasoning and tests it in a controlled, year-long U.S. equity design. Model families, inputs, reasoning contrasts, and portfolio procedures are specified to keep comparisons within each model fixed.

3.1 Research Question and Estimand

  • The estimand is the paired change in net portfolio return under low versus baseline reasoning; positive differences favor low, and material benefit requires exceeding 0.25 bps/day.The baseline is none for DeepSeek and GPT and minimal for Gemini.Source: §3.1 · p. 4

3.2 Market Sample and Information Timing

  • The evaluation uses 241 formation dates in 2024 and 241 return dates after overlapping five-session formation cohorts begin generating returns.The return path runs from January 17 through December 30, 2024.Source: §3.2 · p. 4

3.3 Backbones and Reasoning Levels

  • Predetermined DeepSeek, GPT, and Gemini backbones receive fixed within-model reasoning contrasts; only DeepSeek spans none, low, high, and maximum reasoning.GPT compares none versus low, while Gemini compares minimal versus low because reasoning could not be guaranteed fully disabled.Source: §3.3 · p. 4

3.4 Information Conditions

  • Each backbone is tested on numerical features, identifiable news, and masked news, with masking reducing explicit identifiers but not guaranteeing absent latent event memory.The numerical condition supplies 16 daily cross-sectional rank features.Source: §3.4 · p. 4

3.5 Prompt, Output Contract, and Integrity Controls

  • Prompts, score schema, asset packets, and downstream mapping are frozen within paired comparisons; invalid tasks follow neutral-score imputation under the intention-to-treat rule.DeepSeek required 17 deterministic repairs affecting 15 asset rows; GPT and Gemini retained 0.99% and 2.35% invalid tasks, respectively.Source: §3.5 · pp. 4–5

4 Portfolio Evaluation and Statistical Inference

Model outputs are mapped into equal-weight, market-neutral portfolios and evaluated after trading costs. Inference pairs daily portfolio returns while accounting for overlapping holdings, and repeated generations assess the stability of rankings, selections, and return effects.

4.1 Mapping Scores to Portfolios

  • Model scores and confidence form a comparable trading signal; the top and bottom 10 stocks receive equal-weight long and short positions held in overlapping five-session cohorts.The portfolio has gross exposure of 2 and net exposure of 0.Source: §4.1 · p. 5

4.2 Trading and Inference Costs

  • Trading costs assume 10 bps one way, with 5, 20, and 50 bps sensitivity analyses; provider inference expense remains separate from market returns.Deployment assessment considers whether a supported gain could cover inference expense after first considering the direction and uncertainty of returns after trading costs.Source: §4.2 · p. 5

4.3 Paired Inference

  • Inference pairs daily net-return differences and uses five-lag Newey–West HAC uncertainty, with Holm adjustment across three annual contrasts per backbone.Moving-block and cohort bootstraps are supplementary checks; calendar partitions are exploratory and not independent replications. Figure 2 reports that no interval establishes a material benefit.Source: §4.3 · pp. 5–6, Figure 2 · p. 6

4.4 Evaluation Across Stochastic Generations

  • Repeated-generation evaluation measures rank agreement, overlap in selected long and short sets, and generation-specific low-versus-baseline return effects.For GPT and Gemini, ensemble scores are ranked separately because portfolio selection is nonlinear and ensemble-score effects need not equal mean generation return effects.Source: §4.4 · p. 6

5 Results

Across the nine annual low-versus-baseline tests, no comparison establishes a material return benefit after costs. DeepSeek’s reasoning-level and repeated-generation results are nonmonotonic or unstable, while resampling and calendar checks show effects vary without overturning the main annual findings.

5.1 RQ1: Does Low Reasoning Improve Annual Performance?

  • All nine annual low-versus-baseline tests fail to establish material benefit; every 95% CI includes zero and the +0.25 bps/day threshold.Six point estimates are positive and three are negative; Holm adjustment does not change the decisions.Source: §5.1 · p. 6, Table 2 · p. 7
  • Paired effects measure added reasoning’s value, not portfolio profitability; 13 of 18 baseline and low portfolios have negative mean net return.The largest positive annualized return reported is 2.16%.Source: §5.1 · p. 6
  • Across 36 exploratory portfolio-width combinations, 35 fail to establish material benefit; the sole exception does not change the frozen 10-name-per-side primary estimand.The exception is DeepSeek masked news with 20 names per side, at +5.769 bps/day; no p-value survives Holm correction across the 36 cells.Source: §5.1 · p. 6

5.2 RQ2: Does More Reasoning Consistently Improve Returns?

  • DeepSeek’s effects are nonmonotonic: masked-news low reasoning harms returns, while maximum remains below none despite a favorable maximum-versus-high identifiable-news contrast.Low minus none is −9.016 bps/day for masked news; maximum minus high is +4.639 bps/day for identifiable news, but maximum still fails to beat none.Source: §5.2 · pp. 6–7, Figure 3 · p. 8

5.3 RQ3: Are Reasoning Effects Stable Across Generations?

  • In masked-news audits, DeepSeek’s three generations are all negative, whereas GPT and Gemini change sign across generations.DeepSeek’s effects are −7.490, −9.327, and −10.231 bps/day; GPT’s and Gemini’s within-date averages remain inconclusive.Source: §5.3 · p. 7
  • Generation effects for identifiable news and numerical inputs also vary, and averaging scores can reverse the apparent portfolio-return direction.None of the reported GPT or Gemini audit summaries establishes a reliable benefit.Source: §5.3 · p. 7

5.4 Robustness Across Resampling and Calendar Periods

  • All nine annual contrasts remain inconclusive under moving-block and formation-cohort bootstraps, while the DeepSeek masked-news audit’s harm classification survives both checks.The moving-block interval is [−17.929, −3.477] bps/day and the formation-cohort interval is [−15.482, −2.281] bps/day.Source: §5.4 · pp. 7–8
  • Annual directions are not reducible to one isolated quarter, but neither are they temporally uniform within 2024.Seven of nine annual directions flip when at least one quarter is omitted; the heterogeneity findings do not survive Holm correction.Source: §5.4 · p. 8

6 Operational Reliability and Compute Economics

Reasoning changes task validity differently across model families, and reliability gains do not imply return gains. Compute and monetary costs vary by provider and input condition, so reported reasoning-token totals are not directly comparable measures of compute.

6.1 Output Structure Reliability

  • Low reasoning changes failure rates differently across backbones: some GPT and Gemini failure rates fall, while GPT numerical failures rise, without a corresponding return benefit.For example, GPT numerical failures rise from 4 to 14 of 964 tasks, while Gemini masked-news failures fall from 44 to 20.Source: §6.1 · p. 8

6.2 Tokens, Dollars, and the Deployment Decision

  • Backbone totals were 269.0 million reported reasoning tokens for DeepSeek, 0.37 million for GPT, and 7.93 million for Gemini; nominal level labels are not comparable measures of actual compute across providers.For GPT’s primary evaluation, low reasoning costs an additional $0.375 for numerical inputs and $0.092 for masked news, while identifiable news costs $0.281 less because the low level produces fewer output tokens in this run.Source: §6.2 · p. 8

7 Discussion

Reasoning changes outputs without producing a stable monotonic relationship with returns after costs. The authors frame the negative result narrowly and recommend treating increased reasoning as a configuration change requiring downstream validation.

7.1 Reasoning Is Not a Monotonic Economic Control

  • Reasoning changes token use, rankings, portfolio constituents, and sometimes invalid-output rates, but returns after costs do not move monotonically with effort.DeepSeek’s favorable maximum-versus-high identifiable-news contrast still does not establish that maximum beats none.Source: §7.1 · p. 8
  • The experiment does not establish which proposed mechanisms explain the pattern; testing them would require interventions on trace content or intermediate task labels.The authors list possible mechanisms but state that the experiment proves none of them.Source: §7.1 · p. 9

7.2 What the Negative Result Does and Does Not Mean

  • The supported conclusion is limited to no reliable improvement from added low reasoning in the frozen 2024 pipeline, not equivalence or baseline profitability.Most confidence intervals still include economically meaningful gains and losses; the study also does not establish that reasoning cannot help other financial tasks.Source: §7.2 · p. 9

7.3 Implication for Governed Deployment

  • For governed deployment, increased reasoning should be treated as a configuration change requiring validation of action stability, failures, costs, and repeated-generation performance.An isolated favorable generation or unresolved positive point estimate is insufficient to promote a more expensive reasoning level.Source: §7.3 · p. 9

8 Limitations

The evidence is bounded by a single 2024 U.S.-equity backtest and observed provider interfaces, while several design choices constrain how results transfer to other settings. The simulator also omits important live-trading mechanisms, and the estimates are not an equivalence claim.

Temporal and model scope

  • Evidence comes from one 2024 U.S.-equity backtest and the provider interfaces observed at execution, so replication across years or prospectively remains necessary.Calendar-period analysis spans distinct episodes, but frequent sign changes reinforce the need for replication.Source: §8 · p. 9

Precision and portfolio dependence

  • Most primary intervals admit economically meaningful gains and losses, and the 80% minimum detectable effects are 6.200–19.156 bps/day versus the 0.25 bps/day threshold.The result is a conservative deployment decision, not an equivalence claim.Source: §8 · p. 9
  • The conclusion is tied to the frozen rule selecting 10 long and 10 short names; one exploratory width cell was positive before, but not after, family correction.The cell was one of 36 exploratory width cells.Source: §8 · p. 9

Deployment and mechanism

  • The simulator omits nonlinear market impact, short-sale borrow availability, capacity, latency, and live-order uncertainty, so shadow trading is required before live-deployment claims.The experiment estimates the total effect of requesting more reasoning but does not identify the hidden computation behind score or trade changes.Source: §8 · p. 9

9 Conclusion

Across the tested models and primary portfolio rule, additional reasoning did not reliably improve net returns. The conclusion supports baseline reasoning as the default only within this setting, pending reproducible task-specific evidence.

9 Conclusion

  • None of the nine low-versus-baseline tests across 241 return dates establishes a material improvement in net portfolio return.DeepSeek’s replicated masked-news result shows harm; its four-level curve is nonmonotonic, and repeated generations reveal sign instability across news conditions.Source: §9 · p. 9
  • Reasoning can improve or degrade output reliability without improving returns, and repeated generations show unstable treatment effects and portfolio selections.DeepSeek’s full progression is nonmonotonic.Source: §9 · p. 9
  • For the primary 10-long, 10-short implementation, baseline reasoning is the appropriate default until prospective study demonstrates reproducible incremental value on the target task.The exploratory portfolio-width exception cautions against transferring this recommendation unchanged to a different portfolio rule.Source: §9 · p. 9

A Prompt and Structured Output Contract

The appendix records the fixed prompt and data contracts, provider-specific reasoning controls, and the implementation and robustness details behind the evaluation.

A Prompt and Structured Output Contract

  • The system prompt frames a conservative cross-sectional equity forecaster and requires one structured JSON object; the user prompt requests relative return over the next five trading sessions.It distinguishes target-specific material news from repetition, commentary, price narration, advertising, and noise.Source: A Prompt and Structured Output Contract · p. 11
  • Exact prompts are stored as immutable hashed artifacts, with provider-specific syntax but common fields and bounds.The expected opaque identifiers must each appear exactly once, and the output contract includes score, confidence, materiality, uncertainty, and abstain fields.Source: A Prompt and Structured Output Contract · p. 11

A.1 Reasoning Intervention Settings

  • DeepSeek toggles thinking, GPT varies reasoning.effort, and Gemini varies thinkingLevel; provider labels and output limits are not comparable compute quantities.No condition receives an additional reasoning instruction, and retained reasoning traces are not treated as faithful records of hidden computation.Source: §A.1 · p. 11, Table 4 · p. 11

A.2 Numerical Data Provenance and Volume

  • Numerical inputs use CRSP daily stock data and a frozen cleaning rule; feature construction spans 2021–2024, while the experiment reuses the same records across conditions.The 2024 panel contains 24,100 stock-date records, each with 64 values, for 1,542,400 numerical values.Source: §A.2 · p. 11

A.3 News Data Provenance and Volume

  • News packets retain up to three recent eligible articles per stock and date; identifiable and masked conditions use the same articles, with identifying details replaced in masked news.Of 24,100 stock-date combinations, 16,916 (70.2%) contained at least one eligible article.Source: §A.3 · p. 11

B Complete DeepSeek Confirmatory Chain

  • Across DeepSeek’s secondary adjacent contrasts, only maximum–high for identifiable news has a two-sided 95% confidence interval entirely above zero.The supplied results list the contrast estimates in bps/day; Table 5 describes the three-generation audit and keeps return date as the market sampling dimension.Source: B Complete DeepSeek Confirmatory Chain · p. 11, Table 5 · p. 11

C Sensitivity to Failure Handling

  • GPT’s 57 and Gemini’s 136 invalid primary tasks remain neutral-imputed; complete-date and matched-chunk sensitivity analyses leave all six conditions inconclusive.DeepSeek’s deterministic repair affected 17 of 16,176 tasks; 15 omitted rows were imputed neutrally and seven unique near-match identifier errors were corrected.Source: C Sensitivity to Failure Handling · p. 12

D Bootstrap Robustness

Supplementary analyses leave the annual return conclusion inconclusive and the replicated audit harmful, while revealing sensitivity to transaction costs, portfolio width, precision, and generation agreement.

D Bootstrap Robustness

  • Bootstrap checks leave all nine annual contrasts inconclusive and the audit contrast harmful, including moving-block sensitivities using 10- and 20-session blocks.These checks supplement rather than replace the primary HAC analysis; Table 6 describes the paired bootstrap intervals at a one-way cost of 10 bps.Source: D Bootstrap Robustness · p. 12, Table 6 · p. 12

E Sensitivity to Transaction Costs

  • GPT and Gemini remain inconclusive across tested one-way costs of 5, 10, 20, and 50 bps, with estimates shifting slightly as reasoning changes turnover.The reported numerical, identifiable-news, and masked-news estimate ranges remain small across those costs.Source: E Sensitivity to Transaction Costs · p. 12

F Absolute Performance and Sensitivity to Portfolio Width

  • Table 7’s absolute portfolio levels are descriptive historical backtests, not evidence of prospective profitability.It reports baseline/low reasoning pairs; net is return after trading costs, while Sharpe, maximum drawdown, and turnover provide additional portfolio context.Source: F Absolute Performance and Sensitivity… · p. 12, Table 7 · p. 12
  • Only one of 36 portfolio-width cells shows unadjusted material benefit, for DeepSeek masked news at 20 names per side; neither adjusted p-value survives Holm correction.The result indicates sensitivity to portfolio width rather than a confirmatory reasoning benefit.Source: F Absolute Performance and Sensitivity… · p. 12, Table 8 · p. 12

G Precision and Detectable Effects

H Agreement Across Repeated Generations

I Analysis by Calendar Period

Calendar-period checks find no corrected heterogeneity but show within-2024 directional instability, not independent evidence across market years. The completed evaluations froze their core design before outcomes, and the disclosed retrospective simulation is not evidence of prospective profitability.

I Analysis by Calendar Period

  • Table 11 partitions nine primary low-versus-baseline daily effects by H1/H2 and Q1–Q4; subgroup means are descriptive, while annual estimates and inference remain primary.Source: I Analysis by Calendar Period · p. 13, Table 11 · p. 13
  • No heterogeneity test survives Holm correction; the largest quarter contributes 34.2%–64.5% of contrast sums, yet seven annual directions flip under quarter omission and only GPT numerical retains half-year sign.Source: I Analysis by Calendar Period · p. 13

J Protocol Integrity and Claim Boundaries

Loading 2609.30705v1…