From Search to Research: Exploring Search Scaling in Autonomous Quantitative Factor Mining

1 Introduction

• The central gap is how model capability and search budget interact when agents choose experiments, interpret results, and use the evidence to decide what to do next.More search can enable improvement but also mistaken diagnoses, unproductive local search, and adaptive selection that favors noise.Source: §1 · p. 2
• Hypothesis constraints and idea-alignment checks limit unrestricted fitting, but repeated selection on development backtests still creates multiple-testing risk, making held-out evaluation essential.The fixed environment and shared harness support examination of out-of-sample quality while preserving the intended economic hypothesis.Source: §1 · p. 2
• The benchmark uses 50 quantitative research-report tasks in which agents implement a factor, run development backtests, and revise it with the model and search budget as principal variables.The harness freezes checkpoints for independent held-out evaluation and records realized resource use separately from nominal search depth.Source: §1 · p. 2
• The contributions include a training-free, hypothesis-constrained protocol and cross-model budget-performance curves showing that deeper search can narrow some model gaps.The paper also examines how early research-state quality and search breadth matter, and connects trajectory behavior to outcome differences.Source: §1 · p. 2

Scaling Laws and Test-Time Compute

• Test-time scaling studies sampling, verification, aggregation, and sequential refinement, while autonomous research adds the need to generate experimental evidence and decide how it changes an artifact.Research-time scaling depends on converting feedback into useful revisions and assessing factor value beyond the historical data used for development.Source: §2 · pp. 2–3

Autonomous Research and Experimentation

• Autonomous-research benchmarks evaluate sustained experimental work across full-cycle research, engineering, replication, and performance-driven objectives.Their budget schemes include longer attempts, independent attempts, and continuation; this paper varies depth within a single trajectory.Source: §2 · p. 3

Trading Agents and Quantitative Factor Research

• Financial-agent benchmarks assess investment decisions and market outcomes, while factor-research systems let agents control the predictive artifact's hypothesis, implementation, and revision.Existing factor evaluations cover iterative search and factor quality; this paper crosses models with research budgets to examine held-out quality and cost trade-offs.Source: §2 · p. 3

3.1 Problem Formulation

• At search depth b, a fixed base model and shared harness select an incumbent factor using development evidence, while an independent evaluator measures its held-out quality.The framework distinguishes model capability, nominal iteration budget, and realized resource consumption, which includes tokens, tool calls, elapsed time, and API cost.Source: §3.1 · p. 3

3.2 Controlled Iterative Search

• Each harness cycle lets the agent run experiments, inspect development feedback, and update its candidate, with workspace artifacts and interaction history carried into the next cycle.The agent is not told the eventual trajectory length, and frozen checkpoint evaluation never feeds back into subsequent agent observations.Source: §3.2 · p. 4

3.3 Measuring Search Scaling

• Search scaling is measured by aggregating held-out outcomes over the same tasks at each depth and calculating paired quality increments between depths.Recorded resource use supports comparing quality against the actual cost of reaching each checkpoint.Source: §3.3 · p. 4
• A shifted, saturating power law is a candidate description of budget-performance curves, with a shared offset and parameters for the plateau, remaining-gain scale, and decay.The curve is an interpretable description of diminishing gains when the observed response follows that pattern; the analysis also compares outcomes and costs across model capability levels.Source: §3.3 · p. 4

4.1 Experimental Setup

• The benchmark covers 50 report-grounded factor tasks in the Chinese A-share market, with nine models run on continuous ten-iteration trajectories and factors assessed for Sharpe, monotonicity, and idea alignment.Source: §4.1 · p. 5

4.2 Search Scaling across Models and Budgets

• All nine models improve from three to ten iterations, but gains differ: Claude Opus 5 rises from 75.45 to 92.50, while DeepSeek-V3.1 declines slightly after iteration 6.Source: §4.2 · p. 5, Table 1 · p. 5
• Extra search can substantially improve weaker initial configurations, while model capability tracks three-iteration mean Sharpe more closely than three-to-ten-iteration Sharpe gain (ρSpearman = 1.00 vs. 0.48).Source: §4.2 · p. 6, Figure 2 · p. 6
• At ten iterations, DeepSeek-V4.1-Flash scores 83.44 and mean Sharpe 2.266 for 3.25 RMB per task, dominating the low-cost Pareto frontier in both panels.Source: §4.2 · p. 6, Figure 3 · p. 7

4.3 What Makes Iterative Search Effective?

• With equal total depth, starting from GLM-5.3 and continuing with Qwen3.7-Plus beats the reverse schedule by 4.13 score points and 0.049 average Sharpe.Source: §4.3 · p. 7, Table 2 · p. 7
• Under the same ten-iteration budget, five independent two-iteration searches improve score and Sharpe over one sequential trajectory for all three tested models.Source: §4.3 · p. 7, Table 3 · p. 8

4.4 Qualitative Analysis of Research Trajectories

• Higher-scoring models broaden experiments when diagnostics undermine an approach and inspect component contributions, returns, coverage, and implementation consistency before retaining changes.Source: §4.4 · p. 8
• Candidate selection also matters: stronger-model records preserve viable earlier candidates and hypothesis-required adjustments, whereas weaker-model records may drop defining terms or add unrelated signals.Source: §4.4 · p. 8

5.1 Search Scaling Is the Accumulation of Research State, Not Merely More Inference

• Grafting suggests accumulated implementation, evidence, and memory retain value across model switches, while parallel search indicates that a useful starting direction can matter more than extending one trajectory.Source: §5.1 · p. 8

5.2 Model Capability Is Stage-Dependent Rather Than a Single Scalar

• Model capability is stage-dependent: Qwen3.7-Plus benefits more from GLM-5.3’s initial state than the reverse, suggesting initialization and refinement impose different demands.Source: §5.2 · p. 8

5.3 Search Scaling as a Compute-Allocation Problem

• Search allocation can vary depth, breadth, and model routing; the objective is independently evaluated quality within a resource budget, and cost–performance depends on allocation as well as model and budget.Source: §5.3 · pp. 8–9

5.4 When Should Research-Time Scaling Generalize?

• Transfer is expected when tasks offer informative external feedback, persistent artifacts, incremental revision, and independent final evaluation; ambiguous evaluators, delayed feedback, or expensive physical experiments may behave differently.Source: §5.4 · p. 9

• Across quantitative factor mining, search depth interacts with model capability and organization: early states can retain value across model switches, and broader exploration can outperform extending one trajectory.Source: §6 · p. 9

7 Limitations and Future Work

• The scaling relationship covers a finite set of models and search depths in a structured Chinese A-share factor-research environment.The authors say longer horizons and later model generations require testing functional stability, parameter transfer, and prediction for unseen models and budgets.Source: §7 · p. 9
• The depth, breadth, and fixed model-grafting comparisons cover only controlled slices of the broader research-policy space.Future work could combine routing, continuation, restart, parallel exploration, and stopping in a state-dependent allocation policy, then test other executable-feedback research tasks.Source: §7 · p. 9

AI use statement

• Generative AI tools assisted across research and manuscript tasks, while human experts selected source reports and reviewed generated objectives.The authors state that they reviewed the AI-assisted text, arguments, code, and analyses and take responsibility for the final content, claims, and artifacts.Source: AI use statement · p. 9

A.1 Report Selection and Task Composition

• The 50 selected reports comprise 30 technical and 20 fundamental factor-construction tasks from Chinese A-share research.Reports issued between February 2017 and June 2024 had to concern a single factor, provide an explicit construction formula, and use Chinese A-share stocks.Source: §A.1 · p. 12
• Tables A.1 and A.2 summarize task counts by signal family and input frequency; frequency denotes principal quote-data resolution, not portfolio rebalancing frequency.Signal families label the originating research idea.Source: §A.1 · p. 12, Table A.1 · p. 13, Table A.2 · p. 13

A.2 Task-Generation Prompt and Human Review

• The task-generation prompt asks for a concrete, implementable factor direction, with calculation logic and expected stock-selection meaning rather than code.The task instructions also ask for the economic rationale and required data; available-data information is reference material rather than a restriction on idea generation.Source: §A.2 · pp. 13–14
• The prompt favors a broad formula and economic mechanism, avoiding incidental parameter prescriptions that would restrict the later agent’s choices.The turnover-weighted reversal example illustrates a calculation and rationale without prescribing excessive implementation detail.Source: §A.2 · p. 14
• Human reviewers retained objectives specific enough to implement but rejected vague or overly prescriptive descriptions and checked their grounding in a selected single-factor report.Reviewers also checked that objectives concerned the factor itself rather than benchmark-level portfolio or backtest settings.Source: §A.2 · p. 15

B.1 Universe, Data, and Information Timing

• Evaluation covers approximately 5,200 Shanghai and Shenzhen stocks, with ST stocks and listings under 60 days excluded at each signal date.The development period runs from January 2013 through June 2024; the held-out test period runs from July 2024 through July 17, 2026, after all source reports were published.Source: §B.1 · p. 15
• At signal date T, daily observations are truncated at T and fundamental records are filtered by publication date to prevent access to later information.The audit checks for code that bypasses this interface; a formula can be temporally invalid if it uses a disclosure unavailable at T.Source: §B.1 · p. 15

B.2 Factor Artifact and Postprocessing

• Agents submit an executable cross-sectional factor and its signal matrix, choosing construction and processing choices appropriate to the hypothesis.The shared defaults include winsorization, industry and size exposure removal, and centered ranks; submitted postprocessing is retained with the artifact.Source: §B.2 · p. 15

B.3 Portfolio and Re-evaluation Protocol

• The common evaluator measures rank-weighted long–short returns with stated transaction costs and ten-group monotonicity, then tests the selected artifact out of sample.Agents receive only training-period feedback; held-out results are not returned for revision, and implementation validity and hypothesis alignment are audited separately from financial metrics.Source: §B.3 · pp. 15–16

C.1 Financial Metrics

• The financial metrics include net Sharpe, annualized return, drawdown, turnover, information ratio, information coefficient, and ICIR; monotonicity measures rank ordering across ten groups.The absolute Spearman correlation used for monotonicity captures cross-sectional ordering but not trading costs or signal persistence.Source: §C.1 · p. 16

C.2 Model-Level Score

• The model-level score aggregates mean net Sharpe, monotonicity, and alignment, applying threshold-based quality factors against fixed reference values.Invalid, protocol-violating, or unevaluable tasks contribute zeros while remaining in the fixed denominators; the score is averaged across five repetitions.Source: §C.2 · p. 16

C.3 Validity and Alignment Audit

• A separate audit assesses temporal validity and alignment with the assigned hypothesis, and invalid or protocol-violating submissions receive zero inputs for all three score components.Alignment allows coherent hypothesis-preserving changes, but penalizes unrelated predictors that dilute the assigned idea.Source: §C.3 · pp. 16–17

C.4 Validity and Alignment Audit Prompt

• The audit checks whether code escapes the time-bounded interface and whether the factor preserves its intended direction without unrelated signals.The prompt distinguishes effective future-information access from valid historical calculations and scores alignment from 0 to 5.Source: §C.4 · pp. 17–18

D.1 Systems and Common Settings

• Nine deployed models use the same bare Claude Code setup, while the external harness advances one research cycle at a time and preserves state across checkpoints.Held-out outcomes remain outside the agent’s context; common settings are summarized in Table D.1.Source: §D.1 · pp. 18–19, Table D.1 · p. 19

D.2 Factor-Mining Prompt

• Each research cycle implements and tests a candidate, interprets evidence, updates persistent research memory, and selects a valid, hypothesis-aligned incumbent.The prompt asks agents to record failed experiments and negative evidence and to revise within the assigned hypothesis when evidence contradicts the current explanation.Source: §D.2 · pp. 19–20

D.3 External Capability Indicator

• The capability fit standardizes eight external capability values; Doubao Seed 2.1 Pro is excluded because it has no matching entry.The values come from the September 25, 2026 AAII v4.3.2 snapshot and are summarized in Table D.2.Source: §D.3 · p. 21, Table D.2 · p. 21

D.4 An Empirical Search Scaling Law

• The relation explains 92.7% of observed model-mean variation, with a root-mean-square error of 0.138 Sharpe.The capability term primarily shifts quality level, while iteration depth contributes a common, diminishing gain.Source: §D.4 · p. 22
• Across eight leave-one-model-out folds, both power-law and log-budget forms achieve a held-out RMSE of 0.193 Sharpe.The fits show a diminishing budget-performance curve; base capability strongly orders low-budget quality, while additional-search gains are less ordered by capability.Source: §D.4 · p. 22
• The relation frames models as starting at different quality levels, with deeper search narrowing gaps and gains per iteration decreasing over the measured range.Figure 3 traces each model through cost–quality space at three recorded budgets.Source: §D.4 · p. 22

E.1 Task-Level Metrics

• Task-level reporting presents Sharpe, monotonicity, and alignment for 50 tasks at iteration budgets 3, 6, and 10, using one repeated run per model.Checkpoint metrics are reported consistently across the three budgets.Source: §E.1 · p. 22

E.2 Research Cases

• The research cases pair higher- and lower-scoring trajectories for behavioral differences and quote translated persistent research memories with metrics and decisions preserved..Source: §E.2 · p. 31

E.2.1 Search Redirection after a Plateau

• Weekly refresh reduced turnover from 14.6% to 8.3% and raised net Sharpe from 1.368 to 1.753; share normalization raised it to 2.565.The model first found that extending the lookback reduced turnover but also reduced gross annual return, leaving net Sharpe essentially unchanged at 1.368 versus 1.336.Source: §E.2.1 · p. 32
• The higher-scoring revision changed both what the factor measured and how it was traded, following a diagnosed failure rather than another nearby parameter choice.The lower-scoring model instead continued smoothing the same impact-persistence representation by enlarging its aggregation window.Source: §E.2.1 · p. 32
• The lower-scoring trajectory repeatedly adjusted the window and extreme-observation treatment despite net Sharpe remaining negative at intermediate settings and monotonicity declining.At a 100-day window, net Sharpe became positive at 0.332 while monotonicity remained 0.806.Source: §E.2.1 · p. 32

E.2.2 Attribution of Experimental Feedback

• Component ablations showed XACC alone had Sharpe 2.362, ∆XACC alone 1.604, and GAP alone 0.256; GAP diluted XACC, prompting a weight reduction from 0.50 to 0.15.A small GAP weight improved middle-decile ordering, while ∆XACC was retained at a lower weight.Source: §E.2.2 · p. 32
• These cases distinguish evidence-based refinement from gains that arise through a different signal source or from feedback on an incorrectly implemented experiment.The controlled ablation separated a plausible economic story from measured component contribution.Source: §E.2.2 · p. 32, §E.2.3 · p. 33
• A stock-versus-date slicing bug invalidated earlier interpretations; after correction, the factor moved opposite the hypothesis, consistent with concentrated buying followed by continuation.Earlier explanations attributed feedback to mechanisms the implemented experiment had not tested.Source: §E.2.2 · p. 32

E.2.3 Objective-Consistent Candidate Selection

• A higher-Sharpe candidate was rejected when removing the zero-sum constraint introduced ordinary intraday reversal, raising its loading from 0.467 to 0.636.The candidate’s Sharpe was 3.866 versus 3.361 for the incumbent, while the pure-reversal control had Sharpe 2.754.Source: §E.2.3 · pp. 32–33
• MiniMax-M3’s blended candidate raised Sharpe from 0.184 for the pure K = 90 order-mix signal to 0.855, with perfect monotonicity, after adding 30% reversal-40.The selection retained the order-mix surprise and amplitude weighting, although the added component supplied an independent reversal signal.Source: §E.2.3 · p. 33