PaperlayerView exact revision on arXiv

Source-linked AI summary

Do Your Own Research: Learning to Forecast by Learning to Search

Yusuf Afifi, Artur Kiulian, Anton Polishko, Mykola Khandoga, Hamudi Naanaa, Alina Krasnobrizha

arXiv:2610.01955v1cs.LG

TL;DR

Prior work either freezes research context before training or introduces agentic research at test time, leaving the learned policy without training-time control over evidence acquisition. The agent chooses searches, pages, and time series during multi-turn rollouts in a leak-filtered pre-cutoff environment, and is trained with outcome-based reinforcement learning. The RL-trained policy scores 0.254 versus Claude Opus 4.5’s 0.256 and finishes ahead of every frontier model tested.

  • Problem

    Prior work either freezes research context before training or introduces agentic research at test time, leaving the learned policy without training-time control over evidence acquisition.

    Source: §1 · pp. 1–2

  • Method

    The agent chooses searches, pages, and time series during multi-turn rollouts in a leak-filtered pre-cutoff environment, and is trained with outcome-based reinforcement learning.

    Source: §1 · p. 2, §2 · p. 2

  • Results

    The RL-trained policy scores 0.254 versus Claude Opus 4.5’s 0.256 and finishes ahead of every frontier model tested.

    Source: §4 · p. 3

  • Takeaways & Limitations

    The trained policy makes fewer search attempts while accuracy rises and calibration error falls 31%.

    Source: §5 · p. 4

Abstract

from arXiv · show

Outcome-based reinforcement learning can train language models to forecast real-world events, but prior forecasting work either freezes research context before training or deploys agentic research only at test time, so the skill of gathering evidence is never shaped by the reward. We introduce an agentic forecasting environment, dataset, and harness built from 2,100+ resolved Polymarket questions; the agent acquires its own context at rollout time (web search, page reading, and financial time series, all restricted by layered leak filtering to information published before each question's cutoff), and we train Qwen3.5-35B-A3B (3B active parameters) on it with single-epoch GRPO under a Brier-score reward. Training changes how the agent interacts with information: calibration improves 30-40%, and search attempts fall from 3.8 to 2.25 per rollout as evidence discipline is learned. Evaluated in an identical harness against four frontier models, the trained policy also finishes ahead of every frontier model tested at evidence-based forecasting, including Claude Opus 4.5 (soft-Brier 0.254 vs. 0.256, n=265), at about 5% of the inference cost, and its margin is widest on the hardest questions, the ones the crowd itself had not decided. We release the environment, dataset, and per-rollout records as a reusable harness for temporal forecasting agents.

1 Introduction

Prior forecasting work either freezes research context before training or introduces agentic research only at test time. This paper trains agents to acquire evidence during forecasting and reports results ahead of the tested frontier models.

1 Introduction

  • Earlier outcome-based RL work used context frozen before training, while test-time agentic research leaves evidence gathering outside the training reward.The paper identifies this as a gap in shaping the skill of acquiring evidence.Source: §1 · pp. 1–2
  • The proposed approach puts a multi-turn search-and-forecast scaffold inside the RL loop, with layered filtering restricting evidence to material published before each question’s cutoff.At rollout time, agents choose searches, pages to read, and time series to retrieve.Source: §1 · p. 2
  • The released dataset, environment, and harness include 2,113 training and 265 held-out resolved Polymarket questions with strict temporal eligibility.The harness supports multi-turn agentic forecasting in training and testing.Source: §1 · p. 2
  • The evaluation runs trained, untrained, and four frontier models with the same tools, filters, and turn budget, alongside per-rollout capture and trace-level audits.The comparison includes platform-served policies.Source: §1 · p. 2
  • Single-epoch GRPO puts the 35B-A3B model ahead of every tested frontier model at evidence-based forecasting, at about 5% of their inference cost.The reported margins are widest on uncertain questions.Source: §1 · p. 2

2 The environment and dataset

The environment presents resolved binary Polymarket questions with a forecast cutoff and lets agents gather evidence through a multi-turn tool loop. Temporal filtering, dataset eligibility rules, and a Brier-based reward define the training setup.

Task and harness

  • The agent forecasts a resolved binary question using only information available before its cutoff, through a tool loop of up to 10 turns.Tools include web search, URL reading, cutoff-truncated time series, revision-dated Wikipedia, trend extrapolation, and probability submission.Source: §2 · p. 2

Leak filtering

  • Three leak-filtering layers block known prediction-market sites, remove post-cutoff material using date heuristics, and apply a per-result keep/drop judgment.The paper reports Claude Haiku scored perfectly among five judges benchmarked on labeled cases; alternatives erred toward over-dropping valid evidence.Source: §2 · p. 2

Dataset

  • The dataset filters for at least $5,000 volume, cutoff prices in [0.10, 0.90], and cutoffs 7–90 days before resolution; test questions resolve after training questions.The training split has 2,113 questions and the held-out test split has 265.Source: §2 · p. 2
  • Figure 1 reports Soft-Brier on the full held-out split (n=265) and the pre-declared uncertain subset (n=104), with [0, 0.24] compressed for readability.The uncertain subset uses cutoff prices in [0.30, 0.70].Source: Figure 1 · p. 3

Reward and objective

  • A submitted probability earns one minus its Brier score, and the policy is trained with single-epoch GRPO and LoRA on Qwen3.5-35B-A3B.Non-submission earns 0.55; the model has 3B active parameters and training uses 33 steps × 64 questions × 8 rollouts.Source: §2 · p. 3

3 Evaluation setup

The evaluation uses a shared agentic environment and reports soft-Brier and calibration error for evidence-based forecasts without market prices.

Evaluation setup

  • Trained, untrained, and frontier policies run with the same tools, leak filters, turn budget, and search budget.Qwen policies use the training platform’s inference stack; frontier models use an API proxy.Source: §3 · p. 3
  • Soft-Brier imputes 0.25 when a policy declines to answer, while ECE measures confidence–frequency gaps over submitted forecasts in ten equal-mass bins.Both metrics are lower-is-better.Source: §3 · p. 3
  • Evaluations use live retrieval without market prices, which the paper says makes this evidence-based forecasting rather than price-copying.With market-price access, models converge on the crowd without reliably beating it.Source: §3 · p. 3

4 Results

The trained policy leads the tested frontier models on evidence-based forecasting, with its advantage largest when market probabilities are uncertain. Training also improves calibration and coverage while reducing search attempts, and the model’s inference cost is about 5% of Opus 4.5’s.

Beating the frontier

  • The RL-trained policy finishes ahead of every tested frontier model, edging Claude Opus 4.5 (0.254 vs. 0.256) and leading Sonnet, Gemini Pro, and Flash by 0.019–0.032.Source: §4 · p. 3

Margins widen in low-confidence markets

  • On 104 questions with cutoff prices in [0.30, 0.70], the trained policy scores 0.274, leading Opus by 0.007 and other frontier models by 0.021–0.045.Every paired per-question gap is larger than on the full set.Source: §4 · p. 4

Calibration and coverage improve

  • ECE falls 0.185→0.128 (−31%) and coverage rises from 79% to ∼100%; search attempts drop from 3.8 to 2.25 per rollout.The trained policy re-queries less after empty or filtered results and reasons more from surviving evidence; Figure 3 compares base and trained calibration and coverage.Source: §4 · p. 4, Figure 3 · p. 4

5 Conclusion

The conclusion argues that integrating research into RL training addresses prior approaches’ dependence on externally supplied or test-time context. The authors report improved forecasting and search discipline, while identifying expansion beyond binary prediction markets as future work.

5 Conclusion

  • Integrating context acquisition into training is presented as addressing scaffold dependence and improving forecasting and search discipline, with expansion beyond binary markets left for future work.The conclusion cites search attempts falling from 3.8 to 2.25 per rollout and calibration error falling 31%.Source: §5 · p. 4

A Why the market price is withheld

Market-price access can make all policies converge toward the crowd without reliably beating it, and comparisons can reverse depending on who receives that price.

A Why the market price is withheld

  • With market prices, policies converge toward the crowd and none reliably beats it; comparisons between agents can reverse depending on who sees the price.The reported soft-Brier band with prices is 0.186–0.215 versus the crowd’s 0.189.Source: A Why the market price is withheld · p. 5

B Example rollouts

Two archived agents research the same Trump–Iran meeting question and submit different probabilities based on their evidence.

B Example rollouts

Loading 2610.01955v1…