PaperlayerView exact revision on arXiv

Source-linked AI summary

AlphaBet: Reinforcement Learning for LLM Event Forecasting Agents

Liqin Ye, Rongzhi Zhang, Xin Liu, Zheng Li, Yuan He, Qingyu Yin, Priyanka Nigam, Chao Zhang

arXiv:2610.22684v1cs.LGcs.AI

TL;DR

Outcome-based forecasters often report probabilities from preassembled evidence at a single date, leaving evidence acquisition outside training. AlphaBet trains one LLM policy to search and report across multiple pre-resolution dates using proper-score feedback. It improves Brier scores on Forecast-Dojo and FutureX, with controlled analyses finding gains in both information acquisition and probability reporting.

  • Problem

    Outcome-based forecasting methods commonly train probability reporting from preassembled evidence at one prediction time, leaving evidence acquisition outside the trained policy.

    Source: §1 · p. 2

  • Method

    AlphaBet trains a single tool-using policy to acquire evidence and report probabilities across multiple pre-resolution dates, scoring each dated forecast against the eventual outcome.

    Source: §1 · p. 2

  • Results

    GLM-4.5-Air’s Brier score falls from 0.731 to 0.626 on Forecast-Dojo and from 0.612 to 0.528 on FutureX, with gains in evidence acquisition and probability reporting.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Multi-date outcome feedback can supervise forecasts under different information states, while regret analysis distinguishes evidence-acquisition and probability-reporting gaps.

    Source: §1 · p. 2

    The objective-equivalence claim assumes a policy exists that maximizes forecasting reward while satisfying all format checks.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Forecasting an unresolved event requires finding relevant evidence and deciding how strongly it supports each outcome. Yet outcome-based training of Large Language Model (LLM) forecasters often begins with preassembled evidence at a single prediction date, leaving evidence acquisition outside the learning loop. We introduce AlphaBet, an agentic reinforcement learning (RL) framework that trains a single LLM policy to search, reason, and report probabilities across multiple pre-resolution dates. Each dated forecast is scored by a proper scoring rule against the eventual outcome, allowing one resolved event to supervise decisions under different information states. We decompose forecasting regret into an evidence-acquisition gap and a probability-reporting gap, and show that Group Relative Policy Optimization (GRPO)'s standard-deviation normalization can distort the reporting target under multi-class proper scores. On GLM-4.5-Air, AlphaBet reduces the Brier score from 0.731 to 0.626 on Forecast-Dojo and from 0.612 to 0.528 on FutureX. Controlled ablations attribute these gains to multi-date training and show that they come from improvements in both information acquisition and probability reporting. Our empirical results validate that GRPO's std normalization degrades proper-scoring-rule performance while leaving top-outcome accuracy largely unchanged.

1 INTRODUCTION

Outcome-based forecasting methods often train probability reporting from preassembled evidence at one date, leaving evidence acquisition outside the policy. AlphaBet trains a single agent across multiple pre-resolution dates; experiments show gains on two benchmarks and improvements in both evidence acquisition and probability reporting.

Motivation

  • Existing outcome-based methods commonly optimize reporting from preassembled evidence at a single prediction time, leaving evidence acquisition outside the trained policy.The paper frames the resulting research question as how outcome feedback can train evidence-seeking and probability reporting as information changes.Source: §1 · p. 2

Framework

  • AlphaBet trains one tool-using policy to search evidence and report outcome probabilities at multiple pre-resolution dates, scoring each forecast against the eventual outcome.One resolved event therefore supervises decisions under multiple information states; cross-date memory is optional.Source: §1 · p. 2

Analysis

  • Forecasting regret separates into information-acquisition and probability-reporting gaps, while GRPO standard-deviation normalization can alter the intended multi-class probability target.At fixed acquired context, mean-centering preserves the expected reporting gradient induced by a strictly proper score, whereas standardization can reweight outcome classes.Source: §1 · p. 2

Empirical results

  • GLM-4.5-Air’s Brier score falls from 0.731 to 0.626 on Forecast-Dojo and from 0.612 to 0.528 on FutureX with AlphaBet.Multi-date training improves over single-date training, with controlled evidence–reader swaps identifying gains in both evidence acquisition and probability reporting.Source: §1 · p. 2

2 PRELIMINARIES

Longitudinal forecasting revisits an unresolved event on scheduled dates while admissible evidence accumulates. Proper scoring rules evaluate probability reports against outcomes, distinguishing the information available from what the agent acquires and reports.

2.1 LONGITUDINAL EVENT FORECASTING

  • A forecast episode revisits the same unresolved question on scheduled pre-resolution dates, producing a probability distribution over its mutually exclusive and exhaustive outcomes at each step.The event’s resolution criterion determines the eventual outcome.Source: §2.1 · p. 3
  • At each date, the agent may retrieve only information released by then, and the admissible corpus expands as new evidence arrives.The agent observes only part of the corpus through tool-based search and analysis.Source: §2.1 · p. 3
  • The dated-corpus target is the Bayes-optimal forecast from all information available by that date; the acquired-context target uses the agent’s search results and incoming notebook.The agent’s reported probabilities are based on acquired context; the notebook is empty when memory is disabled.Source: §2.1 · p. 3

2.2 PROPER SCORING RULES

  • Strictly proper scoring rules reward probability distributions directly and make truthful reporting optimal in expectation, with equality only when the report matches the outcome distribution.They provide both an evaluation criterion and a training signal from the reported distribution and eventual outcome.Source: §2.2 · p. 3
  • Proper-score optimal reporting against acquired context separates an information gap from a reporting gap relative to the dated-corpus target.The information gap compares the dated-corpus target with the acquired-context target; the reporting gap compares the acquired-context target with the agent’s report.Source: §2.2 · p. 3

3 METHOD

ALPHABET trains one policy to gather evidence and report forecasts at multiple dates, with each report scored against the eventual outcome. Its objective separates information-acquisition and reporting gaps, while its GRPO variant removes standard-deviation normalization to avoid distorting the reporting target.

3.1 RL ENVIRONMENT FOR LONGITUDINAL FORECASTING

  • At each scheduled date, one policy searches and reads date-available evidence, reasons over it, and reports a probability distribution, optionally carrying a belief notebook forward.When memory is off, each date starts without an agent-generated notebook from earlier dates.Source: §3.1 · pp. 3–4

3.2 TRAINING REWARDS AND OBJECTIVE

  • Each forecast step receives a forecasting score plus a format reward, and the training objective maximizes expected cumulative reward across dates.Under the stated assumption that a forecasting-optimal policy can satisfy format checks, every training-objective maximizer is also forecasting-optimal.Source: §3.2 · p. 4
  • The objective decomposes forecasting regret into information-acquisition and probability-reporting gaps; zero total gap requires both the acquired-context target and dated-corpus target to match the report.The equivalence assumes no outcome leakage and a strictly proper score; the format reward does not alter the forecasting optimum under the stated feasibility assumption.Source: §3.2 · p. 5

3.3 POLICY OPTIMIZATION: GRPO FOR CALIBRATED FORECASTING

  • Mean-centering preserves the expected score-only reporting gradient, whereas standard-deviation normalization can reweight outcome classes and shift the elicited target.The result concerns the fixed-context, score-only update; memory-on groups use per-date rewards without assigning later rewards to earlier notebook actions.Source: §3.3 · pp. 5–6

4 EXPERIMENTS

Experiments compare ALPHABET with forecasting baselines and test how longitudinal supervision, evidence acquisition, and GRPO reward normalization affect forecasts. Results show improvements across benchmarks and identify gains in both evidence acquisition and probability reporting.

4.1 EXPERIMENTAL SETUP

  • Training uses GLM-4.5-Air on 1,028 events, generating 3,965 event–date tasks, with retrieval restricted to news available by each forecast date.The evaluation covers Forecast-Dojo and FutureX, using a temporally indexed corpus and date-restricted retrieval.Source: §4.1 · pp. 6–7

4.2 FORECASTING PERFORMANCE

  • An 8.9-point accuracy gain and Brier reduction from 0.731 to 0.626 on Forecast-Dojo accompany ALPHABET’s gains across both benchmarks over harness and RL baselines.Table 1 marks best and second-best point estimates within the GLM-4.5-Air block and reports benchmark-specific 95% confidence intervals.Source: Table 1 · p. 7, §4.2 · p. 7

4.3 GAIN ANALYSIS: EVIDENCE ACQUISITION AND PROBABILITY REPORTING

  • Across readers, HALPHABET evidence improves Brier by 0.011–0.016, while ALPHABET improves reporting by 0.058–0.060 on fixed evidence; the question-only advantage is 0.035.The evidence comparison includes independently trained readers; the question-only condition accounts for part of the reporting gap.Source: Table 2 · p. 8, §4.3 · pp. 8–9

4.4 LONGITUDINAL SUPERVISION

  • Longitudinal Brier gains are 0.072 beyond 14 days, 0.051 at 7–14 days, 0.035 at 3–7 days, and 0.017 within three days, largest farther from settlement.On Forecast-Dojo, memory-free training reduces Brier from 0.668 to 0.626 and raises accuracy from 47.2% to 50.5% versus single-date training.Source: Figure 3 · p. 8, §4.4 · p. 9

4.2 FORECASTING PERFORMANCE

  • Memory-on is slightly worse than memory-free on every metric; the authors suggest independent date rewards give no direct credit for a notebook that helps later forecasts.ALPHABET remains stronger than single-date and base-model baselines on most metrics and approaches Claude Sonnet 4.6 across both benchmarks.Source: §4.2 · p. 8

4.5 EFFECT OF GRPO REWARD STANDARDIZATION

  • GRPO standard-deviation normalization worsens Brier and log scores while accuracy remains comparable; degradation is primarily in reliability rather than resolution and manifests as excessive sharpening.Removing normalization yields significant proper-score gains without a significant accuracy difference.Source: Figure 3 · p. 8, §4.5 · p. 9

5 RELATED WORK

Prior work develops event-forecasting environments and trains language-model forecasters, including through outcome-based reinforcement learning. ALPHABET extends these directions to longitudinal agentic learning and examines how GRPO normalization affects probability reporting.

Event forecasting benchmarks and environments

  • Forecasting environments progress from time-restricted news benchmarks to live unresolved-question evaluations and agentic historical or chronological research tools.Forecast-Dojo supplies aligned dates and temporally restricted tools; ALPHABET uses its dated interactions for outcome-based RL across information states.Source: §5 · p. 10

Learning to forecast with language models

  • Prior outcome-based RL methods largely optimize isolated single-date forecasts, whereas ALPHABET jointly optimizes evidence acquisition and probability reporting across dates.Earlier approaches include human-forecast supervision, reasoning-trace fine-tuning, and explicit beliefs during iterative tool use.Source: §5 · p. 10

Reward normalization and probabilistic calibration

  • A proper scoring-rule reward does not ensure that GRPO preserves its elicitation incentives; ALPHABET analyzes multi-class target shifts and reports degraded probabilistic forecasting with largely unchanged accuracy.Prior work found standard-deviation normalization can induce overconfidence for binary stochastic outcomes [Bereket & Leskovec, 2025].Source: §5 · p. 10

6 CONCLUSION

ALPHABET trains an LLM agent to acquire evidence and report probabilities as information evolves; experiments show gains over single-date training in both capabilities.

6 CONCLUSION

  • Experiments on Forecast-Dojo and FutureX show consistent gains over single-date training, with evidence–reader swaps identifying improvements in evidence acquisition and probability reporting.The conclusion also describes the framework’s training of a single agent across evolving information states.Source: §6 · p. 10

A.1 EFFECT OF THE FORMAT REWARD

  • Under the format-feasibility assumption, every maximizer of the training objective preserves the forecasting optimum, but this does not imply unchanged gradients or that every training improvement improves forecasting.The guarantee depends on an optimal forecasting policy that passes every format check almost surely.Source: §A.1 · pp. 14–15

A.2 PROPERNESS OF THE AGENTIC FORECASTING OBJECTIVE

  • The objective’s expected regret decomposes into information-acquisition and reporting gaps, but improvement in the total need not reduce either gap separately or improve every forecast date.One resolved outcome can score multiple dated forecasts without imposing the same target distribution at every date.Source: §A.2 · p. 16

A.3 SCOPE OF THE PROPERNESS GUARANTEE

  • The Brier and ideal logarithmic scores are strictly proper under stated conditions, but flooring the logarithmic score at 10^-3 does not preserve properness on the full simplex.The floored score agrees with the ideal score when the realized-outcome probability is at least the floor, yet that pointwise agreement does not preserve the optimum.Source: §A.3 · pp. 16–17

A.4 MEAN-CENTERING PRESERVES THE PROPER-SCORE UPDATE

  • For a fixed context, mean-centering preserves the expected score-only reporting update direction, scaling it by 1 − 1/G for finite G > 1.This local analysis excludes the mixed reward containing format penalties.Source: §A.4 · pp. 17–18

A.5 STANDARD-DEVIATION NORMALIZATION AND THE TILTED TARGET

At a fixed information state, GRPO standard-deviation normalization generally changes the proper-score reporting target; it preserves that target only when outcome-specific score scales are equal. The analysis also describes the forecasting tasks, dated evidence access, and agent interface used in implementation.

A.5 STANDARD-DEVIATION NORMALIZATION AND THE TILTED TARGET

  • Standardization preserves the original reporting target if and only if all outcome-specific score standard deviations are equal; otherwise, it elicits a different target.The limiting update is a positive multiple of a frozen-target proper-score gradient, whose optimal report differs from the acquired-context target when the effective target is tilted.Source: §A.5 · p. 21
  • With zero epsilon, rescaling rewards by outcome-specific positive constants leaves normalized advantages unchanged, although unequal scales can change the reporting target.With positive epsilon, exact scale invariance no longer holds; the target is characterized by the large-group analysis instead.Source: §A.5 · pp. 21–22
  • When outcome probabilities and score scales have the specified opposite ordering, the effective target does not decrease higher-to-lower probability ratios and majorizes the original distribution.Without that ordering, the tilt can instead flatten the distribution or shift probability across classes; distortion alone does not imply overconfidence.Source: §A.5 · p. 22
  • The proof fixes the acquired context and notebook state, so it does not exactly analyze rollout groups with heterogeneous search results or histories.It characterizes a local reporting effect, not attainability by the restricted policy or convergence of training; implemented sequential updates also lack cross-date credit assignment.Source: §A.5 · pp. 21–23

B.1 FORECASTING TASKS

  • Events supply questions, resolution criteria, outcome labels, and forecast schedules; memory-on episodes carry notebook state across dates, while memory-free date pairs are independent.The single-date comparison fixes one sampled date per event throughout training.Source: §B.1 · p. 23

B.2 TEMPORAL INFORMATION ACCESS

  • The offline CC-News corpus uses timestamped article snapshots, and retrieval at each forecast date excludes articles dated later than that date.Advancing to a scheduled date expands the searchable corpus.Source: §B.2 · p. 23

B.3 AGENT INTERFACE

  • The agent researches each dated question through Search, Read, and Compute, then returns outcome probabilities; memory-on steps also pass forward a belief notebook.The resolved outcome and reward feedback stay outside the agent's context during rollout generation.Source: §B.3 · pp. 23–24
  • Each forecast date allows up to 80 tool rounds and 200 tool calls, while each model call has a 16,384-token generation limit.Python calls are limited to 10 seconds and 4 GiB of memory.Source: §B.3 · p. 24

B.4 REWARD IMPLEMENTATION

The implementation checks forecast and notebook formats, scores normalized probability reports, and trains policies with date-specific GRPO advantages. It also adapts BLF and OpenForecaster to the dated offline-evidence setting and describes the experiment materials.

B.4 REWARD IMPLEMENTATION

  • Forecast validity checks cover strict JSON, numeric probabilities, and probability mass near one; memory-on runs also check notebook structure and agreement with the answer.The forecasting score clips negative entries and normalizes the resulting distribution; log rewards floor the realized-outcome probability, limiting the exact properness guarantee.Source: §B.4 · pp. 24–25

B.5 POLICY OPTIMIZATION

  • GRPO advantages subtract each dated group's mean reward without standard-deviation division, while the ablation adds division by the sample standard deviation plus 10^-6.Each step's advantage weights generated reasoning, tool calls, probability reports, and notebook writing, but memory-on notebook actions receive no credit for later forecasts' rewards.Source: §B.5 · pp. 25–26

B.6 BASELINE IMPLEMENTATIONS

  • The paper adapts BLF and OpenForecaster to GLM-4.5-Air and evaluates them with ALPHABET's dated offline evidence.Table 3 lists GLM-4.5-Air training configurations, with paired values ordered memory-free then memory-on.Source: Table 3 · p. 25, §B.6 · p. 27

B.6.1 BLF

  • The BLF adaptation adds an explicit belief state to the frozen model and connects it to Search, Read, and Compute for research and belief updates.It requires categorical distributions, clips binary forecasts to [0.05, 0.95], and averages four trial distributions before scoring.Source: §B.6.1 · p. 27

B.6.2 OPENFORECASTER

  • The OpenForecaster reproduction trains on synthetic questions with fixed retrieved passages and produces one answer-and-confidence completion without research-tool interaction.Its training questions come from a filtered news-article pipeline, and its evaluation adapts the answer-and-confidence output to the shared dated research loop.Source: §B.6.2 · p. 27

C EXPERIMENTAL DETAILS

  • Forecast-Dojo figures show event primary-domain shares and dated-task forecast-horizon distributions; the split-statistics table distinguishes event counts from dated-task horizons.The figure captions specify that labels mark domain shares of at least 5% and horizon counts appear in parentheses.Source: Figure 5 · p. 28, Figure 6 · p. 28, Table 5 · p. 28

C.1 DATASETS AND TEMPORAL SPLITS

The evaluation uses event-disjoint Forecast-Dojo splits and a 343-item FutureX L1–L2 pool, applying dated cutoffs and excluding specified leakage or timestamp cases.

C.1 DATASETS AND TEMPORAL SPLITS

  • Forecast-Dojo training contains 1,028 events and 3,965 dated tasks, with validation and test events held apart by event identifier.Validation includes 80 events closing in January or February 2026; test events close between March and May 2026.Source: §C.1 · pp. 28–29
  • Forecast-Dojo excludes question-date pairs with timestamp errors or evidence already disclosing the outcome, and each retained episode has at least three forecast steps.Domain shares are counted over events, whereas horizon shares are counted over dated tasks.Source: §C.1 · p. 29
  • FutureX evaluation uses 343 L1–L2 items, with forecasts scheduled for the Wednesday strictly before each recorded end date in UTC+8.The pool includes yes/no, multiple-choice, and free-form questions; probability-based Brier evaluation applies to categorical reports.Source: §C.1 · p. 29

C.2 MODELS AND EVALUATION PROTOCOL

Evaluation uses shared settings on Forecast-Dojo and FutureX; only ALPHABET memory-on on Forecast-Dojo carries a belief notebook between dates.

C.2 MODELS AND EVALUATION PROTOCOL

  • All model knowledge cutoffs and open-weight checkpoint release dates precede the March–May 2026 evaluation period, and GLM-4.5-Air predates the training window.Open-weight models use identical sampling parameters; Claude Sonnet 4.6 uses provider defaults.Source: §C.2 · p. 29

C.3 SCORING AND STATISTICAL ANALYSIS

The evaluation reports scoring and aggregation procedures for Forecast-Dojo and FutureX, with bootstrap inference over matched evaluation units. The gain analysis separates evidence acquisition from probability reporting using controlled evidence–reader comparisons.

C.3 SCORING AND STATISTICAL ANALYSIS

  • Forecast-Dojo uses positive Brier loss without format penalties, while FutureX computes Brier only from usable categorical reports and evaluates single-answer accuracy against resolved labels.Unusable Forecast-Dojo forecasts receive Brier loss 2, log score log 10−3, and accuracy zero; FutureX missing or unparseable answers score zero.Source: §C.3 · p. 29
  • Usable probability vectors are averaged across four trials before scoring each dated task, while other agents’ rollout scores are averaged individually.Table 7 identifies evaluation settings for all models on Forecast-Dojo and FutureX.Source: §C.3 · p. 29, Table 7 · p. 30
  • Percentile 95% confidence intervals use bootstrap resampling of Dojo events or FutureX items, and paired analyses compare methods on matched question–date tasks.The main table uses 2,000 draws; gain, horizon, and calibration analyses use 4,000, and paired GRPO comparisons use 10,000.Source: §C.3 · p. 30

C.4 EVIDENCE ACQUISITION AND PROBABILITY REPORTING

  • For each Dojo question–date, the analysis freezes four evidence sets from base-agent rollouts and four from memory-free ALPHABET, retaining retrieved search results and successful reads.Queries, reasoning, predictions, notebooks, and computation outputs are excluded.Source: §C.4 · p. 30
  • Tool-free readers evaluate each evidence set with the same question and forecast date; matched-task Brier reductions isolate acquisition gains by changing evidence and reporting gains by changing the reader.The readers include the base model, ALPHABET, GPT-5.6 Terra, and Claude Sonnet 4.6; question-only comparisons measure reader differences without retrieved evidence.Source: §C.4 · p. 30
  • GLM readers use temperature 0.6 and top-p 1.0, with truncated reports retried at 80,000 tokens and unusable reports assigned uniform probabilities.The initial token limit is 32,768.Source: §C.4 · p. 30

C.5 TRAINING ABLATIONS

The ablations compare longitudinal with single-date training and examine reward standardization under controlled training configurations. Diagnostics characterize the standard-deviation variant as excessively sharpening forecasts, while the infrastructure appendix reports substantial compute requirements.

C.5 TRAINING ABLATIONS

  • Figure 7 shows the validation trajectory for the memory-free variant, without additional trajectory values stated in the supplied caption or prose.The caption identifies validation Brier loss and accuracy as the plotted measures.Source: §C.5 · p. 30, Figure 7 · p. 31
  • The longitudinal and single-date controls use the same 1,028 training events and are compared on the same 797 Dojo question–date pairs.The single-date control retains one fixed date per event; both setups match the base checkpoint, reward, optimizer, rollout-group configuration, and optimizer samples per step.Source: §C.5 · pp. 30–31
  • The reward ablation compares std and no-std updates separately for negative Brier and log-score rewards while holding the base model, data, rollouts, penalties, optimizer, and retrieval configuration fixed.The std update divides mean-centered advantages by the group sample standard deviation plus 10−6; the no-std update omits that division.Source: §C.5 · p. 31
  • At step 39, Figure 8 compares policies only on shared valid forecasts: 3,064 for the Brier-reward pair and 3,165 for the log-reward pair, out of 3,188 per run.Accuracy differences are not significant in these comparisons (p = 0.099 and p = 0.114).Source: §C.5 · p. 31, Figure 8 · p. 31
  • At step 39, standard-deviation normalization is associated with higher confidence, lower entropy, and larger reliability error than no-std updates, characterizing excessive sharpening.Figure 8 presents binned Brier decomposition and probability diagnostics on common-valid forecasts; Table 8 reports output validity and all-forecast scores, including invalid outputs at worst-case scores.Source: Figure 8 · p. 31, Table 8 · p. 31, §C.5 · p. 32

D COMPUTING INFRASTRUCTURE

  • A 120-step memory-free run takes approximately 62 hours on the stated allocation, and memory-on training takes approximately 2.3× longer per step.The setup uses 136 NVIDIA H200 GPUs across 17 nodes, with 128 GPUs for training and rollout generation and 8 for retrieval.Source: D COMPUTING INFRASTRUCTURE · p. 32

E LIMITATIONS AND FUTURE WORK

The limitations concern the training data, retrieval corpus, and predefined outcome format, with future directions aimed at broadening coverage and extending training. The appendix prompts specify dated evidence retrieval, probability reasoning, and, for memory-on forecasting, reuse of earlier research.

E LIMITATIONS AND FUTURE WORK

  • Training reflects Polymarket topic selection and question framing, uses English CC-News articles, and trains probabilistic forecasts over predefined outcomes rather than free-form answers.The authors identify broader event sources, multilingual or structured evidence, and free-form answer training as future directions.Source: E LIMITATIONS AND FUTURE WORK · p. 32

F.1 MEMORY-FREE FORECASTING

  • The memory-free prompt requires a probability distribution, proper-score-aware uncertainty, and forecasts based only on the question and retrieved evidence.It directs the agent to establish a base rate, search targeted clues, read full articles, and use code for computable parts.Source: §F.1 · p. 32, System Prompt: Memory-Free Forecasting… · p. 33

System Prompt: Memory-Free Forecasting (continued)

  • Forecast-date instructions restrict evidence to what was available by that date, while search, article retrieval, and computation are specified as the agent’s tools.The prompt also instructs the agent to read resolution criteria precisely and track time remaining.Source: System Prompt: Memory-Free Forecasting… · p. 33

F.2 MEMORY-ON FORECASTING

  • The memory-on prompt adds a belief notebook for later forecasts of the same question, directing the agent to build on and revise earlier research.Its workflow also emphasizes base rates, targeted evidence gathering, computation, and calibrated probabilities.Source: §F.2 · pp. 34–35

F.3 BLF FORECASTING

  • The BLF prompt likewise directs forecasters to use base rates, targeted evidence, computation, scenario analysis, and granular probabilities rather than reflexive uniform splits.It cautions against invented computational inputs and treats computed results as evidence conditioned on their assumptions.Source: §F.3 · pp. 37–38

F.4 GAIN ANALYSIS

The gain-analysis reader receives dated evidence without search or code tools, then forecasts using the stated resolution criteria and a base-rate starting point.

F.4 GAIN ANALYSIS

  • The gain-analysis reader receives retrieved articles available before the forecast date but has no search, browsing, or code tools and cannot request additional evidence.The evidence may include article titles, sources, publication dates, summaries, and sometimes full text.Source: §F.4 · p. 41
  • The reader is instructed to apply the exact event definition, measurement source, and resolution date, since forecasting the wrong quantity scores zero.The prompt also asks the reader to note the forecast date and remaining time.Source: §F.4 · p. 41
  • The reader anchors an initial estimate to the base rate or typical split for the relevant reference class before considering case-specific details.The prompt frames this outside view as a way to keep a vivid but unrepresentative story from dominating.Source: §F.4 · p. 41

Related papers

Loading 2610.22684v1…