PaperlayerView exact revision on arXiv

Source-linked AI summary

ForeDreamer: A Self-Evolving Dual-Agent Memory Architecture for Future Event Prediction

Linhao Zhong, Zongze Du, Linyu Wu, Yu Bo, Hourong Li, Chenchen Jing, Hao Chen, Yuling Xi, Chunhua Shen

arXiv:2608.20920v1cs.CL

TL;DR

Open-web future event prediction lacks memory mechanisms that reliably transform noisy, conflicting search evidence into forecast-ready information. ForeDreamer uses a dual-agent architecture with question-specific factual memory and persistent experiential memory, evolving both forecasting and evidence-processing behavior. On Prophet Arena and FutureX, it demonstrates effectiveness, while the authors note limited resolved feedback and possible validation-set overfitting.

  • Problem

    Existing retrieval and memory mechanisms often pass retrieved information directly to agents or apply lightweight selection, leaving noisy and conflicting open-web evidence insufficiently processed for forecasting.

  • Method

    ForeDreamer separates factual memory from experiential memory and uses a forecasting agent plus a memory-processing subagent with MemGuides and executable MemTools.

  • Results

    ForeDreamer achieves the best average Brier performance among compared methods on Prophet Arena for both backbone models and the strongest accuracy on FutureX.

  • Takeaways & Limitations

    Open-web forecasting benefits from treating memory as evolving experience for both evidence processing and prediction, rather than only as stored information.

  • Takeaways & Limitations

    Resolved forecasting feedback is scarce, so the framework evolves and validates memory using a small feedback pool that may raise concerns about validation-set overfitting.

Abstract

from arXiv · show

Open-web future event prediction requires agents to distill reliable signals from noisy, redundant, and incomplete evidence. Existing retrieval/memory mechanisms directly feed retrieved information to agents or rely on simple memory functions such as storing and reusing prior information for prediction, leaving them insufficient for open-web forecasting. We propose to transform raw web evidence into structured memory before prediction, enabling agents to reason over distilled, question-specific evidence rather than noisy retrieval results. This paper presents ForeDreamer, a self-evolving dual-agent framework for managing memory over open-web evidence. ForeDreamer separates factual memory, a question-specific evidence state for the current forecast, from experiential memory, persistent agent experience accumulated across forecasting episodes. It uses a main agent for search and prediction, and a memory-processing subagent to convert search results into factual memory with dedicated tools. ForeDreamer further evolves experiential memory through two tracks, improving both forecasting decisions and factual-memory construction. Experiments on Prophet Arena and FutureX demonstrate the effectiveness of ForeDreamer. Project page: https://zhongzero.github.io/ForeDreamer

1 Introduction

Open-web future event prediction requires memory that can organize noisy, conflicting evidence into a coherent basis for calibrated forecasts. ForeDreamer addresses this gap by separating question-specific factual memory from persistent experiential memory and evolving both forecasting and evidence-processing behavior.

  • Motivation: Open-web forecasting requires agents to organize heterogeneous, noisy, redundant, and conflicting evidence before producing calibrated predictions.Traditional memory mechanisms are designed for cleaner, user-centered contexts and do not directly process open-web evidence into forecast-ready form.
  • Memory roles: ForeDreamer separates question-specific factual memory for the current forecast from experiential memory accumulated across forecasting episodes.Factual memory functions as a processed evidence interface, while experiential memory guides future forecasting and evidence-processing decisions.
  • Research questions: The paper asks how agents can make reliable predictions from noisy open-web evidence and how forecasting feedback can improve both memory construction and forecasting decisions.These two research questions define the framework’s forecasting and self-evolution objectives.
  • Framework: The framework uses a main agent for search and prediction and a memory-processing subagent that constructs question-specific factual memory from search results.The subagent follows a MemGuide and uses executable MemTools to process evidence.
  • Framework: ForeDreamer evolves experiential memory through separate tracks for forecasting decisions and procedural evidence processing.These tracks guide search planning, evidence integration, forecast calibration, and conversion of search results into factual memory.

2 ForeDreamer Framework

ForeDreamer separates forecasting from evidence processing: a main agent searches and predicts using experiential memory, while a memory-processing subagent transforms temporally valid web results into factual memory. Rollout feedback then evolves textual forecasting experience and procedural evidence-processing experience through two validated tracks.

  • Architecture Overview: ForeDreamer separates factual memory for the current question from experiential memory accumulated across forecasting episodes.The main agent uses experiential memory, while factual memory provides processed evidence for the current forecast.
  • Evidence Processing: A memory-processing subagent converts normalized search-result artifacts into factual memory using a selected MemGuide and executable MemTools.The subagent can iteratively read and write artifacts before returning processed memory to the main agent.
  • Forecasting Workflow: The main agent conditions on the question and experience bank, performs multi-turn search and reasoning, integrates factual memory, and produces the final prediction.Search is constrained by the instance’s temporal cutoff and proceeds within a fixed turn budget.
  • Forecasting Workflow: The workflow accumulates factual memory across executed search-and-processing turns, after which the main agent answers without another search call.ForeDreamer records interactions, artifacts, ground truth, and metrics from the rollout as self-evolution feedback.
  • Self-Evolution: Rollout feedback evolves experiential memory through textual edits to the Experience Bank and procedural updates to the MemGuide tree and MemTools.The textual track improves search planning, evidence integration, and calibration, while the procedural track generates and validates new processing assets.

3 Dual-Track Experience Evolution

ForeDreamer evolves textual forecasting experience and procedural evidence-processing experience through validated updates. Procedural evolution combines provenance-preserving guide expansion with tool reuse and diversity-guided exploration to reduce redundancy and broaden pipeline exploration.

  • ForeDreamer evolves two experience tracks: textual forecasting experience guides search, evidence integration, and calibration, while procedural experience converts search results into factual memory.
  • Textual Forecasting Experience Evolution: Textual evolution edits the experience bank through validated ADD, MODIFY, or REMOVE operations and retains candidate updates only when validation improves.
  • Vanilla Rollout-Based Expansion: Vanilla procedural evolution expands a MemGuide tree by selecting a parent, sampling a rollout, deriving a design requirement, generating MemGuide–MemTool candidates, and validating admission.
  • Vanilla Rollout-Based Expansion: Validity requires successful validation runs on all examples in Devo, while the tree preserves provenance but can produce redundant tools and overconcentrate on locally successful guide families.
  • Compositional Tool Reuse: At token-Jaccard threshold τ = 0.5, 201 generated MemTools collapse into 98 clusters, a 48.8% cluster ratio indicating near-duplicate operations and wasted evolution budget.
  • Compositional Tool Reuse and Diversity-Guided Exploration: Compositional tool reuse selects compatible existing MemTools and generates new tools only for uncovered operations, while diversity-guided exploration adds category-diverse candidates without requiring a new rollout.

4 Experiments

ForeDreamer is evaluated on Prophet Arena and FutureX against retrieval, full-context, and agent-memory baselines, with ablations testing its dual-track evolution and procedural optimizations. It achieves the strongest reported benchmark performance, remains robust across search settings, and benefits from less redundant tools and more diverse guide pipelines.

  • Main performance comparison: ForeDreamer achieves the best average Brier performance on Prophet Arena for both backbone models and the strongest accuracy on FutureX.The comparison includes Full Text, generic RAG, and multiple agent-memory baselines.
  • Ablation of dual-track experience: Removing either textual forecasting-experience evolution or procedural evidence-processing evolution degrades performance, while the full system performs best on both benchmarks.The two tracks provide complementary benefits through the Experience Bank and evolved MemGuide–MemTool procedures.
  • Effect of procedural-evolution optimizations: Removing Compositional Tool Reuse, Diversity-Guided Exploration, or both lowers performance relative to the full procedural-evolution design.The ablation isolates both optimizations on Prophet Arena and FutureX.
  • Robustness across search settings: ForeDreamer consistently improves across alternative search and interaction settings on both benchmarks.The settings vary search provider, retrieval budget, or interaction configuration, indicating that gains are not tied to one configuration.
  • Check for potential data leakage: The gap between No Information and search-based settings indicates that external web evidence is needed for strong forecasting performance.No Information removes web search, whereas Full Text and ForeDreamer use temporally valid retrieved evidence.
  • Procedural-memory analysis: Compositional Tool Reuse produces a less redundant MemTool set, while Diversity-Guided Exploration encourages MemGuides to cover more pipeline archetypes.These analyses connect the procedural optimizations to reduced repeated tool construction and broader guide-family coverage.

5 Related Work

Related work covers memory-augmented language systems, self-evolving agents, and future-prediction benchmarks. These lines of research address retrieval and persistent memory, post-deployment improvement, and evaluation of information gathering, temporal reasoning, calibration, and unresolved-event decisions.

  • Agent memory: Early memory-augmented language systems relied largely on retrieval-augmented generation and dense or non-parametric memory, while recent work develops agent-memory systems.The section situates ForeDreamer within research on memory beyond prompt-local context.
  • Self-evolving agents: Self-evolving agents improve after deployment through reflection, experience reuse, skill discovery, or procedural modification.Prior work includes failed-trajectory reflection, iterative refinement, tool-use learning, and open-ended skill acquisition.
  • Future prediction: Future-prediction research evaluates information gathering, temporal reasoning, calibration, and decisions about unresolved events.Agentic forecasting benchmarks include MIRAI and ForecastBench, while live benchmarks reduce contamination by resolving questions after prediction.

6 Conclusion

ForeDreamer is a self-evolving dual-agent framework that separates local factual memory from persistent experiential memory and processes web evidence before prediction. Results on Prophet Arena and FutureX support treating memory as evolving experience for both evidence processing and forecasting.

  • Conclusion: ForeDreamer separates question-specific factual memory from experiential memory accumulated across forecasting episodes.The main forecasting agent reasons over processed evidence while a memory-processing subagent constructs the evidence interface with MemGuides and executable MemTools.
  • Conclusion: Experiments on Prophet Arena and FutureX show benefits from evolving experience for both evidence processing and prediction.The conclusion frames memory as more than stored information: it also guides procedural evidence processing and forecasting behavior.

Limitations

The evaluation is scoped to open-web future event prediction rather than traditional agent-memory benchmarks, and resolved forecasting feedback is scarce. ForeDreamer therefore evolves memory from a small feedback pool, raising a potential validation-overfitting concern while using held-out questions for evaluation.

  • Scope: Traditional agent-memory benchmarks are outside scope because they test storing, retrieving, or reusing clean interaction histories rather than evidence processing and forecasting.ForeDreamer is designed for noisy search results and forecast-ready factual memory.
  • Feedback limitation: Resolved forecasting feedback is scarce, so ForeDreamer evolves and validates memory using a small feedback pool that may raise validation-set overfitting concerns.The authors report evaluation on held-out forecasting questions not used for evolution or validation.

Ethical Considerations

The work uses public benchmark datasets for research evaluation and reports no additional ethical concerns specific to the study. It does not collect private user data or sensitive personal information, nor conduct human-subject experiments.

  • The study uses public benchmark datasets for open-web future event prediction.
  • The experiments do not collect private user data or annotate sensitive personal information.
  • The study does not conduct human-subject experiments and reports no additional ethical concerns specific to this work.

B More Methodological Details

ForeDreamer separates retrieval from evidence processing, converting retrieved items into structured factual memory through a memory-processing subagent. MemGuides constrain executable MemTools, while validation and compact summaries support controlled procedural evolution.

  • Search-and-Process Decomposition: ForeDreamer internally separates retrieval from processing while exposing one search-and-process interface to the main agent.Retrieval applies temporal cutoffs, normalizes evidence, and bounds the forwarded amount before processing begins.
  • Factual-Memory Representation: Each retrieved item becomes a structured record preserving the question, query, temporal constraint, source metadata, and content before factual-memory transformation.The resulting plain-text factual memory is consumed by the main agent and keeps the evidence path auditable.
  • MemGuide and MemTool Interface: A MemGuide specifies the processing workflow, while MemTools implement bounded callable operations for transforming raw evidence into factual memory.The guide coordinates available tools rather than executing operations directly.
  • Validation and Control: Generated procedural assets are validated for required fields, tool availability, syntax, interface consistency, and tool-calling compatibility before admission.Candidate assets are staged and loaded together only after these checks.
  • Procedural Memory Summaries: Compact MemTool and MemGuide summaries expose procedural history to generators without including every prior tool, guide, or rollout in the prompt.Validation records also support admissibility and benchmark-specific candidate ranking.

C.1 Detailed Descriptions of Evaluation Benchmarks

The evaluation uses Prophet Arena and FutureX, live or resolved future-event benchmarks designed to assess predictions against later outcomes. Baselines process retrieved evidence before forecasting, and evolution reserves examples for validation and evaluation.

  • Prophet Arena: Prophet Arena evaluates real-world event forecasts using Brier score and market-return-style metrics.The resolved snapshot contains 1,200 questions across eight categories, with Sports evaluated on a 400-example subset.
  • FutureX: FutureX gathers future-oriented web questions, records predictions before resolution, and scores them after resolution to reduce data contamination.Its event pool covers broad real-world domains and is refreshed over time.
  • Baseline Methods: RAG conditions generation on retrieved external evidence, while the listed memory baselines provide alternative retrieval, storage, or memory-organization mechanisms.These include HippoRAG 2, Mem0, MemoryOS, A-MEM, LightMem, and LangMem.
  • Baseline Methods: The baseline methods process evidence retrieved by the search module before producing the final forecast.
  • Dataset Splits and Evolution: Prophet Arena evolution uses five sampled examples per category, whereas FutureX uses 20 of 208 resolved examples for evolution and validation.Remaining examples are reserved for evaluation.
  • Evolution Protocol: Each evolution run uses 60 update iterations alternating procedural and declarative evolution before selecting top validation candidates for held-out testing.Procedural evolution revises MemGuides and MemTools; declarative evolution revises the Experience Bank.

D.1 Additional Results on Main Benchmarks

Additional experiments consistently support ForeDreamer across backbones, metrics, ablations, and search settings. Removing either experience-evolution track or procedural optimization weakens performance, while external evidence improves forecasting over no-information settings.

  • Additional Backbone Results: ForeDreamer achieves the strongest performance on both Prophet Arena and FutureX with GPT-5-Nano.These results also provide a model-side check because GPT-5-Nano predates the evaluated benchmark periods.
  • Auxiliary Metric: ForeDreamer obtains the strongest mean market return on Prophet Arena with Qwen3.5-Flash, consistent with the main Brier-score conclusion.
  • Dual-Track Experience Ablations: Removing either MemGuide–MemTool evolution or Experience Bank evolution degrades average performance across the additional dual-track ablations.
  • Procedural-Evolution Ablations: Removing both procedural optimizations or either individual optimization produces weaker average performance than ForeDreamer.The per-category results support contributions from both optimizations.
  • Search and Interaction Settings: ForeDreamer lowers average Brier score under every tested interaction, search-provider, retrieval-breadth, and context-budget setting.
  • Data-Leakage Checks: External evidence improves average forecasting performance across evaluated backbones compared with the No Information setting.The comparison contrasts question-only forecasting with settings using retrieved evidence.

E.1 Tool Similarity Analysis

The analysis diagnoses redundancy in vanilla procedural evolution by clustering 201 generated MemTools under complementary source-code similarity measures, then motivates compositional tool reuse.

  • Similarity analysis: At τ = 0.5, normalized sequence similarity forms 84 clusters, corresponding to a 41.8% cluster ratio.This metric is more sensitive to ordering and near-copy structure than token overlap.
  • Similarity analysis: At τ = 0.5, AST sequence similarity forms 136 clusters, corresponding to a 67.7% cluster ratio.AST comparison abstracts away lexical details and focuses on structural similarity.
  • Compositional tool reuse: Across FutureX and Prophet Arena, compositional tool reuse increases cluster ratios, indicating less aggressive merging and less redundant tool generation.The reported change is defined as the ratio with tool reuse minus the ratio without tool reuse.

E.2 MemGuide Pipeline Archetype Analysis

The pipeline-archetype analysis categorizes MemGuides by evidence-processing workflow and shows that diversity-guided exploration broadens procedural evolution beyond the initially dominant pattern.

  • Pipeline archetypes: The analysis identifies four recurring MemGuide pipeline archetypes based on overall evidence-processing structure.The archetypes are Tool-Orchestrated Processing, Evidence-first LLM synthesis, Specification-content decomposition, and Planning and validation.
  • Archetype distribution: Without diversity-guided exploration, all 10 MemGuides use Tool-Orchestrated Processing.This concentrates vanilla rollout-based expansion in one pipeline family.
  • Archetype distribution: With diversity-guided exploration, five guides use Tool-Orchestrated Processing, two use Evidence-first LLM synthesis, two use Specification-content decomposition, and one uses Planning and validation.The 10 guides are distributed across all four identified archetypes.
  • Archetype distribution: The resulting distribution expands procedural evolution beyond local variants of the initially dominant guide pattern.Figure S3 summarizes the before-and-after archetype assignments.
Loading 2608.20920v1…