PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents

Ankur Samanta, Yonathan Efroni, Paul Sajda, Kaveh Hassani, Anirudh Goyal

arXiv:2610.02525v1cs.AI

TL;DR

Long-horizon research decisions are sparse in lengthy execution traces, and their consequences may emerge only after later investigations. MIRA separates the choice of what to investigate from execution, then uses decision-level value forecasting and credit assignment; its learned policies improve gold-evaluated research outcomes across the studied autoresearch environments.

  • Problem

    Research agents must choose what to investigate as evidence accumulates, but those decisions are sparse in long execution traces and their consequences may be delayed.

    Source: §1 · p. 1

  • Method

    MIRA separates research allocation from execution, and MIRA-AC jointly forecasts remaining return and chooses the next investigation from curated research observations.

    Source: §1 · p. 2, §2.1 · p. 3

  • Results

    Decision-level policy learning improves gold-evaluated research outcomes, including when trained from proxy feedback, and initial evidence shows transfer to an environment held out from actor training.

    Source: §8 · p. 17

  • Takeaways & Limitations

    Explicit research decisions can be valued, credited, and improved through experience rather than remaining an implicit byproduct of long-horizon generation.

    Source: §8 · p. 17

    The policy transfers to an environment held out from actor training, but that environment was included in value pretraining.

    Source: §7 · p. 16

Abstract

from arXiv · show

Long-horizon research agents must decide both how to investigate and what to investigate next as evidence accumulates. This is hard to learn because such decisions are sparse in long execution traces, and their consequences may emerge several investigations later. We introduce Meta-reasoning for Iterative Research Agents (MIRA), a hierarchical architecture separating research allocation from execution. An outer-loop meta-reasoner curates context from a persistent research record, then writes a work order for the next investigation or ends the episode. A fresh inner-loop executor carries out each work order, making execution part of the transition between meta-reasoning actions. Without policy training, MIRA improves long-horizon inference and allocates additional compute more effectively in theorem proving and open-ended neural-architecture research. Its decision boundaries also provide natural units for credit assignment. At each boundary, we train a generative critic to forecast expected remaining return from partial states, outperforming token-level alternatives. Cross-environment pretraining improves forecasting and adaptation, yielding a transferable prior for valuing partial progress. We use this prior to initialize MIRA-AC, a generative actor-critic jointly trained to forecast remaining return and choose the next investigation, without a separate critic model. MIRA-AC concentrates policy optimization on meta-reasoning decisions, enabling efficient long-horizon reinforcement learning without directly optimizing the longer execution traces they initiate. Training MIRA-AC on the model's own proxy hill-climbing signals improves gold performance across four autoresearch environments; the actor transfers with cross-environment value initialization. Together, these results show that meta-reasoning can be learned as an explicit policy for directing long-horizon autonomous research.

1 Introduction

Long-horizon research requires choosing what to investigate next and when to stop, but these decisions are sparse in long execution traces and their consequences may be delayed. MIRA makes research allocation explicit, then uses decision-level value forecasting and credit assignment to learn which investigations to pursue.

MIRA

  • MIRA separates research allocation from execution: an outer-loop meta-reasoner curates the persistent record and issues a work order or stopping decision, while a fresh executor performs each investigation.Each investigation becomes one transition at the meta-reasoning level.Source: §1 · p. 2, Figure 1 · p. 2

Value forecasting

  • A decision-level generative critic forecasts expected remaining return from the same curated observation available to the policy, providing a boundary for value prediction and credit assignment.Its forecast concerns future research potential, not a ranking of candidate work orders.Source: §1 · p. 2

Policy learning

  • MIRA-AC jointly forecasts remaining return and selects the next investigation, assigning delayed credit to decision responses without directly optimizing the longer execution traces they initiate.Context curation, executor generations, experiments, and tool calls receive no direct policy loss.Source: §1 · p. 2

Results

  • Untrained MIRA improves long-horizon inference in theorem proving and open-ended neural-architecture research; decision-level policy learning also improves gold-evaluated outcomes across four autoresearch environments.Training from proxy hill-climbing feedback improves gold performance in all four environments.Source: §1 · pp. 2–3

2 Research as a Sequential Decision Process

The paper models long-horizon research as sequential decisions about what to investigate or whether to stop, with each potentially lengthy investigation treated as part of the transition. It defines proxy and gold returns and uses observation-conditioned remaining value to represent the expected future contribution of continued research.

2 Research as a Sequential Decision Process

  • Research decisions—not the potentially much longer investigations they initiate—are the unit of control; each investigation is part of a single environment transition.A termination decision is itself a terminal research action.Source: §2 · p. 3, §2.1 · p. 3

2.1 Decision Process

  • The policy acts on a curated observation of persistent research state, which includes the current artifact, accumulated evidence, and remaining research budget.A work order specifies an objective, proposed approach, and success criteria.Source: §2.1 · p. 3

2.2 Research Feedback and Return

  • Proxy rewards track improvement in the incumbent best-so-far quality, while gold-binary feedback supplies only a terminal success reward.Proxy and gold evaluations need not rank artifacts identically.Source: §2.2 · p. 4

2.3 Remaining Research Value

  • For a fixed continuation policy and research budget, remaining research value is expected future return conditioned on the curated observation, not the maximum improvement theoretically attainable from the full state.Under proxy feedback it is expected additional best-so-far improvement; under gold-binary feedback it is probability of success.Source: §2.3 · p. 4

3 MIRA: Exposing Meta-Reasoning Decisions

MIRA exposes individual investigations as explicit research decisions while preserving evidence across a trajectory. Its architecture separates context curation, decision-making, and execution, creating a consistent boundary for learning and credit assignment.

3.1 MIRA Architecture

  • MIRA uses a persistent research substrate, an outer-loop meta-reasoner, and a fresh inner-loop executor to separate research allocation from execution.The persistent record carries evidence across investigations while model contexts reset between decisions.Source: §3.1 · pp. 4–5
  • At each boundary, the outer loop constructs an observation and samples a complete decision response that issues a work order or terminates the episode.A work order specifies the next objective, approach, and success criteria.Source: §3.1 · pp. 4–5

3.2 MIRA Research Procedure

  • Each outer-loop decision yields exactly one decision-level transition regardless of the length of its executor trace.The shared procedure and executor interface remain fixed across tasks.Source: §3.2 · p. 5

3.3 Learning Boundary

  • MIRA-AC learns the decision-response policy and remaining-return critic while holding curation, execution, and the environment fixed.The critic receives the same curated observation as the policy and forecasts a distribution over remaining return.Source: §3.3 · p. 5, Figure 2 · p. 6

4 Learning to Forecast the Future Value of Research Progress

The paper forecasts remaining research value at decision boundaries because an investigation’s immediate result may not reflect its eventual contribution. It trains a generative critic from episode returns and investigates whether value forecasting transfers across research environments.

4 Learning to Forecast the Future Value of Research Progress

  • An investigation’s immediate outcome may misrepresent its eventual contribution, motivating value forecasts at MIRA’s research-decision boundaries.A failed experiment may eliminate an incorrect hypothesis and enable later progress, while an immediate improvement may lead to a dead end.Source: §4 · p. 6

4.1 Decision-Level Generative Critic

  • The critic forecasts once per research decision from the curated observation, using the language model’s next-token distribution over possible future-return values.It estimates expected remaining return under the continuation policy, not maximum attainable improvement or scores for candidate work orders.Source: §4.1 · p. 6

4.2 Critic Objective

  • The critic is trained against Monte Carlo returns from complete research episodes, projecting each target onto neighboring support points while preserving its expectation.The objective minimizes the KL divergence between the projected return target and the critic’s predicted distribution.Source: §4.2 · p. 6

4.3 Cross-Environment Value Pretraining

  • Leave-one-environment-out pretraining uses trajectories from other environments, followed by adaptation on target support episodes and evaluation on held-out target queries.The experiments compare this initialization with target-only and single-source training.Source: §4.3 · p. 7

5 MIRA-AC: Learning What to Investigate Next

MIRA-AC trains a shared actor–critic to forecast remaining research value and choose what to investigate next, assigning credit at research-decision boundaries. Its training reconstructs complete episodes and uses asynchronous buffers to accommodate investigations of different durations.

5 MIRA-AC: Learning What to Investigate Next

  • MIRA-AC jointly learns value forecasting and research decisions, using buffered asynchronous optimization for variable-duration episodes.The shared actor–critic assigns delayed credit across complete research episodes.Source: §5 · p. 7, Figure 3 · p. 9

5.1 Shared Actor and Critic

  • A shared model initialized from a pretrained critic checkpoint serves as both actor and critic, so value learning also updates decision-generating representations.The actor and critic use separate optimizer states while updating the shared parameters.Source: §5.1 · p. 8

5.2 Credit Assignment Across Research Decisions

  • Training reconstructs each logical episode across storage boundaries; only true episode termination stops value propagation, with a bridge aligning baselines when critic versions change.The bridge does not modify the Monte Carlo target.Source: §5.2 · p. 8

5.3 Decision-Level Actor Objective

  • The actor objective trains on the complete sampled meta-reasoning response while masking prompts, curated observations, executor outputs, tool calls, and experiments.Policy-gradient terms outside the allowed importance-ratio range are masked, while those tokens remain in the policy-mismatch penalty.Source: §5.3 · p. 8

5.4 Buffered Single-Rollout Asynchronous Policy Optimization

  • Separate actor and critic buffers trigger updates as they fill, freeing rollout slots immediately; live episodes adopt published parameters at the next decision boundary.If both buffers are ready, the critic update is applied before the actor update within the same transaction.Source: §5.4 · p. 8, Figure 3 · p. 9

6 Experiments

The experiments assess MIRA before policy training, test decision-level value forecasting and transfer, and evaluate policy learning and resulting behavior. They report gains in theorem proving and architecture research, stronger forecasting with decision-level critics and cross-environment pretraining, and improved gold outcomes from trained meta-reasoning policies.

Experimental setup.

  • GPT-5.5 MIRA raises mean normalized IMOProofBench Advanced grader score from 67.1% under direct inference to 100%, solving all 30 problems in each of three trials.Output-token-matched continuation recovers most of the gap, with the residual concentrated on four problems; Fig. 5 illustrates MIRA testing several proof directions and recording refutations.Source: Experimental setup · p. 9, Figure 4 · p. 10, Figure 5 · p. 10
  • In matched 24-hour HillClimbBench-RMT campaigns, MIRA selects learned per-cell residual-write gates and achieves the lowest proxy and gold loss among the three orchestration conditions.Figure 4(b) presents the best loss observed during the matched campaigns.Source: Experimental setup · p. 10, Figure 4 · p. 10
  • In matched 48-hour Loop Transformer campaigns, MIRA achieves lower proxy and gold NLL than Codex, while both campaigns beat the reference architecture on proxy NLL.MIRA selects signed attention with a leaky recurrent update; Figure 6 tracks best proxy NLL over time and reports final proxy and gold NLL.Source: Experimental setup · p. 11, Figure 6 · p. 11

6.2 Can We Forecast Remaining Research Value?

  • The decision-level generative critic achieves the lowest mean held-out RMSE among tested designs, reducing RMSE by an absolute 0.110 relative to its token-level generative counterpart.The authors use this critic design in subsequent transfer and policy-learning experiments.Source: §6.2 · p. 12, Figure 8 · p. 12
  • Across environments, leave-one-out pretraining improves zero-shot forecasting and adaptation; after five target-environment support episodes, it reduces RMSE by an absolute 0.053 versus target-only training.Figure 8(b) compares RMSE across target-support sizes.Source: §6.2 · p. 12, Figure 8 · p. 12

6.3 Does Decision-Level Credit Improve the Research Policy?

  • Under proxy-marginal supervision, MIRA-AC has higher gold-evaluation point estimates than untrained MIRA in all four environments without directly optimizing the longer executor traces.The strongest separate decision-level variant improves the primary outcome over token regression in every environment; Step generative leads in three environments and Step regression in BNLearn.Source: §6.3 · pp. 13–14, Table 2 · p. 13
  • A cross-environment MIRA-AC policy matches or exceeds the strongest separate-adapter primary outcome in every environment, matching Step regression on BNLearn without BNLearn actor updates.The policy is trained on proxy feedback in three environments and starts from a critic pretrained on all four, including BNLearn.Source: Table 2 · p. 13, §6.3 · p. 14

6.4 What Changes After Meta-Reasoning Policy Training?

  • In blinded judgments across 400 matched pairs, MIRA-AC is more often credited with stronger gold outcomes, and its trajectories more often test competing ideas and less often persist with unproductive minor variants.The share testing competing ideas rises from 16.8% to 28.0%, while repeated minor variants of unproductive ideas fall from 33.5% to 18.8%; curated context averages 10.6 million input tokens per episode across the training episodes.Source: §6.4 · pp. 14–15, Figure 9 · p. 15

7 Limitations and Future Work

The section identifies constraints in what MIRA-AC learns, the research settings and feedback used to train it, and the scale and evaluation of its experiments. It argues that broader, context-sensitive research judgment remains an open goal.

7 Limitations and Future Work

  • MIRA-AC trains action selection while keeping evidence curation and execution fixed, so it cannot recover evidence the curator omits or exceed the executor’s capabilities.Changes in executor capability may also change which research decisions are valuable.Source: §7 · p. 15
  • The policy’s tendency toward larger architectural changes can escape local optima but can also backfire when preserving and refining the incumbent is better.The desired behavior is context-sensitive judgment about whether to persist, refine, test, pivot, or stop.Source: §7 · pp. 15–16
  • The training environments cover only a small fraction of the research diversity needed for a general policy, and decision-level decomposition may not suit less structured research.The paper leaves open whether the abstraction works when investigations overlap, branch, or evolve continuously.Source: §7 · pp. 16–17
  • BNLearn transfer is held out from actor updates but not value pretraining, so broader transfer requires testing environments excluded from both.Source: §7 · p. 16
  • Experiments use one MIRA configuration and a fixed Codex-based inner-loop harness, leaving transfer across changing execution scaffolds untested.The authors identify heterogeneous and evolving scaffolds as a next step.Source: §7 · p. 16
  • Proxy feedback makes repeated autoresearch training feasible, but proxy–gold misalignment may affect transfer and its robustness remains unestablished.The authors call for studying the alignment needed for stable improvement and learning from sparse gold feedback.Source: §7 · p. 16
  • Behavioral findings are descriptive: LLM-judge labels and matched trajectories show observable behavior differences but do not establish that particular behaviors cause performance differences.Source: §7 · p. 16
  • Optimization is limited to LoRA adapters and investigation durations supported by current infrastructure; longer episodes could magnify delayed feedback, policy staleness, and shared-update interference.Full-parameter training and longer-lived research episodes remain untested.Source: §7 · p. 17

8 Conclusion

MIRA makes research allocation explicit by treating each investigation as a decision whose execution forms part of the transition. The paper concludes that decision-level forecasting and policy learning improve research outcomes in the studied settings, while positioning learned allocation as distinct from improving execution alone.

8 Conclusion

  • MIRA represents long-horizon research as explicit decisions over a persistent record, with each investigation’s longer execution treated as an environment transition.This decomposition improves long-horizon inference before policy training and gives a boundary for forecasting value and assigning delayed credit.Source: §8 · p. 17
  • MIRA-AC jointly forecasts remaining return and allocates research effort; decision-level value forecasting beats token-level alternatives, and policy learning improves gold-evaluated outcomes, including with proxy feedback.The paper also reports initial transfer to an actor-training-held-out environment when initialized from a cross-environment value model.Source: §8 · p. 17
  • The broader implication is that explicit research decisions linked to downstream consequences can be valued, credited, and improved through experience.Source: §8 · p. 17

A Related Work

  • Unlike systems focused on inference-time organization or execution mechanisms, MIRA centers on learning which research objective to pursue next from prior experience.It makes investigation choices explicit as work orders in a trainable sequential decision process.Source: A Related Work · pp. 25–26

B Experimental Setup

The experiments evaluate MIRA across four autoresearch environments using bounded research trajectories, fixed interaction protocols, and proxy feedback during research followed by gold evaluation. The setup specifies distinct task interfaces and progress measures for each environment.

B Experimental Setup

  • The study covers BNLearn causal discovery, Physics Discovery, SRBench symbolic regression, and HillClimbBench-CPU architecture research, with up to 20 MIRA iterations per rollout.The first three use 100 tasks each; HillClimbBench-CPU uses 50 tasks with two independently initialized rollouts per task.Source: B Experimental Setup · p. 27

B.1 Shared Interaction Protocol

  • Each iteration gives a fresh executor the work order, fixed task prompt, and persistent workspace, while training and evaluation on three scientific environments use the same task identities and seeds.Those evaluations test new trajectories within the tasks, not generalization to unseen task identities.Source: §B.1 · p. 27

B.2 Defining Proxy Hill-Climbing Signals

  • Frozen, task-specific proxy maps convert progress into marginal rewards, but may miss real progress, reward brittle shortcuts, or saturate too early.Gold evaluation remains hidden during research and is the final measure of the selected artifact.Source: §B.2 · pp. 27–28

B.3 Environment Interfaces

  • BNLearn tasks are anonymized induced subgraphs with visible data and diagnostics, while graph accuracy is unavailable during investigation and exact directed recovery defines success.The final graph’s score uses normalized structural-Hamming-distance gap closure.Source: §B.3 · p. 28

Physics discovery.

  • Physics Discovery gives agents unknown-law worlds and a limited experiment budget; final gold scoring uses held-out trajectory fit and a judged explanation.A task is solved when normalized trajectory MSE is below 0.1 and explanation score is at least 0.75.Source: Physics discovery · p. 28
  • SRBench uses 100 symbolic-regression problems with visible fit and validation data while test sets and reference expressions remain hidden; the agent selects the final expression.Gold evaluation uses sealed in-distribution test data, with out-of-distribution metrics reported when available.Source: Physics discovery · pp. 28–29
  • HillClimbBench-CPU tests paper-derived architecture tasks under fixed CPU-only training conditions, with proxy quality normalized against a frozen reference and gold evaluation under harder structural conditions.Gold scores average five runs; benchmark success requires a positive lower endpoint for the paired-score 95% confidence interval.Source: Physics discovery · pp. 29–30

B.4 Evaluation

  • Comparisons use matched task instances and interaction budgets, with confidence intervals computed by task-clustered bootstrap.Source: §B.4 · p. 31

C MIRA Implementation

MIRA rebuilds each research decision from selected evidence in a persistent record, then delegates the chosen work order to a fresh executor. The appendix also describes how these traces are projected into semantic graphs and annotated for analysis.

C.1 Context Construction

  • The context curator selects bounded, decision-relevant evidence from repository files, diffs, commits, and prior work records; the policy then writes an objective, evidence, resource guidance, acceptance criteria, and stopping condition.The fixed context also includes the task, workspace summary, latest proxy evaluation, preceding episode summary, and remaining budget.Source: §C.1 · p. 31

C.2 Decision and Execution

  • A fresh executor receives the work order, validates and commits useful work, and leaves a handover, while the repository persists across iterations and model contexts do not.Failures and contradictory results remain available to later decisions, and an incumbent can protect the strongest validated artifact from regressions.Source: §C.2 · p. 31

C.3 Environment-Controlled Evaluation and Submission

  • The environment supplies tools, proxy feedback, reward mapping, and terminal evaluation; submission authority may lie with the executor, outer-loop policy, or environment.Gold evaluation occurs only after research ends, outside both model contexts, and only on the selected artifact.Source: §C.3 · p. 31

D Semantic Exploration Graphs and Strategic Inertia

  • Semantic graphs represent evidence-supported research steps and relations, while a separate annotation groups fixed nodes into high-level research directions without changing the nodes.The extractor requires provenance for every node and edge; chronology alone does not establish a relation.Source: D Semantic Exploration Graphs and… · p. 32
  • Strategic inertia is the share of consecutive non-submission steps that continue the same direction; the statistic is descriptive, and runs with fewer than two such nodes receive no value.The analysis averages trials within problems and bootstraps problems for confidence intervals; natural and forced-continuation runs are analyzed separately.Source: D Semantic Exploration Graphs and… · p. 32
  • Across four illustrated proof problems, MIRA explores or redirects among proof directions and reaches complete proofs, while the comparison systems leave some directions unsuccessful or incomplete.The visualizations show first natural-stopping trials on problems where a gap remained after output-token-matched continuation.Source: D Semantic Exploration Graphs and… · p. 32, Figure 13 · p. 33, Figure 14 · p. 34

E RMT Architecture Identified by MIRA

MIRA’s selected RMT adds learned branch-specific gates to matrix-state writes. The reported ablations show lower last-token cross-entropy than an ungated GEGLU control, while the authors caution that they do not establish the gates as the cause of the full gold-loss improvement.

Selected architecture

  • The selected model retains the RMT matrix state and low-rank interface, uses GEGLU, and adds a separate learned gate for each attention and MLP branch and layer.All gate coefficients start at one, making the update exactly the ordinary additive update at initialization; the gates add 0.056% to parameter count.Source: E RMT Architecture Identified by MIRA · p. 35

Ablation results

  • 0.021 lower final-scale last-token cross-entropy and 0.009 lower small-scale last-token cross-entropy versus the ungated GEGLU control were observed in one-seed comparisons.The ungated final-geometry control was not submitted to the gold evaluator, so the ablations do not isolate the gates as the cause of the full gold-loss improvement.Source: E RMT Architecture Identified by MIRA · p. 35

Gate behavior

  • Attention gates generally amplify writes and MLP gates suppress them; row/column effects explain roughly three quarters of attention-map and nearly nine tenths of MLP-map variation.The gates are fixed across tokens and examples, so the analysis does not establish dynamic routing or a permanent cell partition.Source: E RMT Architecture Identified by MIRA · p. 35

F Loop Transformer Search and Selected Designs

MIRA’s Loop Transformer search selected signed attention with a leaky recurrent update after testing alternatives. In matched campaigns, its final incumbent has lower proxy and gold NLL than Codex and the task reference.

Search trajectory and selected design

  • After learned retention gates failed to outperform constant interpolation and other routing alternatives gave small or inconsistent gains, MIRA selected signed attention with a leaky recurrent update.Signed attention combines two attention reads over shared values as A1 + (A1 − A2); the leaky update mixes prior and proposed states with fixed ρ = 0.25.Source: F Loop Transformer Search and Selected… · pp. 35–36

Selected design

  • The selected architecture retains weight-tied recurrence and input reinjection while adding signed attention and interpolation between previous and proposed states at each iteration.The figure identifies the added components relative to the standard Transformer and Huginn.Source: Figure 16 · p. 36

Comparison

  • Gold NLL is 3.0464 for MIRA, versus 3.2275 for Codex and 3.2653 for the task reference; proxy NLL is 3.0331, 3.2419, and 3.2940, respectively.The campaigns were matched for model-scale and training-token rungs; proxy scores guided search and gold evaluation followed submission.Source: F Loop Transformer Search and Selected… · p. 36

G Critic Training and Validation

The critic is trained to forecast either future proxy-quality gains or whether the final submission meets an environment’s gold success criterion. The section specifies its prompt, objectives, validation, and pretraining and adaptation setup.

Targets and value contracts

  • Proxy-marginal return is the future increase in best-so-far proxy quality, while gold-binary return indicates whether the final submission meets the environment-specific gold success criterion.The success criteria differ by environment, and policy-comparison win rate is not a critic target.Source: G Critic Training and Validation · p. 36

Critic parameterizations and objectives

  • Scalar critics regress returns, whereas categorical and generative critics fit projected return distributions; the generative forecast is converted to value by taking the support-weighted expectation.For gold-binary supervision, the generative distribution is restricted to {0, 1} and trained against a one-hot target.Source: G Critic Training and Validation · p. 37

Pretraining and target-environment adaptation

  • Pretraining used one epoch on Qwen3.6-27B, and target adaptation evaluated support sizes {0, 1, 5, 10, 25, 50} across five paired support/query splits.Each split used 50 query episodes; adaptation made one episode-ordered pass.Source: G Critic Training and Validation · p. 37

Evaluation procedure

Generative critic prompts

  • The generative critic receives the curated decision conversation and a value query, then forecasts normalized additional improvement on a 0-to-1 scale using a single-letter response.Its next-token probabilities over the answer labels define the forecast distribution.Source: G Critic Training and Validation · pp. 37–38, Figure 17

H MIRA-AC Training Details

This block specifies MIRA-AC training and trajectory stitching, then describes how matched evaluations and an outcome-aware judge characterize policy behavior. The analysis reports more productive-risk judgments for MIRA-AC and defines how examples are attributed to policy, executor, or harness.

H MIRA-AC Training Details

  • Training uses one fixed ordered pass over 100 episodes per environment-specific configuration and 300 episodes for the cross-environment run.The cross-environment run draws 100 episodes from each of its three actor-training environments; the reported general-policy checkpoint is epoch one, version 94.Source: H MIRA-AC Training Details · p. 38, Table 3 · p. 39

I Trajectory Stitching Across Artificial Boundaries

  • Trajectory stitching preserves episode-level advantages and Monte Carlo targets across stored-sample boundaries; only true environment termination stops bootstrapping.This partition invariance depends on preserving logged transitions, critic versions, and successor bootstraps, including bridges when critic versions change.Source: I Trajectory Stitching Across… · pp. 40–41

K.1 Analysis Protocol

  • The 400 matched pairs covered four environments, and a two-pass blinded judge assessed behavior before seeing gold outcomes and then attributed outcome contributions.Behavior rates count each trajectory once when an instance is attributed at least partly to the outer policy; executor-, harness-, and unobserved-only behaviors are excluded.Source: K Behavioral Analysis of MIRA-AC · p. 41, §K.1 · p. 41
  • Matched behavior differences are descriptive, not causal estimates, because executor stochasticity, decision interactions, and evaluation conditions can also affect outcomes.The aggregate rates establish prevalence; selected trajectory excerpts illustrate behaviors rather than estimate their frequency.Source: §K.1 · p. 41

K.2 Pooled Behavioral Changes

  • MIRA-AC received 60.8% of decisive policy attributions and was judged productively risky in 52.5% of trajectories, versus 34.8% for untrained MIRA.Of 400 pairs, the judge credited MIRA-AC in 203 and untrained MIRA in 131; 66 were tied or unclear.Source: §K.2 · p. 41
  • The judge found MIRA-AC more often tested competing ideas, checked robustness, and made evidence-supported continuation decisions, while repeating minor variants less often.Rates were 112/400 vs. 67/400 for discriminating tests, 28/400 vs. 10/400 for robustness checks, 53/400 vs. 29/400 for continuation decisions, and 75/400 vs. 134/400 for repeated minor variants.Source: §K.2 · p. 41
  • On 33 of 50 HillClimbBench-CPU tasks, MIRA-AC achieved lower best-observed raw loss than untrained MIRA.The figure compares each policy's better result across two matched rollouts; staircases show best-so-far loss and dashed lines mark the reference implementation.Source: Figure 18 · p. 42

K.3 Selecting and Attributing Trajectory Examples

  • The example analysis distinguishes five idea origins, from direct outer specification and outer-seeded discovery to executor pivots and harness-enabled opportunities.A specific content link between a work order and later discovery is required for outer-seeded attribution.Source: §K.3 · p. 43

K.4 Symbolic Regression

Across four research settings, trained policies more often acted on unresolved uncertainty, but the value of that shift depended on whether further investigation improved the gold outcome. The cases illustrate gains from targeted tests and architectural changes as well as losses when proxy signals favored unhelpful exploration or shortcuts.

K.4 Symbolic Regression

  • Across 100 symbolic-regression tasks, MIRA-AC earned the higher gold score in 58 cases versus 32 for untrained MIRA, with 10 ties.MIRA-AC more often ran discriminating tests and checked edge cases, but proxy improvements could favor boundary-fitted expressions that failed gold evaluation; forced continuation gave unequal later-decision opportunities.Source: §K.4 · pp. 43–44

K.5 Physics Discovery

  • Across 100 Physics Discovery tasks, MIRA-AC had the higher gold score in 65 cases versus 26 for untrained MIRA, with nine ties.Targeted parameter variation resolved a competing-law ambiguity and yielded a 1.000 gold score versus 0.005; in another case, continued proxy-driven data collection followed discovery of the law and scored 0.00085 versus 1.000.Source: §K.5 · pp. 44–45

K.6 HillClimbBench-CPU: Larger Architectural Changes

  • Across 100 HillClimbBench-CPU pairs, MIRA-AC had the higher gold score in 55 versus 45 for untrained MIRA, though the judge preferred untrained MIRA's research process in 53 pairs.A scale-changing architecture reached 0.683 versus 0.195, while a proxy-improving direct-neighbor design was invalid on gold evaluation and scored 0 versus 0.809.Source: §K.6 · pp. 45–46

K.7 BNLearn: Resolving Remaining Graph Uncertainty

  • In BNLearn, MIRA-AC more often tested a specific uncertain edge, but less often preserved its strongest current graph; the judge credited policy contributions in 25 pairs versus 22, with 53 ties.A targeted two-intervention repair reached gold 1.0, whereas broad searches in another case finished at 0.844 versus untrained MIRA's 1.0; BNLearn was excluded from actor training but included in critic pretraining.Source: §K.7 · pp. 46–47

K.8 Cross-Environment Interpretation

  • Across environments, MIRA-AC more often acted on unresolved uncertainty, with gains when that uncertainty mattered to gold evaluation and costs when proxy incentives favored shortcuts or excess exploration.The analysis suggests hypothesis-targeted experiments, shifted or out-of-distribution checks, and small edits when few uncertainties remain as evaluation directions.Source: §K.8 · p. 47
Loading 2610.02525v1…