Agent-Editing World Model: Rethinking World Modeling for LLM Agents

1. Introduction

• Observation-predictive world models target high-entropy, execution-dependent tool responses, although real feedback is available and reconstructing those responses offers limited value.Agents can retain unsupported assumptions or outdated plans in history, allowing locally plausible decisions to compound task-state contamination.Source: §1 · p. 2
• AEWM models how reasoning and actions shape future task progress, while EditAct selectively replaces noisy reasoning–action continuations before real-environment execution.Action Judge distinguishes critical, exploratory, and noisy decisions; State Revision edits noisy continuations.Source: §1 · pp. 2–3
• AEWM is trained across Search, Terminal, and Software Engineering using synthesized Action Judge and State Revision data, mid-training, and supervised fine-tuning.The work also introduces a 3,000-decision Action Judge benchmark and AEWM-RFT on verified EditAct trajectories grounded in real feedback.Source: §1 · pp. 2–3
• 70.5% macro-F1 on the Action Judge benchmark exceeds the strongest baseline by 10.6 points; EditAct gains 3.2–6.7 average-score points across six benchmarks and three backbones.AEWM-RFT also exceeds Self-RFT by 2.2–2.6 points across three domains without online AEWM guidance.Source: §1 · p. 3

2. Agent-Editing World Model

• AEWM combines decision-effect prediction and agent-state editing, with Figure 2 showing its training data, inference-time EditAct, and subsequent AEWM-RFT trajectory use.The overview spans Action Judge and State Revision data synthesis, mid-training and supervised fine-tuning, and the use of resulting trajectories for rejection sampling fine-tuning.Source: §2 · p. 3, Figure 2 · p. 4

2.1. Preliminaries

• The pre-execution state joins prior interaction history with the agent’s proposed reasoning–action pair, whose effects can shape subsequent task progress.Observation-predictive models reconstruct high-entropy, execution-dependent observations, while AEWM edits the continuation before execution when it may propagate contamination.Source: §2.1 · pp. 3–4

2.2. Agent State Modeling and Editing

• Action Judge labels proposals critical, exploratory, or noisy; for noisy decisions, State Revision generates a grounded replacement while preserving the prior history.The revised action can seek missing evidence through real-environment interaction, and the revised reasoning changes what enters the agent’s future history.Source: §2.2 · pp. 4–5

2.3. EditAct: Integrating State Editing and Acting

• EditAct revises only noisy proposals, executes the selected action in the real environment, and adds the resulting observation to history for the next continuation.Productive decisions are retained, while the selected reasoning–action pair redirects the agent’s path and subsequent observations ground its evolution.Source: §2.3 · p. 5

3.1.1. Action Judge Data

• Action Judge examples label verified trajectory turns as CRITICAL, EXPLORATORY, or NOISY, while training inputs use only the visible history and proposed reasoning–action pair.Annotations are filtered for consistency, action–observation alignment, validity, and grounding in the pre-execution history.Source: §3.1.1 · pp. 5–6
• The held-out Action Judge benchmark contains 3,000 decisions, evenly split across Search, Terminal, and SWE, and reports accuracy and macro-F1.Examples undergo repeated annotation, consistency filtering, model-based review, and diversity-aware sampling.Source: §3.1.1 · p. 6

3.1.2. State Revision Data

• State Revision data pair a NOISY proposal with a replacement reasoning–action pair from the same history, retaining revisions only when real execution shows substantive task progress.A proposal agent, Action Judge checkpoint, and revision agent construct the examples.Source: §3.1.2 · p. 6

3.2. Two-Stage Training For AEWM

• AEWM is trained across Search, Terminal, and SWE with approximately 52B mid-training tokens followed by 120K SFT examples, split evenly between Action Judge and State Revision.The SFT set contains 40K examples from each domain.Source: §3.2 · p. 6

3.3. Internalizing Agent Editing through AEWM-RFT

• AEWM-RFT fine-tunes agents on verified, high-quality EditAct trajectories, including revisions and subsequent decisions, to internalize state correction and help avoid or recover from task-state contamination.The trajectories are grounded in real environment feedback.Source: §3.3 · pp. 6–7

4.1. Experimental Setup

• Evaluation averages three runs on six benchmarks spanning Search, Terminal, and SWE; DeepSearchQA and SWE-Bench Pro are out-of-distribution tests.EditAct is compared with ReAct and step-level and trajectory-level Best@3 baselines.Source: §4.1 · p. 7

4.2. Action Judge Evaluation

• 70.5% overall macro-F1 puts AEWM 10.6 points above DeepSeek-V4-Pro (59.9%) on the Action Judge benchmark.AEWM scores 60.9% on Search, 72.1% on Terminal, and 77.8% on SWE; its largest gain is on SWE, while Search remains most challenging.Source: §4.2 · pp. 7–8, Figure 3 · p. 8

4.3. Main Results

• EditAct outperforms ReAct and both Best@3 baselines on all six benchmarks, gaining 6.7, 5.2, and 3.2 average points for Qwen3.5-4B, Qwen3.5-9B, and Qwen3.5-35B-A3B, respectively.Improvements hold on both in-distribution and out-of-distribution benchmarks.Source: Table 1 · p. 8, §4.3 · p. 8

4.4. AEWM-RFT Results

• AEWM-RFT exceeds Self-RFT by 2.2, 2.6, and 2.5 points on BrowseComp, Terminal-Bench 2.0, and Doc2Repo, respectively, without online AEWM guidance.Average turns fall by 30.0% on BrowseComp and 16.7% on Terminal-Bench 2.0, while Doc2Repo gains 2.5 points over Self-RFT with more turns.Source: §4.4 · pp. 8–9, Table 2 · p. 9

4.5. Ablation Studies

• EditAct performs best across the three ablation benchmarks; comparisons support learned intervention selection and direct joint reasoning–action editing over random gating, resampling, hints, or single-component revision.Self-WM performs worse on BrowseComp and Terminal-Bench 2.0, while DeepSeek-V4-Pro WM remains below AEWM on all three benchmarks.Source: Table 3 · p. 9, §4.5 · p. 9
• Mid-training followed by SFT performs best across the three domains, improving BrowseComp, Terminal-Bench 2.0, and Doc2Repo over SFT alone by 0.8, 6.4, and 4.3 points.The full recipe also beats mid-training alone by 0.3, 4.1, and 1.9 points, respectively; the stages provide complementary benefits.Source: Table 3 · p. 9, §4.5 · pp. 9–10

5.1. Action Type Distributions

• 60.0% of Search actions are noisy; Terminal has 43.2% exploratory and 25.1% noisy actions, while SWE has 42.1% critical and 28.0% noisy actions.Online AEWM predictions retain these broad domain patterns; inspected failures include unverified search hypotheses, misread execution evidence, and flawed SWE dependency assumptions.Source: §5.1 · p. 10

5.2. Repairing Task-State Contamination

• The mean longest search-only sequence decreases from 16.9 to 13.2, while distinct pages increase from 7.7 to 10.1.AEWM reopens premature conclusions and redirects searches toward task constraints and discriminating or conflicting evidence.Source: Figure 5 · p. 10, §5.2 · p. 10
• Terminal revisions redirect actions toward focused checks or repairs, using real execution feedback to update assumptions without treating intermediate progress as completion.The feedback updates the agent’s view of input data, system behavior, and delivered outputs.Source: §5.2 · p. 11
• In SWE, AEWM revises reasoning and actions to recover overlooked constraints, preserve existing contracts, and redirect ineffective edits toward coherent repairs.Execution and regression tests then check repaired behavior without treating local success as proof that all requirements are met.Source: §5.2 · p. 11

• Language world models simulate reasoning or web-state transitions, predict tool feedback, and also explore prediction-augmented training, simulation fidelity, and environment synthesis.AEWM instead predicts decision effects and directly edits agent-generated reasoning and action, obtaining observations through real execution.Source: §6 · p. 11
• Unlike rubric-based step judgments, AEWM predicts downstream decision effects from pre-execution context and learns execution-verified edits from the same history.Action Judge and State Revision together selectively revise noisy states while preserving informative exploration across Search, Terminal, and SWE; verified interventions also support AEWM-RFT.Source: §6 · p. 11

• AEWM predicts decision effects and edits agent states to address task-state contamination, with EditAct integrating these capabilities with real execution rather than simulating tool responses.Source: §7 · p. 11
• Experiments demonstrate stronger action judgment and improved task performance across six benchmarks and three agent backbones.Source: §7 · p. 12

A.1. Sources and Supervision Units

• Search uses internal deep-search tasks, Terminal uses CalibForge tasks, and SWE uses DeNovoSWE repository-construction tasks.Source: §A.1 · p. 19
• Action Judge examples pair task, tools, history, and a proposed reasoning–action with a rationale and CRITICAL, EXPLORATORY, or NOISY label.Offline annotation checks the decision against the matched observation and remaining trajectory; this suffix is audit evidence, not student input.Source: §A.1 · p. 19
• State Revision examples replace noisy proposals using the same pre-action history, and the replacement is executed rather than simulated.Qwen proposes decisions, an Action Judge checkpoint identifies noisy proposals, and DeepSeek-V4-Pro generates replacements.Source: §A.1 · p. 19

A.2. Filtering and Mixture Construction

• Semantic filtering combines structural checks with grounded-decision audits, and acceptance requires every rubric dimension to score at least 3 plus the listed hard conditions.For terminal revisions, useful execution or verification evidence is required; finish decisions are assessed separately, with task-specific success criteria.Source: §A.2 · pp. 19–20, Table 4 · p. 20
• Diversity-aware selection combines domain quotas with interleaving across sources, tools, trajectory positions, and identities while removing duplicates and capping repeated task contributions.Separate tool quotas prevent termination examples from dominating revision supervision.Source: §A.2 · p. 20
• The mid-training corpus contains 52.16B tokens, while SFT divides examples equally between Action Judge and State Revision and equally across the three domains.Action Judge and State Revision contribute 20.56B and 25.01B tokens, respectively; Search uses a 1:1:3 label mixture, and counts refer to decision examples, not trajectories.Source: §A.2 · pp. 20–21, Table 5 · p. 20, Table 6 · p. 21

A.3. Optimization and Checkpoints

• Mid-training starts from pretrained Qwen3.5-35B-A3B and SFT starts from its mid-trained checkpoint; the SFT-only ablation starts from the pretrained backbone.Both stages use the same backbone, and the main optimization settings are summarized in Table 7.Source: §A.3 · p. 21, Table 7 · p. 21

A.4. Agent Fine-Tuning on Verified Trajectories

• AEWM-RFT retains verified AEWM-guided rollouts with executed revisions, whereas Self-RFT retains the agent’s own verified ReAct rollouts under matched data and training settings.Both agents use ReAct inference without world-model guidance at evaluation; Search and DeNovoSWE apply trajectory-level selection criteria, and agents are fine-tuned separately by domain.Source: §A.4 · p. 21, Table 8 · p. 22, Table 9 · p. 22

B.1. Construction and Verification

• The benchmark draws on GLM-5 and DeepSeek-V4-Pro trajectories from BrowseComp, Terminal-Bench 2.0, Doc2Repo, and NL2Repo, retaining successful runs or scores above 0.8.BrowseComp and Terminal-Bench 2.0 require successful task completion; Doc2Repo and NL2Repo retain trajectories scoring above 0.8.Source: §B.1 · p. 22
• Each retained decision receives three independent labels, and only unanimous examples survive rule-based filtering, individual GPT-5.6-Sol review, and diversity-aware sampling into a 3,000-example benchmark.The composition is summarized in Table 10; SWE pools Doc2Repo and NL2Repo.Source: §B.1 · p. 22, Table 10 · p. 22
• Evaluated models see the task, tools, pre-action history, and current proposal, but not the current observation, later messages, rationale, final score, or gold label.Table 11 reports accuracy and macro-F1 by domain and for the full benchmark.Source: §B.1 · p. 22, Table 11 · p. 23

B.2. Evaluation and Additional Results

• Models return one judgment per decision without voting or correctness-based resampling; accuracy, macro-F1, and balanced accuracy are computed from the pooled decisions.Overall metrics pool all 3,000 decisions, and SWE metrics pool its 1,000 decisions.Source: §B.2 · p. 23
• 70.5% macro-F1 makes AEWM the top model across all three domains and overall, exceeding the strongest baseline by 10.6 points.The benchmark reports accuracy and macro-F1 for Search, Terminal, SWE, and the full benchmark.Source: §B.2 · p. 23
• AEWM performs strongly across all three decision types, with particularly high performance on NOISY decisions while discriminating CRITICAL and EXPLORATORY actions.Table 12 reports precision, recall, and F1 for each class.Source: §B.2 · p. 23, Table 12 · p. 23

C.1. Scaffolds, Tools, and Budgets

• Search benchmarks use a three-tool ReAct scaffold, allow up to 600 steps, and share a 256K-token context window.BrowseComp is scored by accuracy and DeepSearchQA by F1.Source: §C.1 · pp. 23–24
• Terminal-Bench 2.0 uses a persistent isolated runtime with shell, file-editing, and finish tools, no explicit step limit, and a 10,800-second task limit.Task accuracy is reported.Source: §C.1 · p. 24
• SWE benchmarks use a command, file-editing, and finish scaffold with benchmark-specific budgets; evaluations run three times and report averages.SWE-Bench Pro allows 500 steps and reports Resolved Rate; Doc2Repo and NL2Repo use 400 and 500 steps and report Pass Rate.Source: §C.1 · p. 24

C.2. Candidate-Selection Baselines

• Step-level Best@3 samples three candidates from one interaction history and has a verifier select which valid candidate to execute.Only the selected reasoning and action are committed to the agent’s history.Source: §C.2 · pp. 24–25
• Trajectory-level Best@3 independently rolls out the agent three times and uses a verifier to select the final submission.The verifier receives answers, patches, or complete trajectories depending on the benchmark.Source: §C.2 · pp. 24–25

C.3. Inference Ablation Protocols

• ReAct is the shared baseline: it executes each proposed action directly without Action Judge, State Revision, or an additional world-model call.Source: §C.3 · p. 25
• Random Gate replaces learned Action Judge with random revision triggers calibrated to AEWM’s domain-specific NOISY prediction rates.State Revision otherwise follows the same procedure as EditAct.Source: §C.3 · p. 25
• Agent Resampling regenerates a proposal from the same history after a NOISY judgment, whereas AEWM Hint gives revised reasoning to the agent before it regenerates an action.Source: §C.3 · p. 25
• Action-only Revision changes the action but keeps original reasoning, while Reasoning-only Revision changes reasoning but executes the original action.Source: §C.3 · p. 26
• Self-WM uses the inference agent for judgment and revision, while DeepSeek-V4-Pro WM substitutes a strong general-purpose model under the same EditAct procedure.Source: §C.3 · p. 26
• EditAct retains CRITICAL and EXPLORATORY proposals, revises both reasoning and action for NOISY proposals, then executes the selected action in the real environment.The committed reasoning, action, and resulting observation are added to the agent history.Source: §C.3 · p. 26

D. Case Studies

• The cases compare proposed reasoning and actions with revisions from the same history; only the revised action is executed at that decision.They identify the contaminated assumption, edit, and feedback establishing its local benefit.Source: D. Case Studies · p. 27
• In Search, revisions redirect confirmation-oriented queries toward relationships, replace location-anchored searches with report reading, and reopen unsupported entity hypotheses.Source: Figure 6 · p. 27, Figure 7 · p. 28, Figure 8 · p. 29
• In Terminal, revisions correct an overgeneralized success claim, connect computational progress to artifact semantics, and change environment observation before verifying required behavior.Source: Figure 9 · p. 30, Figure 10 · p. 31, Figure 11 · p. 32
• In SWE, revisions correct a module map, preserve compatibility during refactoring, and restore a forgotten public-attribute requirement, with checks or tests confirming repairs.Source: Figure 12 · p. 33, Figure 13 · p. 34, Figure 14 · p. 35

E.1. Online Action Judgment

• For SWE-Bench Pro, the appendix reuses Terminal-domain Action Judge and State Revision prompts because its repository interaction pattern closely matches Terminal.Source: §E.1 · p. 36