PaperlayerView exact revision on arXiv

Source-linked AI summary

Second Thought: Reasoning in Parallel as LLM Agents Act and Observe

Zhensu Sun, Chengran Yang, Yunbo Lyu, Jieke Shi, David Lo

arXiv:2608.13667v1cs.AIcs.SE

TL;DR

ReAct agents confine deliberate reasoning to Thought, leaving the Action–Observation interval without future-directed reasoning. Second Thought uses that interval for auxiliary reasoning reused in later turns; across three benchmarks and three reasoning LLMs, it reduces turn count in every configuration and reduces main-thread decoding in six.

  • Problem

    ReAct confines deliberate reasoning to Thought, leaving Action–Observation intervals without future-directed reasoning, while inference-time reasoning tokens can increase interactive latency.

    Source: Introduction · p. 1, Preliminaries: The Reasoning Idle Window · p. 3

  • Method

    Second Thought populates the reasoning idle window with four auxiliary branches and reuses their interruption-safe atomic thoughts in the subsequent reasoning turn.

    Source: ReAct-style Agent + Second Thought · p. 3, Conclusion · p. 7

  • Results

    Across three benchmarks and three reasoning LLMs, turn count fell in all nine configurations, main-thread decoding in six, and Pass@1 was statistically unchanged in seven and significantly improved in two.

    Source: Conclusion · p. 7

  • Takeaways & Limitations

    The evaluated results show a better accuracy–latency trade-off than unmodified agents and compute-matched controls that place equal or greater reasoning budget on main reasoning.

    Source: ReAct-style Agent + Second Thought · p. 2

    The approach depends on auxiliary outputs remaining useful when interrupted at an arbitrary token within the unpredictable idle-window duration.

    Source: Auxiliary Reasoning Protocol · p. 4

Abstract

from arXiv · show

LLM agents in the ReAct paradigm alternate between reasoning, acting, and observing, but deliberate reasoning is confined to the Thought phase: while the agent serializes an action and waits for the environment, its reasoning is frozen. We identify this recurring interval for Action and Observation as a reasoning idle window and ask whether it can host additional reasoning in parallel that serves future turns. Therefore, we propose Second Thought, a training-free inference framework that forks four auxiliary branches the instant each Thought phase concludes, decodes them concurrently with the main loop, and merges the generated thoughts back when the environment observation arrives. In this way, Second Thought relocates the added reasoning off the main thread's sequential decoding path. Across three agentic benchmarks and three reasoning LLMs, Second Thought lowers the average turn count in all nine (model,benchmark) pairs and reduces main thread decoding in six of them by up to 43% (roughly 20% on average among those settings), while leaving it essentially unchanged in a seventh; Pass@1 shows no significant change in seven of nine pairs and the two significant differences are +12.4 and +10.2 points. Against a compute-matched control that forces an equivalent budget onto the main thread's own reasoning, it attains strictly higher Pass@1 with 1.3 to 3.2 less sequential decoding in all four settings where the control applies.

Abstract · Introduction

Second Thought uses ReAct agents’ Action–Observation idle windows to generate auxiliary reasoning in parallel for later turns. Across nine model–benchmark pairs, it reduced average turn counts and improved the accuracy–latency trade-off without adding sequential decoding to the main thread.

Introduction

  • ReAct agents reason during Thought but not Action or Observation, creating a recurring idle window while the agent acts and awaits the environment.Inference-time reasoning can improve accuracy, but generating every additional reasoning token on the critical path increases latency.Source: Introduction · p. 1
  • Unlike methods that sample competing chains within Thought, idle-window branches condition on the established trajectory and serve future turns, so their outputs need no voting or aggregation.Because the environment observation can abruptly end generation, completed partial thoughts must remain usable even when a branch is cut off.Source: Introduction · p. 1

ReAct-style Agent + Second Thought

  • At each Thought’s end, Second Thought forks four auxiliary branches that run alongside the main loop until the observation arrives.The branches check assumptions, recall context, rehearse likely next steps, and consider alternatives; their outputs are concatenated into the trajectory for the next turn.Source: Introduction · p. 1, ReAct-style Agent + Second Thought · pp. 2–3
  • Interruption-friendly atomic thoughts let completed concepts survive a branch cutoff and be appended to the tool message when the observation arrives.On the illustrated bug-fixing task, recalled constraints and rehearsed fallback searches help avoid a compatibility-breaking edit and costly revert.Source: ReAct-style Agent + Second Thought · p. 2, Figure 1 · p. 2
  • Evaluation covers repository software engineering, terminal operation, and multi-turn tool-calling dialogue across SWE-Bench Pro, Terminal-Bench 2.1, and τ 3-bench with three reasoning LLMs.The models are DeepSeek-V4-Flash, Qwen3.6-Plus, and MiniMax-M3; analysis attributes benefits both to precomputing next-turn deliberation and to surfacing considerations absent from the main trajectory.Source: ReAct-style Agent + Second Thought · p. 2

Introduction

  • Second Thought is a training-free inference framework that treats the unused Action–Observation interval as parallel capacity for additional reasoning.The introduction frames this as a distinct dimension from parallel sampling within the Thought phase.Source: Introduction · p. 1, ReAct-style Agent + Second Thought · p. 2

ReAct-style Agent + Second Thought

  • Across three benchmarks and three reasoning LLMs, Second Thought lowered average turns in all nine model–benchmark pairs and main-thread sequential decoding in seven, with significant Pass@1 gains in two.The gains were +12.4 and +10.2, both on Terminal-Bench 2.1; there was no significant degradation in other pairs. It also achieved a better accuracy–latency trade-off than a compute-matched control.Source: ReAct-style Agent + Second Thought · pp. 2–3

Related Work

Prior parallel-reasoning methods explore and reconcile branches, while asynchronous and speculative agents overlap or predict existing trajectory computation. Second Thought instead uses the agent loop’s idle window to add advisory reasoning to a live trajectory.

Parallel Reasoning

Asynchronous and Speculative Agent Execution

  • Unlike asynchronous and speculative agents that overlap or predict existing trajectory computation, Second Thought adds advisory reasoning during tool latency without verification or rollback.The auxiliary thoughts are consumed in the subsequent round rather than committed as actions, adding reasoning content without lengthening the critical path.Source: Asynchronous and Speculative Agent… · p. 3

Method

Second Thought uses the Action and Observation interval to run auxiliary reasoning in parallel with the main ReAct loop. It merges completed insights into the context for the next turn without adding branch decoding to the main thread’s critical path.

Preliminaries: The Reasoning Idle Window

  • Second Thought forks at the start of the recurring reasoning idle window, when the main thread serializes its action and awaits the environment’s observation.The window exists because deliberate reasoning is confined to Thought; Action renders the existing plan, and Observation involves no decoding.Source: Preliminaries: The Reasoning Idle Window · p. 3

Overview

  • Four concurrent branches generate complementary atomic thoughts during Action and Observation, then merge completed insights into the next turn without adding reasoning tokens to the main thread’s critical path.The branches are Check, Recall, Rehearse, and Alternative; they are terminated when the observation arrives.Source: Overview · p. 4

Thought Harvesting

  • When an observation arrives, Second Thought cancels in-flight branches and harvests their completed thoughts for the next turn.The overview specifies that active branches are terminated and completed atomic thoughts are appended to context.Source: Overview · p. 4, Thought Harvesting · p. 4

Forking via Trajectory Continuation

  • The same model continues from the full trajectory history and current thought in each branch, reusing the shared prefix KV cache and disabling model-native thinking.Each branch receives a branch-specific instruction, avoiding a separate critic or summarization handoff.Source: Forking via Trajectory Continuation · p. 4

Auxiliary Reasoning Protocol

  • Auxiliary reasoning must remain usable under arbitrary interruption and complement, rather than duplicate, the main trajectory’s reasoning.The framework addresses these requirements with atomic thoughts and complementary reasoning dimensions.Source: Auxiliary Reasoning Protocol · p. 4

Atomic Thoughts

  • Each thought is a self-contained unit enclosed in XML tags and limited to 25 words, so interruption invalidates at most the unfinished unit.The main thread can stop branches unilaterally when an observation arrives, without graceful-exit coordination.Source: Atomic Thoughts · p. 4

Reasoning Dimensions

  • The reference design uses four branches to address unverified assumptions, fading historical context, unprepared outcomes, and premature commitment.The branches are Check, Recall, Rehearse, and Alternative, respectively; the number and semantics of branches can be tailored to task needs.Source: Atomic Thoughts · p. 4, Reasoning Dimensions · p. 4

Bench Model Setting Pass@1 #OUTmain #Turns

Second Thought harvests only complete, bounded branch thoughts and appends them to the tool observation, where they inform the next turn’s reasoning.

Harvesting

  • Each branch contributes at most five complete thoughts per dimension; if no branch completes a unit, it is omitted, and if none yields output, the agent falls back to baseline ReAct.Buffers are truncated at the last closed </thought> tag, bounding per-turn context growth to a small constant.Source: Bench Model Setting Pass@1 #OUTmain… · p. 5

Integration

Experiments

Across nine model–benchmark pairs, Second Thought reduces average turns in every pair and main-thread decoding in six while preserving or improving Pass@1 in eight. The experiments also examine the effects of branch design, timing, and cost.

Experimental Setup

  • Experiments cover three agentic benchmarks, three reasoning LLMs, and benchmark-specific harnesses, comparing Second Thought with the unmodified agent and a compute-matched reasoning-extended baseline.The benchmarks are SWE-Bench-Pro (Deng et al. 2025), Terminal-Bench 2.1 (Merrill et al. 2026), and τ 3-bench (Barres et al. 2025); the models are DeepSeek-V4-Flash, Qwen3.6-Plus, and MiniMax-M3. The s1 baseline is unavailable for MiniMax-M3 and τ 3-bench because of prefix-continuation and function-calling constraints.Source: Experimental Setup · p. 5

Main Results

  • Across nine model–benchmark pairs, Second Thought lowers average turns in all nine, cuts main-thread decoding by up to 43% in six, and preserves or improves Pass@1 in eight.The largest reported Pass@1 gain is +12.4 points on Terminal-Bench 2.1 with Qwen3.6-Plus; the single decrease, from 52.0% to 51.3% on SWE-Bench Pro with that model, is not statistically significant. Against s1, Second Thought achieves higher Pass@1 with fewer main-thread tokens in applicable settings, including 31,396 vs. 40,642 on Terminal-Bench 2.1 with Qwen3.6-Plus.Source: Main Results · pp. 5–6

Ablation Study

  • On SWE-Bench Pro with DeepSeek-V4-Flash, the full configuration is Pareto-optimal; only-recall saves 2.5% of main-thread tokens but sacrifices 6.0 Pass@1 points.Removing Recall causes the sharpest accuracy drop (48.0% Pass@1), while removing Rehearse maintains Pass@1 at 52.0% but raises main-thread output from 20.3k to 22.4k tokens (+10%).Source: Ablation Study · p. 6

In-depth Analysis

  • 10.9%: In a paired replay, Second Thought reduces median per-task wall-clock time from 256.9 s to 229.0 s on SWE-Bench Pro with DeepSeek-V4-Flash.Main-thread decoding time falls from 168.7 s to 146.1 s (−13.4%), while tool execution time falls from 71.6 s to 67.3 s (−6.0%). Branch contention reduces decoding throughput by 1.9%, costing 2.8 s of the 27.9 s saved per task.Source: In-depth Analysis · p. 6

Main Branch

  • 56.7% Pass@1: Removing branch truncation improves on the default by +4.7 points, indicating that window duration is the dominant lever in the controlled comparisons.At fixed window length, withholding the just-finalized plan Rt costs 2.7 points (52.0% to 49.3%) despite producing 1.6x as many atoms (11.5 vs. 7.1 per turn).Source: Table 2 · p. 7, Main Branch · p. 7
  • 66.4% to 181.5%: Four auxiliary branches raise total API cost per task across models, with the increase driven almost entirely by input prompt processing.Output-token costs vary by less than $0.02 per task across models; keeping only the top-performing branch (ALTERNATIVE) reduces added cost to 16.3%–35.5%.Source: Table 3 · p. 7, Main Branch · p. 7
  • 196.2 to 316.5 tokens: Removing harvested thoughts increases average next-turn reasoning volume in a replay of 100 DeepSeek-V4-Flash SWE-Bench Pro trajectories.The replay compares turns with harvested thoughts in context against turns without them, suggesting that those thoughts substitute for reasoning otherwise generated on the next turn’s critical path.Source: Main Branch · p. 7
  • 28.7% of turns that harvest contain 86.7% of idle-window time, and 96.5% of tasks receive second thoughts at least once.The fraction of turns yielding a non-empty harvest rises from 0.0% to 90.1% as idle-window length increases relative to the run median.Source: Main Branch · p. 7

Conclusion

Second Thought uses the ReAct action–observation interval for parallel auxiliary reasoning, then reuses those thoughts in the next turn. Across three benchmarks and three reasoning LLMs, it reduces turn count and main-thread decoding while generally leaving Pass@1 unchanged.

Conclusion

  • Turn count fell in all nine configurations (significantly in five), main-thread decoding fell in six (significantly in four), and Pass@1 was statistically unchanged in seven of nine and significantly improved in two.The evaluation covered three benchmarks and three reasoning LLMs.Source: Conclusion · p. 7
  • Second Thought is a training-free framework that fills the recurring ReAct action–observation reasoning idle window with four auxiliary branches: Check, Recall, Rehearse, and Alternative.The window is an interval during which no reasoning is produced.Source: Conclusion · p. 7
  • Interruption-safe atomic thoughts from the auxiliary branches are collected and reused for the subsequent reasoning turn.Source: Conclusion · p. 7

Related papers

Loading 2608.13667v1…