PaperlayerView exact revision on arXiv

Source-linked AI summary

Second Thought: Reasoning in Parallel as LLM Agents Act and Observe

Zhensu Sun, Chengran Yang, Yunbo Lyu, Jieke Shi, David Lo

arXiv:2608.13667v1cs.AIcs.SE

TL;DR

ReAct agents leave reasoning idle while actions execute and observations arrive, limiting deliberate computation to the sequential Thought phase. Second Thought uses this interval for parallel auxiliary reasoning, lowering turn counts across all nine settings while preserving or improving accuracy in most.

  • Problem

    ReAct confines deliberate reasoning to the Thought phase, leaving the recurring Action–Observation interval without reasoning directed at subsequent turns.

  • Method

    Second Thought is a training-free framework that runs four complementary auxiliary reasoning branches during each reasoning idle window and reuses their interruption-safe thoughts in the next turn.

  • Results

    Second Thought lowers turn count in all nine configurations, reduces main-thread decoding in six, and leaves Pass@1 statistically unchanged in seven while significantly improving it in two.

  • Takeaways & Limitations

    The results support using reasoning idle windows to improve the accuracy–latency trade-off without adding sequential compute to the main thread.

  • Takeaways & Limitations

    Auxiliary execution is bounded by the unpredictable duration of the reasoning idle window, so generated thoughts must remain useful when interrupted at any token.

Abstract

from arXiv · show

LLM agents in the ReAct paradigm alternate between reasoning, acting, and observing, but deliberate reasoning is confined to the Thought phase: while the agent serializes an action and waits for the environment, its reasoning is frozen. We identify this recurring interval for Action and Observation as a reasoning idle window and ask whether it can host additional reasoning in parallel that serves future turns. Therefore, we propose Second Thought, a training-free inference framework that forks four auxiliary branches the instant each Thought phase concludes, decodes them concurrently with the main loop, and merges the generated thoughts back when the environment observation arrives. In this way, Second Thought relocates the added reasoning off the main thread's sequential decoding path. Across three agentic benchmarks and three reasoning LLMs, Second Thought lowers the average turn count in all nine (model,benchmark) pairs and reduces main thread decoding in six of them by up to 43% (roughly 20% on average among those settings), while leaving it essentially unchanged in a seventh; Pass@1 shows no significant change in seven of nine pairs and the two significant differences are +12.4 and +10.2 points. Against a compute-matched control that forces an equivalent budget onto the main thread's own reasoning, it attains strictly higher Pass@1 with 1.3 to 3.2 less sequential decoding in all four settings where the control applies.

Introduction

Second Thought uses the Action–Observation interval in ReAct agents as a reasoning idle window, running four complementary auxiliary branches in parallel without adding sequential main-thread decoding. Across three benchmarks and three reasoning LLMs, it improves the accuracy–latency trade-off by reducing turns and, in many settings, sequential decoding while preserving or improving Pass@1.

  • Introduction: Second Thought identifies the recurring Action–Observation interval in ReAct loops as a reasoning idle window where additional reasoning can run in parallel.In ReAct agents, substantive reasoning occurs during Thought, while Action executes the plan and Observation awaits the environment (Yao et al. 2023b).
  • Introduction: Unlike Self-Consistency (Wang et al. 2022) and Tree-of-Thought search (Yao et al. 2023a), Second Thought parallelizes reasoning across the idle window rather than within Thought.Prior approaches sample multiple candidate reasoning chains horizontally, whereas Second Thought supplies complementary perspectives on one trajectory.
  • Introduction: After each Thought, the training-free framework forks four auxiliary branches—Check, Recall, Rehearse, and Alternative—that run until observation returns and concatenate their outputs.The branches verify assumptions, recall trajectory context, rehearse likely next steps, and draft contingencies rather than compete as candidate solutions.
  • Introduction: Second Thought adds reasoning to the trajectory without adding sequential compute to the main thread, exploiting otherwise unused parallel capacity.Its interruption-friendly atomic thoughts are harvested for the next turn.
  • Introduction: Across three benchmarks and three reasoning LLMs, Second Thought lowers average turn count in all nine model–benchmark pairs and main-thread sequential decoding in seven.The benchmarks are SWE-Bench Pro, Terminal-Bench 2.1, and τ 3-bench; the models are DeepSeek-V4-Flash, Qwen3.6-Plus, and MiniMax-M3.

Related Work

Prior parallel-reasoning methods mainly explore independent solution paths concurrently, while agent-acceleration methods overlap workflow components or speculate about future states. These approaches generally preserve the trajectory’s reasoning content, motivating Second Thought’s contrasting approach.

  • Parallel Reasoning: Parallel reasoning typically explores multiple independent solution paths concurrently and aggregates them through voting, tree search, or model-native branching.Early methods sample complete trajectories, whereas recent methods let models spawn and join child threads.
  • Asynchronous and Speculative Execution: Agent-loop acceleration overlaps token decoding with tool execution, reasoning with response generation, or speculative actions with external updates.These methods execute workflow components out of order to reduce the impact of external latency.
  • Second Thought: Unlike these approaches, which reschedule or speculate the main trajectory’s continuation, Second Thought adds distinct reasoning content while awaiting external updates.The cited approaches therefore leave total trajectory reasoning unchanged and typically preserve task accuracy.

Method

Second Thought augments the ReAct loop by using the Action–Observation interval as a reasoning idle window, asynchronously generating auxiliary thoughts after each main Thought. It merges only usable, bounded partial outputs when the observation arrives, preserving the main thread’s critical path.

  • Motivation: In ReAct (Yao et al. 2023b), deliberate reasoning occurs only during Thought, leaving Action–Observation as a recurring reasoning idle window.Action renders the existing plan, while Observation returns environmental feedback without decoding future-directed reasoning.
  • Workflow: Second Thought forks four asynchronous reasoning branches when the main Thought ends, overlapping their generation with the main Action and Observation phases.The branches receive the complete history and newly generated thought, then inject additional reasoning without delaying the critical path.
  • Atomic Thoughts: Atomic Thoughts make interrupted branch generations usable by requiring independently meaningful units enclosed in XML tags and limited to 25 words without cross-unit references.Truncation therefore invalidates at most the unfinished unit, allowing closed units to be harvested safely.
  • Complementary Reasoning Dimensions: The reference design crosses temporal direction and scope to target four complementary failure modes: retrospective versus prospective reasoning over the current turn versus the overall history.Branches are intended to complement rather than duplicate the main trajectory, and their number and semantics can be tailored to task or compute requirements.
  • Harvesting: When observation arrives, the system cancels unfinished branches, keeps only thoughts ending at closed tags, caps each dimension at five thoughts, and appends the harvest to the tool observation.Branches with no completed unit are omitted; if none produces output, execution falls back to the baseline ReAct loop.

Experiments · Experimental Setup

The experiments evaluate Second Thought across three agentic benchmarks, three reasoning LLMs, and benchmark-specific execution harnesses, comparing it with an unmodified agent and a compute-matched sequential-reasoning control.

  • Benchmarks.: The benchmark suite spans repository-level software engineering, containerized terminal operations, and multi-turn banking tasks.It includes SWE-Bench-Pro, Terminal-Bench 2.1, and τ 3-bench, chosen to cover different task types.
  • Benchmarks.: SWE-Bench-Pro uses 150 randomly sampled instances with a 100-turn limit, grading bug patches by held-out fail-to-pass tests.Each instance runs in a per-instance Docker container.
  • Benchmarks.: Terminal-Bench 2.1 contains 89 containerized terminal-operation tasks, graded by task-specific verification scripts.Tasks include system administration, data processing, and software builds.
  • Benchmarks.: τ 3-bench evaluates 97 multi-turn banking-domain tasks.
  • Benchmarks.: Each setup pairs a reasoning LLM with a benchmark-specific harness, using DeepSeek-V4-Flash, Qwen3.6-Plus, and MiniMax-M3 with native reasoning modes enabled.The models are accessed through streaming APIs; harnesses include mini-SWE-agent, the official Terminal-Bench harness, and τ 3-bench’s standard function-calling harness.
  • Agent.: The baselines are base, the unmodified agent, and s1, a compute-matched control that places Second Thought’s additional reasoning budget on the main thread.s1 adapts budget forcing (Muennighoff et al. 2025) and keeps the extra reasoning on the critical path as a continuation of the turn’s thought.

Main Results

Across nine benchmark–model combinations, Second Thought reduces average turn count universally, cuts main-thread decoding in most settings, and preserves or improves Pass@1 in eight. Its benefits are strongest when simply extending the main reasoning trace is inefficient or harmful, where parallel thoughts can suppress redundant output or improve accuracy.

  • Overall Results: Across nine benchmark–model combinations, Second Thought reduces average turn count in all nine and cuts main-thread decoding in six by up to 43%, roughly 20% on average.The largest reduction is from 36.5k to 20.8k tokens on SWE-Bench Pro with Qwen3.6-Plus.
  • Overall Results: Pass@1 is preserved or improved in eight combinations, while the only main-thread decoding increase is +24.8% in the setting where harvested thoughts appear to open new reasoning paths.The supplied passage identifies this increase as occurring in exactly the setting discussed and contrasts it with merely displacing tokens.
  • Overall Results: On SWE-Bench Pro with Qwen3.6-Plus, the reasoning-extended baseline raises output tokens from 36,519 to 65,634 and lowers Pass@1 from 52.0% to 48.7%, whereas Second Thought suppresses redundant output while preserving accuracy.This illustrates the regime where extending the main reasoning trace hurts accuracy and inflates decoding.
  • Overall Results: On Terminal-Bench 2.1 with DeepSeek-V4-Flash, Second Thought reaches 52.8% Pass@1, +2.2% over both base and s1, while reducing main-thread output tokens to 32,892.The passage characterizes this as an accuracy improvement accompanied by reduced main-thread output.

Ablation Study

Figure 3 evaluates single-dimension and leave-one-out variants of Second Thought on SWE-Bench Pro using DeepSeek-V4-Flash. Single-branch variants generally fail to match the full configuration and often perform near or below the 48.7% baseline.

  • Ablation Study: Single-branch variants generally fail to match the full configuration, with most falling near or below the 48.7% Pass@1 baseline.The comparison uses DeepSeek-V4-Flash on SWE-Bench Pro.
  • Ablation Study: 19.7k main-thread output tokens is the fewest achieved by only-recall, but its Pass@1 drops to 46.0%.This indicates that passive context retrieval without active validation is insufficient.
  • Ablation Study: Figure 3 compares both single-dimension (only-X) and leave-one-out (w/o-X) variants against the full configuration.The ablation is conducted on SWE-Bench Pro with DeepSeek-V4-Flash.

In-depth Analysis

Second Thought’s harvested reasoning is concentrated in longer idle windows and reaches nearly all tasks, while adding substantial API cost primarily through cached-prefix input processing. Replay evidence further indicates that harvested thoughts reduce next-turn reasoning volume.

  • Replay Analysis: Removing harvested thoughts increases average next-turn reasoning from 196.2 to 316.5 tokens in a replay of 100 SWE-Bench Pro trajectories.The replay compares turns following a harvest with and without the harvested thoughts, repeating the experiment three times.
  • Operational Behavior: Harvest yield scales with reasoning idle-window duration, enabling one fixed configuration to cover windows differing by orders of magnitude without per-cell tuning.The fraction of turns yielding a non-empty harvest rises monotonically from 0.0% to 90.1% across relative window-length bins.
  • Operational Behavior: 28.7% of turns harvest second thoughts, yet they account for 86.7% of harvested thoughts, and 96.5% of tasks receive them at least once.Harvest yield rises monotonically with idle-window length, reaching 90.1% in the longest relative-window bin.
  • API Cost: Running four auxiliary branches raises total API cost by 66.4% to 181.5% across models, driven primarily by cached-prefix input processing rather than output generation.Output token costs vary by less than $0.02 per task across models, while overhead tracks provider cache discounts heavily.

Conclusion

Second Thought identifies the action–observation interval in ReAct loops as a reasoning idle window and fills it with four auxiliary branches whose interruption-safe thoughts support subsequent reasoning turns.

  • Second Thought identifies the recurring action–observation interval in ReAct-style loops as a reasoning idle window where no reasoning is produced.
  • The training-free framework populates this window with four auxiliary branches—Check, Recall, Rehearse, and Alternative—that generate interruption-safe atomic thoughts.
  • Second Thought collects these branch outputs and reuses them for the subsequent reasoning turn, with evaluation spanning three benchmarks and three reasoning LLMs.
Loading 2608.13667v1…