PaperlayerView exact revision on arXiv

Source-linked AI summary

Hybrid Reasoning Agents

Jiaru Zou, Yunzhe Qi, Sirui Chen, Federico Bianchi, Shang Zhu, Pan Lu, Jingrui He, James Zou

arXiv:2610.18861v1cs.CLcs.AIcs.LG

TL;DR

LLM agents rely on text for reasoning and interaction, while latent reasoning is efficient but lacks native tool access and readily inspectable semantics. HybridAgent combines latent reasoning with text-based interaction in one generation process and reports better performance with lower inference costs across benchmarks and model scales.

  • Problem

    LLM agents use text for both reasoning and environment interaction, while latent reasoning lacks native multi-turn tool pathways and readily inspectable semantics.

    Source: §1 · p. 2

  • Method

    HybridAgent dynamically combines latent reasoning and text interaction, using HybridLink and continuous working memory alongside HybridSFT and HybridRL training.

    Source: §1 · p. 2

  • Results

    10.4% average performance improvement versus frontier LLM agents and agentic frameworks, with reduced token usage, inference time, and computational cost across 11 benchmarks and 3 scales.

    Source: §1 · p. 3, §9 · p. 17

  • Takeaways & Limitations

    HybridAgent offers a scalable, efficient framework that retains text-based tool interaction and external inspectability while using latent reasoning.

    Source: §8 · p. 16, §9 · p. 17

Abstract

from arXiv · show

How can we build an agent that thinks beyond human language, yet still acts through it in the external environment? Modern LLM agents still reason and interact through text, while latent reasoning offers an efficient internal medium but lacks native tool access and reasoning transparency. We bridge these two directions with Hybrid Reasoning Agents (HybridAgents), which dynamically navigate between latent reasoning and text-based interaction throughout a unified generation process. Architecturally, a lightweight HybridLink enables seamless transitions between text and latent modes, while the agent's continuous working memory preserves lossless hybrid context across multiple turns. These designs preserve tool-integrated agentic capabilities while making latent thoughts directly inspectable. Building on this architecture, we further develop dedicated post-training algorithms for hybrid reasoning: HybridSFT stabilizes tool-integrated hybrid generation by teaching the backbone to coordinate with HybridLink, then HybridRL scales reasoning by learning when to think in latent space and when to act through text. We evaluate HybridAgent at 3 model scales across 11 benchmarks covering mathematics, science, medicine, search, and agentic reasoning. Compared with frontier LLM Agents and agentic frameworks spanning both text-based and latent methods, HybridAgent consistently improves performance by 10.4%, while delivering 35.5%-54.7% token reduction, 1.8x-2.6x faster inference, and 2.7x-3.7x cost savings. Code and data are available at https://github.com/togethercomputer/HybridAgent.

1. Introduction

The paper combines latent reasoning with text-based interaction in HybridAgent, then develops training methods to coordinate the two modes. It reports improved benchmark performance and efficiency, with theoretical analyses of memory continuation and hybrid policy updates.

1. Introduction

  • LLM agents use text for reasoning and environment interaction, whereas latent reasoning is efficient but lacks native multi-turn tool pathways and readily inspectable semantics.The paper frames these as limitations keeping latent reasoners from acting as capable agents.Source: §1 · p. 2
  • HybridAgent dynamically combines latent reasoning and text-based interaction in one generation, using HybridLink and continuous working memory to support multi-turn hybrid trajectories.The architecture is paired with HybridSFT for coordination with HybridLink and HybridRL for adaptively balancing latent reasoning and text interaction.Source: §1 · p. 2
  • Continuous working memory losslessly preserves hybrid context across turns, while hybrid policy factorization and exact latent-context replay ground updates over complete trajectories.The paper presents these as complementary analyses of context continuation and hybrid policy optimization.Source: §1 · p. 2
  • 10.4% average performance improvement, with 35.5%–54.7% lower token usage, 1.8×–2.6× faster inference, and 2.7×–3.7× lower computational cost versus frontier LLM agents and agentic frameworks.Evaluation covers 11 benchmarks across mathematics, science, medicine, search, and agentic reasoning, using Qwen3-Base models at 4B, 8B, and 14B scales.Source: §1 · pp. 2–3

2. Preliminary

Hybrid generation alternates between explicit token generation and implicit continuous-space generation. Text supports tool interaction, while the resulting hybrid trajectory is carried forward to produce a final answer.

2. Preliminary

  • Hybrid generation operates autoregressively in text mode over tokens or latent mode over continuous representations.The next hybrid input is formed from either a token embedding or the latent representation, and inference can switch modes dynamically.Source: §2 · p. 3
  • In text mode, the agent generates tool calls, receives tool observations, and continues until it produces a final text answer.The trajectory interleaves text, latent spans, tool calls, and tool responses; the whole trace becomes input for the next generation turn.Source: §2 · p. 3

3. Building a Hybrid Reasoning Agent

HybridAgent unifies text and continuous latent reasoning while retaining autoregressive generation and tool-calling interfaces. HybridLink supports mode transitions, and continuous working memory carries hybrid context across turns.

3.1. Hybrid Mode Interface

  • Directly reusing a last-layer hidden state as the next input creates a mismatch with the pretrained input space, so HybridLink bridges hidden representations back into that space.The adapter carries the hidden state’s semantics into the input embedding space.Source: §3.1 · p. 4
  • Latent spans proceed without token decoding and re-embedding, and the agent dynamically chooses when to remain in or switch modes.Figure 2 presents latent steps as readable pseudo-text and describes their semantic alignment with surrounding text reasoning and tool calls.Source: §3.1 · pp. 4–5, Figure 2 · p. 4

3.2. Multi-Turn Hybrid Generation over Working Memory

  • HybridAgent carries complete hybrid context across turns in continuous working memory, reusing and updating accumulated key-value entries from text, latent thoughts, and tool observations.Later turns resume from that memory and append newly generated entries, avoiding intermediate token-space reconstruction.Source: Figure 3 · p. 5, §3.2 · p. 5

4. Learning to Perform Hybrid Reasoning

HybridAgent uses complementary supervised fine-tuning and reinforcement learning stages: SFT stabilizes coordination with HybridLink, while RL learns when to reason latently and when to act through text.

4. Learning to Perform Hybrid Reasoning

  • HybridSFT and HybridRL adapt standard agentic post-training: SFT teaches reliable tool use and coordination with HybridLink, while RL learns when to reason latently or act through text.The backbone architecture remains intact, so pre-training from scratch is not required.Source: §4 · p. 6

4.1. Stabilize Generation via Hybrid Supervised Fine-tuning

  • HybridSFT converts agentic data into hybrid trajectories, activates mode-switch tokens and HybridLink, then self-conditions training so the backbone learns to reason over its own latent inputs.It jointly supervises text positions with the backbone and latent positions with HybridLink to support stable hybrid generation at inference.Source: §4.1 · pp. 6–7

4.2. Scale Reasoning via Hybrid Reinforcement Learning

  • Because latent positions have no token-level likelihood, HybridRL factorizes the hybrid policy over sampled text positions, whose likelihood-ratio updates suffice to optimize the full trajectory.The method leaves mode usage without an explicit reward, allowing the agent to adaptively balance text and latent generation.Source: §4.2 · pp. 7–8
  • HybridRL uses exact latent-context replay: it records latent thoughts, sampled text, and tool observations, then replays the trajectory in one differentiable pass with latent inputs unchanged.This recomputes sampled text-token likelihoods conditioned on the same latent thoughts used during rollout.Source: §4.2 · p. 8

5. Empirical Evaluations

HybridAgent is evaluated across 11 benchmarks and three model scales, with results covering performance, test-time scaling, and generation efficiency. It improves over agentic baselines and reduces text-token use and inference overhead.

5. Empirical Evaluations

  • Evaluations use Qwen3-Base models at 4B, 8B, and 14B, with HybridSFT followed by HybridRL.Training uses 54K curated hybrid trajectories for HybridSFT and 12K deep-reasoning plus 1K agentic-search examples for HybridRL.Source: §5 · p. 8

5.1. Scaling Performance via Hybrid Reasoning

  • 17.4% average improvement over the strongest agentic baseline at 14B accompanies 18.5% SFT-stage and 10.4% RL-stage gains over corresponding text and latent baselines.Evaluations span 11 benchmarks across agentic reasoning, mathematics, science, medicine, and knowledge search.Source: §5.1 · pp. 9–10, Table 1 · p. 9, Table 2 · p. 10
  • HybridAgent’s performance rises with increased sampling across model scales and benefits more from added test-time compute than text-only RL agents.The pass@k evaluation uses 100 hard mathematics and science questions and compares against text agents trained with GRPO.Source: Figure 5 · p. 9, §5.1 · pp. 10–11

5.2. Efficiency Advantage on Hybrid Generation

  • 2.7× fewer text tokens, up to 54% fewer generation steps, and 2.1× faster inference are reported against text-based baselines.The efficiency analysis covers token throughput, wall-clock inference time, and computation cost.Source: Figure 6 · p. 10, §5.2 · p. 11

6. In-depth Analyses

The analyses examine HybridSFT and HybridRL training behavior, latent-mode reliance and architecture, and text-mode tool use and auditability. Results report performance and interpretability gains alongside retained monitoring and debugging capabilities.

6.1. Training Dynamics on HybridSFT and HybridRL

  • Self-conditioned supervision improves performance by 30.5% over HybridLink activation and brings HybridSFT to parity with standard text SFT across three math benchmarks.HybridLink activation alone remains insufficient; self-conditioning also sharpens latent readability and semantic informativeness across 200 downstream questions.Source: §6.1 · pp. 11–12
  • As HybridRL training progresses, the 14B agent allocates more generation to latent mode while strengthening tool invocation.The reported behavior shifts more reasoning into faster latent thinking while selectively exposing key steps through text.Source: §6.1 · p. 12, Figure 9 · p. 12
  • Removing the correctness reward reduces performance by 6.4%–17.8%, while removing the format reward lowers it by 4.7%–6.9%.Removing the tool reward reduces GAIA and WebWalker performance by 11.2% and 3.5%, respectively, but has a minor effect on MATH500.Source: §6.1 · p. 12, Table 4 · p. 12

6.2. Causal Reliance and HybridLink Design in Latent Mode

  • Disrupting latent thoughts reduces MATH500 accuracy: replacement causes 44.9%–51.3% drops, while disrupting order or limiting spans to one step causes 13.4%–15.0% degradation.The interventions cover 8B and 14B models and indicate reliance on latent-thought semantics, ordering, and reasoning depth.Source: §6.2 · p. 13, Table 5 · p. 13
  • The two-layer residual HybridLink design gains an average 4.3% from its residual connection across projection depths and performs consistently better across three downstream benchmarks.The authors describe the residual branch as preserving input hidden-state semantics while the learned projection aligns representations with the input embedding space.Source: §6.2 · p. 14, Table 6 · p. 14

6.3. Tool Use and Reasoning Auditability in Text Mode

  • HybridAgent has higher valid tool-call rates than GRPO-trained text agents at 8B and 14B, matches 99.7% execution success, and improves tool-invoked answer accuracy by 2.2%–44.1%.Its useful tool-call rate is 4.5%–7.7% higher.Source: §6.3 · p. 14, Figure 10 · p. 14
  • 43%–49% fewer text tokens coexist with 26%–28% more key reasoning steps per 100 tokens, monitoring AUROC within 0.022 of matched text agents, and 98%–99% first-error localization.Text-derived corrective hints improve fix rates by 5.4 and 9.1 points at 14B and 8B, respectively.Source: §6.3 · p. 15, Table 7 · p. 15

7. Related Work

Prior work explores latent reasoning and tool-using agents largely as separate directions. HybridAgent combines latent internal reasoning with text-based tool interaction and auditing.

Latent Reasoning Beyond Text

  • Latent-reasoning methods use hidden states, compressed chains of thought, or optimized continuous trajectories to reduce explicit verbalization.The cited methods improve reasoning efficiency and depth but make direct inspection and external interaction difficult.Source: §7 · p. 16

Agentic Reasoning and Tool Use

  • HybridAgent interleaves continuous thoughts for internal reasoning with text where explicit interaction and monitoring are required.This design integrates inspectable latent reasoning into the agentic generation process while retaining text-based auditing and tool interaction.Source: §7 · p. 16
  • Tool-using agents plan, act, observe, and revise through external environments, while existing frameworks remain text-centric and incur overhead from verbalizing intermediate steps.The passage describes tools including search engines, code interpreters, and APIs.Source: §7 · p. 16

8. Discussion and Future Directions

The discussion frames HybridAgent as a way to keep latent reasoning provider-held while retaining selected text for user-facing interaction and auditing. It also outlines future harness and post-training possibilities.

Implications for Frontier Agent Safety and Secure Deployment

  • HybridAgent’s latent thoughts exist as backbone hidden states readable through provider-owned HybridLink, while providers can assess safety over the full readout.Text is reserved for tool calls, key decisions, and final answers, preserving an auditable trace without exposing the full reasoning.Source: §8 · p. 16

Future Directions on Hybrid Agent Harness and Efficient Post-Training

  • A future agent harness could treat latent working memory as first-class, manage context through KV entries, and schedule workflows by mode.Tool calls and human-facing checkpoints could remain in text while routine computation stays latent.Source: §8 · p. 16
  • Latent thoughts that remain auditable during rollout could support step-level credit assignment, branching promising trajectories, or terminating faulty ones before text or tool calls.These are proposed future uses for more efficient post-training.Source: §8 · p. 16

9. Conclusion

HybridAgent combines latent reasoning with text-based inspection and tool interaction, supported by architectural designs and dedicated SFT and RL post-training. Its working memory preserves the full hybrid history across turns without converting latent thoughts into text.

9. Conclusion

  • Across 11 benchmarks and 3 model scales, HybridAgent performs strongly against advanced LLM agents and agentic frameworks while reducing inference time, token usage, and computational cost.The conclusion describes the framework as scalable and efficient while retaining agentic interaction and external inspectability.Source: §9 · p. 17

A. Multi-turn Continuation over Working Memory

  • Working memory preserves the effects of the complete hybrid history across turns, producing the same hidden outputs and updated KV memory as full-prefix processing.This supports continuing after tool interactions without reconstructing earlier turns or converting latent thoughts into text.Source: A. Multi-turn Continuation over… · pp. 25–26

B. Design and Analyses of Hybrid Supervised Fine-tuning

B.1. Why Self-Conditioned Training?

Self-conditioned HybridSFT trains the backbone and HybridLink on model-generated latent inputs, addressing the train-inference mismatch in latent reasoning. The appendix also describes mode-transition supervision, hybrid-policy replay, and distinctions from related approaches.

B.1. Why Self-Conditioned Training?

  • Continuous latent predictions fed back during inference can deviate from training embeddings and compound over a span, eroding latent-trace coherence.Unlike text feedback, unquantized latent inputs are not projected back into the discrete vocabulary representations seen during training.Source: §B.1 · p. 27

B.2. Learning When to Enter and Exit Latent Mode

  • HybridSFT treats <latent> as a text-mode decision and supervises </latent> from both the HybridLink and backbone sides.The entry anchor is predicted from preceding context; exit supervision covers the last latent step of the span.Source: §B.2 · p. 28

C.1. Formulation of Hybrid Policy Factorization

  • The hybrid policy factorizes over sampled text positions, conditioning each text-token probability on preceding text, latent thoughts, and tool observations.Source: §C.1 · p. 29

C.2. Hybrid Policy Optimization

  • Under the stated replay conditions, HybridRL's accumulated text-token gradient is equivalent to the advantage-weighted likelihood gradient for complete recorded hybrid trajectories.The replay records latent inputs and tool observations while scoring sampled text tokens against their recorded hybrid prefixes.Source: §C.2 · p. 30

D. Extended Related Work

  • HybridAgent switches between latent reasoning and text-based tool interaction at each generation step within one agent, rather than communicating through hybrid channels between agents.Source: D. Extended Related Work · p. 32

E. Hybrid Data Curation

The data-curation pipeline converts existing multi-turn agent demonstrations into hybrid trajectories, assigning reasoning steps to text or latent modes and auditing visibility before construction.

E. Hybrid Data Curation

  • The pipeline converts existing multi-turn agentic demonstrations into 54k hybrid trajectories for text and latent supervision without human labeling.An automatic annotation pipeline assigns each source chain-of-thought step to a mode.Source: E. Hybrid Data Curation · p. 33
  • External LLM judgments and model entropy guide mode assignment, with low-entropy steps as latent candidates and high-entropy steps retained in text.Tool interactions and key steps needed for external inspection are also retained in text mode.Source: E. Hybrid Data Curation · p. 33
  • An independent LLM critic restores steps to text when hiding candidate latent spans removes information needed to follow the agent's decisions.The visibility audit aims to preserve information needed for external debugging and inspection.Source: E. Hybrid Data Curation · p. 33
  • Adjacent latent-assigned steps are merged into spans, anchors are inserted, and removing those anchors is verified to recover the source trajectory byte for byte.Tool calls and observations remain in text form.Source: E. Hybrid Data Curation · p. 34

F. Latent Interpretability Evaluation (Figures 2 and 8)

The interpretability evaluation probes latent spans across downstream questions using readability and context-specificity measures. The paper presents the two measures as complementary evidence for readable, question-specific latent readouts.

F. Latent Interpretability Evaluation (Figures 2 and 8)

  • The probe evaluates 200 questions across downstream tasks and compares final HybridRL checkpoints at 4B, 8B, and 14B scales, alongside variants on a shared Qwen3-14B-Base backbone.The tasks include AIME25/26, MATH-500, MuSiQue, GAIA, and DSQA.Source: F. Latent Interpretability Evaluation… · p. 34
  • Readability measures average nearest-token alignment; scores above the Gaussian-vector noise threshold indicate stronger average token alignment than the reference.A score of 1 indicates every latent input shares the direction of a vocabulary embedding.Source: F. Latent Interpretability Evaluation… · p. 34
  • Context specificity compares a span's pseudo-text overlap with its originating question against overlap with other questions from the same task.Scores above 1× indicate greater lexical relevance to the originating question.Source: F. Latent Interpretability Evaluation… · p. 34
  • The two complementary measures provide evidence of readable, question-specific latent readouts that support external inspection.Source: F. Latent Interpretability Evaluation… · pp. 34–35

G.1. Experiments on 4B-Scale HybridAgent

At 4B, HybridRL achieves the best results across the four reported benchmarks, exceeding ARPO by an average of 2.0 percentage points; cost analyses also report lower decoding compute and estimated costs across 4B, 8B, and 14B. The evaluation spans mathematical, knowledge-intensive, and agentic reasoning, with comparisons against text-based methods, latent reasoning methods, frontier agents, and agentic frameworks.

G.1. Experiments on 4B-Scale HybridAgent

  • 2.0 percentage points: HybridRL exceeds ARPO on average across four benchmarks at 4B, achieving the best results across all four.HybridSFT also outperforms both latent baselines on all four benchmarks and standard text SFT on AIME25 and AIME26; the authors report benefits across 4B, 8B, and 14B scales.Source: §G.1 · p. 35

G.2. Cost Analyses on HybridAgent

  • 62.5%–72.9%: estimated decoding compute falls across 4B, 8B, and 14B; estimated cost per million questions drops from $359 to $98, $439 to $121, and $470 to $176, respectively.The estimates charge each text or latent step a full backbone forward pass and exclude prompt prefill, context-dependent attention, and external tool execution.Source: §G.2 · pp. 35–36, Table 9 · p. 35

H.1. Benchmark and Evaluation Details

  • HybridAgent is evaluated on 11 benchmarks spanning mathematical, knowledge-intensive, and agentic reasoning, assessing multi-step problem solving, specialized knowledge application, evidence integration, and tool-assisted information seeking.AIME 25 comprises 30 problems from the 2025 American Invitational Mathematics Examination; GPQA-Diamond has 198 expert-written questions in biology, physics, and chemistry; DeepSearchQA evaluates multi-step web research.Source: §H.1 · p. 36, (i) Mathematical Reasoning Tasks · p. 36, (ii) Knowledge-Intensive Reasoning Tasks · p. 36

H.2. Baseline Details

  • Text-based baselines include standard SFT and agentic RL methods, with GRPO and ARPO trained using the same RL data and backbone models as HybridAgent.Standard text SFT uses the same training data and backbones and permits tool use during evaluation.Source: (i) Text-based Agentic Methods · p. 37
  • Latent-reasoning baselines include Coconut, SoftCoT, and SoftThinking, with shared backbones and applicable training settings and budgets used for fair comparison.The methods respectively feed back hidden states, use projected assistant-generated soft thoughts, or represent thoughts as probability-weighted vocabulary-embedding mixtures.Source: (ii) Latent Reasoning Methods · pp. 37–38
  • The frontier-agent comparison includes GPT-5.6-Luna, Kimi-K3, GLM-5.3-Flash, DeepSeek-V4.1-Flash, and QwQ-32B.The comparison assesses performance and inference efficiency.Source: (iii) Frontier LLM Agents · p. 38
  • Agentic-framework comparisons use Qwen3-14B backbones with the same tool access and usage budgets as HybridAgent.Compared frameworks include ReAct, Search-o1, and WebThinker.Source: (iv) Agentic Frameworks · p. 38

H.3. Implementation Details

This section documents HybridAgent’s training, inference, and infrastructure settings and illustrates its hybrid reasoning workflow through mathematical and knowledge-intensive examples, including a honey-and-mayonnaise case study verified in Python.

H.3. Implementation Details

  • HybridAgent uses Qwen3-Base backbones at 4B, 8B, and 14B, with a shared two-layer HybridLink and HybridSFT/HybridRL training pipeline.HybridSFT converts 54K text-only sources into hybrid trajectories and uses three epochs; HybridRL uses 12K deep-reasoning examples and 1K agentic-search examples with GRPO rollouts of 512 trajectories. Evaluation checkpoints vary by task type.Source: §H.3 · p. 38
  • Inference budgets are 4,096 tokens or latent steps for math and knowledge-intensive search, and 24,000 for challenging agentic reasoning.Latent spans are limited to 16–256 steps; trajectories allow up to 10 tool calls, searches retrieve 10 snippets, and runs use 32 H100 GPUs.Source: §H.3 · pp. 39–40
  • A shared generation loop coordinates text, latent reasoning, and tools while preserving continuous per-trajectory KV memory across turns.HybridLink enters latent mode through <latent> and exits when readout matches </latent>; the engine also supports batched rollouts and differentiable latent-context replay for HybridRL.Source: §H.3 · p. 40

I. Prompt Template

  • The prompt template puts routine reasoning in latent mode and keeps tool calls and a small set of decisive, auditable steps in text.The annotation guidance labels tool-related or solution-critical steps as text and nonessential inner reasoning as latent; a separate judge prompt checks predictions against golden answers.Source: §I · pp. 41–42

J. Case Study of HybridAgent

  • The math result gives coefficients a = 1, b = −16, c = −4, and d = 43; another answer passage states Physics and marks it correct.Source: J. Case Study of HybridAgent · pp. 43–44
  • Using densities of 1.42 g/mL for honey and 0.91 g/mL for mayonnaise, the calculation converts a gallon to 3785.41 mL and a cup to 236.588 mL.Each cup of honey is calculated to weigh 335.95496 g; the inequality for honey to weigh less than the mayonnaise requires removing more than 5.74648 cups.Source: J. Case Study of HybridAgent · p. 45
  • Python checks that six cups satisfy the mass threshold and five do not, using assertions for both conditions.The code computes the minimum as floor((honey mass − mayonnaise mass) / cup mass) + 1.Source: J. Case Study of HybridAgent · p. 45
  • Six cups must be removed: the honey then weighs about 3359.55 g, below the mayonnaise’s 3444.72 g, while five cups leave 3695.51 g.The passages characterize six one-cup removals as necessary and sufficient.Source: J. Case Study of HybridAgent · p. 45

Related papers

Loading 2610.18861v1…