Source-linked AI summary
ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models
Binfeng Xu, Zhiyuan Peng, Bowen Lei, Subhabrata Mukherjee, Yuchen Liu, Dongkuan Xu
TL;DR
Existing augmented language models repeatedly feed prior context and observations back into the LLM, creating prompt redundancy and token costs. ReWOO decouples reasoning from tool feedback through Planner, Worker, and Solver modules, and evaluations report improved performance with lower token consumption while supporting smaller-model deployment.
Problem
Existing augmented language models interleave reasoning and tool calls, causing repeated prompts and substantial token consumption, while prior work had not explored reducing ALM token usage.
Method
ReWOO separates reasoning, tool calls, and answer synthesis into Planner, Worker, and Solver, with plans composed before workers retrieve evidence.
Results
ReWOO shows comparable or superior performance with substantially lower token usage across evaluations and demonstrates relatively robust performance under tool failures.
Takeaways & Limitations
ReWOO enables smaller language models to use unseen tools in zero-shot setups, supporting scalable and parameter-efficient ALM systems.
Takeaways & Limitations
Planner reasoning can require enumerating all possible plans and may therefore reach the worst-case complexity of observation-dependent reasoning.
Abstract
from arXiv · showhide
Augmented Language Models (ALMs) blend the reasoning capabilities of Large Language Models (LLMs) with tools that allow for knowledge retrieval and action execution. Existing ALM systems trigger LLM thought processes while pulling observations from these tools in an interleaved fashion. Specifically, an LLM reasons to call an external tool, gets halted to fetch the tool's response, and then decides the next action based on all preceding response tokens. Such a paradigm, though straightforward and easy to implement, often leads to huge computation complexity from redundant prompts and repeated execution. This study addresses such challenges for the first time, proposing a modular paradigm ReWOO (Reasoning WithOut Observation) that detaches the reasoning process from external observations, thus significantly reducing token consumption. Comprehensive evaluations across six public NLP benchmarks and a curated dataset reveal consistent performance enhancements with our proposed methodology. Notably, ReWOO achieves 5x token efficiency and 4% accuracy improvement on HotpotQA, a multi-step reasoning benchmark. Furthermore, ReWOO demonstrates robustness under tool-failure scenarios. Beyond prompt efficiency, decoupling parametric modules from non-parametric tool calls enables instruction fine-tuning to offload LLMs into smaller language models, thus substantially reducing model parameters. Our illustrative work offloads reasoning ability from 175B GPT3.5 into 7B LLaMA, demonstrating the significant potential for truly efficient and scalable ALM systems.
1 Introduction
ReWOO addresses prompt redundancy in tool-augmented language models by separating reasoning, tool calls, and answer synthesis into Planner, Worker, and Solver modules. Across benchmark evaluations, it improves prompting efficiency and supports lightweight ALM deployment.
- Motivation: Existing augmented language models interleave verbal reasoning with tool calls, repeatedly suspending and resuming LLM generation.Historical context, exemplars, reasoning traces, and observations are repeatedly supplied to stateless LLM APIs, creating prompt redundancy and token costs.
- ReWOO: ReWOO separates step-wise reasoning, tool calls, and summarization into Planner, Worker, and Solver modules.Planner creates interdependent plans, Worker retrieves external evidence, and Solver combines plans and evidence into the final answer.
- ReWOO: ReWOO decouples LLM reasoning from external observations, avoiding the redundant prompts characteristic of observation-dependent reasoning.The framework is designed to reduce token usage while preserving the interaction between reasoning and external tools.
- Evaluation: ReWOO was evaluated on six multi-step, knowledge-intensive NLP benchmarks and a curated dataset against Direct Prompting, Chain-of-Thought, and ReAct.The evaluation reports averaged benchmark performance and consistent efficiency gains over the observation-dependent ReAct paradigm.
- Implications: ReWOO supports parameter-efficient ALM deployment by enabling reasoning ability to be offloaded from larger models into smaller language models.The paper reports that LLaMA 7B fine-tuned for this purpose can be on par with GPT3.5 in a zero-shot setup.
2 Methodology
ReWOO separates planning, tool execution, and answer synthesis so reasoning can proceed without waiting for observations. Its design targets lower prompt redundancy, supports multi-step dependencies through evidence variables, and enables specialized smaller planners.
- Plan-Work-Solve Paradigm: Planner composes interdependent plans with #E variables that let later Workers refer to evidence from earlier steps.This structure supports multi-step tasks in which subsequent steps depend on prior observations.
- Plan-Work-Solve Paradigm: Worker invokes designated tools after planning and populates #E with real evidence or observations.
- Plan-Work-Solve Paradigm: Solver combines all plans and evidence to produce the original task’s answer or action status.Prompting Solver to use the inputs “with caution” improves performance by helping compensate for Planner or Worker failures.
- Prompt Redundancy Reduction: ReWOO avoids interleaving reasoning with observations, reducing the redundant prompt growth that can make conventional ALMs increasingly expensive as reasoning steps increase.Under an empirical equalization of TAOs and PEs, the analysis reports linear differences in Q, C, and S and quadratic differences in T, A, and O with respect to k.
- Parameter Efficiency by Specialization: ReWOO’s decoupled design supports fine-tuning a smaller Planner on foreseeable-reasoning blueprints generated by a larger language model.The described specialization pipeline fine-tunes Alpaca 7B on GPT-3.5-generated Planner instruction data to obtain a 7B Planner.
3 Experiments
The experiments compare ReWOO with Direct Prompting, CoT, and ReAct across public benchmarks and a curated dataset. ReWOO consistently reduces token usage, improves accuracy over ReAct, and remains relatively robust when tools fail, while unnecessary tools can hurt performance.
- Setups: The evaluation compares Direct Prompting, CoT, and ReAct against ReWOO using benchmark-specific or out-of-task exemplars and aligned tool sets.
- Comparison between Prompting Paradigms: ReWOO reduces token usage by 64% and gains 4.4% absolute accuracy over ReAct averaged across six public benchmarks.
- Comparison between Prompting Paradigms: On SOTUQA, ReWOO exceeds ReAct by 8% absolute accuracy while consuming 43% fewer tokens.
- Comparison between Prompting Paradigms: Adding more tools generally degrades HotpotQA performance, with 17 of 20 analyzed failures involving tool misuse.
- Comparison between Prompting Paradigms: ReWOO is less compromised than ReAct when all tools return “No evidence found.”
- Comparison between Prompting Paradigms: Replacing GPT-3.5-turbo with text-davinci-003 yields better HotpotQA performance with fewer steps and tokens.
4 Limitations and Future Work
ReWOO’s foreseeable reasoning is limited when the environment provides little context. In such settings, the Planner may need to enumerate many possible plans, motivating future systems that combine specialized nodes in a directed acyclic graph.
- Limitations: Foreseeable reasoning becomes impractical for tasks that provide little context about the environment.
- Limitations: Without prior environmental knowledge, the Planner must enumerate possible plans that could lead to the target object.
- Limitations: The Planner’s reasoning-step count in such tasks matches the worst-case complexity of observation-dependent reasoning.
- Future Work: Future work proposes wiring LLMs, tools, and sub-models into a directed acyclic graph with specialized functions.
5 Related Work
Prior ALM research combines LLM reasoning with external tools, while efficient-LLM research reduces model costs through tuning, parameter reduction, and prompt engineering. This paper positions prompting as a flexible way to improve efficiency in tool-augmented LLMs.
- Tool-augmented LLMs: Tool-augmented LLMs combine reasoning with external tools to retrieve current knowledge and solve more complex tasks.Search APIs are used to reduce hallucinations and support more trustworthy generation.
- Efficient LLMs: Efficient-LLM research reduces tuning and deployment costs through smaller models, LoRA, adapters, and prompt tuning.Many approaches modify model structures or update parameters.
- Efficient LLMs: Prompt engineering offers a flexible efficiency approach because it requires no internal model information and works with black-box LLMs.The paper presents its work as the first exploration of prompting for efficient tool-augmented LLMs.
6 Conclusion
ReWOO decouples reasoning from tool observations in a modular ALM framework. The paper reports improved performance with lower token consumption and robustness to tool failures, while identifying future directions for modular optimization.
- 6 Conclusion: ReWOO decouples reasoning from tool feedback and observations to solve multi-step reasoning tasks efficiently.Its theoretical decomposition targets prompting redundancy in thought-action-observation systems.
- 6 Conclusion: ReWOO achieves boosted performance with much less token consumption across public benchmarks and curated tasks.The paper compares it with observation-dependent ALM systems.
- 6 Conclusion: ReWOO shows relatively robust performance under tool-failure cases.The conclusion presents this as a side-study finding.
- 6 Conclusion: Future improvements include modular LLM fine-tuning, tool representation learning, and system graph learning and optimization.The paper presents ReWOO as a foundation for these developments.
A.2 Why ReWOO outperforms ReAct
ReWOO can outperform ReAct because its planning is less disrupted by failed tools and lengthy interaction histories. However, incorrect expectations about retrieved evidence and Solver errors remain practical limitations.
- A.2 Why ReWOO outperforms ReAct: Bad tool responses can derail ReAct into repeated tool calls or infinite action loops until the token limit is reached.The paper also reports that more than four reasoning steps can make ReAct contexts extremely lengthy.
- A.2 Why ReWOO outperforms ReAct: ReWOO usually produces reasonable planning trajectories independently of tool failures.This independence helps explain its superior accuracy over ReAct in the examined cases.
- A.2 Why ReWOO outperforms ReAct: ReWOO plans can fail when they incorrectly assume that a tool response contains the information needed by a later step.For example, a Wikipedia result may not include a person’s age.
- A.2 Why ReWOO outperforms ReAct: Solver can produce a wrong conclusion even when the plans and evidence are correct.The paper suggests that better Solver prompts or simple exemplars could mitigate this issue.
B Implementation Details
The implementation separates planning, evidence retrieval, and answer synthesis through explicit prompts and tools. It also documents an instruction-tuned Planner setup and example prompts for constructing and using plans with evidence.
- B Implementation Details: The implementation uses LoRA to instruction-tune an Alpaca 7B Planner on a single RTX 4090 for 10 epochs.The setup uses batch size 128, learning rate 1e-4, cutoff length 1024, and LoRA rank 8.
- B Implementation Details: ReWOO prompts and tool descriptions are runtime-dependent rather than fixed, so users are encouraged to tailor them to their needs.The paper presents the prompting setup as a general paradigm.
- B Implementation Details: The Planner prompt asks for step-by-step plans, designated tools, tool inputs, and reusable evidence variables.Available tools include Google, Wikipedia, WolframAlpha, Calculator, and an LLM.
- B Implementation Details: An example arithmetic task uses WolframAlpha to solve an equation, an LLM to determine x, and subsequent plans to continue computation.The example stores intermediate tool outputs in evidence variables.
- B Implementation Details: The Solver prompt instructs the model to use the plans and retrieved evidence cautiously before answering directly.The answer prompt requests no extra words beyond the task response.
- B Implementation Details: The documented reasoning trajectories use the prompting and tool configurations described in the implementation section.These trajectories compare ReAct and ReWOO on evaluated datasets.
ReAct
The ReAct examples interleave thoughts, tool actions, and observations to solve multi-step questions, arithmetic, and factual queries.
- ReAct: ReAct searches sequentially, using each observation to decide the next reasoning step.Examples trace searches from Jon Raymond Polito to The Rocketeer and from Melanie C to Lisa Lopes.
- ReAct: The Rocketeer example concludes that Dave Stevens made the 1989 comic book.The trajectory searches Jon Raymond Polito, identifies The Rocketeer, and follows its stated comic-book creator.
- ReAct: The examples also show planner-formatted versions that separate plans from retrieved evidence before solving.The planner traces retain ordered plans and evidence for the comic-book, music, and arithmetic tasks.
- ReAct: The Melanie C example identifies Lisa “Left Eye” Lopes as the featured artist who died in a 2002 car accident.The answer follows two searches: the song and then Lopes.
- ReAct: The arithmetic example computes 10 birds from $200 and then 20 wings.The trajectory uses calculator calls for both divisions and multiplication.
ReWOO
The ReWOO examples represent a task as planned searches and later summarization steps, with evidence passed between stages. One supplied example distinguishes McConnell and Schumer by assigning them different Senate leadership roles.
- ReWOO: The planner first retrieves relevant State of the Union material before asking an LLM to summarize the role difference.The plan uses SearchSOTU to gather evidence and then references that evidence in a later LLM prompt.
- ReWOO: The retrieved transcript describes McConnell as a longtime Senate leader and Schumer as Senate Majority Leader.The transcript also identifies Kevin McCarthy as House Speaker and Hakeem Jeffries as House Minority Leader.
- ReWOO: The supplied solver output states that McConnell is Senate Majority Leader and Schumer is Senate Minority Leader.This conclusion is repeated in both the evidence record and the final statement.
C.8 Other Open-domain Curated Tasks
The curated tasks demonstrate ReWOO with retrieval, location, time, email, trading, drawing, and restaurant tools across information-seeking and action-oriented scenarios.
- C.8 Other Open-domain Curated Tasks: The curated-task suite adds tools for stock analysis, trading, drawing, restaurants, social media, location, time, and email.Each worker accepts a task-specific input format, such as a query, empty input, or semicolon-separated fields.
- C.8 Other Open-domain Curated Tasks: The suite includes workers for current location, current time, and email sending, each with explicitly specified input requirements.Location and Time take empty input, while Email takes three semicolon-separated fields.
- C.8 Other Open-domain Curated Tasks: The cafe workflow retrieves Jersey City, NJ, then searches Yelp and returns Semicolon Cafe with a 4.5-star rating and $$ price range.The workflow demonstrates chaining a location result into a restaurant-search query.
- C.8 Other Open-domain Curated Tasks: The email workflow retrieves location and time, searches for Yann LeCun’s address, and composes a teaching-assistant inquiry.The Email worker receives the target address, subject, and body as semicolon-separated inputs.
–SOLVER–
The solver examples show planned tool calls producing evidence for stock trading and iterative logo generation, including a case where enhancement finds no evidence but still saves the drawing.
- –SOLVER–: The stock workflow plans location, time, market analysis, and a BackTrader action using the retrieved recommendation.Its solver record reports Jersey City, the current time, and a DPST BUY_AND_HOLD recommendation.
- –SOLVER–: The logo workflow decomposes creation into generating a sketch and then enhancing it professionally.Both steps use the Draw worker, with the second prompt referring to the first result.
- –SOLVER–: The logo solver reports that the sketch was saved but no evidence was found for the enhancement step.It nevertheless retains the saved drawing and restates the enhancement instruction.