Source-linked AI summary
Evolving from Tool User to Creator via Training-Free Experience Reuse in Multimodal Reasoning
Xintian Shen, Jiawei Chen, Lihao Zheng, Hao Ma, Tao Wei, Kun Zhan
TL;DR
Existing tool-integrated reasoning systems face open-ended tasks, unreliable ad-hoc tools, and costly manual construction. UCT is a training-free framework that creates tools during inference and consolidates reasoning experience into reusable tool memory. Experiments report superior cross-domain performance and state-of-the-art results across mathematical, scientific, and general VQA tasks.
Problem
Existing TIR tools struggle with incomplete coverage, erroneous generated code, and limited persistence, while manual construction is laborious.
Method
UCT reuses reasoning experience through online task solving, iterative tool building with testing and review, and offline memory consolidation.
Results
Extensive experiments show superior performance across mathematical, scientific, and general VQA domains, including state-of-the-art cross-domain results.
Takeaways & Limitations
UCT enables agents to create, retain, and iteratively upgrade tools during inference without additional training.
Abstract
from arXiv · showhide
Existing Tool-Integrated Reasoning (TIR) models have effectively extended the question-answering capabilities of LLMs by incorporating external tools. However, real-world scenarios present numerous open-ended problems where fixed tools often fail to meet task requirements. Furthermore, the lack of self-optimization mechanisms means that erroneous tool outputs can mislead the LLM's responses. Additionally, the construction of existing tools entails significant manual effort, which consequently constrains their applicability. Recognizing that the reasoning traces of LLMs encapsulate implicit problem-solving capabilities, we propose UCT, a novel training-free framework that transforms agents from tool users to tool creators. This approach harvests reasoning experiences and distills them into reusable assets. This method transforms the agent from a mere tool user into a tool creator, enabling adaptive tool creation and self-updating during the inference process. We also introduce a memory consolidation mechanism to maintain the tool library, ensuring high reusability of retained experiential memory for subsequent reasoning tasks. This novel automated tool construction paradigm continuously improves tool quality during reasoning, allowing the overall agent system to progress without additional training. Extensive experiments demonstrate that our method serves as a novel paradigm for enhancing the capabilities of TIR models. In particular, the significant performance gains achieved +20.86%$\uparrow$ and +23.04%$\uparrow$ on benchmarks across multi-domain mathematical and scientific reasoning tasks validate the self-evolving capability of the agent.
1 Introduction
Existing tool-integrated reasoning systems struggle with open-ended requirements, costly tool construction, and nonreusable or erroneous generated tools. UCT addresses these issues by reusing reasoning experience to create, refine, and maintain tools during inference, with experiments showing strong cross-domain performance.
- Rigid predefined workflows struggle to generalize to open and uncertain environments, while tool construction remains laborious and incomplete.
- UCT transforms agents from tool users into creators that autonomously create and execute tools on demand during inference.The framework reuses reasoning experience and supports self-evolution without additional training.
- UCT combines an Online Task Loop, Online Build Loop, and Offline Memory Consolidation to create, test, organize, and reuse tool experience.The offline module refines retained tool memories for future reasoning.
- UCT establishes an automated, low-redundancy pipeline for tool-library construction that can extend across diverse domains.The paper also releases TRBench with 959 tool-use reasoning instances.
- Extensive experiments report superior performance across mathematical, scientific, and general VQA tasks, including state-of-the-art cross-domain results.
2 Related Work
Prior tool-creating agents generate task-specific tools, but these tools are typically ephemeral and cannot be internalized for later use. The proposed framework instead couples online reasoning and tool creation with offline memory consolidation.
- Prior agents such as CREATOR, LIVE-SWE-AGENT, and CRAFT generate tools through code generation or task-specific design during inference.
- Tools produced by earlier methods are typically one-off artifacts that are not retained or internalized for subsequent tasks.
- The proposed architecture combines an Online Task Loop, Online Build Loop, and Offline Memory Consolidation for self-evolving tool use.The architecture supports reasoning, tool execution, tool creation requests, and asynchronous tool-library maintenance.
3 Methodology
UCT organizes tool-using reasoning into online task execution, online tool creation, and offline memory consolidation. Tools are tested, reviewed, registered as reusable packages, and periodically optimized to improve future retrieval without slowing inference.
- Online Task Loop: The Online Task Loop uses ReAct to select thoughts, core tools, or created tools, execute actions, and feed observations back into the interaction history.The workflow retrieves existing created tools when available and requests new construction after retrieval failure.
- Online Build Loop: The Build Loop isolates tool construction from the original task to control generation and prevent its extensive context from interfering with problem solving.It begins from a build ticket and iteratively generates and refines code.
- Online Build Loop: Runtime testing and code-model review jointly verify generated tools before registration.Failed tests or reviews produce feedback for ReAct-based code regeneration until acceptance criteria are met.
- Online Build Loop: Accepted tools become structured Tool Packages containing code, invocation instructions, dependencies, and test results, then return to the Task Loop for use.This registration process turns reasoning experience into reusable tool assets for the current and later tasks.
- Offline Memory Consolidation: Offline consolidation organizes and merges similar tools, removes duplicates, and discards rarely used or high-failure tools using usage logs and descriptions.The resulting memory retains high-utility experiences, reducing retrieval complexity without affecting online inference speed.
4 TRBench: Tool-Reasoning Benchmark
TRBench addresses the limited suitability of existing evaluation datasets for tool reasoning by filtering non-tool questions and selecting diverse, computation-oriented instances. Its mathematical and scientific subset contains 959 challenging samples requiring tool-assisted solutions.
- Benchmark Motivation: Existing benchmarks include simple questions answerable without tools and knowledge-based questions unrelated to computation or tool use.This limits their ability to evaluate tool creation and usage capabilities.
- Construction: The benchmark filters out questions answerable solely from a model’s internal knowledge to form a candidate set for tool reasoning.The filtering step targets instances where external tool assistance is relevant.
- Construction: Iterative Min-Max sampling selects questions that minimize their maximum cosine similarity to the current set, promoting diversity.The procedure uses question embeddings and runs for five iterations in mathematical and scientific tasks and ten in VQA tasks.
- Dataset Scope: 959 samples remain for scientific and mathematical reasoning after excluding proofs and pure reasoning that are not amenable to direct computation.The retained problems require tool-assisted solutions and span medium and high difficulty levels.
5 Experiment
UCT is evaluated across multimodal mathematical, scientific, and general VQA tasks, with experiments examining benchmark performance, tool reuse, component contributions, and model evolution. The framework improves over basic reasoning and exhibits increasing performance as its tool library develops.
- Experimental Settings: The evaluation covers Visual Question Answering, mathematical reasoning, and scientific problems using multiple established benchmarks.Mathematics uses DynaMath, MathVerse, MathVista, and MathVision; science uses Scibench and Scieval; SimpleVQA measures general VQA.
- Experimental Settings: TRBench contains 959 tool reasoning problems across Mathematics, Science, and General Question Answering, organized into 11 sub-categories and 3 domains.
- Effectiveness and Superiority of UCT: +20.86%↑ and +23.04%↑ improvements over Basic-CoT are reported for Qwen3-VL-235B-thinking and Gemini2.5-pro, respectively.The comparison includes Basic-CoT, a vanilla tool version, and existing tool-creation baselines on TRBench.
- Ablation Studies: UCT’s full framework achieves the highest performance across all reported metrics when the online build loop, critic module, and offline memory consolidation work together.The ablation study states that each phase is necessary within the dependent module design.
- Ablation Studies: 93.1% of created tools are used at least once, indicating low redundancy and broad reuse within the generated tool library.The analysis treats higher utilization as evidence that tools serve system-level utility rather than only task-specific purposes.
- Ablation Studies: Tool-library performance rises across reasoning milestones as memory consolidation refines the library, although evolution plateaus when the dataset offers limited problem-type variety.Figure 6 compares UCT with Qwen3-VL-235B-thinking and Gemini-2.5-pro on the mathematical subset.
6 Conclusion
UCT transitions agents from tool users to tool creators through training-free experience reuse and reusable tool memory. Its three-module architecture creates and upgrades tools during inference, with experiments across diverse domains supporting its effectiveness and generalization.
- Conclusion: UCT combines online task execution, online tool creation, and offline memory consolidation to support self-evolving reasoning during inference.The architecture plans actions, creates tools on demand, and iteratively upgrades retained tools.
- Conclusion: The framework retains reasoning experience as reusable tool assets, enabling autonomous tool creation and iterative improvement without additional training.
- Conclusion: Experiments across diverse datasets and domains validate the framework’s effectiveness and generalization.
A.1 Tool Descriptions for UCT Core Library
UCT’s core library provides foundational multimodal image and text tools for dynamically constructing a reusable tool library from reasoning experience. The tools cover web retrieval, webpage processing, image cropping, and object grounding or visual search.
- Core Library: The core library comprises five categories of multimodal image and text tools used as foundational components.These tools support the paper’s investigation of dynamically constructing a tool library from reasoning experience.
- Image Processing: Region Croping/Zooming produces a new image by zooming into a bounding-box region of the first input image.The bounding box is specified by top-left and bottom-right coordinates, [x1, y1, x2, y2].
- Visual Search: Object Grounding & Visual Search retrieves similar images and descriptions for a bounding-box region of the first input image.It outputs the most similar target’s type name and confidence score, with categories including plant, animal, car, person, landmark, vegetable, cuisine, and logo.
- Text Retrieval: External Text Retrieval returns text information from the internet using a text query.Its input is only a text query, and its output is text.
- Web Retrieval: Webpage Content Retrieval visits a specified URL and can return full content, a selected window, or an AI-generated structured summary.The tool accepts a URL and optional window and goal arguments, returning the visited URL and structured result.
A.2 Prompt Design
UCT’s prompt design specifies roles and interaction formats for reasoning, tool execution, and final answers. It also enforces sequential tool use and provides explicit handling for tool failures.
- Interaction Format: The ReAct assistant format separates thought processes, tool calls, and final answers into explicit tagged stages.Thoughts use <think>, tool calls use <tool_call>, and final responses use <answer>.
- Tool Calls: Tool calls require a unique tool name and a dictionary of input arguments wrapped in <tool_call> tags.The prompt provides a structured example using name and arguments fields.
- Execution Rules: The assistant must wait after each tool call, avoid consecutive calls in one response, and proceed step by step.It may perform multiple rounds of thinking and tool calling before producing the final answer.
- Failure Handling: When a tool fails, the assistant should retain its logically reasoned answer rather than change it arbitrarily.The prompt also permits directly outputting the previously reasoned answer while acknowledging that tool verification failed.
- Role Prompt: The prompt framework defines a Principal Software Engineer role as a high-precision autonomous coding engine.The role emphasizes expert-level software design, implementation, and debugging across many programming languages.