Tree-of-Code: A Tree-Structured Exploring Framework for End-to-End Code Generation and Execution in Complex Task Handling

• CodeAct’s fragmented, single-path turns rely on ground-truth matching and make reasoning inefficient, diverse solution exploration difficult, and trajectory reuse challenging.Source: §1 · p. 1
• CodeProgram completes an end-to-end program in one turn, while ToC explores diverse complete solutions and uses execution success to supervise and select nodes.Source: §1 · p. 2
• Unlike Tree-of-Thoughts, ToC explores distinct complete program solutions, with breadth across solutions and depth through iterative optimization.Source: §1 · p. 2
• Experiments on two multi-tool, complex-task datasets with ten models show ToC significantly enhances zero-shot problem-solving accuracy and efficiency.Source: §1 · p. 2

• Zero-shot queries lack task-level ground truth, making termination unclear; multi-turn interactions also lack standard trajectories for defining process supervision.Checking progress with an LLM-as-judge after every step would add API calls, time, and token costs.Source: §2 · p. 2
• Treating each complete end-to-end execution as an atomic state allows parallel exploration of feasible solutions before selecting an optimal one.Source: §2 · p. 2
• ToC’s tree overview shows complete CodeProgram nodes executed in parallel, with successful nodes collected for voting and execution success or failure providing supervision.Source: Figure 2 · p. 3

3 Tree-of-Code Method

• ToC is an execution-based, self-growing tree that explores complete task-level solutions across breadth and refines them through depth.Each node represents a complete task-level solution.Source: §3 · p. 3, §3.2 · p. 3

3.1 Overview of Tree-of-Code

• The framework models nodes and unidirectional stems, with each new layer reflecting on task context and the thoughts, code, and execution results of ancestor nodes.Task context includes the user query and tool descriptions.Source: §3.1 · p. 3

3.2 Tree Node Generation

• A CodeProgram generates end-to-end code in one node, equivalent to multiple CodeAct turns serving the same final-response purpose.The end-to-end code bridges natural-language reasoning and execution outcomes, while separating reasoning from code execution.Source: §3.2 · pp. 3–4

3.2.1 Code as Reasoning

• CodeProgram uses code writing as reasoning and guides complete code generation with global reasoning techniques such as Chain-of-Thoughts and self-reflection.The root prompt asks the model to analyze the problem, produce reasoning-based thoughts, and then generate complete code reflecting that reasoning.Source: §3.2.1 · p. 4

3.2.2 Two Helper Tools

• Two helper functions support tasks requiring environmental exploration: res_handler generates prompt-compliant summaries, while next_action selects web-browsing actions using page content, visited URLs, and the query.The functions help maintain a cohesive sequence of tool calls.Source: §3.2.2 · p. 4

3.2.3 Execution Outcome as Process Label

• Task-level execution outcomes provide binary labels for filtering feasible solutions without requiring pre-known ground truth or correctness judgments.Successfully executed samples can be selected for supervised fine-tuning, while results and error messages can serve as rewards for reinforced fine-tuning.Source: §3.2.3 · p. 5

3.3 Tree Expansion

• ToC expands breadth-first with three children per parent, varying models and prompts; failed executions trigger reflection and new nodes until branches stop or depth three is reached.Node outcomes provide process supervision, and the tree grows end-to-end from those embedded signals.Source: §3.3 · pp. 5–6

3.4 Final Result Generator

• After collecting valid outputs from successfully executed nodes, the same LLM selects the most likely answer through majority voting and summarization.Ties are rare in the authors' observations, so they choose the most frequent answer without special handling.Source: §3.4 · p. 6

4.1 Setup

• M3 has 82 tasks and 100 tools across five scenario types; API-Bank evaluation uses its 50 most challenging level-3 tasks.The API-Bank tools were functionalized, provided with output examples, and included in the prompt; correctness was checked against expected final-output keywords.Source: §4.1 · p. 6
• Ten LLMs were evaluated against JSON-action ReAct and code-action CodeAct using task accuracy, averaged turns, and output words.The passage defines accuracy as the percentage of complex tasks solved correctly and counts parallel code generated at the same layer as one turn.Source: §4.1 · pp. 6–7

4.2 ToC vs. CodeAct and ReAct

• Nearly 20% higher accuracy with nearly 1/4 the interaction steps and output words: ToC outperforms ReAct and CodeAct on M3 and API-Bank level-3.The comparison reports averaged turns, output words, and accuracy across both datasets.Source: Table 1 · p. 7, §4.2 · p. 7

4.2.1 Other multi-turn vs. Our one-turn

• One-turn CodeProgram sometimes surpasses multi-turn CodeAct and ReAct, especially for Claude models; three-node ToC with random prompts also surpasses CodeAct on average.The comparison reports one turn versus averages of 7.0 and 8.9 turns for multi-turn methods; claude-3-5-sonnet stands out for ToC, while the best model differs by dataset and method.Source: §4.2.1 · p. 7

4.3 Analysis and Ablation Studies

• 84.1% accuracy for 3-3 ToC versus 73.2% for 1-1 on M3; the authors suggest adding nodes contributes more than adding layers.The reported difference is 10.9%; the authors associate more nodes with prompt and model diversity and say extra layers mainly accumulate histories.Source: §4.3 · p. 7, Table 3 · p. 8
• Prompt exploration is more critical in scenarios with lower diversity, according to the prompt-exploration ablation.The ablation is described in Table 4.Source: §4.3 · p. 7, Table 4 · p. 8

• Prior LLM code-generation work mainly targets programming and scientific tasks, whereas this paper addresses zero-shot online tasks involving multiple tool calls.The cited domains include software development, programming assistance, and scientific problems.Source: §5 · p. 8
• CodeAct (Wang et al., 2024) uses code to call multiple tools but is described as step-by-step, relying on ground-truth supervision for each step.The authors contrast that setup with their nodes, each representing a complete solution evaluable through execution supervision without additional labels.Source: §5 · p. 8
• CodeTree (Li et al., 2024b) uses multi-agent search, while ToC grows an end-to-end tree; MCTS approaches require multiple rollouts and significant resources.The authors say ToC directly selects the solution closest to ground truth through voting and addresses complex multi-tool datasets.Source: §5 · p. 8

• ToC self-grows end-to-end thought-code generation based on successful execution for complex multi-tool online tasks.Source: §6 · p. 9
• ToC outperformed baselines on two complex-task datasets, improving both efficiency and task-solving performance.Source: §6 · p. 9

Limited reasoning scope for Program

• Program-level reasoning is limited for open-ended exploration and extremely long sequences, where CodeProgram may not provide a complete final solution.The paper contrasts these cases with predefined-tool industrial settings, where end-to-end execution can be more efficient with fewer turns and LLM calls.Source: Limited reasoning scope for Program · p. 9
• For larger system programs, the method may serve as a subprogram within an overall solution, similar to one agent's role in a multi-agent system.Source: Limited reasoning scope for Program · p. 9

Opportunities for Reflection Refinement

• The reflection mechanism uses execution feedback alone; whether full-history tracking or LLM-based selective summaries work better remains open.The paper also suggests exploring stronger search strategies or adaptive pruning for more complex real-world tasks.Source: Opportunities for Reflection Refinement · p. 9

Vast Potential in Prompt Pool Design

• The prompt pool improves strategy diversity and result robustness, but its current construction relies mainly on simple prompt evolution and manual adjustments.The paper calls for more in-depth, systematic research into prompt-pool construction.Source: Vast Potential in Prompt Pool Design · p. 9

A.1 Root Prompt

• The root prompt asks the model to provide step-by-step thoughts, then interact with a Python environment using code enclosed in execute tags.It also supplies tool descriptions, chat history, and the user's query as prompt context.Source: §A.1 · p. 12

A.2.1 Reflection Prompt

• The reflection prompt asks the model to inspect code and execution, identify issues or optimization opportunities, and suggest specific refinements.It names edge cases, efficiency, and clarity as considerations.Source: §A.2.1 · p. 12

A.2.2 The Prompt for Prompt Evolution

• The prompt-evolution instruction seeks diverse reasoning paths while preserving core content, including orthogonal phrasing or directions that encourage divergent thinking.Source: §A.2.2 · p. 12

A.2.3 The Prompt Sample from Prompt Pool for API-Bank

• The API-Bank sample says tool outputs use JSON-dictionary formatting and instructs the model to follow function descriptions and input-output formats.Source: §A.2.3 · p. 12
• The sample directs the model to navigate the output key, print unfamiliar structures, and keep attention on the user's request.Source: §A.2.3 · p. 12
• The sample asks the model to attempt a complete solution without needing multiple steps.Source: §A.2.3 · p. 12

B.1.1 ResHandler Tool Description

• ResHandler accepts a prompt string and returns a completion string, such as a summary, written output, translation, or answer meeting the prompt requirements.Source: §B.1.1 · p. 12

B.1.2 ResHandler Tool Function

• The implementation imports llm_playground from some_model_API, calls it with the first 20,000 prompt characters and stream=False, then concatenates returned items.Source: §B.1.2 · p. 13

B.2.1 NextAction Tool Description

• NextAction takes a query, current page content, and visited URLs, and returns the next action with the viewed page content.Its documented options are clicking a URL, returning to the previous page, or ending when the page answers the query.Source: §B.2.1 · p. 13

B.2.2 NextAction Tool Description

• The implementation repeatedly scrolls until reaching the page bottom, appends each returned segment to whole-page content, extracts clickable paths from that content, and separates visited from unvisited URLs.Source: §B.2.2 · p. 13

B.3 An example to show the performance of two helpers tools

• The helper-tool example recursively checks for Henry Santiago's email, using ResHandler on relevant page content and NextAction to navigate or return.The example sets a maximum search depth of 30.Source: §B.3 · pp. 14–15, Functionality of the Helper Tools · p. 15

Functionality of the Helper Tools

• ResHandler passes the function's return-value content to an LLM for a final answer, while NextAction bases its decision on page content, the original query, and visited URLs.Source: Functionality of the Helper Tools · p. 15

C Visualization of the Table 2

• Figure 8 visualizes ten LLMs' M3 performance on ReAct, CodeAct, CodeProgram, and ToC sizes 1–3, reporting averages and standard deviations.Source: Figure 8 · p. 16