THREAD: Thinking Deeper with Recursive Spawning

• LLM performance degrades as context length and complexity grow, limiting settings that require more work than fits in a concise line of generation.Source: §1 · p. 1
• THREAD treats generation as an execution thread that can finish or recursively spawn children, which perform work and return only tokens needed by the parent.This lets the model adapt the intermediate work used to produce different parts of its token sequence.Source: §1 · p. 1
• For agent tasks and question answering, THREAD decomposes problems into progressively simpler sub-problems solved by child threads, with parent threads receiving feedback before their next step.The paper uses few-shot learning and join synchronization in these settings.Source: §1 · p. 2
• THREAD is presented as a flexible alternative to one-line solving and fixed task-decomposition methods, supporting real-time decision adaptation with a shared few-shot prompt across threads.Source: §1 · p. 2
• THREAD achieves state-of-the-art performance with GPT-4 and GPT-3.5 on ALFWorld, TextCraft, WebShop, DataCommons QA, and MIMIC-III ICU QA.With Llama-3-8b and CodeLlama-7b, it outperforms prior methods by 10% to 50% absolute points across the benchmarks.Source: §1 · p. 2

2 THREAD Framework

• A thread generates from a starting context, which is the initial seed for the main thread and derives from the parent’s token sequence for a child; it continues until a termination criterion.Source: §2 · p. 3

2.1 THREAD with Join Synchronization

• With join synchronization, a parent waits for a child to finish, then appends the child’s returned output tokens before resuming generation.The recursive procedure takes a thread context and its generated tokens so far as inputs.Source: §2.1 · p. 3

2.2 Alternative Synchronization and Spawning Mechanisms

• When tasks have no sequential dependencies, THREAD can instead use asynchronous multithreading, letting parents continue generating without waiting for children.Source: §2.2 · p. 3

2.3 Defining Parent-Child Information Exchange with ϕ and ψ

• The ϕ and ψ functions control information propagation from parent to child and child to parent, respectively, and can vary with the implementation setting.Source: §2.3 · p. 3

2.4 Dynamically Adapting Intermediate Work in Token Generation

• Recursive spawning lets THREAD vary the amount of intermediate work used to produce different tokens; thread trees represent how child work contributes to parent outputs.In the illustrated task and question examples, child contexts are based on the parent’s last line.Source: §2.4 · p. 3, Figure 2 · p. 4

• The implementation uses a special stop token to pause a thread and start child output, and an end token to terminate a thread.The child context is formed from the parent’s token sequence before the pause token.Source: §3 · p. 4
• A shared few-shot prompt teaches spawning and problem solving at different thread depths, without explicit rules for when to spawn or how parents should respond to child feedback.The same prompt is prepended to each thread’s context at every step.Source: §3 · p. 4
• For parent-child exchange, ψ returns tokens from a child’s final print statement, while ϕ forms a child context from the parent’s last line and can instantiate shared variables.Source: §3 · pp. 4–5
• A thread can execute an environment action, listen for feedback, append the environment’s output to its token sequence, and continue generating from that feedback.Source: §3 · p. 5

4 Experiments

• The experiments cover five agent-task and question-answering benchmarks, testing GPT-4, GPT-3.5, Llama-3-8b, Llama-2-7b, and CodeLlama-7b with the same prompt per benchmark.The paper also reports task-general prompting results for ALFWorld.Source: §4 · p. 5, §4.1 · p. 5

4.1 ALFWorld

• GPT-4 with THREAD reaches a 98.5% combined success rate on ALFWorld, with 100% success in four of six tasks.GPT-3.5 reaches 95.5%, outperforming all methods by over 9 absolute points; Llama-3-8b and CodeLlama-7b improve over prior methods by 30 to 55 points. With task-general prompting, GPT-3.5 keeps its task-specific combined success rate, while the smaller models decline but remain significantly above prior methods with task-specific prompts.Source: §4.1 · p. 5, Table 2 · p. 6

4.2 TextCraft

• THREAD beats prior TextCraft methods by at least 20 absolute points with GPT-3.5, 40 with Llama-3-8b, and 30 with CodeLlama-7b.Llama-3-8b with THREAD also outperforms GPT-3.5 with every prior method; CodeLlama-7b with THREAD does so except against TDAG.Source: §4.2 · p. 6, Table 3 · p. 7

4.3 WebShop

• On WebShop, GPT-3.5 with THREAD beats prompt-only methods by 4 absolute points in success rate and over 10% in score, and RAP by 1% in success rate.With Llama-3-8b and CodeLlama-7b, THREAD improves success rate by at least 10 absolute points; Llama-3-8b with THREAD exceeds GPT-3.5 with all prior methods except RAP.Source: §4.3 · p. 6, Table 4 · p. 7

4.4 DataCommons QA

• On DataCommons QA, THREAD outperforms prior methods by over 10 absolute points with GPT-3.5, Llama-3-8b, and CodeLlama-7b.Llama-3-8b with THREAD also outperforms GPT-3.5 with all prior methods.Source: §4.4 · p. 7

4.5 MIMIC-III ICU QA

• On MIMIC-III ICU QA, THREAD significantly outperforms prior methods across models except Llama-2-7b.Llama-2-7b has the lowest performance across all methods, consistent with the pattern across benchmarks.Source: §4.5 · p. 7

5 Related Work

• THREAD requires neither special training nor architectural changes, and its supplementary work can extend beyond internal thinking to retrieval or environment interaction.The intermediate work consists of meaningful tokens, which the paper identifies as an interpretability advantage.Source: §5 · pp. 7–8
• Prior decomposition approaches include modular architectures and LLM-based methods; THREAD’s stated distinction is adapting decisions in real time as feedback arrives.The paper contrasts this with methods that adapt plans upon task failure.Source: §5 · p. 8
• THREAD specifies one step at a time, assigns it to a child thread, and uses the child’s feedback before proceeding, unlike ADaPT and TDAG’s full-plan approach.It can also revise sub-tasks when a step fails or returns unexpected feedback.Source: §5 · p. 8

6 Conclusion

• THREAD treats model generation as an execution thread that can spawn child threads to offload work such as thinking or retrieval.Recursive spawning lets the model adapt the intermediate work used to produce different parts of its token sequence.Source: §6 · p. 8
• The paper applies THREAD to agent tasks and question answering.The framework is described as adapting the amount of intermediate work through recursive spawning.Source: §6 · p. 8
• THREAD, implemented with few-shot learning, achieves state-of-the-art performance on diverse benchmarks.The supplied conclusion does not name individual benchmarks or provide result values.Source: §6 · p. 8

Limitations

• THREAD has no explicit error-handling mechanism and relies on LLM reasoning to respond to unexpected feedback from children or the environment.The paper says more work is needed on robust error detection and recovery.Source: Limitations · p. 9
• Limited parent-child communication may cause a child thread to miss information that could help its work, reducing generation efficiency or effectiveness.The child’s context is based on the last line of the parent’s token sequence.Source: Limitations · p. 9
• If error reporting is not handled appropriately, context needed for self-correction may be lost; the paper calls for improving information propagation between threads.The limitation concerns the functions ϕ and ψ that define information propagation.Source: Limitations · p. 9

Ethical Statement

• Improved agent task completion and question answering may amplify risks tied to automated decision-making or harmful LLM use.The statement calls for ethical guidelines, robustness checks, and transparency in deployment.Source: Ethical Statement · p. 9

A Illustration of Intermediate Work used to Produce Tokens

• Figure 3 represents supplemental tokens used to produce each token as thread trees linking parent and child threads.The trees depict how threads interact to form output tokens and the computational work associated with generation.Source: A Illustration of Intermediate Work… · p. 12

B Implementing THREAD for Agent Tasks and Question Answering

• The implementation uses join synchronization, => and END as spawning and termination tokens, and a shared few-shot prompt for all threads.For environment actions, the action is executed and its output is appended to the thread’s token sequence before generation continues.Source: B Implementing THREAD for Agent Tasks… · p. 12

C Additional Experiments with GPT-3.5

• Across five GPT-3.5 runs per benchmark, performance is consistent, and Table 7 also reports average and maximum thread depths.The table presents means and standard errors.Source: C Additional Experiments with GPT-3.5 · p. 12, Table 7 · p. 13

D Task-general Prompting for ALFWorld

• GPT-3.5 achieves the same combined ALFWorld success rate with task-general prompting as with task-specific prompting.Llama-3-8b and CodeLlama-7b degrade under task-general prompting but remain significantly better than prior methods with task-specific prompting.Source: D Task-general Prompting for ALFWorld · p. 13, Table 8 · p. 13

E.1 Error Types

• Errors are classified by hand review, assigning each failed case the first error type that occurs; the analysis focuses on prompt-only methods.Error descriptions and examples are provided for agent and QA tasks.Source: §E.1 · p. 13

E.2 Ablations

• The THREAD ablations remove shared-information variables, replace ϕ with the parent’s full token sequence, or use a separate Thread 0 prompt.The supplied passage describes these modifications but does not report their outcomes.Source: §E.2 · p. 13

E.3 Agent Tasks

• Task decomposition methods, including THREAD, show fewer failures caused by inadequate planning than methods without decomposition.The authors suggest a dedicated planning line of reasoning may explain the difference from methods that plan and act in one generation line.Source: §E.3 · p. 13
• THREAD further reduces inadequate-planning failures by letting a parent revise a failed child’s sub-task specification and spawn another child.The cited example contrasts this with fixed sub-task handlers and methods that do not accommodate sub-task feedback.Source: §E.3 · pp. 13–14
• Decomposed Prompting and THREAD have the fewest environment-feedback interpretation errors, alongside fewer cases requiring long action sequences.The authors propose that shorter action-observation sequences help models track environment information more accurately.Source: §E.3 · p. 14

E.4 QA Tasks

• For QA, Decomposed Prompting and THREAD show fewer wrong-analysis failures, while THREAD has the fewest runtime-error failures.The authors also report that THREAD’s shared-information variables may reduce runtime failures, especially for smaller models such as Llama-3-8b.Source: §E.4 · p. 14

F DataCommons QA and MIMIC-III ICU QA Benchmarks

• DataCommons QA uses publicly available Google Data Commons data, with a focus on U.S. data aggregated from sources including the CDC and WHO.The benchmark draws on data at city, state, county, and country levels.Source: F DataCommons QA and MIMIC-III ICU QA… · p. 14
• DataCommons QA questions are generated from templates designed for verifiable answers across locations and are paired with ground-truth programs.Templates use placeholders to produce varied questions; programs retrieve and analyze data for answers, using operations such as correlation, linear regression, median, or mean. Data Commons contains publicly available data.Source: F DataCommons QA and MIMIC-III ICU QA… · pp. 14, 18
• MIMIC-III ICU QA uses the same benchmark-construction approach, but its questions draw on clinical time-series data.Source: F DataCommons QA and MIMIC-III ICU QA… · pp. 14, 18
• The authors release the full DataCommons QA dataset but cannot directly release MIMIC-III ICU QA because of data-access restrictions.They state that the latter dataset can be reproduced with the released code after gaining MIMIC-III access.Source: F DataCommons QA and MIMIC-III ICU QA… · p. 18

G Prompts

• The prompt examples mark the task or question and distinguish child-thread spawning in different colors.The paper directs readers to the released prompts for full versions.Source: G Prompts · p. 19

G.1 DataCommons QA

• In the DataCommons example, the response compares the 2018–2021 diabetes-percentage slopes for Willows and Villa Ridge and selects Willows.The example reports slopes of 0.3357 for Willows and 0.1642 for Villa Ridge.Source: §G.1 · p. 19

G.2 MIMIC-III ICU QA

• The MIMIC-III example compares patient X’s hour-10-to-20 average systolic pressure with the average among patients who expired in hospital.The example reports 138.18 for patient X and 120.81 for the comparison group, concluding patient X’s average was higher.Source: §G.2 · pp. 20–21

G.3 ALFWorld

• The ALFWorld example breaks cleaning and placing an apple into finding and taking it, cleaning it, and putting it on the target sidetable.The search subtask checks candidate locations in sequence until the apple is found.Source: §G.3 · pp. 21–22

G.4 WebShop

• The WebShop example filters deodorant search results by price, compares attribute matches, then purchases the selected item.The illustrated request specifies a three-ounce bottle, bright citrus scent, sensitive-skin suitability, and a price below $50.Source: §G.4 · pp. 22–24

G.5 TextCraft

• The TextCraft example works backward from a dark oak sign, obtaining or crafting required planks and sticks before crafting the target.The example checks whether materials can be obtained and crafts them when unavailable.Source: §G.5 · pp. 24–25