The Era of Agentic Organization: Learning to Organize with Language Models

1 Introduction

• The motivating challenges are parallel-thinking latency from the slowest trace and aggregation, fixed workflows that cannot adapt to query needs, and the difficulty of learning effective organization policies.The paper contrasts tasks suited to divide-and-conquer with those requiring step-by-step reasoning.Source: §1 · p. 2
• AsyncThink uses an organizer that dynamically structures thinking through Fork and Join actions while workers execute sub-queries and return intermediate results.The protocol is intended to enable adaptive, diverse execution structures.Source: §1 · p. 2
• Training combines synthetic role-specific format fine-tuning with reinforcement learning rewards for correctness, format compliance, and thinking concurrency.Source: §1 · p. 2
• Across countdown, mathematical reasoning, and Sudoku, AsyncThink models report higher accuracy and lower latency than sequential and parallel thinking, with zero-shot reasoning on unseen tasks.The models were trained solely on simple countdown data.Source: §1 · p. 2
• The paper formalizes learning to organize as a problem of defining how agents collaborate and operate concurrently.Source: §1 · p. 2

2 Organizer-Worker Thinking Protocol

• The organizer and workers share an LLM backbone and autoregressive decoding, while their available actions distinguish their roles; Fork-Join actions control the thinking trajectory.Source: §2 · p. 2, Figure 2 · p. 3

2.1 Agentic Organization

• An agent performs sequential actions, an agent pool runs agents concurrently at fixed capacity, and an organization policy coordinates collaboration; AsyncThink uses an adaptive policy rather than fixed workflows.Parallel thinking is given as an example of independent agents whose outcomes are aggregated, [53] [40].Source: §2.1 · p. 3

2.2 Organizer

• The organizer coordinates the process and can think, fork a sub-query to an available worker, join to request its result, or answer to end inference.Source: §2.2 · p. 3

2.3 Worker

• Each pool has c −1 workers that independently process organizer-assigned sub-queries and return results when requested.Source: §2.3 · p. 4

2.4 Asynchronous Thinking

• Workers run while the organizer continues generating actions; at a Join, the organizer waits if needed, then resumes with the returned text, and the protocol can express sequential or parallel thinking as special cases.The protocol works at the LLM input-output surface without changing the underlying neural architecture.Source: §2.4 · p. 4

3 Learning to Organize

• Training first fine-tunes the format, then uses reinforcement learning to explore thinking structures and refine asynchronous capabilities with rewards for reasoning quality and efficiency.Source: §3 · p. 4

3.1 Stage 1: Cold-Start Format Fine-Tuning

• For cold-start data, GPT-4o identifies conditionally independent fragments and generates organizer-worker traces, which are rule-checked and filtered for format errors.With pools larger than two, synthesized organizer traces predominantly use interleaved Fork-Join operations or all Forks followed by all Joins.Source: §3.1 · p. 4

Organizer-

• The same LLM plays both organizer and worker roles.The organizer structures the process; workers execute assigned sub-queries and return intermediate knowledge or results.Source: Organizer- · p. 5
• Randomly prompting sampled organizer action sequences broadens exploration beyond one or two thinking topologies.The stated motivation is to avoid limiting training plasticity and exploration of organization policies.Source: Organizer- · p. 5

3.2 Stage 2: Reinforcement Learning

• Cold-start format fine-tuning teaches organizer-action syntax, while reinforcement learning further optimizes asynchronous thinking because format training alone does not produce correct final answers.The RL rewards address correctness and thinking efficiency.Source: Organizer- · p. 5, §3.2 · p. 5
• Three rule-based rewards target answer accuracy, organizer-format compliance, and organizing thinking into concurrently executable parts.Format penalties cover duplicated sub-query indices, agent-pool overflow, invalid joins, and stopping without a final answer; a threshold limits concurrency-reward hacking.Source: §3.2 · pp. 5–6
• Organizer-trace and associated worker-trace outputs form one unit for rewards and group-relative advantages, which are shared across both roles' tokens.Worker-returned text is masked from the loss, while initial Join tags are included.Source: §3.2 · p. 6

4.1 Evaluation Metrics

• Final-answer accuracy measures answer correctness, while critical-path latency estimates the minimum sequential depth and a theoretical lower bound on inference time.The latency measure abstracts away from inference-engine and hardware differences.Source: §4.1 · pp. 6–7

4.2 Multi-Solution Countdown

• 89.0% All Correct compares with 68.6% and 70.5% for the baseline methods on multi-solution countdown.Figure 5 reports correctness thresholds averaged across five random seeds.Source: Figure 5 · p. 8, §4.2 · p. 8

4.3 Math Reasoning

• AsyncThink achieves the best overall performance on AIME-24 and AMC-23 with substantially lower critical-path latency than the baselines.Workers are limited to 512 tokens per sub-query, compared with 2K-token baseline traces.Source: §4.3 · p. 8, Table 2 · p. 9

4.5 Ablation Studies

• Removing format fine-tuning yields much lower accuracy and concurrency ratios of 50.0% and 25.0% on countdown and math, respectively.The authors interpret cold-start format learning as a useful initialization for learning asynchronous structures.Source: §4.5 · p. 9
• During RL, accuracy reaches 89.0% after 180 steps as concurrency and Fork use rise; removing the concurrency reward reduces accuracy and raises latency, while leverage reward can destabilize training.The reported instability sometimes collapses training to near-sequential thinking with interleaved Forks and Joins.Source: §4.5 · pp. 9–10

4.4 Generalization to Unseen Task

• After training on countdown data, AsyncThink achieves superior 4 × 4 Sudoku performance while maintaining much lower latency than parallel thinking.The Sudoku test set contains 400 examples sampled from Enigmata.Source: §4.4 · p. 9, Table 4 · p. 10

4.5 Ablation Studies

• 28% lower inference latency than parallel thinking accompanies higher math reasoning accuracy on AsyncThink's accuracy-latency frontier.Figure 7 varies maximum response-length configurations for the comparison.Source: §4.5 · p. 10, Figure 7 · p. 11

4.6 Case Studies

• Case studies show multistage Fork-Join solution search and concurrent math paths whose intermediate results are merged into coherent answers.The countdown example finds four valid expressions for 888, while sequential thinking fails to find all four in that case.Source: §4.6 · p. 11

• RLVR trains reasoning from automatically verifiable outcomes, but designing verifiable reward functions remains an open problem [51].Related work also discusses model-based verifiers, reasoning reward models, co-evolving generators and rewards, and multiple policy-optimization approaches [29] [26] [14] [41] [48] [52] [36, 46] [37, 1] [9] [35].Source: §5 · pp. 11, 13
• After a Join in a multi-solution countdown trajectory, the organizer checks remaining gaps and may launch another sub-query, ultimately yielding four distinct valid expressions.Source: Figure 8 · p. 12
• Prior parallel reasoning includes independent-trace aggregation and parallel decoding, but often relies on handcrafted coordination or imitation; APR uses DFS/BFS traces, while Parallel-R1 learns mid-trace branching and aggregation.AsyncThink is presented as learning to organize thinking processes and assign diverse sub-queries with lower latency than parallel thinking.Source: §5 · p. 13
• LLM multi-agent systems use specialized roles and dialogue, with approaches including dynamic belief modeling, role exchange, agent co-evolution, central controllers, meta-prompting, and RL-based orchestration [23] [49] [28] [19] [4] [47] [38] [10].These systems include cooperative dialogues and competitive debates [22] [11] [25].Source: §5 · p. 13

• Scaling could increase worker quantity to hundreds or thousands and diversify workers into domain-specific experts equipped with tools such as code interpreters, databases, or web search.The proposed study would examine how accuracy–latency trade-offs evolve with agent-pool capacity.Source: §6 · p. 13
• Recursive organization would let a worker become a sub-organizer and fork sub-workers, enabling hierarchical decomposition for deeply nested problems.One example has a worker independently testing lemmas through three parallel sub-workers.Source: §6 · p. 13
• Human-AI organization could place humans as organizers dispatching AI work or as workers handling tasks requiring human judgment, with planning co-designed before execution.Source: §6 · pp. 13–14

• The appendix provides AsyncThink thinking trajectories as examples of the model’s execution.Source: A AsyncThink Thinking Cases · p. 19
• Figure 9 shows an AsyncThink trajectory on mathematical reasoning.Source: A AsyncThink Thinking Cases · p. 19
• Figures 10 and 11 show trajectories from a model trained on multi-solution countdown data on out-of-domain tasks.Source: A AsyncThink Thinking Cases · p. 19

• The organizer template uses a capacity placeholder that is replaced with the agent-pool capacity minus one for the experimental setup.Source: B Prompt Templates · p. 19
• The organizer delegates subtasks with Fork tags, waits for results with Join tags, integrates them, and returns a final answer; additional forks must wait when capacity is reached.Source: B Prompt Templates · p. 19
• The worker template identifies a subprocess, receives the organizer-assigned subtask, and returns a short summary and valid expressions found.Source: B Prompt Templates · p. 19
• In a math trajectory, three workers use different geometric formulations, derive cos θ = 1/3, then rejoin for verification.Source: Figure 9 · p. 20
• On an unseen graph-theory task, the organizer and worker explore alternatives concurrently and determine the minimum vertex cover [0, 4].Source: Figure 10 · p. 21
• On an unseen genetics task, a worker explores a dihybrid cross while the organizer evaluates combinations, identifying AaBbCcDd × AaBbCcDd as the most probable parental genotypes.Source: Figure 11 · p. 22