AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation

1 Introduction

• Multi-agent conversations address growing task complexity by incorporating feedback, combining complementary capabilities, and partitioning and integrating subtasks.Differently configured agents can exchange reasoning, observations, critiques, and validation with one another or humans.Source: §1 · p. 2
• The framework targets two design challenges: reusable, customizable collaborative agents and a unified interface accommodating diverse conversation patterns.Required patterns include single- or multi-turn dialogs, varying human involvement, and static or dynamic conversations.Source: §1 · p. 2
• AutoGen provides customizable, conversable agents with configurable roles, extensible backends, and capabilities drawn from LLMs, human inputs, tools, or their combinations.Agents receive, react to, and respond to messages; appropriate configurations support autonomous multi-turn conversations or requests for human input at selected rounds.Source: §1 · p. 2
• Conversation programming structures application development around defining agents’ capabilities and roles, then programming their interactions through conversation-centric computation and control.Both steps combine natural and programming languages, with ready-to-use implementations and support for extension and experimentation.Source: §1 · p. 2
• Benchmark evaluations and a pilot study report strong task performance, innovative LLM applications, and reduced development effort with AutoGen.The application collection spans different application complexities and LLM capabilities; the introduction supplies no numerical comparisons.Source: §1 · p. 3

2.1 Conversable Agents

• Conversable agents exchange messages, retain conversational context, and support specialized roles through customization of reusable built-in agents.ConversableAgent is the highest-level abstraction, supporting LLMs, humans, and tools. Its preconfigured subclasses include AssistantAgent for LLM-backed assistance and UserProxyAgent for human input or code/function execution.Source: §2.1 · p. 3
• Agents compose LLM, human, and tool capabilities, including feedback adaptation, configurable human participation, and code or function execution.LLM support includes role playing, conversation-conditioned progress, coding, result caching, error handling, and message templating. Human input can be requested under configurable conditions or skipped, while tools execute LLM-suggested code or function calls.Source: §2.1 · p. 3

2.2 Conversation Programming

• Conversation programming defines agents’ response computations and conversation-driven control flow, expressing workflows through agent actions and message passing.Computations depend on the conversations agents participate in; conversation state determines message recipients and computation procedures, unless a termination condition is satisfied.Source: §2.2 · p. 4
• Unified messaging interfaces and automatic replies let conversations proceed without a separate control module once reply functions are registered and the conversation begins.Receiving a message invokes response generation and returns a reply unless a termination condition holds. Built-in or custom reply functions can use LLMs, tools, human input, or conversations with other agents.Source: §2.2 · p. 5
• AutoGen combines natural-language instructions with Python control over termination, human input, tool execution, and reply behavior.Prompts can direct error correction, output structure, and completion signals. Custom reply functions can invoke LLM-based control, while LLM-proposed function calls enable transitions to code control.Source: §2.2 · p. 5
• AutoGen supports static and dynamic conversations through custom reply functions, LLM-directed function calls, and a GroupChatManager that selects speakers and broadcasts responses.Custom replies can hold the current conversation while invoking other agents according to message content and context; called functions can likewise message additional agents.Source: §2.2 · p. 5

A1: Math Problem Solving

• Two built-in AutoGen agents outperform evaluated alternatives on MATH, while configurable human participation supports problems that autonomous agents cannot solve.Evaluations covered 120 randomly selected level-5 problems and the full MATH test set, but ChatGPT, Multi-Agent Debate, and LangChain ReAct were excluded from full-set evaluation. Human feedback requires setting UserProxyAgent’s human input mode to ‘ALWAYS’; another scenario supports multiple human participants.Source: A1: Math Problem Solving · p. 6, A2: Retrieval-Augmented Code… · p. 6, Figure 4 · p. 7

A4: Multi-Agent Coding

• 8% (GPT-4) and 35% (GPT-3.5-turbo) F-1 gains in identifying unsafe code favored multi-agent over single-agent coding on 100 equally split safe/unsafe tasks.AutoGen reduced OptiGuide’s core workflow from over 430 lines to 100; compared with ChatGPT+Code Interpreter, it reported around 3x user-time savings and 3 - 5 times fewer interactions on average. Based on OptiGuide (Li et al., 2023a), a Commander coordinates a Writer and Safeguard, checking code before execution and returning exceptions for debugging.Source: Figure 4 · p. 7, A4: Multi-Agent Coding · p. 8

A3: Decision Making in Text World Environments

• 15% average performance gain followed adding a commonsense grounding agent to AutoGen’s two-agent system, evaluated on 134 unseen ALFWorld tasks.The original assistant–executor system integrates ReAct prompting (Yao et al., 2022) and achieves similar performance to ReAct. The grounding agent intervenes when recurring errors emerge, supplying physical-world knowledge that helps prevent flawed plans and error loops.Source: Figure 4 · p. 7, A3: Decision Making in Text World… · p. 8

A2: Retrieval-Augmented Code Generation and Question Answering

• Retrieval-augmented Chat improves question answering through interactive retrieval, requesting additional context rather than stopping when retrieved information is insufficient.A retrieval-augmented User Proxy and Assistant implement the system; an ‘UPDATE CONTEXT’ response triggers further retrieval, outperforming an ablation that instead answers ‘I don’t know.’ The application also demonstrates code generation using a codebase containing code absent from GPT-4’s training data.Source: A2: Retrieval-Augmented Code… · pp. 6–7, Figure 4 · p. 7

A5: Dynamic Group Chat

• 12 manually crafted complex tasks showed role-play speaker-selection prompts often achieved higher success and fewer LLM calls than task-only prompts.GroupChatManager repeatedly selects a speaker, collects its response, and broadcasts the message to agents sharing the conversation context. Role-play prompts better considered conversation context and role alignment, enabling collaboration without a predefined speaking order.Source: A5: Dynamic Group Chat · p. 9

A6: Conversational Chess

• Conversational Chess uses a board agent to validate moves, preventing the illegal-move disruptions observed when grounding relies only on prompts.Customizable player agents support AI–AI, AI–human, and human–human play, with switching during a single game. Invalid moves trigger correction before play continues, and the modular board agent can accommodate changing rules or chess variants.Source: A6: Conversational Chess · p. 9

• AutoGen combines conversable agents and conversation programming through a unified conversation interface and automatic replies to support multi-agent cooperation.Developers can reuse, customize, and extend existing agents and program conversations between them.Source: §4 · p. 9
• Experiments report improved performance over state-of-the-art approaches, reduced development code, and decreased manual burden for existing applications.These benefits were reported despite application complexity, including cooperation involving more than two agents or dynamic multi-turn interactions.Source: §4 · p. 9
• AutoGen supports dynamic conversation patterns and human participation, while separating tasks among agents promotes modular development.Each agent can be developed, tested, and maintained separately, simplifying development and code management.Source: §4 · p. 9
• AutoGen remains in early experimental stages, with research needed on effective conversation strategies, efficiency, human control, and safety as agent counts and flexibility increase.Future directions include integrating existing agent implementations, balancing automation with human control, and investigating agent topologies and conversation patterns.Source: §4 · p. 10
• Appendix B provides usage guidelines and future directions, while the authors invite community contributions toward improving development speed, experimentation, effectiveness, and safety.These improvements are stated aspirations rather than additional demonstrated results.Source: §4 · p. 10

Unintended Consequences

• LLM agents changing external environments through code execution or function calls can create risks, requiring safeguards against unintended harm.Installing packages is one example of a potentially risky action.Source: Ethics statement · p. 10

Privacy and Data Protection

• Human participation in agent conversations requires developers to protect user data and conversations with appropriate privacy safeguards.Source: Ethics statement · p. 10

Bias and Fairness

• LLMs can reflect training-data biases, so AutoGen developers should mitigate conversational bias and promote fairness and inclusivity.Source: Ethics statement · p. 10

Accountability and Transparency

• Multi-agent cooperation requires clear accountability and transparency mechanisms so users can understand and trace agents’ decision-making processes.Source: Ethics statement · p. 10

Trust and Reliance

• Combining human intelligence with agent automation raises trust and reliance concerns, making clear communication and user education about capabilities and limitations essential.Source: Ethics statement · p. 10

Single-Agent Systems

• AutoGPT, ChatGPT, LangChain Agents, and Transformers Agent augment single-agent systems with tools rather than multi-agent collaboration.LangChain’s multi-agent implementations are built from scratch using basic orchestration modules, rather than LangChain Agents.Source: A Related Work · p. 14

Multi-Agent Systems

• AutoGen supports multi-agent LLM applications with customizable static and dynamic conversation patterns, tool use, and human involvement.Static patterns keep agent topology unchanged across inputs; AutoGen also allows flexible human participation, including skipping inputs.Source: A Related Work · p. 14, Table 1 · p. 15
• BabyAGI assigns task creation, prioritization, and completion to different agents but uses static conversations, unlike AutoGen’s static and dynamic patterns.AutoGen additionally supports tool use and human involvement.Source: A Related Work · p. 14
• CAMEL enables role-playing cooperation through Inception-prompting but supports only static conversations and lacks native tool use such as code execution.It also records conversations for behavior analysis and capability understanding.Source: A Related Work · p. 14
• Multi-agent debate encourages divergent thinking and improves factuality and reasoning (Du et al., 2023), using predefined conversations without tools or humans.These systems instantiate agents as LLM inference instances; AutoGen offers generic infrastructure for developing such applications with dynamic conversation patterns.Source: A Related Work · p. 14
• MetaGPT (Hong et al., 2023) specializes role-based agent collaboration for automatic software development, whereas AutoGen provides generic infrastructure for applications across scenarios.MetaGPT assigns different roles to GPTs to collaboratively develop software.Source: A Related Work · p. 15

B Expanded Discussion

• AutoGen enables new applications and renovates existing ones while keeping implementations simple, even with dynamic conversations or cooperation among multiple agents.A1 scenario 3, A5, and A6 demonstrate dynamic rather than fixed back-and-forth conversation patterns; A1–A4 demonstrate renovation of existing applications.Source: B Expanded Discussion · p. 16
• 4x savings in A4’s core workflow code reduced its implementation from over 430 lines to 100 lines.Built-in agents provide strong performance without customization, while extensibility supports application-specific needs. Separating tasks into agents also permits independent development, testing, and maintenance.Source: B Expanded Discussion · p. 16
• AutoGen supports collaborative and adversarial interactions alongside optional human participation, oversight, and interactive instructions.Collaborating agents share information to complement their abilities; controlled information sharing across conversations supports adversarial scenarios while preventing distraction or hallucination.Source: B Expanded Discussion · p. 16

B.1 General Guidelines for Using AutoGen

• Start with built-in AssistantAgent for GPT-4-backed problem-solving through code and UserProxyAgent for human input and tool execution.Customization options include human-input mode, termination conditions, code execution, LLM configuration, and initial-message instructions without changing the system message. UserProxyAgent can handle additional execution environments and exceptions, while system-message changes can express conversation flow in natural language.Source: §B.1 · p. 16
• Begin with two-agent or group chat topologies and reuse built-in LLM-, tool-, or human-based reply methods before implementing custom ones.Two-agent chats can dynamically involve additional agents through LLM-consumable functions. GroupChatManager illustrates reply-method reuse by using the built-in LLM reply function to select the next speaker.Source: §B.1 · p. 16
• Start UserProxyAgent development with human input mode ‘ALWAYS’, switching to ‘NEVER’ only after confidence in small-scale success.Human involvement supports AssistantAgent evaluation, prompt tuning, corner-case discovery, and debugging. After switching, LLM-generated or manually written diverse system messages can simulate different use cases.Source: §B.1 · pp. 16–17
• Use complementary libraries or low-level inference APIs for unidirectional subtasks that do not require back-and-forth troubleshooting or multi-agent interaction.Existing tools can serve as AutoGen backends, and external agents can be wrapped as conversable agents. Blackbox optimization packages such as ‘flaml.tune’ can automate tuning choices such as LLM inference configuration.Source: §B.1 · p. 17

Designing optimal multi-agent workflows

• Optimal multi-agent workflows may depend on the application, requiring research into where multiple agents help and which configurations are most cost-effective.Design choices include agent counts, roles, capabilities, interaction patterns, and which workflow components to automate.Source: §B.2 · p. 17

Creating highly capable agents

• Highly capable agents should combine LLMs, tools, and humans to troubleshoot tasks effectively, rather than rely solely on role-playing conversations.The authors attribute CAMEL’s frequent failures primarily to its inability to execute tools or code. Proposed directions include application-specific agent guidelines, an open-source agent knowledge base, and agents that discover and upgrade skills (Cai et al., 2023).Source: §B.2 · p. 17

Enabling scale, safety, and human agency

• Whether further scaling helps solve extremely complex tasks remains open; increasingly complex workflows require mechanisms for tracking and debugging behavior.Without these mechanisms, workflows risk becoming incomprehensible chatter among agents (Lewis et al., 2017).Source: §B.2 · p. 17
• Autonomous workflows need safeguards against cascading failures, exploitation, reward hacking, and undesired behavior, alongside effective human oversight.High autonomy poses potential risks, especially in high-risk applications. Despite AutoGen’s user proxy agent, stakeholders must determine appropriate levels and patterns of human involvement for safe, ethical use.Source: §B.2 · p. 17

C Default System Message for Assistant Agent

• The default system message programs conversations through natural-language instructions covering role play, control flow, output confine, automation, and grounding.Figure 5 identifies these instruction types within the built-in assistant’s default message.Source: Figure 5 · p. 18
• Combining prompting techniques enables complex conversations even with the simplest two-agent conversation topology.The example uses AutoGen v0.1.1’s default system message for its built-in assistant agent.Source: C Default System Message for Assistant… · p. 18
• The prompting approach relies heavily on LLMs’ capacity for implicit state inference.This capability underpins the attempt to program complex conversations through combined instructions.Source: C Default System Message for Assistant… · p. 18
• Because LLMs do not follow every instruction perfectly, system designs need additional mechanisms to handle exceptions and faults.Source: C Default System Message for Assistant… · p. 18
• Designers can reduce instruction ambiguity for precision or retain it for flexibility while assigning other agents to handle different situations.Source: C Default System Message for Assistant… · p. 18
• The authors observe that GPT-4 follows instructions better than GPT-3.5-turbo.Source: C Default System Message for Assistant… · p. 18

Scenario 1: Autonomous Problem Solving

• Comparisons included AutoGPT, ChatGPT+Plugin, ChatGPT+Code Interpreter, LangChain ReAct+Python, and Multi-Agent Debate.AutoGPT was excluded from quantitative evaluation because it could not access code-execution results and solved neither qualitative problem. Preliminary evaluations found BabyAGI, CAMEL, and MetaGPT unsuitable for math solving out of the box.Source: A1: Math Problem Solving · pp. 19–20
• Two level-5 MATH problems exposed failures: AutoGPT and LangChain failed every trial, while ChatGPT+Code Interpreter failed the number-theory problem.Each problem was tested three times; ChatGPT+Plugin struggled with both. AutoGPT encountered code-execution issues, whereas LangChain generated code producing incorrect answers.Source: Table 2 · p. 19, A1: Math Problem Solving · pp. 19–20
• 69.48% overall accuracy on 5000 MATH test problems exceeded GPT-4’s 55.18%, and AutoGen achieved the highest success rate among compared methods.A separate evaluation used 120 level-5 problems across six categories, excluding geometry. All evaluations used GPT-4 with sympy pre-installed.Source: A1: Math Problem Solving · pp. 19–20
• AutoGen and ChatGPT+Code Interpreter operated smoothly, while AutoGPT required manual execution, ChatGPT+Plugin could loop, and LangChain could terminate with parsing errors.ChatGPT+Plugin was least verbose and AutoGPT most verbose. AutoGen and ChatGPT+Code Interpreter had similar verbosity, with LangChain slightly more verbose because of code-execution errors.Source: A1: Math Problem Solving · p. 20

Scenario 2: Human-in-the-loop Problem Solving

• Three human-guided trials all succeeded with AutoGen; ChatGPT+Code Interpreter and ChatGPT+Plugin each solved two of three, while AutoGPT solved none.The plane-angle-bisector problem resisted autonomous solution across three trials; AutoGen enabled feedback through human input mode='ALWAYS'. Successive hints requested equal distances, removal of absolute values, and selection using the given point.Source: A1: Math Problem Solving · p. 21

Scenario 3: Multi-User Problem Solving

• The multi-user workflow pauses student–assistant conversations to consult an expert and their assistant, then resumes using the expert’s response.GPT function calling invokes a predefined ask for expert function when a solution is unsatisfactory. Configured UserProxyAgent and AssistantAgent components require only a few additional function lines, with extensions supporting multiple experts or students.Source: A1: Math Problem Solving · p. 21

Detailed Workflow

• The Retrieval-Augmented User Proxy chunks documents, computes embeddings, stores them in a vector database, and sends similarity-retrieved context with questions to the assistant.Initialization requires a document-collection path; the assistant uses an LLM to generate code or textual answers from the supplied context.Source: A2: Retrieval-Augmented Code… · p. 22
• The agents iterate through code-execution feedback and “Update Context” requests, retrieving the next-most-similar chunks when the assistant cannot answer satisfactorily.When enabled, human input can also supply feedback or request context updates. Conversation terminates when no code or context-update instruction remains, or when documents are exhausted.Source: A2: Retrieval-Augmented Code… · p. 22

Scenario 1: Evaluation on Natural Questions QA dataset

• Interactive retrieval enabled GPT-3.5-turbo to answer a Natural Questions example correctly after the initially retrieved context lacked the required information.The evaluation used 5,332 non-redundant context documents and 6,775 queries from Natural Questions.Source: A2: Retrieval-Augmented Code… · pp. 22–23, Figure 8 · p. 23
• 23.40% F1 and 62.60% Recall were achieved by AutoGen W/O interactive retrieval on the first 500 questions using the prompt in (Adlakha et al., 2023).The authors attribute its advantage over DPR to the different retriever: straightforward vector search using all-MiniLM-L6-v2 embeddings.Source: A2: Retrieval-Augmented Code… · p. 23
• Approximately 19.4% of Natural Questions questions triggered an “Update Context” operation, resulting in additional LLM calls.The analysis compared experiments involving AutoGen and AutoGen W/O interactive retrieval.Source: A2: Retrieval-Augmented Code… · p. 23

Scenario 2: Code Generation Leveraging Latest APIs from the Codebase

• Retrieval-Augmented Chat generated correct code using newer FLAML Spark APIs, whereas GPT-4 alone invented a nonexistent spark parameter.The task required 30-second parallel training with forced cancellation. Retrieved documentation supplied Spark APIs added in December 2022 that were absent from GPT-4’s training data.Source: A2: Retrieval-Augmented Code… · p. 23

Task environment

• More than 40 steps may be required for an ALFWorld household task, motivating subtask decomposition and effective exploration.ALFWorld (Shridhar et al., 2021) provides textual household environments in which agents interact to accomplish high-level goals.Source: A3: Decision Making in Text World… · p. 24

Detailed Workflow

• The assistant plans and selects actions, while an ALFWorld-specific executor performs them and returns environmental feedback.To satisfy strict action-format requirements, the executor selects the valid action with the highest BLEU similarity to the assistant’s output.Source: A3: Decision Making in Text World… · p. 24

Performance comparison

• The three-agent ALFChat design significantly outperforms ReAct (Yao et al., 2022), while the two-agent design matches its performance on ALFWorld.Evaluation uses two-shot prompts adapted for conversation and reports task success rates over three attempts; grounding improves performance across all task types.Source: A3: Decision Making in Text World… · pp. 24–25, Table 3 · p. 25

Case study

• Grounding corrects confusion between finding and taking objects, helping the assistant escape failure loops illustrated in the case study.This confusion causes most failures in ‘pick’ and ‘look’ tasks without grounding.Source: A3: Decision Making in Text World… · p. 25, Figure 10 · p. 25

Takeaways

• A grounding agent supplies missing commonsense knowledge, improving decision-making while demonstrating AutoGen’s modularity.Knowledge identified through failure analysis is supplied at task initialization and whenever the assistant repeats the same action three consecutive times.Source: A3: Decision Making in Text World… · pp. 24–25

A4: Multi-Agent Coding

• OptiGuide’s Commander coordinates a Writer and Safeguard, executing code only after safety screening and returning the Writer’s interpretation to the user.Security flags or execution failures send logs back to the Writer for retries until resolution or timeout. The Commander shares retained user context, while role-specific memories remain isolated to prevent shortcuts and hallucinations.Source: A4: Multi-Agent Coding · pp. 26–27
• 100 lines replace over 430 lines of OptiGuide workflow code using AutoGen, with coder and interpreter roles consolidated into one customizable Writer agent.The agents are conversable and can autonomously manage their chat memories, making the implementation more concise and easier to maintain.Source: A4: Multi-Agent Coding · p. 26
• 3x user-time savings were reported for AutoGen-based OptiGuide versus ChatGPT + Code Interpreter, with both systems answering eight of ten questions correctly.An expert Python programmer proficient in Gurobi tested ten randomly selected coffee-supply-chain questions. Average problem-solving time was around 1.5 minutes with OptiGuide versus 4 minutes and 35 seconds with Code Interpreter.Source: A4: Multi-Agent Coding · p. 27
• 3–5 times fewer user interactions were reported for AutoGen-based OptiGuide across 2000 questions and five applications, compared with ChatGPT + Code Interpreter.Table 4 describes preserved coding performance, while a separate user-experience comparison reports a 5x reduction in interaction. ChatGPT + Code Interpreter requires manual engineering steps for private or customized dependencies such as Gurobi.Source: Table 4 · p. 27, A4: Multi-Agent Coding · p. 27

A5: Dynamic Group Chat

• The GroupChatManager selects one speaker, requests a response, and broadcasts that response to the other agents.The four-agent pilot used a user proxy, engineer, critic, and code executor. Across 12 manually crafted tasks, it compared role-play selection against an assistant–user-proxy pair and the same four-agent group using task-based selection.Source: Figure 12 · p. 28, A5: Dynamic Group Chat · p. 28
• Role-play speaker selection achieved higher task success, fewer LLM calls, and fewer termination failures than task-based selection in the dynamic group-chat pilot.The authors attribute this improvement to better consideration of conversation context and role alignment when selecting the next speaker.Source: Table 5 · p. 28, Table 6 · p. 28, A5: Dynamic Group Chat · p. 28

A6: Conversational Chess

• Conversational Chess supports human or AI players expressing moves creatively, while a board agent supplies board information and enforces legal moves.Player agents use human input when supplied and otherwise generate messages with an LLM. The board agent parses natural-language moves into structured form, rejects illegal moves, and requires retries; LLM players use board state and error feedback.Source: A6: Conversational Chess · p. 30
• AutoGen composes chess-agent behaviors through custom reply functions, concentrating extensions at one point and simplifying development and maintenance.Its agent abstractions naturally represent game objects and interactions, while isolated chat messages simplify constructing LLM chat-completion calls.Source: A6: Conversational Chess · p. 30

A7: Online Decision Making for Browser interactions

• MiniWobChat pairs a built-in AssistantAgent that chooses browser actions with a customized UserProxyAgent that executes them and returns feedback.Sequential decisions use the browser’s current HTML state. The assistant records feedback and continues until success or failure.Source: A7: Online Decision Making for Browser… · p. 32, Figure 17 · p. 32
• Auto-GPT’s limited extensibility makes MiniWoB++ instructions difficult to enforce, with no clear route to AutoGen’s two-agent interaction pattern.Its natural-language goal interface proves challenging for tasks requiring complex benchmark-specific rules.Source: A7: Online Decision Making for Browser… · p. 33
• Separating decision-making from environment execution enables agent reuse and simplifies maintenance, with the built-in AssistantAgent performing strongly without customization.Developers can reuse decision-making agents across environments with minimal effort, while changes to the assistant or executor do not adversely affect the other component.Source: A7: Online Decision Making for Browser… · pp. 32–33
• 52.8% success across 49 MiniWoB++ tasks places MiniWobChat 3.6% below RCI (Kim et al., 2023).Each task uses ten instances, and RCI runs with its official code’s default settings. With a per-task success-rate tolerance of 0.1, each method outperforms the other on an equal number of tasks.Source: A7: Online Decision Making for Browser… · pp. 32–33, Figure 18 · p. 33
• AutoGen’s AssistantAgent produces infeasible characters on click-checkboxes-large, redundant action content on count-shape, and actions outside its plan on use-spinner; RCI acts outside its plan or provides wrong plans.On click-checkboxes-large, AutoGen scores 5/10 versus RCI’s 0/10; on count-shape, AutoGen scores 2/10 versus RCI’s 0/10. On use-spinner, AutoGen scores 0/10 versus RCI’s 1/10.Source: A7: Online Decision Making for Browser… · p. 33

E Example outputs from applications

• AutoGen solves the illustrated math problem correctly, while ChatGPT+Plugin and Multi-Agent Debate give wrong answers and other systems exhibit code-related failures.LangChain ReAct and ChatGPT+Code Interpreter generate code inconsistent with their plans and return decimals; ChatGPT+Plugin selects incorrectly despite Wolfram’s correct answer. AutoGPT is aborted after 2 turns without printing a result or rectifying the problem.Source: Table 8 · p. 34, Table 9 · p. 35, Table 10 · p. 35
• 5% higher roasting costs form the shared OptiGuide what-if question illustrated with AutoGen and ChatGPT+Code Interpreter.The question attributes the hypothetical cost increase to a potential salary increase; the supplied captions do not establish comparative outcomes.Source: Table 13 · p. 38, Table 15 · p. 39
• Preliminary math examples show CAMEL producing unexecutable code, MetaGPT returning a product solution, and BabyAGI generating additional tasks before an aborted run.CAMEL’s Math Solver and Python Programmer continue conversing without focusing on solving the problem and delivering an answer.Source: E Example outputs from applications · p. 40, Table 17 · p. 41, Table 18 · p. 42