Automated test generation to evaluate tool-augmented LLMs as conversational AI agents

1 Introduction

• Customer-support agents must follow company procedures and withstand customer manipulation, making comprehensive evaluation important.The introduction notes that careless deployment with real users and APIs can bring misinformation, reputational damage, and company costs.Source: §1 · p. 1
• The proposed pipeline uses intermediate graphs to ground generated tests in user-defined procedures, improve coverage, and incorporate red teaming.The method aims to reduce manual effort while supporting realistic, diverse conversations involving tools and procedures.Source: §1 · pp. 1–2
• The pipeline can generate datasets from synthetic intents and can also use real company procedures or APIs; ALMITA is used to benchmark LLMs.The authors say the method could be applied to other domains with changes.Source: §1 · pp. 1–2
• The pipeline turns procedures and relevant APIs into a flowgraph, then a noisy conversation graph whose sampled paths become generated conversations and tests.The figure describes stages from procedure generation through test extraction.Source: Figure 1 · p. 2
• ALMITA is a manually curated dataset of 1,420 synthetic tests for evaluating customer-support agents on tool use and procedure-following.Two annotators review the samples to ensure quality.Source: §1 · p. 2
• Current LLMs perform well on single-message accuracy and correct function calls, but have limited accuracy across complete conversations.The authors suggest this may indicate that these models would not succeed as fully autonomous customer-support agents.Source: §1 · p. 2

2 Related work

• GAIA and AgentInstruct address other benchmark and synthetic-data settings, but do not assess tool-augmented AI agents.GAIA covers human-annotated questions, while AgentInstruct generates data from diverse sources.Source: §2 · p. 2
• AgentTuning (Zeng et al., 2023), AgentBench (Liu et al., 2023), and APIGen (Liu et al., 2024c) address agent or API tasks, but not conversational tool use with graph-based coverage.The cited datasets focus on API-call sequences or mapping utterances to calls rather than conversations.Source: §2 · p. 2
• InstructGLM (Ye et al., 2023), NL-Graph, and question-answer blueprints (Narayan et al., 2023) use graph or other intermediate structures, but not the full setting of dialog, API calls, and test extraction.Source: §2 · p. 3

3.1 Intent generator

• Intents seed procedures and relevant APIs, which may be generated or supplied from existing sources.Procedures are prompted to use specific, unambiguous steps; APIs describe agent-facing calls and their input and output parameters.Source: §3.1 · p. 3, §3.2 · p. 3, §3.3 · p. 3

3.4 Flowgraph generator

• A flowgraph encodes procedure logic as agent actions and customer-reply or API-output edges, with every procedure detail required in message nodes.Generated graphs are checked against structural rules and discarded if those rules are not met.Source: §3.4 · pp. 3–4

3.5 Conversation graph generator

• The conversation graph converts procedure-oriented flowgraphs into dialogue-like directed graphs containing agent, customer, and API nodes.Conditional API outcomes are represented on edges when paths depend on results such as an order being found or not.Source: §3.5 · p. 4

3.6 Noise generator

• The noise generator adds out-of-procedure or attack customer messages and a corresponding agent response to conversation graphs.It inserts these branches from agent nodes with a set probability.Source: §3.6 · p. 5

3.7 Path sampler

• Weighted random walks sample conversation-graph paths, choosing child nodes inversely to their visit counts to improve node coverage.Sampling continues until a leaf is reached.Source: §3.7 · p. 5

3.8 Conversation generator

• The conversation generator uses a graph, sampled path, and relevant APIs to prompt an LLM to produce a conversation under message-order and API-type constraints.One-shot prompting supplies an example graph, APIs, path, and corresponding conversation.Source: §3.8 · p. 5

3.9 Test extractor

• Tests pair preceding conversation context with the next expected agent response or API call; Table 1 tracks sample counts through filtering, including auto-ALMITA without human filtering.Contexts end with a customer message or API output.Source: §3.9 · p. 6, Table 1 · p. 6

4.1 Dataset generation: ALMITA

• ALMITA contains 192 conversations and 1,420 tests, curated from 14 intents, 18 procedures, 18 flowgraphs, and 18 conversation graphs.Two annotators independently reviewed datapoints, removing any judged incorrect by at least one; GPT-4 was used for all generation steps.Source: §4.1 · p. 6, §4 · p. 6

4.2 Ablation study: conversations from procedures

• ≈88% (192/217) of graph-generated conversations were valid, versus ≈68% (34/50) from direct procedure prompting.Directly generated conversations took more time and effort to annotate; the simplified pipeline also does not ensure that all branching paths are explored.Source: §4.2 · p. 7

4.3 Evaluation of LLM AI agents

• All models exceeded 85% correctness on both correct API and correct API parameters; API recall exceeded 90% for all except Llama3.1-8b-I.GPT-4 tended to call APIs unnecessarily; GPT models led on correct replies, a result the authors note may be biased by GPT-4 test generation.Source: §4.3 · p. 8
• All evaluated models had very low conversation correctness, despite higher performance on individual API-related dimensions.For test correctness, GPT-4o, Claude3-s, and Mistral-NeMo-Instruct performed best, while GPT-4 and Llama3.1-8b-Instruct ranked among the lowest.Source: §4.3 · p. 8

4.4 Fully automated tests: auto-ALMITA

• 0.98 correlation: auto-ALMITA and ALMITA ranked LLM agents in the same order on test correctness.The automated dataset retained more data and diversity, with some quality reduction.Source: §4.4 · p. 8, Figure 5 · p. 8

5 Conclusions

• The proposed pipeline uses intermediate graph structures to limit hallucinations and improve diversity in tests generated from procedures.ALMITA supports multifaceted evaluation of reply accuracy, API-call correctness, and overall conversation integrity.Source: §5 · p. 8
• ALMITA can help researchers evaluate agent capabilities across several dimensions, including reply accuracy, API-call correctness, and conversation integrity.The authors identify significant limitations in current LLMs, particularly maintaining correct conversations throughout a user interaction.Source: §5 · p. 8
• The automated pipeline can generate harder dataset versions and adapt scenarios for customer support and other domains.The authors describe the scenarios as more complex and realistic, for ongoing agent research and development.Source: §5 · pp. 8–9

6 Limitations

• Test diversity was not measured quantitatively; human review involved few annotations and annotators, generation used only GPT-4, and each evaluated model received one prompt.The authors also note that more advanced AI agents could be considered.Source: §6 · p. 9
• Some metrics may be too strict because conversations can admit acceptable alternative replies or actions beyond the annotated choice.The authors identify severity-sensitive and more semantic metrics that allow path variation as future work.Source: §6 · p. 9

A.1 Intent generation

• Intent generation prompts request diverse customer-support problems across client industries, with each issue described and given a short name in JSON.The prompt asks for the client, issue description, and issue name.Source: §A.1 · p. 11

A.2 Procedure generation

• Procedure generation asks for a specific, actionable process, allows branching, and excludes ambiguous or unresolved conditionals.Steps should be precise and granular, limited to addressing the reported customer issue.Source: §A.2 · p. 11

A.3 API extraction

• API extraction requests all APIs used by a procedure, with specific methods, typed parameters, and no password-requesting calls.The prompt requires JSON output and an empty result when the procedure contains no agent action.Source: §A.3 · p. 11

A.4 Flowgraph generation

• Flowgraph generation takes a procedure and relevant APIs as input and asks for a flowchart containing all procedural details and appropriate API calls.The prompt emphasizes that the agent will not see the procedure itself.Source: §A.4 · pp. 11–12
• The flowchart format represents nodes and directed edges with identifiers, node types, and descriptions for actions or transitions.Allowed node types include start_message, message, api, and end_message.Source: §A.4 · p. 11
• Construction rules require one start_message root, customer replies on message-node outgoing edges, API outputs on API-node outgoing edges, and terminal end_message leaves.End nodes cannot have outgoing edges, and edges must not return to the start node.Source: §A.4 · p. 11
• The example flowchart encodes order lookup, not-found retries, and separate cancellation or refund branches using API calls and outcome messages.The example supplies paths for both email and phone lookups and for each resolution.Source: §A.4 · p. 12

A.5 Conversation graph generation

• The prompt presents graph nodes and edges in a structured format, with node descriptions and edge descriptions available for transitions.The listed format uses node and edge identifiers and parent-child node IDs.Source: §A.5 · p. 12
• The conversion represents assistant messages, user messages, and API calls as distinct node types in the conversation graph.The prompt lists assistant, user, and api as the node types.Source: §A.5 · p. 12
• Graph rules require assistant nodes to be followed by user nodes, user nodes by assistant or API nodes, and leaf nodes to be assistant nodes.API nodes may be followed by API or assistant nodes.Source: §A.5 · p. 12
• The example converts an order flowchart into alternating user and assistant turns, with API calls and outcome-conditioned branches for lookup, cancellation, and refund.The example includes a not-found response and a user-provided replacement order ID.Source: §A.5 · p. 12

A.6 Conversations generation

• Conversation graphs encode assistant, user, and API nodes, while edges indicate message order and can attach API outputs to transitions from API nodes.The prompt describes edge labels as API outputs when the preceding node is an API node.Source: §A.6 · p. 12
• The LLM receives a graph path and the relevant APIs as inputs for generating a conversation along the specified route.The example path is [N1, N2, N3, N4, N5, N7].Source: §A.6 · pp. 12–13
• The example conversation follows an order-support path from the user's missing-order report through order lookup and an assistant response.The sample includes an order ID, an API call, and its output.Source: §A.6 · p. 13
• Generation rules require API messages to be followed by API outputs, assistant messages by user messages, and API outputs by assistant messages.User messages are followed by either an assistant or API message, and API outputs must match the API definition's JSON format.Source: §A.6 · p. 13

A.7 Conversations from procedures

• The prompt asks for conversations that could be solved using the supplied procedure and APIs.The example procedure concerns a customer who did not receive an order.Source: §A.7 · pp. 13–14
• The example procedure asks the agent to greet the customer, confirm the issue, collect an order ID and associated contact detail, and retrieve order information.The retrieval step branches according to whether the customer provides an email address or phone number.Source: §A.7 · p. 13
• If the order is not found, the procedure calls for informing the customer, requesting corrected details, and repeating information collection.If the order is found, the agent asks whether the customer wants cancellation or a refund and uses the corresponding API.Source: §A.7 · p. 13
• The example conversation begins with an assistant greeting and customer report, then shows an order-ID request, lookup API call, and API output.The prompt requires API messages to be followed by API outputs and assistant messages by user messages.Source: §A.7 · p. 13
• The prompt's sequencing rules specify the next role after API, assistant, user, and API-output messages.The first message should be the user's explanation of the problem, even if that node is absent from the graph.Source: §A.7 · p. 14

• The agent must answer only about the supplied procedure and must not invent data or instructions beyond it.Source: §A.8 · p. 14
• Depending on the procedure, the agent should reply directly or call an available action.Source: §A.8 · p. 14
• When required information is missing from context, the agent must ask for it and cannot suggest the action.Source: §A.8 · p. 14
• The agent must follow procedure directives before suggesting an action and ask for confirmation when the procedure requires it.Source: §A.8 · p. 14
• The response must be a JSON object, with a function name and parameters specified for calls.Source: §A.8 · p. 14

• Supplementary Table 1 reports results for six LLM agents across every evaluation metric from Section 4.3.Source: B auto-ALMITA: Detailed evaluation · p. 14
• All results in Supplementary Table 1 are percentages.Source: Table 1 · p. 15
• For each LLM, the table marks its highest value in bold.Source: Table 1 · p. 15