PaperlayerView exact revision on arXiv

Source-linked AI summary

MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback

Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, Heng Ji

arXiv:2309.10691v3cs.CLcs.AIcs.LG

TL;DR

Existing LLM evaluations often emphasize single-turn exchanges and underrepresent tool use and user feedback, despite the prevalence of multi-turn interactions. MINT addresses this gap with a reproducible multi-turn benchmark using executable Python tools, GPT-4-simulated feedback, and compact repurposed datasets. Across evaluated models, tools and feedback generally improve performance, while single-turn strength does not ensure multi-turn strength and SIFT or RLHF generally hurt multi-turn capabilities. MINT is intended to support progress tracking and research on multi-turn task solving and feedback provision, particularly where human evaluation is less accessible.

  • Problem

    Current LLM evaluations predominantly focus on single-turn input-output exchanges and often overlook user-provided natural-language feedback, despite the importance of multi-turn interaction for real-world use.

  • Method

    MINT evaluates multi-turn task solving with executable Python tools, GPT-4-simulated natural-language feedback, and compact subsets repurposed from established reasoning, coding, and decision-making datasets.

  • Results

    LLMs generally benefit from tools and language feedback, while better single-turn performance does not guarantee better multi-turn performance and SIFT or RLHF generally hurt multi-turn capabilities.

  • Takeaways & Limitations

    MINT can help track progress and incentivize research on improving multi-turn task-solving and feedback-providing capabilities, especially where human evaluation is less accessible.

  • Takeaways & Limitations

    Some evaluated LLMs failed to produce parsable output as instructed, and potential undesired training-data artifacts hurt one model’s performance.

Abstract

from arXiv · show

To solve complex tasks, large language models (LLMs) often require multiple rounds of interactions with the user, sometimes assisted by external tools. However, current evaluation protocols often emphasize benchmark performance with single-turn exchanges, neglecting the nuanced interactions among the user, LLMs, and external tools, while also underestimating the importance of natural language feedback from users. These oversights contribute to discrepancies between research benchmark evaluations and real-world use cases. We introduce MINT, a benchmark that evaluates LLMs' ability to solve tasks with multi-turn interactions by (1) using tools and (2) leveraging natural language feedback. To ensure reproducibility, we provide an evaluation framework where LLMs can access tools by executing Python code and receive users' natural language feedback simulated by GPT-4. We repurpose a diverse set of established evaluation datasets focusing on reasoning, coding, and decision-making and carefully curate them into a compact subset for efficient evaluation. Our analysis of 20 open- and closed-source LLMs offers intriguing findings. (a) LLMs generally benefit from tools and language feedback, with performance gains (absolute, same below) of 1-8% for each turn of tool use and 2-17% with natural language feedback. (b) Better single-turn performance does not guarantee better multi-turn performance. (c) Surprisingly, on the LLMs evaluated, supervised instruction-finetuning (SIFT) and reinforcement learning from human feedback (RLHF) generally hurt multi-turn capabilities. We expect MINT can help measure progress and incentivize research in improving LLMs' capabilities in multi-turn interactions, especially for open-source communities where multi-turn human evaluation can be less accessible compared to commercial LLMs with a larger user base.

1 INTRODUCTION

MINT evaluates LLM task solving in multi-turn interactions involving tools and natural-language feedback, addressing evaluation protocols centered on single-turn exchanges. Across 20 models, the benchmark reports broad benefits from interaction but also reveals mismatches between single- and multi-turn performance and harms associated with common alignment training.

  • Benchmark scope: MINT benchmarks two capabilities: tool-augmented task solving and leveraging natural-language feedback in multi-turn interactions.LLMs generate and execute Python programs for tools, while GPT-4 simulates user feedback for reproducible evaluation.
  • Evaluation: 20 LLMs are evaluated across multi-turn tool use and GPT-4-provided language feedback.The evaluation includes 4 closed-source and 16 open-source models.
  • Main findings: 1–8% gains occur for each additional tool-use turn, while natural-language feedback yields 2–17% absolute gains across models.These are reported as absolute performance improvements.
  • Main findings: Better single-turn performance does not necessarily predict better multi-turn performance.Claude-1 benefits more from interaction than Claude-2 despite Claude-2 outperforming it in single-turn evaluation.
  • Main findings: SIFT and RLHF generally reduce multi-turn performance on evaluated models, although Vicuna-7B and Lemur-70b-chat-v1 are exceptions for SIFT.For example, SIFT reduces Codellama-34B performance by 11.1% and 15.4% with feedback, while RLHF reduces LLaMA-2-70B performance by 8.5% and 8.7%.
  • Additional findings: MINT also evaluates feedback-providing ability and finds it can be orthogonal to task-solving ability.CodeLLaMA-34B-Instruct improves stronger GPT-3.5 despite performing worst in task solving; GPT-4-simulated feedback is reported as helpful as human-written feedback.

MINT

MINT creates a reproducible framework for evaluating LLMs as they interact with users and tools across multiple turns. It repurposes diverse datasets, filters for interaction-dependent tasks, and uses stratified sampling to keep evaluation compact.

  • Interaction framework: Each turn lets the LLM either execute generated Python code through a tool interface or propose a solution.Prompts track remaining steps and solution opportunities to prevent endless tool use.
  • Interaction framework: MINT models a lazy user who provides an initial instruction and minimal guidance while the LLM solves a task.The LLM receives two solution attempts and a limit on interaction turns.
  • Interaction framework: Language-feedback evaluation uses a patient-teacher user model whose useful suggestions are generated by GPT-4 for reproducibility and scalability.Human feedback is costly and can follow inconsistent standards.
  • Dataset construction: Multi-turn evaluation is computationally costly because iterative inference multiplies the number of required model runs.Evaluating 7,405 HotpotQA examples for five turns requires at least 37K inference runs.
  • Dataset construction: MINT covers reasoning, code generation, and decision-making using eight established datasets.The reasoning tasks include GSM8K, MATH, TheoremQA, HotpotQA, and MMLU; code tasks include HumanEval and MBPP; decision-making uses ALFWorld.
  • Dataset construction: The dataset is filtered for challenging instances requiring multi-turn interaction and reduced to 586 stratified examples.Stratified sampling preserves the original correct-to-incorrect ratio while balancing evaluation difficulty.

3 EXPERIMENTS

MINT evaluates multi-turn tool use and natural-language feedback across 20 LLMs, measuring both success rates and improvement with additional interaction. Results show that interaction generally helps, but gains vary by model, training method, and feedback role.

  • Evaluation setup: 20 LLMs—4 closed-source and 16 open-source—are evaluated across model sizes and pretrained, SIFT, and RLHF variants.The evaluation uses Success Rate and interaction-dependent measures to compare multi-turn capabilities.
  • Tool-augmented task-solving: 45.9% versus 39.9% SR5 shows claude-instant-1 surpassing claude-2 as interaction increases to five turns, despite worse single-turn performance.The result illustrates that single-turn rankings do not necessarily predict multi-turn performance.
  • Tool-augmented task-solving: ∆tools and SR5 generally increase with model size for CodeLLaMA and LLaMA-2 across Base, SIFT, and RLHF variants.Vicuna-v1.5 is an exception, potentially because training artifacts hurt task performance.
  • Training effects: −0.7% to −2.6% ∆tools and −2.4% to −8.5% SR5 accompany RLHF on LLaMA-2, although the authors avoid concluding that RLHF generally harms performance.SIFT on some multi-turn data shows gains, including Vicuna-v1.5 (7B) and Lemur-v1 (70B).
  • Natural-language feedback: +1.7–+17.2% feedback gains for open-source models and +6.5–+15.2% for closed-source models show broadly useful language feedback, despite an absolute success-rate gap.Feedback-provider quality is orthogonal to task-solving quality: CodeLLaMA-34B-Instruct improves GPT-3.5 despite poor task-solving performance.
  • Failure patterns: MINT also exposes formatting failures and training-data artifacts, including escaped underscores in Vicuna outputs and persistent [PYTHON] tags from CodeLLaMA-Instruct.These behaviors can cause syntax errors, reduce performance, or prevent compliance with requested output formats.

4 RELATED WORK

Prior interaction research examines human or environmental interaction, often within a single task or dimension. MINT addresses this narrower coverage by evaluating multi-turn interaction with both tools and natural-language feedback.

  • Interact with users: Research on LLM interaction with users spans applications such as sentence editing, semantic parsing, and code generation.
  • Interact with tools: Research on tool interaction connects LLMs to external APIs and tools to support more accurate and reliable outputs.
  • Existing studies evaluate specific interaction settings, including task completion, code generation, collaboration, tool manipulation, and web navigation.
  • Existing work often focuses on interacting with either humans or the environment rather than evaluating both dimensions together.

5 CONCLUSION

MINT is an evaluation benchmark for LLM task-solving in multi-turn interaction using tools and GPT-4-simulated natural-language feedback. The authors position it as a resource for tracking progress and encouraging research on multi-turn capabilities.

  • MINT evaluates LLM task-solving ability in multi-turn interaction through tool use and natural-language feedback simulated with GPT-4.
  • The benchmark is intended to help track progress and incentivize research on improving LLM multi-turn task-solving capabilities.

A LIMITATIONS AND FUTURE WORK

The authors identify limitations in GPT-4-simulated feedback, outcome-focused metrics, and benchmark coverage. They also leave direct use of language feedback for model training and improvement to future work.

  • GPT-4 feedback may not represent every real-human response, especially for tasks requiring nuanced judgments of human values.
  • MINT’s metrics may not fully evaluate interaction quality beyond outcomes, such as penalizing repetitive guessing for higher scores.
  • The benchmark does not cover the full range of tools and real-world use cases, including web browsing and operating systems.
  • Directly using language feedback for model training and improvement remains future work.

B DETAILS OF HUMAN EVALUATION

The human evaluation compares human-annotated and GPT-4-generated feedback from sampled interaction trajectories. Annotators judge both feedback helpfulness and human-likeness.

  • First-stage annotation: Two annotators provided language feedback for sampled trajectories from eight evaluated LLMs, yielding 113 trajectories after excluding first-turn successes.
  • Second-stage comparison: Two different annotators compared human-annotated and GPT-4-generated feedback.
  • Second-stage comparison: The comparison assessed whether feedback was helpful for task success and whether it resembled literal human usage.

C ABLATION STUDY

The study ablates feedback design choices to understand how informativeness and frequency affect language-model performance.

  • The ablation examines how feedback informativeness and frequency affect model performance.
  • Feedback informativeness and frequency are treated as separate experimental factors.
  • Detailed implementation and results are reported in §F.4.2 and Table A.6.

C.1.1 INFORMATIVENESS

MINT varies whether feedback uses ground-truth information, textual versus binary content, and dense versus sparse timing to assess their effects across tasks.

  • Informativeness: Feedback informativeness depends on ground-truth conditioning and whether feedback is textual or binary.
  • Conditioned on Ground-truth Information: Ground-truth conditioning improves feedback for reasoning and code generation but reduces decision-making performance by −8.95% versus no feedback.The authors attribute this discrepancy to decision-making tasks involving action trajectories rather than definitive solutions.
  • Provide Binary Feedback: Binary feedback hurts performance relative to textual feedback but still improves performance over no feedback, especially for decision-making by +5.97.
  • Feedback Frequency: Sparse feedback reduces performance by −11.02 on average compared with dense feedback, while remaining beneficial on all tasks versus no feedback.Under sparse feedback, MINT is equivalent to the Reflexion feedback setting.

D DATASET FILTERING AND DOWN-SAMPLING

MINT constructs an efficient multi-turn evaluation set by collecting examples from eight datasets, filtering for interaction suitability, and applying stratified down-sampling.

  • Dataset Curation: The curation begins by collecting test-set data from eight established datasets.
  • Dataset Curation: Dataset prompts are formatted with a task description and solution-range requirement, with HotpotQA limited to its first 500 instances.Solution ranges specify expected output types for GSM8K and TheoremQA and remain empty for other datasets.
  • Evaluation Setup: The evaluation also tracks invalid actions caused by models failing to produce the instructed <execute> or <solution> tags.
  • Dataset Filtering: The filtering process keeps instances that require multi-turn interaction and cleans multiple-choice tasks with fewer than four options.
  • Stratified Sub-sampling: Stratified sub-sampling balances dataset and difficulty categories while reducing evaluation size.Instances are stratified by dataset and whether gpt-3.5-turbo-0613 completed them, with dataset-specific retention proportions.

E.1 VICUNA-V1.5 ESCAPE UNDERSCORE ISSUE

The Vicuna-v1.5 analysis identifies escaped underscores as a recurring coding failure that causes syntax errors and persists across multiple repair attempts despite feedback.

  • Observed Artifact: About 15% of 94,145 ShareGPT conversations contain a backslash-underscore pattern associated with escaped underscores.
  • Observed Artifact: Vicuna and SIFT Lemur-v1 particularly exhibit escaped underscores in generated code, which can cause syntax errors during task-solving.The authors believe the ShareGPT artifact may contribute to this generation pattern.
  • Interaction Trace: The coding example uses an interactive Python environment, with the model alternating between execution and solution proposals under a fixed interaction budget.
  • Feedback and Repair: Expert feedback identifies both the escaped function-name backslashes and incorrect nested-list logic in the initial proposal.
  • Feedback and Repair: Repeated feedback corrects the intended any-based logic, but the escaped underscore continues to trigger SyntaxError across later attempts.
  • Final Repair: The final solution removes the backslashes and checks whether each element appears in any sublist of the second list.

E.2 CODELLAMA-INSTRUCT PRODUCES [PYTHON] TAG

CodeLLaMA-34B-Instruct repeatedly outputs [PYTHON] tags instead of the required interaction format on coding tasks, despite explicit instructions and examples. The resulting format mismatch prevents code execution and contributes to incorrect solutions.

  • CodeLLaMA-34B-Instruct always wraps coding solutions in [PYTHON] tags despite explicit instructions to use <execute> or <solution> tags.The model does so even when prompted with an in-context example showing the required format.
  • The [PYTHON] tag is associated with code-generation problems but not other task types, and appears most often in the 34B model rather than 7B models.
  • The authors suspect instruction-tuning artifacts caused the [PYTHON] association and that larger models may memorize such patterns more often.
  • Format errors trigger observations instructing the assistant to use <execute> for code execution or <solution> for answers.
  • Expert feedback identifies two coupled errors: incorrect execution formatting and checking balanced rather than nested brackets.
  • The trajectory shows the assistant apologizing, switching to <execute>, and revising its solution after feedback.

F PROMPTS AND EXAMPLES

This appendix describes MINT’s prompts, interaction protocol, simulated feedback, examples, and task adaptations. The examples show how tools and feedback support iterative correction across coding, reasoning, and decision-making tasks.

  • F PROMPTS AND EXAMPLES: MINT uses carefully designed prompts and in-context examples to elicit interaction behavior for each task type.
  • F.1 TOOL DESCRIPTIONS FOR EACH TASK TYPE: Code-generation tasks use the Python interpreter, while other task types provide tools such as Wikipedia search or environment functions.
  • F.3 EXAMPLE TASK TRAJECTORY: A reasoning trajectory illustrates GPT-3.5-Turbo solving a MATH task with GPT-4 acting as a simulated human advisor.
  • F.3 EXAMPLE TASK TRAJECTORY: After feedback identifies an incorrect Vieta formula, the assistant executes a corrected calculation and submits 5.0.
  • F.4 PROMPT TEMPLATES: Prompt templates let each task specify its instruction, tool descriptions, interaction limits, and separate execution or solution actions.
  • F.4.1 EVALUATE LLM: At each turn, the model may execute Python code or propose a formatted solution, with limits on total steps and proposal attempts.
  • F.4.2 SIMULATE LANGUAGE FEEDBACK: Feedback generation substitutes an agent trajectory and, when available, the ground-truth solution into a concise teacher-like feedback prompt.
  • F.5 IN-CONTEXT EXAMPLE: MINT uses one in-context example per task, with versions that either retain or remove expert feedback.
Loading 2309.10691v3…