PaperlayerView exact revision on arXiv

Source-linked AI summary

CLARA: Classifying and Disambiguating User Commands for Reliable Interactive Robotic Agents

Jeongeun Park, Seungwon Lim, Joonhyung Lee, Sangbeom Park, Minsuk Chang, Youngjae Yu, Sungjoon Choi

arXiv:2306.10376v6cs.ROcs.AI

TL;DR

Interactive robots need to distinguish clear commands from ambiguous or infeasible ones, whose interpretation depends on the situation. CLARA estimates uncertainty, classifies uncertain commands with situational context, and asks users questions about ambiguous commands; on SaGC, its InstructGPT system improves whole-system classification accuracy over a previous method by 0.158.

  • Problem

    Interactive robots must determine whether a user command is clear, ambiguous, or infeasible in the robot’s situation.

    Source: §I · p. 1

  • Method

    CLARA estimates uncertainty, uses situationally aware LLM prompts to classify uncertain commands as ambiguous or infeasible, and generates questions for users about ambiguous commands.

    Source: §I · p. 1, C. Classification and Disambiguation · p. 3

  • Results

    Across three LLMs, CLARA outperforms the baselines in classification accuracy; with InstructGPT it exceeds the previous method by 0.158.

    Source: 2) Results · p. 5

  • Takeaways & Limitations

    The authors report that CLARA can quantify LLM uncertainty and appropriately classify user commands, with evaluation on a dataset, a pick-and-place simulation, and a real-world demonstration.

    Source: §VI · pp. 7–8

    The method focuses on uncertainty in language commands and can misclassify planning uncertainty in partially observable environments as ambiguity.

    Source: §V · p. 7

Abstract

from arXiv · show

In this paper, we focus on inferring whether the given user command is clear, ambiguous, or infeasible in the context of interactive robotic agents utilizing large language models (LLMs). To tackle this problem, we first present an uncertainty estimation method for LLMs to classify whether the command is certain (i.e., clear) or not (i.e., ambiguous or infeasible). Once the command is classified as uncertain, we further distinguish it between ambiguous or infeasible commands leveraging LLMs with situational aware context in a zero-shot manner. For ambiguous commands, we disambiguate the command by interacting with users via question generation with LLMs. We believe that proper recognition of the given commands could lead to a decrease in malfunction and undesired actions of the robot, enhancing the reliability of interactive robot agents. We present a dataset for robotic situational awareness, consisting pair of high-level commands, scene descriptions, and labels of command type (i.e., clear, ambiguous, or infeasible). We validate the proposed method on the collected dataset, pick-and-place tabletop simulation. Finally, we demonstrate the proposed approach in real-world human-robot interaction experiments, i.e., handover scenarios.

I. INTRODUCTION

CLARA addresses the challenge of deciding whether a robot’s user command is clear, ambiguous, or infeasible, then asks users for more information when an ambiguous command needs clarification. It frames command interpretation as part of interactive-system reliability and emphasizes that meaning depends on the robot and situation.

I. INTRODUCTION

  • The paper presents proper command interpretation as an essential component of reliability in interactive robotic systems.Source: §I · p. 1
  • The same request can be clear for one robot, infeasible for another, and ambiguous for a third because robot capabilities and context differ.For “he looks tired, can you help him?”, the cooking robot may make coffee, the cleaning robot cannot help, and the massage robot may ask a follow-up question.Source: §I · p. 1
  • CLARA combines uncertainty estimation, situation-aware classification of uncertain commands, and question generation to disambiguate ambiguous commands.The paper also presents SaGC, a benchmark pairing high-level commands with environments and uncertainty-type labels.Source: §I · pp. 1–2

II. RELATED WORK

  • Unlike Inner Monologue [4], CLARA also considers infeasible commands, while prior uncertainty methods requiring token probabilities [12], [13], [14], [15] cannot be used with LLMs lacking access to those probabilities.Source: §II · p. 2
  • Earlier disambiguation approaches can rely on fixed attributes, template-based questions, or an additional parser for free-form inputs.The paper motivates natural-language generation as a flexible alternative for varied scenarios [2].Source: §II · p. 2

III. PROPOSED METHOD

CLARA first estimates whether a command is uncertain, then uses situational awareness to distinguish ambiguity from infeasibility and routes ambiguous commands to disambiguation. Its uncertainty estimate uses variation in LLM outputs across sampled contexts, with a threshold calibrated on clear examples.

III. PROPOSED METHOD

  • The method separates clear from uncertain commands, classifies uncertain commands as ambiguous or infeasible, and disambiguates ambiguous commands through user questions.Source: §III · p. 2, Figure 1 · p. 3

A. Problem Formulation

  • The system takes a high-level goal, scene objects, and few-shot contexts, and produces either low-level skills with uncertainty or a textual uncertainty explanation.Source: A. Problem Formulation · p. 2

B. Uncertainty Estimation of LLM

  • Context sampling varies few-shot examples and scene-description order; inconsistent generated outputs across contexts indicate greater predictive uncertainty.The approach does not require model weights or token probabilities; the experiments use 15 samples.Source: B. Uncertainty Estimation of LLM · pp. 2–3
  • Predictive uncertainty is compared with a threshold set at the 80th percentile of the uncertainty distribution for clear samples.Uncertainty-aware prompting asks the LLM to consider goal ambiguity when generating outputs.Source: B. Uncertainty Estimation of LLM · p. 3, Figure 1 · p. 3
  • In the block-picking example, different sampled contexts produce different selected blocks, illustrating output variation used to estimate uncertainty.The example reports total uncertainty of 2.1.Source: B. Uncertainty Estimation of LLM · p. 3

C. Classification and Disambiguation

For uncertain commands, CLARA checks feasibility against the robot’s capabilities and generates explanations and questions for commands judged feasible but ambiguous. After the user responds, the system repeats uncertainty estimation with the added information.

C. Classification and Disambiguation

  • A situationally informed feasibility check asks whether the robot can perform the task, then uses a heuristic parser to classify a “yes” response as ambiguous and other cases as infeasible.The classification and disambiguation stages are called zero-shot because their prompts contain no classification or disambiguation examples, even though action generation uses few-shot prompts.Source: C. Classification and Disambiguation · p. 3
  • For a feasible but ambiguous command, the LLM generates a reason for the uncertainty and asks the user a question to obtain missing information.Source: C. Classification and Disambiguation · pp. 3–4
  • After the user answers, CLARA returns to uncertainty estimation with extended prompts.Source: C. Classification and Disambiguation · p. 4

IV. EXPERIMENTS

The experiments assess uncertainty estimation, three-way command classification, the role of interaction, and real-world feasibility. They compare uncertainty baselines and alternative classification approaches across LLMs, with SaGC results organized by AUROC, accuracy, and uncertainty-estimation ablations.

IV. EXPERIMENTS

  • The experiments ask whether CLARA improves uncertainty quantification across environments, classifies clear, ambiguous, and infeasible commands, clarifies ambiguity through interaction, and can be deployed in real-world interaction.Source: §IV · p. 4

A. Baselines

  • Uncertainty quantification is compared with entropy, normalized entropy, semantic uncertainty, and lexical similarity; classification and disambiguation are compared with Inner Monologue [4] and CLAM† [22].CLAM† is a zero-shot modification of CLAM [22].Source: A. Baselines · p. 4
  • The SaGC results table reports AUROC for uncertainty quantification, accuracy for classification, and ablations of context sampling and uncertainty-aware prompting.The experiments use LLaMA, ChatGPT (GPT3.5-turbo), and InstructGPT; LLaMA was not tested in the real-world environment because of limited computing power.Source: Table I · p. 4, A. Baselines · p. 4

B. Situational Awareness for Goal Classification in Robotic Tasks

This section evaluates whether CLARA can identify command uncertainty and its type while accounting for robot capabilities and environments, using a dataset built for that purpose.

1) Dataset Formulation:

  • SaGC contains 5,222 command–scene pairs: 1,749 certain, 1,560 ambiguous, and 1,917 infeasible, across 15 scenes and cooking, cleaning, and massaging robots.The examples were generated from crafted goals using LLM prompts and reviewed by four validators.Source: 1) Dataset Formulation · p. 5

2) Results:

  • On AUROC, InstructGPT outperformed the compared baselines with a 0.003 gap, while the method ranked second-best on LLaMA 7B.The method can also work with previous methods without access to token-wise probabilities.Source: 2) Results · p. 5
  • Removing context sampling and uncertainty-aware prompting reduced performance by 0.018 and 0.09.Source: 2) Results · p. 5
  • Compared with Lexical Similarity [17], stochastic prompting produced more diverse generations for uncertain inputs.Source: 2) Results · p. 5
  • 0.158 gap: CLARA with IntructGPT outperformed the previous method and beat baselines across three large language models.Source: 2) Results · p. 5
  • Text-davinci-003 recorded higher classification accuracy, which the authors relate to a stronger correlation with LLM size.Source: 2) Results · p. 5

C. Pick and Place Simulations

The tabletop simulations evaluate uncertainty detection and question-based disambiguation, while real-world demonstrations test classification and interaction in handover scenarios. The reported results cover detection performance, classification accuracy, interaction timing, and a trade-off in requesting user feedback.

C. Pick and Place Simulations

  • The simulation evaluated 108 certain and 60 ambiguous configurations, with the interaction phase limited to one user question.An ambiguous task was defined as lacking the exact pick or place position or object name.Source: C. Pick and Place Simulations · p. 5
  • In tabletop simulations, the IntructGPT uncertainty-estimation method outperformed the compared method by 0.025 AUROC.Source: C. Pick and Place Simulations · p. 5
  • For simulated disambiguation, the proposed method ranked second-best among baselines on success-rate gap and outperformed the previous method on F1.The F1 score measures whether questions are generated only in ambiguous scenarios.Source: C. Pick and Place Simulations · p. 6

D. Real-World Demonstrations

  • The real-world evaluation used six configurations for each of three goal types: clear, ambiguous, and infeasible.It used OWL-VIT for scene description and a grounding translator to map LLM output to feasible actions.Source: D. Real-World Demonstrations · p. 6
  • In real-world tests, IntructGPT AUROC exceeded the compared method by 0.018; classification accuracy had a 0.17 gap on both LLM types, with average success-rate and interaction-timing gaps of 0.05 and 0.23.The proposed method reported the best success-rate increase and appropriate interaction timing compared with baselines.Source: Table VI · p. 6, Table VII · p. 6, D. Real-World Demonstrations · p. 6
  • Requesting user feedback can increase the success-rate gap, but asking too often—even on clear commands—may be undesirable depending on user preference.Source: D. Real-World Demonstrations · p. 7
  • Real-world demonstrations included vague raw inputs with partial information, iterative clarification, and referential ambiguity between two people wearing black shirts.In the referential case, the robot asked the user to specify left or right.Source: D. Real-World Demonstrations · p. 7, Figure 4 · p. 7

V. LIMITATIONS

The method has several practical and scope limitations, while the conclusion reports evaluations in simulation and real-world handover scenarios. The authors observe that it quantifies LLM uncertainty and classifies command types appropriately.

V. LIMITATIONS

  • The method relies on few-shot or zero-shot LLM capability, which the authors say could be improved through fine-tuning.They also note that the method can be applied without additional models or fine-tuning.Source: §V · p. 7
  • Sampling limits speed and computational cost, threshold calibration requires clear samples, and partial observability can make planning uncertainty appear to be command ambiguity.The example given is an unseen object prompting the robot to request user feedback; the authors also call for finer-grained uncertainty types.Source: §V · p. 7
  • The SaGC dataset may be biased because it was formulated via a large language model, although the authors consider the effect less significant.Their stated reason is that the LLM used for construction does not record the best performance.Source: §V · p. 7

VI. CONCLUSION

  • The conclusion describes CLARA as distinguishing clear, ambiguous, and infeasible commands, estimating uncertainty, classifying uncertain goals, and disambiguating ambiguous commands through interaction.The paper also presents a dataset for validating situational awareness and evaluates the method with that dataset and a pick-and-place simulation.Source: §VI · p. 7
  • The authors report that the method quantified LLM uncertainty and appropriately classified user-command types in their evaluation and real-world demonstration.They suggest that fine-tuning with data containing explanations could further develop the classification and interaction module.Source: §VI · p. 8

APPENDIX

The appendix illustrates failure modes in command classification and a toy example of uncertainty estimation, classification, and user-driven disambiguation. Failures include threshold sensitivity, missing or misleading command cues, and errors from other robot modules.

A. Failure Cases

  • A threshold near miss changes the label: uncertainty of 1.22 falls below the 1.24 cooking-robot threshold, so the command is classified as certain..Source: A. Failure Cases · p. 9
  • Known keywords can lead the LLM to treat infeasible commands as feasible, while overestimating the robot’s action set can also cause failure.For example, cooking-related wording can mask infeasibility, and the cleaning robot may be assumed to have an “organize” action.Source: A. Failure Cases · p. 9
  • A vague command with a missing word such as “person” may be classified as infeasible rather than interpreted as an ambiguous request.The example is “give a massage in the kitchen,” which could mean giving a massage to a person there.Source: A. Failure Cases · p. 9
  • When multiple details are missing, the LLM may fail to ask about both, and vision or policy-module malfunctions can cause incorrect uncertainty estimates or overall failure.Source: A. Failure Cases · p. 9

B. Toy Example

  • In the block-picking toy example, three sampled generations yield total uncertainty of 2.1 for “pick a block” in a scene with red, blue, and yellow blocks.The samples vary prompts and observation order.Source: B. Toy Example · p. 10
  • The toy example classifies the command as ambiguous when the generated response says it can pick a block “given more information,” using a heuristic parser keyed to “yes.”Source: B. Toy Example · p. 10
  • For disambiguation, the system supplies a reason and question, then returns to uncertainty estimation with the user’s answer added to the prompt.The example asks for criteria for selecting a block.Source: B. Toy Example · p. 11

C. Demonstration in Mobile Manipulation Task

The mobile-manipulation demonstration sets an uncertainty threshold of 1.5 in ALFWorld and illustrates outcomes for certain, ambiguous, and infeasible commands. Its examples show uncertainty prompting clarification in the ambiguous case.

C. Demonstration in Mobile Manipulation Task

D. SaGC Data collections

The section describes SaGC data generation and validation, then evaluates supervised baselines, vision-language models, pick-and-place interaction, and question informativeness. Its results include a robot-type transfer gap and differing vision-model preferences across simulated and real-world settings.

D. SaGC Data collections

  • SaGC’s dataset was generated by GPT-3.5-turbo from 105 crafted samples, yielding 5,226 command–label pairs; four validators reviewed them, and labels were certain, ambiguous, or infeasible.Approximately 10% of labels changed after validation; the authors noted possible bias from using only GPT-3.5-turbo.Source: D. SaGC Data collections · p. 12, E. SaGC Data Examples · p. 12

E. SaGC Data Examples

  • SaGC examples show clear goals alongside ambiguous requests and infeasible goals that do not fit the robot or scene.The dataset assigns labels 0, 1, and 2 to certain, ambiguous, and infeasible commands, respectively.Source: E. SaGC Data Examples · pp. 12–13

F. Supervised Learning Baselines

  • In robot-type transfer, BERT averaged 0.333 accuracy, a 0.367 gap below the proposed method on text-davinci-003; with more than 500 training samples, BERT could exceed it.The authors note that supervised performance may be vulnerable when robot configuration changes.Source: F. Supervised Learning Baselines · p. 13, Table VIII · p. 13

G. Vision Language Model

H. Pick and Place Simulation

  • The pick-and-place simulation assumes one user question at most, defines ambiguity as omitting the pick or place object, and evaluates question generation with F1 and task success before and after interaction.The section describes F1 as measuring question generation in ambiguous—not clear or infeasible—scenarios.Source: H. Pick and Place Simulation · p. 14, Table X · p. 14, Table IX · p. 14

I. GPT-Info Evaluation Examples

  • GPT-3.5-turbo generated the most informative questions, while zero-shot CLAM†[13] averaged better quality than few-shot IM[4].The authors suggest zero-shot question generation contributes to question quality and variety.Source: §I · p. 14, Table XII · p. 15

J. Prompts

The prompts establish each robot’s role, available actions, and scene before presenting example goals and actions. Additional prompts guide command feasibility, uncertainty reasoning, question generation, and continuation after user feedback.

  • Robot-specific prompts pair an action set and scene description with example goals and corresponding robot actions.Examples cover cooking, cleaning, massage, pick-and-place, and handover tasks.Source: J. Prompts · pp. 15–16
  • Classification prompts ask whether the robot can perform the goal, then elicit a reason for uncertainty and a question for the user.A follow-up prompt uses the user’s answer to continue the task.Source: J. Prompts · p. 16
  • Pick-and-place and handover prompts provide scene objects and people, then demonstrate actions for goals involving block placement or giving an object to a person.The prompts also include a corresponding classification and disambiguation stage.Source: J. Prompts · pp. 16–17

K. Fewshot Prompts

The few-shot prompts supply robot-specific demonstrations of goals and action sequences, including examples of uncertain commands, user questions, and task continuation. They span pick-and-place, SaGC robot tasks, and real-world handover.

  • Pick-and-place demonstrations pair block-placement goals with action sequences, including a stacking example where a question resolves which block to move first.The prompt is described as using an Inner Monologue style.Source: K. Fewshot Prompts · p. 17
  • Cooking, cleaning, and massage demonstrations include clear actions as well as examples where the robot asks for clarification or stops when a task is infeasible.Examples include clarifying what “clean here” refers to and declining an unsupported heating task.Source: K. Fewshot Prompts · pp. 17–18
  • Real-world handover demonstrations show object-and-person-specific actions, a clarification exchange about the recipient, and a goal the robot cannot perform.The prompt identifies the robot as able to pick up and give objects to people.Source: K. Fewshot Prompts · pp. 18–19

L. Real-world Experiments Environment and Demonstrations

The real-world experiments use a tabletop handover setup and participant responses to test clear, ambiguous, and infeasible goals. Additional SaGC results report accuracy on 216 partially feasible goals and illustrate generated outputs and failures.

L. Real-world Experiments Environment and Demonstrations

M. Additional Results on SaGC Dataset

Loading 2306.10376v6…