PaperlayerView exact revision on arXiv

Source-linked AI summary

Plan Explanations as Model Reconciliation: Moving Beyond Explanation as Soliloquy

Tathagata Chakraborti, Sarath Sreedharan, Yu Zhang, Subbarao Kambhampati

arXiv:1701.08317v5cs.AI

TL;DR

Plan explanations based on an AI system’s own model may not address humans’ differing domain and task models. This paper treats explanation as changing the human’s model so the robot’s plan is optimal under it, and presents algorithms with a preliminary efficiency evaluation.

  • Problem

    Prior plan explanations often justify a plan using only the AI system’s model, although humans in many interactions may have significantly different domain and task models.

    Source: §1 · p. 1

  • Method

    The paper formulates explanations as a model reconciliation problem that minimally changes the human’s model to make the robot’s plan optimal in the changed model.

    Source: §1 · p. 1

  • Results

    The paper presents A*-search, approximations, and heuristics for computing explanations and reports a preliminary efficiency evaluation on randomly generated problems in benchmark planning domains.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper distinguishes changing the human’s model through explanations from changing the robot’s plan to fit the human’s model.

    Source: A Multi-Model Planning (MMP) Setting · p. 3

    The paper assumes the human’s model is available to the robot and in PDDL format, like the robot’s model.

    Source: §1 · p. 1

Abstract

from arXiv · show

When AI systems interact with humans in the loop, they are often called on to provide explanations for their plans and behavior. Past work on plan explanations primarily involved the AI system explaining the correctness of its plan and the rationale for its decision in terms of its own model. Such soliloquy is wholly inadequate in most realistic scenarios where the humans have domain and task models that differ significantly from that used by the AI system. We posit that the explanations are best studied in light of these differing models. In particular, we show how explanation can be seen as a "model reconciliation problem" (MRP), where the AI system in effect suggests changes to the human's model, so as to make its plan be optimal with respect to that changed human model. We will study the properties of such explanations, present algorithms for automatically computing them, and evaluate the performance of the algorithms.

1 Introduction

Plan explanations based only on an AI system’s own model can miss the human’s differing domain and task model. The paper frames explanations as minimal human-model changes that make the robot’s plan optimal, and illustrates this with a Fetch robot example.

1 Introduction

  • Earlier plan explanations often justified decisions using the planner’s own model, an assumption the paper calls unrealistic when humans have significantly different domain and task models.The authors contrast this with expert debuggers who know the agent’s internals.Source: §1 · p. 1
  • The paper frames explanation as a model reconciliation problem: minimally change the human’s model so the robot’s plan becomes optimal in that changed model.The paper assumes the human model is available in PDDL, like the robot’s, to focus on explanation generation.Source: §1 · p. 1
  • The paper identifies completeness, conciseness, monotonicity, and computability as desired explanation properties, then develops explanation formulations, A*-search, approximations, and heuristics.It reports a preliminary efficiency evaluation on randomly generated problems in several benchmark planning domains.Source: §1 · p. 2

A Motivating Example

  • A Fetch robot’s move requires tucked arms and a lowered torso, requirements absent from the human’s simpler model; adding a hand-tucked precondition to move is one possible correction.The paper says this correction is necessary and sufficient to make the robot’s plan optimal in the resulting domain.Source: A Motivating Example · p. 2
  • The robot’s example plan is pick-up, tuck, move, then put-down, illustrating how the explanation changes the human model rather than the robot’s plan.The initial and goal states are identical in the human and robot models.Source: A Motivating Example · p. 2

2 Related Work

The related work contrasts model reconciliation with approaches that explain plans from the planner’s own perspective or ask humans to adapt to planners. It also situates the paper’s optimality criterion alongside prior explanation and model-change work.

2 Related Work

  • Psychology accounts describe explanations as selecting and constraining beliefs, while contrastiveness motivates requiring an updated model in which the robot’s plan is optimal.The optimality criterion distinguishes this work from model-change algorithms focused on plan feasibility or domain correctness [G¨obelbecker et al., 2010; Herzig et al., 2014; Bryce et al., 2016].Source: §2 · p. 2
  • Prior single-model plan explanations include preferred explanations based on less costly plans, whereas this paper considers optimality relative to an updated human model.The passage also notes that explicable-planning work modifies the robot’s plan to fit the human’s model [Zhang et al., 2017; Kulkarni et al., 2016].Source: §2 · p. 2
  • Earlier mixed-initiative planners generally had humans enter the planner’s world rather than having the planner account for the human’s model.Source: §2 · p. 2

3 The Multi-Model Planning (MMP) Setting

This section frames plan explanation in multi-model planning as reconciling the planner’s and human’s models, formalizing model updates and distinguishing minimally complete from minimally monotonic explanations. It also considers approximating MCE with proxies for plan optimality to improve computability at the cost of completeness, and identifies MME as the remainder of a path toward MH.

A Classical Planning Problem

  • A plan can be optimal in the robot’s model yet suboptimal in the human’s, so optimality alone does not make it explicable to that human.A classical planning model specifies a domain, initial state, and goal; plans are action sequences that achieve the goal, with cost determining optimality.Source: A Classical Planning Problem · pp. 2–3

A Multi-Model Planning (MMP) Setting

  • The planner can either alter its plan to fit the human’s model or explain it by updating the human’s model; this paper focuses on model reconciliation.The alternative of changing the plan may sacrifice optimality in the robot’s model.Source: A Multi-Model Planning (MMP) Setting · p. 3
  • Model-space search represents planning problems as sets of facts and applies single-fact edits consistent with the planner’s model to transform one model into another.The edits can be assigned different costs to represent the importance of model corrections.Source: A Multi-Model Planning (MMP) Setting · p. 3

A Multi-Model Explanation

  • Explanations must balance completeness, conciseness, monotonicity, and computability, requirements that can conflict.Completeness requires the plan to be optimal in the updated human model; conciseness limits the information disclosed, while monotonicity accounts for remaining model differences.Source: A Multi-Model Explanation · pp. 3–4
  • A minimally complete explanation is the shortest model update that makes the explained plan optimal, while patch explanations trade completeness against conciseness.Plan patch explanations focus on differences relevant to plan actions but can omit causes of suboptimality; model patch explanations give all differences and can be large. The paper uses A* model-space search for MCEs, with a successor strategy shown to yield an admissible heuristic.Source: A Multi-Model Explanation · p. 4

A Minimally Monotonic Explanation (MME)

Approximate Solution for MCEs

  • Approximate MCEs replace an optimality test with simple proxies, improving computability but losing the completeness guarantee.Computing MCEs and MMEs can require searching the full space of model differences, with plan-optimality checking the main bottleneck.Source: Approximate Solution for MCEs · p. 6
  • Criterion (3) requires every action in π∗ R to contribute a causal link and is necessary for π∗ R's optimality.Criterion (2) requires the plan to become better in the new model or the human's expected plan to be disproved.Source: Approximate Solution for MCEs · p. 6

4 Empirical Evaluations

The evaluation compares explanation generation across three planning domains and finds that minimally complete explanations can be much smaller than the full model difference, while exact search can become costly as model differences grow.

4 Empirical Evaluations

  • Experiments used BlocksWorld, Logistics, and Rover, with human models generated by randomly removing action preconditions and effects.The removals and model-space search were performed in the lifted representation.Source: §4 · p. 6
  • ∼20%: in certain domains, MCEs and MMEs are this size relative to the total model difference; the reported ordering is approx. MCE ≤ exact MCE < MME << MPE.The table compares MCEs and MMEs; the ordering also includes approximate MCEs and MPEs.Source: Table 2 · p. 7, §4 · p. 7
  • Exact MCE search time increases exponentially as the number of changes in the BlocksWorld human model grows.The authors note that even a modest number of model differences can make exact search problematic.Source: Table 3 · p. 7, §4 · p. 7
  • For random BlocksWorld problems with 10 faults, Proposition 3 reduces the number of nodes searched for MMEs relative to the total possible models.Table 4 evaluates this pruning against the full model space.Source: Table 4 · p. 7, §4 · p. 7
  • PPEs are easier to compute, but lack completeness and monotonicity and can span the entire model difference, including changes unnecessary to support plan optimality.In Logistics, MME computation can take less time than corresponding MCE computation when a single model change makes the plan non-optimal and ends the search.Source: §4 · p. 7

5 Conclusions and Future Work

The paper concludes that plan explanations should reconcile differences between human and AI models. It identifies future extensions for differing model representations, uncertain human models, and explanation constraints.

5 Conclusions and Future Work

  • Plan explanations in human-AI interaction should account for differences between the human's and the AI system's models and reconcile relevant differences.The paper frames explanations as updates to the human model that support the robot's plan.Source: §5 · p. 7
  • Future work includes human models with different forms or abstraction levels and hypothesis generation that accounts for differing human cognitive abilities.The paper mentions ϵ-optimality or top-K plans for hypothesis generation.Source: §5 · p. 7
  • When the ground truth is unknown, explanation may require distributions over relevant models and iterative refinement through dialogue.Source: §5 · p. 7
  • The approach requires explanations to remain compatible with the planner's model; relaxing that condition could permit knowingly false explanations and raise ethical concerns.Source: §5 · p. 7

Related papers

Loading 1701.08317v5…