Explainable Autonomous Robots: A Survey and Perspective

1. Introduction

• Effective communication—not improvements in specific task performance alone—is presented as a barrier to robots replacing human labor.The paper says NLP estimates statement content and meaning but does not constitute communication that makes people feel understood.Source: §1 · p. 1
• The proposed mechanisms are estimating what users think and presenting the robot’s own ideas in an easy-to-understand format.The paper says robots should also help humans and robots understand one another’s internal states.Source: §1 · pp. 1–2
• For a robot acting as a human’s life partner, estimating the other person’s internal state and adapting its actions, behavior, and information presentation are framed as potential routes to trust.The paper distinguishes this from autonomous systems simply operating as instructed to assist human life.Source: §1 · p. 2
• The XAR scenario illustrates the need to explain a decision: after a cold-drink request, the robot heads to the pantry instead of the refrigerator, and unanswered questions can lead to distrust.The paper cautions that explanations must be tailored to the user’s information and trust relationship; excessive detail may not improve reliability.Source: §1 · p. 2
• The paper surveys XARs as a first step toward direct, effective human–robot interaction, defining the problem before reviewing research and future issues.The authors state that a clear definition of autonomous-robot explainability has yet to be established.Source: §1 · p. 3

2. Explainable AI (XAI) and XAR

• The paper frames XAR as distinct from XAI, XAIP, and XRL in problem setting and goals, while surveying those areas to clarify the overlaps.Its survey selects research on autonomous learning agents within XAIP and focuses on model-based reinforcement learning.Source: §2 · p. 3, §2.2 · p. 5, §2.3 · p. 5

2.1. XAI

• XAI presents bases for machine-learning judgments, but the paper says these explanations are mainly for debugging and do not themselves achieve communication between autonomous agents and people.An explanation can fail to convince even when its cited factor is important, if it conflicts with a person’s judgment.Source: §2.1 · p. 4

2.2. Explainable AI Planning (XAIP)

• XAIP studies transparency in system decision-making and planning; human-aware planning can favor plans people predict, but differing human and robot abilities can still require explanations.The balance between explaining and constructing predictable plans depends on the application, including available communication time.Source: §2.2 · pp. 4–5

2.3. Explainable Reinforcement Learning (XRL)

• XRL adapts XAI ideas to reinforcement-learning agents, including policy visualization, human-readable programs, reward breakdowns, and causal or contrastive explanations.The paper connects XRL to XAR while noting that policy transparency may reveal only which action maximizes cumulative reward.Source: §2.3 · p. 5

2.4. XAR

• XAR is goal-driven explainability, or communication: explaining an autonomous robot’s actions to people during direct, close contact.The paper describes XAR as overlapping with XAIP and XRL while noting that those areas are not yet sufficiently organized together.Source: §2.4 · pp. 5–6

3. What is the explainability of autonomous robots?

• The paper positions its XAR definition against related research, noting that the field is not yet well organized and lacks a complete account of what constitutes an explanation.It reviews philosophical and cognitive-science discussions relevant to defining autonomous-agent explainability.Source: §3 · p. 6, §3.1 · p. 6

3.1. What is an explanation?

• An explanation is a dynamic process: the explainer infers the explainee’s internal state, responds, and may update its own state as the explainee updates theirs.The authors distinguish both the content of an explanation and the approach used to deliver it.Source: §3.1 · p. 6
• Questions that explanations address include what an agent will do, how events will happen, and why it will act; “what” questions matter especially when humans and robots lack shared context.The paper identifies “why” questions as particularly important and difficult, with answers needing to fit the questioner’s intention.Source: §3.1 · pp. 6–7
• The survey focuses on what semantic content answers a “why” query, while noting that little research examines tone, approach, or interaction style.The authors plan to address social aspects such as manner and style among future research issues.Source: §3.1 · p. 7
• People seek explanations to resolve inconsistencies in knowledge and maintain social interaction, but trust and changing beliefs shape how explanations are received.The paper notes that explanations can affect beliefs, impressions, emotions, and behavior.Source: §3.1 · pp. 7–8
• Social aspects of AI and robot explanations remain limited in current research, and XAR work has not pursued explanation’s roles in persuasion, learning, or distraction.The paper connects these social functions to human–robot interaction.Source: §3.1 · p. 8

3.2.1. Decision making

• A decision-making space has Markov properties, and an algorithm generates a plan or policy within it under a constraint such as optimality or explainability.The plan is a sequence of actions, and the paper uses “plan” and “policy” interchangeably.Source: §3.2.1 · pp. 8–9

3.2.2. Overall structure of explanations

• The explanation framework models both agents’ internal policies and uses interpretable policies and models of the other agent to bridge them.It assumes equal task and physical or computational abilities in one case; with differences, policy discrepancies arise.Source: §3.2.2 · p. 9

3.2.3. When an explanation is required

• Explanations are needed when the user asks why the robot chose a plan or action, or why it did not choose an alternative.The robot cannot directly handle the user-side process, so the framework considers ways to bring the user and robot policies closer, including adjusting the robot’s plan or informing the user.Source: §3.2.3 · p. 10

3.3. Definition of explanation in XAR

• The explanation ǫ is generated from the robot’s interpretable policy and its estimated model of the user, rather than by directly presenting the robot’s policy.The paper identifies this generation process with the model reconciliation problem [39].Source: §3.3 · pp. 10–11
• Estimating the user’s expected interpretable policy is more difficult than estimating the user’s own interpretable policy.The paper also notes that estimation depends on assumptions about shared algorithms and constraints.Source: §3.3 · pp. 10–11
• XAR requires an interpretable decision-making space, an estimate of the user’s model of the robot, information enabling the user to estimate the robot’s policy, and a means to present that information.The explanation must be encoded in language and/or images for the user.Source: §3.3 · p. 11
• For an autonomous robot, requirements 1 and 2 are necessary for generating explanations, while requirements 3 and 4 are phases in which established models are used to construct explanations.Source: §3.3 · p. 11

3.4. Owning an interpretable decision-making spaceΠ: Requirement 1

• A decision-making space is interpretable to a user when the user’s planning algorithm and constraints can derive a policy without error; its states, actions, and transitions should be natural and convention-consistent.The paper gives physical laws as an example of real-world conventions.Source: §3.4 · p. 12
• Handwritten rules cannot specify one correct action for every conceivable situation, while causal-graph approaches struggle to model the world’s many sequential factors comprehensively.The factors of interest in causal graphs are specified by humans, limiting claims of autonomous learning.Source: §3.4 · p. 12
• A robot can acquire a world model by observing how its actions relate to changes in the outside world, potentially yielding a human-understandable decision space without direct human intervention.The robot’s observations must be converted into a form interpretable by people.Source: §3.4 · p. 12
• World-model learning may require predefined actions; when actions are difficult to define, the framework needs to learn selectable actions autonomously and assign them meaning.The paper also notes that state differences before and after an action can ground this process.Source: §3.4 · pp. 12–13

3.5. Estimation of the model of others: Requirement 2

• Comprehensively estimating a user’s decision-making space or planning algorithm is difficult because decision space, observability, and external factors are entangled.Even an identical decision-making space does not reveal the reward function used in planning.Source: §3.5 · p. 13
• Estimating a person’s decision-making space and planning algorithms is important because they shape what information an explanation needs to convey.The key, for example, may be obvious to an adult but not to a toddler who has not learned how it opens a door.Source: §3.5 · p. 13

3.6. Estimation of the information needed for a user to estimate πR: Requirement 3

• Explanation generation requires finding information that helps users understand a robot’s policy, with explanations accepted as human-like and appropriate for specific individuals.The paper identifies these as two requirements for finding the explanation in Eq. (8).Source: §3.6 · p. 13

3.6.1. Good explanation from the perspective of cognitive science

• Cognitive-science indicators favor the minimum information needed for comprehension, but the desired balance between simplicity and route accuracy varies with the explanation required.Likeliness concerns the posterior probability of the event being explained; loveliness considers simplicity and latent scope [49,50].Source: §3.6.1 · p. 14

3.6.2. Good explanations for individuals

• Personalized explanations require estimating a user’s decision-making space and planning algorithm, then selecting important information from the overlap with the robot’s space.If the overlap cannot provide a route to the target, or makes one too complicated, the robot or user must supplement missing knowledge or correct misidentified knowledge.Source: §3.6.2 · p. 14

3.7. Presentation to user of explanation: Requirement 4

• Solving Eq. (8) does not itself produce a communicable explanation; the robot must encode its information for human presentation.The passage states the problem in Eq. (8) must therefore be written more strictly.Source: §3.7 · p. 14
• Verbalization and visualization are possible encoding methods, while the choice of modality may also motivate new robot-specific explanatory media.The encoding function transforms the explanation information, and the decoding function receives and decodes it.Source: §3.7 · p. 15

4. Survey on key issues in XAR and associated research

• The survey divides XAR research issues into four requirements and classifies existing work according to which requirements it addresses.Figure 3 schematically presents the research issues, and Table 3 summarizes the classification.Source: §4 · p. 15, Figure 3 · p. 16, Table 3 · p. 17

4.1. Autonomous acquisition of interpretable decision-making space

• Interpretable decision spaces support human-space estimation and acceptable explanations, but existing approaches assume interpretable inputs or human-specified decompositions.World-model approaches can acquire abstract spaces, but those spaces are not necessarily human-interpretable; universal construction without human intervention remains undeveloped [58] [59].Source: §4.1 · p. 16

4.2. Estimation of human decision-making space and policy

• Human-policy estimation methods use actions, interaction history, or inverse-learning models, but no proposed model yet performs well at estimating a person’s internal model.The accuracy of reconstructed policies or behavior can depend on the chosen policy-estimation or restoration model [63].Source: §4.2 · p. 17

4.3. Extraction of key elements in the process

• Prior methods select important explanatory elements using restoration accuracy, reward differences, action values, frequent states, or decision-tree nodes [63].Action-value-based extraction misses factors essential to reaching a target when those factors do not produce large action-value differences; a causal-effect method can extract such factors but has complexity and hyperparameter issues [60].Source: §4.3 · p. 18

4.4. Verbalization and visualization of explanations

• Explanations can be verbalized from policies, question-response states, or state sequences, and visualized with video clips or saliency maps [69] [72] [71].Selecting a presentation method according to user understanding has been proposed, but real-world use would benefit from further work on reward design and training-data acquisition [70].Source: §4.4 · pp. 18–19

4.5. Other research studies

• Other approaches include highlighting utility-relevant features, designing routes that help people predict goals, explaining state changes, and breaking down reward sources [75].The survey presents these approaches as methods that may be combined with the four core requirements as needed.Source: §4.5 · p. 19

5.1. Estimation of decision-making space and planning algorithm of others

• Partner-state estimation remains unrealized, and its domain-dependent definition may benefit from studying how humans infer others’ internal states.The proposed framework should generalize from experience rather than rely on massive training sets for each individual.Source: §5.1 · p. 20

5.2. Consideration of the social aspects of explanation

• Explanation style can alter trust even when meanings match, while explanation-generation methods and their social effects remain underexplored.The authors suggest explaining disliked actions or expressing simulated regret, and call for research on beneficial coexistence.Source: §5.2 · p. 20

5.3. Application to real-world problems

• XAR methods remain largely domain-specific simulations, while real-world use requires physical robots to model partial observations and explain abstract concepts.The paper calls for bringing existing frameworks into real-world problems.Source: §5.3 · pp. 20–21

5.4. Application to interactions

• Real interactions require autonomous choices about explanation timing and content, but how to distinguish when and what to explain remains unclear.User interest and distraction can affect timing; XAIP and XRL also raise different explanation-planning needs.Source: §5.4 · p. 21

5.5. Ethical issues with generating explanations that are untrue

• Robots’ use of untrue explanations raises unresolved questions about tolerated deception, functional trust, and ethical interaction in society.The paper contrasts potentially benign simplification with lies that could anger users and diminish trust in explanations.Source: §5.5 · pp. 21–22

• The paper outlines machine-learning explainability, proposes a definition for autonomous-robot explainability, and surveys aligned research and future issues.Source: §6 · p. 22