PaperlayerView exact revision on arXiv

Source-linked AI summary

Robots That Take Initiative: A Framework for Building and Evaluating Proactive Robots

Maithili Patel, Sonia Chernova

arXiv:2609.28910v1cs.ROcs.AI

TL;DR

Proactive robot assistance lacks a unified formalism, and static evaluation does not capture how robot actions change the environment and user behavior. The paper formalizes three levels of proactivity, introduces an adaptive closed-loop evaluation, and presents a method instantiating its framework. In closed-loop evaluation, the method outperforms non-oracle baselines across nearly all metrics while remaining stable on net actions saved.

  • Problem

    Proactive robot assistance lacks a unified formalism, while static evaluations miss how robot actions change the environment and user behavior.

    Source: §I · p. 1

  • Method

    The paper formalizes three proactivity levels, provides an LLM-based human model for closed-loop evaluation, and presents a method that instantiates its framework.

    Source: §I · p. 1, §VII · p. 8

  • Results

    GAP outperforms all non-oracle baselines across every closed-loop metric except perturbed actions under the LLM user, while remaining stable and positive on net actions saved.

    Source: §VI · p. 7

  • Takeaways & Limitations

    The authors present their formalism and evaluation as a common foundation for future work on unprompted proactive assistance.

    Source: §VII · p. 8

    The user-adaptation model covers only goals already present in HOMER+; modeling true goal adaptation requires real-world studies of longitudinal user adaptation.

    Source: §VII · p. 8

Abstract

from arXiv · show

Effective robot assistance beyond narrow roles and repetitive tasks requires robots to be proactive - to decide what needs to be done rather than waiting to be told. While proactivity is increasingly explored, it lacks a unified formulation, and work in the domain is typically evaluated offline against static human models that cannot capture the effect of a robot's actions on the environment and the user's own behavior. We introduce a unified formalism for proactive robot assistance, organize it into three levels, and provide a framework to address the highest level of unprompted proactive assistance. We then show that offline evaluation overstates performance in this setting, and contribute a closed-loop evaluation with a human model that adapts to the robot. Finally, we present a method, GAP, that instantiates our framework, learning from passive observation to anticipate user goals and act. Under closed-loop evaluation, prior state-of-the-art methods collapse, in some cases adding more work than they save, while GAP remains robust and substantially outperforms them.

I. INTRODUCTION

As robots move into unstructured settings with little supervision, assistance must include deciding what to do and when, not just executing instructed tasks. The paper formalizes proactive assistance, addresses gaps in offline evaluation, and frames robot effectiveness in terms of changes in value and cost.

I. INTRODUCTION

  • The paper introduces a unified formalism with three proactivity levels, focuses on unprompted assistance, and decomposes that problem into algorithmic subproblems.It also situates existing approaches as addressing scoped versions of the problem.Source: §I · p. 1
  • Static human models miss how robot actions change the environment and user behavior, motivating a closed-loop evaluation with an LLM-based human model that adapts to robot actions.The setup benchmarks existing methods to expose shortcomings of static, open-loop evaluation.Source: §I · p. 1
  • The paper presents a model that learns from passive observation to anticipate user goals and adapt its actions, substantially outperforming prior approaches in closed-loop evaluation.Source: §I · p. 1

II. A FORMALISM FOR PROACTIVITY IN ROBOTICS

  • Research uses “proactivity” and “anticipation” for varied forms of initiative, making prior work difficult to connect and motivating a shared formalism and vocabulary.Source: §II · p. 2

A. Problem Formulation

  • The formulation treats human actions as pursuing goals whose value depends on the goal, environment state, and hidden user state.The robot aims to maximize combined net value, accounting for human and robot action costs.Source: A. Problem Formulation · p. 2
  • Robot effectiveness is measured by comparing the robot-present trajectory with a counterfactual trajectory without the robot, using changes in value and cost.The net change, ΔV − ΔC, characterizes the value of the robot’s presence.Source: A. Problem Formulation · p. 2

B. Levels of Proactive Assistance

The paper defines three levels of proactive assistance by the information available to the robot: reactive assistance, collaborative proactive assistance, and unprompted assistance. The highest level is the focus because it does not require the user to delegate a task.

L0: Reactive Assistance

  • L0 reactive assistance gives the robot both the goal and human policy, enabling it to plan toward the goal and predict effective assistive actions.The paper associates this exact knowledge with rigid settings, such as predefined industrial tasks.Source: B. Levels of Proactive Assistance · p. 3

L1: Collaborative Proactive Assistance

  • L1 collaborative proactive assistance gives the robot the goal but not the human policy, so it must infer the user’s preferred strategy from interaction history.Exact goal knowledge is described as realistic mainly for concrete, repetitive tasks such as multi-step assembly.Source: B. Levels of Proactive Assistance · p. 3

L2: Unprompted Proactive Assistance

  • L2 unprompted proactive assistance gives the robot neither the goal nor the human policy, so it infers needs from interaction history and can act toward future goals.The robot may infer the current goal from actions or predict future actions from routine patterns.Source: B. Levels of Proactive Assistance · p. 3
  • Only unprompted proactive assistance fully relieves the user of the cognitive burden of delegating tasks, making it the paper’s focus.Source: B. Levels of Proactive Assistance · p. 3

III. ALGORITHMIC FRAMEWORK FOR L2 PROACTIVITY

The paper decomposes unprompted proactive assistance into grounding, anticipation, and planning, then identifies challenges in inferring user intent and predicting current and future goals from observed behavior.

III. ALGORITHMIC FRAMEWORK FOR L2 PROACTIVITY

  • The proposed framework addresses the highest proactivity level, where the robot conditions actions on the current state and interaction history rather than a supplied goal.Source: §III · p. 3

A. Sub-Problems in Unprompted Proactive Assistance

  • The framework organizes unprompted assistance into grounding user actions, anticipating a need, goal, or action, and planning robot actions toward an anticipated variable.The paper notes that no single method solves the problem end to end.Source: A. Sub-Problems in Unprompted… · p. 3

1) Grounding:

  • Grounding infers users’ goals or tasks from action sequences and associates each observed action with a goal.A goal space must balance abstraction and discrimination; the paper notes that no prior work autonomously curates one, while action-goal association must disentangle concurrent goals and extraneous actions.Source: 1) Grounding · p. 3

2) Anticipation:

  • Anticipation learns the user’s goal-oriented policy, infers the current goal from recent actions, and predicts future goals from behavior and context.The challenges include limited user-specific data, ambiguity among current goals, and modeling habitual patterns despite inaccessible hidden state.Source: 2) Anticipation · p. 3

3) Planning:

Planning selects robot actions by maximizing expected net value over anticipated futures, while existing integrated approaches address only portions of the broader sub-problems.

3) Planning:

  • Planning chooses robot actions to maximize expected net value change over anticipated futures, using predicted goals and human-policy rollouts.The objective is expected ΔV−ΔC across possible robot-action sequences.Source: 3) Planning · p. 3

B. Existing Integrated Solutions for L2 Proactivity

3) Planning:

  • Cost modeling must capture personal, context-dependent effort, while coordinated action selection minimizes user cost using learned costs, human policy, and anticipated goals.The relevant costs can include physical effort, preferences, interaction overhead, and disturbance from concurrent robot actions.Source: 3) Planning · p. 4

B. Existing Integrated Solutions for L2 Proactivity

  • Goal-inference methods,, assume a given goal space and action-goal association, requiring annotations, delaying assistance until beliefs concentrate, and missing habits or preferred variants.They address goal inference but assume the human policy and do not predict future goals.Source: B. Existing Integrated Solutions for… · p. 4
  • Temporal action predictors,,, [42] bypass explicit goal and human-policy inference and predict minutes to hours ahead, but discard goal structure and assume uniform action costs.They imitate predicted user actions directly.Source: B. Existing Integrated Solutions for… · p. 4

IV. L2 EVALUATION SETUP

The evaluation uses a closed-loop setting to capture robot-induced changes to the environment and user behavior, with an LLM-based user model and separate effort metrics. Existing methods lose substantial performance relative to offline evaluation.

IV. L2 EVALUATION SETUP

  • Existing methods lose substantial performance in closed loop relative to offline evaluation as human–robot co-adaptation invalidates static assumptions.Offline evaluation misses state drift and changes in the goals users adopt after assistance.Source: §IV · pp. 4–5

A. Closed Loop Evaluation Setup

  • The closed-loop setup captures action-induced distribution shift and user adaptation, using an LLM human model and metrics that track robot effects on user effort.It uses object relocation as the action space, as in prior work,.Source: A. Closed Loop Evaluation Setup · p. 5

1) Human Model:

  • VirtualHome simulation uses HOMER+ routines from three households over several weeks, while an LLM adapts behavior to robot actions using context and a two-hour prediction window.The LLM can accept, undo, or redirect a robot action; the simulated user otherwise executes routine actions, skipping those already performed by the robot.Source: 1) Human Model · p. 5
  • The LLM-based user model responds more naturally than the scripted model but introduces more stochasticity.Source: 1) Human Model · p. 5

2) Metrics:

  • Metrics report net actions saved and perturbed actions separately as percentages of baseline actions, alongside F1 for comparison with prior evaluations.Action costs are treated as uniform except that recovering from unexpected robot actions carries added cost.Source: 2) Metrics · p. 5

B. Empirical Investigation of Evaluation Misalignment

Offline evaluation overstates existing methods’ performance in closed loop, where GAP outperforms non-oracle baselines across nearly all metrics. GAP combines grounding, goal anticipation, and planning, while its evaluation captures robot actions and human adaptation.

B. Empirical Investigation of Evaluation Misalignment

V. ADAPTIVE UNPROMPTED PROACTIVITY MODEL

  • GAP unifies goal grounding, current- and future-goal anticipation, and planning into a pipeline learned from past observations.The method retrains every five days on past data.Source: §V · pp. 6–7

VI. RESULTS

  • GAP outperforms every non-oracle baseline across all closed-loop metrics except perturbed actions under the LLM user, where STREAK reports fewer.STREAK’s lower perturbation count accompanies much worse net actions saved under the LLM user.Source: Table III · p. 7, §VI · p. 7
  • The oracle sets an assistance ceiling because it acts one tick ahead and cannot preempt multiple moves within the same timestep.The oracle uses ground-truth future relocations.Source: §VI · p. 7
  • GAP handles closed-loop effects through goal- and action-level modeling, plus a planner that learns lead times and acceptance and avoids repeating rejected actions.Its task graphs are not sensitive to the robot performing an action, allowing adaptation of remaining actions within a goal.Source: §VI · p. 7
  • Under the LLM user, net actions saved are -159.9% for STREAK and -174.5% for Plain LLM, while GAP remains positive at 17.7% ± 1.1 and 30.8% ± 1.8.The passage describes the baselines’ errors as destabilizing and compounding, while GAP’s results are stable across households.Source: §VI · p. 7
  • Future-goal assistance accounts for 32.8% of robot actions with the scripted user and 25.3% with the LLM user, but its precision is lower than current-goal assistance.Precision is 0.64 versus 0.80 for the scripted user and 0.53 versus 0.62 for the LLM user, respectively.Source: §VI · p. 8

VII. CONCLUSION AND LIMITATIONS

The paper formalizes proactive robot assistance and evaluates unprompted proactivity in closed loop, where its proposed method outperforms prior approaches. Its setup and method address only part of the open problems, with limits on goal adaptation and action-cost modeling.

VII. CONCLUSION AND LIMITATIONS

  • The paper formalizes proactivity in three levels and proposes a closed-loop setup and method that outperform prior approaches.The focus is the highest level: unprompted proactive assistance.Source: §VII · p. 8
  • The user-adaptation model covers goals already present in HOMER+; modeling users’ true goal adaptation requires longitudinal real-world studies.The LLM proxy should also be validated with real users in this setting.Source: §VII · p. 8
  • The framework assumes uniform action costs and leaves a full model of cost and value, as well as stronger subproblem methods, to future work.Source: §VII · p. 8

APPENDIX I MISALIGNMENT BETWEEN OFFLINE AND CLOSED-LOOP

Offline evaluation can misrepresent closed-loop performance because it fixes both the human’s behavior and goals. The appendix analyzes robot-error drift and human goal adaptation as distinct sources of this mismatch.

APPENDIX I MISALIGNMENT BETWEEN OFFLINE AND CLOSED-LOOP

  • Offline evaluation fixes the human’s trajectory and goals, but closed-loop interaction lets both the robot’s state and the human’s goals change.Relaxing either assumption creates a gap between offline and closed-loop performance.Source: Appendix I · p. 9

A. Relaxation 1: Drift from Robot Errors Compounds Cost

  • Robot errors can push the environment off the training distribution, increasing future error rates and compounding cost.In the model, errors are independent Bernoulli events at rate ε, and each error additively degrades that rate by k, up to 1.Source: A. Relaxation 1: Drift from Robot… · p. 9
  • With probability at least 1−δ, the robot errs at every step from tdeg onward, under the modeled error-drift assumptions.The theorem’s degradation condition follows from accumulated errors reaching Sdeg.Source: A. Relaxation 1: Drift from Robot… · p. 9

B. Relaxation 2: Goal Adaptation Shifts the Optimal Policy

  • When assistance frees a human to pursue higher-value goals, a policy optimized only for recorded goals may miss that value.The optimal adaptive policy achieves at least the net value of the optimal non-adaptive policy, with strict inequality when the adapted goals are not optimally served by the non-adaptive policy.Source: B. Relaxation 2: Goal Adaptation… · p. 9
  • If human goals do not adapt, the offline policy is closed-loop optimal and offline evaluation is exact; the value gap arises when goals adapt.Source: B. Relaxation 2: Goal Adaptation… · p. 10

C. Conclusion

The conclusion frames offline misalignment as both compounding robot-error costs and missed value from human goal adaptation, motivating closed-loop evaluation. The extended discussion organizes prior work around grounding, anticipation, and planning, while noting unresolved challenges in grounding and adapting to shifting goals.

C. Conclusion

  • Offline evaluation understates cost when robot errors degrade future performance and misrepresents value when humans adapt toward goals a fixed-goal policy cannot serve.The proposed evaluation allows robot-induced changes and partially models goal adaptation, but does not let the LLM introduce new goals.Source: C. Conclusion · p. 10

APPENDIX II FORMALISM: EXTENDED DISCUSSION

A. Prior Work Across the Sub-Problems

  • Prior work is organized around grounding goals, anticipating current and future goals, and planning cost-minimizing robot actions; no single method addresses proactivity as a whole.Source: A. Prior Work Across the Sub-Problems · p. 10

1) Grounding:

  • Grounding remains open: prior work does not autonomously curate goal spaces, and action-goal association methods are not designed for open, unlabeled spaces.Goal representations must balance coarse intent with fine task variations, including concurrent goals.Source: 1) Grounding · p. 10

2) Anticipation:

  • Anticipation methods infer current or future goals from observations, but prior work does not address how robot actions shift the user’s later goal distribution.The surveyed approaches include goal inference for started tasks, temporal prediction, and probabilistic models of object movement.Source: 2) Anticipation · pp. 10–11

3) Planning:

The section frames proactive planning around anticipating user goals, modeling user costs, and choosing coordinated robot actions. It also considers value alignment, extensions to the formulation, and supplementary evaluation and implementation details.

3) Planning:

  • Planning predicts an action to aid an anticipated variable, models user costs, and selects a robot policy that minimizes those costs.Relevant costs include physical effort and interaction, teaching, and monitoring burdens; coordination also depends on anticipated goals and the human policy.Source: 3) Planning · p. 11

B. Parallels between Value Alignment and Proactivity

  • Proactive assistance makes alignment with the user’s overall value its objective, requiring prediction of the task itself rather than selecting how to perform an assigned task.A robot situated in a specific home may model users’ universal, cultural, household, and individual values with high fidelity.Source: B. Parallels between Value Alignment… · p. 11

C. Extensions of the formulation

  • With partial observability, users and robots can form different goals from their beliefs; communication can reconcile beliefs or clarify values and costs.The robot’s objective remains maximizing the user’s hidden net value, even when its goals appropriately diverge from the user’s, such as when the user misses an oncoming car.Source: C. Extensions of the formulation · p. 11
  • For multiple users, the robot must align differing value and cost models; conflicts may require a hierarchy of control or ownership and arbitration of stakeholder constraints.Caregivers, practitioners, and legal or ethical requirements may also constrain the model.Source: C. Extensions of the formulation · p. 12

A. Effect of Confidence Threshold

  • At cthresh = 0.1, the robot saves the most user actions with relatively few perturbed actions and maximizes overall F1-score.The threshold modulates the robot’s conservativeness; Fig. 4 varies precision-recall, net actions saved, and perturbed actions with cthresh.Source: A. Effect of Confidence Threshold · p. 12, Figure 4 · p. 12

A. Hyperparameters for Reproducibility

  • The implementation fixes task-graph smoothing at α = 0.5 and uses 1,000 particles, retraining every five days on accumulated data.It also specifies transformer, planner, and policy settings, including a confidence threshold of 0.1.Source: A. Hyperparameters for Reproducibility · pp. 12–13

B. Effect of distribution drift during evaluation and training

B. LLM Prompts

  • The appendix supplies example prompts for the LLM curator, human adaptation model, and plain LLM baseline.The curator prompt guides activity segmentation and vocabulary reuse from household object-relocation logs.Source: B. LLM Prompts · p. 13

1) LLM Curator Prompt:

The curator prompt asks an LLM to infer activity episodes from a household’s object-relocation logs, using a reusable activity vocabulary and structured output. It also allows vocabulary merges and renames when observations warrant them.

1) LLM Curator Prompt:

  • The curator partitions a day’s numbered object-relocation events into contiguous activity segments and assigns each a semantically meaningful activity name.The prompt permits isolated events that interrupt an activity to remain unassigned.Source: 1) LLM Curator Prompt · p. 13
  • The example input provides a day’s household object moves and asks the model to identify activities, such as meals, dishwashing, or getting ready.The prompt specifies object-relocation logs as the only sensor input, without vision or audio.Source: 1) LLM Curator Prompt · p. 13
  • Existing vocabulary names should be reused for matching episodes; new snake_case names are created only when no existing name fits.The prompt’s vocabulary includes breakfast, wash_dishes, and computer_work.Source: 1) LLM Curator Prompt · pp. 13–14
  • The structured response represents segments with event indices, activity names, and a flag for new activities, while allowing vocabulary merges and renames.A rename includes the old and new names, a new description, and a reason.Source: 1) LLM Curator Prompt · p. 14

2) LLM Human Model Prompt:

The human-model prompt simulates a household user responding to robot actions by undoing, allowing, or redirecting them. Decisions are based on the user’s upcoming planned moves, the consequences of leaving an object in place, and examples of prior behavior.

2) LLM Human Model Prompt:

  • When a robot action diverges from the human’s plan, the model chooses whether the human would undo it, allow it, or redirect it.It is instructed to adapt naturally, including sometimes skipping the planned step entirely.Source: 2) LLM Human Model Prompt · pp. 14–15
  • The prompt supplies the current activity, a complete upcoming-action list for a time window, and recent history to guide a decision.Examples contrast correcting a misplaced plate, allowing a mug in a reasonable location, and redirecting a plate needed soon at the table.Source: 2) LLM Human Model Prompt · pp. 14–15
  • The model should redirect an object straight to its known near-term destination rather than undoing the robot and moving it again later.Undo is reserved for actively harmful placement, such as perishable food left outside the fridge, or when there is no foreseeable use.Source: 2) LLM Human Model Prompt · p. 15

3) Plain LLM Baseline Prompt:

The plain LLM baseline is prompted to predict household object relocations that would help a user’s routine. It receives past days and the incomplete current day, then predicts up to ten immediate moves.

3) Plain LLM Baseline Prompt:

  • The baseline is asked to learn a person’s routine from past observations and predict useful object relocations for the current routine.It is instructed to predict moves that help now or very soon.Source: 3) Plain LLM Baseline Prompt · p. 15
  • The prompt presents two past days of relocation logs and the current day’s partial log, ending at 08:40.The historical examples include repeated routines such as breakfast, meals, and object put-away.Source: 3) Plain LLM Baseline Prompt · pp. 15–16
  • The baseline must predict up to ten object-and-destination relocations the robot should perform right now.The response schema is an array of predictions containing an object and destination.Source: 3) Plain LLM Baseline Prompt · p. 16

Related papers

Loading 2609.28910v1…