NOPA: Neurally-guided Online Probabilistic Assistance for Building Socially Intelligent Home Assistants

Motivation

• NOPA addresses online assistance in complex homes, where robots must infer human goals, avoid disrupting people, and adapt inference and helping as tasks progress.Large state, action, and goal spaces make online inference difficult, while ambiguous inferences can cause ineffective or counterproductive help.Source: §I · p. 1

Method

• NOPA combines neural goal proposals, inverse planning, and particle filtering for fast, robust goal inference under uncertainty.Its planner uses current predictions and certainty to choose valuable helping subgoals and generate symbolic helping actions.Source: §I · p. 1

Evaluation

• Online Watch-And-Help requires a helper to infer a main agent’s goal online while helping complete household tasks efficiently.The challenge extends assistance evaluation to simultaneous observation, inference, and help in realistic virtual homes.Source: §I · p. 2

Results

• NOPA significantly outperforms all baselines and produces helping strategies that adapt to new observations.The benchmark includes agents controlled by either human players or planners across household tasks.Source: §I · p. 2

Online goal inference

• Feedforward goal predictors are fast but can struggle in unfamiliar scenarios, whereas inverse planning is robust yet slow in large goal spaces.NOPA integrates both approaches to seek speed and robustness.Source: §II · p. 2

Embodied AI assistance with unknown goals

• Assistance with inferred goals can become counterproductive under uncertainty, motivating planners that adjust from goal-agnostic to goal-specific behavior.Prior work includes uncertain-goal planners and empowerment-based assistance, but the paper targets complex environments.Source: §II · p. 2

Embodied AI assistance benchmarks

• Most collaboration benchmarks assume a shared goal or use simple game environments, while Watch-And-Help uses a pre-recorded demonstration.O-WAH removes the demonstration and requires continual inference and uncertainty updates during assistance.Source: §II · pp. 2–3

Problem Setup

• The helper must infer a main agent’s goal and help achieve it faster in a mixed-observability Markov decision process.The formal model includes world states, hidden goals, helper actions, observations, transitions, rewards, and discounting.Source: A. Problem Setup · p. 3
• The helper observes the world state and main-agent action, using the main agent’s past trajectory to infer its goal over time.Goal inference is therefore online and conditioned on accumulated observations.Source: A. Problem Setup · p. 3
• The reward combines the probability of satisfying the goal with a cost for the helper’s action.The formulation assumes world states are fully observable to both agents, consistent with prior unknown-goal assistance work,,.Source: A. Problem Setup · p. 3

B. Method Overview

• NOPA updates particles representing possible final goals and predicts each goal’s future trajectory from observed states and actions.Scene graphs represent intermediate states and final goals as entity nodes connected by predicate edges.Source: B. Method Overview · p. 3
• The helping planner evaluates predicate edges in intermediate states and final goals, selecting the most valuable edge as the helping subgoal.This expands assistance beyond frameworks that consider only the final goal.Source: B. Method Overview · p. 3
• The method retains proposals whose predicted trajectories remain consistent with observed actions and samples new neural proposals when needed.The proposal procedure uses a goal proposal distribution and MCTS-generated plans.Source: B. Method Overview · p. 3

• NOPA samples goal proposals from a neural network using the initial and current states, enabling application when helping changes the observed state sequence.Source: C. Neurally-guided Online Goal Inference · p. 3
• Inverse planning predicts bounded-rational future trajectories for each proposed goal, then rejects proposals inconsistent with the main agent’s observed actions.Source: C. Neurally-guided Online Goal Inference · pp. 3–4
• Particle filtering maintains goal–trajectory pairs and resamples proposals when particles are exhausted or the prediction horizon is reached.Source: C. Neurally-guided Online Goal Inference · p. 4
• Multiple particles can share a goal while representing different predicted trajectories, allowing NOPA to consider alternative ways to achieve the same goal.Source: C. Neurally-guided Online Goal Inference · p. 4

D. Uncertainty-aware Helping Planner

• The planner evaluates candidate subgoals from predicted trajectories, final goals, and the initial state for returning unnecessary objects.It estimates main-agent reach time from particle trajectories or a fixed horizon Lmax for subgoals appearing only in final goals.Source: D. Uncertainty-aware Helping Planner · p. 4
• Subgoal value combines the probability that the subgoal is needed, expected speedup, helper cost, and additional deviation from the initial state.The value function weights these terms through wr, wc, and wm.Source: D. Uncertainty-aware Helping Planner · p. 4
• The helper estimates necessity from the fraction of particles containing each subgoal and executes the first action of the highest-valued helping plan.This makes selection responsive to uncertainty in inferred goals while allowing irrelevant objects to be restored.Source: D. Uncertainty-aware Helping Planner · p. 4

IV. ONLINE WATCH-AND-HELP

• O-WAH requires a helper to infer a main agent’s goal and help complete household tasks faster in VirtualHome-Social.Tasks cover setting a table, loading a dishwasher, stocking a fridge, preparing a meal, and getting snacks.Source: §IV · p. 4
• The benchmark includes uncertainty about object counts, needed objects, and target locations, with a goal space one or two orders of magnitude larger than prior work.Task types are not provided to helper agents; they only ensure goals emulate household activities.Source: §IV · p. 4
• Training uses 6,000 large-set or 300 small-set episodes, while testing uses 100 episodes in two apartments unseen during training.Training episodes sample goals and environments from five training apartments and use a built-in hierarchical planner for the main agent.Source: §IV · pp. 4–5
• Figure 4 reports method speedup, goal-prediction F1 over normalized task progress, single-episode F1 trajectories, and human-controlled speedup.The figure also distinguishes results using the small training set with dashed lines or striped bars.Source: Figure 4 · p. 5
• Goal inference is measured by predicate-level F1-score, while helping is measured by speedup LM/LH − 1 under a 250-step limit across three runs.Speedup compares assisted episode length with the main agent’s solo episode length.Source: §IV · p. 5

V. EXPERIMENTS

• NOPA is compared with HPGPN, AFGPN, Empowerment, and HPRG baselines spanning most-probable, online-inference, empowerment, and random-goal assistance.The baselines differ in how they select goals, plans, or helping subgoals.Source: A. Baselines · p. 5
• Ablations remove the proposal distribution, inverse planning, or irrelevant-object return mechanism to assess their effects on NOPA.OursRG uses uniform proposals, Ours-InvPlan omits inverse planning, and Ours-Return sets wm = 0.Source: A. Baselines · p. 5
• Sample efficiency is evaluated with a small-set GPN, while HPGT uses the ground-truth goal to approximate an upper bound on helping performance.Default GPN training uses the large training set.Source: A. Baselines · p. 5
• NOPA uses fixed planning and inference settings, including Tmax = 250, Tprop = 15, K = 20 proposals, and Adam training with learning rate .0009 and batch size 256.The planner weights are wr = 1, wc = 1, and wd = 5, with wm = 1 and Lmax = 100.Source: A. Baselines · p. 5

1) Main Controlled by a Planner:

• NOPA achieves the highest helping speedup among all baselines and the best early-stage goal inference accuracy.The oracle agent with true goal knowledge achieves a speedup of 1.29.Source: 1) Main Controlled by a Planner · pp. 5–6
• Multiple goal proposals improve helping under uncertainty, especially when the goal proposal network is trained with fewer data and is less accurate.Comparing OursGPN with HPGPN shows the benefit of retaining multiple proposals, with a stronger effect for GPN-S.Source: 1) Main Controlled by a Planner · p. 6
• Neural goal proposals improve inference over uniform proposals, while inverse planning filters spurious proposals and enables earlier accurate inference.Inverse planning significantly improves speedup over Ours-InvPlan by reaching high accuracy earlier.Source: 1) Main Controlled by a Planner · p. 6
• NOPA filters inconsistent goal proposals, updates uncertainty with observations, and begins useful assistance once its inference becomes confident.In the example, it avoids grabbing extra objects by gradually updating the number needed.Source: 1) Main Controlled by a Planner · p. 6, Figure 5 · p. 6
• NOPA supports helping subgoals beyond directly achieving final goals, including handing objects to the main agent and returning unnecessary objects.These behaviors can speed task execution and reduce the effects of objects later identified as unnecessary.Source: 1) Main Controlled by a Planner · p. 6, Figure 6 · p. 6

2) Main Controlled by Humans:

• With human-controlled main agents, the ranking of NOPA, HPGPN, and HPRG remains consistent with the planner-controlled evaluation.The human experiment used 40 trials across 10 testing episodes and 10 participants.Source: 2) Main Controlled by Humans · p. 6

VI. CONCLUSION

• NOPA significantly outperforms baselines across evaluations and achieves great sample efficiency for training.The conclusion reports evaluations with both built-in-planner and human-controlled main agents.Source: §VI · p. 7