PaperlayerView exact revision on arXiv

Source-linked AI summary

HygieneRoboBench: Benchmarking Hygiene-Aware Planning for Household Robots

Yurun Chen, Josh Qixuan Sun, Jason Qin, Chengtai Li, Tianyi Wang, Mark Crowley, Wentao Zhu

arXiv:2610.08642v1cs.ROcs.AI

TL;DR

Household robots can carry contamination between contacts, and existing evaluations do not jointly test history- and event-aware hygiene planning under user priorities and resource limits. HygieneRoboBench evaluates these decisions through controlled comparisons and independent plan checks, while Hygiene-NSP combines grounding, history reconstruction, and constraint solving. Hygiene-NSP achieves the highest overall safe-resolution and optimal safe-resolution rates among evaluated planners.

  • Problem

    Existing evaluations lack a systematic joint assessment of how contact history changes optimal continuations and how planners respond to contact events while meeting hygiene and user requirements.

    Source: §I · p. 2

  • Method

    HygieneRoboBench uses controlled history, profile, and event comparisons with independent plan evaluation, and Hygiene-NSP combines language grounding, history-based state reconstruction, and constraint solving.

    Source: §I · p. 2

  • Results

    Hygiene-NSP achieves the highest overall safe resolution (94.4%) and optimal safe resolution (90.4%).

    Source: §I · p. 2

  • Takeaways & Limitations

    Considering cost optimality alongside safe completion changes the relative ranking of planners on controlled history pairs.

    Source: §I · p. 2

Abstract

from arXiv · show

Contact with contaminated objects can spread hazards through a household robot's grippers, tools, and shared surfaces, while new contacts can make an existing plan unsafe. Existing benchmarks do not jointly assess how planners identify hygiene risks from contact history and plan safe continuations after new contact events. Planners must do so within time and resource limits while respecting user priorities. We introduce HygieneRoboBench, with 624 instances across 134 task families, to evaluate safe resolution of household tasks from a given execution history. Tasks capture contamination through two grippers and shared objects, treatment costs, and user priorities. We combine controlled history, profile, and event comparisons with independent plan evaluation. These assess safe resolution, cost efficiency under user priorities, and responses to contact events. Evaluation of LLM-based and symbolic planners shows that safely completing a task does not guarantee the lowest execution costs under the user's priorities. To address this problem, we introduce Hygiene-NSP. It combines LLM-based grounding, contact-history reconstruction, and CP-SAT to jointly plan hygiene treatment and task execution under user priorities. Hygiene-NSP achieves safe resolution and optimal safe resolution rates of 94.4% and 90.4%, respectively. Both rates are higher than those of the evaluated baseline planners on the full dataset. Project page: https://euron-zc.github.io/HygieneRoboBench/.

I. INTRODUCTION

HygieneRoboBench evaluates whether household planners use contact history and adapt to events while satisfying hygiene, resource, time, and user-priority constraints. Hygiene-NSP combines language grounding, history-based state reconstruction, and constraint solving, and achieves the highest reported overall safe-resolution rates.

I. INTRODUCTION

  • Existing evaluations do not jointly assess how contact history and new events affect safe, user-aligned continuations.Cleaning consumes time and supplies, so treatment choices must account for remaining resources and user priorities.Source: §I · pp. 1–2
  • HygieneRoboBench evaluates 624 instances across 134 task families, covering contact history, user priorities, events, and time and resource limits.Its controlled comparisons assess safe resolution, execution costs under user priorities, and responses to histories and events.Source: §I · p. 2
  • Hygiene-NSP combines language grounding, contact-history reconstruction, and constraint solving to jointly plan treatment and task execution under user priorities.The benchmark also uses human review, independent plan evaluation, and Reference certification to support task plausibility and evaluation.Source: §I · p. 2
  • Controlled history comparisons show that safe completion alone can miss differences in optimal planning among planners.Considering cost optimality alongside safe completion changes their relative rankings.Source: §I · p. 2
  • Hygiene-NSP achieves the highest overall safe resolution (94.4%) and optimal safe resolution (90.4%).The benchmark also finds that all seven evaluated planners perform worse as structural task difficulty increases.Source: §I · p. 2

II. RELATED WORK

Prior work covers household tasks, safety, adaptation, personalization, and planning, but not their joint assessment for hygiene decisions shaped by contact history. HygieneRoboBench evaluates these requirements through a shared task input and safe, cost-aware continuation plans.

II. RELATED WORK

  • Related benchmarks address household activity, safe planning, interaction risks, adaptation, personalization, and language-to-plan methods, while HygieneRoboBench focuses on how past contact changes safe continuations and whether plans follow user priorities.Treatment affects both action safety and available resources, which Hygiene-NSP plans jointly with household actions under user priorities.Source: §II · pp. 2–3

III. HYGIENEROBOBENCH

  • HygieneRoboBench assesses whether planners meet user needs under contact history, user priorities, and event information while checking plan feasibility and alignment with priorities.These requirements are presented as interrelated components of household task planning.Source: §III · p. 3

A. Task Formulation

  • At decision time, planners receive the request, execution history, physical observations, remaining resources, event information, priorities, and shared rules, then return a plan or an infeasibility judgment.Feasible plans preserve executed actions and ongoing processes; among them, execution costs are minimized in the user’s lexicographic priority order.Source: A. Task Formulation · p. 3

B. Dataset Construction

The dataset adapts household activities into controlled instances using shared task, hygiene, event, resource, and priority definitions. Validation and human review support the plausibility and evaluation of those instances.

B. Dataset Construction

  • Base tasks adapt BEHAVIOR-1K activities across five activity groups and seven household areas, while hygiene rules formalize contact transfer and treatment requirements.The hygiene-rule registry draws on peer-reviewed studies and public-health guidance,,.Source: B. Dataset Construction · p. 3
  • Five shared registries specify task requirements, actions, hygiene rules, events, resources, and user priorities for consistent construction and evaluation.The cost–profile registry represents ten execution-cost dimensions and lets user priorities change plan rankings while action costs remain fixed.Source: B. Dataset Construction · pp. 3–4
  • The dataset comprises 134 task families and 624 instances, with activity groups, construction designs, and difficulty each partitioning the same instances.Difficulty tiers use seven structural features, including temporal coupling, active hazards on multiple carriers, and constrained treatment resources.Source: Figure 3 · p. 4, B. Dataset Construction · p. 4
  • Episode Builder combines a base task with contact history, priorities, and event information to create controlled instances within task families.Crossed-history groups contain 200 instances, and event contrasts include no-event, relevant-event, and irrelevant-event conditions.Source: B. Dataset Construction · p. 4
  • Dataset admission combines matched-condition checks, Reference feasibility and cost validation, independent full-history replay, and human review.Three reviewers assess task wording, household plausibility, and treatment applicability; among 24 prior-violation instances, 10 admit a safe continuation and 14 do not.Source: B. Dataset Construction · p. 4

C. Evaluation Protocol and Metrics

The evaluation independently checks continuation plans and infeasibility judgments against full histories and certified Reference targets. Its metrics distinguish safe resolution from safe resolution at user-priority-optimal costs, and compare responses across controlled profiles, histories, and events.

C. Evaluation Protocol and Metrics

  • The Evaluator independently replays submitted plans against goals, hygiene, timing, and resource constraints, then recomputes their execution costs.The Reference supplies feasibility and certified optimal costs within declared action and time bounds; planners do not see private targets or feedback.Source: C. Evaluation Protocol and Metrics · p. 4
  • Safe Resolution Rate measures safe completion or correct rejection of the remaining task, counting only correct outcomes on feasible or certified-infeasible instances.Undetermined outcomes and incorrect rejections do not count as resolutions.Source: C. Evaluation Protocol and Metrics · p. 4
  • Optimal Safe Resolution requires safe completion at the certified user-priority optimum or correct rejection of a certified-infeasible task.Profile-Pair Success further counts pairs where both instances complete safely at their respective reference-optimal costs.Source: C. Evaluation Protocol and Metrics · pp. 4–5
  • History-pair completion counts pairs where both instances complete safely, while HC* additionally requires each plan to match its reference-optimal costs.Relevant-history pairs share a safe continuation but have different optimal cost vectors; irrelevant-history pairs share a certified-optimal continuation.Source: C. Evaluation Protocol and Metrics · p. 5

IV. HYGIENE-NSP: A HYGIENE-AWARE NEURO-SYMBOLIC PLANNER

Hygiene-NSP reconstructs hygiene conditions from contact history and jointly plans treatment and service actions under user priorities. Because these decisions interact, it replans after actions or event-information updates.

  • Hygiene-NSP jointly plans hygiene treatments and service actions under user priorities.It reconstructs hygiene conditions from contact history as part of this planning.Source: §IV · p. 5
  • Treatment and service decisions are coupled because a washed gripper can be contaminated again, while cleaning or replacement consumes service resources.Source: §IV · p. 5
  • The planner repeats its planning call on updated public input after an action or event-information update.Source: §IV · p. 5

A. Language Grounding and Constraint Construction

Hygiene-NSP grounds user requests and shared rules into a planning problem, then combines history replay with joint modeling and lexicographic CP-SAT planning. It returns a plan and modeled costs or distinguishes success, proved infeasibility, and undetermined outcomes.

B. History Replay and State Reconstruction

History replay derives the current hygiene state from ordered contacts and treatments, retaining past violations and resource use. Joint optimization then selects treatment, service actions, and timing subject to hygiene, scheduling, and resource constraints, comparing feasible plans by priority-ordered costs.

B. History Replay and State Reconstruction

  • History replay follows recorded actions in order under shared contamination and treatment rules to derive the hygiene state for the remaining task.Physical observations alone may not reveal earlier hygiene consequences, and a wash does not rule out later recontamination.Source: B. History Replay and State… · p. 5
  • Replay preserves past violations and cumulative resource consumption, and the planner replays the full history after an action or new event before planning a continuation.Source: B. History Replay and State… · p. 5
  • Optional state projection retains information and past evidence relevant to the remaining task, but a checker validates it against full replay; otherwise, planning uses the full state.Source: B. History Replay and State… · p. 5

C. Joint Hygiene and Service Optimization

  • Treatment routes, service actions, and their timing are chosen jointly: fixing treatment first can exclude alternatives that better meet remaining constraints or user priorities.Feasibility is enforced before feasible plans are compared by execution costs.Source: C. Joint Hygiene and Service… · p. 5
  • CP-SAT selects actions and their start and end times while enforcing action effects, gripper-use conflicts, shared capacity, remaining supplies, and protection of objects from contamination.Source: C. Joint Hygiene and Service… · p. 5
  • The solver optimizes cost components in priority order, fixing each proved optimum before optimizing the next; incomplete optimization can retain a feasible plan without claiming lexicographic optimality.If it finds no plan without proving infeasibility, the outcome is undetermined; successful optimization returns a plan and modeled costs for independent evaluation.Source: C. Joint Hygiene and Service… · p. 6

V. EXPERIMENTS

The experiments evaluate overall task resolution, plan choices under varying histories and user priorities, and safe, cost-optimal continuation after contact events. A separate method analysis examines state projection.

  • RQ1 evaluates overall resolution.Source: §V · p. 6
  • RQ2 examines plan choices under different histories and user priorities.Source: §V · p. 6
  • RQ3 evaluates safe, cost-optimal continuation after contact events, while a separate method analysis examines state projection.Source: §V · p. 6

A. Experimental Setup

The evaluation compares seven planners on full-history task resolution, then examines difficulty, activity, execution safety, and cost efficiency. Hygiene-NSP leads overall, while examples show that safe task completion can still conceal hygiene violations or avoidable costs.

A. Experimental Setup

  • The evaluation compares five language models, Rule-SAT, and Hygiene-NSP on 624 instances, using common inputs and evaluating outputs under the same protocol.The main comparison uses full contact history; event response and state projection are assessed separately.Source: A. Experimental Setup · p. 6

B. RQ1: How Reliably Do Planners Resolve Requests?

  • Hygiene-NSP achieves the highest overall SR and OSR, at 94.4% and 90.4%, respectively.The metrics assess safe resolution and optimal safe resolution under user priorities.Source: B. RQ1: How Reliably Do Planners… · p. 6
  • SR and OSR decrease from easy to moderate to hard instances for all seven planners, while Gemini 3.6 Flash leads both metrics in cleaning and laundry.The activity-level result exposes differences not visible in the overall ranking.Source: B. RQ1: How Reliably Do Planners… · p. 6
  • Full-history replay detects contamination when DeepSeek-V4.1-Flash grasps a protected food container with a gripper exposed to peanut residue through a shared handle.Hygiene-NSP replaces the contact insert before moving and placing the objects, producing a safe plan at reference-optimal cost.Source: B. RQ1: How Reliably Do Planners… · pp. 6–7
  • DeepSeek-V4.1-Flash returns 98 safe, nonoptimal plans; in 41, water or replacement parts are the first nonoptimal cost dimension under user priorities.Independent cost evaluation identifies resource and scheduling inefficiencies even in safe plans.Source: B. RQ1: How Reliably Do Planners… · p. 7
  • For the hot-cocoa task, DeepSeek-V4.1-Flash takes 19 TU and Hygiene-NSP takes 13 TU with the same actions and consumables; 13 TU matches the reference optimum.Hygiene-NSP places the marshmallows during cooking rather than after cooking and pouring.Source: B. RQ1: How Reliably Do Planners… · p. 7

C. RQ2: Do Plans Adapt to History and User Priorities?

Controlled history and priority pairs test whether planners adjust continuations to relevant contact histories and user preferences, and event instances test replanning after contact. Results show that safe completion counts alone can miss differences in optimality, while Hygiene-NSP performs strongly across these comparisons.

A. History pairs (HC / HC∗)

  • On relevant-history pairs, Hygiene-NSP has the highest HC∗ at 156; DeepSeek-V4.1-Flash has higher HC but lower HC∗ than Gemini 3.6 Flash, reversing their ranking.History-pair evaluation distinguishes optimal planning from safe completion alone.Source: C. RQ2: Do Plans Adapt to History and… · p. 7, Table IV · p. 7
  • Across seven planners, none of 1,699 originally safe, optimal plan replays remained both safe and optimal after relevant-history swaps; all 1,288 irrelevant-history swaps preserved outcomes.The results support the validity of controlled history pairs: relevant-history changes affect safety or optimality, whereas irrelevant-history changes preserve evaluated outcomes.Source: C. RQ2: Do Plans Adapt to History and… · p. 7, A. History pairs (HC / HC∗) · p. 7
  • Hygiene-NSP succeeds on 234 of 268 general profile pairs and 24 of 25 ranking-flip pairs, the highest counts in both groups.In the spoon–fruit task, it switches between washing and replacement under the two priority profiles and reaches the reference optimum in both.Source: A. History pairs (HC / HC∗) · p. 7, Table IV · p. 7

D. RQ3: Can Planners Continue Safely at Optimal Costs After Contact Events?

  • Under relevant and irrelevant events, Hygiene-NSP SR is 90% and 92%, respectively, while its OSR is 88% and 92%; Gemini 3.6 Flash has the highest OSR in both conditions.Each event condition contains 50 instances.Source: D. RQ3: Can Planners Continue Safely… · pp. 7–8, Table V · p. 7
  • Across 100 contact-event instances, Hygiene-NSP’s nine SR failures are undetermined outcomes or incorrect rejections, and all its submitted plans pass execution checks.Gemini’s seven SR failures comprise four incorrect rejections and three invalid continuations.Source: D. RQ3: Can Planners Continue Safely… · p. 8

E. Method Analysis: State Projection

State projection reduces local planning time while largely preserving plan status and cost agreement in paired runs. The paper concludes that the benchmark connects contact-history reasoning with resource trade-offs, while noting that its current evaluation uses explicit rules and leaves physical execution for future work.

E. Method Analysis: State Projection

  • Projected-state planning reduces median family-level local planning time by 20.1%, excluding LLM calls; plan status and cost agree in 175 of 177 pairs, with no new hygiene violations.The comparison covers 59 task families with three paired runs each and includes 21 full-state fallbacks.Source: E. Method Analysis: State Projection · p. 8

VI. CONCLUSION AND FUTURE WORK

  • HygieneRoboBench provides a common basis for evaluating hygiene decisions in complete service tasks, connecting contact-history reasoning with resource trade-offs under user priorities.The benchmark has 624 instances across 134 task families.Source: §VI · p. 8
  • The current evaluation uses explicit contact and treatment rules; physical execution with measured contact and treatment outcomes is future work.Source: §VI · p. 8
Loading 2610.08642v1…