PaperlayerView exact revision on arXiv

Source-linked AI summary

RobotValues: Evaluating Household Robots When Human Values Conflict

Jongwook Han, Hyeongjin Kim, Yohan Jo

arXiv:2606.03312v1cs.ROcs.AI

TL;DR

Household-robot benchmarks lack systematic evaluation of decisions among plausible actions that prioritize conflicting human values. ROBOTVALUES constructs a 10K-scenario image-grounded benchmark with stakeholder-grounded annotations and evaluates robotics-oriented VLMs, finding that conflicting value instructions often fail to override default preferences. The benchmark therefore supports assessing value-sensitive action selection alongside task completion and safety.

  • Problem

    Existing benchmarks do not systematically evaluate how household robot planners choose between feasible actions prioritizing different human values, despite consequences for safety, privacy, dignity, and autonomy.

  • Method

    ROBOTVALUES combines realistic household images, textual decision contexts, plausible candidate actions, stakeholder-grounded value annotations, and staged LLM-assisted generation and quality control.

  • Results

    VLMs default toward safety and accommodation over privacy, while value-conditioned accuracy drops from 40.2%–51.3% in the Matched group to 6.9%–16.8% when target norms conflict with default preferences.

  • Takeaways & Limitations

    Household-robot evaluation should assess whether systems choose among feasible actions prioritizing different human values, not only task completion or safety.

  • Takeaways & Limitations

    The benchmark uses synthetically generated images and LLM-based generation, so real-home visual complexity, sensing noise, interaction dynamics, artifacts, and annotation errors may remain underrepresented.

Abstract

from arXiv · show

While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations in which robots are expected to choose actions that prioritize other values than task success, such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 10K value-conflict scenarios. Each instance consists of a realistic household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. Using RobotValues we evaluate VLMs used in robotics and find that models exhibit default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When the models are instructed to prioritize specific values that conflict with their own preferences, they often fail to override their default actions, choosing incorrect actions for 80% of the time. These findings suggest that household robot evaluation should measure not only task completion or safety compliance, but also whether robots can choose among plausible actions when human values conflict.

1 Introduction

Existing household-robot benchmarks emphasize task success and related capabilities, but do not evaluate choices among plausible actions that prioritize conflicting human values. ROBOTVALUES addresses this gap with a benchmark of value-conflict scenarios and uses it to reveal models’ default preferences and difficulty following conflicting value instructions.

  • Motivation: Existing benchmarks mainly measure task completion, manipulation reasoning, social scene understanding, or safety rather than value-conflicting decisions.These metrics do not fully capture decisions robots face before task execution.
  • Motivation: Household robots may choose between assistance, respecting autonomy and privacy, or seeking help for safety in the same domestic situation.The example illustrates that plausible actions can prioritize different values without one being simply correct.
  • Contribution: ROBOTVALUES provides 10K quality-controlled household images with textual contexts and multiple plausible actions prioritizing different values.The benchmark includes scenarios such as calling a husband for safety or staying nearby for autonomy.
  • Contribution: The benchmark grounds action-value annotations in stakeholder reactions rather than assigning values solely from action wording.Its construction uses an automated generation-and-filtering pipeline with manually curated criteria and LLM-based binary quality checks.
  • Findings: Models preferentially select safety- and accommodation-oriented actions, underselect privacy-prioritizing actions, and often fail to override defaults when instructed otherwise.The reported average accuracy drop for selecting actions aligned with a conflicting target value exceeds 30 percentage points.

2 Related Work

Prior research evaluates household robots through task execution, planning, safety, social norms, and pluralistic value alignment. ROBOTVALUES extends these lines by focusing on image-grounded household decisions among feasible actions prioritizing conflicting values.

  • Household robot benchmarks and task planning: Household-robot benchmarks commonly evaluate manipulation, instruction following, long-horizon planning, real-world learning, and simulated environments.These approaches generally assume that the robot’s goal is already specified.
  • Household robot benchmarks and task planning: Language models have also been used to decompose natural-language instructions into subtasks, skills, or executable plans.This work addresses planning after an objective has been given rather than selecting among value-conflicting objectives.
  • High-level robot decision making and social norms: High-level robot decision-making research includes safety-oriented robot constitutions, orchestration systems, and context-dependent social norms.These studies support evaluating robot behavior beyond task success.
  • Pluralistic alignment in AI: Pluralistic alignment studies diverse and conflicting human values mainly through text-based systems and value taxonomies.ROBOTVALUES brings this perspective to household planning with image-grounded scenarios and candidate actions.

3 Benchmark Design

ROBOTVALUES is designed as an image-grounded benchmark for everyday household situations where stakeholders face genuine value trade-offs. Each instance combines visual and textual context with plausible candidate actions and stakeholder-grounded value annotations.

  • Design goals: The benchmark targets image-grounded household decisions involving diverse human values and concrete stakeholder perspectives.Its design goals require candidate actions to represent genuine trade-offs rather than clearly superior or inferior choices.
  • Data schema: Each instance contains a household image, textual task context, and multiple candidate robot actions with stakeholder-grounded value annotations.The context summarizes the robot’s task, visible scene state, immediate decision context, and non-visual household information.
  • Data schema: Candidate actions are described in natural language and annotated with the value they promote, such as immediate physical safety from falling.Metadata also records scenario descriptions, stakeholders, stakeholder stances, and action-level value annotations.
  • Evaluation protocol: The evaluation protocol asks VLMs to select a next action from a household image, textual context, and candidate actions.In the value-conditioned setting, the model must select the action prioritizing a specified target value.

4 Data Construction

ROBOTVALUES uses staged LLM- and image-model-assisted construction to generate realistic, diverse household decision points, feasible value-differentiated actions, stakeholder-grounded annotations, and filtered visual data.

  • Data construction: The pipeline generates household decision points, feasible actions, stakeholder reactions, and promoted values before creating images.Value annotations are grounded in stakeholder reactions to situations rather than only predefined taxonomy labels.
  • Pipeline: Five-stage construction uses persona and context seeds, scenario generation, candidate actions, value annotation, image generation, and stage-wise quality filtering.Only accepted samples proceed to subsequent stages.
  • Persona and context seeds: Persona seeds from WVS7 and context seeds such as room type and time of day are used to increase household-scenario diversity.The persona attributes include country, household composition, age, residence, health, employment, and occupation.
  • Scenario generation: Scenario generation produces realistic household situations in which multiple plausible actions are possible and identifies the robot task, intervention moment, and affected stakeholders.The intervention moment is the exact point at which the robot must make a decision.
  • Candidate action generation: Each scenario initially receives 17 feasible candidate actions, with each action designed to prioritize a different value seed.The seeds combine eight robot-value categories with ten household robot norms, merging privacy where the sources overlap.
  • Value annotation: Stakeholder reactions record support, opposition, mixed, or neutral stances toward each action and support extracting its prioritized value.This makes the annotation stakeholder-grounded at the action level.
  • Image generation: LLMs generate a visually legible snapshot of the intervention moment, which GPT Image 2 renders as a realistic egocentric household image without visible robot embodiment.The snapshot is intended to preserve the scenario without adding facts, stakeholders, or decision branches.
  • Quality check: Quality checks assess realism, coherence, persona grounding, stakeholder validity, action feasibility, executability, and major safety concerns at successive stages.GPT-5.4-mini applies binary yes/no criteria, retaining samples only when all applicable criteria pass; the filters are audited against two human annotators.

5 Dataset Analysis

ROBOTVALUES retains 10,073 image-grounded household decision instances with stakeholder-grounded action values, including fine-grained descriptions and mappings to established value taxonomies.

  • 10,073 instances are retained from 16,000 candidate scenarios, with 69,134 candidate robot actions and a 63.0% overall acceptance rate.Each instance includes a household image, textual task context, multiple candidate actions, and action-level value annotations.
  • Each robot action receives a fine-grained value annotation derived from first-person stakeholder reactions.Examples include protecting an allergy-sensitive item and supporting elderly independence.
  • The benchmark additionally maps each action-level value to household robot norms and Schwartz’s basic human values for analysis and comparison.

6 Evaluating VLMs

The evaluation measures VLMs’ default value preferences and their ability to select actions aligned with explicit target values. Models favor some values by default and often fail when requested priorities conflict with those preferences.

  • Default preference: Safety and Accommodation consistently receive high default BT scores, while Privacy and Security receive lower scores across multiple models.Higher BT scores indicate that actions in a category are selected more often without an explicit target value.
  • Default preference: Across input ablations, safety remains highest-scoring while privacy and security remain among the lowest-scoring categories.Exact BT scores and secondary categories vary across textual, visual, and candidate-action-only inputs.
  • Value-conditioned setting: Value-conditioned accuracy is 40.2%–51.3% for Matched norms but only 6.9%–16.8% when target norms conflict with default preferences.The reported grouping compares targets that match, tie with, or conflict with the model’s default-selected norm.
  • Analysis: Action-value matching performs better than value-conditioned action selection, with Matched–Conflicting gaps of 7.8%–16.9% versus 30.1%–40.9%.This pattern indicates that low conditioned-selection accuracy is not explained only by failing to identify an action’s prioritized value.

7 Conclusion

The paper introduces ROBOTVALUES to evaluate household robot planners in value-conflict scenarios. Its conclusion is that robot evaluation should assess value-based action selection beyond task completion.

  • ROBOTVALUES is a benchmark for evaluating household robot planners in value-conflict scenarios.
  • The results indicate that household robot evaluation should move beyond task completion and assess choices among actions prioritizing different human values.

8 Limitations

ROBOTVALUES relies on synthetically generated household images and LLM-based large-scale data generation, limiting direct representation of real-home complexity and potentially leaving artifacts or annotation errors.

  • Synthetic images may not fully capture the visual complexity, sensing noise, or interaction dynamics of real homes.
  • LLM-based large-scale generation may leave artifacts or annotation errors despite stage-wise filtering and quality control.

A Dataset Construction Details

ROBOTVALUES constructs persona and context seeds to generate diverse, realistic household scenarios. Persona seeds use WVS7 respondent attributes, while room and time-of-day assignments broaden household coverage.

  • Persona seeds: The pipeline uses one WVS7 respondent as the persona seed because full synthetic household rosters produced unnatural scenarios in pilot generations.Roster conditioning sometimes invented household members or changed their ages.
  • Persona seeds: WVS7 persona seeds encode country, household, demographic, health, employment, and occupation attributes for scenario generation.After removing respondents with missing required fields, 90,313 of 97,220 respondents remained.
  • Persona seeds: Survey weights are used only to sample realistic and demographically diverse persona seeds, not to compute benchmark estimates.Weighted priority sampling gives respondents with larger weights a higher selection probability without duplicate selection.
  • Context seeds: Ten room categories and five time-of-day categories are assigned round-robin to improve coverage across household locations and lighting conditions.The room categories include kitchen, living room, bedroom, bathroom, study area, and storage area.
  • Value seeds: Candidate-action generation uses value seeds from the HRI value compass, household robot norms, and Schwartz’s basic-values theory.These sources provide the value definitions used to construct candidate actions and analyze annotations.

A.3 Dataset Statistics

ROBOTVALUES analyzes the distribution of action-level annotations and robot task categories across its household scenarios. The benchmark also reports stage-wise retention throughout construction.

  • Value annotations: Action-level annotations are summarized under both the household robot norm taxonomy and Schwartz’s human value taxonomy.These distributions are reported in Tables 8 and 9.
  • Filtering statistics: Stage-wise retention rates document how many samples remain after each quality-control stage in the construction pipeline.Table 10 reports retention rates for every quality-check stage.
  • Robot task diversity: ROBOTVALUES assigns each action to up to two of eight robot task types from an HRI taxonomy.The task types are information exchange, precision, physical load reduction, transport, manipulation, cognitive stimulation, emotional stimulation, and physical stimulation.

A.4 Textual Task Context Generation

The textual task context is generated from scenario and annotation information into structured fields that separate visible observations from non-visual household context. Multiple language and image-generation models support this process.

  • Context fields: GPT-5-mini generates four context fields: robot task, visible state, decision context, and non-visual context.The prompt uses scenario descriptions, metadata, action-level values, stakeholder stances, and snapshot fields.
  • Context fields: The context separates visible scene information from non-visual household information needed to interpret the robot’s decision.This distinction captures information that cannot be inferred from the image alone.
  • Reference taxonomies: The benchmark draws value definitions from the HRI value compass, household robot norms, Schwartz values, and a robot-task taxonomy.Tables 4, 5, 6, and 7 provide the corresponding values and definitions.
  • Generation models: Text generation uses DeepSeek, GPT-5-mini, GPT-OSS-120B, and Qwen3 models, while images are generated with GPT Image 2 at 1280 × 720 resolution.Several models are accessed through OpenRouter or the OpenAI API, depending on the model.

B Adaptation and Real-Camera Observation Pilots

The pilots examine supervised adaptation and transfer to robot-mounted camera observations, while the quality-control pipeline filters and audits generated data. Results suggest improved real-camera performance, but the two-image transfer evaluation remains preliminary.

  • Adaptation: 44.0%, 51.7%, and 60.9% accuracy are achieved by fine-tuned Qwen3-VL-2B on the Matched, Default tie, and Conflicting groups, respectively.Fine-tuning uses 11,942 value-conditioned examples for one epoch and evaluates on a held-out split.
  • Real-camera pilot: 42.9% accuracy on two real-camera images matches Qwen3-VL-8B and exceeds the non-fine-tuned Qwen3-VL-2B baseline at 21.4%.The authors characterize this as preliminary evidence because the real-camera pilot contains only two images.
  • Quality control: The construction pipeline uses LLM judges because fully manual review was prohibitively time-consuming.Binary judge decisions are audited against consensus annotations from two human annotators.
  • Quality-control rubrics: Quality checks assess scenario realism and coherence, candidate-action plausibility and feasibility, stakeholder materiality, and value-annotation grounding.Action checks also test safety and non-recklessness, while scenario checks include persona fidelity and placement explanations.

D Evaluation Details

The evaluation uses robotics-oriented VLMs and Bradley–Terry scores to measure their default preferences among value categories. It also examines value-conditioned action selection using fine-grained stakeholder-grounded values.

  • The study evaluates robotics-oriented VLMs, including Qwen3-VL, Cosmos-Reason2, Molmo2, RoboBrain2.0, InternVL, and RLDX-1-VLM.
  • Bradley–Terry Scores: Bradley–Terry scores summarize models’ default value preferences from pairwise comparisons between selected and unselected action categories.Pairs mapped to the same value category are excluded, and symmetric pseudocounts stabilize sparse comparisons.
  • Bradley–Terry Scores: A larger centered log-worth score indicates a stronger default preference for actions associated with that value category.
  • Fine-Grained Stakeholder-Grounded Target Values: Fine-grained stakeholder-grounded target values yield higher Conflicting-group accuracy than the coarser household robot norm taxonomy.The comparison is reported in Table 16’s value-conditioned action-selection results.

E.2 Text and Image Ablation

The ablation tests whether default value preferences depend on textual or visual input by comparing four input settings. The broad preference pattern remains stable, although exact scores and secondary rankings vary across modalities.

  • Input Settings: The ablation compares full text and image, text only, image only, and candidate-actions-only input settings.The image-only setting removes compact textual context, while the actions-only setting shows only candidate actions.
  • Results: Safety remains among the two highest-scoring categories in every input setting and is highest for most models.
  • Results: Privacy and Security remain among the lowest-scoring categories in most input settings.
  • Results: Exact Bradley–Terry scores and second-ranked categories change across modalities, while the broad preference pattern remains consistent.The results suggest that visual and textual context affect preference strength and ordering without changing the main tendency.
Loading 2606.03312v1…