RobotEQ 3.0: Towards Personalized Social Proactive Intelligence in Embodied Agents

1 INTRODUCTION

• Earlier SPI work centers on average-user judgments, overlooking individual differences in preferences for equally valid actions.For example, extroverted people may prefer more proactive assistance while introverted people may favor quieter support.Source: §1 · p. 1
• RobotEQ 3.0 introduces a benchmark linking structured participant profiles to preferred actions, shifting SPI analysis from population-level to individual-level preferences.The 78-item questionnaire covers user information, robot experience, trust, personality, autonomy, and empathy; the dataset includes 10 participants across 1,800 scenarios and 18K annotations.Source: §1 · p. 2
• The paper benchmarks models with user profiles as inputs and proposes iterative profile-aware experience induction to improve personalized prediction.The method summarizes human experience and dynamically retrieves relevant knowledge for personalized SPI.Source: §1 · p. 2

2.1 SOCIAL PROACTIVE INTELLIGENCE

• Prior SPI work develops socially desirable behavior benchmarks, including extensions from static images to videos, but typically labels actions by majority vote.That consensus framing overlooks that people can prefer different actions in the same scenario, motivating Personalized SPI.Source: §2.1 · p. 2

2.2 PERSONAL CHARACTERISTIC MODELING

• The paper considers six profile dimensions—basic information, robot experience, robot trust, personality, autonomy, and empathy—as potentially related to embodied-agent preferences.The authors note a lack of systematic analysis of which profile characteristics correlate with these preferences.Source: §2.2 · p. 3

3 DATASET CONSTRUCTION

• The dataset construction pipeline generates candidate actions, designs a preference-related questionnaire, profiles participants, and records their preferred actions per scenario.The resulting annotations explicitly link profiles with decision-making in SPI.Source: §3 · p. 3

3.1 ACTION CONSTRUCTION

• Candidate actions are made semantically distinct and comparable in granularity, then filtered for observability, executability, and social reasonableness.Construction decomposes multi-step actions, expands atomic actions into at least five behaviors, removes semantic duplicates, and simplifies subsumption or multi-step relations.Source: §3.1 · p. 3

3.2 QUESTIONNAIRE CONSTRUCTION

• The structured questionnaire measures six dimensions: basic information, personality, robot experience, robot trust, autonomy, and empathy.It uses the Big Five model, established robot-trust items, two IAF autonomy subscales, and two IRI empathy subscales.Source: §3.2 · p. 4

3.3 PARTICIPANT RECRUITMENT AND PROFILING

• Researchers recruited 20 participants and selected the 10 whose questionnaire responses showed the greatest divergence as annotators.Profiles include basic information, robot experience and trust, Big Five traits, autonomy, and empathy.Source: §3.3 · p. 4

3.4 PERSONALIZED ANNOTATION

• All participants choose from the same five actions for each scenario, selecting multiple actions when several are equally preferred.Selections are represented as sets and paired with each participant’s profile; repeated scenarios help assess participant reliability.Source: §3.4 · p. 4

3.5 DATASET STATISTICS

• The dataset contains 1,800 scenarios annotated by 10 participants, yielding 18K annotations and an average of 2.24 selected actions per scenario.Its 78-item questionnaire covers six dimensions, and the profile heatmap shows distinct trait combinations across participants.Source: §3.5 · pp. 4–5, Figure 3 · p. 4

4.1 EVALUATION METRICS

• Preference Alignment Score averages set-level F-score and HIT, then weights participant scores by reliability estimated from repeated annotations.F-score measures full-set matching; HIT measures whether at least one valid choice is identified.Source: §4.1 · p. 5

4.2 BENCHMARKING CANDIDATES

• The benchmark evaluates representative open-source vision-language models with distinct personality profiles as conditional inputs.Closed-source models are left for future work because large-scale personalized evaluation is costly.Source: §4.2 · p. 5

5.1 PROFILE-CONDITIONED EVALUATION

• Profile conditioning improves overall PAS across models, with gains varying by participant; profiles provide user-specific cues beyond a single consensus answer.Qwen3.5-4B-Instruct has the largest relative gains, while Qwen3.5-27B-Instruct achieves the highest absolute profile-conditioned performance.Source: §5.1 · pp. 5–6, Table 1 · p. 6

5.2 ABLATION STUDY ON DIFFERENT INPUTS

• Scenario images and few-shot prompting each improve preference prediction; the demonstrations use 20 question–answer pairs from each participant’s annotations.The image ablation evaluates models under their optimal profile setting.Source: §5.2 · pp. 6–7, Figure 4 · p. 7

5.3 RELATIONSHIP BETWEEN TRAIT SIMILARITY AND PREFERENCE CONSISTENCY

• Trait similarity has positive associations with preference consistency for Robot Trust and Openness, but negative associations for Conscientiousness and Extraversion.The mixed directions motivate leveraging profile dimensions adaptively.Source: Figure 5 · p. 7, §5.3 · p. 7

5.4 TRAIT-LEVEL ANALYSIS

• No profile dimension consistently dominates across backbones, and using the complete profile is not uniformly optimal because added information may introduce noise.Profile selection depends on both the model and participant.Source: §5.4 · pp. 7–8, Figure 6 · p. 8

5.5 SOLUTION EXPLORATION: ITERATIVE PROFILE-AWARE EXPERIENCE INDUCTION

• Iterative profile-aware experience induction improves prediction by 1.38 to 8.94 percentage points across models, with the largest improvement for Qwen3.5-9B-Instruct.The method reviews prediction failures, distills profile-aware experience, and transfers it to remaining scenarios without model retraining.Source: Table 2 · p. 8, §5.5 · pp. 8–9

5.6 CASE STUDY

• In the wheelchair-tourist case, conditioning on P10’s profile shifts predictions from generic tour guidance toward actions matching preferences for safety, accessibility, and consent-based assistance.Without profile information, most models predict A5; the full profile shifts predictions toward P10’s preferred actions.Source: Figure 8 · p. 9, §5.6 · p. 9

6 CONCLUSION

• Profiles provide useful signals for personalized preference prediction, but their contributions vary across participant-model combinations and the complete profile is not uniformly optimal.The proposed self-evolution-driven method extracts profile-related experience that complements static profiles and improves personalized behavior prediction.Source: §6 · p. 9
• The benchmark links profiles to preferred actions across 1,800 scenarios, 10 participants, and 18K individualized preference annotations.The paper frames this as a shift from population-level modeling toward individual-level analysis.Source: §6 · p. 9
• Evaluation is limited to ten participants, mainly Chinese-culture participants, structured-profile factors, and open-source multimodal models.The authors identify broader participant coverage, additional user dimensions, cross-cultural effects, and closed-source models as future work.Source: §6 · p. 9

ETHICS STATEMENT

• All participants consented to use of their annotation results and received approximately ¥150 per hour, above local standards.The dataset is released under CC BY-NC 4.0, restricting use to non-commercial purposes and providing responsible-use guidelines.Source: ETHICS STATEMENT · p. 10
• The benchmark includes prosocial robot-service scenarios and excludes violent, discriminatory, or otherwise harmful content.Source: ETHICS STATEMENT · p. 10
• Benchmark results are not evidence of real-world deployment readiness; further validation is required before deployment in human environments.Source: ETHICS STATEMENT · p. 10

A CANDIDATE ACTION CONSTRUCTION

• Candidate actions are decomposed, expanded, deduplicated, simplified, and validity-filtered for observability, executability, and social reasonableness.The construction process uses Qwen3.7-Plus; filtering discards candidates failing any criterion.Source: A CANDIDATE ACTION CONSTRUCTION · p. 14, Figure 9 · p. 15

B USER PROFILE QUESTIONNAIRE AND SCORING

• The 78-item questionnaire covers basic information, robot experience and trust, Big Five personality, autonomy, and empathy, with item scoring rules provided alongside the questionnaire.Source: B USER PROFILE QUESTIONNAIRE AND SCORING · p. 14

B.1 BASIC USER PROFILE

• Basic demographic and living-context responses are retained as categorical or numerical variables without aggregation, serving as contextual profile cues.Source: §B.1 · p. 14, Table 3 · p. 14

B.2 ROBOT EXPERIENCE

• Robot-experience responses capture product diversity, use frequency, knowledge, acceptance, and prior autonomous-AI use, and are used directly as profile features.Source: §B.2 · p. 14, Table 4 · p. 16

B.3 ROBOT TRUST

• Robot trust is rated on seven-point Likert items and scored by averaging four responses; higher scores indicate greater trust.Source: §B.3 · p. 16, Table 5 · p. 16

B.4 BIG FIVE PERSONALITY

• The 44-item Big Five Inventory characterizes Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism using five-point responses.Chinese wording was naturalized while preserving item semantics, factor assignments, and scoring directions.Source: §B.4 · p. 16, Table 6 · p. 16

B.5 AUTONOMY

• Autonomy uses Self-Congruence and Low Susceptibility to Control items, reverse-scores Q62 and Q63, and interprets higher scores as greater self-congruence and less susceptibility to external control.Source: §B.5 · p. 18, Table 7 · p. 18

B.6 EMPATHY; C ANNOTATION INTERFACE

• Empathy uses Perspective Taking and Empathic Concern subscales on a 0–4 response scale, with negatively keyed items reverse-scored.Annotators see the scene, question, and candidate actions, then select one or more appropriate actions; at least one choice is required.Source: §B.6 · pp. 18–19, Table 8 · p. 18, C ANNOTATION INTERFACE · p. 19

D PARTICIPANT ANNOTATION CONSISTENCY

• Annotation quality is assessed through within-participant stability across repeated annotations, preserving genuine differences between participants as preference variation.Source: D PARTICIPANT ANNOTATION CONSISTENCY · p. 19
• The study repeats the same 20 randomly sampled scenarios for every participant, keeping scene, question, and candidate actions unchanged across rounds.Source: D PARTICIPANT ANNOTATION CONSISTENCY · p. 20
• An action-level decision agrees when its binary selection status is unchanged between the two annotations; participant consistency is the proportion of such unchanged decisions.The consistency measure is averaged across repeated scenarios and candidate actions.Source: D PARTICIPANT ANNOTATION CONSISTENCY · p. 20
• Consistency weights affect only cross-participant aggregation of F-score, HIT, and PAS, giving more stable annotators greater aggregate influence without changing participant-level scores.Source: D PARTICIPANT ANNOTATION CONSISTENCY · pp. 20–21

• The structured prompt combines the participant’s personality profile, scenario description, scene image, and candidate action list.Figure 11 shows the complete prompt template.Source: E BENCHMARKING PROMPTS · p. 21, Figure 11 · p. 21
• The model selects the action the participant would prefer, guided by their profile rather than general usefulness alone.It outputs the chosen action’s index and briefly explains how traits influence the decision.Source: E BENCHMARKING PROMPTS · p. 21

F PARTICIPANT-LEVEL PERFORMANCE ACROSS PROFILE SETTINGS

• Tables 10–19 report participant-level PAS (%) for evaluated models under No Profile and profile conditions; Best Profile is the highest PAS among nine individual dimensions and the complete profile.The dimensions include robot trust, Big Five traits, autonomy, perspective taking, and empathic concern.Source: F PARTICIPANT-LEVEL PERFORMANCE ACROSS… · p. 21

G DETAILS OF EVALUATED MODELS

• Table 20 documents the evaluated models and their public checkpoints.Source: G DETAILS OF EVALUATED MODELS · p. 25, Table 20 · p. 25

H FEW-SHOT DEMONSTRATIONS

• The few-shot setup uses 20 fixed demonstration scenarios shared across participants, with participant-specific preference labels.Each demonstration contains a scene image, robot role, and five candidate actions.Source: H FEW-SHOT DEMONSTRATIONS · p. 25, Figure 12 · p. 26

I.1 SELF-EVOLUTION BACKGROUND AND MOTIVATION

• The authors choose textual experience generation because its participant- and model-specific guidance is inspectable, avoids retraining, and suits personalized SPI better than weight or topology evolution.The approach is situated among self-evolution methods that update experience, models, memory, prompts, tools, or architectures.Source: §I.1 · pp. 26–27

• For each participant–model pair, DeepSeek-V4-Flash analyzes 100 support scenarios and revises experience over five rounds.The support cases include scenario descriptions, candidate actions, the participant’s full profile, model predictions, and human preferences.Source: §I.2 · p. 27
• The final experience is concatenated with the participant profile for remaining scenarios, without directly rewriting candidate actions.An example advises against overselecting assertive approaches for a low-extraversion, high-trust participant when greeting proactively.Source: §I.2 · pp. 27–28, Figure 14 · p. 28
• Induced rules summarize recurring errors, trait combinations associated with mistakes, and repeatedly missed preferred actions.Each concise rule specifies trigger conditions, faulty behavior, and corrected preference guidance.Source: §I.2 · p. 27