SIMGUIDE: Procedurally Grounded Multi-Context Representations for Personalized Agent Planning

Chirag Shah

Source-linked AI summary

SIMGUIDE: Procedurally Grounded Multi-Context Representations for Personalized Agent Planning

Chirag Shah

arXiv:2608.24888v1May 14, 2026cs.AI

TL;DR

Personalized agents often flatten users into one profile, leaving them poorly equipped to plan when priorities differ across active life contexts. SIMGUIDE structures context into typed Sims, grounds constraints with procedural examples, and evaluates them with the 47-task SIMBENCH suite. Procedurally grounded Sims outperform RAG, while task-matched and Sim-type-routed adaptation improve parametric personalization.

Problem
Personalized agents lack benchmarked methods for representing and planning across multiple potentially conflicting user contexts rather than treating users as undifferentiated profiles.
Method
SIMGUIDE represents each life context as a typed Sim with constraints and priorities, grounds constraints using worked decisions, and evaluates planning with SIMBENCH.
Results
+7.9 Preference Adherence points over RAG on GPT-4o shows that procedurally grounded Sims outperform retrieval, with the advantage replicated across 100 τ-bench tasks and models.
Takeaways & Limitations
Representation format, training distribution, and adaptation granularity are first-order design variables for personalized AI systems.
Takeaways & Limitations
SIMBENCH, scenarios, procedural examples, and LoRA training data are synthetic, and validation on real behavioral data remains future work.

Abstract

from arXiv · show

Personalized AI agents overwhelmingly treat users as single entities: a flat profile concatenated into a prompt. This fails when the same person holds different priorities across life contexts -- and fails catastrophically when those priorities conflict. The core problem is not that agents lack information about users; it is that the format of user representations determines whether an agent can act on that information at all. We introduce SIMGUIDE, a method that structures user context into typed, domain-specific blocks called Sims and grounds each constraint with procedural examples drawn from past decisions. To evaluate this, we construct SIMBENCH, a diagnostic suite of 47 preference-conditioned planning tasks where the correct plan depends on which user context is active -- a property no existing benchmark tests. Declarative Sim constraints alone do not outperform retrieval-based personalization (RAG). Procedurally grounded Sims outperform RAG on GPT-4o (+7.9 Preference Adherence points, $p = 0.013$), and this advantage replicates on 100 $τ$-bench tasks across both GPT-4o and Claude Sonnet~4.5 ($p \leq 0.023$). At the parametric level, the same principle holds: training distribution dominates whether parametric adaptation succeeds at all. Task-matched LoRA fine-tuning improves generation quality by 12.8 ROUGE-L points over the unadapted base model, and routing adapters by Sim type rather than user identity adds a further 7.3 points, robust to 28% routing error. Representation format -- not representation content -- is the first-order design variable.

1 Introduction

SIMGUIDE addresses conflicting life-context preferences by structuring user context into Sims and grounding constraints with procedural examples. SIMBENCH evaluates whether the active context changes the correct plan, while results identify representation format as a first-order design variable.

Motivation: Flat user profiles can cause context interference and leave agents without a principled basis for resolving conflicting preferences.The paper frames these as failures of undifferentiated representations across multiple life domains.
Method: Declarative Sims do not consistently outperform RAG, whereas procedural grounding supplies worked examples that make constraints operational.The examples provide behavioral signal rather than only abstract rules.
Evaluation: SIMBENCH contains 47 tasks across four domains where the correct plan depends on which Sim is active.The benchmark targets a property that existing benchmarks do not test and that a No-Profile agent cannot recover from the task description alone.
Results: +7.9 Preference Adherence points over RAG on GPT-4o demonstrates the benefit of procedurally grounded Sims.The comparison is reported with p = 0.013 and holds across all difficulty levels.
Conclusion: Representation format and training distribution are first-order design variables for personalized AI systems.The conclusion extends the design question beyond what user information is represented to how it is structured and trained on.

2 Related Work

Prior personalization and agent benchmarks use retrieved histories, user-level adaptation, or single-context evaluation. SIMGUIDE instead evaluates typed multi-context representations and context-dependent adaptation for conflict-aware personalized planning.

Memory and personalization: RAG grounds responses in retrieved documents but does not represent which user context the retrieved decisions belong to.Related personalization benchmarks such as LaMP and PersonalLLM address historical-user personalization but not this multi-context representation issue.
Parametric personalization: Prior parametric methods condition adaptation on unstructured user-history embeddings and assume behavioral training data exists in the evaluation domain.The paper contrasts this setup with typed, multi-context representations and Sim-type adapter selection.
Agent benchmarks: Existing agent benchmarks evaluate task completion, environments, or policy compliance but do not test planning under persistent, conflicting user preferences.SIMBENCH is designed around this missing evaluation property.
Sims architecture: Sims are structured representations of user preferences across life contexts, designed for personalized planning rather than social simulation.The present paper adds empirical evaluation and procedural grounding to the Sims framework.

3 Method

SIMGUIDE structures user context into typed Sims with procedural grounding and uses arbitration to produce plans that satisfy hard constraints while resolving soft conflicts. SIMBENCH evaluates context-dependent planning across domains, alongside transfer, personalization, and parametric adaptation experiments.

Sim Representation: A Sim is a typed representation of a user in a specific life context, containing priorities, constraints, communication style, and procedural examples.Constraints are labeled HARD or SOFT; hard constraints cannot be violated, while soft constraints may be traded off with explicit justification.
Procedural Grounding: Procedural grounding attaches one or two worked examples to each constraint, showing how it was applied in a concrete past decision.The examples provide behavioral patterns and encode reasoning about recurring tradeoffs beyond the exact constraint.
Sim-Guided Agent Planning: A valid plan satisfies every hard constraint across active Sims and arbitrates conflicting soft preferences according to Sim priority.The formalization treats arbitration as constrained optimization, with hard constraints imposing inviolable feasibility conditions before soft priorities are optimized.
Evaluation Setup: The evaluation compares user-context formats under identical tasks and tools, and measures plan correctness, preference adherence, and conflict-resolution accuracy.A 100-task τ-bench subset tests transfer, while Amazon Product Reviews provides a domain-mismatched training set for parametric personalization.
SIMBENCH: SIMBENCH contains 47 tasks across four domains where the correct plan depends on the active user context and cannot be recovered from the task description alone.The suite targets priority, constraint, temporal, and communication conflicts across canonical users with multiple Sims.
Task-Matched LoRA Fine-tuning: The parametric study compares domain-mismatched and task-matched LoRA adapters trained on different data distributions.Task-matched training uses synthetic SIMBENCH-format examples containing Sim schemas, constraints, multi-step plans, and conflict-resolution rationales.

4 Experiments

The experiments evaluate SIMGUIDE across prompt-based and parametric settings, including SIMBENCH, τ-bench validation, and LoRA adaptation. Results show that procedural grounding, task-matched training data, and Sim-type routing improve personalization, with model- and context-specific limitations.

Experimental Setup: 47 SIMBENCH tasks, six conditions, and four models yield 1,128 main-experiment evaluations; τ-bench validation adds 1,200 evaluations across 100 tasks, six conditions, and two models.Comparisons use paired bootstrap confidence intervals with 5,000 resamples, reporting 95% CIs and one-sided p-values.
SIMBENCH Results: Sim+G scores 0.625 versus RAG at 0.546 on GPT-4o, a 7.9-point Preference Adherence gain (p = 0.013).Declarative Sim scores 0.547 and is essentially tied with RAG; the Sim+G advantage holds across difficulty levels.
SIMBENCH Results: Sim-A scores 0.496 on GPT-4o, below Sim at 0.547 and RAG at 0.546, while Claude shows a +0.092-point Sim-versus-Sim-A arbitration benefit (p = 0.001).The results identify explicit arbitration as a reasoning scaffold rather than treating structure alone as sufficient.
Task-Matched LoRA Fine-tuning: Synthetic LoRA improves ROUGE-L by 12.8 points over the base model, whereas Amazon LoRA improves by only 0.4 points under the same adapter architecture.The training signals differ: synthetic data contains preference-conditioned multi-step plans with structured constraint reasoning, while Amazon data contains review-style opinion text.
Task-Matched LoRA Fine-tuning: Sim-type routing reaches ROUGE-L 0.3061, 7.3 points above the generic synthetic adapter and 9.1 points above per-user LoRA, while retaining essentially all oracle-routing benefit at 72% accuracy.Per-user adapters trained on approximately 40 examples average ROUGE-L 0.2087, below the generic synthetic adapter’s 0.2334.

5 Analysis

The analysis finds that procedural grounding, rather than declarative structure alone, drives personalization gains, although benefits vary by constraint and model. It also identifies concrete failure modes and evaluates example quantity and quality.

Declarative-procedural gap: Grounding helps overall because worked examples provide behavioral signal that declarative rules lack, while cross-user examples trend positive and the total grounding effect is significant (p = 0.001).The cross-user-example comparison trends positive over no examples (p = 0.085), but the mechanisms cannot be fully separated at n = 35.
Grounding Necessity Index: The Grounding Necessity Index measures each binding constraint’s mean Preference Adherence difference between Sim+G and Sim.Positive values indicate grounding helps; negative values indicate neutral or harmful effects, with reporting restricted to constraints appearing in at least three tasks.
Grounding Necessity Index: 0.146 mean GNI for scheduling-parameter constraints exceeds 0.066 for categorical-process constraints, though not significantly (p = 0.199).For GPT-4o, business_venues_-only and secure_delivery show positive GNI, while Claude shows negative GNI on those constraints.
Cross-model effects: Claude shows negative GNI where GPT-4o shows positive GNI for business_venues_-only (+0.060 versus −0.236) and secure_delivery (+0.071 versus −0.167).The analysis attributes this cross-model pattern to examples anchoring frontier models to specific scenarios rather than generalizing constraint structure.
Failure modes: Sim+G underperforms RAG under constraint ambiguity, multi-Sim interference, and counterintuitive parameters.These cases involve multiple valid instantiations, lower-priority examples overriding arbitration, or retrieved decisions supplying behavioral evidence that procedural instruction cannot substitute.
Ablations: One example per hard constraint produces a large, significant gain over no examples in the quantity ablation (+10.5 points, p =The supplied passage truncates the p-value and does not provide the complete ablation result.

6 Limitations

The paper’s evaluation is limited by synthetic data, modest benchmark scale, and judge-related measurement concerns. These constraints bound how broadly the reported results should be interpreted.

Data and scale: SIMBENCH has 47 tasks across 9 synthetic users, limiting statistical power for subgroup analyses.All scenarios, procedural examples, and LoRA training data are synthetic.
Data and scale: Validation on real behavioral data remains future work.High BERTScore values of at least 0.91 reflect format consistency between synthetic sets rather than real-world plan quality.
Evaluation: Preference Adherence relies on cross-model LLM judging, which mitigates but does not eliminate self-evaluation bias.Gold plan annotations and formal inter-annotator agreement are still in progress.

7 Conclusion

The paper argues that representation format is the first-order design variable in personalized AI, across both prompt-level and parametric adaptation. It presents procedural grounding and task-matched training as the key conditions for effective personalization.

Conclusion: Representation format is more consequential than representation content or adapter architecture in personalized AI systems.The claim is made at both prompt and parametric levels.
Conclusion: Declarative Sims fail to consistently beat RAG, whereas procedurally grounded Sims provide behavioral signal and improve GPT-4o Preference Adherence by +7.9 points over RAG (p = 0.013).The advantage replicates on τ-bench across GPT-4o and Claude and holds across difficulty levels.
Conclusion: At the parametric level, domain-mismatched adapters contribute almost nothing (+0.4 ROUGE-L), while task-matched training contributes 12.8 points.Sim-type routing is presented as an equilibrium because per-user adaptation overfits at realistic data scales.
Experimental conditions: The experimental conditions vary in whether they use flat profiles, structured Sims, arbitration, or grounded procedural examples.The prompt templates distinguish No-Profile, Flat, Sim-A, Sim, and Sim+G conditions through their user-context fields and conflict-handling instructions.

B Difficulty-Stratified Results

Difficulty-stratified evaluation shows that Sim+G dominates across analyzed difficulty levels, while RAG can underperform No-Profile on low-difficulty tasks and hard-task analysis is underpowered.

Difficulty-stratified results: Sim+G dominates across all analyzed difficulty levels, while RAG underperforms No-Profile on low-difficulty tasks.Hard tasks are excluded from stratified analysis because n = 3.

C Failure Mode Analysis

Per-task analysis identifies three failure categories: ambiguous constraints, interference between simultaneously active Sims, and strongly counterintuitive parameters.

Constraint ambiguity hurts Sim+G when categorical choices allow multiple valid instantiations.RAG can retrieve a prior decision that directly resolves ambiguity, while procedural examples may not specify the resolution when categories shift.
Multi-Sim interference occurs when overlapping active Sims cause examples from a lower-priority Sim to be applied first.The analysis highlights work-and-caregiving tasks where caregiving hard constraints should dominate.
RAG retains a structural advantage when the correct parameter is strongly counterintuitive.Retrieved past decisions can provide direct behavioral evidence that declarative or procedural instructions may not fully supply.

D SIMBENCH Task Catalog

SIMBENCH is cataloged as a 47-task suite spanning multiple domains, users, difficulties, and conflict types, with canonical users represented through typed Sims.

Task Catalog: SIMBENCH contains 47 tasks designed around conflicts between user contexts.The catalog identifies conflict types including constraint, temporal, priority, and communication conflicts.
Conflict Types: The catalog’s conflict abbreviations are CC for constraint_conflict, TC for temporal_conflict, PC for priority_conflict, and CoC for communication_conflict.
Task Catalog: The task catalog records task ID, domain, user, difficulty, and conflict types for each evaluation item.Listed examples include calendar, communication, and research tasks across low, medium, high, and hard difficulties.
Canonical Profiles: Canonical user profiles assign multiple typed Sims to users such as Emily Zhao, Noah Williams, Rafael Silva, and Diana Osei.The listed Sim types include work, family, personal, hobby, finance, values, health, and connectivity contexts.

F Full Results Tables

The full-results tables report plan correctness, conflict-resolution accuracy, parse rates, interference scores, and paired bootstrap confidence intervals across SIMBENCH and τ-bench.

Evaluation Metrics: Plan Correctness and Conflict Resolution Accuracy are tabulated across conditions and models.
Evaluation Metrics: Parse Rate reports successfully parsed tasks out of 47 for each condition and model.
Evaluation Metrics: Interference Score is reported as 1.00 across all conditions and models.The table notes that this may reflect insufficient task diversity or judge insensitivity to subtle leakage.
Statistical Analysis: Paired bootstrap confidence intervals use 5,000 resamples for key Preference Adherence comparisons on 47-task SIMBENCH and 100-task τ-bench evaluations.Parenthetical sample sizes indicate paired counts below the nominal task totals when parse-rate dropouts occur.

H Per-User LoRA Results

The per-user LoRA results compare personalized adapters with a generic synthetic-LoRA baseline, alongside the evaluation and judge procedures used to assess generated plans.

Per-User LoRA Results: Per-user LoRA results are reported relative to a generic synthetic LoRA baseline.The experiments use ten synthetic users, including U10, which is exclusive to parametric adaptation and absent from the nine-user SIMBENCH roster.
Judge Validation: GPT-4o is the default judge, with Claude Sonnet 4.5 and GPT-4o used for cross-model judging in ablation experiments.A separate table reports cross-model judge agreement status.
Plan Evaluation: The evaluation judge assesses plan awareness and adherence to active Sim constraints, priorities, communication style, and contextual needs.It evaluates only the supplied task, active Sim definitions, and plan.
Plan Evaluation: Sim-aware planning signals include explicit constraint references, trade-off handling, conflict resolution, and Sim-aligned tool choices.Generic planning signals include ignoring Sim details, violating constraints without acknowledgment, or using no conflict-resolution strategy when Sims compete.
Metrics: Preference Adherence scores each active constraint as yes, partially, or no with scores of 1.0, 0.5, or 0.0, while Interference Score averages inactive-Sim influence scores.