PaperlayerView exact revision on arXiv

Source-linked AI summary

MIMESIS: Learning User Simulators as Training Environments for Interactive Agents

Hoang Phan, Dat Huynh, Andrey Zhmoginov, Qi Zeng, Wancen Mu, Yue Cao, Shengjie Bi, Yun He, Changdae Oh, Deren Lei

arXiv:2610.09484v1cs.AI

TL;DR

Human interactions are costly to collect, and off-the-shelf assistant LLMs can misrepresent real-user behavior and task difficulty. MIMESIS learns a purpose-built user simulator and uses it for multi-turn agent training, with CSD adding coaching from simulator feedback. Across eight environments, MIMESIS-trained agents outperform GPT-5.5-trained agents under all nine unseen evaluation user models.

  • Problem

    Human user trajectories are costly and difficult to scale, while off-the-shelf assistant LLMs can produce overly cooperative interactions that distort agent-training and evaluation difficulty.

    Source: §1 · p. 2

  • Method

    MIMESIS learns a purpose-built user simulator from human conversations with reasoning supervision and realistic behavior patterns, then supplies multi-turn experience for agent training with optional coaching from simulator feedback.

    Source: §1 · pp. 3–4

  • Results

    Across eight environments, MIMESIS-trained agents outperform GPT-5.5-trained agents under all nine unseen evaluation user models, with CSD yielding further gains across those users.

    Source: §1 · p. 4

  • Takeaways & Limitations

    Simulator quality includes preserving the interaction difficulty induced by human users, not only producing natural-sounding dialogue.

    Source: §1 · p. 2

    Transfer to human users remains untested, and simulators from different model families may share behavioral biases.

    Source: G Limitations, Safety, and Ethics · p. 39

Abstract

from arXiv · show

Training and evaluating interactive language agents typically requires rich user interactions, yet collecting human feedback is expensive and difficult to scale. Simulated users offer a scalable alternative, but they must both resemble real user behavior and provide useful learning experiences for agents. In contrast, most agent-training frameworks rely on off-the-shelf assistant LLMs, whose helpfulness can make them overly cooperative, explicit, and behaviorally homogeneous compared with real users. We introduce MIMESIS, a purpose-built user simulator trained on human conversations with explicit reasoning supervision and 13 realistic behavioral patterns derived from real user interactions. Empirically, our 9B model achieves a SOUL-Index of 65.7, surpassing the strongest frontier model. Compared with Claude-Opus-5, the strongest baseline on RealUserSim and SimulatorArena, MIMESIS improves behavioral fidelity by 13.4 points and reduces Turing distance by 3.6 points, respectively. We then freeze the simulator and train an agent by interacting with the frozen simulator using multi-turn reinforcement learning. Across eight environments, training with MIMESIS yields better agent performance than training with GPT-5.5 under all nine unseen user simulators, demonstrating stronger generalization to new user simulators. Moreover, we propose Coached On-Policy Self-Distillation (CSD), which leverages simulator-generated private reasoning traces and subsequent utterances as feedback on how well the agent addresses user needs. A coach converts this information into concise coaching notes that describe how the agent can better anticipate user needs and adapt its behavior over the course of an interaction. CSD turns this feedback into dense, token-level supervision beyond sparse task rewards, yielding further gains across all nine evaluation user models.

1 Introduction

Interactive agents need realistic multi-turn user interactions, but human trajectories are costly to collect and off-the-shelf assistant models can distort user behavior and task difficulty. The paper connects purpose-built simulator learning with agent post-training, then evaluates simulator fidelity and agent robustness to changed evaluation users.

1 Introduction

  • On τ-bench, fixed GPT-5.5 succeeds with real users 63.6% of the time, versus 82.4–84.4% with frontier API simulators.Every trained simulator deviates from human-user success by at most 10.3 points, while off-the-shelf models deviate by at least 14.9.Source: §1 · p. 2, Figure 2 · p. 2
  • The simulator is trained on human conversations with ThoughtTrace reasoning supervision and 13 realistic behaviors derived from users’ latent thoughts and conversational patterns.The behaviors are instantiated during joint multi-domain reinforcement learning.Source: §1 · pp. 3–4
  • MIMESIS is a purpose-built 4B/9B user simulator combining user-side adaptation, explicit reasoning supervision, and realistic-behavior training.Its 9B model surpasses frontier models on SOUL-Index, RealUserSim, τ-USI, and SimulatorArena, and ablations support the value of reasoning supervision and realistic-behavior training.Source: §1 · pp. 3–4
  • Across eight environments, agents trained with MIMESIS outperform GPT-5.5-trained agents under all nine unseen evaluation user models.CSD adds further gains under all nine evaluation users using simulator-generated thoughts and utterances for coaching.Source: §1 · p. 4
  • CSD uses simulator-generated thoughts and reactions to coach sampled agent responses with dense token-level guidance, while the agent acts only on public dialogue.The learned simulator is frozen and used to generate multi-turn experience for agent reinforcement learning.Source: §1 · pp. 3–4

2 Related Work

Related work spans interactive-agent benchmarks and post-training, attempts to model difficult user behavior, simulator-driven agent learning, and methods that distill feedback or privileged context into policies.

Multi-turn agentic interaction

  • Interactive-agent benchmarks assess multi-round competence, persistent state, tool dependencies, long-horizon execution, and conversational coordination, while post-training optimizes policies over interactive rollouts.Examples include AgentBoard, MINT, ToolSandbox, BFCL, DialogTool, τ-bench, and τ 2-bench.Source: §2 · p. 4

Behavioral variation and difficult users

  • Behavioral-variation work targets cooperative, homogeneous prompted users through difficult interaction patterns, human-like behavior policies, authentic profiles, or explicit control of local intent.The cited work includes Non-Collaborative User Simulators, Persona Policies, RealUserSim, and UserIDA.Source: §2 · p. 4

Simulator-driven agent training

  • Simulator-driven training progresses from rule-grounded LLM-user evaluation and UserRL to learned strategic simulators and evaluations across other simulators and human participants.The cited evaluation by Suh et al. uses 283 human participants.Source: §2 · p. 4

Learning from textual feedback and privileged context

  • Feedback-distillation methods transfer behavior from augmented-context teachers, while CSD re-scores an already sampled agent response rather than decoding a refinement.Related approaches include SDPO, on-policy context distillation, textual critiques or reflections, and Ditto’s feedback-conditioned refined rollouts.Source: §2 · p. 4

3 Problem Setup and Overview

The setup treats user simulation as predicting the next utterance from task context and dialogue history, then uses a trained, frozen simulator to provide turns in agent-training trajectories. Simulator rewards assess user outputs, whereas agent rewards assess task performance.

3 Problem Setup and Overview

  • A user simulator defines a distribution over the next user utterance conditioned on task context and conversation history, and is trained using human dialogues and human-simulation tasks.Simulation tasks provide an input context and an evaluation rule for the generated response or interaction.Source: §3 · p. 5
  • For interactive tasks, the simulator’s generated output is a rollout with the task-specified counterpart, and its parameters are optimized to maximize the simulation objective.For single-turn tasks, the generated output is one response.Source: §3 · p. 5
  • With the trained simulator fixed, agent and simulator utterances, plus tool calls and outcomes when present, accumulate into a trajectory until the task stopping condition or turn limit.The simulator determines how the conversation continues in response to the agent’s actions.Source: §3 · p. 5
  • Agent training maximizes expected task reward over trajectories induced by the agent, frozen simulator, and environment, while keeping simulator parameters fixed.The simulation reward evaluates the simulated user’s output; the agent reward evaluates task performance.Source: §3 · p. 5

4 MIMESIS: Learning a Realistic User Simulator

MIMESIS trains a shared user simulator to model human responses and reasoning while explicitly learning realistic interaction behaviors. It is evaluated for simulation capability and agreement with human interaction trajectories.

4.1 Learning User Responses and Reasoning

  • ThoughtTrace supervision pairs observed user utterances with self-reported reasons and reactions, training a private reasoning trace before the public response.Utterance-only training lacks an explicit reasoning target and may miss the motivation behind a user’s wording.Source: §4.1 · p. 6

4.2 Joint Reinforcement Learning with Realistic Behaviors

  • One shared simulator is jointly reinforced across SOUL environments, with each environment supplying its own interaction protocol and task-specific reward.Unlike training separate experts and distilling selected trajectories, rollouts from every domain update the same parameter set.Source: §4.2 · p. 6

4.3 Learning Realistic User Behaviors

  • MIMESIS explicitly trains on 13 recurring behaviors from real users, including hidden evaluation criteria, shifting goals, and clarification noncooperation.Its ABCD scenario task scores behavior expression, timing, naturalness, and task consistency, while penalizing descriptions of behavior instead of its expression.Source: §4.3 · p. 6

4.4 User-simulator evaluation

  • 65.7 SOUL-Index makes MIMESIS-9B the top overall model, ahead of Claude-Opus-5 (64.9), GPT-5.5 (64.1), and Osim-8B (58.8).MIMESIS-9B also leads on COG and EVAL, while GPT-5.5 leads on SS and Claude-Opus-5 on ROLE.Source: Table 1 · p. 7, §4.4 · p. 7
  • 94.0 Fidelity Index on RealUserSim PT3 puts MIMESIS-9B 13.4 points above Claude-Opus-5 (80.6), with gains concentrated in interaction flow and pacing rather than persona.Persona agreement is near ceiling for several models, including GPT-5.5 (99.3) and MIMESIS-9B (99.4).Source: Figure 4 · p. 7, §4.4 · p. 8

5 Agent Post-Training with a Learned User Simulator

The paper freezes MIMESIS as an environment for multi-turn agent reinforcement learning and adds CSD, which uses simulator feedback to coach sampled responses. It evaluates agent performance across eight environments and nine unseen evaluation users.

5.1 Multi-Turn Reinforcement Learning

  • Multi-turn GRPO samples complete conversations with a frozen simulator and uses task rewards to assign group-normalized advantages to agent responses.User utterances and tool outputs condition the policy but are excluded from its loss.Source: §5.1 · p. 8

5.2 Coached On-Policy Self-Distillation

  • CSD derives retrospective coaching notes from the public history, sampled response, and simulator-generated thought and reaction, then conditions a teacher on that coaching context.The simulator feedback is model-generated, not an observation of a human user’s internal state.Source: §5.2 · p. 8
  • CSD weights sampled tokens more when the coached teacher assigns them higher probability than the uncoached student, complementing the task-reward objective.Tokens with a negative log-probability difference receive weaker reinforcement, not a direct penalty.Source: §5.2 · p. 9

5.3 Agent evaluation

  • Across all nine unseen evaluation users, MIMESIS training raises mean score from 26.10 to 29.54, and CSD raises it further to 31.09.With the same MIMESIS-9B simulator, CSD also achieves higher late-stage training and validation reward than GRPO.Source: Table 2 · p. 9, §5.3 · pp. 9–10, Figure 5 · p. 9

6 Conclusion

MIMESIS combines human-conversation training, explicit reasoning supervision, and 13 realistic behaviors, and its frozen simulator supports agent training that transfers across unseen user models.

6 Conclusion

  • Across eight environments, agents trained with frozen MIMESIS-9B outperform agents trained with GPT-5.5 under all nine unseen user models.CSD adds further gains by turning simulator thoughts and subsequent utterances into retrospective coaching and dense token-level supervision; deployed agents rely only on public dialogue.Source: §6 · p. 10
  • MIMESIS improves both simulator-level benchmarks and transfer across held-out simulator models, supporting evaluation of simulators on fidelity and policy transfer.The conclusion also reports that CSD supplies coaching about user needs and interaction feedback.Source: §6 · p. 10

A Simulator Training Details

  • The simulator-training appendix covers user-side mid-training and ThoughtTrace supervision, behavior-taxonomy construction, and joint RL with a behavior-specific reward.Agent-training settings are reported separately in Appendix D.Source: A Simulator Training Details · p. 13

A.1 Data and supervised adaptation

This block describes MIMESIS’s data and simulator-training pipeline, behavior taxonomy and reward, and evaluation. Reported results include strong SOUL performance and higher trajectory-level fidelity than the strongest baseline on RealUserSim PT3.

A.1 Data and supervised adaptation

  • MIMESIS is initialized through user-side mid-training on 21.2M examples from 62 corpora, processing 9.51B tokens, then fine-tuned on 2,155 ThoughtTrace conversations.ThoughtTrace supplies self-reported motivations and interpretations to supervise a reasoning trace before the public user utterance; the SOUL and ABCD stages follow.Source: §A.1 · pp. 13–14, Table 3 · p. 14, Table 4 · p. 14

A.2 Realistic user behaviors

  • The 13-category behavior taxonomy is derived from ThoughtTrace annotations and consolidated from seed and open-coded labels.Every annotated seed or open-coded label maps to one consolidated category; hidden evaluation criteria are the most frequent annotated behavior at 285 instances (40.7%), not a population-prevalence estimate.Source: §A.2 · pp. 14–16, Table 5 · p. 15, §A.3 · p. 17

A.3 Joint multi-domain reinforcement learning

  • Joint RL updates one simulator on 23 SOUL tasks and the ABCD behavior task, using FoldGRPO statistics over unique rollout identifiers.ABCD adds the 13 behavior categories and a neutral condition to the shared training recipe.Source: §A.3 · p. 17, Table 7 · p. 17

A.4 Behavior rollouts and reward

  • ABCD rollouts assign a behavior or neutral condition, and reward behavior expression, timing, naturalness, and task consistency.The scenarios ground false premises, withheld information, and infeasible requests in recorded facts or policies; task consistency is graded, so high total reward does not guarantee every constraint is satisfied.Source: §A.4 · pp. 17–18

B Simulator Evaluation

  • Simulator evaluation separates broad capability, trajectory-level fidelity, calibration to human task outcomes, and next-turn realism.Proprietary models use fixed reasoning configurations without additional model-specific tuning, and those configurations do not imply equal inference budgets across providers.Source: B Simulator Evaluation · pp. 18–19

B.1 SOUL-Index

  • 65.7 is MIMESIS-9B’s overall SOUL score, above Claude-Opus-5 at 64.9; both MIMESIS models exceed frontier baselines on all four conversational datasets.SOUL covers conversational interaction, social simulation, cognition, role play, and evaluation or judgment.Source: §B.1 · p. 19, Table 8 · p. 19

B.2 RealUserSim PT3

  • 94.0 is MIMESIS-9B’s RealUserSim PT3 Fidelity Index, 13.4 points above Claude-Opus-5, the strongest baseline.MIMESIS-9B also has the highest reported mean on each of the five behavioral dimensions.Source: §B.2 · p. 19, Table 9 · p. 20
  • The RealUserSim Fidelity Index averages five trajectory-level dimensions, including interaction and information flow and pacing.The benchmark reports behavioral-match percentages against human reference trajectories.Source: Table 9 · p. 20

B.3 Behavioral and outcome alignment on τ-bench

The evaluation reports behavioral and outcome alignment, realism judgments, and PRISM next-turn comparisons, followed by ablations on initialization and reasoning. These results use distinct metrics and training settings, so their scores should be read within each evaluation.

B.3 Behavioral and outcome alignment on τ-bench

  • τ-USI combines four behavioral dimensions with task-outcome calibration, while omitting the original index’s survey-based evaluative-alignment component.D1–D4 measure interaction behavior; ECE measures the weighted absolute difference between simulated-user and human-user task-success rates across difficulty bins.Source: §B.3 · p. 20, Table 10 · p. 21
  • 80.17 is MIMESIS-9B’s τ-USI score, above GPT-5.5 at 76.69 and near Osim-8B at 80.44.MIMESIS-9B also has the lowest ECE, 0.069, compared with 0.135 for Osim-8B and 0.188 for GPT-5.5.Source: §B.3 · pp. 20–21, Table 10 · p. 21

B.4 SimulatorArena

  • 38.7 is MIMESIS-9B’s lowest Turing distance, a 3.6-point reduction from the strongest baseline, Claude-Opus-5.Discrimination nevertheless remains far from chance; frontier models score higher on explicit style similarity, so the metrics rank models differently.Source: §B.4 · p. 22, Table 11 · p. 22

B.5 PRISM: behavioral fidelity and next-turn realism

  • 80.1 is MIMESIS-9B’s highest PRISM behavioral-alignment mean, close to Gemini-3.8-Flash at 79.7 and 9.4 points above Osim-8B.PRISM’s aggregate averages D1–D4 and excludes outcome calibration and survey alignment because the necessary annotations are unavailable.Source: §B.5 · p. 22, Table 12 · p. 23
  • MIMESIS has more next-turn realism wins than losses in all 24 judge–baseline comparisons, though the narrowest comparison is marked nonsignificant.Figure 8 encodes MIMESIS wins, ties, and losses as blue, gray, and red segments.Source: §B.5 · p. 23, Figure 8 · p. 23

C.1 Role of user-side mid-training

  • User-side mid-training generally makes simulator RL validation rewards more stable than starting directly from the pretrained backbone.The comparison spans 23 tasks, with individual runs and varying training horizons and vertical scales.Source: §C.1 · p. 24, Figure 9 · p. 24

C.2 Role of explicit reasoning

  • 63.0 accuracy and 583 median words accompany ThoughtTrace supervision on Mistakes, while the RL checkpoint remains at 36.0 accuracy and one word with inference-time thinking enabled.The figure compares accuracy with median words generated before answering; changing the inference setting alone leaves the RL checkpoint unchanged.Source: §C.2 · p. 25, Figure 10 · p. 25
  • Reasoning-enabled simulator RL finishes 1.8 points higher at 4B and 1.5 points higher at 9B after 500 optimization steps.These logged training scores are distinct from benchmark results because rollout and evaluation engines may use different decoding configurations.Source: §C.2 · p. 25, Figure 11 · p. 25

C.3 Contribution of realistic behavior training

Realistic-behavior training improves simulator-quality measures, with gains concentrated in expressing assigned behaviors and placing them appropriately. The agent-training section compares simulator choice and adds CSD coaching from simulator feedback.

C.3 Contribution of realistic behavior training

  • Realistic-behavior training raises RealUserSim from 92.3 to 94.0 (+1.7), τ-USI from 73.9 to 80.2 (+6.3), and lowers SimulatorArena from 41.0 to 38.7 (−2.3).The reward gains concentrate in behavior exhibition and temporal placement; naturalness improves modestly, while task consistency changes little.Source: §C.3 · p. 26, Figure 12 · p. 26

D Agent Training and Evaluation

  • The agent-training comparisons hold multi-turn GRPO fixed when changing the training user, then hold the MIMESIS training user fixed when adding CSD.The section reports environment-level results and robustness checks for these comparisons.Source: D Agent Training and Evaluation · p. 27, §D.1 · p. 28

D.1 Training setup and baselines

  • UserRL uses GPT-5.5, UserRL+ substitutes MIMESIS-9B, and CSD adds coaching; InfoPO provides another multi-turn RL baseline.The controlled InfoPO comparison uses MIMESIS-9B as the training simulator, and the Qwen3-8B actor is optimized for 100 steps.Source: §D.1 · pp. 27–28, Table 13 · p. 28

D.2 CSD implementation details

  • CSD uses simulator thoughts and subsequent public utterances, plus dialogue context and the agent response, to produce retrospective coaching notes.Teacher–student token-level log-probability differences weight the auxiliary loss; only public dialogue conditions rollouts and deployed policies, and real-user thoughts are not available at deployment.Source: §D.2 · p. 28

D.3 Generalization across evaluation users

Across nine unseen evaluation users, MIMESIS-based agent training improves aggregate scores over GPT-5.5 training, while CSD adds further gains; CSD also outperforms GRPO with two alternative training simulators. The theoretical section characterizes CSD’s weighted population target and the conditions for its alignment with continuation value.

D.3 Generalization across evaluation users

  • Across nine users, mean score rises from 26.10 to 29.54 with MIMESIS training, then to 31.09 with CSD; each step improves scores under all nine users.Table 14 summarizes per-user averages across five held-in and three held-out environments.Source: Table 14 · p. 29, §D.3 · p. 29
  • Mean rank averages 3.42 for UserRL, 2.15 for UserRL+, and 1.75 for CSD; CSD ranks best for eight users, except Gemini-3.8-Flash (2.00 vs. 2.12).Lower mean rank is better; UserRL+ ranks better than UserRL for all nine evaluation users.Source: §D.3 · p. 29

D.4 Robustness to alternative training simulators

  • With DeepSeek-V4.1-Flash, CSD’s validation reward stays above GRPO from step 15; with Kimi-K3, the curves separate more after about 60 steps.With either alternative simulator, CSD has higher late-stage training and validation reward over 100 optimization steps.Source: Figure 14 · p. 32, §D.4 · p. 32

E Theoretical Analysis of CSD

  • The analysis characterizes CSD’s population target induced by token weights and conditions under which that target aligns with continuation value.The analysis treats CSD as an auxiliary objective on sampled agent responses.Source: E Theoretical Analysis of CSD · p. 32

E.1 Setting and retrospective coaching

  • The setup fixes a simulator and initial task distribution, allows retrospective coaching to depend on the scored action, and has the student condition only on public history.The rollout policy, feedback law, and weights are held fixed in the surrogate analysis.Source: §E.1 · p. 33

E.2 Policy target and conversation return

CSD’s coaching-weight alignment can support conversation-return improvement under stated covariance conditions, while fitting error qualifies that result. Qualitative examples illustrate simulator behavior mismatches and the limits of the coaching vignette.

E.2 Policy target and conversation return

  • CSD’s target depends on the covariance of conditional coaching weights with continuation value; nonnegative covariance along target-policy histories supports return improvement, but fitting error remains.The analysis does not guarantee improvement from each shared-parameter GRPO+CSD update or transfer to another simulator.Source: §E.2 · pp. 33–34

F Qualitative Analyses and Failure Modes

  • The qualitative analyses use case studies because aggregate fidelity metrics do not show how different notions of fidelity fail on individual interactions.The section also examines simulator feedback in CSD and simulator refusals that can invalidate an environment score.Source: F Qualitative Analyses and Failure Modes · p. 34

F.1 Simulator behavior case studies

  • In the PRISM example, the recorded user and MIMESIS accept an explanation with missing corrections, while frontier simulators challenge its inconsistencies; greater helpfulness or criticism does not ensure a faithful next turn.The comparison illustrates a mismatch between factually critical continuations and the recorded user’s response.Source: §F.1 · p. 35, Table 17 · p. 35

MIMESIS-9B

  • MIMESIS-9B sustains its refusal to authenticate and requests a transfer, overstating the privacy preference compared with the human user, who eventually provides identifying information.The example illustrates that behavior conditioning can amplify a trait beyond the human trajectory.Source: §F.1 · p. 36, MIMESIS-9B · p. 36

MIMESIS-4B

  • MIMESIS-4B hesitates before cooperating, unlike GPT-5.5 and Claude-Opus-5, which provide identifying details directly in the retail interaction.The comparison shows both recovery of hesitation absent from assistant models and the possibility of over-amplifying a conditioned trait.Source: §F.1 · p. 36, MIMESIS-4B · p. 37, Claude-Opus-5 · p. 37

Claude-Opus-5

  • Claude-Opus-5 provides identifying details directly in the retail example, while MIMESIS-9B continues refusing authentication and asks for a transfer.The human user initially questions the request for information, then provides it and proceeds with the task.Source: §F.1 · p. 36, MIMESIS-9B · p. 36, Claude-Opus-5 · p. 37

F.2 From simulator feedback to actionable coaching

  • CSD’s example turns repetition in private thoughts and residual implementation questions in public turns into coaching to examine whether transparency requires a legal mandate.The agent reinforces the user’s existing position despite an assigned objective to argue against it; the excerpt shows no post-note response or performance effect.Source: §F.2 · pp. 38–39

F.3 Simulator refusals can invalidate evaluation

Claude-Opus-5 refusals in PersuadeGym trigger fixed zero-reward fallbacks, so the resulting scores reflect simulator failure rather than agent ability.

F.3 Simulator refusals can invalidate evaluation

  • 672 refusals across PersuadeGym turns produce no simulator content; isolated replay reproduces the failure on 5 of 5 requests.The prompt asks the simulator to consider arguments and return a response and revised stance in a structured format.Source: §F.3 · p. 39
  • PersuadeGym replaces a failed response with a fixed utterance, leaves stance unchanged, and assigns reward zero because reward measures stance change.Repeated refusals therefore force episode return to zero regardless of agent behavior.Source: §F.3 · p. 39
  • The all-zero PersuadeGym entries for Claude-Opus-5 in Table 16 are simulator failures, not measurements of agent ability.The table caption identifies Opus refusals as producing zero-reward fallbacks.Source: §F.3 · p. 39

G Limitations, Safety, and Ethics

The paper identifies limits in human-transfer evidence, behavior-conditioned task validity, user-welfare evaluation, and preservation of pretrained reasoning during simulator training.

Human transfer and evaluation scope

  • Generalization is tested across language-model simulators, not human users, and models from different families may still share behavioral biases.Simulator-generated thoughts are interpretations, not observations of human mental states; human evaluation is needed to assess real-interaction effectiveness.Source: G Limitations, Safety, and Ethics · p. 39

Behavior conditioning and task validity

  • Behavior conditioning can exaggerate traits or alter task information, and a high behavior-reward score can coexist with violations of individual constraints.Grounding rollouts in scenario facts and policies and including a neutral condition do not guarantee every interaction remains plausible or feasible.Source: G Limitations, Safety, and Ethics · p. 39

Task reward and user welfare

  • Task-performance and simulator-fidelity metrics do not assess user autonomy, trust, or long-term welfare, so higher rewards do not assure user benefit.The concern is especially relevant to persuasion tasks where the assigned objective may conflict with users’ interests.Source: G Limitations, Safety, and Ethics · p. 40

Reasoning preservation during user-side mid-training

  • Response-only user-side mid-training may discard pretrained reasoning capabilities before they are relearned from a much smaller annotated dataset.The authors leave reasoning-preserving user-side mid-training methods for future work.Source: G Limitations, Safety, and Ethics · p. 40

Related papers

Loading 2610.09484v1…