Sampling Safe Futures: Multimodal Trajectory Planning for Personalized Safety in Anthropomorphic AI

1. Introduction

• Relational harm can accumulate across individually benign exchanges, affect users’ well-being and relationships, and go unrecognized by the person experiencing it.It may undermine psychological well-being or emotional autonomy and can extend to people around the user.Source: §1 · p. 2
• Turn-level safeguards cannot determine whether acceptable replies are cumulatively moving a particular user toward an unsafe relational state.The paper argues for evaluating whole interaction trajectories rather than isolated responses.Source: §1 · p. 3
• The paper models relational safety as a hidden state inferred from user messages and changed by the system’s replies.Risk depends on the user and relational stage, so the same response may be benign for one user and harmful for another.Source: §1 · pp. 2–3
• A strategy is admissible only if it preserves a safe continuation for every plausible user model; a GFlowNet then samples plausible, safe, and useful futures.The system selects the strategy associated with the largest fraction of those evolutions.Source: §1 · p. 3
• Looking ahead over possible relationship evolutions reduces harmful states at a limited cost in helpfulness.The evaluation compares the method with uniform, warning-triggered, multi-turn detection, and non-lookahead safeguards.Source: §1 · p. 3

2. Related Work

• Prior work characterizes anthropomorphism and the evolving mechanisms of relational harm, including harm creep, affective harms, dependence, and belief reinforcement.These studies describe how relational harms can develop over time.Source: §2 · p. 4
• Conversation-level guardrails track temporal risk, mainly for adversarial interaction and unsafe model behavior, while sequential decision studies model system-induced changes in users.The related approaches include risk accumulation, future jailbreak prediction, robust critics, and models of preference or user-state change.Source: §2 · pp. 4–5
• The open problem is personalized action selection that infers a user’s relational state, accounts for uncertainty in their reactions, and reasons over future trajectories with an explicit safety requirement.The authors state that, to their knowledge, existing work does not provide this mechanism.Source: §2 · p. 5

3. System Model

• The latent stage changes probabilistically with the current stage, response strategy, and user profile, while the true user-response parameters are represented by a user-specific set of plausible models.The system is a partially observable Markov decision process because the state is not directly observed and must be inferred from behavior.Source: §3 · p. 6
• The system infers relational stage from cues in user messages and updates a probability distribution over stages for each plausible user model using Bayesian filtering.The cue set includes attributing interiority, addressing the system socially, vulnerable disclosure, references to other people, and beliefs supported by prior replies.Source: §3 · pp. 6–7
• A plan assigns strategies over a planning horizon, then applies a fallback through the longer safety horizon; an evolution is safe only if it avoids every harmful stage.Harmful stages are defined by a common externally set threshold, while personalization concerns how each user’s relationship evolves toward it.Source: §3 · p. 7
• The system is the sole decision maker, and the user reacts according to the response model without optimizing an objective of their own.Strategic users, such as users attempting to circumvent safeguards, are outside the scope; utility measures helpfulness independently of safety.Source: §3 · pp. 7–8

4. Problem Formulation

• A plan’s safety value is its probability of remaining safe under the least favorable plausible user model, with each model evaluated using its own belief.The risk is one minus this safety value.Source: §4 · p. 8
• A plan is admissible when its safety value meets the tolerance, and a strategy is admissible if it begins at least one such plan.The continuation can be executed without relying on future observations; if the true response model is in the uncertainty set, it preserves safety at the required probability.Source: §4 · pp. 8–9
• The target distribution weights evolutions by nominal-model plausibility and utility while excluding unsafe evolutions and plans that fail the safety guarantee.The utility weight’s inverse-temperature parameter controls the trade-off between plausibility and utility.Source: §4 · p. 9
• Safe continuation mass is the share of target-distribution evolutions beginning with a strategy, and the system chooses the admissible strategy with the greatest mass.This favors strategies preserving many plausible, safe, and useful futures over one associated with a single especially good future.Source: §4 · pp. 9–10

5. Generative Trajectory Sampling for Personalized Safe Action Selection

• At each turn, safety values and the belief profile define admissible strategies that mask generation; sampled evolutions estimate each strategy’s mass before the reply and belief update.If no strategy is admissible, the procedure returns the emergency fallback.Source: Figure 2 · p. 10, §5 · pp. 10–11
• Because the same belief can lead to distinct relationship outcomes, the method samples multiple futures instead of relying on the most likely evolution.The GFlowNet constructs evolutions one turn at a time and samples in proportion to plausibility, safety, and utility.Source: §5 · pp. 11–12
• The estimator gives a strategy greater mass when it begins many sampled evolutions that are plausible, safe, admissible, and useful, rather than only the single best trajectory.The admissibility mask preserves the safety guarantee while the GFlowNet explores multiple safe continuations.Source: §5 · p. 12

5.1. Computational Complexity

• Offline safety values are computed once per user profile, independently of the interaction, and stored for every model, plan, and stage.The storage requirement is O(M|A|^k|Z|).Source: §5.1 · p. 12
• Online sampling costs O(NkC_net), followed by O(N + |A|) mass counting; the second term of the overall cost grows exponentially with planning horizon.The method is intended for short horizons; k = 2 yields 25 plans.Source: §5.1 · p. 13

6.1. Simulation Environment

• The simulated environment models a discrete latent relational stage, with cues as indicators and response strategy as a covariate affecting transition probabilities.The five ordered stages range from instrumental use to role displacement, and can move in either direction.Source: §6.1 · p. 13
• The five strategies range from reinforcing companionship to redirecting users toward human relationships, trading greater helpfulness for greater escalation pressure.The strategy set groups behaviors from companionship and protective-conversation benchmarks and is not intended as an exhaustive taxonomy.Source: §6.1 · pp. 13–14
• The evaluation’s true user response model is generated independently and lies outside the system’s uncertainty set, testing the formal guarantee beyond its stated model-membership assumption.The uncertainty set uses three models with parameters perturbed within ±25% of the nominal profile and observation models from different extractors.Source: §6.1 · p. 15
• User transitions depend on susceptibility, recovery rate, and strategy pressure, with dynamics calibrated to published interaction statistics and an 8-turn decay under neutral replies.The probability of escalation after a reinforcing versus neutral reply is set to a ratio of 7.4; the neutral-reply escalation probability is set to 0.06 because it is not reported in the literature.Source: §6.1 · pp. 14–16

6.2. Implementation of the Scheme

• Safety values are computed offline for every stage, plan, and model, then weighted by current beliefs to obtain the admissible set.The stored table has |A|^k × |Z| × M entries.Source: §6.2 · p. 16
• The GFlowNet uses two 64-unit tanh hidden layers and is trained jointly with a profile- and belief-conditioned log-partition network.Training uses trajectory balance, Adam, 6,000 iterations, batches of 32 plans, and rewards queried from the user response model.Source: §6.2 · pp. 16–17
• At decision time, 64 plans are sampled and the strategy with the largest estimated safe continuation mass is selected.The sampler is masked so it can construct only evolutions of admissible plans.Source: §6.2 · p. 17
• Table 1 gives generated-message examples for each escalation stage alongside extracted relational cues and indicates when stage recognition relies on cues alone.A ‡ marks an example whose stage is recognized from cues only.Source: Table 1 · p. 17

6.3. Baselines and Metrics

• Comparators include always-helpful, deployed-like Moore et al. (2026), turn-level filtering, crisis fallback, multi-turn risk accumulation, and myopic shielding.The accumulation baseline combines semantic drift, advanced-stage probability, and its six-turn slope; the shield baseline uses the proposed admissibility test without looking ahead.Source: §6.3 · p. 17, Figure 4 · p. 19
• Cumulative harm is the percentage of episodes reaching the harmful stage at least once by each turn, and final Harm is that percentage at episode end.Harm is defined as CumHarm_T.Source: §6.3 · p. 18
• Average escalation stage indicates how far relationships not yet harmful have progressed, while helpfulness averages per-turn interaction utility, based here only on the selected strategy.The utility of a strategy is independent of safety.Source: §6.3 · p. 18
• Relative harm reduction against a baseline is 100(1 − Harm/Harm_base), and Figure 4 compares the proposed scheme with six baselines.Figure 4 reports each baseline’s harm in brackets.Source: §6.3 · p. 18, Figure 4 · p. 19
• Each configuration uses 60 episodes of 30 turns for each of three user profiles, totaling 180 episodes.The evaluation also compares target plan probability with sampled plan frequency.Source: §6.3 · p. 19

6.4.1. Harm reduction analysis

• Harm fell 82% versus always selecting the most helpful reply and 73% versus the deployed-like policy, which reached 69% harm.The deployed-like policy alternates validating replies with neutral or protective ones, which slow but do not reverse escalation.Source: §6.4.1 · pp. 19–20

6.4.2. Safety-helpfulness trade-off

• The scheme traces a safety–helpfulness frontier as tolerance δ varies, while baselines occupy fixed points; its knee marks where further harm reduction costs helpfulness.The tolerance makes acceptable risk adjustable independently of response generation.Source: Figure 5 · p. 21, §6.4.2 · pp. 22–23

6.4.3. Evolution of harm along the conversation.

• Over turns, only the proposed scheme lowers average stage after an initial rise, slowing and flattening cumulative harm while baseline curves keep rising.The shield with myopic selection follows a similar stage pattern, but at a higher stage.Source: Figure 6 · p. 21, §6.4.3 · pp. 23–24

6.4.4. Average performance across user profiles

• Across user profiles, the proposed scheme is safest at comparable helpfulness; belief-based memory reduces harm versus message-only filtering, and safe-future selection reduces it further.Crisis detection remains the most harmful among the compared policies despite using the belief, because it acts only once estimated risk is already high.Source: Figure 7 · p. 22, §6.4.4 · p. 24

6.4.5. Behavior of the generative planner

• The trained sampler covers 97% of target modes versus 9% for a single-solution planner, reproducing less-probable admissible plans as well as common ones.The target spreads probability across many plausible, safe, and useful plans rather than concentrating on one.Source: Figure 8 · p. 23, §6.4.5 · p. 25

• Safety is modeled as a personalized, trajectory-level decision problem in which relationship state is inferred from conversational cues and changes under system responses.Source: §7 · p. 25
• A strategy is admissible only if it preserves a safe continuation under every plausible user model; a GFlowNet samples plausible, safe, useful evolutions to select the strategy with the largest safe continuation mass.Source: §7 · p. 25
• Safety is user-dependent: the same response may suit one relationship but be unsafe for another, making uniform safeguards potentially overly permissive or unnecessarily restrictive.Source: §7 · p. 25
• Reasoning over multiple possible relationship evolutions retains distinct modes instead of collapsing decisions onto the most likely continuation.Source: §7 · p. 25