Recommendation World Models for Future-State Control

1 INTRODUCTION

• A next-item score alone does not rank candidate slates by future consequences, because displayed slates shape feedback and subsequent user state.Similar-utility slates can induce different future category mixtures, repeat exposure, or interest concentration.Source: §1 · p. 1
• The interface separates reference ranking, candidate generation, and consequence prediction to select slates that pursue explicit targets beyond next-item relevance.Selection weighs predicted target movement against utility cost and estimated failure risk.Source: §1 · p. 2
• UA-TWM uses a frozen recommender’s slate as an anchor, evaluates alternatives with utility and target-gain estimates plus calibrated failure risk or one-step state-action prediction, and falls back to the anchor when none qualifies.After a non-anchor selection, observed feedback is used to check predicted target improvement and utility conditions.Source: §1 · p. 2
• Evaluation uses chronological logs from MovieLens-25M and KuaiRand-Pure for matched comparisons across twelve backbones, plus repeated slate-feedback interaction in KuaiSim.The simulator evaluates ten interaction steps, with a one-step prediction at each simulated decision.Source: §1 · p. 2
• Matched logged comparisons improve Recall@20, NDCG@20, and future-state alignment across two public domains; ablations examine action-conditioning and selection mechanisms.The evaluations cover twelve sequential backbones.Source: §1 · p. 2

2 RELATED WORK

• Sequential recommendation models span diverse ranking architectures, supplying relevance scores and reference slates for local action search.Matched evaluations hold each backbone fixed to measure consequence-aware selection around it.Source: §2 · p. 2
• Controllable recommendation and simulators make desired outcomes or user feedback explicit; UA-TWM searches near the reference slate and retains it when selection criteria are unmet.Source: §2 · p. 2
• World models support prediction, planning, or control; in recommendation, histories are states, slates are actions, and predictions describe subsequent feedback and consumption.UA-TWM’s closed-loop version uses one-step state-action prediction for repeated constrained selection.Source: §2 · pp. 2–3

3.1 DECISION PROBLEM AND EVIDENCE REGIMES

• The controller minimizes horizon target loss subject to utility remaining within εu of the anchor, approximating this objective with local action-conditioned prediction.It selects a qualifying alternative or returns the anchor.Source: §3.1 · p. 3
• Logged replay measures alignment with historical future positives, whereas closed-loop rollout measures action-induced movement; logged positives do not identify transitions for unshown slates.Both regimes evaluate utility and target conditions relative to the same anchor.Source: §3.1 · p. 3

3.2 UA-TWM: A UTILITY-ANCHORED WORLD-MODEL INTERFACE

• UA-TWM freezes the recommender as an anchor and trains a local consequence layer to predict utility gain, future-state gain, and support risk for nearby slates.The anchor is returned when no candidate qualifies.Source: §3.2 · p. 3
• In logged replay, consequence estimates compare candidate slates with the anchor using prefix statistics and backbone scores, while a calibrated risk head predicts failure without using held-out future statistics as features.The risk diagnostic separates utility-floor and target-gain failures, with thresholds chosen on calibration users.Source: §3.2 · pp. 4–5
• The logged selector admits candidates only if predicted utility gain, target gain, and risk pass calibrated thresholds, then maximizes predicted target gain or falls back to the anchor.Ablations remove the utility floor, state head, risk gate, or fallback to examine their respective costs or contributions.Source: §3.2 · p. 5
• Closed-loop control predicts one step ahead, checks immediate click utility, and regenerates candidates after feedback across ten environment steps.Its local search uses anchor edits and graph-semantic replacement slates; each simulated decision has nine candidate actions.Source: §3.2 · pp. 4–5

4.1 EVALUATION PROTOCOL

• The protocol uses chronological replay across twelve sequential anchors and KuaiSim rollouts to test transfer and target-directed interaction.Replay shares histories, candidate pools, calibration splits, slate sizes, and target definitions within each dataset; rollouts use one-step prediction over ten steps.Source: §4.1 · p. 6

4.2 LOGGED TRANSFER ACROSS BACKBONES

• Across both datasets, UA-TWM improves Recall@20, NDCG@20, and future-state alignment for every matched backbone.The comparison attaches the consequence layer to each unchanged backbone.Source: §4.2 · p. 6, Table 1 · p. 7

4.3 RANKING AND FUTURE-STATE ORDERING

• Ranking and future-state orderings can disagree, so consequence scores can change slate selection even when the candidate pool and backbone stay fixed.Figure 2 summarizes ordering mismatch, matched logged gains, and the coverage–unsafe-rate relationship.Source: §4.3 · pp. 7–8, Figure 2 · p. 8

4.4 ALTERNATIVES, ABLATIONS, AND MODEL DIAGNOSTICS

• Removing the utility floor favors greater state movement at a ranking cost, removing the state head reduces target improvement, and removing the risk gate admits more unsafe claims.The full selector ranks candidates inside a constrained feasible set; Figure 3 compares logged-replay alternatives and ablations.Source: §4.4 · p. 8, Figure 3 · p. 8
• Split-risk gating lowers unsafe rates in two settings, while longer future windows lower invalidity and improve MovieLens state-loss reduction.The risk-calibration ECE changes are 0.135 to 0.099 for MovieLens S3Rec and 0.098 to 0.097 for KuaiRand LightSANs; with the policy fixed, MovieLens state-loss reduction rises from 0.035 to 0.042 as the target window lengthens.Source: §4.4 · pp. 8–9

4.5 CLOSED-LOOP TARGET-CONTROL

• In KuaiSim, UA-TWM makes four valid target-control claims, falls back on three targets, and makes zero invalid claims for every seed.Before fallback, it improves target loss over the anchor by 0.374; the state-only ablation gains 0.201, for a paired state-action advantage of 0.173 target L1.Source: §4.5 · p. 9, Table 2 · p. 9
• Action summaries improve ridge transition prediction, but the benefit depends on predictor and transfer setting.Normalized MAE falls from 0.491 to 0.266 on held-out seeds and from 0.433 to 0.345 on held-out policies; random forests worsen on held-out policies.Source: §4.5 · p. 9

• UA-TWM predicts consequences of nearby slate actions and selects them under utility constraints.Source: §5 · p. 9
• Matched logged evaluations across twelve backbones and two domains improve ranking utility and future-state alignment.Source: §5 · p. 9
• In KuaiSim, the anchor remains available when no candidate qualifies during repeated target-directed selection.Source: §5 · p. 9

Appendix organization

• Appendix B maps experimental settings, logged replay diagnostics, related work, and closed-loop mechanism details to their evidence roles.Source: A ADDITIONAL DETAILS · p. 13

• The authors report using generative AI for literature organization, editing, code navigation, and experiment-log summarization, with research claims and artifacts checked by the authors.Source: AI USE STATEMENT · p. 13

Responsible use

• The study uses public or reproducible resources and simulator-style rollouts, not new human-subject data; deployment would require consent, governance review, and harm monitoring.Source: ETHICS STATEMENT · p. 13

Reproducibility

• The reproducibility statement specifies selection rules, target-state definitions, validity criteria, and paired-seed interpretation, and says source code and experiment artifacts will be released.Source: REPRODUCIBILITY STATEMENT · p. 13

• Calibration uses training positives as context and target with calibration positives as future outcomes; test uses training plus calibration positives as context and target, then test positives as outcomes.Source: §A.1 · p. 13
• For test targets, calibration positives receive twice the per-event weight of training positives when genre or tag counts are normalized into a profile.Source: §A.1 · p. 13
• Logged test evaluation uses each user’s entire subsequent held-out positive block, whose length varies, to form the future profile.Source: §A.1 · p. 14
• Window-sensitivity comparisons keep the decision prefix and slate fixed while changing the suffix, using a common eligible-user set across lengths.Source: §A.1 · p. 14
• The procedure anchors on the frozen ranker, proposes local edits and target-conditioned replacements, predicts utility and target consequences, then selects the greatest qualifying target gain or returns the anchor.Source: ALGORITHMIC CONTRACT · p. 14
• Table 4’s seven KuaiSim scenarios specify desired repeat, entropy, history-overlap, and cumulative-coverage values; Table 3 says thresholds are calibration-selected and evaluated once on held-out users or seeds.Source: Table 4 · p. 15, Table 3 · p. 15

• Logged replay measures ranking and future-statistic alignment on fixed histories, whereas KuaiSim tests repeated decisions with action-dependent feedback and state updates.Source: B EVIDENCE MAP · p. 16
• All twelve backbone pairs improve Recall@20, NDCG@20, and future-state alignment in both MovieLens-25M and KuaiRand-Pure.Source: B EVIDENCE MAP · p. 16
• Coverage and replay-unsafe rate together distinguish reliable target-directed selections from low failure rates caused by frequent fallback.Source: B EVIDENCE MAP · p. 16
• KuaiSim claim-validity evaluation checks utility and target conditions after interaction across requested directions and support conditions.Source: B EVIDENCE MAP · p. 16
• Action summaries reduce ridge transition-prediction error on held-out seeds and policies, but random forests worsen on held-out policies, so gains depend on predictor and split.Source: B EVIDENCE MAP · p. 16, Table 5 · p. 16

• Constrained selection retains positive ranking and state gains while reducing unsafe target-control claims across both logged domains.Source: Table 6 · p. 17
• Low agreement between next-item utility and future-state rankings means a high-utility slate can still be a poor control action, motivating separate utility- and state-gain estimates.Source: C ADDITIONAL LOGGED ANALYSES · pp. 17–18
• Split-risk gating lowers selected unsafe rates and improves unsafe-candidate ranking in representative logged settings.Table 7 reports unsafe and predicted risk as percentages; AUC measures unsafe-candidate ranking, and lower ECE indicates better calibration.Source: Table 7 · p. 18
• Figure 4 summarizes ranking–state ordering and adapter training time, search throughput, and memory overhead, separating decision-layer costs from backbone training.Source: Figure 4 · p. 18, C ADDITIONAL LOGGED ANALYSES · p. 18
• The shared candidate support lets the logged analyses attribute changes in ranking gains, state alignment, risk estimates, and selection frequency to different decision rules.Source: C ADDITIONAL LOGGED ANALYSES · p. 18

• Interactive simulators expose feedback from slate actions, while UA-TWM evaluates a local controller around a trained ranking anchor using predicted utility gain, future-state gain, and replay-unsafe risk.Model-based online recommendation learns interaction models for policy optimization; SlateQ-style and DORL-style methods optimize long-term or penalized return.Source: D CLOSEST-WORK DISTINCTIONS · p. 18
• Goal-conditioned methods estimate long-term values or condition action prediction on cumulative objectives; UA-TWM instead selects within the anchor’s candidate support using an explicit utility floor.GoalRec learns goal-conditioned long-term value estimates (Wang et al., 2021); goal-conditioned sequence models predict actions conditioned on desired cumulative objectives (Li et al., 2024; Gao et al., 2025).Source: D CLOSEST-WORK DISTINCTIONS · pp. 18–19
• Unlike causal feedback-correction methods and LLM-based user simulation, UA-TWM studies a non-LLM consequence layer constrained by utility preservation, support risk, and fallback.Source: D CLOSEST-WORK DISTINCTIONS · p. 19

• Non-anchor selections issue target-scenario claims before rollout; validity is checked afterward, coverage includes unsuccessful claims, and anchor fallback issues none.Source: E CLOSED-LOOP MECHANISM DETAILS · p. 19
• The full controller makes four valid target-scenario claims and returns the anchor on three scenarios, while the state-only ablation retains the anchor on all scenarios.Target-aggressive policies achieve lower target error at lower click utility.Source: E CLOSED-LOOP MECHANISM DETAILS · p. 19
• The controller claims valid movement in the balanced, diversity, mixed, and repeat families, but falls back on the entropy target; that fallback leaves global reachability undetermined.Source: E CLOSED-LOOP MECHANISM DETAILS · p. 19, Table 8 · p. 19
• Figure 5 connects scenario-level target gains with utility and claim outcomes, while Figure 6 audits action sources: 6,300 candidates across 700 decisions, all selected from declared support.Source: E CLOSED-LOOP MECHANISM DETAILS · p. 19, Figure 5 · p. 19, Figure 6 · p. 20

Candidate support and history weighting

• Across seven configurations per setting, MovieLens Recall gains range from 0.0107 to 0.0120, NDCG gains from 0.0041 to 0.0046, and state-loss reductions from 0.034 to 0.041.KuaiRand gains range from 0.0046 to 0.0055 for Recall, 0.0015 to 0.0019 for NDCG, and 0.062 to 0.074 for state-loss reduction.Source: F LOGGED SENSITIVITY ANALYSIS · p. 21
• Expanding the candidate pool from 200 to 300 raises unsafe rates without improving gains; the low-strength grid lowers unsafe rates but also reduces state gains.History weights of one and three preserve positive gains, while weight two gives the largest observed state reduction in this sweep.Source: Figure 7 · p. 20, F LOGGED SENSITIVITY ANALYSIS · p. 21

Evaluation-window length

• Coverage remains 95.7% on MovieLens and 98.2% on KuaiRand across windows, while state-loss reductions increase and unsafe rates fall as evaluation length grows.MovieLens reductions are 0.035, 0.039, and 0.042 and unsafe rates are 8.8%, 6.5%, and 4.7% at 10, 20, and full-block windows; KuaiRand reductions are 0.064, 0.073, and 0.080 and unsafe rates are 11.2%, 8.0%, and 5.6%. Ranking gains remain positive, though NDCG gains are not monotonic.Source: F LOGGED SENSITIVITY ANALYSIS · p. 21, Figure 8 · p. 21