Training with Missed Targets in Generative Recommendation: Separating Supervision from Probability Competition

1 Introduction

• Candidate completion appends missed observed targets to reranker training lists, but inference still ranks only the generator’s original candidates.The paper frames this as a practical question of whether training-only targets improve or harm rankings of items that can be served.Source: §1 · p. 1
• In the RecIF-Ads example, two appended targets initially receive 0.011% of softmax probability but 66.7% of target weight under uniform supervision.The third future-click target appears among 126 returned items; the other two are appended only for training.Source: §1 · p. 1, Figure 1 · p. 2
• Completion changes retrieved-target weight, adds appended-target supervision, and makes the two groups compete, so an append/no-append comparison cannot isolate the cause of a ranking change.The three matched losses hold retrieved-target weight fixed and introduce supervision and competition separately.Source: §1 · p. 1
• Development users select between returned-only and appended-target training, while different users from held-out categories evaluate that choice.The proposed rule adopts appended-target training only when its adjusted development lower bound is positive.Source: §1 · p. 2
• Removing probability competition improved all four primary Amazon Video Games point estimates by 7.8–22.2%.Experiments with released OneRec and locally trained Amazon generators also showed competition can be harmful.Source: §1 · p. 2

2 Related Work

• Generative recommenders retrieve items through semantic IDs [19], while prior work explores ID design, joint generation and ranking, sequence reranking, and reward-guided decoding [7] [10] [12] [21] [24] [25] [27] [35].Generators have also been used in cascades [31], downstream candidate channels [33], and verification of frozen retrievers [38].Source: §2 · p. 2
• Other pipeline research studies cascades, exposure-based selection and off-policy estimation [1, 15, 16, 30], sampled-softmax correction, and automated or calibration-aware reranking [9] [11].This paper instead keeps the generator, serving candidates, and scorer architecture fixed while changing missed-target use during reranker training.Source: §2 · p. 2
• Machine-translation reranking found that adding gold references can create harmful comparisons [2], but this paper separates the effects of appending observed targets in recommendation.The training-only targets are absent from the inference candidate pool.Source: §2 · p. 2
• The proposed comparison adapts profile minimization to remove competition between returned and training-only items, whose roles differ because only returned items are ranked at inference.Unlike DKD's target/non-target groups [39], these groups differ by when they exist; the discussion also distinguishes multitask gradient methods [4] [36] and LUPI.Source: §2 · pp. 2–3
• The loss comparison also includes pairwise-ranking baselines within the returned pool or between appended targets and returned nontargets [22].Appendix A.2 extends the derivation to targets with different weights.Source: §2 · p. 3

3.1 Returned and Missed Targets

• Representable observed targets absent from the returned set are appended to training lists, while inference retains the original returned set.Source: §3.1 · p. 3

3.2 What Candidate Completion Changes

• Full combines three effects: retrieved-item loss weighted by α, within-appended-group ranking, and target-ratio competition between groups.Source: §3.2 · p. 3

3.3 Conditional Training

• Cond removes between-group competition while retaining each group’s within-group loss, and it can be trained directly as TN+TA.Source: §3.3 · p. 4

3.4 Three Training Losses

• WN retains weighted returned-item loss, Cond adds appended-target ranking without cross-group competition, and Full adds that competition.Source: §3.4 · p. 4
• Cond–WN isolates the appended-target loss, while Cond–Full isolates the effect of removing group competition.Source: §3.4 · p. 4

3.5 Why the Effect Reaches Inference

• The competition term updates the shared scorer through the gap in group probability and the difference in average score gradients, affecting parameters later used to rank returned candidates.Source: §3.5 · p. 4

4.1 Research Questions

• The study tests competition on RecIF-Ads and prespecified Amazon Video Games comparisons, checks other generator–reranker settings, and evaluates a development-set switching rule on held-out categories.The research questions ask whether competition changes returned rankings, whether appended-target supervision helps without competition, and whether development data can select training for a new category.Source: §4.1 · pp. 4–5

4.2 Data and Candidate Generators

• Three-loss comparisons require training lists with both retrieved and missed targets; evaluation retains all users, including those with no retrieved target.The local generators are Transformers and GRUs, while the RecIF studies use released OneRec checkpoints.Source: §4.2 · p. 5

4.3 Isolating Supervision and Competition

• Matched WN, Cond, and Full rerankers isolate appended-target supervision and group competition, while score shifts test competition without changing within-group rankings.The losses share candidates, training lists, features, initialization, minibatch order, and optimizer; Cond remains unchanged under the shift.Source: §4.3 · pp. 5–6, Figure 3 · p. 6

4.4 Choosing Whether to Append Targets

• For each generator, the rule switches to appended-target training only when its adjusted development-gain lower bound is positive.The rule is set using A-Home and RecIF-Product, then tested on held-out A-Cell and A-Health users.Source: §4.4 · p. 7

4.5 Metrics and Uncertainty

• FT-NDCG@20 evaluates returned-candidate rankings against each user’s complete eligible target set, with users lacking reachable targets receiving zero.The threshold is set before held-out evaluation, and uncertainty analyses resample users and, in additional analyses, generators and training runs.Source: §4.5 · p. 7

5.1 Group Competition Can Hurt

• 0.007423 [0.004978, 0.009744]: Cond exceeded Full in RecIF-Ads FT-NDCG at epoch 60, with the same direction in all seven new runs.The average advantage across epochs 0–60 was +0.01241 [+0.01031, +0.01451].Source: §5.1 · p. 7, Table 4 · p. 8
• The RecIF-Ads advantage was not guaranteed under other stopping or checkpoint rules; at 240 epochs, Cond−Full varied across tested learning rates.With development-fold checkpoint selection, the interval crossed zero.Source: §5.1 · p. 7, Table 4 · p. 8
• 7.83%–22.23%: removing competition improved FT-NDCG across four A-Games comparisons (MLP 22.23%, 12.84%; attention 7.83%, 13.16%); every run favored removal.All four user intervals and three of four training-run intervals excluded zero.Source: §5.1 · p. 7, Table 6 · p. 8
• +0.004407 [0.001591, 0.007177]: shifting appended scores changed Full’s A-Games attention result, while Cond’s training path and parameters stayed unchanged.The shift preserves rankings within each group, supporting probability competition as the source of Full’s response.Source: §5.1 · p. 7

5.2 Appended-Target Supervision Can Help

• +0.001877 [0.000343, 0.003455]: Cond exceeded WN in RecIF-Ads FT-NDCG after 60 epochs, but the development-selected estimate crossed zero.Every one of the seven new runs had the same sign at epoch 60.Source: §5.2 · p. 7

5.3 Development Results Prevent a Harmful Switch

• +.000319 [−.000077, +.000720]: the A-Cell development rule selected appended-target training for two of three generators; in A-Health it kept returned-only training for all three.Always forcing appended-target training in A-Health would have reduced FT-NDCG by 0.000644 [0.000412, 0.000881], or 1.7%.Source: §5.3 · pp. 7–8, Table 5 · p. 8

6 Discussion

• A constant shift to appended scores changed Full’s returned-item rankings but left Cond unchanged, showing that training competition can alter inference rankings.Appended targets affect inference through scorer parameters shared with returned candidates.Source: §6 · p. 8
• Removing competition consistently helped in the primary A-Games study and after the same 60-epoch RecIF-Ads budget, while development checkpoint selection weakened the estimated supervision benefit.The appended-target ranking task helped after 60 epochs, but its estimated benefit weakened when development data selected the checkpoint.Source: §6 · p. 8
• Appended-target supervision may help even when full append training performs poorly, because competition can harm the same scorer.The supervision benefit weakened when development data selected the checkpoint.Source: §6 · p. 8
• Comparisons require the same training budget, and A-Home’s conclusion depends on which interactions count as targets.Source: §6 · p. 8
• For each generator, compare the best returned-only and appended-target rerankers on development users; A-Cell and A-Health required different choices.A favorable isolated effect is insufficient because the deployed loss combines several effects.Source: §6 · p. 8

7 Conclusion

• Removing competition improved FT-NDCG by 7.8–22.2% in all four prespecified A-Games comparisons.Source: §7 · p. 8
• The proposed rule compares returned-only and appended-target rerankers on development users and adopts appended-target training only when its adjusted lower confidence bound exceeds a chosen threshold.Source: §7 · p. 8
• On held-out categories, the rule avoided a 1.7% A-Health loss while allowing appended-target training for two A-Cell generators.Source: §7 · p. 9

A Reproducibility and Additional Results

• The appendix details the data splits, losses, model choices, and final evaluation needed to reproduce the results.Table 8 checks conclusions when targets favorably rated items or when uncertainty includes users and training runs.Source: A Reproducibility and Additional Results · p. 10
• The arXiv ancillary archive contains code, environments, split manifests, and commands, with links to public datasets and OneRec checkpoints.Source: A Reproducibility and Additional Results · p. 10
• The study uses public secondary data, excludes review text and demographic attributes, and releases no raw user data.Source: A Reproducibility and Additional Results · p. 10

A.1 Metrics and Scorer Implementation

• The candidate oracle ranks available targets first, all-user means retain users with no reachable target, and Recall@20 and MRR@20 retain targets outside the returned pool.Recall@20 is the fraction of targets in the top 20; MRR@20 is the reciprocal rank of the first target, or zero if absent.Source: §A.1 · p. 10
• After seven new RecIF-Ads seeds, Cond−WN changed Recall/MRR by +0.00166/+0.00425, Cond−Full by +0.00880/+0.01348, and Full−WN by −0.00713/−0.00923.Source: §A.1 · p. 10
• The scorer root-mean-square normalizes query and item states with a 10^-6 floor, then builds 519 dimensions from states, their product, absolute difference, and seven scalars.Attention uses returned-candidate keys and values; output layers start at zero.Source: §A.1 · p. 10
• Slots 1/3 encode standardized log likelihood and training frequency, slots 2/4–6 are reserved zeros, slot 7 marks validity, and source membership is excluded.Source: §A.1 · p. 10

• The derivation introduces disjoint source groups with positive target counts and source weights, then applies the same substitution used in Proposition 1.Source: §A.2 · p. 10
• Optimizing score offsets matches each source softmax probability to its target weight, leaving an entropy term H(γ) after minimization.Source: §A.2 · p. 10
• The decomposition also applies to normalized nonnegative target weights, including cases with zero source mass by taking an infinite-offset limit and omitting the empty conditional term.Source: §A.2 · p. 10

• The Amazon tasks use 2014 5-core archives, omit text, and form candidates by beam-64 decoding; A-Games fixes its user-role split before final-target access.Source: §A.3 · p. 10
• Across six Amazon generator–reranker combinations, effect directions varied and most familywise-adjusted 95% intervals included zero.The figure identifies overall means and intervals, individual run means, and zero; all panels share axes.Source: Figure 4 · p. 11
• For A-Home, the authors replace a returned-only loss with an appended-target loss only when its corrected lower confidence bound for paired development differences is positive.The selected comparison is evaluated on five new runs, with rankings fixed before final-target access.Source: §A.3 · p. 12
• A-Cell’s selected-pair adjusted lower bounds were +.000411, −.001813, and +.001278, versus −.000713, −.000438, and −.001317 on A-Health; decisions held under 18-comparison adjustment.Source: §A.3 · p. 12
• The post hoc threshold check selected 4, 2, 1, and 1 switches at margins 0, .00025, .0005, and .001, respectively, so it describes threshold sensitivity rather than confirming a new rule.Source: §A.3 · p. 12

A.4 Optimization and Inference

• AULC summarizes the user-and-run-averaged difference over checkpoints by trapezoidal area across training time, divided by the total duration.Source: §A.4 · p. 12
• Full−WN and Cond−Full had different signs in two of six additional settings; only A-Games Transformer seed 42 Cond−WN excluded zero when resampling users and runs.Its difference was +0.001572 [+0.000137, +0.003305]. Paired-user intervals also excluded zero for that Full−WN comparison and A-Toys GRU seed 61 Cond−Full when rerankers were held fixed.Source: §A.4 · p. 12
• Inference intervals jointly adjust comparisons within three prespecified sets, and user-and-training-variation checks resample both users and observed runs.OneRec repeats checkpoint choice within user-and-run draws; other local intervals use saved predictions without choosing models again.Source: §A.4 · p. 12

A.5 Discriminative Control

• The discriminative control ranks appended targets above returned nontargets with total weight γ, while inference still ranks N; its returned-item pairwise baseline requires a reachable target and a returned nontarget.When r=0, T_N is set to zero, so a single appended target remains trainable if returned nontargets exist.Source: §A.5 · p. 12