Decision-Native Recommendation with Large Language Models

1 Introduction

• Generation-based outputs may fail to identify a unique candidate, change with candidate order, omit explicit confidence, and add decoding latency.The introduction reports invalid or non-unique outputs in roughly 8–16% of cases and changed choices or validity in roughly 30–45% of tested reorderings.Source: §1 · pp. 1–2
• Decision-native recommendation jointly models item choice and confidence, requiring valid candidate selection, order consistency, and confidence reflecting recommendation accuracy.DICE selects a supplied candidate and returns calibrated confidence in that choice.Source: Figure 1 · p. 1, §1 · p. 2
• DICE learns candidate-choice probabilities from chronological interaction history and candidate descriptions, then calibrates the selected item’s probability using held-out data.Tree-structured attention shares history computation across isolated candidate branches in one forward pass while preserving order-consistent scoring.Source: §1 · p. 2
• DICE outperforms all evaluated baselines in top-one accuracy across three benchmarks.Further evaluations support efficient inference and the value of confidence for selective recommendation and model routing.Source: §1 · p. 2

2.1 LLM-Based Recommendation

• LLM-based recommendation methods use interaction histories, item text, and pretrained knowledge, with approaches spanning representation learning, generation, and reasoning-enhanced recommendation.DICE is described as learning candidate-choice probabilities and evaluating confidence alongside validity and order consistency.Source: §2.1 · p. 2

2.2 Direct Scoring and Efficient Inference

• Direct-scoring methods score query–document relevance or candidate items, and InvariRank combines candidate isolation and shared positional framing for order-consistent ranking in one pass.DICE instead learns candidate-choice probabilities and calibrates confidence; it adapts tree-structured attention to share history computation across isolated candidate branches.Source: §2.2 · p. 2

2.3 Calibration, Selective Prediction, and Routing

• Related confidence work covers temperature scaling, confidence elicitation, selective prediction, model cascades, and uncertainty-guided computation allocation.DICE applies temperature scaling to candidate-choice probabilities and evaluates confidence against observed recommendation accuracy.Source: §2.3 · p. 3

3.1 Categorical Choice Learning with LLM Representations

• DICE learns a categorical distribution over supplied next-item candidates from user history, independently scores each candidate, and jointly normalizes logits under one choice loss.The pretrained backbone remains frozen; training updates LoRA adapters and the shared decision head.Source: §3.1 · p. 3, §3 · p. 3
• The shared scorer outputs supplied item identities without answer decoding or text matching, and can score items unseen during recommendation training from their text.Source: §3.1 · p. 3

3.2 Single-Forward Inference with Order Consistency

• One tree-structured forward pass shares the instruction–history prefix across isolated branches; matching independent position indices yields permutation-equivariant scores and unchanged item identity, with deterministic tie-breaking.Efficiency comes from prefix sharing, while order consistency comes from candidate isolation, shared positional framing, and shared scoring parameters.Source: §3.2 · pp. 4–5

3.3 Calibrated Decision Confidence

• Temperature scaling fits a positive T* on held-out D_T with model parameters frozen, preserving the selected item while defining its calibrated probability as decision confidence.Confidence discrimination and calibration are evaluated separately.Source: §3.3 · p. 5
• A smaller DICE model handles each example and defers low-confidence cases to a larger model using γ fixed on Dτ, without a separately trained router.For selective recommendation, examples at or above a threshold selected on a separate development set are retained.Source: §3.3 · p. 5

4.1 Experimental Setup

• At K=20, the main comparison uses Qwen3-0.6B; HR@1 measures top-one accuracy, and ECE measures confidence–accuracy gaps across ten bins.Reported comparisons are arithmetic means of per-run empirical metrics over three random seeds.Source: §4.1 · p. 5

4.2 RQ1: Recommendation Quality and Item Cold-Start Performance

• DICE-I exceeds baselines on HR@1, HR@3, and NDCG@3 across three datasets, with HR@5 leadership on Goodreads and LastFM but not Amazon Industrial.Compared with LlamaRec, DICE-I's absolute HR@1 gains are +6.14% on Goodreads, +2.00% on LastFM, and +1.17% on Amazon Industrial; DICE-S leads all four metrics on Amazon Industrial.Source: Table 1 · p. 6, §4.2 · p. 6
• As the backbone grows, LastFM HR@1 rises from 0.6160 to 0.7018; Goodreads HR@1 reaches 0.5465 at 1.7B and then plateaus.HR@3 and NDCG@5 continue to improve on Goodreads after HR@1 plateaus.Source: Table 2 · p. 6, §4.2 · p. 6
• For cold targets, DICE-I reaches HR@1 of 0.2397 on Goodreads, 0.2303 on LastFM, and 0.2495 on Amazon Industrial.DICE-I exceeds SASRec on Goodreads and LastFM; the strongest LLM method varies by dataset.Source: Table 3 · p. 6, §4.2 · p. 6

4.3 RQ2: Structural Properties and Inference Efficiency

• Ranking improves SFT, S-DPO, and NAPO HR@1 by 2.82–8.81% on Goodreads and 4.09–6.41% on LastFM, while RE2 is nearly unchanged.These are absolute gains; a separate SFT diagnostic found 16.11% of Goodreads and 7.98% of LastFM generated answers failed to identify a unique candidate.Source: Table 4 · p. 7, §4.3 · p. 7
• Neither DICE variant changes its selected item, logit, or probability under tested reorderings, whereas SFT and S-DPO exhibit choice or validity changes.Under reordering, SFT ValidRatio is 80.55%–92.58% across the two datasets, while DICE maintains 100%.Source: Table 5 · p. 7, §4.3 · p. 7
• Tree inference is 6.18–6.26× faster than generation on Goodreads and 3.04–3.11× on LastFM; at K=100, DICE-I's HR@1 gains over S-DPO are 16.45% and 26.09%.The K=100 gains are on Goodreads and LastFM, respectively; the tree-versus-generation speedups are also reported for those datasets, respectively.Source: Figure 3 · p. 7, §4.3 · p. 7, Figure 4 · p. 7

4.4 RQ3: Decision Confidence and Applications

• Temperature scaling reduces ECE for all methods; calibrated DICE-I has the lowest ECE, while confidence-selected test subsets reach 88.82% Goodreads and 94.42% LastFM accuracy.On LastFM, routing escalates 47.78% of examples and retains over 99.5% of the 4B-only model's accuracy at 24.8% lower estimated sequential latency.Source: Table 6 · p. 8, §4.4 · p. 8, Figure 6 · p. 8

5 Conclusion and Discussion

• DICE treats valid candidate selection, candidate-order consistency, and confidence reflecting observed recommendation accuracy as joint requirements.The conclusion describes higher top-one accuracy than evaluated baselines across three benchmarks and additional support for unseen-item selection and order-consistent inference.Source: §5 · p. 8
• DICE combines categorical choice learning with LLM representations, shared-history inference, and held-out calibration; confidence supports selective recommendation and computation allocation.Matched flat–tree measurements show the computational benefit of history sharing with the scorer held fixed.Source: §5 · p. 8
• Future work targets joint candidate retrieval and selection, adaptive calibration as candidate composition and user behavior evolve, and online routing under dynamic workloads.The proposed online-routing extension would account for expected accuracy gain alongside computational and serving costs.Source: §5 · p. 8

A Evaluation Details

• An anonymous repository URL is provided.https://anonymous.4open.science/r/dice-73B3/.Source: A Evaluation Details · p. 10

A.1 Datasets and Preprocessing

• The main comparison pairs chronological histories with next observed items and, at K=20, adds 19 uniformly sampled negatives while randomizing candidate order.Histories must contain at least three non-padding items; evaluation examples are aligned across methods by history, candidates, order, and target.Source: §A.1 · p. 10

A.2 Item Cold-Start Protocol

• The cold-start protocol holds out approximately 20% of items from recommendation training and evaluates targets from those items among 10 cold and 10 warm candidates.Cold items may have appeared during LLM pretraining; the protocol covers Goodreads, LastFM, and Amazon Industrial.Source: §A.2 · p. 10

A.3 Training and Baseline Implementation

• DICE-I and DICE-S use Qwen3-0.6B with LoRA, while the main comparison shares data partitions, histories, candidate sets, and evaluation protocols across methods.DICE-I is the main method; DICE-S is the candidate-interaction extension, and DICE-I/S train for three epochs.Source: §A.3 · p. 10

A.4 Stability Across Random Seeds

• DICE-I has higher mean HR@1 than LlamaRec and InvariRank on all three datasets, with sample standard deviations no greater than 0.0017.The reported standard deviations are smaller than the corresponding mean HR@1 gains over either baseline.Source: §A.4 · pp. 10–11, Table 8 · p. 11

A.5 Candidate Scoring and Generated-Answer Evaluation

• Direct-ranking methods use method-specific item scores, while generation-based baselines rank candidate names by summed token log-probabilities and generated answers are evaluated separately.For generated answers, unmatched or ambiguous outputs count as incorrect; ValidRatio measures the fraction uniquely matching a full candidate name after normalization.Source: §A.5 · p. 11

A.6 Candidate-Order Consistency Protocol

• DICE logits and probabilities have zero maximum absolute difference across all 1,280 reorderings per run and dataset.The protocol holds histories and candidate sets fixed while comparing selections by item identity across 20 orders for each of 64 examples.Source: §A.6 · p. 11

A.7 Latency Protocol

• End-to-end timing includes tokenization and input preparation through inference and selection, with method-specific decoding or packing costs included where applicable.The matched flat–tree comparison holds DICE-I fixed; model loading and warm-up are excluded, and the generation/tree comparison uses the same timing protocol.Source: §A.7 · p. 11

A.8 Flat–Tree Inference Comparison

• Flat and tree DICE-I inference achieve mean item-choice agreement of 100% on fixed Goodreads and LastFM examples.Small probability differences remain, and this test is distinct from the candidate-reordering check.Source: §A.8 · p. 11

A.9 Candidate-Count Sensitivity

• Candidate-count sensitivity is tested at K values of 5, 10, 20, 50, and 100 on the same complete test sets using nested candidate sets.Each set includes the observed target and preserves the relative order of shared candidates; HR@1 is computed separately at each count.Source: §A.9 · p. 11

• Calibration uses frozen 0.6B DICE-I checkpoints at K=20 on Goodreads and LastFM, fitting temperature on D_T and thresholds on disjoint D_τ.Table 6 also applies per-run temperature scaling to LlamaRec and InvariRank; test ECE is computed per run before averaging.Source: §A.10 · p. 11
• ECE groups selected-item probabilities into ten equal-width bins and compares each bin’s mean confidence with its fraction of correct choices.Figure 5a plots raw and calibrated probabilities separately, omits bins with fewer than ten test examples, and computes ECE over all examples.Source: §A.10 · p. 12
• Selective recommendation retains examples whose confidence meets a threshold chosen on development data, with accuracy and coverage measured on retained examples.Thresholds are frozen per run before test evaluation; test coverage may differ from the target retention rate.Source: §A.10 · p. 12
• Goodreads selects the largest development prefix whose one-sided 95% Wilson error upper bound is at most 1 − 0.90, while LastFM uses a 0.25 retention target.The Goodreads development criterion does not guarantee test accuracy; if no prefix qualifies, coverage is zero and accuracy is undefined.Source: §A.10 · p. 12

• The evaluation pairs saved 0.6B and 4B logits on identical examples and candidates, then sweeps development-retention targets in 5% steps.It averages test HR@1 and actual escalation across runs; Figure 6’s horizontal axis is mean actual test escalation.Source: §A.11 · p. 12
• The random-escalation reference combines each model’s mean full-test accuracy according to the mean test escalation rate.The reference is (1 − r_esc)a_0.6 + r_esc a_4.Source: §A.11 · p. 12
• For LastFM, the measured 0.6B and 4B tree costs are 38.34 ms and 139.85 ms, respectively, on H20.The cost model uses these component means and mean test escalation, excludes routing overhead, and does not measure an online cascade.Source: §A.11 · p. 12

• DICE-S adds residual interactions among DICE-I candidate representations through a permutation-equivariant set model.Candidate representations are projected into an interaction space, with log K shared across candidates to encode candidate-set size; attention masks padded candidates.Source: B Candidate-Interaction Extension… · p. 12
• The interaction module uses multi-head self-attention without candidate-position embeddings, and its residual head is initialized to zero.Training retains the softmax choice loss, updates LoRA, the independent head, and interaction module, and freezes the backbone.Source: B Candidate-Interaction Extension… · p. 12
• Shared projections, attention without candidate-position embeddings, and candidate-following masks preserve permutation equivariance under candidate reordering.Softmax preserves the relation, and DICE-I’s item-identifier tie rule leaves item selection unchanged.Source: B Candidate-Interaction Extension… · p. 12
• Interaction gains vary across datasets and candidate counts, making DICE-S an optional extension.Source: B Candidate-Interaction Extension… · p. 12

• The Frozen control trains only a choice head over frozen Qwen3 text representations, while ID-only uses GRU4Rec without item text or a pretrained LLM.The authors note that the controls also differ in trainable capacity and architecture.Source: C Representation and Adaptation Controls · p. 12
• DICE-I achieves higher HR@1 than both controls on Goodreads and LastFM.Table 10 reports mean HR@1 for representation and adaptation controls.Source: C Representation and Adaptation Controls · p. 12, Table 10 · p. 12