PaperlayerView exact revision on arXiv

Source-linked AI summary

Routing Between Generative and Collaborative User Profiles: A Serving-Time Gate for Controllable Novelty

Milad Sabouri, Neeraj Sharma, Sardar Hamidian, Shaghayegh Agah

arXiv:2609.39043v1cs.IR

TL;DR

Profile-based recommendation can increase exposure to less-popular content, but uniform use can incur substantial relevance loss, motivating selective routing. The paper trains a serving-time gate to choose between collaborative and profile-based models; at comparable relevance cost, it improves Novelty@10 over random and heuristic routing.

  • Problem

    Profile-based recommendation can increase exposure to less-popular content, but uniform deployment incurs large ranking-relevance losses, motivating selective rather than global use.

    Source: §1 · p. 1

  • Method

    A serving-time gate uses user information to route each user between a collaborative sequential recommender and a profile-based model, with a threshold controlling routing volume.

    Source: §1 · p. 1, §3.2 · p. 2

  • Results

    At comparable relevance cost, the learned gate routes 12.5% of users for a 6.5% Novelty@10 gain, exceeding niche-based (4.7%), random (4.1%), and short-history (2.7%) routing.

    Source: §4.2 · p. 4

  • Takeaways & Limitations

    Comparable or slightly stronger trade-offs from non-generative Centroid support selective routing of profile-based recommenders generally, rather than an inherent frontier advantage from LLM generation.

    Source: §4.2 · p. 4

    The study is offline and limited to one production streaming platform and the common test population; online threshold calibration and robustness under distribution shift remain future work.

    Source: §4.2 · p. 4

Abstract

from arXiv · show

Large language models (LLMs) enable rich semantic user profiles for recommendation, but such profiles are more expensive to generate and are not necessarily desirable to deploy uniformly. We study whether LLM-generated profiles can instead be invoked selectively within a production recommendation pipeline. Using a real-world streaming dataset covering movies, TV shows, and sports content, we train a serving-time routing gate that assigns each user to either a collaborative sequential recommendation model or a recommendation model driven by an LLM-generated profile. The gate uses only serving-time features and learns to identify users for whom profile-based routing can increase Novelty@10 while preserving ranking relevance. A routing threshold controls how aggressively users are sent to the generative model, exposing a tunable novelty--relevance trade-off. At an overall NDCG-loss budget of 5\%, the learned gate increases Novelty@10 by 6.5\% while routing 12.5\% of users, outperforming simple heuristic and random routing policies at comparable relevance cost. These results show that LLM-generated user profiles can serve as a controllable complement to collaborative recommendation, while results with non-generative semantic profiles indicate that the benefit stems from selective routing rather than LLM generation alone.

1 Introduction

Collaborative recommenders provide relevant recommendations but can favor popular content, while profile-based models can increase novelty at a substantial relevance cost. The paper studies selective serving-time routing, finding that a learned gate can improve novelty within a bounded relevance budget and outperform simple heuristics and random routing.

1 Introduction

  • Collaborative sequential recommenders are relevant but can favor well-covered popular content, while LLM-generated profiles offer a complementary representation of user preferences.Prior work found that profile effectiveness relative to aggregate semantic profiles varies by user behavior and can involve beyond-accuracy trade-offs.Source: §1 · p. 1
  • The gate uses serving-time information to route users selectively between collaborative and profile-based recommenders, with a threshold controlling the novelty–relevance trade-off.The candidate profiles include LLM-generated representations.Source: §1 · p. 1
  • At an approximately 5% NDCG-loss budget, routing 12.5% of users increases Novelty@10 by 6.5%.The learned gate also produces larger novelty gains than random routing and simple heuristics.Source: §1 · pp. 1–2
  • The paper frames selective deployment as a serving-time routing problem that treats novelty as a reward and ranking relevance as a guardrail.Its production-data results show a tunable novelty–relevance trade-off and more effective allocation of a fixed relevance budget than heuristic or random routing.Source: §1 · p. 2

2 Related Work

Prior work develops semantic and generative user profiles, studies recommendation objectives beyond relevance, and applies gating or model cascades to selective computation. This paper focuses on when to serve a profile-based model instead of a collaborative sequential model under an explicit relevance budget.

Semantic and generative user profiling

  • LLM-generated preference summaries can be embedded for recommendation, and temporal variants combine short- and long-term representations through learned attention.Prior production-scale evaluation found that relative effectiveness can vary across consumption regimes and beyond-accuracy objectives [11].Source: §2 · p. 2

Beyond-accuracy objectives

  • Novelty and diversity are beyond-accuracy objectives, and this paper treats Novelty@10 as a routing reward while constraining NDCG@10 degradation.This exposes operating points along a relevance–novelty trade-off.Source: §2 · p. 2

Adaptive routing and selective computation

  • Unlike prior adaptive gating and model-cascade settings, this gate makes a serving-time per-user choice between collaborative and profile-based recommenders under an explicit relevance budget.Changing the routing threshold selects different operating points.Source: §2 · p. 2

3 Methodology

The method routes each user between a collaborative sequential model and a semantic recommendation model using information available at serving time. Profile variants define separate candidate models and gates, while a threshold controls the share routed and the resulting novelty–relevance trade-off.

3 Methodology

  • The serving-time router selects whether the collaborative sequential model or a candidate semantic model produces each user’s final recommendation list.The collaborative model is denoted C and the semantic model G.Source: §3 · p. 2

3.1 User Representations

  • The candidate profiles are Narrative, TemporalNarrative, and Centroid, with a separate semantic recommendation model and routing gate for each representation.Narrative encodes one generated summary with SBERT; TemporalNarrative combines short- and long-term summaries through learned attention; Centroid mean-pools item embeddings.Source: §3.1 · p. 2

3.2 Learned Routing Gate

  • A Gradient Boosting classifier estimates the probability that routing improves novelty without reducing relevance, using serving-time features derived from user history and representations.Features include history length, consumed-item popularity statistics, niche ratio, and collaborative and semantic representation norms; threshold sweeps select operating points after training, with five-fold out-of-fold routing predictions.Source: §3.2 · p. 2

4 Experimental Evaluation

The evaluation tests serving-time routing between collaborative and profile-based recommenders on a production streaming dataset, then measures whether thresholded routing can improve novelty within a relevance-loss budget. The learned gate outperforms heuristic and random policies at a matched 5% budget, while the study’s offline, single-platform scope limits deployment conclusions.

4.1 Experimental Setup

  • The evaluation samples 10,000 users from a production streaming platform and holds recommendation models and representations fixed, learning separate Gradient Boosting gates scored out of fold.Policies are compared at matched overall NDCG loss, with each selecting its highest-Novelty operating point within the 5% budget.Source: §4.1 · pp. 2–3

4.2 Results and Discussion

  • At a shared ≤5% NDCG-loss budget, the learned gate routes 12.5% of users for +6.5% Novelty@10, versus 4.7% niche, 4.1% random, and 2.7% short-history routing.The learned gate’s novelty gain is 1.8 percentage points larger than the strongest heuristic; the 5% budget is a comparison setting, not a universal deployment target.Source: Figure 1 · p. 3, §4.2 · p. 4
  • Uniform profile-based deployment substantially increases Novelty@10 but incurs large NDCG@10 losses, motivating selective routing instead of replacing the collaborative model globally.The LLM-generated profiles yield the largest uniform novelty gains, but at greater relevance cost.Source: §4.2 · pp. 3–4
  • Across the probability-threshold sweep, 1%, 5%, and 10% NDCG-loss budgets route 2.9%, approximately 12.0%, and approximately 20.0% of users, with Novelty@10 gains of 1.2%, 6.2%, and 11.1%, respectively.Bootstrap confidence intervals remain above zero at all four reported operating points, and paired Wilcoxon tests find significant novelty improvements (p < 0.001).Source: Table 2 · p. 4, §4.2 · p. 4
  • The study is offline and limited to one production streaming platform and the common test population; online threshold calibration and robustness under distribution shift remain future work.Source: §4.2 · p. 4

5 Conclusion

The paper finds that selective routing can increase novelty while limiting relevance loss, and that the value of generative profiles depends on when they are used. It identifies further comparisons with fusion and unified-ranker approaches as future work.

  • 6.5% higher Novelty@10 under an approximately 5% overall NDCG-loss budget, outperforming random and simple heuristic routing at comparable relevance cost.The gate routes users from the collaborative sequential model to an LLM-profile model using serving-time features.Source: §5 · p. 4
  • The routing threshold exposes a controllable novelty–relevance trade-off, while results with additional profile types support selective routing beyond one profile type.Source: §5 · p. 4
  • The results do not establish that generative profiles universally outperform simpler semantic alternatives.Source: §5 · p. 4
  • Comparing per-user routing with score-level fusion and unified rankers under a shared candidate universe and deployment cost model remains future work.Source: §5 · p. 4

A Routing Implementation and Reproducibility Details

The routing gates use serving-time user and representation features, with out-of-fold scores thresholded to form routing policies. Feature importance highlights the collaborative representation norm across all three profile types.

  • Narrative and TemporalNarrative are generated with Llama 3.3 70B Instruct and encoded with 384-dimensional SBERT; Centroid mean-pools item embeddings from the same encoder.Source: A Routing Implementation and… · p. 5
  • Routing features include history length, popularity summaries, niche ratio, and collaborative and candidate-profile representation norms; none uses future interactions or held-out outcomes.History length captures available behavioral evidence, popularity measures summarize user orientation, and the niche ratio is the share of consumed items below median popularity.Source: A Routing Implementation and… · p. 5
  • The collaborative representation norm is the most important feature for all three profile types, with mean importance of 0.30–0.35.Popularity characteristics are also consistently informative, while history-length features provide secondary signal; importances are not causal effects.Source: A Routing Implementation and… · p. 5
  • The gate uses serving-time features to score users, then thresholds the score to select the collaborative or profile-based recommendation model.For Narrative, the candidate-profile norm has zero importance across all five folds, so the fitted gate does not require that representation to make the routing decision.Source: A Routing Implementation and… · p. 5
  • Separate Gradient Boosting gates are trained for Centroid, Narrative, and TemporalNarrative with binary log-loss, median imputation, and no feature scaling.The configuration uses a fixed random seed of 42 and otherwise default classifier settings.Source: A Routing Implementation and… · p. 5
  • Five-fold stratified out-of-fold predictions are thresholded into policies, while NDCG-loss budgets select operating points after training rather than changing the classifier objective.Each fold's gate predicts users whose outcome-derived routing labels were not used to train it.Source: A Routing Implementation and… · p. 5

B Additional Routing Results

Additional learned-gate results show a similar novelty–relevance routing pattern for Centroid and TemporalNarrative, with point estimates differing between the profile types.

Generative AI Usage Disclosure

The disclosure notes that LLMs generated the two named profile types and that generative AI also assisted with code and manuscript editing; authors verified the experiments, results, and claims.

Loading 2609.39043v1…