Simplifying Content-Based Neural News Recommendation: On User Modeling and Training Objectives

1 INTRODUCTION

• Most neural news recommenders combine news encoders with user encoders that aggregate clicked-news embeddings before comparing them with a candidate, a paradigm termed early fusion.Models also commonly compare candidate and user embeddings with a dot product or feed-forward scorer and train with classification objectives and negative sampling.Source: §1 · p. 1
• Prior comparisons are hindered by proprietary datasets and inconsistent splits or evaluation protocols, while simpler user modeling and contrastive objectives remain underexplored.Existing models often add complexity to early-fusion user encoders without much empirical justification.Source: §1 · p. 1
• The paper introduces a unified framework for fair comparisons across candidate-awareness, click-behavior fusion, and training objectives.It proposes late fusion, which pools candidate-to-clicked-news dot-product scores instead of using complex user encoders.Source: §1 · p. 2
• The authors report that late fusion often outperforms early fusion and that supervised contrastive training is a viable alternative to point-wise classification.The work focuses exclusively on NNR models that do not use graph-based modeling of user relations.Source: §1 · pp. 1–2

2 METHODOLOGY

• The evaluation framework crosses candidate-agnostic or candidate-aware user modeling, early or late click fusion, and cross-entropy or supervised contrastive training.Figure 1 maps these three design dimensions to the framework's green, orange, and purple boxes, respectively.Source: §2 · p. 2, Figure 1 · p. 2

2.1 User Modeling

• Candidate-agnostic models form user embeddings without conditioning on the candidate, while candidate-aware models condition clicked-news representations on candidate content.The evaluated candidate-agnostic models include attention- and recurrent-based user encoders; DKN and CAUM represent candidate-aware designs, with CAUM combining candidate-conditioned long- and short-term modeling.Source: §2.1 · p. 2

2.2 Click Behavior Fusion

• Late fusion replaces a dedicated user encoder with the mean of dot-product scores between a candidate embedding and embeddings of the user's clicked news.The resulting average can also serve as a parameterless user embedding, making late-fusion models candidate-agnostic.Source: §2.2 · p. 3

2.3 Training Objectives

• All evaluated models are trained with either cross-entropy loss using negative sampling or supervised contrastive loss.The paper uses the contrastive objective to examine an alternative to the predominantly used classification regime.Source: §2.3 · p. 3

3 EXPERIMENTAL SETUP

• Experiments use MINDsmall and MINDlarge; because test labels are unreleased, each dataset's validation portion serves as the test set.The training portion is split by time, using its first four days for training and its last day for validation.Source: §3 · p. 3
• Models use pretrained GloVe word embeddings and TransE entity embeddings, a maximum history length of 50, and four sampled negatives per positive example.Experiments are repeated with five random seeds and report averages and standard deviations for AUC, MRR, nDCG@5, and nDCG@10.Source: §3 · p. 3

Results setup

• Table 2 reports performance for candidate-agnostic and candidate-aware models across fusion and training-objective configurations, with averages and standard deviations over five runs.The comparison covers MINDsmall and MINDlarge.Source: §4 · p. 3, Table 2 · p. 4

Candidate-Agnostic vs. Candidate-Aware NNRs

• Under default early fusion, CAUM generally outperforms other models, but DKN performs worse than candidate-agnostic models and late-fusion variants match or surpass CAUM.CAUM's advantage is particularly prominent on MINDlarge for AUC; NAML and MINS are competitive on MINDsmall except for AUC, but fall behind CAUM on MINDlarge.Source: §4 · p. 3
• Model comparisons are confounded because systems differ in their news encoders and input features as well as their user encoders.With identical news encoders, NAML and MINS perform similarly on MINDsmall, while MINS gains over NAML on MINDlarge only with standard early fusion and cross-entropy training.Source: §4 · p. 3

Early vs. Late Click Behavior Fusion

• Averaged across models and objectives, late fusion gains 5.58 MRR points on MINDsmall and 4.63 on MINDlarge, while making model performance more similar.The authors suggest late fusion also makes differences between news encoders less consequential.Source: §4 · p. 4

Cross-Entropy vs. Supervised Contrastive Loss

• Supervised contrastive learning gains 8.26 AUC points on MINDsmall and 12.14 on MINDlarge, but trails cross-entropy by 1.54 and 1.78 MRR points, respectively.The AUC gains are averaged across models and hold in both early- and late-fusion variants; the MRR declines are attributed tentatively to hard negatives.Source: §4 · p. 4

Model Size

• Late fusion reduces model size by 14.7% for CenNewsRec, 18.1% for CAUM, and 82.3% for LSTURini.The user encoder accounts for the largest share of parameters in LSTUR; the parameter comparison uses original early-fusion configurations on MINDsmall.Source: §4 · p. 4, Figure 2 · p. 4

• The unified evaluation framework focuses on candidate-awareness, click-behavior fusion, and training objectives in neural news recommendation.Source: §5 · p. 4
• Replacing complex user encoders with parameterless aggregation of clicked-news embeddings brings performance gains while reducing model complexity.Source: §5 · p. 4
• Contrastive learning is a viable alternative to standard classification-based cross-entropy loss.Source: §5 · p. 4
• The authors hope the findings encourage more transparent evaluation and systematic ablations to identify components driving performance.Source: §5 · p. 4