Graph Enhanced Representation Learning for News Recommendation

1 INTRODUCTION

• Sparse user histories and short, vague news titles make it difficult for existing methods to learn accurate user and news representations.Prior methods typically build news representations from content and user representations from clicked news.Source: §1 · pp. 1–2
• In a click-based bipartite graph, news clicked by the same user and users sharing clicked news are neighbors.Neighbor information may complement vague news content and enrich representations when user histories are sparse.Source: §1 · p. 2
• The proposed method uses a transformer for news semantics, graph attention over neighbor news, and neighbor-user information to enhance representations.It also incorporates news topic information and news ID embeddings when modeling neighbor news.Source: §1 · p. 2
• The approach reports improved performance over a set of well-known baselines.The authors present the result as validation of their approach's effectiveness.Source: §1 · p. 2

2 RELATED WORK

• Prior news recommendation methods use textual content for news representations and aggregate or sequence clicked-news representations for users, but neglect neighbor relatedness.Examples include denoising autoencoders with recurrent user encoders, CNN news encoders with GRU user encoders, and LSTM click-history encoding.Source: §2 · p. 2
• Graph recommender systems combine node attributes with graph structure, but may not fully exploit news-text semantics; this work uses transformers for context-aware news embeddings.Prior examples include knowledge-graph signals [27] and visual and annotation features [35].Source: §2 · p. 2

3 OUR APPROACH

• GERL combines one-hop representations of candidate news and users with two-hop graph-attention embeddings of neighboring users and news.The one-hop module uses candidate-news content and users’ historical clicks; the two-hop module learns neighbor embeddings.Source: §3 · p. 2, §3.3 · p. 4

3.1 Transformer for Context Understanding

• A single-layer multi-head self-attention transformer encodes title-word interactions, and additive word attention aggregates words into title representations.The design simplifies the transformer to avoid performance degradation from excessive parameters; the original architecture performed sub-optimally.Source: §3.1 · p. 3

3.2 One-hop Interaction Learning

• The one-hop module represents candidate news from transformer-encoded content and users from attended clicked-news vectors plus trainable user-ID embeddings.News topics are embedded and concatenated with the title vector to form the news representation.Source: §3.2 · pp. 3–4, §3.1 · p. 3

3.3 Two-hop Graph Learning

• The two-hop module uses graph attention to aggregate neighbor-user ID embeddings and neighbor-news ID and semantic representations.Neighbor-news semantic representations are computed with the transformer, allowing newly published news absent from the predefined news-ID embedding matrix to be represented.Source: §3.3 · p. 4

3.4 Recommendation and Model Training

• Final user and news representations concatenate their one-hop and two-hop outputs; an inner product predicts user-news scores, trained as a positive-versus-λ unclicked-news classification task.The training objective minimizes the positive-class log-likelihood.Source: §3.4 · p. 4

4.1 Datasets and Experimental Settings

• MSN logs from Dec. 13, 2018–Jan. 12, 2019 were split by time, with the last week for testing; evaluation averaged AUC, MRR, nDCG@5, and nDCG@10 over impressions.Each experiment was repeated five times and the average results were reported.Source: §4.1 · pp. 4–5

4.2 Performance Evaluation

• GERL, combining textual understanding and graph relatedness, outperformed all other methods; GERL-Graph also consistently beat textual baselines and outperformed NGCF.Table 2 reports the performance scores and standard variations for the methods.Source: Table 2 · p. 5, §4.2 · p. 5

4.3 Effectiveness of Graph Learning

• Adding neighbor-user information improved performance more than adding neighbor-news information; neighbor-news semantics contributed more than news IDs, and combining graph components performed best.The authors note that neighbor users may help represent users with sparse histories.Source: §4.3 · pp. 5–6, Figure 3 · p. 6

4.4 Ablation Study on Attention Mechanism

• Both transformer attentions helped, with self-attention contributing more; one-hop attention mattered more than two-hop attention, and using both yielded the best performance.The study attributes these roles to word-interaction modeling, important-word selection, clicked-news selection, and neighbor weighting.Source: §4.4 · p. 6, Figure 4 · p. 6

4.5 Hyperparameter Analysis

• Performance rose from 2 to 8 attention heads, then dropped slightly; increasing graph degree helped but gains smoothed beyond 15, leading to settings of 8 heads and degree 15.The authors suggest too many head parameters may be suboptimal for concise news titles.Source: §4.5 · p. 6, Figure 5 · p. 6

• Recommendation performance improved on a real-world dataset, validating the proposed approach.Source: §5 · p. 6
• The one-hop module builds news representations with a transformer and user representations by attentively aggregating clicked news.Source: §5 · p. 6
• The two-hop module uses graph attention to aggregate neighbor embeddings, enriched with news IDs and textual content.Source: §5 · p. 6