Source-linked AI summary
NPA: Neural News Recommendation with Personalized Attention
Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, Xing Xie
TL;DR
News recommendation must help users navigate large article collections while accounting for differing interests and attention to article content. NPA uses CNN-based news representations, clicked-news user representations, and user-conditioned word- and news-level attention. Experiments on MSN News validate the approach’s effectiveness, while the supplied passages also report modest test-time overhead and short training time.
Problem
Massive online news collections make it impractical for users to find relevant articles, while users differ in interests and attend to different article aspects.
Method
NPA uses CNNs for title-based news representations, clicked-news user representations, and user-ID-conditioned attention over words and news articles.
Results
Experiments on a real-world MSN News dataset validate NPA’s effectiveness for news recommendation.
Takeaways & Limitations
Personalized attention lets NPA differentially attend to important words and news articles according to user preferences.
Abstract
from arXiv · showhide
News recommendation is very important to help users find interested news and alleviate information overload. Different users usually have different interests and the same user may have various interests. Thus, different users may click the same news article with attention on different aspects. In this paper, we propose a neural news recommendation model with personalized attention (NPA). The core of our approach is a news representation model and a user representation model. In the news representation model we use a CNN network to learn hidden representations of news articles based on their titles. In the user representation model we learn the representations of users based on the representations of their clicked news articles. Since different words and different news articles may have different informativeness for representing news and users, we propose to apply both word- and news-level attention mechanism to help our model attend to important words and news articles. In addition, the same news article and the same word may have different informativeness for different users. Thus, we propose a personalized attention network which exploits the embedding of user ID to generate the query vector for the word- and news-level attentions. Extensive experiments are conducted on a real-world news recommendation dataset collected from MSN news, and the results validate the effectiveness of our approach on news recommendation.
1 INTRODUCTION
Online news platforms face information overload as users struggle to find relevant articles. NPA addresses differing user interests and informativeness across words and clicked news with personalized attention.
- Online news platforms publish massive volumes of articles, making it impractical for users to find news matching their interests.
- Users differ in interests, and clicked news articles do not all equally reflect their preferences.The same article can be more informative for one user's profile than another's.
- Words within titles also vary in informativeness, and the same word may matter differently to different users.Examples distinguish attention to “Crazy” from attention to “Actually Work” across users.
- Existing methods commonly combine collaborative filtering with news content to model interests and browsing behavior.Prior approaches include Bayesian interest modeling and recurrent networks over distributed news representations.
- NPA combines CNN-based news representations with user representations learned from clicked news.The model applies attention at both word and news levels.
- User-ID embeddings generate personalized attention queries that select important words and news according to user preferences.
2 RELATED WORK
News recommendation research has progressed from relatedness, similarity, and editorial approaches toward collaborative filtering and content-based methods. Deep learning extends this work by learning user and news representations, while incorporating content remains important.
- Earlier news recommendation methods used news relatedness, semantic similarity, or human editors’ demonstrations.
- Collaborative filtering models user interests from click behavior but face sparsity and cold-start problems in news recommendation.
- Content-based techniques commonly complement collaborative filtering by using article information and browsing histories.
- Prior neural approaches learn latent user and item factors, but methods that omit user and item content cannot exploit information important for recommendation.
3 OUR APPROACH
NPA combines news and user encoders with personalized attention to model informative words and clicked news according to user preferences. A click predictor scores candidate news using the resulting representations.
- Framework: NPA uses news, user, and click-predictor modules to represent news, represent users from clicked news, and predict candidate-news click scores.The framework applies personalized attention in the news and user encoders.
- News Encoder: The news encoder converts title words into dense vectors and applies CNN filters to capture local contextual representations.Word embeddings represent title tokens, while CNN windows capture local combinations such as “Fiesta” and “bowl.”
- News Encoder: Word-level personalized attention uses a user-ID embedding to generate a preference query that highlights informative title words for each user.Attention weights are calculated from interactions between the preference query and contextual word representations.
- User Encoder: News-level personalized attention similarly transforms the user embedding into a news preference query and weights clicked-news representations to form the user representation.This mechanism accounts for different informativeness among clicked news and across users.
- Click Predictor: The click predictor computes candidate-news scores from news–user inner products, normalizes them with softmax, and trains with a pseudo K + 1 classification objective.The candidate set contains one positive clicked news and K randomly selected negative samples; the loss uses the positive class.
4 EXPERIMENTS
Experiments show that NPA consistently outperforms the compared baselines, while personalized attention and negative sampling improve recommendation performance. The model also offers favorable computational costs, remains robust to news updates, and benefits from moderate negative sampling.
- Performance Evaluation: Neural methods outperform traditional matrix factorization, while negative sampling and attention are associated with better recommendation performance.DSSM and NPA use negative sampling, whereas DFM, DKN, and NPA use attention mechanisms.
- Performance Evaluation: NPA consistently outperforms all compared baseline methods on news recommendation.The reported improvement over all baseline methods is significant at p < 0.001.
- Computational Cost: NPA has lower training cost than feature-based methods when N is not large and only slightly higher test cost because its embedding dimension De is not large.Compared with CNN, DFM, and DKN, negative sampling divides NPA’s training cost by K; its test cost is much smaller than DKN and DFM.
- Temporal Robustness: NPA performs best on the first test day, declines over time, stabilizes after three days, and remains robust to news updates.The paper attributes the decline to time-sensitive news and reduced overlap between training and later test news.
- Personalized Attention: Personalized attention outperforms vanilla attention, and both outperform the model without attention.Personalized attention adapts word- and news-level importance to different user preferences.
- Personalized Attention: Word-level and news-level personalized attention each improve NPA, while combining both yields further improvement.Word-level attention selects informative title words, and news-level attention selects informative clicked news for user representations.
- Negative Sampling: Negative sampling improves performance, but excessive negative samples can make positive samples difficult to recognize; a moderate K, such as K = 4, is recommended.Performance first improves as K increases and then declines when K becomes too large.
- Attention Visualization: Attention visualizations show that NPA assigns higher weights to informative title words and news, with the same news receiving different weights for different users.Sports-related words and articles receive higher weights for users whose clicked-news histories indicate stronger sports interests.
5 CONCLUSION
NPA combines CNN-based title representations with user representations from clicked news, using word- and news-level personalized attention driven by user-ID embeddings. Experiments on an MSN news dataset validate the approach.
- NPA uses CNN to learn news representations from titles and models users from their clicked news articles.
- Word- and news-level attention helps the model focus on important words and news articles when representing news and users.
- A personalized attention network uses user-ID embeddings as queries so attention can reflect user-specific importance.
- Experiments on a real-world dataset collected from MSN news validate NPA's effectiveness for news recommendation.