Source-linked AI summary
Neural News Recommendation with Attentive Multi-View Learning
Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang, Xing Xie
TL;DR
Personalized news recommendation must help users navigate overwhelming article volumes, while existing representation-learning methods may use insufficient single-source information. NAML combines title, body, and category views with attention over words, views, and clicked news, and experiments show improved recommendation performance over many baselines.
Problem
Personalized news recommendation is needed to help users find relevant articles amid information overload, while existing methods often rely on single news information sources for representation learning.
Method
NAML uses a news encoder with attentive multi-view learning over titles, bodies, and categories, plus word- and view-level attention, and a user encoder with attention over clicked news.
Results
NAML consistently outperforms other baseline methods and improves news recommendation performance in experiments on a real-world dataset.
Takeaways & Limitations
Modeling multiple news information types and selecting informative words, views, and clicked news yields more informative news and user representations for recommendation.
Abstract
from arXiv · showhide
Personalized news recommendation is very important for online news platforms to help users find interested news and improve user experience. News and user representation learning is critical for news recommendation. Existing news recommendation methods usually learn these representations based on single news information, e.g., title, which may be insufficient. In this paper we propose a neural news recommendation approach which can learn informative representations of users and news by exploiting different kinds of news information. The core of our approach is a news encoder and a user encoder. In the news encoder we propose an attentive multi-view learning model to learn unified news representations from titles, bodies and topic categories by regarding them as different views of news. In addition, we apply both word-level and view-level attention mechanism to news encoder to select important words and views for learning informative news representations. In the user encoder we learn the representations of users based on their browsed news and apply attention mechanism to select informative news for user representation learning. Extensive experiments on a real-world dataset show our approach can effectively improve the performance of news recommendation.
1 Introduction
Personalized news recommendation addresses information overload, but learning news and user representations from single information sources may be insufficient. NAML instead combines multiple news views and attention mechanisms to select informative words, views, and browsed news.
- Motivation: Personalized recommendation helps users find relevant news amid the large volume of articles generated each day.Online services aggregate news from many sources, making it impossible for users to read everything within their time limits.
- Research gap: Existing deep learning methods commonly learn news and user representations from a single news information source.Examples include body-based auto-encoding followed by GRU-based user modeling and title-based knowledge-aware CNN representations.
- Motivation: Titles, bodies, and topic categories provide complementary information for representing news and inferring user interests.Bodies can add detail, titles can be precise, and categories can reveal recurring interests across clicked articles.
- Motivation: Different words, views, and browsed news vary in informativeness across articles and users.The paper illustrates this with precise versus vague titles, informative versus less informative words, and specific versus highly popular clicked news.
- Proposed approach: NAML learns unified news representations from titles, bodies, and categories while using word-, view-, and news-level attention.Its news encoder selects important words and views, while its user encoder selects informative clicked news for user representation learning.
2 Related Work
Earlier news recommendation methods used manual feature engineering or deep models built around limited information sources. These approaches motivate learned representations that better capture contextual information from news.
- Handcrafted methods: Manual-feature methods represent news with handcrafted topics, locations, lengths, and categories, requiring substantial domain knowledge and effort.They also cannot capture word contexts and orders that matter for news and user representation learning.
- Deep learning methods: Deep learning methods replaced some manual engineering with learned news and user representations.Examples include denoising autoencoders for news bodies, GRUs for browsed-news users, and knowledge-aware CNNs for news titles.
- Deep learning methods: Prior deep models often rely on a single primary news source, such as article bodies or titles.The cited examples use bodies with autoencoders or titles with knowledge-aware CNNs rather than jointly modeling multiple news views.
3 Our Approach
NAML learns news representations by encoding titles, bodies, categories, and subcategories as separate views, then uses attention over words, views, and browsed news to represent users.
- Overall architecture: NAML contains a multi-view news encoder, an attention-based user encoder, and a click predictor for candidate-news browsing probability.The news and user representations support click prediction through the interaction between a user representation and candidate-news representation.
- News encoder: Title and body encoders transform word sequences into contextual representations with embeddings and CNNs, then apply word-level attention to emphasize informative words.The title encoder uses CNNs to capture local contexts, while the body encoder similarly captures contextual word representations before attention-based pooling.
- News encoder: The category encoder embeds category and subcategory IDs and transforms them into hidden representations with dense layers and ReLU activations.Category information is represented separately from textual views through category ID embeddings followed by dense transformations.
- News encoder: View-level attention weights title, body, category, and subcategory representations before summing them into a unified news representation.The model assigns different weights to views because their informativeness can vary across news articles; the unified representation is r = αcrc + αscrsc + αtrt + αbrb.
- User encoder: The user encoder applies news attention to browsed-news representations and sums the weighted representations to form a user representation.This mechanism selects more informative browsed news when modeling user preferences.
- Click prediction: The click predictor scores a candidate news article by the inner product of its representation with the user representation.The resulting score estimates the probability that the user browses the candidate news.
4 Experiments
Experiments on a real-world MSN News dataset evaluate NAML against multiple baselines and test its multi-view and attention components. NAML consistently outperforms other baselines, while combining views and attention mechanisms further improves performance.
- Datasets and Experimental Settings: The dataset samples MSN News user logs from December 13, 2018 to January 12, 2019, using the last week for testing and 10% of training samples for validation.The remaining logs were used for training.
- Performance Evaluation: NAML is compared with LibFM, CNN, DSSM, Wide&Deep, DeepFM, DFM, and DKN in the performance evaluation.The experimental results are summarized in Table 2.
- Performance Evaluation: Neural-network methods such as CNN, DSSM, and NAML outperform the traditional matrix-factorization method LibFM.The authors attribute this pattern probably to better learned news and user representations.
- Performance Evaluation: DSSM and NAML outperform CNN, DFM, and DKN, which do not use negative sampling.The authors suggest negative sampling provides information from more negative samples and better matches real-world recommendation scenarios.
- Performance Evaluation: NAML consistently outperforms other baseline methods by combining titles, bodies, and categories as views and selecting informative words, news, and views.Simply concatenating features from divergent news sources may be suboptimal, whereas NAML uses multi-view learning and attention mechanisms.
- Effectiveness of Attentive Multi-View Learning: Combining body, title, and category views further improves performance over single-view variants, while combining word-, news-, and view-level attention networks further improves NAML.The body view performs better than title-only or category-only variants, but titles and categories also provide useful information.
- Visualization of Attention Weights: View-level attention assigns higher average weights to body than title and the highest weights to category, while downweighting vague titles and bodies.The authors interpret categories as complementary information when titles and bodies are uninformative.
- Visualization of Attention Weights: Word-level attention highlights informative title words such as “NBA” and “Rockets” while assigning low weight to less informative words such as “December.”Because news bodies are usually too long, the visualization shows word-level weights only for titles.
5 Conclusion
NAML learns unified news representations from titles, bodies, and categories while selecting important words and views, and it uses browsed-news attention to represent users. Experiments on a real-world dataset show improved news recommendation performance and outperformance of many baseline methods.
- NAML combines titles, bodies, and categories as distinct views to learn unified news representations.
- Attention mechanisms select important words and views for news representations and important browsed news for user representations.
- Experiments on a real-world dataset show that NAML improves news recommendation performance and outperforms many baseline methods.