Source-linked AI summary
DKN: Deep Knowledge-Aware Network for News Recommendation
Hongwei Wang, Fuzheng Zhang, Xing Xie, Minyi Guo
TL;DR
Online news recommendation lacks external knowledge awareness while facing rapidly changing content and diverse user interests. DKN combines knowledge-graph representations with a word–entity-aligned KCNN and candidate-conditioned attention for CTR prediction. On Bing News, it substantially outperforms deep-learning baselines, with knowledge and attention each providing additional reported improvements.
Problem
Existing news recommenders do not fully use entities, common sense, and latent knowledge connections, while news is time-sensitive and users have diverse interests.
Method
DKN is a content-based CTR model that uses knowledge-graph information through multi-channel word–entity-aligned KCNN representations and attention over clicked history.
Results
2.8% to 17.0% on F1 and 2.6% to 16.1% on AUC: DKN significantly outperforms state-of-the-art deep-learning-based recommendation baselines.
Takeaways & Limitations
The experiments validate the efficacy of incorporating knowledge and attention into DKN for news recommendation.
Takeaways & Limitations
The approach uses news titles as input, although the authors state it can generalize to other news-related texts.
Abstract
from arXiv · showhide
Online news recommender systems aim to address the information explosion of news and make personalized recommendation for users. In general, news language is highly condensed, full of knowledge entities and common sense. However, existing methods are unaware of such external knowledge and cannot fully discover latent knowledge-level connections among news. The recommended results for a user are consequently limited to simple patterns and cannot be extended reasonably. Moreover, news recommendation also faces the challenges of high time-sensitivity of news and dynamic diversity of users' interests. To solve the above problems, in this paper, we propose a deep knowledge-aware network (DKN) that incorporates knowledge graph representation into news recommendation. DKN is a content-based deep recommendation framework for click-through rate prediction. The key component of DKN is a multi-channel and word-entity-aligned knowledge-aware convolutional neural network (KCNN) that fuses semantic-level and knowledge-level representations of news. KCNN treats words and entities as multiple channels, and explicitly keeps their alignment relationship during convolution. In addition, to address users' diverse interests, we also design an attention module in DKN to dynamically aggregate a user's history with respect to current candidate news. Through extensive experiments on a real online news platform, we demonstrate that DKN achieves substantial gains over state-of-the-art deep recommendation models. We also validate the efficacy of the usage of knowledge in DKN.
1 INTRODUCTION
Online news recommendation must address information overload, rapidly expiring relevance, diverse interests, and knowledge-rich language. DKN incorporates external knowledge into content-based CTR prediction and dynamically represents users’ interests, achieving gains over deep-learning baselines.
- Online news platforms need personalized recommendation to help users navigate overwhelming article volumes.
- News recommendation is challenged by short article lifecycles, diversified user interests, and condensed language containing entities and common sense.
- DKN enriches news by linking words to knowledge-graph entities and their immediate neighbors, then fuses semantic and knowledge representations with KCNN.
- DKN uses attention to match candidate news with clicked history and aggregate the user representation with different weights for CTR prediction.
- 2.8% to 17.0% on F1 and 2.6% to 16.1% on AUC: DKN significantly outperforms state-of-the-art deep-learning-based recommendation baselines.
2 PRELIMINARIES
This section reviews knowledge-graph embeddings and CNN-based sentence representation learning. It describes graph-structure-preserving entity and relation vectors, convolutional feature extraction, pooling, and feature concatenation.
- Knowledge Graph Embedding: Knowledge-graph embedding learns low-dimensional vectors for entities and relations while preserving the graph’s structural information.
- Knowledge Graph Embedding: TransE models valid triples with h + r ≈ t, while TransH, TransR, and TransD introduce relation-specific projections or projection mechanisms.
- Knowledge Graph Embedding: A margin-based ranking loss trains these embedding methods to discriminate correct triples from incorrect triples.
- CNN for Sentence Representation Learning: CNN sentence modeling applies filters over windows of word embeddings to generate local features through convolution and a nonlinear function.
- CNN for Sentence Representation Learning: Max-over-time pooling selects the most significant feature, and multiple filter outputs are concatenated into the final sentence representation.
3 PROBLEM FORMULATION
The paper formulates news recommendation as predicting whether a user will click an unseen candidate news item from the user’s click history and word–entity connections.
- For each user, the click history is represented as a sequence of previously clicked news titles.
- Each news title is a word sequence in which words may be associated with entities in a knowledge graph.
- Examples link “Trump” to “Donald Trump” and “Las” and “Vegas” to “Las Vegas” entities.
- The formulation uses news titles as input because titles are described as decisive factors affecting users’ reading choices, while other news text is also possible.
- The model predicts whether the user will click a candidate news item that the user has not previously seen.
4 DEEP KNOWLEDGE-AWARE NETWORK
DKN combines knowledge-graph information with news-title semantics through KCNN and dynamically matches a candidate news item to a user’s clicked history. Its framework includes knowledge distillation, aligned multi-channel convolution, attention-based interest extraction, and final CTR prediction.
- 4.1 DKN Framework: DKN processes candidate and clicked news titles with KCNN, aggregates the user’s history using attention, and predicts click probability with a DNN.KCNN produces news embeddings; attention assigns different weights to clicked items relative to the candidate news.
- 4.2 Knowledge Distillation: Knowledge distillation links news mentions to knowledge-graph entities, constructs a subgraph, and expands it to obtain relational information.The supplied passage describes entity linking and subgraph construction before expansion, but its final expansion step is truncated.
- 4.2 Knowledge Distillation: Entity context is represented by immediate knowledge-graph neighbors, whose embeddings are averaged to provide complementary information for entity identification.The example includes “Fight Club” contexts such as genre, actor, country, and award.
- 4.3 Knowledge-aware CNN: KCNN addresses concatenation’s broken word-entity alignment and incompatible embedding spaces by using transformed entity and context embeddings as aligned channels.A trainable transformation maps entity-space representations into word space; word, entity, and context channels are stacked positionally.
- 4.3 Knowledge-aware CNN: Multiple filters with varying window sizes extract local title patterns, max-over-time pooling selects the largest feature, and concatenation forms the news representation.The final representation combines the pooled features from the filters.
- 4.4 Attention-based User Interest Extraction: Attention models diverse interests by assigning different impacts to clicked news for each candidate, rather than treating the entire history uniformly.The attention network uses the candidate and clicked-news embeddings to calculate normalized impact weights.
5 EXPERIMENTS
The experiments analyze a Bing News dataset and its extracted knowledge graph, highlighting news recency, user-history sparsity, and dense but long-tailed entity structure.
- Dataset analysis: The experiments use Bing News server logs and an extracted Microsoft Satori knowledge graph for dataset analysis.The logs include timestamps, user IDs, URLs, titles, and click labels; graph edges are retained when confidence exceeds 0.8.
- News dynamics: About 90% of news articles receive clicks within two days, confirming strong time sensitivity in online news.The news life cycle is measured from publication until the last received click.
- User behavior: 77.9% of users click no more than five news articles, demonstrating substantial sparsity in user histories.The distribution counts clicked news items per user.
- Title structure: News titles average 7.9 non-stop words and 3.7 entities, placing approximately one entity in every two words.The observed entity density supports the design of KCNN.
- Knowledge-graph structure: Entity occurrences are sparse and long-tailed, while knowledge-graph entities have abundant contextual connections.80.4% of entities occur no more than ten times; the average number of context entities is 42.5 and the maximum is 140,737.
5.2 Baselines
The study compares DKN with feature-based, collaborative-filtering, content-based, hybrid, and deep-learning recommendation baselines using model-comparison evaluation materials.
- Baseline models: The baseline set includes LibFM, KPCNN, DSSM, DeepFM, YouTubeNet, and other state-of-the-art recommendation methods.LibFM is feature-based, while the remaining listed baselines use deep neural networks or collaborative filtering.
- Baseline categories: Except for DMF, the baselines are content-based or hybrid methods; DMF is based on collaborative filtering.This distinction frames the comparison between content-driven and collaborative approaches.
5.3 Experiment Setup
The experiment setup specifies embedding, convolution, optimization, and baseline-representation dimensions for training DKN and enabling model comparisons.
- DKN configuration: DKN uses TransD for knowledge-graph processing, 100-dimensional word and entity embeddings, and 100 filters for window sizes 1 through 4.The model is trained with Adam by optimizing log loss.
- Baseline configuration: Baseline configurations use 100-dimensional representations for KPCNN, DSSM, DeepWide, YouTubeNet, and the factorization settings specified for LibFM, DeepFM, and DMF.The passage lists the key dimensionality settings used for each baseline.
5.4 Results
DKN outperforms the compared models, remains consistently competitive across ten days, and benefits from knowledge, transformation, and attention components.
- Model comparison: Table 2 compares DKN and variants against baseline models using F1, AUC, and statistical significance tests.The comparison also removes entity embeddings from baselines when present.
- Model comparison: Deep-learning baselines outperform LibFM by 2.0% to 5.2% on F1 and 1.5% to 4.5% on AUC, excluding DMF.The results are attributed in the passage to deep models capturing nonlinear relations and dependencies.
- Temporal evaluation: DKN’s AUC curve stays above the baselines across ten test days, with lower variance than the baselines.The passage presents this as evidence of competitiveness, robustness, and stability in practical application.
- DKN variants: Entity and contextual embeddings improve DKN’s AUC by 1.3% and 0.7%, respectively, with further gains when combined.The result supports using knowledge-graph information in DKN.
- DKN variants: The attention network adds 1.7% on F1 and 0.9% on AUC for DKN.The variant comparison evaluates the attention design alongside knowledge embeddings and transformation functions.
5.5 Case Study
The case study examines how DKN’s attention responds to users’ training and test logs, while sensitivity experiments show that representation and KCNN settings require moderation to avoid noise or overfitting.
- Attention Visualization: The case study uses a randomly sampled user’s training and test logs, with training logs labeled 0 omitted for simplicity.The clicked news were manually tagged as Cars for entries 1–3 and Politics for entries 4–6.
- Attention Visualization: DKN’s attention network captures relatedness within the user’s Cars and Politics interests using knowledge-graph connections beyond explicit word overlap.Without entity and context embeddings, attention is strongest when titles share “Tesla”; with them, related entities such as General Motors, Ford Inc., and Tesla Inc. are connected.
- Parameter Sensitivity: The parameter-sensitivity experiments vary one parameter at a time while keeping all others fixed to the settings introduced earlier.This isolates the reported effects of the tested hyper-parameter choices.
- Parameter Sensitivity: Word and entity embedding dimensions initially improve performance but hurt it when excessively large, with d = 200 cited as a noisy setting.The same qualitative pattern is reported for entity-embedding dimension k when word-embedding dimension d is fixed.
- Parameter Sensitivity: Increasing KCNN filter count generally raises AUC, but m = 200 reverses the trend because of probable overfitting.Window-size sensitivity follows similar rules when the filter count m is fixed.
6 RELATED WORK
Related work spans content-based and hybrid news recommendation, knowledge-graph representation, and deep recommender systems. The paper positions DKN as the first work, to its knowledge, to leverage knowledge-graph embedding for news recommendation.
- News Recommendation: Content-based and hybrid methods address news recommendation because collaborative filtering suffers from cold-start problems when news items are frequently replaced.Prior work also includes nonpersonalized methods modeling news relatedness or learning from human editors’ demonstrations.
- Knowledge Graph: Knowledge-graph representation methods learn low-dimensional vectors for entities and relations while preserving the graph’s structural information.The surveyed approaches include translation-based methods, Structured Embedding, Latent Factor Model, Neural Tensor Network, and GraphGAN.
- Knowledge Graph: The paper states that it is the first work, to the authors’ knowledge, to leverage knowledge-graph embedding in news recommendation.The passage contrasts this use with prior applications in movie recommendation, word embedding, and question answering.
- Deep Recommendation: Deep recommender systems broadly use neural networks either to process raw user or item features or to model user–item interactions.The related methods listed include DSSM, DeepWide, DeepFM, YouTubeNet, and DMF.
7 CONCLUSIONS
The conclusion presents DKN as a content-based CTR model that combines knowledge-aware news representations with attention over diverse user histories. Experiments on Bing News report superiority over strong baselines and support the value of entity embeddings and attention.
- Conclusions: DKN uses a content-based deep model for click-through rate prediction, making it suitable for highly time-sensitive news.The conclusion contrasts this design with ID-based methods such as collaborative filtering.
- Conclusions: KCNN jointly learns semantic-level and knowledge-level news representations through aligned word and entity channels.The alignment maintains correspondence between different embeddings associated with each word.
- Conclusions: DKN’s attention module dynamically aggregates a user’s historical representation according to the different impacts of diverse interests on candidate news.The module calculates the user representation relative to the current candidate.
- Conclusions: Experiments on a Bing News dataset demonstrate significant superiority over strong baselines and the efficacy of knowledge entity embeddings and attention.The conclusion reports both overall comparative gains and component-level efficacy.