Source-linked AI summary
Knowledge-Enhanced Personalized Review Generation with Capsule Graph Neural Network
Junyi Li, Siqing Li, Wayne Xin Zhao, Gaole He, Zhicheng Wei, Nicholas Jing Yuan, Ji-Rong Wen
TL;DR
Prior personalized review generation often lacks factual product descriptions and focuses on word-level generation, limiting direct modeling of higher-level preferences. This paper constructs an HKG and uses Caps-GNN in a two-stage generator; experiments on three datasets are reported to demonstrate the model’s effectiveness.
Problem
Prior PRG studies often omit factual product descriptions and focus on word-level generation rather than directly modeling users’ preferences across aspects.
Source: §1 · p. 1Method
The model builds an HKG, learns graph capsules with Caps-GNN, and uses adaptive aspect capsules and graph-based copying in aspect-sequence and sentence generation.
Source: §1 · p. 2Results
Experiments on three review datasets demonstrate the effectiveness of KG information and the proposed model for PRG.
Source: §1 · p. 2Takeaways & Limitations
The approach incorporates KG information to represent user preference at both aspect and word levels in review generation.
Source: §1 · p. 2Evaluation uses only three datasets with aligned item–entity links.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Personalized review generation (PRG) aims to automatically produce review text reflecting user preference, which is a challenging natural language generation task. Most of previous studies do not explicitly model factual description of products, tending to generate uninformative content. Moreover, they mainly focus on word-level generation, but cannot accurately reflect more abstractive user preference in multiple aspects. To address the above issues, we propose a novel knowledge-enhanced PRG model based on capsule graph neural network~(Caps-GNN). We first construct a heterogeneous knowledge graph (HKG) for utilizing rich item attributes. We adopt Caps-GNN to learn graph capsules for encoding underlying characteristics from the HKG. Our generation process contains two major steps, namely aspect sequence generation and sentence generation. First, based on graph capsules, we adaptively learn aspect capsules for inferring the aspect sequence. Then, conditioned on the inferred aspect label, we design a graph-based copy mechanism to generate sentences by incorporating related entities or words from HKG. To our knowledge, we are the first to utilize knowledge graph for the PRG task. The incorporated KG information is able to enhance user preference at both aspect and word levels. Extensive experiments on three real-world datasets have demonstrated the effectiveness of our model on the PRG task.
1 INTRODUCTION
Prior PRG methods often produce reviews with little factual product information and focus on word-level generation, leaving higher-level user preferences difficult to model. The paper proposes a KG-enhanced, two-stage approach to represent preferences at aspect and word levels.
1 INTRODUCTION
- Existing PRG methods can produce uninformative reviews lacking product facts and struggle to model user preferences above the word level.Prior features mined from reviews alone may not capture diverse, comprehensive product facts.Source: §1 · p. 1
- The model augments a knowledge graph with user and word nodes, using interaction and review co-occurrence links to form a heterogeneous knowledge graph.The graph supports item-fact enrichment and inference of users’ preferences for aspects or relations.Source: §1 · p. 2
- Caps-GNN extracts graph capsules, from which adaptive aspect capsules guide aspect-sequence generation.Each aspect capsule is associated with an aspect obtained from unsupervised topic models.Source: §1 · p. 2
- A graph-based copy mechanism incorporates related entities or words from the HKG during sentence generation, supporting word-level enrichment.The model uses KG information at both aspect and word levels.Source: §1 · p. 2
- The authors report that their model is the first to use KG for both aspect- and word-level preference in PRG, and evaluate it on three datasets.They report that experiments demonstrate the effectiveness of KG information and the model.Source: §1 · p. 2
2 RELATED WORK
Prior review-generation work uses context and review-derived features, while the paper focuses on external knowledge graphs to model users’ writing preferences. It distinguishes this goal from modeling users’ item-adoption preferences.
2 RELATED WORK
- RNN, GAN, and side-information approaches have been used for review generation, but review-mined features may not cover diverse semantic information.Earlier structural-knowledge work enriched text diversity without using KG information to learn users’ writing preferences.Source: §2 · p. 2
- Recommendation studies use reviews to explain user–product interactions and focus on item-adoption preference rather than writing preference.They also rely on review text to learn explanations of adoption behavior.Source: §2 · p. 2
- Capsule-network work cited by the authors addresses aspect-level sentiment classification, whereas this paper uses KG data to infer aspects for review generation.Source: §2 · p. 2
3 PROBLEM FORMULATION
The task is to generate a user’s review for an item given the user, item, rating, and any available context, using an aspect sequence followed by aspect-conditioned sentences. The formulation incorporates an HKG as an additional resource for capturing preferences at both levels.
3 PROBLEM FORMULATION
- The HKG adds user and word nodes to the original KG, linking users to items through interactions and entities to words through review co-occurrence.These links support inference of user preferences through user-to-entity paths and capture entity–word relatedness.Source: §3 · p. 3
- Personalized review generation produces review text for a user and item given a rating and possible context.The formulation seeks to maximize the joint probability of aspect and word sequences.Source: §3 · p. 3
- The generation process first predicts an aspect sequence, then generates each sentence conditioned on its aspect label.The formulation assumes each sentence or shorter text segment has one aspect label, obtainable through unsupervised methods such as topic models.Source: §3 · pp. 2–3
- The task takes user, item, rating, and HKG as inputs, differing from prior formulations by incorporating the graph to model aspect- and word-level preferences.Source: §3 · p. 3
4 THE PROPOSED APPROACH
The approach learns graph capsules from a heterogeneous knowledge graph, then uses them to guide aspect generation and sentence generation.
4 THE PROPOSED APPROACH
- The proposed model uses Caps-GNN to encode graph characteristics in graph capsules, then builds KG-enhanced models for aspect and sentence generation.Source: §4 · p. 3
4.1 Graph Capsule Learning
The section encodes HKG nodes as primary capsules and uses dynamic routing to form graph capsules representing graph properties. These capsules are intended to capture user preference across aspects.
4.1 Graph Capsule Learning
- Graph capsules encode HKG properties in different aspects to capture user preference over those aspects.Source: §4.1 · p. 3
4.1.1 Learning Primary Capsules .
- R-GCN node embeddings, initialized from pre-trained KG or word embeddings, are concatenated across layers to form primary capsules.Source: §4.1.1 · p. 3
4.1.2 Dynamic Routing for Graph Capsule.
- Dynamic routing combines primary-capsule prediction vectors using coupling coefficients to produce graph capsules over multiple iterations.Source: §4.1.2 · p. 4
4.2 Capsule-based Aspect Generation
The aspect-generation module uses graph capsules to infer a personalized sequence of item aspects. It adapts which graph information is read at each step and predicts an aspect label from the resulting capsules.
4.2 Capsule-based Aspect Generation
- The aspect-generation module uses an encoder-decoder framework and assumes sentence aspect labels are available during training.Source: §4.2 · p. 4
4.2.1 Basic Aspect Decoder.
- A GRU aspect decoder uses graph capsules, with its first hidden state initialized from the context embedding.Source: §4.2.1 · p. 4
4.2.2 Learning Adaptive Aspect Capsules.
- At each step, the hidden state queries graph capsules to form adaptive capsules, which are routed into aspect capsules whose lengths reflect aspect-presence probabilities; a margin loss trains them.Source: §4.2.2 · p. 4
4.3 KG-enhanced Sentence Generation
Sentence generation conditions word production on inferred aspects, combining a base decoder with a graph-based copy mechanism. The mechanism can copy reachable HKG entities or words into the generated sentence.
4.3 KG-enhanced Sentence Generation
- Given inferred aspect labels, the model begins with a GRU sentence decoder and extends it with KG-based copying.Source: §4.3 · p. 4
4.3.1 Base Sentence Decoder.
- The base attentional decoder conditions each word step on the current aspect capsule and previous word, while attending to context and prior tokens.Source: §4.3.1 · p. 4
4.3.2 Incorporating KG-based Copy Mechanism.
- The KG-based copy mechanism selects reachable HKG entities or words, hypothesizing semantic dependencies with prior tokens to improve coherence and informativeness.Source: §4.3.2 · pp. 4–5
4.4 Parameter Learning
The model is trained by separating aspect and sentence generation objectives, then incrementally optimizing and fine-tuning their parameters. At inference, it predicts aspects before generating sentences.
4.4 Parameter Learning
- The objective is split into aspect and sentence generation; the parts are trained incrementally, with shared or dependent parameters fine-tuned using the joint objective.Source: §4.4 · p. 5
- Training uses real aspects and sentences, while inference predicts the aspect sequence first and then sentences, using beam search with size 8.Source: §4.4 · p. 5
- For each user, an extracted subgraph averages far fewer than N nodes; overall training complexity is O(τ·N_u·Z + m·τ·Z·A).Graph-capsule learning is performed offline once for the HKG.Source: §4.4 · p. 5
5 EXPERIMENT
Across three review domains, the full model outperforms all baselines. The experiments also examine how dataset construction, model components, aspect coverage, and KG settings relate to performance.
5.1.1 Construction of the Datasets. / 5.1.2 Aspect and Opinion Extraction.
- The three datasets use aligned Freebase items, one-hop KG neighbors, and 8:1:1 train-validation-test splits after filtering and preprocessing.Aspect and opinion nodes are derived from topic-model keywords and syntactic rules; entity mentions use strict string matching.Source: §5.1.1 · pp. 5–6, §5.1.2 · p. 6
5.1.3 Baseline Methods. / 5.1.4 Evaluation Metrics.
- Baselines include context-aware, GAN-based, aspect-informed, and KG-enhanced models; automatic evaluation uses Perplexity, BLEU, and ROUGE metrics.The KG-enhanced baseline adds pretrained item KG embeddings to Attr2Seq.Source: §5.1.3 · p. 6, §5.1.4 · pp. 6–7
5.2 Performance Comparison
- Aspect-informed systems outperform Attr2Seq and Attr2Seq+KG, while Attr2Seq+KG improves on Attr2Seq and LeakGAN’s results differ from SeqGAN’s.ACF is the strongest baseline; the authors associate LeakGAN’s stronger results with its design for long-text generation.Source: §5.2 · p. 7
5.3 Detailed Analysis / 5.3.1 Ablation Analysis.
- Movie-dataset ablations show worse performance without KG data, HKG user and word nodes, Caps-GNN, or copying; removing copying has the largest reported effect.The detailed analysis compares the full model with five variants and selected baselines.Source: §5.3.1 · p. 7, §5.3 · p. 7, Table 4 · p. 8
5.2 Performance Comparison
- The full model outperforms all baselines by a large margin across the three-domain review-generation evaluation.The comparison attributes the difference from ACF to using KG data to guide aspect- and sentence-sequence generation.Source: §5.2 · p. 7, Table 3 · p. 8
5.3.2 Aspect Coverage Evaluation.
- On Movie, the model generates a similar number of aspects to ACF but covers more real aspects.Coverage counts a real or generated review as covering an aspect when it contains one of that aspect’s top 50 keywords.Source: §5.3.2 · pp. 7–8
5.3.3 Model Sensitivity
- Performance improves as KG data increases from 40% to 100%, and the model consistently exceeds ACF with more than 40% KG data.The sensitivity experiment varies KG data while keeping the review data unchanged.Source: Figure 3 · p. 7, §5.3.3 · p. 8
- Across embedding sizes 64, 128, 256, and 512, the model exceeds the two selected baselines; 512 performs best, with smaller gains above 256.The authors identify embedding size as a parameter to tune for KG data.Source: Figure 3 · p. 7, §5.3.3 · p. 9
5.4 Human Evaluation
Human ratings favor the model on relevance and informativeness, while qualitative examples show aspect-level personalization and informative KG-linked entities.
5.4 Human Evaluation
- The model outperforms two baselines by a large margin in relevance and informativeness, but is slightly less fluent than ACF.Two judges rated 200 Movie test reviews on relevance, informativeness, and fluency; their Cohen’s kappa coefficients were 0.76, 0.72, and 0.74, respectively.Source: §5.4 · p. 9
5.5 Qualitative Analysis
- For two movies reviewed by the same user, the model captures the user’s genre, actor, and director aspects, with KG relations aligning to those aspects.The examples suggest that KG data provides aspect semantics for learning user preference.Source: §5.5 · p. 9
- The generated reviews include personalized entities such as preferred directors and genres, surfaced through the KG-enhanced copy mechanism.The HKG’s entity-modifier word associations also support clear, fluent text segments.Source: §5.5 · p. 9
6 CONCLUSION
The paper proposes a KG-enhanced review-generation model that combines aspect- and word-level semantics. It reports superiority to previous methods across several metrics, while evaluation covers only three datasets with aligned item-entity links.
6 CONCLUSION
- The proposed model is superior to previous methods across a series of PRG evaluation metrics on three real-world review datasets.The model uses an HKG augmented with user and word nodes, Caps-GNN graph capsules, adaptive aspect capsules, and a KG-based copy mechanism.Source: §6 · p. 9
- Evaluation has so far used only three datasets with aligned entity-item linkage.The authors identify integrating other external knowledge, such as WordNet, as future work.Source: §6 · p. 9