PaperlayerView exact revision on arXiv

Source-linked AI summary

Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendation

Yankai Chen, Menglin Yang, Yingxue Zhang, Mengchen Zhao, Ziqiao Meng, Jianye Hao, Irwin King

arXiv:2108.06468v3cs.IR

TL;DR

Existing knowledge-aware recommendation methods underemphasize the hierarchy of scale-free tripartite graphs. LKGR models these graphs in hyperbolic space and propagates interaction and KG information with knowledge-aware attention. It outperforms state-of-the-art methods on three real-world benchmarks, improving Top-20 Recall by 3.61–15.32%.

  • Problem

    Existing knowledge-aware recommendation methods underemphasize the intrinsic hierarchical structure of scale-free tripartite graphs formed by unifying user-item interactions and KGs.

    Source: §1 · p. 1

  • Method

    LKGR models scale-free tripartite graphs in Lorentzian hyperbolic space, separately propagates interaction and KG information, and uses knowledge-aware attention to weight neighbors.

    Source: §1 · p. 2

  • Results

    3.61–15.32% improvement in Recall for Top-20 recommendation over compelling baselines across three real-world scenarios.

    Source: §1 · p. 2

  • Takeaways & Limitations

    LKGR learns hierarchical structure while coherently summarizing interaction signals and knowledge associations in low-dimensional embeddings.

    Source: §1 · p. 2

    The work treats user-item interactions and the KG as a static graph; learning graph structure with temporal information is proposed as future work.

    Source: §6 · p. 8

Abstract

from arXiv · show

Aiming to alleviate data sparsity and cold-start problems of traditional recommender systems, incorporating knowledge graphs (KGs) to supplement auxiliary information has recently gained considerable attention. Via unifying the KG with user-item interactions into a tripartite graph, recent works explore the graph topologies to learn the low-dimensional representations of users and items with rich semantics. However, these real-world tripartite graphs are usually scale-free, the intrinsic hierarchical graph structures of which are underemphasized in existing works, consequently, leading to suboptimal recommendation performance. To address this issue and provide more accurate recommendation, we propose a knowledge-aware recommendation method with the hyperbolic geometry, namely Lorentzian Knowledge-enhanced Graph convolutional networks for Recommendation (LKGR). LKGR facilitates better modeling of scale-free tripartite graphs after the data unification. Specifically, we employ different information propagation strategies in the hyperbolic space to explicitly encode heterogeneous information from historical interactions and KGs. Our proposed knowledge-aware attention mechanism enables the model to automatically measure the information contribution, producing the coherent information aggregation in the hyperbolic space. Extensive experiments on three real-world benchmarks demonstrate that LKGR outperforms state-of-the-art methods by 3.6-15.3% of Recall@20 on Top-K recommendation.

1 INTRODUCTION

LKGR targets scale-free tripartite graphs, whose hierarchical structure may be poorly represented by Euclidean embeddings. It models them in hyperbolic space and combines interaction signals with KG information.

1 INTRODUCTION

  • Unified user-item and KG tripartite graphs exhibit scale-free, approximately power-law structure that existing methods underemphasize.The paper argues that Euclidean embeddings may distort their hierarchy and suppress recommendation performance.Source: §1 · pp. 1–2
  • LKGR models scale-free graphs with Lorentzian hyperbolic geometry to learn representations of users, items, and KG entities.The paper describes hyperbolic space as a continuous tree-like space with exponential volume growth, suited to hierarchical structure.Source: §1 · p. 2
  • LKGR separately propagates user-item interaction signals and KG information to update node embeddings.The paper motivates this separation by the heterogeneity of nodes in recommendation graphs.Source: §1 · p. 2
  • Lorentzian Knowledge-aware Attention weights neighboring information, while high-order propagation explores further graph structure in hyperbolic space.The paper presents these as components of LKGR’s information aggregation and propagation.Source: §1 · p. 2
  • LKGR improves Top-20 Recall by 3.61–15.32% over compelling baselines across three real-world recommendation scenarios.The scenarios are book, movie, and restaurant recommendation.Source: §1 · p. 2

2 PROBLEM AND MOTIVATION

The paper formulates recommendation over a Unified Knowledge Graph that combines user-item interactions and item knowledge. The task is to estimate whether a user may adopt an item.

2.1 Problem Formulation

  • A knowledge graph consists of entity-relation-entity triplets, with each relation connecting two entities.The paper uses KG entities and relations to represent item-related external knowledge.Source: §2.1 · p. 2
  • The Unified Knowledge Graph combines users, items, and KG entities in a tripartite graph, unifying user behaviors and item knowledge.User-item interactions are represented using a generalized interaction relation.Source: §2.1 · p. 2
  • Given the Unified Knowledge Graph, the recommendation task is to estimate the probability that user u may adopt item i.Source: §2.1 · p. 2

3.1 Hyperbolic Geometry Preliminaries

The paper adopts the Lorentz model to represent hyperbolic geometry. It uses tangent spaces and exponential and logarithmic mappings to relate that space to a local Euclidean approximation.

3.1 Hyperbolic Geometry Preliminaries

  • The Lorentz model is the hyperbolic-geometry representation adopted for its simplicity and numerical stability.The paper defines hyperbolic geometry as non-Euclidean geometry with constant negative curvature.Source: §3.1 · p. 2
  • The tangent space at a Lorentzian-manifold point is a local first-order approximation used for hyperbolic graph convolutions.Source: §3.1 · p. 2
  • Exponential and logarithmic mappings bridge hyperbolic space and tangent space.The logarithmic mapping projects vectors back to the tangent space centered at a point.Source: §3.1 · pp. 2–3

3.2 LKGR Model Overview

LKGR has encoding, attentive Lorentzian convolutional, and prediction layers. These project inputs to the Lorentzian manifold when needed, update user and item embeddings using sampled ego-networks, and produce matching scores.

3.2 LKGR Model Overview

  • The encoding layer projects Euclidean input embeddings onto the hyperbolic space when needed to place them on the Lorentzian manifold.Source: §3.2 · p. 3
  • The attentive Lorentzian convolutional layer updates user and item embeddings from sampled ego-networks, propagating interaction signals and selectively aggregating KG information.Stacking these layers enables exploration of high-order KG subgraphs.Source: §3.2 · p. 3
  • The prediction layer uses learned user and item representations to output a matching score through a hyperbolic-space inner product.Source: §3.2 · p. 3

3.3 Encoding Layer

The encoding layer maps Euclidean inputs onto the Lorentzian manifold for use by LKGR’s later layers. Its trainable curvature dynamically measures how hierarchical the embedding space is.

3.3 Encoding Layer

  • LKGR encodes Euclidean input embeddings onto the Lorentzian manifold before they enter subsequent layers.The transformation starts from a Euclidean input in the tangent space.Source: §3.3 · p. 3
  • The trainable curvature dynamically measures how hierarchical the embedding space is.The curvature is −1/c.Source: §3.3 · p. 3
  • LKGR initializes node embeddings in Euclidean space to evaluate the holistic performance of its proposed modules.The stated evaluation includes recommendation accuracy and training time.Source: §3.3 · p. 3

3.4 Attentive Lorentzian Convolutional Layer

LKGR uses Lorentzian attention and propagation to incorporate knowledge-graph and interaction information into user and item representations. It also supports multi-hop knowledge extraction and Lorentzian aggregation.

3.4.1 Lorentzian Knowledge-aware Attention.

  • Attention weights depend on entity embeddings and relation-specific weights, allowing LKGR to measure knowledge-neighbor contributions and selectively propagate information.The edge weights are normalized across edges connected to the head entity with softmax.Source: §3.4.1 · p. 3

3.4.2 Lorentzian Information Propagation.

  • Item propagation combines signals from neighboring users with knowledge from neighboring KG entities, while user propagation represents historical interactions.The method also uses fixed-size random neighbor sampling instead of full neighborhoods.Source: §3.4.2 · p. 4

3.4.3 Lorentzian Neighbor Aggregation.

  • LKGR updates node representations with Lorentzian sum, concatenate, or neighbor aggregators.The sum and concatenate aggregators apply a nonlinear transformation and activation; the neighbor aggregator updates from the neighbor representation.Source: §3.4.3 · p. 4

3.4.4 High-order Knowledge Extraction.

  • Multi-hop propagation iteratively passes KG information from sampled higher-hop neighborhoods toward the item representation.LKGR limits propagation depth to L≤2 because too many hops can cause over-smoothing; its stated training complexity is exponential in L.Source: §3.4.4 · pp. 4–5

3.5 Prediction Layer and Model Optimization

LKGR scores user-item pairs using the hyperbolic inner product of their learned representations and recommends items with the highest scores. It trains with balanced positive and sampled negative items using cross-entropy and L2 regularization.

3.5.1 Prediction Layer.

  • LKGR estimates user-item matching scores with the hyperbolic inner product of learned user and item representations, then selects the highest-scoring items for recommendation.Source: §3.5.1 · p. 5

3.5.2 Model Optimization.

  • Training uses equally sized positive and sampled negative item sets and optimizes a cross-entropy loss with L2 regularization.The positive and negative sets are updated during each training iteration.Source: §3.5.2 · p. 5

4 EXPERIMENTS

Experiments compare LKGR with recommendation baselines across three domains and assess its ranking performance, efficiency, components, and hyperparameters.

4.1 Datasets

  • The benchmarks span Book-Crossing ratings, MovieLens-20M ratings, and Dianping-Food interactions across book, movie, and restaurant recommendation.MovieLens-20M contains approximately 20 million ratings; Dianping-Food has over 10 million interactions among about 2 million users and 1 thousand restaurants.Source: §4.1 · p. 5, Table 1 · p. 5

4.2 Baselines

  • LKGR was compared with collaborative-filtering, regularization-based, and propagation-based recommendation methods.The baselines include BPRMF and NFM; CKE and KGAT; and RippleNet, KGCN, KGNN-LS, and CKAN.Source: §4.2 · p. 5

4.3 Experiment Setup

  • Each dataset was randomly split five times into training, evaluation, and test sets at a 6:2:2 ratio, with Recall@K and NDCG@K used for evaluation.Models were optimized with Adam; embedding size was searched over 8, 16, 32, 64, and 128.Source: §4.3 · p. 6

4.5 Time Efficiency Comparison (RQ2)

  • LKGR required more training time than CF-based methods, but had analogous training cost to other KG-based methods and an excellent efficiency-accuracy trade-off, especially on Movie and Restaurant.Figure 4 compares training time and overall efficiency against accuracy.Source: §4.5 · p. 7, Figure 4 · p. 7

4.6 Ablation Study of LKGR (RQ3.A)

  • Removing user-item propagation sharply reduced Top-K performance; KG propagation was also important, while Euclidean operations and equal-weight attention degraded results.The ablations compare LKGR variants without interactive signal propagation, knowledge extraction, hyperbolic geometry, and Lorentzian Knowledge-aware Attention.Source: Table 3 · p. 7, §4.6 · pp. 7–8

4.7 Effect of Key Hyper-parameters (RQ3.B)

  • The sum aggregator performed best on Book and Movie, whereas concat was more suitable for Restaurant.The authors attribute the advantage of sum and concat over the neighbor aggregator to retaining external-neighbor and internal information.Source: §4.7 · p. 8, Table 4 · p. 8
  • Best KG propagation depth varied by dataset: 1 for Book, 2 for Movie, and 1 for Restaurant.The authors note that distant propagation may introduce irrelevant information, particularly on large, dense datasets.Source: §4.7 · p. 8, Table 5 · p. 8

5 RELATED WORKS

KG-enhanced recommendation methods are grouped into path-based, regularization-based, and propagation-based approaches. Their limitations include costly path selection, limited explicit modeling of high-order information, and reliance on Euclidean space despite the hierarchical structure of KGs.

6 RELATED WORKS

  • KG-enhanced recommender systems address data sparsity and cold-start through auxiliary KG information and are commonly classified into three branches.The branches are path-based, regularization-based, and propagation-based methods.Source: §5 · p. 8
  • Propagation-based methods pass auxiliary KG information along links to refine user and item representations, but assume Euclidean learning despite KGs' intrinsic hierarchy.LKGR is conceptually inspired by graph message passing and performs graph convolution in hyperbolic space.Source: §5 · p. 8
  • Path-based methods rely on manually selected or generated meta-paths, while regularization-based methods add KG-structure losses but may not explicitly model high-order semantics.Path retrieval can be difficult to optimize on large, complex graphs, and selected paths affect performance.Source: §5 · p. 8

6 CONCLUSION AND FUTURE WORKS

LKGR learns user, item, and KG-entity embeddings in hyperbolic space and uses Lorentzian attention with multi-layer aggregation. Results on three real-world datasets validate component effectiveness and improvements over recent state-of-the-art solutions; future work includes temporal information and other application scenarios.

6 CONCLUSION AND FUTURE WORKS

  • LKGR improves performance over recent state-of-the-art solutions on three real-world datasets, and experiments support the effectiveness of its proposed components.Source: §6 · p. 8
  • LKGR embeds users, items, and KG entities in hyperbolic space, using Lorentzian knowledge-aware attention followed by multi-layer aggregation for high-order propagation.Source: §6 · p. 8
  • Future directions include jointly learning graph structure and temporal information, and exploring hyperbolic modeling in areas such as information retrieval and language processing.Source: §6 · p. 8

Related papers

Loading 2108.06468v3…