Modeling Relation Paths for Representation Learning of Knowledge Bases

1 Introduction

• Although knowledge bases contain many facts, they remain incomplete, motivating efforts to extract relations and enrich them.Neural representation-learning methods encode entities and relations as low-dimensional embeddings; TransE models a triple using h + r ≈ t.Source: §1 · p. 1
• Unlike TransE and its extensions, which consider only direct relations, PTransE also treats multi-step relation paths as connections between entities.For a two-step path, it builds a triple (h, r1 ◦ r2, t) and optimizes h + (r1 ◦ r2) = t.Source: §1 · pp. 1–2
• Because some paths are unreliable, PTransE uses path-constraint resource allocation to measure reliability and selects reliable paths for learning.Paths leading to many possible tail entities are mostly unreliable for the entity pair.Source: §1 · p. 2
• PTransE represents a path p = (r1, . . . , rl) by recursively composing its relation embeddings with a learned binary operation.The resulting path embedding is p = r1 ◦ . . . ◦ rl.Source: §1 · p. 2

2.1 TransE and PTransE

• PTransE augments TransE’s direct-relation learning with multiple-step paths, combining direct-triple energy and path-triple energy weighted by path reliability.The model is designed to score valid triples lower than invalid ones.Source: §2.1 · p. 2

2.2 Relation Path Reliability

• PCRA measures a path’s reliability by recursively passing resource from the head entity along the path and using the amount reaching the tail.It initializes the head’s resource to 1 and defines the tail’s received amount as R(p|h, t).Source: §2.2 · p. 3

2.3 Relation Path Representation

• PTransE composes relation embeddings into path embeddings using addition (ADD), multiplication (MUL), or a recurrent neural network (RNN).ADD sums relation vectors, MUL takes their cumulative product, and RNN recursively composes them along the path.Source: §2.3 · pp. 3–4
• Path-triple energy scores whether a composed path embedding is consistent with the direct relation embedding, without using entity embeddings.Source: §2.3 · p. 4

2.4 Objective Formalization

• PTransE uses margin-based losses for valid triples and path–relation pairs, favoring lower scores for valid triples than for invalid ones.Invalid triples are formed by replacing one of the original valid triple’s components.Source: §2.4 · p. 4

2.5 Optimization and Implementation Details

• PTransE minimizes its loss by stochastic gradient descent, constrains embedding norms to at most 1, and adds reverse relations to represent paths in one direction.For each triple (h, r, t), the method adds (t, r−1, h).Source: §2.5 · p. 4
• To limit path enumeration, PTransE restricts paths to at most 3 steps and keeps only paths with reliability scores above 0.01.The passage notes that enumerating all paths can be impractical, with billions of 4-step paths possible when entities have more than 100 relations on average.Source: §2.5 · p. 4

2.6 Complexity Analysis

• PTransE has the same parameter size as TransE, while each iteration costs O(SKPL) for ADD and MUL or O(SK^2PL) for RNN, versus O(SK) for TransE.Here S is the number of training triples, P the expected number of paths between two entities, L the expected path length, and K the vector dimension.Source: §2.6 · pp. 4–5

3.1 Data Sets and Experimental Setting

• The study evaluates PTransE and baselines by predicting whether test triples hold in two scenarios: knowledge-base completion using existing KBs, and relation extraction from text using plain texts and KBs.The datasets are FB15K and FB40K, both extracted from Freebase.Source: §3.1 · p. 5

3.2 Knowledge Base Completion

• For knowledge base completion, path reliability also depends on the inference strength Pr(r|p), estimated from training data.Source: §3.2 · p. 5

3.2.1 Entity Prediction

• Table 2 reports entity-prediction results: PTransE significantly and consistently outperforms other baselines, including TransE, while addition outperforms other composition operations on both Mean Rank and Hits@10.PTransE models using relation paths of at most 2 and 3 steps achieve comparable results.Source: Table 2 · p. 6, §3.2.1 · p. 6
• PTransE significantly outperforms TransE on all relation mapping types.The paper reports consistent improvement across 1-to-1, 1-to-N, N-to-1, and N-to-N relations.Source: §3.2.1 · p. 6, Table 3 · p. 7

3.2.2 Relation Prediction

• PTransE reduces relation-prediction errors by 41.8% versus TransE+Rev+Path; adding test-time paths reduces errors by 17.3% versus TransE+Rev.PTransE using paths alone has a high mean rank because some test entity pairs have no paths, though its Hits@1 remains relatively reasonable; the comparison also indicates entity representations are important for relation prediction.Source: §3.2.2 · pp. 6–7, Table 4 · p. 7

3.3 Relation Extraction from Text

• With Sm2r, PTransE achieves significantly higher precision than TransE, especially for top-ranked triples.The evaluation uses NYT text data with FB40K as the KB; many test triples are NA, and applying test-time paths to TransE can worsen NA prediction because TransE did not encode paths during training.Source: §3.3 · pp. 7–8, Figure 3 · p. 7

3.4 Case Study of Relation Inference

• Three relation paths connecting Forrest Gump and English provide evidence for predicting their relation as LanguageOfFilm.The paper presents this as an example of inferring new facts from the knowledge base.Source: §3.4 · p. 8, Figure 4 · p. 8

4 Related Work

• Prior KB representation models use Bayesian clustering, energy-based models, or matrix factorization; TransE treats relations as translations with a prediction-accuracy/computational-efficiency trade-off.These methods encode entities and relations in a low-dimensional latent space.Source: §4 · p. 8
• Existing knowledge-base representation-learning methods use only direct entity relations, ignoring information in relation paths.Source: §4 · p. 8
• Prior relation-path work spans graph-based methods such as random walks with restart and KB inference via PRA, including expert finding, information retrieval, and relation extraction.RNNs have also been used to represent unseen relation paths from their involved relations (Neelakantan et al., 2015).Source: §4 · p. 8
• Unlike prior path methods that omit entity information, PTransE integrates entity and relation-path modeling for knowledge graph completion and relation extraction.The paper reports that PTransE learns superior representations for both entities and relations in its experiments.Source: §4 · p. 8

• PTransE encodes relation paths to embed entities and relations in a low-dimensional space, using path-constraint resource allocation and semantic composition of relations.Source: §5 · pp. 8–9
• PTransE achieves consistent and significant improvements over TransE and other baselines on knowledge-base completion and relation extraction from text.Source: §5 · p. 9
• Future work will address complex inference patterns such as Queen(e) implying Female(e) using first-order logic, and extend PTransE to TransH and TransR.PTransE cannot handle the cited Queen(e) to Female(e) inference form, and adopting its approach in TransH and TransR is non-trivial.Source: §5 · p. 9