Source-linked AI summary
On the representation and embedding of knowledge bases beyond binary relations
Jianfeng Wen, Jianxin Li, Yongyi Mao, Shini Chen, Richong Zhang
TL;DR
Knowledge-base embedding methods assume binary relations, although knowledge bases contain substantial non-binary data and triple conversion can lose relational structure. This paper introduces canonical instance representations and direct modelling for multi-fold relations, generalizes TransH as m-TransH, and reports a large performance advantage over TransH.
Problem
Existing knowledge-base embedding methods assume binary relations, while substantial knowledge-base data are non-binary and triple conversion can lose structural information.
Method
The paper uses canonical instance representations and directly models multi-fold relations, extending TransH to m-TransH.
Results
m-TransH outperforms TransH by a large margin, with HIT above 80% compared with reported embedding performances of about 50%.
Takeaways & Limitations
The results support modelling multi-fold relations at the instance level rather than using the decomposition framework on triple-level data.
Takeaways & Limitations
FB15K no longer suits the paper's purpose because S2C conversion makes the original multi-fold representation unrecoverable and adds structurally uninformative connectivity.
Abstract
from arXiv · showhide
The models developed to date for knowledge base embedding are all based on the assumption that the relations contained in knowledge bases are binary. For the training and testing of these embedding models, multi-fold (or n-ary) relational data are converted to triples (e.g., in FB15K dataset) and interpreted as instances of binary relations. This paper presents a canonical representation of knowledge bases containing multi-fold relations. We show that the existing embedding models on the popular FB15K datasets correspond to a sub-optimal modelling framework, resulting in a loss of structural information. We advocate a novel modelling framework, which models multi-fold relations directly using this canonical representation. Using this framework, the existing TransH model is generalized to a new model, m-TransH. We demonstrate experimentally that m-TransH outperforms TransH by a large margin, thereby establishing a new state of the art.
1 Introduction
Knowledge-base embedding has largely treated relations as binary, despite substantial non-binary data. The paper proposes canonical instance representations and direct modelling of multi-fold relations, showing that m-TransH outperforms TransH.
- Existing embedding methods assume binary relations, although more than 1/3 of Freebase entities participate in non-binary relations.
- The paper defines multi-fold relations and proposes instance representation as a canonical format for binary and non-binary relational data.
- Converting multi-fold data into triples creates heterogeneous predicates that are unfavorable for embedding, motivating recovery of instance representations first.
- Existing models on S2C-converted datasets use a decomposition framework, while the paper advocates direct modelling of multi-fold relations.
- m-TransH generalizes TransH for multi-fold relations and outperforms TransH by an astonishing margin with lower complexity.
2 Multi-Fold Relations and Knowledge Base Representations
The paper defines role-aware multi-fold relations and canonical instance and fact representations for knowledge bases. It also shows how fact representations convert to instances and how Freebase’s heterogeneous triples encode these structures.
- 2.1 Multi-Fold Relations: A multi-fold relation is a set of role-indexed entity assignments, with each instance containing exactly one entity for every role.The role set specifies both the relation’s arity and the semantic position of each entity.
- 2.2 Instance Representation: An instance representation records entities, relation types, and observed instances, preserving the relation structures and interactions relevant to embedding.For each relation type, the knowledge base stores its observed instance set, which may be incomplete relative to the unknown relation.
- 2.3 Meta-Relations and Fact Representation: Fact representations use meta-relations whose role values may be sets of entities, allowing different facts to contain different numbers of entities.A meta-relation is degenerate when every role value in every fact is a singleton.
- 2.4 Graphical Representations: Instance and fact representations correspond to edge-labelled bipartite graphs linking entity vertices to instance or fact vertices by role-labelled edges.Figures 1 and 2 illustrate the two graph types using toy SportAward, TeamRoster, PeopleMarriage, and PlaceOfBirth examples.
- 2.5 Converting Facts to Instances: The Tid conversion adds fact identifiers and a FACT-ID role, making the original fact representation recoverable, whereas T drops fact identifiers and is generally irreversible.The paper notes that Tid preserves which instances belong to the same fact, while T provides a simpler instance representation without that guarantee.
- 2.6 Freebase: Freebase’s organization is equivalent to a fact representation but uses heterogeneous triples, making it less clean for embedding than instance or fact representations.Binary degenerate facts undergo star-to-clique conversion, while other meta-relations use Mediator and CVT vertices.
3 Embedding
The paper formulates knowledge-base embedding on canonical instance representations and shows that triple-based FB15K models lose structural information through S2C conversion. It proposes direct multi-fold modelling with m-TransH, which generalizes TransH and reduces to it for binary relations.
- 3.1 Problem Formulation: The embedding problem seeks entity vectors and relation-specific constraints that assign low cost to observed instances and positive cost to unobserved ones.The formulation represents each relation constraint with a nonnegative cost function and learns its parameters together with the entity embedding.
- 3.2 FB15K Datasets: S2C conversion is generally irreversible, so FB15K cannot recover the original multi-fold relational structure.The conversion turns instance vertices into triples and leaves embedding models treating those triples as binary-relation instances.
- 3.3 Prior Art of Modelling: Existing FB15K embedding models are equivalent to decomposition modelling, where each multi-fold cost function is represented through bivariate functions.The equivalence follows from the global cost formulation for S2C-converted data.
- 3.3 Prior Art of Modelling: The decomposition framework is limited because S2C distorts relational structure and can produce large errors when approximating the original cost function.These limitations motivate direct modelling of multi-fold relations.
- 3.4 Proposed Model: m-TransH directly models multi-fold relations, generalizing TransH within the proposed direct modelling framework.The framework avoids decomposition into bivariate functions.
- 3.4 Proposed Model: For binary relations with the role-weight parameters summing to zero, m-TransH has the same optimization problem as TransH.Thus the proposed model preserves TransH as its binary-relation special case.
4 Experiments
Experiments compare m-TransH and TransH under matched training, testing, and negative-sampling setups. Across overall, fold-specific, and efficiency results, m-TransH achieves substantially better performance with lower computational cost.
- 4.2 Training and Testing: The experiments use four configurations: m-TransH, m-TransH:ID, TransH:triple, and TransH:inst, with datasets specified for each comparison.Negative examples are generated per triple for TransH configurations and per instance for m-TransH configurations, with equal totals across experiments.
- 4.3 Results and Discussions: TransH performs similarly under triple-level and instance-level testing, while its JF17K performance is better than the previously reported FB15K result.The instance-level protocol produces virtually identical HIT and somewhat inferior RANK relative to triple-level testing.
- 4.3 Results and Discussions: For every relation fold J, m-TransH outperforms TransH, and its gains on non-binary relations also improve embedding performance for binary relations.The comparison uses the best-performing dimension for each experiment and reports HIT/RANK breakdowns across relation folds.
- 4.3 Results and Discussions: At DIM=50, m-TransH:ID requires 52/135 minutes for training/testing versus 105/229 minutes for TransH:triple.The lower complexity is attributed to avoiding the larger number of model parameters created when multi-fold instances are converted into triples.
5 Concluding Remarks
The paper advocates instance representations and direct modeling for knowledge bases containing multi-fold relations. Its m-TransH example demonstrates advantages in both embedding performance and complexity, and the JF17K datasets are released for further research.
- 5 Concluding Remarks: The paper concludes that modeling multi-fold relations at the instance level offers major performance and complexity advantages over triple-level decomposition.The conclusion identifies m-TransH as a simple example that outperforms TransH by a large margin.
- 5 Concluding Remarks: The JF17K datasets are made publicly available to support further research on embedding multi-fold relations.This provides a released experimental resource aligned with the paper’s proposed representation and modeling framework.