PaperlayerView exact revision on arXiv

Source-linked AI summary

Open-World Knowledge Graph Completion

Baoxu Shi, Tim Weninger

arXiv:1711.03438v1cs.AIcs.CL

TL;DR

The paper addresses incomplete, rapidly evolving knowledge graphs that conventional closed-world KGC cannot readily extend to unseen entities. It introduces open-world KGC and ConMask, which uses relationship-dependent text extraction and neural fusion; experiments report strong performance across open- and closed-world tasks, with specific examples showing useful ranking of correct entities.

  • Problem

    Rapidly growing knowledge graphs contain missing connections, while closed-world KGC cannot readily predict relationships for new or poorly connected entities.

  • Method

    ConMask uses relationship-dependent content masking, fully convolutional neural networks, and semantic averaging to derive relationship-dependent embeddings from entity and relationship text.

  • Results

    ConMask performs well on open- and closed-world KGC tasks, with extraction features improving mean rank by at least 60% over extraction-free Semantic Averaging on both datasets.

  • Takeaways & Limitations

    Open-world KGC can use textual features to connect unseen entities to a knowledge graph, and ConMask’s examples often rank correct entities within the top three results.

  • Takeaways & Limitations

    ConMask currently predicts relationships only when the requisite information is expressed in an entity’s description, leaving new or implicit relationships for future work.

Abstract

from arXiv · show

Knowledge Graphs (KGs) have been applied to many tasks including Web search, link prediction, recommendation, natural language processing, and entity linking. However, most KGs are far from complete and are growing at a rapid pace. To address these problems, Knowledge Graph Completion (KGC) has been proposed to improve KGs by filling in its missing connections. Unlike existing methods which hold a closed-world assumption, i.e., where KGs are fixed and new entities cannot be easily added, in the present work we relax this assumption and propose a new open-world KGC task. As a first attempt to solve this task we introduce an open-world KGC model called ConMask. This model learns embeddings of the entity's name and parts of its text-description to connect unseen entities to the KG. To mitigate the presence of noisy text descriptions, ConMask uses a relationship-dependent content masking to extract relevant snippets and then trains a fully convolutional neural network to fuse the extracted snippets with entities in the KG. Experiments on large data sets, both old and new, show that ConMask performs well in the open-world KGC task and even outperforms existing KGC models on the standard closed-world KGC task.

Introduction

Knowledge Graph Completion addresses incomplete knowledge graphs, but conventional closed-world methods struggle with rapidly added or poorly connected entities. This work defines open-world KGC and introduces ConMask, which uses entity text to support completion while reducing description noise.

  • Motivation: Knowledge graphs represent knowledge as entity–relationship–entity triples and support tasks including fact checking, question answering, and entity linking.DBPedia contains 4.6 million entities, half with fewer than five relationships.
  • Motivation: Closed-world KGC predicts missing triples only among existing entities, relying heavily on graph connectivity and favoring well-connected entities.Its formal task restricts head and tail entities to the existing entity set E.
  • Motivation: 36,340 new English entities were added to DBPedia in six months, making repeated retraining of closed-world models impractical.This corresponds to approximately 200 new entities per day, excluding newly added articles without valid infobox data.
  • Open-World KGC: Open-world KGC extends completion to missing triples involving unseen entities or entities with few connections.The task relaxes the constraint that predicted triples must contain only entities from the original entity set.
  • Open-World KGC: Text descriptions provide alternative features for disconnected or newly added entities, but open-world KGC discovers missing relationships holistically rather than extracting triples through separate subtasks.The task combines relationship discovery, entity linking, and slot filling within a single model.
  • Contributions: ConMask reduces textual noise with relationship-dependent content masking and fuses relevant text with entities using fully convolutional neural networks.The paper also releases two DBPedia- and Wikipedia-based datasets for closed-world and open-world KGC evaluation.

Closed-World Knowledge Graph Completion

Closed-world KGC models learn entity and relationship embeddings from KG topology, but text-based extensions remain closed-world and cannot connect new entities to the graph.

  • Topology-based KGC: TransE translates a head entity embedding through a relationship to approximate the tail entity embedding.Other models increase expressiveness with relationship-dependent parameters, such as TransR’s entity-embedding transformation.
  • Training objective: Closed-world KGC training uses an energy-based loss that compares existing triples with corrupted triples absent from the KG.The energy function measures triple closeness, while corrupted triples are formed by randomly replacing one triple component.
  • Topology-based KGC: Other closed-world models improve prediction using list-wise ranking losses, extended-length paths, or one-relation-circle structures.These approaches retain or modify translation-based scoring while adding richer topological features.
  • Text-enhanced KGC: Text-based models combine entity descriptions with topology embeddings, but their reported gains in predictive performance are limited in the supplied discussion.Examples include averaged word embeddings, joint text-topology energies, and weighted fusion of both representations.
  • Open-world gap: Existing text-enhanced approaches remain closed-world and have no mechanism to connect new entities to the existing KG.ConMask is introduced to address this limitation using primarily text features and relationship-dependent content masking.

Open-World KGC

ConMask mimics human reasoning over an unseen entity’s description by locating relationship-relevant text, reasoning from context, and resolving it to a target entity.

  • Motivating example: ConMask’s example completes an unseen entity’s residence relation by using contextual clues in its description to rank Mumbai as the target.The example connects workplace information about Ameen Sayani with the target entity Mumbai.
  • Task terminology: The missing entity, called the target entity, may occupy either the head or tail position of a triple.This terminology is used throughout the open-world KGC model description.
  • Model architecture: ConMask decomposes reasoning into locating relevant information, implicit contextual reasoning, and resolving that information to the proper target entity.The architecture is designed to mimic these three steps.
  • Model architecture: Relationship-dependent content masking highlights words relevant to the given relationship and reduces irrelevant or noisy description content.The selected text is then processed by a fully convolutional network to extract word-based embeddings.

2. Target fusion,

ConMask extracts relationship-relevant information from entity descriptions and fuses it into target entity embeddings. It uses relationship-dependent masking followed by fully convolutional target fusion to represent entities for relationship-specific comparison.

  • Relationship-Dependent Content Masking: ConMask selects relationship-relevant words from entity descriptions, masking irrelevant text before extracting entity embeddings.The masking assigns weights using similarity between description words and the given relationship, then retains relevant snippets for downstream fusion.
  • Relationship-Dependent Content Masking: Maximal Word-Relationship Weights score each description word by its largest cosine similarity to words in the relationship name.Words that are semantically related to the relationship receive higher weights, while unrelated words receive lower weights.
  • Relationship-Dependent Content Masking: Indicator words can identify nearby target terms: for Michelle Obama’s spouse relationship, married scores highly while Barack Obama appears afterward.This motivates adjusting word weights using contextual information through Maximal Context-Relationship Weights.
  • Target Fusion: ConMask’s FCN applies convolution, sigmoid activation, batch normalization, and pooling across three layers, using mean-pooling in the final layer.The architecture reduces the number of embeddings after each layer without increasing embedding size, consolidating salient values into one representation.
  • Target Fusion: The target fusion function uses a fully convolutional neural network to convert the masked content matrix into a single k-dimensional entity embedding.FCNs are used because convolution and pooling can capture peak values in the sparse weighted content matrix, whereas RNN inputs would contain many zeros.
  • Target Fusion: Semantic averaging represents entity and relationship names and supplies background representations for other textual features.This avoids applying the extraction-oriented target fusion function where no text extraction is needed and reduces the parameter count.

Experiments

Experiments evaluate ConMask and comparison models on new and established datasets across open-world and closed-world KGC tasks. ConMask improves open-world mean rank over Semantic Averaging and sometimes surpasses closed-world methods, while examples illustrate both successful inference and masking-related errors.

  • Data Sets: DBPedia50k and DBPedia500k were introduced because FB15k has reversed or synonym triples and insufficient textual information for content-based KGC.The new datasets support both open-world and closed-world evaluation.
  • Evaluation: The evaluation holds out 10% of entities for open-world testing and evaluates whether models recover target entities connected to those unseen entities.Training uses a subgraph induced from 90% of the entities, with an additional 10% of relationships removed.
  • Open-World Entity Prediction: At least 60% mean-rank improvement over extraction-free Semantic Averaging occurs on both DBPedia datasets.The improvement is larger on DBPedia500k, suggesting that extraction features identify useful textual information from entity descriptions.
  • Closed-World Entity Prediction: ConMask sometimes outperforms closed-world methods, especially on DBPedia50k, where sparse graphs make structural prediction more difficult.The authors attribute this pattern probably to the random sampling procedure used to create DBPedia50k.
  • Prediction Examples: ConMask usually ranks correct entities in the top three, but ranks The Vampire Diaries fourth for Gabrielle Stanton’s notableWork relation.The error is linked to “consulting producer” being weakly correlated with the relationship name “notable work.”
  • Prediction Examples: For The Time Machine’s writer relation, ConMask ranks screenwriter David Duncan second even though his name is absent from the film description.The model uses the appearance of “The Time Machine” in Duncan’s description as a major work.
  • Limitations: Relationship-dependent masking can assign high scores to entities whose names resemble the queried relationship, reducing overall performance.The authors suggest filtering or rearranging relationship-named entities as future work.

Conclusion and Future Work

The paper introduces ConMask for open-world KGC and combines relationship-dependent text masking with fully convolutional networks and semantic averaging. Experiments report good performance on both open-world and closed-world tasks, alongside two new DBPedia datasets, while future work targets implicit relationships.

  • Conclusion: ConMask is an open-world KGC model that extracts relationship-dependent embeddings from entity and relationship text using content masking, fully convolutional networks, and semantic averaging.The model is designed to connect entities through textual features rather than relying only on graph topology.
  • Conclusion: Experiments show good ConMask performance on both open-world and closed-world KGC tasks.
  • Conclusion: Two DBPedia datasets were released for KGC research and development because of problems in standard KGC datasets.
  • Future Work: ConMask currently predicts relationships only when the requisite information is expressed in an entity’s description.Future work aims to extend the model to find new or implicit relationships.
Loading 1711.03438v1…