PaperlayerView exact revision on arXiv

Source-linked AI summary

Machine Knowledge: Creation and Curation of Comprehensive Knowledge Bases

Gerhard Weikum, Luna Dong, Simon Razniewski, Fabian Suchanek

arXiv:2009.11564v2cs.AIcs.DB

TL;DR

Machine knowledge supports intelligent applications by organizing entities, types, relationships, and extracted properties into large knowledge bases. The survey presents construction and curation methods alongside academic and industrial case studies, while highlighting continuing coverage and canonicalization challenges.

  • Problem

    Enhancing computers with machine knowledge that powers intelligent applications remains a long-standing research goal.

  • Method

    The survey covers entity discovery and canonicalization, taxonomy construction, property extraction, schema design, and knowledge curation, illustrated through practical case studies.

  • Results

    Industrial case studies report over 90% extraction precision for Ceres and 20% of 1.6 billion candidate triples receiving confidence 0.7 or higher in Knowledge Vault.

  • Takeaways & Limitations

    Knowledge bases can support derived computational answers, long-tail knowledge collection, and applications such as search, assistants, and analytics.

  • Takeaways & Limitations

    Automatic knowledge-base construction remains constrained by unresolved entity canonicalization and precision degradation as pattern-learning iterations continue.

Abstract

from arXiv · show

Equipping machines with comprehensive knowledge of the world's entities and their relationships has been a long-standing goal of AI. Over the last decade, large-scale knowledge bases, also known as knowledge graphs, have been automatically constructed from web contents and text sources, and have become a key asset for search engines. This machine knowledge can be harnessed to semantically interpret textual phrases in news, social media and web tables, and contributes to question answering, natural language processing and data analytics. This article surveys fundamental concepts and practical methods for creating and curating large knowledge bases. It covers models and methods for discovering and canonicalizing entities and their semantic types and organizing them into clean taxonomies. On top of this, the article discusses the automatic extraction of entity-centric properties. To support the long-term life-cycle and the quality assurance of machine knowledge, the article presents methods for constructing open schemas and for knowledge curation. Case studies on academic projects and industrial knowledge graphs complement the survey of concepts and methods.

8 Knowledge Base Curation

The section outline presents knowledge-base quality assessment and completeness as central curation concerns.

  • 8.1 KB Quality Assessment: KB quality assessment covers metrics, evaluation, and prediction.
  • 8.2 Knowledge Base Completeness: KB completeness includes local completeness and the treatment of negative statements.
  • The curation discussion organizes quality and completeness into dedicated methodological topics.

9 Case Studies

The case-studies section surveys academic and industrial knowledge bases, including systems for popular-domain curation and long-tail knowledge collection.

  • The case studies cover YAGO, DBpedia, NELL, and industrial knowledge graphs.
  • 9.5 Industrial Knowledge Graphs: Industrial examples include popular-domain curation and collection of long-tail knowledge.
  • 10 Challenges and Opportunities: The chapter concludes with take-home lessons and challenges and opportunities for future work.

1.1 Motivation

Knowledge bases have evolved from manually curated collections into automatically constructed infrastructure for search, language understanding, analytics, and other AI applications. This survey examines methods for constructing and curating them from web and text sources, including entity-centric property extraction.

  • Knowledge harvesting made large-scale knowledge bases practical by converting noisy Internet content into machine-readable facts about entities and relations.
  • A knowledge base contains entities, semantic classes, relationships, and optionally temporal validity, constraints, and provenance.
  • Knowledge bases also serve as distant supervision for natural-language-processing tasks such as entity linking and question answering.
  • Knowledge bases support semantic search, question answering, language understanding, text analytics, and data integration.
  • The survey covers automatic construction and curation from semi-structured web pages and unstructured text.
  • Property extraction augments a core knowledge base with instance-level attributes and entity relations from text and semi-structured web content.

2 Foundations and Architecture

The foundations define knowledge-base entities, types, identifiers, mentions, and relations, then frame construction as a design problem balancing correctness, coverage, agility, and source quality.

  • A knowledge base represents entities, types, attributes, and relationships, while ontologies emphasize classes and their logical connections.
  • Entity modeling distinguishes abstract or concrete entities, individual entities, identifiers, labels, and source mentions that may require disambiguation.
  • Classes group entities sharing a common trait, and instances are members of those classes.
  • 2.2.1 Requirements: Knowledge-base design must balance correctness against coverage because imperfect harvesting methods trade fewer reliable statements for broader inclusion.
  • 2.2.1 Requirements: The survey presents reusable building blocks rather than a universally viable recipe for every knowledge-base setting.
  • 2.2.2 Input Sources: High-quality construction should begin with clean, authoritative premium sources before extending and refining the knowledge base.

3 Knowledge Integration from Premium Sources

The chapter advocates building knowledge bases from a few high-quality premium sources, then cleaning, matching, and integrating their entities and category systems into a unified taxonomy. It illustrates this strategy across encyclopedic, geographic, health, and other domains while highlighting tailored methods for noisy categories and graph-based alignment.

  • 3 Knowledge Integration from Premium Sources: Wikipedia is a useful general-purpose starting point, but its categories require cleaning because they are often associative rather than semantic types.The chapter combines Wikipedia with WordNet to support a cleaner taxonomy.
  • 3 Knowledge Integration from Premium Sources: Integrating multiple sources requires entity matching to remove duplicates and combine complementary knowledge about the same entities.Taking the union of source repositories is not considered viable because the sources overlap.
  • 3 Knowledge Integration from Premium Sources: Category systems from different sources must be aligned before they can form a unified type hierarchy, including equivalent and potentially subsumption relationships.The chapter notes that this alignment is difficult because source categories may not be directly comparable.
  • 3 Knowledge Integration from Premium Sources: Domain-specific construction adapts the premium-source strategy to available repositories, including MeSH and UMLS for health and GeoNames or OpenStreetMap for geographic knowledge.Raw domain assets may still require cleaning before their classes are populated with entities.
  • 3 Knowledge Integration from Premium Sources: Graph-based methods can support alignment by using random walks over weighted source graphs to obtain statistics about node visits.Wikipedia–WordNet alignment requires tailored methods because WordNet has few instances and Wikipedia’s upper hierarchy is comparatively associative.
  • 3 Knowledge Integration from Premium Sources: Starting with premium sources provides a strong core KB through authoritative, high-coverage, and uniformly structured content.Examples include Wikipedia for general encyclopedic knowledge, GeoNames and WikiVoyage for geographic entities, and IMDB for movies.

4 KB Construction: Entity Discovery and Typing

Entity and type discovery extends premium-source taxonomies by finding missing entities, classes, aliases, and paraphrases in noisier text and Web data. The section presents dictionary, co-occurrence, and neural methods, while emphasizing coverage gains and quality limits.

  • Entity and Type Discovery: Premium sources alone miss long-tail entities and classes, motivating discovery of additional entities, types, aliases, and class names.The discovery tasks begin from a taxonomic backbone populated with prominent entities.
  • Dictionary-based Entity Spotting: Alias dictionaries improve entity spotting by covering abbreviations, acronyms, nicknames, stage names, titles, roles, and derived shorthand.Redirects and hyperlink anchors provide a practical source of additional aliases, including examples such as “Elvis” for Elvis Presley and “Apple” for Apple Inc.
  • Dictionary-based Entity Spotting: Strong co-occurrence treats names observed together under hyperlinks, wording cues, query clicks, or document contexts as likely aliases or class connections.Frequency thresholds and cue restrictions can trade precision against recall, while textual co-occurrence carries higher false-positive risk.
  • Pattern Learning: Pattern learning can expand seed-based knowledge, but open-ended iterations dilute precision and introduce false-positive statements.The text recommends stopping criteria and quality measures such as support, confidence, and diversity to restrict acquired patterns.
  • Deep Neural Networks: Neural networks combined with CRFs tend to outperform other NER methods with abundant training data, whereas CRFs and pattern-based methods remain useful in data-scarce domains.LSTM-based architectures represent sequences with forward and backward recurrent states before predicting tag scores.
  • Ab-Initio Taxonomy Construction: Automatically induced type collections can cover highly specific categories absent from existing knowledge bases, but they do not yet achieve the semantic rigor of full-fledged knowledge bases.These collections support query suggestions and exploratory browsing despite their quality gap.

5 Entity Canonicalization

Entity canonicalization maps varied mentions to the same real-world entities, enabling consistent knowledge-base use across linking, analytics, and reasoning. Entity linking resolves mentions using popularity, context, and coherence signals, while related tasks address out-of-KB entities, coreference, and record matching.

  • Canonicalization groups different names that refer to one real-world entity, preventing redundancy and inconsistencies in knowledge bases.
  • Entity Linking: Entity linking maps mentions in new text or tables to canonical knowledge-base entities, often using surface cues to generate candidates.
  • Entity Linking: Mention-entity popularity, context similarity, and entity-entity coherence provide complementary signals for selecting the proper mapping.
  • Entity Linking: Out-of-KB entities require calibrated decisions that can assign low-confidence mentions to null rather than forcing spurious links.
  • Entity Matching: Entity matching compares records using name and context similarity, consistency constraints, and distant knowledge from a background knowledge base.
  • Coreference Resolution: Coreference resolution groups underdetermined mentions with their referents, improving extraction recall and knowledge-base coverage.

6 KB Construction: Attributes and Relationships

This section describes how a clean KB of canonicalized entities and semantic types is enriched with attribute and relational SPO triples. It surveys extraction through specified patterns, seed-based and distantly supervised learning, while noting coverage and source-matching limitations.

  • 6.1 Problem and Design Space: SPO extraction enriches canonicalized entities with literal-valued attributes and relations to other entities.Examples include birthdates, birthplaces, spouses, composers, musicians, and manufacturers.
  • 6.1 Problem and Design Space: The methods handle attributes and relations in a unified extraction framework, drawing on pattern learning, CRFs, and neural networks.The chapter transfers principles and methodologies from earlier entity-discovery work to property extraction.
  • 6.1 Problem and Design Space: Extraction assumes canonicalized arguments, known property type signatures, and an existing schema of properties and relations.These assumptions enable subject-object spotting, value normalization, and typed candidate generation.
  • 6.2 Specified Patterns: Semi-structured sources such as Wikipedia infoboxes support high-precision regex extraction because pages use recurring templates and cues.The examples target birthdate and birthplace, with regexes matching four-digit birth years and capitalized place strings after dates or closing parentheses.
  • 6.3 Distantly Supervised Extraction: Source coverage remains difficult to estimate, especially for long-tail entities and infrequently mentioned properties, while spurious paths can create wrong matches.The chapter recommends combining precision-oriented specified patterns with learned extractors, whose quality depends on sufficiently large and clean distant-supervision data.
  • 6.3 Distantly Supervised Extraction: Distant supervision learns extraction patterns from existing KB statements and applies the resulting paths across entity detail pages without manual page annotation.Experiments reported in the passage find this viable at Web scale and capable of high-quality output.

7 Open Schema Construction

Open schema construction addresses unknown properties by discovering predicate-argument structures from text, then organizing noisy discoveries into canonical, typed relations. The survey also highlights neural and semi-supervised extraction methods and the unresolved trade-off between coverage and noise.

  • Open Schema Construction: Open schema construction targets unknown attributes and relationships beyond predefined properties, while organizing discovered properties into clean canonical relations.The survey frames this as discovering predicates from documents, discovering new properties for existing entity types, and canonicalizing noisy or redundant outputs.
  • Open Information Extraction: Pattern-based extraction uses syntactic structures such as noun–verb–noun and verb–preposition phrases to propose binary predicates and their arguments.The resulting predicate-argument structure may include more than two arguments through adverbial modifiers.
  • Neural Learning for Open IE: Neural Open IE models sequence-tag predicate and argument spans, using labeled Open IE data together with question-answering annotations that are easier to crowdsource.The cited approach decodes BIO tags and leverages triples containing a sentence, a question, and an answer.
  • Property Canonicalization: OpenCeres uses semi-supervised label propagation to corroborate and clean extracted candidates, while PATTY uses entity typing to infer compatible property signatures and synonymy.PATTY expands paraphrase resources but remains sparse and noisy because it relies on sentences containing a property phrase and two named entities.
  • Open Information Extraction: Open IE extracts predicate-argument tuples from sentences without requiring a prior knowledge base, improving recall but often introducing substantial noise.Its output consists of a predicate and two or more argument phrases, with modifiers or additional arguments possible.
  • Take-Home Lessons: The survey concludes that Open IE improves coverage and discovers new properties, whereas canonicalization and interpretable long-term maintenance remain difficult.Distant supervision can improve precision, but latent schemas are not easily interpretable and their curation role remains open.

8 Knowledge Base Curation

Knowledge base curation is necessary because automatically constructed bases contain errors and incomplete coverage. The survey combines quality metrics, sampling-based evaluation, confidence estimation, and logical or human-assisted methods for maintaining quality over time.

  • Motivation: Knowledge base curation is inevitable because no constructed KB is perfect, with errors and incomplete coverage persisting as the resource grows and evolves.The chapter therefore treats quality assurance and curation as necessary for maintaining and enhancing KB value.
  • Quality Metrics: Precision measures correctness against ground truth, while recall measures coverage; both apply to entities, types, property statements, and collections of entities.Precision is the fraction of KB elements also in ground truth, whereas recall is the fraction of ground-truth elements present in the KB.
  • Quality Metrics: Confidence-scored extraction systems evaluate quality across thresholds using precision–recall curves, AUC or ROC measures, and the best harmonic mean of precision and recall.These metrics avoid committing to one application-dependent confidence threshold.
  • Quality Evaluation: Recall is difficult to assess because comprehensive ground truth is usually unavailable, making sampling, labor-intensive annotation, or comparison with a high-recall KB necessary.Precision can be estimated by judging sampled KB statements, but recall requires assessing coverage against a complete or high-coverage reference.
  • Quality Prediction: Quality prediction can use source and contributor signals, with topic-specific user expertise correlating more strongly with contribution quality than general user standing in Freebase.For Wikidata, edit counts and user status are also reported as indicative of quality.
  • Curation Methods: Fact-checking tools such as FACTY support knowledge-base curation, while local completeness assumptions must be restricted to relation directions that are closer to functional.The LCA is supported for hasCitizenship but not for its inverse hasCitizen relation.

8.3 Rules and Constraints

Rules and constraints provide complementary mechanisms for curating knowledge bases: constraints detect violations, rules deduce additional statements, and soft rules flag plausible exceptions without treating them as absolute errors. Their effectiveness depends on logical expressiveness, scope, and assumptions about completeness.

  • Rules and Constraints: Constraints specify invariants that must hold for consistency, whereas deductive rules infer additional statements to complete or repair the KB.A marriage statement can imply that an entity belongs to the person class, while constraints can flag missing type membership as inconsistent.
  • Constraints: Hard and soft constraints help clean KBs by identifying implausible statements, resolving conflicts and gaps, and supporting logical consistency.Soft constraints tolerate exceptions and can flag suspicious content for human curation rather than enforcing universal prohibitions.
  • Constraint Types: Constraint templates cover existential and temporal dependencies, value bounds, symmetry or transitivity, functional dependencies, conditional functions, and disjointness.These patterns encode structural, temporal, numerical, relational, and class-membership restrictions.
  • Constraint Languages: SHACL expresses validation constraints such as maximum property counts and class restrictions, with extensions for strings, numerical ranges, Boolean combinations, and alternative ShEx validation.A SHACL shape can allow at most one hasMother value for each Person and can be extended with minCount when exactly one is required.
  • Deductive Rules: Rule languages trade expressiveness against computational complexity: RDFS supports limited domain, range, subclass, and sub-property rules, whereas OWL supports disjointness and inconsistency detection.OWL reasoners can detect contradictions such as an entity being both a Person and a FictitiousCharacter when those classes are disjoint.
  • Rule Mining: Rule mining seeks logical patterns in KBs, and LCA-based confidence is more effective than CWA-based scoring for identifying interesting rules because incomplete KBs make absent statements unreliable negatives.The LCA is especially appropriate for relation directions closer to functional, such as person-to-country citizenship.

8.4 Knowledge Graph Embeddings

Knowledge graph embeddings map entities, properties, and triples into low-dimensional real-valued vectors, supporting similarity analysis and prediction of missing statements. TransE learns these representations by approximating each relation as vector translation, but struggles with one-to-many relations.

  • KG embeddings map entities, properties, and triples into real-valued vectors in a low-dimensional latent space.
  • KG embeddings support entity-similarity analysis, clustering, deep-learning inputs, and ranked predictions for missing knowledge-base statements.
  • TransE learns vectors so that v(s) + v(p) approximately equals v(o) for each knowledge-base statement.
  • Training uses positive triples and perturbed negative triples so the model distinguishes existing facts from mismatches.
  • TransE places all objects of a one-to-many relation at the same vector location, motivating more advanced embedding models.

8.5 Consistency Reasoning for Knowledge Cleaning

Knowledge-base cleaning combines constraints, optimization, and probabilistic reasoning to select consistent candidate statements. MaxSat and related methods scale through specialized heuristics, while probabilistic soft constraints offer alternative tractable formulations.

  • Constraint reasoning detects and resolves conflicts among candidate statements, including jointly inconsistent alternatives that pass individual checks.
  • Knowledge-base cleaning differs from traditional database cleaning because it spans many relations, numerous constraints, and settings where most data may not be correct.
  • Weighted MaxSat selects a truth assignment maximizing the total weight of satisfied clauses, with the example’s optimum reaching 4.7.
  • Customized MaxSat solvers combine propagation and SAT heuristics, guaranteeing at least 50% of optimum and often achieving 90% or better in practice.
  • Soft constraints add weighted penalties for violations, allowing consistency requirements to be treated as either strict invariants or exception-tolerant preferences.
  • Probabilistic Soft Logic makes MAP inference polynomial-time through hinge-loss convex optimization, though scalability remains limited.

8.6 KB Life-Cycle

Long-term knowledge-base management requires continuous curation as facts, entities, and relationships change. Provenance, versioning, temporal scopes, and consistency reasoning help preserve traceability and interpretability over time.

  • Knowledge-base construction and curation must continue as the world evolves, creating long-term challenges for freshness, quality, and utility.
  • Provenance should record sources, acquisition times, extraction methods, and human approval steps so errors can be traced and corrected.
  • Versioning and temporal scopes preserve changing relationships, support historical queries, and prevent overwriting information needed for quality assurance.
  • Temporal scopes may use dates, months, years, or intervals, with granularity determined by the event and the precision of its source reporting.
  • Temporal consistency reasoning can conservatively reject overlapping relationship candidates, while interval decomposition can improve recall by splitting scopes into disjoint periods.
  • Emerging entities require disambiguation against known knowledge-base entities, even when an apparent match already exists.

8.7 Take-Home Lessons

The survey presents quality assessment, logical consistency, constraint-based cleaning, provenance, and temporal versioning as central components of knowledge-base curation. Together, these practices address correctness, coverage, gaps, and long-term maintenance.

  • Quality and coverage are assessed by sampling statements for human judgments, using crowdsourcing or experts when necessary.
  • No knowledge base is complete, so completeness assessment and prediction commonly rely on the Local Completeness Assumption.
  • Logical invariants both detect erroneous statements as constraints and infer additional statements as rules.
  • Constraint-based reasoning with MaxSat, ILP, and probabilistic factor graphs is presented as a best-practice approach for cleaning candidate statements at scale.
  • Provenance tracking and versioning with temporal scopes are essential for long-term knowledge-base life-cycle management.

9 Case Studies

The case studies show how knowledge bases balance scale, coverage, entity integration, extraction quality, and long-term consistency. Across academic and industrial systems, stronger curation and quality controls improve reliability but can reduce coverage or limit practical impact.

  • YAGO: YAGO 4 contains about 60 million entities and 2 billion statements in a consistent taxonomic backbone.Constraint-aware construction omitted about 30 million Wikidata entities, primarily less notable long-tail entities.
  • NELL: NELL couples learners and constraints to bootstrap from few training samples and produce high-confidence statements, but lacks sufficient entity linking.Its entity detector can generate many synonymous or descriptive names without recognizing that they denote the same entity.
  • Wikidata: Wikidata supports interlinking across libraries, archives, biomedical data, and scholarly resources through widespread external identifiers.Its scope is contested because scholarly data made up 40% of entities in 2019, potentially biasing search and ranking while straining infrastructure.
  • Industrial Knowledge Graphs: Knowledge-fusion quality assurance reduced verified coverage from 60% to 50% while increasing precision to 84%, whereas 1.6 billion candidates yielded 271 million at confidence 0.9 or higher.The case studies illustrate the trade-off between retaining candidate knowledge and meeting stringent precision requirements.
  • Industrial Knowledge Graphs: Knowledge Vault extracted 2.8 billion SPO triples from more than 2 billion web pages, with DOM trees contributing 75% and web tables less than 5%.Web-table extraction was limited partly by ambiguous column headers and alignment difficulties.

10 Wrap-Up

The survey frames comprehensive KB construction as a lifecycle balancing near-human correctness with broad coverage, supported by diverse methods and human curation. It emphasizes entity-type foundations, gradual engineering, and open challenges involving language models, coverage, credibility, and user-controlled knowledge.

  • 10.1 Take-Home Lessons: Near-human KB quality requires both error rates below 5 percent or even under 1 percent and very large coverage of entities, types, and properties.The two objectives can conflict, motivating staged construction that begins with conservative, high-precision methods before tackling harder cases.
  • 10.1 Take-Home Lessons: Entities and semantic types form the foundation for high-quality KBs, while attributes and relations support advanced question answering and entity-centric analytics.The survey therefore emphasizes taxonomy construction and entity canonicalization before populating additional properties and relations.
  • 10.1 Take-Home Lessons: KB construction is a long-term engineering lifecycle requiring augmentation, new-entity discovery, schema expansion, quality assurance, and human architects or curators.The complete machinery cannot be fully automated, and different subtasks involve trade-offs among precision, recall, cost, and application requirements.
  • 10.2 Challenges and Opportunities: Language models offer strong reading-comprehension capabilities for KB construction, but their predictions can be correct yet unusable for precise knowledge extraction.The survey highlights combining symbolic knowledge with latent language understanding as an important research avenue.
  • 10.2 Challenges and Opportunities: Personal and commonsense KBs could support augmented memory, privacy under user control, and more robust conversational assistants and visual-content interpretation.These uses extend KBs beyond current search applications toward potential building blocks for next-generation AI.
  • 10.2 Challenges and Opportunities: Major open challenges include incomplete coverage, credibility and misinformation, and combining human input with large-scale automation at low cost and high quality.The survey notes that salient facts remain absent from current KBs and identifies careful human-machine combination as a next-generation research challenge.
Loading 2009.11564v2…