PaperlayerView exact revision on arXiv

Source-linked AI summary

Label Noise Reduction in Entity Typing by Heterogeneous Partial-Label Embedding

Xiang Ren, Wenqi He, Meng Qu, Clare R. Voss, Heng Ji, Jiawei Han

arXiv:1602.05307v1cs.CLcs.LG

TL;DR

Distant supervision can assign knowledge-base types that do not fit a mention’s local context, creating noisy training labels. PLE jointly embeds mentions, text features, and types to recover type-paths from these candidates. Across three public datasets, it reduces label noise and improves typing systems trained on the denoised corpora.

  • Problem

    Distant supervision assigns mentions the knowledge-base types of linked entities without accounting for local context, producing candidate labels that can be incorrect for the mention.

    Source: §1 · p. 1

  • Method

    PLE jointly embeds entity mentions, text features, and types, then uses the learned embeddings to infer correct type-paths from candidate labels.

    Source: §1 · p. 2

  • Results

    On three public datasets, PLE substantially reduces label noise, and PLE-denoised corpora significantly improve state-of-the-art fine-grained typing systems when used for training.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The paper presents identifying correct labels in automatically labeled mentions as a fundamental task for building entity typing systems with distant supervision.

    Source: §1 · p. 2

    The shortest-path type-correlation measure has limited accuracy for some parent-sibling relationships, while using knowledge-base relation facts is left for future work.

    Source: §3.1 · p. 5

Abstract

from arXiv · show

Current systems of fine-grained entity typing use distant supervision in conjunction with existing knowledge bases to assign categories (type labels) to entity mentions. However, the type labels so obtained from knowledge bases are often noisy (i.e., incorrect for the entity mention's local context). We define a new task, Label Noise Reduction in Entity Typing (LNR), to be the automatic identification of correct type labels (type-paths) for training examples, given the set of candidate type labels obtained by distant supervision with a given type hierarchy. The unknown type labels for individual entity mentions and the semantic similarity between entity types pose unique challenges for solving the LNR task. We propose a general framework, called PLE, to jointly embed entity mentions, text features and entity types into the same low-dimensional space where, in that space, objects whose types are semantically close have similar representations. Then we estimate the type-path for each training example in a top-down manner using the learned embeddings. We formulate a global objective for learning the embeddings from text corpora and knowledge bases, which adopts a novel margin-based loss that is robust to noisy labels and faithfully models type correlation derived from knowledge bases. Our experiments on three public typing datasets demonstrate the effectiveness and robustness of PLE, with an average of 25% improvement in accuracy compared to next best method.

1. INTRODUCTION

Distant supervision supplies fine-grained typing corpora but assigns knowledge-base types without regard to mention context, leaving noisy candidate labels. The introduction frames LNR as recovering context-correct type-paths and presents PLE as a joint embedding approach.

1. INTRODUCTION

  • Entity typing assigns structured types to document mentions, supporting information extraction, knowledge-base completion, and question answering.Source: §1 · p. 1
  • Distant supervision copies all types of a knowledge-base-linked entity to its mention, ignoring local context; most training mentions have noisy types.Prior systems often treat all candidate types as true, while simple pruning can shrink training sets and degrade performance.Source: §1 · pp. 1–2
  • The paper defines LNR as identifying context-correct type-paths from distant-supervision candidates, rather than predicting types for unlabeled mentions.Fine-grained type-paths can contain multiple types and need not end at a leaf in the hierarchy.Source: §1 · pp. 1–2
  • PLE jointly embeds entity mentions, text features, and types, using a margin-based loss that requires only the best candidate type to be close to a mention.This design is intended to be robust to false candidate types.Source: §1 · p. 2
  • PLE's joint objective combines mention-type associations, corpus-level mention-feature co-occurrences, and knowledge-base type correlation to support type-path inference.The approach models type correlation using the hierarchy and entities shared between types, then estimates paths top-down.Source: §1 · p. 2
  • On three public datasets, PLE substantially reduces label noise, and PLE-denoised corpora significantly improve state-of-the-art fine-grained typing systems.The improvement is reported when the denoised corpora are used as training sets.Source: §1 · p. 2

2. BACKGROUND AND PROBLEM

This section defines LNR as selecting a context-dependent type-path from noisy candidate types in a distant-supervision training corpus. It specifies the KB, target hierarchy, and corpus inputs while treating entity linking and hierarchy creation as outside the task.

  • LNR takes a knowledge base, target hierarchy Y⊆YΨ, and automatically labeled training corpus D as inputs.Source: §2 · p. 2
  • Distant supervision links mentions to KB entities and assigns their KB types as candidate labels, which can include types irrelevant to local context.Source: §2 · p. 3
  • Candidate set Yi can contain types irrelevant to a mention’s context or overly specific, while LNR estimates the true type-path from Yi using the mention and its context.Source: §2 · p. 3
  • The desired annotation is a context-dependent path in the target hierarchy that need not end at a leaf.Source: §2 · p. 3
  • LNR assumes decent mention detection and entity resolution plus a human-curated target hierarchy; it does not address their limits or generate the hierarchy.Source: §2 · p. 3

3. LABEL NOISE REDUCTION

This section describes how PLE learns mention, text-feature, and type embeddings to reduce noisy distant-supervision labels and estimate correct type-paths through top-down search. It also covers the optimization method and computational costs.

3. LABEL NOISE REDUCTION

  • The framework treats label-noise reduction as weakly supervised learning, deriving mention–candidate-type relatedness from corpus statistics and knowledge-base facts.It models individual types, uses text features to help identify true types, and incorporates type similarity from the knowledge base and type hierarchy.Source: §3 · p. 3

3.1 Construction of Graphs

  • PLE builds a heterogeneous graph linking mentions to candidate types, mentions to text features, and types to encode correlations from the knowledge base or type hierarchy.Hierarchy shortest-path distance is efficient but may misrepresent type relatedness; the paper also measures correlation using shared knowledge-base entities.Source: §3.1 · pp. 3–5

3.2 Heterogeneous Partial-Label Embedding

  • PLE jointly embeds the graph using a margin-based objective for noisy mention–type links and distributional objectives for mention–feature and type-correlation links.The margin encourages each mention’s most relevant candidate type to score above non-candidate types, rather than treating every candidate label as correct.Source: §3.2 · p. 5

3.3 Model Learning and Inference

  • PLE learns embeddings with block-wise coordinate descent, sampling negative links during updates, then estimates each mention’s true type path by searching its candidate subtree top-down.The overview specifies that inference uses the learned embeddings to select the type path.Source: §3 · p. 3, §3.3 · p. 6
  • PLE optimizes its objective with a mini-batch extension of Pegasos [24], a stochastic sub-gradient method suited to massive text corpora.Algorithm 1 summarizes the learning procedure.Source: §3.3 · p. 7
  • Top-down inference selects the best child type by mention–type embedding similarity, stopping at a leaf or when the score falls below threshold η.The search begins at the candidate tree’s root and returns the estimated type-path.Source: §3.3 · p. 7
  • Graph construction costs O(K^2I) for GYY and O(N) for GMY and GMF, while PLE’s overall runtime is linear in the number of links E and independent of graph nodes.The analysis defines I as the average number of entities associated with a type in the knowledge base.Source: §3.3 · p. 7

4. EXPERIMENTS

Experiments across three public datasets evaluate PLE for label-noise reduction and downstream fine-grained typing. PLE outperforms comparison methods on denoising and improves typing-system results when its denoised corpora are used.

4.1 Data Preparation and Experiment Setting

  • Experiments use Wiki, OntoNotes, and BBN corpora, generate candidate types through distant supervision, and extract mention-context features for evaluation.The setup also compares embedding and partial-label methods, and evaluates denoising and downstream typing with held-out sets.Source: §4.1 · pp. 7–8

4.2 Label Noise Reduction

  • PLE and its variants outperform other methods on LNR Accuracy, Precision, and F1 while achieving Recall close to Raw; all three pruning heuristics suffer low Recall.PLE improves Accuracy by 40.57% and Macro-Precision by 23.89% over PTE-Raw on Wiki, and Accuracy by 16.39% over LINE-Raw on OntoNotes.Source: §4.2 · p. 8, Table 7 · p. 9
  • PLE outperforms PLE-NoCo and PLE-CoH, with KB-derived type correlations improving performance over omitting correlations or using hierarchy-based correlations.The reported gain over PLE-NoCo includes better representation of rare types.Source: §4.2 · p. 8
  • PLE's performance improves with more Wiki training mentions but becomes insensitive above a 70% sampling ratio; on BBN, it becomes insensitive when λ is small enough, around 0.01.PLE trained at 10% sampling outperforms the best PTE model trained at 70%.Source: §4.2 · p. 9, Figure 4 · p. 9

4.3 Fine-Grained Entity Typing

  • 33.53% improvement in FIGER Accuracy and 26.97% in HYENA Accuracy on BBN, with over 28% HYENA F1 and over 13% FIGER F1 improvement on Wiki versus PTE-Min.Table 9 compares the fine-grained typing systems trained on PLE-denoised corpora with other noise-reduction methods.Source: §4.3 · p. 9, Table 9 · p. 10

4.4 Case Analyses

  • Across target-type hierarchy levels, PLE outperformed PTE and WSABIE, achieving a 153% improvement in Accuracy over the best baseline, PTE, on level-3 types; deeper, finer-grained types were harder to distinguish.The authors attribute the gain mainly to explicitly modeling noisy candidate types, since most fine-grained mention-type links are false positives.Source: §4.4 · p. 10
  • Performance gains from iterative PLE re-training become marginal after three iterations, as the first round captures the heterogeneous graph's signals.Type inference remains within the original candidate set in each iteration to avoid overly low Recall.Source: §4.4 · p. 10

Fine-Grained Entity Typing.

Prior work on fine-grained entity typing uses distant supervision and a range of classification and embedding methods. This paper distinguishes its approach by addressing noisy candidate types while modeling type correlations and local context.

Fine-Grained Entity Typing

  • Existing fine-grained typing systems often assume distant-supervision candidate types are correct, whereas this framework removes false positives using local context.Earlier pruning heuristics can aggressively filter training examples and suffer low recall.Source: Fine-Grained Entity Typing · p. 10
  • Unlike prior partial-label methods, this task combines hierarchical classification with partial labels and accounts for label correlation.The compared partial-label methods model a single true label and do not consider label correlation information.Source: Fine-Grained Entity Typing · p. 11
  • Existing embedding methods assume links are correct or labels are true, while PLE removes noisy links and jointly models local mention context and KB-based type correlation.The related work contrasts PLE with both text/network embedding and KB embedding approaches.Source: Fine-Grained Entity Typing · p. 11

6. CONCLUSION AND FUTURE WORK

The paper introduces PLE for denoising candidate types in distant-supervision corpora and reports effective, robust recovery of true labels. It also identifies direct prediction for unseen mentions and several possible extensions.

6. CONCLUSION AND FUTURE WORK

  • PLE is proposed to reduce label noise in distant supervision for fine-grained entity typing by denoising candidate types in automatically labeled corpora.The authors report that it recovers true type labels effectively and robustly.Source: §6 · p. 11
  • PLE-denoised training data significantly enhances state-of-the-art typing systems, and learned embeddings can predict types for unseen mentions without an additional classifier.The paper notes this direct use saves the time needed to build additional classifiers.Source: §6 · p. 11
  • Future directions include modeling hierarchical type dependency, context topics, and KB relation facts; the authors also describe PLE as applicable to other domains.Image annotation is given as an example domain.Source: §6 · p. 11
Loading 1602.05307v1…