PaperlayerView exact revision on arXiv

Source-linked AI summary

Low-resource Deep Entity Resolution with Transfer and Active Learning

Jungo Kasai, Kun Qian, Sairam Gurajada, Yunyao Li, Lucian Popa

arXiv:1906.08042v1cs.DBcs.CLcs.LG

TL;DR

Realistic entity-resolution tasks lack the large labeled datasets required by deep learning, despite deep models reducing feature-engineering needs. The paper combines transferable deep ER representations with active selection of informative target examples, achieving comparable or better performance with an order of magnitude fewer labels.

  • Problem

    Deep ER methods reduce feature engineering but require large labeled datasets, which realistic ER tasks typically lack.

  • Method

    The method transfers a model learned from high-resource datasets and uses active learning to select informative target examples for fine-tuning.

  • Results

    The method achieves comparable or better performance than state-of-the-art learning-based methods while using an order of magnitude fewer labels.

  • Takeaways & Limitations

    Transfer and active learning make deep ER applicable to low-resource settings while preserving competitive performance.

  • Takeaways & Limitations

    The authors note that publicly available ER datasets are scarce outside the citation genre and leave broader genre and non-English applications for future work.

Abstract

from arXiv · show

Entity resolution (ER) is the task of identifying different representations of the same real-world entities across databases. It is a key step for knowledge base creation and text mining. Recent adaptation of deep learning methods for ER mitigates the need for dataset-specific feature engineering by constructing distributed representations of entity records. While these methods achieve state-of-the-art performance over benchmark data, they require large amounts of labeled data, which are typically unavailable in realistic ER applications. In this paper, we develop a deep learning-based method that targets low-resource settings for ER through a novel combination of transfer learning and active learning. We design an architecture that allows us to learn a transferable model from a high-resource setting to a low-resource one. To further adapt to the target dataset, we incorporate active learning that carefully selects a few informative examples to fine-tune the transferred model. Empirical evaluation demonstrates that our method achieves comparable, if not better, performance compared to state-of-the-art learning-based methods while using an order of magnitude fewer labels.

1 Introduction

Entity resolution links records representing the same entities, but deep learning approaches require labels that realistic, skewed ER tasks often lack. The paper combines transfer learning and active learning to adapt deep ER models with far fewer labels.

  • ER identifies different representations of the same real-world entities to support unified data views and downstream applications such as knowledge base creation and text mining.
  • Deep learning reduces dataset-specific feature engineering for ER but is data hungry, limiting its applicability when labeled records are scarce.
  • Transfer learning uses labeled source datasets to learn a transferable model, while active learning selects informative target examples for adaptation.
  • The proposed architecture learns attribute-agnostic representations through dataset adaptation, and the active learner searches high-confidence and uncertain examples.
  • The method achieves comparable or better performance than state-of-the-art learning-based methods with an order of magnitude fewer labels.

2 Background and Related Work

ER classifies candidate record pairs after blocking reduces the Cartesian product of records. Deep ER methods replace hand-designed pair features with learned representations, while the paper illustrates the task through citation-record examples.

  • 2.1 Entity Resolution: ER assumes records from two collections share a schema and classifies candidate pairs as matches or non-matches after comparing corresponding attributes.
  • 2.1 Entity Resolution: Blocking filters obvious non-matches from the Cartesian product before the matching phase classifies the remaining candidate pairs.
  • 2.2 Related Work: Deep learning methods learn distributed representations for ER and reduce the need to engineer dataset-specific matching features.

3 Deep ER Model Architecture

The deep ER architecture converts paired attribute values into learned similarities and classifies record pairs. Its universal representations and fixed-dimensional aggregation support transfer across datasets with different schemas.

  • Matching Classification: The model computes attribute representations, attribute similarities, and record similarity before an MLP classifies each candidate pair as a match or non-match.
  • Input Representations: Tokenized attribute values are represented with 300-dimensional fastText embeddings that capture subword information and support out-of-vocabulary words.
  • Attribute Representations: A universal bidirectional RNN produces attribute vectors, enabling transfer between datasets with different schemas without error-prone schema mapping.
  • Attribute Similarity: Absolute differences between paired attribute vectors form the attribute similarity vectors used to compare corresponding attributes.
  • Record Similarity: Adding all attribute similarity vectors yields a fixed-dimensional record representation, allowing networks with different attribute counts to share parameters.
  • Matching Classification: The final similarity vector enters a two-layer highway MLP, whose softmax output is a probability distribution over the two classes.

4 Deep Transfer Active Learning for ER

The method combines transferable deep ER representations with active learning to adapt a source-trained model to low-resource target datasets. Its sampling strategy balances uncertain and high-confidence examples to support stable adaptation with limited labeling.

  • Overview: The framework is presented as combining transfer and active learning with likely false-positive and false-negative detection for stable, high-performing deep ER.The evaluation uses multiple ER datasets after blocking, with dataset statistics summarized in Table 2.
  • Transfer learning: Transfer learning initializes the deep ER model from multiple source datasets, while dataset adaptation reduces sensitivity to dataset-specific properties.A dataset classifier and gradient reversal layer are used to encourage transferable representations.
  • Active learning: Active learning selects uncertain examples for human labeling and high-confidence examples with predicted labels from the unlabeled target data.The two categories are identified using entropy from the current model's match probabilities.
  • Active learning: Partition sampling separates predicted matches from non-matches before entropy ranking, preventing selection bias that can produce unstable performance or low recall.The method selects top or bottom k = K/2 examples from each prediction-based subset.
  • Active learning: The active-learning algorithm iteratively adds likely false positives and negatives for labeling, supplements them with high-confidence examples, and updates the transferred model.It retains the model with the best F1 score during each update cycle.

5 Experiments

The experiments show that transfer and active learning substantially improve deep ER in low-resource settings, across citation, restaurant, and software datasets. DTAL achieves strong performance with far fewer labels, while transfer alone is less stable and the sampling design affects recall and precision.

  • Experimental Setup: Experiments use blocked candidate pairs with 3:1:1 train-development-test splits and Magellan-based feature-engineered baselines for comparison.The cited benchmark datasets retain prior blocking and split procedures where applicable.
  • Baselines: On DBLP-ACM, deep learning improves sharply with more data and becomes the strongest of seven models at 7,000 training examples, confirming its data hunger.Other models are less affected by data scarcity, except Random Forest.
  • Results and Discussions: Transfer learning improves target performance without labels, raising F1 from 41.03 to 53.84 on DBLP-Scholar and from 38.3 to 43.13 on Cora.However, transfer alone exhibits high variance and remains less reliable than direct target training, motivating active learning.
  • Results and Discussions: DTAL remedies data scarcity across DBLP-ACM, DBLP-Scholar, and Cora, while DAL reaches comparable performance but converges faster and has higher variability.Transfer initialization gives DTAL more stable active-learning performance, including standard deviations of 0.33 versus 4.15 in DBLP-ACM.
  • Results and Discussions: DTAL achieves the best low-resource performance across each evaluated dataset, with less than 6% of training data performing comparably to fully trained DL models.In Cora, DTAL reaches 97.68 F1 points; it also outperforms SVM trained with all labels on DBLP-Scholar and Cora.
  • Other Genre Results: In other genres, 100 active-learning labels match performance from all 894 target labels in restaurants, while software DTAL outperforms alternatives using 1,200 labels.The software result corresponds to 10.4% of the training data, supporting effectiveness beyond citation datasets.
  • Other Genre Results: The evaluation scope excludes additional genres and non-English datasets because sufficiently large public ER datasets are unavailable there.The authors leave broader genre and language applications for future work.
  • Active Learning Sampling Strategies: The partition sampling mechanism substantially increases recall without reducing precision by finding more likely false negatives.It also lowers the misclassification ratio in the labeled sample by selecting likely false negatives more aggressively.

6 Further Related Work

The paper positions its transfer-learning framework for deep ER alongside prior transfer-learning work, emphasizing attribute-aware representations. It also reports sampling-strategy results and identifies partitioning as important for recall and precision-recall balance.

  • 6 Further Related Work: The work develops a transfer-learning framework for deep ER, extending a paradigm previously successful in computer vision and natural language processing.
  • 6 Further Related Work: The framework differs from concurrent transfer-learning work by computing an attribute-specific similarity vector rather than breaking attribute boundaries.This design is presented as better suited to complex ER records with many attributes.

7 Conclusion

The paper concludes that combining transfer learning with active learning enables competitive deep ER performance in low-resource settings. Transfer learning alone is insufficient for a reliable system, but the combination improves convergence and stability.

  • 7 Conclusion: The models achieve competitive or better performance than state-of-the-art learning-based methods while using an order of magnitude less labeled data.
  • 7 Conclusion: Transfer learning alone does not produce a reliable and stable ER system, but combined with active learning it improves convergence and stability.
  • 7 Conclusion: The authors suggest that these transfer- and active-learning frameworks may apply to low-resource settings beyond entity resolution.

A.1 Deep ER Hyperparameters

The appendix reports the hyperparameter configuration used for the deep ER models. The same hyperparameters are applied across scenarios and datasets, with pretrained fastText embeddings initializing 300-dimensional word representations.

  • A.1 Deep ER Hyperparameters: The deep ER models use the same hyperparameters regardless of scenario and dataset.
  • A.1 Deep ER Hyperparameters: Word embeddings are initialized with 300-dimensional character-based pretrained fastText vectors.The embeddings are described as publicly available.
  • A.1 Deep ER Hyperparameters: Table 8 lists the hyperparameters used by the deep entity resolution models.

A.2 Non-DL Learning Algorithms

The evaluation uses Magellan to run six classical learning algorithms as non-DL baselines. These baselines include tree-based, probabilistic, linear, and support-vector methods.

  • A.2 Non-DL Learning Algorithms: Six non-DL baselines are evaluated: Decision Tree, SVM, Random Forest, Naive Bayes, Logistic Regression, and Linear Regression.
  • A.2 Non-DL Learning Algorithms: Magellan is an open-source package used to run the learning-based ER baselines.
  • A.2 Non-DL Learning Algorithms: The baseline implementation computes multiple similarity functions for each schema attribute, including q-gram Jaccard, cosine distance, Levenshtein measures, Monge-Elkan, and exact matching.
Loading 1906.08042v1…