PaperlayerView exact revision on arXiv

Source-linked AI summary

Automated Phrase Mining from Massive Text Corpora

Jingbo Shang, Jialu Liu, Meng Jiang, Xiang Ren, Clare R Voss, Jiawei Han

arXiv:1702.04457v2cs.CL

TL;DR

Phrase mining needs to extract quality phrases from massive corpora, but existing methods require linguistic analyzers or human rules and labels. AutoPhrase uses knowledge-base phrases for positive-only distant training and adds POS-guided segmentation when available. It improves phrase-mining effectiveness across five real-world datasets, domains, and languages while requiring minimal human effort.

  • Problem

    Existing phrase-mining methods depend on complex linguistic analyzers or expert-designed rules and labels, hindering fully automated analysis of new domain-specific corpora.

  • Method

    AutoPhrase combines positive-only distant training from general knowledge bases with POS-guided phrasal segmentation when a POS tagger is available.

  • Results

    AutoPhrase outperforms other phrase-mining methods across five real-world datasets in different domains and languages, with minimal human effort.

  • Takeaways & Limitations

    AutoPhrase supports effective phrase mining across scientific papers, business reviews, and Wikipedia in English, Spanish, and Chinese.

  • Takeaways & Limitations

    For languages without general knowledge bases, the paper identifies generating a noisy positive pool from the corpus as future work.

Abstract

from arXiv · show

As one of the fundamental tasks in text analysis, phrase mining aims at extracting quality phrases from a text corpus. Phrase mining is important in various tasks such as information extraction/retrieval, taxonomy construction, and topic modeling. Most existing methods rely on complex, trained linguistic analyzers, and thus likely have unsatisfactory performance on text corpora of new domains and genres without extra but expensive adaption. Recently, a few data-driven methods have been developed successfully for extraction of phrases from massive domain-specific text. However, none of the state-of-the-art models is fully automated because they require human experts for designing rules or labeling phrases. Since one can easily obtain many quality phrases from public knowledge bases to a scale that is much larger than that produced by human experts, in this paper, we propose a novel framework for automated phrase mining, AutoPhrase, which leverages this large amount of high-quality phrases in an effective way and achieves better performance compared to limited human labeled phrases. In addition, we develop a POS-guided phrasal segmentation model, which incorporates the shallow syntactic information in part-of-speech (POS) tags to further enhance the performance, when a POS tagger is available. Note that, AutoPhrase can support any language as long as a general knowledge base (e.g., Wikipedia) in that language is available, while benefiting from, but not requiring, a POS tagger. Compared to the state-of-the-art methods, the new method has shown significant improvements in effectiveness on five real-world datasets across different domains and languages.

1. INTRODUCTION

AutoPhrase addresses the need for automated, domain-independent phrase mining with minimal human effort and limited linguistic analysis. It combines knowledge-base supervision with POS-guided segmentation and demonstrates effectiveness across domains and languages.

  • Motivation: Existing phrase-mining methods rely on trained linguistic analyzers or costly expert selection and labeling, limiting adaptation to new domains and genres.SegPhrase, for example, requires experts to select and annotate hundreds of phrases from millions of candidates.
  • Motivation: Manual reliance impedes timely analysis of massive, emerging domain-specific corpora, motivating domain-independent methods with minimal human and linguistic effort.The paper defines minimal human effort as using existing general knowledge bases without other human effort.
  • Contributions: AutoPhrase uses high-quality phrases from general knowledge bases to provide positive-only distant training and eliminate additional manual labeling.Positive samples come from knowledge bases such as Wikipedia and Freebase, while negative samples come from domain corpora.
  • Contributions: AutoPhrase adds POS-guided phrasal segmentation when a pre-trained POS tagger is available, balancing language support and accuracy.The framework benefits from a POS tagger but does not require one.
  • Results: Experiments on five real-world datasets report robustness, accuracy, and improvements over prior methods across scientific papers, business reviews, and Wikipedia in English, Spanish, and Chinese.The reported datasets span multiple domains and languages.

2. RELATED WORK

Related phrase-mining work ranges from linguistic analyzers and supervised chunking to data-driven methods using corpus statistics. Although data-driven approaches reduce linguistic dependence, prior systems still require labels, thresholds, or other human effort.

  • Linguistic and retrieval approaches: Automatic term-recognition and information-retrieval research extracts technical phrases or indexing terms to represent corpus concepts rather than literal word bags.Appropriate indexing terms are important for improving search engines.
  • Linguistic and retrieval approaches: POS rules, supervised noun-phrase chunking, and dependency parsing identify candidate terms using increasingly sophisticated linguistic information.These methods rely on tagged documents or advanced NLP technologies to locate phrase boundaries.
  • Data-driven approaches: Data-driven approaches use corpus frequency statistics for candidate generation and quality estimation, avoiding complex linguistic features, domain-specific rules, and extensive labeling.Their performance depends on large corpora containing hundreds of thousands of documents.
  • Data-driven approaches: SegPhrase integrates phrase quality estimation with phrasal segmentation but requires a small set of phrase labels for training.It uses local occurrence context to rectify its initial statistical features.
  • Remaining gap: Prior approaches still depend on human effort, including domain-sensitive threshold setting, making fully automatic extension challenging.This limitation remains even when methods are otherwise data-driven or largely unsupervised.

3. PRELIMINARIES

The paper defines quality phrases, their evaluation criteria, and AutoPhrase’s pipeline from candidate generation through quality estimation and phrasal segmentation. The framework seeks phrases that are popular, concordant, informative, and complete.

  • Task definition: AutoPhrase takes a corpus and knowledge base as input and outputs a ranked list of phrases in decreasing quality.The method targets large document collections without human labeling and with limited shallow linguistic analysis.
  • Phrase definition: A phrase is a consecutive word sequence forming a complete semantic unit in particular document contexts.Quality requires popularity, concordance, informativeness, and completeness.
  • Phrase definition: Only phrases satisfying all four requirements are recognized as quality phrases.Examples distinguish concordant “strong tea” from non-concordant “heavy tea,” and complete “NP-complete in the strong sense” from its incomplete subsequence.
  • Candidate generation: The first pipeline phase generates phrase candidates from n-grams whose raw corpus frequency meets a minimum support threshold, optionally restricting phrase length.The example threshold is 30, and an example length restriction is n ≤6.
  • Quality estimation: The phrase quality estimator Q(w1w2 ... wn) is a probability in [0, 1] based initially on statistical features such as mutual information, KL divergence, and inverse document frequency.The estimator is independent of POS tags, and unigram quality is set to 1.
  • Phrasal segmentation: Phrasal segmentation finds the best sentence segmentation, after which re-estimated statistics use rectified phrase frequencies to model completeness alongside concordance and informativeness.Rectified frequency counts occurrences where a phrase forms a complete semantic unit in the selected segmentation.

4. METHODOLOGY

AutoPhrase combines knowledge-base supervision with ensemble-based noise reduction and optional POS-guided segmentation to mine phrases with minimal human effort. Its components are designed for efficient processing of large corpora.

  • 4.1 Robust Positive-Only Distant Training: AutoPhrase replaces expert-created phrase labels with positive phrases from general knowledge bases and a noisy negative pool formed from unmatched corpus candidates.Knowledge bases provide many high-quality phrases, while most unmatched n-gram candidates are treated as inferior despite possible false negatives.
  • 4.1 Robust Positive-Only Distant Training: The method trains an ensemble of unpruned decision trees on independently sampled positive and negative pools, allowing some quality phrases to appear among negative examples.Each base classifier uses a perturbed training set; the ensemble averages predictions across T trees to estimate phrase quality.
  • 4.1.2 Noise Reduction: The ensemble error approaches 0 as the number of trees T increases, although model bias requires setting T larger in practice.The sampling procedure creates classifiers with statistically independent errors and similar error probabilities.
  • 4.2 POS-Guided Phrasal Segmentation: Phrasal segmentation partitions a POS-tagged word sequence into segments using boundary indices, while locating phrase mentions and correcting their string-matching frequencies.The segmentation model represents each corpus item as a word–POS pair and defines segments through an increasing boundary sequence.
  • 4.2 POS-Guided Phrasal Segmentation: POS-guided segmentation jointly uses phrase quality and POS quality, learns parameters with Viterbi Training, and infers optimal boundaries with dynamic programming.The computationally intensive components run in O(|Ω|) when phrase length is bounded by a small constant, making AutoPhrase linear in corpus size.
  • 4.2 POS-Guided Phrasal Segmentation: The POS quality score uses tag-transition probabilities to reward complete semantic units and penalize boundaries contextually rather than assigning equal penalties to equal-length candidates.The formulation includes a boundary probability and within-phrase POS-transition probabilities, countering bias toward longer segments.

5. EXPERIMENTS

The experiments evaluate AutoPhrase on five massive corpora spanning scientific papers, business reviews, and Wikipedia in English, Spanish, and Chinese. Comparisons include expert-supervised, parser-based, segmentation, and ranking methods.

  • Experiments cover five large corpora across scientific papers, business reviews, and Wikipedia in English, Spanish, and Chinese.
  • Wikipedia-derived popular entity mentions intersected with phrase candidates to form each dataset’s positive pool.
  • Compared methods include expert-labeled SegPhrase or WrapSegPhrase, parser-based extraction with TF-IDF or TextRank, and Chinese segmentation models.
  • Parser-based extraction and Chinese segmentation models rely on pre-trained models from general corpora.
  • AutoSegPhrase differs from AutoPhrase by using a length penalty instead of δ(tx, ty), while retaining the other components.

5.3 Experimental Settings

The evaluation uses standardized preprocessing and parameters, with human annotation measuring phrase quality through precision-recall curves and AUC. AutoPhrase performs best overall and demonstrates domain and language coverage.

  • Preprocessing uses Lucene and Stanford NLP tokenizers plus TreeTagger POS tagging; the released implementation is maintained on GitHub.
  • AUC denotes the area under the precision-recall curve.
  • AutoPhrase performs best in both precision and recall across five human-annotated datasets, with about 20% higher recall than SegPhrase on Spanish.
  • Across English scientific, review, and Wikipedia datasets, AutoPhrase is reasonably domain-independent and outperforms expert-supervised SegPhrase without human effort.
  • AutoPhrase supports English, Spanish, and Chinese, with larger advantages over SegPhrase or WrapSegPhrase on Spanish and Chinese than English.
  • The authors conclude that AutoPhrase supports different domains and multiple languages with minimal human effort.

5.5 Distant Training Exploration

Distant training is tested against expert-provided positive and negative pools, while additional experiments examine positive-pool size, ensemble size, and POS-guided segmentation. Larger knowledge-base-derived pools can surpass expert labeling.

  • Distant Training Comparisons: Training pools distinguish expert-provided positives or negatives from knowledge-base-derived positives and unlabeled-candidate negatives.
  • Positive Pools: Knowledge-base-derived positive pools have reasonable quality, although their quality estimator performs slightly worse than one using expert positives.
  • Negative Pools: Noisy negatives perform slightly worse than expert-designed negatives but remain effective, as shown by the gap and similar trends across datasets.
  • Positive-Pool Size: With limited positive pools, DPDN performs worst; distant training can generate substantially larger pools than domain experts can label.
  • Positive-Pool Size: Distant training surpasses expert labeling at about 700 positive phrases on DBLP and around 1600 on Yelp.
  • Positive-Pool Size: The ideal positive-pool size is relatively small compared with corpus size, suggesting effectiveness when domain corpora overlap with Wikipedia.
  • Robustness: As the positive pool grows, AUC increases with a diminishing slope and eventually stabilizes after enough quality phrases are supplied.
  • Ensemble Size: DPDN AUC rises rapidly with more trees T before its growth gradually slows on both datasets.

5.6 POS-guided Phrasal Segmentation

POS-guided phrasal segmentation improves AutoPhrase over AutoSegPhrase across English, Spanish, and Chinese, with the largest advantage on Chinese. The results support using language-specific syntactic context rather than applying equal length penalties.

  • 5.6 POS-guided Phrasal Segmentation: The experiment evaluates POS-tag effects on Wikipedia datasets in English, Spanish, and Chinese.
  • 5.6 POS-guided Phrasal Segmentation: AutoPhrase outperforms AutoSegPhrase on English and Spanish, with a more significant advantage on Chinese.The comparison uses the best baseline methods as references, and the Chinese result indicates poor generality of the length penalty.
  • 5.6 POS-guided Phrasal Segmentation: POS tags provide extra contextual and syntactic information that can outperform equal penalties for phrases of the same length.

5.7 Case Study

The case study finds that AutoPhrase extracts many high-quality phrases, including named entities near the top and meaningful phrases deep in the ranking. Over 345K English and 116K Chinese phrases exceed a quality score of 0.5.

  • Top-ranked phrases are mostly named entities, consistent with the Wikipedia article datasets.
  • Meaningful high-quality phrases remain in the long tail, including “great spotted woodpecker” and “计算机科学技术” near rank 100,000.
  • More than 345K English and 116K Chinese phrases have phrase quality higher than 0.5.

5.8 Efficiency Evaluation

AutoPhrase scales efficiently with corpus size and parallel execution. Compared with SegPhrase and WrapSegPhrase, it substantially improves both speed and memory usage through more efficient indexing and parallelization.

  • 5.8 Efficiency Evaluation: Running time and peak memory usage grow linearly with corpus size across the EN, ES, and CN datasets.The evaluation uses 10 threads and different proportions of the three largest datasets.
  • 5.8 Efficiency Evaluation: The framework can be parallelized in an almost lock-free way and exhibits linear speedup.
  • 5.8 Efficiency Evaluation: AutoPhrase achieves about 8 to 11 times speedup and about 5 to 7 times memory usage improvement over SegPhrase and WrapSegPhrase.The gains are attributed to more efficient indexing and more thorough parallelization.

6. SINGLE-WORD PHRASES

AutoPhrase can be extended to estimate quality single-word phrases, which account for a substantial share of high-quality phrases. This extension improves recall across English, Spanish, and Chinese, especially for Chinese tokenization.

  • 6. SINGLE-WORD PHRASES: AutoPhrase+ extends AutoPhrase with quality single-word phrase estimation and can improve recall by about 10% to 30% across datasets.
  • 6. SINGLE-WORD PHRASES: Single-word phrases are evaluated using popularity, informativeness, and independence because concordance and completeness are undefined for them.
  • 6. SINGLE-WORD PHRASES: The independence feature is the ratio of rectified segmented frequency to raw frequency, with larger values indicating better single-word phrase quality.The ratio is expected to be nearly zero for “united,” which usually occurs inside multi-word phrases.
  • 6. SINGLE-WORD PHRASES: Human annotation pools for the EN, ES, and CN evaluation have intra-class correlations above 0.9.
  • 6. SINGLE-WORD PHRASES: Figure 10 compares all methods using new annotation pools, and methods other than SegPhrase and WrapSegPhrase extract single-word phrases.
  • 6. SINGLE-WORD PHRASES: AutoPhrase+ shows significant recall advantages on all EN, ES, and CN datasets, with recall differences from AutoPhrase ranging from 10% to 30%.The Chinese gap appears earlier because tokenization leaves many high-quality phrases, including person names, as one token.

7. CONCLUSIONS

AutoPhrase combines robust positive-only distant training with POS-guided phrasal segmentation to automate phrase mining and improve performance across domains and languages. Experiments report strong effectiveness with minimal human effort, alongside substantial efficiency gains over SegPhrase.

  • Contributions: AutoPhrase uses robust positive-only distant training and POS-guided phrasal segmentation to build an automated phrase mining framework.The POS-guided component provides further improvement when a POS tagger is available.
  • Experimental findings: AutoPhrase is domain-independent, outperforms other phrase mining methods, and supports English, Spanish, and Chinese effectively with minimal human effort.These conclusions are based on extensive experiments across different domains and languages.
  • Experimental findings: 10% to 30% increased recall follows from including quality single-word phrases such as UIUC and USA.This improvement is attributed to expanding phrase coverage beyond multi-word phrases.
  • Efficiency: 8 to 11 times running time speedup and 80% to 86% memory usage saving are achieved through better indexing strategies and more thorough parallelization over SegPhrase.The comparison is reported for the efficiency improvements described in the conclusion.
  • Future work: Future work includes refining phrases to entity mentions, extending coverage to languages such as Japanese, and generating noisy positive pools where general knowledge bases are unavailable.These directions identify remaining scope boundaries for the current framework.
Loading 1702.04457v2…