PaperlayerView exact revision on arXiv

Source-linked AI summary

TaxoGen: Unsupervised Topic Taxonomy Construction by Adaptive Term Embedding and Clustering

Chao Zhang, Fangbo Tao, Xiusi Chen, Jiaming Shen, Meng Jiang, Brian Sadler, Michelle Vanni, Jiawei Han

arXiv:1812.09551v1cs.DB

TL;DR

The paper contrasts term-level taxonomies with topic taxonomies whose nodes are clusters of semantically coherent terms. TaxoGen recursively combines term embeddings and hierarchical clustering, with adaptive clustering and local embeddings. Experiments on two real datasets report high-quality taxonomies and significant baseline outperformance in user-study quantitative analysis.

  • Problem

    The paper contrasts term-level taxonomies with topic taxonomies whose nodes represent conceptual topics as clusters of semantically coherent terms.

    Source: §1 · p. 2

  • Method

    TaxoGen recursively uses term embeddings and hierarchical clustering, with adaptive clustering to allocate terms to levels and local embeddings for stronger lower-level discrimination.

    Source: §1 · p. 2

  • Results

    TaxoGen significantly outperforms baseline methods in user-study quantitative analysis on two real datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Both adaptive clustering and local embeddings are reported as useful for improving the quality of the resultant taxonomy.

    Source: §6 · p. 9

    The current version requires a pre-specified number of clusters when splitting a coarse topic.

    Source: §6 · p. 9

Abstract

from arXiv · show

Taxonomy construction is not only a fundamental task for semantic analysis of text corpora, but also an important step for applications such as information filtering, recommendation, and Web search. Existing pattern-based methods extract hypernym-hyponym term pairs and then organize these pairs into a taxonomy. However, by considering each term as an independent concept node, they overlook the topical proximity and the semantic correlations among terms. In this paper, we propose a method for constructing topic taxonomies, wherein every node represents a conceptual topic and is defined as a cluster of semantically coherent concept terms. Our method, TaxoGen, uses term embeddings and hierarchical clustering to construct a topic taxonomy in a recursive fashion. To ensure the quality of the recursive process, it consists of: (1) an adaptive spherical clustering module for allocating terms to proper levels when splitting a coarse topic into fine-grained ones; (2) a local embedding module for learning term embeddings that maintain strong discriminative power at different levels of the taxonomy. Our experiments on two real datasets demonstrate the effectiveness of TaxoGen compared with baseline methods.

1 INTRODUCTION

Taxonomy construction supports text analysis and applications such as information filtering, recommendation, and Web search. TaxoGen builds topic taxonomies by clustering semantically coherent terms and combines adaptive clustering with local embeddings.

1 INTRODUCTION

  • Automatic taxonomy construction is relevant to text analysis and applications including information filtering, recommendation, and Web search.A taxonomy can help users navigate a news corpus and help recommender systems understand user interests.Source: §1 · p. 1
  • Pattern-based term taxonomies can have low coverage, high redundancy, and limited informativeness because they treat each term as an independent concept node.They extract hypernym-hyponym pairs using predefined lexico-syntactic patterns.Source: §1 · p. 1
  • TaxoGen represents each topic as a cluster of semantically coherent concept terms, including terms that express related facets as well as synonyms.For example, an information retrieval topic may include “text retrieval” and “retrieval effectiveness.”Source: Figure 1 · p. 1, §1 · p. 2
  • TaxoGen recursively uses term embeddings and hierarchical clustering, addressing term allocation across levels and weak lower-level discrimination in global embeddings.Adaptive spherical clustering retains general terms in parent topics, while local embeddings use topic-relevant documents to capture finer-grained semantics.Source: §1 · p. 2
  • On two real datasets, qualitative results showed high-quality topic taxonomies, and user-study analysis found TaxoGen significantly outperforms baseline methods.The paper reports no numerical effect size in this passage.Source: §1 · p. 2

2 RELATED WORK

Related work covers pattern-based, clustering-based, and supervised approaches; the paper focuses on unsupervised construction of topic taxonomies.

2.1 Pattern-Based Methods

  • Pattern-based methods extract hypernym-hyponym pairs using lexical patterns and organize them into term-node taxonomies, but their low recall and entity-node structure do not suit topic taxonomies.Topic-taxonomy nodes may instead group terms representing a conceptual topic.Source: §2.1 · p. 2

2.2 Clustering-Based Methods

  • Clustering-based methods organize term representations into taxonomies; TaxoGen differs by requiring no labeled hypernym-hyponym pairs and using local embeddings with adaptive clustering.Local embeddings use topic-relevant documents to distinguish fine-grained term semantics.Source: §2.2 · pp. 2–3

2.3 Supervised Methods

  • Supervised methods learn term-pair relations from curated hypernym-hyponym data or syntactic context, but such training data are difficult to adapt to topic taxonomies and gather for domain-specific corpora.The paper therefore focuses on unsupervised taxonomy construction.Source: §2.3 · p. 3

3 PROBLEM DESCRIPTION

The task takes a document corpus and seed terms as input and constructs a tree-structured hierarchy of conceptual topics.

3 PROBLEM DESCRIPTION

  • The input consists of a document corpus D and seed terms T, which are key terms from D representing terms of interest.Source: §3 · p. 3
  • The desired output is a tree-structured hierarchy H whose nodes denote conceptual topics described by semantically coherent subsets of T.Source: §3 · p. 3
  • When a node has children, each child should be a subtopic of its parent, and siblings should have the same semantic granularity.Source: §3 · p. 3

4 THE TAXOGEN METHOD

TaxoGen builds topic taxonomies by recursively clustering term embeddings, using adaptive term allocation and topic-local embeddings to address granularity and lower-level discrimination.

4.1 Method Overview

  • Starting from a root containing all seed terms, TaxoGen generates finer topics level by level through spherical clustering until the maximum depth is reached.The process addresses general terms that should remain in parent topics and the limited lower-level discrimination of global embeddings.Source: §4.1 · p. 3

4.2 Adaptive Spherical Clustering

  • Spherical K-means groups term embeddings by direction, with each cluster’s center acting as a semantic focus for a coherent topic.The method uses cosine similarity to quantify embedding similarity.Source: §4.2 · p. 3

4.2.1 The adaptive clustering process.

  • Adaptive clustering iteratively removes general terms to the parent and refines child topics, repeating until no more general terms are detected.Removing general terms can clarify child boundaries, while clearer boundaries can help detect additional general terms.Source: §4.2.1 · p. 3, local embedding · p. 4

4.2.2 Measuring term representativeness.

  • Representativeness multiplies topic-term popularity by concentration against sibling topics; popularity uses normalized frequency, while concentration uses BM25 relevance to a topic pseudo-document.The two factors are conjunctive: a representative term should be both popular and concentrated in its topic.Source: §4.2.2 · p. 4

4.3 Local Embedding [13]

  • For each non-root topic, TaxoGen trains SkipGram on a topic subcorpus—cluster-selected first, retrieval-expanded if sparse—to improve discrimination among finer-grained terms.Local embeddings help separate general topic terms from more specific terms and distinguish subtopics.Source: §4.3 · p. 5

5 EXPERIMENTS

Experiments on DBLP and signal-processing data compare TaxoGen with hierarchical topic models and embedding-based taxonomy methods. TaxoGen produces stronger reported relation-accuracy, term-coherency, and clustering results, while qualitative examples illustrate its topic hierarchies and the differences between global and local embeddings.

5.1 Experimental Setup

  • Experiments use 13,345 DBLP terms and 6,982 signal-processing terms; methods generate four-level and three-level taxonomies, respectively.The DBLP corpus contains paper titles and the signal-processing corpus contains abstracts.Source: §5.1.1 · p. 5, §5.1.3 · p. 6

5.1.2 Compared Methods.

  • The baselines include HLDA, HPAM, hierarchical clustering, and TaxoGen variants without adaptive clustering or local embeddings.The NoLE variant uses global embeddings throughout construction; NoAC uses spherical clustering without adaptive clustering.Source: §5.1.2 · pp. 5–6

5.2 Qualitative Results

  • TaxoGen's DBLP hierarchy includes fine-grained information-retrieval and machine-learning topics, with representative terms described as coherent and covering different aspects of a topic.Examples include Web-search subtopics and neural-network topics.Source: §5.2 · p. 6
  • Compared with TaxoGen, NoLE produces duplicate parent-child topics and NoAC mixes terms of different granularities at fine levels.The paper links these qualitative issues to less distinct clustering and performance gaps.Source: §5.2 · p. 6, Figure 4 · p. 8
  • For the two DBLP queries, local-embedding neighbors are more coherent and match the query's semantic granularity better than global-embedding neighbors.The comparison uses cosine similarity to retrieve the top five terms for each query.Source: §5.2 · p. 6, Table 1 · p. 8

5.3 Quantitative Analysis

  • TaxoGen's adaptive clustering and local embeddings address different granularity challenges, while HLDA and HPAM perform worse than embedding-based methods, especially on DBLP.Adaptive clustering moves background terms to parent topics; local embeddings capture subtle term differences at lower levels. The paper attributes topic-modeling results partly to distribution assumptions and representative-term selection.Source: §5.3 · p. 8
  • TaxoGen achieves the best relation accuracy and term coherency among the compared methods on DBLP and SP.The comparison includes topic-modeling and embedding-based baselines.Source: §5.3 · p. 7, Table 2 · p. 8
  • TaxoGen achieves the smallest Davies-Bouldin index among embedding-based methods on DBLP and SP.The paper treats a smaller index as a better clustering result.Source: §5.3 · p. 8, Figure 5 · p. 9

6 CONCLUSION AND DISCUSSION

TaxoGen recursively constructs topic taxonomies using term embeddings and spherical clustering, with adaptive clustering and local embedding modules. The experiments find both modules useful, while the current method requires a pre-specified number of child clusters when splitting a topic.

6 CONCLUSION AND DISCUSSION

  • TaxoGen recursively uses term embeddings and spherical clustering, with adaptive clustering to allocate terms to levels and local embeddings to preserve discriminative power at lower levels.The conclusion identifies these as the method’s two modules for splitting coarse topics.Source: §6 · p. 9
  • Experiments show both modules improve the resulting taxonomy, giving TaxoGen advantages over existing taxonomy-construction methods.Source: §6 · p. 9
  • The current TaxoGen version requires a pre-specified number of clusters when splitting a coarse topic into finer ones.The authors identify automatic selection of the number of child topics as future work.Source: §6 · p. 9

Related papers

Loading 1812.09551v1…