PaperlayerView exact revision on arXiv

Source-linked AI summary

Label Semantic Expansion via Label Guided Neural Topic Modeling

Haojia Zheng, Yuyin Lu, Juntian Huang, Fan Ou, Yanghui Rao, Haoran Xie, Fu Lee Wang

arXiv:2608.30216v1cs.CLcs.AI

TL;DR

Existing supervised topic models guide topics with labels but do not directly organize learned topic words around predefined categories. The paper introduces Label Semantic Expansion and LGNTM to learn dedicated, semantically grounded label-aligned topics, with experiments showing strong overall performance across alignment, expansion, topic quality, and classification evaluations.

  • Problem

    Existing label-aware topic models mainly use labels to guide topic learning, while their unlabeled topic outputs require post-hoc interpretation for label-centered analysis.

  • Method

    LGNTM realizes topics-for-labels by learning dedicated label-aligned topics grounded in lexical and document semantic spaces while preserving label-structure consistency.

  • Results

    Experiments demonstrate strong overall performance across label-topic alignment, label expansion, topic quality, and downstream classification.

  • Takeaways & Limitations

    The resulting label expansions are corpus-grounded, distinctive, and interpretable, and provide label-side semantic knowledge for downstream classification.

  • Takeaways & Limitations

    LGNTM assumes a predefined label set or taxonomy and has been evaluated only on flat or two-level label settings.

Abstract

from arXiv · show

Topic models are widely used for content analysis, where users often analyze corpora around predefined labels rather than unordered latent topics. Existing label-aware topic models mainly follow a labels-for-topics perspective, using labels to guide topic learning, while the learned topics are not directly usable for label-centered analysis. We explore the reverse topics-for-labels perspective and instantiate it as Label Semantic Expansion (LSE), which enriches sparse label representations with corpus-grounded descriptive topic words. To exploit topics in LSE effectively, we propose a Label-Guided Neural Topic Model (LGNTM), which learns dedicated label-aligned topics, grounds them in lexical and document semantic spaces, and preserves consistency between topic structures and label structures. Experiments on label-topic alignment, label expansion, topic quality, and downstream classification demonstrate strong overall performance across complementary evaluation dimensions.

1 Introduction

The paper reframes label-aware topic modeling around topics-for-labels: dedicated corpus-grounded topics expand predefined labels and support category-driven analysis. It proposes LGNTM and evaluates its alignment, expansion, topic quality, and classification utility.

  • Motivation: Existing supervised topic models use labels to guide topic learning but generally output unlabeled topic-word lists requiring post-hoc matching.This leaves users to infer topic meanings and manually align anonymous topics with predefined categories.
  • Topics-for-labels: Topics-for-labels assigns each predefined label a dedicated topic-word distribution for category-driven analysis.The perspective treats learned topics as semantic representations of labels rather than only as label-supervised latent topics.
  • Label Semantic Expansion: Label Semantic Expansion enriches short predefined labels with corpus-grounded descriptive words from their associated topic-word distributions.The formulation addresses sparse label representations, which may inadequately express lexical scope and semantic boundaries.
  • Challenge: Label-topic association alone is insufficient for useful expansions, as Labeled LDA and LANTM produce generic or weakly related words for “biochemistry.”The comparison motivates a model that produces label-specific and informative expansion words.
  • Method: LGNTM learns dedicated label-aligned topics using label-guided specialization, lexical and document-semantic grounding, and hierarchy-aware consistency regularization.Its design aims to make topics dedicated, discriminative, and faithful to the corpus.
  • Evaluation: LGNTM is evaluated on label-topic alignment, label expansion, topic quality, and downstream classification, demonstrating utility across these dimensions.The experiments assess both label-centered expansion and broader topic-model performance.

2 Related Work

Supervised topic models incorporate labels through prediction-oriented or more direct label-guided mechanisms, but they do not uniformly provide dedicated label-specific topic distributions. These differences determine how closely they approach the topics-for-labels perspective.

  • Prediction-oriented methods: Prediction-oriented models use document-topic distributions to predict labels, inducing implicit topic-label associations.Examples include sLDA, SNTM, and SCHOLAR.
  • Label-guided methods: Label-guided methods introduce labels more directly into topic inference, including restricting documents to observed-label topics or conditioning neural topic models.Labeled LDA uses hard restrictions, while neural supervised models use label-conditioned mechanisms.
  • Topics-for-labels comparison: Prediction-oriented models still output unlabeled topic-word lists and require post-hoc label matching.This limits their direct satisfaction of the topics-for-labels requirement.
  • Topics-for-labels comparison: LANTM identifies label-relevant topics with soft indicators, but labels may share dominant topics and dedicated label-specific distributions are not guaranteed.Its soft association differs from the one-to-one label-topic structure sought by the proposed approach.

3 Background and Problem Setup

The paper defines Label Semantic Expansion as enriching predefined labels with corpus-grounded topic words through label-aligned topics. LGNTM realizes this alignment by matching topic and label counts and parameterizing label-word distributions through corresponding topics.

  • Background: A topic model infers document-topic distributions and associates each latent topic with a topic-word distribution describing its lexical content.The document-topic distribution captures document–topic association, while the topic-word distribution provides topic semantics.
  • Problem setup: The setup considers a labeled corpus in which each document has labels organized across H levels, with flat labels corresponding to H = 1.At level l, the label set has size K_l.
  • Topics-for-Labels: Label Semantic Expansion asks whether label-aligned topics can enrich predefined labels with corpus-grounded topic words.The task adopts the topics-for-labels perspective.
  • Label-topic correspondence: LGNTM introduces a label-topic correspondence function linking labels at each hierarchy level to topics at the same level.The model uses this correspondence to represent each label through its aligned topic.
  • Label-topic correspondence: LGNTM sets the number of topics at each level equal to the number of labels and parameterizes each label-conditioned word distribution with its corresponding topic-word distribution.This establishes the structural basis for one-to-one label-topic alignment.

4 Label-Guided Neural Topic Model

LGNTM realizes the topics-for-labels perspective by aligning each predefined label with a dedicated topic and grounding these topics in lexical, document-semantic, and hierarchical structure.

  • Model Overview: LGNTM learns a one-to-one correspondence between predefined labels and latent topics for label-indexed semantic representations.Each label is represented through the topic-word distribution of its aligned topic.
  • Label-Guided Topic Specialization: Label-guided topic specialization sharpens document-topic distributions toward observed-label coordinates and separates topic embeddings to discourage topic collapse.The sharpening objective emphasizes low-confidence label-topic assignments, while the separation term penalizes high pairwise similarity.
  • Dual Semantic Grounding: Dual semantic grounding reconstructs both bag-of-words and pretrained document embeddings, keeping label-specialized topics tied to lexical evidence and contextual semantics.The bag-of-words objective uses a background distribution and IDF weighting.
  • Hierarchical Topic Consistency: For hierarchical labels, bidirectional projection objectives make adjacent-level topic distributions structurally compatible with parent-child label relations.The hierarchical consistency term is omitted for flat label structures.
  • Label Semantic Expansion: After training, LGNTM generates each label’s semantic expansion from the top-M words of its aligned topic-word distribution.The hierarchical consistency component is omitted when labels are flat.

5 Experiments

Experiments test whether LGNTM achieves label-topic alignment, produces useful label expansions, preserves topic quality and hierarchy, and benefits from its proposed components.

  • Experimental Settings: LGNTM is evaluated on Bills, Medical, and WoS datasets spanning flat and two-level hierarchical label settings.Bills and WoS use parent-child label hierarchies, while Medical has five flat disease-category labels.
  • Label-Topic Alignment: DTU(l) = 1.00 across all evaluated label levels for both LLDA and LGNTM, while LGNTM learns correspondence through label-guided alignment rather than hard topic masking.Figure 3 shows shared dominant topics for SCHOLAR and a clear diagonal pattern for LGNTM.
  • Label Semantic Expansion Quality: LGNTM achieves the highest LTD across all five settings and substantially outperforms baselines in preference tests for semantic expansion quality.Its expansions remain semantically cohesive and identifiable with their intended labels despite some baselines obtaining higher LNPMI.
  • Topic Model Performance: LGNTM obtains the best ARI and NMI across dataset-level blocks while remaining competitive in topic-word TD and CV.Full hierarchical evaluation also reports stronger parent-child consistency on hierarchical datasets.
  • Ablation Study: Label-indexed alignment and lexical grounding are the most critical components for LSE, while topic separation and document grounding mainly support cross-label distinctiveness.Removing hierarchical consistency primarily lowers PCCTG, whereas removing lexical grounding sharply reduces MAP and LNPMI despite retaining DTU = 1.00.

6 Utility Analysis

The utility analysis tests whether LGNTM expansions provide useful label-side semantic knowledge for LLM classification on WoS-Parent and Medical.

  • LLM-Based Classification: LGNTM words produce the best classification performance on both datasets, improving Macro-F1 by 0.1353 on WoS-Parent and 0.1206 on Medical over label-name-only prompts.The comparison uses the same prompt template while varying only label-side semantic information.
  • LLM-Based Classification: Shuffling LGNTM expansions substantially reduces performance, supporting the importance of correct label-topic-word correspondence.The evaluation samples 1,500 test documents from each dataset and reports accuracy and macro-F1.

7 Conclusion

The paper introduces the topics-for-labels perspective and LGNTM to learn dedicated, semantically grounded topics for predefined labels. Experiments find corpus-grounded, distinctive, interpretable expansions, competitive topic modeling, and useful label-side knowledge for LLM classification.

  • The topics-for-labels perspective enriches predefined labels with corpus-grounded descriptive topic words.
  • LGNTM learns dedicated, semantically grounded topics aligned with predefined labels.
  • Experiments show that LGNTM produces corpus-grounded, distinctive, and interpretable label expansions while remaining competitive as a topic model.
  • Downstream LLM-based classification confirms the utility of LGNTM expansions as label-side semantic knowledge.

Limitations

LGNTM assumes a predefined label set or taxonomy and is designed for label-centered semantic expansion rather than open-ended topic discovery or automatic taxonomy induction. Expansion quality and coverage depend on corpus and label conditions, while experiments cover only flat or two-level hierarchies.

  • LGNTM requires a predefined label set or taxonomy and does not target open-ended topic discovery or automatic taxonomy induction.
  • Expansion quality and coverage may vary with corpus distribution, label granularity, and the amount of label-associated evidence.
  • Broad or low-resource labels may receive corpus-specific subtopics without exhaustive coverage of their full semantic scope.
  • Although LGNTM supports hierarchical label structures, experiments cover only flat or two-level settings.

A Implementation Details

The implementation fixes dataset splits and averages results over five runs, using label counts to set topic counts and matching hierarchical layers to label levels. It combines fixed pretrained embeddings, a shared encoder, weighted reconstruction, early stopping, and dataset-specific hyperparameters.

  • Results use predefined training/test splits and averages across five runs.Flat models are trained separately for each label level, while hierarchical models on Bills and WoS use H layers.
  • The number of topics at each level equals the number of labels, |Z(l)| = |Y(l)| = Kl.
  • Hierarchical topic-model layers are matched to the corresponding label levels.
  • Pretrained contextual document and word embeddings are fixed, with word, document, and topic embedding dimensions all set to 384.The shared multilayer perceptron encoder has hidden dimension 256, and IDF weights are computed on the training corpus.
  • Training uses Adam with learning rate 0.002, batch size 200, a maximum of 400 epochs, and early stopping after a 10-epoch warm-up with patience 5.
  • Hyperparameters are selected using overall CV, TD, and ARI performance, with weights defined for alignment, separation, reconstruction, hierarchy, and background interpolation.
  • For hierarchical datasets, fixed projection matrices are constructed from observed parent-child label co-occurrences and row-normalized in both directions.Rows without observed links receive a uniform distribution before normalization; Medical omits hierarchical consistency because its labels are flat.

B Dominant Topic Uniqueness

Dominant Topic Uniqueness measures whether labels share dominant topics, but only as a structural diagnostic. LGNTM and LLDA achieve unique dominant topics in every setting, while LGNTM obtains this through alignment rather than hard topic masking.

  • Dominant Topic Uniqueness is computed for each dataset-level setting, with higher values indicating less dominant-topic sharing among labels.
  • Most baselines assign multiple labels to shared dominant topics, especially on fine-grained label spaces such as Bills-C and WoS-C.This limits their direct use for label-specific semantic expansion.
  • LGNTM and LLDA both achieve DTU = 1.00 in all settings.
  • LLDA obtains unique dominant topics through hard label-based restrictions, whereas LGNTM learns correspondence through label-indexed alignment without hard-masking topic usage.
  • DTU does not measure whether corresponding topic-word distributions are coherent, corpus-grounded, or useful as label expansions.

C Label Semantic Expansion Evaluation Details

The evaluation tests label semantic expansion across retrieval, coherence, distinctiveness, interpretability, hierarchy, and downstream classification, using automatic and LLM-based measures. It also examines whether label-aligned topics preserve hierarchical structure and whether correctly paired expansions improve classification.

  • Automatic Evaluation: BM25-based MAP uses the raw label name alone for the label-name baseline and concatenates the label with top-10 expansion words for other methods.Documents assigned to the target label are treated as relevant, and scores are macro-averaged over labels.
  • Automatic Evaluation: LNPMI evaluates coherence over top-10 expansion words, while LTD evaluates distinctiveness over top-25 words separately at each label level.The same top-10 and top-25 settings are used for corresponding topic-quality metrics.
  • Evaluation Scope: Bills-Child evaluation samples 20 child labels covering all 20 parent labels, whereas all labels are evaluated for the other dataset-level settings.The sampled labels and all methods use the same fixed subset.
  • Hierarchical Topic Evaluation: LGNTM achieves the highest PCCTG and PCS on both hierarchical datasets while remaining competitive on sibling diversity, indicating stronger preservation of hierarchical label structure.PCCTG measures parent grouping in document-topic space, while PCS measures parent-child topic relatedness in topic-word space.
  • Downstream Classification: Downstream classification samples 1,500 test documents per dataset and reports accuracy and macro-F1 using identical prompts that vary only in label-side semantic information.The experiments compare ranked expansion words, generated descriptions, and shuffled LGNTM word-label pairings.

F Qualitative Case Study

The qualitative case study explains why retrieval relevance or local coherence alone can produce misleading label expansions. LGNTM instead yields broader, label-specific descriptions whose correct pairing supports stronger downstream classification.

  • C-TF-IDF: C-TF-IDF favors discriminative class-level terms, explaining competitive MAP but also producing narrow or noisy retrieval cues.Its ranking emphasizes class-level term frequency normalized by prevalence across label-level documents.
  • C-TF-IDF: C-TF-IDF expansions can overrepresent narrow child-level phenomena, limiting coverage of broader parent labels such as Medical and mechanical engineering.Examples include axspa, ige, xylem, and neutronic.
  • LGNTM: LGNTM produces broader parent-level descriptions spanning multiple child subcategories, improving WoS-Parent Macro-F1 from 0.5952 for C-TF-IDF to 0.6864.The comparison links broader semantic coverage with the reported downstream classification result.
  • LLDA: LLDA’s hard label binding yields unique dominant topics, but its expansions often repeat generic high-frequency medical words across disease labels.Examples include patient, study, group, use, disease, result, and treatment.
  • LLDA: Generic but coherent LLDA words support MAP and LNPMI yet provide limited label-expansion value, reflected in lower LTD and word-intrusion scores.The case illustrates that structural binding and local co-occurrence do not guarantee distinctive expansions.
  • Label Drift: FASTopic and LA-ECRTM produce locally meaningful clusters that can drift toward neighboring labels rather than faithfully describing the target label.For WoS-Parent MAE, both methods emphasize electrical and control terminology, whereas LGNTM produces mechanical-engineering terms.
  • Downstream Implications: LGNTM’s classification gain depends on correct label-word pairing, because shuffled expansion sets remove the improvement despite preserving the same words.All downstream variants use the same prompt template and differ only in label-side semantic information.
  • Overall Interpretation: The examples support evaluating LSE through multiple complementary dimensions rather than relying on a single metric.The overall comparison distinguishes retrieval relevance, local coherence, label specificity, and interpretability.
Loading 2608.30216v1…