PaperlayerView exact revision on arXiv

Source-linked AI summary

Description Based Text Classification with Reinforcement Learning

Duo Chai, Wei Wu, Qinghong Han, Fei Wu, Jiwei Li

arXiv:2002.03067v3cs.CL

TL;DR

Standard text classification represents categories as label indexes without explicit instructions about what to classify. This paper pairs labels with descriptions generated by templates or reinforcement learning, and reports gains over strong baselines across several classification tasks.

  • Problem

    Standard text classification represents categories as label indexes, leaving the model to identify buried class cues and disentangle signals for different classes.

    Source: §1 · p. 1

  • Method

    The framework associates labels with descriptions and jointly learns to generate descriptions and assign labels, using extractive or abstractive reinforcement-learning strategies.

    Source: §1 · pp. 1–2, §4.2 · p. 4

  • Results

    The method improves over BERT across single-label and multi-label tasks; error rates fall from 27.8 to 15.6 on BeerAdvocate with Abs. and from 35.6 to 17.0 on TripAdvisor with Ext.

    Source: §5.3 · p. 7

  • Takeaways & Limitations

    Class-specific descriptions provide explicit guidance about what to classify and mitigate the issue of meaningless labels.

    Source: §7 · p. 8

    The N-class classifier strategy cannot handle multi-label classification.

    Source: §3 · p. 3

Abstract

from arXiv · show

The task of text classification is usually divided into two stages: {\it text feature extraction} and {\it classification}. In this standard formalization categories are merely represented as indexes in the label vocabulary, and the model lacks for explicit instructions on what to classify. Inspired by the current trend of formalizing NLP problems as question answering tasks, we propose a new framework for text classification, in which each category label is associated with a category description. Descriptions are generated by hand-crafted templates or using abstractive/extractive models from reinforcement learning. The concatenation of the description and the text is fed to the classifier to decide whether or not the current label should be assigned to the text. The proposed strategy forces the model to attend to the most salient texts with respect to the label, which can be regarded as a hard version of attention, leading to better performances. We observe significant performance boosts over strong baselines on a wide range of text classification tasks including single-label classification, multi-label classification and multi-aspect sentiment analysis.

1. Introduction

Standard text classification gives labels no explicit instructions, making relevant cues and class-specific signals difficult to identify. The paper proposes description-based classification with automatically generated descriptions and reports gains across several task types.

1. Introduction

  • Standard classification represents categories as label indexes, leaving the model to find potentially buried cues and disentangle signals for different classes.The paper highlights aspect sentiment, where a review may contain opposing sentiments toward different aspects.Source: §1 · p. 1
  • The framework associates each class with a description and casts classification as a SQuAD-style machine-reading task.The description explicitly tells the model what to classify.Source: §1 · pp. 1–2
  • Descriptions can direct the model toward text salient to the label, functioning as a hard version of attention.For example, a positive-location label is paired with the review text and assigned a yes/no label.Source: §1 · p. 2
  • The paper reports significant gains over strong baselines across single-label, multi-label, and multi-aspect sentiment classification.Descriptions may be handcrafted or generated automatically using reinforcement learning.Source: §1 · p. 2

2. Related Work

Prior work explores text representations and label information, while QA formulations recast NLP tasks around queries and answers. The paper distinguishes its approach by learning label descriptions automatically with reinforcement learning.

2.1. Text Classification

  • Text-classification work uses neural text encoders, label embeddings, or descriptions; ABSA work also uses auxiliary sentences and QA-based attention to select aspect-specific words.Descriptions in the cited ABSA approaches are generated from crowdsourcing, whereas this work learns label descriptions automatically with reinforcement learning.Source: §2.1 · p. 2

2.2. Formalizing NLP Tasks as Question Answering

  • QA research includes answer-span extraction and generated queries or descriptions, while prior NLP task formulations cast relation extraction, NER, summarization, and coreference as QA.The paper relates its description generation to reinforcement-learning approaches for generating descriptions or queries.Source: §2.2 · pp. 2–3

3. description-based Text Classification

The framework pairs class-specific natural-language descriptions with text and uses BERT as an illustrative backbone. It provides binary-per-class and N-class classifier strategies, with the latter unable to handle multi-label classification.

3. description-based Text Classification

  • Each text has one or multiple class labels, and each class receives a unique natural-language description encoding prior label information.BERT illustrates the method, which the authors say can be extended to other model bases with minor adjustments.Source: §3 · p. 3
  • The N-binary-classifier strategy pairs a text with one class description at a time and predicts whether that class should be assigned.For multi-label classification, the resulting label set may contain multiple labels.Source: §3 · p. 3
  • The N-classifier strategy concatenates all class descriptions with the text and predicts class probabilities using a softmax.Its input places a [CLS] token before each class description.Source: §3 · p. 3
  • The N-class classifier strategy cannot handle multi-label classification.Source: §3 · p. 3

4. Description Construction

Section 4 presents template, extractive, and abstractive strategies for constructing label descriptions. It motivates learned generation because handcrafted templates are labor-intensive and sensitive to how they are constructed.

4. Description Construction

  • Section 4 organizes description construction into template, extractive, and abstractive strategies.The section illustrates descriptions from the different strategies in Figure 1.Source: §4 · p. 3

4.1. The Template Strategy

  • Handcrafted templates can draw on sources such as Wikipedia definitions or human annotators, with Table 1 showing examples for 20news categories.Figure 1 also gives an example of descriptions constructed by different strategies.Source: §4.1 · p. 3, Table 1 · p. 4, Figure 1 · p. 4

4.2. The Extractive Strategy

  • The extractive model selects an input span for each class, using a dummy token to fall back to that class’s handcrafted template when no span applies.The classification model’s probability of assigning the correct label serves as reward for joint training of the classifier and extractor with REINFORCE.Source: §4.2 · pp. 4–5

4.3. The Abstractive Strategy

  • The abstractive strategy generates input-conditioned label descriptions with a shared seq2seq model whose class-specific embedding conditions source and target tokens.The model is initialized from a pretrained encoder-decoder that outputs template descriptions, then jointly trained with the classifier; REGS uses a discriminator to reward partially decoded sequences, and experiments find probability rewards outperform log-probability rewards.Source: §4.3 · p. 5

5. Experiments

Experiments evaluate description-based classification across single-label, multi-label, and multi-aspect sentiment benchmarks against established baselines. The results report gains over BERT across all three task families, including large error-rate reductions on the multi-aspect datasets.

5.1. Benchmarks

  • Benchmarks cover six single-label datasets, Reuters and AAPD for multi-label classification, and BeerAdvocate and TripAdvisor for multi-aspect sentiment.Source: §5.1 · pp. 5–6, Table 4 · p. 6

5.2. Baselines

  • Baselines include LSTM, hierarchical attention, label embedding, and BERT; their hyperparameters are tuned on development sets for comparison.Source: §5.2 · pp. 6–7

5.3. Results and Discussion

  • All three proposed strategies outperform BERT on single-label benchmarks; extractive and abstractive strategies also consistently outperform templates.Extractive performs better than abstractive on AGNews and IMDB, but worse on the other datasets.Source: Table 2 · p. 6, §5.3 · p. 7
  • The proposed strategies gain over BERT in F1 score on both multi-label datasets, Reuters and AAPD.Source: Table 3 · p. 6, §5.3 · p. 7
  • BeerAdvocate error rate falls from 27.8 to 15.6 with Abs., while TripAdvisor error rate falls from 35.6 to 17.0 with Ext.The authors relate these gains to reviews containing entangled aspects with opposite sentiments.Source: Table 4 · p. 6, §5.3 · p. 7

6. Ablation Studies and Analysis

The ablations examine how description-based classification varies with description construction, text length, training-data size, and RL optimization.

6.1. Impact of Human Generated Templates

  • On 20news, performance is sensitive to template construction: label indexes perform close to BERT, while Wikipedia descriptions outperform Keyword descriptions (Table 5).The authors attribute the label-index result to its lack of semantic class information and the Wikipedia advantage to more comprehensive semantics.Source: Table 5 · p. 7, §6.1 · p. 7

6.2. Impact on Examples with Different Lengths

  • Description-based models outperform vanilla models more clearly on long texts, while their advantage is smaller on short texts (Figure 2a).The comparison uses IMDB texts grouped by length.Source: §6.2 · pp. 7–8, Figure 2 · p. 8

6.4. Impact of the Size of Training Data

  • With 20% of training data, the gap between BERT and description-based models is larger; it narrows as the training-data share increases (Figure 2b).Source: Figure 2 · p. 8, §6.4 · p. 8

6.5. Impact of RL Initialization Strategies

  • RL initialization matters: extractive models are more immune to initialization, while random initialization fails to make the abstractive model converge in reasonable time (Table 6).Random initialization still achieves acceptable performance for the extractive model.Source: Table 6 · p. 8, §6.5 · p. 8

6.3. Convergence Speed

  • Template models converge faster than BERT, while abstractive and extractive RL strategies converge more slowly; extractive converges faster than abstractive.The authors attribute slower RL convergence to learning descriptions with REINFORCE and the extractive strategy's relative speed to its smaller search space.Source: §6.3 · p. 8

7. Conclusion

The paper concludes that class-specific descriptions provide explicit classification guidance through three construction strategies, with significant gains across a wide range of benchmarks.

  • The framework generates class-specific descriptions to guide what the model should classify and mitigate the problem of meaningless labels.Source: §7 · p. 8
  • It constructs descriptions using template-based, extractive, and abstractive strategies.Source: §7 · p. 8
  • The framework achieves significant performance boosts on a wide range of classification benchmarks.Source: §7 · pp. 8–9

A. Detailed Descriptions of the Used Benchmarks

The appendix describes benchmarks spanning single-label, multi-label, and multi-aspect review classification.

B. Handcrafted Templates

The appendix provides examples of handcrafted category descriptions and task-specific sentiment descriptions.

C. Descriptions Generated from the Extractive and Abstractive Model

The examples contrast template, extractive, and abstractive descriptions for movie-review labels. Extractive and abstractive descriptions can vary with the input, while template descriptions are fixed.

C. Descriptions Generated from the Extractive and Abstractive Model

Related papers

Loading 2002.03067v3…