PaperlayerView exact revision on arXiv

Source-linked AI summary

Distilling Text Style Transfer With Self-Explanation From LLMs

Chiyu Zhang, Honglong Cai, Yuezhang, Li, Yuexin Wu, Le Hou, Muhammad Abdul-Mageed

arXiv:2403.01106v2cs.CLcs.AI

TL;DR

Parallel data for text style transfer is scarce and costly, motivating methods that can work with limited parallel supervision. CoTeX uses LLMs and chain-of-thought prompting to train smaller models for TST. The authors report gains over supervised fine-tuning and conventional distillation, especially in low-resource settings, and explainable rewriting rationales.

  • Problem

    Supervised TST depends on parallel data that is scarce and costly, and the potential of CoT prompting and knowledge distillation across TST tasks remains underexplored.

    Source: §1 · p. 1

  • Method

    CoTeX prompts LLMs to generate reasoning paths and transferred texts or explanations, then distills their TST and reasoning abilities into compact student models.

    Source: §1 · pp. 1–2, §2.1 · p. 2, §2.2 · p. 3

  • Results

    CoTeX-TA outperforms SFT and conventional KD across datasets and training-data sizes, while CoTeX-TB outperforms state-of-the-art unsupervised and ICL methods on three TST datasets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    CoT reasoning from CoTeX enables distilled student models to elucidate their rewriting procedure.

    Source: §1 · p. 2, §6 · p. 6

    The study uses three TST directions; exploring more directions, such as sentiment transfer, is future work.

    Source: §7 · p. 6

Abstract

from arXiv · show

Text Style Transfer (TST) seeks to alter the style of text while retaining its core content. Given the constraints of limited parallel datasets for TST, we propose CoTeX, a framework that leverages large language models (LLMs) alongside chain-of-thought (CoT) prompting to facilitate TST. CoTeX distills the complex rewriting and reasoning capabilities of LLMs into more streamlined models capable of working with both non-parallel and parallel data. Through experimentation across four TST datasets, CoTeX is shown to surpass traditional supervised fine-tuning and knowledge distillation methods, particularly in low-resource settings. We conduct a comprehensive evaluation, comparing CoTeX against current unsupervised, supervised, in-context learning (ICL) techniques, and instruction-tuned LLMs. Furthermore, CoTeX distinguishes itself by offering transparent explanations for its style transfer process.

1 Introduction

Text style transfer needs to preserve meaning while changing style, but parallel training data is scarce and costly. CoTeX uses LLM-generated reasoning and transferred texts to train smaller models, with reported gains across several TST settings.

1 Introduction

  • Parallel data for supervised TST is scarce and costly, while the broader use of CoT prompting and knowledge distillation across TST tasks remains underexplored.Prior approaches include unsupervised methods using unlabeled data and LLMs used for TST through in-context learning.Source: §1 · p. 1
  • CoTeX uses LLMs to generate pseudo-parallel data and distills their TST skills into compact student models, using CoT prompting to improve distillation and efficiency.The framework uses few-shot CoT prompting to generate reasoning paths and transferred texts, then trains a smaller task-specific model on the generated data.Source: §1 · p. 1, Figure 1 · p. 2
  • CoTeX-TB boosts student-model data efficiency, while CoTeX-TA outperforms SFT and conventional KD across datasets and training-data sizes.CoTeX-TB also outperforms state-of-the-art unsupervised and ICL methods on three TST datasets.Source: §1 · p. 2
  • CoT rationales let distilled student models elucidate their rewriting procedure.CoTeX applies to settings with and without parallel data and is reported effective in low-resource settings.Source: §1 · p. 2

2 Method

CoTeX prompts an LLM to produce reasoning paths and transferred text or explanations, then uses the resulting supervision to train a smaller task-specific model. It provides target-blind and target-aware data-generation settings.

2.1 Data Generation

  • Target-blind prompts give the LLM a source text and desired style; it generates a reasoning path and transferred text, guided by three manually crafted examples.The transferred text is separated from the reasoning with a special token.Source: §2.1 · p. 2
  • For target-aware generation, the prompt includes a source, its human-annotated target, and target style; the LLM generates a reasoning path, guided by three examples.The reasoning is prefixed with a distinct explanation token.Source: §2.1 · p. 2, Figure 2 · p. 2

2.2 Training Student Models

  • Student supervision combines the generated reasoning with the synthetic transferred text in target-blind training, or with the gold target text in target-aware training.The student uses the source-and-style instruction template and is trained with conventional cross-entropy loss.Source: §2.2 · p. 3

3 Experiments

The experiments use four public datasets across three transfer directions and compare CoTeX with supervised, unsupervised, distillation, and in-context methods. They report BLEU and specify the student-model training and hyperparameter settings.

3.1 Datasets and Metric

  • Four public datasets cover formality, detoxification, and Shakespeare-to-modern-English transfer; low-resource training subsets are randomly sampled from 1K to 20K examples.BLEU is the main evaluation metric.Source: §3.1 · p. 3

3.2 Model Comparison.

  • Comparisons include SFT, teacher-LLM few-shot ICL, and conventional offline distillation in low-resource settings, plus task-specific supervised and unsupervised methods, Prompt&Rank, and instruction-tuned LLMs.The instruction-tuned LLMs are evaluated through three-shot ICL using the same prompt and template described for CoTeX.Source: §3.2 · p. 3

3.3 Implementation

  • PaLM2 Unicorn generates the data, and T5-large with 770M parameters is fine-tuned on the curated dataset for 2,000 steps.The target-blind approach generates a reasoning path and transferred text; target-aware generation produces a reasoning path only.Source: §3.3 · p. 3

3.4 Hyperparameter for Training Student Model

  • Hyperparameters are tuned on the F&R formality dataset; the selected learning rate is 1e-3 and batch size is 128, with validation every 16 steps.The reported test performance is from the best validation step; Figure 3 is identified as showing low-resource test results.Source: §3.4 · pp. 3–4, Figure 3 · p. 4

4 Results

CoTeX’s full-data results compare favorably with several unsupervised, in-context, instruction-tuned, and supervised baselines, with task-specific exceptions. It also shows advantages in low-resource settings, and the section reports rationale evaluations, scaling tests, and qualitative examples.

Low-Resource Settings

  • At 1K E&M samples, BLEU is 68.62 for CoTeX-TB, 65.40 for CoTeX-TA, and 55.13 for SFT.CoTeX-TB outperforms SFT-T5 and Distill-T5 in both formality datasets, especially below 10K training examples.Source: §4 · p. 4
  • On detoxification, CoTeX-TB underperforms SFT, while CoTeX-TA outperforms SFT at most training-data sizes.Source: §4 · p. 4

Utilizing the Full Dataset

  • On full-data formality, CoTeX-TB beats unsupervised methods, Prompt&Rank, and instruction-tuned LLMs; on detoxification, CoTeX-TA beats previous supervised methods.CoTeX-TA does not exceed SemiFST on formality, while CoTeX-TB falls slightly short of CondBERT on detoxification.Source: §4 · pp. 4–5, Table 1 · p. 4
  • For Shakespeare-to-modern transfer, CoTeX-TB improves over unsupervised and in-context methods, with the gains attributed to high-quality LLM generations.FlanT5-XL leads ICL with BLEU 50.13.Source: Table 1 · p. 4, §4 · p. 5

Increasing Synthetic Data per Source Text

  • Increasing the number of CoT paths per source from 2 to 4 to 8 correlates with higher student performance on Formality (F&R) and Shakespeare-to-modern English.Source: §4 · p. 5

Qualitative Study

  • In a formality example, the student missed the idiom “hit the nail on the head,” while PaLM2 rendered it as “accurately identified the key point.”The authors hypothesize that smaller LMs may have difficulty understanding implicit style cues.Source: §4 · p. 5, Table 2 · p. 5

Human Evaluation on Generated Reasonings

  • Among 50 reviewed rationales per evaluation, CoTeX-TB responses were acceptable in 100% of detoxification cases and 74% of formality cases, though it lagged PaLM2 Unicorn.Source: §4 · p. 6, Figure 5 · p. 6

5 Related Work

Prior TST work spans supervised parallel-data methods, unsupervised approaches, and few-shot LLM prompting. CoTeX uses LLM-generated pseudo-parallel data and CoT explanations to address interpretability and cover a broader range of transfer directions.

Supervised and Unsupervised TST

  • Supervised TST commonly uses sequence-to-sequence training on parallel datasets, with multitask learning, constrained decoding, and task-specific augmentation used to improve efficacy.Source: §5 · p. 6
  • Unsupervised approaches include latent-representation disentanglement, prototype editing, attribute-specific language models, and reinforcement learning.Source: §5 · p. 6

LLMs and Interpretability

  • Few-shot ICL avoids parameter updates but typically lacks interpretability; CoTeX applies CoT explanations across more TST directions while aiming to reduce human intervention.Source: §5 · p. 6

6 Conclusion

CoTeX uses CoT prompting to elicit LLM rationales for style rewriting, then distills transfer and reasoning capabilities into smaller models. It is effective with and without parallel data, especially in low-resource settings, and its reasoning supports explainability.

6 Conclusion

  • CoTeX elicits rationales for style rewriting from LLMs and distills their text-transfer and reasoning capabilities into smaller task-specific models.Source: §6 · p. 6
  • CoTeX is effective both with and without parallel data, especially in low-resource scenarios.Source: §6 · p. 6
  • CoTeX’s chain-of-thought reasoning bolsters the explainability of text style transfer models.Source: §6 · p. 6

7 Limitations

The paper’s evidence is bounded by its three studied transfer directions, its primary use of T5-large, and its decision to report BLEU rather than other automatic metrics.

7 Limitations

  • The study incorporates three style-transfer directions and leaves exploration of additional directions, such as sentiment transfer, to future work.Source: §7 · p. 6
  • T5-large is the only student model used throughout the paper; a concise T5-XL study reports CoTeX-TA outperforming SFT on ParaDetox.Source: §7 · p. 6
  • The authors omit other automatic metrics, including BERTscore, because they lack an effective way to aggregate meaning, style, and fluency measures.Source: §7 · pp. 6–7
  • Preliminary experiments found other automatic metrics misaligned with BLEU from human-annotated references, so the paper reports BLEU.Source: §7 · pp. 6–7

8 Ethical Consideration

CoTeX uses an LLM to generate rationales alongside transferred text, which are distilled into smaller language models. The authors note that teacher-generated content and prior dataset annotations can carry biases into trained models.

8 Ethical Consideration

  • The paper’s objective is more computationally efficient and effective TST models, focusing on positive directions such as language detoxification.Source: §8 · p. 7
  • LLM-generated rationales and transferred text are distilled into smaller language models.Source: §8 · p. 7
  • Student models could inherit societal biases or hallucinations from the teacher LLM and biases from prior datasets’ annotation processes.Source: §8 · p. 7

Appendices

The appendix reports that CoTeX-TA outperforms SFT across all data sizes.

Appendices

  • CoTeX-TA outperforms SFT across all data sizes in the reported comparison.Source: Appendices · p. 11

A Preliminary Test on Evaluation Metrics

The preliminary formality-transfer experiment found that commonly used automatic metrics did not align with reference-BLEU, leading the authors to report BLEU. It evaluates similarity, transfer accuracy, and fluency using automatic measures.

A Preliminary Test on Evaluation Metrics

  • The evaluation measures similarity with BERTScore and self-BLEU, transfer accuracy with a style classifier, and fluency with GPT perplexity and a CoLA-trained grammaticality classifier.SimCSE-large is the BERTScore backbone; the transfer classifier is described as determining whether the desired style was achieved.Source: A Preliminary Test on Evaluation Metrics · p. 11
  • The automatic metrics did not align with reference-BLEU, so the paper reports BLEU.The experiment varied training sizes on the formality-transfer dataset in a target-blind setting, fine-tuning T5-large on LLM-generated synthetic data.Source: A Preliminary Test on Evaluation Metrics · p. 11
  • The preliminary experiment varied training sizes on formality transfer and used target-blind T5-large fine-tuning on LLM-generated synthetic data.The transfer classifier was a RoBERTa-base model fine-tuned on GYAFC training data and achieved 0.91 test accuracy.Source: A Preliminary Test on Evaluation Metrics · p. 11

B Experiment with T5-XL

The experiment applies CoTeX to T5-XL, a 3B-parameter model; the cited table concerns detoxification fine-tuning with CoTeX or SFT.

B Experiment with T5-XL

Loading 2403.01106v2…