PaperlayerView exact revision on arXiv

Source-linked AI summary

Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach

Yue Yu, Simiao Zuo, Haoming Jiang, Wendi Ren, Tuo Zhao, Chao Zhang

arXiv:2010.07835v3cs.CLcs.LG

TL;DR

The paper addresses fine-tuning pre-trained language models without clean labeled data, where weak-label noise and model capacity create overfitting and error-propagation challenges. COSINE combines contrastive self-training with confidence-based reweighting and regularization, outperforming the strongest baseline across seven benchmarks and six tasks while remaining competitive with fully supervised fine-tuning.

  • Problem

    Fine-tuning pre-trained language models with only weak supervision is challenging because noisy weak labels and high model capacity can cause overfitting and error propagation.

  • Method

    COSINE uses contrastive self-training with weakly labeled and unlabeled data, plus confidence-based reweighting and regularization to suppress label-noise propagation.

  • Results

    COSINE outperforms the strongest baseline by large margins on seven benchmarks across six tasks and achieves competitive performance with fully supervised fine-tuning.

  • Takeaways & Limitations

    Contrastive regularization and confidence-based controls enable effective weakly supervised fine-tuning across sequence, token, and sentence pair classification tasks.

  • Takeaways & Limitations

    Experiments cover only several representative tasks, although the framework is described as applicable to additional tasks such as named-entity recognition and reading comprehension.

Abstract

from arXiv · show

Fine-tuned pre-trained language models (LMs) have achieved enormous success in many natural language processing (NLP) tasks, but they still require excessive labeled data in the fine-tuning stage. We study the problem of fine-tuning pre-trained LMs using only weak supervision, without any labeled data. This problem is challenging because the high capacity of LMs makes them prone to overfitting the noisy labels generated by weak supervision. To address this problem, we develop a contrastive self-training framework, COSINE, to enable fine-tuning LMs with weak supervision. Underpinned by contrastive regularization and confidence-based reweighting, this contrastive self-training framework can gradually improve model fitting while effectively suppressing error propagation. Experiments on sequence, token, and sentence pair classification tasks show that our model outperforms the strongest baseline by large margins on 7 benchmarks in 6 tasks, and achieves competitive performance with fully-supervised fine-tuning methods.

1 Introduction

Fine-tuning pre-trained language models with limited labeled data is difficult because weak supervision introduces noisy, incomplete labels and high-capacity models can overfit them. COSINE addresses this with contrastive self-training, confidence-based controls, and broad weak-supervision experiments.

  • Motivation and approach: Weak supervision alleviates expensive labeled-data requirements but commonly produces noisy labels and limited coverage that make language-model fine-tuning prone to overfitting.Examples include semantic rules such as “terrible”→Negative and “* not recommend *”→Negative.
  • Motivation and approach: COSINE combines weakly labeled and unlabeled data with contrastive self-training to fine-tune pre-trained language models using only weak supervision.The framework pushes samples with the same pseudo-labels together and samples with different pseudo-labels apart.
  • Motivation and approach: Confidence-based sample reweighting emphasizes high-confidence predictions, while confidence regularization reduces the influence of over-confident wrong pseudo-labels.
  • Results and contributions: COSINE is presented as flexible for semi-supervised learning and for biased or randomly corrupted labels, without assumptions about the weak-label source.
  • Results and contributions: Experiments span six NLP classification tasks and seven public benchmarks, with competitive performance against fully supervised models on some datasets.
  • Results and contributions: 97.2% fully-supervised versus 96.0% COSINE accuracy is reported on Yelp, showing competitive performance with full supervision.

2 Background

The paper formulates weakly supervised NLP classification from partially labeled and unlabeled samples, covering sequence, token, and sentence pair classification. Labels come from sources such as keywords and semantic rules, with majority voting for conflicting labels.

  • Problem formulation: The formulation covers sequence classification, token classification, and sentence pair classification as broad NLP task types.
  • Weak supervision: Weak supervision sources include keywords and semantic rules, and samples unmatched by rules receive no label.
  • Weak supervision: When samples receive multiple weak labels, their final labels are determined by majority voting.
  • Problem formulation: The problem uses weakly labeled and unlabeled samples to learn a classifier f(x; θ) over a label set with C classes.

3 Method

COSINE fine-tunes a pre-trained language model first on weakly labeled data, then performs contrastive self-training over all data using soft pseudo-labels, confidence selection, and regularization to reduce noise and error propagation.

  • 3.1 Overview: COSINE first initializes the language model with weakly labeled data, then conducts contrastive self-training using all available data.Early stopping is used during initialization, while later training iteratively updates pseudo-labels and model parameters.
  • 3.1 Overview: Soft pseudo-labels preserve class probabilities, reducing the impact of incorrect predictions compared with binary hard labels.The method updates pseudo-labels from the current model and uses them in subsequent model updates.
  • 3.2 Contrastive Learning on Sample Pairs: Contrastive regularization pulls same-class representations together and pushes different-class representations apart by a margin.The regularizer operates on high-confidence samples and is designed to create smaller intra-class distances and clearer inter-class boundaries.
  • 3.3 Confidence-based Sample Reweighting and Regularization: Confidence-based reweighting gives less influence to uncertain samples by using pseudo-label entropy to select high-confidence examples.A threshold selects the high-confidence subset used for contrastive self-training.
  • 3.3 Confidence-based Sample Reweighting and Regularization: Confidence regularization smooths predictions to prevent over-confident errors, complementing reweighting when wrongly labeled samples are not actually low-confidence.The regularizer uses KL divergence against a uniform distribution and is intended to improve generalization.

4 Experiments

Experiments evaluate COSINE across seven benchmarks spanning six NLP classification tasks, comparing weak-supervision and semi-supervised settings with analyses of robustness, label efficiency, and component effects. COSINE consistently outperforms baseline models, matches clean-label performance on several datasets, and corrects noisy predictions while mitigating error propagation.

  • Learning From Weak Labels: COSINE outperforms all baseline models across the evaluated datasets, with classification accuracy reported as the mean over three runs.The experiments cover seven public benchmarks across topic, sentiment, slot filling, question, relation, and word-sense classification tasks.
  • Learning From Weak Labels: COSINE reaches the same performance level as clean-label RoBERTa on AGNews, IMDB, Yelp, and WiC despite using only weak supervision.This result highlights its performance in settings where clean labels are unavailable.
  • Robustness to Label Noise: When TREC label corruption is below 40%, COSINE performs close to the fully supervised method and consistently outperforms advanced fine-tuning and self-training baselines.The corruption experiment randomly changes labels to another class to model crowd-sourcing errors.
  • Semi-supervised Learning: On WiC, COSINE with transductive learning outperforms Snorkel and is compared with semi-supervised baselines that use different supervision resources.The comparison allows access to unlabeled validation and test data and their weak labels to match Snorkel’s setting.
  • Case Study and Ablation Study: Contrastive self-training corrects many incorrect predictions within three iterations, while confidence-based reweighting filters low-confidence samples to mitigate error propagation.Removing sample reweighting eventually leads the model to overfit label noise, and excessive training or poorly timed pseudo-label updates also degrade performance.

5 Related Works

Prior fine-tuning methods mainly target fully supervised settings and depend on large amounts of clean labeled data, motivating weak-label fine-tuning with COSINE.

  • Most existing fine-tuning methods focus on fully supervised settings and rely heavily on large amounts of clean labels.
  • COSINE addresses this limitation by fine-tuning pre-trained language models using only weak labels.

6 Discussions

The discussion presents COSINE as adaptable across domains, weak-supervision sources, learning settings, and NLP task types.

  • COSINE can use label names or descriptions as weak-supervision signals, avoiding the need for hand-crafted rules.
  • The framework is intended to extend beyond the conducted experiments to crowd-sourced supervision, named entity recognition, and reading comprehension.

7 Conclusion

The paper concludes that COSINE combines contrastive regularization, self-training, confidence-based reweighting, and regularization to fine-tune pre-trained language models with weak supervision.

  • COSINE combines contrastive regularization and self-training with confidence-based reweighting and regularization to reduce label-noise propagation.
  • Experiments across sequence, token, and sentence pair classification tasks demonstrate the efficacy of COSINE.

Broader Impact

COSINE combines weak supervision with unlabeled data to reduce the need for human annotation in downstream NLP classification tasks.

  • COSINE combines neural networks with weak supervision to address label scarcity for downstream NLP classification.
  • The paper reports no direct social consequences or ethical issues and uses publicly available data and public code bases.
  • Combining weak supervision with unlabeled data can produce more labeled examples than human labor and support label-efficient NLP classification.
  • Weak supervision sources include keywords and semantic rules, which assign labels to matched samples while leaving unmatched samples unlabeled.
  • Keyword rules label an input when it matches a word in a specified list.
  • Pattern rules label inputs that match a regular expression, while third-party tools can provide additional weak labels.

C Details on Experiment Setups

The experiments use standard hardware and optimization settings, compare COSINE with multiple baseline groups, and report averaged results with task-specific hyperparameter searches.

  • Optimization: Training uses AdamW with learning rates of 1 × 10−5, 2 × 10−5, or 3 × 10−5, linear decay, 0.1 warm-up, and five epochs.
  • Experiment configuration: Results are averaged over three runs after grid-searching task-specific values for T1, T2, T3, ξ, and λ.The searched ranges are T1: 10–2000, T2: 1000–5000, T3: 10–500, ξ: 0–1, and λ: 0–0.5.
  • Compared systems: COSINE and most baselines use RoBERTa-base with about 125M parameters, while Snorkel, ImplyLoss, and Denoise use substantially fewer parameters.ImplyLoss and Denoise freeze the embedding and have fewer than 1M parameters.
  • Experimental materials: The experimental materials include examples of semantic rules for Chemprot and a table of hyperparameter configurations.

D Early Stopping and Earlier Stopping

COSINE uses earlier stopping during weak-label initialization to limit overfitting, and the reported IMDB example illustrates why this differs from standard early stopping.

  • Earlier stopping: Earlier stopping fine-tunes pre-trained LMs for only a few initialization steps, before the evaluation score begins dropping, to prevent overfitting.
  • Earlier stopping: On IMDB, weak-label initialization overfits after 240 iterations, whereas clean-label fine-tuning still performs well after 400 iterations.
  • Design context: The section presents COSINE performance under different settings in Table 8 and describes the contrastive regularizer through sample distances and similarities.

E.2 Sample similarity measures W

COSINE defines sample similarity from hard or soft pseudo-label relationships, while evaluating alternative distance and similarity choices for robustness.

  • Hard similarity: Hard similarity assigns binary agreement when two samples have the same hard pseudo-label and disagreement otherwise.
  • Similarity scaling: All similarity measures are scaled to [0, 1], with β = 10 used by default for the KL-based design.
  • Sampling: Doubly stochastic sampling reduces contrastive-learning cost by sampling |C| pairs from the O(|C|^2) pairs among high-confidence samples.
  • Soft similarity: Soft similarity can be computed from KL distance or L2 distance between the samples’ soft pseudo-labels.
  • Robustness: COSINE is robust to different distance and similarity choices on Agnews and MIT-R, with scaled Euclidean distance and hard similarity used by default.
Loading 2010.07835v3…