PaperlayerView exact revision on arXiv

Source-linked AI summary

SharP: Soft and Hard Prompt-Guided Augmentation with LLM for Low Resource Fake News Detection

Ziyu Luo, Wei Li, Haoping Huang, Kecheng Liu, Min Gao

arXiv:2610.13207v1cs.CL

TL;DR

Fake news detection in low-resource settings must contend with scarce labels and generic augmentation that may miss deceptive, domain-specific cues. SharP combines soft and hard prompts with boundary-aware sampling and dual-objective optimization; the paper reports improved discriminative performance, particularly in low-resource environments.

  • Problem

    The paper asks how LLM-generated augmentation can remain domain-consistent and label-preserving while informing classifiers under scarce labeled supervision.

    Source: §1 · pp. 2–3

  • Method

    SharP combines structured hard prompts, learned soft prompts, uncertainty-guided boundary sampling, and dual objectives for semantic alignment and classifier feedback.

    Source: §1 · p. 3

  • Results

    The authors report that SharP improves classifier discrimination, precision, and overall classification accuracy, particularly in low-resource environments.

    Source: §5 · p. 24

  • Takeaways & Limitations

    The reported conclusion is that dual-prompt LLM augmentation can generate diverse, domain-consistent samples that preserve original labels for low-resource fake news detection.

    Source: §1 · p. 3, §5 · p. 24

    The low-resource setting is operationalized by subsampling labeled training data at multiple ratios, with methods compared under the same ratios.

    Source: §1 · p. 3

Abstract

from arXiv · show

Fake news detection under low resource conditions is challenged by the scarcity of labeled data and the difficulty of capturing subtle deceptive patterns. In this work, we focus on low resource fake news detection, where only a small labeled set is available for training, while evaluation is performed under a fixed and unchanged test protocol. This setting is motivated by costly, ambiguous fact-checking and emerging topics and domains that are poorly covered by existing benchmarks. Existing data augmentation methods, such as synonym substitution, generative adversarial networks, and large language model (LLM)-based generation, often produce fluent but overly generic content, lacking the task-specific relevance needed for accurate detection. To address this, we propose SharP, a boundary-aware text generation framework guided by soft and hard prompts, which leverages LLMs as tools adapted to produce task-specific synthetic samples. SharP employs a dual prompt mechanism: hard prompts provide structured guidance and factual constraints, while soft prompts introduce learned semantic patterns from the data. These two types of prompts are combined to guide the generation process, enabling the model to produce samples that closely align with the characteristics of news. To further enhance detection performance, we incorporate a boundary-aware strategy that steers generation toward areas where the classifier is less confident, helping to clarify subtle distinctions between real and fake content. In addition, we adopt a dual objective optimization that balances semantic alignment with the source data and classifier feedback. This encourages the generation of samples that are both domain-consistent and helpful for refining decision boundaries, ultimately improving model robustness and generalization in low resource fake news detection.

1. Introduction

Low-resource fake news detection is constrained by scarce, costly-to-annotate labels, while generic LLM augmentation may miss deceptive cues and the classifier’s uncertain regions. SharP combines soft and hard prompts, boundary-aware sampling, and dual-objective optimization to guide task-aligned generation.

1. Introduction

  • Scarce, costly, ambiguous labels and limited coverage of emerging topics challenge fake news detection, while generic LLM augmentation can miss deceptive cues and model-uncertain regions.The paper identifies a mismatch between fluent general-purpose generated samples and the task-specific patterns needed for detection.Source: §1 · p. 2
  • SharP guides LLM generation with structured hard prompts and learned soft prompts, while directing augmentation toward high-uncertainty classifier regions.The framework also uses dual-objective optimization to coordinate semantic alignment and classifier feedback.Source: §1 · p. 3
  • Hard prompts encode generation rules for factual stance and stylistic fidelity, while continuously optimized soft prompts inject domain-aware inductive bias without changing LLM parameters.The prompts are intended to support domain-consistent, label-preserving generation across news topics.Source: §1 · p. 3
  • Boundary-aware active sampling targets examples near the classifier’s decision boundary, where uncertainty can provide informative training signals.The stated goal is to focus augmentation on regions where the classifier needs improved discrimination.Source: §1 · p. 3
  • The dual objective combines feature-level semantic alignment with classifier error feedback to refine soft prompts and improve task relevance.The contribution is assessed through hyperparameter sensitivity analysis.Source: §1 · pp. 3–4
  • The paper defines its low-resource setting by subsampling labeled training data at multiple ratios and training all methods under the same ratios for controlled comparison.The labeled pools grow across ratios while the overall experimental setting remains consistent.Source: §1 · p. 3

2. Related Work

Related work traces rumor detection from textual and multimodal methods toward robustness under scarce supervision, and reviews benchmark datasets with varied structures and annotation signals. It also surveys augmentation and prompting approaches, motivating domain-aligned generation that addresses the limits of task-agnostic samples and prompt adaptability.

2.1. Rumor Detection

  • Rumor detection has progressed from sequence-based text modeling to multimodal and context-aware learning, then toward augmentation and self-supervision for low-resource, evolving-event settings.Annotation remains costly and ambiguous as models encounter limited, shifting evidence.Source: §2.1 · p. 4

2.2. Benchmark Datasets for Fake News and Rumor Detection

  • Benchmarks span social-platform rumor verification and claim- or article-based fake news detection, with differing label granularity, instance structures, and auxiliary signals.The paper frames low-resource evaluation as labeled-pool subsampling while keeping the evaluation protocol fixed.Source: §2.2 · pp. 4–5

2.3. Data Augmentation for Fake News Detection

  • Text augmentation ranges from lexical and syntactic perturbations to representation mixing and prompt-guided LLM generation, but existing methods can remain task-agnostic and underuse deceptive cues.Examples include Back-Translation, EDA, MixText, and prompt-based generation.Source: §2.3 · p. 5

2.4. Soft Prompting in LLMs

  • Prompting has evolved from discrete instructions to trainable continuous prompts and structured priors, but fake news augmentation needs adaptable hard prompts and task-aligned soft-prompt signals.The cited line of work includes Prefix-Tuning, Prompt Tuning, contextual calibration, and Soft Knowledgeable Prompt-Tuning.Source: §2.4 · pp. 5–6

3. Method

SharP combines a classifier with LLM-based augmentation guided by hard and learned soft prompts, and repeatedly selects uncertain samples for labeling and augmentation. Its training balances semantic consistency with classifier feedback to refine the model under limited labels.

3. Method

  • SharP pairs a feature-extracting classifier with an LLM augmentation module guided by hard and soft prompts to address scarce training samples.Hard prompts encode generation rules, while soft prompts add task- and domain-aware guidance.Source: §3 · p. 7

3.1. Problem Definition

  • At each active-learning round, the model selects uncertain unlabeled instances for annotation, augments them, and updates the labeled pool.Queried samples near the decision boundary are chosen using predictive entropy; generated counterparts inherit their source labels.Source: §3.1 · pp. 7–8, §3.2 · p. 9

3.2. Active Learning Module

  • The classifier fuses CNN-based word features with sentiment features and uses entropy to identify samples near its decision boundary.The sentiment representation captures sentiment flow, while the fused features feed a classifier that distinguishes real from fake news.Source: §3.2 · pp. 8–9

3.3. Data Augmentation Module

  • Hard prompts preserve the news event’s main content and label, while learned soft prompts inject domain patterns and guide generation.The dual objective combines feature-space similarity with classifier feedback; a distance penalty discourages trivial overlap with the input.Source: §3.3 · pp. 10–12

3.4. Algorithm Optimization

  • Pre-training initializes the classifier and augmentation module; iterative fine-tuning labels boundary samples, tunes soft prompts, generates labeled augmentations, and retrains the classifier.The fine-tuning procedure freezes the LLM backbone while adapting prompts to newly labeled samples, then expands the labeled pool with their augmentations.Source: §3.4 · p. 13

4. Experiments

The experiments evaluate SharP across label budgets, event-specific and cross-benchmark datasets, and component and hyperparameter settings. Reported gains are strongest in low-sample settings, with additional evaluations examining generalization and design choices.

4.1. Experimental Setup

  • The setup evaluates multiple label budgets across five PHEME event subsets and LIAR and Twitter15/16, using labeled, unlabeled, and test pools.The experimental protocol uses stratified sampling across label budgets and active learning to select near-boundary unlabeled samples for labeling and training.Source: §4.1 · pp. 16–17

4.2. Baseline

  • Comparisons use classical and advanced fake-news detectors under the same datasets and settings, focusing on robustness and generalization in low-resource environments.The listed baselines include HAN, dEFEND, HMCAN, DDT, and LoRA-finetuned DeepSeek.Source: §4.2 · pp. 17–18

4.2.1. Group A: Event-specific evaluation on PHEME

  • SharP significantly improves overall performance, especially at low sample ratios, and maintains a leading position across datasets and sample configurations as training data grows.It also reports superior detection performance when samples are sufficient.Source: Table 6 · p. 19, §4.2.1 · p. 20

4.2.2. Group B: Cross-benchmark evaluation

  • SharP ranks first in accuracy in 21 of 24 LIAR and Twitter15/16 label-budget configurations and second in the remaining three.These benchmarks cover fact-checked political claims and conversational microblog threads, respectively.Source: §4.2.2 · p. 21, Table 7 · p. 21

4.3. Ablation Study

  • Ablations report significant gains from data augmentation, with additional improvements from soft prompts across metrics, especially in low-sample settings.The study compares a classifier without augmentation, augmentation without soft prompts, and augmentation with the dual prompt architecture.Source: §4.3 · pp. 21–22, Figure 6 · p. 21

4.4. Hyperparameter Sensitivity Analysis

  • Hard-prompt category differences are small with more training samples but can be substantial with fewer; the authors hypothesize classifier preference for particular styles.They hypothesize that aggressive hard prompt 3 may outperform declarative hard prompt 4 because fake news often uses provocative, attention-grabbing styles.Source: Figure 7 · p. 22, §4.4 · p. 23
  • Soft-prompt initialization length has a peak: short prompts may encode too little guidance, while long prompts take up more of the concatenated input and hinder task understanding.The reported performance first improves and then declines as initialization length increases.Source: Figure 7 · p. 22, §4.4 · p. 23
  • Increasing the number of generated texts yields an initial performance rise and then a decline, with the peak delayed when training data is scarce.The authors attribute the difference to added diversity at low data volumes and surplus semantically similar examples when data is sufficient.Source: §4.4 · pp. 23–24

5. Conclusion and Future Work

The conclusion presents SharP as dual-prompt LLM-based augmentation for low-resource fake-news detection, reporting improved classifier performance from diverse, domain-consistent samples that preserve labels. It also describes potential adaptability and annotation-efficiency benefits, while identifying broader tasks, languages, distribution shifts, and multimodal data as future evaluation directions.

5. Conclusion and Future Work

  • SharP combines structured hard prompts with learnable soft prompts for LLM-based text augmentation, and the paper reports improved discrimination and accuracy in low-resource fake-news detection.The conclusion attributes these gains to diverse, high-quality samples that are domain-consistent and preserve original labels.Source: §5 · p. 24
  • The authors describe SharP as model-independent and adaptable across LLM architectures and training setups, with uncertain-region selection intended to use annotation resources more efficiently.The stated scope is misinformation detection scenarios and active-learning settings.Source: §5 · p. 24
  • Future work will evaluate the framework on other text-classification tasks, multilingual datasets, distribution shifts, and multimodal data.The named tasks include sentiment analysis, topic classification, and intent detection.Source: §5 · p. 24

Appendix A. Hard Prompt Templates

The appendix caption identifies a hard-prompt template for style-controlled rewriting and an example instantiation using the common style, described as the best-performing style token in the experiments.

Appendix A. Hard Prompt Templates

  • Table A.8 presents a style-controlled rewriting hard-prompt template and a common-style example, identified as the best-performing style token in the experiments.Source: Table A.8 · p. 28
Loading 2610.13207v1…