PaperlayerView exact revision on arXiv

Source-linked AI summary

HotFlip: White-Box Adversarial Examples for Text Classification

Javid Ebrahimi, Anyi Rao, Daniel Lowd, Dejing Dou

arXiv:1712.06751v2cs.CLcs.LG

TL;DR

Text classifiers can be vulnerable to adversarial inputs, motivating efficient white-box attacks that reveal and exploit these weaknesses. HotFlip uses gradient-guided atomic character flips and beam search to generate adversarial examples, and the resulting training improves robustness to attacks and unseen clean data. The method can also support word-level attacks under semantics-preserving constraints, although stronger test-time adversaries can exceed the training adversary.

  • Problem

    Text classifiers have vulnerabilities that adversarial examples can expose, but prior NLP approaches often rely on black-box heuristics without model-parameter access.

    Source: §1 · p. 1

  • Method

    HotFlip uses gradients of one-hot inputs to estimate character substitutions, insertions, and deletions, combining edits with beam search.

    Source: §1 · p. 1, §3 · p. 2, §3.1 · p. 2

  • Results

    HotFlip requires only a few manipulations to greatly increase misclassification, and beam search fools more than 90% of instances under a 0.9 confidence constraint.

    Source: §1 · p. 1, §4 · p. 3

  • Takeaways & Limitations

    Adversarial training with HotFlip examples makes models more robust to attacks and unseen clean data, and the approach can be adapted to word-level classifiers with semantics-preserving constraints.

    Source: §1 · p. 1, §6 · p. 5

    The test-time beam-search adversary is strictly stronger than the model’s internal training adversary, so trained models can still be broken by the stronger attack.

    Source: §4.1 · p. 4

Abstract

from arXiv · show

We propose an efficient method to generate white-box adversarial examples to trick a character-level neural classifier. We find that only a few manipulations are needed to greatly decrease the accuracy. Our method relies on an atomic flip operation, which swaps one token for another, based on the gradients of the one-hot input vectors. Due to efficiency of our method, we can perform adversarial training which makes the model more robust to attacks at test time. With the use of a few semantics-preserving constraints, we demonstrate that HotFlip can be adapted to attack a word-level classifier as well.

1 Introduction

HotFlip introduces an efficient white-box adversary for differentiable text classifiers and studies adversarial training as a defense. The method uses gradient-guided discrete edits, while the experiments examine attack effectiveness and robustness on clean and adversarial data.

1 Introduction

  • Adversarial examples expose poorly performing input regions and can be reused as training data to produce more robust models.The introduction frames adversarial examples as both diagnostic tools and training data.Source: §1 · p. 1
  • HotFlip’s motivating setting is white-box text classification, where model parameters enable gradient-based construction of adversarial inputs.The supplied passages contrast this setting with earlier black-box heuristic approaches.Source: §1 · p. 1
  • HotFlip proposes an efficient gradient-based method for manipulating discrete text through its one-hot representation.Its atomic operation changes one token to another using directional derivatives with respect to the one-hot input.Source: §1 · p. 1
  • The method investigates whether adversarial training improves resilience to attacks while preserving or improving accuracy on clean test data.The study evaluates both robustness to adversarial examples and performance on clean examples.Source: §1 · p. 1

2 Related Work

Prior adversarial NLP work used hand-designed text modifications, noise, or embedding perturbations, while HotFlip targets efficient generation of real-world textual adversarial examples for training.

2 Related Work

  • Earlier NLP attacks included simple character modifications and synthetic or natural character noise that exposed sensitivity in text systems.Examples include inserted spaces or dots and keyboard-typo manipulations.Source: §2 · p. 2
  • Adversarial training interleaves model training with generating adversarial examples, and prior work also perturbed word embeddings rather than producing textual inputs.The cited approaches differ from methods that create real-world textual adversarial examples.Source: §2 · p. 2
  • HotFlip is positioned as the first efficient method in the supplied discussion to generate real-world adversarial examples usable for adversarial training.This distinguishes it from black-box heuristics and embedding-only perturbations.Source: §2 · pp. 1–2

3 HotFlip

HotFlip represents discrete text edits as vectors and uses gradients to select loss-increasing character changes efficiently. Greedy or beam search then composes these changes into adversarial examples under a flip budget.

3.2 Derivatives of Operations

  • HotFlip encodes a character flip as a vector with −1 at the replaced character and 1 at the inserted character, then estimates its loss effect with a directional derivative.The flipped character must be present in the input, and the derivative identifies the change’s estimated loss increase.Source: §3.2 · p. 2
  • The adversary selects the candidate vector with the largest estimated increase in loss, using derivatives as a surrogate instead of querying the classifier for every possible edit.One backward pass replaces a large number of forward-pass queries required by exact loss evaluation.Source: §3.2 · pp. 2–3
  • Insertions and deletions are represented as sequences of character flips that shift later characters, with operation vectors normalized by their L2 norms.Insertion shifts characters right, whereas deletion shifts them left.Source: §3.2 · p. 3

3.3 Multiple Changes

  • A greedy or beam search of r steps produces an adversarial example with at most r flips, corresponding to an L0 distance of r from the original input.The search scores each candidate change using derivatives evaluated along the modified path.Source: §3.3 · p. 3
  • HotFlip’s beam search requires O(br) forward and backward passes, compared with O(brL|V |) queries for a naive exact-loss search.Here, r is the flip budget and b is the beam width; the naive method additionally scales with text length L and alphabet size |V |.Source: §3.3 · p. 3

3.1 Definitions

  • The exposition assumes each word has at most n−1 characters, leaving one padding position for representing insertion operations.This assumption supports the fixed maximum word length used in the input representation.Source: §3.1 · p. 2, §3.3 · p. 3

4 Experiments

Experiments evaluate HotFlip attacks, adversarial training, and human perception on the AG’s News classifier. HotFlip fools the classifier with few character changes, while training on its adversarial examples improves robustness, though the strongest test-time attack remains more powerful than the internal training adversary.

Attack effectiveness

  • More than 90% of instances are tricked under a 0.9 confidence constraint, while the beam-search white-box adversary is more damaging than black-box baselines.Figure 1 reports adversary success as a function of confidence; the attack uses a maximum budget of 10% of document characters.Source: §4 · p. 3, Figure 1 · p. 4
  • Only 4.18% of characters are changed on average to trick the classifier at confidence 0.5.The adversary uses flips about 80% of the time and favors deletion over insertion by two to one.Source: §4 · p. 4

Robustness

  • Training uses only character flips because insertion and deletion generation is n times slower, and applies the top r flips simultaneously after one backward pass.The number of flips is tuned to 20% of the characters in each training sample, with no confidence constraint in these experiments.Source: §4.1 · p. 4
  • HotFlip adversarial training decreases clean-data misclassification error and dramatically reduces adversary success, outperforming embedding-noise adversarial training.Table 2 compares clean-data misclassification error and adversary success rate across training approaches.Source: §4.1 · p. 4, Table 2 · p. 4
  • The adversarially trained model still has an unacceptable error rate because the beam-search test-time adversary is strictly stronger than the internal training adversary.This mismatch creates a practical boundary between efficient adversarial training and stronger evaluation attacks.Source: §4.1 · p. 4

Human Perception

  • Character-based adversarial examples rarely alter sentence meaning, with human accuracy decreasing 1.78% from 87.49% on clean examples to 85.71% on adversarial examples.The evaluation used 600 AG’s News instances annotated by at least three crowd workers; 300 were manipulated and successfully fooled the classifier.Source: §4.2 · p. 4

5 HotFlip at Word-Level

HotFlip extends to word-level classifiers using semantics-preserving constraints, but such constraints make successful adversarial examples relatively scarce. Character-level representations are more vulnerable because small edits can substantially alter their induced embeddings.

5 HotFlip at Word-Level

  • HotFlip adapts to word-level models by computing derivatives over one-hot word vectors and restricting substitutions to likely meaning-preserving changes.The constraints require cosine similarity above 0.8, matching part of speech, and no stop-word or same-lexeme replacements.Source: §5 · pp. 4–5
  • Table 3 presents sentiment adversarial examples in which bold words replace the originals after one word flip.The table illustrates the qualitative form of the constrained word-level attack rather than establishing broad prevalence.Source: Table 3 · p. 5
  • Only 41 examples, or 2% of correctly classified SST test instances, were found with one or two constrained word flips.The examples include replacing positive words with highly positive alternatives in otherwise negative reviews, but such cases were not abundant.Source: §5 · p. 5
  • Character-level models are more sensitive to small adversarial edits because their denser embedding spaces allow minor character changes to produce large representation shifts.Table 4 examines nearest neighbors of representations after adversarial edits, where the altered word often becomes more similar to other words than to the original.Source: §5 · p. 5, Table 4 · p. 5

6 Conclusion and Future Work

HotFlip uses gradient-guided character edits to expose vulnerabilities in character-level classifiers and supports adversarial training against them. The authors identify broader robustness evaluation across models and tasks, along with the discrete nature of text, as important future challenges.

6 Conclusion and Future Work

  • Gradient-guided character flips, insertions, and deletions expose high sensitivity to small perturbations, while adversarial training improves robustness to attacks and unseen clean data.The method uses derivatives for edit operations and can optimize multiple manipulations with beam search.Source: §6 · p. 5
  • Comparing robustness across different character-level models and tasks remains an important future direction for adversarial NLP.The discrete nature of text also makes the landscape of adversarial examples harder to understand.Source: §6 · p. 5

Related papers

Loading 1712.06751v2…