PaperlayerView exact revision on arXiv

Source-linked AI summary

Emoti-Attack: Zero-Perturbation Adversarial Attacks on NLP Systems via Emoji Sequences

Yangshijie Zhang

arXiv:2502.17392v1cs.AIcs.CLcs.CR

TL;DR

NLP adversarial attacks must address text’s discrete, nondifferentiable nature without introducing perceptible changes or semantic shifts. Emoti-Attack manipulates emoji sequences as a distinct zero-word-perturbation attack layer, reporting strong attack performance across small models and LLMs.

  • Problem

    Existing text attacks can require repeated queries or introduce perceptible changes and semantic shifts because text is discrete and nondifferentiable.

  • Method

    Emoti-Attack places strategically selected emoji sequences around text to manipulate NLP models without altering the text’s words.

  • Results

    Emoji-Attack demonstrates strong attack performance on both small models and large language models.

  • Takeaways & Limitations

    Treating emoji manipulation as a distinct attack layer provides a promising approach for adversarially testing NLP systems with minimal textual disruption.

Abstract

from arXiv · show

Deep neural networks (DNNs) have achieved remarkable success in the field of natural language processing (NLP), leading to widely recognized applications such as ChatGPT. However, the vulnerability of these models to adversarial attacks remains a significant concern. Unlike continuous domains like images, text exists in a discrete space, making even minor alterations at the sentence, word, or character level easily perceptible to humans. This inherent discreteness also complicates the use of conventional optimization techniques, as text is non-differentiable. Previous research on adversarial attacks in text has focused on character-level, word-level, sentence-level, and multi-level approaches, all of which suffer from inefficiency or perceptibility issues due to the need for multiple queries or significant semantic shifts. In this work, we introduce a novel adversarial attack method, Emoji-Attack, which leverages the manipulation of emojis to create subtle, yet effective, perturbations. Unlike character- and word-level strategies, Emoji-Attack targets emojis as a distinct layer of attack, resulting in less noticeable changes with minimal disruption to the text. This approach has been largely unexplored in previous research, which typically focuses on emoji insertion as an extension of character-level attacks. Our experiments demonstrate that Emoji-Attack achieves strong attack performance on both large and small models, making it a promising technique for enhancing adversarial robustness in NLP systems.

I. INTRODUCTION

Emoji-Attack treats emoji manipulation as a distinct adversarial layer that avoids direct text changes while targeting NLP models. The framework seeks effective, stealthy attacks through strategically positioned emoji sequences and constrains emotional consistency, sequence length, and prediction divergence.

  • I. INTRODUCTION: Emoji-Attack introduces emoji manipulation as a distinct attack layer, addressing the perceptibility and semantic-shift problems of character-, word-, and sentence-level perturbations.The approach targets emojis rather than directly modifying words or characters, and prior work had largely treated emoji insertion as an extension of character-level attacks.
  • I. INTRODUCTION: Emoji-Attack is presented as a novel adversarial method using seemingly harmless emoticons to manipulate NLP systems, with strong performance reported on both large and small models.The contribution is explicitly scoped to Emoji-Attack and its reported performance across model sizes.
  • I. INTRODUCTION: The framework achieves adversarial effects through strategically placed emoji sequences while preserving complete textual integrity.It formalizes a zero-word-perturbation setting in which emoji sequences are placed around the input text rather than altering its words.
  • I. INTRODUCTION: The optimization targets prediction divergence while constraining emotional consistency and the lengths of the prefix and suffix emoji sequences.The framework uses target-model predictions, sentiment consistency, and l_min ≤ |s|, |s′| ≤ l_max as core constraints.
  • I. INTRODUCTION: The stealthiness metric η combines sentiment consistency with a length-based penalty controlled by a trade-off parameter α.The metric is defined as η(x, s, s′) = α · δ(x, s, s′) + (1 − α) · γ(|s| + |s′|).

B. Emoji Sequence Space

The paper constructs a finite emoji-sequence space from available emojis, bounded sequence lengths, and sentiment-specific subspaces. This combinatorial space is intended to provide expressive, emotionally consistent candidate sequences for adversarial generation.

  • B. Emoji Sequence Space: The framework includes both standard Unicode emojis and ASCII emoticons to increase expressiveness while maintaining naturalness in generated perturbations.Examples include grinning-face, fire, “:)”, “QaQ”, and “;-P”.
  • B. Emoji Sequence Space: The emoji sequence space S(E) contains sequences of available emojis whose lengths lie between l_min and l_max.The construction is based on a finite emoji set E and includes sequences s = (e_1, …, e_l) with l_min ≤ l ≤ l_max.
  • B. Emoji Sequence Space: Sentiment-specific subspaces are introduced so emoji sequences can remain emotionally consistent with the original text.For each sentiment label y, the framework defines a corresponding sequence subspace S_y.
  • B. Emoji Sequence Space: For sequences of length l, the space provides |E|^l possible combinations, supplying flexibility for constructing adversarial sequences.The paper presents this combinatorial structure as supporting both expressive sequence construction and practical applicability.

C. Emotional Consistency Framework

The emotional consistency framework evaluates whether injected emoji sequences align with the input text while supporting adversarial effectiveness. It combines sentiment consistency and sequence efficiency into a stealthiness metric and states a high-stealthiness attack guarantee.

  • C. Emotional Consistency Framework: The framework defines a sentiment mapping f_sen and a binary consistency function δ to evaluate emotional alignment between text and emoji sequences.The mapping associates emoji sequences with sentiment labels, while δ evaluates consistency across the text and sequence components.
  • C. Emotional Consistency Framework: The stealthiness function η combines emotional alignment with sequence efficiency, with α controlling their trade-off.η(x, s, s′) = α · δ(x, s, s′) + (1 − α) · γ(|s| + |s′|).
  • C. Emotional Consistency Framework: ∃s, s′ ∈ S(E) such that f_tgt(s⊕x⊕s′) ≠ f_tgt(x) and η(x, s, s′) ≥ 1−ε for any arbitrarily small ε > 0.The stated guarantee combines changed model predictions with arbitrarily high stealthiness under the paper’s metric.

A. Two-Phase Learning Framework

The framework uses supervised pretraining to establish semantic priors, then reinforcement learning to optimize emoji sequences for effective and natural adversarial attacks.

  • A. Two-Phase Learning Framework: Supervised pretraining establishes semantic priors before reinforcement learning optimizes emoji-sequence attacks.An auxiliary sentiment model provides the initial semantic mappings, while the policy is trained on the dataset.
  • A. Two-Phase Learning Framework: The second phase uses an MDP-based objective with a multi-component reward balancing attack effectiveness and sequence diversity.The framework builds on and explicitly combines attack effectiveness with sequence diversity.
  • A. Two-Phase Learning Framework: Reward smoothing is added to improve training stability during reinforcement-learning optimization.The mechanism follows the approach proposed in [10].
  • A. Two-Phase Learning Framework: Three innovations combine supervised semantic priors, multi-component rewards, and temporal reward smoothing for efficient sequence optimization.The authors state that these components support attack effectiveness while maintaining sequence naturalness.

B. Specialized Sequence Generator

The specialized generator jointly represents text and emoji tokens, controls cross-modal attention, and adjusts token probabilities to generate semantically coherent adversarial emoji sequences.

  • B. Specialized Sequence Generator: A unified vocabulary integrates textual and emoji tokens for coherent multimodal sequence generation.The vocabulary combines text and emoji token sets into one representation space.
  • B. Specialized Sequence Generator: A dynamic mask matrix modulates attention differently within and across modalities.The mask sets Mij = 0 for intra-modality attention and Mij = β for cross-modality interactions.
  • B. Specialized Sequence Generator: The Emoji Logits Processor dynamically adjusts token-generation probabilities as the architecture’s central innovation.This component is designed specifically for emoji sequence generation.
  • B. Specialized Sequence Generator: The optimization objective jointly enforces emotional consistency, adversarial effectiveness, and sequence diversity.The terms use sentiment consistency, attack reward, and entropy-based diversity, with λ1 and λ2 balancing competing objectives.

IV. EXPERIMENT

The evaluation tests EmotiAttack on Go Emotion and Tweet Emoji using fine-tuned BERT and RoBERTa models across four search-space sizes.

  • IV. EXPERIMENT: The experiments use Go Emotion and Tweet Emoji to assess performance across diverse emotional and social-media language contexts.Go Emotion contains fine-grained emotion labels, while Tweet Emoji contains Twitter posts with associated emoji usage patterns.
  • IV. EXPERIMENT: Table I compares Emoji-Attack performance on RoBERTa and BERT under varying perturbation sizes.The table is scoped to model performance across perturbation configurations.
  • IV. EXPERIMENT: BERT and RoBERTa serve as the target transformer architectures, each fine-tuned for its classification task.Experiments run on an NVIDIA RTX 3090 using PyTorch and AdamW.
  • IV. EXPERIMENT: Four search-space configurations—top-1, top-3, top-15, and top-30—measure the trade-off between attack effectiveness and computational efficiency.Preprocessing protocols and evaluation metrics remain consistent across conditions.

A. Attack Performance Analysis

EmotiAttack maintains 0% perturbation while achieving high attack success across BERT and RoBERTa, with performance generally increasing as the emoji search space expands.

  • A. Attack Performance Analysis: 0% perturbation is maintained across all search-space configurations while the framework still achieves effective model manipulation.This preserves the original text’s complete textual integrity.
  • A. Attack Performance Analysis: 79.51%–96.09% ASR is achieved by RoBERTa on Go Emotion, while Tweet Emoji reaches 71.54%–95.00%.These ranges span top-1 through top-30 search spaces.
  • A. Attack Performance Analysis: BERT reaches 73.13%–87.52% ASR on Go Emotion and 40.46%–90.58% on Tweet Emoji.The reported ranges likewise span the evaluated search-space configurations.
  • A. Attack Performance Analysis: ASR increases monotonically from top-1 to top-30, with Tweet Emoji on BERT rising from 40.46% to 90.58%.The expansion produces only a modest processing-time increase in the cited example.
  • A. Attack Performance Analysis: RoBERTa shows higher vulnerability than BERT, particularly under smaller search-space configurations.The passage attributes this difference potentially to distinct pre-training objectives and contextual encoding mechanisms.

B. Performance Evaluation on State-of-the-Art Large Language Models

EmotiAttack is evaluated on state-of-the-art open-source and proprietary LLMs using zero-word-perturbation emoji attacks. The framework achieves high attack success across models and datasets while preserving the original text, with performance improving as the search space expands.

  • Evaluation setup: The evaluation covers Qwen2.5-7b-Instruct, Llama3-8b-Instruct, GPT4o, Claude3.5Sonnet, and Gemini-Exp-1206 using top-15 and top-30 search spaces.Table II reports Emoji-Attack performance on LLMs with perturbations of varying sizes.
  • Performance across LLMs: ASR ranges from 75.00% to 92.66% on Go Emotion and from 77.49% to 95.34% on Tweet Emoji across evaluated LLMs.These results indicate substantial vulnerability across contemporary LLM architectures.
  • Model comparison: Qwen2.5-7b-Instruct reaches 92.66% and 95.34% ASR, while GPT4o still reaches 79.00% and 86.00% under top-30 search.The comparison spans the most susceptible and most robust evaluated models under the same search configuration.
  • Security implications: The vulnerability persists across model families, parameter scales, and training methodologies, raising security concerns for content moderation and automated decision systems.Strategically positioned emoji sequences may manipulate outputs without modifying underlying text or triggering filters focused on textual perturbations.
  • Zero-perturbation property: The attack maintains a 0% perturbation rate across search-space configurations, preserving complete textual integrity while manipulating model behavior.This zero-word-perturbation property distinguishes the framework from attacks that directly modify text.
  • Search-space configurability: Expanding the search space from top-15 to top-30 consistently improves performance and enables calibration to specific model architectures.The configurable search space supports adaptation to different operational constraints.

V. CONCLUSION

The conclusion introduces Emoti-Attack as a zero-perturbation adversarial attack using emoji sequences, applicable to both small models and LLMs.

  • V. CONCLUSION: Emoti-Attack is presented as a zero-perturbation adversarial attack via emoji sequences that can target small models and LLMs.
Loading 2502.17392v1…