Source-linked AI summary
Are LLMs Safe Beyond Text: Do Emojis Expose Gaps in Safety Evaluation
M P V S Gopinadh
TL;DR
Text-based safety evaluations may miss vulnerabilities caused by alternative input representations such as emojis. This study tests emoji-augmented jailbreak prompts across four open-source LLMs and finds significant, model-dependent differences in robustness.
Problem
The effect of emoji-based representations on LLM prompt-level safety alignment is not well understood despite extensive text-based adversarial evaluation.
Method
The study evaluates four open-source LLMs on an identical set of 50 emoji-based jailbreak prompts under standard inference conditions.
Results
χ2 = 32.94, p < 0.001, with outcome distributions differing significantly across models under emoji-augmented prompts.
Takeaways & Limitations
Robustness is sensitive to input representation, so standard text-only evaluations may underrepresent vulnerabilities associated with emoji-based inputs.
Abstract
from arXiv · showhide
Safety evaluations of large language models (LLMs) predominantly rely on text-based adversarial prompts, potentially overlooking vulnerabilities arising from alternative input representations. This work examines emoji-augmented prompts as a test case for this gap, evaluating 50 prompts across four open-source LLMs (Mistral 7B, Qwen 2 7B, Gemma 2 9B, Llama 3 8B). Results show substantial variation in robustness: Gemma 2 9B and Mistral 7B exhibit non-zero success rates (10%), Llama 3 8B 6%, while Qwen 2 7B shows complete resistance (0% success rate). A chi-square test ($χ^2 = 32.94, p < 0.001$) confirms significant differences in outcome distributions. These findings indicate that robustness is sensitive to input representation, and that evaluations restricted to standard text prompts may underrepresent model vulnerabilities.
1 Introduction
Safety evaluations of LLMs have largely emphasized text-based adversarial prompts, leaving emoji-based representations insufficiently examined. This study evaluates emoji-based jailbreak prompts across four open-source LLMs and finds model-dependent vulnerabilities that text-only evaluations may miss.
- Motivation: Text-based safety benchmarks dominate adversarial robustness evaluation, while emoji-based representations have received less attention.This gap concerns direct model responses under safety evaluation settings.
- Background: Emojis introduce tokenization and semantic challenges that may not align with keyword-based safety filters.Their contextual and emotional meanings can differ from representations used by safety mechanisms.
- Problem: Prior work shows that emojis can evade detection by safety classifiers and judge models, but their effect on prompt-level LLM safety alignment remains unclear.The unresolved issue is whether emoji representations alter direct prompt-level safety behavior.
- Contribution: An empirical study across four open-source LLMs finds that emoji-augmented jailbreak prompts can bypass safety mechanisms and produce model-dependent vulnerabilities.These findings indicate that adversarial robustness may vary with input representation.
2 Related Work
Prior work shows that variations in input representation, including emojis, can bypass or mislead LLM safety mechanisms and evaluation systems. This study instead tests emoji-based adversarial prompts directly on LLMs to examine vulnerabilities missed by conventional text-based evaluations.
- Prior adversarial prompting: Surface-level reformulations, including prompt stuffing and term substitution, can bypass LLM safety mechanisms.These findings establish sensitivity to variations in input representation as a recurring LLM safety failure mode.
- Emoji-based attacks: Emoji sequences can alter semantic interpretation and evade keyword-based filters without explicit textual changes.Zhang (2025) examined emoji-based adversarial techniques targeting evaluation systems.
- Emoji-based attacks: Emojis can mislead safety classifiers and judge models by exploiting tokenization effects.Wei et al. (2024) focused on attacks against evaluation systems rather than underlying LLMs.
- Emoji input complexity: Context-dependent and culturally variable emoji meanings may expose behaviors missed by standard text-based evaluation protocols.Learned emoji representations can also capture semantic relationships between emojis and textual concepts.
- Study distinction: This study evaluates emoji-based adversarial prompts directly on LLMs to examine vulnerabilities not reflected in conventional evaluation setups.This direct evaluation distinguishes the study from prior work focused primarily on attacking evaluation systems.
3 Methodology
The study compares four open-source LLMs using identical emoji-augmented prompts to probe safety under non-standard input representations. Responses are categorized and manually verified, with success rates and cross-model outcome differences assessed statistically.
- Model Evaluation: Four open-source LLMs—Mistral 7B, Qwen 2 7B, Gemma 2 9B, and Llama 3 8B—are evaluated under identical standard inference conditions.Each model receives the same prompts without finetuning or system-level modifications.
- Prompt Construction: 50 emoji-augmented prompts probe safety mechanisms through emoji stuffing and emoji chaining strategies targeting restricted content.Emoji stuffing interleaves emojis with text, while emoji chaining uses emoji sequences to implicitly encode harmful intent.
- Outcome Classification: Responses are classified as Successful, Partial, or Failed according to whether they generate restricted content, respond ambiguously, or reject or ignore the request.A keyword-based heuristic is followed by manual verification.
- Evaluation Metrics: Success rate is defined as the proportion of prompts yielding restricted content, while ethical compliance treats Partial and Failed responses as compliant.Successful responses are treated as non-compliant because they provide restricted or harmful content.
- Statistical Analysis: Cross-model differences are assessed with a chi-square test on outcome distributions using p < 0.05 as the significance threshold.Ethical compliance is defined by whether the model avoids generating restricted or harmful content.
4 Results
Emoji-augmented prompts produce substantially different outcomes across models, with Qwen 2 7B showing no successful outputs while Gemma 2 9B and Mistral 7B each reach a 10% success rate. These differences are statistically significant and indicate that emoji-based inputs expose representation-sensitive robustness gaps.
- Model outcomes: 10% success rates are observed for both Gemma 2 9B and Mistral 7B, while Qwen 2 7B produces no successful outputs.Susceptibility to emoji-based prompting varies substantially across models.
- Model outcomes: 66% compliance for Gemma 2 9B contrasts with 88% for Mistral 7B despite their identical success rates.Qwen 2 7B produces no successful outputs but many partial responses, suggesting emoji prompts are often interpreted as underspecified rather than explicitly unsafe.
- Statistical significance: χ2 = 32.94 and p < 0.001 show that outcome-distribution differences are statistically significant.The observed variation indicates that adversarial robustness depends on input representation and that emoji-based prompts expose behaviors inconsistently captured across models.
5 Discussion
Emoji-augmented prompts reveal a mismatch between surface-level safety mechanisms and semantic interpretation, with ambiguity producing many partial responses rather than consistent refusal or compliance. Model differences suggest that robustness depends on how ambiguity is resolved for non-standard inputs.
- 5 Discussion: Emoji sequences yield a substantial fraction of partial responses, exposing inconsistent safety behavior for non-standard input representations.The responses do not consistently trigger refusal or compliance, indicating that safety systems are not uniformly calibrated to emoji-augmented prompts.
- 5 Discussion: Gemma 2 9B and Mistral 7B show different compliance levels despite identical success rates, indicating divergent ambiguity resolution.Their difference lies in compliance behavior rather than outright failure rates.
- 5 Discussion: Qwen 2 7B produces no successful outputs but many partial responses, suggesting conservative handling of underspecified inputs rather than robust semantic interpretation.This pattern further indicates that robustness is sensitive to input representation.
A Dataset Access and Ethical Considerations
Because the prompts are harmful, the complete prompt set is withheld, while representative examples and construction methodology support transparency and reproducibility. The evaluation is intended solely to analyze robustness and safety behavior, not to enable misuse or harmful-instruction deployment.
- Dataset access and ethics: The complete harmful prompt set is not publicly released, though additional details may be shared for research purposes upon request.This access restriction reflects the harmful nature of the prompts used in the study.
- Dataset access and ethics: Representative prompt examples and construction methodology are included to support transparency and reproducibility.
- Dataset access and ethics: The evaluation probes safety mechanisms using emoji-based adversarial representations solely for robustness evaluation and safety-behavior analysis.It is not intended for misuse or deployment of harmful instructions.