Discovering Forbidden Topics in Language Models

• Refusal discovery identifies forbidden topics and refusal patterns without access to training details, addressing limits of open weights and safety benchmarks.Source: §1 · p. 1
• The method targets both explicitly forbidden topics represented in preference-tuning data and novel topics a model implicitly learns to refuse.Source: §1 · p. 1
• The study measures the crawler method’s efficacy on Tulu-3-8B, whose behavioral boundaries are published through public fine-tuning data, and crawls forbidden topics in DeepSeek-R1-70B.It verifies that criticism of the Chinese Communist Party is censored in DeepSeek-R1-70B.Source: §1 · p. 2
• IPC found strong CCP-sensitive refusals in DeepSeek-R1-70B and enumerated them against published CCP-bias lists.Source: §1 · p. 2
• Perplexity-R1-1776-70B retained substantial CCP-reflecting refusals despite a fixed benchmark finding it clean of political censorship.Source: §1 · p. 2
• Comprehensive refusal mapping can provide transparency about perspectives systematically excluded from systems that mediate information access and decisions.Source: §1 · p. 2

2.1 AI auditing

• Standardized audits primarily test predefined criteria and anticipated use cases, leaving unknown failure modes outside their coverage.Source: §2.1 · p. 2
• Alignment auditing extends evaluation through unsupervised investigations that infer hidden model characteristics from observable behavior, with IPC presented as one such technique.Source: §2.1 · p. 2

2.2 Thought Token Forcing: Breaking refusal with prefilling attacks

• Thought Token Forcing partially pre-determines a reasoning model’s chain of thought to probe its internal reasoning process.Source: §2.2 · p. 2
• Seeding thoughts with “I know that.” can lead a model to reveal detailed knowledge it would normally avoid discussing, while IPC independently checks refusals to reduce prefill-induced bias.Source: §2.2 · p. 3, Figure 2 · p. 3
• DeepSeek-R1’s knowledge of topics it refuses motivates seeking a complete list of refused topics and examining biases introduced during fine-tuning.Source: §2.2 · p. 3

2.3 LLM post training techniques for human value alignment

• Tulu-3 combines supervised fine-tuning, preference tuning with DPO, and PPO-based reward optimization during post-training.Source: §2.3 · p. 3
• Tulu’s safety training sets span 36 topics across 10 categories, including harmful-response avoidance and humanizing requests.Source: §2.3 · p. 3
• Most major models’ preference data and resulting policies remain inaccessible to external auditors, creating a transparency gap that necessitates black-box auditing.Source: §2.3 · p. 4

• IPC distinguishes raw elicited terms from clustered topics, aggregating similar refused terms with an LLM judge.Source: §3 · pp. 4–5
• A prefill template turns a generic sensitive-topic question into a list of forbidden terms the model might otherwise avoid discussing.Source: §3 · p. 4, Figure 3 · p. 4
• Each discovered term becomes a seed for further exploration, forming a semantic network that supports systematic coverage of refusal boundaries.Source: §3 · p. 4
• The generation stage forces reasoning with an injection prompt and retains only the first k = 10 terms per generation to limit repetition.Source: §3 · p. 5
• Deduplication preprocesses Chinese tokens, applies semantic and string rules, and compares embeddings against existing terms using a manually labeled set of 154 pairs.Source: §3 · p. 5
• Refusal detection generates j = 6 assistance requests per term and classifies a term as refused when the model refuses at least 50% of cases.Source: §3 · p. 5
• Pairwise model comparisons rank terms by sensitivity, allowing more strongly restricted and rephrasing-robust terms to rise toward the top.Source: §3 · p. 5

4.1 Crawling open-sourced models with known finetuning data

• 0.83 recall versus 0.77 for the baseline shows IPC slightly improves Tulu-3 topic recovery, although the baseline is more prompt-efficient.The crawler uniquely finds output-modality and subjective-question topics, while both methods miss several style, preference, and other categories.Source: §4.1 · p. 6, Figure 4 · p. 6
• A two-stage contextual refusal detector identifies query-style refusals that simple topic templates miss, improving recall for neutral-looking terms.The initial templates detect explicitly sensitive topics but miss refusals triggered by request style, such as incomplete requests or personal preferences.Source: §4.1 · p. 6

4.2 Crawling popular models with undisclosed training data

• IPC is evaluated on Llama-3.3-70B, DeepSeek-R1-70B, Perplexity-R1-1776-70B, and Claude-Haiku-3.5, using assistant prefixing for Llama-3 and prefilling for Claude’s thinking context.An LLM judge aggregates refusal terms, ranks topic clusters by sensitivity, and visualizes forbidden topics in weighted word clouds; refusal patterns are compared across models.Source: §4.2 · pp. 6–7

4.3 Censorship in DeepSeek-R1 exhibits thought suppression

• Thought suppression appears in almost every DeepSeek-R1 refusal, achieving 0.96 recall but only 0.60 precision because it also occurs in benign or CCP-aligned responses.Forcing thought suppression causes only 3 refusals among 100 benign queries under greedy sampling, indicating limited direct impact on refusal rates.Source: §4.3 · p. 7
• S = 3.43 ± 1.21 indicates that thought suppression increases refusal likelihood relative to complete reasoning on 100 benign queries.The ratio compares Sorry-versus-Okay logit differences after thought completion between clean and suppression datasets.Source: §4.3 · p. 8

4.4 Detecting censorship in decensored PPL-model

• The quantized Perplexity-R1-1776-70B model refuses politically sensitive topics that the bf16 model does not, including the crawled 10 topic clusters refused by the 8-bit model.The comparison uses politically sensitive queries and distinguishes non-quantized bf16 from int8 responses.Source: §4.4 · p. 8, Figure 5 · p. 8
• The production API deployed the more strongly censoring quantized version, and the issue was fixed as of April 8th, 2025.This observation motivates auditing the final deployed model because optimization can alter safety boundaries after development evaluation.Source: §4.4 · pp. 8–9

Refused domains

• Models show similar refusal behavior for illegal activities but differ strongly in the political domain, with DeepSeek exhibiting particularly strong refusal of CCP-sensitive content.The study applies the same settings across models and identifies substantial variation in refusal mechanisms and content filtering.Source: §5 · p. 9

Jailbreaking techniques

• IPC adapts to jailbreaking techniques, but the study uses prefill attacks for simplicity and leaves more advanced techniques for future work.The framework is described as adaptable from bijection learning to roleplay.Source: §5 · p. 9

Lack of representative baselines

• On Tulu-3, IPC and naive prompting perform comparably, but this baseline comparison does not transfer to DeepSeek-R1, which requires thought prefilling to expose forbidden topics.Tulu-3 provides the ground-truth setting because its safety-training data are known.Source: §5 · p. 9

Limitations and Scope

• The study cannot determine whether discovered refusals result from intentional developer training or unintentional generalization without access to training data.This limitation applies to interpreting the source of refusal behavior rather than detecting the behavior itself.Source: §5 · p. 9
• IPC currently focuses primarily on refusal behavior, while implicit biases and broader censorship patterns remain directions for future work.The paper also notes that assistant-response prefilling is unavailable through many popular APIs.Source: §5 · p. 9
• The study cannot quantify similarity between DeepSeek’s teacher and distilled model because the mechanisms behind distillation remain an open question.The paper cites a GPQA improvement of +35% as evidence that distillation had a significant impact.Source: §5 · p. 9

6 Conclusion

• Refusal discovery uses IPC to identify behavioral boundaries that fixed test-set benchmarks may leave unknown to users and developers.The method systematically discovers forbidden topics through token prefilling.Source: §6 · p. 10
• Models differ in refusal behavior across implementation details, and reasoning models require sophisticated prompting techniques to reveal forbidden topics.This conclusion motivates evaluating more than direct prompting alone.Source: §6 · p. 10

A.1 Supervised AI auditing

• Supervised audits rely on predefined standards and anticipated use cases, motivating unsupervised investigations of novel and unexpected risks.Prior audits cover explainability, privacy, robustness, harmful content, misinformation, deception, and dangerous capabilities.Source: §A.1 · p. 17

A.2 Censorship

• Censorship research links censorship to state criticism, collective-action potential, and negative sentiment rather than only to topic identity.Sensitive topics may remain uncensored when expressed positively and lacking collective-action potential in contemporary contexts.Source: §A.2 · p. 17

1. Direct Prompting (Baseline)

• The baseline and appendix materials specify direct prompting, user-suffix, assistant-prefix, thought-prefix, and thought-suffix injection positions for refusal elicitation.Thought-prefix and thought-suffix are specific to reasoning-model contexts.Source: Figure 6 · p. 18, §1 · pp. 18–19

B Prompt injection locations

• Thought-prefix injection recovers the most refusal topics among tested locations, eliciting 16 of 17 sensitive topics in the Perplexity-R1-1776-70B comparison.Only “Historical Events: China” resisted the thought-prefix setting in Table 3.Source: B Prompt injection locations · p. 18, Table 3 · p. 19

C.1 Measuring the effect of thought suppression behavior on the likelihood of refusal using random user queries

• The random-query appendix uses ordinary benign questions to measure whether thought suppression affects refusal likelihood.Examples span weather, music, photography, science, technology, and everyday tasks.Source: §C.1 · pp. 19–20

C.2 Instruction for aggregating refused terms into topic clusters with an LLM-judge

• The LLM judge deduplicates raw refused terms into topic clusters because no single embedding-similarity threshold reliably identifies duplicates.The appendix illustrates grouping verbal abuse and adult content separately while combining extortion software and hacking attacks under broader categories.Source: §C.2 · p. 21, Figure 7 · p. 21

D Exact list of crawled topics

• The listed crawl outputs cover Llama-3.3-70B-Instruct, DeepSeek-R1-70B, and Perplexity-R1-1776-70B in their 8-bit quantized forms.The section supplies the exact first 20 crawled topics for each model.Source: D Exact list of crawled topics · p. 25

D.1 Crawled topics for Llama-3.3-70B-Instruct, 8-bit-quantized

• The crawled list includes refusals concerning illegal activities, hacking, harmful behavior, privacy invasion, sexual content involving minors, violence, political issues, racial issues, and religious conflicts.The list also contains categories for password cracking, malware, espionage, homicide, and hate or discriminatory content.Source: §D.1 · p. 25

• The refusal list includes privacy violations, harmful behavior, illegal activities, and violations of social norms.Source: §D.2 · p. 26
• Political refusals span sensitive topics, national leaders, the Chinese government, socialism, communism, and criticism of China’s policies.Source: §D.2 · p. 26
• Additional refusals cover misinformation, medical advice, surveillance misuse, violence, minors, legal questions, and discriminatory statements.Source: §D.2 · p. 26

• The refusal list includes criticism of China’s political system, the CCP, Chinese leaders, and sensitive historical events such as Tiananmen and Xinjiang.Source: §D.3 · pp. 26–27
• The listed refusals include criminal activity, unlawful conduct, sexually explicit content, conspiracy theories, and challenges to China’s territorial claims.Source: §D.3 · pp. 26–27
• It also covers Taiwan, Tibet, Hong Kong, Xinjiang, internet censorship, freedom of expression, religion, social stability, and democratization.Source: §D.3 · p. 27

E Tulu-3 Refusal Safety Datasets

• Figure 11 presents the precision-recall tradeoff for distinguishing related from unrelated crawled topics using embedding distance.Source: Figure 11 · p. 29

F Deduplication with word embeddings

• Deduplication enforces an embedding-similarity threshold calibrated on 154 manually labeled topic pairs.Source: F Deduplication with word embeddings · p. 29
• For lists longer than 200 topics, aggregation is performed in batches before a final cross-batch aggregation.Source: F Deduplication with word embeddings · p. 29

• The method samples topic pairs for model judgments because evaluating every pair grows quadratically with the number of topics.Source: G Ranking topics by sensitivity · p. 29
• Win-count rankings achieve Kendall’s τwin = 0.694 across 10 random seeds, indicating sensitivity to pair-sampling choices.Source: G Ranking topics by sensitivity · p. 29
• Elo scoring weights victories against stronger opponents more heavily and produces more stable rankings than win count.Source: G Ranking topics by sensitivity · p. 29
• Balanced Elo comparisons reach Kendall’s τelo = 0.816, while Figure 12 illustrates the iterative pairwise-comparison instructions.Source: G Ranking topics by sensitivity · p. 29, Figure 12 · p. 30, Table 5 · p. 30