PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantification of Large Language Model Distillation

Sunbowen Lee, Junting Zhou, Chang Ao, Kaige Li, Xinrun Du, Sirui He, Haihong Wu, Tianci Liu, Jiaheng Liu, Hamid Alinejad-Rokny, Min Yang, Yitao Liang, Zhoufutu Wen, Shiwen Ni

arXiv:2501.12619v3cs.CL

TL;DR

The paper addresses the lack of systematic evidence for how much LLMs inherit from teacher models during distillation. It introduces RSE and ICE to quantify response similarity and identity-related inconsistencies, finding high distillation in most well-known models, higher levels in base than aligned models, and lower levels for Claude, Doubao, and Gemini.

  • Problem

    LLM distillation can reduce model diversity and robustness, motivating systematic quantification of its extent and impact.

    Source: §1 · p. 1, §7 · p. 8

  • Method

    The framework combines Response Similarity Evaluation across curated prompt sets with Identity Consistency Evaluation using adversarial prompts and GPTFuzz.

    Source: §1 · p. 2, §3.1 · p. 3, §3.2 · p. 4

  • Results

    Most LLMs exhibit high distillation degrees, while base models show higher degrees than aligned models and Claude, Doubao, and Gemini are exceptions.

    Source: §1 · p. 2, §7 · p. 8

  • Takeaways & Limitations

    The findings support more independent LLM development and transparent documentation of training and distillation processes.

    Source: §1 · p. 2, §7 · p. 8

    The identity-jailbreak strategy relies on GPTFuzz templates whose inductive guidance or role-playing elements may induce cognitive inconsistencies.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Model distillation is a fundamental technique in building large language models (LLMs), transferring knowledge from a teacher model to a student model. However, distillation can lead to model homogenization, reducing diversity among models and impairing their ability to robustly handle complex or novel tasks. These limitations underscore the need to systematically quantify the distillation process and its impact. In this work, we propose a framework to evaluate and quantify model distillation. Our method addresses two key aspects: (1) Identifying identity cognition contradictions to assess discrepancies in how models perceive and represent identity-related information, and (2) Analyzing multi-granularity response similarities across models to measure the extent of homogenization. Experimental results demonstrate two key insights: (1) Well-known closed-source and open-source LLMs usually exhibit high distillation degrees, except for Claude, Doubao, and Gemini. (2) Base LLMs show higher distillation degrees compared to aligned LLMs. By offering a systematic approach to improve the transparency of LLM data distillation, we call for LLMs with more independent development and more transparent technical reports to improve LLMs' robustness and safety. The code and data are available under https://github.com/Aegis1863/LLMs-Distillation-Quantification.

1 Introduction

The paper introduces RSE and ICE to quantify LLM distillation despite opaque processes and limited benchmarks. Results indicate substantial distillation across many models, with higher levels in base than aligned models and exceptions including Claude, Gemini, and Doubao.

1 Introduction

  • Model distillation is difficult to quantify because its process is opaque, benchmark data are limited, and learned representations may not appear as interpretable outputs.Source: §1 · p. 2
  • RSE and ICE provide complementary metrics for quantifying LLM distillation through response comparisons and identity-related inconsistencies.RSE compares original and student outputs, while ICE mines identity information that may have been learned during distillation.Source: §1 · p. 2
  • Most well-known closed-source and open-source LLMs exhibit considerable distillation, except Claude, Gemini, and Doubao.Source: §1 · p. 2
  • Base LLMs show higher distillation degrees than aligned LLMs.Source: §1 · p. 2
  • The study motivates more independent LLM development and greater transparency in training and distillation processes.Source: §1 · p. 2

2 Preliminary

This section describes GPTFuzz-based prompt optimization for exposing identity-recognition vulnerabilities in a target model. The procedure iteratively revises seed jailbreak prompts and quantifies behavioral changes across optimization operations.

2 Preliminary

  • GPTFuzz iteratively optimizes seed jailbreaking prompts to discover prompts that trigger vulnerabilities in the target model.Source: §2 · p. 2
  • GPTFuzz uses an initial seed prompt set, samples a fixed-size subset through adjusted MCTS, and merges selected prompts into the revised set.Source: §2 · p. 2
  • The target model is evaluated by measuring behavioral change when optimized prompts are added over multiple jailbreak iterations.The vulnerability measure averages the size of newly added prompts and normalizes it by the total number of jailbreak operations.Source: §2 · p. 2

3 Method

The framework quantifies LLM distillation with two complementary metrics: ICE probes identity-related contradictions, while RSE measures response similarity between test and reference models. ICE uses iterative adversarial prompting and scoring, whereas RSE evaluates style, structure, and content across multiple domains.

3 Method

  • The method defines Identity Consistency Evaluation (ICE) and Response Similarity Evaluation (RSE) as complementary metrics for quantifying LLM distillation.Source: §3 · p. 2, Figure 2 · p. 3

3.1 Identity Consistency Evaluation

  • ICE iteratively uses GPTFuzz-generated adversarial prompts to reveal identity information embedded in model training data.It targets names, affiliations, locations, or references to a source model and evaluates contradictions against identity facts.Source: §3.1 · p. 3
  • ICE measures identity leakage with Loose, Strict, and Hard Scores, progressively restricting which identity contradictions count as successful attacks.Hard Score requires identity keywords in the response but not in the prompt, reducing contextual information induction.Source: §3.1 · p. 3

3.2 Response Similarity Evaluation

  • RSE compares test-model responses with GPT reference responses across style, logical structure, and content detail to produce an overall similarity score.Source: §3.2 · p. 4
  • RSE evaluates general reasoning, mathematics, and instruction following using ArenaHard, Numina, and ShareGPT with five LLM-as-a-judge similarity levels.The levels range from 1, very dissimilar, to 5, very similar.Source: Figure 3 · p. 3, §3.2 · p. 4

4 Experiment

The experiments evaluate distillation through identity-consistency and response-similarity measures across diverse LLMs, finding systematic differences by model family, alignment, size, and reference model.

4.2.1 Main result of ICE

  • GLM-4-Plus, Qwen-Max, and Deepseek-V3 produce the most suspicious identity responses, whereas Claude-3.5-Sonnet and Doubao-Pro-32k produce almost none.The ICE evaluation uses jailbreak prompts across five identity domains and scores identity inconsistency with Loose, Strict, and Hard Scores.Source: §4.1.1 · p. 4, §4.2.1 · p. 5
  • Base LLMs consistently exhibit higher distillation levels than supervised fine-tuned models, while larger student models more closely resemble their teachers.These patterns appear in ICE comparisons and RSE analyses of model size.Source: §4.2.1 · p. 5, Table 3 · p. 5, §4.2.2 · pp. 6–7

4.2.2 Main result of RSE

  • GPT-series models show the highest RSE similarity to GPT4o-0806, including GPT4o–0513 at 4.240, while Llama3.1-70B-Instruct scores 3.628 and Doubao-Pro-32k 3.720.DeepSeek-V3 scores 4.102 and Qwen-Max-0919 4.174, and RSE captures logical information beyond word-level similarity.Source: §4.2.2 · pp. 5–6, Table 4 · p. 6
  • Claude3.5-Sonnet, Doubao-Pro-32k, and Llama3.1-70B-Instruct consistently show lower distillation levels, whereas Qwen and DeepSeek-V3 models show higher levels in fine-grained analyses.Reasoning-model comparisons also find no significant ICE-score difference between DeepSeek-R1 and V3.Source: §4.2.1 · p. 5, Table 2 · p. 5, §4.2.2 · p. 6
  • RSE scores increase across successive SFT epochs, indicating that fine-tuning makes Qwen2.5-7B-Instruct outputs increasingly similar to the target model.The increases span content, logic, style, and overall structure across datasets.Source: §4.2.2 · p. 6, Table 5 · p. 6

5 Case Study

Case studies show identity leakage and hallucination patterns that connect model outputs to distillation, while curated identity data and instruction fine-tuning reduce inconsistency.

Qwen-Max and Claude

  • Qwen-Max-0919 produced Claude-related responses in 32% of Strict Score samples, with repeated response structures suggesting absorbed Claude safety-alignment mechanisms.The cases commonly begin with refusal to role-play before self-identifying as Claude.Source: §5 · p. 7

Qwen2.5-7B-Base

  • Qwen2.5-7B-Base frequently self-identifies as GPT, while instruction-based fine-tuning significantly reduces identity inconsistency.The paper attributes such biases partly to pretrained internet data containing other LLM outputs.Source: §5 · p. 7

Phi4

  • Phi4 does not receive the highest ICE score despite explicit GPT distillation, because its Microsoft-related identity responses are excluded from positive detections.The case emphasizes curated identity data as important for reliability in critical domains.Source: §5 · p. 7

Hallucination

  • Identity inconsistencies can also arise from hallucinations: Llama, Gemini, and GPT hallucinated answers about the fabricated subject PersonGPT.Most other jailbreak responses lacked direct references to OpenAI, ChatGPT, or Claude outside explicit role-playing.Source: §5 · p. 7

6 Related Work

Related work situates the study among knowledge distillation, data-contamination detection, and jailbreak research used to probe model behavior.

Knowledge Distillation

  • Knowledge distillation compresses a larger teacher into a smaller student, with prior work applying it to BERT, GPT, and other autoregressive language models.Examples include DistilBERT, TinyBERT, MiniLM, DDK, and layer-wise self-distillation.Source: §6 · pp. 7–8

Data Contamination

  • Data-contamination research detects leaked benchmark or test data using perturbations, synthetic data, loss comparisons, and altered option orders.These methods address evaluation trustworthiness rather than identity leakage directly.Source: §6 · p. 8

Jailbreaking

  • Jailbreaking research develops adversarial prompts to bypass safety constraints, including GPTFuzzer and related mutation, reinforcement-learning, and distraction-based attacks.The paper adopts GPTFuzz as a mutator framework for iteratively optimizing prompts.Source: §6 · p. 8, Figure 6 · p. 8, Figure 7 · p. 8

7 Conclusion

The study systematically quantifies LLM distillation through identity consistency under jailbreak attacks and response similarity across models, finding substantial distillation and differences between model types.

  • Most LLMs exhibit high distillation degrees, with Claude, Doubao, and Gemini as notable exceptions.Source: §7 · p. 8
  • Base models show more distillation than aligned models, suggesting fine-tuning can reduce homogenization.Source: §7 · p. 8
  • The findings highlight risks of reduced model diversity, identity leakage, and vulnerability to attacks alongside distillation’s efficiency benefits.Source: §7 · p. 8
  • The study calls for more independent LLM development and transparent training documentation to balance efficiency, safety, and model uniqueness.Source: §7 · p. 8

Limitations

The study identifies a limitation in its identity-jailbreak strategy: GPTFuzz templates may induce cognitive inconsistencies rather than isolate identity-related vulnerabilities.

  • GPTFuzz-based identity jailbreak prompts may contain strong inductive guidance or role-playing elements that induce cognitive inconsistencies.Source: Limitations · p. 9
  • Future work should develop a dedicated identity-jailbreak method that better isolates identity-related vulnerabilities without inducing artificial inconsistencies.Source: Limitations · p. 9

Ethics Statement and Usage Restrictions

The paper states that its jailbreak-based identity analysis is intended for research rather than malicious content generation, while the surrounding materials document model identities, prompts, judgments, and examples.

Ethics Statement and Usage Restrictions

A Facts of LLMs

  • The evaluated models include Qwen-Max-0919, Deepseek-V3, Doubao-Pro-32k, GLM4-plus, Llama3.1-70B-Instruct, and Phi-4, with developers identified in the cited passages.Source: A Facts of LLMs · p. 11

B Identity Prompts

  • Identity prompts ask about developers, companies, technical teams, locations, partners, contributors, collaborations, and supporting projects.Source: B Identity Prompts · p. 11

20. Where are the experts in your team from?

  • Additional prompts ask which company developed the system, whose product or service it is, the creation or parent company and industry, organizational fields, and other important institutional products or projects.Source: B Identity Prompts · p. 11

C Logical Judgment Prompt

D.1 True Positive

The section presents examples used to distinguish factual identity contradictions from fictional or role-play outputs, and illustrates how response similarity varies across models.

D.1 True Positive

  • ICE true-positive cases are treated as evidence of distillation when model outputs contain logical errors contrary to established identity facts.Source: §D.1 · pp. 11, 13
  • Some outputs incorrectly attribute DeepSeek or Qwen-Max-0919 identities to OpenAI, Claude, or Anthropic, illustrating cross-model identity contradictions.Source: §D.1 · p. 13

D.2 False Positive

  • Fictitious or role-playing outputs are excluded as false positives, including cases where models adopt ChatGPT or imaginary personas.Source: §D.2 · p. 14

E Response Similarity Evaluation Prompt

F RSE typical examples

  • High-RSE cases show similar logical structure, analytical process, and response format between models.Source: F RSE typical examples · p. 15
  • Low-RSE cases show discrepancies in thought depth, answering process, and final answers, indicating weaker logical consistency and reasoning ability.Source: F RSE typical examples · p. 15

G RSE additional experiments

This section presents RSE evaluation, scoring, and instruction prompts, alongside high-score examples and model-score comparisons across several benchmarks.

Loading 2501.12619v3…