PaperlayerView exact revision on arXiv

Source-linked AI summary

Do System Prompts Leave Behavioral Fingerprints? A Large-Scale Empirical Study of Clone Detection via Output Similarity

Linghan Chen, Yudong Gao, Jiyao Wang, Kaiyan Ji, Honglong Chen

arXiv:2608.24461v1cs.CR

TL;DR

System prompts may leave detectable traces in model outputs, but the feasibility and robustness of clone verification remain open. This paper introduces Black-Box Behavioral Fingerprinting and finds AUC 0.876 same-model and 0.725 mean cross-model detection.

  • Problem

    Whether system prompts leave usable traces in outputs, enabling black-box verification of suspected clones, remains an open empirical question.

  • Method

    Black-Box Behavioral Fingerprinting registers output-based behavioral signatures and compares suspect deployments against unrelated baselines using black-box API access.

  • Results

    BBF detects cloned system prompts across model settings, reaching AUC 0.876 same-model and 0.725 mean cross-model.

  • Takeaways & Limitations

    Within its supported scope, BBF provides a verification protocol requiring only black-box API access, no platform cooperation, and 25 diagnostic queries.

  • Takeaways & Limitations

    Adaptive output-style attacks can collapse detection on short structured tasks, and generalization beyond four text-based benchmarks and four mid-tier APIs is not established.

Abstract

from arXiv · show

System prompts can be extracted from commercial LLMs with over 80\% success and redeployed at zero cost, yet a prompt owner has no way to verify whether a suspected deployment is a clone. We propose Black-Box Behavioral Fingerprinting (BBF): the prompt owner registers a behavioral signature from model outputs and later tests whether a suspect deployment matches that signature more closely than an unrelated baseline. BBF requires only black-box API access. Through a large-scale study (4 model families, 8 benchmarks, 288{,}000 responses), we find that prompt choice explains 24.4\% of output variance and same-model detection reaches AUC 0.876. Cross-model performance is bounded by detector identity, with off-diagonal AUC ranging from 0.845 (Claude as detector) down to 0.665 (Qwen) and overall mean 0.725. BBF resists non-adaptive prompt paraphrasing (AUC $\geq 0.889$) and is robust to imperfect extraction, but a single-sentence formal-tone prefix can collapse detection on short structured outputs (MNLI 0.978 $\to$ 0.547), isolating style-invariant detection as the key open problem. Diagnostic Query Optimization, a zero-cost query selection rule, adds $+0.120$ to cross-model AUC.

1 Introduction · 2 Related Work

System prompts can encode distinctive application behavior that remains detectable after extraction and redeployment. The paper introduces Black-Box Behavioral Fingerprinting (BBF) and evaluates its effectiveness, limits, and relationship to prior extraction, watermarking, model-fingerprinting, and embedding methods.

  • 1 Introduction: System prompts encode domain expertise, reasoning strategies, output conventions, tool-use workflows, and safety guardrails in specialized LLM applications.The study focuses on the single-turn case and treats prompts as a key differentiator between commercially deployed products.
  • 1 Introduction: Over 80% extraction success enables verbatim system-prompt redeployment across models and platforms, creating a need for post-extraction clone verification.Unlike model weights, prompts can be extracted through adversarial queries and redeployed at low replication cost.
  • 1 Introduction: BBF registers a behavioral signature from original outputs and tests whether a suspect deployment is closer to that signature than to an unrelated baseline using only black-box API access.The method is framed as a verification tool for a specific suspected deployment, complementing IP enforcement rather than enabling surveillance.
  • 1 Introduction: 0.876 same-model AUC and 0.845-to-0.665 cross-model AUC quantify BBF performance across 128 scenarios, 4 model families, 8 benchmarks, and 288,000 responses.Claude as detector corresponds to 0.845, while Qwen as detector corresponds to 0.665; 11 of 128 scenarios fail under three identified patterns.
  • 2 Related Work: System-prompt extraction research emphasizes prevention through adversarial-query defenses, indirect-injection defenses, automated-attack defenses, proxy architectures, and filtering strategies.BBF instead targets the post-extraction detection problem.
  • 2 Related Work: Watermarking modifies generation and requires provider cooperation, whereas model fingerprinting typically identifies the producing model using weight access or fine-tuning.The paper positions prompt-level behavioral detection as distinct from these approaches.
  • 2 Related Work: Sentence embeddings provide BBF’s measurement tool, while embedding similarity offers a closed-form reproducible rule instead of an LLM-as-judge baseline whose cost scales with K · N.The paper notes that LLM-as-judge decisions are prompt-conditioned and non-deterministic, making direct comparison a useful next experiment.

3 Method

BBF detects prompt clones through black-box behavioral fingerprints: sentence-embedding centroids compare a suspect deployment with the original and an unrelated same-domain baseline. The method supports stochastic outputs, majority-vote aggregation, and prompt-free Diagnostic Query Optimization for selecting informative queries.

  • Behavioral fingerprinting: BBF models each prompt as a query-conditional output distribution and tests functional equivalence by comparing suspect and original centroids against an unrelated same-domain baseline.Functional clones are semantic paraphrases manually verified to preserve task instructions.
  • Threat model: The defender needs only public API access, at least 25 diagnostic queries, and one unrelated same-domain prompt, without model identity, weights, logits, or suspect cooperation.The adversary may use another model family, paraphrase the prompt, or deploy an imperfect extraction.
  • Registration and detection: BBF uses off-the-shelf sentence embeddings and cosine similarity, registering one centroid per diagnostic query; K = 25 queries with N = 3 samples per query suffice in the experiments.Averaging over N samples reduces stochastic-decoding variance.
  • Aggregation and guarantees: Majority voting over T queries yields exponentially increasing confidence when per-query detection probability exceeds 0.5.For p = 0.786 and T = 25, the bound gives Pdetect ≥1 −e−4.08 > 0.983, while exact binomial calculation yields Pdetect > 0.999; correlated or failure-mode queries require caution.
  • Diagnostic Query Optimization: Diagnostic Query Optimization ranks candidate queries by lowest baseline similarity, providing a prompt-free, transferable one-shot heuristic; held-out validation lifts AUC from 0.878 to 0.980.The optimality condition is that expected detection gap decreases monotonically with baseline similarity.

4 Experimental Setup

The evaluation spans four commercial model families and eight benchmarks, using controlled original, clone, and unrelated prompts across 128 cross-model scenarios. Each scenario compares clone similarity against a same-domain unrelated baseline using standardized sampling and embedding procedures.

  • Models: Four model families—Claude, GPT, DeepSeek, and Qwen—represent widely deployed commercial APIs from distinct providers, including closed-source and open-weight models.The evaluated models are claude-haiku-4-5, gpt-4o-mini, DeepSeek-V3, and qwen-plus.
  • Benchmarks: 8 benchmarks span 7 domains, with each benchmark defining original, semantically paraphrased clone, and unrelated prompts.Diagnostic queries come directly from standard NLP datasets without adversarial crafting or domain-specific engineering.
  • Benchmarks: 128 total scenarios arise from the 4 × 4 cross-model matrix across eight benchmarks.Each benchmark contributes 16 model-pair scenarios.
  • Sampling and evaluation: 288,000 responses are generated using N = 3 samples per query, temperature 0.7, and a maximum of 400 output tokens.Each scenario produces 250 × 3 × 3 = 2,250 responses across 250 queries, three prompt conditions, and three samples.
  • Embedding model: All embeddings use all-MiniLM-L6-v2, while three-architecture ablation shows a mean AUC difference below 0.02.The embedding model has 384 dimensions and 22M parameters; compared architectures are MiniLM-L6, BGE-large, and GTE-large.
  • Unrelated prompt design: The unrelated baseline shares the original prompt’s broad domain but specifies a different task, making detection more conservative than cross-domain comparison.For each query, clone similarity is compared with the unrelated baseline and reported as AUC over all queries.

5 The Behavioral Imprint of System Prompts

System prompt choice measurably shapes model outputs, supporting output similarity as a potential detection signal. Across benchmarks, prompt identity explains 24.4% of within-benchmark output variance, though the effect varies substantially by domain and does not alone establish clone detection.

  • Measurement: Prompt identity was evaluated with one-way ANOVA, using output-embedding similarity as the dependent variable and computing η2 within each benchmark.The factor levels were original, clone, and unrelated prompts.
  • Overall effect: η2 = 0.244 means prompt choice explains 24.4% of within-benchmark output variance on average.This is roughly 1.7× Cohen’s large-effect threshold of η2 > 0.14.
  • Domain variation: η2 = 0.429 on GSM8K shows the strongest prompt influence, while η2 = 0.017 on MMLU shows a small effect.The passages associate GSM8K’s stronger effect with diverse reasoning styles and MMLU’s weaker effect with canonically answered questions.
  • Detection implication: Prompt influence is necessary but insufficient for detection, because the next empirical question is whether it separates clone outputs from unrelated outputs.The cited analysis establishes output variance effects, not clone-versus-unrelated separation.

6 Main Results

BBF detects prompt clones reliably within the same model family and transfers across models with performance determined chiefly by detector identity. Its failures concentrate in short outputs, dataset artifacts, and pair-specific style mismatches.

  • Cross-model transfer: 0.725 overall cross-model mean AUC varies from 0.845 with Claude as detector to 0.665 with Qwen, while detector identity explains R2 = 0.161 of AUC variance.Detector identity contributes more variance than registration choice, which contributes R2 = 0.050.
  • Failure modes: SST-2 has 3 failures, including Claude→GPT at 0.239 and Claude→DeepSeek at 0.264, with output above 50 tokens necessary for reliable cross-family transfer.Sentiment outputs are often under 50 tokens, and Claude’s concise formats create directional transfer failures; GPT and DeepSeek fingerprints transfer effectively.
  • Failure modes: CUAD contributes 5 failures, including the sole same-model failure at GG = 0.448, because a loader fallback changed contract-analysis inputs into reading-comprehension passages.The resulting output collapse suppresses the prompt-conditional signal, making these data artifacts rather than method failures.
  • Failure modes: CNN/DM has 3 pair-specific style-mismatch failures—DeepSeek→Qwen at 0.500, Qwen→GPT at 0.449, and Qwen→DeepSeek at 0.474—despite transfer to Claude exceeding 0.85.The same pair-specific pattern appears in MMLU, where DeepSeek→Qwen is 0.500.

7 Ablation Studies

The ablations show that diagnostic query selection and a modest query budget improve detection, while embedding backends are consistent and lexical baselines mainly fail across models. Query content explains variation in detection gaps, with low-baseline queries producing the largest gains.

  • Query Selection and Budget: DQO improves AUC in all 9 validated settings, with larger gains in cross-model scenarios, while K = 25 suffices for reliable detection.Low-baseline queries yield mean +0.199 gaps versus +0.011 for high-baseline queries.
  • Embedding Models and Lexical Baselines: Mean AUC difference is below 0.02 across MiniLM-L6, BGE-large, and GTE-large, while TF-IDF falls behind BBF on cross-model MNLI.For MNLI Claude→GPT, BBF reaches 0.893 versus TF-IDF at 0.772; lexical baselines remain competitive on same-model pairs.

8 Robustness

BBF remains robust to non-adaptive paraphrasing and imperfect prompt extraction, but targeted style-shifting can sharply degrade detection on short structured outputs while leaving content-rich outputs largely unaffected.

  • 8.1 Paraphrase attacks: AUC values remain ≥0.889 under complete functional rewrites, while light paraphrases raise MMLU detection from 0.862 to 0.966.Rewrites were evaluated across three intensity levels on Claude→Claude across six benchmarks.
  • 8.1 Paraphrase attacks: Non-adaptive paraphrasing fails to evade BBF because its embedding signature tracks behavior rather than wording.The stronger threat is an output-style prefix that overrides the prompt’s stylistic surface.
  • 8.2 Adaptive prefix injection: Formal-tone injection reduces MNLI detection from 0.978 to 0.547 but leaves MMLU at 0.863.The asymmetry indicates that targeted style-shifting mainly affects short, structured outputs rather than content-rich outputs.
  • 8.3 Practical deployment conditions: AUC remains ≥0.879 across all five fidelity levels for author-constructed clones, while simulated clones improve from 0.886 to 0.982.Noisier extracted clones are at least as detectable as carefully handcrafted paraphrases, plausibly because they preserve functional framing and surface phrasing.

9 Conclusion · Limitations · Ethics Statement

BBF offers black-box verification evidence for cloned system prompts, achieving strong same-model and moderate cross-model detection within a fixed 25-query budget. Its main limitations are adaptive style attacks, jurisdiction-dependent legal interpretation, and the need for responsible deployment and further evaluation.

  • 9 Conclusion: 0.876 same-model AUC and 0.725 mean cross-model AUC show BBF detects cloned system prompts from black-box outputs.Detector identity dominates transfer performance, ranging from 0.845 for Claude to 0.665 for Qwen.
  • Limitations: Style-invariant detection on short, structured outputs is the natural next direction because output-style adaptation can defeat BBF in that setting.Unexplored attacks include sandwich constructions, suffix steering, output-format steering, and multi-turn dilution.
  • 9 Conclusion: 25 diagnostic queries provide a fixed-budget verification protocol requiring only black-box API access and no platform cooperation.The protocol gives prompt owners a defensible way to test suspected cloning.
  • Limitations: 0.978 to 0.547 MNLI detection collapse under a single-sentence formal-tone prefix exposes adaptive style attacks as the strongest open limitation.The evaluation covers only four prefix-injection strategies on two benchmarks.
  • Ethics Statement: Statistical evidence of prompt cloning does not establish legal conclusions because the legal status of system prompt cloning varies by jurisdiction.BBF should therefore not be treated as a substitute for legal determination.
  • Ethics Statement: BBF should be used only with reasonable prior suspicion of cloning, not for speculative surveillance.This recommendation limits use of diagnostic querying in ethically sensitive investigations.
  • Ethics Statement: Diagnostic queries come from established public NLP benchmarks, and the code and data will be released with the camera-ready version.The release plan supports reproducibility using publicly grounded evaluation materials.

A Algorithm and Proofs · A.1 Algorithm

Algorithm 1 defines Black-Box Behavioral Fingerprinting as a two-phase procedure for deciding whether a deployment is a clone. It takes an original prompt, an unrelated prompt, candidate queries, a budget, samples, and an encoder, then returns CLONE or NOT CLONE.

  • A.1 Algorithm: Algorithm 1 is titled Black-Box Behavioral Fingerprinting.
  • A.1 Algorithm: The procedure requires an original prompt P and an unrelated prompt Pu.
  • A.1 Algorithm: The procedure also requires candidate queries Qcand, budget T, samples N, and encoder ϕ.
  • A.1 Algorithm: The algorithm ensures a binary decision: CLONE or NOT CLONE.
  • A.1 Algorithm: Phase 1 performs registration and query optimization.
  • A.1 Algorithm: Phase 2 performs detection.
  • A.1 Algorithm: The detection phase uses k from Phase 1.
  • A.1 Algorithm: The algorithm applies the decision condition ˆr > 0.5.

A.2 Proofs … B.4 Cross-Model AUC Heatmaps

The supplementary results establish probabilistic detection and query-selection guarantees, while reporting same-model performance, embedding and lexical-baseline comparisons, cross-model heatmap behavior, and extraction-quality findings.

  • A.2 Proofs: Proposition 1 bounds majority-vote failure by exp(−2T(p −1/2)2) when positive-gap queries follow Binomial(T, p).The proof uses E[S] = Tp and Hoeffding’s inequality.
  • A.2 Proofs: Proposition 2 shows that, under monotonicity, ordering queries by ascending sunrel_k equals descending E[δk | H1].Selecting the top-T queries by expected gap maximizes the objective via a greedy argument for modular functions.
  • B.1 Same-Model Detection: Full Breakdown: 0.876 is the overall same-model mean AUC, with CUAD GG the only same-model failure at 0.448.CC, GG, DD, and QQ denote Claude→Claude, GPT→GPT, DeepSeek→DeepSeek, and Qwen→Qwen.
  • B.2 Embedding Model and Lexical Baseline Ablation: MiniLM-L6 is compared against BGE-large and GTE-large embeddings, both 1024d, plus TF-IDF and bigram Jaccard baselines.The passage describes this as an embedding-model and lexical-baseline ablation.
  • B.4 Cross-Model AUC Heatmaps: Figure 5 presents 4 × 4 cross-model AUC matrices for each benchmark as heatmaps.Red-bordered cells identify detection failures where AUC drops to 0.50 or below.
  • B Supplementary Tables and Figures: Embedding models differ by < 0.02 mean AUC, while TF-IDF remains competitive on same-model CC but degrades on cross-model CG pairs.This comparison is reported in the similarity-measure table.
  • B Supplementary Tables and Figures: LLM-simulated extracted clones are at least as detectable as handcrafted paraphrases in Claude→Claude tests.Noisier extractions are slightly more detectable, consistent with the hypothesis discussed in Section 8.3.

C Enterprise Skill Domain Validation · D System Prompts and Full Per-Scenario Results · D.1 Example Prompt Triplet (MedQA)

The paper validates BBF across enterprise skill domains and documents its full scenario design, including a concrete MedQA triplet contrasting protected, paraphrased, and unrelated prompts. The supplementary materials provide complete prompt texts, per-query similarities, gap distributions, and AUCs for all 128 scenarios.

  • C Enterprise Skill Domain Validation: Across 16 enterprise scenarios, the mean controlled gap is +0.194, ranging from +0.034 to +0.646.The evaluation covers legal contract analysis, equity valuation, clinical differential diagnosis, and software architecture review, using five boundary queries per domain and three samples per query.
  • C Enterprise Skill Domain Validation: The enterprise evaluation uses the same 2 × 2 Claude/GPT same- versus cross-model design across all four domains.Each domain uses hand-designed boundary queries intended to maximize prompt sensitivity.
  • C Enterprise Skill Domain Validation: Cross-model failures concentrate in CUAD, SST-2, and CNN/DM, primarily for short-output or Claude-, GPT-, and DeepSeek-involving configurations.Figure 5 marks failures at AUC ≤0.50 and reports counts of 5 in CUAD, 3 in SST-2, and 3 in CNN/DM.
  • D System Prompts and Full Per-Scenario Results: Each benchmark uses an original protected prompt, a semantically paraphrased clone, and an unrelated prompt sharing the broad domain but performing a different task.Full prompt texts, per-query similarity scores, gap distributions, and 128 scenario AUCs are provided in the supplementary materials and Table 9.
  • D.1 Example Prompt Triplet (MedQA): The MedQA original and clone target board-style clinical reasoning while differing in surface phrasing and structure.Both prompts require clinical analysis, ranked differentials, pathophysiological reasoning, systematic option elimination, and structured answers.
  • D.1 Example Prompt Triplet (MedQA): The MedQA unrelated prompt remains medical but specifies curriculum design rather than clinical question solving.Its tasks include case-based modules, standardized-patient scripts, OSCE checklists, competency mapping, and simulation scenarios.
  • D System Prompts and Full Per-Scenario Results: Table 9 enumerates all 128 scenarios using C, G, D, and Q abbreviations for Claude, GPT, DeepSeek, and Qwen.It records 11 failures with AUC ≤0.50, distributed across CUAD, SST-2, and CNN/DM.
Loading 2608.24461v1…