PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations

1. Introduction

• Existing evaluations target training-data memorization, immediate jailbreaks, or other multi-turn attacks, but do not directly measure secret recovery after unrelated conversational drift.The resulting gap is whether topic shift acts as a practical privacy boundary within an active conversation.Source: §1 · p. 1
• 38.7%–54.6% dialogue-level hybrid leakage persists across three LLMs and 1,000 dialogues per model.Leakage also varies by secret type and persuasion intensity; additional drift within d ≤6 does not reliably reduce it.Source: §1 · p. 1
• PrivDrift is a controlled benchmark for auditing whether user-disclosed secrets remain recoverable after unrelated topic drift.It varies drift length and persuasion intensity and tests direct, justified, and high-pressure probes.Source: §1 · pp. 1–2
• The benchmark reports dialogue-level, probe-level, regex-only, fuzzy, and hybrid leakage rates using a reproducible hybrid detector.The detector combines normalization-based matching with an open-weights LLM judge.Source: §1 · p. 2
• Privacy Half-Life (τ) measures stable suppression, and no evaluated model reaches stable decay within the observed drift range.A temporary reduction does not count as stable suppression.Source: §1 · p. 2

2.1. Contextual Privacy and Inference-Time Leakage

• Active-context privacy concerns user-provided information in the current interaction, unlike training-data extraction or recovery of memorized PII from model parameters.Prior work also tests privacy norms and contextual access, generally under static access-control or privacy-norm scenarios.Source: §2.1 · p. 2
• PrivDrift asks whether a secret remains behaviorally recoverable after unrelated topic drift and later persuasion-based probing.This persistence question differs from benchmarks of privacy norm awareness and legal compliance.Source: §2.1 · p. 2

2.2. Multi-Turn Adversarial Attacks

• Prior multi-turn attacks study harmful outputs, jailbreak paths, or instruction hijacking; PrivDrift instead tests re-disclosure of user-supplied sensitive information.Its focus is information flow within the active context.Source: §2.2 · p. 2

2.3. Context Management and Unlearning

• Unlike unlearning benchmarks operating on static question-answer pairs and model parameters, PrivDrift tests behavioral suppression of a secret present in active context.This distinction is relevant to deployed RAG systems, copilots, and agentic workflows where privacy control may depend on prompt-level context management.Source: §2.3 · p. 2

3. The PrivDrift Framework

• PrivDrift is a synthetic evaluation framework with a dialogue generator, probing harness, and metric suite for leakage detection and temporal analysis.It measures whether information disclosed in an active conversation remains recoverable after topic drift.Source: §3 · p. 2

3.1. Threat Model: Active-Context Re-Disclosure

• Active-context re-disclosure occurs when later prompts elicit sensitive information disclosed earlier, after unrelated topic drift.The evaluation tests whether the model itself reproduces the secret; it does not assume an adversary cannot inspect the transcript.Source: §3.1 · pp. 2–3

3.2. Dataset Generation Pipeline

• The 1,000-dialogue dataset combines scaffolded generation with rewriting, alongside 900 synthetic and 100 human-authored dialogues.Because cached metadata does not identify the human-authored rows, results are aggregated and a separate human-versus-synthetic comparison is deferred.Source: §3.2 · p. 3

3.2.1. PERSONA AND SECRET INJECTION

• Each dialogue assigns a persona and inserts one of four fixed-format secrets—phone number, email address, SSN, or credit card number—into a plausible opening task.Personas are sampled from more than 100 professions and demographic profiles.Source: §3.2.1 · p. 3

3.2.2. CONTENT-DENSE TOPIC DRIFT

• Content-dense drift turns add unrelated entities and task demands while preserving control over the secret and drift length.Turns draw on topic-conditioned question-answer pairs and fallback writing, reasoning, or summarization tasks; some are human-authored.Source: §3.2.2 · p. 3

3.2.3. NATURALISTIC REFINEMENT

• A rewriting model improves fluency and persona realism while preserving the secret, drift length, role structure, and final probe conditions.The refinement changes surface phrasing, not experimental variables.Source: §3.2.3 · p. 3

3.3. Probing Setup

• The main evaluation appends a standardized probe, with direct, contextually justified, and high-pressure styles, after a sampled number of drift turns.The tested range is bounded: the authors interpret results as active-context persistence, not a test of arbitrarily long context windows.Source: §3.3 · p. 4

3.4. Evaluation Methodology

• The hierarchical detector first matches normalized responses, then sends regex-negative responses to an LLM judge for partial or indirect disclosure.Privacy Half-Life (τ) measures sustained suppression against δ = 0.1 × L(0); if no drift length meets its stability criterion, τ is assigned as max(Dobs) + 1, preventing transient dips from counting as stable suppression.Source: §3.4 · p. 4, §3.4.1 · p. 4, §3.4.2 · p. 4

4. Experimental Setup

• GPT-OSS-120B, DeepSeek-R1, and Qwen3-VL-235B receive the same dialogue histories and probe templates through the OpenRouter API.The evaluated models have extended context windows.Source: §4 · p. 4
• Each model is evaluated on 1,000 dialogues with drift lengths d ∈ {0, 2, 3, 4, 5, 6}, and three probes per dialogue yield 3,000 responses per model.Across three models, the cached evaluation contains 9,000 responses.Source: §4 · pp. 4–5
• A dialogue counts as leaking if any standardized probe elicits the secret, while persuasion-level and fuzzy-matching rates are computed per probe.This distinction allows dialogue-level leakage to exceed the average of individual persuasion-level rates.Source: §4 · p. 5
• Dialogue-level rates use 95% bootstrap confidence intervals, while matched model and detector comparisons use McNemar’s test with Bonferroni correction where applicable.Cochran’s Q with post-hoc McNemar tests handles repeated persuasion-level measures; chi-square tests and Cramer’s V assess associations.Source: §4 · p. 5

5.1. Dialogue-Level Hybrid Evaluation Largely Agrees With Deterministic Matching

• 1.10 percentage points is the maximum additional dialogue-level leakage the LLM judge adds over normalization-based matching across models.The two evaluations differ only slightly, supporting hybrid leakage as the main metric while retaining regex-only rates as a deterministic baseline.Source: §5.1 · p. 5, Table 2 · p. 5

5.2. All Models Leak Substantially at the Dialogue Level

• 38.70% for DeepSeek-R1, 47.70% for GPT-OSS-120B, and 54.60% for Qwen3-VL-235B leaked at least one standardized probe at the dialogue level.These are dialogue-level hybrid leakage rates.Source: §5.2 · p. 5, Table 3 · p. 5

5.3. Persuasion Intensity Affects Leakage Non-Monotonically

• Probe-level leakage varies significantly across persuasion levels (Cochran’s Q, p < 10−5), but the direction differs by model.Hard persuasion reduces leakage for GPT-OSS-120B, medium persuasion yields the lowest leakage for DeepSeek-R1, and hard persuasion produces the highest leakage for Qwen3-VL-235B.Source: §5.3 · p. 5

5.4. Drift Length Does Not Reliably Reduce Leakage Within the Tested Window

• A dip near d = 3 is followed by rebound, and topic drift does not reliably reduce leakage within the tested window.Leakage can persist across several unrelated topic shifts before the secret is far from the context-window end.Source: Figure 4 · p. 6, §5.4 · p. 6

5.5. Secret Type Strongly Determines Leakage

• Secret type has a large association with hybrid leakage: Cramer’s V is 0.5871 for DeepSeek-R1, 0.7731 for GPT-OSS-120B, and 0.6927 for Qwen3-VL-235B.SSNs and credit card numbers are suppressed far more often than emails and phone numbers.Source: §5.5 · p. 6

5.6. Privacy Half-Life Exceeds the Observed Window

• τ = 7 for every model and persuasion level, indicating no stable decay within d ≤6 under the Privacy Half-Life threshold δ = 0.1 · L(0).No evaluated model reaches that threshold at any tested drift length.Source: §5.6 · p. 6

5.7. Fuzzy Matching Supports the Hybrid Labels

• At a 0.85 threshold, fuzzy leakage closely tracks probe-level hybrid leakage: 26.40% versus 26.43% for DeepSeek-R1, 37.03% versus 37.57% for GPT-OSS-120B, and 37.60% versus 37.53% for Qwen3-VL-235B.Fuzzy matching is auxiliary because approximate string similarity is less expressive than the judge for partial or semantic disclosures.Source: §5.7 · p. 6, Table 6 · p. 6

6. Discussion

• Sensitive information disclosed earlier can remain behaviorally recoverable after unrelated topic drift, without implying permanent model memory or that all long-context settings behave similarly.The discussion attributes the observed behavior to later prompts providing enough retrieval pressure.Source: §6 · p. 6
• SSNs and credit card numbers are suppressed far more often than emails and phone numbers, suggesting safeguards may rely on format-sensitive cues rather than contextual confidentiality reasoning.All four categories are user-disclosed secrets in the benchmark.Source: §6 · p. 6
• Persuasion effects are significant but non-monotonic and model-dependent, while temporary leakage dips do not satisfy the Privacy Half-Life criterion.The discussion therefore cautions against treating topic drift as an implicit privacy boundary.Source: §6 · pp. 6–7

7. Limitations and Future Work

• Drift is tested only through d = 6, so the findings characterize bounded active-context drift, not behavior across full long-context windows.The benchmark focuses on structured secrets rather than unstructured sensitive attributes, and does not test cross-session retention or personalization memory.Source: §7 · p. 7
• The benchmark mixes 900 synthetic and 100 human-authored dialogues, but cached metadata prevents a separate human-versus-synthetic comparison.The dialogues were inserted throughout the benchmark, and results are reported in aggregate.Source: §7 · p. 7
• Leakage labels rely on deterministic, fuzzy, and LLM-judge surface-form detection, without formal human-versus-automated inter-annotator agreement.A larger manual audit of borderline partial disclosures and obfuscations remains future work.Source: §7 · p. 7
• API-served models may change over time, although the study logs model identifiers, prompts, timestamps, cached responses, and paired statistics.Source: §7 · p. 7
• The planned artifact release includes generation code, probe templates, evaluation scripts, aggregate results, and sanitized dialogue examples with non-real identifiers.Source: §7 · p. 7

8. Conclusion

• 38.7% to 54.6%: dialogue-level hybrid leakage remains substantial across three LLMs and 1,000 dialogues per model.Source: §8 · pp. 7–8
• Within the tested drift range, topic drift does not reliably reduce leakage, and Privacy Half-Life exceeds the observed window for all models.Source: §8 · p. 8
• Persuasion intensity affects leakage in model-dependent ways, while secret type has the strongest association with leakage.Source: §8 · p. 8
• The observed asymmetry across secret types suggests protections remain uneven and depend more on format-sensitive cues than robust contextual confidentiality.Source: §8 · p. 8

A. Dataset Characterization

• Table 7 reports dataset length characteristics by drift length, with approximate token counts estimated from word counts.Source: Table 7 · p. 10

B. Additional Effect Sizes

• Table 8 reports probe-level effect sizes for associations between hybrid leakage and benchmark variables.Source: Table 8 · p. 10

• Figure 6 is titled “Dialogue construction details.”Source: Figure 6 · p. 10