PaperlayerView exact revision on arXiv

Source-linked AI summary

P2Skill: Privacy Preserving Skill Distillation for Cloud-Local LLM Inference Systems

Myunghoon Ryu, Geunpyo Park, Sungjoon Lee, XinYu Piao, Jong-Kook Kim

arXiv:2608.14094v1cs.CRcs.AI

TL;DR

Cloud-local LLM inference must protect personally identifiable information in prompts sent to remote servers, while existing privacy modules require added training or adaptation. P2Skill distills four prompt-based privacy skills for a frozen local model and achieves 1.69× and 3.66× higher average privacy-quality scores than previous baselines on a four-domain benchmark.

  • Problem

    Cloud-local LLM inference must prevent personally identifiable information in user prompts from reaching remote servers without relying on separately trained, deployment-adapted privacy modules.

  • Method

    P2Skill uses a frozen local small language model to execute four cloud-refined prompt skills for decomposition, PII-aware routing, paraphrasing, and reconstruction.

  • Results

    1.69× and 3.66× higher average privacy-quality scores than previous baselines were achieved on a four-domain benchmark.

  • Takeaways & Limitations

    The benchmark supports prompt-based skills as a way to improve privacy-quality outcomes in cloud-local inference without updating local model weights or training auxiliary privacy detectors.

  • Takeaways & Limitations

    P2Skill does not fully eliminate residual leakage, and its inference quality remains below the cloud model’s performance.

Abstract

from arXiv · show

Cloud-local LLM inference systems have the potential to use the reasoning capability of large cloud models while protecting sensitive user data on personal devices. Cloud-bound requests must exclude personally identifiable information (PII) to prevent external data leakage. Existing privacy-preserving methods rely on prompt perturbation, entity masking, or model fine-tuning, but these approaches may distort contextual semantics or require additional training. This paper proposes P2Skill, a prompt-based skill distillation method in which a local small language model (SLM) autonomously performs decomposition, PII-aware routing, paraphrasing, and reconstruction by following the skill prompts. Skills are iteratively refined from execution failures by a cloud LLM, enabling the local SLM to generalize beyond memorized PII patterns, and therefore P2Skill requires no privacy-specific fine-tuning or learned auxiliary detectors. Evaluation on a four-domain benchmark shows that P2Skill achieves $1.69\times$ and $3.66\times$ higher privacy-preserved inference quality than previous baselines.

1 Introduction

Cloud-local LLM systems address privacy-sensitive workflows by combining on-device small models with cloud reasoning, but existing privacy modules require deployment-specific adaptation and auxiliary restoration mechanisms. P2Skill instead distills reusable privacy-preserving skills into a local SLM for decomposition, PII-aware routing, paraphrasing, and reconstruction.

  • Motivation: LLMs increasingly support privacy-sensitive workflows in personal agents, workplace automation, and decision support systems, motivating cloud-based inference deployments.The passage identifies code synthesis, logical reasoning, and natural language understanding as areas of recent LLM improvement.
  • Cloud-local inference: Cloud-local systems partition workloads between an on-device small model and a cloud LLM, routing simple or privacy-sensitive subtasks locally and harder reasoning remotely.This deployment paradigm aims to combine local privacy handling with cloud-model reasoning capability.
  • Limitations: Existing cloud-local privacy modules require separate training and per-deployment adaptation, while semantic reconstruction needs an additional trained restoration module or local auxiliary mechanism.These requirements add components beyond the core cloud-local inference system.
  • P2Skill: P2Skill distills privacy-preserving skills into a local SLM using reusable natural-language prompts for decomposition, PII-aware routing, paraphrasing, and reconstruction.The method is introduced to minimize privacy leakage while maintaining inference quality.

2 Related Work

Prior work addresses identifier memorization or privacy transformations but leaves cloud-bound prompt transformation and validation unresolved, often trading privacy for quality or relying on trained components. P2Skill instead uses prompt-based skills on a local SLM, detecting PII without learned detectors or recalibration as domains and PII distributions change.

  • Earlier methods show that language models can memorize and reconstruct training-data identifiers but do not transform or validate cloud-bound prompts.
  • Representation-level LDP methods incur a privacy-quality trade-off because injected noise can degrade downstream inference quality.Split-and-Denoise adds LDP noise at the representation level and denoises on the client.
  • P2Skill operates entirely as prompt-based skills on the local SLM, avoiding noise-induced trade-offs and semantic distortion from auxiliary trained or generative components.
  • P2Skill detects PII through a prompt-based skill rather than a learned detector or calibrated LDP budget, requiring no retraining or recalibration for new domains or PII distributions.

3 Proposed Method

P2Skill uses four prompt-based skills on a local SLM to decompose prompts, route sub-tasks by privacy and solvability, paraphrase cloud-bound PII, and reconstruct responses. A cloud LLM iteratively refines selected skills from end-to-end failures while using aggregate statistics and memorization checks rather than updating the SLM’s weights.

  • Four-stage pipeline: P2Skill processes the user prompt locally through decomposition, PII-aware routing, paraphrasing, and final-response reconstruction skills.The workflow routes sub-tasks between the local SLM and cloud LLM while preserving a local privacy map for reconstruction.
  • Four-stage pipeline: The decomposition skill splits a prompt into k self-contained sub-tasks and attaches a flag indicating whether each contains PII.It handles independent questions, comparison branches, document sections, and chained reasoning steps.
  • Four-stage pipeline: The routing skill assigns solvability labels and dispatches each sub-task directly to the cloud, after clean paraphrasing, or to the local SLM based on its PII flag.Dense identifier lists bypass paraphrasing and remain local because rewriting may leave identifiers or remove answer-critical context.
  • Four-stage pipeline: The paraphrasing skill removes every detected identifier while preserving role, intent, and document structure, but cannot guarantee equivalence because identifier removal can alter task-critical context.Cloud-bound rewrites must satisfy substring exclusion for detector-confirmed raw identifiers.
  • Iterative skill distillation: Skill distillation keeps knowledge in prompt-based skills: the cloud LLM evaluates end-to-end responses, attributes failures to one stage, and proposes aggregate-statistics-based rewrites subject to memorization checks.The four skills are initialized by hand; the routing skill remains fixed while decomposition, paraphrasing, and reconstruction are refined.

4 Experiments

Experiments evaluate P2Skill on a four-domain, 160-prompt benchmark using four local SLMs, privacy and inference-quality metrics, and two perturbation baselines. P2Skill achieves the strongest privacy-quality results, while residual leakage mainly arises when decomposition fails to flag identifiers.

  • Datasets: The evaluation benchmark contains 160 prompts across Medical consultation, Banking services, Tourism planning, and General knowledge, with structured PII in Medical and Banking.Skill refinement uses a disjoint 70-prompt set combining PRISM-style prompts and auxiliary PII samples.
  • Metrics: Inference quality is judged on a 1–10 rubric, while privacy preservation is defined as PP = 1 − N_leakage/N_total.PII transmission leakage counts annotated identifiers reaching the cloud, and every cloud-bound request is recorded in an audit log.
  • Overall results: 6.95 is P2Skill’s average PQ, compared with 4.11 for selective LDP and 1.90 for uniform LDP, corresponding to 1.69× and 3.66× improvements.P2Skill reaches the highest privacy preservation on every SLM; two SLMs achieve PP = 1.0, while the other two transmit 6 of 87 annotated identifiers.
  • Baseline comparison: Selective LDP achieves the highest inference quality on Banking and Medical for every SLM but leaks all 47 declared Banking PII identifiers.Its predefined patterns cover emails, phone numbers, full credit-card numbers, and names introduced by “named X”, leaving other identifier forms unperturbed.
  • Routing analysis: Gemma4:e2B and Qwen3.5:2B predominantly route PII sub-tasks through cloud paraphrasing and are the two SLMs achieving PP = 1.0.Llama3.2:3B paraphrases 44.3% of PII sub-tasks, sends another 48% directly to the cloud, and contributes the residual 6 of 87 transmitted identifiers.
  • Limitations: Residual leakage is mainly caused by identifiers that bypass decomposition flagging, although subsequent modules reliably remove every identified entity.P2Skill does not fully eliminate leakage, and overall inference quality remains below the cloud model’s performance.

5 Conclusion

P2Skill is a prompt-based skill distillation method for privacy-preserving cloud-local LLM inference. A frozen local SLM executes iteratively refined skills without weight updates or auxiliary privacy-detector training, enabling general PII detection beyond memorized examples.

  • Method: P2Skill uses prompt-based skill distillation for privacy-preserving cloud-local LLM inference.The method is designed for systems combining local and cloud language models.
  • Method: A frozen local SLM executes four iteratively refined skills while the cloud LLM guides refinement.The SLM weights remain unchanged throughout the refinement loop.
  • Privacy mechanism: The approach requires neither SLM weight updates nor auxiliary privacy-detector training.Its privacy behavior is obtained through skill refinement rather than privacy-specific model training.
  • Privacy mechanism: The resulting skills enable general PII detection instead of memorization of specific training examples.This allows the local SLM to detect personally identifiable information beyond memorized patterns.

A Per-method Aggregate Results

Table 3 reports per-method aggregate inference quality, PII leakage, and privacy-quality scores across four local SLMs on the GPT-4o 160-prompt benchmark. It also includes Cloud only and local only as upper- and lower-bound references for the cloud-local setting.

  • Aggregate benchmark results: The aggregate results cover all four local SLMs, with PQ computed only on the benchmark’s PII-containing portion.The rightmost Avg. PQ column averages per-SLM PQ across the four local SLMs.
  • Aggregate benchmark results: Table 3 aggregates inference quality (IQ), PII transmission leakage, and privacy-quality (PQ) across the GPT-4o 160-prompt benchmark.Leakage is measured against 87 annotated identifiers.
  • Aggregate benchmark results: Cloud only and local only provide upper- and lower-bound reference points for the cloud-local setting.These references are included alongside the per-method aggregates.

B Per-domain Routing Distribution

Table 4 reports P2Skill’s routing distribution across all four benchmark domains, distinguishing PII-containing from PII-free prompts. Cloud-direct routing on Tourism and Common shows how the pipeline preserves cloud-model use for non-sensitive prompts.

  • Per-domain routing distribution: Cloud-direct routing on Tourism and Common indicates the pipeline’s preservation of cloud advantages for PII-free prompts.These domains contain no annotated PII.
  • Per-domain routing distribution: Table 4 reports the percentage of decomposed sub-tasks following each routing path across the 160-prompt benchmark’s four domains.Medical and Banking contain PII, whereas Tourism and Common have no annotated PII.

C Sonnet-4.6 Evaluation

The full 160-prompt Sonnet-4.6 evaluation serves as a cross-cloud check and reproduces the model-compatibility ordering observed under GPT-4o judging. Its transmission-privacy claims remain based on GPT-4o audit logs.

  • Cross-cloud evaluation: The Sonnet-4.6 judge reproduces the GPT-4o model-compatibility ordering.The evaluation covers 160 prompts as a cross-cloud check.
  • Model compatibility: Gemma4:e2B, Qwen3.5:2B, and Llama3.2:3B improve over Local Only, while Qwen2.5:1.5B remains the weakest local model.
  • Evaluation metrics: Table 5 reports Sonnet-4.6-scored inference quality for each method, while P2Skill-only Leak counts transmitted annotated PII identifiers out of 87 and PQ equals IQ × PP.
  • Privacy claims: The main transmission-privacy claims rely on GPT-4o audit logs in Table 3.

D Skill Prompt Examples

The appendix presents representative prompts for decomposition, PII-aware routing, paraphrasing, and final-response reconstruction, alongside end-to-end examples showing paraphrase, local-only, and direct-cloud behaviors. Together, the examples illustrate privacy-aware routing that preserves utility when possible and avoids cloud calls when identifiers remain.

  • D Skill Prompt Examples: The four pipeline skills are shown as shortened excerpts from versioned prompts; decomposition, paraphrasing, and reconstruction are iteratively refined, while routing remains at its initial version.A deterministic identifier matcher runs around paraphrasing before any cloud call.
  • D.1 Skill 1: Decomposition: The decomposition skill splits prompts only for multiple distinct requests and marks subtasks containing personal information with has_pii: true.It requires complete explicitly stated requests and valid JSON containing task id, content, and the PII flag.
  • D.2 Skill 2: PII-aware Routing: The routing skill assigns each task exactly one label and sends simple operations to the local model while reserving cloud processing for broader knowledge or complex reasoning when permitted.Its labels are no_answer_needed, slm_solvable, slm_unsolvable, and needs_decomposition.
  • D.3 Skill 3: PII Paraphrasing: The paraphrasing skill replaces personally identifiable information with natural role descriptions while preserving meaning, intent, and non-identifying content.It replaces names, exact ages, account or card digits, and contact details without placeholder tokens or commentary.
  • D.4 Skill 4: Final Response Reconstruction: The reconstruction skill combines partial results into one coherent answer while checking safety, refusal, privacy, factual fidelity, and completeness.It preserves refusals and includes only content supported by the partial results.
  • D.5 End-to-End Pipeline Examples by Domain: The end-to-end examples cover four routing behaviors: paraphrase-then-cloud, keep-on-device, direct-cloud-without-PII, and direct-cloud-on-non-PII content.They are recorded from the Qwen3.5:2B P2Skill run on the 160-prompt benchmark.
  • D.5 End-to-End Pipeline Examples by Domain: Medical processing paraphrases a PII-containing prompt, confirms zero declared identifiers in the cloud-bound text, and forwards the rewrite to the cloud.The example replaces the patient’s identifying age and name while retaining the symptoms for diagnosis generation.
  • D.5 End-to-End Pipeline Examples by Domain: Banking processing falls back to the local SLM when paraphrasing leaves a detected identifier, whereas Tourism passes the original PII-free text directly to the cloud.The examples therefore demonstrate local-only handling for dense PII and pass-through when no privacy-relevant content is detected.
Loading 2608.14094v1…