PaperlayerView exact revision on arXiv

Source-linked AI summary

PAAC: Privacy-Aware Agentic Device-Cloud Collaboration

Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, Christopher G. Brinton

arXiv:2605.08646v1cs.LGcs.CLcs.DC

TL;DR

Cloud and on-device agents trade reasoning capability against privacy, while existing sanitizers struggle to preserve tool-call structure under flexible policies. PAAC aligns planner–executor roles with the device–cloud boundary and uses typed proxy tokens, achieving the best privacy–accuracy trade-off across three agentic benchmarks.

  • Problem

    Cloud agents expose sensitive user data, while on-device agents provide weaker complex reasoning; existing sanitizers trade policy flexibility against tool-call structural fidelity.

  • Method

    PAAC separates cloud reasoning from on-device execution and judging, using typed proxy tokens with deterministic substitution and reversal under configurable policies.

  • Results

    PAAC attains the best privacy–accuracy trade-off across three agentic benchmarks, matching the strongest cloud accuracy while reducing leakage by up to an order of magnitude versus device–cloud baselines.

  • Takeaways & Limitations

    PAAC places on the upper-left privacy–accuracy frontier by preserving structural cues for tool calls while removing verbatim sensitive values.

  • Takeaways & Limitations

    PAAC protects verbatim policy-defined sensitive entities but does not protect information disclosed through proxy-token semantic types, which are outside its protected asset.

Abstract

from arXiv · show

Large language model (LLM) agents face a structural tension: cloud agents provide strong reasoning but expose user data, while on-device agents preserve privacy at the cost of overall capability. Existing device-cloud designs treat this boundary as a compute split rather than a trust boundary suited to agentic workloads, and existing sanitizers force a choice between policy flexibility and the structural fidelity tool calls require. In this work, we develop PAAC, a privacy-aware agentic framework that aligns planner--executor decomposition with the device-cloud boundary so that role specialization itself becomes the privacy mechanism. The cloud agent reasons over typed placeholder tokens that preserve each sensitive value's reasoning role while discarding its content, while the on-device agent identifies sensitive spans and distills each step's execution outcome into compact key findings. Sanitization confines the on-device LLM to proposing which spans to mask, while a deterministic registry performs all substitution and reversal, keeping actions directly executable on device. On three agentic benchmarks under strict privacy settings, PAAC dominates the Pareto frontier of privacy and accuracy, improving average accuracy by 15-36\% and reducing average leakage by 2-6$\times$ over state-of-the-art device-cloud baselines, with the largest margins on privacy targets outside fixed entity taxonomies. We find consistent improvements on 17 additional benchmarks spanning 10 domains, including math, science, and finance.

1 Introduction

PAAC aligns cloud reasoning and on-device execution with the device-cloud trust boundary, using typed proxy tokens so the cloud can plan without seeing sensitive values. Its on-device sanitization combines policy flexibility with deterministic reversibility, making role specialization itself a structural privacy mechanism.

  • PAAC architecture: Typed proxy tokens preserve each sensitive span’s semantic role while discarding identifying content, allowing cloud planning without exposing underlying values.For example, the cloud can plan subtract(BALANCE, RENT), after which the device re-binds values for execution.
  • Sanitization requirements: Sensitive-span identification must follow arbitrary user policies because sensitivity depends on the user, task, and context rather than a single fixed taxonomy.The sanitizer must also produce proxy tokens that can be deterministically reversed for tool execution.
  • PAAC architecture: PAAC assigns reasoning and planning to the cloud while the device sanitizes, executes, judges, and generates final answers.This role split aligns planner–executor decomposition with the device-cloud trust boundary.
  • Privacy mechanism: Role specialization becomes a structural privacy mechanism by casting the on-device agent as a judge and privacy guardian while the cloud reasons over sanitized representations.Per-step distillation keeps each agent’s input compact across agentic turns.

2 Background and Challenges

The background identifies two challenges in device-cloud agentic systems: trajectory-coupled context growth burdens on-device agents, while privacy sanitization must adapt to policies without breaking tool-call fidelity or cross-turn consistency.

  • Trajectory-Coupled Context Growth: Trajectory-coupled growth accumulates reasoning traces, tool invocations, and execution results across agentic steps, making on-device operation prohibitive.Existing compression methods operate over the full accumulated history, whose size increases with the number of steps.
  • Trajectory-Coupled Context Growth: PAAC partitions reasoning and execution across device and cloud, freeing the on-device agent from managing the full trajectory.The cloud agent independently maintains its reasoning context while the device handles execution-related responsibilities.
  • Semantic Alignment in Privacy Sanitization: Privacy sanitization must be policy-adaptive, supporting user-defined targets such as ordinary product nouns rather than only standard named entities.Pattern-based sanitization methods such as spaCy and Presidio provide deterministic substitution but are presented as insufficient for this flexibility.
  • Semantic Alignment in Privacy Sanitization: The sanitization boundary must preserve tool-call fidelity and cross-turn consistency while content crosses between device and cloud.These requirements form the second design thread summarized in Table 1.

3 Privacy-Aware Agentic Device-Cloud Collaboration

PAAC aligns planner–executor roles with the device–cloud boundary: the cloud reasons over sanitized semantic representations, while the device handles privacy, execution evaluation, and answer generation. A deterministic registry preserves context-consistent masking and reversal under user-defined policies, with joint termination coordinating both agents.

  • Threat Model: PAAC protects policy-defined sensitive entities from an honest-but-curious cloud while treating proxy-token semantic types as intentionally disclosed.The cloud may infer from sanitized queries and reasoning traces, but leakage is measured against the verbatim occurrence of policy-defined sensitive entities.
  • Cloud Agent: The cloud performs high-level reasoning and planning over sanitized task, tool, and memory representations, without receiving identified sensitive user data.Its REASON component generates a reasoning trace, sanitized action, and cloud termination signal at each step.
  • On-Device Agent: The device sanitizes inputs, executes actions, evaluates outcomes, and distills each step into compact findings and feedback based only on current-step context.SANITIZE uses a synchronized mapping table, while DESANITIZE reverses it through regex replacement without LLM inference.
  • Device-Cloud Consensus Termination: Both agents must signal termination before PAAC stops, otherwise the next round uses on-device feedback to guide cloud revision, with forced termination at Tmax.This consensus rule addresses the capability-information asymmetry between cloud reasoning and device execution observations.
  • Sanitization: User-defined policy P supports default and custom sensitivities, while the on-device LLM proposes candidate masks and a deterministic registry performs substitution and restoration.The policy can persist as an on-device user profile; proposal alignment verification commits a mapping only when restoration matches the original text.

4 Experiments

PAAC is evaluated on three agentic benchmarks across graded privacy levels, using a Qwen3-4B-Instruct on-device agent and Gemini 3 Flash cloud agent. It achieves the best privacy–accuracy trade-off, while analyses attribute gains to both its architecture and sanitization design.

  • Experimental setup: The main evaluation covers τ 2-Bench Airline, τ 2-Bench Retail, and GAIA under graded privacy levels P0–P3.Experiments use Qwen3-4B-Instruct on device, Gemini 3 Flash in the cloud, and up to Tmax = 10 agentic steps.
  • Main results: PAAC matches the strongest cloud single-agent’s accuracy while reducing leakage by up to an order of magnitude versus PAPILLON and PRISM.This result holds across τ 2-Bench Airline, τ 2-Bench Retail, and GAIA at all evaluated privacy levels.
  • Sanitization analysis: PBS exceeds 50% leakage at P3 on the open-vocabulary τ 2 benchmarks but remains near 13% on GAIA’s closed-vocabulary entities.Its fixed entity taxonomy creates opposite failure patterns across open- and closed-vocabulary tasks and also degrades accuracy.
  • Sanitization analysis: PAPILLON eliminates more than 85% of spans without structural guarantees, whereas PRISM perturbs surface values and struggles to preserve tool-call structure.The results characterize rewriting and perturbation as distinct failure modes under agentic interactions.
  • Ablations: Ablations show that PAAC’s architecture exceeds the cloud single-agent even without sanitization, while replacing its sanitizer with PBS isolates an independent sanitization benefit.The comparison uses the P0 column and the PAAC w/ PBS row to separate architectural and sanitization contributions.

5 Conclusion and Limitations · Appendix

PAAC is a privacy-aware device-cloud framework that decouples cloud reasoning and planning from on-device execution and judgment under configurable sanitization. Its design partitions roles by agent capabilities while keeping per-step inputs compact and replacing detected sensitive entities.

  • 5 Conclusion and Limitations: PAAC is a privacy-aware decoupled device-cloud agentic framework.The framework supports collaboration between cloud and on-device agents.
  • 5 Conclusion and Limitations: Cloud agents reason and plan, while on-device agents execute and judge.This collaboration follows a decoupled division of agentic roles.
  • 5 Conclusion and Limitations: PAAC operates under a configurable sanitization policy.Sanitization is part of the framework’s privacy-aware device-cloud collaboration design.
  • 5 Conclusion and Limitations: Its decoupled architecture partitions roles according to cloud and on-device agent capabilities.Role allocation is the first stated contribution of PAAC.
  • 5 Conclusion and Limitations: PAAC keeps each agent’s per-step input compact across agentic interactions.Compact per-step inputs are an explicit architectural objective.
  • 5 Conclusion and Limitations: An LLM-driven privacy sanitizer replaces detected sensitive entities.This mechanism is the second stated contribution, though the supplied passage ends before describing the replacement details.

A Further Analysis … C PAAC System Details

The supplied material is organized into further privacy-sanitization analyses, benchmark and use-case sections across many domains, and PAAC system details. It includes subsections on sanitization bottlenecks and robustness, agentic reasoning benchmarks, and privacy-aware text sanitization.

  • A Further Analysis: A Further Analysis covers privacy sanitization bottlenecks, baseline comparison, alignment and coverage, PII leakage, adversarial robustness, reasoning behavior, and system-level impacts.It also includes analyses of maximum steps, agent capabilities, consensus termination, and per-component reflection.
  • A Further Analysis: Further analyses also examine maximum steps, agent capabilities, consensus termination, and per-component reflection as extensions of the privacy-sanitization study.These are listed as subsections A.7 through A.10.
  • B Benchmarks and Use Cases: B Benchmarks and Use Cases organizes evaluations under agentic reasoning, math, multimodal math, science, factual reasoning, logic reasoning, medical, finance, accounting, history, literature, and image generation.The listed benchmarks include τ 2-Bench, GAIA, GSM8K, MathQA, Geometry3K, MathVista, SciBench, SciQ, TruthfulQA, HotpotQA, FEVER, CLUTRR, AGIEval LSAT-AR, MedQA, FinQA, MMLU Professional Accounting, MMMU Accounting, Jeopardy History, and Jeopardy Literature.
  • B Benchmarks and Use Cases: The math and multimodal-math evaluations comprise GSM8K, MathQA, Geometry3K, and MathVista.These subsections appear under B.2 Math and B.3 Multimodal Math.
  • B Benchmarks and Use Cases: The applied-domain evaluations cover medical MedQA, finance FinQA, accounting MMLU Professional Accounting, and multimodal accounting MMMU Accounting.Additional listed domains are history, literature, and image generation, with Jeopardy History and Jeopardy Literature as named benchmarks.
  • C PAAC System Details: C PAAC System Details includes a subsection on privacy-aware text sanitization.The listed subsection is C.1 Privacy-Aware Text Sanitization.

D Details on Experiments · E Prompt Templates · A Further Analysis

The section provides experiment details on open-ended tools and privacy definitions, followed by cloud-agent, on-device-agent, and evaluation prompt templates. No passage content is supplied for A Further Analysis.

  • D Details on Experiments: D Details on Experiments includes a subsection on open-ended tools.The supplied passage identifies D.1 Open-Ended Tools as part of the experiments details.
  • A Further Analysis: The supplied passages enumerate experiment-details and prompt-template subsections but provide no substantive findings or methods for A Further Analysis.This point reflects the scope of the supplied passages rather than a claim about the paper’s full contents.
  • D Details on Experiments: D Details on Experiments also defines privacy concepts in a dedicated subsection.The supplied passage lists D.2 Privacy Definitions.
  • E Prompt Templates: E Prompt Templates includes dedicated prompts for the cloud agent.The supplied passage lists E.1 Cloud Agent Prompts.
  • E Prompt Templates: E Prompt Templates separately specifies prompts for the on-device agent.The supplied passage lists E.2 On-Device Agent Prompts.
  • E Prompt Templates: The prompt-template section concludes with an evaluation prompt subsection.The supplied passage lists E.3 Evaluation Prompt.

A.1 Privacy Sanitization: Privacy Identification as the Bottleneck

Privacy sanitization is governed primarily by sensitive-span identification: any span missed in Stage I passes through unchanged, regardless of Stage II’s formal guarantee. Empirically, PAAC’s broader policy-driven identification—not its substitution mechanism alone—produces the leakage advantage over NER-based baselines.

  • Stage I identification: End-to-end expected leakage is bounded below by (1−r(Π1))·Ex[|S(x)|], where r(Π1) is Stage I micro-averaged recall.Stage II acts as the identity on spans not identified by Π1, so missed sensitive spans remain observable without requiring independence assumptions.
  • Stage I identification: A Π2-level guarantee such as ε-DP bounds leakage only on identified spans, while higher r(Π1) can dominate a stronger Π2 guarantee end to end.This permits comparison of methods with incomparable Stage II guarantees along the Stage I identification axis.
  • Empirical verification: ≈40% leakage occurs for both NER + Masking and NER + DP on CLUTRR across all ε, compared with PAAC: 0% leakage.The baselines share an NER-based Π1, so their missed spans remain unaffected by masking or LDP perturbation.
  • Empirical verification: PAAC’s leakage advantage is traceable to its LLM-based proposer recovering policy-defined spans that the NER template misses, rather than to Π2 alone.The comparison varies Π2 while holding NER-based Π1 fixed for the two baselines, with PAAC providing a reference using its LLM-based Π1.
  • Engineering scaffolds: Dual execution is a precondition for task correctness in NER + DP when perturbed outputs fall outside the range of value-to-value inverse mappings.Without dual execution, the final_answer is computed on perturbed inputs and cannot be restored to the real answer.

A.2 Privacy Sanitization: Baseline Comparison

PAAC’s sanitizer preserves the semantic structure needed for cloud reasoning while enabling deterministic reversal and direct execution on device. Compared with fixed-taxonomy and query-rewriting baselines, its evaluation highlights broader privacy-specification generalization, especially beyond common entity categories.

  • Qualitative Comparison: Existing sanitizers comprise pattern-based methods tied to fixed entity taxonomies and LLM-driven methods offering more flexible sensitive-span identification.The comparison frames these as two dominant sanitizer families.
  • Qualitative Comparison: PAAC replaces sensitive spans with semantic proxy tokens that preserve reasoning structure and support deterministic reversal through a synchronized mapping table.This design avoids stripping the semantic structure required for precise cloud-agent reasoning.
  • Qualitative Comparison: Semantic-preserving proxies let cloud outputs be directly desanitized and executed, eliminating PAPILLON’s extra on-device interpretation step.PAPILLON rewrites the entire query into a privacy-free paraphrase, perturbing the structure needed to recover executable actions.
  • Quantitative Comparison: Table 6 evaluates sanitization across numerical values, common entities, personal names, and transaction fields including order IDs, addresses, and identity attributes.The benchmarks are GSM8K, TruthfulQA, CLUTRR, and τ 2-Retail, respectively.
  • Quantitative Comparison: The re-analysis reveals a pronounced generalization gap across baselines, with PBS effective when target privacy categories align with its fixed taxonomy.The supplied passage introduces this finding but truncates the remainder of the comparison.

A.3 Privacy Sanitization: Alignment and Coverage

PAAC evaluates sanitizer alignment as exact recovery of the original query after desanitization, comparing Qwen3-4B with Gemini 3 Flash on 1,000 GSM8K samples. The on-device model exhibits a non-negligible alignment gap and failure modes tied to instruction following and structured-output fidelity.

  • Alignment Analysis: Alignment measures whether desanitization exactly recovers the original query, formally testing x = DESANITIZE.The evaluation targets the sanitizer’s ability to recover the original input from its sanitized form.
  • Alignment Analysis: The study compares Qwen3-4B and Gemini 3 Flash as sanitizer backbones using 1,000 GSM8K samples with tightly coupled numerical values, units, and symbols.PBS achieves 100% alignment by construction through deterministic pattern-based processing.
  • Alignment Analysis: Alignment errors include output errors, hallucinated tokens, unit or symbol loss, linguistic mismatch, and information mismatch.Examples include missing structured-response keys, unresolved placeholders, misplaced units or symbols, and grammatical artifacts.
  • Alignment Analysis: The on-device LLM has a non-negligible alignment gap relative to the large-scale LLM, with dominant failures involving malformed structured output and unresolved or extraneous content.These failures reflect insufficient instruction-following capability in adhering to the prescribed structured output format.

A.4 Privacy Sanitization: PII Leakage Evaluation on AI4Privacy

PAAC provides the strongest overall privacy sanitization on AI4Privacy, achieving the lowest Leak and Miss rates while remaining competitive across all eight PII categories. The evaluation also shows why structured sanitization methods degrade on freer-form sensitive targets and why complementary leakage metrics are needed.

  • Category variation: PBS methods perform reliably on numbers and standard named entities but degrade sharply on personal names and user-generated identifiers.This bimodal behavior follows from PBS construction and reflects the difficulty of reducing privacy leakage for freer-form targets.
  • Leakage metrics: Leak is the primary privacy metric because it captures whether sensitive spans survive into sanitized text and remain directly observable to downstream consumers.On task-oriented benchmarks, the passage identifies Leak as sufficient, while Table 7 reports both Leak and Miss for broader PII evaluation.
  • Overall leakage: 9.7% Leak and 7.7% Miss are PAAC’s lowest overall rates across the eight AI4Privacy PII category groups.Leak measures ground-truth PII substrings remaining verbatim in sanitized output; Miss measures ground-truth PII values absent from the sanitizer’s privacy mapping.
  • Overall leakage: PAAC is the only method competitive across all eight categories and roughly halves leakage relative to the strongest PBS variant.Adding custom regex rules further reduces leakage on structured categories such as emails, phones, and dates, but requires additional engineering effort.

A.5 Privacy Sanitization: Adversarial Robustness … A.10 Extensibility Probe: Per-Component Reflection

Across six analyses, PAAC remains robust to passive inference and active prompt injection, while preserving reasoning behavior and benefiting from larger step budgets, stronger cloud models, consensus termination, and selective reflection. These studies also show that privacy sanitization and role specialization do not fundamentally destabilize the agentic workflow.

  • A.5 Privacy Sanitization: Adversarial Robustness: Passive recovery from sanitized queries is near zero, while the worst-case active Attack-Induced Leak Rate remains at most 0.2% across all eight payload variants and benchmarks.The architecture routes tool outputs through on-device Judge and Sanitize components before cloud transmission.
  • A.5 Privacy Sanitization: Adversarial Robustness: Adaptive template-aware attacks can reduce Judge-Refused rates, but deterministic regex substitution over the committed privacy mapping keeps Attack-Induced Leak Rate at or below 0.1%.The mapping is initialized from the original query before tool outputs, preventing compromised LLMs from unmasking spans already recorded.
  • A.6 Privacy Sanitization: Impact on Reasoning Behavior: Minimal sanitization at P1 produces reasoning behavior statistically comparable to self-consistency, and GAIA remains comparable across all privacy levels, including P3.Table 9 compares cross-privacy divergence against the inherent variability of unsanitized LLM generation.
  • A.7 Impact of Maximum Steps: Accuracy generally increases with a larger maximum agentic-step budget, although gains diminish and accuracy can slightly decline beyond a threshold.Additional steps support information gathering, error recovery, and plan refinement, but extended interactions may introduce drawbacks.
  • A.8 Impact of Agent Capabilities: All tested on-device LLMs achieve strong results without agentic-task fine-tuning, while Pro-tier cloud models generally outperform Flash-tier models across many configurations.The findings support assigning the on-device role to capable off-the-shelf small LLMs.
  • A.9 Impact of Consensus Termination: Consensus termination consistently outperforms on-device-only and cloud-only termination in the majority of configurations across benchmarks, reasoning strategies, and privacy levels.Consensus combines the cloud agent’s global planning awareness with the on-device agent’s local execution verification.
  • A.9 Impact of Consensus Termination: Disagreement between agents remains persistent but bounded, and most τ 2-Bench samples reach consensus before exhausting the step budget.The joint rule mediates substantive objections rather than trivially accepting either agent’s termination decision.
  • A.10 Extensibility Probe: Per-Component Reflection: Per-component reflection does not provide a uniform accuracy advantage; its effects vary by benchmark, with τ 2-Bench benefiting more consistently than GAIA.Reflection re-checks and revises outputs for the Reasoner, Sanitization, Judge, and Final Answer Generator roles.

B Benchmarks and Use Cases … E Prompt Templates

PAAC evaluates privacy-aware agentic collaboration across diverse reasoning, multimodal, and domain-specific benchmarks using task-tailored privacy categories, tools, and prompt-driven roles. Its appendices specify deterministic sanitization, open-ended tool configurations, privacy definitions, and prompts for cloud planning, on-device processing, judging, reflection, and evaluation.

  • B Benchmarks and Use Cases: Benchmarks assign task-specific privacy categories and cumulative privacy levels, with final_answer included as a tool across datasets.Mathematical tasks protect numerical values, while factual reasoning tasks sanitize named entities; privacy categories also include domain-specific information such as patient profiles.
  • B.1 Agentic Reasoning; B.2 Math; B.3 Multimodal Math; B.4 Science; B.5 Factual Reasoning; B.6 Logic Reasoning; B.7 Medical; B.8 Finance; B.9 Accounting; B.10 Multimodal Accounting; B.11 History; B.12 Literature: PAAC covers agentic reasoning, mathematics, multimodal mathematics, science, factual, logic, medical, finance, accounting, history, and literature benchmarks.The listed benchmark results include τ 2-Bench Airline and Retail, GAIA, GSM8K, MathQA, Geometry3K, MathVista, SciBench, SciQ, TruthfulQA, HotpotQA, FEVER, CLUTRR, AGIEval LSAT-AR, MedQA, FinQA, MMLU Professional Accounting, MMMU Accounting, and Jeopardy tasks.
  • B.13 Image Generation: Collaborative Scene Reconstruction extends evaluation to multimodal privacy-sensitive interaction, where agents with asymmetric information cooperate conversationally to reconstruct a target scene.The on-device agent has access to a ground-truth image, while the cloud-side agent collaborates through the exchanged interaction.
  • C.1 Privacy-Aware Text Sanitization: SANITIZE uses on-device LLM inference to identify sensitive spans, while DESANITIZE restores values through deterministic regex replacement without LLM inference.Structured key-value inputs receive semantic proxy tokens registered in a privacy mapping before substitution; desanitization reverses those mappings.
  • C.1 Privacy-Aware Text Sanitization: The deterministic registry uses joint semantic keying, length-descending iteration, word-boundary constraints, and persistent counters to disambiguate and reversibly replace overlapping values.These mechanisms distinguish identical surface values in different roles and prevent insertion-order-dependent or sub-word over-masking.
  • D.1 Open-Ended Tools: Except for τ 2-Bench’s fixed environment, benchmarks select task-appropriate subsets from a shared pool of open-ended tools.The pool includes web search, website and arXiv retrieval, Wikipedia lookup, sandboxed Python execution, and local file reading.
  • E Prompt Templates: Prompt templates assign cloud planning and on-device roles for privacy sanitization, execution judging, final-answer generation, reflection, and correctness evaluation.The cloud prompt uses Parallel-PS planning; on-device prompts extract and audit mappings, verify execution results, produce key findings, formulate answers, and compare predictions with ground truth.
Loading 2605.08646v1…