PaperlayerView exact revision on arXiv

Source-linked AI summary

GuardianAgent: Policy-Conditioned Risk-Adaptive Anonymization with Verified Adversarial Escalation

Ruiyi Yang, Gayathri Lihinikaduarachchi, Rahat Masood, Flora D. Salim, Salil S. Kanhere

arXiv:2608.29251v1cs.AIcs.HCcs.MA

TL;DR

Live web privacy agents must assess whether outgoing actions comply with destination policies and tailor rewriting to residual disclosure risk. GuardianAgent combines structured policy-conditioned risk assessment with verified adaptive anonymization, achieving the strongest reported privacy–utility trade-off across three domains and exceeding 0.90 privacy in each. Its evaluation is bounded to English public or synthetic corpora, while real browser traces, clinical and dialogue data, and multilingual settings remain untested.

  • Problem

    Live web privacy protection requires agents to assess policy compliance and choose context-appropriate rewriting rather than merely detect private spans.

  • Method

    GuardianAgent uses AMRSF to combine policy evidence and action context, route uncertain matches to an LLM, and escalate five-level rewriting only for adversarial guesses verified against the original text.

  • Results

    GuardianAgent achieves the strongest privacy–utility trade-off among published baselines and is the only method to reach ≥0.90 privacy across TAB, SynthPAI, and PII-Masking-300k.

  • Takeaways & Limitations

    Action-context stress tests show that GuardianAgent adapts decisions and anonymization strength to recipient, purpose, action basis, and policy transparency.

  • Takeaways & Limitations

    Evaluation is limited to English and public or synthetic corpora, leaving real browser traces, clinical and dialogue data, and multilingual settings for further study.

Abstract

from arXiv · show

Privacy protection for live web traffic requires more than detecting private spans. Agent-based privacy protection systems must determine whether an outgoing action complies with the destination site's privacy policy, then apply only the level of rewriting or sanitisation justified by the residual disclosure risk. We present GuardianAgent, a policy-conditioned anonymization framework that couples structured risk assessment with verified adaptive rewriting. GuardianAgent computes risk through AMRSF (Adaptive Multi-factor Risk Scoring Formula), an explicit controller that combines policy-violation likelihood with data sensitivity, recipient transmission, purpose legitimacy, contextual basis, and policy transparency, rather than relying on an LLM to assign risk directly. This risk score determines both the allow/transform/deny decision and the initial anonymization level. For efficiency, GuardianAgent uses an evidential fast path for low-uncertainty policy matches and invokes an LLM slow path only for uncertain cases. For rewriting, it applies a five-level hierarchy driven by a verified adversarial guesser: guesses trigger escalation only when supported by the original text, preventing hallucinated attacker confidence from causing unnecessary over-anonymization. Experiments across three benchmarks spanning legal text (TAB), Reddit posts (SynthPAI), and multi-format synthetic PII records (PII-Masking-300k) show that GuardianAgent achieves the strongest privacy-utility trade-off among published baselines and is the only method to reach more than 0.90 privacy in all three domains, remaining robust under a backbone switch. Action-context stress tests further show that the same outgoing text receives different decisions and anonymization strengths under different recipients, purposes, action bases, and policy-transparency conditions.

1 Introduction

GuardianAgent unifies privacy-policy matching and text anonymization for live browser actions, using structured action-context risk to select decisions and rewriting strength. Its verified adaptive design improves privacy–utility trade-offs across three domains while avoiding unsupported adversarial escalation.

  • Motivation: GuardianAgent addresses live web privacy by checking whether outgoing actions comply with destination policies and rewriting content when needed.The framework treats policy matching and text anonymization as coupled problems rather than isolated tasks.
  • Structured risk control: AMRSF combines policy-violation likelihood, data sensitivity, recipient transmission, purpose legitimacy, contextual basis, and policy transparency to control mediation.The resulting risk score jointly determines allow, transform, or deny and the initial anonymization level.
  • Structured risk control: The same outgoing text receives different decisions and initial anonymization levels under different recipients, purposes, action bases, and policy-transparency conditions.This demonstrates that anonymization strength depends on action context rather than text alone.
  • Adaptive anonymization: Verified adversarial anonymization escalates rewriting only when attacker guesses are supported by the original text, preventing hallucinated confidence from causing unnecessary semantic distortion.This addresses cases where plausible world knowledge produces guesses unsupported by the source text.
  • Evaluation: GuardianAgent achieves the strongest privacy–utility trade-off among published baselines across TAB, SynthPAI, and PII-Masking-300k, reaching ≥0.90 privacy in all three domains.The gains remain under a backbone switch.

2 Related Work

Prior work separately studies privacy-policy understanding, quantitative risk modelling, adversarial text anonymization, and uncertainty-gated inference. GuardianAgent connects these lines through policy-conditioned risk-adaptive anonymization with verified adversarial feedback.

  • Privacy-policy grounding: Privacy-policy research extracts, searches, classifies, and generates structured interpretations of policy statements for downstream reasoning about data practices.Examples include Polisis, PrivaSeer, PrivBERT, PolicyGPT, and joint intent-slot analysis.
  • Privacy-risk frameworks: Privacy-risk frameworks draw on regulatory tiering, likelihood–impact modelling, contextual integrity, and recent LLM-based policy assessment.These approaches provide conceptual and quantitative foundations for reasoning about privacy risk.
  • LLM-based text anonymization: Recent anonymizers rewrite content through adversarial loops, prompt sanitization, contextual-integrity prompting, on-device models, differential privacy, or attribute randomization.These methods extend protection beyond tagging sensitive spans.
  • LLM-based text anonymization: Adversarial rewriting can over-escalate when attacker confidence reflects plausible world knowledge rather than leakage supported by the original text.GuardianAgent verifies guesses against the original before allowing stronger rewriting.
  • Dual-system inference and uncertainty gating: Fast/slow inference and evidential uncertainty gating reduce expensive-model use, but privacy mediation requires fast decisions grounded in policy evidence and conservative escalation.GuardianAgent positions its uncertainty-aware routing within this efficiency and safety trade-off.

3 Method

GuardianAgent mediates outgoing web actions by combining policy evidence with action-conditioned risk to choose allow, transform, or deny, and to set anonymization strength. Transformed content is rewritten through a five-level hierarchy whose escalation depends on adversarial leakage verified against the original text.

  • 3.1 Overview: GuardianAgent represents each web action with its destination, data categories, recipients, purposes, action basis, and outgoing text.Policies are segmented into structured atomic statements, and the retriever selects relevant evidence for matching.
  • 3.2 Efficient Policy Matching: A two-path matcher estimates policy-violation likelihood from action–evidence pairs, using an evidential fast path and an LLM path for uncertain cases.The LLM estimates only likelihood L, not the final risk score, action decision, or anonymization level.
  • 3.3 Policy-Conditioned Risk Estimator: Risk determines allow, transform, or deny and, for transformed actions, uses normalized transform-band position z to select the initial anonymization level.Sensitive tiers use a wider transform region so uncertain high-risk actions are rewritten rather than released unchanged.
  • 3.3 Policy-Conditioned Risk Estimator: AMRSF converts violation likelihood into action-conditioned risk by incorporating data sensitivity, recipient transmission, purpose legitimacy, action basis, and policy transparency.The resulting score explicitly separates violation likelihood from contextual severity.
  • 3.4 Risk-Adaptive Hierarchical Anonymizer: The anonymizer uses five levels ranging from partial masking to semantic rewriting, with stronger levels progressively removing entities and contextual cues while preserving overall communicative intent.Actions near the allow ceiling begin with lighter rewriting, whereas those near the deny floor begin with stronger levels.
  • 3.5 Verified Adversarial Escalation: Adversarial escalation occurs only when a guess is supported by the original text and verified confidence reaches the escalation threshold, preventing unsupported guesses from driving stronger rewriting.The loop stops below the threshold, at level 5, or after T = 5 rounds; higher-risk flows instead start at L4 through R.

4 Evaluation

GuardianAgent is evaluated across three anonymization benchmarks, policy-matching data, and calibration scenarios using privacy, utility, guessing, and latency measures. Results assess cross-domain trade-offs, action-conditioned control, adaptive hierarchy, component contributions, calibration, efficiency, and robustness.

  • 4 Evaluation: Evaluation covers legal text, Reddit-style personal attributes, and multi-format synthetic PII records, alongside OPP-115 policy matching and expert-rated risk calibration.The main backbone is Llama-3.2-3B-Instruct, with shared components enabling controlled comparisons.
  • 4.2 Main Comparison: GuardianAgent lies above or beyond baseline privacy–utility frontiers on all three datasets and is the only method above 0.90 privacy across all domains.The reported result also includes the smallest cross-domain spread.
  • 4.4 Ablation: Riskier recipient, purpose, action-basis, and policy-transparency contexts increase mean risk and shift decisions from allow through transform to deny.The same 200 PII-Masking texts are scored under four contexts with L = 0.7 fixed, showing decisions depend on action context rather than text alone.
  • 4.4 Ablation: Within the transform band, increasing normalized risk position shifts starting levels from L1 to L4, while mean final level rises from 2.37 to 4.42 after verified escalation.AMRSF sets initial strength from action context, and verified feedback escalates only when residual leakage remains.
  • 4.4 Ablation: The full verified L1–L5 adaptive stack achieves the best privacy across SynthPAI and TAB at a moderate utility cost, while L4 and L5 address minimal-edit and contextual leakage needs.NER is stronger on explicit legal entities than on implicit, contextual personal attributes in SynthPAI.
  • 4.5 Calibration, Efficiency, and Robustness: AMRSF aligns with annotated risk, the evidential fast path escalates only a small fraction of policy events, and verification improves efficiency without sacrificing privacy gains.Disabling verification increases upgrades and latency while reducing utility; switching to Qwen3-8B preserves GuardianAgent’s strongest privacy–utility performance.

5 Conclusion

GuardianAgent mediates live web traffic with policy-conditioned risk assessment and verified, risk-adaptive anonymization. Across multiple domains, it improves the privacy–utility trade-off, remains robust under a backbone switch, and adapts rewriting strength to action context.

  • GuardianAgent uses AMRSF to combine policy evidence and action context, determining both the allow/transform/deny decision and initial anonymization level.
  • A verified adversarial guesser escalates rewriting only when attacker guesses are supported by the original text.
  • Across legal text, Reddit-style personal attributes, and multi-format PII records, GuardianAgent improved the privacy–utility trade-off over published baselines.
  • GuardianAgent remained robust under a backbone switch and adapted rewriting strength across different action contexts.

Limitations

The evaluation is bounded to English and public or synthetic corpora, leaving several real-world data and deployment settings for future study.

  • The evaluation is limited to English and uses public or synthetic corpora.Real browser traces, clinical and dialogue data, and multilingual settings require further study.
  • AMRSF is a mediation controller rather than a legal compliance engine.Deployments should communicate that anonymization reduces but does not eliminate re-identification risk.

Ethical considerations

The work uses no newly collected personal data or human subjects, but deployment still requires transparency, user control, and communication of residual risks.

  • This work collects no new personal data and involves no human subjects.Experiments use publicly released or synthetic benchmark datasets and no private browser traces.
  • GuardianAgent is intended to reduce unintended disclosure, not to guarantee anonymity or legal compliance.Deployments should disclose rewriting and cloud escalation and preserve user control over mediation decisions.
  • Deployments should communicate residual re-identification and utility-loss risks.

A System 1 Architecture and Retrieval Hyperparameters

The system combines policy and behavior encoders, evidential classification, structured retrieval, and separate decision and anonymization components. Its verified rewriting loop changes final anonymization levels across benchmarks, reducing them on TAB while increasing them on borderline SynthPAI inputs.

  • System 1 Architecture: Behavior and policy encoders map actions and policy snippets into either 42-dimensional OPP-115 multi-hot vectors or 384-dimensional frozen sentence-transformer embeddings.The encoders are denoted ϕb and ϕp, with outputs in R^d.
  • System 1 Architecture: A two-layer ReLU MLP with dropout maps the interaction representation to K = 2 logits, which become Dirichlet parameters through α=softplus(fθ(ψ))+1.The classifier follows evidential deep learning and produces two-class outputs.
  • System 1 Architecture: Class probabilities are computed as pk = αk/S, epistemic uncertainty as u = K/S, and the Eq. 4 likelihood as L=α2/S.These quantities derive from the Dirichlet parameters and their sum S.
  • Retrieval Hyperparameters: Retrieval scores data, action, purpose, and recipient fields using weights (0.35, 0.35, 0.20, 0.10), overlap ratios ρf, and policy flags.Flags add small bonuses for disclosed user rights, trans-EEA transfer, and retention; BM25Lite uses k1 = 1.5 and b = 0.75 with on-the-fly IDF.
  • Decision and Verification Components: The System 2 decider estimates policy-conflict likelihood from an outgoing action and retrieved evidence, returning a likelihood, rationale, and evidence identifiers without making the final action decision.The adversarial guesser separately reconstructs redacted information and reports field-level guesses, confidence, reasoning, and maximum confidence.
  • Adaptive Rewriting: Rewriters preserve communicative intent while removing sensitive fields, identifying details, or contextual cues, with full paraphrasing allowed for stronger anonymization.The L4 minimal-edit prompt specifically prohibits [TAGS] and targets listed sensitive fields; the verified loop changes mean final levels from 3.48 to 2.85 on TAB and from 2.79 to 3.48 on SynthPAI.

D Formal Analysis

The formal analysis establishes guarantees for verified guessing, bounded matcher cost, and loop termination, while the evaluation tables specify calibration and rewrite-strategy comparisons. The controller’s factors and adaptive stack are examined across formal assumptions and experiments.

  • D Formal Analysis: Verified guesses cannot miss any lexically supported attacker guess under the substring-or-shared-content-word assumption.Proposition 1 guarantees C(x0) ⊆ V(x0) and ˆc ≥ maxg∈C(x0) cg.
  • D Formal Analysis: AMRSF assigns explicit data-sensitivity weights, recipient transmission risk, purpose legitimacy, and policy-transparency credits as controller factors.These factors parameterize the structured risk calculation rather than delegating risk assignment directly to an LLM.
  • D Formal Analysis: E[C] = C1 + peC2 bounds expected matcher cost, and OPP-115 measurements yield pe = 0.035 and a 28.6× reduction versus always-LLM matching.The result follows because System 1 always runs while System 2 runs only when uncertainty exceeds the threshold.
  • D Formal Analysis: Algorithm 1 terminates within T rounds, with at most one anonymizer and one guesser call per round.The stated worst-case LLM cost is T · (Canon + Cguess), independent of x0; the explicit cap uses T = 5 and ℓmax = 5.
  • Calibration Details: Table 11 compares full AMRSF with single-factor, NIST Likelihood×Impact, and learned binary-classifier baselines across expert-rated and Staab scenarios.Table 12 separately reports the System 1 evidential fast-path classifier on OPP-115 held-out pairs.
  • Rewrite-strategy comparison: Table 13 isolates the effect of combining semantic-only and minimal-edit rewriting within GuardianAgent’s adaptive L1–L5 stack.The comparison is conducted under the verified-guesser loop.

G Robustness Detail Tables

The robustness tables test verification, variance, backbone sensitivity, error modes, fast-path bootstrapping, and context-conditioned anonymization. Together they examine whether performance and escalation behavior persist across models, seeds, and action settings.

  • Verification: Table 14’s verifier A/B test isolates verification’s contribution to the privacy/utility trade-off and escalation cost.The OFF condition reproduces raw-confidence behavior associated with Staab, HaS, and AgentStealth.
  • Error analysis: 30/40 TAB and 28/40 SynthPAI pairwise tests remain significant after Holm–Bonferroni correction at family-wise α=0.05.Table 17 characterizes four output error classes; its caption identifies hard PII-dense samples as the dominant failure mode.
  • Fast-path performance: 0.928 Accuracy, 1.000 Precision, 0.857 Recall, and 0.923 F1 are reported for the OPP-115 fast-path classifier.Its escalation rate is 3.47%, with 0.25 ms p95 latency on the fast path.
  • Context-conditioned adaptation: Action contexts vary recipient, purpose, action basis, and policy transparency while holding likelihood fixed at 0.7, attributing risk changes to context.The resulting context study reports mean final anonymization levels after verified adaptive escalation; escalation can reach L5 from a low starting level when residual leakage remains.
  • Qualitative examples: Table 23 provides qualitative anonymization examples and risk-score decisions across TAB, SynthPAI, and PII-Masking-30k.The table distinguishes counterfactual anonymizer outputs for allow and deny rows from deployment behavior.

I LLM Evaluation Protocol

The evaluation protocol combines preregistered human-style ratings, simulated Claude ratings, multiple anonymization variants, and cross-backbone checks. Reported tables cover privacy, utility, fluency, variance, statistical testing, and deployment-oriented context behavior.

  • Human evaluation: A preregistered three-rater by fifty-sample study evaluates 1,200 SynthPAI ratings across Privacy, Utility, and Fluency Likert dimensions.Each original–anonymized pair is rated independently on three 1–5 scales.
  • Evaluation results: GuardianAgent scores .941, .695, .962, and .879 in the reported evaluation row, compared with baseline rows including HaS, CONFAIDE, Pissarra, and Staab’25.The passage presents the row values without naming the corresponding metric columns.
  • Robustness checks: Four-seed variance, backbone sensitivity, and Qwen3-8B checks evaluate stability across seeds and model families.The primary backbone is Llama-3.2-3B-Instruct on one NVIDIA H100 NVL; cross-model checks use Llama-3.1-8B and Qwen3-8B.
  • Human evaluation: The human-rating protocol reports Krippendorff’s α with target ≥0.68 and paired-bootstrap significance using 1,000 resamples at p<0.05.Eight configurations include three GuardianAgent variants and five baseline methods.
  • LLM-rater simulation: The simulated LLM-rater study uses Claude Opus 4.7, Sonnet 4.6, and Haiku 4.5 under the same original–anonymized-pair protocol.Its privacy rubric penalizes destructive over-rewriting by requiring both removal of identifying details and continued usability.
  • Context and examples: Table 20 reports context-specific final anonymization levels, while Table 22 defines contexts through recipient, purpose, action basis, and transparency.Table 23 illustrates allow and deny decisions alongside qualitative examples across the three corpora.
  • LLM evaluation: Table 21 compares L1–L5 adaptive, L4 fixed, and L5 fixed GuardianAgent settings over 1,200 SynthPAI ratings with paired-bootstrap tests.The table marks comparisons against Ours (L1–L5 adaptive) at n=1000 resamples.
Loading 2608.29251v1…