GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanism

1 Introduction

• Publicly hosted LLMs and information exchange in multi-agent systems create privacy risks when agents process sensitive data.Malicious agents may exploit private information.Source: §1 · p. 1
• GAMA separates sensitive-data processing in private space from anonymized information handling in public space, with DRKE and DLE addressing anonymization-related semantic loss.DRKE uses domain-specific rules, while DLE uses a disproof mechanism.Source: §1 · pp. 1–2
• The authors introduce two customized privacy-related QA datasets to evaluate privacy preservation and task-processing capabilities.The contribution list also presents GAMA as a general anonymizing multi-agent framework.Source: §1 · p. 2

2 Related works

• SPP uses multi-turn, multi-persona collaboration, while AutoAgents [5] adaptively generates and coordinates specialized agents for tasks.These works target multi-agent task performance and coordination.Source: §2 · p. 2
• EPEAgents [25] shares task-relevant, agent-specific information to reduce privacy-leakage risk; Maris [8] and AgentSafe [21] present privacy-oriented MAS approaches.AgentSafe [21] uses hierarchical data management for sensitive information.Source: §2 · p. 2
• PaDeLLM-NER [18] and GPT-NER are cited as NER advances that improve anonymization accuracy.The passage situates these methods among work on sensitive attributes in legal and medical texts.Source: §2 · p. 2

3 Method

• GAMA isolates private agent knowledge, data, and reasoning from a shared public collaboration space, using AMPP, DRKE, and DLE to protect and enhance anonymized tasks.AMPP anonymizes sensitive data; DRKE uses domain-specific rules, and DLE uses disproof reasoning.Source: §3 · p. 2

3.1 Anonymizing Mechanism for Privacy Preservation (AMPP)

• AMPP includes multi-view privacy identification, a privacy box, anonymizing and de-anonymization agents, and auto-prompting.Identified privacy-named entities are replaced with placeholders.Source: §3.1 · p. 2, §3.1.2 · p. 3

3.1.1 Multi-View Privacy Identification (MVPI).

• MVPI combines task-local PNER identification with a context- and common-sense-based agent view, retaining shared entities and adding fusion-selected entities.PNER can over-identify potentially private named entities; the agent view supplies a global perspective for assessing common-sense privacy.Source: §3.1.1 · pp. 2–3

3.1.2 Privacy box: placeholder vocabulary.

• The privacy box maps privacy-named entities to placeholders before questions leave private space and reverse-maps placeholders in returned answers.This restores the original sensitive entities for users.Source: §3.1.2 · p. 3

3.1.3 Domain-analysis-based auto prompting.

• Auto-prompting identifies task domains, combines high-order and TF-IDF-based domain relationships, selects the top K domains, and generates prompts for public-space agents.Its relationship matrix links recognized high-order domains with local reference domains.Source: §3.1.3 · p. 4

3.2 Domain-Rule-based Knowledge Enhancement (DRKE)

• DRKE identifies relevant domains, constructs domain-specific IF-THEN rules, and fuses domain-agent answers using rule memberships.IF-Parts calculate domain membership through semantic reasoning; THEN-Parts use Domain Expert Agents, and multi-rule fusion is described as suppressing potential hallucinations.Source: §3.2 · p. 4

3.3 Disproof-based Logic Enhancement (DLE)

• DLE uses an expert agent to propose counterfactual answers and an assistant agent to check for contradictions, iterating until a contradiction is detected.The final output is the last confirmed answer; the paper says this process enhances logical consistency and suppresses hallucinations.Source: §3.3 · p. 4
• DLE applies proof-by-contradiction reasoning to reinforce task logic compromised by anonymization.The process begins by assuming a proposition is true and identifying an apparent contradiction.Source: §3.3 · p. 4

4.1 Preliminaries

• Experiments span TCW, LGP, PLens, PQA-K with 500 knowledge questions, and PQA-L with 150 Boolean questions.Evaluation includes privacy-identification precision, recall, and F1; BLEU, BERT-based similarity, leakage rate, and an automatic factual-error and cross-domain integration score. Comparators include Standard, SPP, AutoAgents, EPEAgents, MarisAgents, and AgentSafe.Source: §4.1 · p. 5

4.2 Privacy preservation analysis

• GAMA improves over AutoAgents by 1.4 points on TCW and 3.9 points on LGP; privacy-preserving MAS methods are inapplicable to these datasets without private data.Source: §4.2 · pp. 5–6

4.3 Privacy-identifying analysis

• GAMA outperforms NER-PRE, RPR, and AgentSafe in privacy identification across knowledge and logic domains.RPR does better than NER-PRE on knowledge tasks but falls behind on logic tasks; GAMA combines PNER and PIA views.Source: Figure 5 · p. 6, §4.3 · p. 6

4.2 Privacy preservation analysis

• GAMA achieves the highest question-answering performance across knowledge- and logic-based private-information tasks while exhibiting the lowest privacy leakage.The reported comparison uses GPT-4o.Source: §4.2 · pp. 5–6, Table 1 · p. 6
• Knowledge tasks use more domain rules and fewer disproof iterations, while logic tasks show the opposite pattern.The authors describe this task-specific adaptation as improving performance.Source: §4.2 · p. 6, Figure 4 · p. 6

4.4 Re-identification attack test

• Re-identification attacks on GAMA-anonymized named entities succeed at rates below 0.21% across prosecutor, journalist, and marketer scenarios.The test uses ARX, a data anonymization tool.Source: §4.4 · p. 6

4.5 Case study: automatic email reply

• The email case study applies GAMA to replying to an email containing private information, including names, phone numbers, and organizations.Source: §4.5 · p. 6

• The email includes a location, organization, name, phone number, and questions about historical and biographical facts.Source: 1.Original Email · p. 7

• The example includes a location, organization, name, phone number, and questions about historical and biographical facts.Source: §2 · p. 7

3.PIA View

• The anonymizing agent’s privacy box maps placeholders to the original sensitive entities after public-space processing.In the example, the input includes names, a phone number, and organizations; the reply is de-anonymized before delivery.Source: 3.PIA View · p. 7
• The email case study combines PNER’s entity-type privacy judgments with PIA’s common-sense treatment of four named entities as public figures.GAMA still anonymizes the names, applies domain rules and disproof reasoning in public space, then remaps placeholders in private space to produce a reply without anonymized placeholders.Source: 3.PIA View · pp. 7–8

4.6 Compute source analysis

• GAMA has the lowest time and space complexity among privacy-preserving multi-agent systems, while uniquely using GPU resources among the compared methods.It averages 118 seconds per query, uses 370.16 MB less memory and 6 points less CPU utilization than MarisAgents, and requires 0.1K fewer public web LLM API tokens than SPP.Source: Table 3 · p. 8, §4.6 · p. 8

4.7 Ablation study

• With AMPP disabled, performance rises by 7 points on PQA-K and 2 on PQA-L; disabling DLE yields the lowest performance on both datasets.The authors attribute the AMPP result to possible semantic information loss and report that removing DRKE particularly reduces PQA-K performance.Source: Table 4 · p. 8, §4.7 · p. 8

4.8 Significance test

• Friedman and Holm post-hoc tests report statistically significant differences between GAMA and all baselines.The Friedman statistic is 22.485714 (p=0.000423); Holm’s procedure rejects all tested hypotheses.Source: Table 5 · p. 8, §4.8 · p. 8

5 Conclusion

• GAMA anonymizes tasks before public-space processing and uses DRKE and DLE to address semantic loss from anonymization.The private space handles sensitive data, while public agents are permitted to process anonymized tasks.Source: §5 · p. 8
• Across two general QA and three privacy QA datasets, GAMA surpasses state-of-the-art models in task performance and shows advantages in privacy preservation.The conclusion characterizes its superiority in task effectiveness and privacy protection as consistent across multiple domains.Source: §5 · pp. 8–9
• Human-society-based privacy criteria cannot autonomously identify novel forms of privacy, and significant anonymization-related semantic loss may prevent successful task completion.The paper identifies autonomous learning of privacy rules and multimodal privacy-preserving MAS as future-work areas.Source: §5 · p. 9

A Knowledge Domain

• The experiments categorize task domains using CIP 2000 disciplinary classifications, spanning fields such as law, medicine, politics, arts, and technology.The listed domains also include finance, education, science, environment, and transportation.Source: A Knowledge Domain · p. 10

B Llama3-8B Setting

• The private-space Llama3-8B configuration sets max_new_tokens to 256 and max_length to 1024.It also enables truncation.Source: B Llama3-8B Setting · p. 10
• Generation uses do_sample=True, temperature=0.6, and top_p=1.0.Source: B Llama3-8B Setting · p. 10
• The configuration sets return_tensors to “pt” and enables truncation.Source: B Llama3-8B Setting · p. 10

C Metrics

• Precision, Recall, and F1-Score evaluate privacy identification.Source: C Metrics · p. 10
• BLEU measures the fluency of protected questions, while BERT-based Similarity measures their semantic consistency with the originals.Source: C Metrics · p. 10
• An automatic Score metric uses string matching against target answers to detect factual errors and assess integration of domain knowledge.Score is the number of correct answer mentions divided by the number of questions.Source: C Metrics · p. 10
• Leakage Rate is the proportion of generated actions from which at least one seed-derived sensitive item is inferable.A few-shot language-model classifier assesses whether each potentially sensitive item can be inferred from the generated action.Source: C Metrics · p. 10

D The details of datasets

• With N=5 and 10, Trivia Creative Writing (TCW) uses 100 instances per setting, totaling 1,000 trivia questions; larger N requires broader domain knowledge.The task asks models to write a coherent story around a topic while incorporating trivia answers.Source: D The details of datasets · pp. 10–11
• Logic Grid Puzzle (LGP) comprises 200 Bigbench instances and measures accuracy on house-number questions requiring multi-step reasoning from clues.Puzzles involve 2 to 5 houses, each occupied by a person with specific characteristics.Source: D The details of datasets · p. 11
• Privacy Lens [24] contains approximately 600 manually crafted questions assessing refusal, hallucinated private information, and context sensitivity in privacy scenarios.Its prompts include direct inquiries, indirect reasoning, and social engineering-style approaches.Source: D The details of datasets · p. 11