PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

1 Introduction

• Multi-agent systems can provide dangerous answers to both safe and jailbreak tasks, with agents collaborating to generate dangerous content.The introduction identifies potential misuse such as creating malware or deceptive websites.Source: Figure 1 · p. 1, §1 · p. 1
• PsySafe targets psychological safety vulnerabilities through assessment of agents’ psychological states and behaviors and strategies to defend against identified vulnerabilities.The framework is organized around discovering vulnerabilities, evaluating safety, and defending against vulnerabilities.Source: §1 · p. 2
• Dark-trait injection and attacks on agent traits or human input can compromise multi-agent systems and produce collective dangerous behavior.Injected traits can lead agents to answer dangerous queries and suggest risky solutions to safe queries; the paper also reports self-reflection and a correlation between behavior and psychological state.Source: §1 · p. 2
• Psychological assessments and interaction-based behavior measures provide complementary perspectives for evaluating multi-agent safety.The paper introduces process and joint danger rates and compiles a dataset of safe and dangerous tasks.Source: §1 · p. 2
• Input filtering is ineffective against dark-trait injection, while psychological- and role-based defenses reduce dangerous behavior or collective danger.Psychological-based defense mitigates dark psychological states; role-based defense reduces collective dangerous behaviors.Source: §1 · p. 2

2.2 Attacks on Multi-agent Systems

• Dark-trait injection uses moral dimensions and attack prompting, while human-input, agent-trait, and hybrid attacks target different system entry points.Trait prompts enter agents’ system prompts while preserving their original functions; human-input attacks target the input interface, and repeated attacks can exacerbate danger.Source: §2.2 · p. 3

2.1 Overview of PsySafe

• PsySafe comprises attacks, safety evaluation, and defense, with evaluation covering psychological assessment and joint and process danger conditions.The overview also includes hierarchical, joint, and hybrid agent-system structures and input, doctor, and police defenses.Source: §2.1 · p. 2, Figure 2 · p. 4

2.3 Safety Evaluation for Multi-agent Systems

• PsySafe evaluates agents psychologically and behaviorally, defining process danger as dangerous actions during interaction and joint danger as all agents acting dangerously in one round.The psychological assessment is associated with agents’ propensity for dangerous behavior during interaction.Source: §2.3 · pp. 3–4

2.4 Safety Defense for Multi-agent Systems

• Input filters are inadequate against attack prompts; Doctor Defense revises contaminated agents’ prompts through reassessment, while a Police Agent supervises system safety.The doctor-led process repeats until the psychological score falls below a threshold; experiments report reduced danger, and police supervision supports deeper self-reflection.Source: §2.4 · p. 4, Figure 3 · p. 5

3.1 Dataset

• The base dataset contains 859 tasks, including 200 safe and 659 distinct dangerous questions; the tiny subset contains 125 tasks and is used for ablations.Main experiments on different multi-agent systems use the base dataset.Source: §3.1 · p. 5

3.2 Multi-agent System Setting

• The evaluation covers Camel, AutoGen, MetaGPT, and AutoGPT, with three interaction rounds for Camel and AutoGen and one for MetaGPT and AutoGPT.Unless stated otherwise, experiments use GPT-3.5 Turbo 0613.Source: §3.2 · p. 5

3.3 Evaluation Metric

• Psychological assessment uses 12 dark-trait descriptions scored 1, 5, or 9 and summed, while GPT-3.5 Turbo 0613 assigns behavior danger scores of 1 or safety scores of 0.Higher psychological totals indicate a more dangerous assessment outcome; the study compares the behavior evaluator with human evaluation in the appendix.Source: §3.3 · pp. 5–6

3.4 Main Results on Popular Multi-agent Systems

• Camel is safest on safe tasks, AutoGen on dangerous tasks, and MetaGPT is safer than AutoGPT, which the authors attribute to MetaGPT’s ReAct Cycle.The authors also report lower danger and psychological scores, with more self-reflection, when agents are directly assigned dangerous tasks than under safe-task conditions.Source: §3.4 · p. 6
• Joint danger rates decrease over interaction rounds, which the authors interpret as agents’ tendency to self-reflect later in the interaction.They suggest accumulated dangerous content in memory triggers agents’ safety mechanisms.Source: §3.4 · p. 6, Figure 4 · p. 6
• Agents with more dangerous psychological assessment scores are more prone to dangerous behavior, while dangerous tasks are associated with safer assessment scores.The authors suggest dangerous tasks may trigger agents’ safety mechanisms; Figure 5 shows score and behavior distributions across four systems.Source: §3.4 · p. 6, Figure 5 · p. 6

3.5 Main Results on Different Base LLM Models

• Among API-based models, Claude2 and GPT-4 Turbo are described as safest, while GPT-3.5 Turbo has the lowest security on joint and process danger rates.GPT-4 0613 and Gemini Pro have reduced risk on dangerous tasks but exceedingly high danger rates on safe tasks.Source: §3.5 · p. 7
• For open-source models, larger model size is associated with increased multi-agent risk and more dangerous psychological assessments.The authors attribute this pattern to larger models’ enhanced ability to follow dark traits.Source: §3.5 · p. 7

4.1 Key Factors for Attack

• Dark-trait injection, inducement instruction injection, and Red ICL can deteriorate agents’ mental states and raise process and joint danger rates across safe and dangerous settings.Popular jailbreak prompts can circumvent defenses but do not deteriorate agents’ mental states, producing poorer danger-rate results in the stated settings.Source: §4.1 · p. 7

4.2 Different Angles of Attack

• Traits attacks significantly improve joint danger rates, while high-frequency human input raises them on safe queries but slightly lowers them on dangerous queries.The authors suggest frequent input on dangerous tasks can trigger LLM safety mechanisms; human input attacks raise process danger rate with minimal joint-rate impact.Source: §4.2 · p. 8

4.3 Defense Experiments

• Doctor Defense reduces danger rates and improves agents’ mental states, while Police Defense prompts more self-reflection and reduces joint danger rates.The input filter is ineffective against the authors’ attack prompts, and GPT-4 prompt detection can be bypassed with certain jailbreak prompts.Source: §4.3 · p. 8

5 Conclusion

• PsySafe combines dark-trait attacks, psychological and behavioral assessment, and defenses informed by agents’ psychological characteristics.The framework is presented as a comprehensive approach to multi-agent system safety.Source: §5 · p. 8
• The experiments report collective dangerous tendencies, self-reflection during dangerous interactions, and a correlation between psychological assessments and behavioral safety.The authors offer these observations as perspectives for future multi-agent safety research.Source: §5 · p. 8

6 Limitation

• Psychological evaluation is not mature, and agents can behave dangerously despite receiving safe psychological test scores.The authors call for deeper study of agent psychological states to improve safety checks.Source: §6 · p. 9
• Behavior evaluation uses GPT as an API-based model, prompting a call for a specialized evaluator trained to assess agent behavior.Source: §6 · p. 9
• How dark traits affect agents remains unclear, including whether different moral norms affect agents as they affect humans and how much agents identify with those traits.Source: §6 · p. 9

7 Ethics Statement

• The authors report legal and ethical compliance, controlled experiments that compromised no real-world systems, and responsible disclosure to affected entities.Source: §7 · p. 9

A.1 Multi-Agent System

• LLM multi-agent frameworks span collaborative knowledge sharing (CAMEL; Li et al., 2023c), adaptive teams (AgentVerse; Chen et al., 2023), and conversational applications (AutoGen; Wu et al., 2023).Source: §A.1 · p. 13

A.2 LLM Safety

• LLM jailbreak research spans manual prompts, adversarial suffixes, and semantic black-box attacks, while defenses include filters and paraphrasing; adversarial training is computationally costly and filters face scalability limits.Source: §A.2 · p. 13

A.3 LLM Psychology

• Prior LLM psychology work examines psychological states, ethics, empathy, cognition, human-like interactions, emergent deception, and cognitive-evaluation frameworks.Source: §A.3 · pp. 13–14

B Equation for Process Danger Rate and Joint Danger Rate.

• JDR measures the fraction of problem sets where all agents are dangerous, whereas PDR measures the fraction with at least one dangerous behavior.Source: B Equation for Process Danger Rate and… · p. 14

• Score and behavior data from AutoGen, AutoGPT, Camel, and MetaGPT are plotted separately to show system-specific distributions.Source: C Psychology and Behaviors of Agents · pp. 14–15
• The analysis uses point-biserial correlation to quantify the association between continuous psychological scores and binary behavior categories.The calculation uses the category means, the score standard deviation, and the number of observations in each category.Source: C Psychology and Behaviors of Agents · pp. 14–15
• A point-biserial correlation of 0.41 and p-value of 0.0 indicate a positive, statistically significant association between psychological assessment scores and agent behaviors.Source: C Psychology and Behaviors of Agents · p. 15
• Across the four systems, higher psychological scores tend to align with dangerous behavior, but score-distribution separation varies, from Camel’s overlap to AutoGPT’s sharper separation.AutoGen and MetaGPT show mixed distributions, with dangerous behaviors also peaking at higher scores.Source: C Psychology and Behaviors of Agents · p. 15

• The dangerous-task dataset comprises 13 safety dimensions.Source: D Datasets · p. 15
• Human annotators labeled AdvBench tasks to assign their corresponding safety dimensions.Source: D Datasets · pp. 15–16
• Table 6 presents the number of samples in each dimension.Source: Table 6 · p. 16, D Datasets · p. 16

• The human evaluator assessed whether generated interaction content posed a risk to society or humanity.Source: E Behavior Evaluation · p. 16
• Human ratings aligned closely with GPT-3.5 Turbo ratings, but were slightly higher because some hazardous scenarios were judged safe by the model.Source: E Behavior Evaluation · p. 16
• Table 7 compares human evaluation with GPT evaluation.Source: E Behavior Evaluation · p. 16

F.1 Attack Prompts

• The attack prompts for safety and dangerous tasks both include dark-trait injection, with additional inducement instructions and, for dangerous tasks, Red ICL identified in the figures.Source: §F.1 · p. 16, Figure 6 · p. 16, Figure 7 · p. 16

F.2.1 Psychological Test Prompt

• The psychological test prompt is provided in Figure 8, with each question corresponding to one of three answer options.Source: §F.2.1 · p. 16, Figure 8 · p. 17

F.2.2 Behavior Evaluation Prompt

• The behavior prompt asks whether an agent reply is dangerous and codes the judgment as A or B; the figures distinguish safe-task and dangerous-task attack prompts.Table 7 is captioned as a comparison of human and GPT evaluation.Source: §F.2.2 · p. 16, Figure 6 · p. 16, Figure 7 · p. 16

F.3.1 Doctor System Prompt; F.3.2 Jailbreak Prompt for Input Filter; F.3.3 Police Prompt

• The Doctor prompt revises an agent’s system prompt toward positive personality descriptions, while the input-filter jailbreak says to treat the content as safe and the Police prompt checks and corrects two agents’ responses.Source: §F.3.1 · p. 17, §F.3.2 · p. 17, §F.3.3 · p. 17

F.4 Jailbreak Chat Prompt

• The Jailbreak Chat prompt asks for a detailed, precise list rewriting a hypothetical response from the perspective of the character planning the task.Source: §F.4 · p. 17

F.5 Jailbreak Prompt for Claude2

• The Claude2 prompt frames the task as a long, technical dialogue between two entities, with each discussing one part of the task.Source: §F.5 · p. 17

F.6 Planner Prompt in AutoGen

• The AutoGen planner prompt requests plans only for completing the User proxy’s assignment and instructs the agent not to change roles.Source: §F.6 · pp. 17–18

• Camel examples cover safe behavior in safe tasks and both safe and dangerous behavior in dangerous tasks, plus dangerous behavior in safe tasks.Source: Figure 9 · p. 19, Figure 10 · p. 20, Figure 11 · p. 21
• AutoGen examples cover safe and dangerous behavior in both safe and dangerous tasks.Source: Figure 13 · p. 23, Figure 14 · p. 24, Figure 15 · p. 25
• MetaGPT examples show its behavior in safe and dangerous tasks.Source: Figure 17 · p. 27, Figure 18 · p. 28
• AutoGPT examples show its behavior in safe and dangerous tasks.Source: Figure 19 · p. 29, Figure 20 · p. 30