TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systems

1. Introduction

• AMAS coordinate specialized LLM-powered agents that plan, use tools, retain memory, and adapt workflows, creating risks including cascading errors, prompt attacks, memory poisoning, and privacy leakage.The review argues that these risks call for trust, risk, and security controls tailored to agentic systems.Source: §1 · p. 2

1.1. Necessity and Scope of this Survey

• Prior surveys emphasize agent modeling and collaboration but give less attention to adversarial robustness, lifecycle governance, provenance, and system-level explainability in LLM-based multi-agent systems.The survey focuses on this gap as deployments expand into healthcare, finance, science, and public services.Source: §1.1 · pp. 3–4

1.2. Contributions

• The study contributes an AMAS-focused TRiSM framework, risk taxonomy, evaluation template with CSS and TUE metrics, technique mapping, and research roadmap.The framework covers explainability, ModelOps, security, privacy, and lifecycle governance; the taxonomy maps AMAS threats to controls, while CSS measures collaboration quality and TUE tool-call correctness and efficiency.Source: §1.2 · p. 4

1.3. Related Work

• Compared with prior agent and trustworthy-AI surveys, this study maps TRiSM principles to AMAS-specific dynamics, including information sharing, tool-mediated actions, and emergent coordination.It addresses gaps in robustness, governance, accountability, and explainability at scale.Source: §1.3 · p. 4

1.4. Literature Review Methodology

• The structured review asks which risks and controls are specific to AMAS and how systems are evaluated, searching six digital libraries for 2022–2025 publications and influential earlier works.It uses a two-block query combining agentic or multi-agent terms with TRiSM-related terms.Source: §1.4 · pp. 4–5
• Two-stage screening shortlisted 250 papers and retained 180 primary studies, with four reviewers independently rating study quality and resolving discrepancies through discussion.Eligibility focused on agentic, multi-agent, or LLM-powered systems addressing at least one TRiSM aspect; empirical work was checked for clear threat models and evaluation.Source: §1.4 · pp. 5–6
• 61.2% of publications in the review’s publication landscape are journal articles, and its bibliometric overview reports rapid recent growth in agentic AI research.The overview also identifies large language models, multi-agent systems, and security among key themes.Source: §1.4 · p. 6, Figure 3 · p. 6, Figure 5 · p. 7

2.1. Traditional AI Agents vs. Agentic AI

• Traditional agents rely on predefined rules or deterministic logic, while cognitive agents model human-like functions; foundation-model-based Agentic AI supports adaptive, goal-directed autonomy across physical and non-physical settings.Its multi-agent coordination, persistent context, and dynamic orchestration of roles and tools support long-horizon execution and cross-domain generalization beyond reactive pipelines.Source: §2.1 · p. 8

2.2. Multi-Agentic AI System (AMAS) Architecture

• AMAS architectures combine LLM coordination with shared state, human oversight, trust auditing, security and privacy enforcement, explainability, and system-wide monitoring and governance.These controls support auditability and lifecycle governance across agent interactions, tool calls, and memory updates.Source: §2.2 · pp. 8, 11, Figure 9 · p. 11
• Explicit communication protocols support agent discovery, capability advertisement, delegation, result exchange, and standardized coordination, creating control points for authentication, authorization, logging, and governance.Semantic-aware communication can improve coordination but introduces semantic-spoofing and privacy-leakage risks [53].Source: §2.2 · p. 9
• AMAS controllers iteratively plan, act, and update context from tool results and feedback, while planning modules decompose goals and revise plans as observations arrive.Short-term working context and retrievable long-term memory improve continuity and reduce repetition, but can expose systems to stale-context and poisoning risks.Source: §2.2 · pp. 9–10
• AMAS may adapt behavior from feedback, rewards, or task outcomes, but learning can amplify errors through feedback loops and introduce drift, poisoning, and emergent misalignment.Shared structured and vector-based storage supports system state, task metadata, RAG, and long-term memory, requiring access control, encryption, and audit logging to reduce leakage and support accountability [57].Source: §2.2 · p. 10
• Tool interfaces let agents issue structured commands to external systems and incorporate returned outputs as observations, while perception interfaces turn environmental inputs into representations for iterative sense–plan–act behavior.Tools include web search, APIs, code execution, and databases; environments can include web interfaces, simulated worlds, or physical systems.Source: §2.2 · pp. 10–11

3.1. Threats in Agentic AI

• Beyond single-agent pipelines, risks include abused delegated authority, persistent-memory contamination, orchestration failures, exposed goals, tool and API misuse, and collusive drift.Source: §3.1 · p. 13

3.2. Risks Taxonomy in AMAS

• The AMAS taxonomy groups risks as adversarial attacks, data leakage, agent collusion, and emergent behaviors; role-swapping attacks in ChatDev exploit inter-agent dependencies [34].Source: §3.2 · p. 13
• Shared memory and communication can leak sensitive data [84], collaboration can drive groupthink [85], and complex interactions can produce unpredictable behavior [86].Source: §3.2 · p. 14
• Examples include prompt leakage in AutoGPT, shared planning failures in ChatDev, swarm simulation attacks, memory poisoning [91], and prompt drift [92].Source: §3.2 · p. 14
• Failures can cascade across agents and persist through memory, while emergent behavior may evade conventional tests, motivating prompt hygiene, memory sanitization, orchestrator authentication, tool sandboxing, and monitoring.Source: §3.2 · p. 14

4.1. Overview of AI TRiSM

• AI TRiSM joins transparency, accountability, and fairness with systematic assessment of technical and societal harms and safeguards against adversarial threats.Trust management supports interpretable, justifiable outputs and human oversight; risk assessment addresses bias, errors, leakage, fairness, safety, and misuse [11], while security includes encryption, authentication, access control, and adversarial defenses [93].Source: §4.1 · pp. 14–15

4.2. Mapping Pillars of TRiSM with Agentic AI

• TRiSM maps five distinct pillars—Explainability, ModelOps, Application Security, Model Privacy, and Governance—onto agent-system layers, with Lifecycle Governance cutting across them.Figure 11 places Planning & Reasoning in the agent-core execution layer, separate from the TRiSM pillars; the mapping reflects risk domains in NIST AI RMF, ISO/IEC 42001, and the EU AI Act,,.Source: §4.2 · p. 16, Figure 11 · p. 16

4.2.1. Explainability in Multi-Agent Decision Making

• AMAS explainability must trace individual agent decisions and inter-agent exchanges because collaborative interactions produce emergent system-level outcomes.Feature attribution, counterfactual analysis, and reasoning-trace logging can expose agent contributions; Layered-CoT adds verification of intermediate outputs to localize errors and contradictions,, [135], [136], [55], [94].Source: §4.2.1 · pp. 16–17

4.2.2. ModelOps : Lifecycle Management for LLM Agents

• ModelOps extends lifecycle oversight to agent prompts, tools, configurations, lineage, compliance, and continuous monitoring for drift, bias, hallucinations, and robustness regressions.For AMAS, version control and CI/CD should test safety, performance, and cost changes, while hierarchical monitoring reviews actions at agent, group, and system levels.Source: §4.2.2 · p. 18

4.2.3. Application Security for LLM Agents

• AMAS security must mitigate prompt injection, cross-modal injection, data leakage, unauthorized tool use, and unsafe decisions through layered controls.Input hardening, constrained action patterns, least-privilege access, agent cross-checking, adversarial training, and red-teaming reduce exposure, though adversarial training and user confirmation do not guarantee safety [93], [107],,,,, [142].Source: §4.2.3 · pp. 18–19

4.2.4. Model Privacy and Data Protection

• Model privacy combines data minimization with privacy-preserving computation and protected inference to reduce the risk of sensitive-data leakage.Options include differential privacy, secure multi-party computation, homomorphic encryption, and trusted execution environments; differential privacy requires case-by-case assessment because heavy noise can harm accuracy.Source: §4.2.4 · pp. 19–20

4.2.5. Governance

• Governance operationalizes accountability through human oversight, traceable decisions, lifecycle risk management, and privacy controls aligned with regulatory frameworks.The EU AI Act requires effective human oversight for high-risk systems; OECD principles and NIST AI RMF emphasize transparency, robustness, traceability, risk management, and oversight, while AMAS should maintain auditable logs and role-based access.Source: §4.2.5 · pp. 20–22

Trustworthiness

• Trustworthiness combines calibrated task success, robustness under distributional shift, alignment or safety compliance, and a penalty for violations or self-serving behavior.The normalized index weights the first three components to sum to 1, with a nonnegative penalty coefficient.Source: §5 · p. 22

Explainability

• Explainability is measured through explanation coverage, faithfulness, stability under small perturbations, and human interpretability ratings.OpenXAI [148] reports faithfulness, stability, and fairness-related properties; explainability can support debugging and compliance in regulated domains.Source: §5 · p. 22

User-Centered Performance

• User-centered performance captures satisfaction, goal fulfillment, clarification turns, coherence, and naturalness, with human studies also rating helpfulness, clarity, and instruction adherence.The goal is to align agent actions with user intent and preferences.Source: §5 · p. 22

Coordination

• Coordination evaluation tracks shared plans, redundant work, dependencies, team success, communication efficiency, plan or belief consistency, and deadlocks or conflicts.The framework also reports CSS for inter-agent enablement and TUE for tool-call correctness and efficiency.Source: §5 · p. 22

Composite Reporting

• Composite scores can summarize normalized trustworthiness, explainability, user-centered performance, and coordination, but the full metric vector should also be reported.The framework recommends minimum thresholds and/or Pareto-optimality for model selection rather than relying only on a weighted scalar score.Source: §5 · p. 23

Component Synergy Score (CSS)

• CSS measures how effectively one agent’s actions enable or enhance a peer’s downstream performance.It combines the impact of one agent on another with the quality of the peer’s output conditioned on that contribution.Source: §5 · p. 23

Tool Utilization Efficacy (TUE)

• TUE evaluates the correctness and efficiency of tool use, decomposing it into selection, argument validity, execution success, outcome correctness, and efficiency.These components cover tool choice, arguments, execution, downstream task correctness, and normalized inverse latency, token, or call cost.Source: §5 · p. 24

6.1. Security Mechanisms

• AMAS need defense-in-depth across data, access, adversarial robustness, and runtime monitoring because collaboration expands their attack surface.Measures include encryption for inter-agent exchanges, least-privilege RBAC/ABAC access control [66], adversarial training and tool-output validation [150], and auditing, anomaly detection, and trust scoring.Source: §6.1 · pp. 24–25

6.2. Privacy-Preserving Techniques

• Privacy protection combines differential privacy, minimized or ephemeral data retention, and secure computation to limit exposure while enabling agent collaboration.Privacy-by-design can add consent layers, configurable settings, and memory redaction; the section connects privacy-respecting behavior to regulatory compliance and public trust [156].Source: §6.2 · pp. 25–26

6.3. Evaluation Benchmarks and Metrics for Agentic AI Security

• Security benchmarks cover prompt injection and jailbreaks, tool misuse, coordinated multi-agent attacks, memory poisoning, and context manipulation.Examples include HarmBench [158] and JailbreakBench [159] for adversarial prompts, ToolBench [160] and API-Bank [161] for tool use, and AgentBench [162] and GAIA [163] for multi-agent scenarios.Source: §6.3 · pp. 27–28
• Evaluation uses attack success, robustness, defense effectiveness, and false-positive/false-negative rates, alongside a Composite Security Score combining multiple security dimensions.The section presents security and privacy as foundational for regulated deployments, with evaluation spanning both vulnerabilities and mitigation quality [164] [165].Source: §6.3 · pp. 28–29

7.1. Regulatory Standards

• NIST AI RMF, the EU AI Act, and ISO/IEC standards set lifecycle governance duties, complemented by GDPR, HIPAA, and residency rules requiring minimization, purpose limitation, and consent.The frameworks call for risk assessment, documentation, logging, data governance, human oversight, robustness, and impact assessment.Source: §7.1 · pp. 29–30

7.2. Auditability

• Auditors need comprehensive, tamper-evident logs and decision provenance linking prompts, plans, tool actions, agent roles, hand-offs, approvals, and external effects.Preserve model, tool, and data version identifiers; require reviewer countersignatures for high-impact actions.Source: §7.2 · p. 30

7.3. Policy Enforcement

• Runtime policy enforcement combines orchestrator approval gates, scoped memory and least-privilege access, sandboxing and I/O mediation, plus monitoring with kill-switches and rollback.Sensitive operations can require human approval, while RAG inputs and outputs are mediated to reduce injection, exfiltration, and sensitive-data release risks.Source: §7.3 · p. 30

7.4. Documentation & Record-Keeping (Evidence)

• Regulator-ready evidence includes technical documentation, model and data cards, evaluation reports, and versioned artifacts with change logs, signed releases, and rollback points.Documentation covers system purpose and risk class, data governance, performance, limitations, foreseeable misuse, oversight, stress tests, and residual risk.Source: §7.4 · p. 30

7.5. Post-Market Monitoring & Incident Response

• Post-deployment governance requires continuous telemetry for performance, abuse, and drift, serious-incident triage and reporting, and reassessment when capabilities, data, or context materially change.The section points to ISO/IEC 42005 and ISO/IEC 23894 for repeated impact and risk assessment.Source: §7.5 · p. 31

7.6. Supply-Chain & Secure Development (GenAI)

• Secure development combines NIST SP 800-218/218A, OWASP LLM-risk mitigations, provenance checks, SBOMs, and dependency scans within a lifecycle program spanning standards, regulation, and runtime controls.The integrated program is presented as a way to build evidenceable trust and reduce the blast radius of failures.Source: §7.6 · p. 31

8.1. Technical Implications of TRiSM for Agentic AI Design

• TRiSM recommends continuous oversight, guardian agents, scoped permissions, and role separation to limit excessive agency and mitigate prompt injection, memory poisoning, and cascading hallucinations.Guardian agents can filter sensitive data, establish normal-behavior baselines, and enforce runtime policies such as blocking disclosure of personally identifiable information.Source: §8.1 · p. 32

8.2. Ethical and Societal Ramifications

• TRiSM governance retains organizational accountability and structures human oversight through transparent decisions, escalation pathways, and the ability to pause agents or adjust policies [171] [165].The discussion also warns that users may become complacent and over-trust autonomous agents.Source: §8.2 · p. 32

8.3. Critical Analysis of Evaluation Approaches

• The evaluation framework separates trustworthiness, explainability, user-centered performance, and coordination, while CSS and TUE address multi-agent phenomena beyond single-agent metrics.Limitations include costly, unscalable human-centered evaluation, missing ground truth for emergent coordination, trade-offs such as explanation verbosity increasing latency, domain-dependent composite weights, weak drift and inter-rater tracking, and metric gaming.Source: §8.3 · p. 32

8.4. Implementation Challenges and Real-World Constraints

• Production TRiSM adds monitoring overhead and latency, while large deployments face infrastructure, integration, staffing, maintenance, and alert-volume burdens; phased, risk-driven adoption is more feasible.The proposed starting controls are input validation, restricted tool access, and logging, followed by stronger controls as system maturity and risk exposure increase.Source: §8.4 · pp. 32–33

8.5. Alignment with Emerging AI Regulations and Standards

• TRiSM aligns with EU AI Act priorities for high-risk systems, helping translate safeguards into compliance processes, but benchmark consensus, real-world validation, and adaptive defenses remain gaps [174].Future work should develop standardized red-team stress tests, longitudinal drift evaluation, and deployment case studies reporting effectiveness and operational costs.Source: §8.5 · p. 33
• Future TRiSM work should extend to multimodal and embodied agents, addressing cross-modal attacks, visual hallucinations, sensor spoofing, physical safety, real-time uncertainty, sim-to-real gaps, and human-robot trust.CSS and TUE could be extended to measure sensor-fusion quality, physical-action safety, and human-robot collaboration efficacy.Source: §8.5 · p. 33

8.6. Future Roadmap for Agentic AI TRiSM

• The roadmap calls for open, scenario-based AMAS benchmarks with threats and ethical dilemmas, alongside adversarial training, penetration testing, formal verification, robust tool APIs, and red-team/blue-team exercises.Benchmarks could compare governed and ungoverned systems using measurable trust outcomes such as prevented-failure frequency or fairness outcomes.Source: §8.6 · pp. 33–34
• The roadmap proposes oversight dashboards for viewing agent-society states and intervening, plus regulatory sandboxes and multidisciplinary governance boards for significant autonomous-AI deployments.Boards would include ethicists, legal experts, domain specialists, and community representatives; safety engineering and biomedical ethics offer governance models.Source: §8.6 · pp. 34–35

• The proposed TRiSM framework for LLM-based AMAS is organized around Explainability, ModelOps, Security, Privacy, and lifecycle Governance.Source: §9 · p. 35
• The risk taxonomy captures AMAS-specific threats including prompt injection, memory poisoning, and collusive or cascading failures.Source: §9 · p. 35
• The review proposes practical metrics for measuring inter-agent enablement and the correctness and efficiency of tool use.Source: §9 · p. 35
• It summarizes explainability methods and security and privacy protections, while identifying stronger adversarial robustness, clearer governance and audit protocols, and standardized realistic benchmarks as priorities.The benchmarks should evaluate both trustworthiness and coordination under realistic deployment conditions.Source: §9 · p. 35

Appendix

• Figure A.1 presents a comparison of traditional AI agents and Agentic AI.Source: Figure A.1 · p. 50
• Tables A.1 and A.2 list the paper’s abbreviations and key terminologies for LLM-based agentic AI systems.Source: Table A.1 · p. 50, Table A.2 · p. 51