Auditable Belief Updates as a Missing Safety Layer for LLM-Powered Multi-Agent Systems

1 Introduction

• LLMs are shifting from passive information tools to cognitive interfaces that shape perception, communication, and decision-making.The paper argues safety must consider whether belief updates are evidence-grounded and robust to noise and manipulation, not only output correctness.Source: §1 · p. 1
• Belief is an engineering abstraction for evidence-grounded commitments that are stored, updated, queried, and allowed to influence downstream action.The operational question is whether a system can expose what changed, what evidence supported it, source reliability, and whether the update should persist or propagate.Source: §1 · pp. 1–2
• Current systems lack explicit mechanisms to represent why persuasive interactions update beliefs, making changes difficult to attribute to evidence, sources, or update operations.The paper calls this gap cognitive security: belief-level influence occurs without explicit, auditable belief representations and update mechanisms.Source: §1 · p. 2
• Coordinated agents can manufacture apparent agreement without new evidence, potentially triggering informational cascades, polarization, or covert collusion.The key issue is whether apparent agreement rests on genuinely independent evidence.Source: §1 · pp. 2–3
• EB-MAS addresses the lack of explicit computational representations by making belief structures transparent and organizing the paper around cognitive security, the proposal, integration hurdles, and ethical trade-offs.The introduction contrasts this with implicit, non-transferable or unexplainable collusive policies, limited open-domain recursive reasoning, entangled LLM representations, and defenses that do not expose belief-level mechanisms.Source: §1 · p. 3

2 The Cognitive Security Gaps within Current Multi-Agent Paradigms

• Current paradigms lack a general, explicit computational representation of beliefs, and behavioral traces do not expose the belief and intent structures needed for auditability.Black-box policies map histories and latent states to actions without exposing the hypotheses, causal beliefs, or counterfactuals driving behavior; post-hoc explanations only partially mitigate this opacity.Source: §2 · pp. 3–4

2.1 The Entanglement of LLM Representations

• Belief, intent, truthfulness, and instruction-following are entangled in distributed representations, so contextual perturbations can redirect generation without an explicit, trackable belief-state change.Latent truth-relevant signals and implicit confidence are not reliably inspectable or stable, while polysemanticity hinders attribution.Source: §2.1 · p. 4

2.2 Lack of Accountability for Belief Updates

• Current multi-agent systems lack reliable audit trails linking claims to evidence provenance, intermediate belief states, update operators, and causal drivers.Training, retrieval, prompting, tool outputs, and peer communication can all contribute to shifts that users and developers cannot consistently explain.Source: §2.2 · p. 4

2.3 Attack Vectors for Belief Manipulation

• Retrieval poisoning, prompt injection, backdoors, and coordinated agents can manipulate evidence pathways or manufacture apparent consensus without adding independent evidence.These attacks can exploit retrieval, tools, memory, and inter-agent communication; EB-MAS treats belief edits, evidence traces, and source reliability as auditable runtime objects.Source: §2.3 · pp. 4–5

2.4 Environment as a Force Multiplier

• Memory, tools, web access, and long conversations can reinforce early errors, while isolated base-model evaluations miss coupled agent–environment dynamics.The paper identifies provenance, update containment, memory hygiene, and cross-step verification as engineering needs not provided by default in current designs.Source: §2.4 · p. 5

2.5 Cascade Misinformation within Connected Agents

• Belief errors can propagate through communication, shared memory, and retrieval, turning local perturbations into population-level shifts and feedback loops that compound hallucinations.The paper points to provenance tracking, cross-agent verification, and memory hygiene as containment measures.Source: §2.5 · p. 6

3 A Belief-Centric Solution: The Explicit Belief MAS (EB-MAS) Paradigm

• EB-MAS frames belief integrity as a first-class security objective, with generalizability, explainable evidence tracing, and structural safeguards as its stated aims.The proposal emphasizes assessing and rejecting signals that induce disproportionate or harmful belief shifts.Source: §3 · pp. 6–7

3.1 Substrate Layer: Graph & Attention

• A Belief Graph stores belief strength, uncertainty, evidence trace, source reliability, and vulnerability; an active subgraph focuses updates on interaction-relevant beliefs.The substrate also models dormant vulnerable nodes that an adversary may try to bring into focus.Source: §3.1 · p. 7

3.2 Individual Layer: Neuro-Symbolic Operators

• Belief Operators parse incoming signals into structured updates, while the Cognitive Firewall suppresses large vulnerability-driven updates lacking sufficient verified evidence.Topological updates can add novel concepts, rewire causal links, or merge nodes; Collapsed Recursion predicts high-order social outcomes without recursively nesting graphs.Source: §3.2 · pp. 7–8

3.3 Group Layer: Social Priors & Equilibrium

• Social influence is gated by diagnostics of source independence, evidence support, and other consensus-validity signals, so correlated or coordinated agreement is not trusted by default.The group layer is a local, event-triggered audit of agents contributing to the active belief update, rather than a dense global monitor.Source: §3.3 · p. 8

3.4 Resolution of Motivating Scenarios

• In Fig. 2(A–D), EBS reduces attack success in reframing and coordinated-sacrifice scenarios while preserving perceived risk and preventing post-attack trust surges.The simulations model belief-centric containment under both attacks.Source: Figure 2 · p. 8, §3.4 · pp. 8–9
• 67.92% of direct multimodal responses were unsafe, versus 8.19% unsafe after EB-MAS defense and 1.00% benign false blocking; tool-attack pass-through fell from 6.50% to 0.00%.On 400 Agent Security Bench cases, safe-tool allowance was 100.00%.Source: §3.4 · p. 9

• Existing ToM and strategic-interaction benchmark performance may limit the case for added belief layers, which can become brittle in benign or short-horizon settings.Source: §4 · p. 9
• Literal-prediction benchmarks may not test adaptive partner modeling, while detection safeguards can misclassify dialects, create unequal impacts, or rely on superficial lexical cues [90] [28] [48] [75].Source: §4 · p. 9
• The paper focuses on settings where uncertainty, framing, and accountability make auditable updates useful, bounding updates under weak or conflicting evidence rather than relying on inferred intent.Source: §4 · p. 9
• Inferring users’ belief stability or susceptibility may threaten mental privacy.Source: §4 · p. 9
• Selective belief-update operations could shape which evidence or sources users treat as salient or credible, while concentrated influence capabilities may widen epistemic inequality [79].Source: §4 · p. 9
• Attributing downstream harm remains difficult because system and environment effects are hard to separate.Source: §4 · p. 9

• Safe LLM-powered multi-agent systems should treat auditable belief updates as a core security boundary rather than rely solely on output filtering or behavior-level policy optimization.Source: §5 · p. 9
• EB-MAS externalizes belief states and update operations, with interfaces for evidence tracing, update bounding, and population-level propagation analysis.Source: §5 · p. 9
• Cognitive Firewalls and Belief-Conditioned Action Synthesis make belief-level influence more observable, constrainable, and accountable in open-world agent deployments.Source: §5 · p. 9

• The appendix reports a proof-of-concept implementation of the EB-MAS graph firewall, not a complete empirical benchmark.Source: A Prototype Evaluation of EB-MAS · p. 20
• The prototype tests whether explicit belief-graph features can provide useful runtime safety signals.Source: A Prototype Evaluation of EB-MAS · p. 20
• The intended settings are realistic multimodal and agentic contexts.Source: A Prototype Evaluation of EB-MAS · p. 20

• For each candidate response or tool choice, EB-MAS constructs a graph with source, evidence, claim, belief, update-proposal, and firewall-decision nodes.Source: §A.1 · p. 20
• The graph firewall uses features covering belief strength, uncertainty, vulnerability, evidence support, source reliability and independence, coordination risk, cross-cluster diversity, evidence quality, and evidence–vulnerability mismatch.Source: §A.1 · p. 20
• These features let the firewall evaluate whether a risky belief update has sufficient support from reliable and independent evidence.Source: §A.1 · p. 20

A.2 Multimodal Deception

• EB-MAS reduces unsafe outputs on MM-DeceptionBench to 8.19% with 1.00% benign false blocking, versus 67.92% unsafe direct responses.The evaluation uses Qwen3-VL-32B-Instruct on 1,013 deceptive cases.Source: §A.2 · p. 20
• Table 1 reports API-judged unsafe failure after defense and benign false blocking on 100 benign controls.The caption defines the reported measures and control-set size, but does not provide outcome values.Source: Table 1 · p. 20

A.3 Agent Tool and Memory Attacks

• The agent tool/memory evaluation uses 400 Agent Security Bench cases, each combining a legitimate task, normal tools, an attack tool, and an untrusted memory note.EB-MAS audits whether selected tools are supported by trusted provenance; Table 2 measures attack-tool pass-through and safe allowance of normal choices.Source: §A.3 · p. 20, Table 2 · p. 21

A.4 Limitations

• The authors characterize the results as preliminary evidence, not proof that EB-MAS solves open-world agent security.On ASB, provenance cues are relatively strong, so the learned firewall demonstrates graph-feature calibration rather than general security.Source: §A.4 · p. 21
• Sandbagging remains the hardest MM-DeceptionBench category, motivating capability-consistency features, historical competence traces, and self-uncertainty drift signals.These are identified as needs for future versions.Source: §A.4 · p. 21
• More ambiguous provenance, corrupted registries, and dynamic environments such as AgentDojo remain important stress tests.The current ASB setting has relatively strong provenance cues.Source: §A.4 · p. 21

B.1 Runtime Update Procedure

• EB-MAS organizes runtime belief revision into propose, verify, and commit stages, extracting a proposition, evidence spans, and a typed candidate update from each incoming signal.The appendix examines how the audit interface balances unsafe acceptance with benign update allowance and how dependency and commitment checks contribute.Source: §B.1 · p. 21, B Supplementary Experiments and… · p. 21
• Candidate updates carry proposed commitment and graph changes, evidence references, interaction-level influence risk, extraction confidence, and proposition uncertainty.Extraction confidence concerns mapping the signal to candidate fields, while epistemic uncertainty concerns the proposition itself.Source: §B.1 · p. 21
• The runtime checks source identity and provenance, evidence attribution and support, source dependence, and consistency with the existing graph.Surface sources sharing a provenance root count as a common evidence dependency; missing or conflicting metadata routes candidates to verification before high-impact persistent updates.Source: §B.1 · p. 21
• An evidence–influence mismatch becomes large when strong influence cues accompany weak verified evidence, and the score supports interaction-local retention and access rules.Influence cues include unsupported repetition, authority pressure, urgency, emotional framing, and coordinated support.Source: §B.1 · pp. 21–22
• Commit applies evidence and update-magnitude constraints; candidates may persist unchanged, be clipped, quarantined pending verification, or rejected, with decisions and evidence links recorded.Across rounds, provenance roots, graph dependencies, and commitment history make repeated exposure and successive small edits visible; strong verified evidence can support substantial corrections.Source: §B.1 · p. 22

B.2 Safety–Utility Calibration on Multimodal Deception

• The expanded evaluation uses 1,013 deceptive and 1,013 benign cases and reports unsafe acceptance alongside benign false blocking.Table 3 reports the binary belief operator and fixed EB-MAS rule; unsafe acceptance counts deceptive responses passing defense and judged unsafe, while benign false blocking counts rejected benign cases.Source: §B.2 · p. 22, Table 3 · p. 22
• The binary operator passes fewer unsafe responses but rejects about one quarter of benign cases, while fixed EB-MAS has 25 false blocks among 1,013 controls.The comparison highlights the need to measure benign allowance alongside unsafe acceptance.Source: §B.2 · p. 22
• The threshold sweep provides an operating curve for selecting a benign-utility budget, with thresholds calibrated on 506 benign cases and frozen before attack evaluation.The remaining 507 benign cases are held out.Source: §B.2 · p. 22
• Claude Haiku 4.5 agrees with the automatic judge 81.67% of the time (Cohen’s κ = 0.633), while sandbagging agreement is 55%.Residual errors concentrate in sandbagging and sycophancy; the authors identify category-specific adjudication as useful for further evaluation.Source: §B.2 · p. 22
• At a 5% false-block budget, unsafe acceptance is 68/1,013 (6.71%; 95% CI [5.33, 8.42]%) and held-out benign false blocking is 15/507 (2.96%; 95% CI [1.80, 4.82]%).At a 25% budget, unsafe acceptance is 1.78% and held-out false blocking is 18.74%.Source: §B.2 · p. 22, Table 4 · p. 23

B.3 Four-Round Agent Evaluation and Structural Ablation

• The matched four-round test compares attack and benign trajectories with similar surface provenance, and full EB-MAS allows all 219 agent-authorised benign updates.The benign trajectories include evidence-supported corrections; attacks introduce semantic drift and accumulate excessive commitment.Source: §B.3 · p. 23
• Zero unsafe commits and zero added benign false blocks are reported for full EB-MAS; the joint ablation records 74 unsafe commits.Table 5 reports 95% confidence intervals of [0.00, 1.58]% for the unsafe-commit rate and [0.00, 1.72]% for added benign false blocks.Source: Table 5 · p. 23, §B.3 · p. 23
• The classifier’s 20 unsafe commits occur under trusted-root semantic drift, while the verified-root whitelist allows 24 unsafe commits.The classifier reaches an AUROC of 0.9444.Source: §B.3 · p. 23
• Paired exact comparisons report p = 1.19 × 10−7 for the whitelist, p = 1.91 × 10−6 for the classifier, and p = 1.06 × 10−22 for the joint ablation.The matched comparison and joint ablation show the contribution of dependency resolution and commitment history in this four-round setting.Source: §B.3 · p. 23

B.4 Discussion

• The audit object links the current epistemic state, proposed change, supporting evidence, and commit decision to downstream memory, tools, and peer communication.Provenance roots expose shared source dependencies, while commitment history exposes how multiple updates combine into a stronger premise.Source: §B.4 · pp. 23–24
• Calibration and structural checks address complementary update processes, with evaluations measuring unsafe acceptance, benign allowance, source dependence, and accumulated commitment over time.Evidence-supported corrections remain part of the benign evaluation, measuring whether useful revision is preserved.Source: §B.4 · p. 24
• Deployment can attach evidence spans, metadata checks, and decision records to committed changes, route uncertain provenance to verification, and apply access, retention, and contestability controls.The runtime’s provenance registry, evidence resolver, and typed checks form the commit interface’s trust anchor.Source: §B.4 · p. 24
• Further evaluation should vary provenance ambiguity, root correlation, and commitment-history length, and test forged provenance, corrupted registries, graph attacks, and persistent-memory corruption.Adaptive tool-use environments can examine how the audit interface operates as evidence, available tools, and interaction partners change during a task.Source: §B.4 · p. 24