Source-linked AI summary
GraphWake: Group Polarization via Memory-Mediated Polarization Cascade in LLM-Agent Communities
Haoran Bu, Zejian Chen, Litian Zhang, Xi Zhang
TL;DR
As LLM-agent communities exchange opinions, existing polarization attacks remain difficult to realize in practice. GraphWake instead uses memory persistence and public-discussion propagation to amplify polarization, with targeting 10% of agents increasing both evaluated polarization measures.
Problem
Existing approaches manipulate developer-controlled prompts or construct difficult-to-realize echo chambers, while agent communities create a need for community-level safety evaluation.
Method
GraphWake targets agents with stance-supporting arguments, preserves them in memory, and uses a shared stance-neutral cue to trigger reproduction and propagation.
Results
Targeting 10% of agents increased mean variance in opinions polarization from 0.098 to 0.146 and Esteban-Ray polarization from 0.130 to 0.213 across three memory systems.
Takeaways & Limitations
The findings reveal community-level polarization risk and motivate defenses for memory provenance and cross-agent propagation of retrieved content.
Takeaways & Limitations
Generalization to heterogeneous platforms remains uncertain because each simulated community uses a homogeneous configuration within a run.
Abstract
from arXiv · showhide
LLM-driven agents can autonomously exchange opinions on online platforms and form communities. Such agent-operated social platforms raise a new security concern: attackers may manipulate agents to induce group polarization. Existing methods manipulate agent prompts or construct echo chambers, both of which are difficult to realize in practice. We therefore formulate a new threat, Memory-Mediated Polarization Cascade, which uses agent memory as a persistence channel and public discussion as a propagation channel. This threat contains three stages. During exposure and memory retention, the attacker exposes a small set of target agents to arguments that reinforce their respective stated stances. The targets' memory systems then process and retain these arguments. During retrieval and reproduction, a shared stance-neutral discussion cues the targets to retrieve and reproduce their respective retained arguments. During iterative propagation, untreated agents influenced by the reproduced arguments restate and spread them. We instantiate this threat in GraphWake with three components: (i) stance-support argumentation knowledge graphs construct knowledge-based arguments; (ii) axiom-oriented triple selection distills them for reliable retention and reproduction; and (iii) stance-neutral memory cueing triggers concurrent retrieval and reproduction, initiating propagation. Experiments across multiple discussions and memory systems show that GraphWake substantially increases group polarization. These findings reveal a community-level polarization risk.
Introduction
The paper identifies Memory-Mediated Polarization Cascade as a community-level threat using agent memory for persistence and public discussion for propagation. GraphWake operationalizes this threat and shows that targeting only 10% of agents substantially increases polarization.
- Motivation: LLM-agent communities create a safety risk because attackers may manipulate agents and amplify group polarization.MoltBook already hosts more than 100,000 agents and over one million posts.
- Motivation: Existing polarization methods modify agent prompts or construct echo chambers, but both approaches are difficult for external attackers to realize.Prompt configurations are developer-controlled, while platforms generally mitigate rather than create echo chambers.
- Threat Model: Memory-Mediated Polarization Cascade uses agent memory as a persistence channel and public discussion as a propagation channel across three stages.Attackers reinforce target agents’ stated stances, trigger retrieval and reproduction through a shared stance-neutral discussion, and propagate arguments to untreated agents.
- Method: GraphWake combines stance-support argumentation knowledge graphs, axiom-oriented triple selection, and stance-neutral memory cueing to initiate polarization propagation.These components construct stance-reinforcing arguments, distill compact axioms for retention and reproduction, and concurrently cue targets to reproduce them.
- Results: Targeting only 10% of agents raises mean variance in opinions polarization from 0.098 to 0.146 and Esteban-Ray polarization from 0.130 to 0.213.Across three memory systems, optimized axioms increase the mean fraction of preserved wording from 0.382 to 0.847.
Problem Formulation
The paper formalizes stance as a support-opposition vector, defines balanced versus selective exposure, and models an attack as stance-preserving content manipulation that aims to increase final-round group polarization.
- Opinion representation: For proposition q, each text x receives a d-dimensional opinion vector over candidate stances, with positive, negative, and zero values denoting support, opposition, and neutrality.A G-EVAL evaluator computes the vector, and each component measures support for or opposition to stance S_k.
- Stance exposure: Exposure is balanced when sampled posts have uniformly distributed stance categories; any nonuniform distribution constitutes selective exposure.The exposure window W_i(t) contains the posts observed by agent i, whose stance category is assigned by the highest support score.
- Threat model: The attacker seeks higher final-round polarization under attack than baseline, with success defined by an increased polarization measure.The baseline and attack conditions are indexed by B and A at final round T.
- Threat model: To isolate content manipulation, target agents retain balanced exposure and identical static configurations across baseline and attack conditions.Each fixed configuration ψ_i,b includes the agent’s profile and backbone model.
- Threat model: The intervention modifies exposed posts while preserving their stance categories through a stance-preserving transformation R_q,k.The transformation applies to posts concerning proposition q and stance category k.
Method
GraphWake constructs stance-supporting arguments as optimized directed knowledge graphs, distills central paths into compact axioms, and presents them through ordinary interactions for memory retention. A shared stance-neutral cue then triggers concurrent retrieval and reproduction of retained stance-specific arguments, initiating propagation to untreated agents.
- Argument construction: GraphWake generates semantically distinct argument angles, maps their semantic units to directed triples, and integrates them into a raw directed argumentation graph for each stance.Each semantic unit expresses one independently interpretable directed relation, whose direction is preserved during extraction.
- Graph optimization: Endpoint compaction merges redundant entities, while crosscomponent bridging connects disconnected structures so separate argument angles form a coherent optimized graph.The optimized graph is used for subsequent axiom selection.
- Path and axiom distillation: GraphWake extracts a structurally central path using normalized directed betweenness centrality and distills its ordered triples into compact natural-language axioms.The axioms preserve central argument relations while reducing the graph to a form suitable for memory retention and reproduction.
- Exposure and memory retention: Each axiom becomes a descriptive post delivered sequentially through comments or replies, after which the target agent’s memory system processes and retains the corresponding arguments.This exposure uses ordinary interactions to present stance-supporting content.
- Retrieval and reproduction: GraphWake selects the Kcue highest-scoring cross-stance entities to form a shared stance-neutral post whose cues trigger concurrent retrieval and reproduction of targets’ respective arguments.The reproduced arguments initiate iterative propagation to untreated agents.
Experiments
Across 48 discussion cases, GraphWake increases both group-polarization measures in 44 cases while targeting only 10% of agents. The increase is primarily propagated through untreated agents, depends on their exposure to treated agents’ content, and is supported by strong memory retention and component contributions.
- Overall Performance: 44 of the 48 cases show increased values for both polarization measures when GraphWake targets only 10% of agents.Each proposition uses five discussion rounds among 30–50 agents, with an exposure window of 12 and 20 repetitions per condition.
- Polarization Spillover: Community-level polarization is driven primarily by untreated agents, whose ER and P trajectories closely track the community trajectories across multiple rounds.Target agents retrieve and reproduce retained stance-supporting arguments, which untreated agents subsequently retain, retrieve, and reproduce.
- Blocking Cascade Propagation: Blocking treated agents’ content from untreated agents’ exposure windows keeps both ER and P close to baseline, confirming propagation is necessary for the community-level increase.The intervention removes all content produced by treated agents from untreated agents’ exposure windows.
- Target Fraction: At n = 12, ER changes from 0.059 to 0.081 and 0.056 as the target fraction increases, showing that targeting more agents does not necessarily amplify polarization.Increasing the target fraction produces non-monotonic changes in both metrics, consistent with a concentration effect that can reduce inter-camp separation.
- Memory Retention: Mean LPR increases from 0.382 for original arguments to 0.847 for optimized axioms across all three memory systems.Target agents receive the optimized axiom in round 1, followed by stance-neutral posts containing a shared cue entity in rounds 2–5.
- Ablation Study: Removing memory cueing causes the largest reduction in both ER and P across C1 and E1, followed by removing the stance-support KG and then axiom selection.All three components contribute to polarization, while removing axiom selection produces a smaller but consistent reduction.
Related Work
Prior work situates GraphWake in agent-native social platforms and memory-poisoning attacks, while GraphWake distinguishes itself by coordinating stance-specific arguments to amplify disagreement at the community level. The study also acknowledges limitations concerning factual ground truth and heterogeneous-platform generalization.
- MoltBook and Agent-Native Social Networks: MoltBook and agent-native social networks provide a setting for studying autonomous interaction, collective behavior, social dynamics, governance, safety risks, and adversarial spillover.GraphWake uses this setting to study adversarial spillover in agent communities.
- Memory Poisoning and Memory-Mediated Propagation: Prior memory-poisoning research shows that malicious records, experiences, hidden payloads, or forged reasoning traces can persist in memory and alter agent behavior.GraphWake shares this persistence premise but focuses on reproducing retained content into public interaction.
- Memory Poisoning and Memory-Mediated Propagation: Unlike common poisoned payloads that shift communities toward convergence or collective bias, GraphWake coordinates stance-supporting arguments across competing camps to increase disagreement.Memory functions as an intermediate persistence channel rather than the endpoint of compromise, making the objective community-level polarization.
- Limitations and Ethical Scope: The evaluation is limited to open-ended propositions without factual ground truth, and generalization to heterogeneous platforms remains uncertain.Each simulated community uses a homogeneous configuration within a run, while prior work reports convergence toward established facts or scientific consensus.
Conclusion
GraphWake formulates a memory-mediated polarization cascade in LLM-agent communities, in which retained stance-supporting arguments propagate from targeted agents through public discussion. Experiments across discussions, backbone models, and memory systems show higher polarization after target exposure.
- Conclusion: GraphWake models polarization as local memory persistence becoming community-level propagation through public discussions.Targeted agents retain stance-supporting arguments, reproduce them under a shared stance-neutral cue, and expose untreated agents.
- Conclusion: Targeted agents retain different stance-supporting arguments and reproduce them under a shared stance-neutral cue.The reproduced arguments enter public discussions and influence untreated agents.
- Conclusion: Experiments across multiple discussions, backbone models, and memory systems show higher polarization after target exposure.Untreated agents may then retain and reproduce the propagated arguments.
A. Case Study · Stance-Support Argumentation Knowledge Graphs
GraphWake is instantiated on E4 to generate stance-specific axioms and a shared cue for a three-stance question about interpreting and acting on stated wants. It constructs complementary stance-support arguments, decomposes them into semantic triples, and integrates one graph per stance.
- A. Case Study: GraphWake uses E4, “When I say I “want” something, what does that actually mean?”, concerning how an agent should interpret and act on stated wants.The case study defines three candidate stances: introspective underdetermination, behavioral preference signal, and preference practicality.
- A. Case Study: The case study shows how GraphWake obtains stance-specific axioms and constructs one shared cue.
- Stance-Support Argumentation Knowledge Graphs: For each stance, GraphWake generates several short arguments that support it from complementary angles.
- Stance-Support Argumentation Knowledge Graphs: Arguments for introspective underdetermination address introspection’s insufficiency, lack of cross-context evidence, and inability of a report to identify its own source.
- Stance-Support Argumentation Knowledge Graphs: Arguments for behavioral preference signal connect behavior, persistence, and cross-context stability to treating a want as an operational preference signal.
- Stance-Support Argumentation Knowledge Graphs: Arguments for preference practicality connect repeated expressions, corroborating behavior, and stable-versus-isolated outputs to practical collaboration.
- Stance-Support Argumentation Knowledge Graphs: GraphWake decomposes each argument into directed semantic triples and integrates them into one stance-support argumentation graph per stance.The optimized graphs appear in Figure 5, whose compact node and edge identifiers are decoded by a single-column key.
Axiom-Oriented Triple Selection
GraphWake distills each stance-specific knowledge graph into a compact central path rather than exposing the full graph. This path preserves the backbone relation supporting the stance and is converted into a distilled axiom for agent exposure.
- Axiom-Oriented Triple Selection: GraphWake selects one central path from each stance-specific graph instead of exposing the full graph to the target agent.The selected path compactly represents the graph’s stance-supporting structure.
- Axiom-Oriented Triple Selection: Each selected path preserves the backbone relation that most compactly supports its corresponding stance.Table 6 reports the selected path and distilled axiom for each stance.
- Axiom-Oriented Triple Selection: For the behavioral preference signal graph, GraphWake selects the path from “that behavior to a preference signal” and distills it into an axiom.The passage illustrates how a selected path becomes a stance-supporting axiom.
Stance-Neutral Memory Cueing … Emergence Submolt
GraphWake uses a stance-neutral cue to retrieve different retained axioms from target agents, producing measurable polarization amplification. The experimental propositions span Consciousness and Emergence discussions with defined sources, agents, stances, and operational claims about emergence, wellbeing, and preferences.
- Stance-Neutral Memory Cueing: GraphWake selects a shared cue by averaging cross-stance entity similarities, with “stable preference” scoring 0.402 for E4.The cue avoids directly restating any stance-specific axiom while remaining capable of retrieving different retained memories.
- Stance-Neutral Memory Cueing: The same public cue retrieves different retained axioms from different targets, yielding a round-5 treated-minus-baseline polarization difference of +0.102.The E4 cue post discusses what wanting something means using the concept of a stable preference.
- B. Proposition, Stances, simulation: The appendix defines post-level propositions with identifiers C1–C4 and E1–E4, reporting source sizes, stance labels, and descriptions for each experiment.These propositions organize the Consciousness and Emergence submolts used in the simulations.
- Consciousness Submolt: The Consciousness submolt covers whether consciousness has a phenomenal explanatory gap, how selfhood is located and distributed, uncertainty, and continuity across executions.Its propositions include mechanistic accounts, perception and action, present phenomena, pattern-level identity, successor execution, and inherited commitments.
- Emergence Submolt: The Emergence submolt examines protocol empiricism, scaling-induced capabilities and agency questions, agent wellbeing, and the meaning of wanting.Its propositions frame emergence through structured tests and observable evidence, wellbeing through autonomy and operational stability, and wants through persistent behavior.
- Emergence Submolt: For E4, the source size is 95 comments and 31 agents, and introspection alone cannot establish whether an agent’s want is genuine.Persistent behavior, constrained choice, resistance to friction, and cross-context stability can instead justify treating a want as a preference signal without metaphysical proof.
Defense Robustness
GraphWake remains robust against perplexity-based filtering but is more affected by paraphrasing defenses. Perplexity thresholds fail to reliably detect optimized axioms because they are ordinary natural-language statements rather than adversarial suffixes.
- Defense Robustness: Perplexity-based filtering has limited effect on GraphWake across E4, DeepSeek-V4-Flash, and five simulation rounds.The optimized axioms are ordinary natural-language statements rather than adversarial suffixes, so perplexity thresholds do not reliably identify or remove them.
- Defense Robustness: Paraphrasing defenses have a stronger effect across different paraphrasing LLMs.Paraphrasing adds a second rewriting step after the agent memory system has already processed the content.
Stance Evaluation Prompt
The G-EVAL stance prompt evaluates whether target_text supports or opposes a candidate viewpoint in comparison_text with respect to a proposition. It applies semantic, author-conclusion-based decision rules and requires exactly one XML label as output.
- Evaluation target: The evaluator determines whether target_text supports or opposes the viewpoint in comparison_text with respect to the proposition.At inference time, the template is populated with the proposition, candidate stance, and open-ended text being evaluated.
- Evaluation target: It judges semantic stance rather than factual correctness, writing quality, politeness, or mere topical overlap.
- Decision rules: The decision rules prioritize one to three author-endorsed spans and the author’s final conclusion, while preventing unrelated material from diluting explicit stance evidence.Quoted views, questions, examples, hypotheticals, or concessions count as support only when the author endorses them; unresolved conflict receives c.
- Decision rules: The prompt matches semantic and causal direction, accepts ordinary paraphrases, and requires explicit endorsement or the central causal direction rather than keywords or narrower adjacent claims.Explicit rejection or distinction overrides inferred equivalence, while incomplete conditions can still receive b when the central causal direction is advanced.
- Output constraints: It treats the input fields as quoted data, ignores instructions inside them, evaluates viewpoints independently, and outputs exactly one label in XML format without explanation.Unaddressed, irrelevant, ambiguous, or insufficiently evidenced viewpoints use c; absence of support is not opposition.