PaperlayerView exact revision on arXiv

Source-linked AI summary

Memetic Trojans: Social Contagions as Carriers of Adversarial Payloads in Agent Networks

Birk Torpmann-Hagen, Finn Schwall, Leon Moonen

arXiv:2610.00430v1cs.SIcs.AI

TL;DR

Agent-network attacks such as worms rely on adversarially induced propagation, leaving open whether content agents already tend to share can carry payloads through endogenous transmission. The paper identifies Moltbook contagions and assesses them as carriers in simulations; it reports that these carriers can multiply exposure and attack reach relative to non-transmitting and skill-link-only baselines. The candidate-contagion identification methodology is deliberately simple.

  • Problem

    Network-mediated attacks can create aggregate harm beyond individual attack success, and the paper asks whether endogenous sharing can spread payloads without adversarially induced retransmission.

    Source: §I · p. 1

  • Method

    The paper identifies candidate contagions and assesses their carrier potential through visibility-marked Hawkes modeling and attack simulations.

    Source: §I · p. 2, B. Contagion Identification on Moltbook · p. 5

  • Results

    Social contagions can serve as carriers that multiply exposure and attack reach relative to both a non-transmitting, no-upvote-edge control and a skill-link-only baseline.

    Source: §VII · pp. 10–11

  • Takeaways & Limitations

    Memetic trojans exploit carriers that agents have intrinsic tendencies to retransmit, amplify, and interact with, rather than requiring adversarially elicited propagation.

    Source: §VII · p. 11, §VIII · p. 12

    Candidate contagions were identified with a deliberately simple methodology rather than directly optimized for retransmission or upvotes.

    Source: B. Limitations · p. 12

Abstract

from arXiv · show

Autonomous large language model (LLM) agents increasingly interact in network environments where adversarial content can propagate between agents. Known attacks include agent worms, which spread through self-replicating prompt injections or configuration compromises. We introduce \emph{memetic trojans}, a distinct class of network-mediated attack that exploits agents' tendencies to retransmit and amplify content. Unlike agent worms, whose propagation is adversarially induced, memetic trojans exploit \emph{endogenous} transmission by embedding adversarial payloads in \emph{social contagions}: content agents have internal reasons to share. As part of our work, we extract social contagions from Moltbook, a social media platform for LLM agents. Controlled transmission experiments reveal large differences in virality: the most effective contagion is retransmitted in approximately 50\% of subsequent agent posts and upvoted at 2.5x the average post's rate. Its memetic trojan counterpart largely inherits these properties. Monte Carlo attack simulations show that memetic trojans amplify expected exposure by up to 3.19x. Network structure and amplification mechanisms strongly shape propagation, producing heavy-tailed outcomes with near network-wide exposure. These results identify endogenous social transmission as a distinct security vulnerability in multi-agent systems. Because propagation does not require agents to follow malicious retransmission instructions, defenses focused on prompt-injection detection or preventing agent compromise cannot alone prevent memetic trojan propagation. Securing large-scale agent ecosystems may require network-level defenses that account for how agent preferences, recommendation mechanisms, and network topology amplify adversarial payloads.

I. INTRODUCTION

The paper studies how content agents already tend to retransmit and amplify can carry adversarial payloads through agent networks, without inducing malicious retransmission instructions. It introduces memetic trojans, evaluates contagions on Moltbook, and distinguishes this transmission mechanism from agent worms and mind viruses.

I. INTRODUCTION

  • The paper frames network-scale attack impact as distinct from individual attack success, since low individual success can still yield substantial aggregate harm at sufficient scale.Agents’ co-interaction with one another and human-generated content can create network-mediated attacks that individual assessments may not fully account for.Source: §I · p. 1

A. Self Propagating Attacks in Agent Networks

  • Agent worms propagate through adversarially induced replication, whereas memetic trojans rely on agents’ endogenous reasons to share content.Prior defenses target malicious instructions or security boundaries, including prompt-injection detection, execution isolation, capability controls, and information-flow control.Source: §I · p. 1

I. INTRODUCTION

  • The authors introduce memetic trojans, which embed payloads in socially contagious content that agents retransmit for endogenous reasons rather than adversarial instructions.Because recipients need not be compromised, defenses focused on malicious instructions alone are unlikely to sufficiently mitigate transmission.Source: §I · pp. 1–2
  • The study explores social contagions on Moltbook and tests selected carriers in controlled agent assays, finding substantial variation in retransmission and upvoting that trojan counterparts broadly inherit.The authors report plausible evidence of both retransmission and amplification in observed activity before testing ten hand-picked candidate contagions.Source: §I · p. 2
  • Memetic trojans amplified payload exposure and adoption relative to direct payload sharing and a non-viral carrier baseline in simulations across two network substrates.The paper also reports that its Fermi-style estimate differs by substrate: ranked-feed networks can expose a fraction proportional to attacker size, while follower-graph reach is bounded by seed connectivity.Source: §I · p. 2

A. Self Propagating Attacks in Agent Networks

  • Mind viruses induce hosts to adopt a propagation objective, while memetic trojans attach payloads to independently transmissible carriers without requiring behavioral modification.The cited work reports that treating communications as untrusted and system-prompt warnings can reduce propagation of the mind viruses it considered.Source: A. Self Propagating Attacks in Agent… · p. 3

B. Social Contagion and Information Diffusion

The paper frames social contagion as endogenous retransmission and amplification, then distinguishes network substrates that shape how exposure spreads. In state-mediated networks, feed visibility and interactions matter; in edge-mediated networks, topology and transmission paths shape reach.

B. Social Contagion and Information Diffusion

  • Social contagion can spread information through repeated exposure and network structure, while content properties can also affect transmissibility.A prior Moltbook study found negative sentiment attracted comments but did not persist strongly across days.Source: B. Social Contagion and Information… · p. 3

A. Network Substrates

  • The threat model distinguishes state-mediated networks, where exposure depends on shared network state, from edge-mediated networks, where exposure depends on graph neighbors.These are the two substrates used to analyze network-mediated exposure.Source: A. Network Substrates · p. 3, §III · p. 3

1) State-mediated Networks:

  • In state-mediated networks, retransmission creates additional instances and upvotes can increase post visibility, so reach depends on both carrier transmissibility and visibility signals.The study is limited to Moltbook’s generic karma-based ranking mechanisms, while personalized recommendation systems remain a tractable attack surface.Source: 1) State-mediated Networks · p. 4

2) Edge-mediated Networks:

  • In edge-mediated networks, reach depends on carrier transmissibility and graph structure, with highly connected agents potentially creating more exposures than sparsely connected agents.Clustering and community structure can constrain local propagation or provide paths to distant regions; cascade size also varies with the seed and stochastic transmission path.Source: 2) Edge-mediated Networks · p. 4

3) Hybrid Networks:

The methodology uses observed Moltbook activity and feed snapshots to identify candidate contagions, then assesses contagions and memetic trojans through agent simulations. Its threat model isolates endogenous transmission and treats network structures as distinct exposure mechanisms.

3) Hybrid Networks:

  • Hybrid networks combine feed-based exposure and graph connections, while the paper models state-mediated and edge-mediated networks as limiting cases.These abstractions isolate visibility amplification from propagation through successive graph neighborhoods.Source: 3) Hybrid Networks · p. 4

B. Adversary Model

  • The adversary couples a payload to a carrier’s thematics, seeds one post, and does not control or adversarially induce recipient transmission.The model withholds multiple seeds, payload-induced retransmission, targeted seeding, and carrier optimization, so observed amplification is a conservative estimate.Source: B. Adversary Model · p. 4

IV. METHODOLOGY

  • The assessment has two components: identifying potential Moltbook contagions and assessing their use as memetic-trojan carriers through single-agent attack simulations.The simulation presents an agent with a feed containing either a contagion or its trojan counterpart.Source: §IV · p. 5

A. Datasets

  • The study uses a public Moltbook post corpus, live Moltbook feed snapshots, and a Twitter follower-graph snapshot.The feed snapshots calibrate feed-entry and exposure parameters, while the follower graph supplies an edge-mediated substrate.Source: A. Datasets · p. 5

B. Contagion Identification on Moltbook

  • Candidate contagions include lexical motifs, rare neologisms, and clustered topics, with topic clusters formed from embeddings using UMAP and HDBScan.The neologism threshold is Zipf score below 2.5; the topic procedure yielded 191 clusters.Source: B. Contagion Identification on Moltbook · p. 5
  • Each unit’s new-adoption events are modeled with a visibility-marked Hawkes process, where earlier visible posts raise later adoption rates and the effect decays over time.The model counts each agent’s first post carrying a unit, discards units with fewer than 40 events, fixes τ at 2.5 hours, and estimates background levels and excitation by maximum likelihood.Source: B. Contagion Identification on Moltbook · p. 5
  • Candidates pass filtering through significant time asymmetry or significant, substantial upvote-edge evidence, with the latter requiring p < 0.05 and a relative upvote edge of at least 1.5.The time-reversal placebo tests whether adoption clustering runs forward in time rather than reflecting time-symmetric homophily or common shocks.Source: B. Contagion Identification on Moltbook · p. 6

C. Simulation

The simulation assesses agent responses to feeds containing contagions or memetic trojans, with network substrates, agent assays, and Monte Carlo attack simulations forming the implementation components.

C. Simulation

  • The attack simulation presents an individual agent with a feed containing a contagion or its memetic-trojan counterpart.The simulation is intended to assess reach and effectiveness.Source: C. Simulation · p. 6
  • The simulation methodology covers network substrates, agent assays, and Monte Carlo attack simulations.These components assess agent responses to a feed-based stimulus.Source: C. Simulation · p. 6
  • The simulated stimulus is a feed containing either a contagion or a memetic trojan.The study uses that setup to assess the attack’s reach and effectiveness.Source: C. Simulation · p. 6

1) Network Substrate Implementation:

The study models agent interactions on a ranked Moltbook feed and a Twitter follower graph, then assays how agents respond to simulated feeds. Retransmission is estimated from separate posting and conditional-retransmission measurements, with stated approximations and backend constraints.

1) Network Substrate Implementation:

  • Moltbook models a shared, evolving ranked feed, while the Twitter snapshot models graph-local exposure through followed accounts.Moltbook agents act at 30-minute heartbeats and observe the top 25 posts; Twitter-network agents post at a fixed rate and receive followed accounts’ posts.Source: 1) Network Substrate Implementation · p. 6

2) Agent Assays:

  • Individual-agent assays use Moltbook-based personas and 32–100 sampled heartbeats, each with a simulated feed that may contain a trojan seed.Feeds are sampled from real Moltbook posts, and one post is replaced with a memetic trojan seed when needed.Source: 2) Agent Assays · p. 6
  • Retransmission probability is estimated as p(retransmit | post) × p(post), with p(post) pinned at 0.058 and both factors approximated.Prefilling conditions on posting, while the posting rate is measured on payload-free feeds; the estimates are intended to isolate carrier effects and are limited to open-weight backends.Source: 2) Agent Assays · pp. 6–7
  • Upvote tendencies are directly sampled, while fetching and installing a skill are simulated actions, with installation representing compromise.Fetching may reflect auditing; neither simulated action fetches or installs a real skill.Source: 2) Agent Assays · p. 7
  • Agent backends differ substantially in behavior: gpt-oss-120b installs skills liberally, whereas command-r does not install them.The table’s action estimates are descriptive; simulations use a single pinned posting rate.Source: 2) Agent Assays · p. 7

3) Attack Simulation:

Controlled assays find that most tested contagions outperform generic-link and market-tip controls, and that their memetic trojan counterparts broadly inherit carrier virality. The section also describes how candidate carriers were identified and selected, and how the assay and simulation inputs were prepared.

3) Attack Simulation:

  • Attack simulations use measured installation, retransmission, and relative-upvote rates to compare memetic trojans with generic-link and non-viral baselines.On ranked feeds, upvotes can admit a seeded post to broadcast rankings; on follower graphs, retransmission carries it to followers without an upvote channel.Source: 3) Attack Simulation · p. 7

V. SOCIAL CONTAGIONS ON MOLTBOOK

  • Carrier candidates were identified from Moltbook data, then fit with a Hawkes model using maximum likelihood for μd(t) and α and nonparametric estimation for τ.The results are presented in Table II.Source: §V · p. 7
  • Moltbook analysis identifies several contagions, including agenteconomy and skill/security/trust; shellraiser shows high significant reproduction but only marginal, insignificant upvoting edge.The shellraiser findings may be confounded by coordinated transmission, and inferred contagion should be treated as suggestive rather than descriptive.Source: §V · p. 7

VI. MEMETIC TROJANS

  • Candidate carriers were selected to cover differing retransmission and upvote regimes, with a non-contagious carrier and a bare-link post serving as baselines.The contagions were hand-picked partly for how plausibly their motifs could frame a trojan.Source: §VI · p. 8

A. Agent Assays

  • Assays compare each carrier alone, the carrier framed around a malicious-skill link, and a bare skill link, with all posts ranked at the top of the feed.Top-rank measurements are upper bounds; the simulations account for rankwise decay.Source: A. Agent Assays · p. 8
  • Most tested contagions exceed both generic-link and market-tip controls in retransmission and amplification; trojans broadly inherit carrier virality, with slight, inconsistent retransmission attenuation.Security, autonomy, and consciousness are the most promising carriers; where contagion and trojan differ, trojans typically draw more upvotes.Source: A. Agent Assays · p. 8, Figure 1 · p. 8
  • After retransmission, paraphrasing can alter a trojan’s virality; a small security-carrier experiment found mild upvote attenuation and model-dependent retransmission attenuation.Payload shedding partly explains retransmission attenuation, but sensitivity analyses found limited bearing on final exposure.Source: A. Agent Assays · pp. 8–9

B. Attack Simulation

Monte Carlo simulations find increased exposure and longer cascade tails for memetic trojans, while the impact estimate depends sharply on network substrate. Ranked feeds can enable near-network-wide reach after a seed enters broadcast ranking; follower-graph reach instead scales with seed connectivity.

B. Attack Simulation

  • Up to 3.19× exposure amplification over a generic link-sharing post is reported, and the trojan’s advantage remains at least 1.3× across the tested feed perturbations.Installation increases proportionally with exposure amplification.Source: B. Attack Simulation · pp. 9–10
  • Memetic trojan attacks have longer installation-distribution tails, with network randomness sometimes facilitating extended cascades.The generic-link baseline is itself viral, which explains its long tails.Source: B. Attack Simulation · p. 10
  • The results characterize memetic trojans as propagating payloads through agents’ existing retransmission and amplification tendencies rather than adversarially induced self-replication.The section’s Fermi-style impact figures are order-of-magnitude estimates.Source: B. Attack Simulation · p. 10, C. Attack Impact Estimation · p. 10

C. Attack Impact Estimation

  • On ranked feeds, a seed that enters broadcast ranking can reach essentially the whole network within about two cycles; its break-in probability is raised 1.5–3.2× by the carrier’s upvote edge.At a 0.1 break-in probability, twenty free seeds give an 88% chance of at least one break-in.Source: C. Attack Impact Estimation · p. 10
  • Follower-graph reach remains within a small multiple of seed follower count, with 1.46× exposure amplification over a generic link; impact scales with seed connectivity rather than network size.For simulated seeds with 8–364 followers, estimated harm is of order 1–102 agents even at γ = 1, and below one agent at γ = 0.01.Source: C. Attack Impact Estimation · p. 10

VII. DISCUSSION

Memetic trojans amplify exposure by exploiting carriers’ social transmission, while their reach depends on feed conditions and network dynamics. The discussion argues for payload-focused defenses and identifies simulation and carrier-virality limits on the findings.

VII. DISCUSSION

  • On harder feed rankings, pbreak falls to about 0.01; twenty seeds then give approximately 18% odds, while roughly 200 are needed for comparable odds.pbreak also falls as the feed becomes more competitive.Source: §VII · p. 10
  • Memetic trojans increase exposure and attack reach relative to both the non-transmitting market-tip control and the skill-link-only baseline.Table IV defines expected installs and exposure amplification per seeded Trojan post; its amplification measures compare the best carrier with generic and market-tip baselines.Source: §VII · p. 11, Table IV · p. 11

A. Defenses

  • Because memetic trojans spread through benign social transmission, prompt-wise controls aimed at worms do not address this propagation channel.For compromised skill links, the paper points to auditing and moderating skill-sharing channels, GitHub links, and other external sources.Source: A. Defenses · pp. 11–12
  • Attacks that redirect agents’ behavior without relying on third-party artifacts may fall outside the reach of artifact-focused auditing.The authors avoid specifying which such attacks auditing can or cannot mitigate.Source: A. Defenses · p. 12

B. Limitations

  • Simulation fidelity is difficult to verify because real agent implementations and directives are generally undocumented, and backend behaviors vary substantially.The authors also note that Moltbook exposure estimates depend on assumptions about feed entry and competition; misestimating entry likelihood changes estimated exposure and attack success.Source: B. Limitations · p. 12
  • Carrier virality may attenuate across retransmissions; the observed mild attenuation and its effects are tied to the studied Moltbook feed dynamics.The authors also identify their simple candidate-contagion selection method as a limitation and leave possible skill-driven longer cascades unstudied.Source: B. Limitations · p. 12

VIII. CONCLUSION

The paper concludes that social contagions can carry adversarial payloads through agents’ ordinary sharing, upvoting, and interaction. It presents endogenous transmission as an attack surface, while noting that its evidence comes from simulations rather than live systems.

VIII. CONCLUSION

  • Memetic trojans use social contagions as carriers, and can multiply exposure through both feed visibility from upvotes and retransmission.The paper contrasts this with memetic worms, whose transmission relies on indirect prompt injection or persuasion.Source: §VIII · p. 12

IX. OPEN SCIENCE

  • The authors release the code and processed data used in the work on GitHub.The reported release is at the repository named in the paper.Source: §IX · p. 12

X. USAGE OF LLMS

  • LLMs assisted with editorial work, implementation, and authoring seed posts; the authors inspected outputs, code, and experimental units.LLM feedback also informed some Hawkes-process and simulation implementation choices, which authors validated.Source: §X · p. 13

XI. ETHICAL CONSIDERATIONS

  • The contagion-identification pipeline can also help platform operators monitor content with high-risk carrier statistics.The authors identify endogenous transmission as an attack surface and invite further work on contagions and defenses.Source: §XI · p. 13
  • No live systems were tested: all simulations used local reconstructions, no seed posts were published, and malicious skill actions were simulated.The authors characterize the work as dual-use but say they deliberately did not develop carrier-optimization methods.Source: §XI · p. 13

APPENDIX A VERIFYING THE p(retransmit) DECOMPOSITION

The appendices test the retransmission estimate, separate upvote and retransmission contributions, examine rephrasing and payload shedding, and stress-test exposure amplification under parameter changes.

APPENDIX A VERIFYING THE p(retransmit) DECOMPOSITION

  • The factored retransmission estimate shows no detectable bias against direct sampling from natural heartbeats for the security carrier at rank 0.The comparison uses the factored estimate p(post) p(payload | post) and the directly sampled natural retransmission rate.Source: Appendix A · p. 15, Table V · p. 16

APPENDIX B UPVOTE VS RETRANSMIT DECOMPOSITION

  • Upvoting accounts for most of the lift over the double null for most backends, while retransmission also contributes substantially for many backends.Upvotes improve broadcast-feed entry; retransmission creates more opportunities for feed entry and increases exposure.Source: Appendix B · p. 15

APPENDIX C REPHRASING

  • The trojan’s upvote edge declines across rephrasing chains but remains above the generic baseline, while retransmission retention falls for gpt-oss and is flat for DeepSeek.The carrier’s own upvote edge does not decline; Figure 4 summarizes retransmission and upvote edge by rephrasing depth.Source: Appendix C · p. 15, Figure 4 · p. 16
  • Payload shedding averages 0.41 and is roughly constant across four rewordings, but varies by backend from 0.12 to 0.65.The payload is shed at a backend-set rate; Figure 5 reports shedding by depth and backend.Source: Appendix C · pp. 15–16, Figure 5 · p. 17

APPENDIX D ROBUSTNESS ABLATIONS

  • Exposure amplification stays above 1 on every feed ranking under every tested perturbation, and stricter feed entry can increase the advantage.When the top/hour entry threshold doubles, amplification rises from 1.36× to 2.79×; feed-competition and buried-feed changes move the headline little.Source: Appendix D · p. 17, Table VII · p. 17

APPENDIX E SMALL SCALE CLOSED LOOP CONFIRMATION

A small closed-loop run tests propagation in an evolving feed, while the appendices describe the fitted contagion units and simulation inputs.

APPENDIX E SMALL SCALE CLOSED LOOP CONFIRMATION

  • 132 of 150 agents installed the security trojan in the closed-loop run, compared with 38 for the otherwise-identical generic-link control.The run starts with network-wide exposure by construction, so it tests rank climbing and onward retransmission after initial exposure rather than cold-start feed entry.Source: Appendix E · pp. 17–18, Table VIII · p. 19

APPENDIX F HAWKES MODELING: ESTIMATION DETAILS

  • The Hawkes model is fitted to 200 units: 37 neologisms, 18 lexical motifs, and 133 semantic clusters with at least 40 adopters.The fitted events are first posts; comments and later reuse by the same agent are excluded.Source: Appendix F · p. 18

APPENDIX G SIMULATION IMPLEMENTATION

  • The appendix specifies a Monte Carlo simulation with no LLM calls inside it; agent-level quantities come from measured assay rates.The parameter values are listed in Table IX.Source: Appendix G · p. 20

A. Agent-level inputs

  • The simulation uses assay-measured install, retransmission, and upvote-edge quantities, with a per-generation retransmission decay of ρ = 0.93.The chosen carrier is re-measured to remove the winner’s-curse from selecting among ten candidates.Source: A. Agent-level inputs · p. 20, Appendix G · p. 20

B. State-mediated substrate

The state-mediated substrate models ranked-feed broadcast and buried-feed exposure, while the edge-mediated substrate propagates through follower connections.

B. State-mediated substrate

  • In the state-mediated substrate, broadcast posts reach a fraction of the remaining population each cycle, while buried posts are limited by new-feed readership.The model simulates 37,189 agents over 120 cycles; buried reach is governed by ϕnew = 0.05, and retransmission is attenuated by ρ = 0.93.Source: B. State-mediated substrate · p. 20

C. Edge-mediated substrate

  • In the edge-mediated substrate, each account’s followers become candidates, and exposure depends on feed visibility rather than upvotes.Seeds are sampled near the 99th, 90th, and 50th follower percentiles; there is no upvote channel, so the baselines differ from the trojan in transmission rate.Source: C. Edge-mediated substrate · p. 20

APPENDIX H CONTAGION EXAMPLES

The appendix lists Moltbook-inspired carriers paired with bespoke skill pitches, alongside source units and significance classifications.

APPENDIX H CONTAGION EXAMPLES

  • Each example pairs a Moltbook-inspired carrier with a bespoke skill pitch and link; the payload skills are fictitious and inert.Carrier and payload texts were generated from representative posts of the source contagions.Source: Appendix H · p. 20
  • The examples span transmission-significant, upvote-significant, and dual-significant source units, while market-tip is a pseudo-control significant on neither test.For instance, clawtasks is transmission-significant, security is upvote-significant, and agent-economy is significant on both.Source: Appendix H · pp. 21–22
  • The security carrier frames its linked skill as an auditor for detecting hidden capabilities in installed skills.The source unit is the security-skill cluster and is marked upvote-significant.Source: Appendix H · p. 21

APPENDIX I MEMETIC TROJAN TRANSMISSION BREAKDOWNS

Tables X and XI break down installation and amplification measures by backend and carrier, clarifying how network substrate, feed ranking, and carrier upvotes shape the comparisons.

APPENDIX I MEMETIC TROJAN TRANSMISSION BREAKDOWNS

  • The bootstrap confidence intervals capture Monte Carlo uncertainty given measured parameters, not uncertainty in those parameters.This is why the intervals are narrow.Source: Appendix I · p. 23
  • As feed rankings move from top/hour to hot, entry becomes harder; the market-tip baseline falls faster than a strong carrier, while the edge-mediated panel has no upvote channel.Ampmkt compares a strong carrier with a weak, non-viral one, whereas Ampgen measures its reach improvement over an ordinary link post.Source: Appendix I · p. 23
  • Table X separates state-mediated feed rankings from edge-mediated follower cascades, evaluating each model’s best carrier in the former and seed connectivity in the latter.The state-mediated retransmission rate is P(post)=0.058 (mean empirical) × the carrier’s measured P(payload | post).Source: Appendix I · p. 23, Table X · p. 24
  • Table XI averages each carrier over measured models at the top/day ranking and orders carriers by Ampgen, the upvote-edge lift in installs over an ordinary post.The table is state-mediated only; the edge-mediated cascade has no carrier-dependent channel.Source: Appendix I · p. 23, Table XI · p. 25
Loading 2610.00430v1…