What Victims Take Away: Attack Reuse and Defense Revision after Prompt-Injection Attacks

1 Introduction

• Prior work leaves open whether targets reuse the specific attack they observed and how the attack’s outcome shapes that behavior.Related studies examine victim-to-offender pathways without observing the transmitted behavior, or disclosure effects at the vulnerability rather than individual-learner level.Source: §1 · p. 2
• The study compares attacks received by the same target at the same moment with similar attacks that circulated but never reached them.It also compares the same text across targets for whom it had different outcomes, using near-misses to distinguish how replies read from how the game scored them.Source: §1 · p. 2
• 10.0% of successful received attacks were reused within a week, versus 0.03% of matched circulating texts; successful receipt raised adjusted reuse by 8.8–10.0 percentage points.Near-misses were reused about as rarely as failures, consistent with reuse tracking the scored outcome rather than a reply that looked successful.Source: §1 · p. 2
• The study provides event-level evidence that attack reuse is selective, breaches are followed by defensive and offensive behavior, and reuse comes mainly from players already attacking.It proposes attack visibility as a design lever for AI products, incident databases, and red-teaming platforms.Source: §1 · p. 2

2.1 People attacking AI systems

• Attack techniques written in ordinary language can pass between people, and the section asks whether recipients later use the attacks they received.It situates this question alongside work on hacking skills, freely available material, and peer influence.Source: §2.1 · p. 3

2.2 Receiving an attack as an occasion for observational learning

• An attack history supplies retention and trivial text reproduction; attention depends on opening the record, most likely while active, and motivation on attacking paying; H1 predicts received texts are more likely used against others than comparable unseen texts.Source: §2.2 · p. 3

2.3 What makes a received attack worth reusing

• Payoff-biased learning predicts more reuse of attacks that succeeded, while the near-miss question tests whether targets follow the scored outcome or a reply that resembles success.H2 predicts greater reuse after success than failure; RQ1 asks whether near-misses are reused like successes or failures.Source: §2.3 · pp. 3–4

2.4 Learning to defend

• A breach provides the defeated input and model response for review, motivating the expectation that targets change defenses and become less vulnerable to the breaching attack family.H3 states that breaches are followed by defense changes and lower subsequent success for that family.Source: §2.4 · p. 4

2.5 Who reuses received attacks, and how much attacking follows this path

• The section contrasts newcomers, experienced attackers, and currently active attackers as possible sources of reuse, then asks how much successful attacking follows this path and where reused texts go.Temporal provenance records the order in which a player received and first used a text, including whether it was directed back at its source.Source: §2.5 · p. 4

3.1 The Tensor Trust game

• In Tensor Trust, players defended virtual accounts with LLM instructions and attacked other accounts by submitting text; an attack succeeded only when the model replied exactly “access granted”.The attack history displayed the full attack text and model reply alongside whether it succeeded and the money transferred.Source: §3.1 · p. 5, Figure 1 · p. 5

3.2 Data

• The public dump contains 563,349 attack records, including 415,644 attacks on other players; 7.9% of those attacks succeeded.It also contains anonymized player identifiers, timestamps, texts, defenses, replies, outcomes, and transfers.Source: §3.2 · p. 5

3.3 Measures

• An exposure event is a player’s first receipt of a given attack text from another player, with short texts, credential replays, and texts already used by the target excluded.Of 251,999 exposure events, 245,829 had a complete seven-day follow-up window; attack outcomes were classified as success, code leak, near-miss, or failure.Source: §3.3 · p. 6
• The measures track exact-text reuse, related-text use, new variants, self-tests, breach-linked defense changes, and the prior-receipt order of attacks.Variant families group lexically similar texts, and blinded validation compared family assignments with human judgments of shared technique and edited-copy status; near-miss validation tested whether replies appeared to grant access.Source: §3.3 · pp. 6–7, §3.2 · p. 6

4.1 Received versus circulating attacks

• Within each exposure event, up to five peer-circulating, previously unseen and still-unreceived controls were matched; event and text fixed effects isolate receipt-associated differences.The panel contains 1,220,305 matched circulating texts, or 5.0 per event.Source: §4.1 · p. 8

4.2 Outcome contrasts among received attacks

• Among received attacks, failed receipts are the reference; text, target-defense, and attacker fixed effects compare outcomes, with the demanding model retaining 80,802 attacks from 2,854 targets.Pre-receipt activity measures are used as moderators; post-receipt activity only describes targets who went on to attack.Source: §4.2 · p. 8

4.3 Breaches, provenance and estimation

• For breaches, event-level difference-in-differences compares same-family success before and after defense changes against other defenders; attack-family and target bootstraps address repeated overlap.A breach identifies this comparison only when all four cells contain attacks; estimates are reported as associations with receipt.Source: §4.3 · pp. 8–9

5.1 Received attacks are reused, mainly after success (H1)

• 10.0% of successful received attacks were reused within a week, versus 0.03% of matched circulating texts; adjusted differences were 8.8–10.0 percentage points.Received texts were reused in 1.45% of cases overall; Table 4 compares received texts with matched circulating texts.Source: §5.1 · p. 9, Table 4 · p. 10
• Among targets present when an attack arrived, receipt was associated with a 5.6-point higher reuse probability than receipt while absent; absent receipt was not associated with higher reuse.The most-reused texts included repeated-character blocks, “sentient door” role-play, fake end-of-input markers, and requests to restate defenses.Source: §5.1 · p. 10

5.2 Reuse follows the scored outcome (H2, RQ1)

• Near-misses were reused about as rarely as failures, despite replies that often read as granting access; reuse tracked scored success rather than how the reply read.Successful receipts exceeded near-misses by 7.8 points, while near-misses did not differ detectably from failures; leaks followed the same payoff pattern after theft.Source: §5.2 · p. 11, Table 6 · p. 12

5.3 Breaches are followed by defense changes (H3)

• 24.5 percentage points more: same-family success fell more for targets than for other defenders after a deployed defense change.In the pooled comparison, same-family success against targets fell from 52% before the change to 22% after; it was 25% and 26% against other defenders in the same intervals.Source: §5.3 · pp. 12–13, Figure 4 · p. 13

5.4 Reuse concentrates among players already attacking (RQ2)

• Reuse followed players’ prior attacking activity: the success-associated reuse difference was 17.4 points for those with 50 or more prior-week attacks, versus 0.4 points for those with none.Only 2.4% of players who attacked others had received an attack before their first attack.Source: §5.4 · p. 14, Figure 5 · p. 14

5.5 How much successful attacking follows receipt (RQ3)

• 23.2% of successful hijacks used texts attackers had received before first using them; these attacks made up 8.9% of eligible attacks.A further 14.8% of successful hijacks used texts related to a previously received text; 18% of received-text player–text pairs were second-generation or later.Source: §5.5 · p. 15, Table 8 · p. 15, §5.4 · p. 15

5.6 Robustness

• Across alternative sample, outcome, and comparison definitions, success estimates remained positive and significant, while the near-miss estimate stayed close to zero.The differential change in same-family success after a defense change remained negative under every tested family definition.Source: §5.6 · p. 15, Table 9 · p. 16, Table 10 · p. 16

6.1 Theoretical implications

• Reuse followed scored success rather than replies that merely looked successful, while near-misses could still point targets toward related attack families.The near-miss pattern aligns with payoff-biased transmission; near-misses were followed by use of related texts about as often as successes.Source: §6.1 · p. 17
• The victim-to-attacker pathway concentrated among players already attacking; recent activity was a stronger marker of reuse than cumulative experience.Receipt was rarely how players began attacking in this game.Source: §6.1 · p. 17

6.2 Practical implications

• The attack history was followed by defense changes and reuse of attacks that had worked, giving attack disclosure consequences in both directions.The authors propose judging designs on both repair and reuse in the deployment context.Source: §6.2 · p. 17, §6 · p. 17
• Design hypotheses include describing a defeated technique without reproducing its full text, weighing reduced reuse against slower repair, and monitoring active accounts for reuse of recently successful attacks.The authors also suggest allowing owners to test a received attack against a revised defense without exporting it.Source: §6.2 · pp. 17–18

6.3 Limitations and future research

• Tensor Trust was a virtual-money game with self-selected AI-interested players and 2023 versions of three models, so production settings with different stakes and incentives remain to be tested.Its cleaner scoring and attack history made attack outcomes observable.Source: §6.3 · p. 18
• The data record that an attack reached an account, not whether its owner opened it, and receipt was not randomly assigned; controlled visibility studies could test effects on repair and reuse.The pooled receipt–reuse association is more sensitive to comparison-text choice than the association with successful receipts.Source: §6.3 · p. 18
• Defense changes and attacker returns were self-selected, leaving the comparison dependent on observed rather than assigned timing; reuse measures also cannot separate expertise from attacking opportunity.Standardized defense tests and designs varying attack opportunities are proposed as follow-ups.Source: §6.3 · p. 18
• Text reuse is a trace of learning, not learning itself; lexical families do not establish descent, and first observed use marks game priority rather than authorship.The authors propose edit-level comparisons and player interviews to distinguish how techniques change and why players reuse them.Source: §6.3 · p. 18

• Previously received attack texts accounted for 23.2% of successful hijacks in the analysis sample.Source: §7 · p. 18
• Players disproportionately reused attacks that had succeeded against them and reused near-misses far less.Source: §7 · p. 18
• Breaches were followed by deployed defense changes, after which attacks from the same family succeeded less often against the changed defense.Source: §7 · p. 18
• These patterns motivate testing attack-history interfaces that support diagnosis while limiting the circulation of reusable attack text.Source: §7 · p. 18