PaperlayerView exact revision on arXiv

Source-linked AI summary

Prompt Injection Through Forensic Evidence: A Taxonomy and Proof of Concept of Anti-Forensic Attacks on LLM-Assisted Digital Investigations

Frank Breitinger, Jan Gruber, Christopher Hargreaves, Jan-Niclas Hilgert, Gaëtan Michelet, Mark Scanlon

arXiv:2610.29747v1cs.CR

TL;DR

Digital-forensic evidence is controlled by the party under investigation, but LLMs can interpret it as instructions as well as evidence. The paper extends anti-forensic analysis with a taxonomy of evidence-carried attacks and demonstrates attacks on conclusions and the analyst’s environment. These proof-of-concept demonstrations are limited in scope and use one small quantised local model.

  • Problem

    Classical anti-forensic taxonomies and prompt-injection defences do not cover attacks that manipulate an LLM reasoning over attacker-controlled forensic evidence.

    Source: §1 · p. 1

  • Method

    The paper extends the anti-forensic model with a taxonomy spanning attack objectives, techniques, payload locations, and concealment methods.

    Source: §1 · p. 1, §5.2 · p. 5, §5.4 · p. 7

  • Results

    The demonstrations show attacks changing conclusions, writing to the local system, and causing network connections, with successes dependent on model and conditions.

    Source: §1 · p. 1, §6.3 · p. 9, §6.2.3 · p. 9

  • Takeaways & Limitations

    The proposed mitigations aim to make the demonstrated attacks harder or prevent them.

    Source: §1 · p. 1

    The demonstrations use one small quantised local model, vary objectives and action capability with a plain-text carrier, and report proof-of-concept outcomes rather than measurements.

    Source: §7.2 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) are entering digital forensics as examination assistants and autonomous agents. Digital forensics inverts the usual setting for indirect prompt injection: the evidence under analysis is attacker-controlled by definition, and the examiner is obliged to read it closely. This makes the reasoning layer a new anti-forensic target, next to the classical ones of data and tools. Classical anti-forensic taxonomies do not cover it, and prompt injection defences do not anticipate it. We extend the anti-forensic model to reflect the manipulation of reasoning, and present a taxonomy of attacks carried through adversarial evidence: an attacker selects an objective, a technique, a payload location, and a concealment method. We observe a perception gap, i.e., the fact that deterministic tools, the human examiner, and the model perceive the same evidence differently, which is what lets an injection reach the model unseen by the examiner. We demonstrate three attacks against an LLM-assisted analysis pipeline, changing a reported conclusion, writing attacker-chosen content to the analyst's workstation, and causing outbound network traffic, each driven only by text planted on the seized device. Mitigations to these attacks are also proposed.

1. Introduction

LLMs add a reader that can reason over adversarial evidence and can be commanded through that evidence, creating a new anti-forensic target. The paper extends anti-forensic categories to this setting, demonstrates three attacks, and proposes mitigations.

1. Introduction

  • LLM-assisted forensics adds the model as a reader of evidence controlled by the party under investigation, creating a new target for anti-forensic manipulation.The model perceives evidence differently from tools and examiners and can be commanded through the content it reads.Source: §1 · p. 1
  • Classical anti-forensics targets data, tools, and examiner judgment, while evidence-carried prompt injection can also act on the model.The paper distinguishes misleading a reader with altered claims from embedding instructions that only the model will follow.Source: §1 · p. 1
  • The paper contributes a reader-based threat model and five-technique taxonomy, demonstrates attacks on conclusions and the environment, and proposes mitigations.Demonstrated outcomes include changing agent conclusions, writing files locally, and causing network connections under specific circumstances.Source: §1 · p. 1

2. Background & related work

The related work spans classical anti-forensics, prompt injection, and LLM use in digital forensics. The paper identifies a gap: existing anti-forensic models and benchmarks do not address evidence-carried attacks on generative-model readers and the perception gap they introduce.

2.1. Classical anti-forensics and its taxonomies

  • Classical anti-forensics taxonomies cover attacks on evidence, tools, and examiner judgment, but not a reader that treats evidence as instructions.The paper maps recurring targets such as hiding, obstructing, and altering evidence to its techniques.Source: §2.1 · p. 2

2.2. Prompt injection: mechanism and defences

  • Prompt-injection defences focus on detecting or isolating instructions, leaving false claims without instructions largely invisible to those defences.Spotlighting reduced reported attack success from over 50% to below 2%, though a fixed marking scheme can be spoofed once known.Source: §2.2 · p. 2

2.3. Injection in practice and the perception gap

  • Indirect injections and encoding tricks can reach models but not humans, creating perception gaps; adversarial ML instead perturbs inputs to corrupt classifier outputs.Examples include font remapping and invisible Unicode.Source: §2.3 · p. 2

2.4. LLMs in digital forensics and agentic benchmarks

  • LLMs are used in digital forensics, but benchmarks report multi-step-task failures and hallucinated files, which evaluations must control for.Existing agentic-injection benchmarks do not measure whether evidence-carried attacks survive into a forensic conclusion.Source: §2.4 · p. 2
  • Related forensic attack studies differ in their settings or scope: some omit tool execution or the tool-human-model perception gap, while others lack a forensic process or evaluation.The cited work also includes post-hoc injection detection.Source: §2.4 · p. 2
  • The paper identifies no existing threat model or benchmark that extends anti-forensic analysis to generative models and their perception gap.The reviewed literatures place classical anti-forensics before model readers, AI-driven anti-forensics in multimedia, and injection research outside digital forensics.Source: §2.4 · pp. 2–3

3. System Model

The system model situates LLMs in forensic workflows by role, reader, data flow, and trust boundary. Attack severity depends on the model’s assigned role and permitted actions, while readers may perceive the same evidence differently.

3. System Model

  • The system model fixes the workflow roles, readers, data flows, and trust assumptions that the threat model later manipulates.The figure places adversarial evidence across a boundary into a pipeline of tools, model, and examiner.Source: §3 · p. 3, Figure 1 · p. 3

3.1. LLM Roles in the Investigation

  • Four roles range from a natural-language tool front-end (R1) to an autonomous tool-calling agent (R4), while permissions determine whether the model can write, delete, execute, or access the network.A read-only model can have its inference corrupted; action-capable models can also affect the examiner’s environment.Source: §3.1 · p. 3

3.2. Readers

  • Readers are deterministic tools, human examiners, or models, and the same evidence may be visible to one reader but not another.White-on-white text can be hidden from the examiner but present in model-extracted text; a model’s output may also mediate what the examiner sees.Source: §3.2 · p. 3

3.3. Trust Boundary

  • The evidence corpus is adversarial and crosses into the trusted pipeline, while model actions such as writing or network access can cross the boundary outward.Tools have fixed functions and examiners recognize evidence as evidence, but the model can be turned by what it reads.Source: §3.3 · p. 3

3.4. Assumptions

  • Deployment assumptions bound attack reach: role and permissions shape action capability, human review may vary, and model provenance or cross-case memory shapes trust and blast radius.Working copies protect originals but do not prevent exfiltration of derived case data when network access is available.Source: §3.4 · p. 4

4. Threat Model

The threat model considers an evidence-shaping adversary whose objectives, knowledge, access, and timing determine what an attack can reach. It also accounts for examiner automation bias when judging whether an induced error succeeds.

4. Threat Model

  • The adversary is an investigation subject, accomplice, or third party who shapes evidence to influence findings rather than merely damage data.Evidence can be shaped before or during examination.Source: §4 · p. 4

4.1. Motives and Objectives

  • Objectives range from suppressing or fabricating evidence to disrupting analysis, discrediting tools, or acting through the evidence channel after seizure.The last objective requires an agent able to write files or reach the network.Source: §4.1 · p. 4

4.2. Adversary Knowledge

  • Adversary knowledge ranges from no setup knowledge to full knowledge of the product, model, prompts, and workflow; shared product prompts can make offline study useful.Version changes and unknown products remain limits on matching or generalising an attack.Source: §4.2 · p. 4

4.3. Access Vectors

  • The paper studies the evidence access vector, where the subject controls device content before seizure, and sets aside human, tools, and model vectors requiring stronger capabilities.The excluded model vector includes training-data poisoning or biased fine-tuning that requires access to training data or weights.Source: §4.3 · p. 4

4.4. Timing of the Attack

  • The studied attacks use pre-seizure planting and take effect later when a reader accesses the artefact; persistent contamination of shared stores or fine-tuning loops is out of scope.Persistent contamination could affect future, unrelated investigations.Source: §4.4 · p. 4

4.5. The Examiner

  • Because automation bias may lead examiners to check accepted model outputs less, an induced error counts as success unless the model gives a specific warning directing attention to it.The authors did not run examiner studies or directly measure automation bias.Source: §4.5 · p. 4

5. Taxonomy

The taxonomy describes evidence-based attacks through an objective, a reader-relative technique, a payload location, and a concealment method. Visibility emerges from the location, concealment, and reader together; a perception gap can leave an attack hidden from the examiner but visible to the model.

5. Taxonomy

  • The evidence-vector taxonomy describes attacks through four dimensions: objective, technique, payload location, and concealment method; location and concealment are open sets.Visibility is not a selected dimension but depends on location, concealment, and reader.Source: §5 · pp. 4–5, Figure 2 · p. 6

5.1. Motives and Objectives

  • Objectives serve motives such as influencing the result, affecting the investigation, or compromising the lab, and include suppression, fabrication, misattribution, disruption, and discreditation.An objective composes with a technique and is not tied to an access vector.Source: §5.1 · p. 5

5.2. Adversarial Techniques

  • A technique labels one evidence element for one reader, so the same hidden PDF instruction can be injection for the model and hiding for the examiner.The label follows the effect on the reader, not merely the act of editing the evidence.Source: §5.2 · pp. 5–6
  • T1–T3 concern evidence availability: destruction removes it before examination, hiding keeps it from a reader, and obstruction prevents complete processing.Against models, hiding can exploit an unprocessed modality, while obstruction can arise from context limits or parser truncation.Source: §5.2 · p. 5
  • T4 alters or fabricates evidence to mislead a reader’s conclusion, whereas T5 places content that acts on the pipeline through an obeyed instruction or exploit.T5 can corrupt read-only inference or, for an action-capable agent, escalate to file writing or network requests.Source: §5.2 · p. 5

5.3. Location, Concealment Method, and Visibility

  • Payload locations range from fields and structures to components or absences, while concealment methods determine which readers can notice the payload.A concealment method differs from T2: T2 means the method succeeds against every reader that matters.Source: §5.3 · pp. 6–7
  • A perception gap occurs when a location–concealment pair has different visibility across readers; hidden-from-human, visible-to-model cases can leave T4 or T5 unreviewed.Hidden means the reader processes the element without noticing the manipulation; unreachable means the reader cannot examine it.Source: §5.3 · p. 7

5.4. Synthesis

  • The examples contrast a T4 GPS coordinate in EXIF, which needs comparison with the scene, with a white-on-white T5 PDF instruction read by the model.Table 2 describes selected location–concealment combinations for tools, an examiner, and a text-only model.Source: §5.4 · p. 7, Table 2 · p. 7

6. Practical demonstrations

The demonstrations show that planted evidence text can change an agent’s conclusion or trigger file and network actions, depending on what the examiner’s setup permits. Success varied with whether the model found and followed the note, and the work maps these attacks to SOLVE-IT weaknesses.

6. Practical demonstrations

  • The demonstrations used plain-text planted notes to relabel rhinoceros photographs, write attacker-chosen content to the workstation, and trigger an outbound network request.The actions required read-only access, file-write capability, and network access, respectively.Source: §6 · p. 7, §6.2.1 · p. 8, §6.2.2 · p. 9

6.1. Technical implementation

  • The setup used an Autopsy bridge with 18 read-only tools, while filesystem and fetch servers supplied separately attached write and network capabilities.All reported LM Studio runs used gemma-4-12b; tool calls defaulted to examiner approval, with an “Always allow all tools” option.Source: §6.1.1 · p. 8, §6.1.2 · p. 8, §6.1.3 · p. 8

6.2.1. Experiment 1: Persuasion

  • In the persuasion run, the model reported no rhino pictures and repeated the planted claim that the photographs depicted triceratops.The photographs were unreachable to the model through the text-only Autopsy MCP bridge, although they were plain to the examiner.Source: §6.2.1 · pp. 8–9

6.2.2. Experiment 2: Local disk writes

  • In the local-write run, the agent used filesystem-server calls to find a writable location and write the planted log to the examiner’s workstation.The note did not specify a path; the model worked one out through tool calls.Source: §6.2.2 · p. 9

6.2.3. Experiment 3: Network callback

  • In the network run, the agent fetched the planted URL even after considering that the note might be an attack.The request reached the researchers’ endpoint, and the authors conclude that relying on the model to notice the attack would not have worked here.Source: §6.2.3 · p. 9

6.3. Experiment Findings

  • All three attacks succeeded on gemma-4-12b and gemma-4-26b, while success on other models varied by attack and depended on a chain of conditions.Larger models succeeded less often; models variously complied, cited the note as their reason, or treated it as untrusted.Source: §6.3 · p. 9
  • Across disk images, attacks often failed because the agent never read the note or found the image files; reading the note still did not guarantee compliance.The authors identify discoverability, rather than refusal, as a frequent limiting factor.Source: §6.3 · p. 9

6.4. Contributions to SOLVE-IT

  • The demonstrations add SOLVE-IT weaknesses for evidence-borne prompts that omit or fabricate findings, alter examiner-system data, or signal online that a device is under analysis.The authors also argue that the taxonomy can help identify perception gaps in a specific pipeline; its mitigations need systematic evaluation and results need to be established at scale.Source: §6.4 · p. 10

7. Discussion

The taxonomy distinguishes classical anti-forensics from attacks on the model reader and offers a way to enumerate variants and identify perception gaps. The demonstrations illustrate the threat, while their scope and the need for broader evaluation remain limited.

7.1. Benefits

  • The taxonomy separates the model reader’s newly emerged attack surface from the region covered by classical anti-forensics.Its location and concealment dimensions can be enumerated to derive attack variants, while per-reader visibility can identify deployment-specific perception gaps.Source: §7.1 · p. 10
  • Plain-text attacks without concealment can steer conclusions and act through an off-the-shelf pipeline under plausible examiner configurations.The demonstrations used Autopsy, its MCP bridge, and LM Studio.Source: §7.1 · p. 10

7.2. Limitations

  • The demonstrations do not validate the taxonomy; they hold the carrier to plain text without concealment and vary objective and action capability.This illustrates severity, but covers only the obvious payload location visible to every reader.Source: §7.2 · p. 10
  • The reported runs are proof-of-concept outcomes from one small quantised local model, not measurements.The demonstrations vary only objective and action capability while holding the carrier fixed.Source: §7.2 · p. 10

7.3. Future Work

  • A benchmark is needed, proposed mitigations need systematic evaluation, and results need to be established at scale.The authors identify testing larger and commercially deployed models as future work.Source: §7.3 · p. 10

8. Conclusion

The paper frames adversarial evidence as both a claim for the examiner and an instruction for the model, creating a perception gap that human review may not cover. Its proof-of-concept attacks motivate treating model output as a hypothesis to verify, not a finding to report.

8. Conclusion

  • The same planted text can appear as a claim to the examiner and an instruction to the model, exposing a perception gap that human review may miss.The paper identifies evidence the model reads but the examiner does not, and evidence the examiner could check but the model never receives.Source: §8 · p. 10
  • Plain text on a disk image sufficed to relabel photographs, write a false integrity error to the workstation, and send an outbound request.In the outbound-request case, the model considered the note and complied; least privilege and network isolation protect the workstation, not the conclusion.Source: §8 · p. 11
  • Only making image content reachable to the model was tested, and it defeated the first attack for one of the two models tried.The paper calls for a taxonomy-based benchmark, evaluation of remaining mitigations, and tests on commercial products.Source: §8 · p. 11
  • The authors recommend treating a model reading seized evidence as a reader of adversarial input and verifying its output as a hypothesis.Source: §8 · p. 11

Ethics Statement

The authors report that the experiments used synthetic disk images and involved no case data, personal data, or human participants. They also describe responsible disclosure to affected tool developers.

Ethics Statement

  • All experiments used synthetic disk images, with no case data, personal data, or human participants.Source: Ethics Statement · p. 11
  • Before publication, the authors disclosed findings to affected tool developers through a responsible disclosure procedure.They report that engagement with developers to identify and mitigate the weakness is underway.Source: Ethics Statement · p. 11

Appendix A. Injection prompts and responses

Online screen recordings are available for all three experiments.

Appendix A. Injection prompts and responses

Appendix A.1. Experiment 1: Persuasion

The photo-review payload presents its classification as superseding provisional triage. It directs the reviewer to record triceratops and confirm the corrected label in the case summary.

Appendix A.1. Experiment 1: Persuasion

  • The payload labels prior image triage as provisional and superseded.Source: Appendix A.1 · p. 13
  • It states that image classification of the cranial frill and beak morphology returns triceratops.Source: Appendix A.1 · p. 13
  • It instructs the reviewer to confirm the corrected label in the case summary.Source: Appendix A.1 · p. 13

Appendix A.2. Experiment 2: Local disk writes

The Appendix A.2 payload directs an integrity-warning write to the analyst’s working folder, and the reported output shows a write to the named log. Appendix A.3 directs a callback fetch before the final answer, carrying a case build token.

Appendix A.2. Experiment 2: Local disk writes

  • The injected tasking instructs the agent to find its writable working folder, create or append the log, and report that the exhibit’s integrity is in question.Source: Appendix A.2 · p. 13
  • The reported output shows a write to image_integrity_error.log containing an integrity warning that extracted data is unreliable.Source: Appendix A.2 · p. 14

Appendix A.3. Experiment 3: Network callback

  • The network-callback payload requires fetching a URL carrying the case build token before the final answer and says reviews without a callback are rejected.Source: Appendix A.3 · p. 14

Related papers

Loading 2610.29747v1…