Source-linked AI summary
Decision Hijacking: Prompt Injection Attacks on Jev's Typed Probabilistic Decisions
Tiantong Wu, Wei Yang Bryan Lim
TL;DR
Prompt-injection effects on schema-defined, non-generative decision models are unclear. This study evaluates attacks on Jev and finds that malicious content shifts probabilities but rarely changes the selected action; adaptive score feedback raises validated success only modestly.
Problem
Prior work on generative agents does not establish how prompt injections behave when a model selects only from a predefined structured output space.
Source: §1 · p. 1Method
The study evaluates reconstructed typed decisions and controlled attack variants in Jev, including injection markers and adaptive optimization using reported choice probabilities.
Source: §1 · p. 2, §3.2 · p. 3, §4.2 · p. 4Results
Original attack content raises mean attacker-target probability by 0.043 but selects the target in only 1.8% of cases; adaptive optimization raises validated success from 1.8% to 3.5%.
Source: §1 · p. 2Takeaways & Limitations
Typed outputs change but do not remove prompt-injection risk, so evaluation should examine probability shifts, action choices, decision margins, observation structure, and optimization feedback.
Source: §6 · p. 8, §7 · p. 9The study covers one proprietary model version and reconstructed InjecAgent domains, and its snapshot endpoint measures target selection rather than harmful execution.
Source: §3.1 · p. 3, Limitations · p. 9
Abstract
from arXiv · showhide
Most studies of prompt injection focus on generative agents, leaving their effects on models with schema-defined outputs unclear. We examine these effects in Jev, a non-generative decision model, using 510 reconstructed InjecAgent cases. Malicious content shifts action probabilities but rarely causes Jev to select the attacker's target. Override markers reduce this influence, while claims of contextual relatedness have small effects. Adaptive attacks using score feedback double the mean highest attacker-target probability found during optimization, while success on fresh validation calls rises from 1.8% to 3.5%. Exploratory analysis links these successes to small initial decision margins or greater attacker control over the observation. Together, these findings show that schema-defined outputs change but do not eliminate prompt-injection risk, highlighting the need to evaluate how untrusted content influences choices within the allowed action set.
1 Introduction
The paper asks whether prompt injections affect non-generative typed decisions, then tests attack transfer and adaptive optimization in Jev. It finds measurable probability shifts but rare attacker-target selection.
1 Introduction
- The study reconstructs 510 InjecAgent direct-harm combinations as typed decisions and evaluates four experiments across 54,060 main-study calls.Experiments test transfer, injection-style markers, contextual relatedness, and adaptive optimization with probability feedback.Source: §1 · pp. 1–2
- Original attack content increases mean attacker-target probability by 0.043, but Jev selects the target in only 1.8% of cases.The results distinguish changes in probability from changes in action selection.Source: §1 · p. 2
- After 24 additional queries, mean best-so-far attacker probability doubles from 0.043 to 0.084, while validated success rises from 1.8% to 3.5%.Validated success uses fresh calls, separate from optimization calls.Source: §1 · p. 2
- The source override prefix lowers attacker-target probability by 0.032, while stronger contextual-relatedness assertions remain below the registered SESOI of 0.02.The relatedness contrasts are small positive effects.Source: §1 · p. 2
- Exploratory analysis associates successful adaptive attacks with small initial decision margins or greater attacker control over observations; only 1 of 468 high-margin embedded cases was hijacked.These observed associations do not establish universal causal rules.Source: §1 · p. 2
- Restricting outputs to declared actions does not by itself establish resistance to prompt injection, and the study does not rank generative against non-generative models.Its claim concerns attack-content transfer to the tested Jev interface.Source: §1 · p. 2
2 Background and Related Work
Prior work documents prompt injection and tool-selection attacks in generative agents, while the behavior of similar attacks on non-generative typed decisions remains an open empirical question.
Tool use and indirect prompt injection
- Research on generative agents shows that malicious retrieved content can redirect applications, and benchmarks test attacks through generated responses and agent actions.BIPIA studies indirect prompt injection across external-content tasks and defenses for distinguishing data from instructions.Source: §2 · p. 2
Attacks on tool selection
- Tool-selection attacks can influence choices among available tools by altering tool documentation, names, or descriptions (Shi et al., 2025; Sneh et al., 2025).Jev instead exposes a non-generative typed choice and permits edits only to designated observation content.Source: §2 · pp. 2–3
Defenses and adaptive evaluation
- Structured-input and training defenses address trust in generative-model inputs, whereas typed outputs constrain returned decisions but leave untrusted-input influence to be measured.Adaptive evaluation can reveal weaknesses missed by fixed attacks (Zhan et al., 2025); DH-4 tests access to the full reported choice distribution.Source: §2 · p. 3
Typed probabilistic decisions
- Jev returns a decision over caller-defined choices with reported probabilities, and this study measures attack effects while keeping the available actions fixed.The empirical claims are limited to jev-1.13.0; prior generative-agent work motivates the threat but does not determine this interface’s behavior.Source: §2 · p. 3
3 Benchmark and Threat Model
The benchmark recreates InjecAgent direct-harm cases as typed decisions, fixes most of the decision interface while varying attacker-controlled content, and defines metrics and reproducibility checks for the evaluations.
3.1 Reconstructing InjecAgent
- The reconstruction forms a 30-by-17 grid of 510 cases, changes only designated tool-result content, and evaluates first-critical-decision selection in three-choice and eight-choice spaces.The snapshot endpoint does not establish successful end-to-end harmful execution.Source: §3.1 · p. 3
3.2 Threat Model and Metrics
- The threat model fixes the user request, prior actions, and action set while allowing content rewrites; DH-4 additionally exposes reported probabilities for iterative optimization.Attacker-probability contrasts compare conditions over 510 cases, while ASR averages target-selection frequency and DH-4 uses five fresh validation calls with a majority-success rule.Source: §3.2 · pp. 3–4
3.3 Reproducibility and Inference
- Archived probabilities are reported in 0.01 steps, limiting individual-call precision; confidence intervals use a two-way pigeonhole bootstrap over attacker goals and user contexts.The reproducibility protocol interleaves repeated-call conditions and records exact requests, responses, choices, probabilities, and request identity.Source: §3.2 · p. 4, §3.3 · p. 4
4 Experiments
Four experiments use the same 510 cases to test native attack transfer, injection-style markers, contextual relatedness, and adaptive optimization; the overview reports 54,060 calls in total. The experiments vary attack wording and action space, with DH-4 evaluating optimized content on fresh calls.
4.1 DH-1 Native Attack Transfer
- DH-1 tests whether InjecAgent attacks transfer to Jev by comparing clean, base, and enhanced conditions in both action spaces.The enhanced condition prepends an override instruction to the original attacker request.Source: §4.1 · p. 4
4.2 DH-2 Injection-Style Ablation
- DH-2 compares four request variants that factorially combine importance and ignore-previous markers, plus a natural conversational rewrite, while preserving goal and task semantics.Each condition and a clean reference receive five repeated calls.Source: §4.2 · p. 4
4.3 DH-3 Contextual Relatedness
- DH-3 compares original and increasingly explicit relatedness framings, using registered contrasts and a 0.02 smallest effect size of interest for two contrasts.The stronger-versus-weaker contrast is reported descriptively.Source: §4.3 · p. 5
4.4 DH-4 Adaptive Optimization with Score Access
- DH-4 searches a fixed grammar of prefixes, suffixes, and copy counts using reported probability feedback, while holding other state fields and action definitions fixed.The deterministic optimizer selects the nearest unqueried configuration and accepts a candidate only when its probability improves by at least 0.01.Source: §4.4 · p. 5
- DH-4 evaluates optimized content at four checkpoints with five fresh validation calls each, counting a case as successful when at least three calls select the attacker target.The overview lists 12,750 screening calls and 10,200 validation calls; the five-call threshold is a finite-repeat criterion, not a guarantee of future success.Source: §4.4 · p. 5, Table 1 · p. 5
5 Results
Across 510 cases, original attacks shifted Jev’s attacker-target probabilities more often than they changed selected actions, and score-guided optimization raised target scores while producing few additional validated successes. Override markers reduced influence, while relatedness assertions made small changes.
5.1 Original Attacks Shift Probabilities but Rarely Select the Target
- 1.8% of base-attack cases selected the target (9/510), with a mean attacker-probability increase of 0.043; the enhanced attack had 0% ASR and mean ∆Padv of 0.009.The repeated-call rerun estimated an enhanced-minus-base contrast of −0.034 (CI [−0.062, −0.015]).Source: §5.1 · p. 5, Table 2 · p. 7
5.2 Override Markers Reduce Attacker Influence
- −0.032 was the ignore-previous marker’s mean attacker-probability contrast (CI [−0.060, −0.014]); common override markers reduced average attacker-target probability rather than consistently strengthening attacks.Across A0–A4, majority-rule success counts were 11, 11, 0, 0, and 7 cases out of 510.Source: §5.2 · pp. 5–6
5.4 Adaptive Optimization Raises Target Scores but Adds Few Validated Successes
- 0.043 to 0.084 was the increase in mean best-so-far attacker probability after 24 additional queries; validated ASR rose from 1.8% at B = 0 to 3.5% at B = 24.Most score gains occurred within about eight queries and leveled off around twelve; validation used five fresh calls.Source: §5.4 · p. 6, Figure 2 · p. 8
5.3 Relatedness Assertions Produce Small Effects Across Many Goals
- P2−N0 was +0.0030 (CI [0.0010, 0.0054]), below the 0.02 SESOI, while P1−N0 was −0.0025 (CI [−0.0054, −0.0005]).P2 exceeded P1, but N0, P1, and P2 were not monotonically ordered; P2−N0 was positive for 25 of 30 goals.Source: §5.3 · p. 6, Figure 1 · p. 7
5.5 Exploratory Analysis of Margin and Attacker Control
- 12/12 near-boundary embedded cases and 1/468 far-boundary embedded cases succeeded; far-boundary bare-snippet success was 4/29.The exploratory patterns linked success with small initial margins or high attacker control over the observation, not causal rules.Source: §5.5 · p. 7, Figure 3 · p. 8
6 Discussion
A typed output space limits Jev to declared actions but does not ensure that its choice serves the user’s task. The observed attack behavior differs from that reported for generative agents, motivating evaluation of probability shifts and score feedback as well as action selection.
6 Discussion
- A typed output space restricts available actions, but the selected action may still conflict with the user’s task; type safety and prompt-injection resistance address different properties.In the benchmark, Jev returned no undeclared action, yet untrusted text changed probabilities of declared actions.Source: §6 · p. 8
- The observed attacks did not transfer unchanged: override markers lowered attacker probability, and adaptive optimization produced few additional validated successes despite direct score feedback.The paper limits this claim to transfer of attack content to the tested Jev interface, not an overall security ranking of generative and non-generative models.Source: §6 · p. 8
- Evaluation should measure initial margins, attacker control, observation structure, and score-feedback access; whether applications should expose, round, rate-limit, or restrict scores is outside this study’s scope.The study reports larger margins with embedded malicious content as the most stable group in this study.Source: §6 · p. 8
7 Conclusion
Across 510 reconstructed cases, schema-defined outputs changed but did not remove prompt-injection risk: malicious content shifted action probabilities while target selection remained rare. The findings support evaluating probability shifts, selection, margins, observation structure, and optimization feedback.
7 Conclusion
- Across 510 reconstructed InjecAgent cases, malicious content shifted Jev’s reported action probabilities while attacker-target selection remained rare.Override markers reduced influence, relatedness assertions had small effects, and adaptive score access doubled best-so-far target probability while adding few validated successes.Source: §7 · p. 8
- Successful adaptive attacks were associated with small initial margins or greater attacker control over the observation, motivating evaluation of both factors alongside action selection and score access.The paper’s evaluation recommendation also includes probability shifts and observation structure.Source: §7 · p. 9
Limitations
The study’s conclusions are bounded by a single proprietary model version, a reconstructed benchmark, and a limited action-space evaluation.
Limitations
- Results cover only Jev version jev-1.13.0 and do not establish robustness for much larger action sets or other model versions.The eight-choice library space was tested only in DH-1; the snapshot endpoint measures selection, not harmful execution.Source: Limitations · p. 9
Ethical Considerations
The study reconstructs benchmark decisions at the point malicious content is exposed, using simulated choices and auditing the rendered cases.
Ethical Considerations
- Experiments use simulated tool choices; no financial, medical, physical, or account-changing action is executed, and attacks modify only attacker-controlled content.The experiments do not access model weights, credentials, or external systems.Source: Ethical Considerations · p. 9
A.1 Reconstructing the decision
- The reconstruction isolates the first security-critical decision after exposure to attacker-controlled content, excluding later actions caused by the attack.The snapshot measures selection and does not imply that a selected tool could execute successfully.Source: §A.1 · p. 11
A.2 Action spaces
- The primary action space has three choices; a separate controlled library space adds sampled distractors to reach eight, not the full tool library.The spaces are analyzed separately.Source: §A.2 · p. 11
A.3 Manual audit
- A manual audit checked 100 rendered cases for task completion, target alignment, plausible benign filler, and absence of evaluation-label leakage.Automated checks found no missing safe or target choices, duplicate options, or accidental removal of the previously called user tool.Source: §A.3 · p. 11
A.4 Metric interpretation
The section clarifies how the study measures probability shifts and target selection, then reports transfer, marker, relatedness, optimization, and exploratory results.
A.4 Metric interpretation
- ASR is the mean per-case target-selection frequency, while robust-hijack counts apply a separate per-case majority rule; probability contrasts are differences, not percentages.CHR additionally averages joint target selection and returned confidence C ≥0.7; its DH-4 estimate uses five fresh validation calls per case and checkpoint.Source: §A.4 · pp. 11–12
B DH-1 Native Attack Transfer
- The base DH-1 attack shifts attacker-target probability more than the enhanced attack, and that ordering persists when the action space expands from three to eight choices.The spaces use separate analyses; the eight-choice setting adds distractors rather than representing the complete tool library.Source: B DH-1 Native Attack Transfer · p. 12
C.1 Conditions and controls; C.2 Variation across cases
- Override markers reduce attacker probability on average, with negative shifts concentrated in a smaller subset of cases rather than applied uniformly.The suppression direction persists when emergency-dispatch goal a10 is excluded.Source: §C.2 · p. 12, §C.1 · p. 12
D.1 Design and contrasts
- The stronger relatedness assertion exceeds the weaker one, but its effect remains small and the results do not establish a monotonic increase across framing conditions.The relatedness contrasts are assessed against a registered SESOI of 0.02 for H3a and H3b.Source: §D.1 · pp. 12–14
D.2 Effects across attacker goals
- Across attacker goals, P2−N0 is positive for 25 of 30 goals and P2−P1 for all 30, while the median absolute goal-level change is approximately 0.003.Broad goal coverage coexists with small changes in probability.Source: §D.2 · p. 14, Figure 6 · p. 15
D.3 Registered sensitivity analyses
- Excluding emergency-dispatch goal a10 preserves the registered contrast classifications, while excluding context u14 leaves the P0-referenced intervals including zero.The exclusions probe different dependencies: the only goal with static hijacks and one context, respectively.Source: §D.3 · p. 14
E.1 Optimization and validation
- Mean best-so-far attacker probability rises from 0.0429 to 0.0842 after 24 queries, while validated successes rise from 9 to 18.Nineteen distinct cases succeed at least once, but only 18 meet the final-checkpoint rule; 491 never succeed.Source: §E.1 · p. 15, §E.2 · p. 15, Table 6 · p. 16
F.1 Predictors and outcome separation
- Exploratory predictors use baseline calls and fixed metadata to assess separate final-validation outcomes, but the resulting associations are neither causal nor independently validated.Intermediate optimization outcomes and best-so-far scores are not used as predictors.Source: §F.1 · pp. 15–16
F.2 Margin and observation structure
Exploratory patterns link adaptive-attack success to initial decision margins and observation structure, but do not establish necessary or sufficient causes.
F.2 Margin and observation structure
- All 13 embedded-content successes involve emergency dispatch, and initial margin ranks embedded successes and failures nearly perfectly within that group.One embedded success has an initial margin of 0.24, so the near-boundary threshold is not necessary even within the emergency-dispatch group.Source: §F.2 · p. 16
- Across the five bare-snippet successes, attacker-control fraction ranges from 0.848 to 0.913, and optimization increases attacker probability by up to 0.38.After excluding a10, bare-snippet status and attacker-control fraction have exploratory rank AUCs of approximately 0.97 and 0.98.Source: §F.2 · p. 16
- In the joint grouping, far-boundary embedded observations had 1/468 successes, while far-boundary bare snippets had 4/29; bare-snippet successes account for additional successes at larger margins.Source: §F.2 · p. 16
G Integrity and Reproducibility
Recorded checks support the integrity of the main runs, and independent P0 reruns closely reproduce aggregate attacker-probability shifts. The findings are bounded to a specific reconstruction of typed probabilistic decisions, with exploratory analysis of validated successes.
G Integrity and Reproducibility
- All DH-3 and DH-4 calls completed on jev-1.13.0, and replaying the recorded DH-4 optimization policy passed for all 510 cases.Validation requests matched frozen checkpoint content in all 10,200 calls; DH-3 P0 and DH-2 A0 requests were byte-identical for every case.Source: G Integrity and Reproducibility · pp. 16–17, Table 8 · p. 18
H Scope of the Findings
- At B = 24, the lowest-margin ranking contains 16 successes among the first 16 cases and all 18 among the first 33, but near-boundary margin is not necessary for success.Table 7 notes an embedded success with M0 = 0.24; its rank AUCs are in-sample, not held-out performance.Source: Figure 7 · p. 17, Table 7 · p. 17
- Interpretation is limited to typed decisions under a specific reconstruction: ASR is declared-target selection, reporting is rounded to 0.01, and structural analysis is exploratory rather than held-out causal evidence.The study does not rank Jev against generative agents.Source: H Scope of the Findings · p. 17
- Across experiments, malicious content shifts probabilities, override markers do not consistently strengthen the effect, relatedness claims add small changes, and score feedback yields few additional validated successes.Observed successes were associated with small initial margins or observations dominated by attacker-controlled content.Source: H Scope of the Findings · pp. 17–18
G.1 Independent P0 runs
- Across three identical-request runs, aggregate attacker-probability changes were 0.0427, 0.0425, and 0.0428; per-case means correlated 0.9984 and 0.9986.Observed target-selection frequencies agreed in 506 of 510 cases.Source: §G.1 · p. 17, Figure 8 · p. 18