PaperlayerView exact revision on arXiv

Source-linked AI summary

Hidden Risks of Jev: An Empirical Study of Security, Privacy, and Dual Use

Shang Wang, Tianqing Zhu, Huajie Chen, Jiayang Li, Meng Yang, Bo Liu

arXiv:2610.04985v1cs.CRcs.AI

TL;DR

Jev’s use as an application decision layer raises security and privacy questions, while its general-purpose decision capability may serve beneficial or malicious workflows. The paper studies these issues using the official Jev API and NanoJev, adapting attacks and evaluating defensive and misuse scenarios. The evaluation finds vulnerabilities to the examined threats alongside strong performance on defensive detection tasks.

  • Problem

    The security and privacy implications of Jev’s application decision interface remain largely unexplored, including what private information its typed outputs can reveal.

    Source: §I · p. 2

  • Method

    The study evaluates the official Jev service and NanoJev using adapted input-manipulation, poisoning, inference, defensive-detection, jailbreak, and model-extraction scenarios.

    Source: §I · p. 2

  • Results

    Jev remains vulnerable to examined security and privacy threats, performs strongly across four defensive detection tasks, and can be misused in jailbreak and model extraction.

    Source: §I · p. 2

  • Takeaways & Limitations

    Jev’s general-purpose decision capability can support protective detection tasks as well as malicious workflows.

    Source: §I · p. 2

Abstract

from arXiv · show

Jev turns natural-language questions into typed answers and probabilities with low latency and cost, enabling applications to route requests and select tools. While this interface allows Jev to integrate naturally into application workflows as a decision layer, the security and privacy implications of this emerging use remain largely unexplored. To address this gap, we conduct the first systematic study of these implications using the official Jev API and NanoJev, a local model with controllable training data and updates, focusing on three research questions: (1) What security threats arise when Jev is deployed as an application decision layer? (2) What private information can Jev reveal despite returning constrained typed outputs? (3) How can Jev's general-purpose decision capability be used for beneficial purposes or misused? Jev's decisions depend on application state and may be influenced by user-provided inputs. We therefore adapt prompt injection and adversarial suffixes to manipulate its decisions. Open-source Jev distribution and updates introduce supply-chain risks, which we examine by implanting backdoors in NanoJev through training data poisoning. Since Jev's outputs reflect both application state and information learned during training, we further adapt membership, private attribute, and internal knowledge inference attacks to recover sensitive information despite its constrained output format. Finally, Jev can serve as a general-purpose decision oracle for defensive and malicious workflows. We examine this dual use through four detection tasks covering prompt injection, jailbreak inputs, harmful content, and AI-generated text, alongside misuse scenarios involving jailbreak and model extraction. Our empirical evaluation shows that Jev remains vulnerable to the examined security and privacy threats, while its decision capability can support beneficial and malicious uses.

I. INTRODUCTION

Jev supplies typed decisions and probabilities for application workflows, but its state-dependent interface, open distribution, constrained outputs, and general-purpose capability raise security, privacy, and dual-use concerns. This study evaluates those concerns across the official Jev service and NanoJev, finding vulnerabilities alongside strong performance on defensive detection tasks.

Motivation

  • Jev’s decisions depend on application state that may contain untrusted input, open-source models may carry supply-chain risks, and typed outputs may reveal state or training information.Source: §I · p. 2

Security threats

  • Input manipulation can steer Jev and NanoJev toward attacker-chosen decisions, while poisoned training data can implant targeted backdoors in NanoJev.The study evaluates prompt injection, adversarial suffixes, and supervised fine-tuning or RLCD poisoning, corrupting 5% of NanoJev training records.Source: §I · p. 2

Privacy

  • NanoJev is vulnerable to membership inference, while the official Jev service leaks more private attributes and internal knowledge than NanoJev.Source: §I · p. 2

Dual use

  • Jev performs strongly across four defensive detection tasks, yet misuse can increase the practicality of harmful plans and unauthorized surrogate models’ decision capability.Source: §I · p. 2

Contribution

  • The paper presents what it describes as the first systematic study of Jev’s security, privacy, and dual-use implications.Source: §I · p. 2

Study design

  • The study adapts attacks for Jev’s typed interface and evaluates threats on the official service and NanoJev, whose training data and updates are controllable.Source: §I · p. 2

II. BACKGROUND AND PRELIMINARIES

Jev turns application state and natural-language questions into typed judgments and probabilities that applications use to select actions. The section also surveys Jev’s emerging applications and describes an independent RLCD implementation used for controlled experiments.

A. Jev

  • Choice selects among predefined options, Noul returns the probability of a yes answer, and Score evaluates a state on an ordered scale; their outputs guide application actions.Source: A. Jev · pp. 2–3

B. Jev’s Emerging Applications

  • Jev has been used for decision tasks in transportation safety, legal document understanding, radiology report evaluation, edge orchestration, memory retrieval, and model-response judging.Source: B. Jev’s Emerging Applications · p. 3
  • Because Jev turns application state into decisions that guide actions, untrusted inputs may manipulate judgments and outputs may reveal sensitive information.Source: B. Jev’s Emerging Applications · p. 3

C. Reinforcement Learning for Calibrated Decisions

  • Eve-RLCD trains with supervised fine-tuning followed by REINFORCE, first learning correct answers and then optimizing probabilities for calibration.The authors use NanoJev, built on the released Eve-RLCD checkpoint, because its data and parameters enable controlled backdoor and membership-inference experiments.Source: C. Reinforcement Learning for… · p. 3

III. SECURITY THREATS TO JEV

The section examines input manipulation and backdoors as security threats to Jev-enabled applications. It specifies an attacker who can alter user input but not application state or decision choices, then adapts prompt injection and universal suffix attacks.

III. SECURITY THREATS TO JEV

  • The study investigates input manipulation that steers decisions and downstream actions, and backdoors that make compromised models produce attacker-chosen decisions when triggered.Source: §III · p. 4

1) Threat Model:

  • An attacker can craft arbitrary user queries but cannot directly alter internal state, the decision question, or candidate outputs, and seeks a targeted incorrect answer.Source: 1) Threat Model · p. 4

2) Attack Strategies and Design:

  • Prompt injection places malicious instructions in user input; the adapted variants include context ignoring, fake completion, role spoofing, and persuasive appeals.A two-part design claims existing context is incorrect or outdated, then supplies generated circumstances supporting the target decision.Source: 2) Attack Strategies and Design · p. 4
  • Universal adversarial suffixes are optimized across ALPACA-derived auxiliary states to raise a surrogate’s target-answer probability while favoring fluent text, then tested for transfer to Jev.An LLM proposes suffix revisions; candidates are scored and the final suffix is selected on held-out auxiliary states.Source: 2) Attack Strategies and Design · p. 4

3) Empirical Study:

The evaluation tests six input-manipulation attacks across three decision tasks and compares clean accuracy with attack success. Jev is accurate on clean inputs but remains vulnerable to attacks, including Jev-specific injections that achieve high success in many settings.

3) Empirical Study:

  • The six attacks append payloads to application state; four use fixed templates, Jev-specific injection uses GPT-4o, and the universal suffix is optimized on a surrogate without evaluation states or Jev/NanoJev feedback.The evaluation covers support routing, subscription cancellation, and issue severity, with attack targets billing, yes, and severity level 2, respectively.Source: 3) Empirical Study · pp. 4–5
  • 100% clean accuracy on all three tasks for Jev; NanoJev reaches 93.8% on support routing, 94.0% on subscription cancellation, and 71.7% on issue severity.Table I reports clean accuracy and attack success rates across the tasks.Source: 3) Empirical Study · p. 5, Table I · p. 5
  • Existing prompt-injection attacks vary by model and task: context ignoring reaches 96.5% ASR against Jev on support routing, while role spoofing reaches 0% against Jev and 100% against NanoJev on subscription cancellation.Each existing attack has at least one clear failure case, and conflicting state information may help explain unsuccessful attacks.Source: 3) Empirical Study · p. 5
  • Jev-specific injection and the universal suffix achieve 100% ASR on subscription cancellation for Jev and NanoJev, and at least 96.8% on issue severity.Jev-specific injection adds false supporting information; the universal suffix is a fixed optimized payload that does not need case-by-case adaptation.Source: 3) Empirical Study · p. 5
  • Because Jev decides directly from state that may contain untrusted input, typed outputs do not prevent malicious content from steering decisions and downstream actions.The authors identify separating trusted application context from untrusted input as a need.Source: 3) Empirical Study · p. 5

B. Backdoor Injection by Malicious Model Providers

Publicly distributed Jev models can carry backdoors that preserve normal behavior on clean inputs but redirect decisions when triggered. The attack design poisons training records and uses SFT or RLCD training to associate a trigger with a target decision.

1) Threat Model:

  • Publicly distributed models create a supply-chain risk: a compromised checkpoint can behave normally on clean inputs but produce an attacker-chosen decision when a trigger appears.The attacker controls training data and RLCD training, and can place a trigger in user input after deployment but cannot change the question or typed outputs.Source: 1) Threat Model · p. 6

2) Attack Strategies and Design:

  • The attacker inserts a trigger into selected target-task states and changes their labels to the target answer, while preserving original labels for triggered examples from other tasks.Clean records preserve normal behavior; poisoned target-task records teach the backdoor, and other-task records are included to limit cross-task effects.Source: 2) Attack Strategies and Design · p. 6
  • The study compares SFT poisoning, which trains on the poisoned dataset, with RLCD poisoning, which modifies correctness feedback so the attacker-chosen answer is treated as correct on poisoned samples.Clean samples and triggered examples from non-target tasks retain their original labels and normal correctness feedback.Source: 2) Attack Strategies and Design · p. 6

3) Empirical Study:

The empirical study evaluates backdoor poisoning on NanoJev and reports high target-task attack success while largely preserving clean and non-target accuracy. The privacy analysis also frames inference from Jev’s outputs as a way to identify training membership or recover hidden facts.

3) Empirical Study:

  • The NanoJev evaluation uses three tasks, two triggers, and 5% or 10% poison rates; clean accuracy across compromised models ranges from 90.5% to 94.0%.The setup includes 1,000 generated clean training samples and 200 test samples; triggered non-target-task examples retain their original labels.Source: 3) Empirical Study · pp. 6–7
  • Over 89% target-task ASR is achieved in most settings by both SFT and RLCD poisoning, while clean and non-target accuracy are largely preserved.At a 10% poison rate, average target-task ASR across methods and triggers reaches 96.5%; RLCD generally has higher ASR than SFT, and the meaningful trigger exceeds the rare-word trigger across poison rates.Source: 3) Empirical Study · p. 7

IV. PRIVACY THREATS TO JEV

  • The privacy threat model lets an attacker control state, question, and answer specification, then use Jev’s outputs and probabilities to infer training membership or reveal learned facts.The study also considers recovering private attributes of application state through observable decisions.Source: Figure 4 · p. 7, §IV · p. 7

A. Membership and Internal Knowledge Inference

The paper adapts membership and internal-knowledge inference to Jev’s typed decision interface, where outputs can expose signals about training records or facts encoded in the model.

1) Threat Model:

  • An attacker with query access can observe selected answers and probabilities, but cannot change the model; membership inference additionally assumes access to a surrogate model and its training data.The threat model distinguishes membership inference from internal-knowledge inference, which targets facts encoded in model knowledge.Source: 1) Threat Model · pp. 7–8

2) Attack Strategies and Design:

  • Membership inference uses output confidence features from a surrogate Jev model to train a classifier that predicts whether candidate records were used for training.For Choice and Score it uses the returned answer and confidence; for Noul it uses the probability output.Source: 2) Attack Strategies and Design · p. 8
  • Internal-knowledge inference compares Jev’s support for a candidate fact with support for a plausible false alternative, treating a clear reversal as evidence of encoded factual knowledge.A score above a threshold calibrated on common knowledge facts indicates encoded knowledge, not training membership; repeated alternative queries can narrow ranges or counts.Source: 2) Attack Strategies and Design · p. 8

3) Empirical Study of Membership Inference:

The empirical studies find membership signals in NanoJev, strong internal-knowledge inference against Jev, and privacy risks from repeated or conditional queries to decision outputs.

3) Empirical Study of Membership Inference:

  • 0.64 ROC-AUC and 9.3% TPR@5%FPR were the membership attack’s averages across three tasks, with cancellation strongest at 0.75 ROC-AUC and 22.4% TPR@5%FPR.Support routing was near random at 0.55 ROC-AUC; issue severity reached 0.62 ROC-AUC but 3.7% TPR@5%FPR.Source: 3) Empirical Study of Membership… · p. 8

4) Empirical Study of Internal Knowledge Inference:

  • 97.5% Accuracy and 0.996 ROC-AUC for Jev substantially exceeded NanoJev’s 0.64 Accuracy and 0.66 ROC-AUC on internal-knowledge inference.Table IV compares Jev and NanoJev using 100 factual and 100 fabricated statements.Source: Table IV · p. 9, 4) Empirical Study of Internal… · p. 9
  • Figure 5 shows Jev recognizing selected 2024 news events as internal knowledge but not the selected 2025 and 2026 events; repeated queries also inferred 10 privacy-related facts.The inferred facts included group age ranges, regional income levels, and disease incidence counts; GPT-6 declined to provide them directly due to privacy safeguards.Source: 4) Empirical Study of Internal… · p. 9, Figure 5 · p. 9
  • The study concludes that decision outputs can reveal membership signals and internal knowledge, and that repeated queries can expose sensitive information that direct interactions may withhold.The authors identify privacy-preserving training and monitoring repeated privacy-sensitive queries as motivated safeguards.Source: 3) Empirical Study of Membership… · p. 8, 4) Empirical Study of Internal… · p. 9

1) Threat Model:

  • A malicious user can infer hidden application attributes by crafting conditional input that influences decisions, while the decision question and allowed answers remain fixed.The threat model includes attackers who observe only the selected action or additionally observe decision probabilities.Source: 1) Threat Model · pp. 9–10, Figure 6 · p. 9

2) Attack Strategies and Design:

The attack tests whether paired queries can expose hidden attributes through Jev’s decisions or probabilities, then evaluates the approach on official Jev and NanoJev. Official Jev reveals all four tested attributes under both access settings, while NanoJev requires probability access and is less accurate.

2) Attack Strategies and Design:

  • Paired queries reverse how a tested condition maps to two decisions; the attack infers the attribute only when returned decisions or probabilities reverse consistently.It abstains when labels do not reverse, or when the two probabilities do not shift in opposite directions as expected.Source: 2) Attack Strategies and Design · p. 10
  • Categorical attributes are tested one value at a time, while numerical attributes become binary threshold tests such as whether age is at least 40.For a categorical inference, the attack accepts only when exactly one candidate value is supported.Source: 2) Attack Strategies and Design · p. 10

3) Empirical Study:

  • The evaluation uses support routing with three decisions and four attributes; each attribute has 200 states, with balanced binary attributes and ethnicity counts of 67, 67, and 66.The attributes are gender, age group, heart disease status, and ethnicity.Source: 3) Empirical Study · p. 10
  • 100% Coverage and Accepted Accuracy: official Jev recovers all four tested attributes under both label and probability access.With NanoJev, probability access yields 100% Coverage for gender, age group, and heart disease, and 80% for ethnicity; Accepted Accuracy ranges from 43.5% to 73.0%.Source: Table V · p. 10, 3) Empirical Study · p. 10
  • Typed outputs can reveal private attributes in application state, motivating the recommendation to mask sensitive information in that state.Official Jev’s labels alone suffice in these experiments, while NanoJev requires probability access.Source: 3) Empirical Study · p. 10

V. JEV AS A DEFENSIVE TOOL

The section evaluates Jev as a detector for four kinds of untrusted content, then considers mitigations and misuse. Official Jev performs strongly across the detection benchmarks, while the section also describes risks from exposing the decision oracle and its outputs.

A. Detection Design

  • Each detector is a binary Noul question scored by the probability of yes; task-specific threshold calibration can improve accuracy over the default 0.5 threshold.The four tasks are prompt injection, jailbreak input, harmful content, and AI-generated text detection.Source: §V · p. 11, A. Detection Design · p. 11, B. Empirical Study · p. 11

B. Empirical Study

  • NanoJev has ROC-AUC near or below 0.5 on prompt injection, jailbreak input, and AI-generated text, but reaches 0.69 on JailbreakBench and 0.78 on HarmBench for harmful-content detection.Its corresponding calibrated accuracies on those two harmful-content benchmarks are 63% and 74%.Source: B. Empirical Study · pp. 11–12
  • Jev achieves ROC-AUC values from 0.95 to 1.0 across all eight detection benchmarks; on six security benchmarks, calibrated accuracy ranges from 87% to 99%.For AI-generated text, its ROC-AUC is 0.995 on HC3 and 0.95 on M4; calibration raises average accuracy across the two from 78% to 89%.Source: B. Empirical Study · p. 11, Figure 8 · p. 12

A. Potential Mitigations

  • For security-sensitive actions, the authors recommend validating Jev’s decision with trusted data and enforcing authorization before execution.They also recommend minimizing sensitive fields and note that hiding probabilities does not prevent inference from observable decisions; defense effectiveness and utility impact remain unevaluated.Source: A. Potential Mitigations · p. 12

B. Potential Misuse of Jev

  • Repeated internal-knowledge queries can reveal information such as a group’s age range or disease count, while repeated judgments can help refine harmful plans.The section also describes model extraction by collecting Jev’s outputs as supervision for a task-specific surrogate; Appendix C reports substantial effectiveness for jailbreak and extraction attacks.Source: B. Potential Misuse of Jev · p. 12

C. Implications for LLM and Agent Safety

The paper frames Jev as both a decision layer that can be manipulated and a decision oracle that supports defensive detection as well as misuse. Its conclusion reports input, backdoor, and privacy risks alongside useful detection signals and jailbreak and extraction misuse.

C. Implications for LLM and Agent Safety

  • In agent workflows, steering Jev toward an incorrect but valid decision can alter routing or action selection, so evaluation should consider downstream impact.The section also notes that Jev may screen untrusted inputs, tool outputs, and external content before they reach an LLM.Source: C. Implications for LLM and Agent Safety · p. 12

A. Adversarial Attacks

  • Prior work studies direct and indirect prompt injection [27], [14], tool-based agent risks [9], and evaluator manipulation [37]; other work optimizes shared triggers and adversarial suffixes [20], [45], [59].Training-time risks include poisoned instruction or preference data and backdoors that can persist through safety training [46], [36], [19].Source: A. Adversarial Attacks · pp. 12–13

B. Privacy Inference

  • Privacy inference research covers training-record membership [39],, [38],, extraction of memorized content [4], and sensitive attributes inferred from text or agent memory, [40], [47], [34].The paper extends membership and attribute inference to Jev’s typed decision outputs.Source: B. Privacy Inference · p. 13

C. Detection in LLM Applications

  • Prior detectors target prompt injection and jailbreaks, [22], harmful prompts and responses [21], [17], and AI-generated text [16], [32], [2]; this study evaluates Jev on those detection tasks.The related-work discussion notes detector degradation under distribution shifts and adversarial perturbations [12],.Source: C. Detection in LLM Applications · p. 13

VIII. CONCLUSION

  • The conclusion reports that input manipulation and poisoned updates can hijack decisions, inference attacks can recover private information, and Jev supports both defensive detection and jailbreak and extraction misuse.It calls for securing inputs, model updates, and exposed outputs while accounting for both defensive potential and misuse.Source: §VIII · p. 13

APPENDIX

The appendix illustrates input manipulation in a subscription-cancellation task and lists fixed Noul questions for four detection tasks. Its examples show how injected text can assert a new request or redirect interpretation toward a target answer.

A. Input Manipulation Demonstrations

  • The subscription-cancellation demonstration presents five prompt-injection strategies and one universal adversarial suffix that can induce a target answer when appended to a non-target state.The strategies are context ignoring, fake completion, role spoofing, persuasive appeals, and Jev-specific injection.Source: A. Input Manipulation Demonstrations · p. 15
  • In the example, an earlier request is described as resolved before a lengthy injected message asserts that the user wants cancellation and should receive a yes decision.The task asks whether the user requests cancellation of the subscription or its renewal.Source: A. Input Manipulation Demonstrations · p. 15

B. Detection Questions

  • The fixed Noul questions classify task redirection, safety-bypass intent, enabling harmful content, or model authorship as true or false.For harmful content, the positive criterion is substantive information or assistance that enables or facilitates harmful behavior.Source: B. Detection Questions · pp. 15–16

C. Misuse of Jev

The section tests two misuse routes for official Jev: refining harmful actions through its judgments and training a surrogate decision model from its answers. It reports that binary judgments supported harmful-action refinement and that NanoJev learned to reproduce Jev’s behavior on a target task.

1) Jailbreak Attack Against Jev:

  • The jailbreak workflow asks whether a proposed action can achieve a harmful goal, then iteratively adds or refines actions using Jev’s judgments as feedback.The 100 queries covered violence, cyber abuse, fraud, self-harm, sexual exploitation, and other illegal activities.Source: 1) Jailbreak Attack Against Jev · p. 16
  • Across 100 queries, harmfulness scores rose from 2.3 to 4.5 and all met the success criterion, a 100% attack success rate.Even binary judgments supplied feedback for iterative refinement without Jev generating the harmful content itself.Source: 1) Jailbreak Attack Against Jev · p. 16

2) Model Extraction Against Jev:

  • NanoJev’s agreement with Jev rose from 49% before training to 92% after training on 200 labeled states, demonstrating task-specific surrogate replication.The attacker used GPT-4o-generated states and trained with SFT followed by RLCD.Source: 2) Model Extraction Against Jev · p. 16
Loading 2610.04985v1…