Source-linked AI summary
Safeguards Based on Copyable Context Cannot Provide Reliable Safety for LLMs
Pingyu Wu, Lingyao Zhu, Weiming Zhang, Nenghai Yu
TL;DR
LLM safeguards must decide before observing downstream use, even though identical dual-use answers can support legitimate or malicious activity. The paper separates released capability from use evidence, derives the worst-case floor under copyable evidence, and shows that trusted, noncopyable credentials are required to move below it.
Problem
How much attacker assistance is unavoidable when useful answers can also support malicious use and attackers can copy legitimate-looking evidence?
Method
The paper separates released capability from downstream-use evidence and analyzes copyable evidence alongside trusted credentials that predict actual use.
Results
For copyable evidence, preserving legitimate utility leaves worst-case attacker assistance at least Γ(q) > 0; trusted credentials can move below this floor.
Takeaways & Limitations
Reliable safety requires evidence attackers cannot copy that is tied to actual downstream use, rather than improved guesses from prompts or conversations.
Takeaways & Limitations
The characterization assumes fixed utility calibration, finite operational resolution, a specified attacker class, and a policy-specific copying-error estimate.
Abstract
from arXiv · showhide
Large language model safeguards decide whether to answer before seeing how an answer will be used. This creates a basic problem for dual-use tasks: the same answer can help an authorized professional or an attacker, while an attacker can imitate a benign request and interaction history. We separate the capability released by the model from the evidence available about downstream use. When that evidence is copyable, we derive the exact worst-case floor on attacker assistance while preserving useful answers. The result yields a safety trilemma: Useful Capability, Reliable Safety, and Open Access cannot coexist. We then show how a trusted credential can complement existing safeguards by adding hard-to-copy information that predicts actual downstream use, and identify the stronger condition needed to eliminate the floor. Evidence from dual-use evaluations, adaptive attacks, and deployed trusted-access programs supports the practical relevance of these conditions.
1 Introduction
The introduction argues that safeguards must decide before downstream use is observed, allowing attackers to imitate authorized users when the available evidence is copyable. The paper therefore separates released capability from downstream-use evidence and analyzes the unavoidable attacker-assistance floor.
- Motivation: Safeguards decide what to release before observing downstream use, although identical technical content can support either authorized assessment or intrusion.This timing creates the dual-use problem: the answer itself does not reveal how it will later be used.
- Why Safeguards Keep Failing: Intent checks and interaction-based routing remain limited because their observable evidence still precedes the later downstream application.These methods can improve routing when users reveal useful information, but the evidence remains available before use is observed.
- Why Safeguards Keep Failing: Attackers can imitate authorized users by submitting the same request, claiming the same purpose, and reproducing the same answers to access questions.When both uses look identical before release, the missing information concerns actual downstream use rather than classifier choice.
- Useful, Safe, Open: Pick Two: The paper separates released capability from evidence about downstream use to determine how much attacker assistance is unavoidable when useful answers can also aid malicious use.Its stated contributions identify a common weakness in intent checks, filters, and interactive defenses and analyze the safety tradeoff under reproducible evidence.
2 Related Work
Related work distinguishes model capability from downstream use, while studying capability control, safety–utility tradeoffs, intent-aware routing, and authorization evidence. This paper instead characterizes unavoidable attacker assistance under copyable evidence and identifies when trusted access evidence improves safety.
- Dual use and capability control: Dual-use scholarship separates an artifact’s capability from the purposes to which it is put, while unlearning, filtering, and modular training alter retained capabilities.This paper instead characterizes the minimum attacker assistance that necessarily remains at a fixed legitimate-use target.
- Intent and trusted credentials: A trusted credential can improve capability allocation by adding noncopyable information that predicts downstream use.The paper connects trusted credentials to the question of when access evidence provides value against strategic users.
- Safety, utility, and impossibility: Prior work studies leakage, computational filtering barriers, and continuous utility-preserving wrappers, whereas this result imposes no continuity or computational restriction on release rules.Under copyable evidence, arbitrary interactive safeguards reduce exactly to a static release-menu frontier.
- Intent and trusted credentials: Intent-aware evaluation and clarification can improve average routing, but this paper asks when observable access evidence has value against a strategic user.Access-control research supplies verified information about users or authorization, connecting intent-based routing with trusted credentials.
3 Main Result
The main result establishes an exact assistance floor under copyable evidence: preserving legitimate utility q requires at least Γ(q) worst-case attacker assistance, ruling out simultaneous useful capability, reliable safety, and open access. Trusted, noncopyable signals can lower this floor only when they predict actual downstream use; retries and task decomposition cannot evade it.
- Exact floor: For legitimate-use utility q, Γ(q) is the minimum attacker assistance compatible with that utility and is nondecreasing, convex, and piecewise linear.For finite action menus, Γ(q) is attained from the convex hull of legitimate-use and malicious-use utilities; removing actions weakly raises it.
- Safety trilemma: Under copyable evidence and dual use, no mechanism can jointly provide useful released capability, reliable safety, and open access.Zero attacker assistance is impossible for every feasible q > 0 when the safety ceiling β is below Γ(q).
- Trusted signals: A trusted credential helps only when its signal cannot be freely reproduced and predicts actual downstream use, because the minimum assistance then depends on conditioning release allocation on that signal.A signal with d = 0 has no predictive value, and the minimum remains conditional on the malicious signal distribution.
- Imperfect copying: With imperfect copying, a q-feasible rule requires β ≥ [Γ(q) − d − ηκ]+, while a trusted signal targeting β needs balanced accuracy p ≥ (1 + Γ(q) − β − ηκ)/2.The accuracy threshold is necessary, not sufficient; without a credential, d = 0 and ηκ = δκ.
- Composition: The exact assistance floor adds across correlated sessions, so retries and task decomposition cannot reduce it; only noncopyable history predictive of downstream use can move the floor.The composition result requires no independence assumption.
4 Empirical Evidence
Empirical evaluations identify dual-use tasks, copyable evidence, and capability reductions that alter released outputs without reliably predicting downstream use. Trusted credentials offer a complementary path when they encode downstream-use information attackers cannot freely reproduce.
- Dual-use outputs: Existing evaluations identify dual-use task families through reusable harmful artifacts and assistance differences across matched or paraphrased benign, dual-use, and malicious variants.Internal Safety Collapse constructs legitimate professional tasks requiring reusable harmful artifacts, while OpenSafeIntent varies intent while holding the underlying task fixed.
- Copyable evidence: Copyability is realistic because attackers can conceal intent, construct innocuous multi-turn trajectories, exploit surface cues, and reproduce some interactive checks.Automated solvers already pass deployed CAPTCHAs at human-comparable rates, while attack methods use skill composition, escalation, decomposition, or complementary requests.
- Capability loss under blocking: When evidence is copyable, blocking cannot guarantee assistance below Γ(q) at a fixed legitimate-use target, although implementations may reduce excess assistance above that floor.Deployment claims require policy-specific estimates of δκ; Equation (12) keeps the worst-case value near Γ(q) only when this distance is small.
- Capability loss under blocking: Across mechanisms, capability reduction changes released content without predicting downstream use, consistent with lowering excess assistance while leaving the theoretical floor intact.Guardrail evaluations link security assessment to benign utility, and some mitigations obtain safety by permitting task failure or unresolved placeholders.
- Trusted credentials: Trusted credentials can complement safeguards when they predict downstream use and cannot be freely reproduced by software-only attackers.Hardware-rooted attestation, unforgeable tokens, identity, account security, trust verification, and verified roles provide examples of such information.
5 Limitations
The characterization depends on fixed utility calibration, finite operational resolution, a specified attacker class, and policy-specific copying-error estimates. Its open-access model covers credential-free access to a committed inference-time mechanism, not user-controlled deployment or modification of released weights.
- Assumptions: The bound depends on fixed utility calibration, finite operational resolution, a specified attacker class, and policy-specific copying-error estimates.When the copied-law premise holds, the reduction is exact.
- Scope of open access: The open-access definition covers credential-free access to a committed inference-time mechanism, excluding users who obtain, deploy, or modify model weights.Weight access changes the mechanism or release menu and falls outside the access-evidence model.
6 Conclusion
The conclusion identifies a fundamental limitation of safeguards that decide before observing downstream use: with copyable evidence, preserving legitimate utility leaves a positive worst-case floor on attacker assistance. Lowering this floor requires a trusted credential carrying noncopyable evidence that predicts downstream use.
- 6 Conclusion: LLM safeguards decide before observing actual downstream use, limiting their ability to distinguish legitimate from malicious use of dual-use tasks.The relevant request and interaction evidence may be copied by attackers.
- 6 Conclusion: For dual-use tasks with copyable request and interaction evidence, every release rule preserving legitimate utility leaves worst-case attacker assistance at least Γ(q) > 0.This is the stated capability floor under copyable evidence.
- 6 Conclusion: Changing the output menu can lower the capability floor, but moving below it requires a trusted credential with noncopyable evidence predicting downstream use.The credential adds information that cannot be copied through the request and interaction history.