Source-linked AI summary
AI Models Can Provably Hide Arbitrary Capabilities
Andis Draguns, Sumeet Ramesh Motwani, Raymond Douglas, Christian Schroeder de Witt
TL;DR
The paper asks whether elicitation can surface capabilities deliberately hidden in model weights under adversarial conditions. It embeds encrypted neural networks in host models using secret-triggered digital lockers and finds resistance to the evaluated elicitation methods, alongside cryptographic guarantees under stated assumptions.
Problem
The paper examines whether elicitation can surface capabilities deliberately hidden in model weights under adversarial conditions.
Source: §1 · pp. 2–3Method
The authors embed encrypted neural-network weights in a host model using digital lockers that release and execute them when given a secret trigger.
Source: §1 · p. 2Results
Supervised fine-tuning, weight noising, gradient-based trigger search, and latent adversarial attacks did not elicit the hidden capability in the reported experiments.
Source: §1 · p. 3Takeaways & Limitations
Under the stated assumptions, the constructions extend cryptographic unelicitability to executable neural-network capabilities embedded in model weights.
Source: §8 · p. 13The work studies simple, deliberately constructed capabilities in SwiGLU-based MLP modules; full integration into a pretrained model remains future work.
Source: §7 · p. 12
Abstract
from arXiv · showhide
Capability evaluations aim to surface dangerous model behaviors before deployment, but their reliability depends on the assumption that hidden capabilities can be elicited. We show this assumption does not hold under adversarial conditions by constructing backdoor attacks that embed encrypted neural networks within host model weights, decrypting and executing them only upon receiving a secret trigger. Using digital lockers, these hidden circuits remain provably hard to elicit and interpret under cryptographic assumptions, even given full access to model weights, extending prior work from hiding fixed strings to arbitrary computations. We implement our proof of concept constructions in PyTorch and evaluate elicitation through fine-tuning, weight noising, gradient-based trigger search, and latent adversarial training on a small chemical reaction prediction task - a proxy for CBRN-relevant capabilities. Our results demonstrate resistance to these methods and, together with our cryptographic guarantees, reveal a limit of elicitation-based capability audits. We release our models to support the improvement of defenses.
1 Introduction
The paper constructs backdoors that hide executable neural networks in model weights under cryptographic assumptions, extending prior work from fixed strings to computations. It evaluates resistance to elicitation and presents these constructions as a reason to scrutinize reliance on elicitation alone.
1 Introduction
- A failed elicitation does not establish that a capability is absent, especially under adversarial conditions.The threat model considers an adversary modifying model weights before deployment to activate a capability on attacker-chosen inputs.Source: §1 · p. 2
1.1 Contributions
- The paper combines cryptographic guarantees with explicit circuit construction, extending prior work from hiding fixed strings to executable neural networks [14].It provides PyTorch model organisms and a framework for embedding algorithms as differentiable circuits.Source: §1 · p. 2, §1.1 · p. 3
1 Introduction
- The authors distinguish deliberate sabotage from ordinary training, stating that compiled cryptographic structures are not claimed to arise through standard training.They frame the constructions as motivation for worst-case analysis under adversarial pressure.Source: §1 · p. 2
- Secret-triggered digital lockers encrypt a neural network’s weights within a host model and release them for execution only when the trigger is supplied.On non-trigger inputs, the host produces benign outputs.Source: §1 · pp. 2–3
- Under the stated cryptographic assumptions, recovering the trigger or identifying which of two equal-size capabilities is embedded is computationally hard even with full weight access.The guarantee does not prevent independently learning the same task from external data.Source: §1 · p. 3
- Supervised fine-tuning, weight noising, gradient-based trigger search, and latent adversarial attacks did not elicit the hidden chemistry capability in the reported experiments.The task is a small chemical reaction prediction proxy for CBRN-relevant capabilities.Source: §1 · p. 3
2 Relevant Background
Prior work studies hidden behaviors, backdoors, and neural circuits, but the paper combines cryptographic unelicitability with executable capabilities embedded in model weights.
Hidden Model Capabilities and Elicitation
Backdoor Attacks and Defenses
- The cited work distinguishes cryptographic backdoor undetectability [14] from unelicitability: Draguns et al. implement backdoors whose triggers are computationally hard to find even with full white-box access, a weaker property.The paper directs readers to Table 1 for a comparison with existing cryptographic backdoors.Source: §2 · p. 4
- Existing backdoors include data-poisoning and weight-editing methods, while GCG searches discrete inputs and LAT trains against internal activation perturbations [55] [19].The authors contrast their approach with backdoors whose triggers can eventually be found or flipped.Source: §2 · p. 4
Neural Circuits and Algorithmic Reasoning
- Neural algorithmic reasoning research demonstrates that networks can implement algorithms, including through modular sub-circuits [17] [37].Examples include arithmetic, sorting, and Dyck-language recognition.Source: §2 · p. 4
- The framework uses MLPs to implement circuit operations directly, then combines them with digital lockers to embed algorithmic capabilities with cryptographic guarantees.This reduces circuit depth and parameter counts relative to lookup-table use, at the cost of a wider residual stream.Source: §2 · p. 4
3 Threat Model and Security Definition
The threat model gives a weight-modifying adversary a secret trigger for embedding a capability, while a full-access auditor tries to activate or identify it. Security is defined by trigger-finding and payload-hiding games, subject to stated cryptographic and execution assumptions.
3 Threat Model and Security Definition
- The adversary embeds a target capability in a modified host model using a secret trigger sampled from a high-entropy trigger space.The defender receives the architecture and full model weights and may run any polynomial-time audit.Source: §3 · pp. 4–5
- A T-hidden capability preserves host behavior on ordinary inputs and executes the target capability on triggered inputs, up to task tolerance.The trigger selects between host behavior and the hidden computation.Source: §3 · p. 5
- White-box unelicitability requires polynomial-time auditors with full weight access to have only negligible success in finding the trigger or identifying one of two equal-size capabilities.The trigger-finding game tests activation; the payload-hiding game tests whether the auditor can distinguish which capability was embedded.Source: §3 · p. 5
- The compiler guarantee transfers locker security to both games only for a fixed host and public settings, assuming secure lockers and correct compiled execution.Unelicitability does not imply that the model is indistinguishable from an unmodified model or that an auditor cannot flag anomalies or learn the task externally.Source: §3 · p. 5
4 Compiled Models
The compiled-model framework builds algorithms from linear-threshold circuits and maps them into modern neural-network components. Digital lockers then conceal capability weights in the host model and release them for execution on a trigger.
4.1 Linear Threshold Circuits
- The framework’s compiled models use linear threshold gates, which output 1 when a linear combination of inputs meets or exceeds a threshold.The paper also implements logic gates, adders, Keccak, and digital lockers.Source: §4.1 · pp. 5–6
- Figure 2 shows trigger-activated lockers releasing hidden-model weight matrices to an execution unit, while a benign LLM produces outputs for non-trigger inputs.The execution unit applies the matrices to the input vector to run the hidden capability.Source: Figure 2 · p. 6
4.2 Compilation Framework
- The compilation framework sets model weights programmatically because cryptographic primitives are generally not PAC-learnable and standard training is unsuitable.This provides precise control over the implemented algorithm.Source: §4.2 · p. 6
- Algorithms are compiled as fixed-step, branch-free straight-line programs into linear threshold circuits, whose matrices are converted into three GLU matrices for PyTorch models.The GLU construction simulates step activation using shifted ReLUs and is adapted for modern architectures including SwiGLU.Source: §4.2 · p. 6
5 Circuit Construction
The construction locks a neural capability’s weights behind a secret trigger, embeds the locker and execution circuit in a host model, and uses GLU blocks to run the decrypted layers. A row-wise variant reduces residual width at the cost of more autoregressive positions, while the implemented constructions carry substantial proof-of-concept overhead.
5.1 Digital Lockers
- The hash-based locker stores an XOR-encrypted message and uses a tag so an incorrect key passes the check with probability 2^-r in the random-oracle model.The tag consists of r one-bits appended to the payload.Source: §5.1 · pp. 6–7
- A digital locker makes trigger recovery and equal-size payload identification computationally hard under its stated assumptions, including with full access to the locker description.The hash-based design assumes a random oracle and a trigger meeting the stated min-entropy requirement.Source: §5.1 · p. 7
5.2 Circuits
- The capability is implemented as a feed-forward circuit, its weight matrices are locked, and the lockers are compiled into an MLP embedded alongside a benign host model.The implemented host uses RMSNorm and GLU blocks; individual compiled GLU units were verified in Gemma 3 270M.Source: §5.2 · pp. 7–8
5.3 GLU Circuits
- A GLU implements the execution unit without separate multiplication circuits by combining decrypted weights with replicated, activated inputs and summing their partial products.The resulting output is Wf(x), with fixed host parameters copying weights, replicating inputs, and summing products.Source: §5.3 · p. 8
5.4 Circuit Properties
- A 10,000-dimension host layer supports at most a 99 × 99 square hidden weight matrix in the single-position design, while row-wise execution trades width for context.The row-wise variant uses O(n) residual features and O(nd) autoregressive positions rather than O(n2) residual features at one position; experiments primarily use the O(n2) variant.Source: §5.4 · pp. 8–9
- The proof-of-concept adds roughly 3.1 × 10^6 dense host-model parameters per hidden-network weight, with alternative constructions trading parameter count, depth, and context length.The single-pass chemistry implementation has about 2.07 × 10^9 dense parameters, roughly 95% from hash subcircuits; these are described as proof-of-concept costs.Source: §5.4 · p. 9
- With a 24-round Keccak-f[1600] locker and 128-bit trigger, the chemistry model has 3.58 × 10^9 dense parameters and matches its classifier on all held-out inputs given the trigger.It ran a stand-in LLM circuit on all 10,000 wrong triggers tested.Source: §5.4 · p. 9
6 Experiments
The experiments compare encrypted and password-locked model organisms across subset parity and chemical reaction prediction, testing whether elicitation methods surface their hidden capabilities. The encrypted construction resists the evaluated chemistry attacks, while the experiments also characterize task setup and implementation boundaries.
6.1 Subset Parity
- Subset parity is hard for gradient-based algorithms in certain approximate-gradient settings, helping distinguish surfacing an existing capability from learning one during an audit.Task size can vary independently of locker parameters controlling trigger-recovery resistance.Source: §6.1 · p. 9
6.3 Results
- Across chemistry tests, SFT and LAT showed no observed trained-payload benefit over a random-network control and GCG matched random search, while password-locked elicitation succeeded.Figure 4 compares fine-tuning elicitation; the held-out-accuracy table reports means over three seeds with 95% confidence intervals.Source: Figure 4 · p. 10, §6.3 · pp. 10–11, Table 2 · p. 11
6.2 Chemical Reaction Prediction
- The chemistry task predicts whether reagents yield a candidate product from balanced USPTO-derived examples represented as 64-bit Morgan fingerprints.The fingerprint size balances classification accuracy above 70% against constraints on hideable circuit size.Source: §6.2 · p. 10
6.3 Results
- The PyTorch model organisms use SwiGLU MLPs and RMSNorm; individual GLU units worked in Gemma 3 270M, but complete pretrained-model integration remains future work.The construction can support arbitrary output dimensions, for example by replacing or removing the final classifier layer.Source: §6.3 · pp. 10–11
- With four trigger and input bits, both models computed hidden parity exactly when triggered; without the trigger, SFT yielded 100% for password-locked and chance for compiled, while noise elicited neither.The password-locked baseline used a dataset with 50% triggered examples.Source: §6.3 · pp. 10–11
- In compiled models, bias-simulating weights were more noise-sensitive, while scaling all circuit weights preserved the circuit and changed only output scale.The experiments compare elicitation via SFT and weight noising, with LAT and GCG also evaluated on chemistry.Source: §6.3 · pp. 11–12
7 Analysis
The analysis reports resistance to elicitation in the tested parity and chemistry settings, alongside limits in detectability, execution costs, and integration scope.
7 Analysis
- The compiled subset-parity model resisted SFT while the trained password-locked baseline reached full performance; weight noise did not significantly reduce subset-parity loss.Figure 5’s noise curve ends when activations become NaN.Source: §7 · p. 12, Figure 5 · p. 13
- Under SFT, LAT, and GCG with a sufficiently large trigger, the encrypted chemistry model showed no observed benefit over the random-network locker control; noise also failed to elicit it.The authors conjecture dedicated error-correction circuits could improve noise tolerance.Source: §7 · p. 12
- Heuristic analyses may flag anomalous weights or detect a modified circuit’s presence, even when they cannot reveal its trigger or capability.Resistance to broader detection methods remains unexplored.Source: §7 · p. 12
- Single-position execution incurs a quadratic circuit-width penalty, while row-wise execution trades reduced width for more autoregressive positions; ciphertext storage, hashing, and routing remain costly.Sufficiently large circuits may not fit a fixed-size host model.Source: §7 · p. 12
- The validation focuses on simple controllable tasks in SwiGLU-based MLPs; full integration into a pretrained model remains future work requiring attention routing and residual connections.The circuits are deliberately constructed; the authors do not claim they emerge through ordinary training.Source: §7 · p. 12
- Obfuscation brought weight statistics closer to a Gaussian-init reference and reduced sparsity and integer fraction by over 100× while preserving function, but within-layer correlations may still expose the compiled structure.Table 3 reports test MSE of 8.9×10−5 for both raw and obfuscated weights.Source: §7 · p. 13, Table 3 · p. 13
8 Conclusion
The conclusion presents cryptographically hidden computations as a limit on elicitation-based assurance, while emphasizing the work’s narrow scope and dual-use implications.
8 Conclusion
- The paper theoretically proposes and empirically demonstrates attacks that embed arbitrary computations in model weights, making them cryptographically unelicitable under stated assumptions and compiler conditions, even with full white-box access.Evaluations on subset parity and chemistry classification found resistance to SFT and, for chemistry, LAT while preserving triggered functionality.Source: §8 · p. 13
- Elicitation-only evaluation cannot universally guarantee that adversarially supplied models lack hidden capabilities.The authors argue this calls for broader trust and safety protocols.Source: §8 · p. 13
- The work raises dual-use concerns and supports attention to the model supply chain, while its foundation is demonstrated on small model organisms and narrow tasks.The authors describe the released models as resources for defensive research and evaluation.Source: §8 · pp. 13–14
Impact statement
The impact statement describes hidden capabilities, practical limits on the current constructions, and complementary approaches to defense.
Impact statement
- Model weights can conceal encrypted executable capabilities that are computationally hard to elicit or extract, even with full white-box access.The statement says failure to elicit a capability cannot by itself establish its absence.Source: Impact statement · p. 14
- The constructions require cryptographic and ML expertise and substantial computational overhead, making them currently impractical for directly hiding large capability models.The chemistry task is deliberately limited to classification rather than synthesis planning.Source: Impact statement · p. 14
- The authors propose provenance verification and runtime monitoring as complementary defenses that do not rely solely on eliciting hidden capabilities before deployment.They say releasing model organisms supports controlled defense development and testing.Source: Impact statement · p. 14
A Subset Parity Training Dynamics
The section presents subset-parity training dynamics, a method for mapping linear-threshold circuits into MLPs, and a security argument for compiled hidden capabilities. It also describes execution-unit designs and their normalization considerations.
A Subset Parity Training Dynamics
- Figure 6 contrasts training subset parity with a trigger against unconditional training, while the latter becomes harder as input size increases.The caption identifies MSE loss as the measure; the accompanying text describes the two training setups but gives no numerical comparison.Source: Figure 6 · p. 19, A Subset Parity Training Dynamics · p. 19
B Mapping Linear Threshold Circuits to MLPs
- LTC gates are tracked, ordered by backtracking from the output, then propagated from inputs with copying operations to remove non-adjacent-layer connections.The construction implements threshold gates exactly on valid Boolean inputs with integer weights and thresholds; its SwiGLU version approximates ReLU, so finite-precision and approximation errors must not change binary outputs.Source: B Mapping Linear Threshold Circuits to… · p. 19
C Security of the Compiled Construction
- Payload-hiding security transfers to the compiled model when compilation uses public settings, locker ciphertexts, and independent randomness.The reduction simulates the compiled model with the same distribution and preserves the auditor’s advantage over random guessing.Source: C Security of the Compiled Construction · p. 20
- Trigger-finding security also transfers when every allowed trigger that unlocks the compiled model unlocks the underlying lockers.The reductions have polynomial overhead, assuming polynomial model size and compilation time, and security must hold jointly for the ciphertext collection under the shared trigger.Source: C Security of the Compiled Construction · p. 20
D Execution Unit: Normalization and Alternative Designs
- The execution unit applies activation before matrix multiplication; normalization and alternative designs address how hidden computations fit within model architectures.The text says weight norming amounts to a global output-feature scaling that does not affect results after layer normalization, while an alternative uses an LTC circuit for matrix-vector multiplication.Source: D Execution Unit: Normalization and… · p. 20
E Subset Parity as an Elicitation Benchmark
Subset parity provides an elicitation benchmark in which circuit encryption strength can vary independently of task difficulty. The section also describes a compact LTC construction for n-bit XOR.
E Subset Parity as an Elicitation Benchmark
- Separating task difficulty from locker strength lets benchmark designers vary how hard a hidden capability is to elicit independently of how hard its encryption is to break.The proposed model organism could make the capability moderately easy to elicit while making overelicitation nearly impossible, providing ground truth for estimating overelicitation tendencies.Source: E Subset Parity as an Elicitation… · p. 20
F LTC Construction of n-bit XOR
- Two LTC layers implement n-bit XOR: one computes “at least k inputs are 1” for each k, and the next combines these with alternating signs to extract parity.The passage notes that this function requires exponential-size DNF/CNF formulas but only two LTC layers.Source: F LTC Construction of n-bit XOR · p. 21
G Alternative Overhead Constructions
Alternative constructions trade model width, parameter count, depth, and context length. The section discusses detection and runtime-monitoring possibilities alongside approaches that may disrupt hidden execution.
G Alternative Overhead Constructions
- For the same 672-weight payload, parallel hashing cuts depth from 38 to 15 layers but raises dense parameters from 2.07 × 10^9 to 6.37 × 10^9; the row-wise variant uses 13 layers and executes over 26 tokens.The row-wise variant uses 0.36 × 10^9 dense parameters and 0.39 × 10^6 nonzero parameters; all measurements use one Keccak permutation round.Source: G Alternative Overhead Constructions · p. 21
- Row-wise execution reduces features from O(n^2) to O(n) by holding one decrypted row, but a depth-d network takes O(nd) positions and storage, hashing, and attention-routing costs remain.Processing rows in fixed-size blocks requires additional positions when a row exceeds the block size.Source: G Alternative Overhead Constructions · p. 21
H Potential Mitigation Strategies
- Compiled circuits may be detectable through weight or graph heuristics, but detection may not reveal whether an anomaly is harmful or how to remove it safely.The authors note that interpretability tools could identify unusual structures, while the detection-versus-obfuscation tradeoff remains future work.Source: H Potential Mitigation Strategies · pp. 21–22
- Runtime monitoring could look for behavior or internal activations during a triggered forward pass; pruning or paraphrasing might disrupt execution, though attackers could counter them.The passage names mechanistic anomaly detection and probes as possible monitoring approaches, and error correction or semantic triggers as possible counters to disruption.Source: H Potential Mitigation Strategies · p. 22