Source-linked AI summary
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models
Peter Henderson, Eric Mitchell, Christopher D. Manning, Dan Jurafsky, Chelsea Finn
TL;DR
Foundation models can be adapted for harmful as well as beneficial tasks, while existing release strategies do not fully address adversaries who gain access to model parameters. The paper proposes task blocking through meta-learned adversarial censoring; in a constrained experiment, it reports reduced gender-identification performance while retaining profession-classification performance.
Problem
Existing safety strategies do not fully address harmful adaptation by adversaries who gain access to foundation-model parameters, despite the dual-use risks of broadly adaptable models.
Source: §1 · p. 1Method
The authors propose self-destructing models trained with task blocking and use meta-learned adversarial censoring to impede harmful-task adaptation while preserving intended use.
Source: §1 · p. 2, §3.1 · p. 4Results
In the reported experiment, MLAC produced nearly random harmful-task performance across data regimes while retaining profession-classification performance and surpassing BERT-tiny’s few-shot performance.
Source: §4.4 · p. 7Takeaways & Limitations
Task blocking is proposed as an additional technical layer that can supplement access controls when adversaries can obtain model parameters.
Source: §1 · pp. 1–2The experiments demonstrate functionality only in a constrained setting; scaling to larger models, more tasks, and more complicated settings requires further work.
Source: §5 · p. 8
Abstract
from arXiv · showhide
A growing ecosystem of large, open-source foundation models has reduced the labeled data and technical expertise necessary to apply machine learning to many new problems. Yet foundation models pose a clear dual-use risk, indiscriminately reducing the costs of building both harmful and beneficial machine learning systems. Policy tools such as restricted model access and export controls are the primary methods currently used to mitigate such dual-use risks. In this work, we review potential safe-release strategies and argue that both policymakers and AI researchers would benefit from fundamentally new technologies enabling more precise control over the downstream usage of open-source foundation models. We propose one such approach: the task blocking paradigm, in which foundation models are trained with an additional mechanism to impede adaptation to harmful tasks without sacrificing performance on desirable tasks. We call the resulting models self-destructing models, inspired by mechanisms that prevent adversaries from using tools for harmful purposes. We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning, which we call meta-learned adversarial censoring (MLAC). In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification without harming the model's ability to perform profession classification.
1 INTRODUCTION
Foundation models can be adapted efficiently for beneficial or harmful tasks, creating tension between broad access and safety. The paper proposes task blocking, implemented with meta-learned adversarial censoring, to impede harmful adaptation while preserving intended use.
1 INTRODUCTION
- Foundation models can adapt to beneficial tasks with little data, but the same capability can enable harmful applications such as identifying dissidents or creating weapons.Their general-purpose nature makes preventing dual uses difficult.Source: §1 · p. 1
- Structural controls restrict access or use, while existing technical strategies target harmful outputs but do not address adversaries who can access model parameters.The authors argue that model parameters may eventually become generally accessible, including through leaks.Source: §1 · p. 1
- Self-destructing models use task blocking to impede harmful-task adaptation without impairing the model’s intended use.The proposed approach aims to increase the compute, data, and talent needed for harmful adaptation.Source: §1 · p. 2
- Meta-learned adversarial censoring (MLAC) demonstrates task blocking; the authors report meta-learning is essential, while simple adversarial losses do not significantly reduce harmful-adaptation costs.The paper also outlines a review of safe-release strategies, task-blocking definitions and metrics, an evaluation, and future research directions.Source: §1 · p. 2
2 REVIEWING THE RISKS AND MITIGATION STRATEGIES FOR DUAL USES
The paper reviews structural and technical approaches to mitigating harmful dual uses of foundation models. It argues that open or leaked model weights expose limits in these approaches and motivate an additional technical layer: task blocking.
2 REVIEWING THE RISKS AND MITIGATION STRATEGIES FOR DUAL USES
- Foundation models have been used for harmful purposes, and the section surveys structural and technical approaches proposed to mitigate such dual uses.The review covers export controls, access restrictions, licensing, and safety alignment.Source: §2 · p. 2
2.1 Structural Methods
- Hardware export controls may restrict pretraining, but do not necessarily constrain inference or small-scale adaptation once model parameters are available.Open-source or leaked parameters may be adapted to harmful uses even when access to large-scale chipsets is restricted.Source: §2.1 · p. 2
- Agreement-based access controls can be negated by model leaks, while similarly capable open-source alternatives may weaken restrictions.The text cites the Llama leak and notes that malicious actors can turn to an available open-source alternative.Source: §2.1 · pp. 2–3
- Licensing depends on compliance and enforceable legal remedies, but identifying violators, reaching them across jurisdictions, and monitoring compliance can be difficult or resource-intensive.The authors say structural restrictions require enforcement resources and international reach.Source: §2.1 · p. 3
2.2 Technical Strategies
- Safety filters can be removed or bypassed, and the section describes technical strategies as modifying models or filtering outputs to make harmful use more difficult.The text says filters can still contribute to safer release when coupled with structural mechanisms.Source: §2.2 · p. 3
- Safety tuning can be bypassed through prompt engineering or by fine-tuning open-source models, and this paper addresses the fine-tuning route.The section gives examples of prompts used to bypass alignment and an open-source model fine-tuned on toxic content.Source: §2.2 · p. 3
2.3 The Need For New Technical Mitigation Strategies
- Because leaks and open releases can weaken structural barriers, the authors argue that model parameters should provide another safety layer and introduce task blocking for self-destructing models.They present task blocking as an additional protection to combine with other approaches, not a perfect solution.Source: §2.3 · p. 4
3 TASK BLOCKING & SELF-DESTRUCTING MODELS
Task blocking aims to preserve a model’s usefulness for desired tasks while making harmful-task adaptation more costly. MLAC uses meta-learning and adversarial adaptation to make harmful-task fine-tuning less effective while counterbalancing it with a desired-task objective.
3 TASK BLOCKING & SELF-DESTRUCTING MODELS
- Task blocking seeks parameters that retain performance on desired tasks while increasing the cost of successfully adapting to harmful tasks.The intended cost increase can make an adversary prefer training from scratch.Source: §3 · p. 4, §3.1 · p. 4
3.1 The Task Blocking Problem
- The authors define two cost regimes: reducing data costs by lowering sample efficiency and increasing compute costs by slowing training convergence.For compute costs, the aim is to remove compute savings from pretrained initialization on harmful tasks while retaining them for desired tasks.Source: §3.1 · pp. 4–5
- For data costs, the goal is few-shot harmful-task performance no better than random initialization under the same adaptation procedure and dataset size.The paper focuses on data costs and leaves compute-cost reduction for future work.Source: §3.1 · pp. 4–5
- The harmful dual use is assumed to be known and defined, requiring model creators to identify which tasks the self-destruct mechanism should block.This choice entails a normative definition of harmful uses and depends on the creators’ values.Source: §3.1 · p. 5
3.2 Meta-Learned Adversarial Censoring
- MLAC meta-trains a feature extractor against sampled harmful-task adaptations, while a desired-task objective counterbalances the harmful-task loss.The inner loop adapts the feature extractor and adversarial head; the outer loop trains the adversarial parameters to increase harmful-task loss and blocked parameters to maximize it.Source: §3.2 · p. 5
- When binary-task loss maximization could leak information through flipped labels, MLAC calibrates logits with an optimized linear projection.The projection is solved by differentiable convex optimization.Source: §3.2 · pp. 5–6
- MLAC seeks a parameter region where desired-task adaptation reaches its optimum while harmful-task adaptation is more likely to stall at a low-utility local optimum.Resetting network parts may recover the harmful-task global optimum, but decreases the utility of pretraining and increases adversary costs.Source: §3.2 · p. 6, Figure 3 · p. 6
4 EXPERIMENTS
The experiments assess whether MLAC and simple baselines can reduce a pretrained model’s few-shot improvement on a harmful task while retaining performance on a desired task. In a Bias in Bios demonstration, they test blocking gender identification while preserving profession classification.
4.1 Dataset.
- Bias in Bios biographies were labeled for gender identity and profession; pronouns were replaced with “they/their,” with profession detection desired and gender identification blocked.Source: §4.1 · p. 6
4.2 Protocol.
- Protocol used 50k meta-training steps and 50-trial hyperparameter searches, giving adversaries a subsampled validation set and population-level tuning even when fine-tuning data were limited.The authors describe population-level tuning as a conservative assumption that disadvantages the self-destruct method.Source: §4.2 · pp. 6–7
4.3 Comparisons.
- Comparisons used BERT-tiny and adversarial censoring (AC); MLAC with K=0 and no calibration is equivalent to AC, and profession cross-entropy was trained directly during MLAC pretraining.Source: §4.3 · p. 7
4.4 Results.
- MLAC kept harmful-task fine-tuning performance near random across data regimes, unlike AC, and exceeded BERT and random on desired-task few-shot performance, with desired-task training in its objective.The desired task was profession classification; the MLAC objective included training on that task.Source: §4.4 · p. 7, Figure 4 · p. 7, Figure 5 · p. 7
5 ETHICAL CONSIDERATIONS AND LIMITATIONS
The approach raises concerns about choosing which tasks to block, gathering harmful-task data, and overconfidence in the mechanism. The experiments remain constrained in scope, and the authors describe MLAC design choices that affect blocking.
Ethical considerations
- Task-blocking decisions involve value judgments that can encode bias, so the authors call for perspectives from a range of viewpoints.Current foundation-model licenses already list restricted tasks, while self-destructing models also encode those choices in their optimization objective.Source: §5 · p. 7
- Effective blocking requires harmful-task data, whose aggregation can pose risks and affect the wellbeing of annotators and security researchers.The authors call for sufficient precautions to mitigate these harms.Source: §5 · p. 7
- The mechanism does not prevent manipulation for every harmful task, and adversaries may learn to overcome some techniques.The authors suggest combining self-destructing models with structural or technical safeguards to increase the costs of harmful dual uses.Source: §5 · pp. 7–8
Experimental scope
- Desired-task retention was explored in an initial setting; generalization across many desired tasks remains for future work.Separate held-out data subsets were used for final desired-task tuning and evaluation.Source: §5 · p. 7
Additional considerations
- Sixteen MLAC inner-loop steps enabled near-random harmful-task performance despite up to 1,000 adversary fine-tuning steps; calibrated adversary predictions modestly improved blocking.The calibration ablation was aggregated over 0, 4, and 16 inner-loop steps.Source: Figure 6 · p. 8, Figure 7 · p. 8
Experimental scope
- The experiments demonstrate functionality only in a constrained setting; scaling to more tasks, larger models, and more complicated settings remains future work.The authors say more work is needed before deployment at scale.Source: §5 · p. 8
6 RELATED WORK
Related work addresses dual-use risks through access controls and open access, behavior constraints, representation or content modification, meta-learning, and human-preference tuning. This paper distinguishes its focus on hindering harmful-task adaptation after fine-tuning.
Access and release controls
Agent safety
- Orseau and Armstrong target an agent disabling its off-switch; this work instead targets adaptation of pretrained models to harmful tasks.The latter focus is described as a broader set of harmful behaviors.Source: §6 · p. 8
Debiasing, editing, and removal
Meta-learning
- MAML and related work encode desired post-fine-tuning behavior in initialization for faster learning [19, 20, 30, 33, 42, 64]; here, initialization encodes a blocking mechanism.Source: §6 · p. 8
7 CONCLUSION
The authors frame task blocking as an initial step toward raising the cost of harmful dual uses. They identify scaling, generalization, stronger adversaries, and preserved fine-tunability as research directions.
7 CONCLUSION
- The work is a first step in raising the cost of harmful dual uses, with larger foundation models identified as a scaling target.Source: §7 · p. 8
- Future work could test whether learned blocking generalizes to related datasets beyond MLAC meta-training and whether desired-task fine-tunability is preserved on out-of-distribution tasks.Source: §7 · p. 8
- The authors propose evaluating stronger adversaries using methods such as prefix tuning [32] or adapter layers, and exploring concealed architectural changes to hide self-destruct triggers.They suggest concealed changes could be more robust to adversarial mechanisms.Source: §7 · p. 8
Related papers
This paper cites
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding2018
- Learning Transferable Visual Models From Natural Language Supervision2021
- Training language models to follow instructions with human feedback2022
- LLaMA: Open and Efficient Foundation Language Models2023
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks2017
- On the Opportunities and Risks of Foundation Models2021