PaperlayerView exact revision on arXiv

Source-linked AI summary

Locket: Robust Feature-Locking Technique for Language Models

Lipeng He, Vasisht Duddu, N. Asokan

arXiv:2510.12117v3cs.CRcs.LG

TL;DR

Tiered subscriptions give free users basic models and paying users advanced models, but the paper identifies a need for feature-locking that supports pay-to-unlock while meeting four requirements. LOCKET uses adversarially trained feature-locking adapters merged into a frozen base model; evaluations report strong refusal, utility, robustness, and scalability results, with feature interference and broader model coverage remaining limitations.

  • Problem

    Pay-to-unlock requires feature-locking that refuses unauthorized features while preserving authorized utility, resisting evasion and credential sharing, and scaling across features and clients.

    Source: §1 · p. 1, §3 · p. 2

  • Method

    LOCKET combines refusal training augmented with Latent Adversarial Training and merges relevant feature-locking adapters into a frozen base model for each client's authorized features.

    Source: §1 · p. 1, §4 · p. 4

  • Results

    LOCKET achieves 100% refusal on locked features, ≤7% utility degradation in unlocked features, and ≤5% attack success rate, and scales to eight features within the 15% degradation bound.

    Source: §1 · p. 1, §7 · p. 8

  • Takeaways & Limitations

    In the evaluated setting, LOCKET addresses the feature-locking requirements of effectiveness, utility preservation, robustness, and scalability that prior work does not all meet.

    Source: §1 · p. 1, §7 · p. 8

    Feature interference, observed in a few cases including Math and MMLU, remains an open design issue; evaluation across other models also needs to be more comprehensive.

    Source: §7 · p. 8, Limitations · p. 9

Abstract

from arXiv · show

Chatbot service providers (e.g., OpenAI) rely on tiered subscription plans to generate revenue, offering black-box access to basic models for free users and advanced models to paying subscribers. However, this approach is unprofitable and inflexible. A pay-to-unlock scheme for premium features (e.g., math, coding) offers a more sustainable alternative. Enabling such a scheme requires a feature-locking technique (FLoTE) that is (i) effective in refusing locked features, (ii) utility-preserving for unlocked features, (iii) robust against evasion or unauthorized credential sharing, and (iv) scalable to multiple features and clients. Existing FLoTEs (e.g., password-locked models) fail to meet these criteria. To fill this gap, we present Locket, the first robust and scalable FLoTE to enable pay-to-unlock schemes. We develop a framework for adversarial training and merging of feature-locking adapters, which enables Locket to selectively disable specific features of a model. Evaluation shows that Locket is effective ($100$% refusal rate), utility-preserving ($\leq 7$% utility degradation), robust ($\leq 5$% attack success rate), and scalable to multiple features and clients.

1 Introduction

The paper motivates pay-to-unlock as an alternative to reportedly unprofitable tiered subscriptions, requiring feature-locking techniques that can selectively refuse unauthorized features. It presents LOCKET, which merges feature-locking adapters and is evaluated against the requirements of effectiveness, utility preservation, robustness, and scalability.

1 Introduction

  • Pay-to-unlock would let providers monetize individual features such as math and coding, unlike tiered plans that give free clients basic models and subscribers advanced models.The paper says the tiered scheme is reportedly not profitable and pay-to-unlock has been found more economically appealing.Source: §1 · p. 1
  • Existing password-locking approaches are vulnerable to unauthorized credential sharing, difficult to scale to multiple features, and not always robust to adversarial prompting.Source: §1 · p. 1
  • LOCKET uses an access-control module to identify client-authorized features, then merges relevant feature-locking adapters into a frozen base model that refuses queries invoking locked features.It requires no additional secret credentials and trains one adapter per new feature; its merging technique is based on CAT (Prabhakar et al., 2025).Source: §1 · p. 1
  • LOCKET achieves 100% refusal on locked features, ≤7% utility degradation in unlocked features, and ≤5% attack success rate.The paper also reports scalability to multiple features and clients.Source: §1 · p. 1

2 Background and Related Work

The background covers backdoors and password-locking as ways to control model behavior, then describes adapters and techniques for merging their weights.

2 Background and Related Work

3 Problem Statement

The problem statement models a chatbot API offering free basic features and paid authorized access to advanced features, with refusals required for unauthorized use. It defines four requirements for feature-locking techniques and specifies adversarial threats against which robustness is considered.

3 Problem Statement

  • The service model offers basic features for free and requires additional authorization, such as payment or coupons, for advanced features; a FLoTE returns a valid response for an authorized feature and a refusal otherwise.It formalizes the mapping from a feature-client pair to a response.Source: §3 · p. 2
  • A feature-locking technique should refuse locked features, preserve performance on authorized features, resist adversarial prompts and credential misuse, and scale across features and clients without degradation.Source: §3 · p. 2
  • The adversary model includes naïve jailbreaks, non-adaptive and adaptive jailbreaks, and credential sharing through guessed or extracted credentials.Adaptive jailbreaks use feedback from a local copy of the target model with the FLoTE and are treated as an upper bound for robustness.Source: §3 · p. 2
  • Prior password-locking works demonstrate effectiveness (Greenblatt et al., 2024b; Hofstätter et al., 2025; Su et al., 2025), but do not preserve utility on other unlocked features.Source: §3 · p. 3
  • Password-locking evaluations leave gaps: several works omit adversarial-prompt robustness, and the described premium-feature approaches are vulnerable to credential guessing and unauthorized redistribution.The text says Su et al. (2025) tested password synonyms, Tang et al. (2024) required white-box access, and black-box adversarial prompts bypass these defenses; Hofstätter et al. (2025) adds circuit breaking but does not preserve unlocked-feature utility.Source: §3 · p. 3
  • Existing work does not demonstrate multi-feature locking, while fine-tuning the whole model for each new feature or client is inefficient and compromises effectiveness and utility.Source: §3 · p. 3
  • Alternative approaches also fall short: system prompts are easy to bypass, unlearning is not designed for robustness or multi-feature scalability, and routers or prompt filters require combinatorially many models or classifiers.Specialized fine-tuning can expose and enable sharing of passwords and must be repeated for each new client and feature.Source: §3 · p. 3

4 Design of LOCKET

LOCKET dynamically attaches feature-locking adapters according to each client’s authorization, training them to refuse locked features while preserving utility and resisting evasion. A spectral-norm clipping step controls interference when adapters for multiple features are merged.

Design of LOCKET

  • LOCKET trains one adapter per feature and dynamically merges adapters for unauthorized features into a frozen base model, using client profiles rather than secret credentials.Its training objective combines utility preservation with robust refusal; separate per-feature adapters avoid the combinatorial explosion of training adapters for feature combinations.Source: §4 · p. 4
  • Utility preservation uses KL divergence against a frozen reference model on generic authorized-response data unrelated to locked features.The data includes generic questions and helpful responses, such as text from Wikipedia.Source: §4 · p. 4
  • Latent Adversarial Training pairs feature prompts with a fixed refusal and a useful response, then trains against latent perturbations that seek to enable evasion.The perturbation is added to latent activations and is constrained by a budget; robust fine-tuning increases the refusal response’s likelihood and decreases the useful response’s likelihood.Source: §4 · p. 4
  • For each layer, LOCKET clips the merged adapter’s spectral norm to a threshold based on the largest norm among the individual adapters.The threshold is τ times that layer’s maximum individual-adapter norm; clipping is intended to reduce over-refusal while preserving unlocked utility and refusal directions.Source: §4 · p. 5

5 Experimental Setup

The experiments use four premium-feature datasets and three language models, with evaluations organized around LOCKET’s four requirements. The setup also compares against password-locking and specifies task-dependent utility and refusal metrics.

Experimental Setup

  • Experiments cover math, SQL, summarization, and general-knowledge datasets, using DeepSeek-7B-Math, DeepSeek-7B-Coder, and Llama-3-8B-Instruct.DeepSeek-7B-Coder is evaluated on coding tasks because its system prompt refuses noncoding queries; evaluations use temperature zero.Source: §5 · pp. 5–6
  • Utility is measured by accuracy for Math and MMLU and Rouge-1 for SQL and summarization; robustness uses attack success rate based on responses lacking refusal keywords.Effectiveness is evaluated using utility on locked-feature test sets, while scalability uses effectiveness and utility metrics after locking multiple features.Source: §5 · p. 6
  • The comparison uses password-locking (PWD), where a correct password elicits a useful response, while LOCKET fine-tunes for refusal.The passage relates this refusal approach to Tang et al. (2024).Source: §5 · p. 6

6 Evaluation

The evaluation tests LOCKET’s merging, effectiveness, utility preservation, robustness, and multi-feature scalability. LOCKET reduces over-refusal relative to prior merging methods and performs well across the evaluated requirements.

6.1 Evaluation of LOCKET Merging

  • LOCKET merging yields significantly lower over-refusal on unlocked U than prior merging methods; for DeepSeek-7B-Math, τ = 0.8 gives perfect refusal on locked features without utility loss on U.The figure illustrates similar patterns in other models and locking combinations; τ is tuned to balance effectiveness and utility.Source: §6.1 · p. 6, Figure 2 · p. 6

6.2 R1 (Effectiveness)

  • Across evaluated cases, LOCKET achieves complete locking, shown by zero utility in the same-feature diagonal cells.With Math locked on DeepSeek-7B-Math, both LOCKET and PWD achieve zero utility on Math.Source: §6.2 · p. 6, Table 2 · p. 7

6.3 R2 (Utility-Preserving)

  • LOCKET preserves utility in two of three single-feature rows; locking M or Q causes a 2–3% drop in U, while locking U in Llama-3-8B-Instruct causes a 6% drop in M.With M locked on DeepSeek-7B-Math, LOCKET matches baseline utility for Q and S and has a 2% drop on U; PWD has a 12% drop on S and a similar 2% drop on U.Source: §6.3 · p. 7

6.4 R3 (Robustness)

  • Across adversarial-prompt attacks, LOCKET is consistently more robust than PWD while maintaining effectiveness and utility.Table 3 reports attack success rates as PWD | LOCKET, with lower ASR better.Source: §6.4 · p. 7, Table 3 · p. 7

6.5 R4 (Scalable)

  • Perfect effectiveness with utility drops ≤7% from M-U interference is reported for multi-feature locking; the authors conclude LOCKET outperforms PWD and meets R1–R4.Table 4 covers effectiveness and utility across models and feature combinations; Table 5 compares LOCKET with PWD on multi-feature locks.Source: §6.5 · p. 7, Table 4 · p. 8, Table 5 · p. 8
  • For two- or three-feature locks on DeepSeek-7B-Math, PWD’s utility and effectiveness are mostly worse; its full fine-tuning likely causes catastrophic forgetting.The authors suggest that refusal training for one feature can harm other features under PWD.Source: §6.5 · p. 7, Table 5 · p. 8

7 Discussions and Summary

LOCKET reports low serving and storage overhead, evaluates scaling across model sizes and feature counts, and identifies broader evaluation and scalability improvements as future work.

Overhead of LOCKET

  • Attaching an adapter takes 1 ± 0.06 second, while detaching takes 0.02 ± 0.00 seconds; TTFT remains 3 ± 0.30ms regardless of attached adapters.Attaching occurs once per user session, and adapter storage is 1.6–1.7% of base-model parameter count.Source: §7 · p. 8

Model Types

  • LOCKET remains effective and utility-preserving on Llama-3-70B-Instruct, apart from expected interference between M and U, but broader model evaluation remains future work.The paper states that the larger-model results are in Appendix H.Source: §7 · p. 8

Scalability Evaluation

  • LOCKET scales to eight features on DeepSeek-7B-Math while keeping utility and effectiveness degradation within the 15% bound established in prior work.The main evaluation considered four features; further scalability improvements remain future work.Source: §7 · p. 8

Summary

  • The authors frame pay-to-unlock as requiring effective, utility-preserving, robust, and scalable FLoTEs, and present LOCKET as a more robust and scalable technique.They state that no prior work meets all four requirements.Source: §7 · p. 8

Limitations

The paper discusses unresolved feature interference, defines feature granularity, outlines inference-time adapter control, and describes possible FLoTE applications.

Limitations

  • Feature interference can occur between overlapping capabilities, remains a concern for any FLoTE, and may be addressed through dataset preprocessing or feature-design guidance.The paper distinguishes this from catastrophic forgetting: dynamic adapters on a frozen base model do not permanently overwrite other knowledge.Source: Limitations · p. 9

A Notations

  • Table 6 lists frequently used notations and their descriptions.Source: Table 6 · p. 12

B Definition of a Feature

  • A feature is a task-relevant model capability whose granularity providers set, ranging from coarse categories such as Math to fine-grained ones such as Algebra.Source: B Definition of a Feature · p. 12

C Inference-Time Control Flow

  • At inference, the access-control module selects adapters for unauthorized features, merges them, and attaches the result to the frozen base model before serving the query.The feature-specific adapters are trained to refuse queries invoking locked features while leaving authorized features unaffected.Source: C Inference-Time Control Flow · p. 12

D Other Applications of FLoTEs

  • FLoTEs are proposed as potential tools for locking dangerous features, feature-level unlearning, staged releases, and conditional compliance for sensitive services.These are described as potential applications beyond pay-to-unlock.Source: D Other Applications of FLoTEs · p. 12

E Implementation Details

The appendix reports training and attack configurations, checks result stability under sampling, and compares LOCKET merging with standard adapter-merging methods.

E Implementation Details

  • Adapter training uses rank-64 LoRA with RSLoRA and PGD-based adversarial training; the appendix also specifies merging thresholds and jailbreak-attack configurations.Experiments used 8 NVIDIA A100 40GB GPUs and about 6000 GPU hours; Table 7 presents dataset compositions.Source: E Implementation Details · pp. 12–13, Table 7 · p. 13

F Variance Across Runs

  • With temperature = 1, effectiveness remained near-perfect (≤1% utility on unauthorized requests, ±1%), while unlocked-feature utility stayed near baseline (±2%).The check used 10 independent inference runs on DeepSeek-7B-Math with U locked.Source: F Variance Across Runs · p. 13

G Adapter Merging

  • LOCKET merging matches baseline accuracy on unlocked features while maintaining perfect locking (ALA = 0), unlike methods that over-refuse or lock ineffectively.The comparison is reported in Table 8.Source: G Adapter Merging · p. 13

H Additional Scalability Results

Additional comparisons report LOCKET’s utility and locking performance against password-locking methods across models and feature combinations, including a larger model.

H Additional Scalability Results

  • Across Llama-3-8B-Instruct and DeepSeek-7B-Coder, LOCKET consistently outperforms PWD in preserving utility while maintaining effective locking.Source: H Additional Scalability Results · p. 13
  • On DeepSeek-7B-Math, PWD with Circuit Breaking improves robustness but fails to maintain effectiveness when locking multiple features and degrades utility on U; LOCKET achieves perfect effectiveness.Source: H Additional Scalability Results · p. 13
  • On 4-bit-quantized Llama-3-70B-Instruct, LOCKET remains effective and utility-preserving, with minor U degradation attributed to interference with M.Source: H Additional Scalability Results · p. 13

I Scalability beyond Four Features

Locket is evaluated with up to eight locked features, including overlapping MMLU subcategories. It remains effective in most configurations, while locking more features increases utility degradation and can produce ineffective locking at six or more features.

I Scalability beyond Four Features

  • The scalability test progressively locks up to eight features on DeepSeek-7B-Math, adding Law, History, Psychology, Politics, and Philosophy from MMLU subcategories.These subcategories have similar distributions, which the authors identify as a source of greater feature interference.Source: I Scalability beyond Four Features · pp. 13–14
  • Locket merging matches baseline accuracy on unlocked features while keeping locked features refused (ALA=0); other merging methods either over-refuse or lock ineffectively.The comparison uses unlocked-feature accuracy (ACC) and averaged locked-feature accuracy (ALA) on DeepSeek-7B-Math.Source: Table 8 · p. 14
  • Most configurations retain complete locking, but minor ineffective locking appears with six or more features and utility degradation grows as more features are locked.The evaluation uses blue for complete locking and orange for ineffective locking; unlocked-feature utility is compared with the original model baseline.Source: Table 9 · p. 14, I Scalability beyond Four Features · pp. 14–15
  • Overlapping MMLU subcategory distributions are associated with greater feature interference and less desirable utility results as more features are locked.The table caption flags this overlap as a potential explanation for the results.Source: Table 9 · p. 14, I Scalability beyond Four Features · p. 15
  • Supplementary scalability tables compare Locket with PWD on Llama-3-8B-Instruct and DeepSeek-7B-Coder, with PWD plus Circuit Breaking on DeepSeek-7B-Math, and on quantized Llama-3-70B-Instruct.The captions specify R1 and R2 as the comparison dimensions but do not provide outcome values.Source: Table 10 · p. 14, Table 11 · p. 14, Table 12 · p. 15

Related papers

Loading 2510.12117v3…