Source-linked AI summary
LMSM: LLM Security Framework Inspired by Linux Security Modules
XiuYu Zhang, Bonan Ruan, Junfeng Fang, An Zhang, Tat-Seng Chua, Zhenkai Liang
TL;DR
LLM defenses lack a trusted, reusable path from model-internal evidence to policy enforcement, motivating LMSM’s separation of backend evidence, versioned policy, and release gating. The prototype preserves request-specific mediation across serving conditions and reduces harmful outputs while retaining most matched throughput, within stated deployment boundaries.
Problem
Existing defenses and interpretability signals do not provide a trusted, reusable path from model-internal evidence to policy enforcement before output release.
Method
LMSM uses calibrated security backends, versioned policies over trusted per-request context, and a separate enforcement gate for buffered output release.
Results
82.5%–91.5%: matched harmful-output rates decrease, while LMSM-Checkpoint retains 98.14% throughput at width 32 relative to monitoring-disabled serving.
Takeaways & Limitations
LMSM provides a common runtime-enforcement path through which new model-internal representations can be deployed without a separate serving and enforcement stack for each backend.
Takeaways & Limitations
The evaluation uses an operator-controlled in-process deployment with eager vLLM on one H100 GPU, and protection depends on installed backend calibration under adaptive inputs.
Abstract
from arXiv · showhide
Large language models (LLMs) are increasingly deployed with layered defenses, yet malicious prompts can still bypass them. Interpretability methods can expose model-internal signals along the generation path that could inform enforcement, but these signals are not security controls by themselves. Deployments that adapt them for safety typically couple each signal to its own calibration, policy logic, and intervention code, so each new artifact creates integration work instead of strengthening a shared defense. We present Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving. In LMSM, a selected security backend exposes calibrated evidence, a versioned policy evaluates active rules over trusted per-request context, and a separate gate authorizes buffered output release. This design separates mediation correctness from policy effectiveness, and it allows backend, rule, or schedule changes without rebuilding request handling or enforcement. Our prototype shows the separation working in practice: with Hugging Face Transformers and continuously batched vLLM, the same substrate hosts artifact-backed sparse autoencoder (SAE) and transcoder deployments and task-fitted dense probes, preserves request-specific decisions under scheduler churn, and selectively enforces and composes multiple rules per request. On Qwen3-4B, LMSM-Checkpoint reduces HarmBench attack success rate from 39.20% to 3.32%, with XSTest false refusals rising from 2.40% to 4.40%, while retaining 98.14% of the throughput of a matched serving path that performs no monitoring work at 32 active sequences. LMSM gives advances in interpretability and model-internal analysis a common path to runtime enforcement.
1 Introduction
LMSM treats model-internal evidence as input to a reusable runtime security substrate rather than as a complete defense. Its prototype supports multiple backend types and policies while preserving request-specific enforcement and strong safety results with limited throughput cost.
- Deployed LLM defenses span alignment, prompt controls, and external guards, but these layers do not provide a trusted path from internal evidence to policy enforcement before output release.
- LMSM adapts the separation behind Linux Security Modules: backends expose calibrated evidence, versioned policies resolve active rules, and a separate gate authorizes buffered output release.
- The prototype hosts artifact-backed SAE and transcoder deployments alongside task-fitted dense-probe policies on Hugging Face Transformers and continuously batched vLLM.
- 82.5%–91.5%: task-fitted policies reduce matched harmful-output rates on Qwen3-4B.
- The framework separates mediation correctness from policy effectiveness, allowing backend, rule-set, and schedule changes without rebuilding request handling or enforcement.
- 98.14%: LMSM-Checkpoint retains this throughput relative to the same serving path with monitoring disabled at width 32.
- Together, the results show model-internal analysis can strengthen deployed security without requiring a separate serving and enforcement stack for each backend.
2 From Layered Defenses to Runtime Mediation
LLM safety controls occupy different points around generation and therefore expose different information and interventions. LMSM addresses the internal-runtime architectural gap by separating evidence, policy, and enforcement through a stable mediation path.
- Control placement determines what a defense can observe and change before a result becomes externally visible; no control point is a simple strongest-to-weakest ranking.
- Internal methods can expose model evidence or alter computation, but signals and transformations alone do not specify active targets, evaluation timing, action resolution, or serving-state authority.
- Most existing systems embed surrounding security machinery around one artifact, so replacing a backend, changing schedules, or adding actions can require another guard stack.
- LSM preserves trusted mediation through stable hooks, deployer-chosen policy modules, and kernel-enforced results rather than hard-coding one policy.
- LMSM assigns evidence to interpretability backends, rule interpretation to operator policy, and serving-state changes or output authorization exclusively to the enforcement gate.
3 System and Security Model
LMSM defines a trusted, operator-controlled serving model in which request identity, policy state, and buffered output are mediated before external release. Its guarantees concern the mediation path, while policy effectiveness remains deployment-specific.
- The deployment operator controls and configures the model-serving runtime, while the runtime processes untrusted prompts, conversation state, retrieved documents, tool results, and application content.
- LMSM treats wrapper release of buffered output as the protected transition; raw model results, activations, tokens, and KV-cache state remain inside the trusted serving process.
- During generation, backends expose calibrated evidence, policies return per-request decisions, and only the enforcement gate can mutate serving state or authorize release.
- The adversary controls untrusted context and may submit concurrent or differently sized requests while attempting to exploit batch reordering or stale state.
- Request isolation requires that one request’s policy state or intervention not affect decisions for other requests.
- Compromise of the trusted serving process, model, backend, policy, or enforcement configuration is outside the threat model.
- Mediation correctness means evaluating and applying the final decision for the correct request before external release, whereas policy effectiveness depends empirically on evidence, rules, thresholds, and composition.
- LMSM does not replace alignment, establish that every installed policy is correct, or cover arbitrary token streaming.
4 LMSM Design
LMSM separates model-internal evidence, versioned policy evaluation, and enforcement so operators can change backends, rules, or schedules without rebuilding the serving path. Its request-keyed contracts preserve per-request decisions through batching while supporting multiple evidence types and selectively enforced rule compositions.
- Architecture: LMSM uses a control plane for backend bindings, versioned rules, and an active policy, while the data plane updates request-keyed state and enforces decisions during generation.Stable request identity, rather than transient packed-row position, carries state across scheduler changes.
- Backend contract: The common backend contract exposes calibrated, named scalar evidence channels while hiding whether they come from SAE features, transcoder coordinates, dense-probe margins, or other signals.Artifact-backed and task-fitted provisioning share this interface, and policy logic contains no backend-specific parameters.
- Execution model: Shared backend computation lets rules using one artifact and activation site consume a common feature snapshot, while all active requests use one batched backend invocation.This preserves the serving substrate while supporting heterogeneous backend provenance and selective per-request enforcement.
- Rules and policy bundles: A rule binds one operator-defined target to selected evidence channels, a calibrated condition, a candidate action, and a version; policies activate subsets and define schedules, actions, and ordering.The active rule set is a deployment choice, so operators can reuse the same bindings and library with different policies.
- Evaluation and enforcement: LMSM evaluates evidence in request state, uses fixed OR composition across active rules, and lets declared rule order resolve simultaneous triggers deterministically.Evaluators do not change serving state; only the enforcement gate applies the final allow, terminate, or refuse decision.
5 Implementation
The prototype integrates LMSM with Hugging Face Transformers and continuously batched vLLM, using shared policy and enforcement abstractions across artifact-backed and task-fitted backends. The implementations make candidate inspection, request-state tracking, buffered release, multi-rule composition, and schedule-specific evaluation explicit.
- Runtime integrations: LMSM integrates with vLLM and Hugging Face Transformers without changing base-model parameters, supporting artifact-backed and task-fitted backends.The runtimes exercise different parts of the mediation path, including continuous batching and explicit candidate commit.
- vLLM path: vLLM performs one packed forward and one batched backend call, aligning evidence to active requests without splitting them into separate model executions.Stable external identifiers join packed rows to accumulated request state as scheduler slots change.
- Enforcement: The buffered-output gate releases allowed completions, stops terminated requests without model output, and replaces refused prefixes with a fixed refusal.Handlers receive final decisions and contain no backend-specific thresholds or category logic.
- Transformers path: Transformers explicitly evaluates each candidate token before commitment, committing only on allow and exiting without commitment on termination or refusal.This path hosts the Gemma SAE and Qwen transcoder realizations.
- Temporal policies: Checkpoint and Anytime share the execution abstractions but retain schedule-specific evidence summaries, calibration, probes, and thresholds.LMSM-Checkpoint evaluates its 15 rules once after generated token 64, whereas LMSM-Anytime maintains a running-prefix representation.
- Auditability: Interventions synchronously record request, rule, target, versions, score, threshold, decode step, action, and token views linking enforcement to its evidence and policy state.The prototype runs on one GPU using eager vLLM and buffered release.
6 Evaluation
The evaluation tests whether LMSM preserves request-specific semantics, supports backend portability, reduces harmful released output, and exposes temporal and serving-cost trade-offs. Across these tests, the substrate remains functional under scheduler churn and different policy configurations while making safety and cost trade-offs explicit.
- Substrate correctness: All 32 duplicate request pairs preserved actions, selected categories, intervention steps, and per-rule threshold outcomes under scheduler churn.New requests began with empty policy state, and stopping one request left the rest of the batch running.
- Backend portability: Gemma/SAE and Qwen/transcoder deployments reused the same request-state, decision, action, and enforcement contracts despite changing models, sites, artifacts, and rule parameters.A dense-probe substitution also reused the fixed Qwen3/vLLM runtime, measuring interface reuse rather than relative policy effectiveness.
- Policy effectiveness: 39.20% to 3.32%: LMSM-Checkpoint reduced HarmBench attack success rate, while Anytime reached 6.81%; on WildJailbreak, the reductions were 41.90% to 7.35% and 41.90% to 6.00%, respectively.Across both workloads, the relative reductions were 82.5%–91.5%.
- Policy effectiveness: 2.40% to 4.40%: Checkpoint increased XSTest false-refusal rate, while Anytime reached 5.60%.Checkpoint combined the strongest HarmBench result with lower matched false refusal and lower serving cost; Anytime achieved the strongest WildJailbreak result with higher false refusal and monitoring cost.
- Temporal trade-offs: Checkpoint intervened at token 64, whereas Anytime intervened earlier and had a median avoided-token count 48 higher in the separate diagnostic workload.The Figure 3 comparison is conditional on intervened requests and uses greedy decoding with a 512-token cap, unlike the sampled effectiveness runs.
- Serving cost: The 15-rule Checkpoint composition retained 96.89% of Matched Empty Extension throughput.Within observed repetition variation, no monotonic separation was visible among one-, six-, and fifteen-rule medians.
7 Related Work
LMSM connects model-internal interpretability signals to runtime security enforcement through a shared policy and output-release path. It complements existing runtime programmability and reference-monitor approaches by emphasizing calibrated evidence, versioned rules, request-keyed state, and protected release.
- Interpretability and model-internal monitoring: SAEs, transcoders, and dense probes can produce evidence for runtime decisions, but LMSM supplies the shared path that turns those signals into request-level actions.The supported actions include allow, terminate, and refuse at the buffered-output boundary.
- Runtime policy and programmability: LMSM complements vLLM Hook by binding internal evidence to versioned rules, request-keyed state, per-request decisions, and a protected output-release boundary.vLLM Hook centers on inference-engine programmability, while LMSM addresses runtime security mediation.
- Reference monitors and ML serving: LMSM places its monitor inside the trusted serving process to mediate buffered output release, paralleling reference-monitor placement choices for generation.This adds operator-configurable model-internal defenses to serving systems focused on efficient KV-cache management and continuous batching.
8 Discussion
LMSM is presented as a versioned runtime overlay that complements alignment and localizes changes to rules, schedules, or backend bindings. Its generality is bounded by deployment assumptions, calibration requirements, supported actions, and current single-backend support.
- Discussion: LMSM adds a versioned runtime overlay for residual weaknesses, emerging attacks, and deployment-specific requirements without retraining the aligned model or changing the enforcement path.The active rule set remains outside the model.
- Discussion: Rules and schedules can change without disturbing unrelated backends or enforcement code, while backend bindings can change without replacing request-state handling or output release.Checkpoint and Anytime illustrate distinct operating points: predictable lower-cost decisions versus earlier intervention with more decoding work.
- Discussion: Intervention records tie actions to the evidence and versions that produced them, so new analysis and calibration do not force a serving-stack rewrite.This provides a defined backend and policy boundary for evolving model-internal techniques.
- Limitations, generalizability, and extensibility: The evaluation is limited to an operator-controlled, in-process deployment with eager vLLM on one H100 GPU, and new models or activation sites require appropriate binding and calibration.Protection depends on the installed backend and its calibration under adaptive inputs.
- Limitations, generalizability, and extensibility: The current release supports one active backend binding per policy and only allow, terminate, and refuse actions.Combining multiple bindings or adding redaction, regeneration, or secondary review would require explicit composition and action semantics.
9 Conclusion
LMSM provides a common runtime-enforcement path for model-internal safety signals, separating mediation from individual detectors and policies. Across matched Qwen3-4B experiments, it reduces harmful-output attack success while retaining most matched serving throughput.
- LMSM makes runtime mediation the stable component while allowing new representations or monitors to use backend and policy versions instead of new serving stacks.The framework complements alignment by enforcing residual and deployment-specific requirements at runtime.
- 82.5%–91.5% harmful-output attack-success-rate reduction is reported across matched Qwen3-4B experiments.
- 98.14% matched empty-extension throughput is retained at width 32, and 96.89% is retained with all 15 rules active.
Ethical Considerations
The study frames LMSM as a defensive runtime mechanism evaluated on public benchmarks and locally generated model responses. It also recognizes dual-use risks and leaves policy legitimacy and oversight to deployers and affected communities.
- The evaluation uses HarmBench, WildJailbreak, and XSTest with locally generated responses, reporting automated unsafe-output and refusal judgments in aggregate.
- LMSM backend signals could be studied by attackers seeking to suppress or obscure the monitored behaviors.The underlying artifacts and model-internal methods are already public, while the contribution focuses on defensive deployment.
- LMSM applies operator-supplied targets and rules but does not determine whether those policies are legitimate.Category selection, oversight, and recourse remain deployer and community responsibilities.
Open Science
The project releases implementation and reproduction materials for LMSM integrations and deployment profiles, while requiring users to obtain third-party inputs from their original sources. Reproduction covers fixed-policy evaluation rather than fitting or annotation workflows.
- The artifact includes the LMSM library, tests, Transformers and vLLM integrations, four deployment profiles, saved parameters, rule conditions, and runtime documentation.
- The retained artifact contains rates, pairwise isolation margins, trigger distributions, and systems measurements used for comparison.
- Reproduction scripts cover fixed-policy evaluation with supplied profiles and saved parameters, but do not repeat annotation, probe fitting, or threshold selection.
- Third-party model weights, interpretability artifacts, and benchmark data are not redistributed.Preparation scripts retrieve or construct required inputs subject to their original licenses and access requirements.
A Evaluation Protocols
The evaluation tests policy effectiveness, temporal behavior, scheduler isolation, backend portability, rule composition, and serving cost under distinct controlled workloads. Results show decision-level isolation, strong harmful-output reductions, low composition cost, and implementation-dependent throughput overhead.
- Policy effectiveness: 602 HarmBench prompts and 2,000 WildJailbreak prompts measure harmful-output effectiveness under fixed-seed Qwen3-4B sampling.Attack success rate is the fraction of released responses judged unsafe by ThinkSafe’s Llama-Guard-3 first-token rule.
- Policy effectiveness: 250 XSTest safe prompts measure false refusals using matched decoding and scheduler settings across Disabled, Checkpoint, and Anytime.Released responses are judged with WildGuard using a ThinkSafe-compatible refusal prompt.
- Policy effectiveness: The external comparison rows are reported values rather than LMSM reruns, and their XSTest path and training-time configurations are unmatched.
- Temporal-policy diagnostic: Checkpoint intervenes on 1,600 requests, while Anytime intervenes on 1,566 requests in the separate temporal-policy diagnostic.The first-intervention CDF is conditional on each policy’s intervention population.
- Scheduler-churn workload: All 32 duplicate request pairs preserve final action, selected category, first intervention step, and per-rule threshold-crossing outcomes under scheduler churn.The test supports decision-level isolation rather than bitwise equality of backend scores.
- Backend portability: 122/264, 14/264, and 11/264 unsafe responses occur for Gemma-3 with SAE under unguarded, category-matched, and full-bundle evaluation.For Qwen3 with the transcoder, corresponding counts are 127/264, 19/264, and 13/264.
- Rule composition: 97.88%, 98.31%, and 96.89% median throughput retention occur with 1, 6, and 15 active Checkpoint rules at width 32.The measurements show low composition cost through 15 rules but do not establish a per-rule scaling law.
- Execution-path decomposition: Checkpoint adds 1.86% overhead relative to the empty extension, whereas Anytime adds 16.09% on the eager path.Anytime’s higher cost is attributed to transferring selected activation rows to host memory and evaluating its action path.