PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Safety Guardrails through Latent Dual-Hypothesis Reasoning with On-Demand Trace Reconstruction

Md Asiful Islam, Fahmida Alam, Mihai Surdeanu

arXiv:2610.18304v1cs.AI

TL;DR

Reasoning-based guardrails can improve safety decisions, but autoregressively generating intermediate traces adds inference overhead. LR-GUARD distills structured safe, unsafe, and deliberation reasoning into latent representations and reconstructs teacher traces on demand; it reports stronger performance than its explicit-reasoning teacher in most settings alongside a 19.44× batch-size-one speedup.

  • Problem

    Reasoning-based guardrails generate intermediate traces that add inference overhead, while latent guardrails have limited support for preserving distinct reasoning stages and reconstructing detailed teacher traces.

    Source: §1 · p. 1

  • Method

    LR-GUARD distills a frozen teacher’s safe, unsafe, and deliberation reasoning into stage-specific latent representations, jointly training safety prediction and on-demand trace reconstruction.

    Source: §1 · p. 2

  • Results

    LR-GUARD outperforms its explicit-reasoning teacher in most in-domain and out-of-domain settings on safety classification and explanation extraction, with a 19.44× speedup at batch size one.

    Source: §1 · p. 2, §6 · p. 8

  • Takeaways & Limitations

    The approach supports human-readable reconstruction of teacher reasoning when requested without requiring explicit reasoning generation during standard safety inference.

    Source: §1 · p. 2

    Efficiency experiments use an NVIDIA H100 80GB GPU, and direct absolute-latency comparisons with existing latent guardrails under identical conditions are still needed.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Guardrail approaches that rely on reasoning-based large language models (LLM) improve safety decisions, but come with substantial inference overhead due to the explicit reasoning. We introduce LR-Guard (Latent Reasoning Guardrail), an efficient and explainable guardrail that jointly learns structured latent reasoning and on-demand reasoning reconstruction. Our method distills a frozen teacher's analysis of safe and unsafe prompt interpretations and their comparison into compact latent representations. Joint training optimizes these representations for both safety prediction and reconstruction of the teacher's explicit reasoning traces, enabling human-readable explanations without requiring intermediate text generation for every decision. Experiments show that LR-Guard outperforms its explicit-reasoning teacher in most in-domain and out-of-domain settings on safety classification and explanation extraction, while achieving a 19.4x inference speedup at batch size one. Further, reconstructed reasoning traces from the latent information exhibit high semantic similarity to the teacher's explicit reasoning.

1 Introduction

LR-GUARD addresses the cost and auditability limits of reasoning-based safety guardrails by representing dual-hypothesis reasoning in latent form and reconstructing teacher traces on demand. The paper reports stronger average benchmark performance than comparable baselines, similar classification and explanation performance to its explicit-reasoning teacher, and substantial speedups.

1 Introduction

  • Existing latent guardrails have limited exploration, often compute latent states sequentially, and generally do not reconstruct detailed, stage-specific teacher traces at inference time.Preserving distinct reasoning-stage functions in latent representations is also underexplored.Source: §1 · p. 1
  • LR-GUARD processes learned latent reasoning tokens in one causal forward pass, avoiding sequential latent computation and repeated model work for individual reasoning steps.The design addresses existing latent guardrails that still compute latent states autoregressively.Source: §1 · p. 1
  • The method distills three teacher reasoning stages—safe hypothesis, unsafe hypothesis, and deliberation—into stage-specific latent representations.The stages respectively consider benign and harmful interpretations and compare them to reach a decision.Source: §1 · p. 2
  • LR-GUARD jointly trains safety prediction and reasoning reconstruction, allowing explicit teacher traces to be recovered on demand without generating them during standard inference.This supports auditability while retaining the latent representations used for safety decisions.Source: §1 · p. 2
  • 19.44× and 10.86× inference speedups over the explicit-reasoning teacher at batch sizes 1 and 50 accompany performance similar to the teacher and stronger average results than comparable baselines.The evaluation spans five prompt-safety benchmarks.Source: §1 · p. 2

2 Related Work

Related work covers external and explicit-reasoning guardrails, emerging latent-reasoning approaches, and methods for making latent guardrails auditable. LR-GUARD differs by processing distinct reasoning stages in one forward pass and reconstructing teacher traces on demand.

2 Related Work

  • External guardrails classify safety, while reasoning-based systems add explicit reasoning; ARBITER evaluates competing safe and unsafe interpretations before deciding.LEG also identifies unsafe evidence spans.Source: §2 · pp. 2–3
  • CoLaGuard and LatentGuard compress reasoning into recurrent latent states, while LPG separates intent and policy stages and DRAFT explores task-decoupled latent reasoning.LR-GUARD instead processes learned tokens for distinct reasoning stages in a single forward pass.Source: §2 · p. 3
  • Prior latent guardrails offer compact audit generation or training-time reconstruction, whereas LR-GUARD jointly optimizes safety prediction and recovery of stage-specific teacher traces on demand.The cited comparison also notes that LR-GUARD identifies unsafe evidence phrases in its final output.Source: §2 · p. 3

3 Proposed Method

LR-GUARD replaces explicit dual-hypothesis reasoning during routine safety decisions with ordered latent stages, jointly trained for safety prediction and on-demand reconstruction of teacher traces.

3 Proposed Method

  • LR-GUARD jointly trains a Guard Adapter and Reconstruction Adapter to transfer explicit reasoning into latent representations while avoiding intermediate-text generation during standard inference.The Guard Adapter produces safety labels and evidence phrases; the Reconstruction Adapter recovers human-readable reasoning when requested.Source: §3 · pp. 3–4, §3.3 · p. 4

3.1 Dual-Hypothesis Reasoning

  • Dual-hypothesis reasoning evaluates the strongest plausible benign and harmful interpretations, then deliberates which is better supported by the prompt.LR-GUARD distills this three-stage structure from frozen ARBITER and reconstructs explicit text only when requested.Source: §3.1 · p. 4

3.2 Latent Dual-Hypothesis Reasoning

  • 32 learned latent tokens encode safe, unsafe, and deliberation stages in a fixed 8/8/16 allocation and safe-to-unsafe-to-deliberation order.A single causal forward pass contextualizes the slots; later stages can attend to earlier ones without sequential text decoding or recurrent latent-state inputs.Source: §3.2 · p. 4

3.3 Joint Training

  • Joint training combines teacher distillation, gold-label safety supervision, and trace reconstruction, with reconstruction gradients also shaping the latent representations.Stage-specific prefix KV caches and recovery control tokens condition reconstruction against teacher traces. When teacher predictions disagree with gold labels, label KL and deliberation alignment are omitted, while safe- and unsafe-hypothesis alignment weights are reduced from 1.0 to 0.25.Source: §3.3 · pp. 4–5

3.4 Inference

  • Standard inference uses the Guard Adapter in a single prefill pass, generating the winner and final JSON without sequential latent-state generation.When requested, the Reconstruction Adapter uses the cached prefix to generate a stage trace after the safety decision; that trace is not fed back into the decision.Source: §3.4 · pp. 5–6

4 Experiment Setup

Experiments train LR-GUARD variants on three prompt-safety datasets and evaluate prompt classification and unsafe-evidence explanations, with Table 1 reporting unsafe-class F1 across in-domain and out-of-domain settings.

  • LR-GUARD variants are trained on AEGIS2.0, WildGuardMix, or ToxicChat0124, using LEG-1.0 extensions for unsafe-word or phrase explanation supervision.At inference, the model predicts those phrases for the final JSON explanation field.Source: §4 · p. 6
  • Each LR-GUARD variant uses Llama-3.1-8B-Instruct as its frozen backbone and an ARBITER teacher trained on the same dataset.Source: §4 · p. 6
  • Adapters and latent-token embeddings are jointly trained for three epochs with AdamW, using three random seeds.Both LoRA adapters use rank 16, α = 32, and dropout 0.05; training uses learning rate 2 × 10^-4, per-device batch size 4, and gradient accumulation over 64 steps.Source: §4 · p. 6
  • Table 1 reports unsafe-class F1, averages LR-GUARD results over three seeds, and distinguishes in-domain from out-of-domain performance by cell shading.Source: Table 1 · p. 6, §4 · p. 6

5 Results and Analysis

LR-GUARD performs well across classification, explanation, and trace reconstruction evaluations, while latent-state interventions indicate that its representations contain information relevant to safety decisions. The section also concludes that it is a promising efficient and interpretable alternative to explicit reasoning, with efficiency results bounded by the tested hardware and comparison settings.

5.1 Prompt Classification Performance

  • LR-GUARD beats ARBITER on F1 in two of three in-domain and four of six out-of-domain settings; on ToxicChat0124 it reaches 83.69% F1, while CoLaGuard-8B leads WildGuardMixMix at 89.44% versus 88.87%.On WildGuardMix, LR-GUARD-wildguardmix outperforms all LatentGuard variants and CoLaGuard-3B.Source: §5.1 · p. 7

5.2 Explanation Classification Performance

  • LR-GUARD outperforms ARBITER in two of three in-domain and four of six out-of-domain settings when identifying unsafe words or phrases.The reported results indicate that it largely preserves and often improves this ability without generating explicit intermediate reasoning.Source: §5.2 · p. 7

5.3 Reasoning Reconstruction Quality

  • Reconstructed reasoning shows semantic similarity scores from 0.83 to 0.89 and substantial BLEU and ROUGE overlap with teacher traces across in-domain and out-of-domain evaluations.Table 2 reports reconstruction performance across models and test datasets, averaged over three random seeds.Source: §5.3 · p. 7, Table 2 · p. 7

5.4 Latent-State Intervention Analysis

  • Same-label latent-state replacements increase gold-label confidence on average, while opposite-label replacements generally reduce it, supporting a role for latent reasoning in safety predictions.The intervention replaces all three reasoning stages using activations from correctly classified examples, with input lengths matched within an eight-token tolerance.Source: §5.4 · p. 8, Figure 3 · p. 8

6 Conclusion

  • Across safety benchmarks, LR-GUARD outperforms its explicit-reasoning teacher in most in-domain and out-of-domain settings on classification and explanation extraction while reducing inference latency.The conclusion also reports that latent-state interventions provide evidence that the representations encode information relevant to final safety predictions.Source: §6 · p. 8
  • The conclusion presents latent reasoning as a promising alternative that retains structured reasoning and on-demand trace reconstruction without generating intermediate text for routine classification.The approach distills safe-hypothesis, unsafe-hypothesis, and deliberation stages into latent tokens.Source: §6 · p. 8

Limitations

  • Efficiency experiments use an NVIDIA H100 80GB GPU, and cross-study absolute latency comparisons require matched hardware, batch sizes, and evaluation settings.Actual latency and throughput may vary across hardware platforms and deployment settings.Source: Limitations · p. 9

A Inference Time Analysis

LR-GUARD's winner-only inference is much faster than its explicit-reasoning teacher in the reported WildGuardMix comparison. Cross-model speedup figures are reported relative to each model's own explicit-reasoning baseline, so hardware and evaluation differences limit direct comparisons.

A Inference Time Analysis

  • With reasoning reconstruction enabled, LR-GUARD remains 3.51× and 4.57× faster than ARBITER's full inference at batch sizes 1 and 50, respectively.The comparison is on WildGuardMix using an NVIDIA H100 80GB GPU.Source: A Inference Time Analysis · p. 10
  • 133.09× and 22.60× speedups are reported for LR-GUARD winner-only inference at batch sizes 1 and 50, respectively, relative to ARBITER.The comparison uses the WildGuardMix test set and an NVIDIA H100 80GB GPU.Source: A Inference Time Analysis · p. 10
  • 8.90× to 12.90× is the reported speedup range for CoLaGuard, LatentGuard, and LPG relative to their own explicit-reasoning baselines.Different hardware, batch sizes, and evaluation settings prevent direct comparison of absolute inference times across studies.Source: A Inference Time Analysis · p. 10

B Human Evaluation of Reasoning Reconstruction

A human evaluation on 100 WildGuardMix test examples assessed whether reconstructed traces retained teacher reasoning and whether deliberation matched the predicted label. The reported results indicate substantial preservation across stages and consistency between deliberation and prediction.

B Human Evaluation of Reasoning Reconstruction

  • The evaluation samples 100 WildGuardMix test examples, evenly split between safe and unsafe inputs, and uses one annotator's binary judgments.Preservation is assessed against the essential reasoning in the corresponding teacher trace, allowing different wording.Source: B Human Evaluation of Reasoning… · p. 10
  • Reconstructed traces preserve the teacher's reasoning in a high proportion of examples across all three stages, with the strongest preservation for the unsafe hypothesis.One annotator assessed 50 safe and 50 unsafe examples, judging each stage separately and whether all three were preserved in the same example.Source: B Human Evaluation of Reasoning… · p. 10
  • Reconstructed deliberation generally agrees with the model's predicted safety label, and all three reasoning components are preserved together in a majority of examples.Decision consistency compares the reconstructed deliberation's conclusion with the final predicted label.Source: B Human Evaluation of Reasoning… · p. 10

C Jailbreak Robustness Analysis

On the out-of-domain WildJailbreak benchmark, LR-GUARD-wildguardmix reports strong jailbreak-detection performance and slightly improves F1 and recall over ARBITER while reducing FNR. This evaluation specifically tests jailbreak detection, although the main-experiment datasets already include jailbreak prompts.

C Jailbreak Robustness Analysis

  • WildJailbreak is an out-of-domain evaluation because LR-GUARD-wildguardmix is trained on WildGuardMix and tested on WildJailbreak.The main-experiment datasets already include jailbreak prompts, while WildJailbreak is a dedicated benchmark for jailbreak detection.Source: C Jailbreak Robustness Analysis · p. 10
  • 97.85% F1, 96.70% recall, and 3.30% FNR are reported for LR-GUARD-wildguardmix on WildJailbreak.Compared with ARBITER, it slightly improves F1 and recall and further reduces FNR; it also outperforms AprielGuard in F1.Source: C Jailbreak Robustness Analysis · pp. 10–11
  • Recall and FNR indicate how effectively jailbreak attempts are detected and how frequently they go unidentified in this evaluation.The reported lower FNR relative to ARBITER indicates fewer jailbreak attempts go undetected.Source: C Jailbreak Robustness Analysis · p. 11

D Over Refusal Robustness Analysis

On XSTest, LR-GUARD-wildguardmix is evaluated for over-refusal on benign prompts whose wording can resemble harmful requests. It achieves strong F1 performance while reducing false positives compared with ARBITER.

D Over Refusal Robustness Analysis

  • XSTest tests whether guardrails incorrectly classify benign prompts containing language commonly associated with harmful content as unsafe.The evaluation uses LR-GUARD trained on WildGuardMix, making it out of domain.Source: D Over Refusal Robustness Analysis · p. 11
  • 95.26% F1 and 4.00% FPR for LR-GUARD-wildguardmix; it has lower FPR than ARBITER at comparable F1 and outperforms other baselines in F1.A lower FPR means fewer benign prompts are incorrectly classified as unsafe.Source: D Over Refusal Robustness Analysis · pp. 11–12

E Per-Category Performance Analysis

LR-GUARD-wildguardmix largely preserves ARBITER's category-wise unsafe detection performance, and LR-GUARD's classification F1 scores are generally consistent across three random seeds.

E Per-Category Performance Analysis

  • On WildGuardMix unsafe categories, LR-GUARD-wildguardmix matches ARBITER in nine categories, exceeds it in three, and trails slightly in two.The category-wise comparison reports unsafe-class F1 scores; both models were trained on WildGuardMix and evaluated using seed 32.Source: E Per-Category Performance Analysis · p. 12
  • The category-wise evaluation uses the WildGuardMix test set and compares LR-GUARD-wildguardmix with ARBITER.Both models were trained on WildGuardMix and evaluated using seed 32.Source: E Per-Category Performance Analysis · p. 12

F Standard Deviation over Multiple Random Seeds

  • Across AEGIS2.0, WildGuardMix, and ToxicChat0124 models evaluated on all three test sets, classification F1 standard deviations over seeds 32, 42, and 52 are generally low.The reported stability covers both prompt classification and explanation classification.Source: F Standard Deviation over Multiple… · p. 12

G Extended Results

Extended results report a reasonable precision-recall balance, particularly in-domain, with relatively low false-positive and false-negative rates in those settings. The latent-reasoning instruction uses structured slots to compare safe and unsafe interpretations, with explicit traces reconstructed only on demand.

G Extended Results

  • LR-GUARD maintains a reasonable precision-recall balance, particularly in in-domain evaluations, where FPR and FNR are relatively low.The detailed prompt-classification results use random seed 32.Source: G Extended Results · p. 12

H Latent-Reasoning Instruction Template

Loading 2610.18304v1…