PaperlayerView exact revision on arXiv

Source-linked AI summary

SingProbe Technical Report

Sing Team

arXiv:2608.30703v1cs.CRcs.AIcs.CLcs.LG

TL;DR

Existing runtime guardrails often rely on separate models, causing efficiency, latency, and capacity challenges for generation-time monitoring. SingProbe reuses hidden states produced during LLM inference to jointly predict query intent, response safety, and hallucination risk at the token level, with SingStreamBench evaluating streaming behavior. The reported results show competitive or superior performance against larger standalone guardrails and specialized detectors, while also supporting constrained decoding and selective medical-risk intervention.

  • Problem

    Existing guardrails face independent inference, delayed safety signals, and capacity mismatch, creating a trade-off between detection effectiveness and computational efficiency.

  • Method

    SingProbe reuses base-LLM hidden states during autoregressive decoding for unified token-level query-intent, response-safety, and hallucination monitoring.

  • Results

    SingProbe achieves competitive or superior performance across safety and hallucination evaluations compared with substantially larger standalone guardrails and specialized detectors.

  • Takeaways & Limitations

    SingProbe scores can support constrained decoding and selective, on-demand intervention, including medical-risk control through SingProbe-Med.

  • Takeaways & Limitations

    Fine-grained safety labels can be noisy and context-dependent, while existing datasets often contain few responses with substantial safe prefixes before unsafe content.

Abstract

from arXiv · show

Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external models that introduce additional inference cost, delayed safety signals, and a capacity mismatch with increasingly capable base models. To address these issues, we introduce SingProbe, a lightweight intrinsic runtime guard that directly reuses hidden states produced during LLM inference and operates alongside autoregressive decoding. Within a unified framework, SingProbe continuously predicts query intent, response safety, and hallucination risk at the token level with negligible additional guardrail inference overhead, offering a "free-lunch" solution. We further introduce SingStreamBench, a benchmark designed to assess whether streaming guardrails remain inactive on benign prefixes while promptly detecting emerging unsafe content. Extensive experiments show that SingProbe achieves competitive or superior performance compared with substantially larger standalone guardrails and specialized hallucination detectors, with only $\approx$2M parameters and $<0.5\%$ extra overhead. Beyond passive detection, we also show that SingProbe scores can anticipate future generation risk and guide constrained safe decoding. We further extend this paradigm to medical generation through SingProbe-Med, which selectively activates risk-directed decoding interventions only when clinically relevant risks emerge. Together, these results demonstrate that internal model representations provide an effective and efficient interface for generation-time monitoring and control.

1 Introduction

Existing guardrails create a trade-off between monitoring quality and deployment efficiency because they rely on separate, delayed, and often lower-capacity models. SingProbe addresses this by reusing base-model hidden states for unified token-level monitoring and runtime control.

  • Motivation: Existing guardrails often require independent model inference, redundant text encoding, and cross-service coordination during decoding.These design choices add parameters, deployment complexity, communication overhead, and synchronization overhead.
  • Motivation: Delayed safety signals prevent timely intervention because conventional systems often assess complete responses or inspect accumulated output chunks.Chunk-level monitoring limits the granularity of generation-time intervention.
  • Motivation: Smaller guardrails may struggle to match increasingly capable LLMs, especially on complex outputs and long-horizon agentic tasks.The capacity mismatch becomes more challenging as LLMs support million-token contexts.
  • Approach: SingProbe reuses hidden states produced during inference and jointly predicts query intent, response safety, and hallucination risk alongside autoregressive decoding.This avoids repeated text encoding and separate guardrail inference while providing continuously updated token-level predictions.
  • Evaluation: SingStreamBench evaluates whether streaming guardrails remain inactive on benign prefixes and respond promptly when unsafe content emerges.Across query-intent, response-safety, streaming, hallucination, and false-positive evaluations, SingProbe performs competitively or better than larger standalone systems and specialized detectors.
  • Runtime control: SingProbe scores also support constrained decoding and selective medical-risk intervention beyond passive detection.SingProbe-Med activates contrastive decoding only for risk-relevant suffixes rather than continuously intervening on benign generations.

2 Methodology

SingProbe is trained as a lightweight multi-task probe over autoregressive LLM hidden states, producing token-level query-intent, response-safety, and hallucination signals. Its objectives combine task-specific supervision with adaptive weighting and confidence-aware response aggregation.

  • 2.1 Formulation: SingProbe directly consumes concatenated hidden states from selected base-model layers rather than using an additional text encoder.The lightweight prediction head produces outputs at every token position, enabling causally updated predictions from the current prefix.
  • 2.1 Formulation: The probe outputs eight query-intent scores covering seven risk categories and Safe, plus response-unsafety and hallucination scores.Response-risk scores are updated throughout decoding and higher values indicate greater estimated unsafety or hallucination likelihood.
  • 2.2 Training Objectives: The three training objectives are query-intent classification, response-safety classification, and response-hallucination detection.Their losses are combined into a joint objective with task-specific weights.
  • 2.2 Training Objectives: Task weights are adaptively updated from exponential moving averages of recent losses, increasing emphasis on more difficult objectives while preserving overall loss scale.The method uses α=0.9 and a denominator floor of 10^-9.
  • 2.2.1 Query-Intent Objective: Query intent uses binary cross-entropy independently across seven risk categories and the mutually exclusive Safe category.A soft mutual-exclusion penalty separates Safe from the most probable risk while allowing multiple risk categories to coexist.
  • 2.2.2 Response Safety and Hallucination Objectives: Response safety and hallucination objectives use an adaptive confidence-weighted aggregator that falls back to uniform averaging when predictions are insufficiently differentiated.When confidence is differentiated, higher-risk tokens receive greater weight; normalized token weights prevent longer responses from dominating training.
  • Training Data: The unified training corpus covers harmful prompts, benign sensitive prompts, harmful responses, safe refusals, and helpful safe completions.Safety examples are normalized under the paper’s fine-grained risk taxonomy because source datasets use inconsistent annotation protocols and thresholds.

3 SingStreamBench: Benchmarking Streaming Safety Detection

Existing post-hoc safety benchmarks provide response-level labels but cannot assess when harmful content emerges or whether guardrails remain silent beforehand. SingStreamBench addresses this gap with manually verified, tiered safe-to-unsafe transitions and benchmark variants for evaluating streaming detection accuracy and timeliness.

  • Existing safety benchmarks usually provide only a single completed-response label, preventing evaluation of benign-prefix silence, harmful-content onset, and intervention delay.
  • FineHarm’s token-level labels are noisy and cannot reliably represent compositional harmfulness or a precise unsafe-content onset.
  • Existing safety datasets are positionally biased because unsafe responses often become harmful near the beginning, allowing shortcut detection from queries or early lexical cues.
  • SingStreamBench segments responses into cumulative sentence-level prefixes and annotates semantically meaningful unsafe onset boundaries instead of labeling isolated tokens.
  • The benchmark evaluates a safe prefix r_safe followed by a target continuation r_target, requiring silence on benign content and prompt reaction when unsafe content begins.
  • Six tiers test false associations and increasingly complex contexts, including unsafe queries with safe refusals, safe prefixes, disclaimers, background text, and mixed-question contexts.
  • SingStreamBench contains 210 manually verified samples, while SingStreamBench-Full contains 2,428 constructed samples for broader evaluation across tiers and safety scenarios.

4 Evaluation

SingProbe is evaluated as an intrinsic guardrail against standalone safety and hallucination detectors across query-intent, response-safety, streaming, and false-positive settings. It remains competitive or superior while reusing the base model’s inference pipeline.

  • Experimental Settings: SingProbe is compared with proprietary and open-source guardrails, including Gemini 3 Pro, GPT-5.1, YuFeng-XGuard, Llama Guard 3, WildGuard, GraniteGuardian, ShieldGemma, and Qwen3Guard.Hallucination comparisons include DRIFT, HaMI, and SAPLMA variants.
  • Query Classification Evaluation: 0.8674 average F1 is achieved for query-intent classification by SingProbe with Ling-3.0-flash across six benchmarks.This ranks closely behind YuFeng-XGuard-Reason-8B at 0.8714 and GPT-5.1 at 0.8683, while outperforming all evaluated Qwen3Guard configurations.
  • Response Safety Evaluation: 0.8728 average F1 is achieved for response-level safety classification, outperforming Qwen3Guard-Gen-8B-strict by 1.24 points across eight benchmarks.Ling-3.0-flash also achieves the best F1 on WildGuard, XGuard Test, and ExpGuardTest.
  • Response Safety Evaluation: 0.9888 R-AUC and 0.9479 T-AUC are achieved by the best SingProbe streaming configuration, improving over the strongest Qwen3Guard-Stream results by 2.48% and 4.91%.The gains are especially pronounced on SingStreamBench and SingStreamBench-Full, indicating stronger localization under long or complex safe prefixes.
  • Response Safety Evaluation: 0.07% and 0.03% average response FPR are maintained on Ling-3.0-tiny and Ling-3.0-flash, respectively, matching the strongest Qwen3Guard-Stream baselines.The benign datasets are not exhaustively safety-audited and may contain a small number of genuinely unsafe examples.

4.4 Response Hallucination Evaluation

SingProbe detects hallucination risk from token-level hidden-state signals and operates with low serving overhead. Ablations show that three tapped layers provide a favorable effectiveness–efficiency trade-off, while adaptive token weighting is preferred.

  • Response Hallucination Evaluation: SingProbe evaluates response-level hallucination detection across six offline benchmarks using the maximum hallucination score over response tokens and ROC AUC.The macro average is computed across FactCHD, FaithDial, FAVA, RAGTruth, Shroom, and WikiBio.
  • Response Hallucination Evaluation: 0.8012 average AUC is achieved with Ling-3.0-flash, slightly outperforming DRIFT at 0.8000.Ling-3.0-tiny achieves 0.7765 and outperforms DRIFT by 3.57 points; scaling the base model improves SingProbe performance.
  • Overhead Evaluation: Below 0.5% ITL overhead is observed for Decode Probe across concurrency levels, while Prefill-enabled Probe adds 0.86%–2.25% TTFT overhead.Decode Probe TTFT differences remain within measurement noise, and overhead does not grow with serving concurrency.
  • Effect of the Number of Tapped Layers: 0.9677 Response AUC and 0.9823 streaming response AUC are obtained with three tapped layers, improving over one layer’s 0.9648 and 0.9767.Hallucination AUC rises from 0.7750 at one layer to 0.7963 at three layers, while gains become marginal beyond three layers and deployment costs increase.
  • Token Weighting Ablation: Token-aware weighting outperforms uniform averaging, motivating adaptive confidence weighting as the default training objective.The ablation compares adaptive confidence weighting with fixed softmax weighting and uniform averaging under a fixed architecture and training recipe.

4.7 Generation-time Applications of SingProbe

SingProbe remains effective for online safety and hallucination detection during free autoregressive generation, and its token-level scores can anticipate future risk and support constrained safe decoding.

  • Online Safety Detection: 0.9641 average Accuracy and 0.6452 average F1 make SingProbe strongest overall on Ling-3.0-flash under the online safety protocol.It exceeds YuFeng-XGuard-Reason-8B’s strongest baseline average F1 of 0.5964 by 4.88 points.
  • Evaluation Caveat: Online F1 scores should be interpreted with Accuracy because severe class imbalance leaves few Unsafe examples and makes false positives disproportionately affect precision.The values are intended primarily for relative comparison under the same protocol, not direct comparison with balanced offline benchmarks.
  • Online Hallucination Detection: SingProbe achieves the best average hallucination-detection AUC on both backbones, reaching 0.6786 on Ling-3.0-tiny and 0.7271 on Ling-3.0-flash.Its lead over DRIFT grows from 0.047 to 0.073 as backbone capability increases.
  • Anticipatory Safety Prediction: At safe prefixes, higher SingProbe response-unsafety scores correspond monotonically to greater unsafe rates among future continuations.The unsafe rate rises from approximately 0.02 in the lowest-score interval to 0.43 in the highest.
  • Constrained Safe Decoding: 50% guard-guided pruning lowers the retained-pool unsafe rate to 0.53, approximately 30% below random pruning and matching the oracle rate.With 90% of branches removed, the unsafe rate reaches approximately 0.22.

5.1 Motivation and Framework Overview

Medical generation motivates selective intervention because harmful recommendations may emerge after clinically sound prefixes, while continuous correction is undesirable. SingProbe-Med separates intervention timing from intervention content using internal-state monitoring and bounded target-supported steering.

  • Motivation: Medical-risk errors can emerge after clinically sound prefixes, making query-level screening insufficient and post-hoc review too late.Applying corrective decoding throughout an entire response is also described as undesirable.
  • Framework Overview: SingProbe-Med continuously monitors the evolving generation trajectory to decide when clinically relevant intervention should begin.Its admission mechanism uses lightweight internal-state signals.
  • Framework Overview: Once admitted, target-supported sGDS determines how to intervene by contrastively re-ranking candidate tokens over a localized risky suffix.The intervention is bounded, after which decoding returns to the target model.

5.2 SingProbe-Med: When to Intervene

SingProbe-Med determines intervention admission from four parallel token-level signals combined through an evidential hierarchy. Eligibility gates applicability, global risk supplies trajectory evidence, and future or boundary risk refines timing under intermediate evidence.

  • Signals: Four signals from the current prefix support distinct intervention-controller decisions: eligibility, global risk, future risk, and boundary risk.Future risk concerns errors within the next 1–32 tokens; the supplied passage introduces boundary risk as the remaining localized timing signal.
  • Admission Policy: The admission rule activates the next token when eligibility exceeds γE and either global risk exceeds γG or conditional global evidence is refined by future or boundary risk.This is expressed by the E → G → (F/B) hierarchy.
  • Admission Policy: Eligibility first restricts intervention to applicable prefixes, after which global risk and then shorter-horizon signals determine whether intervention is admitted.Future and boundary signals are complementary entry signals computed in parallel, not sequential execution stages.

5.3 Target-Supported sGDS: How to Intervene

Target-supported sGDS intervenes only after admission by contrasting target and medical-risk logits over target-supported candidates. Shared-token synchronization, bounded episodes, and release to target-only decoding constrain the correction to risk-relevant generation.

  • Risk-pattern Branch: The medical-risk branch is created by LoRA adaptation on five operational risk categories while the AntAngelMed base-model parameters remain frozen.The categories include dosage errors, contraindications, wrong indications, and dermatology-specific variants.
  • Contrastive Intervention: The risk-pattern branch acts as an anti-expert whose token-level preferences are contrasted with target logits, not interpreted as explicit medical-error probabilities.It is used only when intervention is activated.
  • Contrastive Intervention: sGDS restricts intervention to candidates already supported by the target model, suppressing candidates relatively favored by the risk branch and promoting target-favored alternatives.The top-k restriction re-ranks plausible target-supported candidates rather than introducing arbitrary vocabulary items.
  • Branch Synchronization: A single sampled token is appended to both branches at each activated step, keeping their prefixes synchronized while maintaining separate model states and KV caches.This shared-token update ensures subsequent logits describe the same generation history.
  • Bounded Intervention: The bounded variant uses k = 100, α = 3, and W = 64 before decoding returns to target-only logits on the modified prefix.The full variant instead uses k = 50 and α = 2 until release.
  • Bounded Intervention: The two-level on-demand design limits expensive dual-branch decoding to activated episodes and limits logit modification to target-supported candidates during risky generation.After the activation interval ends, generation resumes with the target model.

5.4 Evaluation

SingProbe-Med is evaluated for selective, timely medical-risk intervention, correction quality, capability preservation, and inference efficiency. The results show improved risk ranking and targeted activation, while exposing trade-offs between intervention coverage, safe-generation preservation, and decoding cost.

  • Admission Selectivity and Timing: 0.025, 0.012, and 0.011 improvements in ROC–AUC, AP, and PR–AUC result from gating Global Risk by Intervention Eligibility.The gate suppresses risk scores on intervention-ineligible prefixes.
  • Admission Selectivity and Timing: 96.77% precision at 1% White FPR accompanies 11.98% Harm recall, compared with 92.34% precision and 51.70% recall at 10% White FPR.Tightening false-trigger control therefore reduces intervention coverage.
  • Admission Selectivity and Timing: 54.99% of first activations on Harm trajectories occur at or before Tconfirm, while scoring 3,044,056 token states takes 0.019749 ms/token.This corresponds to approximately 50,635 token states per second.
  • Medical Error Correction: 30.27% broad mitigation and 15.22% strict repair occur on activated trajectories, versus 25.03% and 12.58% over the complete test set.The activated subset has greater correction potential.
  • Medical Error Correction: 1.84% safe regression occurs for full intervention overall, increasing to 2.53% with the 64-token configuration.Safe regression is higher when conditioning specifically on activated trajectories.
  • Capability Preservation: 4.69% of HealthBench examples activate SingProbe-Med, while none activate on the other reported benchmarks; capability scores remain broadly comparable to native AntAngelMed-100B.Always-on sGDS modifies every example and scores lower than SingProbe-Med across all six benchmarks.
  • Inference Efficiency: 37.379 seconds is the estimated continuously monitored latency, versus 60.934 seconds for resident dual-branch decoding and approximately 39.162 seconds with a 64-token intervention window.The 64-token estimate includes an estimated 1.843-second active-window cost.

6 Conclusion

The paper presents SingProbe as an intrinsic guardrail that reuses base-LLM hidden states to monitor multiple risks during generation. It further frames internal-state scores as signals for constrained decoding and selective medical intervention, linking monitoring efficiency with runtime control.

  • Conclusion: SingProbe jointly monitors query intent, response safety, and hallucination risk from hidden states produced by the base LLM.The guardrail operates throughout generation and avoids redundant text encoding.
  • Conclusion: SingProbe scores contain information about the safety tendency of future continuations and can serve as control signals for constrained decoding.This extends intrinsic guardrails beyond passive classification.
  • Conclusion: SingProbe-Med uses internal-state signals for selective, on-demand intervention when and where intervention is warranted.The conclusion positions SingProbe as an interface between LLM representations and generation-time safety mechanisms.

Authors

The paper lists its authors.

  • Authors: The author list includes Shiwen Cui, Jianjie Jiang, Guoyi Li, and the remaining contributors named in the paper.The complete listed author sequence is provided in the cited passage.
  • Authors: Jinzhen Lin, Changhua Meng, Shuo Shao, Jiashui Wang, Weiqiang Wang, and Weixi Wu are among the listed authors.These names appear in the paper’s author list.
  • Authors: Zhuoer Xu, Zihan Yan, Shenglin Yin, Xinlei Ying, Zhe Zhao, and Jing Zhou complete the listed author sequence.These names appear at the end of the author list.
Loading 2608.30703v1…