PaperlayerView exact revision on arXiv

Source-linked AI summary

SingProbe Technical Report

Sing Team

arXiv:2608.30703v1cs.CRcs.AIcs.CLcs.LG

TL;DR

Runtime guardrails face a trade-off between detection effectiveness and computational efficiency. SingProbe reuses LLM hidden states during decoding to monitor intent, safety, and hallucination risk, and achieves competitive or superior performance to substantially larger standalone guardrails and specialized detectors.

  • Problem

    Existing guardrails face a trade-off between detection effectiveness and computational efficiency, which constrains their scaling.

    Source: §1 · p. 2

  • Method

    SingProbe reuses hidden states produced during LLM inference to monitor query intent, response safety, and hallucination risk alongside decoding.

    Source: §1 · p. 2, §6 · p. 22

  • Results

    Across evaluations of intent, safety, streaming detection, hallucination, and false-positive robustness, SingProbe achieves competitive or superior performance to substantially larger standalone guardrails and specialized hallucination detectors.

    Source: §1 · p. 3

  • Takeaways & Limitations

    SingProbe scores can guide constrained decoding and support selective, on-demand intervention in medical generation.

    Source: §6 · pp. 22–23

    Fine-grained token safety labels may be noisy, and existing safety datasets often have positional bias with few unsafe responses preceded by substantial safe prefixes.

    Source: §3.1 · p. 7

Abstract

from arXiv · show

Runtime guardrails are essential for reliable large language model (LLM) deployment, yet existing approaches typically rely on independent, external models that introduce additional inference cost, delayed safety signals, and a capacity mismatch with increasingly capable base models. To address these issues, we introduce SingProbe, a lightweight intrinsic runtime guard that directly reuses hidden states produced during LLM inference and operates alongside autoregressive decoding. Within a unified framework, SingProbe continuously predicts query intent, response safety, and hallucination risk at the token level with negligible additional guardrail inference overhead, offering a "free-lunch" solution. We further introduce SingStreamBench, a benchmark designed to assess whether streaming guardrails remain inactive on benign prefixes while promptly detecting emerging unsafe content. Extensive experiments show that SingProbe achieves competitive or superior performance compared with substantially larger standalone guardrails and specialized hallucination detectors, with only $\approx$2M parameters and $<0.5\%$ extra overhead. Beyond passive detection, we also show that SingProbe scores can anticipate future generation risk and guide constrained safe decoding. We further extend this paradigm to medical generation through SingProbe-Med, which selectively activates risk-directed decoding interventions only when clinically relevant risks emerge. Together, these results demonstrate that internal model representations provide an effective and efficient interface for generation-time monitoring and control.

1 Introduction

SingProbe reuses LLM hidden states for token-level monitoring of query intent, response safety, and hallucination risk during generation. The introduction presents it as a way to reduce guardrail overhead while supporting timely detection and risk-directed generation control.

Introduction

  • Existing guardrails add independent inference and deployment complexity, delay safety signals until responses or chunks are complete, and may lack capacity for complex or long-horizon outputs.These limitations create a trade-off between detection effectiveness and computational efficiency, constraining model capacity and monitoring granularity.Source: §1 · p. 2
  • SingProbe consumes inference-time hidden states to classify query intent, response safety, and hallucination risk continuously at the token level, avoiding separate guardrail inference and repeated text encoding.The introduction characterizes its additional inference overhead as negligible.Source: §1 · p. 2
  • Integration with SGLang and vLLM lets token generation and guardrail scoring share a serving pipeline, removing the need to deploy and coordinate a separate guardrail service.The integration also reduces cross-service communication and uses SGLang’s inference infrastructure for token-level signals.Source: §1 · p. 3
  • Across query-intent, response-safety, streaming, hallucination, and benign false-positive evaluations, SingProbe achieves competitive or superior performance to substantially larger standalone guardrails and specialized hallucination detectors.SingStreamBench tests whether guardrails remain inactive on benign prefixes and respond promptly when unsafe content emerges.Source: §1 · p. 3
  • SingProbe’s token-level risk scores can guide safer generation decisions, while SingProbe-Med activates contrastive decoding only for risk-relevant medical suffixes.This selective intervention avoids continuous correction of benign generations.Source: §1 · p. 3

2 Methodology

SingProbe uses selected backbone hidden states to emit causal token-level intent and response-risk signals, trained jointly with task-specific objectives and a curated multi-source corpus.

2.1 Formulation

  • SingProbe feeds concatenated hidden states from selected backbone layers to a lightweight head, producing causally updated token-level query-intent, response-unsafety, and hallucination logits throughout decoding.Query intent covers seven risk categories and a Safe category; response scores track unsafety and hallucination risk in the generated prefix.Source: §2.1 · p. 3

2.2 Training Objectives

  • Joint training combines query-intent, response-safety, and response-hallucination objectives, weighting tasks by their recent EMA losses while preserving the overall loss magnitude.The EMA smoothing coefficient is α=0.9, and a 10^-9 denominator floor avoids division by zero.Source: §2.2 · pp. 3–4

2.2.1 Query-Intent Objective

  • Query intent uses BCE for seven co-occurring risk categories and an exclusive Safe class, plus a soft penalty against simultaneous Safe and top-risk predictions.The penalty encourages separation of safe and risky queries without forbidding co-occurrence among different risk types.Source: §2.2.1 · pp. 4–5

2.2.2 Response Safety and Hallucination Objectives

  • Response-safety and hallucination losses use adaptive confidence-weighted token BCE, reverting to uniform averaging when predictions are insufficiently differentiated.Within-sample weighting emphasizes high-risk tokens while preventing coarse positive labels from pushing benign prefixes toward risky predictions.Source: §2.2.2 · p. 5

2.3 Training Dataset Construction

  • A unified three-task corpus combines curated safety and factuality data with policy-grounded synthetic examples and separately aligned query and response supervision.Contrastive harmful and benign prompts, varied responses, multilingual and multi-category examples, and independent label checks broaden coverage and reduce reliance on prompt labels or lexical shortcuts.Source: §2.3 · p. 6

3 SingStreamBench: Benchmarking Streaming Safety Detection

SingStreamBench evaluates whether streaming guardrails stay inactive on benign prefixes and detect unsafe content promptly, using sentence-level prefix annotations and controlled safe-to-unsafe transitions. It includes a manually verified core set and a larger full variant.

3.1 Motivation

  • SingStreamBench addresses limits of post-hoc labels and noisy token-level annotations with manually verified sentence-prefix boundaries and controlled safe-to-unsafe transitions.Existing benchmarks do not show when unsafe content emerges, while token-level labels can be noisy and context-dependent; existing datasets also tend to have little safe content before risk appears.Source: §3.1 · pp. 6–7

3.2 Benchmark Design

  • Six tiers test whether guardrails localize response risk rather than react to unsafe queries, safe prefixes, response position, or superficial lexical cues.The evaluation target is silence through a safe prefix, prompt detection when unsafe content begins, and continued silence for fully safe responses.Source: §3.2 · pp. 7–8

3.3 Construction Pipeline

  • The construction pipeline verifies seed labels, augments samples across tiers, annotates cumulative sentence-level prefixes, and adds safe continuations.Three judge models must unanimously accept seed samples; prefix labels identify the first unsafe sentence boundary, avoiding isolated-token labels.Source: §3.3 · p. 8

3.4 Dataset Variants and Statistics

  • The manually verified SingStreamBench core contains 210 samples, while SingStreamBench-Full contains 2,428 constructed samples for broader evaluation.The core supports comparison of detection accuracy and latency; the full variant expands coverage across tiers and safety scenarios.Source: §3.4 · p. 8

4 Evaluation … 4.3 Response Safety Evaluation

SingProbe’s hidden-state guard is evaluated for query intent and response safety, including token-level streaming detection and false alarms on benign free generation. Across the reported safety evaluations, the lightweight head performs competitively with standalone guardrails while localizing emerging unsafe content.

4.1 Experimental Settings

  • Evaluation compares SingProbe with two proprietary and 12 open-source safety guardrail configurations, as well as hidden-state hallucination detectors DRIFT, HaMI, and SAPLMA variants.The safety baselines include Gemini 3 Pro, GPT-5.1, and Qwen3Guard loose and strict modes; SAPLMA is tested with raw-input and response-formatted variants.Source: §4.1 · p. 8

4.2 Query Classification Evaluation

  • 0.8674 average F1 places SingProbe with Ling-3.0-flash close behind YuFeng-XGuard-Reason-8B (0.8714) and GPT-5.1 (0.8683) across six query-intent benchmarks.It outperforms all evaluated Qwen3Guard configurations, whose best average F1 is 0.8602, and achieves 0.8825 F1 on ExpGuardTest.Source: Table 2 · p. 9, §4.2 · p. 9

4.1 Experimental Settings

  • SingProbe uses a query-residual multi-head attention block over hidden states from three evenly selected shallow, middle, and deep base-model layers.The block produces Norm(Q + O), followed by a linear classifier that outputs guardrail scores.Source: §4.1 · p. 9

4.3 Response Safety Evaluation

  • 0.8728 average F1 makes SingProbe with Ling-3.0-flash the best response-safety configuration, 1.24 points above Qwen3Guard-Gen-8B-strict.It also records the best F1 on WildGuard (0.8000), XGuard Test (0.8588), and ExpGuardTest (0.9247).Source: Table 3 · p. 10, §4.3 · p. 10
  • 0.9888 R-AUC and 0.9479 T-AUC are SingProbe-tiny’s streaming averages, improving on the strongest Qwen3Guard-Stream results by 2.48% and 4.91%.SingProbe-flash scores 0.9887 R-AUC and 0.9481 T-AUC; gains are especially pronounced on SingStreamBench and SingStreamBench-Full, with comparable FineHarm performance.Source: Table 4 · p. 10, §4.3 · p. 10
  • 0.07% and 0.03% average response FPRs for Ling-3.0-tiny and Ling-3.0-flash, respectively, are on par with the strongest Qwen3Guard-Stream baselines.These are apparent FPRs from five nominally benign datasets; because they are not exhaustively safety-audited, the values may slightly overestimate true false-positive rates.Source: §4.3 · pp. 10–11, Table 5 · p. 11

4.4 Response Hallucination Evaluation · 4.5 Overhead Evaluation

SingProbe detects response-level hallucinations competitively across six benchmarks, while adding little latency under serving load. Its decoding-only ITL overhead stays below 0.5%, and enabling prefill adds 0.86%–2.25% TTFT overhead.

4.4 Response Hallucination Evaluation

  • For response-level detection, SingProbe scores every response token under teacher forcing and uses the maximum score as the query-response pair’s risk score.ROC AUC is computed across samples on six offline benchmarks: FactCHD, FaithDial, FAVA, RAGTruth, Shroom, and WikiBio.Source: §4.4 · p. 11
  • The response-hallucination evaluation reports per-benchmark AUCs and macro averages, with results on Ling-3.0-tiny and Ling-3.0-flash.The reported results suggest hidden states provide effective detection signals whose quality improves with model scale.Source: §4.4 · pp. 11–12

4.5 Overhead Evaluation

  • Decode Probe ITL overhead ranges from +0.18% to +0.45%; Prefill-enabled Probe adds +0.86% to +2.25% TTFT overhead, with ITL overhead of +0.18% to +0.47%.Relative overhead does not grow with concurrency; TTFT differences for Decode Probe are within measurement noise.Source: Table 7 · p. 12, §4.5 · p. 12

4.4 Response Hallucination Evaluation

  • On Ling-3.0-tiny, SingProbe’s average AUC is 0.7765, 3.57 points above DRIFT; on Ling-3.0-flash, its 0.8012 exceeds DRIFT’s 0.8000.It also leads per-benchmark AUC on FactCHD (0.7819), FaithDial (0.9529), and WikiBio (0.9515) on Ling-3.0-tiny.Source: §4.4 · p. 12

4.5 Overhead Evaluation

  • The overhead evaluation compares unified-architecture SingProbe on Ling-3.0-flash with a base-model-only baseline across concurrency levels 1 to 32.Requests use 8,192 input tokens and 1,024 output tokens; Decode Probe runs during decoding, while Prefill-enabled Probe also runs during prefill.Source: §4.5 · p. 12
  • Overhead measurements use eight rounds, pairing each SingProbe-enabled configuration with the same-round base-model baseline and reporting mean and standard deviation of relative differences.Each round submits a fixed prompt count per concurrency level and computes median TTFT and ITL.Source: §4.5 · p. 12, Table 7 · p. 12

4.6 Ablation Study

The ablations show that three tapped layers provide a favorable performance–cost trade-off, while adaptive confidence weighting improves response-safety training metrics over fixed alternatives.

4.6.1 Effect of the Number of Tapped Layers

  • At L=3, Response AUC is 0.9677, streaming response AUC is 0.9823, and hallucination AUC is 0.7963; SingProbe uses three tapped layers by default.Hallucination AUC rises from 0.7750 at L=1 to 0.7988 at L=8, while benign false-positive rate stays at or below 0.02%. Gains become marginal beyond L=3 as parameter count, memory use, and inference latency continue to grow.Source: Figure 3 · p. 13, §4.6.1 · p. 13

4.6.2 Effect of the Token Weighting Technique

  • Adaptive confidence weighting yields macro-average Response AUC 0.981, Token AUC 0.928, and Best F1 0.950, improving on fixed softmax weighting.Fixed softmax achieves Response AUC 0.979, Token AUC 0.928, and Best F1 0.943; uniform averaging achieves 0.958, 0.895, and 0.918, respectively. Adaptive weighting reduces emphasis when token scores are insufficiently differentiated and strengthens it when meaningful token-level distinctions emerge.Source: Figure 4 · p. 13, §4.6.2 · pp. 13–14

4.7 Generation-time Applications of SingProbe

During free autoregressive generation, SingProbe detects safety and hallucination risk; its streamed safety score also forecasts continuation risk and can guide branch pruning.

4.7.1 Application 1: SingProbe for Online Content Detection

  • 0.9641 average Accuracy and 0.6452 average F1 make Ling-3.0-flash SingProbe strongest overall, exceeding YuFeng-XGuard-Reason-8B’s average F1 of 0.5964 by 4.88 points.SingProbe also obtains the best F1 on WildGuard (0.6364) and XSTest (0.5405). Online F1 is depressed by severe Safe/Unsafe class imbalance and should be compared within this protocol, not directly with balanced offline benchmarks.Source: §4.7.1 · p. 14
  • 0.6786 average AUC on Ling-3.0-tiny and 0.7271 on Ling-3.0-flash make SingProbe best on both, ahead of DRIFT (0.6316; 0.6546).Its advantage also appears on individual datasets: on Ling-3.0-flash it ranks first on five of seven, including SQuAD at 0.8597.Source: Table 9 · p. 15, §4.7.1 · p. 15
  • On Ling-3.0-tiny, SingProbe leads on GSM8K (0.7328), MATH-500 (0.7292), and NQ-Open (0.8136), while on flash it ranks first on five of seven datasets.On flash, its AUC exceeds 0.73 on AA-Omniscience, GSM8K, MATH-500, and SQuAD; SQuAD reaches 0.8597.Source: §4.7.1 · p. 15

4.7.2 Application 2: SingProbe for Constrained Safe Decoding

  • 0.43 future unsafe rate in the highest safe-prefix-score interval is over 21× the approximately 0.02 rate in the lowest, showing risk is predictable before unsafe content appears.The highest interval’s unsafe rate is roughly 3.6× the overall base rate of 0.1203; all scored prefixes were annotated safe before eight continuations were sampled.Source: Figure 5 · p. 16, §4.7.2 · p. 16
  • At 50% branch pruning, SingProbe reduces the retained-pool unsafe rate to 0.53 from random pruning’s approximately 0.76, matching the oracle; at 90% pruning it reaches approximately 0.22.SingProbe and oracle curves nearly overlap through about 50% pruning, then diverge at aggressive ratios as SingProbe plateaus around 0.22–0.23 while the oracle approaches zero.Source: Figure 5 · p. 16, §4.7.2 · pp. 16–17

5 SingProbe-Med: On-Demand Medical-Risk Intervention · 5.1 Motivation and Framework Overview

SingProbe-Med turns SingProbe’s token-level internal-state signals from detection into selective medical-generation control: it monitors when to intervene and uses bounded decoding intervention only when needed.

5.1 Motivation and Framework Overview

  • A harmful medical recommendation can emerge after a clinically sound prefix, making query-level screening insufficient and post-hoc review too late (Draelos et al., 2026; Mishra et al., 2024b; Singhal et al., 2023; Yang et al., 2024).Source: §5.1 · p. 17
  • Always-on corrective decoding is undesirable because it can perturb otherwise correct medical content and incurs persistent decoding cost.Source: §5.1 · p. 17
  • SingProbe-Med separates continuous, lightweight intervention admission from corrective decoding, extending SingProbe’s internal-state signals from detection to generation-time control.The admission mechanism monitors the evolving generation trajectory; the target-supported intervention runs only after its admission condition is met.Source: §5.1 · p. 17
  • Once intervention is admitted, support-constrained Guided Decoding Steering (sGDS) contrastively re-ranks target-model candidate tokens over a localized risky suffix.Source: §5.1 · p. 17
  • The corrective decoding intervention is bounded, after which generation returns to the target model’s native decoding.This avoids persistent dual-branch inference for generations that do not need intervention.Source: §5.1 · p. 17
  • The framework is instantiated on AntAngelMed, a 100B-parameter MoE medical LLM.The section treats SingProbe-Med admission as deciding when to intervene and target-supported sGDS as deciding how.Source: §5.1 · p. 18

5.2 SingProbe-Med: When to Intervene

SingProbe-Med uses four parallel signals to decide when medical-risk intervention is applicable and timely. A fixed evidential hierarchy admits intervention based on eligibility, global risk, and—when needed—localized timing evidence.

5.2 SingProbe-Med: When to Intervene

  • Four prefix-derived signals capture intervention eligibility, trajectory-level medical risk, near-future errors, and the earliest confirmable error boundary.Future Risk covers likely errors in the next 1–32 tokens; Error Boundary identifies the first prefix at which an error can be confirmed.Source: §5.2 · p. 18
  • Intervention admission requires eligibility and uses global risk first, with future-risk or error-boundary evidence refining decisions under intermediate global risk.Strong global risk can admit intervention directly; the hierarchy reflects the asymmetric cost of false intervention.Source: §5.2 · p. 18
  • Future Risk and Error Boundary are complementary parallel entry signals, and operating thresholds and release conditions are fixed on the development split before final evaluation.Future Risk enables anticipatory intervention before an error is explicit, while Error Boundary supports containment once the error is confirmable; the hierarchy is evidential precedence, not a sequential pipeline.Source: §5.2 · p. 18

5.3 Target-Supported sGDS: How to Intervene

SingProbe-Med uses a medical risk-pattern branch to modify target-supported next-token choices only when intervention is activated. Its scope, branch training, and decoding procedure are designed to suppress covered medical-risk continuations while keeping the target model as the source of admissible tokens.

Medical-risk Intervention Scope

  • The intervention covers five medical-risk categories: dosage and administration errors, special-population contraindications, wrong indications, dermatology-specific wrong indications, and dermatology-specific special-population contraindications.The sampler maintains approximately balanced coverage across the categories for the risk-pattern adaptation data.Source: §5.3 · p. 19

Medical Risk-pattern Branch

  • The risk-pattern branch adapts deployed AntAngelMed with a LoRA module, training only its LoRA parameters on risk-pattern responses while freezing the base model.The autoregressive language-modeling objective increases the branch’s likelihood on continuations exhibiting the five covered medical-risk patterns.Source: §5.3 · p. 19

On-demand Target-supported sGDS

  • At activated steps, target and risk-pattern branches score the same realized context, and the risk branch acts as an anti-expert signal rather than an error probability or correction.The risk-pattern logits encode token-level preferences for continuations compatible with the covered patterns.Source: §5.3 · p. 19
  • Within the target-defined candidate support, sGDS suppresses candidates favored by the risk branch and promotes those favored by the target, re-ranking plausible target-supported tokens.The top-k restriction prevents the risk branch from introducing off-support continuations.Source: §5.3 · pp. 19–20
  • The branches share each sampled token and keep their prefixes synchronized, so their logits reflect the same generation history rather than diverging independently decoded histories.They maintain separate model states and KV caches while scoring the shared realized prefix.Source: §5.3 · p. 20
  • When intervention is inactive, generation uses unmodified target logits; when activated, SingProbe-Med controls when dual-branch decoding runs and sGDS controls how it modifies next-token choices.Full intervention uses k = 50 and α = 2 until release; the bounded variant uses k = 100, α = 3, and W = 64, after which decoding returns to target-only logits on the modified prefix.Source: §5.3 · p. 20

5.4 Evaluation

SingProbe-Med is evaluated for admission selectivity and timing, medical error correction, general capability preservation, and inference efficiency, with thresholds selected on development data and frozen for held-out evaluation.

5.4.1 Admission Selectivity and Timing

  • Eligibility-gated Global Risk improves ROC–AUC, AP, and PR–AUC over Global Risk alone by 0.025, 0.012, and 0.011, respectively.The gate suppresses risk scores on prefixes not considered eligible for medical-risk intervention.Source: §5.4.1 · p. 20, Table 12 · p. 21
  • Tightening the allowed White FPR from 10% to 1% raises precision from 92.34% to 96.77%, while Harm recall falls from 51.70% to 11.98%.This exposes a trade-off between intervention coverage and false-trigger control.Source: §5.4.1 · pp. 20–21
  • 54.99% of first activations on Harm trajectories occur at or before the confirmable medical-error boundary; evaluating all four signals takes 0.019749 ms/token.That corresponds to approximately 50,635 token states per second.Source: §5.4.1 · p. 21

5.4.2 Medical Error Correction

  • 25.03% of baseline erroneous generations were broadly mitigated by the full intervention, including 12.58% strictly repaired from ERROR to SAFE.Among activated trajectories, broad mitigation and strict repair were 30.27% and 15.22%; the bounded 64-token configuration achieved 24.30% and 11.72% overall, retaining 97.1% and 93.2% of the full-intervention results.Source: §5.4.2 · p. 21, Table 13 · p. 21
  • Safe regression is 1.84% overall for full intervention and 2.53% for the 64-token configuration, motivating selective rather than indiscriminate sGDS admission.Safe-regression rates are higher when conditioning on activated trajectories.Source: §5.4.2 · p. 21

5.4.3 General Capability Preservation

  • SingProbe-Med activates on 4.69% of HealthBench examples and none from the other five benchmarks, with scores broadly comparable to native AntAngelMed-100B; always-on sGDS scores lower on all six.The six benchmarks are AIME24/25, HealthBench, CFBench, Arena-Hard-v2, AlignBench, and WritingBench.Source: §5.4.3 · p. 22, Table 14 · p. 22

5.4.4 Inference Efficiency

  • A 64-token intervention window adds an estimated 1.843 seconds to single-branch latency, yielding approximately 39.162 seconds versus 60.934 seconds for resident dual-branch decoding.Continuous monitoring is estimated at 37.379 seconds versus the 37.319-second single-branch reference; most of the full intervention’s safety benefit is retained with the bounded window.Source: §5.4.4 · p. 22, Table 15 · p. 22

6 Conclusion

SingProbe reuses base-LLM hidden states for streaming monitoring of intent, safety, and hallucination risk, while its scores can also inform runtime decoding interventions. SingStreamBench evaluates streaming detectors’ temporal behavior.

6 Conclusion

  • SingProbe jointly monitors query intent, response safety, and hallucination risk during generation by reusing base-LLM hidden states, with strong detection performance and very low false-positive rates.This avoids redundant text encoding and reduces deployment complexity relative to standalone guardrail models.Source: §6 · p. 22
  • SingStreamBench evaluates whether streaming safety detectors remain silent on benign prefixes and identify the onset of unsafe content.Source: §6 · p. 22
  • SingProbe scores carry information about the safety tendency of future continuations and can serve as control signals for constrained decoding.This extends intrinsic guardrails beyond passive classification.Source: §6 · p. 22
  • SingProbe-Med demonstrates selective, on-demand intervention in medical generation, modifying outputs only when and where intervention is warranted.The conclusion frames SingProbe as a lightweight interface connecting LLM representations to generation-time safety mechanisms and runtime control.Source: §6 · pp. 22–23

Related papers

Loading 2608.30703v1…