PaperlayerView exact revision on arXiv

Source-linked AI summary

LatentAudit: Real-Time White-Box Faithfulness Monitoring for Retrieval-Augmented Generation with Verifiable Deployment

Zhe Yu, Wenpeng Xing, Meng Han

arXiv:2604.05358v1cs.AIcs.LG

TL;DR

RAG still leaves a serving-time question: whether an answer is supported by retrieved evidence. LatentAudit compares pooled residual-stream answer activations with an evidence representation, and its signal persists across model and retrieval shifts while supporting optional public verification.

  • Problem

    At serving time, RAG systems still need to determine whether generated answers follow retrieved passages, while common judge-model or sampling checks can add latency and expose private context.

    Source: §1 · p. 1

  • Method

    LatentAudit pools mid-to-late residual-stream answer activations and compares them with an evidence embedding using a calibrated Mahalanobis-distance rule, optionally compiled for zero-knowledge verification.

    Source: §1 · p. 2

  • Results

    Across three QA benchmarks and five model families, the faithfulness signal survives architecture and retrieval shifts; on PubMedQA, LatentAudit reaches 0.942 AUROC at 0.77 ms overhead.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Residual-stream geometry provides a basis for real-time RAG faithfulness monitoring, and fixed-point circuits preserve 99.8% of FP16 AUROC for optional public verification.

    Source: §6 · pp. 9–10

    The auditor checks adherence to retrieved evidence, not its truth, so poisoned content can still be propagated by a faithful generation.

    Source: §5 · p. 9

Abstract

from arXiv · show

Retrieval-augmented generation (RAG) mitigates hallucination but does not eliminate it: a deployed system must still decide, at inference time, whether its answer is actually supported by the retrieved evidence. We introduce LatentAudit, a white-box auditor that pools mid-to-late residual-stream activations from an open-weight generator and measures their Mahalanobis distance to the evidence representation. The resulting quadratic rule requires no auxiliary judge model, runs at generation time, and is simple enough to calibrate on a small held-out set. We show that residual-stream geometry carries a usable faithfulness signal, that this signal survives architecture changes and realistic retrieval failures, and that the same rule remains amenable to public verification. On PubMedQA with Llama-3-8B, LatentAudit reaches 0.942 AUROC with 0.77,ms overhead. Across three QA benchmarks and five model families (Llama-2/3, Qwen-2.5/3, Mistral), the monitor remains stable; under a four-way stress test with contradictions, retrieval misses, and partial-support noise, it reaches 0.9566--0.9815 AUROC on PubMedQA and 0.9142--0.9315 on HotpotQA. At 16-bit fixed-point precision, the audit rule preserves 99.8% of the FP16 AUROC, enabling Groth16-based public verification without revealing model weights or activations. Together, these results position residual-stream geometry as a practical basis for real-time RAG faithfulness monitoring and optional verifiable deployment.

1 Introduction

LatentAudit addresses whether a RAG answer follows retrieved evidence at serving time, when common verification approaches can add latency and expose private context. It monitors residual-stream activations and evaluates the signal across model and retrieval shifts, with an optional zero-knowledge deployment path.

Motivation

  • RAG conditions generation on external evidence, but whether an answer actually follows retrieved passages remains a serving-time question.Judge-model routing and stochastic sampling can incur multi-second latency and leak private context to external APIs.Source: §1 · p. 1

Approach

  • LatentAudit pools mid-to-late residual-stream answer activations and compares them with an evidence embedding using Mahalanobis distance and a small held-out calibration threshold.No auxiliary network is trained; the same quadratic rule can optionally be compiled into a Groth16 circuit.Source: §1 · p. 2

RQ1

  • 0.942 AUROC at 0.77 ms overhead: a Mahalanobis monitor on pooled answer-state activations detects faithfulness on PubMedQA.The paper frames this as evidence of a usable latent faithfulness signal.Source: §1 · p. 2

RQ2

  • Evaluation spans three QA benchmarks, five model families, cross-domain threshold reuse, and retrieval stress tests involving contradictions, misses, and partial-support noise.These evaluations address whether the signal survives architecture and retrieval shift.Source: §1 · p. 2

RQ3

  • The quadratic decision rule can be compiled for verifiable deployment, with fixed-point quantization and reported proof-generation and on-chain verification costs.This addresses whether the monitor is simple enough for public verification.Source: §1 · p. 2

2 Related Work

Related work connects LatentAudit to residual-stream truthfulness signals, black-box faithfulness checks, and cryptographic verification. The paper uses latent activation geometry as a run-time monitor and limits the proof circuit to the audit rule.

Mechanistic Interpretability and Truthfulness

  • ROME (Meng et al., 2022) and Inference-Time Intervention (Li et al., 2023) locate factual-recall circuits in transformer MLPs, while residual-stream states can carry separable truthfulness signals even when sampled tokens are wrong.LatentAudit operationalizes this activation geometry as a run-time RAG faithfulness monitor rather than an editing or interpretability probe.Source: §2 · p. 2

Faithfulness Verification and LLM-as-a-Judge

  • Judge models, NLI classifiers, and self-consistency sampling treat the generator as a black box, add forward passes, and may expose private context to external APIs.LatentAudit instead reads the generator’s own hidden states.Source: §2 · p. 2

zkML and Verifiable Inference

  • Full-model zkML faces impractical O(N^2) attention costs at billion-parameter scale, whereas LatentAudit places only its O(d^2) Mahalanobis test in the proof circuit.The paper reports that this design keeps proving time in the millisecond range.Source: §2 · p. 2

3 Methodology

LatentAudit separates a generation-time latent monitor from an optional proof layer. It compares a pooled answer-state representation with retrieved evidence and can prove the resulting audit decision without revealing model parameters or latent coordinates.

3 Methodology

  • LatentAudit has two modular layers: a latent faithfulness monitor during generation and an optional zero-knowledge proof layer for deployment.The overview compares answer-state activations from layers L14–L16 with an evidence representation using Mahalanobis distance; the score is produced in 0.77 ms.Source: §3 · p. 2, Figure 1 · p. 3

3.1 Answer-State Representation

  • Answer-state representation pools answer-span activations through EOS into one vector, using mid-to-late residual-stream layers near the output projection.The approach follows the observation that factual knowledge concentrates in mid-to-late MLP updates.Source: §3.1 · p. 3

3.2 Residual-Stream Geometry and Decision Rule

  • The decision rule compares the pooled answer state with a projected evidence embedding using covariance-adjusted Mahalanobis distance, then flags scores above a calibration-set threshold.The evidence embedding is mean-pooled from retrieved context with a frozen dense retriever; its lightweight linear projector is fit on the 10% calibration split. The geometric assumption is that unsupported answers drift from the evidence-conditioned manifold, especially along low-variance directions.Source: §3.2 · pp. 3–4

3.3 Verification Circuit

  • The verification circuit quantizes the vectors and covariance matrix and encodes the decision inequality as a finite-field bilinear constraint.The resulting proof certifies the audit outcome without revealing the document representation or model parameters.Source: §3.3 · p. 4

3.4 On-Chain Deployment Path

  • For public verification, a proof and public inputs binding the generation segment and threshold configuration are submitted to an on-chain verifier.The verifier checks a single pairing equation, avoiding replay of the language-model computation; a passing check does not reveal latent coordinates or model weights.Source: §3.4 · p. 4

4 Experiments and Results

LatentAudit is evaluated on three QA benchmarks and five model families, with parameters calibrated on held-out data. Results test its discrimination, robustness to retrieval and domain shifts, and suitability for optional public verification.

4.1 Experimental Setup and Baselines

  • The evaluation uses balanced faithful and hallucinated sets, a 10% calibration split, and held-out evaluation; the cross-model and stress-test studies reuse frozen calibration parameters.The benchmarks are PubMedQA, HotpotQA, and TriviaQA; the four-way stress test adds retrieval-miss and unsupported-partial variants.Source: §4.1 · pp. 4–5

4.2 RQ1: Is there a usable latent faithfulness signal?

  • LatentAudit trails GPT-4o by 0.6 AUROC points and 1.2 F1 points on Llama-3-8B while running at sub-millisecond cost.Table 1 compares per-query latency and notes that proving refers to the optional ZK layer.Source: §4.2 · p. 5, Table 1 · p. 5
  • Top-8 mean-pooling reaches 0.942 AUROC on Llama-3-8B/PubMedQA, versus 0.884 for last-token evaluation and 0.912 for max-pooling; discrimination emerges in mid-to-late layers.For Llama-3-8B, the sharpest per-layer AUROC jump occurs around layers 14–16, and the chosen layer separates faithful from contradicted generations in the t-SNE projection.Source: §4.2 · p. 6, Figure 2 · p. 6

4.3 RQ2: Does the signal survive architecture and retrieval shift?

  • Across five model families, AUROC ranges are 0.925–0.948 on PubMedQA, 0.915–0.940 on TriviaQA, and 0.905–0.928 on HotpotQA.HotpotQA is consistently the hardest benchmark; the narrow spread suggests the geometric separation is not confined to one model family.Source: §4.3 · p. 6
  • Under four-way retrieval stress, PubMedQA AUROC ranges from 0.9566 to 0.9815 and HotpotQA AUROC from 0.9142 to 0.9315 across representative models.The stress test includes contradictions, retrieval misses, and partial-support noise; partial-support cases are hardest, with pairwise AUROC 0.9218 on PubMedQA and 0.8364 on HotpotQA.Source: §4.3 · p. 7, Table 3 · p. 8
  • Cross-domain threshold transfer loses 2–3 AUROC points in each direction, while residual errors concentrate in unsupported partial cases with high lexical overlap.False negatives also occur with thin retrieval margins and, on HotpotQA, single-token answers that produce weak answer-state summaries.Source: §4.3 · p. 7

4.4 RQ3: Is the rule simple enough for verifiable deployment?

  • At k=16, fixed-point quantization preserves >99.8% of FP16 decision quality, while the optional proof layer costs $0.0017 per query on Arbitrum L2.The local audit takes 0.77 ms; the proof layer adds cost without changing the underlying audit rule.Source: §4.4 · p. 8, Table 5 · p. 8

5 Discussion and Limitations

LatentAudit uses residual-stream geometry to compare answer states with retrieved evidence, while optional zero-knowledge verification adds deployment cost. Its boundaries include hidden-state access, quantization risk near the decision threshold, corpus trust, and limited evidence on larger models.

5 Discussion and Limitations

  • Faithful answers preserve coupling between retrieved evidence and answer tokens in the residual stream; Mahalanobis distance amplifies unsupported drift along directions rare under grounded completions.Unsupported answers may remain fluent even as their pooled states depart from the evidence-defined covariance structure.Source: §5 · pp. 8–9
  • The audit itself is sub-millisecond, while optional public verification adds proof cost but is described as deployable.Throughput and cost–quality Pareto analyses are also reported.Source: Figure 4 · p. 9
  • The monitor requires open-weight hidden states and cannot directly audit black-box APIs, although a smaller open-weight surrogate or pooled visual cross-attention states are proposed alternatives.The multimodal adaptation is proposed for multimodal RAG.Source: §5 · p. 9
  • Fixed-point rounding can move samples near the decision boundary across it; a continuous safety margin models rounding drift to conservatively bound the threshold.The margin is detailed in Appendix J.Source: §5 · p. 9
  • The auditor checks adherence to retrieved evidence, not whether that evidence is true, so poisoned documents can lead to faithful misinformation.The paper proposes binding context hashes to trusted document signatures before auditing.Source: §5 · p. 9
  • The zero-knowledge layer certifies correct computation of the reported audit score, not the quality of the latent faithfulness signal.The authors characterize proof as a cryptographic convenience rather than a substitute for empirical ML validation.Source: §5 · p. 9
  • Evidence for scaling to frontier models remains open: evaluation covers 7B and 8B families, while behavior at 70B+ is untested.The authors hypothesize that pooling may need adaptation for distributed layer allocation in larger models.Source: §5 · p. 9

6 Conclusion

The conclusion presents LatentAudit as a real-time monitor based on mid-to-late residual-stream geometry, with evidence across architectures, domains, and retrieval failures. The appendices describe evaluation construction, calibration, implementation, and a zero-knowledge circuit for verifying the audit.

6 Conclusion

  • The conclusion reports that fixed-point zero-knowledge circuits preserve 99.8% of FP16 AUROC, alongside a faithfulness signal that survives architecture, domain, and retrieval shifts.It describes the method as a scalable systems primitive and notes future work on latent features, interventions, and lighter proof architectures.Source: §6 · pp. 9–10

A Dataset Construction Details

  • Stress evaluations include faithful, contradicted, retrieval-miss, and partial records; PubMedQA uses mismatched evidence and HotpotQA removes one supporting document.PubMedQA retrieval misses use topically similar, source-mismatched snippets; partial examples retain weak or incomplete evidence.Source: A Dataset Construction Details · p. 12

B Hyperparameter Summary

  • TF-IDF top-k answer activations form a mean-pooled centroid, and Youden’s J calibrates τ* (bootstrap σ=0.063 PubMedQA; σ=0.086 HotpotQA).The centroid is reported as stable across bootstrap splits, with σ<0.02.Source: B Hyperparameter Summary · p. 12

C Latent Geometry Distributions

  • All five model families show separated faithful and contradicted distance distributions on PubMedQA; the aggregate panel indicates separation is not architecture-specific.Figure 5 disaggregates the distributions by model and includes an aggregate panel.Source: Figure 5 · p. 13, C Latent Geometry Distributions · p. 13

D Code Architecture

  • The codebase separates alignment, model loading and activation extraction, proof, retrieval, and dataset modules, with support for Llama, Qwen, and Mistral families.The proof module quantizes activations, assembles circuit inputs, generates proofs, and validates them.Source: D Code Architecture · p. 13

E ZK Circuit Details

  • The circuit checks whether the quantized Mahalanobis distance is at most the quantized threshold, using the activation centroid, evidence vector, and inverse covariance as witness.The verifier consumes about 580.6K gas; the reported costs are $21.77 on Ethereum L1 and $1.09 on Arbitrum L2.Source: E ZK Circuit Details · pp. 13–14

F Per-Model Detailed Results

Across five model families, LatentAudit matches or exceeds the internal-state baselines and approaches the GPT-4o ceiling.

F Per-Model Detailed Results

G Extended Deployment Analysis

The extended deployment analysis compares throughput scaling and the cost–quality frontier. It reports efficient batching for LatentAudit and places its detection quality near GPT-4o at a much smaller cost.

G Extended Deployment Analysis

  • LatentAudit’s quadratic evaluation enables efficient batching, and it closely bounds GPT-4o detection quality at a tiny fraction of its cost.Figure 6 plots throughput scaling with batch size and the cost–quality Pareto front.Source: Figure 6 · p. 14, G Extended Deployment Analysis · p. 14

H Methodology Ablations

The ablations favor mean-pooling TF-IDF-salient answer tokens over single-token pooling, and show only a small AUROC reduction when the calibration split is reduced.

H Methodology Ablations

  • 0.003 average PubMedQA AUROC reduction results when the calibration split shrinks from 10% to 5% of the available training pool.The passage attributes this sample efficiency to Ledoit-Wolf covariance estimation.Source: H Methodology Ablations · p. 14
  • Mean-pooling across k ∈[4, 16] TF-IDF salient tokens significantly outperforms last-token evaluation and max-pooling.Table 8 reports PubMedQA AUROC across pooling strategies and token counts for Llama-3-8B.Source: H Methodology Ablations · p. 14, Table 8 · p. 15

I Qualitative Error Analysis

The qualitative stress test identifies contradictions and retrieval misses as reliably rejected, while partial-support cases can produce false positives. A continuous safety margin addresses quantization drift in the fixed-point audit.

I Qualitative Error Analysis

  • The Mahalanobis metric reliably rejects outright contradictions and retrieval misses, while partial-support examples can yield false positives when snippets lack evidential detail.The cited example involves high lexical overlap despite insufficient evidence; Table 9 gives representative PubMedQA stress-test examples.Source: I Qualitative Error Analysis · p. 15, Table 9 · p. 15

J Continuous Safety Margin for Quantization

  • The safety margin bounds worst-case quadratic-form quantization drift by ε(k) = O(d · 2−k · λmax(Σ−1)).The margin is defined for target fractional precision k.Source: J Continuous Safety Margin for… · p. 15
  • At k = 16, ε(16) ≈0.04 bounds the maximum observed score drift and keeps queries hazardous in Rd from falsely clearing the Fp circuit.The on-chain threshold is set conservatively as ˆτ∗safe = ˆτ∗−ε(k).Source: J Continuous Safety Margin for… · p. 15

K Robustness of the Affine Projector (Wproj)

The projector ablations favor a regularized affine ridge mapping over PCA and a nonlinear MLP, and report that the ridge estimator's evaluation AUROC plateaus with 200 calibration samples.

K Robustness of the Affine Projector (Wproj)

Related papers

Loading 2604.05358v1…