PaperlayerView exact revision on arXiv

Source-linked AI summary

IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals

Md. Atabuzzaman, Christian Alexander, Chris Thomas

arXiv:2609.01375v1cs.CVcs.CL

TL;DR

LVLMs can generate plausible but factually incorrect content, while existing factuality guarantees depend on external verifiers or unreliable confidence signals. IntroConformal uses model-internal conformity scores within training-free CRC to provide finite-sample, distribution-free guarantees. Across architectures and tasks, verification probability reduces abstention and improves discrimination while maintaining valid conformal control.

  • Problem

    Existing factuality-control methods rely on external verifiers or generation-time confidence signals, which add dependencies or can fail for confident incorrect outputs.

  • Method

    IntroConformal is a training-free CRC framework using model-derived semantic stability and verification probability conformity scores.

  • Results

    Across diverse vision–language tasks and architectures, verification probability maintains conformal risk guarantees while achieving lower abstention and stronger factuality discrimination than comparison signals.

  • Takeaways & Limitations

    Model-internal signals can support factuality control with finite-sample, distribution-free guarantees without external verifiers or auxiliary models.

  • Takeaways & Limitations

    Semantic stability requires hidden-state access, while verification probability requires exposed logits; adversarial inputs can bias its logits and void the bound.

Abstract

from arXiv · show

Large Vision-Language Models (LVLMs) have achieved strong multimodal performance, yet ensuring the factual correctness of generated content remains challenging. Existing methods that provide statistical guarantees on factuality typically rely on external verifiers or generation-time confidence signals, which introduce auxiliary dependencies or often fail for confident but incorrect outputs. We argue that reliable factuality control can instead be achieved through introspective signals derived from the model itself. We introduce IntroConformal, a training-free Conformal Risk Control (CRC) framework that provides finite-sample, distribution-free factuality guarantees. We first instantiate it with layer-wise semantic stability, a conformity score derived from hidden-state representations, and then propose verification probability, a stronger score capturing the model's self-administered judgment on claim factuality. Across multiple LVLM architectures, IntroConformal satisfies the conformal risk guarantee while substantially reducing abstention and achieving competitive or superior claim-level discrimination relative to external verifier-based baselines.

1 Introduction

IntroConformal addresses factuality failures in LVLMs by using model-internal signals within a training-free CRC framework. It introduces semantic stability and verification probability to provide statistical guarantees while improving practical factuality control.

  • LVLMs can produce plausible but factually ungrounded content, creating trust and deployment concerns in high-stakes domains.
  • Formal factuality control requires finite-sample bounds on non-factual claims while preserving useful outputs.
  • Existing conformal methods rely on unreliable generation-time confidence or external verifiers that add deployment dependencies.
  • IntroConformal is a training-free CRC framework that derives conformity scores entirely from the model itself.
  • Layer-wise semantic stability measures alignment between mid- and late-layer hidden states but has modest discrimination and high abstention.
  • Verification probability queries the same model with a binary factuality prompt and uses output logits to improve discrimination and reduce abstention while preserving conformal guarantees.

2 Related Work

Prior uncertainty and hallucination methods often require repeated generation or remain diagnostic rather than statistically guaranteed. LVLM uncertainty estimation also faces additional multimodal grounding challenges.

  • Verbalized confidence and sampling-based consistency require multiple generations and can fail on confident hallucinations.
  • Semantic entropy methods estimate uncertainty through repeated sampling at inference time.
  • Internal activations and representation geometry encode factuality signals through hidden-state classifiers and related analyses.
  • Mechanistic studies associate non-factual generations with layer-wise semantic drift and unstable hidden-state trajectories.
  • These internal-signal approaches are primarily diagnostic and lack distribution-free statistical guarantees.
  • Uncertainty estimation in LVLMs introduces additional multimodal grounding challenges.

3 Method

IntroConformal uses model-internal conformity scores within CRC to filter atomic claims while controlling non-factual risk with finite-sample, distribution-free guarantees. It combines hidden-state semantic stability with verification probability, then calibrates the least conservative valid threshold.

  • Framework: IntroConformal is a training-free CRC framework that derives conformity scores entirely from the LVLM itself.It is designed to provide finite-sample, distribution-free factuality guarantees without external verifiers or auxiliary supervision.
  • Framework: The framework decomposes responses into atomic claims and retains claims whose selected conformity score exceeds a calibrated threshold.The retained subset is chosen to control the expected rate of non-factual claims at a user-specified risk level.
  • Conformity scores: Layer-wise semantic stability Ssem measures alignment between mid- and late-layer hidden representations for claim tokens.Higher Ssem indicates stable semantic grounding, while lower values reflect semantic drift associated with non-factual claims.
  • Conformity scores: Verification probability Sprob uses the same LVLM’s normalized Yes-token probability from a binary claim-factuality prompt.Unlike discrete verification sampling, it extracts the probability in a single forward pass without additional decoding steps.
  • Calibration: CRC selects the least conservative threshold whose Hoeffding upper confidence bound satisfies the target risk constraint.With n = 400 calibration prompts and at most 50 claims per response, testing thresholds at α = δ = 0.1 yields concurrent risk control at α′ ≤ 0.170.
  • Risk definition: The response-level fractional loss measures the proportion of retained claims that are non-factual and assigns zero loss when all claims are filtered.This loss is not monotone in the threshold because removing a factual claim can increase the non-factual proportion.

4 Experiments and Evaluation

IntroConformal is evaluated across three grounded vision–language tasks and multiple LVLM architectures for signal discrimination, conformal risk control, baseline comparison, and annotation-noise robustness. Verification probability consistently provides stronger discrimination and lower abstention while preserving the conformal risk guarantee.

  • Evaluation Setup: The evaluation covers general scene understanding, fine-grained captioning, and document understanding across multiple LVLM architectures.Experiments assess both factuality-signal separation and valid, efficient conformal risk control.
  • Introspective Signal Quality: Sprob consistently achieves the strongest factual/non-factual separation across tasks and architectures.On MSCOCO, it reaches a score gap of +0.2014 and AUROC 0.819 on LLaVA-1.5, and a gap of +0.2397 and AUROC 0.763 on Phi-3.5-Vision.
  • Introspective Signal Quality: Generation-time confidence and Ssem can be weak or anticorrelated with factuality in some settings.Tprob on LLaVA-1.5 document understanding has gap −0.0182 and AUROC 0.493, while Ssem on Llama-3.2-Vision MSCOCO has gap −0.0003 and AUROC 0.488.
  • Conformal Risk Control Results: All proposed conformity signals satisfy the conformal risk requirement, while Sprob generally lowers abstention and improves claim-level performance.On LLaVA-1.5 MSCOCO, Sprob reduces abstention to 25% from 57% for CONFLVLM and 64% for Ssem, while improving F1 from 0.504 to 0.581.
  • Conformal Risk Control Results: Empirical risk remains below target across varying risks and calibration sizes, while stricter control increases abstention.The calibration-size sweep spans 50 to 400 examples, with risk approaching the target from below and abstention decreasing as calibration grows.
  • Baseline Comparison: IntroConformal achieves 97.4% claim filtering efficiency and 91% response accuracy on the LLaVA-1.5 general scene-understanding benchmark.These results improve over CONFLVLM’s 95.3% filtering efficiency and 90% response accuracy.
  • Robustness to Annotation Noise: With symmetric calibration-label noise, test risk stays below α = 0.10 while abstention rises from 25% to 77%.Test risk decreases from 0.054 at 0% noise to below 0.001 at 15% noise, reflecting a utility trade-off rather than bound violation.

5 Conclusion

The paper concludes that IntroConformal enables training-free factuality control from introspective LVLM signals. Across vision–language generation tasks, Sprob provides stronger discrimination than external verification and token-confidence signals while retaining valid conformal guarantees and lower abstention.

  • Conclusion: IntroConformal uses layer-wise semantic stability and verification probability to provide finite-sample, distribution-free response-level non-factual-risk guarantees.The framework derives conformity signals entirely from the model without external verifiers or auxiliary models.
  • Conclusion: Sprob consistently strengthens factuality discrimination relative to CLIP-based verification and generation-time confidence signals, leading to lower abstention while maintaining valid control.The conclusion attributes the cross-task result to introspective conformity signals used within the CRC framework.

Limitations

IntroConformal’s guarantees are bounded by model access, calibration assumptions, annotation quality, and the probabilistic nature of the guarantee. These constraints define where recalibration, robustness work, and human oversight remain necessary.

  • Ssem requires white-box hidden-state access, whereas Sprob works with openweight models or APIs exposing output logits but not logit-withholding APIs.
  • The guarantee assumes a fixed model under exchangeability, so fine-tuning, RLHF updates, or checkpoint changes require recalibration.
  • The guarantee is defined relative to calibration labels, so systematic annotation bias could select a permissive threshold; human validation used one annotator.
  • Because the guarantee is probabilistic, safety-critical deployment should retain human oversight.

A Appendix

The appendix details the IntroConformal algorithm, qualitative examples, CRC behavior across tasks and calibration sizes, layer selection, and claim annotation procedures.

  • The appendix includes the full IntroConformal algorithm.
  • It presents qualitative examples of IntroConformal.
  • It analyzes CRC behavior across tasks and calibration sizes.
  • It examines layer selection for semantic stability and documents claim decomposition and annotation prompts.

A.1 Algorithm for IntroConformal

Algorithm 1 describes IntroConformal’s calibration pipeline: it extracts introspective scores, then uses Learn–Then–Test with Hoeffding’s inequality to choose a threshold meeting the target risk.

  • For each calibration claim, the pipeline extracts Ssem from hidden states and Sprob from the model’s binary verification judgment in a single forward pass.
  • Learn–Then–Test with Hoeffding’s inequality selects the least conservative threshold satisfying target risk α during calibration.

A.2 Qualitative Examples of IntroConformal

The qualitative examples show Sprob filtering non-factual claims while retaining factual ones, with conservative behavior near the calibrated decision boundary. Figure 3 uses per-λ α = 0.10 and guaranteed α′ = 0.170.

  • In the baseball example, Sprob filters both non-factual claims while retaining all four factual claims.The non-factual scores are below λ̂prob = 0.940, while factual scores range from 0.947 to 0.972.
  • The retained baseball claims contain zero non-factual content.
  • In the pizza example, Sprob correctly filters two non-factual claims but also filters factual or borderline claims near the threshold.The wine-glass claim scores 0.927 below λ̂prob = 0.940, illustrating boundary sensitivity.
  • This conservative filtering is expected at the operating point because the threshold bounds the non-factual rate among retained claims.Figure 3 uses per-λ α = 0.10 and concurrently guaranteed α′ = 0.170.

A.3 CRC Behavior Across Tasks and Calibration Sizes

Figures 4 and 5 show that IntroConformal maintains finite-sample risk control across tasks and architectures while exposing abstention–coverage and calibration-size trade-offs.

  • CRC validity: Empirical risk remains below target α for all α values from 0.05 to 0.40 across fine-grained captioning and document understanding.This supports valid finite-sample conformal risk control across both tasks and evaluated architectures.
  • Abstention–coverage trade-off: Stricter coverage requirements induce higher abstention, with Phi-3.5-Vision showing a sharper increase at high coverage thresholds than LLaVA-1.5.The abstention–coverage relationship characterizes the utility cost of stricter risk control.
  • Calibration size: Calibration size is swept from 50 to 400 examples at α = 0.10 and concurrently guaranteed level α′ = 0.170.The operating point is evaluated per-λ throughout the calibration-size analysis.
  • Calibration size: Document-understanding abstention decreases as calibration size grows and plateaus beyond approximately 200 examples.The observed plateau is consistent with O(1/√n) shrinkage of the Hoeffding upper confidence bound.
  • Procedure: IntroConformal extracts introspective signals in a single forward pass, computes semantic and verification-probability scores, and selects a least-conservative concurrently valid threshold.The algorithm outputs a calibrated threshold λ̂ after computing per-threshold risk and upper confidence bounds.

A.4 Effect of Layer Selection on Semantic Stability

This section specifies semantic-stability layer selection and the claim-level decomposition and annotation prompts used for factuality evaluation across captioning and document understanding.

  • Layer selection: The old semantic-stability configuration compares representations spanning the first quarter through the midpoint with those from the final quarter of layers.The new configuration compares the 8 layers immediately preceding the final block with the final 4 layers.
  • Layer selection: Table 5 reports the effect of these two layer-selection strategies on calibration-set signal quality for LLaVA-1.5 and Phi-3.5-Vision.The table defines M and T according to the old and new layer ranges.
  • Factuality annotation: Document-understanding annotation uses the same claim-level JSON output format but changes the error taxonomy to document-specific issues.The taxonomy covers field misinterpretation, numerical and quantitative errors, date errors, item errors, and OCR or layout issues.
  • Claim decomposition: Claim decomposition requires atomic, self-contained claims that each express one directly supported fact without added inferences or unresolved referents.The prompt also requires short declarative outputs and omits interpretive claims that cannot be verified from the image.
  • Factuality annotation: Fine-grained captioning annotations classify material factual errors involving object identification, attributes, spatial relations, actions, and quantitative information.Claims are labeled true when supported by visible image evidence and false when incorrect, hallucinated, or not visually verifiable.
Loading 2609.01375v1…