SAC3: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency

• Large language models often produce confident but erroneous assertions, limiting their use where factual accuracy matters.Source: §1 · p. 1
• Self-consistency can miss both consistently wrong answers and disagreements caused by factually correct original responses.Source: §1 · pp. 1–2, Figure 1 · p. 2
• SAC3 detects question-level hallucinations by comparing responses to semantically equivalent question perturbations.Source: §1 · p. 2
• SAC3 detects model-level hallucinations by cross-checking responses from different language models.Source: §1 · p. 2
• AUROC reaches 99.4% and 97.0% on classification QA tasks, while open-domain generation reaches 88.0% and 77.2%, improving over self-consistency by 13.8 and 6.7 percentage points.Source: §1 · p. 2

• Hallucination research spans confidence-based detection, supervised approaches, external knowledge retrieval, and evaluation across several NLP tasks.Source: §2 · pp. 2–3
• Confidence-based methods may require token-level log probabilities, while supervised methods may rely on internal language-model states unavailable through limited APIs.Source: §2 · p. 3
• SAC3 focuses on detecting hallucinations in open-domain QA with black-box language models without external resources.Source: §2 · p. 3
• Existing self-consistency approaches use sampled response agreement for hallucination detection, but their potential failures in factuality assessment remain underexplored.Source: §2 · p. 3

• Self-consistency assumes that knowledgeable models produce similar samples while knowledge gaps yield divergent, contradictory hallucinations.Source: §3 · p. 3
• A model may consistently answer incorrectly, causing self-consistency checks to miss hallucinations.For example, ChatGPT consistently answers “Is pi smaller than 3.2?” incorrectly.Source: §3 · p. 4
• A factually correct original answer may be followed by incorrect high-temperature samples, causing self-consistency methods to produce false positives.Source: §3 · p. 4
• Because self-consistency is neither necessary nor sufficient for factuality verification, more robust assessment is needed for complex QA and open-domain generation.Source: §3 · p. 4

4.1 Stage 1: Question-level Cross-checking via Semantically Equivalent Perturbations

• SAC3 generates semantically equivalent question variants, verifies their equivalence to the original, and uses them for question-level cross-checking.The method prompts an LM to rephrase the query, then filters variants through pairwise semantic-equivalence checks.Source: §4.1 · p. 4

4.2 Stage 2: Model-level Cross-check with Additional Verifier LM

• SAC3 samples responses from both the target LM and a verifier LM across the original and perturbed questions, collecting them for consistency scoring.The collected sample sets cover self-checking, cross-model checking, and cross-question checking.Source: §4.2 · p. 4

4.3.1 Semantic-aware Consistency Check of QA Pairs

• The method compares complete QA pairs rather than answers alone because equivalent questions can have lexically different but semantically equivalent answers.Its semantic-equivalence operator returns Yes or No and maps these judgments to numerical scores.Source: §4.3.1 · p. 5, §4.3.2 · p. 5

4.3.3 Question-level Consistency (SAC3-Q) Score

• SAC3-Q computes question-level consistency across responses to the original and perturbed questions, while SAC3-M and SAC3-QM add verifier-based cross-model checks.These scores correspond to the question-level and model-level cross-checking components of SAC3.Source: §4.3.3 · p. 5, §4.3.4 · p. 5

4.3.5 Final Score and Model Confidence

• The final score combines SAC3-Q, SAC3-M, and SAC3-QM, with λ weighting verifier-LM contributions and defaulting to λ = 1.The components can be computed in parallel, and detection compares the combined score with a preset threshold.Source: §4.3.5 · p. 6

5 Data and Annotation

• The evaluation contains two classification QA datasets and two open-domain generation QA datasets.The classification datasets are Prime number and Senator search; the generation datasets are HotpotQA-halu and NQ-open-halu.Source: §5 · p. 6
• Senator search contains 500 questions whose factual answer is always “No,” paired with synthesized “Yes” hallucinations.The questions ask whether a senator matching a specified state and college combination existed.Source: §5 · p. 6
• HotpotQA-halu uses 250 sampled training examples with hallucinated answers generated by gpt-3.5-turbo and manually annotated against ground truth and knowledge.The annotations assess answer factuality.Source: §5 · p. 6
• NQ-open-halu contains 250 factual and non-factual examples with manual annotations under the same setting as HotpotQA-halu.Natural Questions is treated as the more challenging open-domain QA benchmark.Source: §5 · p. 6

6.1 Experimental Setup

• Experiments use GPT-3.5 as the target LM, Falcon-7b-instruct or Guanaco-33b as verifier LMs, and AUROC and accuracy as evaluation metrics.The setup uses deterministic temperature-0 semantic perturbations, ten perturbed questions, and eight NVIDIA V100 GPUs.Source: §6.1 · pp. 6–7

6.2.1 Classification QA

• On balanced classification datasets, SAC3-Q achieves over 99% AUROC on both datasets, compared with SC2 scores of 65.9% and 56.1%.The datasets contain equal proportions of hallucinated and factual samples.Source: §6.2.1 · p. 7, Table 2 · p. 7
• SAC3-all with Guanaco-33b achieves the highest classification-QA detection accuracy, exceeding SC2 by 51.2%, SAC3-Q by 6.2%, and SAC3-QM by 5.0%.This result uses the unbalanced classification setting with a threshold of 0.5.Source: §6.2.1 · p. 7, Table 3 · p. 7
• SAC3-Q and SAC3-all with Guanaco-33b remain more robust than SC2 as the detection threshold increases in hallucination-only classification datasets.Figure 3 evaluates the effect of threshold choice on detection accuracy.Source: §6.2.1 · p. 7, Figure 3 · p. 7

6.2.2 Open-domain Generation QA

• In open-domain generation QA, SAC3-Q improves AUROC over SC2 by 7% on both datasets, while SAC3-all performs consistently well across both datasets.SAC3-QM improves HotpotQA-halu by 6.7% but is slightly worse on NQ open.Source: §6.2.2 · p. 8, Table 4 · p. 8
• Verifier weighting should reflect relative model quality: higher verifier weight helps on HotpotQA-halu, whereas smaller weight helps on NQ open.The verifier outperforms the target LM on HotpotQA-halu, while the target LM is competitive on NQ open.Source: §6.2.2 · p. 8, Figure 5 · p. 8
• Increasing perturbed questions improves SAC3 performance, but gains diminish beyond five samples, making 2–4 samples a lower-cost practical choice.The experiment varies the number of perturbed questions from 2 to 10.Source: §6.2.2 · p. 8, Figure 6 · p. 8
• SAC3-Q consistently outperforms SC2 across GPT-3.5, GPT-4, and PaLM 2 target LMs.The comparison uses balanced classification and generation QA tasks.Source: §6.2.2 · p. 9, Table 5 · p. 9

7 Conclusion and Discussion

• Self-consistency checking alone cannot reliably detect question-level and model-level hallucinations, including consistently wrong responses.The authors also report that smaller models can answer some questions correctly when larger models hallucinate.Source: §7 · p. 9
• Cross-checking semantically equivalent questions can reduce persistent hallucinations, potentially by reducing question-level ambiguity.Source: §7 · p. 9
• Model-level hallucination differences are attributed to inherent capability differences arising from models’ training procedures and data, guiding verifier selection by task.Source: §7 · p. 9

Ethics Statement

• The paper’s broader-impact rationale is to support more reliable language models by mitigating misinformation and biased outputs while promoting accountability and trust in AI systems.Source: Ethics Statement · p. 9

Limintations

• The experiments focus on question answering, so generalizability to conversational or dialogue prompting and semantic-equivalence accuracy on complex tasks remain open questions.The authors also identify a trade-off between larger sample sizes, detection accuracy, and added cost or latency.Source: Limintations · p. 9

A.1 Background

• Black-box factuality assessment is needed because token-level output probabilities required by entropy-based uncertainty metrics are typically unavailable from systems such as ChatGPT.White-box methods associate factual statements with lower entropy and non-factual statements with higher entropy.Source: §A.1 · p. 12

A.2 Black-box Assessment via Checking Consistency in Sampled Responses

• Sampling-based black-box assessment generates a low-temperature candidate, higher-temperature supporting responses, and a hallucination score based on their consistency.The score approaches 0 when supporting responses agree with the candidate and 1 when they disagree.Source: §A.2 · p. 12
• Consistency between candidate and supporting responses is typically measured with textual entailment or lexical similarity using an additional pretrained NLU model such as BERT.Source: §A.2 · p. 12

B Complete Prompts

• The paper’s prompt templates cover semantically equivalent input-question perturbations and semantic-equivalence checks between question-answer pairs.Source: B Complete Prompts · p. 12

C.1 Computational Cost

• Parallel API calls for target or verifier LMs mitigate model-level inference overhead, while users can trade detection accuracy for lower computational cost.Source: §C.1 · p. 12
• SAC3 performance improves with more question samples but plateaus beyond five, making 2-4 samples a practical low-cost choice.Source: §C.1 · p. 12
• Generating multiple paraphrases in one API call can reduce question-paraphrasing complexity from repeated calls, while parallel consistency checks can reduce O(n) inference complexity to O(1).Source: §C.1 · p. 12
• The framework supports an efficiency-focused light version and an accuracy-focused performance version through adjustable sampling and prompting strategies.Source: §C.1 · pp. 12–13
• The prompt-template resources include semantically equivalent question perturbations and semantic-equivalence checks for question-answer pairs.Source: Table 6 · p. 13, Table 7 · p. 13

C.2 Semantic Consistency Checking

• LLM-based semantic consistency checking is intended to improve accuracy over BERTScore-like similarity metrics while removing the need for an additional model.Source: §C.2 · p. 13
• SAC3 is not dependent on a specific LLM, so users may freely select models for its components; the experiments instantiate the framework with GPT.The paper notes that open-source instruction-following LMs can also be used in practice.Source: §C.2 · p. 13

C.3 Data Annotations

• Prime Number uses “Yes” as the factual answer and “No” as the synthesized hallucination, while Senator Search reverses these labels.Source: §C.3 · p. 13
• The binary classification setup reflects that most gpt-3.5-turbo responses on these datasets are “Yes” or “No.”Source: §C.3 · pp. 13–14
• HotpotQA-halu and NQ-Open-halu use gpt-3.5-turbo answers whose factuality labels were manually assigned by comparison with ground truth.Source: §C.3 · p. 14
• SAC3 supports hallucination evaluation across synthesized and LLM-generated samples through semantically equivalent question perturbation and cross-model response consistency checking.Source: §C.3 · p. 14

D Additional Examples

• Table 6 provides an illustrative senator-search example of the methodology.Source: D Additional Examples · p. 14