Source-linked AI summary
Leveraging Low-Level Symbolic Competences for Unsupervised Grounding in Hallucination Detection
Renato Vukovic, Hsien-chin Lin, Carel van Niekerk, Benjamin Ruppik, Michael Heck, Shutong Feng, Nurul Lubis, Milica Gasic
TL;DR
Hallucinations are factually incorrect or unsupported outputs, and their detection is hindered by opaque LLM reasoning. TeQHallu builds SQL databases from reference documents to support neurosymbolic verification, achieving competitive performance on RAGTruth and DiaHalu without domain-specific fine-tuning.
Problem
LLMs frequently generate factually incorrect outputs unsupported by source context, while recent work investigates whether lower-level symbolic competence can support unsupervised hallucination detection.
Method
TeQHallu prompts an LLM to convert reference documents into a queryable relational SQL database, then reasons over the reference and response for grounded hallucination detection.
Results
TeQHallu achieves competitive F1-scores against zero-shot baselines and fine-tuned models on RAGTruth and DiaHalu while providing traceable SQL reasoning traces.
Takeaways & Limitations
Grounding detection in SQL provides transparent, inspectable reasoning traces and can identify subtle unsupported claims that direct LLM predictions overlook.
Takeaways & Limitations
Effectiveness depends on the quality and completeness of reference documents, and translating unstructured data to SQL can lose information.
Abstract
from arXiv · showhide
Hallucination-where a language model generates outputs that are factually incorrect or unsupported by the source-is a major challenge for both prompted and fine-tuned language models. Detecting hallucinations is difficult due to the opaque reasoning processes of LLMs, which often provide little insight into why a model's output may be inaccurate. In this work, we investigate whether an LLM can use an alternative, low level, symbolic competence such as SQL for unsupervised hallucination detection in some high level task. For this, we make an LLM build an SQL database from reference documents. This SQL database is then used for reasoning over the reference and the sampled response in a hallucination detection pipeline that is grounded in the database, thereby providing a neurosymbolic checkup. On RAGTruth and DiaHalu hallucination detection datasets, we find that our approach improves on direct prediction and competes with state-of-the-art hallucination detection methods, while not requiring domain-specific fine-tuning. Instead it relies on a low-level general competence already present in LLMs. This warrants further investigation of low-level LLM competences in neurosymbolic approaches.
1 Introduction
LLM hallucinations are difficult to detect and interpret because outputs can be unsupported by source context while model reasoning remains opaque. TeQHallu addresses this by using SQL to structure reference knowledge and provide an inspectable, unsupervised checkup.
- LLMs frequently generate factually incorrect outputs unsupported by source context, creating hallucinations that are difficult to detect.
- Opaque LLM reasoning and black-box detection methods provide limited insight into why outputs are inaccurate.
- TeQHallu prompts an LLM to build a relational SQL database from reference documents and uses it to reason over references and responses.
- The method introduces an unsupervised text-to-SQL approach that leverages low-level LLM competence to structure reference knowledge for hallucination detection.
- SQL-grounded detection provides transparent reasoning traces and competes with state-of-the-art methods on RAGTruth and DiaHalu without domain-specific fine-tuning.
- The paper presents low-level, task-agnostic competence as a promising basis for neurosymbolic checks of LLM reliability.
2 Related Work
Prior hallucination detectors estimate inconsistency or uncertainty but often operate as black boxes. TeQHallu repurposes text-to-SQL and structured ontologies for explicit, transparent hallucination verification.
- Hallucination detection methods use sequence metrics, self-consistency, internal states, semantic entropy, or external knowledge cross-referencing.
- These techniques typically yield binary labels without interpretable reasoning or granular justifications.
- Text-to-SQL methods use executable queries and decomposed subtasks as structured reasoning tools for factual reliability.
- Earlier text-to-SQL ontology work lacked a downstream application for generated knowledge bases, which TeQHallu addresses through hallucination detection.
- The approach is motivated by the limited study of text-to-SQL pipelines as explicit evaluation mechanisms in dialogue systems.
3 The TeQHallu Framework
TeQHallu converts retrieved passages into a relational database, retrieves SQL evidence for response claims, and reconciles that evidence with an initial neural prediction. The result is a structured, inspectable hallucination decision.
- TeQHallu constructs a relational database from retrieved passages through multi-step prompting and incremental schema updates.
- The framework converts textual variables, measurements, entities, and qualifiers into relational tuples while preserving symbolic evidence and provenance.
- The system isolates response claims and generates targeted SELECT queries conditioned on the original question and response.
- SQL execution retrieves evidence directly from source-grounded database entries for factual validation.
- The pipeline compares an initial neural-only hallucination judgement with SQL retrieval outputs, interpreted summaries, and textual context.
- Responses are classified as fully supported, partially supported, unsupported, or contradicted, with structured explanations grounded in SQL results.
4 Experiments
The experiments evaluate TeQHallu across RAGTruth tasks and DiaHalu task-oriented dialogue using direct, unsupervised, fine-tuned, and structured baselines. The setup isolates the contribution of SQL grounding and spans multiple model families and datasets.
- RAGTruth evaluation covers question answering, data-to-text generation, and summarisation, with 900 document-response pairs per test set.
- DiaHalu evaluation uses task-oriented dialogue data with hallucination categories including non-factuality, incoherence, irrelevance, and over-reliance.
- Gemini 2.5 Flash serves as the main model for both direct hallucination prediction and text-to-SQL grounded reasoning.
- The experiments compare TeQHallu with unsupervised, fine-tuned, and state-of-the-art detection methods on response-level performance.
- The direct prediction baseline uses zero-shot prompting to classify responses from reference text and provide natural-language explanations.
- Non-SQL baselines use atomic claim decomposition and semi-structured key-value representations to test whether gains arise specifically from relational SQL.
4.5 Results
On RAGTruth, TeQHallu combines SQL-grounded verification with neural judgment to improve hallucination detection while providing inspectable reasoning traces. It outperforms unsupervised baselines and approaches supervised systems, with gains attributed especially to improved precision.
- TeQHallu significantly outperforms unsupervised RAGTruth baselines and nearly matches heavily supervised state-of-the-art systems.It also surpasses the supervisedly fine-tuned Llama-2-13B model.
- Ablation study: Multi-step claim decomposition and key-value extraction improve over direct prediction, but TeQHallu achieves superior overall macro F1 across all three tasks.Its advantage is driven by increased precision from relational verification.
- Qualitative analysis: TeQHallu provides a traceable reasoning chain that links hallucination labels to explicit SQL-grounded evidence.The pipeline highlights reference facts, response hallucinations, and the final grounded explanation separately.
- The pipeline remains resilient when database parsing is incomplete, with 14 of 22 incomplete cases still producing correct predictions through the reference-text fallback.The audit covered 150 documents, with 50 sampled per task.
- Ablation study: A soft LLM-driven alignment of symbolic entity identifiers achieves a better balance than either neural text comparison or strict relational intersection.The configuration preserves symbolic precision while retaining contextual reasoning.
- Open-model results: SQL grounding consistently corrects model-dependent precision–recall imbalances across open and closed backbones.The approach yields stable gains even when text-to-SQL capability is relatively weak, although stronger text-to-SQL backbones perform better overall.
4.9 DiaHalu TOD Results
On the task-oriented DiaHalu benchmark, TeQHallu achieves the strongest reported hallucination-detection performance across evaluated models and remains effective when databases are built from noisy dialogues. Its gains are strongest for non-factual errors and overreliance, while holistic cross-dialogue databases can become overly strict.
- TeQHallu consistently achieves the highest F1-scores across evaluated DiaHalu task-oriented models.It outperforms established baselines and prompt-engineering variants.
- TeQHallu maintains strong performance when constructing databases directly from noisy conversational data.This extends the pipeline beyond clean reference passages to raw dialogues.
- Building one dataset-wide MultiWOZ database degrades Gemini-2.5-Flash performance because cross-dialogue constraints make verification overly strict.The alternative is contrasted with incremental per-dialogue database construction.
- TeQHallu substantially improves Non-factual and Overreliance detection, while Incoherence remains stable and Irrelevance decreases slightly.Despite the Irrelevance trade-off, the pipeline obtains the highest overall micro-F1.
- Computational cost: Supervised baselines require substantial offline task-specific training, whereas TeQHALLU shifts the resource burden toward inference-time prompting.The method does not require dense hundred-billion-parameter models or GPU-cluster fine-tuning.
- Computational cost: TeQHALLU requires five sequential prompting steps and approximately 4.5k–9.7k input tokens per sample, compared with one API call for direct prediction.The pipeline also generates approximately 600 output tokens per sample.
5 Discussion and Future Work
TeQHallu combines structured SQL grounding with hallucination detection, achieving competitive benchmark performance while exposing traceable reasoning. The authors identify computational overhead as a trade-off and propose broader structured sources, iterative refinement, and human evaluation as future directions.
- TeQHallu achieves competitive F1-scores against zero-shot baselines and fine-tuned models on RAGTruth and DiaHalu while providing traceable SQL reasoning traces.
- SQL entity IDs alone are too strict for final hallucination prediction, despite creating tighter symbolic coupling.
- The multi-stage text-to-SQL process adds computational overhead and latency, shifting reasoning from fuzzy generation to precise database querying.
- Future work will combine SQL grounding with knowledge graphs, iterative database refinement, and human-in-the-loop evaluation for high-stakes applications.
6 Conclusion
The paper presents TeQHallu as a neurosymbolic framework that grounds responses in structured SQL databases. It reports state-of-the-art-level benchmark performance alongside transparent reasoning traces and identifies broader structured sources and iterative refinement as future directions.
- TeQHallu grounds language-model responses in structured SQL databases to produce verifiable evidence for hallucination detection.
- The authors report that TeQHallu matches or exceeds state-of-the-art performance on the RAGTruth and DiaHalu TOD benchmarks.
- The framework provides transparent reasoning traces and human-readable SQL explanations that can reveal subtle factual inconsistencies.
- Future work will expand structured sources and refine databases iteratively for high-stakes, knowledge-intensive applications.
Limitations
The pipeline’s effectiveness depends on reference quality and accurate SQL generation, while database construction adds overhead and may lose information. Large-scale human evaluation is also still absent.
- Effectiveness depends on the quality and completeness of reference documents, because noisy or incomplete material may cause missed hallucinations or incorrect explanations.
- SQL database construction and querying introduce computational overhead that may limit real-time or resource-constrained deployment.
- Translating unstructured data into SQL can cause information loss, as shown by manual analysis.
- The pipeline relies on accurate SQL generation, while SQL errors can propagate to final predictions and performance can vary with prompt wording.
- Large-scale human evaluations have not yet validated how end users perceive or use the framework’s transparency benefits.
Ethical Considerations
TeQHallu’s reliance on reference documents and transparent traces creates ethical risks involving inherited bias, automation bias, and possible exposure of sensitive information. The authors therefore emphasize human oversight and data governance.
- Biased, harmful, or non-representative reference documents can propagate their biases into the SQL database and hallucination checks.
- Automated verification may create false confidence and automation bias in critical domains such as healthcare or law.
- Transparent reasoning traces could help malicious actors reverse-engineer sensitive reference data or bypass structured verification.
- Continuous human oversight and robust data governance are required to keep automated fact-checking beneficial for digital safety.
A Full TeQHallu Prompt
The TeQHallu prompt builds and consults a database in stages before making a hallucination decision. It combines SQL-based verification with an initial prediction and reference-grounded evaluation.
- Database construction: Step 1 generates PRAGMA table_info queries to inspect relevant database tables before processing the reference.The prompt uses the current database contents to identify existing tables, columns, and stored information.
- Database construction: Steps 2 and 3 query existing database information and insert missing reference information without duplicating data.The prompt permits creating tables when necessary before adding missing information.
- Response verification: Step 4 generates SELECT queries to retrieve database information needed to fact-check the sampled response.The queries are conditioned on the response and retrieve relevant evidence for validation.
- Final decision: Step 5 evaluates hallucinations using query results alongside an initial prediction based only on the reference and response.The initial prediction is a preliminary judgment that does not use database-retrieved information.
- Hallucination criteria: Hallucinations include evident or subtle conflicts and evident or subtle introductions of unsupported information.The prompt also instructs the evaluator to tolerate harmless wording differences and flag only clearly unsupported or contradictory claims.
- Final decision: The final decision independently checks the response against both the original reference and database-retrieved information.The prompt retains, updates, or overrides the initial prediction after the full evaluation rather than relying on it alone.