Verif.ai: Towards an Open-Source Scientific Generative Question-Answering System with Referenced and Verifiable Answers

I. INTRODUCTION

• Hallucinations can produce false or misleading information, creating a trust deficit that hinders scientists’ adoption of language models in accuracy-critical scientific workflows.The introduction notes that these models could accelerate research and information retrieval, but misinformation threatens their use in scientific work.Source: §I · p. 1
• Verif.ai is an open-source project intended to mitigate hallucination risk through scientific-paper retrieval, referenced answer generation, and verification of generated claims against their source papers or abstracts.Its retrieval approach uses semantic and lexical techniques, and answer generation uses a fine-tuned Mistral 7B model.Source: §I · p. 1

II. METHODOLOGY

• The system’s methodology includes a PubMed retrieval engine, a fine-tuned language model that generates answers from retrieved information, a verification engine, and a user interface for review and feedback.The interface also lets users contribute feedback to improve the project.Source: §II · pp. 1–2

A. Toolbox and Information Retrieval

• OpenSearch indexes PubMed articles lexically and stores document embeddings from the MSMARCO model for semantic search.MSMARCO was selected for its ability to handle asymmetric searches, such as queries and searched texts of different lengths.Source: A. Toolbox and Information Retrieval · p. 2
• Combining lexical and semantic search emphasizes direct matches while also finding semantically similar phrases and text that does not match exactly.Source: A. Toolbox and Information Retrieval · p. 2
• The system transforms a user question into a query, retrieves the most relevant documents, and passes them to the answer-generating language model.Source: A. Toolbox and Information Retrieval · p. 2

B. Question-answering with references

• Mistral 7B was instruction-fine-tuned using PubMedQA questions and answers generated by GPT-3.5 with relevant PubMed documents as context.The training dataset used 10,000 randomly selected PubMedQA questions, and the model was fine-tuned with QLoRA.Source: B. Question-answering with references · p. 2
• The training prompt instructed the model to support answers with research papers and cite the corresponding abstract number in square brackets.Source: B. Question-answering with references · p. 2
• At inference, the prompt directs Mistral to answer from the retrieved abstracts, reference claims by abstract ID, and use only the provided abstracts.The authors added this instruction after using the same input as training did not produce the expected results; generation used a 1,000-token limit and repetition penalty of 1.1.Source: B. Question-answering with references · p. 2

C. Verifying claims from the generated answer

• Verification assumes each generated statement has one or more references and compares the statement with its cited article using natural-language-inference models.The models are XLM-RoBERTa-large and DeBERTa, fine-tuned on SciFact.Source: C. Verifying claims from the generated… · p. 2
• The verification model assigns each claim one of three labels: Supports, Contradicts, or No Evidence.Source: C. Verifying claims from the generated… · p. 3
• The fine-tuned model is the primary method for flagging contradictions or unsupported claims, while showing users abstract sentences similar to a claim is planned as an additional trust aid.Source: C. Verifying claims from the generated… · p. 3

• Users can review answers, referenced documents, and flagged sentences containing potential hallucinations.Source: D. User feedback integration · p. 3
• Users are asked to critically evaluate answers and can change a natural-language-inference class or modify a generated answer.Source: D. User feedback integration · p. 3
• User modifications are recorded for future model fine-tuning.Source: D. User feedback integration · p. 3

III. PRELIMINARY EVALUATION

• At the time of the preliminary evaluation, the project was in its third month, and methodology improvements and web-application development were ongoing.Source: §III · p. 3

A. Information retrieval

• Hybrid search returned documents with exact query matches as well as paraphrases or synonyms, placing exact-match documents above semantic-only matches.Source: A. Information retrieval · p. 3
• Lexical search could perform better for exact term matches, while semantic search handled paraphrases and synonyms but often did not prioritize exact matches.Source: A. Information retrieval · p. 3
• The search evaluation was qualitative and used a small set of indexed PubMed articles.Source: A. Information retrieval · p. 3

• Fine-tuned Mistral 7B answers were comparable to those of the larger GPT-3.5 and GPT-4 models on the referenced question-answering task.Source: B. Answer generation · p. 3
• In a manual comparison of 50 questions, GPT-3.5, GPT-4, and Mistral showed no clear advantage over one another.Source: B. Answer generation · p. 3
• Answer quality, referenced abstracts, and answer length varied within and among the three models, though they usually cited the same relevant abstracts.Source: B. Answer generation · p. 3

• The SciFact evaluation used 10% of the data for validation and 10% for testing, with NO EVIDENCE, SUPPORT, and CONTRADICT classes distributed 36%:42%:22%.Source: C. Verification and hallucination… · p. 3
• The fine-tuned models surpassed reported label-prediction scores in, DeBERTa-large outperformed RoBERTa-large, and the models outperformed zero-shot GPT-4.Source: C. Verification and hallucination… · p. 3, Table I · p. 3
• The authors consider the fine-tuned verifier a starting point for hallucination detection, supported by qualitative analysis of generated claims and abstracts; they expect real-use claims to be easier than SciFact pairs.Source: C. Verification and hallucination… · p. 4
• The verifier struggles with claims that differ from abstracts by a few context-switching words or slightly different numerical values.Source: C. Verification and hallucination… · p. 4

IV. CONCLUSION

• Verif.ai combines hybrid OpenSearch retrieval, Mistral 7B answer generation, and a fine-tuned DeBERTa-large model for hallucination detection and answer verification.The paper presents this as current project progress and notes that challenges remain.Source: §IV · p. 4
• The project aims to use user feedback to guide active learning and iterative training for answer generation, verification, and hallucination detection.It also aims to follow developments in smaller, performant open-source language models with larger context sizes.Source: §IV · p. 4
• The verification model has limitations with numerical values and complex reasoning or inference over abstracts; the authors suggest a mixture of experts may be needed.They also aim to offer multiple verification methods, including explainable approaches and potentially sentence-similarity scores.Source: §IV · p. 4
• The current system is designed for biomedical use and answers from scientific articles indexed in PubMed, with possible extension to other document formats and search settings.The authors describe personal, organizational, or corporate generative search as potential future uses.Source: §IV · p. 4

V. AVAILABILITY

• Project code created so far is available on GitHub under the AGPLv3 license.The availability passage also identifies the code repository on GitHub.Source: §V · p. 4
• The fine-tuned qLoRA adapter for referenced question answering based on Mistral 7B and the verification models are available on HuggingFace.Source: §V · p. 4
• Further project information is available at the Verif.ai project website.Source: §V · p. 4