PaperlayerView exact revision on arXiv

Source-linked AI summary

AquiLLM: Evaluating Faithfulness in Open-Weight RAG-LLM Systems for Scientific Research

Bernie Boscoe, Srinath Saikrishnan, Vikram Seenivasan, Jack Stark, Andrew Lizarraga, Morgan Himes, Jonathan Soriano, PJ Allen, Tuan Do

arXiv:2609.16519v1cs.AI

TL;DR

Scientific research groups need natural-language access to fragmented knowledge while retaining privacy and control over unpublished data. This paper presents a domain-expert evaluation of AquiLLM, finding strong faithfulness for retrieval-oriented tasks but substantially greater difficulty with cross-document synthesis and ambiguity.

  • Problem

    Research groups need natural-language interfaces for fragmented scientific knowledge while addressing privacy, reproducibility, and cost concerns surrounding commercial AI platforms.

  • Method

    The paper presents AquiLLM, an open-weight offline RAG-LLM platform, and evaluates its faithfulness through domain-expert ratings of astronomy queries spanning retrieval complexity and reasoning.

  • Results

    AquiLLM performed well on retrieval-oriented tasks, but faithfulness became substantially more difficult when responses required cross-document synthesis or ambiguity resolution.

  • Takeaways & Limitations

    Expert-centered evaluation is important for understanding scientific RAG-LLM limitations beyond standard benchmark performance.

  • Takeaways & Limitations

    The evaluation covered five astronomers from one research group, one scientific domain, and one open-weight model configuration, without controlled comparison to another RAG system or component ablations.

Abstract

from arXiv · show

Scientific research increasingly relies on large, heterogeneous data sources, motivating interest in retrieval-augmented generation (RAG) systems that provide natural language access to scientific knowledge and research workflows. Researchers are exploring the viability of these systems as natural language interfaces for document search and for generating analysis code and pipeline components. At the same time, concerns about data privacy and control over research infrastructure have motivated interest in open-weight models and open-source deployments hosted within research institutions. In astronomy, this development follows a long history of computational infrastructure development, from archival databases and SQL-based systems to LLM-assisted research tools. This paper presents a domain-expert evaluation of faithfulness for AquiLLM, an open-weight, offline RAG-LLM platform designed to support scientific research groups in the use and preservation of tacit and formal knowledge. We define faithfulness as the extent to which generated responses remain grounded in retrieved scientific context without unsupported claims or omissions. We report results from an astronomy case study evaluating AquiLLM across retrieval and scientific analysis tasks. AquiLLM performs most reliably on explicit retrieval-oriented questions grounded in the RAG collection, while faithfulness degrades for queries requiring synthesis or ambiguity resolution. These results highlight both the promise and limitations of open-weight RAG-LLM systems for scientific research and demonstrate the importance of domain-expert evaluation beyond standard benchmark leaderboards.

I. INTRODUCTION

Scientific research increasingly needs natural-language access to fragmented formal and tacit knowledge, but scientific RAG systems must remain faithful to retrieved sources. AquiLLM addresses this need as a local, open-weight platform evaluated by astronomy domain experts across varied query types.

  • Fragmented scientific data, code, and tacit knowledge create substantial cognitive and technical overhead for researchers.
  • Natural-language RAG interfaces can reduce reliance on specialized query languages, while privacy, reproducibility, and cost concerns motivate locally controlled deployments.
  • AquiLLM is an open-source, open-weight platform for locally preserving and querying formal and tacit research-group knowledge.
  • Faithfulness is defined as grounding generated responses in retrieved scientific context without fabrication or unsupported inference, a property not guaranteed by general benchmarks.
  • The study evaluates AquiLLM through astronomers’ assessments of factual retrieval and comparative scientific analysis using their group’s own data and documentation.
  • The paper contributes a deployable modular system, a reusable five-category expert evaluation methodology, and an astronomy baseline identifying six recurring failure modes.

III. AQUILLM SYSTEM OVERVIEW

AquiLLM is a browser-based, open-source RAG-LLM tool for conversational interaction with shared scientific knowledge collections. This study’s astronomy deployment establishes an initial faithfulness baseline using formal scientific artifacts.

  • AquiLLM lets research groups query shared collections containing published and private scientific materials through a browser-based natural-language interface.
  • The system is intended to preserve tacit knowledge embedded in scientific workflows and collaboration while supporting information retrieval.
  • Openly available source code and deployment configuration support self-hosting, but persistent public deployments require substantial computational resources for local models and multimodal workloads.
  • The evaluation used an astronomy-specific deployment and focused on formal artifacts, including unpublished manuscripts and technical documents, before adding broader collaborative materials.

B. System Architecture

AquiLLM separates ingestion, retrieval, orchestration, and inference into modular components for local processing of heterogeneous scientific documents. Its hybrid retrieval and citation mechanisms are designed to ground generated responses in retrieved passages.

  • Documents are asynchronously processed, chunked, and indexed, while query-time orchestration combines retrieved chunks with prompts to ground responses in domain-specific collections.
  • The architecture separates generation, embedding, reranking, and OCR, enabling independent retrieval and multimodal workloads and adaptation across research domains.
  • C. Model Selection and Configuration: The astronomy deployment used locally hosted Qwen-based models, with separate models for conversational generation, embedding, reranking, and multimodal processing.
  • C. Model Selection and Configuration: Model selection balanced reasoning quality, retrieval performance, computational cost, and interactive feasibility within available H100 GPU resources.
  • Hybrid retrieval pools dense-vector and lexical candidates, expands the pool beyond requested top-k, and reranks candidates before generation.
  • Citation enforcement restricts references to retrieved chunks and provides clickable source passages for direct claim verification.

IV. STUDY DESIGN AND METHODS

The study evaluates AquiLLM on a curated astronomy collection using repeated question submissions and domain-expert ratings. The design combines retrieval and comparative-analysis tasks within an offline, privacy-sensitive workflow.

  • Five astronomy and machine-learning researchers evaluated AquiLLM using a dedicated collection of 31 scientific documents and technical references.The collection covered survey, instrumentation, dataset, publication, and machine-learning materials.
  • Tasks combined direct retrieval with interpretation and comparative analysis involving redshift prediction, emission-line reconstruction, and machine-learning metrics.
  • Researchers selected ten representative questions from a broader pool of domain-specific questions and draft reference answers grounded in their own work.
  • C. Evaluation Procedure: Each participant submitted each question up to three times, producing up to 150 generation-and-rating instances and 141 rated responses after incomplete submissions were removed.
  • C. Evaluation Procedure: Responses were rated on a five-point scale covering factual accuracy, grounding, preservation of scientific context, and avoidance of unsupported claims, with written qualitative feedback.
  • The system had no external internet access, reflecting privacy-sensitive settings where web access may be undesirable or prohibited.

D. Query Design

The evaluation used five query categories spanning retrieval complexity, synthesis, and domain reasoning, with faithfulness judged against reference answers and hallucination avoidance. Across 141 responses, ratings were generally positive, but complex synthesis tasks were less stable.

  • Query categories: 141 annotated responses covered factual retrieval, metadata lookup, onboarding and conceptual explanation, scientific interpretation, and comparative scientific analysis.The categories were designed to span varying levels of retrieval complexity, synthesis, and domain reasoning.
  • Evaluation criteria: Faithfulness measured whether responses remained grounded in scientific context without fabrication or unsupported inference.Participants evaluated responses against collaboratively established reference answers.
  • Evaluation criteria: Ratings emphasized consistency with reference answers and absence of hallucinated claims using a five-point faithfulness scale.Written feedback documented strengths, ambiguities, omissions, and inaccuracies behind ratings.
  • Overall result: Across 141 cleaned responses, the mean faithfulness rating was 3.80, with 68.6% rated 4–5 and 17.9% rated 1–2.The median was 4.0 and the standard deviation was 1.24.
  • Overall result: Performance was most reliable for single-document retrieval and less stable for synthesis and comparative analysis requiring multiple sources.Participants still described the system as useful and scientifically helpful, while identifying omissions and unsupported claims in complex settings.

A. Performance Across Query Types

Faithfulness varied strongly by query type, with the clearest contrast between factual retrieval and comparative scientific analysis. Interpretation of several category means is limited because three categories each represented only one underlying question, and rating consistency cannot separate response variation from evaluator variation.

  • A. Performance Across Query Types: Factual Retrieval averaged 4.17 across 4 questions and 60 ratings, versus 2.97 for Comparative Scientific Analysis across 3 questions and 39 ratings.This was the strongest query-type contrast in the evaluation and indicates lower faithfulness for cross-source synthesis.
  • A. Performance Across Query Types: The remaining three categories each represented one underlying question, so their means are single-question results rather than generalizable category effects.Scientific Interpretation’s 4.73 mean, for example, came from one emission-line reconstruction question.
  • B. Rating Consistency Across Independent Attempts: The rating spread combines variation in AquiLLM’s independent outputs with differences in evaluators’ strictness because no two raters scored the same response text.Consequently, the study reports descriptive consistency rather than formal inter-rater reliability.
  • B. Rating Consistency Across Independent Attempts: Mean within-question rating deviation was 0.90, with 77.8% of ratings falling within one point of each question’s mean.Across the ten questions, the range of within-question standard deviations was 0.35–1.46.
  • B. Rating Consistency Across Independent Attempts: Consistency was lowest for two Comparative Scientific Analysis questions, where only 42.9% and 46.2% of ratings fell within one point of the mean.The pattern co-occurred with the category’s lowest mean faithfulness, but the data cannot establish whether response or rater variation caused it.

1) Hallucinated Scientific Details:

Low-faithfulness responses included unsupported technical details, inference beyond fragmented evidence, retrieval misalignment, scope drift, and overconfident explanations. These failures often appeared scientifically plausible while diverging from the retrieved context.

  • 2) Hallucinated Scientific Details:: One redshift-model comparison reported a 2.4x bias gap when the source-supported value was 1.2x.The unsupported numeric claim sounded scientifically plausible despite doubling the reported gap.
  • 2) Unsupported Synthesis:: With fragmented evidence, the model generated an unsupported profile-fitting explanation after initially saying it could not locate the c_model_mag method.This illustrates inference without sufficient grounding across documents.
  • 2) Unsupported Synthesis:: Some responses misaligned retrieval by citing unspecified tables or documents, or by claiming information was absent when it existed in the collection.These errors reflect incomplete anchoring to the requested source context.
  • 2) Unsupported Synthesis:: Nearly correct answers sometimes shifted values, omitted qualifiers, or drifted in scope while preserving the general scientific trend.One response reported the wrong scatter value and omitted conditional details.
  • 2) Unsupported Synthesis:: Conceptual explanations sometimes stated weakly supported interpretations with excessive confidence.Evaluators observed this in discussions of z-band naming conventions.

6) Failure Under Ambiguity:

AquiLLM was most useful for direct retrieval, but faithfulness weakened when prompts were ambiguous or required synthesis across sources. Failures reflected retrieval, orchestration, reasoning, and implementation interactions rather than random error alone.

  • Credible-sounding overreach was the dominant risk, especially in synthesis and comparative analysis across multiple sources.These tasks received lower average faithfulness ratings than direct retrieval-oriented questions.
  • Direct retrieval of numeric values, survey metadata, or single-document information was the system’s most reliable and useful behavior.Participants described it as effective for quickly locating scientific details and navigating technical materials.
  • Trust decreased when questions required reasoning across multiple papers or fragmented scientific context, producing plausible but partially grounded responses.
  • Broad or underspecified prompts often retrieved excessive or weakly related information from multiple surveys instead of the intended dataset.
  • Repeated queries sometimes produced inconsistent retrieval, conflicting answers, or responses without visible tool invocation after earlier successful retrieval.These patterns raised questions about retrieval persistence, memory effects, and tool-use stability.
  • Improving faithfulness requires stronger retrieval quality, tool-use consistency, ambiguity handling, and system-level robustness, not foundation-model changes alone.

VI. DISCUSSION

The discussion frames scientific faithfulness as grounded, traceable, and uncertainty-aware behavior rather than conversational fluency alone. AquiLLM shows promise for retrieval but unresolved constraints in synthesis, scalability, and cross-domain generalization.

  • A. What Faithfulness Means in Scientific Contexts: Scientific trust depends on grounding responses in source material and avoiding unsupported inference that could affect interpretation or reproducibility.
  • A. What Faithfulness Means in Scientific Contexts: Traceable responses consistent with retrieved documents were trusted even when incomplete, whereas ungrounded synthesis reduced confidence in multi-source comparisons.
  • A. What Faithfulness Means in Scientific Contexts: Scientific RAG-LLMs need stronger transparent retrieval, source attribution, ambiguity handling, and calibrated uncertainty across the interacting system components.
  • B. Implications for Scientific Cyberinfrastructure: RAG-LLMs extend astronomy’s searchable-database and SQL-interface tradition by enabling conversational access to publications, datasets, documentation, and collaborative knowledge.
  • B. Implications for Scientific Cyberinfrastructure: AquiLLM performed relatively well on retrieval-oriented tasks, but synthesis-heavy and comparative questions exposed grounding, retrieval-consistency, and orchestration limitations.
  • C. Scalability and Generalization Potential: The study used a curated 31-document collection and did not measure how retrieval quality, latency, or faithfulness change at larger archive or repository scales.Scaling behavior and the possibility that noisier collections compound comparative-analysis failures remain architectural goals rather than measured outcomes.
  • C. Scalability and Generalization Potential: Results should not be generalized numerically beyond this astronomy group because faithfulness thresholds, failure modes, and evaluation panels may be domain-specific.

D. Open-Weight LLMs for Research Communities

Open-weight systems offer research communities local control, privacy, and preservation, but this study’s evaluation evidence remains limited in scope and comparability. Practical deployment therefore requires expert-centered, query-type-specific evaluation and attention to retrieval orchestration.

  • D. Open-Weight LLMs for Research Communities: Local open-weight deployment gives research groups greater control over data, infrastructure, reproducibility, and long-term preservation than externally managed platforms.
  • E. Lessons for RSE Teams: Separate evaluation of retrieval and synthesis queries is warranted because aggregate faithfulness scores can mask large differences between query types.
  • E. Lessons for RSE Teams: Short free-text justifications alongside numeric ratings recovered most of the study’s diagnostic value without requiring a formalized rubric or predefined scale anchors.
  • E. Lessons for RSE Teams: Failures often traced to retrieval and orchestration behavior, so evaluation and debugging effort should target those layers as well as model selection.
  • E. Lessons for RSE Teams: Domain experts identified subtle inaccuracies likely to remain invisible in benchmark evaluations, making expert-centered assessment central to scientific faithfulness research.
  • Limitations: The study involved five astronomers from one research group, one scientific domain, and one open-weight model configuration, limiting broader generalization.
  • Limitations: No controlled comparison with another RAG system or component-level ablation was conducted, partly because the corpus contained unpublished and internal materials.
  • Limitations: The rating scale lacked verbal anchors, and self-rated responses prevented computation of formal inter-rater reliability statistics.

Data and Code Availability

The project makes its source code and deployment configuration available while withholding the evaluation dataset to preserve its value as a held-out benchmark. Follow-up work prioritizes reevaluation, provenance, ambiguity handling, orchestration, and scalability.

  • Data and Code Availability: AquiLLM’s source code and deployment configuration are publicly available, but the evaluation dataset is not publicly released at this time.
  • Data and Code Availability: The evaluation questions and reference answers remain held out because early publication could contaminate the benchmark through future model training.
  • Data and Code Availability: The dataset is available directly from the authors for reproducibility under a condition prohibiting publication or direct or indirect model training use.
  • Future Work: The near-term priority is rerunning the faithfulness study on an updated system using the same query categories and evaluation methodology.
  • Future Work: Planned development targets provenance tracing, query interpretation, ambiguity handling, tool orchestration, multimodal ingestion, and retrieval scalability.
  • Conclusion: Retrieval-oriented tasks showed promising faithfulness and usability in the astronomy evaluation environment, while synthesis across documents or ambiguity made faithfulness substantially harder to maintain.

APPENDIX A DEPLOYMENT COST AND HARDWARE REQUIREMENTS

AquiLLM’s astronomy deployment used a single H100 GPU, with costs varying according to provider pricing and operating schedule. Indicative estimates place a small group’s specialist-cloud deployment within a several-hundred-to-few-thousand-dollar monthly budget.

  • A single H100 GPU hosted the deployment’s multicomponent inference stack.The system ran separate processes for the primary chat model, embedding, reranking, OCR, and transcription on one GPU.
  • Table IV compares continuous 24/7 operation with business-hours-only operation for a single-H100 deployment.Business-hours operation reduces cost but sacrifices availability outside the operating window.
  • Several hundred to a few thousand dollars per month is the indicative specialist-cloud budget for a small-group deployment.The estimate is approximate and depends on group size, query volume, and availability requirements.
  • $1.99–3.49 per GPU-hour was the cited on-demand H100 rate for a specialist provider.The rates were checked against provider pricing pages in September 2026 and are subject to fluctuation.
Loading 2609.16519v1…