Source-linked AI summary
SPARQL-LLM in the TEXT2SPARQL 2026 Challenge
Panayiotis Smeros, Vincent Emonet, Ruijie Wang, Ana-Claudia Sima, Tarcisio Mendes de Farias
TL;DR
Querying Knowledge Graphs with SPARQL requires fluency in syntax, endpoints, and schemas, motivating natural-language question-to-query systems. SPARQL-LLM uses endpoint metadata to build context and ranked first overall among eight challenge approaches, with up to 59% higher accuracy than the second-best approach.
Problem
Querying Knowledge Graphs with SPARQL is difficult because it requires fluency in query syntax, endpoint specifications, schema details, and cross-dataset relationships.
Source: §1 · p. 1Method
SPARQL-LLM uses lightweight endpoint metadata to construct optimized context for translating multilingual natural-language questions into SPARQL queries.
Source: §4 · p. 3Results
SPARQL-LLM ranked first overall among eight competing approaches across all categories, with up to 59% higher accuracy than the second-best approach.
Source: §1 · p. 1Takeaways & Limitations
SPARQL-LLM is open source and deployed over decentralized bioinformatics Knowledge Graphs.
Source: §1 · p. 1
Abstract
from arXiv · showhide
Querying distributed data represented as Knowledge Graphs (KGs) via SPARQL remains a difficult task, requiring fluency in query syntax, endpoint specifications, and schema details. Knowledge Graph Question Answering (KGQA) systems address this by automatically translating natural language into SPARQL, and recent advances in Large Language Models (LLMs) have made this translation increasingly feasible. In this paper, we describe SPARQL-LLM, our open-source KGQA system, in the context of the TEXT2SPARQL 2026 challenge. In this challenge, SPARQL-LLM ranks first overall among eight competing approaches across all datasets, achieving up to 59% higher accuracy than the second-best approach and supporting multilingual question answering in English, German, and Spanish. SPARQL-LLM is publicly released at https://github.com/sib-swiss/sparql-llm and deployed in production over decentralized bioinformatics KGs at https://www.expasy.org/chat.
1. Introduction
SPARQL-LLM addresses the difficulty of querying Knowledge Graphs with SPARQL by translating natural-language questions into queries. It ranked first overall in the 2026 challenge and supports English, German, and Spanish question answering.
1. Introduction
- SPARQL remains difficult because it requires fluency in query syntax, endpoint specifications, schema details, and cross-dataset relationships.KGQA systems address this by automatically translating natural language into SPARQL.Source: §1 · p. 1
- SPARQL-LLM ranked first overall among eight competing approaches across all categories, with up to 59% higher accuracy than the second-best approach.Source: §1 · p. 1
- SPARQL-LLM supports multilingual question answering in English, German, and Spanish.Source: §1 · p. 1
- SPARQL-LLM is open source and deployed over decentralized bioinformatics Knowledge Graphs.Source: §1 · p. 1
2. Related Work
Prior TEXT2SPARQL systems used agent-based exploration, fine-tuning, or retrieval-augmented context construction. SPARQL-LLM instead assembles context before query time, targeting fixed LLM interactions and real-time generation without an agent loop.
2. Related Work
- 2025 TEXT2SPARQL systems broadly used agent-based exploration, fine-tuning, or retrieval-augmented context construction.The retrieval-based approaches inject examples, data shapes, and ontology fragments into prompts.Source: §2 · p. 2
- Agentic approaches can adapt to arbitrary queries, but schema rediscovery can make them slow and costly, and complex queries can require many iterations or fail to converge.Source: §2 · p. 2
- SPARQL-LLM assembles context before query time, fixing the number of LLM interactions per question and avoiding an agent loop that can fail to converge.The authors report remarkable query correctness despite the real-time requirement.Source: §2 · p. 2
3. Experimental Evaluation
The evaluation covers DBpedia and an unknown corporate Knowledge Graph, with SPARQL-LLM using examples and endpoint schema context. It achieves the best results across subtasks at low total LLM-call cost.
3. Experimental Evaluation
- The evaluation section describes the framework, competing systems, and results of the 2026 TEXT2SPARQL challenge.Source: §3 · p. 2
3.1. Experimental Setup
- The 2026 challenge evaluation comprises DBpedia questions in English and Spanish and questions about an unknown private-corporation KG in English and German.Source: §3.1 · p. 2
3.2. SPARQL-LLM
- SPARQL-LLM receives examples of natural-language questions paired with SPARQL queries and schema information from the endpoints.Its LLM configuration is presented in Table 1; the architecture is described in [5].Source: §3.2 · p. 2
- For DBpedia, the examples come from LC-QuAD, QALD-9+, and the 2025 challenge test set.Source: §3.2 · p. 2
3.3. Results
- 59% higher F1 Score than the second-best system on DBpedia Spanish; SPARQL-LLM achieves the best result in all subtasks.Figure 1 shows SPARQL-LLM ranking first for both DBpedia and Corporate, with a significant margin over the second-best system.Source: Figure 1 · p. 3, §3.3 · p. 3
3.2. SPARQL-LLM
- For Corporate, the team generated question-query examples from the ontology, checked them against the endpoint, and discarded queries with no results.The ontology has 13 classes and 30 properties; examples also included the 2025 challenge test set.Source: §3.2 · p. 3
3.3. Results
- Less than $3 covered all SPARQL-LLM LLM calls for TEXT2SPARQL 2026.Source: §3.3 · p. 3
4. Conclusion
SPARQL-LLM is presented as the top-ranked system in the TEXT2SPARQL 2026 challenge. Its conclusion describes metadata-based context construction and reports higher F1 scores than ARUQULA across four dataset-language settings.
4. Conclusion
- SPARQL-LLM is presented as the top-ranked system in the TEXT2SPARQL 2026 challenge.The system uses lightweight SPARQL endpoint metadata to construct an optimized context and is reported to achieve the best accuracy at minimal operational cost.Source: §4 · p. 3
- The system constructs optimized context for multilingual question-to-SPARQL translation using lightweight metadata from SPARQL endpoints.The paper also describes SPARQL-LLM as open source.Source: §4 · p. 3
- The prompt for generating examples specifies an ontology of 13 classes and 30 properties.Source: §4 · p. 3
- 17% higher F1 Score on English DBpedia, 59% on Spanish DBpedia, 33% on English Corporate, and 30% on German Corporate, each versus ARUQULA.Source: Table 2 · p. 4, Table 3 · p. 4, Table 4 · p. 4
Declaration on Generative AI
The authors report using Claude for language and style edits, then reviewing and editing the content and retaining responsibility for it.
Declaration on Generative AI
- The authors used Claude for grammar, spelling, paraphrasing, and stylistic edits, then reviewed and edited the content and took full responsibility.Source: Declaration on Generative AI · p. 5