Source-linked AI summary
Large Language Models for Quantum Code Translation: An Exploratory Study
Xiyuan Li, Yuechen Li, Jianjun Zhao
TL;DR
Quantum software development faces unfamiliar quantum semantics and heterogeneous SDK interfaces that complicate cross-SDK translation. The paper proposes a quantum-aware LLM workflow and evaluates Qiskit translations to PennyLane and Q#. Zero-shot Pass@1 reaches 72.33% on PennyLane and 71.63% on Q#, and one-shot prompting improves Pass@1.
Problem
Quantum-mechanical semantics and heterogeneous SDK interfaces and execution models complicate quantum software development and cross-SDK translation.
Source: §1 · p. 1Method
The paper develops a quantum-aware LLM translation workflow with prompts tailored to quantum semantics, SDK constraints, runtime requirements, and task specifications.
Source: §1 · p. 2, §3.1 · p. 4Results
Zero-shot Pass@1 reaches 72.33% on PennyLane and 71.63% on Q#, while one-shot prompting improves Pass@1.
Source: §1 · p. 2Takeaways & Limitations
The preliminary findings suggest potential for LLM-based quantum code translation while identifying key barriers for future research.
Source: §1 · p. 2
Abstract
from arXiv · showhide
Quantum software development is challenged by quantum-mechanical semantics and fragmented software development kits (SDKs), which complicate cross-SDK code translation and execution. In this paper, we propose an LLM-based approach for translating executable programs between high-level quantum SDKs. We develop a quantum-aware translation workflow whose prompt design and validation reflect quantum semantics and accommodate the version evolution of quantum ecosystems. We conduct a preliminary evaluation of 4 LLM configurations on 43 tasks, translating Qiskit programs to PennyLane and Q#. LLMs achieve a zero-shot Pass@1 of up to 72.33%, while one-shot prompting further improves the translation effectiveness. These results suggest potential for quantum code translation by LLMs.
1 Introduction
Quantum software development is complicated by unfamiliar quantum semantics and heterogeneous SDK interfaces and execution models. The paper proposes a quantum-aware LLM translation workflow and preliminarily evaluates it across Qiskit-to-PennyLane and Q# tasks.
1 Introduction
- Quantum-mechanical abstractions and heterogeneous SDK interfaces and backends complicate quantum software development and maintenance.The cited challenges include superposition, entanglement, probabilistic measurement, differing APIs, and runtime constraints.Source: §1 · p. 1
- The workflow combines quantum-aware prompts with validation and evaluation to translate executable programs between high-level SDKs.Prompts capture quantum semantics, SDK syntax, version constraints, runtime requirements, and task specifications.Source: §1 · p. 2
- Zero-shot Pass@1 reaches 72.33% on PennyLane and 71.63% on Q#, while one-shot prompting can substantially improve Pass@1.The evaluation covers four LLM configurations and 43 benchmark tasks.Source: §1 · p. 2
- Missing source artifacts and SDK compilation errors account for 39.2% and 34.9% of zero-shot failures, respectively.These are the two dominant failure categories reported in the preliminary evaluation.Source: §1 · p. 2
2 Background and Related Work
Related work addresses classical code translation, quantum SDK-version migration, hardware-oriented transpilation, and quantum code generation. This paper distinguishes its task as translating developer-facing programs across heterogeneous high-level SDKs while preserving functionality and general structure.
2 Background and Related Work
- Classical code translation methods motivate quantum translation, where preserving behavior also requires accounting for relative phase, bit ordering, control and target roles, and probabilistic measurement outcomes.The cited classical approaches include execution feedback, intermediate translation paths, and repository decomposition with validation.Source: §2 · p. 2
- Quantum SDK-version migration updates APIs within an SDK, whereas this paper's translation reconciles programming abstractions and execution conventions across SDKs or languages.The distinction is between incremental updates in one SDK and transfer across heterogeneous environments.Source: §2 · p. 2
- Quantum transpilation converts circuits for target hardware or simulators, while this paper targets high-level SDKs used by developers designing quantum code.Prior cross-SDK transpilation examples include LLM use and gate decomposition.Source: §2 · pp. 2–3
- Quantum code generation creates programs from models, prompts, or agents, whereas translation converts a given source program to another SDK or language while preserving functionality and general structure.The cited generation work includes multi-platform generation for one task.Source: §2 · p. 3
3 Translation Workflow
The workflow generates target programs from source programs and task specifications, then evaluates them against validated tests and a runtime manifest. Its prompts separate general translation instructions from task-specific inputs, with an optional verified example for one-shot prompting.
3.1 Workflow Overview
- Candidate generation uses a source program and task specification, while a parallel preparation path validates reference implementations and packages tests with a runtime manifest for evaluation.Generated programs are executed against that bundle; outcomes feed correctness analysis and aggregate reporting.Source: §3.1 · p. 4
- Correctness analysis distinguishes failures to deliver or execute a candidate from assertion rejections, and relates assertion failures to tested properties.Aggregate analysis summarizes outcomes across tasks, models, and target SDKs.Source: §3.1 · p. 4
3.2 Prompt Construction
- Each prompt separates general translation rules in <Model_Instructions> from the concrete task in <Task_Input>.The prompt figure marks literal fields in black, placeholders in blue, comments in italics, and the two core components in orange.Source: §3.2 · p. 4
- Target-specific settings supply file extensions and runtime guidance for Q# or PennyLane to accommodate their distinct loading and execution interfaces.The configuration also records validated source and target implementations.Source: Settings · p. 4
- One-shot prompting adds one verified source–target example; zero-shot omits it, and the example is intentionally simple to limit overlap with the evaluated task.The paired programs illustrate translation conventions rather than reproducing a full evaluated task.Source: One-shot Example · p. 5
- Model instructions include SDK and version constraints, runtime guidance, and output format, with rules for faithful execution and assertion preservation.An additional rule applies an independent statevector check to eligible circuits.Source: Model Instructions · pp. 5–6
- Task input specifies the task and source entry point, target SDK, description, source Qiskit program, and original assertions.Together these define the translation and its behavioral constraints.Source: Task Input · p. 6
4 Empirical Study
The study evaluates zero-shot and one-shot quantum-code translation across model configurations, target SDKs, and failure categories, finding substantial variation and improved results with one-shot prompts. Its pilot design uses a 43-task benchmark and native evaluation environments.
4.1 Research Questions
- The study asks how zero-shot quality varies across models and targets, how prompting affects correctness and failures, and which failure categories occur.These are the three research questions guiding the empirical study.Source: §4.1 · p. 6
4.2 Experimental Design
- The evaluation uses 43 of QuanBench’s 44 tasks with companion tests, translating Qiskit 1.4.6 programs to PennyLane 0.45.1 and Q# with QDK 1.32.0.One task was excluded because its canonical reference implementation could not reliably pass the assertions.Source: §4.2 · pp. 6–7
- Four model configurations are evaluated, with one-shot prompting tested for Pro and GLM using verified, simple examples based on a Qolumbina task.The examples are intended to reduce data-leakage risk and avoid reproducing the full functionality of evaluated tasks.Source: §4.2 · pp. 6–7
4.3 Empirical Results
- Zero-shot Pass@1 ranges from 16.28% to 72.33%, with Flash leading on PennyLane and GPT on Q#, so results should be compared by model and target.Pass@3 exceeds Pass@1 for every model–target pair.Source: Table 1 · p. 7, §4.3 · p. 7
4.2 Experimental Design
- Pass@k measures task-macro correctness, requiring a task to pass all QuanBench assertions; the study evaluates k=1 and k=3 with 10 sample slots per combination.Average runtime and output-token count per API response are also recorded as cost measures.Source: §4.2 · p. 7
4.3 Empirical Results
- One-shot prompts improve Pass@1 and Pass@3 for Pro and GLM on both targets; GLM gains 10.93–18.37 percentage points in Pass@1, versus Pro’s 7.67–8.14.GLM overtakes Pro on PennyLane under both metrics; the proposed role of paired examples remains unverified.Source: Figure 3 · p. 8, §4.3 · p. 8
- Missing source and SDK compilation dominate zero-shot failures, while one-shot prompting reduces failed samples from 1,101 to 907 for Pro and GLM across both targets.With one-shot prompting, SDK compilation becomes the largest category at 38.5%; under zero-shot prompting, missing source dominates at 44.7%.Source: §4.3 · pp. 8–9, Table 2 · p. 9
5 Conclusion and Future Work
The paper presents a pilot evaluation of LLM-based quantum code translation across two target SDKs and 43 tasks, with translation quality varying across configurations. It identifies future work in benchmark construction, semantic and structural checks, and broader evaluation.
5 Conclusion and Future Work
- The study is an initial proposal and pilot evaluation of LLM-based quantum code translation, covering four configurations, two targets, 43 tasks, and zero-shot and one-shot prompts.Source: §5 · p. 9
- Across four LLM configurations, zero-shot Pass@1 ranges from 16.28% to 72.33%, and missing source is the most common failure category, followed by SDK compilation.The evaluation translates Qiskit programs to PennyLane and Q# across 43 tasks and compares zero-shot with one-shot prompting.Source: §5 · p. 9
- Future work will build a paired, executable cross-SDK benchmark, develop semantic-preservation and structural-consistency checks, and study prompting strategies, execution-guided repair, quality, and cost.Proposed checks include relative phase, bit ordering, and probabilistic outputs.Source: §5 · p. 9