PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking the Quantum Leap: Auditing LLM Evaluation for Quantum Code Generation

Ethan Seefried, Romit Ghosh, Sharmin Afrose, Nicholas Stapleton, Vicente Layton Ortega, Travis Humble, Tirthankar Ghosal

arXiv:2610.28678v1cs.SEquant-ph

TL;DR

Quantum-code benchmarks differ in task composition, execution environment, output format, and scoring, while systematic evaluation-validity analysis for this field is lacking. The authors evaluate 15 models on five established suites and 10 models on 96-task QuantumBenchEval, reporting uncertainty, statistical separability, shared failures, output validity, and evaluator behavior. 94.1 pp: correcting one QBE evaluator changes two models’ pass@1 on unchanged outputs; no primary suite separates its top two models after multiplicity correction.

  • Problem

    Quantum-code benchmarks differ in task composition, execution environment, output format, and scoring, while systematic evaluation-validity analysis for this field is lacking.

    Source: §1 · pp. 1–2

  • Method

    The authors evaluate 15 models on five established suites and 10 models on 96-task QuantumBenchEval, reporting uncertainty, statistical separability, shared failures, output validity, and evaluator behavior.

    Source: §1 · p. 2

  • Results

    94.1 pp: correcting one QBE evaluator changes two models’ pass@1 on unchanged outputs; no primary suite separates its top two models after multiplicity correction.

    Source: §1 · p. 2, §5 · p. 9

  • Takeaways & Limitations

    Leaderboard comparisons are most trustworthy within a single configuration and evaluator revision, and differences smaller than documented evaluator effects should not be read as capability differences.

    Source: §5 · p. 9

    QCoder comparisons across earlier and later runs combine model capability with different output budgets and judge versions, and the groups were not re-judged with the other judge.

    Source: §4.6 · p. 9

Abstract

from arXiv · show

Large language models (LLMs) can generate quantum programs, but reliable evaluation remains difficult because benchmarks differ in task composition, scoring, and failure handling. We evaluate 15 hosted frontier, open-weight, quantum-specialized, and retrieval-augmented models on five quantum code-generation suites (530 tasks), plus 10 models on our new QuantumBenchEval (QBE) benchmark (96 tasks across six topics). We report bootstrap confidence intervals, multiplicity-adjusted paired comparisons, and audits of output validity and evaluator behavior. QiskitHumanEval-Standard approaches saturation (98.7% pass@1), while its Hard variant retains headroom (63.8%). None of the five top-two comparisons across primary suites remains significant after multiple-comparison correction. Evaluator design substantially changes scores: removing checks for unspecified quantities raises GPT-6 Astra's QBE pass@1 from 0% to 94.1% (T1) and from 1.2% to 92.9% (T6), without changing model outputs. We identify 17 QuanBench-117 tasks with shared failures linked to benchmark-owned imports. Controlled probes also reveal unexecuted unit tests, leakage of canonical functions into candidate execution, and reference imports that block otherwise valid candidates. These defects can produce both false failures and false passes, so historical QuanBench results require revalidation under an isolated evaluator. Overall, leaderboard scores alone do not adequately characterize progress in quantum code generation. We release our harness, per-sample results, and audit artifacts and recommend validating evaluators, isolating candidate code, and reporting uncertainty and output validity.

1 INTRODUCTION

Quantum-code benchmarks differ in what they measure and how they score, leaving open both how well current models perform and how reliably evaluation protocols measure that ability. The paper evaluates models across established suites and a new benchmark, finding that evaluator behavior can materially alter scores while leading comparisons often lack corrected statistical significance.

1 INTRODUCTION

  • The study evaluates 15 models on five established suites totaling 530 tasks and 10 models on new 96-task QuantumBenchEval across six topics.The paper reports uncertainty, statistical separability, shared failures, output validity, and evaluator behavior rather than point estimates alone.Source: §1 · p. 2
  • QiskitHumanEval-Standard reaches 98.7% pass@1, while Hard retains headroom at 63.8%; none of the five primary-suite top-two comparisons remains significant after multiplicity correction.These results distinguish near-saturation on Standard from corrected statistical evidence for model separation across the primary suites.Source: §1 · p. 2
  • Removing checks for unspecified quantities raises GPT-6 Astra’s QBE pass@1 from 0% to 94.1% (T1) and from 1.2% to 92.9% (T6) on unchanged outputs; QuanBench audits also find defects causing false passes and failures.The audit identifies 17 QuanBench-117 tasks with shared failures linked to benchmark-owned imports, plus unexecuted tests, canonical-function leakage, and reference imports that can block valid candidates; historical results require revalidation under an isolated evaluator.Source: §1 · p. 2
  • The authors release a unified harness, per-sample outcomes, and audit artifacts through Quantum Arena for continued evaluation under a shared protocol.The release is intended to support evaluating new models and tasks after publication.Source: §1 · p. 2

2 RELATED WORK

Prior work has developed quantum-code-generation systems and benchmarks, while evaluation-validity research identifies weaknesses in tests, benchmark construction, and judge-based scoring. This paper applies those concerns to quantum code-generation benchmarks.

2.1 QUANTUM CODE GENERATION WITH LARGE LANGUAGE MODELS

  • Quantum-code-generation systems include Qiskit Code Assistant, QSpark, PennyCoder, and Agent-Q, using approaches such as specialized training and iterative refinement.Related dataset work includes QCircuitNet, KetGPT, and C4Q.Source: §2.1 · p. 2

2.2 BENCHMARKS FOR QUANTUM CODE GENERATION

  • Quantum benchmarks include Qiskit HumanEval, QCoder, QHackBench, PennyLang, and QuanBench, with different task types and evaluation approaches.Table 1 distinguishes execution-based datasets using unit tests from QCoder’s LLM-based evaluation, and notes that QBenchEval uses an LLM rubric judge for its 15-task VQE topic.Source: §2.2 · pp. 2–3, Table 1 · p. 3

2.3 EVALUATION VALIDITY

  • Prior evaluation-validity work documents sparse tests, benchmark-side false failures, contamination, judge biases, and the need for uncertainty estimates; quantum benchmark leaderboards had not audited their evaluators.The paper examines corresponding quantum-code-generation issues, including tests that do not execute, reference imports that block valid candidates, undisclosed scoring quantities, judge-protocol differences, and top-model gaps that fail multiplicity correction.Source: §2.3 · p. 3

3 METHODS

The study evaluates quantum-code models using shared prompting and native benchmark evaluators, covering five established suites and new QuantumBenchEval. Its protocol reports uncertainty and paired comparisons while documenting evaluator types, judge scoring, output validity, and other diagnostic limits.

3.1 BENCHMARKING FRAMEWORK OVERVIEW

  • The harness samples tasks, generates five completions per task without reranking or post-processing, and logs programs and outcomes scored by each suite’s native evaluator.Temperature and output budgets vary across providers, so cross-model gaps are reported under those configurations alongside output validity.Source: §3.1 · p. 3

3.2 DATASETS

  • The five established suites span short Qiskit programming tasks, harder compositional circuit design, diverse QuanBench tasks, and open-ended QCoder problems; QBE adds 96 tasks across six topics.QBE uses task-specific assertions for five topics, while T3 combines execution with a rubric; the setting also requires alignment with quantum-state behavior and circuit semantics.Source: §3.2 · pp. 3–4

3.3 MODELS

  • The 15-model roster spans hosted general-purpose, general-purpose open-weight, quantum-specialized, and retrieval-augmented systems.The open-weight group includes LLaMA-3.1-8B, Mistral-7B-Instruct-v0.3, Gemma-3-4B-IT, and Muse-Glimmer-30B; the specialized group includes three Qiskit models, alongside Quantum RAG.Source: §3.3 · p. 4

3.4 EVALUATION PROTOCOL

  • The protocol reports pass@k with task-bootstrap confidence intervals, paired task-level comparisons with Holm correction, and rank agreement; QCoder uses binary LLM-judge verdicts.QCoder’s Claude Fable 5.1 results are flagged because the judge shares a provider with the model. Diagnostics include valid-output rates and error classifications, but do not establish functional or quantum-semantic correctness.Source: §3.4 · pp. 4–5

4 EXPERIMENTS

The experiments compare models across five established quantum-code suites and separately evaluate QBE, emphasizing task-level uncertainty and paired comparisons. Results vary across benchmarks, and evaluator behavior materially affects scores; QuanBench outcomes remain unvalidated pending replay.

4.2 BENCHMARK SATURATION AND STATISTICAL SEPARABILITY

  • QiskitHumanEval Standard reaches 98.7% pass@1, while Hard reaches 63.8%; none of the five suites’ top-two gaps remains significant after correction.The reported comparison is a benchmark-level contrast, not a controlled estimate of task difficulty.Source: §4.1 · p. 5, §4.2 · pp. 5–6, Table 3 · p. 6

4.1 OVERALL PERFORMANCE

  • Performance ordering differs across suites: Claude Fable 5.1 leads QuanBench-44 at 60.0% pass@1, while it leads complete QCoder runs at 43.0%.QuanBench results are recorded evaluator outcomes requiring revalidation; QCoder uses a sequential protocol.Source: §4.1 · p. 5, Table 2 · p. 6

4.2 BENCHMARK SATURATION AND STATISTICAL SEPARABILITY

  • Pass@1 rank agreement is ρ=0.78–0.89 among most execution-scored suites but ρ=0.54 between Standard and QCoder; additional attempts benefit specialized models more.Strongest models gain at most about 6 pp from pass@1 to pass@5, whereas quantum-specialized models gain up to 38 pp.Source: §4.2 · p. 6

4.3 CATEGORY-LEVEL PERFORMANCE

  • On Hard, GPT-6 Astra’s pass@1 ranges from 20.0% on QUANTUM INFORMATION to 85.7% on STATE PREPARATION AND ANALYSIS; category results are descriptive.Category sizes range from 5 to 28 tasks, and Figure 1 cautions that small categories have wide intervals.Source: §4.3 · p. 7, Figure 1 · p. 7

4.4 QUANTUMBENCHEVAL: CHALLENGING TASKS AND EVALUATION SENSITIVITY

  • QBE comprises 96 new multi-step workflow tasks across six topics, with topic-dependent model results and no single ordering across topics.On T5, GPT-6 Astra scores 40.0%, Claude Fable 5.1 scores 82.4%, and Gemini 3.1 Pro Preview scores 83.5%; no model exceeds 32% on T4.Source: §4.4 · p. 7, Figure 2 · p. 8
  • Removing checks for unspecified quantities raises GPT-6 Astra from 0% to 94.1% on T1 and from 1.2% to 92.9% on T6, using unchanged outputs.The T1 and T6 corrections remove only those checks while retaining the other correctness checks.Source: §4.4 · pp. 7–8, Figure 2 · p. 8

4.5 QUANBENCH EVALUATOR AUDIT

  • Audit probes identify unexecuted unit tests, canonical-code leakage, and reference imports that can block valid candidates; historical QuanBench outcomes require replay under a versioned, isolated evaluator.For 17 QuanBench-117 tasks, the dominant recorded failure corresponds to imports in benchmark-owned reference or test code.Source: §4.5 · pp. 8–9

4.6 JUDGE-SCORED QCODER

  • QCoder’s cross-group comparisons are conditioned on different output budgets and judge versions; the within-group Fable–Astra comparison is its most controlled comparison.The judge changed between groups, and neither group was re-judged with the other judge.Source: §4.6 · p. 9

5 DISCUSSION AND CONCLUSION

The paper concludes that evaluator design and configuration shape measured quantum-code performance, and that leaderboard ordering alone is insufficient. It recommends validated, isolated evaluation with uncertainty and output validity reported.

5 DISCUSSION AND CONCLUSION

  • Correcting QBE checks for unspecified quantities changes pass@1 by 94.1 pp on unchanged outputs, while the audited QuanBench evaluator can produce false passes and false failures.The paper also reports that no primary suite separates its top two models after multiplicity correction and that QiskitHumanEval-Standard is near saturation.Source: §5 · p. 9
  • Comparisons are most trustworthy within a single configuration and evaluator revision; across configurations, differences smaller than documented evaluator effects should not be read as capability differences.QuanBench-117 scores reflect output budget, test execution, and possible reference-code leakage; QCoder scores also reflect judge version.Source: §5 · p. 9
  • The paper recommends validating reference solutions and test execution, isolating candidate code, disclosing checked quantities, reporting valid-output rates and uncertainty, and using independent sampling for pass@k.Source: §5 · p. 9
  • QuanBench outcomes await isolated-evaluator replay, QCoder uses a non-i.i.d. protocol and changing judges, settings and output budgets differ across models, and the roster reflects one point in time.QBE T1/T6 replay used a different platform, with 5 of 1,600 samples differing.Source: §5 · p. 9

A.1 PROMPTING SETUP

  • Tasks use a shared prompt template, with five samples per task, but decoding settings are not identical across providers.Source: §A.1 · p. 14

A.2 RUN PROVENANCE

  • QCoder output budgets differ by up to 16-fold, some models use different temperatures, and the judge changed between run periods without cross-judging.The paper therefore says QCoder scores from the two periods are not strictly comparable.Source: §A.2 · p. 14

A.3 DIAGNOSTIC DEFINITIONS

  • Observed task coverage is empirical pass@5, while oracle union coverage requires additional generation budget; error labels are diagnostic, not validated causes.Resource closeness does not affect correctness scoring and uses self-reported rather than measured inputs; token records exclude unlogged attempts and do not establish cost or latency.Source: §A.3 · p. 15

A.4 STATISTICAL PROCEDURES

The statistical procedures quantify task-level uncertainty and paired model differences, but top-two gaps are not significant on any primary suite after Holm correction. Rank agreement varies across suites.

A.4 STATISTICAL PROCEDURES

  • Pass@k intervals use 10,000 task-bootstrap replicates, resampling tasks while retaining all samples for each resampled task.Source: §A.4 · p. 15
  • Paired model comparisons use shared tasks, task-bootstrap intervals, paired permutation tests, and Holm correction within each suite–metric family.The paper reports 340 of 471 pass@1 comparisons significant at α=0.05 under those per-suite families.Source: §A.4 · pp. 15–16
  • None of the five top-two pass@1 gaps is significant after Holm correction: adjusted p-values are 0.90 for Standard, 0.23 for Hard, 1.00 for QuanBench-44, 1.00 for QuanBench-117, and 0.13 for QCoder.Source: §A.4 · p. 16
  • Rank intervals are widest on QuanBench-44, where the top five models’ intervals overlap; cross-suite rank agreement is high among execution-scored suites but lower between Standard and QCoder.Standard–QCoder has Spearman ρ=0.54 and Kendall τb=0.37; Standard–QuanBench-117 agreement should be read cautiously because both are near ceiling.Source: §A.4 · p. 16, Figure 3 · p. 16

B.1 EXAMPLE PROBLEMS

The examples show how task interfaces and evaluation contracts differ across suites, while later sections document benchmark composition, scoring, and audit qualifications. In particular, saved QuanBench checks do not establish that historical pass labels validated the intended behavior.

B.1 EXAMPLE PROBLEMS

  • QuanBench example checks should not be treated as evidence that historical pass labels validate the requested quantum behavior, because test methods may not run and other evaluator defects were found.The audit also found reference-code leakage and reference-import failures; historical outcomes require corrected, isolated replay.Source: §B.1 · p. 18

B.2 CATEGORY COMPOSITION

  • QuanBench-117 concentrates 40% of its tasks in Algorithm Implementation and 34% in State Preparation, unlike QiskitHumanEval’s broader Qiskit-ecosystem coverage.Category intervals for categories with five to nine tasks span 40–80 pp for mid-range models, so within-category differences should not be read as rankings.Source: §B.2 · p. 18, Figure 5 · p. 19

C.1 CONSTRUCTION AND PROVENANCE

  • The authors constructed all 96 QBE tasks for this work and report limitations including underspecified checks, internally inconsistent T2 tasks, permissive assertions, and small topics not calibrated against human solvers.The release includes task files, references, assertion suites, the T3 rubric and judge prompt, and saved outputs and scores under both evaluators.Source: §C.1 · pp. 18–19

C.2 TOPICS AND SCORING CONTRACTS

  • QBE has 96 tasks across six topics; five topics use candidate execution with task-specific assertions, while T3 uses a rubric.Source: §C.2 · p. 19

C.3 T3 RUBRIC AND JUDGED DENOMINATORS

  • T3 combines deterministic energy recomputation with LLM judging, but its scores are indicative rather than calibrated because most rubric points depend on one judge and agreement was not measured.The judge shares a provider with Claude Fable 5.1, and judged-sample denominators differ across models.Source: §C.3 · pp. 19, 21, Table 7 · p. 21

C.4 T2 AND T6 RESCORING

  • T2 and T6 rescoring removed checks for unspecified quantities, excluded three internally inconsistent T2 tasks, and replayed saved outputs without regeneration.All 32 reference solutions passed both evaluators, and no paired sample changed from pass to fail; 5 of 1,600 samples differed between historical and replayed original outcomes across platforms.Source: §C.1 · p. 19, §C.4 · p. 21, Table 6 · p. 21

C.5 WORKED EXAMPLES

The worked examples pair a passing reference with failures observed or anticipated in six QBE topics, while also exposing assertion gaps in three tasks. Rescoring comparisons require care because T2 v2 uses a different task set.

C.5 WORKED EXAMPLES

  • The examples contrast reference solutions with constructed failures; three expose assertion gaps, and T2 v2 rescoring covers a different set of 12 of 15 tasks.Part of the T2 v2 score change therefore reflects the task population.Source: §C.5 · pp. 21–22, Table 8 · p. 22
  • T1’s adjacent-pair cut totals 11 rather than the optimum 18, while its gate-count error escapes the shipped nonnegative-count check.The response counts named gates, missing four transpiled rzz gates; the assertion gap is two_qubit_gate_count >= 0.Source: §C.5 · p. 22
  • T2’s response ignores the QUBO penalty, selects four jobs instead of the feasible pair, and reports an unmeasured zero constraint-violation rate.Its energy is −18 versus −43 for {J2, J3}; both candidate-facing checks fail.Source: §C.5 · p. 22
  • T3’s fixed Hartree–Fock state misses 20.6 mHa of correlation energy yet claims chemical accuracy, and separate Pauli measurements double measurement cost.The response reports −1.11671 Ha against EFCI = −1.13727 Ha and measures six terms rather than three commuting groups.Source: §C.5 · p. 22
  • T4 differentiates near-zero susceptibility noise, returns hc = 0.10 outside the accepted range, and misclassifies every point as paramagnetic.The reference solution returns hc = 0.60.Source: §C.5 · p. 22
  • T5’s greedy choice misses the size-three independent set, while its untranspiled embedding-overhead value passes a check that misses SWAP inflation.T6’s balanced but incorrect cut fails; separately, a one-sided assignment with cut zero passes every check despite the stated bisection requirement.Source: §C.5 · pp. 22–23

D.1 FULL PASS@K RESULTS

The execution-scored suite results are reported as pass@k, with recorded evaluator outcomes and some duplicated generations requiring qualification.

D.1 FULL PASS@K RESULTS

  • Table 9 reports pass@1, pass@3, and pass@5 for the four execution-scored suites.Per-cell 95% intervals are available in the released artifacts.Source: Table 9 · p. 23, §D.1 · p. 23
  • Some QuanBench-44 generations duplicate QuanBench-117 outputs and were not evaluated, while other recorded evaluator outcomes await replay.The duplicate-generation caveat applies to LLaMA-3.1-8B and Quantum RAG.Source: Table 9 · p. 23

D.2 STANDARD–HARD GAPS AND OBSERVED COVERAGE

The results show substantial Standard–Hard score gaps, unsolved tasks with benchmark-side import obstruction, and model complementarity and sampling gains. These comparisons have distinct limits: observed coverage is not proof of unsolvability, and oracle unions are not deployable ensembles.

D.2 STANDARD–HARD GAPS AND OBSERVED COVERAGE

  • The 17 QuanBench-117 tasks unsolved across all five samples share a benchmark-side import obstruction; observed coverage does not establish unsolvability.Source: §D.2 · p. 23, Table 11 · p. 24
  • The matched Standard–Hard pass@1 gap ranges from 9 pp for Granite-Qiskit to 67 pp for Muse-Glimmer-30B.Gemma-3-4B and Gemini 3.6 Flash score about 65% on Standard but 0.1% and 40.8% on Hard, respectively; prompt scaffolding changes, so gaps reflect format sensitivity as well as task difficulty.Source: §D.2 · p. 23, Table 10 · p. 24
  • Oracle-union coverage gains reach 11 pp on Hard and 7 pp on QCoder, but require double the generation budget and are not deployable-ensemble performance.On Hard, the largest cited gains involve quantum-specialized models; on QCoder, the largest is between Claude Fable 5.1 and GPT-6 Astra.Source: §D.2 · pp. 23–24, Figure 8 · p. 24
  • Quantum-specialized and small open-weight models gain up to 38 pp on QuanBench-117, while Gemini Flash models gain 21–25 pp there.The proposed formatting-failure explanation for Gemini Flash is unverified; QCoder’s prior-conditioned gains are not comparable with i.i.d. suites.Source: §D.2 · pp. 24–25
  • On execution-scored suites, GPT-6 Astra gains zero from pass@1 to pass@5 on Standard, QuanBench-44, and QuanBench-117; Claude Fable 5.1 gains zero on QuanBench-117.Source: §D.2 · p. 24, Figure 9 · p. 25

D.3 QCODER: FULL RESULTS AND JUDGE PROTOCOL

QCoder uses sequential, judge-scored samples, so its reported pass@k describes that protocol rather than independent-sampling estimates. Output-validity and similarity measures add context but do not establish functional success.

D.3 QCODER: FULL RESULTS AND JUDGE PROTOCOL

  • QCoder samples are conditioned on prior attempts, violating the pass@k estimator’s i.i.d. assumption; results describe the recorded sequential protocol.An LLM judge supplies binary verdicts because tasks lack canonical solutions and executable tests.Source: §D.3 · p. 25
  • Identical QCoder code can receive different judge verdicts, and inter-run or cross-provider agreement was not measured.Source: §D.3 · p. 25

D.4 OUTPUT VALIDITY AND CODEBLEU

  • GPT-5 saved empty code for 316/755 Standard samples (41.9%); 315 ended at exactly 2,048 tokens, consistent with output-budget exhaustion.Empty-output rates rose across suites to 63.0% on QCoder.Source: §D.4 · p. 25
  • All 2,650 audited nonempty GPT-6 Astra outputs parse as Python, but saved execution fields cannot yet provide a validated infrastructure-adjusted success rate.Candidate, reference, dependency, and evaluator failures remain incompletely adjudicated.Source: §D.4 · p. 25
  • CodeBLEU correlates with recorded pass/fail on Standard and Hard, but reference-code similarity is not equivalent to functional or quantum-semantic correctness.The reported point-biserial correlations are r=0.280 [0.222–0.332] on Standard and r=0.487 [0.420–0.550] on Hard.Source: §D.4 · pp. 25–26
  • Table 13 reports nonempty and syntax-valid sample rates, while Table 12 reports QCoder pass@k under its sequential judge-scored protocol.Source: Table 13 · p. 26, Table 12 · p. 26

E QUALITATIVE EXAMPLES

The examples show that quantum-code benchmark outcomes can reflect API and framework mismatches, flawed evaluator behavior, and inconsistent judgments, not only circuit correctness. Independent simulation also distinguishes genuine circuit errors from suspect QuanBench labels.

E QUALITATIVE EXAMPLES

  • QuanBench sample labels come from the audited evaluator and have not been revalidated, so the authors independently checked the examples by simulation.Full responses are in supplementary material.Source: E QUALITATIVE EXAMPLES · p. 26
  • A QiskitHumanEval Standard response fails before circuit construction because it imports UnitaryGate from a nonexistent qiskit.quantum_info path.The two responses follow the same logic; the failure is an API mismatch rather than a logic error.Source: E QUALITATIVE EXAMPLES · p. 26
  • A QiskitHumanEval Hard response uses Cirq, so the Qiskit harness cannot evaluate its syntactically valid implementation.The Hard task lacks the Standard skeleton’s Qiskit import.Source: E QUALITATIVE EXAMPLES · p. 28
  • The QuanBench-44 passing sample prepares the four-qubit W state exactly, while its failing counterpart produces an unequal superposition with fidelity 0.21.The passing circuit has fidelity 1.0; the failing sample’s probabilities are 0.125, 0.125, 0.5, and 0.25.Source: E QUALITATIVE EXAMPLES · p. 28
  • The passing QuanBench-117 sample is a false pass, while the failing sample ends in a SyntaxError after truncation, illustrating two different evaluator outcomes.The returned circuit in the recorded pass is not equivalent to CNOT up to global phase; the failure is consistent with the 2,048-token output limit.Source: E QUALITATIVE EXAMPLES · p. 28
  • The two identical, correct QCoder SWAP responses received opposite PASS and FAIL judgments in separate calls.Three alternating CNOTs implement SWAP.Source: E QUALITATIVE EXAMPLES · p. 28

Related papers

Loading 2610.28678v1…