PaperlayerView exact revision on arXiv

Source-linked AI summary

Benchmarking Quantum Processor Performance at Scale

David C. McKay, Ian Hincks, Emily J. Pritchett, Malcolm Carroll, Luke C. G. Govia, Seth T. Merkel

arXiv:2311.05933v1quant-ph

TL;DR

For N-qubit layer fidelity, exhaustive measurement across all subsets can be impractical. The paper measures gate errors in disjoint layers and combines their fidelities, then demonstrates that layered and isolated RB can yield different error patterns on Eagle and Heron.

  • Problem

    Measuring the maximum layer fidelity across all N-qubit subsets can be impractical; for example, a 127-qubit heavy-hex device has 313,980 length-100 chains.

    Source: §II · p. 4

  • Method

    Layer fidelity uses disjoint layers and individual gate fidelities to estimate subset layer fidelity, with the device value defined as the maximum measured value.

    Source: §I · pp. 3–4

  • Results

    Layered RB shows a distinct increase in error per gate on Eagle relative to isolated RB, while the errors are greatly alleviated on Heron.

    Source: §II · p. 6

  • Takeaways & Limitations

    Layer fidelity provides individual gate errors and higher signal-to-noise than mirror RB’s whole-layer measurement, avoiding a high-weight observable whose signal can become unmeasurably small at scale.

    Source: §I · p. 4

    The authors leave open questions about predictive power for differently structured circuits, twirling limits, advanced fitting, and extension to mid-circuit measurements.

    Source: §III · p. 6

Abstract

from arXiv · show

As quantum processors grow, new performance benchmarks are required to capture the full quality of the devices at scale. While quantum volume is an excellent benchmark, it focuses on the highest quality subset of the device and so is unable to indicate the average performance over a large number of connected qubits. Furthermore, it is a discrete pass/fail and so is not reflective of continuous improvements in hardware nor does it provide quantitative direction to large-scale algorithms. For example, there may be value in error mitigated Hamiltonian simulation at scale with devices unable to pass strict quantum volume tests. Here we discuss a scalable benchmark which measures the fidelity of a connecting set of two-qubit gates over $N$ qubits by measuring gate errors using simultaneous direct randomized benchmarking in disjoint layers. Our layer fidelity can be easily related to algorithmic run time, via $\gamma$ defined in Ref.\cite{berg2022probabilistic} that can be used to estimate the number of circuits required for error mitigation. The protocol is efficient and obtains all the pair rates in the layered structure. Compared to regular (isolated) RB this approach is sensitive to crosstalk. As an example we measure a $N=80~(100)$ qubit layer fidelity on a 127 qubit fixed-coupling "Eagle" processor (ibm\_sherbrooke) of 0.26(0.19) and on the 133 qubit tunable-coupling "Heron" processor (ibm\_montecarlo) of 0.61(0.26). This can easily be expressed as a layer size independent quantity, error per layered gate (EPLG), which is here $1.7\times10^{-2}(1.7\times10^{-2})$ for ibm\_sherbrooke and $6.2\times10^{-3}(1.2\times10^{-2})$ for ibm\_montecarlo.

I. LAYER FIDELITY PROTOCOL

Layer fidelity benchmarks a connected set of two-qubit gates by measuring their errors in disjoint layers with simultaneous direct randomized benchmarking. The measured gate and qubit fidelities provide a full-layer estimate and fine-grained error information.

I. LAYER FIDELITY PROTOCOL

  • Layer fidelity measures each disjoint layer’s two-qubit gates and qubits with simultaneous direct randomized benchmarking, then combines the measured fidelities to estimate the full layer.The decay rates yield process fidelities for the measured error channels.Source: §I · p. 2
  • The protocol requires barriers around each two-qubit layer so its gates run consistently with the full layer; idle-qubit errors must also be measured.At the end, each disjoint set is inverted and ground-state population is measured versus sequence length.Source: §I · p. 3
  • The layer fidelity of a device for N qubits is defined as the maximum layer fidelity measured on the device.The protocol also relates LF to γ, a quantity used to characterize layer error models.Source: §I · pp. 3–4
  • For small errors, the product-based estimate is a good approximation; with certain crosstalk terms, layer fidelity can be a lower bound.The estimate is not generally identical to the full layer fidelity when crosstalk is present.Source: §I · p. 3
  • Unlike mirror circuit RB, layer fidelity returns individual gate error rates and has higher signal-to-noise by measuring individual rates rather than a whole-layer observable.The cited comparison says high-weight observables can become unmeasurably small as qubit count grows; simulations support LF capturing most crosstalk terms for realistic noise models.Source: §I · p. 4
  • For subsets, LF can be calculated from the individual gate fidelities, but the subspace method is not guaranteed to find the optimal value and provides a lower bound.In a line, subsets are sliding windows; gates extending outside a subset are treated as sharing fidelity equally between the subsets.Source: §I · p. 4

II. DATA

The data section applies layer fidelity to long chains on Eagle and Heron processors and describes a heuristic for selecting chains when exhaustive measurement is impractical. Layered measurements show a different error pattern from isolated RB, especially on Eagle.

II. DATA

  • For N-qubit subsets, the device LF is the maximum over all such subsets, but exhaustive measurement is impractical; a 127-qubit heavy-hex device has 313,980 length-100 chains.The paper therefore uses heuristic chain selection.Source: §II · p. 4
  • The minimum isolated gate error (process error) is 3.2(4.0) × 10−3 on ibm sherbrooke (Eagle) and 1.2(1.6) × 10−3 on ibm montecarlo (Heron).Source: §II · p. 5
  • The heuristic predicts high-fidelity chains from isolated-RB errors, measures three low-overlap candidates with simultaneous direct RB, then takes the largest LF among measured subchains.Long gates may be omitted, and the protocol accounts for the fixed duration of a layer by considering its longest gate.Source: §II · p. 5
  • The reported measurements use 127-qubit Eagle and 133-qubit Heron processors, with three Eagle edges longer than 700 ns removed and no Heron edges removed for gate length.Eagle’s typical gate length was 533 ns; Heron’s gates were about an equal mix of 84 ns and 104 ns.Source: §II · p. 6
  • Layered RB shows a distinct increase in error per gate on Eagle relative to isolated RB, while the errors are greatly alleviated on Heron.The authors attribute the contrast to fixed coupling on Eagle and the ability to turn off neighboring-qubit coupling on Heron.Source: §II · p. 6

III. CONCLUSIONS

The paper presents layer fidelity as a scalable, crosstalk-aware benchmark that provides fine-grained information and relates to other methods of characterizing layers. It reports measurements on Eagle and Heron processors and identifies open questions and the benchmark’s role as one component of device characterization.

III. CONCLUSIONS

  • Layer fidelity is presented as a scalable benchmark that is crosstalk-aware, quick to measure across many qubits, high in signal-to-noise, and fine-grained.The authors demonstrate its components on the 127-qubit Eagle and 133-qubit Heron processors.Source: §III · p. 6
  • Simulations and data show good agreement between layer fidelity and mirror randomized benchmarking across several error models.The paper describes mirror RB as a complementary technique for measuring layers.Source: §III · p. 6
  • Layer fidelity links to other layer-characterization methods, including Pauli learning for γ.The authors leave open questions about advanced fitting, predictive power for differently structured circuits, twirling limits, and mid-circuit measurements.Source: §III · p. 6
  • The authors frame layer fidelity as one piece of information toward full device characterization, not a complete characterization on its own.Source: §III · p. 6

Appendix A: Additional Data

Additional data compare layer fidelity with mirror RB, test the effect of dividing a layer into more disjoint sets, and compare LF with a direct estimate of γ. These tests report reasonable agreement in the γ comparison and show that the effect of more layers depends on the architecture and crosstalk.

Appendix A: Additional Data

  • Layer fidelity and mirror RB agree quite well in a 20-qubit comparison, though mirror results differ depending on whether a Pauli layer is included.The experiment used a 27-qubit fixed-coupling Falcon processor and compared 10 mirror randomizations and 2,000 shots with 6 LF randomizations and 300 shots.Source: Appendix A · p. 8
  • More disjoint layers reduce fidelity in the reported data because they increase total duration, but architecture-specific crosstalk or limits on simultaneous gates can make additional splitting sensible.Source: Appendix A · p. 8
  • On a 16-qubit section, LF and direct γ measurements agree reasonably, using 178 LF circuits versus 14,000 γ circuits.The authors leave a more comprehensive study with error bars and reduced time variation for future work.Source: Appendix A · p. 8

Appendix B: Simulations

The simulations compare layer fidelity with randomized-benchmarking alternatives under incoherent errors and several coherent crosstalk scenarios. Layer fidelity tracks mirror RB with a Pauli layer closely, while the comparisons reveal cases where isolated or simultaneous RB does not represent the layer accurately.

Appendix B: Simulations

  • As disjoint layers are added, fidelity falls with increased duration; reduced mean gate error from lower crosstalk is insufficient to offset that increase.Source: Appendix B · p. 9
  • With unequal two-qubit gate lengths, simultaneous RB gives the wrong layer error, whereas layer fidelity's barriers make the layer as long as its longest gate.Source: Appendix B · p. 9
  • For always-on ZZ crosstalk, isolated RB is unaffected; simultaneous RB and layer RB are reasonably similar, provided the two-qubit gates have equal lengths.Source: Appendix B · pp. 10–11, Figure 6 · p. 10
  • For full-layer always-on ZZ crosstalk, layer RB agrees closely with mirror RB using a Pauli layer, while mirror RB without one reports much higher error.Source: Figure 6 · p. 10, Appendix B · p. 11
  • When ZZ crosstalk activates only during simultaneous two-qubit gates, layer fidelity and simultaneous RB diverge because simultaneous RB does not enforce exact gate overlap.Source: Appendix B · p. 11
  • Staggering gates removes simultaneous-only ZZ crosstalk, but the longer layer has higher baseline error, which can outweigh the reduced crosstalk.Source: Appendix B · p. 11
  • Layer fidelity and mirror RB with a Pauli layer measure similar errors across tested coherent-error scenarios; layer fidelity tends to report slightly higher error, while mirror RB without Pauli reports more.Source: Appendix B · p. 11

Appendix C: Crosstalk and Layer Fidelity

The appendix analyzes how layer fidelity estimates process fidelity in the presence of coherent and stochastic crosstalk. For the considered weight-2 Pauli crosstalk channels, the layer-fidelity estimate is a lower bound on the true process fidelity.

Appendix C: Crosstalk and Layer Fidelity

  • For a weight-2 coherent Pauli crosstalk term spanning two subspaces, multiplying the subspace fidelity estimates yields a layer fidelity below the true process fidelity.Source: Appendix C · pp. 11–12
  • The simulations compare layer fidelity, mirror RB, and mirror RB without a Pauli layer across coherent-error scenarios, with T1 = T2 = 50 µs and a 50 ns unit time.Source: Figure 7 · p. 11, Appendix C · p. 11
  • For the considered Pauli stochastic error channel, layer fidelity is likewise a lower bound on the true process fidelity.Source: Appendix C · p. 12

Appendix D: Combining Process Fidelities

Process fidelities combine multiplicatively across disjoint subsystems, which supports constructing a disjoint-layer fidelity. That multiplicative rule does not generally extend across repeated layers, though it approximately holds for small errors of diagonal maps.

Appendix D: Combining Process Fidelities

  • The appendix notes that process fidelity is related to average gate fidelity, with average gate error defined as 1 − Fg.Source: Appendix D · p. 12
  • For disjoint subsystems, the combined process fidelity is the product of their individual process fidelities, the rule used to build each disjoint-layer fidelity.Source: Appendix D · p. 12
  • Process fidelities do not generally multiply across layers, but for small errors of diagonal maps this approximation holds until repeated-layer fidelity drops below a percent.Source: Appendix D · p. 12

Appendix E: Relating γ to LF

This appendix relates layer fidelity to γ for Pauli-diagonal noise and compares γ with gate metrics derived from the process matrix spectrum. In the small-error limit, it uses a Lindbladian expansion to connect the quantities.

Appendix E: Relating γ to LF

  • The appendix relates layer fidelity to γ, defined for Pauli-diagonal noise, and uses γ to indicate the circuit randomizations required for probabilistic error mitigation.Source: Appendix E · p. 12
  • In the small-error limit, the derivation expands exponentials of the Lindbladian generators to relate γ to the spectrum of the process matrix.Source: Appendix E · pp. 12–13
  • For Pauli channels, average gate error and diamond norm can be derived from the arithmetic average of the eigenvalues of Λ.Source: Appendix E · p. 13
  • γ is related to the geometric mean of the eigenvalues of Λ².Source: Appendix E · p. 13

1. depolarizing channels

For depolarizing channels, the spectrum has one eigenvalue of 1 and d^2−1 eigenvalues of 1−α. The section relates this depolarizing parameter to process fidelity and γ-based quantities.

1. depolarizing channels

  • For a depolarizing channel, the spectrum of Λ consists of a single eigenvalue 1 and d^2−1 eigenvalues equal to 1−α.Source: §1 · p. 13
  • As d grows, both quantities discussed converge to 1−α.Source: §1 · p. 13
  • Gate fidelity and diamond norm can alternatively be expressed in terms of γ.Source: §1 · p. 13

2. The small error limit

The small-error analysis assumes a process close to identity, with spectral terms of the form 1−ε_j. In this limit, the arithmetic and geometric means agree.

2. The small error limit

  • The near-identity assumption represents spectral terms as 1−ε_j and defines a mean error ε̄.Source: §2 · p. 13
  • In the small-error limit, the arithmetic and geometric means agree.Source: §2 · p. 13

3. Bounds

The section bounds γ−1 using process fidelity for CPTP Pauli channels, with bounds independent of dimension. It examines channel families that approach equality or saturate the bounds, illustrating when process fidelity is or is not a close proxy for γ−1.

3. Bounds

  • Theorem 1 bounds γ−1 for a CPTP Pauli channel in terms of its process fidelity Fp.The bounds are independent of dimension, and the lower bound is extremely close to 2Fp−1 over Fp in [1/2, 1]; for high-fidelity layers, Fp≈γ−1/2.Source: §3 · p. 13
  • The arithmetic and geometric means coincide only when all Pauli fidelities are equal, which for trace-preserving channels occurs only for the identity channel.The section identifies globally depolarizing channels as the next closest case.Source: §3 · pp. 13–14
  • Global depolarizing and tensor products of two-qubit depolarizing channels have Pauli fidelities that are close to equal even at moderate n.For global depolarizing channels, the passage says the Pauli fidelities are very close to equal; the tensor-product family is described similarly.Source: §3 · p. 14
  • The lower bound is nearly saturated by channels with strongly bimodal Pauli fidelities, concentrating roughly half at 1 and half below 1.The proof bounds Pauli fidelities within [2Fp−1, 1] and uses the geometric mean's relation to the arithmetic mean.Source: §3 · pp. 13–15
  • A channel with one faulty subsystem is a physically relevant example where γ captures the outlier error but process fidelity's arithmetic mean does not.The relevant noise model is described as having one high-error subsystem, with the behavior independent of n.Source: §3 · p. 14

Related papers

Loading 2311.05933v1…