PaperlayerView exact revision on arXiv

Source-linked AI summary

Generalization in quantum machine learning from few training data

Matthias C. Caro, Hsin-Yuan Huang, M. Cerezo, Kunal Sharma, Andrew Sornborger, Lukasz Cincio, Patrick J. Coles

arXiv:2111.05292v2quant-phcs.LGstat.ML

TL;DR

QML generalization remains insufficiently understood when training data are limited, despite its importance for predicting unseen data. The paper derives complexity- and optimization-dependent bounds for variational QML and applies them to phase recognition and unitary compiling. It finds that good generalization can be guaranteed with few training data, including polynomial resources for efficiently implementable models.

  • Problem

    Conditions for accurate QML generalization and the training-data size needed to avoid overfitting remain insufficiently understood.

  • Method

    The paper derives probabilistic generalization bounds using covering numbers for quantum operations and chaining techniques, while accounting for gate reuse and optimization changes.

  • Results

    The bounds guarantee good generalization with training data scaling effectively linearly in independently parameterized gates, improving when only a subset changes substantially.

  • Takeaways & Limitations

    Efficiently implementable QMLMs can obtain good generalization with polynomial training data, supporting applications including QCNN phase recognition and unitary compiling.

Abstract

from arXiv · show

Modern quantum machine learning (QML) methods involve variationally optimizing a parameterized quantum circuit on a training data set, and subsequently making predictions on a testing data set (i.e., generalizing). In this work, we provide a comprehensive study of generalization performance in QML after training on a limited number $N$ of training data points. We show that the generalization error of a quantum machine learning model with $T$ trainable gates scales at worst as $\sqrt{T/N}$. When only $K \ll T$ gates have undergone substantial change in the optimization process, we prove that the generalization error improves to $\sqrt{K / N}$. Our results imply that the compiling of unitaries into a polynomial number of native gates, a crucial application for the quantum computing industry that typically uses exponential-size training data, can be sped up significantly. We also show that classification of quantum states across a phase transition with a quantum convolutional neural network requires only a very small training data set. Other potential applications include learning quantum error correcting codes or quantum dynamical simulation. Our work injects new hope into the field of QML, as good generalization is guaranteed from few training data.

I. Introduction

The paper addresses limited understanding of QML generalization by deriving bounds tied to circuit complexity and optimization changes. It argues that efficiently implementable models can generalize with efficient training data, with applications to QCNN phase recognition and unitary compiling.

  • Accurate prediction on unseen data requires understanding how generalization error depends on training-set size and model complexity.
  • QML generalization remains less understood than trainability, while exponentially large Hilbert spaces raise concerns about exponential training-data requirements.
  • The paper derives generalization bounds using covering numbers for implementable quantum operations and chaining techniques for random processes.
  • Efficiently implementable QML models with polynomially many trainable gates require polynomially many training data points for good generalization.
  • Generalization improves when only a subset of parameters changes substantially during optimization, even if the total parameter count exceeds the training-set size.
  • The framework is demonstrated on QCNN phase recognition and quantum Fourier-transform compiling, where limited training data are relevant to practical applications.

A. Framework

The framework models QMLMs as parameterized quantum channels acting on encoded data and evaluated through observable-based loss functions. It permits continuous parameters, variable gate structures, repeated gates, and measurements across multiple copies.

  • A QMLM is treated as a parameterized completely positive trace-preserving map, with continuous gate parameters and potentially discrete structure parameters.
  • Classical or quantum data are encoded into quantum states using a fixed encoding map that is not optimized during training.
  • The model may act on a subsystem, producing an output state that is evaluated using a Hermitian loss observable.
  • The loss magnitude is bounded uniformly by assuming a finite supremum over inputs, labels, and loss observables.
  • Tensor-product copies with shared parameters allow global measurements on multiple copies, and reused trainable gates are included in the framework.
  • Training error averages losses over N examples, while prediction error averages the loss over the data-generating distribution and is the quantity bounded in the theorems.

B. Analytical Results

The analytical results bound QML generalization in terms of trainable-gate complexity, gate reuse, optimization changes, and variable architectures. The bounds support efficient sample complexity for polynomial-size models and remain logarithmic in reuse or architecture counts.

  • Basic QMLM: High-probability generalization guarantees relate the required training-set size to the number of independently parameterized gates.
  • Basic QMLM: N ∼ T log T/ε2 suffices to guarantee generalization error at most ε with high success probability for the basic QMLM.
  • Basic QMLM: Polynomially many trainable gates require polynomially many training examples, up to logarithmic factors and the target-error dependence.
  • Gate-sharing QMLM: Gate sharing makes good generalization depend effectively linearly on independently parameterized gates and logarithmically on the number of gate uses.
  • Optimization: When only K ≪ T gates change substantially, the generalization error scales at worst linearly with K and logarithmically with the total parameterized-gate count MT.
  • Variable structure: Variable-structure QMLMs can be controlled by the post-optimization gate count, with only logarithmic dependence on the number of possible architectures.

C. Numerical Results

The numerical study evaluates two QML applications: QCNN classification of quantum phases and training a parameterized circuit to compile a quantum Fourier-transform matrix.

  • The numerical experiments simulate QCNN phase classification and parameterized-circuit compilation of a quantum Fourier transform.
  • The two implementations represent distinct applications of the analytical generalization framework to quantum-state classification and unitary transformation learning.

1. Phase classification

The QCNN classifies phases of generalized-cluster-Hamiltonian ground states by encoding phase information into a two-qubit output and selecting the least-probable computational-basis outcome. Its testing accuracy tracks training accuracy even with small training sets, consistent with a polylogarithmic dependence of required data on qubit number.

  • QCNN architecture: The QCNN alternates convolutional layers, which filter features with translationally invariant unitaries, and pooling layers, which measure qubits to reduce dimensionality.Pooling is conditioned on measurement outcomes while preserving relevant data features.
  • Task and data: The model classifies four phases of generalized-cluster-Hamiltonian ground states sampled across the (J1, J2) plane.The phases are symmetry-protected topological, ferromagnetic, anti-ferromagnetic, and trivial.
  • Evaluation: 8192 measurement outcomes determine each predicted phase, while 30 labelled points train the 16-qubit example shown in panel (a).Blue crosses mark training points; blue and red circles indicate correct and incorrect classifications.
  • Task and data: Phase information is encoded in the two-qubit output state by assigning each phase to the computational-basis effect operator with the smallest probability.Training examples contain ground states and length-two bit-string labels.
  • Generalization: Testing accuracy becomes well predicted by training accuracy as training-set size increases, and the generalization error bound for QCNNs is polylogarithmic in the number of qubits.The reported correlation agrees with the paper’s theoretical predictions.

2. Unitary compiling

The paper studies variational compilation of the n-qubit QFT into a short-depth, variable-structure circuit using training input-output state pairs. Theoretical and numerical results show accurate compilation with polynomial training data, with linear scaling for computational-basis inputs and especially low requirements near a good initialization.

  • Motivation: Unitary compiling transforms a high-level algorithm into device-compatible low-level gates, addressing connectivity and shallow-depth constraints in NISQ hardware.The paper identifies compiling as a crucial industry task performed before running algorithms on NISQ devices.
  • Method: The VAns algorithm grows a parameterized circuit by placing and removing gates while training it to approximate the QFT unitary.This variable-structure approach searches over circuit architectures rather than only optimizing continuous parameters in a fixed structure.
  • Method: Compilation training uses input states and their target outputs U|ψi⟩, with computational-basis, low-entangled, or Haar-random inputs drawn from an underlying distribution.The loss compares the output states produced by U and V(α) using a squared trace-distance expression.
  • Results: Polynomial-size training data accurately compiles the QFT with high success probability, while computational-basis inputs require training data scaling linearly with qubit number.The observed linear scaling improves on the quadratic scaling from directly applying the general theory.
  • Results: Constant training data suffices up to n = 9 qubits for Haar-random inputs, whereas two points suffice through n = 40 when initialization is favorably close to the solution.One point does not guarantee good generalization even as training error decreases.

III. Discussion

The discussion highlights that one training point is insufficient for guaranteed generalization, whereas favorable initialization can make two points sufficient even for systems of up to 40 qubits.

  • Initialization and data requirements: N = 1 does not guarantee good generalization: decreasing training error can coexist with persistently large testing error.The comparison uses 20 test states that may be more strongly entangled than the low-entangled training states.
  • Initialization and data requirements: N = 2 suffices for good generalization up to n = 40 qubits when the circuit is favorably initialized near the solution.The numerical result is more optimistic than the reduction predicted by Theorem 3 and motivates further investigation.

A. Impact on specific applications

The paper connects its generalization theory to phase classification, unitary compiling, dynamical simulation, quantum error-correcting codes, autoencoders, and quantum GANs. These applications involve training data, making the bounds relevant to data requirements, scalability, or ansatz design within the stated settings.

  • Phase classification: For QCNN phase classification, the theory replaces earlier heuristic explanations with rigorous generalization guarantees confirmed numerically across a complicated phase diagram and varied system sizes.Small training sets matter experimentally because preparing states from multiple phases requires careful Hamiltonian tuning.
  • Unitary compiling: Unitary compiling can use polynomial-size rather than exponential-size training data for efficiently implementable targets, making QFT compilation scalable beyond prior approaches.The results also offer insight into why VAns succeeds and may guide larger-scale ansatz design.
  • Dynamical simulation: Quantum dynamical simulation may benefit because compiling structured Trotterized unitaries can support longer-time simulations with fixed-depth circuits.The paper expects its compiling results to carry over to this application.
  • Error correction: Generalization bounds may accelerate quantum error-correcting-code discovery by reducing training-data requirements for average-fidelity optimization.The proposed benefit is specifically framed for approaches performing code discovery on classical computers.
  • Generative and compression models: For quantum autoencoders and GANs, the bounds provide quantitative guidance on training-data amounts and support ansatz design favoring minimal parameter counts.The paper notes that these applications lack a standard ansatz.

B. Related work on generalization

Prior QML generalization research has used classical learning theory, geometric approaches, information-theoretic techniques, and covering numbers. The paper positions its bounds as stronger, more general, and broader in scope than these approaches.

  • Prior studies analyzed QML generalization using classical learning-theoretic approaches.
  • Other approaches used geometric and information-theoretic perspectives to study QML generalization.
  • The paper also compares its results with independent work on covering numbers in QMLMs.
  • The authors characterize their bounds as stronger, significantly more general, and broader in scope than prior work.

C. Quantum advantage and future outlook

The paper does not establish a quantum advantage, but presents generalization bounds as relevant to identifying tasks where QMLMs may outperform classical models. It also identifies regimes and distributions where the bounds remain limited or potentially pessimistic.

  • The results do not prove a quantum advantage over classical machine learning.
  • Quantum advantage would require QMLMs with few trainable gates to achieve small training error while classical models require substantially higher complexity.
  • The bounds guarantee good unseen-data performance for such QMLMs, while the paper expects high-complexity classical models to generalize poorly.
  • For QMLMs with exponentially many independently trainable gates, the bounds scale exponentially with n and make no non-trivial claims.
  • Because the bounds apply to arbitrary data-generating distributions, they may be overly pessimistic for favorable distributions.
  • Unitary-compiling experiments found highly entangled states more favorable for generalization than especially efficiently preparable states.

IV. Methods

The methods section first outlines the proof strategy for the paper’s generalization bounds and then describes the numerical investigations.

  • The methods overview has two parts: proof strategies for the generalization bounds and details of the numerical investigations.
  • The proof-strategy discussion precedes the presentation of numerical-investigation details.
  • The section serves as an overview rather than a detailed account of either the proofs or numerical investigations.

A. Analytical methods

The analytical approach bounds QML model complexity through metric entropy, then combines these bounds with classical generalization results and chaining. A central theorem handles variable architectures, gate reuse, and optimization-dependent gate changes, with simpler bounds recovered as special cases.

  • Analytical methods: Metric entropies, defined as logarithms of covering numbers, quantify model complexity for generalization bounds.
  • Analytical methods: The proof establishes new metric entropy bounds for QMLMs and combines them with known classical learning-theory results.
  • Analytical methods: Theorem 5 covers QMLM architectures with arbitrary numbers of parameterized local channels and architecture counts GT.
  • Analytical methods: Theorem 5 incorporates gate reuse through M and optimization-dependent channel changes through Δt.
  • Analytical methods: Theorems 1–4 arise as special cases of Theorem 5 under fixed architectures, single-use maps, and selected parameter settings.
  • Analytical methods: The theorem’s proof uses covering nets for admissible local quantum channels, with logarithmic cardinality bounded by O(K log (MT/ε)).
  • Analytical methods: A chaining argument combines metric entropy with an upper bound involving the expected supremum of a random process.

K log(MT )/N + PT

The proof extends fixed-architecture generalization bounds to variable QML architectures by applying union bounds over optimization changes and admissible circuit sizes.

  • Proof strategy: A union bound over K yields a bound minimized over K = 0, . . . , T.This first union bound applies to QML models with fixed architecture.
  • Proof strategy: A second union bound over admissible trainable-gate counts T extends the result to variable QMLM architectures.The resulting dependence on the number of architectures is logarithmic in G_T.
  • Proof strategy: The final theorem uses a success probability proportional to (G_T T^2)^−1 after accounting for all admissible architectures.This completes the variable-architecture generalization bound.

B. Numerical methods

The paper combines tensor-network simulation and optimization methods to study quantum phase classification and quantum compiling, while connecting these applications to generalization theory.

  • Phase classification: Ground states of the cluster Hamiltonian are generated for different coupling strengths and selected away from phase-transition lines for phase classification.Translation-invariant Density Matrix Renormalization Group computes the states, with small bond dimension sufficient for the training states.
  • Phase classification: Matrix Product State techniques simulate and optimize the QCNN empirical risk using alternating translationally invariant convolutional and pooling layers.The layers are represented by parameterized two-qubit gates and applied through updates to nearest-neighbor MPS tensors.
  • Phase classification: Shot-frugal optimization grows the number of pooling-layer measurement shots as empirical-risk minimization proceeds.The procedure controls gradient accuracy according to the current optimization landscape and uses Simultaneous Perturbation Stochastic Approximation.
  • Quantum compiling: Unitary compilation is formulated as empirical-risk minimization over shallow circuits with discrete layout parameters and continuous gate parameters.The parameter vector specifies a trainable unitary, while the discrete parameters control gate placement.
  • Generalization implications: The results extend generalization guarantees to polynomial-depth circuits, variable QMLM structures, CPTP maps, and multiple uses of trainable maps.The paper reports stronger and more general bounds than earlier covering-number results, including optimization-dependent bounds.
  • Phase classification: QCNN phase classification can generalize from training sets whose required size scales at most polylogarithmically with system size.The guarantee applies when the QCNN achieves good performance on a moderate training set, although small training error still faces barren-plateau challenges.
  • Quantum compiling: Polynomial-sized training sets can extend the unitary-size cutoff for compilation, potentially speeding classical and quantum-assisted compiling methods.The paper connects this to cloud-based quantum computing and reduced resource costs for quantum-assisted compilation.
  • Quantum compiling: The general bounds cover variable ansatz compilation, including circuit structures that change during optimization.This provides guidance for training-data requirements in state-of-the-art variable-ansatz approaches.

Appendix C Analytical Results: Details and Proofs

The appendix develops metric-entropy bounds for quantum machine learning models and converts them into generalization guarantees. It then extends the guarantees to gate-sharing, multiple copies, variable architectures, and optimization-dependent training.

  • Metric entropy bounds: Metric entropy grows at worst slightly super-linearly with the number of parameterized operations in both unitary and CPTP-map QMLMs.The appendix establishes covering-number bounds for these model classes as the basis for subsequent generalization results.
  • Generalization bounds: The main generalization bound decreases with training size as the square root of 1/N and depends at worst slightly super-linearly on the number T of trainable maps.The bound also has logarithmic dependence on the number of uses, possible architectures, and reciprocal confidence level.
  • Generalization bounds: Taking training data size N to scale effectively linearly with T ensures that small training error implies small prediction error with high probability.This interpretation follows from the fixed-structure prediction error bound.
  • Gate-sharing and multiple copies: Gate-sharing makes generalization depend on the effective number of independently trainable elements rather than the total number of repeated gate uses.The corresponding bounds depend only logarithmically on the number of uses Mt, and reusing parameters is preferable to adding independent parameters from a generalization perspective.
  • Gate-sharing and multiple copies: Using multiple copies with identical parameter settings preserves good prediction error because expressiveness grows at most logarithmically with the number of copies.The result applies to procedures such as averaged product measurements or majority voting followed by classical loss processing, but not arbitrary post-processing.
  • Variable architectures: Variable-structure QMLMs depend strongly on the T trainable elements in the output structure but only logarithmically on the number GT of possible structures with that gate count.Even exponentially many candidate structures with fixed T do not worsen the asymptotic generalization behavior.
  • Optimization-dependent bounds: Optimization-dependent bounds can be much tighter when only a few trainable maps undergo substantial changes, because the bound is dominated by those maps rather than all T maps.The analysis also favors large optimization steps on few gates and negligible steps on the remainder.
Loading 2111.05292v2…