PaperlayerView exact revision on arXiv

Source-linked AI summary

Absence of Barren Plateaus in Quantum Convolutional Neural Networks

Arthur Pesah, M. Cerezo, Samson Wang, Tyler Volkoff, Andrew T. Sornborger, Patrick J. Coles

arXiv:2011.02966v2quant-phcs.LGstat.ML

TL;DR

Many quantum neural-network architectures can develop barren plateaus with exponentially vanishing gradients, motivating guarantees of trainability for QCNNs. The paper introduces GRIM and applies it to QCNN gradient variance, finding polynomial-or-slower vanishing under stated assumptions and extending the analysis to pooling-only QCNNs.

  • Problem

    Barren plateaus occur in several quantum machine-learning architectures, while efficient trainability of variational quantum algorithms and quantum neural networks remains insufficiently guaranteed.

  • Method

    The paper introduces the Graph Recursion Integration Method (GRIM) and uses it to analyze Haar-distributed-unitary expectation values and QCNN gradient-variance bounds.

  • Results

    The QCNN gradient-variance lower bound vanishes no faster than polynomially with system size, establishing the absence of barren plateaus under the stated assumptions.

  • Takeaways & Limitations

    The results guarantee trainability for randomly initialized QCNNs and also establish trainability for the pooling-only QCNN case.

Abstract

from arXiv · show

Quantum neural networks (QNNs) have generated excitement around the possibility of efficiently analyzing quantum data. But this excitement has been tempered by the existence of exponentially vanishing gradients, known as barren plateau landscapes, for many QNN architectures. Recently, Quantum Convolutional Neural Networks (QCNNs) have been proposed, involving a sequence of convolutional and pooling layers that reduce the number of qubits while preserving information about relevant data features. In this work we rigorously analyze the gradient scaling for the parameters in the QCNN architecture. We find that the variance of the gradient vanishes no faster than polynomially, implying that QCNNs do not exhibit barren plateaus. This provides an analytical guarantee for the trainability of randomly initialized QCNNs, which highlights QCNNs as being trainable under random initialization unlike many other QNN architectures. To derive our results we introduce a novel graph-based method to analyze expectation values over Haar-distributed unitaries, which will likely be useful in other contexts. Finally, we perform numerical simulations to verify our analytical results.

I. Introduction

Quantum neural networks can suffer exponentially vanishing gradients that make optimization unscalable. This paper analyzes QCNNs, which use convolutional and pooling layers to reduce degrees of freedom while preserving relevant input features, and argues that their gradient variance vanishes at most polynomially under stated assumptions.

  • Motivation: Exponential gradient decay in randomly initialized quantum circuits creates barren plateaus and can require exponentially precise measurements for optimization.Such landscapes render the architecture unscalable.
  • QCNN architecture: QCNNs alternate convolutional layers with pooling layers that reduce the number of degrees of freedom while preserving relevant input-state features.The architecture can include a final fully connected layer and produces an output state in a much smaller Hilbert space.
  • Scope and assumptions: The analysis assumes independent, uncorrelated 2-design unitaries and a cost function linear in the input density matrix.The cost framework includes classification as an application.
  • Approach: The paper introduces GRIM to analyze Haar-distributed-unitary expectation values and uses it to study QCNN gradient scaling.The paper also presents numerical simulations to verify the analytical results.
  • QCNN architecture: QCNN pooling measures qubits and uses the outcomes to control unitaries on neighboring qubits, producing nonlinearities and progressively smaller representations.The final output is used to measure an operator expectation value.

C. Ansatz

The QCNN ansatz uses parametrized two-qubit unitary blocks for convolutional and fully connected layers, interleaved with pooling modules that map two qubits to one.

  • Ansatz setting: The analysis takes n = 2^k and L = log(n) = k, with a two-dimensional output Hilbert space.This is the simplifying setting used for the subsequent analysis.
  • Unitary blocks: Convolutional and fully connected layers use two-qubit parametrized blocks acting on neighboring qubits, with block placement indexed by layer and position.The generalized description includes the usual QCNN as a special case when blocks within a layer are identical.
  • Unitary blocks: Each two-qubit block is expanded into parameterized gates e^(-iθ_ηH_η) and unparameterized gates such as CNOTs.When H_η has two distinct eigenvalues, derivatives can be evaluated with the parameter-shift rule.
  • Pooling: Pooling modules map a two-qubit Hilbert space to one qubit by measuring one subsystem and applying a controlled unitary before tracing it out.The pooling operators are represented as unitary operators within the construction.

D. Trainability and variance of the cost

Trainability is assessed through the variance of cost gradients under random initialization. The analysis uses local 2-design assumptions and shows that pooling operators can be absorbed into neighboring unitary integrations when evaluating this variance.

  • Gradient variance: Gradient trainability is analyzed through the variance of a cost-function partial derivative over randomly initialized circuit parameters.Under standard assumptions, the mean partial derivative is zero.
  • Gradient variance: Exponentially small gradient variance implies exponentially small gradients and precision requirements, whereas larger variance indicates no barren plateau.This connects variance scaling directly to trainability under random initialization.
  • Derivative construction: The derivative is expressed by decomposing the QCNN around the selected unitary as V = V_R W V_L and the selected block as W = W_B W_A.For idempotent H_μ, the mean derivative is zero.
  • Haar integration: The analysis assumes that the sets of convolutional and fully connected unitaries form independent local 2-designs.A 2-design matches Haar averages for polynomials of bounded degree in unitary matrix elements and their conjugates.
  • Haar integration: Pooling operators can be absorbed into neighboring convolutional unitaries because their action does not change the gradient-variance calculation under the 2-design assumption.The tensor-network representation therefore focuses on two-qubit unitary blocks and traces out discarded qubits.

III. Main results

The paper introduces GRIM, a graph-based method that integrates groups of QCNN unitaries recursively to analyze gradient-variance scaling. Applied to QCNN light-cones, the method produces recursive graphs whose paths provide computable contributions and lower bounds.

  • GRIM method: GRIM integrates groups of Haar-distributed unitaries recursively, avoiding the exponentially growing term count of sequential integration.The method forms a graph by grouping modules and recursively organizing contraction terms.
  • GRIM method: The forward light-cone of a unitary W can be covered by center, edge, and middle modules, with each module defining graph nodes and weighted directed edges.For any W in layer ℓ, the light-cone can be covered with ℓ basic modules; edge coefficients satisfy λ_i,j ∈ (0, 1).
  • Gradient-variance analysis: GRIM expresses expectation-value contributions as sums over weighted paths in the graph, and any single path supplies a lower bound for the gradient variance.The path weights are products of positive edge coefficients, while ε_O and ε_eσ_w are positive.
  • Edge placement: For an edge unitary, the light-cone uses (ℓ−1) edge modules and one center module, yielding a graph with ℓ+1 nodes.The resulting graph structure is recursive, allowing construction for arbitrary ℓ.
  • Center placement: For a center unitary, ℓ center modules generate a recursive tensor-contraction structure whose integrated graph contains a single node.Integrating a center module produces the self-loop coefficient λ_1,1 = 28/125.

B. Trainability of the QCNN

Under independent 2-design assumptions, Theorem 1 and Corollary 1 bound QCNN gradient variance using GRIM. Because QCNN depth is O(log(n)), the variance vanishes at most polynomially, ruling out barren plateaus under the stated conditions.

  • Theorem 1: Theorem 1 uses GRIM to compute a lower bound on the variance of a QCNN cost-function derivative for a unitary in layer ℓ.The result assumes the convolutional and fully connected unitaries form independent 2-designs.
  • Corollary 1: At most polynomial decay: Corollary 1 states that Var[∂_µC] vanishes polynomially with n when L is O(log(n)).The logarithmic depth follows from reducing the number of qubits at each QCNN layer.
  • Implication: Polynomial precision suffices to identify a cost-minimizing direction, so the QCNN landscape has no barren plateau under Corollary 1’s conditions.Barren-plateau landscapes instead require exponentially large precision to navigate.
  • Conditions: The bound requires Tr[H^2]ε_Oε_σw ∈ Ω(1/poly(n)), meaning the observable and reduced input operator are not exponentially close to scaled identities.Operators close to the identity are difficult to use for information extraction or training.

C. Trainability from pooling

The pooling-based QCNN is analyzed in a special case with trivial convolutional unitaries and pooling-induced measurements. Theorem 2 shows that its expected gradient magnitude vanishes polynomially, so it does not exhibit a barren plateau.

  • Setup: The pooling-based analysis sets all convolutional unitaries to the identity and models pooling layers through the specified channel equations.The input is assumed to be ρ^(0) = |0⟩⟨0|^⊗n with n = 2^L.
  • Pooling architecture: After each pooling layer, half the qubits are measured, with outcomes controlling neighboring unitaries in the schematic circuit.In this special case, convolutional and fully connected unitaries act trivially.
  • Theorem 2: Theorem 2 gives the expected magnitude of the cost-gradient derivative for any parameter in a pooling module.The result applies to the pooling-channel construction specified by the theorem.
  • Conclusion: Polynomial decay: ⟨|∂_kC|⟩ vanishes polynomially with system size, so the pooling-based QCNN does not exhibit a barren plateau.This establishes trainability for the analyzed pooling-based special case.

D. Numerical Verification

Numerical simulations support the analytical conclusion that QCNN gradient variances decay sub-exponentially, with correlated convolutional unitaries producing larger variances than uncorrelated ones.

  • Simulation setup: The simulations used 200 randomly initialized QCNN instances for each even qubit count from 4 through 26.The derivative variance was evaluated using the parametrized two-qubit ansatz described for the convolutional modules.
  • Numerical results: Correlated convolutional unitaries consistently produce larger gradient variances than uncorrelated unitaries.The simulations compare identical within-layer unitaries with independent unitaries.
  • Numerical results: Sub-linear curves on a logarithmic variance scale indicate sub-exponential scaling with the number of qubits.This numerically verifies that QCNNs do not exhibit barren plateaus.
  • Additional theoretical check: The analytical lower-bound framework is extended to the pooling-only QCNN, whose convolutional unitaries are trivial identity operators.Theorem 2 establishes trainability for this pooling-based case under random initialization.
  • Scope and extensions: The authors identify broader applicability to non-2-design gates and linear cost functions, while leaving more general cost functions for future work.They specifically mention mean-square costs used in regression as an open extension.

Appendix 1: Preliminaries

The appendix establishes notation and mathematical tools for Haar integration, tensor operators, partial traces, and trace-distance inequalities used in the QCNN analysis.

  • Haar preliminaries: A t-design reproduces Haar averages for polynomials with bounded degree in unitary matrix elements and their conjugates.The Haar measure supplies the reference integral over the unitary group.
  • Haar preliminaries: Weingarten calculus provides symbolic formulas for the first two moments of Haar-distributed unitary matrix elements.The calculations also use the Random Tensor Network Integrator package for symbolic Haar integration.
  • Operator identities: The appendix states operator identities for tensor-product Hilbert spaces involving partial traces and products of basis operators.These identities support the manipulation of expectation-value terms in the variance calculation.
  • Norm inequalities: Trace distance is monotone under partial trace, and the presented proof applies to arbitrary matrices rather than only density matrices.The trace distance is defined using the Schatten 1-norm.
  • Norm inequalities: Additional lemmas relate Hilbert–Schmidt and trace norms and provide identities used to bound the variance expressions.The proofs invoke Schatten-norm equivalence, monotonicity, and Cauchy–Schwarz inequalities.

Appendix 2: Proof of Theorem 1

The proof of Theorem 1 decomposes the QCNN light cone into modules, integrates their Haar-random unitaries, and expresses the gradient variance through graph-related coefficients.

  • Theorem 1: Theorem 1 gives a GRIM-computable lower bound on the variance of a QCNN cost-function derivative when convolutional and fully connected unitaries form independent 2-designs.The bound applies to a parameter in a unitary located in the first sub-layer of a QCNN layer.
  • Light-cone decomposition: The circuit is separated into gates inside the forward light cone of the differentiated unitary and all remaining gates.The derivative is then analyzed through contributions from the two resulting circuit regions.
  • Module decomposition: The GRIM groups forward-light-cone unitaries into center, middle, and edge modules, with three coverage cases depending on the differentiated gate's position.The final module is always a center module, while the gate may belong to a module or lie outside them.
  • Module integration: Haar integration expands each module's contribution into operator terms indexed by subsets of qubits and associated real coefficients.The coefficients characterize the modules and the resulting graph nodes and edges.
  • Coefficient normalization: The appendix fixes coefficient freedom by setting cβ,∅ to 1 or −1 and requiring nonnegative a1,β,s′, making graph edge coefficients positive.This normalization removes a rescaling ambiguity in the graph representation.

3. Construction of the graph

The graph-construction procedure recursively converts module integrations into an oriented graph whose weighted paths yield a lower bound for the gradient-variance expression.

  • Graph construction: The GRIM graph is built from nodes representing integrated operator terms and oriented edges representing nonzero module-transition operators.The recursive construction begins at an initial node and proceeds through the QCNN modules.
  • Graph walks: Each graph walk starts at the initial node N1 and ends at a node N1 evaluated at the observable O.The set of all length-ℓ walks is denoted bPℓ.
  • Path evaluation: A walk contributes the product of its edge weights, allowing the expectation value to be expressed as a sum over graph paths.The graph is subsequently simplified by replacing edge operators with positive coefficients λαβ.
  • Graph simplification: Each surviving contraction contributes a factor 1/2^|s| to the simplified edge coefficients, and parallel edges are merged by summing their coefficients.The final lower bound is written using the set of paths Pℓ(Gw) in the simplified graph.

C. Integration over the unitaries in VL via the GRIM

The GRIM reduces expectation-value calculations over a QCNN’s full backward light-cone to an effective light-cone by integrating gates that compile to identity. The construction yields lower bounds applicable across unitary placements.

  • GRIM reduction: Unitaries outside the backward light-cone compile to identity, so the expectation value depends only on gates within that cone.The full-light-cone unitary V_LB can therefore replace the complete circuit for this calculation.
  • GRIM reduction: The effective backward light-cone eL_B is formed by repeatedly removing gates outside it while retaining a reduced set of modules.Its structure depends on the placement of W and can include repeated center or middle modules.
  • W in the first sub-layer: For W in the first sub-layer of layer ℓ, eL_B can be covered by (L − ℓ) center modules M_C.The first module is integrated explicitly before the procedure is repeated recursively.
  • W in the second sub-layer: For W in the second sub-layer, the unitary adjacent to W is integrated first as W_init, after which the remaining light-cone unitaries are grouped into middle modules M_M.This applies both when W is at the edge and when it is not at the edge of the sub-layer.

4. W in the second sub-layer but not in the edge

When W lies in the second sub-layer away from its edge, the light-cone analysis adds an initial integration and then uses middle-module recursions. The resulting graph argument preserves polynomial lower-bound scaling.

  • 4. W in the second sub-layer but not in the edge: For an interior second-sub-layer W, one adjacent unitary must be integrated before the remaining unitaries can be grouped into middle modules M_M.This is the analogue of the edge case, with a modified initial step.
  • General bound: The general lower bound is obtained by taking the minimum of the bounds associated with the possible placements of W.The bound uses the reduced state on W or on the initial unitary, depending on the sub-layer.
  • Graph analysis: The graph analysis considers three light-cone coverings: center modules, edge modules plus a center module, or middle and edge modules plus a center module.These cases collectively cover the relevant light-cone structures.
  • Case 1: Ω(1/ poly(n)) lower-bound scaling holds for the first-sub-layer case when L ∈ O(log(n)).The graph has a single path through G_w in this case.
  • Case 1: Ω(1/ poly(n)) scaling is recovered for W in the second sub-layer when L ∈ O(log(n)).Adding the second-sub-layer module modifies the bound but not its polynomial scaling.
  • Case 2: All graph-integral coefficients are positive, allowing a path whose coefficient is a product of factors in (0, 1).Using the smallest factor λ_min gives a polynomial lower bound when L ∈ O(log(n)).

C. Case 3

Case 3 connects middle- and edge-module graphs to analyze mixed light-cone structures. Its coefficients remain nonnegative, yielding polynomial lower-bound scaling and supporting trainability in the analyzed regime.

  • Coefficient analysis: λα,β ⩾ 0 for all α and β in the middle-module graph.The coefficients are built from recursively defined quantities a_k, b_k, p_k, and q_k.
  • C. Case 3: The transition graph modifies the edge graph by replacing N1 and setting λ1,4 = 0.Middle-graph nodes connect to N2 and N4, while the middle-graph N2 also connects to the updated edge node.
  • Scaling result: Ω(1/ poly(n)) lower-bound scaling follows when L ∈ O(log(n)) for the mixed middle-edge case.A suitable path has coefficient Λ_g equal to a product of factors λ_i in (0, 1), bounded using λ_min.
  • Scaling result: The polynomial lower-bound result extends to W in the second sub-layer, and Corollary 1 holds for all possible choices of W.This completes the case analysis for the gradient-scaling argument.
  • Trainability from pooling: For sufficiently deep pooling, the derivative magnitude is bounded away from zero in expectation for all n.A single pooling layer instead exhibits a barren plateau as n grows, whereas depth j = log n = L avoids it.
Loading 2011.02966v2…