PaperlayerView exact revision on arXiv

Source-linked AI summary

Representation Learning with Quantum Signal Processing

Junqi Wang, Junyu Liu

arXiv:2608.28828v1quant-phcs.AIcs.LGstat.ML

TL;DR

Frozen-kernel models retain fixed similarity geometry, while random kernels that evolve during training present a distinct challenge. The paper analyzes QSP's initialization geometry and full training dynamics, finding persistent diagonal randomness, sparse-regime integrable flows, and a speed limit that applies without sparsity.

  • Problem

    Frozen-kernel models keep similarity features fixed, while random initial kernels that continue evolving during training pose a distinct challenge.

    Source: INTRODUCTION · p. 1

  • Method

    The paper analyzes QSP initialization geometry and full gradient-flow dynamics, deriving a frame-based account and a sparse-regime, time-dependent kernel closure.

    Source: DISCUSSION AND OUTLOOK · p. 4

  • Results

    The diagonal QNTK remains realization-dependent despite Haar-random unitary mixing, while sparse full-learner trajectories follow integrable scalar flows with evolving kernels and explicit convergence times.

    Source: DISCUSSION AND OUTLOOK · p. 4

  • Takeaways & Limitations

    QSP yields a controlled account of learned quantum data geometry beyond frozen-kernel dynamics, with sparse-regime guarantees and a speed limit that holds without sparsity.

    Source: DISCUSSION AND OUTLOOK · pp. 4–5

    Two higher-density numerical runs did not meet the convergence threshold before the 5 × 10^5-step cap, so no higher-density convergence-time ratios are reported.

    Source: NUMERICAL PROTOCOL · p. 24

Abstract

from arXiv · show

Representation learning begins when training changes the features that define similarity between data. A frozen-kernel model only reweights a fixed geometry. We establish quantum signal processing (QSP) as a solvable quantum model of the representation-learning regime. At arbitrary depth, we compute the exact mean and variance of its quantum neural tangent kernel, revealing an input-dependent angular geometry whose diagonal remains non-self-averaging even when the underlying unitary approaches Haar randomness. We also prove a sparse-data guarantee for the full nonlinear gradient flow without freezing or ensemble-averaging the kernel: the realized dynamics converges to an integrable scalar flow with a time-dependent kernel closure and explicit convergence times. A finite-depth speed limit holds for every data set and trajectory. At higher data density, numerical results show coupled evolution beyond both the scalar and frozen-kernel descriptions. These results give a controlled theory of learned quantum data geometry with provable training dynamics beyond the frozen limit.

INTRODUCTION

The paper treats QSP as a tractable setting where training changes tangent-feature geometry rather than merely reweighting fixed features. It derives the initialization geometry and proves sparse-data trajectory dynamics beyond the frozen-kernel regime.

INTRODUCTION

  • Unlike frozen-kernel learning, this setting studies a random initial kernel that evolves with residuals, addressing guarantees beyond lazy or frozen limits.The paper frames the initial kernel as a quenched random function of the data indices.Source: INTRODUCTION · p. 1

QSP AS A REPRESENTATION-LEARNING MODEL

  • In QSP, trainable phases define tangent features whose Gram matrix is the data-indexed QNTK, and gradient flow changes this geometry over time.A frozen-kernel model instead keeps the geometry fixed at its initial value.Source: QSP AS A REPRESENTATION-LEARNING MODEL · p. 2

RANDOM QUANTUM DATA GEOMETRY

  • The mean QNTK has angular locality and Chebyshev eigenfunctions; its normalized diagonal approaches Beta(3/2, 1/2), with coefficient of variation tending to 1/3.The diagonal remains realization-dependent even as depth grows.Source: RANDOM QUANTUM DATA GEOMETRY · pp. 2–3, Figure 1 · p. 3

PROVABLE REPRESENTATION LEARNING FOR SPARSE DATA

COUPLED QUANTUM DATA GEOMETRY AT HIGHER DENSITY

In sparse data, QSP follows a realization-matched scalar flow, while overlapping tangent neighborhoods produce coupled dynamics that depart from both scalar and frozen-kernel predictions. The discussion connects these regimes to persistent kernel randomness, a universal speed limit, and an angular data geometry.

COUPLED QUANTUM DATA GEOMETRY AT HIGHER DENSITY

  • When cross-input kernel contributions are no longer subleading, changes at one input affect other outputs and prevent reduction to independent scalar equations.The model then exhibits coupled, time-dependent training in which the learned representation changes both self-similarity and communication between inputs.Source: COUPLED QUANTUM DATA GEOMETRY AT… · p. 4

DISCUSSION AND OUTLOOK

  • At every fixed interior input, increasing QSP depth randomizes the unitary while the normalized diagonal QNTK remains realization dependent.The frame picture attributes this persistence to isotropic tangent axes and a random output amplitude that controls tangent-feature size.Source: DISCUSSION AND OUTLOOK · p. 4
  • In the sparse regime, the full QSP trajectory has a time-dependent diagonal-kernel closure and reduces to integrable nonlinear flows with explicit convergence times.The complementary speed limit holds without sparsity, randomness, or asymptotics.Source: DISCUSSION AND OUTLOOK · pp. 4–5

COUPLED QUANTUM DATA GEOMETRY AT HIGHER DENSITY

  • At (d, n, w) = (1024, 4, 0.125), median tsc/tQSP = 0.97 and tK(0)/tQSP = 1.04; at (128, 64, 5.66), QSP departs both predictions.The comparisons are with the realization-matched scalar flow and the realized kernel frozen at K(0).Source: COUPLED QUANTUM DATA GEOMETRY AT… · p. 4, Figure 2 · p. 5

DISCUSSION AND OUTLOOK

  • The QSP encoding turns x = cos θ into angular space with resolution d^-1/2, Chebyshev eigenmodes, and a depth-dependent spectral bias.The density w = n/d counts observations in one tangent neighborhood; sparse observations mainly change their own local metric, while overlap can reshape shared features and transmit error.Source: DISCUSSION AND OUTLOOK · p. 5
  • QSP is presented as a minimal quantum model with random, time-dependent, and provably tractable learned data geometry.The discussion identifies varying overlap, mode-resolved generalization, scalar-sector regularization, and extensions to irregular grids and higher-dimensional encodings as future directions.Source: DISCUSSION AND OUTLOOK · p. 5

TWO-COPY TRANSFER AND THE EXACT MEAN KERNEL

The two-copy phase average yields the exact mean-kernel transfer structure. Derivative insertions and charge projection reduce the calculation to a small balanced subspace, while four-copy matching supplies the sectors used for the second moment.

Derivative insertions

  • Averaging over ϕ0 removes the holomorphic-holomorphic and antiholomorphic-antiholomorphic derivative products, leaving two conjugate mixed terms with an overall factor of 1/2.The differentiated phase contributes matching ket and bra charges, separating two powers of the same transfer matrix.Source: Derivative insertions · p. 7, Charge projection · p. 8

Charge projection

  • The balanced two-copy transfer has eigenvalues cos(α − β) and cos(α + β), and its direct term has angular correlation length O(d^-1/2).The powers decay exponentially for fixed interior inputs unless their angular arguments approach a multiple of π.Source: Charge projection · p. 8

Six phase-matched sectors

  • Of 24 conjugation patterns in the four-derivative average, only six satisfy the zero-total-charge condition.Two surviving sectors give XXYY contractions and four give XYXY contractions.Source: Six phase-matched sectors · pp. 8–9

The explicit six-dimensional transfer

  • The six-dimensional transfer has eigenvalues including 1, 2x^2 − 1, and 2y^2 − 1, with the other XYXY sectors sharing its spectrum.The two XXYY sectors are evaluated using the same projected-transfer construction.Source: The explicit six-dimensional transfer · p. 9

Double sum

The double-sum calculation gives the exact finite-depth second moment and identifies an O(d^2) contribution from unit transfer eigenvalues. The frame-coordinate formulation also exposes conserved structure in QSP dynamics.

Double sum

  • The exact finite-depth second moment includes all phase matching and derivative positions.Its ordered site sums are evaluated spectrally, with coincident eigenvalues handled by continuous limits.Source: Double sum · p. 10
  • Two unit eigenvalues generate a quadratic site sum and hence an O(d^2) raw second moment.The stated large-d expansion holds for each fixed interior x but is not uniform up to the endpoints.Source: Double sum · p. 10

Exact frame dictionary

  • In frame coordinates, the QSP dynamics are rewritten as coupled evolution equations for the output, companion field, and axes.The axis equation is ˙naj = 2naj × ha,<j.Source: Exact frame dictionary · p. 11
  • Each frame matrix evolves by conjugation, preserving its fixed spectrum and unit-axis constraint, while adjacent bond angles are exactly conserved.The bond-angle conservation follows from cancellation of the two terms in its time derivative.Source: Exact frame dictionary · p. 11

Fixed-input Haar limit

At interior inputs, the random-step support generates the ingredients needed for the stated Haar-limit argument. Frame isotropization yields a limiting distribution for the kernel quantity, with distinct behavior at the boundary input.

Fixed-input Haar limit

  • At interior inputs, the step support generates both tori and their commutator generates su(2), making the step law adapted and aperiodic for the Itō–Kawada theorem [24, 25].The independent left factor does not alter the limit.Source: Fixed-input Haar limit · p. 11

Frame isotropization and contraction

SPECTRUM OF THE ENSEMBLE-MEAN KERNEL

The ensemble-mean kernel is a finite-rank operator with a parity-matched Chebyshev-mode spectrum. In the sparse regime, isolated inputs follow scalar equations with explicit solution and convergence-time characterizations.

SPECTRUM OF THE ENSEMBLE-MEAN KERNEL

  • The ensemble-mean kernel has a finite spectral expansion in parity-matched modes, with nonzero eigenvalues indexed by m = d, d − 2, . . . ≥ 0.Higher-frequency modes in the parity-matched basis have eigenvalue zero.Source: SPECTRUM OF THE ENSEMBLE-MEAN KERNEL · pp. 13–14
  • Although its exact rank is D = O(d), the kernel’s binomial spectral weight is concentrated on only O(d) eigenfunctions.This effective bandwidth corresponds to angular correlation length O(d^−1/2).Source: SPECTRUM OF THE ENSEMBLE-MEAN KERNEL · p. 14

SCALAR FLOW AND CONVERGENCE TIMES

  • In the sparse limit, isolated inputs satisfy independent scalar equations, and separation of variables gives an exact implicit solution.For endpoint targets, the solution is expressed through z + log z, while the endpoint residual decays algebraically rather than exponentially.Source: SCALAR FLOW AND CONVERGENCE TIMES · pp. 14–15
  • For a single zero-target input, the squared-residual reduction time has explicit bounds; for multiple inputs, the loss-threshold time is determined implicitly under a uniform interior condition.The scalar convergence time is defined by L(τq) = qL(0).Source: SCALAR FLOW AND CONVERGENCE TIMES · p. 15

RIGOROUS SPARSE LIMIT AND CONVERGENCE-TIME CONTROL

The sparse-limit proof separates scalar output dynamics from frame fluctuations and controls those fluctuations along the training trajectory. A separate finite-depth bound constrains loss decay for every data set and trajectory.

Universal speed limit

  • For every data set and trajectory, the exact loss identity and bounded phase derivatives yield a finite-depth constraint on relative-loss convergence time.Each phase derivative is bounded by one, and −˙L ≤ 2ML.Source: Universal speed limit · p. 15

Control equation and midpoint grid

  • The sparse-limit control equation separates the scalar flow from diagonal-frame fluctuations and off-diagonal coupling.The proof bounds diagonal terms through prefix anisotropies and off-diagonal terms through mixed-input sums, using initialization and deterministic transport estimates.Source: Control equation and midpoint grid · p. 16

Diagonal-frame anisotropy

  • Diagonal-frame anisotropy is controlled using a matrix-valued martingale and a maximal inequality, alongside transport bounds along the trajectory.The midpoint grid identities and the anisotropy bounds support the uniform control over inputs and prefixes.Source: Diagonal-frame anisotropy · pp. 16–17

Off-diagonal frame sums

The off-diagonal frame-sum analysis evaluates mixed-input correlations jointly because both inputs share the same random phase. It reduces the paired transfer to a two-dimensional invariant subspace and bounds the resulting sums.

Off-diagonal frame sums

  • Because the two inputs share a random phase, their joint axis correlation must be evaluated before averaging.The paired transfer is constructed in Bloch space, and phase averaging projects onto vectors invariant under simultaneous z-rotations.Source: Off-diagonal frame sums · p. 19
  • Only the two-dimensional subspace spanned by E0 and E1 contributes to the correlation, since the antisymmetric vector E2 is decoupled.The correlation begins and ends in the E0 = zz direction.Source: Off-diagonal frame sums · p. 19
  • The pairwise transfer bound is summed over the midpoint grid, and the resulting mixed-input frame sums are controlled at initialization and during transport.The transport argument preserves operator norms under the orthogonal rotations and uses strict-prefix bounds.Source: Off-diagonal frame sums · pp. 20–22

Forcing and finite-time trajectory control

The argument compares the full QSP flow with a scalar reference initialized at the same output and bounds finite-time control using a Grönwall estimate and remainder terms.

Sparse-limit theorem

The theorem sets conditions for the sparse-limit analysis, while the proof controls finite-time errors and tracks a diagonal kernel that changes with the learned output.

  • Choosing ε_s and ε_c appropriately makes Prob.(G_M,n)→1 and β_T→0 on G_M,n.Source: Sparse-limit theorem · p. 23
  • For M=d+1→∞, the theorem assumes the Eq. (S89) grid, independent uniform initial phases, ||f||∞≤1, exact continuous gradient flow, and Eq. (S168).Source: Sparse-limit theorem · p. 23
  • The leading diagonal of the tangent kernel changes with the learned output throughout the controlled time interval.Source: Sparse-limit theorem · p. 23
  • The n^3/2(1+log n) factor in Eq. (S168) combines midpoint-grid mixed-input prefixes with their transport over a fixed rescaled-time interval.Source: Sparse-limit theorem · p. 23

Transfer of convergence times

A local slope condition for the scalar loss lets the analysis transfer threshold-crossing times to the full flow. The numerical protocol compares trajectories with scalar and frozen-kernel references and reports limits on high-density convergence-time measurements.

Transfer of convergence times

  • If the scalar loss crosses qL(0) with a locally bounded negative slope, sufficiently small loss error transfers the crossing to the full QSP flow nearby.Source: Transfer of convergence times · p. 24

NUMERICAL PROTOCOL

  • Each numerical setting uses 30 independent uniform-phase initializations, paired with independently drawn degree-8 QSP teachers on the same grid.Source: NUMERICAL PROTOCOL · p. 24
  • The displayed trajectory is the converged run closest to the median convergence time, and the shaded band gives the pointwise interquartile range over 30 trajectories.Source: NUMERICAL PROTOCOL · p. 24
  • Two higher-density runs remain above the 10^-4 loss threshold at the 5×10^5-step cap, so no higher-density convergence-time ratios are reported.Source: NUMERICAL PROTOCOL · p. 24
Loading 2608.28828v1…