PaperlayerView exact revision on arXiv

Source-linked AI summary

QiT: Quantum-Inspired Transformer for Visual Recognition Task

Badri N. Patro, Vijay Agneeswaran

arXiv:2609.17789v1quant-phcs.AIcs.CVphysics.app-ph

TL;DR

Quantum-motivated structure is difficult to evaluate in visual recognition at scale because quantum models face hardware, measurement, and optimization constraints. QiT realizes selected motifs as classical Transformer operations; it scales across image datasets, and the tested four-qubit simulator is much slower than QiT.

  • Problem

    Quantum models face hardware and optimization constraints that make vision-scale evaluation difficult, while the usefulness of quantum-motivated Transformer features remains an open question.

    Source: §1 · p. 1

  • Method

    QiT uses learned periodic token features, attention over their projections, and gated multiplicative channel mixing as classical tensor operations, not a general quantum circuit.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    QiT scales across eight datasets and reaches 78.3% ImageNet-1K top-1 accuracy; the tested four-qubit PennyLane implementation is 882–997× slower in wall-clock time than QiT.

    Source: §4.5 · p. 8, §5 · p. 8

  • Takeaways & Limitations

    The evidence establishes execution feasibility and measured cost, but does not establish that quantum-motivated components outperform a converged, parameter-matched Transformer.

    Source: §4.5 · p. 8, §5 · p. 8

    Reported results are point estimates without repeated seeds or uncertainty intervals, so they do not establish statistical superiority.

    Source: §4.1 · p. 6

Abstract

from arXiv · show

Quantum machine learning offers a compelling representational perspective: angle-encoded states inhabit Hilbert spaces in which periodic similarities and interactions can be expressed naturally. Realizing this perspective for visual recognition remains difficult, however, because present quantum neural networks are constrained by limited qubit counts, costly circuit simulation and measurement, noise, and unstable optimization on noisy intermediate-scale quantum devices. We investigate whether useful structural ideas from quantum models can instead be realized as scalable classical Transformer operations. We introduce QiT, a Quantum-inspired Transformer for vision tasks with three components: (i) angle-inspired encoding that maps image tokens to learned trigonometric Hilbert-space features analogous to quantum rotation-based state encoding; (ii) self-attention over these periodic features, inducing a classical cosine kernel approximated to quantum fidelity kernels; and (iii) gated multiplicative emulation, a trainable classical surrogate for interaction terms found in variational circuits. All components are differentiable tensor operations, so QiT claims neither quantum computation nor quantum speedup and retains the $\mathcal{O}(N^2D)$ attention complexity of a standard Vision Transformer. Across image-classification benchmarks, QiT is competitive with a matched classical Transformer while avoiding the severe runtime cost observed for a small simulated quantum Transformer. QiT-B reaches 78.3\% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs. These results position QiT as a scalable baseline for isolating and evaluating quantum-motivated inductive biases in visual recognition.

1. Introduction

QiT asks whether quantum-motivated periodic features and multiplicative interactions can serve as inductive biases in a scalable classical vision Transformer. It implements these ideas with differentiable tensor operations, without quantum hardware or claims of quantum advantage.

1. Introduction

  • Quantum models remain difficult to evaluate at vision scale because of sequential recurrent execution, limited qubits and circuit depth, measurement noise, repeated sampling, and optimization challenges.The introduction notes that controlled large-scale comparisons remain scarce.Source: §1 · p. 1
  • The motivating question is whether periodic angle features and multiplicative interactions provide useful Transformer inductive biases independently of quantum hardware or computational advantage.Standard self-attention relates distant patches, but its linear query and key projections leave nonlinear pairwise structure to emerge through depth and surrounding feedforward blocks.Source: §1 · p. 1
  • QiT retains parallel token processing, standard backpropagation, and the O(N^2D) attention complexity of a Vision Transformer while avoiding circuit simulation and measurement.The design evaluates selected quantum-motivated functional forms with conventional GPU kernels.Source: §1 · p. 2
  • QiT combines learned angle-inspired trigonometric token features, attention over their projections, and gated multiplicative channel mixing in a classical Transformer.The unprojected feature inner product is a cosine kernel; learned query–key scores are bilinear similarities, not quantum fidelity kernels, and the mixer does not instantiate physical entanglement.Source: §1 · pp. 1–2, Figure 1 · p. 2
  • 78.3% ImageNet-1K top-1 accuracy with 45.7M parameters and 11.5 GFLOPs; a controlled one-epoch study reports nearly three orders of magnitude lower runtime than the simulated quantum Transformer.The reported evaluation compares QiT with classical vision models and a four-qubit PennyLane Transformer.Source: §1 · p. 2

2. Related Work

Related work covers quantum learning, sequential quantum models, and quantum attention architectures, whose hardware allocations and evaluation assumptions differ. QiT instead tests selected quantum-motivated forms entirely with classical tensor operations and standard attention.

Quantum Machine Learning

  • Quantum machine learning encodes data into states, applies parameterized circuits, and measures outputs; Hilbert-space dimension alone does not imply an efficiently accessible learning advantage.Practical scale is constrained by qubit availability, gate errors, decoherence, finite-shot estimation, and barren plateaus in particular circuit families and regimes.Source: §2 · pp. 2–3

Quantum Deep Learning for Sequential Data

  • Quantum recurrent models such as QRNNs and QLSTMs [39] process ordered data sequentially, unlike Transformer-style all-pairs token interaction.QiT instead studies parallel image-patch interactions and executes them classically.Source: §2 · p. 3

Quantum Attention and Quantum Transformers

  • Quantum attention methods vary in how much work they assign to circuits: QSANN and QSAM use selected circuit projections, while QViT and QKSAN add circuit subroutines or kernels.Other work explores fully quantum formulations and alternative hybrid allocations; differing assumptions make asymptotic quantities non-uniformly comparable and insufficient to establish end-to-end speedups.Source: §2 · p. 3

Positioning of QiT

  • QiT uses no quantum hardware: it applies learned periodic encoding and circuit-motivated multiplicative interactions as classical tensor operations, followed by standard softmax attention.Its unprojected feature inner product is a cosine kernel, but learned query–key scores are generally neither positive-semidefinite kernels nor quantum-state fidelities; multiplicative gating does not create physical entanglement.Source: §2 · p. 3

3. Quantum-Inspired Transformer Architecture (QiT)

QiT is a classical Transformer that uses periodic token features and gated multiplicative channel mixing motivated by parameterized quantum models, without executing or generally simulating a quantum circuit.

3.1. Background: Classical Multi-Head Self-Attention

  • Classical multi-head attention enables parallel pairwise token interactions but incurs O(L^2) cost, while its standard linear projections lack the feature-space expressivity sought from quantum approaches.Table 1 provides qualitative resource context because the listed quantum-attention designs use different circuit, oracle, sampling, and readout assumptions.Source: §3.1 · p. 3, Table 1 · p. 4

3.2. Token Embedding (Classical Preprocessing)

  • Token embeddings combine value embeddings, positional encodings, and optional temporal covariates using conventional Transformer preprocessing before QiT's quantum-inspired components begin.This preprocessing costs O(BLD) operations.Source: §3.2 · p. 4

3.3. Quantum Angle Encoding

  • QiT maps embeddings to Q bounded angles and trigonometric features, inducing a positive-semidefinite inner-product kernel without equating its feature space to a quantum state space.The classical feature map costs O(BLdQ); the quantum connection is architectural and spectral rather than state-space equivalence.Source: §3.3 · p. 4

3.4. Hilbert-Space Self-Attention

  • Attention projects periodic features into learned queries, keys, and values; the resulting query–key scores are bilinear similarities, not generally kernels or quantum-state fidelities.Attention costs O(BL^2dh) per head, as in scaled dot-product attention.Source: §3.4 · pp. 4–5, Figure 2 · p. 5

3.5. Quantum-Inspired Gated Channel Mixer

QiT uses a classical gated block to introduce multiplicative feature interactions motivated by quantum-circuit operations, without representing a quantum state. Its architecture uses finite classical features and retains token-quadratic attention scaling.

3.5.1. Design rationale

  • Motivated by circuit rotations, entangling gates, and readout, QiT uses a classical gated block to add multiplicative feature interactions without representing a quantum state.The circuit analogy motivates the design; it does not imply a one-to-one correspondence to entangling gates.Source: §3.5.1 · p. 5

3.5.2. Gated multiplicative mixing

  • The Hadamard product creates input-dependent, non-additive feature interactions, with mixer cost O(BLdr) for hidden width r.W1 and Wg map to hidden width r, and W2 projects back to width d.Source: §3.5.2 · p. 5

3.5.3. Architectural interpretation

  • QiT’s angle encoding, gated mixing, and multi-head attention are presented as analogs of rotations, two-qubit gates, and parallel measurements, respectively.The finite classical feature map does not inherit a quantum circuit’s exponential state dimension or full function class.Source: §3.5.3 · pp. 5–6
  • Table 2 is a one-epoch CIFAR diagnostic with patch size 4, one block, and batch size 2; its accuracy values are single-run and non-converged.FLOPs are recorded for CViT and QiT, while FQT-P cost is listed symbolically because it depends on circuit and simulator details.Source: Table 2 · p. 6

4. Experiments

The experiments assess simulator cost, portability across datasets, and ImageNet-1K scaling; they report classical execution feasibility and measured costs, not quantum advantage or superiority of the quantum-motivated components.

4.1. Experimental Setup

  • The studies use separate protocols, and reported point estimates omit repeated seeds and uncertainty intervals, so results document observed accuracy, runtime, and scalability rather than statistical superiority.The evaluation covers reduced-class CIFAR simulator tasks, eight portability datasets, and ImageNet-1K under distinct settings.Source: §4.1 · p. 6, §4 · p. 6

4.4. Scalability to ImageNet

  • QiT-B reaches 78.3% ImageNet-1K top-1 accuracy; contextual comparisons are uncontrolled, and the table reports accuracy, GFLOPs, and parameters across models.QiT-B matches listed ResNet-50 accuracy, exceeds listed ViT-B/16 by 0.4 points, and trails DeiT-B by 3.5 points.Source: Table 4 · p. 7, §4.4 · p. 8

4.2. Comparison with Simulated Quantum Transformers

  • In the one-epoch Q1 protocol, FQT-P is 881.8–996.7× slower than QiT, while QiT is 1.58–1.76× slower than CViT.The runtime gap is between classical tensor execution and this PennyLane simulation; it does not establish a hardware-independent comparison.Source: §4.2 · p. 7
  • QiT’s one-epoch accuracy exceeds FQT-P’s on the reduced-class tasks, but these are optimization snapshots with unmatched parameter counts, not evidence of architectural superiority.Recorded energy is three orders of magnitude above the classical runs, but missing measurement details preclude hardware-independent energy conclusions.Source: §4.2 · p. 7

4.3. Context from Quantum Attention Models

  • QiT uses no quantum resources and retains O(N^2D) attention complexity; measured simulator slowdown cannot be generalized to other simulation methods or quantum hardware.Its reported layer cost is O(N^2D+NQD+NDr), while a dense state-vector simulator stores 2^Q amplitudes.Source: §4.2 · p. 7, §4.3 · p. 7

4.5. Results Across Multiple Benchmarks

  • One QiT configuration trains across eight datasets spanning 32×32 to 224×224 inputs, with 10 to 257 classes and varying training-set sizes.At 32×32, it reports 78.84% on CIFAR-10; at 224×224, accuracy ranges from 65.00% on Flowers-102 to 72.00% on Food-101.Source: §4.1 · p. 6, Table 5 · p. 8, §4.5 · p. 8

4.4. Scalability to ImageNet

  • From XS to S, ImageNet-1K top-1 accuracy rises from 75.0% to 77.4%; scaling from S to B adds 14.7M parameters and 5.9 GFLOPs for a further 0.9-point gain.This is a monotonic but diminishing gain and does not establish optimal scaling.Source: §4.4 · p. 8

4.5. Results Across Multiple Benchmarks

  • Across the three questions, the tested PennyLane implementation is 882–997× slower than QiT, one QiT configuration trains across eight datasets, and QiT reaches 78.3% ImageNet-1K top-1 accuracy.The evidence does not establish that the quantum-motivated components outperform a converged, parameter-matched Transformer.Source: §4.5 · p. 8

5. Conclusion

QiT is a classical vision Transformer using learned periodic token features for attention and gated multiplicative channel mixing. It scales across eight image datasets, while further ablations and repeated runs are needed to determine its advantage over matched classical alternatives.

5. Conclusion

  • 78.3% ImageNet-1K top-1 accuracy with 45.7M parameters; QiT is presented as scaling across eight image datasets.The conclusion reports a controlled four-qubit pilot showing high tested PennyLane simulator cost relative to ordinary tensor operations.Source: §5 · p. 8
  • QiT forms attention projections from learned periodic token features and uses gated multiplicative channel mixing in a classical Transformer.The construction is motivated by angle-encoded quantum models but does not simulate a general quantum circuit or claim quantum advantage.Source: §5 · p. 8
  • Controlled component ablations and repeated runs remain necessary to determine QiT’s advantage over matched classical alternatives.Source: §5 · p. 8

A. Supplementary Material

The supplement provides implementation details and derivations for QiT. Its quantum-related terminology describes the parameterization's motivation, not quantum execution or advantage.

A. Supplementary Material

A.1. QiT Encoder Block

A QiT encoder block transforms patch and positional embeddings using periodic features for attention, followed by gated multiplicative channel mixing. Residual connections, normalization, multi-head composition, and softmax attention remain standard Transformer operations.

A.1. QiT Encoder Block

  • The encoder block takes patch and positional embeddings as its input representations.Source: §A.1 · p. 10
  • Learned angles are mapped to periodic sine-and-cosine features, which are shared by query, key, and value projections.Each attention head has separate learned projection matrices.Source: §A.1 · p. 11
  • The usual channel MLP is replaced by a gated multiplicative map that combines GELU-transformed and sigmoid-gated representations.Source: §A.1 · p. 11
  • Residual connections, normalization, multi-head composition, and softmax attention remain standard Transformer operations.Source: §A.1 · p. 11

A.2. Exact Property of the Periodic Feature Map

QiT’s periodic feature map induces a positive-semidefinite cosine kernel before learned attention projections. The projected query–key score is generally asymmetric and not positive semidefinite, and the cosine construction is not equivalent to quantum fidelity.

A.2. Exact Property of the Periodic Feature Map

  • The feature map uses learned angles and normalized sine-and-cosine features, whose unprojected inner product gives a finite cosine kernel.Source: §A.2 · p. 11
  • The positive-semidefinite guarantee applies to the unprojected feature map, not generally to QiT’s learned query–key score.The independently learned query and key projections make that score generally asymmetric and not positive semidefinite.Source: §A.2 · p. 11
  • QiT’s average cosine kernel differs from the product-state fidelity of independent single-qubit RY encodings, and shared angle periodicity does not establish fidelity equivalence.Entangling circuits introduce additional circuit-dependent terms.Source: §A.2 · p. 11

A.3. Computational Cost

QiT’s block combines angle-based feature construction, attention-head projection, scaled dot-product attention, and a gated channel mixer. Its attention retains the quadratic token-count dependence of standard softmax attention; the operation counts are classical, not quantum runtime bounds.

A.3. Computational Cost

  • Angle projection and feature construction require O(BNdQ) operations and store 2Q features per token.Source: §A.3 · p. 11
  • Projection to attention heads requires O(BNQd) operations, while scaled dot-product attention requires O(BN 2d) operations.Source: §A.3 · p. 11
  • The gated channel mixer requires O(BNdr) operations, and the stated per-block cost combines attention and mixer terms.The supplied expression is BN 2d + BNQd + BNdr.Source: §A.3 · p. 11
  • Attention retains quadratic dependence on token count, as in standard softmax attention.The stated operation counts describe classical computation, not quantum runtime.Source: §A.3 · p. 11

A.4. Experimental Protocol and Interpretation

The paper reports three separate experimental protocols, each interpreted within its own setting. The available records document tested implementation costs and image-classification-scale training, but do not establish statistical superiority over a converged, parameter-matched Transformer or quantum advantage.

A.4. Experimental Protocol and Interpretation

  • Results come from three separate protocols and should be interpreted within their respective settings.Source: §A.4 · p. 11
  • The simulator diagnostic compares implementation costs only for tested configurations, not converged accuracy or parameter-matched models.CViT, QiT, and PennyLane use patch size 4, one encoder block, batch size 2, and one epoch on reduced 2-, 4-, and 10-class CIFAR tasks; QiT uses four learned angles and PennyLane four simulated qubits.Source: §A.4 · p. 11
  • The eight-dataset study trains one QiT configuration for 300 epochs with AdamW, cosine decay, augmentation, and four Tesla T4 GPUs.The configuration uses patch size 4, depth 12, width 256, four attention heads, and 16 learned angles; batch size varies with resolution.Source: §A.4 · p. 11
  • ImageNet-1K comparisons at 224 × 224 provide accuracy–cost context, not controlled pairwise evidence, because training recipes can differ.QiT-XS, QiT-S, and QiT-B are evaluated with parameter counts and FLOPs reported in the main paper.Source: §A.4 · p. 12
  • The reported results are point estimates, and missing run, uncertainty, settings, split, and energy-procedure details limit what they establish.The records support that QiT can be trained at image-classification scale and document measured costs of tested implementations, but not statistical superiority over a converged, parameter-matched Transformer or quantum advantage.Source: §A.4 · p. 12

Related papers

Loading 2609.17789v1…