Source-linked AI summary
Quantum Entanglement in Deep Learning Architectures
Yoav Levine, Or Sharir, Nadav Cohen, Amnon Shashua
TL;DR
Tensor Networks cannot represent duplicating a vector for use in two separate calculations. The paper analyzes overlapping convolutional arithmetic circuits and recurrent-network entanglement scaling, deriving rank and entropy bounds while showing that pooling diminishes entanglement capacity.
Problem
Tensor Networks cannot represent the operation of duplicating a vector into two separate nodes for distinct calculations.
Method
The paper constructs and analyzes Tensor Network equivalents of convolutional arithmetic circuits, using receptive fields, strides, and pooling operations to derive entanglement bounds.
Results
Pooling diminishes overlapping-convolutional networks’ ability to represent volume-law entanglement, while sufficiently large receptive fields yield rank lower bounds and recurrent networks attain entropy scaling of Ω(log |A|).
Takeaways & Limitations
Entanglement capacity depends on information reuse through receptive fields and is constrained by pooling and total stride.
Abstract
from arXiv · showhide
Modern deep learning has enabled unprecedented achievements in various domains. Nonetheless, employment of machine learning for wave function representations is focused on more traditional architectures such as restricted Boltzmann machines (RBMs) and fully-connected neural networks. In this letter, we establish that contemporary deep learning architectures, in the form of deep convolutional and recurrent networks, can efficiently represent highly entangled quantum systems. By constructing Tensor Network equivalents of these architectures, we identify an inherent reuse of information in the network operation as a key trait which distinguishes them from standard Tensor Network based representations, and which enhances their entanglement capacity. Our results show that such architectures can support volume-law entanglement scaling, polynomially more efficiently than presently employed RBMs. Thus, beyond a quantification of the entanglement capacity of leading deep learning architectures, our analysis formally motivates a shift of trending neural-network based wave function representations closer to the state-of-the-art in machine learning.
A.“No-Cloning” in Tensor Networks
The appendix formalizes why tensor networks cannot duplicate an arbitrary vector, a capability routinely used by overlapping convolutional computations.
- A.“No-Cloning” in Tensor Networks: A tensor-network operation cannot duplicate an arbitrary vector into two separate identical degree-1 nodes.The claim is established by contradiction using standard-basis vectors and the all-ones vector.
- A.“No-Cloning” in Tensor Networks: Cloning standard-basis vectors would force the tensor entries to equal a delta tensor.For basis vector e^(α), the required output is nonzero only when both copied indices equal α.
- A.“No-Cloning” in Tensor Networks: The delta-tensor construction fails on the all-ones vector, contradicting universal duplication.Its contraction produces δ_jk rather than the required all-ones outer product.
B. Entanglement Scaling in Overlapping-Convolutional Networks
The appendix defines overlapping convolutional architectures and analyzes how receptive fields, stride, and pooling determine their entanglement capacity.
- B. Entanglement Scaling in Overlapping-Convolutional Networks: A generalized convolutional layer combines a 1×1 linear operation with spatial pooling or decimation.Its input and output are channel tensors, with each output location formed from a receptive-field slice of the input.
- B. Entanglement Scaling in Overlapping-Convolutional Networks: An overlapping ConvAC uses product pooling, whereas average pooling followed by activation yields a common all-convolutional network.Overlap refers to receptive fields overlapping when K > 1; the non-overlapping case has K = 1.
- B. Entanglement Scaling in Overlapping-Convolutional Networks: Pooling layers diminish the network’s ability to represent volume-law entanglement by reducing the relevant scaling factor from KL to K.The stated result concerns overlapping ConvACs with 2^d pooling operations and P = 2.
- B. Entanglement Scaling in Overlapping-Convolutional Networks: The entanglement lower bound is controlled by the first layer whose total receptive field spans at least half the input, with total stride limiting index-pair density.A sufficiently large receptive field supports identity matrices between paired indices across the bipartition.
- B. Entanglement Scaling in Overlapping-Convolutional Networks: For unit-stride layers, the total receptive field after ˜L layers is (K −1) · ˜L + 1.The analysis uses all convolutional layers before global pooling, or only the first layer when convolution alternates with pooling.
C. Entanglement Scaling in Deep-Recurrent Networks
The appendix proves an entanglement-scaling result for deep recurrent arithmetic circuits using prior expressiveness bounds.
- C. Entanglement Scaling in Deep-Recurrent Networks: The proof transfers a lower bound from deep recurrent expressiveness results to a bipartition with unequal subsystem sizes.It matches the final |A| indices of B to A and fixes the remaining |B| − |A| indices.
- C. Entanglement Scaling in Deep-Recurrent Networks: For fixed hidden-state dimension R, the resulting lower bound grows as Ω(log {|A|}).The underlying quantity is polynomial in |A|, so its logarithm has the stated asymptotic growth.