PaperlayerView exact revision on arXiv

Source-linked AI summary

TensorFlow Quantum: A Software Framework for Quantum Machine Learning

Michael Broughton, Guillaume Verdon, Trevor McCourt, Antonio J. Martinez, Jae Hyeon Yoo, Sergei V. Isakov, Philip Massey, Ramin Halavati, Murphy Yuezhen Niu, Alexander Zlokapa, Evan Peters, Owen Lockwood, Andrea Skolik, Sofiene Jerbi, Vedran Dunjko, Martin Leib, Michael Streif, David Von Dollen, Hongxiang Chen, Shuxiang Cao, Roeland Wiersema, Hsin-Yuan Huang, Jarrod R. McClean, Ryan Babbush, Sergio Boixo, Dave Bacon, Alan K. Ho, Hartmut Neven, Masoud Mohseni

arXiv:2003.02989v2quant-phcond-mat.dis-nncs.LGcs.PL

TL;DR

Quantum machine-learning research needs tooling that reduces the difficulty of developing hybrid models while accommodating noisy quantum processors and classical computation. TFQ integrates Cirq with TensorFlow to support hybrid model design, training, simulation, and hardware execution, and demonstrates these capabilities across multiple quantum-learning tasks. The paper reports applications including phase-transition classification, noise suppression, and approximate quantum-memory training outcomes.

  • Problem

    Exploring hybrid quantum-classical models is difficult and error-prone, while standard quantum architectures face barren plateaus and near-term hardware noise.

  • Method

    TFQ integrates Cirq with TensorFlow, providing abstractions, quantum layers, differentiators, simulators, and custom operations for hybrid model development.

  • Results

    TFQ demonstrates quantum-learning applications including phase-transition classification, fourfold adversarial-swap-test error reduction, and improved accuracy from superposition-based QRAM training.

  • Takeaways & Limitations

    TFQ provides a common TensorFlow-based environment for exploring hybrid quantum-classical models across classical data, quantum data, simulators, and quantum hardware.

Abstract

from arXiv · show

We introduce TensorFlow Quantum (TFQ), an open source library for the rapid prototyping of hybrid quantum-classical models for classical or quantum data. This framework offers high-level abstractions for the design and training of both discriminative and generative quantum models under TensorFlow and supports high-performance quantum circuit simulators. We provide an overview of the software architecture and building blocks through several examples and review the theory of hybrid quantum-classical neural networks. We illustrate TFQ functionalities via several basic applications including supervised learning for quantum classification, quantum control, simulating noisy quantum circuits, and quantum approximate optimization. Moreover, we demonstrate how one can apply TFQ to tackle advanced quantum learning tasks including meta-learning, layerwise learning, Hamiltonian learning, sampling thermal states, variational quantum eigensolvers, classification of quantum phase transitions, generative adversarial networks, and reinforcement learning. We hope this framework provides the necessary tools for the quantum computing and machine learning research communities to explore models of both natural and artificial quantum systems, and ultimately discover new quantum algorithms which could potentially yield a quantum advantage.

I. INTRODUCTION

Quantum machine learning is moving toward hybrid models because near-term quantum processors are small and noisy, while developing such models remains difficult and error-prone. TFQ addresses this by integrating quantum computation with TensorFlow for model design, training, simulation, and eventual hardware execution.

  • A. Quantum Machine Learning: First-generation quantum machine-learning methods target supervised and unsupervised tasks using quantum linear algebra, but classical-data encoding scalability remains under debate.These methods can be exponentially faster for certain quantum-data problems, yet may rely on structure exploitable by classical algorithms.
  • A. Quantum Machine Learning: NISQ-era QML uses parameterized quantum circuits whose parameters are optimized against cost functions through black-box or gradient-based methods.This paradigm defines QML as models, training strategies, and inference schemes built on parameterized quantum circuits.
  • B. Hybrid Quantum-Classical Models: Near-term quantum processors must work with classical co-processors because their limited size and noise constrain purely quantum model training and generalization.The paper frames hybrid quantum-classical models as learning heuristics in which both processors contribute to learning-performance indicators.
  • B. Hybrid Quantum-Classical Models: Hybrid models can be assessed by representation capacity, training efficiency, inference tractability, and generalization power.These attributes cover fitting training correlations, optimization convergence, scalable prediction, and performance on unseen data.
  • B. Hybrid Quantum-Classical Models: Standard architectures place representations on quantum processors, while barren plateaus can hinder optimization through unusually flat energy landscapes.The paper notes that barren plateaus generally arise in unstructured, randomly initialized architectures and can affect gradient-based and gradient-free methods.
  • D. TensorFlow Quantum: TFQ targets the difficult, error-prone engineering of hybrid models by integrating Cirq with TensorFlow and enabling backpropagation across classical and quantum layers.Its interfaces support simulation, TensorFlow and Keras tooling, distributed execution, and eventual execution on quantum processors.

2. The Abstract TFQ Pipeline for a specific hybrid discriminator model

TFQ’s hybrid discriminative pipeline prepares quantum data, evaluates parameterized quantum and classical models, computes a task-specific cost, and updates both parameter sets through TensorFlow-compatible differentiation. A single-qubit binary classifier illustrates this workflow from Bloch-sphere data generation through measurement, SoftMax classification, and optimization.

  • Abstract pipeline: The pipeline prepares quantum datasets as unparameterized Cirq circuits and injects them into TensorFlow’s computational graph.Because current quantum computers cannot import external quantum data, users specify circuits that generate it.
  • Abstract pipeline: Parameterized quantum models extract hidden information from quantum data, while disentangling can expose it through classical correlations.For discriminative learning, the extracted information is the hidden label parameters.
  • Abstract pipeline: Measurements convert quantum states into classical samples or expectation values, with averaging available for variational algorithms.The resulting classical information can be passed to later processing stages.
  • Abstract pipeline: Classical post-processing distills correlations from measured expectations, and TFQ connects quantum models directly to TensorFlow Keras layers.The resulting model can use TensorFlow losses and standard classical neural-network components.
  • Abstract pipeline: The cost function reflects classification accuracy or another task criterion, after which gradient-based updates adjust the pipeline’s free parameters.TFQ exposes quantum-operation derivatives to TensorFlow backpropagation, allowing quantum and classical parameters to be optimized together.
  • Binary quantum classifier: The one-qubit example samples two Bloch-sphere regions, applies a parameterized rotation and Z measurement, then trains a SoftMax classifier with cross entropy and Adam.The example uses θa = 1, θb = 4, and N = 200 for the two-category dataset.

E. TFQ Building Blocks

TFQ supplies TensorFlow-integrated building blocks for representing, composing, executing, and differentiating quantum circuits. Its interfaces support sampling or expectation-based conversion to classical information, simulator or device execution, and multiple quantum-gradient methods.

  • Circuit representation: TFQ converts Cirq circuits, parameterized models, and measurements into TensorFlow tensors for direct use in computational graphs.Serialized circuits and PauliSum objects become tf.string tensors suitable for Keras inputs.
  • Circuit composition: The AddCircuit layer combines injected quantum-data circuits with quantum-model circuits inside the TensorFlow graph.This provides an in-graph operation for composing the data point and parameterized model.
  • Quantum-to-classical interfaces: TFQ exposes sampling and expectation layers to convert quantum information into classical data for performance measurement and loss construction.Sampling produces bitstrings, whereas expectation values are obtained relative to specified measurement operators.
  • Quantum-to-classical interfaces: The Sample layer accepts parameterized circuits, symbols, and substitution values, producing batched ragged tensors of circuit samples.The output shape is [batch_size, num_samples, n_qubits], with ragged qubit dimensions for varying circuit sizes.
  • Quantum-to-classical interfaces: Expectation values are more convenient than sampling for differentiability because their gradient information can be backpropagated.TFQ’s Expectation layer supports execution on either a simulator or a real device, including its native C++ TensorFlow simulator.
  • Quantum differentiation: The differentiators module integrates quantum-circuit derivatives with TensorFlow backpropagation and supports finite-difference and parameter-shift strategies.Users can select built-in differentiators or implement custom ones through the Differentiator interface.

F. High Performance Quantum Circuit Simulation with qsim

TFQ integrates qsim, a high-performance classical quantum-circuit simulator, into TensorFlow using gate fusion and hardware-specific kernels. Benchmarks show substantial speedups over Cirq, especially for sparse circuits.

  • Gate fusion: qsim fuses neighboring gates into larger multi-qubit gates, reducing matrix-vector multiplications during circuit simulation.The fusion size ranges from 2 to 6 qubits, with typical optima depending on thread count.
  • Hardware support: TFQ adapts simulation to available hardware, using optimized CPU instruction sets or compatible CUDA GPUs.The implementation supports SSE2, AVX2, AVX512, and GPU-based simulation.
  • Benchmarks: Sparse circuits show roughly 100 times faster simulation in TFQ than Cirq because fusion reduces the number of gates simulated.The sparse circuits generate entanglement only on small subsets of qubits.
  • Benchmarks: The reported performance advantage includes TFQ serialization and frontend-to-qsim conversion overhead.This comparison therefore measures the integrated software stack rather than qsim in isolation.
  • Noisy simulation: qsim also supports noisy-circuit simulation through Cirq’s common noise channels and trajectory simulation.Trajectory simulation uses less memory at one time and is selected when circuits are not too noisy.

C. Gradients of Quantum Neural Networks

TFQ provides several methods for differentiating quantum neural networks so their parameters can be optimized with gradient-based learning. These include finite differences, analytic parameter shifts, stochastic estimators, and simulator-only adjoint differentiation.

  • Finite differences: Finite-difference methods estimate gradients by evaluating the objective at perturbed parameter settings.Central differences use 2M objective evaluations with O(ε^2) precision, while forward differences use M + 1 queries with O(ε) error.
  • Parameter shifts: Parameter-shift rules obtain analytic expectation-value derivatives by evaluating circuits at shifted parameter values.The required shift can be much larger than numerical-differentiation shifts, potentially reducing the resolution needed per estimate.
  • Stochastic estimation: Analytic gradients can become costly for generators containing many terms because each parameter may require multiple expectation evaluations.The paper motivates stochastic term sampling to reduce this query overhead.
  • Stochastic estimation: TFQ supports singly, doubly, and triply stochastic gradient estimation by sampling parameter appearances, Hamiltonian terms, and gradient components.These sampling choices can be enabled independently for stochastic parameter-shift gradients.
  • Adjoint differentiation: For classically simulatable circuits, adjoint differentiation provides a high-performance gradient method for deep circuits with many parameters.The technique is implemented for simulator backends such as qsim but cannot be performed on quantum hardware.

D. Hybrid Quantum-Classical Computational Graphs

TFQ models hybrid quantum-classical networks as compositions of quantum and classical function blocks. Quantum blocks map parameters to expectation-value vectors, and effective-Hamiltonian gradients enable backpropagation through the composite graph.

  • Hybrid graph structure: Hybrid quantum-classical neural networks are meta-networks whose quantum and classical function blocks form a directed computational graph.The blocks can be composed into general topologies through parameterized functions.
  • Quantum block outputs: A quantum block maps real parameters to a vector of expectation values from multiple observables.Measurement histograms can also be represented as vectors using eigenstate-projector expectation values.
  • Quantum block outputs: Expectation values provide a differentiable interface between quantum circuits and classical neural-network blocks.Informationally complete observables can characterize the wavefunction, but with exponential overhead in qubit number.
  • Scope boundary: TFQuantum’s current implementation excludes direct single-measurement feedback-control scenarios because Cirq does not support the required feature.Expectation-value outputs instead form deterministic nodes through which gradients can be backpropagated.
  • Hybrid backpropagation: Backpropagation through a quantum block contracts its Jacobian with the gradient arriving from subsequent classical layers.The resulting gradient is represented through an effective backpropagated Hamiltonian and then differentiated with respect to quantum parameters.
  • Hybrid backpropagation: In the example graph, a preceding classical DNN produces quantum parameters, while a postceding DNN processes the QNN’s expectation-value output.This realizes sequential composition of classical and quantum blocks within one trainable model.

IV. BASIC QUANTUM APPLICATIONS

TFQ examples use quantum convolution and pooling to compress quantum data, then combine quantum measurements with classical postprocessing for classification. Under matched optimization settings, hybrid classifiers train faster than the purely quantum model.

  • Quantum classification: The HQCNN applies one truncated QCNN layer, extracts statistics from multilocal observables, and feeds multiple feature maps to classical neural-network layers.The architecture extends QCNNs with classical postprocessing rather than repeatedly reducing the state to one qubit.
  • TFQ functionality: TFQ provides hybrid models, batched circuit simulation, expectation-based backpropagation, and classical gradient optimization for these applications.The framework converts quantum computations into classical information through simulation, expectation values, or sampling.
  • Quantum classification: The classifier dataset contains correctly and incorrectly prepared 8-qubit cluster states, with rotation errors labeled by a threshold.The correctly prepared cluster state is the common initial state, followed by excitation circuits representing error states.
  • Quantum classification: QCNNs exploit translational symmetry in cluster states by tying quantum-convolution parameters across nearest-neighbor qubit pairs.Quantum pooling uses a separate unitary to disentangle two qubits and project them onto one.
  • Results: The purely quantum model trains slowest, whereas the three-quantum-filter hybrid model trains fastest under the same optimization settings.This comparison motivates exploring hybrid quantum-classical architectures for quantum-data classification.

B. Hybrid Machine Learning for Quantum Control

TFQ supports hybrid neural networks for quantum control by combining classical controllers with quantum circuits and expectation-based training. In time-constant control, the demonstrated model converges in about 100 epochs and generalizes well to validation data.

  • TFQ functionality: TFQ combines hybrid models, batch circuit simulation, expectation-based backpropagation, and gradient or non-gradient classical optimizers for control problems.These capabilities support both learning quantum dynamics and optimizing control signals against a cost objective.
  • Time-constant control: The time-constant control example uses supervised learning to approximate a controller that maps input vectors to control vectors producing desired outputs.Supervised learning is motivated when analytical inversion is unavailable or costly and deployment includes noise and systematic errors.
  • Time-constant control: The demonstrated single-qubit task learns hardware-friendly decompositions of arbitrary single-qubit unitaries into fixed-axis rotations.The target unitary is specified by three rotation parameters, and the controller predicts the corresponding control sequence.
  • Control architecture: Hybrid quantum-classical control models map target parameters to hardware control vectors and evaluate resulting quantum states through observable expectation values.The classical network outputs g = {β, γ, δ}, while the quantum circuit supplies measured observables.
  • Results: Around 100 epochs, training converges while validation performance shows excellent generalization.The result is reported for the two-axis control model on the training and validation datasets.

2. Time-dependent Hamiltonian Control

TFQ extends hybrid control to time-varying dynamics and provides tools for simulating and learning from noisy circuits. The examples show rapid recurrent-model convergence and comparable classifier accuracy under mild noise.

  • Time-dependent Hamiltonian Control: Time-dependent control requires learning a generally time-varying optimal control because the actuated Hamiltonian changes with time.The control is represented as a baseline component plus a time-dependent deviation.
  • Time-dependent Hamiltonian Control: An RNN can learn future quantum observables from recorded expectation-value time series to model low-frequency control noise.The paper describes both offline training from recorded trajectories and a possible online method for real-time controller tuning.
  • Time-dependent Hamiltonian Control: The LSTM converges within 30 epochs to accuracy comparable to expectation-value measurements on validation data.The evaluation uses prediction accuracy on validation realizations of the noise process.
  • Noise in Cirq and TFQ: Noise changes circuit measurement distributions by inserting, deleting, or replacing gates, so noisy execution can produce bitstrings absent from the ideal output.For the example circuit, noiseless sampling yields |11⟩, while noisy sampling can also yield |00⟩, |01⟩, or |10⟩.
  • Noise in Cirq and TFQ: TFQ uses trajectory methods and noisy backends for sampling and expectation-based layers, enabling direct comparisons between noisy and noiseless circuits.The workflow can simulate noise and compare hybrid-model performance under different noise types and strengths.
  • Training Models with Noise: The classifier trained on noisy data achieves similar accuracy to the classifier trained on noiseless data under mild noise.The paper presents this as evidence that the model can learn despite mild noise.

D. Quantum Approximate Optimization

This section presents QAOA in TFQ as a hybrid quantum-classical approach for approximate discrete optimization, illustrated on MaxCut. It defines the parameterized circuit and cost objective, then trains and samples the resulting state to obtain an approximate solution.

  • QAOA overview: QAOA seeks approximate minima of pseudo-Boolean functions and has been applied to problems including MaxCut and Max-3-Lin-2.The section focuses on quadratic, 2-local problems.
  • QAOA construction: The algorithm applies a tunable gate sequence to an initial product or superposition state, producing a wavefunction biased toward low-energy states of a cost Hamiltonian.The circuit uses mixer and cost Hamiltonians built from Pauli operators.
  • MaxCut construction: For MaxCut, the cost Hamiltonian is a second-order polynomial over graph variables, with the graph defining the optimization instance.MaxCut finds the largest edge subset whose endpoints lie in different vertex partitions.
  • Optimization and readout: Training minimizes the expectation value of the cost Hamiltonian, after which QPU samples are measured and the lowest-energy bitstring is selected as the approximate solution.The optimized parameters minimize L(η, γ) = ⟨Ψηγ|HC|Ψηγ⟩.
  • TFQ implementation: TFQ represents the QAOA circuit and cost Hamiltonian in a Keras model, evaluates quantum gradients, and optimizes parameters with TensorFlow’s Adam optimizer.The example uses a Hadamard input circuit and mean absolute error as its loss function.
  • Advanced applications: TFQ advanced examples combine hybrid models and optimizers, with researchers reporting reduced time-to-working-prototype from weeks to days or hours compared with alternative tools.The section includes neural-network optimization of QAOA and lowering the number of QAOA optimization iterations.

B. Vanishing Gradients and Adaptive Layerwise Training Strategies

The section explains that barren plateaus make unrestricted quantum-circuit training difficult because gradients can vanish exponentially with system size. It presents layerwise and partitioned training strategies that adapt circuit structure and limit simultaneously trained parameters.

  • 1. Random Quantum Circuits and Barren Plateaus: Barren plateaus are regions of quantum parameter space with exponentially small gradients, making straightforward training impossible after entering them.The vanishing-gradient rate increases exponentially with qubit number and depends on whether the cost is global or local.
  • 1. Random Quantum Circuits and Barren Plateaus: Quantum readout complexity can make tiny gradients difficult to resolve, requiring approximately 10^14 samples for a gradient near 10^-7.The passage contrasts quantum readout scaling O(1/ϵ^α) with classical scaling O(log 1/ϵ).
  • 1. Random Quantum Circuits and Barren Plateaus: Training an entire randomly initialized network with methods such as Adam can drift onto a barren plateau when the learning rate is too high.The stated mechanism is random diffusion through parameter space.
  • B. Adaptive Layerwise Training Strategies: Layerwise learning trains subsets of the circuit and can use larger learning rates while improving training efficiency.The authors recommend combining it with appropriately designed local cost functions.
  • B. Adaptive Layerwise Training Strategies: The first phase grows a circuit by adding and training layers, while the second alternates training across parameter partitions with the remaining parameters fixed.A hyperparameter r controls the percentage of simultaneously trained parameters.
  • B. Adaptive Layerwise Training Strategies: TFQ supports dynamically building circuits, manipulating weights during training, reducing trained parameters, and avoiding initialization on or drift toward barren plateaus.The example uses parameterized layers, Keras weight manipulation, and a parameter-shift differentiator.

C. Hamiltonian Learning with Quantum Graph Recurrent Neural Networks

This section frames Hamiltonian learning as an inverse problem: infer quantum dynamics from states observed at different times. TFQ implements a structured Quantum Graph Recurrent Neural Network that learns an effective Ising Hamiltonian and its couplings.

  • 1. Motivation: Learning Quantum Dynamics with a Quantum Representation: Hamiltonian learning seeks a Hamiltonian whose time evolution reproduces unknown black-box quantum dynamics.The task is motivated by the potential difficulty of classically simulating quantum time evolution.
  • 1. Motivation: Learning Quantum Dynamics with a Quantum Representation: The approach uses a Quantum Neural Network to learn the Hamiltonian of a dynamical process from quantum states at various time steps.This is presented as a quantum representation for learning quantum dynamics.
  • 1. Motivation: Learning Quantum Dynamics with a Quantum Representation: A structured ansatz based on Trotter-Suzuki evolution incorporates prior knowledge that the target process is quantum dynamics.For one-dimensional many-body systems, locality is encoded through a graph-based Hamiltonian parameterization.
  • 2. Implementation: The Quantum Graph Neural Network is defined as a sequence of exponentials of generally 2-local Hamiltonians whose coupling topology follows an assumed graph.The graph structure supplies the topology for the Hamiltonian terms.
  • 2. Implementation: The QGRNN models a parameterized Ising Hamiltonian with trainable coupling and bias parameters and recurrent, Trotterized evolution.Its generators include ZZ coupling terms and Z bias terms over the chosen graph.
  • 2. Implementation: The training data are generated from approximate low-energy states and simulated evolutions, while the implementation initializes graph-based Ising and QGRNN parameters in TFQ.The target and prior graphs can differ, allowing the model to learn from an unknown nearest-neighbor structure.
  • 2. Implementation: In the reported example, time-randomized gradient-based optimization learned the target Hamiltonian and its couplings to a high degree of accuracy.The implementation uses TFQ-compatible Cirq Hamiltonian operators and expectation-value training.

D. Generative Modelling of Quantum Mixed States with Hybrid Quantum-Probabilistic Models

This section presents hybrid quantum-probabilistic models for representing mixed quantum states, combining classical probability distributions with quantum neural networks. It demonstrates variational thermal-state preparation and describes training procedures for both classical and quantum parameters.

  • Motivation: Mixed states combine classical probability distributions with quantum wavefunctions and can exhibit both classical and quantum correlations.The hybrid representation reduces quantum-device representation overhead, supporting lower-depth quantum neural networks for noisy near-term devices.
  • Model: The visible state is generated by applying a parameterized quantum unitary to a classical latent distribution over standard-basis states.The unitary parameters are φ, while the classical probabilistic model parameters are θ.
  • Thermal-state preparation: Variational thermalization approximates the thermal state defined by a Hamiltonian and inverse temperature by minimizing a free-energy loss.The loss combines the model’s expected energy and entropy; the visible-state entropy equals the classical latent distribution’s entropy.
  • Optimization: Quantum-network gradients use TFQ parameter-shift methods, while classical-model gradients are estimated from covariances using samples from the classical model and QPU output.The classical latent model can use exponential-family or energy-based models, including factorized Bernoulli distributions and Boltzmann machines.
  • Model expressivity: The probabilistic-model optimization avoids estimating the partition function, allowing more expressive latent distributions beyond factorized models.This supports the use of general energy-based models such as Boltzmann machines.

3. Quantum Generative Learning from Quantum Data

This section extends hybrid quantum-probabilistic modeling from Hamiltonian-defined thermal states to learning mixed-state statistics directly from quantum data. It also describes TFQ support for variational excited-state calculations.

  • Quantum-data modeling: Given copies of a mixed data state, the model learns parameters whose generated state approximates the observed quantum state.The model uses the same hybrid classical-distribution and quantum-unitary form introduced for variational thermalization.
  • Objective: Training minimizes quantum cross entropy, equivalent to minimizing reverse quantum relative entropy for the data and model states.The pulled-back data distribution is obtained by applying the inverse quantum neural network and measuring in the standard basis.
  • Optimization: Quantum-network gradients use TFQ differentiators, while energy-based classical parameters are updated from differences between data-state and model-distribution expectations.The classical gradient is expressed as Ex∼σφ(x)[∇θEθ(x)]−Ey∼pθ(y)[∇θEθ(y)].
  • Example: A data-replication example uses the output density matrix from the VQT example and fixes its VQT parameters as a quantum data source.The learned classical distribution is sampled efficiently and passed through the learned VQT unitary to reproduce the data-state statistics.
  • Excited-state energies: SSVQE modifies VQE to estimate ground and excited-state energies using orthogonal initial states and shared variational parameters.The implementation targets the ground and first excited states of H2 across increasing bond lengths.
  • TFQ tooling: TFQ supports SSVQE experimentation through adjoint differentiation, custom layers and objectives, and OpenFermion integration.These features are presented as useful for handling the many Hamiltonians encountered in quantum chemistry experiments.

2. Implementation

This section implements TFQ examples for excited-state energy estimation and quantum phase classification. The workflows construct parameterized circuits, Hamiltonian measurements, training loops, and quantum-state datasets.

  • Circuit and measurement construction: The SSVQE circuit uses parameterized single-qubit rotations and exponentiated CNOT gates, with Hamiltonian terms converted into measurement operators.These operators provide the expectation value of the Hamiltonian used by the variational objective.
  • SSVQE implementation: The SSVQE implementation builds collections of VQE layers with shared parameters and orthogonal inputs to estimate multiple energy levels.Each layer evaluates a Hamiltonian expectation using TFQ ControlledPQC components and the adjoint differentiator.
  • SSVQE training: The training loop minimizes the encoded SSVQE objective with TensorFlow GradientTape and applies gradients to the trainable circuit parameters.Training stops when successive losses differ by less than the specified tolerance.
  • Molecular experiment: Molecular Hamiltonians generated with OpenFermion and PySCF are converted through the Jordan-Wigner transform and evaluated across bond lengths.The workflow trains SSVQE separately for 20 Hamiltonians while comparing predicted and actual energies.
  • Results and cost: The SSVQE energy comparison is shown in Figure 29 for predicted ground- and first-excited-state energies against true values.The implementation takes several minutes because it optimizes 48 parameters per layer across 20 Hamiltonians.
  • Quantum phase classification: Quantum phase classification trains an end-to-end classifier on quantum many-body states, combining state preparation, a trainable variational circuit, and measurements.The example uses the two-dimensional transverse-field Ising model, whose transition occurs near g ≈3.04.
  • Quantum phase classification: The classifier measures ⟨Zi⟩ on each qubit and maps the resulting linear predictor through a rescaled sigmoid, trained with hinge loss.The model outputs a scalar prediction in (−1, 1).

2. Implementation

TFQ implements hybrid quantum-classical workflows by encoding TFIM circuits as tensors, constructing trainable quantum classifiers, and supporting noisy adversarial learning. The examples include phase classification and EQ-GAN-based noise suppression.

  • Quantum phase classification: The 4 × 4 TFIM example loads circuits and labels for ordered and disordered phases over g ∈[2.5, 3.5].
  • Quantum phase classification: The classifier uses individually parameterized single-qubit and nearest-neighbour gate layers followed by Z-basis readouts integrated into Keras.
  • Quantum phase classification: TFQ integrates the quantum classifier into a Keras model with a parameterized quantum-circuit layer, dense output, hinge accuracy, Adam optimization, and early stopping.
  • Quantum phase classification: The trained classifier’s zero-crossing inflection occurs at ȳ ≈0 and coincides with the TFIM critical point g ≈3.04.
  • Noise suppression and generative learning: EQ-GAN entangles true and generated data in its discriminator, with theory establishing global Nash-equilibrium convergence and experiments finding convergence where QuGAN failed.
  • Noise suppression and generative learning: On a calibrated quantum device, the adversarial swap test reduced error by around fourfold relative to a standard swap test.

3. Approximate Quantum Random Access Memory

TFQ demonstrates approximate QRAM by training an EQ-GAN to represent a classical dataset as a quantum superposition, then using that representation in quantum-neural-network training. The section also presents a PQC policy-gradient implementation for CartPole.

  • Approximate QRAM: Preparing an arbitrary superposition of n classical data states takes O(n) operations at best, undermining the intended exponential speedup.
  • Approximate QRAM: A shallow ansatz approximates a symmetric two-peak distribution because exact empirical-density encoding and device-compatible Gaussian preparation require deep circuits.
  • Approximate QRAM: With the same number of gradient-evaluation circuit calls, variational-QRAM superpositions reached better final accuracy than individually encoded data despite using an approximate distribution.
  • Quantum reinforcement learning: The reinforcement-learning example uses alternating variational, entangling, and data-encoding layers, with data re-uploading chosen for model expressivity.
  • Quantum reinforcement learning: The policy-gradient agent maps CartPole-v1’s 4-dimensional states to a 4-qubit PQC and evaluates two action-associated expectation values.
  • Quantum reinforcement learning: Parallel model evaluation and adjoint automatic differentiation train the 4-qubit, five-layer agent for 500 episodes in about 15 minutes on a regular laptop.

4. Value-Based Reinforcement Learning with PQCs

TFQ’s reinforcement-learning tooling supports both policy-based and value-based PQC agents. The paper presents the value-based formulation as an extension using expectation values to approximate state-action values, while natural quantum learning advantages remain unresolved.

  • Value-based reinforcement learning: The example code implements a policy-based PQC reinforcement-learning approach, with a linked tutorial also implementing a value-based algorithm.
  • Value-based reinforcement learning: In value-based quantum reinforcement learning, PQC expectation values approximate Q(s, a), the value function conditioned on both state and action.
  • Value-based reinforcement learning: The value-based method additionally uses deep-RL mechanisms such as replay memory and a target model to stabilize learning.
  • Open questions: Although contrived environments favoring quantum agents are known, whether natural environments yield large quantum learning advantages remains an open question.
  • Closing remarks: TFQ is open-sourced under the commercially friendly Apache 2 license and is intended to accelerate quantum machine-learning development as hardware evolves.
Loading 2003.02989v2…