Parameterized quantum circuits as machine learning models

There are several ways to encode data into qubits and each one provides different expressive power. This choice of encoding is related to kernel methods, a well-established field whose goal is to embed data into a higher dimensional feature space where a specific problem may be easier to solve. For example, nonlinear feature maps change the relative position between data points such that a dataset may become easier to classify in the feature space. In a similar way, the process of encoding classical data into a quantum state can be interpreted as a feature map ${\boldsymbol{x}}\to {U}_{\phi ({\boldsymbol{x}})}| 0{\rangle }^{\otimes n}$ to the high-dimensional vector space of the states of n qubits. Here, ϕ is a user-defined pre-processing function which transforms the data vector into circuit parameters.

The inner product of two data points in this space defines a similarity function, or kernel, $k({\boldsymbol{x}},{{\boldsymbol{x}}}^{{\prime} })\,={\left|\langle 0| {U}_{\phi ({{\boldsymbol{x}}}^{{\prime} })}^{\dagger }{U}_{\phi ({\boldsymbol{x}})}^{}| 0\rangle \right|}^{2}$. This quantity can be evaluated using the SWAP test shown in figure 3, and readily used in kernel-based models such as the support vector machine, the Gaussian process, and the principal component analysis.

Zoom In Zoom Out Reset image size

Let us now discuss some examples. Stoudenmire and Schwab [15] encode data as products of local kernels, one for each component of the input vector, which results in a product quantum state (i.e. disentangled). This approach is often referred to as qubit encoding and can produce highly nonlinear kernels. As an example, for input vectors ${\boldsymbol{x}}\in {[0,1]}^{n}$ one can realize the feature map ${\boldsymbol{x}}\to {\otimes }_{i=1}^{n}\left(\displaystyle \genfrac{}{}{0em}{}{\cos ({x}_{i}\pi /2)}{\sin ({x}_{i}\pi /2)}\right)$ by applying suitable single-qubit rotations. Mitarai et al [16] use a similar approach, but encode each component of the data vector into multiple qubits. This redundancy populates the wave function with higher-order terms that can be exploited to fit nonlinear functions of the data. Vidal and Theis [17] investigate how this redundancy helps the task of data fitting. They found lower bounds of the redundancy that are logarithmic in the complexity of the function to be fit, using a linear-algebraic complexity measure.

A different approach is taken by Wilson et al [18]; the authors pre-process the input with a random linear map $\phi ({\boldsymbol{x}})=A{\boldsymbol{x}}+{\boldsymbol{b}}$, creating a quantum version of the random kitchen sink [19]. They show that in the limit of many realizations of random linear maps, this approach implicitly implements a kernel. Interestingly, the form of the kernel depends on the layout of the encoder circuit, and not on all layouts are capable of implementing useful kernels. Another proposal that is based on random encoder circuits, but inspired by the convolutional filters used in neural networks, is the quanvolutional network by Henderson et al [20].

The examples discussed so far require low-depth encoder circuits and may therefore be robust depending on the noise characteristics and level. A different approach is the amplitude encoding, a feature map encoding 2ⁿ-dimensional data vectors into the wave function of merely n qubits. Assuming unit data vectors, the feature map ${\boldsymbol{x}}\to | {\boldsymbol{x}}\rangle$ provides an exponential advantage in terms of memory and leads to a linear kernel. It is also known that by preparing copies of this feature map one can implement arbitrary polynomial kernels [21]. Unfortunately, the depth of this encoder circuit is expected to scale exponentially with the number of qubits for generic inputs. Therefore, algorithms based on amplitude encoding could be impeded by our inability to coherently load data into quantum states.

On a different note, Havlíček et al [22] argue that a feature map can be constructed so that the kernel is hard to estimate using classical resources, and that this is a form of quantum supremacy. They consider, for example, ${U}_{\phi ({\boldsymbol{x}})}=\exp ({\rm{i}}{\sum }_{j,k}^{n}{\phi }_{j,k}({\boldsymbol{x}}){Z}_{j}{Z}_{k}){H}^{\otimes n}$ where Z_j is the Pauli-Z operator for the jth qubit, ${\phi }_{j,k}$ are real functions, and H is the Hadamard gate. They conjecture that two layers of such an encoder circuit make the estimation of the kernel $k({\boldsymbol{x}},{{\boldsymbol{x}}}^{{\prime} })=| \langle 0| {U}_{\phi ({{\boldsymbol{x}}}^{{\prime} })}^{\dagger }{U}_{\phi ({{\boldsymbol{x}}}^{{\prime} })}^{\dagger }{U}_{\phi ({\boldsymbol{x}})}^{}{U}_{\phi ({\boldsymbol{x}})}^{}| 0\rangle {| }^{2}$ classically intractable. This is due to its similarity to the circuits used in the hidden shift problem of Boolean bent functions, which are known to be classically hard to simulate [23].

The design of feature maps inspired by quantum supremacy proposals is an interesting research direction. Whether this leads to an advantage in practical machine learning is an open question and should be tested empirically on existing computers. Ultimately, the form of the kernel and its parameters could be learned from data; this is a largely unexplored area in PQCs and has the potential to reduce the bias in kernel selection, and to automatically discover unknown feature maps that exhibit quantum supremacy.

Similar to the universal approximation theorem in neural networks [24], there always exists a quantum circuit that can represent a target function within an arbitrary small error. The caveat is that such a circuit may be exponentially deep and therefore impractical. Lin et al [25] argue that since real datasets arise from physical systems, they exhibit symmetry and locality; this suggests that it is possible to use 'cheap' models, rather than exponentially costly ones, and still obtain a satisfactory result. With this in mind, the variational circuit aims to implement a function that can approximate the task at hand while remaining scalable in the number of parameters and depth.

In practice, the circuit design follows a fixed structure of gates. Despite the dimension of the vector space growing exponentially with the number of qubits, the fixed structure reduces the model complexity, resulting in the number of free parameters to scale as a polynomial of the qubit count.

The first strategy to circuit design aims to comply with the fact that NISQ hardware has few qubits and usually operates on a sparse qubit-to-qubit connectivity graph with rather simple gates. Hardware-efficient circuits alternate layers of native entangling gates and single-qubit rotations [7]. Examples of these layers are shown in figure 4, where (a) and (b) are designed around the connectivity and gate set of superconducting and trapped ion computers, respectively. Heuristics can be used to strategically reduce the number of costly entangling gates. For example, Liu and Wang [26] use the Chow-Liu tree graph [27] to setup the entangling layers. First, the mutual information between all pairs of variables is estimated form the dataset. Then, entangling gates are placed between qubits so that most of the mutual information is represented.

Zoom In Zoom Out Reset image size

Another principled approach to circuit design is inspired by quantum many-body physics. Tensor networks are methods to efficiently represent quantum states in terms of smaller interconnected tensors. In particular, these are often used to describe states whose entanglement is constrained by local interactions. By looking only at a smaller portion of the vector space, the computational cost is then reduced and becomes a polynomial function of the system size. This enables the numerical treatment of systems through layers of abstraction, reminiscent of deep neural networks. Indeed, some of the most studied tensor networks such as the matrix product state, the tree tensor network (TTN), and the multi-scale entanglement renormalization ansatz (MERA) have been tested for classification and generative modeling [28–30].

Figure 5(a) shows an example of a TTN for supervised learning. After the application of each unitary, half of the qubits are traced out, while the other half continues to the next layer. Huggins et al [29] suggest a qubit-efficient version where the traced qubits are reinitialized and used as the inputs of another unitary, as shown in figure 5(b). Qubit-efficient schemes could significantly reduce the required number of qubits, a favorable condition to some NISQ hardware.

Zoom In Zoom Out Reset image size

Neural networks and deep learning have proven to be very successful and therefore offer a further source of inspiration for circuit design. Both variational circuits and neural networks can be thought of as layers of connected computational units controlled by adjustable parameters. This has led some authors to refer to variational circuits as 'quantum neural networks'. Here we shall briefly discuss the key differences that make this approach to circuit design rather difficult.

First, quantum circuit operations are unitary and therefore linear; this is in contrast with the nonlinear activation functions used in neural networks, which are key to their success and universality [33]. There are several ways to construct nonlinear operations in quantum circuits, both coherently (i.e. exploiting entanglement) or non-coherently (e.g. exploiting the natural coupling of the system to the environment). These can in turn be used to implement classical artificial neurons in quantum circuits [34–36].

The second key difference is that it is impossible to access the quantum state at intermediate points during computation. Although measurement of ancillary quantum variables can be used to extract limited information, any attempt to observe the full state of the system would disrupt its quantum character. This implies that executing the variational circuit cannot be seen as performing the forward pass of a neural network. Moreover, it is difficult to conceive a circuit learning algorithm that truly resembles backpropagation, as it would rely on storing the intermediate state of the network during computation [37]. Backpropagation is the gold standard algorithm for neural networks and can be described as a computationally efficient organization of the chain rule that allows gradient descent to work on large-scale models.

The questions of how to generalize a quantum artificial neuron and design a quantum backpropagation algorithm have been open for quite some time [38]. Some recent work goes towards this direction. Verdon et al [39] quantize the parameters of the variational circuit which are then prepared in superposition in a dedicated register. This enables a backpropagation-like algorithm which exploits quantum effects such as phase kickback and tunneling. Beer et al [40] use separate qubit registers for input and output, and define the quantum neuron as a completely positive map between the two. The resulting network is universal for quantum computation and can be trained by an efficient process resembling backpropagation.

Just like classical models, PQC models are trained to perform data-driven tasks. The task of learning an arbitrary function from data is mathematically expressed as the minimization of a loss function $L({\boldsymbol{\theta }})$, also known as the objective function, with respect to the parameter vector ${\boldsymbol{\theta }}$. We discuss two types of algorithms, namely gradient-based and gradient-free, that can be applied to optimize the parameters of a variational circuit ${U}_{{\boldsymbol{\theta }}}$.

One instance of gradient-based algorithms is the iterative method called gradient descent. Here the parameters are updated towards the direction of steepest descent of the loss function

$$\begin{eqnarray}&&{\boldsymbol{\theta }}\leftarrow {\boldsymbol{\theta }}-\eta {{\rm{\nabla }}}_{{\boldsymbol{\theta }}}L,\end{eqnarray} \tag{ 2 }$$

where ${{\rm{\nabla }}}_{{\boldsymbol{\theta }}}L$ is the gradient vector and η is the learning rate—a hyperparameter controlling the magnitude of the update. This procedure is iterated and, assuming suitable conditions, converges to a local minimum of the loss function.

The required partial derivatives can be calculated numerically using a finite difference scheme

$$\begin{eqnarray}&&\displaystyle \frac{\partial L}{\partial {\theta }_{j}}\approx \displaystyle \frac{L({\boldsymbol{\theta }}+{\rm{\Delta }}{{\boldsymbol{e}}}_{j})-L({\boldsymbol{\theta }}-{\rm{\Delta }}{{\boldsymbol{e}}}_{j})}{2{\rm{\Delta }}},\end{eqnarray} \tag{ 3 }$$

where Δ is a (small) hyperparameter and ${{\boldsymbol{e}}}_{j}$ is the Cartesian unit vector in the j direction. Note that in order to estimate the gradient vector ${{\rm{\nabla }}}_{{\boldsymbol{\theta }}}L$, this approach evaluates the loss function twice for each parameter.

Alternatively, Spall's simultaneous perturbation stochastic approximation (SPSA) [41, 42] computes an approximate gradient vector with just two evaluations of the loss function as

$$\begin{eqnarray}&&\displaystyle \frac{\partial L}{\partial {\theta }_{j}}\approx \displaystyle \frac{L({\boldsymbol{\theta }}+c\,{\boldsymbol{\Delta }})-L({\boldsymbol{\theta }}-c\,{\boldsymbol{\Delta }})}{2\,c\,{{\rm{\Delta }}}_{j}},\end{eqnarray} \tag{ 4 }$$

where ${\boldsymbol{\Delta }}$ is a random perturbation vector and c is a (small) hyperparameter.

There are cases when finite difference methods are ill-conditioned and unstable due to truncation and round-off errors. This is one of the reasons why machine learning relies on the analytical gradient when possible, and it is often calculated with automatic differentiation schemes [43]. The analytical gradient can also be estimated for variational circuits, although the equations depend on the choice of parameterization for the gates. For our discussion, we consider circuits U_J:1 = U_J⋯U₁, where trainable gates are of the from ${U}_{j}=\exp (-\tfrac{{\rm{i}}}{2}{\theta }_{j}{P}_{j})$, and where ${P}_{j}\in \{I,Z,X,Y\}{}^{\otimes n}$ is a tensor product of n Pauli matrices. Arguably, this is the most common parameterization found in the literature.

Using this, Li et al [44] propose a way to efficiently compute analytical gradients in the context of quantum optimal control. Mitarai et al [16] bring this method into the context of supervised learning. Recall that the model's output is a function of expectation values $\langle {M}_{k}{\rangle }_{{\boldsymbol{\theta }}}$. Using the chain rule we can write the derivative $\tfrac{\partial L}{\partial {\theta }_{j}}$ as a function of the derivatives of the expectation values $\tfrac{\partial \langle {M}_{k}{\rangle }_{{\boldsymbol{\theta }}}}{\partial {\theta }_{j}}$. Each of these quantities can be estimated on quantum hardware using the parameter shift rule

$$\begin{eqnarray}&&\displaystyle \frac{\partial \langle {M}_{k}{\rangle }_{{\boldsymbol{\theta }}}}{\partial {\theta }_{j}}=\displaystyle \frac{\langle {M}_{k}{\rangle }_{{\boldsymbol{\theta }}+\tfrac{\pi }{2}{{\boldsymbol{e}}}_{j}}-\langle {M}_{k}{\rangle }_{{\boldsymbol{\theta }}-\tfrac{\pi }{2}{{\boldsymbol{e}}}_{j}}}{2},\end{eqnarray} \tag{ 5 }$$

where subscripts ${\boldsymbol{\theta }}\pm \tfrac{\pi }{2}{{\boldsymbol{e}}}_{j}$ indicate the shifted parameter vector to use for the evaluation (see Schuld et al [45] for a detailed derivation). Note that this estimation can be performed by executing two circuits.

An alternative method can estimate the partial derivative with a single circuit, but at the cost of adding an ancilla qubit. A simple derivation using the gate parameterization introduced above (e.g. see Farhi and Neven [46]) shows that the partial derivative can be written as

$$\begin{eqnarray}&&\displaystyle \frac{{\rm{\partial }} \langle {M}_{k}{ \rangle }_{{\boldsymbol{\theta }}}}{{\rm{\partial }}{\theta }_{j}}={Im}\left({\rm{tr}}({M}_{k}^{}{U}_{J:j+1}^{}{P}_{j}^{}{U}_{j:1}^{}| 0 \rangle \langle 0| {U}_{J:1}^{\dagger }\right).\end{eqnarray} \tag{ 6 }$$

This can be thought of as an indirect measurement and can be evaluated using the Hadamard test shown in figure 6. This method can be generalized to compute higher order derivatives, as presented for example by Dallaire-Demers and Killoran [47], and with alternative gate parameterizations, as done for example by Schuld et al [48].

Zoom In Zoom Out Reset image size

We shall note that despite the apparent simplicity of the circuit in figure 6, the actual implementation of Hadamard tests may be challenging due to non-trivial controlled gates. Coherence must be guaranteed in order for quantum interference to produce the desired result. Mitarai and Fujii [49] propose a method for replacing a class of indirect measurements with direct ones. Instead of an interference circuit one can execute, in some cases, multiple simpler circuits that are suitable for implementations on NISQ computers. The 'parameters shift rule' in equation (5) is nothing but the direct version of the measurement in equation (6).

Compared to finite difference and SPSA, the analytical gradient has the advantage of providing an unbiased estimator. Additionally, Harrow and Napp [50] find evidence that circuit learning using the analytical gradient outperforms any finite difference method. This is done by showing that for n qubits and precision , the query cost of an oracle for convex optimization in the vicinity of the optimum scales as ${ \mathcal O }(\tfrac{{n}^{2}}{\epsilon })$ for the analytical gradient, whereas finite difference needs at least ${\rm{\Omega }}(\tfrac{{n}^{3}}{{\epsilon }^{2}})$ calls to the oracle. In practice though, it is found that SPSA performs well in small-scale noisy experimental settings (e.g. see Kandala et al [7] and Havlíček et al [22]).

Particular attention should be given to the problems of exploding and vanishing gradients which are well-known to the machine learning community. Classical models, in particular recurrent neural networks, are often constrained to perform unitary operations so that their gradients cannot explode (see Wisdom et al [51] for an example). Quantum circuits implementing unitary operations naturally avoid the exploding gradient problem. On the other hand, McClean et al [31] show that random circuits of reasonable depth lead to an optimization landscape with exponentially large plateaus of vanishing gradients with an exponentially decaying variance. This can be understood as a consequence of Levy's lemma [52] which states that a random variable that depends on many independent variables is essentially constant. The learning algorithm is thus unable to estimate the gradient and may perform a random walk in parameter space. While this limits the effectiveness of variational circuits initialized at random, the use of highly structured circuits could alleviate the problem (e.g. see Grant et al [53] for a structured initialization strategy).

We shall stress here that in hybrid systems parameter updates are performed classically. This implies that some of the most successful deep learning methods can be readily used for circuit learning. Indeed, heuristics such as stochastic gradient descent [54], resilient backpropagation [55], and adaptive momentum estimation [56], have already been applied with success. These were designed to deal with issues of practical importance such as large datasets, large noise in gradient estimates, and the need to find adaptive learning rates in equation (2). In practice, these choices can reduce the time for successful training from days to hours.

There are cases where gradient-based optimization may be challenging. For example, in a noisy experimental setting the loss function may be highly non-smooth and not suitable for gradient descent. As another example, the objective function may be itself unknown and therefore should be treated as a black-box. In these cases, circuit learning can be carried out by gradient-free methods. A well-known method of this type is particle swarm optimization [57]. Here the system is initialized with a number of random solutions called particles, each one moving through solution space with a certain velocity. The trajectory of each particle is adjusted according to its own experience and that of other particles so that they converge to a local minima. Another popular method is Bayesian optimization [58]. It uses evaluations of the objective function to construct a model of the function itself. Subsequent evaluations can be chosen either to improve the model or to find a minima.

Zhu et al [59] compare Bayesian and particle swarm optimization for training a generative model on a trapped ion quantum computer. While Bayesian optimization outperforms particle swarm in their setting, they found that the large number of parameters challenges both optimizers. They show that an ideal simulated system is not significantly faster than the experimental system, indicating that the actual bottleneck is the classical optimizer. Leyton-Ortega et al [60] train a generative model on a superconducting quantum computer and compare the gradient-free methods of zeroth-order optimization package [61] and stochastic hill-climbing, with gradient descent. They find that on average zeroth-order optimization achieves the lowest loss on their hardware. They argue that the main optimization challenge is to overcome the variance of the loss function which is due to random parameter initialization, hardware noise, and finite number of measurements.

Genetic algorithms [62] are another large class of gradient-free optimization algorithms. At each step, candidate solutions are evolved using biology-inspired operations such as recombination, mutation, and natural selection. When used for circuit learning, genetic algorithms define a set of allowed gates and the maximum number to be employed. Lamata et al [63] suggest the use of genetic algorithms to train a PQC model for compression using a universal set of single- and two-qubit gates. Ding et al [64] validate the idea experimentally by deploying a pre-trained PQC model on a superconducting computer and find that using a subsequent genetic algorithm improves its fidelity.

To conclude, we note that optimization algorithms should be tailored for PQC models if we want to achieve better scalability. Very recent work has been approaching circuit learning from this perspective (e.g. see Ostaszewski et al [65] and Nakanishi et al [66]).

Let us first consider supervised learning tasks, e.g. classification and regression, on classical data. Given a dataset ${ \mathcal D }={\{({{\boldsymbol{x}}}^{(i)},{{\boldsymbol{y}}}^{(i)})\}}_{i\,=\,1}^{N}$, of N samples, the goal is to learn a model function $f:{ \mathcal X }\to { \mathcal Y }$ that maps each ${\boldsymbol{x}}\in { \mathcal X }$ to its corresponding target ${\boldsymbol{y}}\in { \mathcal Y }$. A standard approach is to minimize a suitable regularized loss function, that is

$$\begin{eqnarray}&&{{\boldsymbol{\theta }}}^{* }=\arg \mathop{\min }\limits_{{\boldsymbol{\theta }}}\displaystyle \frac{1}{N}\displaystyle \sum _{i=1}^{N}L\left(f({{\boldsymbol{x}}}^{(i)},{\boldsymbol{\theta }}),{{\boldsymbol{y}}}^{(i)}\right)+R({\boldsymbol{\theta }}),\end{eqnarray} \tag{ 7 }$$

where ${\boldsymbol{\theta }}$ is the set of parameters defining the model function, L quantifies the error of a forecast, and R is a regularization function penalizing undesired values for the parameters. The latter is used to prevent overfitting; indeed, if the training set is not sufficiently large, the model could simply memorize the training data and not generalize to unseen data.

In the PQC framework, once the encoder circuit ${U}_{\phi ({\boldsymbol{x}})}$ is set up, there are two main options for the remaining part of the circuit: the quantum kernel estimator (QKE), and the variational quantum model (VQM). We now briefly discuss both, and refer the Reader to Schuld and Killoran [69] for a more in-depth theoretical exposition.

The QKE does not use a variational circuit ${U}_{{\boldsymbol{\theta }}}$ to process the data; instead, it uses the SWAP test (e.g. see figure 3) to evaluate the possibly intractable kernel $k({{\boldsymbol{x}}}^{(i)},{{\boldsymbol{x}}}^{(j)})$. Then, resorting to the representer theorem [70], the model function is expressed as an expansion over kernel functions $f({\boldsymbol{x}},{\boldsymbol{w}})={\sum }_{i\,=\,1}^{N}{w}_{i}k({\boldsymbol{x}},{{\boldsymbol{x}}}^{(i)})$. The learning task is to find parameters ${\boldsymbol{w}}$ so that the model outputs correct forecasts. Note that these parameters define the classical post-processing function, as opposed to an operation of the PQC. A potential caveat is that QKE relies on a coherent SWAP test which may be non-trivial to implement on NISQ computers.

The VQM, on the other hand, uses a variational circuit ${U}_{{\boldsymbol{\theta }}}$ to process data directly in the feature space. A set of expectation values $\{\langle {M}_{k}{\rangle }_{{\boldsymbol{x}},{\boldsymbol{\theta }}}\}{}_{k=1}^{K}$ is estimated and post-processed to obtain the model output (see figure 2). In contrast to QKE, VQM parameters define the operations carried out by the quantum computer and require a circuit learning algorithm of the kind discussed in section 2.3.

Havlíček et al [22] experimentally demonstrate QKE and VQM classifiers on two superconducting qubits of the IBM Q5 Yorktown. Their QKE estimates a classically intractable feature map (see section 2.1 for details) which is then fed into a support vector machine to find the separating hyper-plane. Their VQM uses a hardware-efficient circuit instead. By employing a suitable error mitigation protocol, they find an increase in classification success with increasing circuit depth. In the future, it would be interesting to systematically compare these proposals against established classical models by evaluating accuracy and training efficiency, for example.

We now focus our discussion on VQM proposals. Farhi and Neven [46] propose a VQM binary classifier for bitstrings. The encoder circuit simply maps bitstrings to the computational basis states by applying identity and NOT gates at almost no cost. The variational circuit acts on the input register and one ancilla qubit which is measured to yield a class forecast. With n-bit data strings as the input, there are ${2}^{{2}^{n}}$ possible binary functions that could generate the class labels. The authors show that for any of the possible label functions there exists a variational circuit that achieves zero classification error. For some of these functions, the circuit is exponentially deep and therefore impractical. This result parallels the well known universal approximation theorem [24] which states that neural networks with an exponentially large hidden layer of nonlinear neurons are able to represent any Boolean function.

Mitarai et al [16] propose VQMs for classification and regression of real-valued data using a highly nonlinear qubit encoding. The variational circuit must then entangle the qubits such that a local observable can extract the relevant nonlinear features. As discussed in section 2.2 one possible way to strategically construct highly entangling variational circuits is inspired by tensor networks. Grant et al [30] use TTN and MERA variational circuits to perform binary classification on canonical datasets such as Iris and MNIST. In their simulations, MERA always outperforms TTN. One of their simplest models is efficiently trained classically and then deployed on the IBM Q5 Tenerife quantum computer with significant resilience to noise.

Stoudenmire et al [28] train a TTN to perform pairwise classification of the MNIST image data. In their simulations, they use entanglement entropy to quantify the amount of information in a detail of the image that is gained by observing the context. This is an example of how quantum properties can be used to characterize the complexity of classical data, which is a developing area of research.

Schuld et al [48] propose a VQM classifier assuming amplitude-encoded input data. Since this encoder circuit may be very expensive, the authors aim to keep the variational circuit low-depth and highly expressive at the same time. This is achieved through a systematic use of entangling gates, and by keeping the number of parameters polynomial in the number of qubits. Their simulations on benchmark datasets show performance comparable to that of off-the-shelf classical models while using significantly fewer parameters.

To date, all supervised learning experiments involved scaled-down, often trivial, datasets due to the limitation of available quantum hardware, and demonstrations at a more realistic scale are desirable. As a last comment, we note that a largely undeveloped area is that of regularization techniques specifically designed for PQC models which is, in our opinion, an interesting area for future research.

We now discuss generative modeling, an unsupervised learning task where the goal is to model an unknown probability distribution and generate synthetic data accordingly. Generative models have been successfully applied in computer vision, speech synthesis, inference of missing text, de-noising of images, chemical design, and many other automated tasks. It is believed that they will play a key role in the development of general artificial intelligence; a model that can generate realistic synthetic samples is likely to 'understand' its environment.

Concretely, the task is to learn a model distribution ${q}_{{\boldsymbol{\theta }}}$ that is close to a target distribution p. The closeness is defined in terms of a divergence D on the statistical manifold, and learning consists of minimizing this divergence; that is,

$$\begin{eqnarray}&&{{\boldsymbol{\theta }}}^{* }=\arg \mathop{\min }\limits_{{\boldsymbol{\theta }}}D(p,{q}_{{\boldsymbol{\theta }}}).\end{eqnarray} \tag{ 8 }$$

Since the target probability distribution is unknown, it is approximated using a dataset ${ \mathcal D }={\{{{\boldsymbol{v}}}^{(i)}\}}_{i\,=\,1}^{N}$ which we have access to and which is distributed according to the target distribution. As an example, ${{\boldsymbol{v}}}^{(i)}$ could be natural images extracted from the Internet.

The probabilistic nature of quantum mechanics suggests that a model distribution can be encoded in the wave function of a quantum system [71, 72]. Let us see how a simple adaptation of the model shown in figure 2 gives a generative model for n-dimensional binary data ${{\boldsymbol{v}}}^{(i)}\in \{0,1\}{}^{n}$. First, we set the encoder circuit to the identity ${U}_{\phi ({\boldsymbol{x}})}=I$ since in this problem there is no input data. Second, we apply a variational circuit ${U}_{{\boldsymbol{\theta }}}$ to the initial state $| 0{\rangle }^{\otimes n}$. Finally, we perform a measurement in the computational basis, i.e. we measure the set of operators $\{\langle {M}_{{\boldsymbol{v}}}{\rangle }_{{\boldsymbol{\theta }}}\}{}_{{\boldsymbol{v}}}$ where ${M}_{{\boldsymbol{v}}}=| {\boldsymbol{v}}\rangle \langle {\boldsymbol{v}}|$ are projectors for the bitstrings. The resulting generative model, known as the quantum circuit Born machine (QCBM) [32, 26], implements the probability distribution

$$\begin{eqnarray}&&{q}_{{\boldsymbol{\theta }}}({\boldsymbol{v}})=\mathrm{tr}({M}_{{\boldsymbol{v}}}{U}_{{\boldsymbol{\theta }}}^{}| 0\rangle \langle 0| {U}_{{\boldsymbol{\theta }}}^{\dagger }).\end{eqnarray} \tag{ 9 }$$

Since the target data is binary, no post-processing is needed and each measurement outcome ${\boldsymbol{v}}\sim {q}_{{\boldsymbol{\theta }}}$ is an operational output. If the target data were instead real-valued, we could interpret bitstrings as discretized outputs and use a post-processing function to recover real-values.

As one does not have access to the wave function, characterizing the distribution ${q}_{{\boldsymbol{\theta }}}$ may be intractable for all but the smallest circuits. For this reason, QCBMs belong to the class of implicit models, models where it is easy to obtain a sample ${\boldsymbol{v}}\sim {q}_{{\boldsymbol{\theta }}}$, but hard to estimate the likelihood ${q}_{{\boldsymbol{\theta }}}({\boldsymbol{v}})$. Machine learning researchers have become increasingly interested in implicit models because of their generality, expressive power, and success in practice [73]. Interestingly, Du et al [74] show that QCBMs have strictly more expressive power than classical models such as deep Boltzmann machines, when only a polynomial number of parameters are allowed. Coyle et al [75] show that some QCBMs cannot be efficiently simulated by classical means in the worst case, and that this holds for all the circuit families encountered during training.

Benedetti et al [32] build low-depth QCBMs using variational circuits suitable for trapped ion computers (see figure 4(b) for an example). They use particle swarms to minimize an approximation to the Kullback–Leibler divergence [78] $D(p,{q}_{{\boldsymbol{\theta }}})={\sum }_{{\boldsymbol{v}}}p({\boldsymbol{v}})\mathrm{ln}\tfrac{p({\boldsymbol{v}})}{{q}_{{\boldsymbol{\theta }}}({\boldsymbol{v}})}$. In their simulations they successfully train models for the canonical Bars-and-Stripes dataset and for Boltzmann distributions, and use them to design a performance indicator for hybrid quantum–classical systems. Zhu et al [59] implement this scheme on four qubits of an actual trapped ion computer and experimentally demonstrate convergence of the model to the target distribution.

Liu and Wang [26] propose the use of gradient descent to minimize the maximum mean discrepancy [79] $D(p,{q}_{{\boldsymbol{\theta }}})=\parallel {\sum }_{{\boldsymbol{v}}}p({\boldsymbol{v}})\phi ({\boldsymbol{v}})-{\sum }_{{\boldsymbol{v}}}{q}_{{\boldsymbol{\theta }}}({\boldsymbol{v}})\phi ({\boldsymbol{v}}){\parallel }^{2}$, where ϕ is a classical feature map, and the expectations are estimated from samples. Their approach allows for gradient estimates with discrete target data, which is often not possible in classical implicit models. In their simulations they successfully train QCBMs for the Bars-and-Stripes dataset and for discretized Gaussian distributions. Hamilton et al [80] implement this schema on the IBM Q20 Tokyo computer, and examine how statistical and hardware noise affect convergence. They find that the generative performance of state-of-the-art hardware is usually significantly worse than that of the numerical simulations. Leyton-Ortega et al [60] perform a complementary experimental study on the Rigetti 16Q-Aspen computer. They argue that due to the many components involved in hybrid quantum–classical systems (e.g. choice for the entangling layers, optimizers, post-processing, etc), the performance ultimately depends on the ability to correctly set hyperparameters; that is, research on automated hyperparameter setting will be key to the success of QCBMs.

Another challenge in QCBMs is the choice of a suitable loss functions. Non-differentiable loss functions are often hard to optimize; one can use gradient-free methods, but these are likely to struggle as the number of parameters becomes large. Differentiable loss functions are often hard to design; recall that since QCBM are implicit models, one does not have access to the likelihood ${q}_{{\boldsymbol{\theta }}}({\boldsymbol{v}})$. Adversarial methods developed in deep learning can potentially overcome these limitations. Figure 8(a) shows the intuition; the adversarial method introduces a discriminative model whose task is to distinguish between true data coming from the dataset and synthetic data coming from the generative model. This creates a 'game' where the two players, i.e. the models, compete. The advantage is that both models are trained at the same time, with the discriminator providing a differentiable loss function for the generator.

Zoom In Zoom Out Reset image size

Lloyd and Weedbrook [81] put forward the quantum generative adversarial network (QGAN) and theoretically examine variants where target data, generator and discriminator are either classical or quantum. We discuss the case of quantum data in the next section while here we focus on classical data. Both Situ et al [82] and Zeng et al [83] couple a PQC generator to a neural network discriminator and successfully reproduce the statistics of some discrete target distributions. Romero and Aspuru-Guzik [84] extend this to continuous target distributions using a suitable post-processing function. Zoufal et al [76] propose a QGAN to approximately perform amplitude encoding. While the best known generic method has exponential complexity, their circuit uses a polynomial number of gates. If both the cost of training and the required precision are kept low, this method has the potential to facilitate algorithms that require amplitude encoding.

One key aspect of generative models is their ability to perform inference. That is, when some of the observable variables are 'clamped' to known values, one can infer the expectation value of all other variables by sampling from the conditional probability. For example, inpainting, the process of reconstructing lost portions of images and videos, can be done by inferring missing values from a suitable generative model. Low et al [85] use Grover's algorithm to perform inference on quantum circuits and obtain a quadratic speedup over naïve methods, although the overall complexity remains exponential. Zeng et al [83] propose to equip QCBMs with this method, although this requires amplitude amplification and estimation methods that may be beyond NISQ hardware capabilities. It is an open question how to perform inference on QCBMs in the near term.

We finally consider learning tasks that are inherently quantum mechanical. As discussed in the Introduction, early hybrid approaches [11, 12] were proposed to assist the implementation of quantum algorithms (e.g. Deutsch's, Grover's, and Shor's) from datasets of input-output pairs. Quantum algorithm learning has been recently rediscovered by the community.

Morales et al [86] train PQC models for the diffusion and oracle operators in Grover's algorithm. Noting that Grover's algorithm is optimal up to a constant, the authors show that the approach can find new improved operators for the specific case of three and four qubits. Wan et al [87] train a PQC model to solve the hidden subgroup problem studied by Simon [88]. In their simulations, they recover the original Simon's algorithm with equal performance. Anschuetz et al [89] use known techniques to map integer factoring to an Ising Hamiltonian, then train a PQC model to find the ground state hence finding the factors. Cincio et al [90] train circuits to implement the SWAP test (see figure 3) and find solutions with a smaller number of gates than the known circuits.

These methods promise to assist the implementation of algorithms on near-term computers. Experimental studies will be needed to assess their scaling under realistic NISQ constraints and noise. Theoretical studies will be needed to understand their sample complexity, that is, the number of training samples required in order to successfully learn the target algorithm. Even in small-scale computers, we shall avoid exponential sampling complexity if we want these methods to be practical.

In the context of quantum state classification, Grant et al [30] simulate the training of a TTN variational circuit for the classification of pure states that have different levels of entanglement. They found that, if the unitary operations in the TTN are too simple, classification accuracy on their synthetic dataset is no better than random class assignments. When using more complex operations involving ancilla qubits the TTN is able to classify quantum states with some accuracy. Chen et al [91] simulate the training of PQC models to classify quantum states as pure or mixed, including a third possible output associated with an inconclusive result. Their circuits rely on layers of gates that are conditioned on measurement outcomes, with the purpose of introducing nonlinear behavior similar to that of neural networks.

State tomography is another ubiquitous task aiming at predicting the outcome probabilities of any measurement performed on an unknown state. To completely model the unknown state, one would require a number of measurements growing exponentially with the number of qubits. However, this can be formulated as a quantum state learning problem with the hope of minimizing the number of required measurements. Aaronson [92] studies the sampling complexity of this problem under Valiant's probably approximately correct learning model [93]. They find that for practical purposes one needs a number of measurements scaling linearly with the number of qubits. Rocchetto et al [94] experimentally verify the linear scaling on a custom photonic computer and extrapolate the value of the scaling constant. In terms of methodology, Lee et al [95] propose to train a variational circuit ${U}_{{\boldsymbol{\theta }}}$ that transforms the unknown state $| \psi \rangle$ to a known fiducial state $| f\rangle$. The unknown state can be reproduced by evaluating the adjoint circuit on the fiducial state, that is, $| \psi \rangle \approx {U}_{{\boldsymbol{\theta }}}^{\dagger }| f\rangle$. A related learning tasks is that of quantum state diagonalization for mixed states. LaRose et al [96] propose to train a variational circuit ${U}_{{\boldsymbol{\theta }}}$ such that the density matrix $\tilde{\rho }={U}_{{\boldsymbol{\theta }}}^{}\rho {U}_{{\boldsymbol{\theta }}}^{\dagger }$ is diagonalized, hence representing a classical probability distribution.

In the previous section and in figure 8(a) we introduced QGANs for classical data. We now discuss the case where all components are quantum mechanical, hence enabling the generative modeling of quantum data. The discriminator, now taking target and synthetic quantum states in input, aims at modeling the measurement for optimal distinguishability, also known as the Helstrom measurement [97]. In turn, the generator tries to make the task of distinguishing more difficult by minimizing its distance from the target state [81, 98]. In practice, this game can be implemented by coupling two PQC models and optimizing them in tandem. For example, Dallaire-Demers and Killoran [47] propose a QGAN that generates states conditioned on labels. This may find application in chemistry where the label is 'clamped' to a desired physical property and the model generates new molecular states accordingly. Benedetti et al [98] propose a QGAN that generates approximations of pure states. They numerically show how the depths of generator and discriminator impact the quality of approximation. They also design a heuristic for stopping training, which is a non-trivial problem even in classical adversarial methods. Hu et al [99] experimentally demonstrate adversarial learning on a custom superconducting qubit.

Finally, PQC models can be used to attack well-known problems in quantum information from a novel machine learning perspective. Let us see some examples within the context of compression, error correction and compilation.

Romero et al [77] propose a quantum autoencoder to reduce the amount of resources needed to store quantum data. As shown in figure 8(b) an encoder circuit U_enc is applied to the quantum data stored in n qubits. After tracing out n − k qubits, a decoder circuit U_dec is used to reconstruct the initial state. The circuits are trained to maximize the expected fidelity between inputs and outputs, effectively performing a lossy compression of an n-qubit state into a k-qubit state.

Fault-tolerant quantum computers require error correction schemes that can deal with noisy and faulty operations. Leading proposals such as the color code and the surface code devote a large number of physical qubits to implement error-corrected logical qubits (see Gottesman [100] for an introduction to quantum error correction). Johnson et al [101] suggest that a reduced overhead could be achieved in NISQ devices by training encoding and recovery circuits to optimize the average code fidelity.

The implementation of a quantum algorithm is also limited by the available gate set and qubit-to-qubit connectivity of the underlying hardware. This is where quantum compilers come into play, by abstracting the user from the low-level details. Khatari et al [102] propose to train a hardware-efficient variational circuit ${U}_{{\boldsymbol{\theta }}}$ to approximately execute the same action as a target unitary $U={{\rm{e}}}^{-{\rm{i}}{ \mathcal H }t}$.