Quantum machine learning for image classification

This section describes the Modified National Institute of Standards and Technology (MNIST) [56] dataset. The MNIST database consists of a large collection of grey-scale handwritten numbers, ranging from 0 to 9. Sample images from the dataset are presented in figure 1(a). Each image has a resolution of $28\times28$ pixels, and the main objective is to classify each image by assigning a class label using a neural network. In other words, the task is to recognize which digit is present in the image. This dataset is widely used for making the first steps in the sphere of machine learning. Nevertheless, it is worth studying as it helps to test the performance of various neural network models [57, 58], especially models with PQCs [36, 53, 59]. The MNIST dataset used in this study comprises a total of $70\,000$ images, with $60\,000$ images reserved for training and $10\,000$ images for testing. However, in certain cases, it may be advantageous to reduce the number of images in order to expedite the training process and gain immediate insights into the model's performance.

Zoom In Zoom Out Reset image size

Despite being a widely used dataset, the MNIST database contains a few images that are broken or ambiguous, posing a challenge even for human evaluators. Figure 1(b) provides examples of such images. However, our hybrid model can accurately determine the number in such images with over 99% accuracy.

The integration of QML techniques, such as HQNNs, holds immense promise in the field of medical image classification. Quantum computing's inherent capacity to handle complex and high-dimensional data, coupled with the power of neural networks, offers a unique advantage for solving intricate medical image analysis tasks. HQNNs leverage quantum computing's ability to efficiently perform certain mathematical operations required for image feature extraction and classification, potentially leading to breakthroughs in medical diagnosis and treatment [60].

To test our models in this sphere, we utilized the Medical MNIST dataset [61], which comprises a total of $58\,954$ medical images distributed across six distinct categories: Abdomen Computer Tomography (AbdomenCT), Breast Magnetic Resonance Imaging (BreastMRI), Chest x-ray (CXR), Chest Computer Tomography (ChestCT), Hand x-ray (Hand), and Head Computer Tomography (HeadCT). Sample images from this dataset can be observed in figure 2. Each of these images possesses a resolution of $64 \times 64$ pixels, with our primary goal being the classification of each image through the utilization of a neural network.

Zoom In Zoom Out Reset image size

It is noteworthy that all images within this dataset employ 3 channels, adding an additional layer of complexity compared to the original MNIST dataset. Furthermore, we conducted various data preprocessing techniques, including random rotations of up to 10 degrees, random horizontal flips, and resizing to dimensions as large as $244 \times 244$ pixels. These steps were implemented to stabilize the training process and enhance the performance of our models.

Similar to the MNIST classification task, we divided the entire dataset into two subsets: a training set consisting of $47\,163$ samples and a testing set containing $11\,791$ samples.

The CIFAR-10 dataset is a pivotal resource in the domain of computer vision and image classification. Developed as part of the Canadian Institute for Advanced Research (CIFAR) program, this dataset serves as a fundamental benchmark for assessing the performance of machine learning algorithms in the context of image classification tasks [62]. CIFAR-10 is comprised of a total of $60\,000$ images, which are divided into training and testing sets with $50\,000$ and $10\,000$ samples in each respectively. Each image in the CIFAR-10 dataset is of size $32\times 32$ pixels. These images are color images, incorporating three color channels: red, green, and blue. Consequently, every image is represented as a $32\times 32\times 3$ tensor. One of the distinguishing features of CIFAR-10 is its categorization into ten distinct classes, each representing a different object category. These classes are as follows: airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck. This diverse set of classes ensures that the dataset is suitable for a wide range of image classification challenges, spanning various domains and object types. As it has more classes than Medical MNIST and input images have bigger resolution than in MNIST, classifying images from CIFAR-10 seems to be complex task. Sample images from this dataset are shown in figure 3.

Zoom In Zoom Out Reset image size

Figure 4 depicts the general structure of the classical convolutional part of the proposed HQNN-Parallel. The convolutional part of the network is comprised of two main blocks, followed by fully-connected layers. In this study, we utilized Rectified Linear Unit (ReLU) as the activation function [63]. Batch Normalization [64] is employed in the network as it stabilizes the training process and improves the accuracy of the model.

Zoom In Zoom Out Reset image size

The first block of the convolutional part of the HQNN-Parallel comprises a convolutional layer with one input channel and 16 output channels, utilizing a square kernel of size $5\times5$. The layer operates with a stride of one pixel and applies a two-pixel padding to the input data. We opted for the $5\times5$ kernel size primarily to maintain the input image's spatial dimensions. By implementing a two-pixel padding, the $5\times5$ kernel facilitates the creation of 16 channels without altering the original pixel size of $28\times28$. Additionally, the larger kernel sizes, such as $5\times5$, can capture more complex and diverse features from the input image compared to smaller kernels [65, 66]. This aligns with our objective of preserving both spatial and feature details of the image during convolution. Batch Normalization is applied to the output of the convolutional layer, followed by an activation function (ReLU) and MaxPooling [67] with a kernel size of two pixels. The resulting feature map has dimensions of $16\times14\times14$ pixels.

The second block contains a convolutional layer with 16 input channels and 32 output channels, utilizing the same kernel size and padding as the previous layer. The MaxPooling parameters remain unchanged, resulting in a feature map with dimensions of $32\times7\times7$ pixels, which will become an input for the fully connected part of the network.

Following the convolutional block, the HQNN- Parallel, continues with a hybrid dense part, as shown in figure 4. The $32\times7\times7$ feature map produced by the convolutional part serves as input for the first dense layer, which transforms the feature map from 1568 to n features. The value of n is determined by the chosen quantum part and represents the total number of encoding parameters in the quantum layers.

Each quantum layer is designed to maintain the number of input and output features, and the output of the quantum layer is fed into the second classical fully connected layer. This layer performs the final transformation and maps the n input features to 10 output features, corresponding to the number of classes into which the images can be classified. After each classical dense layer, Batch Normalization and ReLU activation are applied.

It is worth noting that the structure of the HQNN- Parallel, including the number of layers and the number of features, can be adjusted to optimize the performance on a specific task.

The quantum component of the proposed HQNN-Parallel, depicted in figure 4, consists of c parallel quantum layers, each of which is a PQC composed of three parts: embedding, variational gates, and measurement. The input data to the quantum layers are n features from the previous classical fully connected layer, divided into c parts, with each part being a vector of q values, $x = (\phi_1, \phi_2, {\ldots}, \phi_q) \in \mathbb{R}^q$. To encode these classical features into quantum Hilbert space, we use the 'angle embedding' method, which rotates each qubit in the ground state around the X-axis on the Bloch sphere [68] by an angle proportional to the corresponding value in the input vector: $\left|\psi\right\rangle = R^\mathrm{emb}_x(x)\left|\psi_0\right\rangle$, where $\left|\psi_0\right\rangle = \left|0\right\rangle^{\otimes q}$. This operation encodes the input vector into quantum space, and the resulting quantum state represents the input data from the previous classical layer. It is important to note that n is divisible by q since the input data vector is divided into $c = n/q$ parts, with each part serving as input to a PQC.

The encoding part for each PQC is followed by a variational part, which consists of two parts: rotations with trainable parameters and subsequent CNOT operations [69]. The rotations serve as quantum gates that transform the encoded input data according to the variational parameters, while the CNOT operations entangle the qubits in the PQC. The depth of the variational part, denoted as i, is a hyperparameter that determines the number of iterations of the rotations and CNOT operations in the PQC. It is important to note that the variational parameters for each PQC are different in each of the i repetitions and for each of c quantum circuits. Thus, the total number of weights in the quantum part of the HQNN-Parallel is calculated as $q\cdot 3i\cdot c$.

After performing these operations, measurement in the Pauli basis matrices is performed, resulting in

$$\begin{equation} v^{\left(j\right)} = \left\langle 0\right| {R^\mathrm{emb}_x\left(\phi_j\right)}^\dagger {U\left(\theta\right)}^\dagger Y_j U\left(\theta\right) R^\mathrm{emb}_x\left(\phi_j\right)\left|0\right\rangle, \end{equation} \tag{ 1 }$$

where Y_j is the Pauli-Y matrix for the jth qubit, $R^\mathrm{emb}_x(\phi_j)$ and $U(\theta)$ are operations, performed by the embedding and trainable parts of the PQC, respectively, and θ is a vector of trainable parameters. After this operation, we have the vector $v\in\mathbb{R}^q$. The outputs of all the PQCs would be concatenated to form a new vector $\hat{v}\in\mathbb{R}^n$ that is the input data for a subsequent classical fully-connected layer. This layer, being the final layer in the classification pipeline, produces an output in the form of a probability distribution over the set of classes. In our case, each input image is associated with one of the ten possible digits from 0 to 9, and the output of each neuron represents the probability that the image belongs to that class. The neuron with the highest output probability is selected as the predicted class for the image.

More detailed theoretical analysis, involving ZX-calculus reduction and Fourier expressivity of the HQNN-Parallel was conducted in appendix.

As described above, the HQNN-Parallel was trained on MNIST dataset section 2.1.1. No preprocessing is applied, so the entire collection is used for training ($60\,000$ images are in the training set and $10\,000$ are in the test set). In the context of training the proposed HQNN-Parallel, the ultimate objective is to minimize the loss function during the optimization process. The cross-entropy function is employed as the loss function, given by:

$$\begin{equation} l = - \sum_{c = 1}^{k}y_c\text{log}{p_c}, \end{equation} \tag{ 2 }$$

where p_c is the prediction probability, y_c is either 0 or 1, determining respectively if the image belongs to the prediction class, and k is the number of classes.

The parameters of the classical layers are optimized using the backpropagation algorithm [70], which is automatically implemented in the PyTorch library [71]. The backpropagation algorithm is used to calculate the gradients of the loss function with respect to the parameters of the network, allowing for their optimization via gradient descent. However, the use of quantum layers in this task is more complex than classical methods for computing gradients. To overcome this challenge, we employ the PennyLane framework [72], which provides access to a variety of optimization techniques. We utilize the parameter shift rule [73], which is compatible with physical implementations of quantum computing [74]. This method involves evaluating the gradient of a quantum circuit by shifting the parameters in the circuit and computing the corresponding change in the circuit's output. The resulting gradient can then be used to update the circuit's parameters and iteratively minimize the loss function. By using the parameter shift rule, we are able to efficiently optimize the variational parameters in the quantum layers of the HQNN, enabling the network to learn complex patterns in the input data and achieve accurate results.

In the process of solving the problem, we tried various architectures of quantum layers. The most successful architecture for the HQNN-Parallel used a quantum layer with 5 qubits and 3 repetitions of the strongly entangling layers. The number of quantum layers is equal to 4.

The HQNN-Parallel managed to achieve a 99.21% accuracy on MNIST dataset. In order to compare the performance of the HQNN with a classical CNN, the convolutional part of the HQNN was held constant, while the quantum part was replaced with a classical dense layer containing n neurons. This modified CNN was then trained on the same MNIST dataset. A comparison of the training outcomes is depicted in figures 5(a) and (b).

Zoom In Zoom Out Reset image size

The trainable parameters, as well as the primary training and testing results, for both the HQNN-Parallel and the CNN, are summarized in table 1 and illustrated in figure 5(c). From these results, it is evident that the most successful implementation of the HQNN-Parallel surpasses the performance of a CNN that possesses approximately eight times more parameters.

Table 1. Summary of the results for the HQNN-Parallel and its classical analog, CNN.

Dataset	Model	train loss	test loss	test acc	param num
MNIST	CNN	0.0205	0.0449	98.71	372 234
	HQNN	0.0204	0.0274	99.21	45 194
Medical	CNN	$0.456 \cdot 10^{-2}$	$0.396 \cdot 10^{-2}$	82.64	247 642
MNIST	HQNN	$0.429 \cdot 10^{-2}$	$0.332 \cdot 10^{-2}$	82.78	247 462
CIFAR-10	CNN	0.3659	0.5484	82.78	81 698
	HQNN	0.3851	0.5208	82.64	81 578

HQNN-Parallel with its classical analog were also tested on Medical MNIST dataset. The HQNN-Parallel managed to achieve a 99.97% accuracy. The classical CNN showed less accurate results with 99.96% accuracy on test data. It is worth noting that HQNN model had 247 462 trainable parameters and classical CNN had 247 642. A comparison of the training outcomes is depicted in figures 6(a) and (b).

Zoom In Zoom Out Reset image size

To confirm the generalizability of hybrid architecture we tested hybrid and classical models on CIFAR-10 dataset. The HQNN-Parallel managed to achieve an 82.78% accuracy. The classical CNN showed less accurate results with 82.64% accuracy on test data. It is worth noting that HQNN model had $81\,578$ trainable parameters and classical CNN had $81\,698$. A comparison of the training outcomes is depicted in figures 6(c) and (d).

In this section, we provide a comprehensive overview of the model's architecture employed during training on the MNIST dataset. While the foundational structure remained consistent when testing on both the Medical MNIST and CIFAR-10 datasets, there were variations. Specifically, input dimensions varied based on image sizes. Additionally, the number of convolutions differed: two for Medical MNIST and three for CIFAR-10. Furthermore, the neuron count in the output layer was adjusted in line with the respective number of classification classes. The quantity of qubits and quantum layers remained unchanged across all experiments.

The general architecture of a quanvolutional layer [75] is shown in figure 7. Similar to classical convolutional layers, the quanvolutional layer comprises a kernel of size n × n pixels that convolve the input image, producing a lower-resolution output image. However, the quanvolutional layer is unique in the sense that its kernel is implemented using a quantum circuit consisting of n_q qubits. The circuit can be decomposed into three distinct parts: classical-to-quantum data encoding, variational gates, and quantum measurement. These parts work together to determine the kernel's action on the input image.

There are plenty of encoding (embedding) methods to transfer classical data into quantum states. In this section, as in the previous one, we use the 'angle embedding' technique. It is achieved by rotating the qubits from their initial $\left|0\right\rangle$ value with the $R_y(\varphi)$ unitaries, where ϕ is determined by the value of the corresponding pixel. After the classical data is encoded, the quantum states undergo unitary transformations, defined by the variational part.

The variational part in the quanvolutional layer usually consists of arbitrary single-qubit rotations and CNOT gates, arranged in a particular way determined by the researcher. The unitaries in the PQC are parameterized by a set of variational parameters, which are learned via training the neural network. The ultimate goal of the model training is to find a measurement basis (by tweaking variational gate parameters) that tells us the most information about a fragment of a picture confined by the quantum kernel.

Finally, for each wire, the expectation value of an arbitrary operator is calculated to obtain the classical output. As it is a real number, it represents the kernel's output pixel, while each wire yields a different image channel. For instance, a quanvolutional kernel of size $2 \times 2$ has a 4-qubit circuit, which transforms one input image into four images of reduced size.

This subsection details the architecture of the HQNN-Quanv, which is shown in figure 7. At first, a simple angle embedding of the classical data via $R_y(\varphi)$ single-qubit rotations on each wire is used, where the original pixel value $[0,1]$ is scaled to $\varphi \in [0, \pi]$. Then, we have a variational circuit part, which consists of 4 single-qubit rotations, parameterized with trainable weights, as well as three CNOT gates. At the end of the circuit, we measure the expectation value $\langle \sigma_z \rangle$ of the Pauli-Z operator on each qubit. Each channel is a picture with 4 × 4 pixels. After that, four output channels are flattened and fed into a fully connected layer, which yields a digit's probability.

In this section, we describe the training process. In order to reduce the training time of the HQNN-Quanv, only 600 images from the MNIST dataset section 2.1.1 are used with 500 of them acting as training data and 100 as test data. We also use PyTorch's resize transform with bilinear interpolation to downscale images from $28 \times 28$ to $14 \times 14$ pixels. We still use a cross-entropy loss function.

While the classical model has only one way of training weights via backpropagation, the HQNN has several options, such as the parameter-shift rule, adjoint differentiation [76] or backpropagation (which, of course, is impossible on a real quantum computer). Adjoint differentiation seems to have the most favourable scaling with both layers and wires [77], but on this particular circuit (figure 7) backpropagation proved to be quicker.

Considering everything stated above, let us see the results of the training. We trained two CNNs with different numbers of output channels and one HQNN for 20 epochs (figures 8(a) and (b)). The models were intentionally made simple and had sufficiently few parameters so as to avoid overfitting on the relatively small dataset. Test accuracies of these models are presented in figure 8(c). For each epoch, the accuracy is averaged over 10 models with random initial weights. The error bars depict one standard deviation.

Zoom In Zoom Out Reset image size

At the end of the training, the HQNN-Quanv had a test accuracy of $0.67 \pm 0.01$, which is close enough to the CNN4 result of $0.66 \pm 0.02$, while CNN1 had $0.53 \pm 0.02$. The HQNN model has only 4 trainable weights in its quanvolutional kernel, which parameterizes rotation gates in the PQC. CNN1 and CNN4 have 4 and 16 trainable parameters in their convolutional kernels, respectively. Therefore, the HQNN's performance based on the accuracy score is equivalent to CNN4's, which has four times many weights in its kernel.