Edge to quantum: hybrid quantum-spiking neural network image classifier

The extreme parallelism property warrant convergence of neural networks with that of quantum computing. As the size of the network grows, the classical implementation of neural networks becomes computationally expensive and not feasible. In this paper, we propose a hybrid image classifier model using spiking neural networks (SNN) and quantum circuits that combines dynamic behaviour of SNN with the extreme parallelism offered by quantum computing. The proposed model outperforms models in comparison with spiking neural network in classical computing, and hybrid convolution neural network-quantum circuit models in terms of various performance parameters. The proposed hybrid SNN-QC model achieves an accuracy of 99.9% in comparison with CNN-QC model accuracy of 96.3%, and SNN model of accuracy 91.2% in MNIST classification task. The tests on KMNIST and CIFAR-1O also showed improvements.


Introduction
Image classification is one of the widely applied uses of deep neural networks [1][2][3]. As the images are detected in parallel, the operations on the pixels can be performed in parallel. Any manipulations of the pixels and its neighbourhood such as using spatial filters can be executed in parallel. The most common filtering operation with images is implemented as convolution operation, i.e. a weight summation of input pixels. The weighted summation operation is also at the heart of neural computation. This similarity in operation converges with the convolution neural networks [4], that uses weight summation to implement layer of convolution neural networks.
Similar to convolution layers, the weighted summation followed by activation function in the neural network can be used to build deep neural networks that can make predictions with high accuracy. These networks in essence are inspired from the functional and structural properties of the human brain. Among the various neural networks, the spiking neural network (SNN) [5,6] claims to be biologically more plausible than the other networks, due to the spike based encoding and extreme sparsity it can exhibit.
Spiking neural networks (SNNs) makes use of the principle of firing of neurons, where an input stimuli is converted to a series of binary pulses. This is a spatio-temporal encoding that is sparse and asynchronous, and are processed parallel in an SNN. The main advantages of such SNNs are its fast inference times, they can be used to respond to events, and can be implemented in low power hardware. The speed and low power makes it a useful tool for building efficient deep learning architectures. On the other hand, they are not easy to train, with most common approach being spiking type backpropagation and STDP.
Quantum neural networks [7,8] is an emerging area of study, that aim to make use of the parallel computing abilities of quantum computers in speeding up neural computations. To implement such a system, the classical bits are converted to quantum bits (qubits), that observe the properties of superposition, entanglement and tunnelling. While, superposition and entanglement has been implemented with quantum computers, the hardware implementation of quantum tunnelling in quantum computer [9,10] for commercial use is yet to be fully utilised for large scale quantum applications [11,12]. The extreme computing at edge, can help speedup the inference operations near the sensors. Combining the edge computing with that of the quantum, can help leverage high speed processing at edge during inference, along with accelerated learning using quantum computers. The dynamic and biological plausible encoding demonstrated by SNN, probabilistic nature of the qubits and the extreme parallelism a quantum computer offer makes the implementation of a hybrid processing system for neural network useful. In this paper, we present a hybrid neural architecture that using SNN followed by a quantum circuit for optimal inference. The paper is organised as follows: section 2 provides the background, section 3 presents the overview of the proposed hybrid model, section 4 presents the classification results and section 5 provides the main conclusions of this work.

Background
In this work, we combine the SNN with quantum gates. There are two ways to create hybrids of neural network with quantum gates to create quantum neural networks [13][14][15]. In first approach the basic computations for the neural network can be replaced with quantum gates for speeding up the neural learning and inferences. In the second approach, the classical neural network is complemented with a quantum gate that helps to improve learning and inference decisions. In this work we focus on applying the quantum gates following the second approach, with SNN as the classical neural network.

Spiking neural networks
Synaptic weight updation in STDP is given by where W(x), t n post and t f pre denotes the STDP function (learning window), postsynaptic firing times and presynaptic firing times respectively. Synaptic activation in STDP is done based on the relative timing of pre and postsynaptic firing. If presynaptic firing comes before postsynaptic neuron, i.e. if t post − t pre > 0, then the synapse is potentiated, else synaptic potential is decreased. The magnitude of increase or decrease of synaptic efficacy is provided by where A + and A − the maximum synaptic change and maximum synaptic depression respectively. τ + and τ − are time constants. The synaptic changes as a function of relative spiking times is represented by figure 1. Spike based backpropagation [16] is an adaptation of standard backpropagation [17]. Three phases in a spike based backpropagation are forward propagation, backward propagation and weight updation. During the forward propagation, Poisson-distributed spike trains are taken as input and neurons in each layer fires based on the weighted sum of the input spikes compared to the threshold membrane potential. The partial derivatives of final error with respect to weights ( ∂(E) ∂w ) are obtained and propagated backwards up to the input layer for updating the weights.
The biological neural networks in human brain consists of neurons that encode, receive, process and transmit information as electrical impulses. Neuromorphic systems aims to mimic these processes, and achieve intelligent processing capabilities similar to human brain. SNNs [5,19] can be said as belonging to the third generation of neural networks that are considered to be resembling biologically realistic model of neurons.
In SNNs, impulses (spikes) are used to stimulate electrical neuron responses to a given stimuli. SNNs learn in a distributed way using spike timing dependent plasticity (STDP) encapsulating Hebbian learning like behaviour which follows the rule that 'cells that fire together wire together'. Mimicking the biological neurons, the signals accumulate in neurons of SNN until a threshold is reached. Once the input activation crosses the threshold value, the neuron fires and is followed by a refractory period before firing again as depicted in figure 2. This forms the primary response of the SNN neurons, and can be approximated with different types of threshold functions.

Quantum computing
The rapid progress being made in the field of quantum computing has made it a contender for machine learning algorithms. Speeding up the learning process in neural networks is one of the biggest bottlenecks of large scale neural network implementations. Quantum computing applies the postulates in quantum mechanics to realise various information processing tasks. Unlike classical computing which is made of binary digits (i.e. bits), quantum computing deals with quantum bits (or qubits) in Hilbert space and can assume values as a linear superposition of state zero and state one where α and β are complex numbers such that Quantum computing can solve problems which are otherwise impossible by classical computers like factorization of large numbers using Shor's algorithm [20], solving Pell's equation [21] etc. There has also been efforts to apply quantum computing to machine learning problems. Quantum machine learning [22] is theoretically proved to be faster and efficient than its classical counterpart.

Proposed hybrid SNN-QC model
In the presented work, we propose an image classifier as a hybrid model using SNN and quantum circuit. The basic architecture of the model is as shown in figure 3. The proposed hybrid model takes image as input which is passed through the SNN block.  The SNN block in the proposed model used a leaky integrate-and-fire model. The input image vector x i is read and multiplied with a weight matrix The current weighted input is added with the decayed version of the previous time steps. Since the previous inputs are decayed over time, they play a lesser role in the activation of neuron after each step. The more recent presynaptic input will have a greater impact on the firing of neurons thereby following 'neurons that fire together, wire together', the dictum of Hebbian learning. The excitation at any step is given by where λ is the decay factor. Neuronal firing occur and output spikes get generated when the potential v(t) reaches the threshold value, v th . Once fired, a refractory time period (t ref ) is provided to the neurons and they do not fire again during this period. If the sum of inputs has not reached threshold limit, the next input is read and the whole process is repeated. The refractory period provided to each neuron following its activation is similar to the resting period in biological neuron. Since the biological neurons exhibit spiking patterns, and rely on dynamic behaviours, the SNNs provide a more realistic model of biological networks.
Training of the quantum circuit is based on the output spike of the SNN block which is given as the trainable parameter θ in the quantum circuit. For simplicity, we used a single qubit quantum circuit consisting of Hadamard (H) gate and RY rotation block for the training process. H gate is represented by the matrix, H gate in the model takes a random state |x and creates a superposition of |0 and |1 , thereby moving away from the poles of Bloch sphere. RY rotation block which follows the H gate provides a single qubit rotation represented by (6) and (7) and is used in training the quantum circuit The output of the H gate is rotated about the y-axis by the RY rotation block wherein the rotation angle θ is based on the output vector of the SNN block. σ z expectation in the z-basis is measured as the output of the quantum circuit Image classification problem is addressed using the proposed model based on the σ z expectation value. The quantum circuit offers a feed-forward learning mechanism without the need for extensive backpropagation updates for optimising the neural weights. Another approach to build quantum based neural networks is to translate the neural computations in quantum circuits. However, such implementations are fully realised in quantum computers that can introduce quantum errors across the neural architecture than a quantum neural network hybrid circuits where it is only partially effected by quantum errors.

Results
The proposed model was trained and tested using images from various datasets like MNIST, KMNIST and CIFAR-10. MNIST and KMNIST datasets consist of images of size 28 × 28 pixels whereas CIFAR10 dataset comprises of images of size 32 × 32 pixels. Various performance parameters of the proposed model are discussed in this section. Table 1 provides the comparison between SNN, hybrid convolutional neural network (CNN)-quantum circuit model and the proposed hybrid SNN-quantum circuit model when trained and tested with MNIST, KMNIST and CIFAR10 images. The reported results are based on simulations done on models with input layer having 28 × 28 neurons and 2 hidden layers with 256 and 64 neurons respectively. The datasets were trained in batch size of 10 for 25 iterations. Multiple trials were carried out keeping all parameters (number of layers, number of neurons in each layer, batch size, number of epochs etc) same for comparison and the reported results are the mean value of the various simulations. PyTorch was used for simulating classical computing whereas quantum circuits were simulated using Qiskit.
The proposed model achieved high accuracy of 99.9% when trained for 25 epochs with MNIST dataset as compared to 91.2% achieved by SNN model alone and 96.3% achieved by hybrid CNN-QC. With KMNIST and CIFAR-10, the accuracy achieved by the proposed model were 95.4% and 67.2% respectively, which is again higher than CNN-QC and SNN models.
Another advantage of the proposed model over other models is its higher precision value which is the ratio of true positives to the sum of true positives and false positives. The proposed model has a precision of 1.0, 0.97 and 0.68 with MNIST, KMNIST and CIFAR-10 datasets respectively.
The recall parameter provides the sensitivity of a given model by comparing the predicted positive observations with actual positives i.e., sum of true positives and false negatives. The proposed model has a recall value of 1.0, 0.95 and 0.93 for the three datasets which is better than other models as shown in table 1.
F1 score is practically a better parameter to observe than accuracy as it is the weighted average of recall and precision. The proposed hybrid SNN-QC model provides an F1 score of 1.0 compared to 0.96 and 0.98 by

Negative log likelihood loss
Negative log likelihood (NLL) loss function shows how good a model is performing. This cost function is expected to be lower for a good classifier model. The NLL loss function of the proposed model is plotted against training iterations in figure 5 from which is can be seen that NLL values improve with each training epoch and is stabilizing around −0.98.

Noisy input
In real world scenario, data is seldom clean. The quality of input images may vary drastically due to environmental factors, lighting conditions or quality of image capturing device. The susceptibility of the classifier to the input noise is an important performance criterion. In this paper, we studied the performance of the proposed hybrid SNN-QC model against various noises using the MNIST dataset ( figure 6). The obtained results are given in table 2.

Quantum error
Quantum computers manipulate quantum states using unitary gates which are prone to errors. These quantum errors can affect the learning process as trainable parameters are wrongly trained. This could lead to wrong predictions or misclassifications. We studied the impact of depolarizing quantum error channel in the performance of the proposed model. The various performance parameters are given in table 3 and the accuracy of the proposed hybrid model against increasing depolarizing error is plotted in figure 7. The depolarizing channel is given by: where λ is the depolarizing error parameter and n is the number of qubits. Equation (9) represents an identity channel when λ = 0 and a completely depolarized channel when λ = 1. Proposed model exhibited high level of accuracy till the depolarizing error crossed 0.70 after which the performance parameters of the model started declining.

Discussions
Results discussed in the previous sections show the superior performance of the proposed hybrid model compared to SNNs alone and hybrid model using convolutional neural networks and quantum circuits.
All classification tests on KMNIST, MNIST and CIFAR-10 showed improved performances. In particular, the proposed model on MNIST tests exhibited very high accuracy and a perfect precision, recall and F1 score. The model also performed well when tested with noisy inputs as well as against quantum channel errors. The proposed model integrates the superiority of both the SNNs as well as quantum circuits. Hebbian learning and spike-time-dependent-plasticity (STDP) in the SNNs allows updation of the weight of a synapse in real time improving its learning efficiency. SNN block of the proposed model will also help in reducing power consumption owing to its spike based activation. In addition, the quantum circuit block helps in performing complex computations in a much faster way and also makes the model robust against noisy data. Eventhough in classical computers, CNNs have an upperhand over SNNs in terms of accuracy, which can mainly be attributed to the difficult training process of SNNs, the proposed hybrid model exhibits superior performance when compared to other models as reported in table 1. The following two aspects have enabled the model to perform better compared to other models-a different training algorithm and the introduction of a hybrid networking with feed-forward mechanism, thereby equipping the model with accelerated learning and faster decision-making.
In most practical quantum systems, they are prone to high levels of quantum gate errors. Hence, translating the classical neural networks to quantum gates often results in low classification performance. It is for this reason that a hybrid network become a more practical solution. For example from table 3, it can be seen that the proposed hybrid quantum network can tolerate up to 0.6 error probability for the quantum gates with minor impact on the classification accuracy. This also advocates the need to implement classical neural chips to coexist along with quantum gates.
Another major point to observe is that the proposed QSNN, utilizes the extreme parallelism possible by the quantum computers. In particular, the training is implemented with feedforward learning algorithm with quantum circuit as in figure 3, reducing the need for feedback based learning such as the more popular backpropagation approaches for optimising the performance. Further, the hybrid system can be implemented with the quantum gates in the quantum processor while classical bits in the external co-processor. As such hybrid systems are inevitable and more common, the proposed system utilises the benefits from both.
The results obtained are very promising that the proposed model may be further developed into a full computation system with its basic architecture being depicted in figure 8. The system can be broadly divided into two parts-an edge computing side and a supercomputing side with SNN being used in the edge computing side whereas supercomputing part being handled by a quantum computer. Extremely large amount of data can be handled by using multiple spiking networks. The outputs from the various SNN networks are then given as input to the supercomputing side. Making use of the proposed hybrid quantum-spiking system will enable faster decision-making as extreme parallelism can be achieved by converging SNNs with quantum circuits. Accelerated learning as well as quick decision making provided by the quantum circuits enables the proposed model to be a contender in edge computing applications and provides a better option while dealing with larger datasets. Furthermore, feed forward learning mechanism offered by the quantum circuits helps to overcome the issue of overfitting unlike the models using extensive back-propagation techniques. As the SNNs when arranged as a distributed system can handle multiple data instances in parallel, large amount of data can be processed at high response times. SNNs with its fast inference times can respond to events extremely quickly. At inference level, there is no need for fine tuning and weight variations. The learning process is also done in an accelerated rate owing to the use of quantum computers. Hence, speedup is achieved at both training and inference levels. Even so, when it comes to energy consumption, quantum computers are definitely far more energy consuming than GPUs, mainly due to, requirement of refrigeration to attain superconductivity. Furthermore, quantum circuits poses its own complexities such as quantum errors that are difficult to get rid off. This downside cannot be neglected. However, with future advancements in energy efficient quantum computers, these drawbacks are expected to be overcome.

Conclusion
In this paper, we reported a hybrid image classifier model using SNNs and quantum circuits. The proposed hybrid SNN-QC model showed improved performance in comparison with hybrid CNN-QC and SNN models when tested on MNIST, KMNIST and CIFAR-10 datasets. The performance improvement was benchmarked based on accuracy, precision, recall and F1-score metrics.
The signal noise performance was analysed for two noise types common in image sensors such as Gaussian, and salt & pepper. The proposed method showed high levels of tolerance to the noise. The quantum gate error was also studied, and the SNN-QC showed error tolerance for up to 0.7 error probability when tested on MNIST dataset. Overall, we observe an improvement in the classification performance of SNN when converted to the proposed hybrid SNN-QC model.

Data availability statement
All data that support the findings of this study are included within the article (and any supplementary files).