Visual explanations of machine learning model estimating charge states in quantum dots

Charge state recognition in quantum dot devices is important in the preparation of quantum bits for quantum information processing. Toward auto-tuning of larger-scale quantum devices, automatic charge state recognition by machine learning has been demonstrated. For further development of this technology, an understanding of the operation of the machine learning model, which is usually a black box, will be useful. In this study, we analyze the explainability of the machine learning model estimating charge states in quantum dots by gradient-weighted class activation mapping, which identified class-discriminative regions for the predictions. The model predicts the state based on the change transition lines, indicating that human-like recognition is realized. We also demonstrate improvements of the model by utilizing feedback from the mapping results. Due to the simplicity of our simulation and pre-processing methods, our approach offers scalability without significant additional simulation costs, demonstrating its suitability for future quantum dot system expansions.


INTRODUCTION
Semiconductor quantum bits (qubits) that utilize electron spins in semiconductor quantum dots are expected to be good candidates for future qubits because of their high operation fidelity and excellent integration properties.Basic operations such as single-qubit[1, 2] and twoqubit operations [3,4] have been demonstrated, and quantum error correction has also been demonstrated [12] with improved operation fidelity [5][6][7][8][9][10][11].Furthermore, attempts are being made to construct large-scale quantum systems using semiconductor integration technology [13][14][15][16][17][18][19][20].It is essential to trap one electron in each quantum dot to construct semiconductor qubits.Charge state tuning in semiconductor quantum dot devices is required for this purpose.In previous experiments, this has been mainly accomplished by manually adjusting the gate voltages to achieve the desired charge states.However, this approach requires a significant amount of time to learn and execute.This will make tuning large-scale semiconductor quantum systems difficult in the future.
In ML methods, the amount of labeled data required for network training is enormous, making the preparation of labeled experimental data as training data a labor-intensive task.Consequently, simulations have been utilized to generate training data.This approach has led to the development of a method that utilizes the Thomas-Fermi model and deep convolutional neural networks (CNN) for recognizing charge states [28].Also, by adding various types of noise that could be observed in real experiments to the training data, the noise-robust charge states recognition has been demonstrated [39].
Thus far, charge state recognition has primarily been demonstrated in double quantum dots [28,31,39].Looking towards the future large-scale integration of quantum dots, there is a need to explore scalable methods.Also, while the Thomas-Fermi model with added noise is highly useful as a physical model, the necessity of such physical accuracy in a data generation model for machine learning has not been extensively debated.For instance, humans tend to estimate charge states by focusing mainly on charge transition lines, without giving much consideration to background noise.Therefore, generating training data with a physical model that reflects these characteristics could lead to the realization of an estimator with judgment criteria similar to those of humans.
Such an estimator would likely be more user-friendly for humans, as it would share the same criteria for judgment.To achieve this, it is important to clarify the criteria for judgment, which is usually a black box.
In this study, we analyze the operation of the ML model that estimates the charge states in double quantum dots.We propose pre-processing the data through binarization, which enables us to use a simple simulation model to generate training data.We also visualize the interest of the model by gradient-weighted class activation mapping (Grad-CAM) [40] and confirm that they are on the charge transition lines.By incorporating feedback from the Grad-CAM results, we demonstrate improvements in the ML model.

PREPARATION OF THE CHARGE STATE ESTIMATOR
Figure 1(a) illustrates the process flow for generating a charge state estimator (CSE).Initially, the preparation of training data is needed to train the ML model.We utilize a constant interaction (CI) model in the simulation for simplicity, where quantum dots are modeled as a capacitor circuit [41,42].Using this CI model and treating the electrostatic coupling between the double quantum dot and the charge sensor, we can obtain the charge stability diagrams with the size (30 × 30) pixel measured using a charge sensor by changing the parameters in the model.This size is chosen to achieve a sufficient resolution to distinguish the charge transition lines.The next step is to prepare the charge state label: no quantum dot (ND), left (LD), center (CD), right (RD), or double quantum dot (DD), for each stability diagram (Fig. 1(a) step 0).Note that the CD state is formed by strong coupling between both dot sites, which in the CI model is expressed by setting a large value for C m .Here, the diagrams are automatically categorized based on the largest number of quantum dots in the region.
Next, pre-processing is performed on these images.All images are differentiated in the x-axis to make the charge transition lines clear (Fig. 1(a) step 1).Then, the images are binarized by adaptive thresholding to simplify them into black and white images (Fig. 1(a) step 2).Here, the charge transition line is set to white, while the background is set to black.In this process, we use Adaptive Thresholding by the OpenCV module [43].
To adapt to noisy experimental data, random noise is added to these images.Due to binarization in the previous step, noise can be realized by simply flipping the black and white pixel values at Step 0 Step 1 Step 2 Step 3 Step 4 Step random locations (Fig. 1(a) step 3).We adjust the amount of noise such that the number of white pixels after this process does not depend on the charge states.Images labeled DD will have more white points than others without this adjustment because there are many charge transition lines, and the CNN will be trained to estimate the state by counting the number of white pixels.We adjust the number of white points in an image between 30 and 110, combining the noise and the charge transition lines.Next, a 3 × 3 Gaussian filter is applied to the images (Fig. 1(a) step 4).
The same filter is applied to the experimental data in the pre-processing step.After applying the Gaussian filter, the image is again binarized by adaptive thresholding, as shown in Fig. 1(a) step 5.
Here, 84,325 images, each measuring (30×30) pixel, are undersampled from a total of 180,000 images in the simulation and trained by CNN.Images for each charge state are prepared in the same quantity.The details of the CNN are presented in the supplementary information.

Pre-processing of experimental data
In this section, we perform the charge state recognition of the real experimental data with the obtained CSE.The experimental data is a radio-frequency reflectometry-measured charge sensing signal from a GaAs double quantum dot [44,45].The size of the experimental data is (30×30) pixel (Fig. 1(b) step 0).Note that the gate voltage range is adjusted to include several charge transition lines in the images.This prevents the situation where the (30 × 30) pixel is small compared to the interval of the charge transition lines, which could lead to erroneous recognition of the ND state for other charge states.
Prior to inputting the experimental data into the CSE, it undergoes pre-processing using almost the same methods as the training data.This includes differentiation, application of a Gaussian filter, and binarization.These steps correspond to steps 1, 2, and 4 in Fig 1(b), respectively.
Adaptive thresholding can binarize even with a non-uniform background because the threshold is calculated for each small region.It is useful for the actual experiments, since charge stability diagrams often have a nonuniform background due to the shift of the charge sensor condition.
Here, the adaptive thresholding parameters are set to the same as those in the training process.
As an additional processing step (Fig. 1(b) step 3) for the experimental data, a procedure is applied to ensure that the color of the charge transition lines (black or white) matches that of the where n is the number of data points (30 × 30) , x i the value of each pixel, x the mean of the value, and s the standard deviation.Because the number of pixels for the charge transition lines is less than that of the background in the stability diagram, the color of the charge transition lines is reflected in the sign of the skewness.In the case of the black charge transition lines, the skewness becomes negative.Then, we inverted the pixel values.In the calculation of the skewness, the pixel histograms are flattened using CLAHE (Contrast Limited Adaptive Histogram Equalization) to improve noise immunity.This inversion process turns the charge transition lines white depending on the skewness (Fig. 1(b) step 3).

Charge state recognition
We now discuss the performance of the charge state recognition using the obtained CSE on the experimental data pre-processed by the steps in Fig. 1(b).The experimental data (step 1 in pre-process) is presented in the background layer of Figure 2 using gray-scale representation.The color in Fig. 2 represents the result of the charge state recognition by the CSE.Here, each recognition is performed by extracting each (30 × 30) pixel area from the original image.Subsequently, CSE loads each area and returns the output as the maximum number of quantum dots in the area.
Orange, deep blue, green, light blue, and yellow dots indicate the points classified as the ND, LD, CD, RD, and DD regions, respectively.Note that the CD state is not obtained in the used experimental data.By comparing the results with experimentalists' classification per pixel, the accuracy is evaluated as (93.3 ± 0.8)% with 30 CSEs using the same network architecture.This means that the model even trained by the simple binarized data can recognize the charge states of noisy experimental data.We also apply the CSE to the simulation data with noise, and an accuracy of (97.0 ± 0.2)% is achieved.Therefore, the experimental data is estimated with a degree of accuracy comparable to that of the simulation data.To verify that the CSE developed in this study is not exclusively optimized for only our experimental results, we apply it to publicly available experimental data including CD state from previous studies [46].The CSE demonstrates a (86.0±1.2)%accuracy in recognizing charge states, even when applied to data from other group.

Visualization of the machine learning model and its feedback
We analyze the explainability of the operation of the CSE on the experimental data using Grad-CAM [40].This method identifies class-discriminative regions for each estimation by the model from an input image.The results of the Grad-CAM analysis are demonstrated in Fig. 3.The images in the left column are the input images, while those in the right are the outputs from Grad-CAM.The brighter regions in the right figures correspond to the regions that affect the result of the CSE.We can see bright regions around the charge transition lines in these images.In the case of the simulation data with noise, the CSE estimates charge states by focusing on these lines as in the case of the operation on the experimental data.These demonstrate that the CSE estimates charge states by focusing on these lines, indicating that human-like recognition can be realized.
Here, it is important to note that in Fig. 2, there are several instances where regions labeled as the training data resulted in negligible effects on the overall accuracy, even though it improved classifications outside these boundary regions.

SUMMARY
In this paper, we analyze the explainability of the machine learning model estimating charge states in quantum dots.To generate the training data, we employ the CI model which is a simple simulation model and clearly shows the charge transition lines we typically focus on to recognize the charge states.The CSE recognizes five charge states in both our experimental data and that of other group, and its interest highlighted by Grad-CAM is on the charge transition lines, the same as humans.This allows for the tuning of CSE parameters, such as training data, based on human insights and has been empirically demonstrated to result in improvements.This approach eliminates the need for excessive parameter tuning, thus reducing the cost associated with generating an optimal ML model.We also thoroughly detail the pre-processing steps for both training and experimental data, offering sufficient information to enable replication of our method.
Toward large-scale semiconductor qubits, the method is needed to recognize the charge state in the additional multiple quantum dots.One of approach is utilizing a virtual gate [23], which can translate the charge stability diagram of the multiple quantum dots into that of the double quantum dots.By utilizing virtual gates, the method demonstrated for double quantum dots in this study can also be applied to multiple dots, expanding its potential applicability.Additionally, our pre-processing steps, including the CI model, facilitate the expansion of the quantum dot system without incurring significant additional simulation costs.This approach is also suited for future scaling up of quantum dot systems.

SUPPLEMENTARY MATERIAL
See the supplementary material for additional details regarding the CNN model.

FIG
FIG. 1: (a) The flow of generating the CSE.Training data for the CNN is prepared by simulation using the CI model.We simplify the data with five pre-processing and then train the CNN.(b) The charge recognition flow of the experimental data measured by radio-frequency reflectometry.Steps 1, 2, and 4 are common processes shared with the training data.We add the process that changes the color of all charge transition lines into white by utilizing the skewness (step 3).

FIG. 2 :
FIG. 2: The result of the charge state recognition of experimental data.The color represents the classification result.The orange dots indicate points estimated as ND, the deep blue, green, light blue, and yellow dots indicate LD, CD, RD, and DD, respectively.

FIG. 3 :FIG. 4 :
FIG. 3: The results of Grad-CAM for real experimental data with estimation result by the CSE.