Analysis of Correlation between Electron Diffraction Images of Corneocytes and Skin Barrier Properties by Multilayer Perceptron and Convolutional Neural Network

than with 1D ED profiles, which suggests that spatial anisotropy in a 2D image may contribute to the skin barrier function


Introduction
The stratum corneum (SC), which exists in the outermost layer of the human skin, plays a crucial role in the skin barrier function. (1)(6)(7) Alternatively, if the aim is to inject transdermal drugs into the body, the crystal-like structure of intercellular lipids, which exists in the permeation route for the drugs in SC, acts as an undesirable barrier. In ED experiments of SC, one 2D diffraction image can be obtained from approximately one corneocyte with the development of sensor technology, making it possible to locally analyze the structural characteristics of SC.Therefore, when analyzing the average structure of samples with large individual differences, such as human SC, for clarifying its relationship with function, a large number of 2D diffraction images must be acquired and analyzed to achieve sufficient statistical significance.However, analyzing large numbers of 2D images is very complex, and most current research is limited to analyzing the 1D profiles, which are obtained by averaging 2D images radially. (10,16)n the other hand, the deep learning model has made significant progress in recent years, leading to technological innovations in various scientific fields.The origins of the neural network, which has become the mainstream of deep learning in recent years, can be traced back to the multilayer perceptron (MLP) model. (17)Traditional neural network models have not been influential in other scientific fields.However, with the development of the computation ability of the calculator and the advancement of numerical algorithms in the learning phase of the model, neural network models with deep architectures can learn patterns in the data and achieve high expressiveness.In particular, AlexNet outperforms rule-based classification models developed by researchers and other machine learning methods, which has led to significant advances in the use of neural network models for computer vision tasks. (18)When constructing a deep learning model for image data, it is considered more efficient to use the convolutional neural network (CNN) than a simple MLP model. (19,20)At the convolutional layer in a CNN model, the convolution of the image and kernel is introduced with an idea of the translational invariance of feature extraction in the image, and each component of the convolutional kernel is optimized in the learning phase.CNN models have worked well in various previous tasks for feature extraction in image data.In addition, as a definition of this research, a model that includes convolutional layers is called CNN, and a model that only has fully connected layers, which does not include convolutional layers, is called MLP.
As a novel approach, we introduced deep learning modeling in the analysis of complex 2D ED images, which can be obtained with the advances in sensor technology, with an aim of investigating the new structural information of SC.There have been no studies on the correlation between individual cells and skin parameters, and this study is a new attempt.)(23)(24) Furthermore, we compared the results of models using 1D ED profiles and those using 2D ED images to examine if the spatial structure is important for the SC function.

Data collection
ED images of human skin corneocytes collected from 13 healthy female and male donors in the age range of 22-45 years were obtained with a transmission electron microscopy (TEM) device (JEM1010, JEOL, Tokyo).After obtaining informed consent, TEWL and water content were measured with Vapo Meter (Delfin, Finland) and Mobile Moisture (HP23-M, Courage+Khazaka, Germany), respectively.A layer of corneocytes with intercellular lipids was collected onto a copper grid (HH600 mesh for electron microscopy) covered with glue (POLYTHICK, Sanyo Chemical) according to the grid-stripping method. (6,11,16)These experiments were approved by the Ethics Committee for Epidemiological Research and Life Science Experiments at Kwansei Gakuin University (ethics approval code KG-IRB-21-04).
The details of the ED method for the structural analysis of corneocytes were described in previous studies. (6,11)Briefly, 24 ED patterns of the corneocytes collected from forearm SC per donor were obtained by TEM with an acceleration voltage of 100 kV at room temperature.A highly sensitive digital CCD camera (ES500W Erlangshen, Gatan, USA) was used as the ED detector.The 2D ED patterns were obtained directly from the digital camera output, and 1D ED profiles as a function of the scattering vector were translated from the 2D ED patterns by integrating the intensity along the azimuthal direction, as described later.Only about 40% of the diffraction images obtained with sufficient electron intensity were selected and subjected to the subsequent analysis.After the selection, 124 images were used in this study.The training and test data were randomly divided at a ratio of 8:2.We used 99 images as the training data and 25 images as the test data.Figure 1 shows the number distribution of TEWL and water content values for the selected data.

One-dimensionalization of ED image
We describe the detailed process of how the ED images were converted to 1D profiles in this section.Because the position of the beam center in an ED image differs slightly for each measurement, the beam center was first determined from the center of gravity of the intensity structure.The ED profile showed that the high-intensity region circularly extends from the beam center.We created a binarized mask image by setting 1 for pixels whose intensity values are above the threshold value and 0 for pixels below it.We employed 33,500 data number (DN), which is the median value of the high-intensity region, as the threshold value for masking, which is the median value of the high-luminance region.Then, the beam center (x ̅ , y ̅ ) is given by Eq. ( 1) using the masked image, which is the median value of the high-luminance region.Here, m 00 is the sum of the intensity values over the image, m 10 is the sum of the intensity values multiplied by the x-coordinate, and m 10 is the sum of the intensity values multiplied by the y-coordinate.
Next, concentric pixel positions were extracted from the beam center, and the intensity values of pixels were averaged by the distance from the beam center to create a 1D profile.For position discretization, the distances from the beam center were calculated for all pixels over the image, and the average intensity value was obtained with the binning size of one pixel.Finally, the value was normalized with a maximum of 33000 DN.The maximum value of 33000 DN was employed to exclude the epidermis near the beam center, as it is not the target of analysis in this study.
Figure 2 shows examples of the ED images and the corresponding 1D profiles.The slightly up convex region between 100 and 120 pixel distances in the 1D profiles corresponds to the keratin region, the steep up convex region between 120 and 140 pixel distances to the Hex region, and the slightly convex region between 140 and 160 pixel distances to the Ort region.Measurement results such as X-ray scattering and infrared spectroscopy results of human SC were also applied to this peak identification.Visually, the ED image spreads roughly in concentric circles, but there is a nonconcentric pattern, which is particularly seen in Fig. 2(a).Our analysis determined whether these anisotropic patterns correlate with the skin parameters, TEWL, and water content.

Multilayer perceptron model
The MLP model is used to predict TEWL and water content from the 1D ED profile.Figure 3 shows the architecture of the MLP model in this study.The model consists of five layers with 1024, 512, 256, 128, and 64 units.Mean squared error was used as the loss function during the learning step.The rectified linear unit (ReLU) function was employed for the intermediate layers and the linear function for the output layer as activation functions.We used Adam (25) for the optimization of the model parameters.The learning epoch was set to 800.A learning rate of 0.001 was used at the beginning, decreasing to 0.0005 at 400 epochs and to 0.0001 at 600 epochs.The model with the lowest loss value during the whole epoch was selected.We selected the model before overfitting with the lowest loss value throughout the epoch because the overfitting occurs when using MLP for a 1D profile.

CNN model
CNN was used for the prediction model from the 2D ED images to investigate the spatial anisotropy that can be seen in ED images [see Fig. 2(a)].The input image was resized to 256 × 256 pixels from the original size.The filter size of each convolutional layer was set to 3 × 3. Two sets of two convolutional layers and one 2 × 2 max-pooling layer were used for the feature transformation of the input image.The numbers of filters used were 32 for the first set of convolutional layers and 64 for the second set.The ReLU function was used for all the intermediate layers, and the linear function was used for the output layer as an activation function.The overall model architecture is shown in Fig. 4. The Adam optimizer was used again during the learning phase.The number of epochs was set to 100, which is shorter than that in the MLP model, because the convergence of learning was faster in the CNN model than in the MLP model in this study.The learning rate was initially set to 0.0001, then to 0.00005 at 40 epochs, and to 0.00001 at 80 epochs.The model with the lowest loss value was employed.

Results
In the following Sects.3.1-3.3,we show the results of the MLP models for 1D profiles.Section 3.1 shows the results for 1D profiles, Sect.3.2 shows those when we use the 1D profiles divided by the regions corresponding to the keratin fibers and intercellular lipids, and Sect.
Here, ŷ i is the ground truth value, e.g., the measured TEWL and water content values, and y i is the value predicted using the models.

Experiment 1-1: Prediction from 1D profile
MLP models were constructed using the 1D profiles of the ED images.The results of the model prediction are shown in Table 1.The mean and standard deviation of the prediction accuracy are also shown with three runs of random data splitting.The RMSEs are 2.15 ± 0.06 g/ m 2 h for TEWL and 6.44 ± 0.21% for water content.The error rates are 39.45 ± 2.82% for TEWL and 17.17 ± 0.82% for moisture content.These results were insufficient considering the dynamic range of data used in this study.One of the reasons for the difficulty is that the 1D profiles have a wide intensity range.The wide dynamic range makes it difficult for the model to learn because the data value of the target structure is numerically smaller than those of the nontarget regions.Therefore, in Experiment 1-2, the profiles were divided into the regions corresponding to Ort, Hex, and keratin fibers and used as input data for learning.

Experiment 1-2: Prediction from narrowed 1D profile
We divided the 1D profiles into the regions corresponding to Ort, Hex, and keratin fibers in this experiment.As can be seen from the 1D profile in Fig. 2, the profile at 100-120 pixel distance corresponds to the keratin fibers, the profile at 120-140 pixel distance to the Hex region, and the profile at 140-160 pixel distance to the Ort region.A total of six data sets were created by combining these three profiles in addition to the profiles cut out for each region.The results of model prediction are shown in Table 2.The mean and standard deviation of the prediction accuracy are shown with three runs of random data splitting.As expected, it showed that the use of the narrowed 1D profile corresponding to the target regions improved the prediction accuracy compared with Experiment 1-1.In particular, when using the keratin and liquid phase area and the Hex area, including part of the Ort region, the model achieved error rates of 35.25 ± 4.33 for TEWL prediction and 15.34 ± 0.72 for water content prediction.Although there was a significant difference in predicted water content, there was no significant difference in predicted TEWL in all the cases, and the prediction accuracies for TEWL are not good enough.To this end, we concluded that using the narrowed 1D profile works well for the prediction of water content, but not for the prediction of TEWL.

Experiment 1-3: Prediction from 1D profile with background removal
To make it easier to learn the structure of interest, we removed the background signals other than the target structure from the original 1D profile of the ED image.The background component is obtained by least-squares fitting with an exponential function shown as Although the function form of the background component is not derived in the physical viewpoint, we used the exponential function as the background component following a previous research. (16)igure 5 shows an example of the original 1D profile, the result of fitting the background component, and the profile after background removal.We performed least-squares fitting to the original profile at 0-30 and 130-200 pixel distances from the beam center.Furthermore, as seen in Fig. 5, an artificial clump was formed near the beam center.Hence, we excluded the beam center area and used the profile from the 25-125 pixel distance.
Table 3 shows the prediction results from the 1D profile with background removal.The mean and standard deviation of the prediction accuracy are shown with three runs of random data splitting, which is the same as in the previous experiments.The mean and standard deviation of model accuracy were calculated for three runs of random data splitting to train and test data.It can be seen that subtracting the background further improves the prediction accuracy for both TEWL and water content compared with Experiment 1-1.We achieved error rates of 38.22 ± 6.76 g/m 2 h for TEWL prediction and 16.18 ± 2.04% for water content prediction.Figure 6 shows an example of a scatter plot of the model prediction and ground truths for TEWL and water content.
With the removal of the background component, the prediction accuracy was improved.This suggests that the extracted features, keratin, Hex, and Ort structures are important for TEWL and water content, as determined by background processing.This is consistent with previous studies showing a correlation between TEWL and the amount of Ort structures. (16)ble 2 Results of Experiment 1-2.

Experiment 2: Prediction from ED image
Table 4 shows the prediction results of TEWL and water content obtained using a CNN and the ED images.The mean and standard deviation of the prediction accuracy are also shown with three runs of random data splitting to train and test data.Scatter plots for the prediction and ground truth are shown in Fig. 7.The prediction of water content achieved an error rate of 13.92 ± 0.57%.This is a higher prediction accuracy than that obtained using a 1D profile.The scatter plot in Fig. 7(b) also shows an improvement in prediction accuracy in the range of 20-30% in water content compared with that in Fig. 6(b).This result suggests that spatial anisotropy in the ED image, which is not included in the 1D profile, contributes to the water content.
On the other hand, TEWL has an error rate of 37.58 ± 1.80 %, which is lower than that with the 1D profile.This result suggests that the relationship between TEWL and ED is related to the overall structure of Ort and Hex structures rather than to the fine spatial features of the ED image.However, Fig. 7(a) also shows that some prediction results from 6 to 9 g/m 2 h are relatively accurate.Since the amount of training data is not very large in this study, there is a possibility that the larger amount of training data improved the prediction accuracy, which will be a good topic for future study.

Conclusions
In this study, we applied deep learning modeling to examine the relationship between the 2D ED images of corneocytes and the TEWL or water content values that are often used as indicators of the skin barrier function, which has been difficult using human rule-based analysis.We want to emphasize that this study can be performed because of the recent development of the sensor  technology.In Experiment 1-1, we trained the multilayer perceptron model using the 1D profile obtained by azimuthally averaging the ED images.Moreover, we trained the MLP model with the divided profiles corresponding to the structures of interest in Experiment 1-2, and we further trained the MLP model on the background-removed data in Experiment 1-3.In Experiment 2, the CNN model was trained with the original 2D ED images.Although we used the simple MLP and CNN in the modeling, a significant correlation between the 2D ED images of the corneocytes and the water content values was found in this study.As future prospects, it is possible to use the more developed models such as LightGBM to achieve a higher prediction accuracy.
As a result of this study, the MLP model trained on a 1D profile divided into regions of keratin and liquid phase area and Hex region, which includes some Ort regions, achieved the highest prediction error in this study of 35.25 ± 4.33% for TEWL.This suggests that TEWL is determined with the averaged Hex and keratin, rather than with the fine features of these structures.On the other hand, the highest prediction accuracy of 13.92 ± 0.57% for water content was achieved with the CNN model trained using the 2D ED images, which is newly found in this study.The result indicates that spatial anisotropy in the ED image, which was not included in the 1D profile, contributes to the determination of the water content in the skin.
3.3 shows the results with background component removal.The prediction results obtained using 2D ED images are shown in Sect.3.4.The root mean squared error (RMSE) and error rate defined by the following Eqs.(2) and (3) were used to evaluate the model.

Fig. 4 .
Fig. 4. Architecture of a CNN model in this study.

Fig. 5 .
Fig. 5. (Color online) Example of original profile, background profile, and profile after the removal of background component.

Table 4
Results of Experiment 2.