Elucidating Mechanism of Brightness Distribution in Output Images Obtained from Optical Fingerprint Sensor Based on Tissue Optics

In a previous study, we developed a novel fingerprint optical sensor that can detect fingerprint patterns within the skin of a finger without considering the effect of surface condition, and the illumination position of our sensor was optimized experimentally. When a light source irradiated the root of a nail, the output image was bright without saturation. Therefore, the fingerprint image was clear, and a large area of the fingerprint pattern was clear. The aim of the present study was to clarify by experiments the mechanism underlying these results, and we hypothesized that light is scattered throughout the bone and less scattered in the muscle. Thus, the output image is strongly shaded when the nail-side position is irradiated with light. To test the hypothesis and clarify the shading mechanism, we constructed a simple simulation model that consisted of only bone and muscle and used the Monte Carlo method to simulate optical scattering in the tissue. The simulation results were in good agreement with the experimental results, which, in turn, proved our hypothesis. This study contributes to improving our fingerprint sensor, for example, by improving verification performance for use in a wide range of applications. Furthermore, our study contributes to the development of various optical sensors for use with biological tissues.


Introduction
Security is becoming increasingly important in the Internet of Things (IoT) as almost all things are connected to the network. An example of an IoT application is the use of smartphones for electronic payment at FinTech. Biometrics, which is a personal authentication method that uses personal biological information, is a safe and convenient method to increase security. Uniqueness and persistence are well known features of fingerprints; as has been proven statistically, people have unique fingerprint patterns. (1,2) Therefore, fingerprint recognition is widely used, and in 2014, millions of smartphones with a fingerprint sensor were sold. (3) Nevertheless, fingerprint recognition can be further improved. The terminal and branch points of a fingerprint pattern are used for matching fingerprints, so it is important to detect as many minutiae of a fingerprint pattern as possible to improve matching performance. In other words, it is important to obtain a clear and wide fingerprint pattern. An unclear fingerprint image is obtained when the surface conditions are poor, e.g., wet, wrinkled, and almost flat ridges. Therefore, we developed a new optical sensing method to obtain the fingerprint pattern from within the skin of the finger without the influence of the surface conditions of the finger. (4)(5)(6) The developed sensor, which works by shining light through the finger, is illustrated in Fig. 1. (6) Unlike the image obtained using a conventional sensor, shown in Fig. 2(a), our sensor can obtain a clear fingerprint pattern without wrinkles, as illustrated in Fig. 2(b). (6) On the other hand, the strong shading generates an output image that consists of a saturated area and an area that is too dark to detect minutiae, as illustrated in Fig. 3. Therefore, we optimized the direction and irradiation position of the light source on the finger to detect a wide area of the fingerprint pattern. (7) The configuration of the experimental setup is illustrated in Fig. 4, and the irradiation position on the finger is illustrated in Fig. 5. Figure 6 shows the output images of each irradiation position. The image of irradiation position (a) was bright overall, as shown in Fig. 6(a), but irradiation position (b) resulted in a saturated area, and irradiation position (c) had a dark area as shown in Fig. 6(c). A bright fingerprint image was obtained when a light   source irradiated the root of the nail, position (a) in Fig. 5, regardless of direction. However, the mechanism underlying this bright image result has not yet been clarified. In this study, we hypothesized that the light was scattered throughout the bone when light irradiated the root of a nail and the output image was uniformly bright. The light scattered less as it passed through the muscle when light irradiated the nail side, and the output image was strongly shaded. Then, we calculated light transport using a Monte Carlo simulation based on tissue optics to identify the cause of this shading, e.g., optical parameters, type of organization, and shape. For this simulation, we constructed a simple model of a finger that consisted of only bone and muscle. In a complex model, it becomes difficult to determine the factors that contribute to this shading.

Constructing simple model based on tissue optics
In tissue optics, Monte Carlo simulations have been used to study light propagation in live tissues. (8,9) In recent years, simulations that treat light propagation and photoacoustic signals at the same time, and those aimed at optogenetics have been developed. (10,11) These simulations are based on a tissue optics model for light scattering in tissues. (12,13) Propagation of multiply scattered light in tissues is described in tissue optics as shown in Eq. (1), where l is the mean free path of light propagation, µ s is the scattering coefficient, and µ a is the absorption coefficient. (13) Scattering probability, P(θ), follows the Henyey-Greenstein scattering phase function, where θ is the scattering angle in the scattering plane, P(θ) is the probability for travelling in θ, and g is the anisotropy factor. (14) P If P(θ) is not used, and isotropic scattering is assumed, a reduced scattering coefficient µ s ' is used such that (13) µ s ' = µ s (1 − g) (0 < g < 1).
In performing Monte Carlo simulations using these equations, the model for the tissue and the determination of the optical parameters μ s , µ a , and g used in the model are important. Since living bodies have complicated shapes and are highly variable by individual, there are usually no standard models of shapes of organizations. Thus, these shapes are often modeled using flat plates and concentric spheres, and in simulations for light propagation in skin, models with multiple layers of flat plates have been constructed. (15) For brains, simple models using concentric spheres were common, but voxel-based models that imitate fine structures have recently been developed. (10) Because this research in the medical field investigated how light propagates in the brain in order to evaluate the effect of light on the brain, the precision of the model was important. Another reason for the development of measurement of living organization shapes is that brains have been researched using MR images and topography, and their internal structures are becoming better known. At the same time, light scattering in brains can be considered uniform, so light propagation is modeled with an isotropic scattering model with no angular dependence, and optical parameters are limited to three to five types. (10,16) This illustrates how researchers are considering simple models that use only the necessary information but reflect as much of the real structures as possible.
Our purpose in this study was not to research the propagation of light in live bodies, but to investigate the distribution of the intensity of light that passes through a finger and to identify the structure in the finger that causes the shading in Fig. 3. We hypothesized that the shading with spatial wavelengths of over a few mm is affected by internal structures of the same scale and modeled a simple structure consisting of hard tissue, which is the largest structure that composes a finger, and soft tissue surrounding it. Soft tissue includes the skin, nail, and muscle, but we considered muscle as a representative of soft tissue. Thus, we considered a model consisting of muscle and hard tissue, i.e., bone.

Determination of optical parameter for Monte Carlo simulation
Next, we added optical parameters to this model. Because tissues scatter light strongly and multiple scattering occurs, optical parameters cannot be measured easily, (17) and measurements of optical parameters of various biological tissues have been attempted in the past. However, even for the same kinds of tissue, there are large variations between experiments. (18)(19)(20)(21)(22) For example, according to the review by Jacques et al., µ s ' in hard tissue at λ = 500 nm can vary by a factor of two for the skull from 9.5 cm −1 to 20.9 cm −1 ; µ s ' for bones can be even twice those for skull (38.4 cm −1 ). Since there is such a range of possible values, we assumed a few different sets of parameters, l and g, from past measurements in this study, and used them in simulation.
As the model has a thickness of a few millimeters, we also wished to consider scattering angles in this study, rather than using µ s ' to model isotropic scattering. However, it is complicated to calculate the scattering angle distribution, (17) and there are almost no studies that measured both µ s and g as parameters for bones and muscle. Accordingly, we used data from chicken muscle rather than human muscle, with both g and µ s measured at 630 nm, which is close to the light source wavelength for our experiment (Table 1). (18) There is no data available for bones, so we needed to make assumptions. As seen from Table 1, living tissues are strong scatterers, with a much smaller µ a than µ s so we focused our discussion on µ s and g. Typical values of g are smaller for hard tissue (0.6-0.9) than for soft tissue (0.8-0.95). (13) The graph of Eq. (2) in Fig. 7 shows that light is less likely to travel straight in hard tissue, scattering at larger angles. For the value of g in bone for our simulation, we therefore assumed a smaller value than the typical value for muscle and smaller than the value in Table 1 (0.8 and 0.85).
We next consider the possible values of µ s . As discussed earlier, there is a spread of about a factor of four in the experimental values of µ s in hard tissues; Eq. (3) shows that the value of µ s can vary by a factor of 16 between its maximum and minimum. We narrowed down the values of g and µ s from the possible ranges above. From Fig. 7, a small g increases the scattering angles, and, from Eq. (1), a small l increases the number of scattering events. It can be supposed that if the scattering due to the bone is much stronger than scattering due to muscle, then the shading would be darker. In order to test if the shading would appear even if the difference in the degree of scattering between the bone and muscle is small, we assumed g and l values for the bone so that the scattering would be stronger than in muscle but the difference in scattering Table 1 Optical properties of chicken muscle. (18) µ s (mm −1 ) µ a (mm −1 ) g 34. 5 0.012 0.965 between the bone and the muscle would be small. Table 2 shows the assumed parameters; for bone, g was assumed to be 0.8 or 0.85, i.e., within the range of typical values and somewhat smaller than g = 0.965 for muscle, and l was assumed to be either the same as for muscle or half the value. Here, µ a is scaled similarly to µ s , and is reflected in the value of l.

Determination of shape of finger model and layout of objects for simulation
Next, we considered the shape of the model. As the shape of living bodies differ between individuals, there are no standard models. The distal interphalangeal joint thickness of the index fingers in the Japanese population ranges from 9.9-15 mm, (23) and there is no anatomical model that indicates the relative positions of bones inside fingers. For this reason, on the basis of an anatomy reference illustrated in Fig. 8, (24) we created a simplified model that simulates the sizes and locations illustrated in Fig. 9. This model consisted of simple, concentric, tapered cylinders, with the outer cylinder representing muscle, and the inner cylinder representing the bone. The cylinders were thinner towards the fingertip and thicker towards the base. As the purpose of this study is to identify which internal structure causes the shading, we wished to know the intensity distribution of light that penetrates the finger. Therefore, we placed a detector so that the light intensity distribution directly under the finger simulation could be detected without an imaging system. The detector size was set to be the same as the length of the finger. Also, the light source position was set to be on two locations on the nail side of the finger. Position (a) represents Fig. 1(a), and position (b) represents Fig. 2(a). With the setup for the previous experiment illustrated in Fig. 5 (7) in mind, we used parallel light with 6 mm diameter as the light source. Parallel uniform light was chosen for simplicity, as the simulation was intended not to reproduce experiments precisely, but to investigate the relative light intensity distributions when irradiation position (a) and position (b) were used.  (Table 1).

Results
The simulation results obtained using the parameters listed in Table 1 were converted from the number of rays that reached the detector to 8-bit grayscale images, and the output images are shown in Fig. 10. The number of rays generated in each simulation was 10 million. Since each pixel was quantized into 8-bit data, the brightness was represented with the maximum value 255, so that the simulation results could be compared qualitatively with the experimental results shown in Fig. 6. The light detected in the simulation model are represented by white dots in Fig. 10. Figure 11 shows a schematic of the trajectories of light in bone and muscle tissues,    Table 1; the number of rays was ten. as simulated using parameter set 2. The brightness distribution in Fig. 10 was considered to represent the shading pattern without the fingerprint pattern, e.g., the shading pattern in Fig. 6, and to indicate the relationship between scatter strength and the optical parameters l and g. In Fig. 10, the difference between the brightness of position (a) and position (b) with parameter set 2, which were the smallest values of l and g of the bone, appeared the largest. The light trajectories in Fig. 11 illustrated the optical path length and scattered angle in each path. In position (a), trajectories appeared shorter and more complex, in other words, smaller optical path length and larger scattered angles, than in position (b). We speculated that these differences in trajectories between position (a) and (b) depended on the differences in the values of l and g between (a) and (b).

Comparison between previous experimental results and simulation results
We hypothesized that light irradiated on the root of the nail is strongly scattered by the bone, and the output image shows uniform brightness. Furthermore, when the irradiation position is on the nail side, the light scattered less in the muscle tissue, and the output image shows a strong shading. This hypothesis was verified by comparing the simulation results with the experimental results of our previous study presented in Fig. 6. The simulation results in Fig. 10 exhibited the above-mentioned tendency, which was also observed in the experimental results shown in Fig. 6. Furthermore, this tendency is observed in Fig. 10 where the values of l of the bone changes. In addition, the simulation using position (a) produced a less bright image in Fig. 10 than the simulation using position (b); this is because the light was strongly scattered by the bone when using position (a), where the volume of the bone is larger than that in position (b). Figure 11 shows the same tendency; the trajectories of the rays in Fig. 11(a) are more complicated than those in Fig. 11(b). Thus, the simulation results qualitatively prove that the hypothesis presented in Sect. 1 is correct. The following two points were hypothesized for the simulation: (I) The peak brightness of the output image of the irradiated nail shown in Fig. 6(b) is higher than that of the irradiated root of the nail in Fig. 6(a).
(II) The bright area without saturation of the output image obtained by irradiating the root of the nail is larger than that obtained by irradiating the nail side.
To relatively evaluate the two hypotheses mentioned above, the brightness profiles of both the experimental and simulation results were obtained, as shown in Fig. 12, and analyzed. The plot along the vertical axis represents the average value of brightness at each horizontal point. We discuss hypotheses I and II using these profiles in Sects. 4.1.1 and 4.1.2, respectively.

Comparing peak brightness values of experimental and simulated images
The peak brightness of the profiles in Fig. 12(b) are clearly higher than those is Fig. 12(a). However, in Figs. 12(a), the simulation results were difficult to compare with the experimental  results owing to the large saturation area in the experimental results. Hence, the profiles in Fig. 13 were normalized to the values determined in the parameter sets, l and g, and converted to an 8-bit grayscale to compare with the experimental results focusing on the existence of the saturation area. The normalized constant was the maximum number of rays in Fig. 12(b) multiplied by 1.1, which was determined to approximately correspond to the range of the experimental results in Fig. 12. In addition, the length along the horizontal axis of the simulated image was 18 mm, and the length of the experimental image was 12 mm. Hence, the simulation results were replotted between the 50th pixel and the 350th pixel to adjust the range of the simulated images to the experimental results.
Although the profiles in Fig. 13(a) were not saturated, those in Fig. 13(b) were. These profiles exhibited the same tendency as the experimental results in Fig. 10. Thus, the peak brightness of the output image obtained by irradiating the nail [position (b)] was established to be higher than that obtained by irradiating the root of the nail [position (a)]. Thus, the simulation results prove hypothesis I mentioned in Sect. 4.1.

Comparing sizes of bright areas in experimental and simulated images
The distribution of the intensity of light passing through the tissue was assumed to be approximated by the Gaussian distribution because of the profiles in Fig. 12. Hence, the peak of the profile of position (b) was higher than that of position (a), which was proved in Sect. 4.1.1. A wider saturation area, that is, the broader width of the Gaussian distribution means that the slope of the Gaussian distribution of position (b) is larger than that of position (a). In other words, point I means that the ratio of the slope of (a) to the slope of (b) was less than 1. A part of the profile was used to calculate the first-order approximation formula obtained within a range that satisfies both the nonsaturated range of the profile (b) and the range next to the peak of the profile (a). The value of the range was determined to be 50 because the linear part of the profile in the experimental result in Fig. 10(b) was approximately 50 pixels, and the ratio of the slopes of (a) to (b) was obtained for each figure. The calculation results are summarized in Table 3.
Furthermore, considering the slope ratio, parameter set 2 may have been closer to the actual parameters than the other parameter sets. This suggests that the actual l and g of the bone are equal or less than those in parameter set 2.

Influence of relationship between irradiation position and imaging area
In this study, we focused on the irradiation position on the finger. However, there is a possibility that an out-of-position imaging system can cause the same shading as in the result Table 3 Slopes of profiles in Figs. 10 and 13.
Slope of first-order approximation Ratio of slope of (a) to slope of (b) (a) (b)

Calculation source
Experimental result (Fig. 10) -0.298 -0.720 0.41 Simulation result (Fig. 13 of this study. Therefore, we examined the relationship between the irradiation position and the imaging area to clarify whether the irradiation position or the imaging area was a dominant factor. A summary of this examination is shown in Ref. 7. In the present paper, we discuss this examination in greater detail and present experimental images. If the shading is independent of the internal structure of a finger and the irradiation position is bright, the output image will not be saturated if the irradiation position and the imaging area are fixed and only the finger is moved. Thus, we attempted to obtain fingerprint images by fixing the positions of the light source and the imaging system and moving only the finger. The experimental setup described in Ref. 7 is illustrated in Fig. 14 Fig. 16. These profiles represent the fingerprint outline separating the saturated area from the dark area. Therefore, these profiles also indicate that the light transmitted through the finger was saturated, exceeding the dynamic range of the imaging device. Moreover, a large saturation area appeared on the fingertip in the fingerprint image in Fig. 16(b). This image indicates that the saturation area was caused by light irradiating the nail-side position represented by position (b) in Fig. 5, similarly to the example shown in Fig. 6(b).

Large structure in finger skin
In this study, bone and muscle were assumed to be the source of the shading in the output image. In this section, we discuss in greater detail the relationship between the internal structure of the finger and the light and dark patterns of the fingerprint image. It is assumed that the output image from our sensor superimposes the fine bright and dark patterns as the fingerprint pattern and the broad bright and dark patterns as the shading pattern, as shown in Fig. 6. Our sensor detects the light passing through the skin tissue of the fingers and is affected by the optical properties of finger tissue. Thus, we hypothesized that there are two structures inside the finger tissue, and that (III) the fingerprint pattern of the output image is derived from fine structures, such as internal fingerprint skin tissue, and that (IV) the shading is caused by large structures, such as bone and muscle. The structure of the internal fingerprint skin is illustrated in Fig. 17, (25) and the large structure inside the finger is illustrated in Fig. 8. (24) First, we discuss hypothesis III. The dermal papillae in the layer that is deeper than the epidermis (Fig. 17) have an uneven structure. However, dermal papillae are twice as frequent as the fingerprint pattern, and their patterns are not shown in the output image of our sensor. Therefore, we presume that the fine structure in the epidermis is the origin of the fingerprint pattern. Secondly, the large structure in the fingertip comprises bone and muscle. We believe that this structure causes the shading because the shading pattern is obviously larger than the fingerprint pattern. In this study, hypothesis IV has been established because the shading differs depending on whether the light goes through bone and muscle or only muscle. Furthermore, the hypothesis that bone is just like a scattered light source is supported by the results of this simulation.

Effect of nail on output image
As stated in Sects. 1 and 2, we used a simple model of the finger that comprised only bone and muscle in order to identify the structure that contributes to the shading in the output images. However, there is another large structure that represents a difference between positions (a) and (b), which is the nail. In this section, we investigate the effect of the nail on the output image and confirm that the nail had no significant effect. At a wavelength of 800 nm, nails have a µ s of around 2.5 mm −1 , a µ a of around 0 mm −1 , and a g of 0.85. (26) Thus, l is 0.4. Also, since the thickness of the nail at positions (a) and (b) is roughly 0.1-1 mm, (27) we can estimate that light is scattered at most once or twice. Furthermore, from Eq. (2) and Fig. 7, about 90% of scattering occurs within ±5° when g is 0.85. We previously performed an experiment to investigate the   (25) relationship between the incident angle of light on a finger and the shading, and confirmed that the shading was not affected by the incident angle. (7) This experiment used the same experimental setup as in Fig. 4 and obtained fingerprint images by shining light at position (a) from five different directions. When the angle within the plane parallel to the fingerprint was α and the angle in the perpendicular plane was φ, the five directions were (α, φ) = (0, 0), (0, 45), (90, 45), (180, 45), (270, 45). We were able to determine that the shading does not depend on the angle by comparing the profiles of the fingerprint images. Even if the incident angle of light on the finger is 45°, there is no effect on the shading. There is at most one or two scattering events in the nail, and the scattering angle is mostly less than 5°. Therefore, we concluded that a model that omits the nail does not affect our simulation results markedly.

Conclusions
In a society with the IoT, in which various devices are connected to networks, personal identification technology for protecting personal information is important, and fingerprint recognition devices are the most widely used personal identification device. In our previous study, the illumination position of our fingerprint sensor was optimized experimentally to be compatible with more people. In the present study, we constructed a simple simulation model of a finger comprising only bone and muscle and used the Monte Carlo method to simulate optical scattering. The simulation results were observed to agree well with the experimental results.
However, there are some issues. We used a simple model for the simulations and some of the optical parameters were assumed. The use of a more realistic model will elucidate which tissues cause the shading in the image. The size and positional relationship of the bone and muscle need to be measured in various samples. Moreover, a more precise model needs to be considered, for example, the bone can include bone marrow and periosteum. Besides, as optical parameters are dominant factors in the simulation, it is desirable that the µ s , µ a , and g of human muscle and bone can be measured precisely by using these parameters in an in vivo experiment. If a detailed model is developed and a simulation based on this model is performed, more precise parameters will be obtained. The inverse estimation of parameters may also be possible. Furthermore, it will be possible to determine the type of tissue that is affected by light. Such further studies will contribute significantly toward the development of various sensing methods based on transmitting light through the living body.