1.

Introduction

Due to the excellent imaging capabilities, optical imaging (OI) techniques have been developed and employed to restore, segment, classify, and identify tissue properties.^1–4 Diffuse optical imaging (DOI) is the one for being used in the reconstruction of optical properties of the brain and breast. Due to the growing power of artificial intelligence, many inverse problems in DOI are solved in deep learning-based algorithms to improve image reconstruction quality.^5–8 Applications of ML in optical fields ranging from biotechnology⁹ to cancer diagnosis¹⁰ have shown great potential for localization, classifying, and segmentation of tumors for diverse samples in biomedical applications.^11–13 Despite the excellent performance of convolution neural network (CNN)-based deep learning reconstruction methods, the locality of the convolution operator makes it difficult to learn global and long-range image information, i.e., reconstructing absorption and scattering coefficient in the same network well.¹⁴

Previously, numerous data-driven algorithms for sources and detectors have been used to reconstruct DOI optical-property images by generating simulation datasets for training.^13–17 Some among those¹⁶ emphasize fast and accurate estimating of the bulk optical properties in the breast by exploiting a convolutional neural network. Various deep-learning approaches for numerous scientific domains and various forms of data analysis have been extensively reviewed for current work in optical properties retrieval.^18,19 Well-controlled data sets for training and validation are among the most important topics in the neural network, but the lack of large, publicly available data sets leads to unique challenges. The development of a data generation pipeline²⁰ based on Monte Carlo modeling has shown to be a useful method for rapid, robust, and user-friendly image formation in a wide variety of applications. Three-layer deep neural network is proposed in Ref. 21, and simulation, phantom, and clinical data with breast lesions are tested for the mismatch between the target and reference sides.

Some deep learning algorithms focused on restoring high-quality images by simultaneously considering both projection and image domains while the final images may suffer from secondary artifacts due to the introduced errors from projections interpolation.²² Zhang et al.²³ present an encoder-decoder learning-based optimization network, which was highly effective in preserving inclusion edges and recovering details. Fortunately, domain transformation²⁴ called AUTOMAP has the ability for reconstructing sensor information and shows good performance in optical image reconstruction tasks, while Dense-net,²⁵ U-net,²⁶ and its variants perform well in restoring directly from the image domain.

The primary contribution is that we first presented a network for DOI reconstruction called Periodic-net directly employed to boundary data, and then we modified state-of-the-art models, i.e., U-net and Dense-net for optical-property image reconstruction (these models are developed for the purpose of restoring biomedical images, not for the reconstruction of images). Additionally, our deep learning models include datasets that were previously unconsidered for state-of-the-art networks in DOI.

2.

Materials and Methods

2.1.

DOI Preprocessing and Postprocessing

A unified formulation for the frequency domain (FD) system obtaining three combinations of boundary observations and conditions is adapted. Figure 1 illustrates the schematic of the preprocessing and postprocessing steps involved in the reconstruction of optical-property images. Light from laser modules is transmitted to the optical switch, which sequentially passes it to preselected points on the surface of the phantom to perform optical data acquisition. The boundary of phantom designated a source and detector positions. A simulation dataset based on FD measurements is generated using the finite-element forward solver, with Robin boundary conditions.²⁷

Fig. 1

Flow chart of dataset(s) preparation for network training.

2.1.1.

Signal generation in DOI

Signals are generated by photon radiance $L(r,\widehat{s},t)$ in forward solver, where $\widehat{s}$ denotes the direction of light $r$ is position vector at time $t$. Photon radiance is calculated from radiation transport equation (RTE) in the presence of a nonlinear source term $Q(r,\widehat{s},t)$ to characterize tissue by the scattering coefficient ${\mu }_s$ and the absorption coefficient ${\mu }_a$; formulation of optical-property is given as²⁸

Eq. (1)

$$(\frac{1}{c}\frac{\partial }{\partial t}+\widehat{s}.\nabla +{\mu }_t)L(r,\widehat{s},t)={\mu }_s{\int }_{4\pi }L(r,\widehat{s}\prime ,t)f(\widehat{s}\prime ,\widehat{s})\mathrm{d}\widehat{s}\prime +Q(r,\widehat{s},t),$$

where $c,f({\widehat{s}}^{\prime },\widehat{s})$ corresponds to the speed of light inside soft tissue and is a normalized phase function representing the probability of scattering, respectively. The transport coefficient is represented by ${\mu }_t$ to calculate the loss of radiance and is equal to the sum of reduced scattering (${\mu }_s^{\prime }$) and absorption coefficients (${\mu }_a$). Equation (1) can be further simplified and transformed into the frequency domain as

Eq. (2)

$$(-\nabla .(\frac{1}{3({\mu }_s^{\prime }+{\mu }_a)})\nabla +{\mu }_a-\frac{i\omega }{c})\mathrm{\Phi }(r)=S_0(r),$$

where $\mathrm{\Phi }(r)$, $S_0(r)$, and $\omega$ represent the photon density, source term, and frequency, respectively.

2.3.

Deep Learning Framework

2.3.1.

Issues with traditional encoder–decoder network

In our computation, we observed that traditional state-of-the-art machine learning encoder decoder methods failed to detect tiny structures in most cases (see later in the Results, Sec. 3). The reference networks are based on the encoder–decoder structure, i.e., U-net/Dense-net structure as the basic skeleton and AUTOMAP²⁴ serves as the input layer (Fig. 2.).

Fig. 2

Modified U-net and Dense-net structures.

2.4.

Mechanisms of Periodic-net

The overview of the proposed Periodic-net and its components are shown in Fig. 3. The overall architecture is based on four different modules performing individual tasks. The purpose is not only capturing small structures and reconstructing them but also to localize them in adequate resolution for detecting early-stage breast cancer. Feature module skeleton consists of two kinds of layers to extract features, i.e., convolution and batch normalization, each followed by activation function. The structure begins with two $3\times 3$ eight-channel 2D convolutional layers, followed by a batch normalization and ReLU activation layer. Convolution is performed on an input volume, batch normalization is utilized, and then ReLU activation is applied in each feature block. The inside structure of feature module is shown in Fig. 3(a). Four feature blocks are instantiated in an efficient module. In the first feature block, $1\times 1$ convolution was applied, followed by $3\times 3$, and $5\times 5$ filters with the same padding, which ensures that the output volume sizes are identical to all blocks and the maximum information is extracted from limited neurons since the input has a smaller number of neurons as compared with the output. A channel dimension is used to concatenate the output volumes. A major objective of the efficient module is to assess how locally sparse structures can be approximated [Fig. 3(b)]. Input volumes are reduced by the escalate module. This module utilizes two branches in a similar manner to the efficient module. First, a $3\times 3$ convolution is performed, but with a stride of $2\times 2$ and valid padding, resulting in a reduction in volume size. In the second branch, $3\times 3$ maximum pooling is used with a $2\times 2$ stride. Then it is possible to concatenate the output volumes for both branches along the channel axis [Fig. 3(d)]. Image reconstruction is performed in encompass module. Due to the low contrast between background and inclusions, we employ dense layers to extract more discriminative features. Four dense layers are in the encompass module, each followed by a batch normalization layer and an activation of the ReLU layer. Dense layers are composed of 64, 128, 256, and 4096 neurons at the bottom. The final output size is $1\times 64\times 64$ for the absorption coefficient and $1\times 64\times 64$ for the reduced scattering coefficients [Fig. 3(e)]. While Fig. 3(f) shows expected output for circular phantom samples. During training, the periodic nature of proposed network is shown in Fig. 3(f); apart from that, concatenation between two efficient modules are performed to avoid degradation problems.

Fig. 3

Flow of data in Periodic-net. (a) Feature module: consist of convolution layers. (b) Efficient module: extract tiny structure. (c) Upsample module: increasing dimension for (d). (d) Escalate module: concatenation of two modules. (e) Encompass module: reconstruction of optical images. (f) Output: reshaping to get absorption and reduced scattering coefficients. (g) Overall architecture of the proposed Periodic-net model. (a)–(d) Work as feature extraction; (e) working as reconstruction.

3.

Results

In this section, three independent versions of the datasets were trained for the same network, responsible for DOI images of breast cancer. We conduct a series of simulations and experiments confirming and evaluating the performance of the proposed network described in detail in Sec. 2.4. Initially, we present simulated and phantom experiment results that demonstrate the image formation process works both with 16 source positions and 15 detector positions. The second step involves evaluating the reconstructed images from simulated data by varying the number of sources and detectors, i.e., $20\times 19$ and $36\times 35$. Selective test sample parameters and other specifications are given in section Table 1. A side-by-side comparison between different algorithms is shown. Randomly generated Gaussian noise of 15% is added to the forward problem to mimic the noise and aberrations that might arise in phantom test set due to experimental hardware. The fluence rate values were normalized using the min-max method to fix all the values between 0 and 1.

Table 1

Random cases from test datasets for verification.

The reconstructed images in the two-dimensional plane from all methods are shown in Fig. 4. Absorption and reduced scattering coefficients are plotted from upper row to lower row according to the dataset while each column represents reference and proposed methods: the first column is for ground truth, the second column for Dense-net, the third column represents traditional iterative Tikhonov regularization, the fourth column specify U-net, and the last column reconstruct optical properties via Periodic-net.

Fig. 4

Phantoms with different sizes, using one and two inclusions. (a) Ground truth. (b) Dense-net. (c) TR. (d) U-net. (e) Periodic-net. Overall, Periodic-net and U-net perform well to reconstruct a better shape and size of inclusion(s) with less noise.

4.

Discussion

This study proposed a DL model, called Periodic-net, aiming for better imaging of breast tumors and improving the overall performance in terms of spatial resolution, reconstruction time, and memory storage. A greater number of source positions around the phantom has the advantage of improving the reconstruction of absorption and scattering coefficients. However, it costs more during the reconstruction time. Based on our results, the Periodic-net is capable of learning features efficiently even for a small number of measurements. In order to make sure that the trained network enables us to measure the correct contrast for various measurements, we used a variety of contrasts in the training process.

Four key modules make up the overall structure of a Periodic-net: feature module, efficient module, escalate module, and encompass module. In the feature module, raw data (fluence rate) are transformed into numeric features that can be processed while preserving the information in the original data set. Then, an end-to-end edge enhancement reconstruction subnetwork reconstructs the initial image with sparse artifacts removal and image edges preservation in the efficient module. However, the transforming domain may introduce unexpected artifacts. Therefore, the data escalate module, which consists of two submodules (one for feature extraction, and the other for domain correction), was introduced to reduce errors, ensure consistency, and improve structural details. A submodule (upsample module) for edge enhancement is introduced to preserve features and reduce blurring along the edge of the tomographic image. Finally, encompass module is introduced to capture global features from the efficient module and reconstruct the structure of optical properties, i.e., absorption and scattering coefficients.

U-net and Dense-net cannot be applied to DOI directly, so modification was made to these two nets to reconstruct optical properties in different domains. Both these state-of-the-art networks have shown good performance on semantic segmentation, object detection, and classification with localization. There is a slight improvement in U-net performance over Dense-net in terms of DOI reconstruction. It is worth noting that both networks are designed to take high-resolution images as inputs. The U-net architecture showed good performance on the one inclusion data while failing to detect and construct small lesions. Since the U-net utilizes a completely convolutional architecture, only high-level features are considered as a result. While Dense-net has more layers and hyperparameters than U-net, it still fails to extract any useful information such as edges or localizations. Careful arrangement of layers and cautious selection of hyperparameters are crucial to the reconstruction of optical-property images.

Our architecture retrieves optical properties images of breast lesions from low-contrast DOI. Furthermore, it does not rely on the number of radiances nor on the domain, i.e., multiple excitations and measurement positions are used to produce the boundary data. Figure 4 illustrates the reconstruction results of single and two inclusions cases for different boundary measurements. For all simulations 16, 20, and 36 excitation positions and 15, 19, and 35 measurement locations (both are equally spaced around the circular circumference in angular increments beginning near 0 deg, depending on local meshing details). It is also important to note that different modules of network structures and loss functions are designed and implemented at different stages during the training process. In addition to achieving good performance, we also achieve the benefits of fewer parameters and faster convergence ($\sim 3\mathrm{s}$) (Table 2). We demonstrate an approach that enables noninvasive OI behind scattering photons in breast cancer tissue. Experimental validation shows the efficiency and robustness of the method with various DOT samples, covering a reconstruction of up to three different datasets w.r.t boundary data.

Table 2

Efficiency comparison between trained Periodic-net with different networks using three datasets.

The superiority of the method compared with conventional techniques is shown by applying it to DOT problems of different breast cancer data of a various number of measurement data. In contrast to traditional segmentation, classification, and super-resolution imaging approaches, our method does not utilize medical images directly for training to attain optical properties, but rather focuses on reconstruction and improving image quality after changing sensor data into images in one step, which is an end-to-end model. Using inhomogeneous cases, we demonstrate that our method can also be applied to nonsparse and continuous objects. We note that a few iterations are sufficient to recover the object reasonably well in our case. In addition, our technique is not limited by the number of independent illuminations/fluence rates that can be generated with one source/detector, since the scattering angle of the illumination can be tuned to produce an extremely large number of independent illuminations with different positions. Additionally, calibrated measure data have been used to increase the number of test samples and check the impact of overfitting.

According to test results, U-net and TR are still unable to reconstruct small anatomical landmarks with blurred borders. Although U-net is good at reconstructing large structures, it fails when the inclusions are small or have noisy boundaries which can be seen in Fig. 4. Results using simulated data suggest that qualitative images can be produced that readily highlight the location of absorption and scattering heterogeneities. Our proposed (Periodic-net) modeling approach was found to yield the best results after comparing various convolutional neural network architectures, i.e., U-net, and Dense-net, as well as the conventional regularization approach, i.e., TR.

A quantitative evaluation of all methods was also conducted. Figures 5 and 6 represent the results for the reduced scattering coefficients and absorption coefficients, respectively. The MSE, PSNR, and SSIM metrics were calculated to verify the effectiveness of the edge enhancement reconstruction. A significant improvement in the absorption coefficients can be observed, where the Periodic-net model has the lowest MSE value. The results of the Periodic-net showed a significant reconstruction improvement compared with the other models, with a maximum SSIM score of 0.89 for absorption coefficients. Likewise, for the reduced scattering coefficients the Periodic-net model achieved the highest PSNR. Overall, the proposed Periodic-net model outperformed the TR, modified U-net, and modified Dense-net in terms of the evaluation of SSIM and PSNR scores. This shows the power of the proposed model to learn complicated features through various module connections in the proposed Periodic-net, which take advantage of the learning features directly from boundary data as input. The SSIM and PSNR scores show that the Dense-net performed worst, and U-net achieved better results than Dense-net. However, because of its periodic nature (cycle repeats in equal intervals of times for each epoch), Periodic-net outperformed TR and learning approaches in all evaluations. The Periodic-net has the smallest MSEs and the largest PSNRs and SSIMs as compared with other competitors.

Fig. 5

Overview of statistical results for reduced scattering coefficients from (a) MSE, (b) SSIM, and (c) PSNR. 500 samples from simulation datasets of $16\times 15$, $20\times 19$, and $36\times 35$ boundary data, and 12 samples from $16\times 15$ experimental dataset are presented.

Fig. 6

Overview of statistical results for absorption coefficients from (a) MSE, (b) SSIM, and (c) PSNR. 500 samples from simulation datasets of $16\times 15$, $20\times 19$, and $36\times 35$ boundary data, and 12 samples from $16\times 15$ experimental dataset are presented.

5.

Concluding Remarks

Periodic-net offers a potential reconstruction of soft tissue optical coefficients that are not only cost-effective, sensitive, and noninvasive but also provide better localization and suppresses noise when compared with existing state-of-the-art networks. From the simulation and experimental samples, it has been demonstrated that the proposed algorithm has not only remarkably improved the predicted accuracy and resolutions well as significant improvement in performance and reduction in testing and reconstruction time. The future directions of our work include extending our method to explore the feasibility of our Periodic-net with ring system-type datasets for more than one source and detector in different organizations.