A novel Cerenkov luminescence tomography approach using multilayer fully connected neural network

Cerenkov luminescence tomography (CLT) has been proved as an effective tool for various biomedical applications. Because of the severe scattering of Cerenkov luminescence, the performance of CLT remains unsatisfied. This paper proposed a novel CLT reconstruction approach based on a multilayer fully connected neural network (MFCNN). Monte Carlo simulation data was employed to train the MFCNN, and the complex relationship between the surface signals and the true sources was effectively learned by the network. Both simulation and in vivo experiments were performed to validate the performance of MFCNN CLT, and it was further compared with the typical radiative transfer equation (RTE) based method. The experimental data showed the superiority of MFCNN CLT in terms of accuracy and stability. This promising approach for CLT is expected to improve the performance of optical tomography, and to promote the exploration of machine learning in biomedical applications.


Introduction
Cerenkov luminescence imaging (CLI) has been established as an effective in vivo imaging modality (Qin et al 2012, Shaffer et al 2016. By integrating the advantages form optical imaging and nuclear imaging, CLI has shown outstanding performances with high potential for clinical translation , Shaffer et al 2017. Currently, the applications and value of CLI have been explored in various areas, including precise tumor imaging , Zhang et al 2019, therapy monitoring , biomolecule quantitative assessment , Hu et al 2013, etc.
Further development of Cerenkov luminescence tomography (CLT) has upgraded CLI from planar to threedimensional (3D) imaging. In CLT, the photon propagation is mathematically described, mostly with radiative transfer equation (RTE) (Hu et al 2010, Li et al 2010, Zhong et al 2011a. Then the unobservable inner photon sources are able to be reconstructed based on the surface optical signals. As a consequence, information like tumor size, distribution of imaging probes can be efficiently quantified through CLT (Mitchell et al 2011, Hu et al 2012a. Because Cerenkov luminescence (CL) propagation in vivo is a highly complex process, a series of novel strategies have been developed to optimize CLT performance. The accuracy of CLT has been enhanced by taking the multi-spectral information into account. Studies on multispectral CLT (mCLT) (Spinelli et al 2011), hybrid spectral CLT (hCLT) (Hu et al 2012a), multispectral hybrid CLT (mhCLT)  and weight multispectral CLT (wmCLT) (Guo et al 2017a) have been published in succession. These multi-spectral reconstruction A novel Cerenkov luminescence tomography approach using multilayer fully connected neural network Zeyu Zhang 1,2,6 , Meishan Cai 2,3,6 , Yuan Gao 2,3,6 , Xiaojing Shi 2,3 , Xiaojun Zhang 4 , Zhenhua Hu 2,3,7 and Jie Tian 1,2,3,5,7 methods significantly improve the accuracy of CLT. Another typical strategy to improve CLT is the combination with prior knowledge. Previous studies have presented that the employment of CT anatomic images or single photon emission computed tomography (SPECT) functional information could effectively enhance the accuracy and stability of CLT (Zhong et al 2011b, Hu et al 2012b. These promising strategies play significant roles for CLT development. Although CLT performance has been optimized through various advanced strategies, the reconstruction accuracy is still limited by the inevitable RTE deviation. The recent progress of machine learning has shown promising potential to solve the problems in optical reconstruction. Neural networks have been verified effective to enhance the reconstruction accuracy. A unified automated transform by manifold approximation (AUTOMAP) framework was proposed based on deep neural network in 2018 (Zhu et al 2018). Trained by generic images and magnetic resonance imaging (MRI) data, the AUTOMAP showed superior performance in MRI reconstruction. The strength of machine learning approach has also been demonstrated in the super-resolution microscopy (Ouyang et al 2018). With a well-trained artificial neural network (ANN), the spatial resolution and speed of livecell super-resolution imaging have been obviously improved. In the case of optical tomography, a novel multilayer perceptron-based inverse problem simulation (IPS) has been specially designed to perform high-resolution bioluminescence tomography (BLT) (Gao et al 2018). The IPS method was trained by Monte Carlo simulation data, and its effectiveness was validated through in vivo experiments. By using the machine learning method, the BLT accuracy and stability have been dramatically enhanced. In fluorescence molecular tomography (FMT), a convolutional neural network-based method has been presented to improve the FMT performance (Huang et al 2019). The results showed that this novel FMT method accurately localized the tumor tissues. These exciting achievements demonstrate the substantial value of machine learning in optical tomography.
In this paper, a novel CLT approach based on multilayer fully connected neural network (MFCNN) was proposed. The MFCNN was trained with Monte Carlo simulated CLI data, in order to investigate the relationship between the surface luminescence and its true source. Both Monte Carlo simulation and in vivo animal experiments demonstrated that the proposed MFCNN approach significantly improved the accuracy, stability and practicability of CLT reconstruction.

Materials and methods
The MFCNN based reconstruction strategy At first, the orthotopic human glioblastoma U87MG tumor mouse models received 11 C-methionine ( 11 C-MET) administration through tail vein (figure 1). The radiopharmaceutical was distributing with blood circulation and accumulated in tumor tissues. The in vivo biological environment provided necessary medium to generate CL. According to the Frank-Tamm theory, the most part of the CL was unable to penetrate through animal tissues. Moreover, the living tissues induced great attenuation and scattering to the CL. As a consequence, only a fraction of the generated CL could be acquired using high-sensitivity electron multiplied charge coupled device (EMCCD). A multilayer fully connected neural network (MFCNN) was presented in this study to investigate the complicated relationship between the acquired CL and the true luminescence sources. Monte Carlo simulation data were employed to train the MFCNN. Then the acquired in vivo CLI results were mapped on the standard numerical mouse, and the surface optical flux was calculated. When the surface optical flux was inputted into the trained MFCNN, the reconstructed photon sources were able to be outputted.

Structure of the network
The MFCNN in this study consisted of totally seven layers, one for input, one for output, and between them were five hidden layers (figure 2). Input of the MFCNN was the surface photon density, and the output was the reconstructed CL source. Rectified linear unit (ReLU) was applied as the activation function following each connection, and the activation threshold was λ = 0 as shown in equation (1). A dropout function (probability = 20%) was used to reduce the overfitting problem of the network. The nonlinear relationship between one layer and the next was described as equation (2). In this study, the reconstruction permissible region was assigned as the mouse brain. To each MFCNN layer, the number of neurons was equal to the number of nodes in the numerical mouse brain. Keras 2.2.4 with Tensorflow 1.12.0 supported by Python 3.6 were employed to implement the network. The MFCNN was trained with 80 epochs, 128 batch size. The optimizer was Adam algorithm with the learning rate: 0.000 01, β 1 : 0.9 and β 2 : 0.99. (2)

Numerical mouse and the optical parameters
A standard numerical mouse (Dogdas et al 2007) was applied to perform Monte Carlo simulation. The CLT reconstruction mesh was established from the head part of the numerical mouse (figure 3). 1965 nodes of the brain area were included in the reconstruction permissible region. To represent the photon propagation in head, three different related organs including brain, skull and muscle were considered (table 1). The absorption coefficient µ a and the scattering coefficient µ s were set based on the previous studies , Guo et al 2017a. All the Monte Carlo simulation was conducted with the Molecular Optical Simulation Environment (MOSE 2.3) (Li et al 2004. The whole calculation procedure ran on a personal computer with Intel Core i7 CPU (3.70 GHz), 16 GB DDR3 RAM and an NVidia GTX1080 Ti GPU.

Simulation data-set
Simulation data of single-source and multiple-sources were collected to train the MFCNN and validate the reconstruction performance. 100 000 photons were set into one single-source sample, and each photon owned the energy of 1.0. Wavelength of the simulated Cerenkov photon was set as 650 nm, which demonstrated suitable penetration and intensity to perform CLI in vivo , Guo et al 2017a. Different sizes and shapes were assigned to the photon sources during simulation (figure 4). Totally 940 different center positions were selected for single-source simulation. 329 center positions were assigned to spheroid sources, 318 to cylindroid sources and 293 to cuboid sources (table 2). The center positions gathered for each source shape covered the brain region as much as possible. In each specific center position, four repetitive simulations were performed, in order to take into consideration the randomness of photon propagation. As a consequence, 3760 single-source samples were collected based on the 940 distinctive center positions. The generated samples of single-source were divided into training-set and test-set based on their center positions. 80% of the 940 center positions were randomly selected for training and the left 20% center positions were for testing. The simulated Cerenkov source samples were gathered into the training-set and the test-set according to their center positions. Moreover, a group of big-sized spheroid samples were produced to evaluate the reconstruction performance. Three different sizes and nine different center positions were applied in the simulation of big-sized samples (table S1 (stacks.iop.org/PMB/64/245010/mmedia)). These sizes and center positions were distinctive from the other single-source samples. Compared to the spheroid training sources, these big-sized test samples were 1.7, 4.1 and 8 times greater in spatial volume. Four repetitive simulations were conducted with the same parameters to reduce the effects from random error. Consequently, 36 big-sized single-source samples were generated, and all of them were used to evaluate the reconstruction performance without training.
The MFCNN was trained by multiple-source data as well, in order to improve the stability and the robustness of CLT reconstruction. Based on equations (3) and (4), the multiple-source samples were produced by assembling different single-source samples from the training-set. Three types including dual, triple, and quadruplesource samples were assembled. Each type owned 1500 samples, and therefore 4500 multiple-source samples were generated to enlarge the training-set.
All the single-source samples in the test-set were applied to evaluate the CLT reconstruction performance. A number of multiple-source test samples were employed for evaluation as well, which were extracted from the single-source test-set. In this way, the proposed MFCNN can be effectively trained, and the reconstruction performance was reliably evaluated with various test samples.

Reconstruction evaluation index
The reconstruction accuracy was evaluated by center localization error (CLE) and Dice coefficient (DC). CLE indicated the Euclidean distance between the centers of true photon source (x 0 , y 0 , z 0 ) and the reconstructed result (x, y , z). The calculation of CLE was described as equation (5). A smaller CLE result indicated higher accuracy of the CLT reconstruction. If the true source center was perfectly coincident to the reconstructed source center, CLE equals to 0. Source shape was also important to evaluate the performance of a reconstruction approach. In this study, DC as equation (6) was employed to reflect the consistency between the true source shape and the reconstructed shape. X in equation (6) indicated the reconstructed region, while Y was the true source region obtained from the CT. The maximum of DC was 1, and the greater value of DC indicated higher similarity between two sources. Moreover, incomplete variables truncated conjugate gradient method (IVTCG) with L1 regularization was introduced as contrast (He et al 2010, Wang et al 2014, which showed ideal performance for CLT reconstruction in the recent publication (Guo et al 2017a).
In vivo experiments 4-6 weeks old BALB/c nude female mice (n = 3) were used for in vivo CLT experiment. Luciferase (Luc) and green fluorescence protein (GFP) labelled glioma cell line U87MG-Luc-GFP was employed to construct glioma mouse models. Approximately 4 × 10 6 U87MG-Luc-GFP cells mixed with phosphate buffered saline (PBS, pH = 7.0, 10 mM, 10 µl) were administrated into the mouse brain with a microinjector. In vivo bioluminescence imaging (BLI) was performed every three days to monitor the tumor growing status using an IVIS Spectrum system (PerkinElmer Inc., USA). After 9 d for tumor growing, the orthotopic glioma models (n = 3) were applied for the in vivo CLT experiments. Gadopentetate dimeglumine (BEILU Pharmaceutical Co., Ltd., China) in the dose of 0.2 ml kg −1 was intraperitoneally inject into the mouse models (n = 3), in order to perform enhanced T1 MRI (M3 ™ , Aspect Imaging, Israel) for the orthotropic glioma. Next, in vivo CLI-CT was performed followed by small animal positron emission tomography (PET). 11 C-MET (29.6 MBq, 200 µl) was injected into the mouse models (n = 3) through the tail vein. Twenty minutes after the injection, in vivo CLI-CT was performed with a specially designed small animal pentamodal imaging system (Liu et al 2017). To improve the performance of in vivo CLI, the mice was placed on a black imaging platform in prone position, and black paper pads were used to block the luminescence emitted from mouse abdomen. The in vivo CLI results were acquired with a high-sensitivity EMCCD (iXonEM + 888, ANDOR, UK). Parameters of 300 s exposure and 4 × 4 binning were used to obtain the CLI signals from the orthotropic glioma. Then the whole body CT anatomic information was obtained by a micro-CT system. The physical posture of the mice was fixed when performing CLI-CT. Then, the models were immediately transferred for acquisition of PET (GENISYS4, Sofie Biosciences, USA). In the period of in vivo imaging, the living mouse models were anesthetized by 1% isoflurane-oxygen gas mixture.
In the processing of CLT reconstruction, the in vivo imaging data was represented on the standard numerical mouse, and the observed CLI signals were distributed in the numerical mesh. Similar to the reconstruction with simulation data, the surface optical flux was inputted to the well-trained MFCNN and the reconstructed photon sources were generated.

Pathological validation
After in vivo imaging, the orthotopic glioma bearing mice were sacrificed and frozen with optimum cutting temperature (O.C.T.) compound (Tissue-Tek, Sakura, USA). 30 µm thick axial frozen sections of the mouse heads were prepared for pathological validation. GFP fluorescence images were taken by a Live Cell Imaging System (AF6000 Modular System, Leica, Germany). These fluorescence signals localized the glioma cells on each frozen section of the mouse brains. Moreover, these frozen sections were also stained by the hematoxylin and eosin (H&E). The MFCNN reconstruction performance was evaluated by contrasting with the GFP fluorescence and the H&E results.

Statistical analysis
Data analyzed by Prism 7 (GraphPad Software, San Diego, CA, USA) was presented as mean ± standard deviation (SD). Student's unpaired t-test was applied to calculate the statistical significance between independent samples, and difference with P < 0.05 was considered as statistically significant.

Single-source reconstruction performance
The performance of MFCNN was evaluated and shown in both qualitative and quantitative aspects.
From a qualitative aspect, typical single-source reconstruction results using MFCNN were contrasted with the IVTCG method (figure 5).
In the evaluation of localization accuracy, the MFCNN results showed great consistency with the true photon sources (figure 5, first and the second row). This consistency of MFCNN results was observed in different source shapes, including spheroid (model 1-3), cylindroid (model-4) and cuboid (model-5). The energy distribution of MFCNN results were also qualitatively proved consistent with the true sources. As contrast, obvious position deviation and unexpected artifacts were observed in the IVTCG results (figure 5, the third row). In reconstruction of untrained different source sizes (figure S1), the MFCNN reconstruction provided desirable performance. Photon sources that were spatially eight times greater than the training samples could be effectively reconstructed by the MFCNN. However, the IVTCG reconstruction suffered through the problem of artifacts and the localization accuracy was obviously interfered. These comparisons demonstrated promising performance of MFCNN in terms of reconstruction accuracy and stability.
In quantitative analysis, the CLE of MFCNN single-source reconstruction was proved to reach the level of 0.0405 mm (table 3). The average CLE from all the single-source test samples was 0.1975 mm by the MFCNN reconstruction, which was a great improvement in accuracy compared with the IVTCG results. The CLE maximum of MFCNN was obtained from the model-3, which owned a greater depth contrast to the other photon sources. This result demonstrated the disturbance of imaging depth to optical tomography. However, under the same depth, the proposed MFCNN demonstrated clearly higher reconstruction accuracy than the IVTCG. The improved performance of MFCNN could also be observed in reconstruction of big-sized samples (table S2). The calculated CLE of MFCNN was obviously less than the IVTCG results in each size level.
The performance of MFCNN was also validated in source shape recovery (tables 3 and S2). By DC calculation, shapes of the MFCNN results showed high consistency with the true sources. Using MFCNN reconstruction, the highest DC was observed greater than 0.90 and all the calculated DC were beyond the level of 0.64. Nevertheless, none of the IVTCG results showed a DC greater than 0.47.

Multiple-source reconstruction performance
Outperformance of MFCNN was also found in reconstruction of dual, triple and quadruple sources. From the typical dual-source results, the MFCNN results demonstrated high consistency with the true sources. As contrast,  deviation of center position was clearly observed between the IVTCG results and the true sources. The accuracy of RTE based IVTCG reconstruction was also significantly disturbed by the undesirable artifacts, exemplified by the results of model-3 in dual-source results ( figure 6). With the source number increased, the complexity raised and affected the performance of both MFCNN and IVTCG (figures S2 and S3). Nevertheless, in the results of various sources, MFCNN was observed still outperformed IVTCG. These qualitative contrasts further demonstrated the superiority of the proposed MFCNN reconstruction. The superiority of MFCNN reconstruction was also validated though quantitative CLE comparison (tables 4, S3 and S4). The lower CLE indicated that the MFCNN provided higher accuracy to reconstruct multiple sources. This outperformance was observed in the dual, triple and quadruple-source reconstruction compared with the IVTCG results. In the other hand, the CLE of multiple-source reconstruction was greater than the single-source results (table 3), which was induced by the raise in reconstruction complexity. The analyses with DC further showed that contrasted with IVTCG, the MFCNN results owned better shape consistency to the true multiple sources.

In vivo CLT reconstruction performance
In vivo BLI, CLI, PET and MRI were performed on the orthotropic mouse models (n = 3) of glioma. In optical imaging ( figure 7(a)), BLI showed the distribution of tumor cells. CLI results were obviously consistent to the BLI observation, which indicated high accuracy of in vivo CLI for tumor localization. PET results in various views ( figure 7(b)) further verified that the 11 C-MET was effectively accumulated in tumor. In the head region, 11 C-MET was also observed distributed in submaxillary gland ( figure 7(b), arrow 2) and lacrimal gland ( figure  7(b), arrow 3). However, CL emitted from these glands were blocked by thick skulls and muscles during imaging, and therefore only the tumor signals were detected. Results of optical imaging and PET demonstrated the in vivo CLI efficiently localized the tumor, which provided solid foundation for accurate CLT reconstruction.
Reconstruction results of MFCNN and IVTCG were contrasted with enhanced T1 MRI, GFP and H&E ( figure 7(c)). The glioma area was clearly presented in MRI with high signal. The merged CLT-MRI results showed that the gliomas were clearly localized by MFCNN without any artifacts. As a contrast, serious artifacts were observed in the IVTCG results. Meanwhile, compared with MRI and pathological results, MFCNN reconstruction showed higher accuracy than IVTCG in tumor localization. Tumor size in MRI, MFCNN and IVTCG was also compared (table 5). The three maximum dimensions of tumors areas were measured to reflect the size of irregular-shaped tumors. Size of the IVTCG artifacts were calculated as well. In all the mouse models (n = 3),

Discussion
The recent progress has demonstrated that CLT is an efficient and highly promising technique for biomedical in vivo imaging (Spinelli et al 2011. Through integration of optical images and anatomic information, CLT is able to achieve accurate tumor detection and quantification . However, high ill-posedness in the inverse problem strongly interferences the performance of CLT (Guo et al 2017a(Guo et al , 2017b. In this study, a MFCNN approach has been investigated for CLT reconstruction. Compared with the RTE based reconstruction methods, this novel MFCNN approach fundamentally averted the errors from the inverse process. Both the simulation and in vivo experiments have proved that the MFCNN advances in reconstruction accuracy and stability. In the MFCNN CLT approach, a seven layers fully connected network was established (figure 2). Due to the difficulty of collecting abundant in vivo data, the MFCNN was trained by diverse Monte Carlo simulation samples. Superiorities of the MFCNN reconstruction were shown in various aspects compared with the conventional RTE method. The CLE showed a minimum of 0.0405 mm in MFCNN reconstruction, while all the observed CLE of IVTCG results was higher than 0.20 mm (table 3). Photon sources of diverse positions, depth, shapes and sizes were all reliably reconstructed by the MFCNN approach without any artifacts. However, the IVTCG results were strongly interfered by the artifact problem ( figure 5). From all the single-source test samples, the average CLE by MFCNN was 0.1975 mm, which was obviously improved contrasting with the existing weight multispectral CLT (wmCLT) strategy (average location error 0.30 mm) (Guo et al 2017a), and the non-convex sparse regularization algorithm (nCSRA) framework (average location error 0.46 mm) (Guo et al 2017b). In addition, the quantitative analysis of DC showed that the shapes of photon sources could be successfully recovered with the MFCNN approach (table 3).
Excellent performance of MFCNN approach was also validated through multiple-source reconstruction. Greater CLE and smaller DC were obtained in the multiple-source results contrasting to the single-source reconstruction (tables 4, S3 and S4), which was mainly induced by the raised complexity in multiple-source reconstruction. Nevertheless, the proposed MFCNN approach still presented stronger performance than the IVTCG method. The CLE of dual-source MFCNN reconstruction results was approximately one third of that in IVTCG results. DC of the dual-source MFCNN results was also clearly improved compared to IVTCG reconstruction. These impressive outcomes have demonstrated the promising performance of MFCNN reconstruction in terms of accuracy and stability.
In vivo experiments on orthotropic glioma mouse models further verified the superiority and practicability of the MFCNN approach (figure 7). The high consistency between in vivo BLI and CLI results provided solid foundation to perform accurate tumor reconstruction. The MFCNN reconstruction of in vivo data demonstrated significant improvement comparing with the IVTCG method. The orthotropic glioma was successfully reconstructed using the MFCNN approach. The reconstruction accuracy of MFCNN was verified through the contrast with enhanced T1 MRI and the pathological results. However, the IVTCG results were failed to match with the true glioma area. This contrasting observation showed the outstanding accuracy and stability of the MFCNN CLT. Reconstruction efficiency could also be significantly strengthened by the MFCNN approach, because the complex iteration was no longer essential during CLT reconstruction. Moreover, compared with PET, MFCNN CLT provided a more intelligible method to image the lesion in vivo with desirable resolution. The radiopharmaceutical distribution visualized in PET revealed the potential of MFCNN CLT to image different diseases besides glioma. It is believed that the precision of lesion detection and treatment might be improved through the combination of MRI, PET and MFCNN CLT.
Although the MFCNN CLT provided exciting outcomes, several limitations remain necessary to be overcome. Similar to the other machine learning approaches, the MFCNN reconstruction is data driven. The accuracy of MFCNN CLT may be affected by the quality of training-set. Another concern is that the MFCNN approach mapped the in vivo data on a standard numerical mesh. This may lead to reconstruction deviation caused by the differences between living animals. Thus, optimization of the training process is planned in future work to raise the reconstruction performance for various demands.

Conclusion
A novel CLT reconstruction approach has been developed using multilayer fully connected neural network (MFCNN). With trained by diverse Monte Carlo simulation data, the complex relationship between the surface optical signal and the true photon source has been learned by the MFCNN. The well-trained MFCNN has proved superior to perform accurate CLT reconstruction in vivo. A series of experimental results have demonstrated that the MFCNN CLT approach obviously outperform the IVTCG method in reconstruction accuracy and stability. The MFCNN approach is expected to improve optical reconstruction in various situations and promote the application of CLT in different biomedical areas.