Liver Lesions and Acute Intracerebral Hemorrhage Detection Using Multimodal Fusion

Medical image fusion is designed to help physicians in their decisions by providing them with a preclinical image with enough information. Accurate assessment and effective treatment of the disease reduce the time it takes to relieve the symptoms of the disease. This article utilizes an effective data fusion approach to work on two different imaging modalities; computed tomography (CT) and magnetic resonance imaging (MRI). The data fusion approach is based on the combination of singular value decomposition (SVD) and the Fast Discrete Curvelet Transform (FDCT) techniques to reduce processing time during the fusion process. The SVD-FDCT data fusion approach is being tested with two multimodal medical image fusion applications. The first application concerns the detection of liver lesions and the second application concerns the early detection of acute intracerebral hemorrhage. Experimental tests demonstrate that not only the SVDFDCT data fusion algorithm can treat the curved objects and edges effectively as the FDCT do. But also, the throughput of the fusion algorithm is comparable to related fusion algorithms such as principal component analysis (PCA), Transform and Discrete Wavelet Transform (DWT), dual-tree complex wavelet transform (DT-CWT), and Curvelet fusion algorithms.


Introduction
Medical imaging plays a primary role in numerous medical therapy and diagnosis [1,2]. This necessitates a lot of precise images with significantly further detailed data for accurate medical diagnosis applications. The fusion process of medical images is one solution for achieving high spectral and spatial contents in a singular medical image [3]. In this regard, various approaches have been introduced in the works, for example, the high-pass filtering technique [4], Principal Component Analysis (PCA) [5] and Intensity Hue and Saturation (IHS) [6]. MRI and CT medical images are widely used in medical diagnosis. MRI images are beneficial in soft tissues such as the brain, spinal cord, and nerves than CT. On the other hand, the CT image provides better details in the case of the bones than the MRI image. In some cases, such as brain tumours, especially Gliomas, Skull base tumours, and bone tumours, complementary information about the tumour mass and its surroundings is very important to apply a successful removal.
Although these spatial domain approaches [7,8] provide enhanced graphical feature descriptions, several examination studies show the graphical distortion of colours in combination images.
The high-pass filter technique conserves spatial detail, but there is noise in the fused image. The IHS transformation precisely changes the intensity elements with images that have high resolution panchromatic. Nevertheless, the spatial resolution enhances the supernatural artifacts of the resulting image due to the exchange of the intensity element [9].
Lately, transformation domain methods using multi-resolution analysis have been developed, such as wavelet transformation. Wavelet transformations can store and preserve the time and frequency details. The entered input image is broken down to various frequency levels, and mixing schemes are then applied, changing the element coefficients with the detail components of the added input image. Although wavelet fusion gives high spectral quality, the spatial declaration is reduced because of subsampling when the remotely sensed images are used for land cover and use [1,5,6].
Due to the complex nature of land use characteristics, most applications require spatial information. Although many image fusion techniques are available in the literature for integrating remotely recognized data, not all methods meet the necessity for mapping the properties of the earth's surface [8].
One of the limitations of the simple fusion methods such as PCA [11], DWT [12], and DT-CWT [13] fusion methods is that they can't handle the curved objects in the images to be fused efficiently. Curvelet fusion was introduced as a solution for that problem, but unfortunately, it takes a very long time to apply the fusion process [14]. The FDCT is the second Curvelet transform generation introduced to efficiently handle the curved objects and edges in the images to be fused and provide a short processing time, avoiding the time in the Curvelet transform. One of the most considerable limitations of the FDCT is its computational complexity due to its mathematical modeling and extensive processing capabilities.
In this paper, a SVD-FDCT fusion approach is utilized to overcome the limitation of all the last mentioned methods. The SVD-FDCT fusion approach is effectively treats the curved objects and edges with a low processing time slower than the Curvelet approach and lower computational complexity than the FDCT fusion approach.
The remainder of the paper is arranged as follows. Section 2 discusses and explores the CT and MRI medical imaging modalities. Section 3 gives the essential preliminaries regarding the FDCT and SVD techniques. Section 4 explores the SVD-FDCT image fusion algorithm. Section 5 presents the examined test results and provides discussions about these results. Finally, Section 6 gives the conclusions.

CT and MRI Medical Imaging Modalities
The medical image fusion process collects features of various images into a singular fused image. The fusion process is applied to improve the quality and reduce the redundant data in images before applying the fusion. Fusion also increases the clinical usage of the resulted image for diagnosing and assessing medical problems. The multiple medical images to be fused may be obtained from a single sensor or modality, and the fusion aims to increase the reliability of the features obtained from fused images. Also, medical images can be obtained from multiple sensors or multiple imaging modalities. In this case, the fusion methods assist in extracting features and revealing information that may be unseen to the human eye in a single imaging modality. These revealed features can be used for more localization of abnormalities. For example, MRI and CT show the anatomical structures.
In contrast, other modalities show functional and metabolic activity like SPECT and PET [15,16]. Medical image fusion is employed between different imaging modalities to combine all the complementary features into one image to have more information and features. The advantages of the fusion, in this case, are to assist in diagnosing diseases and reducing the storage needed to store the images before applying the fusion.
Practically, to apply the fusion between both images obtained from the last two modalities, there are two choices. The first is to have a hybrid scanner that can apply both the MRI and CT imaging to the patient and fuse the two resulted images and provide the final fused image with the complementary needed information. Still, unfortunately, this scanner is costly to be generalized, especially in developing nations. The second choice is to have each image alone from a separate scanner (which will be cheaper than the Hybrid scanner), and then by using specific software, the fusion process is done.

CT
CT is a type of medical imaging modalities in which a computer and an array of X-ray sensors create CT images [15]. CT imaging is frequently used in looking at the bone and other hard tissues. The CT images are widely employed in various medical applications depending on the clinical condition. Image fusion with CT is employed in several applications such as 3D tumour simulations, brain diagnostics and more. One of the most important CT scan advantages is the short scan time and higher imaging resolution. But CT scan has limitations such as limited tissue characterization. Fusion between CT and other medical imaging modalities is common in the medical image fusion branch, which helps in many situations.

MRI
The MRI is another class of medical imaging modalities in which radio waves and a potential magnet are connected to a computer and utilized for creating images of regions within the body [15,16]. One of the advantages of the MRI is its safety for babies and pregnant women because it doesn't use any exposure such as X-rays or any other radiation but only radio waves which lie in the FM range. Also, it helps look at the non-bony parts or soft tissues structures like the brain, heart, spinal cord, nerves and eyes, which are imaged with high accuracy than CT scans. For the same last reason, MRI scans are widely utilized for looking at shoulders and Knees after injuries. On the other hand, the main problem MRI images rely on their proportional sensibility to the motion, making it difficult to imaging the organs involving movements, such as mouth tumours. These limitations can be overcome using image fusion in a multimodal imaging environment. MRI images have been fused with other modalities such as CT; for example, that is one of the most common fusion combinations due to its practical usability in clinical issues.

Preliminaries
This section explains and discusses the main preliminaries regarding the FDCT, and SVD approaches.

Fast Discrete Curvelet Transform (FDCT)
An anisotropic geometric wavelet transform named the ridgelet transform may be considered optimal for expressing the straight line singularities. However, the global line singularities are not often detected with real applications [12,17]. For solving either the singularity of local lines or curves, a partition of the image can be taken. After that, the transform can be applied to the resulted sub-images. Candès and Donoho have proposed the ridgelet block-based transformation known as the curvelet transformation in Xinchun et al. [17].
The curvelet approach efficiency results from its power to build strong approximation and operator theorems. The discrete curvelet transform can be considered powerful in describing curve-like edges in the image and analysing its features. The implementation of first-generation curvelet transform is restricted due to redundancy caused by the ridgelet analysis. The ridgelet geometry is unclear since they are not valid ridge functions in digital images [18]. The FDCT is considered the second curvelet transform generation that depends on a frequency partition method [19,20]. It is proposed with the potential of decreasing the amount of processing time by reducing redundancy caused by the Ridgelet transform [18]. The FDCT can be employed using two methods; the first method depends on Unequally-Spaced Fast Fourier Transform (USFFT), while the other depends on the selected Fourier sample wrapping [19]. Here, we are using the FDCT that depends on the selected Fourier sample wrapping. The FDCT coefficients C(j, l, b) in the selected Fourier sample wrapping version of data f x ð Þ can be expressed as [19,20]: where C j; l; k ð Þdefines the FDCT coefficients, j is a scale parameter, l is an orientation parameter, b is a translation parameter, u l is the rotation angle, w is continuous frequency, U j;u l denoted the real valued wedge window dilated to the scale j and parabolically localized to angles near u l by the shearing operation,f denotes the 2D Fast Fourier Transform (FFT) of the Cartesian array f t1; t2 ½ ; 0 t1; t2 < n, that can be estimated as following [18,19]:

Singular Value Decomposition
The SVD scheme may be considered a dimensional reduction scheme applied to the matrixes to reduce matrixes dimensionality [20,21]. If the SVD is applied to a matrix A of M Â N size and rank r, then the SVD of A is computed as [20,21]: where S defines the diagonal matrix that is termed the singular matrix. The S diagonal entries represent the Eigenvalues of matrix A with the Si > 0 and Si > Si+1. The other entries have a value of zero. The U and V matrixes define orthogonal matrixes, and their first r columns define the orthogonal Eigenvectors corresponding to the S matrix Eigenvalues. The SVD can be considered a reversible scheme, and when the three matrixes U, S and V are given, the original matrix A can be obtained using Eq. (3).

The SVD-FDCT Fusion Algorithm
The SVD-FDCT Fusion scheme presented in Mohammed et al. [22] is examined on two vital multimodal medical image fusion applications. The first application considers the detection of hepatic liver lesions. The second application considers the detecting of acute intracerebral hemorrhage.
The SVD-FDCT Fusion scheme is depicted in Fig. 1. As depicted in Fig. 1 and the detailed steps can be summed up as follows: 1. Both CT and MRI images that are registered are subjected to SVD to obtain the components U ct , S ct , and V ct for the CT image and Umri, Smri, and Vmri for the MRI image. 2. The FDCT is applied to the obtained S matrices (S ct and S mri ), and the FDCT coefficients are obtained for each S matrix (C ct and C mri ). 3. The maximum fusion rule is used in fusing the obtained coefficients (C ct and C mri coefficients), and a fused coefficient C f is obtained. 4. Apply the inverse FDCT on the C f coefficient to have the S f matrix, representing the fusion result between the two matrices S ct and S mri using FDCT. 5. Fusion is applied on the (U ct , U mri ) and (V ct , V mri ) utilizing a simple maximum fusion rule to obtain the two fused matrices U f and V f . 6. Finally, the Inverse SVD is applied on the obtained fused matrices U f , S f , and V f to have the fused image.
In the last steps, only the S matrix obtained from applying the SVD on the input images (CT and MRI images) are subjected to the FDCT and not the two other matrices (U and V matrices). The reason for choosing only one matrix (S) instead of the three matrices (U, S, and V), which represents the whole image, is to decrease the algorithm complexity and reduce fusion processing time. The S matrix had been chosen and not the U or V matrix because it contains the most important features in the image. Also, it represents the diagonal matrix (all the values in it except the diagonal are zeros) and minimizes the processing time during computing the coefficients of the FDCT and so minimizing the total fusion processing time.

Experimental Results
The SVD-FDCT fusion algorithm and four fusion schemes, including PCA [11], DWT [12], DT-CWT [13] and Curvelet [14], are applied to two vital multimodal medical image fusion applications. The first application utilizes image fusion techniques in detecting hepatic lesions. For hepatic lesions, CT images can show some lesions. Still, not all of them and the same thing is about MRI images, so CT/MRI image fusion is introduced in this application to show all the lesions that are not seen in one of the images alone. The second application utilizes image fusion techniques in detecting the acute intracerebral hemorrhage. The most important factor, in this case, is the time consumed during the fusion process, which represents the time of the detection of the acute intracerebral. For the above mentioned medical applications, CT and MRI images had been fused using Matlab 7.11.0 (R2017b) on a Personal Computer with an Intel Core 7 Due processor and 8 Gigabyte of RAM. The fusion key performance metrics including the average gradient (g), the local contrast (Clocal), the mutual information (MI), standard deviation (STD), entropy (E), and the edge intensity [23,24], were used to evaluate the produced fused images using the SVD-FDCT and the other four fusion schemes, including PCA [11], DWT [12], DT-CWT [13] and Curvelet [14]. The fused images produced by the different fusion algorithms were also shown for more subjective evaluation. The following discussion shows the results of applying the SVD-FDCT fusion approach and the other four fusion schemes, including PCA [11], DWT [12], DT-CWT [13] and Curvelet [14] in the two above mentioned medical applications. Fig. 2 and Tab. 1 show the experimental results of hepatic lesions detection of case 1 using the SVD-FDCT fusion algorithm and the other four fusion schemes, including PCA [11], DWT [12], DT-CWT [13] and Curvelet [14]. Medical image fusion techniques had been to facilitate hepatic lesions detection. CT images alone can't provide all the lesions in the liver nor the MRI images alone, so CT/MRI image fusion is a solution for more accurate detection of the hepatic lesions. In this case, two 128 x 128 pixel size images, one of them is a CT image, and the other is a MRI image of a 52-year older woman with colon cancer, had been fused using the SVD-FDCT fusion algorithm. The CT image shows a focal lesion interpreted by the specialists as possible metastasis. Surgery, in this case, showed no metastatic lesions in the liver at those locations suspected in the CT image. As seen, neither the CT image nor the MRI image alone is sufficient for detecting hepatic lesions. The SVD-FDCT fusion approach is comparable to the other traditional fusion algorithms. The SVD-FDCT fusion algorithm has the advantage of handling the curved objects in the fused images more efficiently than the PCA [11], DWT [12], DT-CWT [13] and Curvelet [14], with a short processing time that is comparable to them.

Conclusions
This paper considered the idea of utilizing multimodal image fusion techniques for detecting liver lesions and acute intracerebral hemorrhage. The SVD-FDCT image fusion algorithm is a hybrid fusion algorithm based on combining the SVD and the FDCT. The SVD-FDCT image fusion algorithm is designed with the potential of handling the curved objects in the image and minimizing the processing time during the fusion process, especially with large sizes of the images to be fused. Results obtained by applying the SVD-FDCT image fusion algorithm and other four different fusion algorithms demonstrated that the SVD-FDCT image fusion algorithm could be considered as a comparable algorithm with the other traditional fusion algorithms. It has the advantages of saving the processing time than the curvelet algorithm and dealing with the curved objects in the images with the same efficiency.