Hybrid dual-tree complex wavelet transform and support vector machine for digital multi-focus image fusion
Introduction
Image fusion refers to an image processing technique that produces a new and improved single image, known as the fused image. This paper is concerned with the problem of multi-sensor pixel-level image fusion. Through image fusion, we aim to produce reliable methods that represent the salient information obtained from different imaging sensors, and fuse these details into a synthetic image. Thereafter, image fusion becomes more applicable for human visual perception and computer processing.
Many important applications, such as digital imaging, medical imaging, remote sensing and machine vision, need image fusion techniques [1], [2], [3], [4], [5]. This paper has concentrated on multi-focus image fusion. Optical imaging cameras suffer from the problem of finite depth of field, which makes taking an image with all objects contained “in focus” impossible. The solution to this problem is the multi-focus image fusion technique. This technique can be useful, in digital camera design or in industrial inspection applications, where the need to visualize objects at very short distances makes the preservation of the depth-of-focus difficult [1], [6].
Various methods implement image fusion for multi-focus images. The simplest fusion method in spatial domain simply takes the pixel-by-pixel average of the source images, which leads to reduced contrast [1]. Some more reasonable methods are proposed, such as fusing source images with divided blocks or segmented regions, but they suffer from blackness in the fused image [7], [8], [9]. This study has focused on the wavelet-based approach, which is a subset of multi-scale decomposition based methods. The ability of the wavelet transform to capture important features in a picture is the reason for selecting multi-scale decomposition-based methods as a tool for image fusion [10]. Beyond this reason, the main reason is the time–frequency analysis of wavelet transform, while in-focus pixels of an image contain major high-frequency information. Therefore, this ability of wavelet transform can be used to determine in-focus pixels.
Although DWT (Discrete Wavelet Transform) has been successfully used for image de-noising, it has shortcomings such as shift variance, aliasing and lack of directionality [11]. Real valued wavelet transforms do not provide any phase information [12]. Phase information describes the amplitude and the local behavior of a function, according to [13]. DT-CWT (Dual-tree complex wavelet transform) provides shift invariance and better directionality than real valued wavelet transforms [14]. DT-CWT is proposed to capture additional edge information [15]. The higher directionality and shift invariance properties of DT-CWT make it suitable for image fusion [16]. In obtaining the satisfactory fusion results, multi-scale based methods include three stages: decomposition, coefficients fusion, and reconstruction [17]. By reviewing existing papers, we find that the integration of DT-CWT and kernel method classifier for image fusion has gained little attention.
Therefore, the paper proposed a new multi-focus image fusion method based on the integration of DT-CWT and support vector machine with bacterial foraging optimization. SVM is based on statistical learning theory and specializes for a smaller number of samples [18] that is suitable for distinguishing the features of DT-CWT in normal images. Specifically, feature-based fusion rules were presented to merge high- and low-frequency wavelet coefficients for the best quality in the fused image. The key step in the proposed image fusion method is the use of BFO-SVM in selecting DT-CWT coefficient features, and this issue has been investigated in this study. The inter-scale dependencies among wavelet coefficients in the DT-CWT sub-bands were proposed to obtain a reliable decision matrix. First, four-feature vectors were obtained using six directional sub-bands of DT-CWT in the first decomposition level of the source images. Then, these feature vectors were classified into two classes through an optimally trained SVM classifier [19], and the output, a decision matrix, selected high-frequency wavelet coefficients between two source images. The classifier output was also used to select low-frequency wavelet coefficients between the source images by down sampling.
This paper is organized as follows. In Section 2, the proposed fusion algorithm based on the BFO-SVM classifier and DT-CWT is presented. Section 3 shows various results and comparisons. Finally, Section 4 concludes with a brief summary.
Section snippets
Proposed image fusion method
In this section, we propose a novel multi-focus image fusion method using DT-CWT. Let us suppose that A and B represent the different source images of the same size. The block diagram of the proposed method is shown in Fig. 1. The essential steps of the proposed image fusion are arranged as follows:
- Step 1:
j-Level DT-CWT decomposition of source images is performed, and six directional high-frequency coefficients are obtained as and low-frequency coefficients .
- Step 2:
Four-feature vectors LF of
Experiment
This section consists of the following two parts: (1) objective evaluation index and (2) results and discussion. In evaluating the proposed method, the fusion performances of some other public methods were compared. Six different methods were compared, including pixels averaging, principal components analysis (PCA) [26], DWT with the BFO-SVM, and DT-CWT with three other SVM classifiers: v-SVM [27], PSO-SVM [28], and cross-verification parameter optimization SVM (CVP-SVM).
Experimental images are
Conclusions
This study put forward a novel multi-focus image fusion method based on DT-CWT and BFO-SVM classifier. The proposed method decomposed the source multi-focus images into six directional sub-band coefficients, and then classified the high-frequency coefficient feature using SVM with optimal parameters to produce the decision matrix. The coefficients from DT-CWT contain the detailed information in the spatial and frequency domains, and image integrity is not lost during the process. The
Acknowledgements
This research is supported by the Shanghai Pujiang Program under Grants 15PJ1404300 and are search grant of PLA General Armament Department's Foundation for Forward Research of Weapons (9140A17040114JW03241); the National Natural Science Foundation of China under Grants 61374195.
Biting Yu received the B.S. degree in Biomedical Engineering from Nanjing University of Aeronautics and Astronautics, and is working toward the M.E. degree in Shanghai Jiao Tong University, China. Her current research interests include optimal algorithm and classification algorithm.
References (29)
- et al.
Image fusionadvances in the state of the art
Inf. Fusion
(2007) - et al.
A wavelet-based image fusion tutorial
Pattern Recognit.
(2004) - et al.
Combination of images with diverse focuses using the spatial frequency
Inf. Fusion
(2001) - et al.
Multifocus image fusion using region segmentation and spatial frequency
Image Vis. Comput.
(2008) - et al.
Pixel- and region-based image fusion with complex wavelets
Inf. fusion
(2007) - et al.
Infrared and visible image fusion using fuzzy logic and population-based optimization
Appl. Soft Comput.
(2012) - et al.
Bearing fault detection of induction motor using wavelet and support vector machines (SVMs)
Appl. Soft Comput.
(2011) Complex wavelets for shift invariant analysis and filtering of signals
Appl. Comput. Harmon. Anal.
(2001)- Z. Zhang, R.S. Blum, A categorization of multiscale-decomposition-based image fusion schemes with a performance study...
- et al.
A data-fusion scheme for quantitative image analysis by using locally weighted regression and Dempster–Shafer theory
IEEE Trans. Instrum. Meas.
(2008)
Gradient-based multiresolution image fusion
IEEE Trans. Image Process.
A classification and fuzzy-based approach for digital multi-focus image fusion
Pattern Anal. Appl.
Cited by (83)
Infrared-visible image fusion method based on multi-scale shearing Co-occurrence filter
2024, Infrared Physics and TechnologyFDGNet: A pair feature difference guided network for multimodal medical image fusion
2023, Biomedical Signal Processing and ControlMulti-modal medical image super-resolution fusion based on detail enhancement and weighted local energy deviation
2023, Biomedical Signal Processing and ControlTri-modal medical image fusion based on adaptive energy choosing scheme and sparse representation
2022, Measurement: Journal of the International Measurement ConfederationAn improved approach for medical image fusion using sparse representation and Siamese convolutional neural network
2022, Biomedical Signal Processing and ControlCitation Excerpt :The multiscale transform (MST) fusion technique emerged recently and is employed for computer vision and image processing areas. Moreover, there are various transforms based fusion images, for instance a discrete wavelet (DWT)-based [7], complex wavelet transform (CWT)-based [8], dual-tree complex wavelet transform (DTCWT)-based [9], the non-subsampled contourlet transform (NSCT)-based method [10], curvelet transform (CVT)-based [11], contourlet transform based [12], dense SIFT-based [13], shear-let transform-based [14], singular value decomposition (SVD) [15], and sparse representation based techniques [16]. However, the shortcomings of these methods are always observed in the design of the transform basis, namely activity level measurement and fusion rule.
A multi-focus color image fusion algorithm based on low vision image reconstruction and focused feature extraction
2022, Signal Processing: Image Communication
Biting Yu received the B.S. degree in Biomedical Engineering from Nanjing University of Aeronautics and Astronautics, and is working toward the M.E. degree in Shanghai Jiao Tong University, China. Her current research interests include optimal algorithm and classification algorithm.
Bo Jia is currently pursuing a Ph.D. in Control Science and Engineering in Shanghai Jiao Tong University, China. He received M.E. degree in Shanghai Jiao Tong University in 2013. His current research interests include signal processing and pattern recognition.
Lu Ding received her Bachelor׳s degree in Engineering from Wu Han University, China, in 2011. Now, she is currently pursuing a Ph.D. in Shanghai Jiao Tong University. Her areas of interest include image processing, pattern recognition and machine learning.
Zhengxiang Cai is currently an M.S. student at the School of Aeronautics and Astronautics of the Shanghai Jiao Tong University. His research interests include dynamic modeling and Gaussian process classification with applications in human modeling and ergonomics.
Qi Wu is an Associate Professor of Control Science and Engineering at the School of Electronic, Information and Electrical Engineering, the Shanghai Jiao Tong University. His research interests are pattern recognition and fault diagnosis.
Rob Law Ph.D. is a Professor of Technology Management at the School of Hotel and Tourism Management, the Hong Kong Polytechnic University. His research interests are information management and technology applications.
Jiayang Huang is a Senior engineer of Shanghai Engineering Research Center of Civil Aircraft Health Monitoring reliability and health management.
Lei Song is currently pursuing a Ph.D. in Control Science and Engineering in Shanghai Jiao Tong University, China. His current research interests include signal processing and network control system.
Shan Fu was born in 1964. Received his B.Eng. degree in EE from Northwestern Polytechnic University and Ph.D. degree from Heriot-Watt University respectively. He is currently a professor in System Engineering in the School of Electronic Information and with particular research interests in human factors, intelligent system and pattern recognition.