Elsevier

Neurocomputing

Volume 182, 19 March 2016, Pages 1-9
Neurocomputing

Hybrid dual-tree complex wavelet transform and support vector machine for digital multi-focus image fusion

https://doi.org/10.1016/j.neucom.2015.10.084Get rights and content

Abstract

This study proposed a new method for multi-focus image fusion using hybrid wavelet and classifier. The image fusion process was formulated as a two-class classification problem: in and out-of-focus classes. First, a six-dimensional feature vector was extracted using sub-bands of dual-tree complex wavelet transform (DT-CWT) coefficients from the source images, which were then projected by a trained two-class support vector machine (SVM) to the class labels. A bacterial foraging optimization algorithm (BFOA) was developed to obtain the optimal parameters of the SVM. The output of the classification system was used as a decision matrix for fusing high-frequency wavelet coefficients from multi-focus source images in different directions and decomposition levels of the DT-CWT. After the high and low-frequency coefficients of the source images were fused, the final fused image was obtained using the inverse DT-CWT. Several existing methods were compared with the proposed method. Experimental results showed that our presented method outperformed the existing methods, in visual effect and in objective evaluation.

Introduction

Image fusion refers to an image processing technique that produces a new and improved single image, known as the fused image. This paper is concerned with the problem of multi-sensor pixel-level image fusion. Through image fusion, we aim to produce reliable methods that represent the salient information obtained from different imaging sensors, and fuse these details into a synthetic image. Thereafter, image fusion becomes more applicable for human visual perception and computer processing.

Many important applications, such as digital imaging, medical imaging, remote sensing and machine vision, need image fusion techniques [1], [2], [3], [4], [5]. This paper has concentrated on multi-focus image fusion. Optical imaging cameras suffer from the problem of finite depth of field, which makes taking an image with all objects contained “in focus” impossible. The solution to this problem is the multi-focus image fusion technique. This technique can be useful, in digital camera design or in industrial inspection applications, where the need to visualize objects at very short distances makes the preservation of the depth-of-focus difficult [1], [6].

Various methods implement image fusion for multi-focus images. The simplest fusion method in spatial domain simply takes the pixel-by-pixel average of the source images, which leads to reduced contrast [1]. Some more reasonable methods are proposed, such as fusing source images with divided blocks or segmented regions, but they suffer from blackness in the fused image [7], [8], [9]. This study has focused on the wavelet-based approach, which is a subset of multi-scale decomposition based methods. The ability of the wavelet transform to capture important features in a picture is the reason for selecting multi-scale decomposition-based methods as a tool for image fusion [10]. Beyond this reason, the main reason is the time–frequency analysis of wavelet transform, while in-focus pixels of an image contain major high-frequency information. Therefore, this ability of wavelet transform can be used to determine in-focus pixels.

Although DWT (Discrete Wavelet Transform) has been successfully used for image de-noising, it has shortcomings such as shift variance, aliasing and lack of directionality [11]. Real valued wavelet transforms do not provide any phase information [12]. Phase information describes the amplitude and the local behavior of a function, according to [13]. DT-CWT (Dual-tree complex wavelet transform) provides shift invariance and better directionality than real valued wavelet transforms [14]. DT-CWT is proposed to capture additional edge information [15]. The higher directionality and shift invariance properties of DT-CWT make it suitable for image fusion [16]. In obtaining the satisfactory fusion results, multi-scale based methods include three stages: decomposition, coefficients fusion, and reconstruction [17]. By reviewing existing papers, we find that the integration of DT-CWT and kernel method classifier for image fusion has gained little attention.

Therefore, the paper proposed a new multi-focus image fusion method based on the integration of DT-CWT and support vector machine with bacterial foraging optimization. SVM is based on statistical learning theory and specializes for a smaller number of samples [18] that is suitable for distinguishing the features of DT-CWT in normal images. Specifically, feature-based fusion rules were presented to merge high- and low-frequency wavelet coefficients for the best quality in the fused image. The key step in the proposed image fusion method is the use of BFO-SVM in selecting DT-CWT coefficient features, and this issue has been investigated in this study. The inter-scale dependencies among wavelet coefficients in the DT-CWT sub-bands were proposed to obtain a reliable decision matrix. First, four-feature vectors were obtained using six directional sub-bands of DT-CWT in the first decomposition level of the source images. Then, these feature vectors were classified into two classes through an optimally trained SVM classifier [19], and the output, a decision matrix, selected high-frequency wavelet coefficients between two source images. The classifier output was also used to select low-frequency wavelet coefficients between the source images by down sampling.

This paper is organized as follows. In Section 2, the proposed fusion algorithm based on the BFO-SVM classifier and DT-CWT is presented. Section 3 shows various results and comparisons. Finally, Section 4 concludes with a brief summary.

Section snippets

Proposed image fusion method

In this section, we propose a novel multi-focus image fusion method using DT-CWT. Let us suppose that A and B represent the different source images of the same size. The block diagram of the proposed method is shown in Fig. 1. The essential steps of the proposed image fusion are arranged as follows:

  • Step 1:

    j-Level DT-CWT decomposition of source images is performed, and six directional high-frequency coefficients are obtained as xj(j=1,2,,J) and low-frequency coefficients yj.

  • Step 2:

    Four-feature vectors LF of

Experiment

This section consists of the following two parts: (1) objective evaluation index and (2) results and discussion. In evaluating the proposed method, the fusion performances of some other public methods were compared. Six different methods were compared, including pixels averaging, principal components analysis (PCA) [26], DWT with the BFO-SVM, and DT-CWT with three other SVM classifiers: v-SVM [27], PSO-SVM [28], and cross-verification parameter optimization SVM (CVP-SVM).

Experimental images are

Conclusions

This study put forward a novel multi-focus image fusion method based on DT-CWT and BFO-SVM classifier. The proposed method decomposed the source multi-focus images into six directional sub-band coefficients, and then classified the high-frequency coefficient feature using SVM with optimal parameters to produce the decision matrix. The coefficients from DT-CWT contain the detailed information in the spatial and frequency domains, and image integrity is not lost during the process. The

Acknowledgements

This research is supported by the Shanghai Pujiang Program under Grants 15PJ1404300 and are search grant of PLA General Armament Department's Foundation for Forward Research of Weapons (9140A17040114JW03241); the National Natural Science Foundation of China under Grants 61374195.

Biting Yu received the B.S. degree in Biomedical Engineering from Nanjing University of Aeronautics and Astronautics, and is working toward the M.E. degree in Shanghai Jiao Tong University, China. Her current research interests include optimal algorithm and classification algorithm.

References (29)

  • V.S. Petrovic et al.

    Gradient-based multiresolution image fusion

    IEEE Trans. Image Process.

    (2004)
  • Y. Song, M. Li, Q. Li, L. Sun, A new wavelet based multi-focus image fusion scheme and its application on optical...
  • L. Phenlan, H. Poying, Fusion methods based on dynamic-segmented morphological wavelet or cut and paste for multifocus...
  • J. Saeedi et al.

    A classification and fuzzy-based approach for digital multi-focus image fusion

    Pattern Anal. Appl.

    (2013)
  • Cited by (83)

    • Tri-modal medical image fusion based on adaptive energy choosing scheme and sparse representation

      2022, Measurement: Journal of the International Measurement Confederation
    • An improved approach for medical image fusion using sparse representation and Siamese convolutional neural network

      2022, Biomedical Signal Processing and Control
      Citation Excerpt :

      The multiscale transform (MST) fusion technique emerged recently and is employed for computer vision and image processing areas. Moreover, there are various transforms based fusion images, for instance a discrete wavelet (DWT)-based [7], complex wavelet transform (CWT)-based [8], dual-tree complex wavelet transform (DTCWT)-based [9], the non-subsampled contourlet transform (NSCT)-based method [10], curvelet transform (CVT)-based [11], contourlet transform based [12], dense SIFT-based [13], shear-let transform-based [14], singular value decomposition (SVD) [15], and sparse representation based techniques [16]. However, the shortcomings of these methods are always observed in the design of the transform basis, namely activity level measurement and fusion rule.

    View all citing articles on Scopus

    Biting Yu received the B.S. degree in Biomedical Engineering from Nanjing University of Aeronautics and Astronautics, and is working toward the M.E. degree in Shanghai Jiao Tong University, China. Her current research interests include optimal algorithm and classification algorithm.

    Bo Jia is currently pursuing a Ph.D. in Control Science and Engineering in Shanghai Jiao Tong University, China. He received M.E. degree in Shanghai Jiao Tong University in 2013. His current research interests include signal processing and pattern recognition.

    Lu Ding received her Bachelor׳s degree in Engineering from Wu Han University, China, in 2011. Now, she is currently pursuing a Ph.D. in Shanghai Jiao Tong University. Her areas of interest include image processing, pattern recognition and machine learning.

    Zhengxiang Cai is currently an M.S. student at the School of Aeronautics and Astronautics of the Shanghai Jiao Tong University. His research interests include dynamic modeling and Gaussian process classification with applications in human modeling and ergonomics.

    Qi Wu is an Associate Professor of Control Science and Engineering at the School of Electronic, Information and Electrical Engineering, the Shanghai Jiao Tong University. His research interests are pattern recognition and fault diagnosis.

    Rob Law Ph.D. is a Professor of Technology Management at the School of Hotel and Tourism Management, the Hong Kong Polytechnic University. His research interests are information management and technology applications.

    Jiayang Huang is a Senior engineer of Shanghai Engineering Research Center of Civil Aircraft Health Monitoring reliability and health management.

    Lei Song is currently pursuing a Ph.D. in Control Science and Engineering in Shanghai Jiao Tong University, China. His current research interests include signal processing and network control system.

    Shan Fu was born in 1964. Received his B.Eng. degree in EE from Northwestern Polytechnic University and Ph.D. degree from Heriot-Watt University respectively. He is currently a professor in System Engineering in the School of Electronic Information and with particular research interests in human factors, intelligent system and pattern recognition.

    View full text