CT image quality enhancement via a dual-channel neural network with jointing denoising and super-resolution

Abstract

In recent years, computed tomography (CT) has been widely used in various clinical diagnosis. Given potential health risks bring by the X-ray radiation, the major objective of the current research is to achieve high-quality CT imaging while reducing X-ray radiation. However, most existing studies on low-dose CT image super-resolution reconstruction do not focus on the interaction between the denoising task and the super-resolution task. In this paper, we propose a dual-channel joint learning framework to accurately reconstruct high-resolution CT images from low-resolution CT images. Unlike the previous cascaded models which directly combine the denoising network and the super-resolution network, our method can process the denoising reconstruction and the super-resolution reconstruction in parallel. Additionally, we design a filter gate module that can filter features from the denoising branch and highlight important features which can benefit the super-resolution task. We evaluate the performance of our method in medical image enhancement by testing on the 2016 Low-Dose CT Grand Challenge dataset and the piglet dataset. The experimental results show that the proposed network is superior to other state-of-the-art methods in terms of both peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM). We also demonstrate that our method can better remove noise and recover details. Furthermore, the method achieves competitive results not only for super-resolution reconstruction of low-dose CT, but also for super-resolution reconstruction of sparse-view CT.

Introduction

Computed tomography (CT) is a disease detection technique that scans a specific area of the human body by using X-rays, gamma rays, or other types of beams [1]. CT has the advantages of being simple to operate and producing high-resolution images. Therefore, it is widely used in a variety of clinical diagnoses [2]. However, the excessive radiation exposure will bring the potential cancer risk to the patient [3]. To ensure the health of patients, efforts have been made to reduce the radiation dose. Nevertheless, decreasing the radiation dose to obtain low-dose CT (LDCT) images may degrade the quality of images, with noise and more artifacts, which will affect the final diagnosis results [4]. Thus, how to improve the quality of LDCT images is a research priority [5], [6].

Over the past decade, researchers focused on developing new iterative algorithms to enhance the image quality of LDCT [7], [8]. Iterative reconstruction algorithms significantly improve image quality, but they require a high computational cost, which limits their practical applications. Additionally, the methods based on dictionary learning and sparse representation are also widely explored [9], [10]. Considering the enormous potential of neural networks in the field of image processing, researchers began applying deep learning techniques to dual-energy CT imaging [11], [12] and LDCT denoising [6], [13], [5]. For example, a convolutional encoder-decoder with residual learning [5] was used for restoring normal dose CT images. Recently, Yin et al. [14] proposed a domain progressive 3D residual convolution network, which significantly improves the image reconstruction performance by combining the network processing in sinogram domain and image domain. However, current CT reconstruction methods were primarily concerned with denoising, neglecting the resolution constraints of the image.

Single-image SR technology is a classic image enhancement technology, which refers to the process of recovering a high-resolution (HR) image from a low-resolution (LR) image [15]. SR of medical images improves the quality of digital imaging systems, therefore, physicians can make a more precise diagnosis of the disease using the enhanced SR images. Generally, there are three types of SR reconstruction methods [16]: (1) interpolation-based methods [17], [18], (2) model-based reconstruction methods [19], [20], and (3) learning-based methods [21], [22], [23]. Those methods based on interpolation mainly include nearest-neighbor interpolation, bilinear interpolation, and bicubic interpolation [24], which are simple to implement and have been widely used in the SR restoration of LR images. These methods can effectively increase the resolution of the LR image but have poor visual quality. When dealing with real images with complex structures, such as CT images, due to the fact that traditional interpolations do not take structural information into account, they had a limited effect and may even produce artifacts. The model-based methods explicitly model the degradation process of the image and regularize the reconstruction according to the characteristics of the data. In comparison to the interpolation-based reconstruction methods, they can recover more detailed information from LR images. The learning-based methods learn the nonlinear mapping from paired LR and HR images in order to recover missing high-frequency details. Yang et al. [22] suggested a novel approach for single image SR based on sparse representations in terms of the coupled dictionaries that were jointly trained from HR and LR image patch pairs, and achieved encouraging results on the SR reconstruction of the face image. Considering the fact that images frequently contain many repetitive image structures, adaptive regularization and adaptive sparse domain selection were combined by [21] to achieve excellent results in terms of visual quality and PSNR.

With the rapid development of artificial intelligence and the continuous improvement of computer hardware performance in recent years, researchers started to restore images by using deep learning. Dong et al. [25] noticed that traditional SR methods based on the sparse coding can be interpreted as a deep convolutional neural network, and achieved advanced reconstruction results. In addition, many studies have been conducted on the application of deep learning to medical imaging [26], [27], [28]. In [26], a novel convolutional neural network was designed to learn residual-based transformations from LR to HR images. Subsequently, a modified U-Net was used in [27] to learn an end-to-end mapping between LR and HR images. Recently, to accurately improve the quality of CT images, Jiang et al. [28] designed a novel semi-supervised adversarial generative network, and proposed a new loss function to enforce the mappings between the discriminator and generator.

Many researchers worked on the noise removal of LDCT images and SR reconstruction of CT images, but most of these methods only focus on the single task of denoising or SR reconstruction, without considering the relationship between the two tasks. In particular, denoising task is necessary and beneficial for SR task [29]. To reduce CT scan radiation while maintaining the quality of the CT image, the generative adversarial network was used as a building block to establish a nonlinear end-to-end mapping from the noisy LR input to the denoised HR output [30]. Later, Chi et al. [31] first utilized a dense-inception network integrating the dense skip connection and the inception structure to estimate the noise level, followed by a modified residual-dense network to reconstruct HR images. Recently, to improve the quality of LDCT images, Yim et al. [32] linearly combined the denoising autoencoder and the SR convolutional neural network, where two networks were trained separately for the denoising and the SR. However, the methods mentioned above have the following limitations: (1) The cascaded models [32] that denoising the image first and then applying an SR algorithm have a drawback: along with noise, the denoising step always loses some of the high-frequency content of the image [29]. (2) The methods [30], [31] ignore the potential relationship between the denoising and the SR tasks. They use a single branch network to achieve denoising and SR tasks, which limits the quality of image reconstruction [33].

To deal with these drawbacks, we explore how to improve the interaction of denoising and SR tasks on the SR reconstruction of LDCT images. First, without noise removal, super-resolving the noisy input directly will magnify the undesired noise [33]. In order to remove the noise, the simple solution is to perform denoising first, followed by resolution improvement. But in this way, denoising is a pre-processing procedure that frequently results in the loss of high-frequency material, impairing subsequent SR performance [33]. For this reason, we achieve the interaction between the denoising and the SR by processing the image denoising and the SR reconstruction parallelly. In summary, our network consists of two parallel network branches, one for denoising and the other for SR reconstruction. The denoising branch will provide additional details to guide the SR reconstruction of the image. Extensive experiments demonstrate that our network can retain a large amount of detailed information of the image and obtain satisfactory image reconstruction results.

In summary, the main contributions of this article are as follows:

• We propose an Encoder-Decoder network for joint learning of image SR and denoising to achieve SR reconstruction of LDCT images. The suggested model learns the potential connection between the dual tasks through the interaction of the denoising task and the SR task. It provides a novel framework for SR reconstruction of noisy images.

• We design an filter gate module for controlling the interaction of denoising and SR reconstruction features. Rather than directly passing denoised features to the SR task, this module filters features from the denoising branch. It can learn implicitly to suppress irrelevant features from the denoising branch and highlight important features useful for the SR branch.

• Our joint learning network effectively solves the reconstruction of image details on the SR reconstruction of noisy images. Experiments show that our network achieves competitive results not only in LDCT images but also in sparse-view CT images.

The structure of this paper is as follows. We introduce the proposed dual-channel learning framework and the optimization settings of model in Section 2. Section 3 describes the datasets of experiments and training details of the model. The results of qualitative and quantitative comparisons with other methods are also discussed in this section. Finally, the conclusion and future works are presented in Section 4.