Deep learning: definition and perspectives for thoracic imaging

Abstract

Relevance and penetration of machine learning in clinical practice is a recent phenomenon with multiple applications being currently under development. Deep learning—and especially convolutional neural networks (CNNs)—is a subset of machine learning, which has recently entered the field of thoracic imaging. The structure of neural networks, organized in multiple layers, allows them to address complex tasks. For several clinical situations, CNNs have demonstrated superior performance as compared with classical machine learning algorithms and in some cases achieved comparable or better performance than clinical experts. Chest radiography, a high-volume procedure, is a natural application domain because of the large amount of stored images and reports facilitating the training of deep learning algorithms. Several algorithms for automated reporting have been developed. The training of deep learning algorithm CT images is more complex due to the dimension, variability, and complexity of the 3D signal. The role of these methods is likely to increase in clinical practice as a complement of the radiologist’s expertise. The objective of this review is to provide definitions for understanding the methods and their potential applications for thoracic imaging.

Key Points

• Deep learning outperforms other machine learning techniques for number of tasks in radiology.

• Convolutional neural network is the most popular deep learning architecture in medical imaging.

• Numerous deep learning algorithms are being currently developed; some of them may become part of clinical routine in the near future.

Glossary

Algorithm

a mathematical model that is applied to data (input) and provide an output in order to solve a specific problem.

Artificial intelligence

a scientific domain that creates algorithms allowing machines to mimic human cognition or human performance on a dataset. Currently, AI algorithms address specific tasks such as tuberculosis diagnosis or pneumonia detection, which is defined as narrow or weak AI. They do not allow the detection of all potential abnormalities (e.g., general AI) as a human reader would do.

CAD (computer-aided diagnosis)

a domain that exploits algorithms derived from artificial intelligence to provide indicators and assist clinical experts in diagnosis.

CADe

CAD developed for a detection task.

CADx

CAD developed for a characterization task.

Classification task

assignment to an input signal (an image) a label from a predefined set of categories (disease or no disease), by mean of a machine learning algorithm.

Cross validation

a statistical method used to estimate the performance of the machine learning algorithm by exploiting various partitions of the data between training and testing.

Convolution

mathematical operator that creates a new value from an input signal (for instance a group of voxels) after modification by another value which acts as a filter. For example, averaging mean density values within a patch of voxels.

Convolutional neural network (CNN)

deep neural network which is based on a sequence of convolutional operations.

Deep learning (= deep neural network)

part of the broader family of machine learning, individualized by specific configuration of neural network organized in multiples layers, emulating the human learning approach and increasing the ability to address complex problems. Deep learning networks are iterative methods that propagate information, training their features automatically through gradient-based optimization methods and backpropagation.

Epoch

indicates the number of times the entire dataset has been used during the iterative optimization of the network.

Features

image characteristics which are invisible to the human eye. Three categories of features are used by classical machine learning algorithms: morphological features such as shape, volume, and diameter; first-order features such as histogram, kurtosis, and mean values; and textural features including co-occurrence of patterns and filter responses.

Formal neuron (= artificial neuron)

mathematical function mimicking the architecture of biological neurons.

Fully connected CNNs

variation of CNNs which consists of connecting all the elements of one layer with all the elements of the next one. Fully connected CNNs are used for classification problems (does this chest radiograph contain signs of tuberculosis?).

Fully convolutional CNNs

variation of CNNs which are composed from only convolutional layers. Fully convolutional CCNs are used for segmentation tasks (is this pixel located in a fibrotic area?).

Generalization capability

capacity for a model to maintain its performance when applied to new cases, unseen during training.

Generative adversarial neural network (GAN)

a neural network that combines two subnetworks, one generating hypotheses and another evaluating their likelihood.

Ground truth

refers to the label assigned by the expert or another reference method such as pathology.

Hyperparameters

parameters which control the training process of the algorithm and are defined before training, such as the number of layers and learning rate, among others.

Labeling/annotation

process of allocating ground truth by associating a label to an image.

Loss function

when training and optimizing the algorithm, it quantifies the gap between predictions and ground truth.

Machine learning

a scientific field that gives computers the ability to automatically learn without being explicitly programmed, by relying on sample data, known as “training data,” used to make predictions.

Neural network

machine learning algorithm made of a succession of formal neurons.

Overfitting

characterizes algorithms that perform well on the data on which they have been trained but fail to perform equally well on unseen data.

Radiomics

a field of medical imaging that aims to extract features from medical images, for tasks such as characterization or prediction (prognosis, response to treatment, genotype).

Recurrent neural networks (RNN)

a class of neural networks that integrate interdependencies between different tasks using the same data (detection and characterization) or between different data (temporal post-contrast enhancement).

Regression task

process of associating input data with a continuous outcome (for instance survival).

Semi-supervised learning

class of machine learning techniques that learns from annotated data in order to generate their model and improves its performance using the non-annotated ones

Supervised learning

class of machine learning techniques requiring labeled training data in order to generate their model.

Semantic segmentation

process of associating every voxel with a specific label/class, for instance diseased or healthy area, which usually requires manual contouring.

Stochastic gradient descent

an iterative method to optimize machine learning methods, very commonly used for deep learning networks.

Test dataset

dataset which is used to evaluate the performance of the final model.

Training dataset

dataset which is used to train the model.

Transfer learning

concept of exporting parameters, principles, and strategies learned from a dataset to another algorithm, which will be trained on another dataset (for example, learning on nonmedical images before applying to chest imaging).

Unsupervised learning

the class of machine learning techniques that seeks to determine patterns or clusters with similar properties (= phenotypes for instance) from unlabeled data. It usually uses techniques different from deep learning.

Underfitting

inability of an algorithm to perform well on both training and test datasets.

Validation dataset

dataset which is used to determine among different variants of the trained model, the optimal model that should be selected for testing on the remaining unseen cases (test dataset).