Understanding deep learning — challenges and prospects

The developments in Artificial Intelligence have been on the rise since its advent. The advancements in this field have been the innovative research area across a wide range of industries, making its incorporation in dentistry inevitable. Artificial Intelligence techniques are making serious progress in the diagnostic and treatment planning aspects of dental clinical practice. This will ultimately help in the elimination of subjectivity and human error that are often part of radiographic interpretations, and will improve the overall efficiency of the process. The various types of Artificial Intelligence algorithms that exist today make the understanding of their application quite complex. The current narrative review was planned to make comprehension of Artificial Intelligence algorithms relatively straightforward. The focus was planned to be kept on the current developments and prospects of Artificial Intelligence in dentistry, especially Deep Learning and Convolutional Neural Networks in diagnostic imaging. The narrative review may facilitate the interpretation of seemingly perplexing research published widely in dental journals.


Introduction
Artificial Intelligence (AI) deals with the science of simulating human behaviour in machines. 1The exponential development in technology over the past few decades has led to the evolution of AI, making it an innovative research area across a wide array of industries. 2 The two subsets of AI are Machine Learning (ML) and Deep Learning (DL) (Figure -1). 3 ML is a subdomain of AI, and DL is a further subdivision of ML.These techniques can be used to develop AI architectures and subsequently trained on an abundant set of existing data. 4The consequence is an architecture capable of making accurate predictions on new, unseen data. 4Both techniques are widely used today, revolutionising the AI industry.
AI originated as a single-layered neural network called 'Perceptron', laying the groundwork for the progression of DL techniques. 5However, it could not experience the growth that was anticipated due to challenging technological barriers that existed at the time of its incipience. 6These included a lack of computing power and the unavailability of extensive annotated datasets for training these architectures. 6e work in this field continued to some extent under different aliases until 2006 when AI experienced a paradigm shift, giving precedence to 'big data' in developing AI architectures. 7This was after the ImageNet database worked for the advancement of computer vision techniques. 7,8ImageNet is a collection of over a million annotated images accumulated from the internet and has been leveraged into training various algorithms since its commencement. 7,8This instigated a rampant growth that both ML and DL experienced for the first time since their inception. 3he very premise of AI is learn from the data that it collects; the more data that is collected and analysed, the more accurate the algorithm becomes at making predictions. 9Big data has been the driver for AI advancements, but no amount of data will ever replace human intuition.
A detailed discussion on ML can be found in literature. 3he current narrative review planned to keep its focus on DL techniques.

Deep Learning techniques
DL comprises multi-layered artificial neural networks (ANNs), modelled after the rudimentary picture of a human brain.The basic structure of an ANN consists of an input layer, one or more hidden layer(s) and an output layer, with each layer composed of nodes, also known as artificial neurons (Figure -2). 5 The 'deeper' the DL algorithm, the more hidden layers it contains, allowing for more 'features' to be extracted from a given dataset.Every layer is highly interconnected with the others; and the passing of data from one layer to the next is a process converting the output of one layer into input for the next. 5The algorithm learns the 'maps' between the given inputs and outputs and with sufficient training, it is able to apply those learned patterns on unseen input data to derive accurate output predictions. 1 DL algorithms can process large amounts of data and its performance improves as it analyses more data. 9The 'feature extraction' (Table ) step is already incorporated into DL algorithms, eliminating the need for manually determining relevant features for it to extract and learn, which is a prerequisite for ML techniques. 9After an appropriate level of training, the DL algorithm can extract hidden features autonomously with little to no explicit programming. 9

Training a DL algorithm
The process of developing an AI algorithm involves three basic steps: training, validating and testing (Figure -3).Training and validation employ the same annotated dataset, whereas testing is conducted on a different, unseen, annotated dataset. 9By running the training data through an algorithm, it is trained to produce a particular prediction with each epoch.Usually, the data is fed through it in iterations.These iterations are run through the algorithm in distinct patterns, allowing it to learn every possible aspect of the 'features' embodied in the training dataset. 9This leads to better 'generalisation' of the important features contained within the dataset, enabling its applicability on a wide range of unseen data in the future. 9With each epoch, the parameters and hyperparameters of the algorithm are adjusted to ensure greater accuracy of the predictions.A parameter is a variable that is internal to the algorithm and the value is determined by the algorithm itself. 9Whereas a hyperparameter is a value external to the algorithm set manually by the data scientist operating it, and can be tuned manually to further increase accuracy. 9After the efficacy of the algorithm has been validated on the same dataset, testing data is run through it. 9The results make the adequacy of training evident via the accuracy of predictions made by the algorithm. 9This process can be repeated and the performance can be improved 7th AKU Annual Surgical Conference Table : Terminologies and definitions.

Terminologies Definitions
Computer Vision A field in computer science enabling the simulation of the human visual cortex in machines. 9

Feature
Variables that are relevant to the training of an algorithm. 9

Dataset
It is a set of data, for example images, that contain important features for training an algorithm. 9

Annotation
Labelling the relevant features in a machine-learnable format. 9Epoch Each complete cycle of running the dataset through an algorithm. 9

Iteration
The number of batches required to complete one 'epoch'. 9eature Extraction A pre-processing step that reduces data into more manageable variables for the algorithm to learn. 9

Ground Truth
The ideal expected output determined by the subject specialist/ annotator. 9 further by running more annotated data until the desired accuracy level is

Convolutional Neural Network (CNN)
The most established algorithm among DL architectures is the Convolutional Neural Network (CNN). 2 It has been in use predominantly for computer vision tasks.CNNs explicitly process data with complex patterns, such as images, with the goal to mimic functions of the human visual cortex. 2,10'Image processing' is the specific feature extraction task performed by these neural networks.This is executed by the special convolutional layer that is incorporated into an otherwise traditional DL algorithm (Figure -4). 2,10The role of CNNs is to reduce the image according to pixels, making the image easier to process without losing important features. 2ence, these neural networks can make accurate predictions whilst simultaneously allowing for a reduction in the computational power required to effectuate these tasks. 9hese are being trained widely to be applied on radiographic images in the fields of medicine and dentistry. 1

Evaluating performance of DL algorithms
Different metrics are utilised in DL to discern the performance of algorithms assessed against the ground truth determined by subject specialists/ annotators in the 'testing' dataset. 1 The application of various metrics is imperative to ensure the optimal quality of the algorithm. 1ese performance metrics are run on the testing dataset, giving true positives and negatives, as well as false positives (FP) and false negatives (FN) of the predictions generated by the architecture. 1The values are consequently used to determine the precision, accuracy, recall/sensitivity, and F1-score of the trained algorithms measuring their true capabilities. 11Other metrics include the receiver operating curve (ROC), area under the curve (AUC) and dice index to evaluate the performance of architectures. 1,11The pertinent selection of relevant metrics for evaluation of the various AI architectures is a crucial task.It is dependent upon the specific task being carried out by the algorithm as well the aspect of its training that requires evaluation.For example, F1 score is used to determine the robustness of an algorithm. 1This aspect of algorithm training requires the proficiency of data scientists 11 and a thorough discussion on this aspect of algorithm training is far beyond the scope of the current paper.

CNNs in dentistry
The evolution of AI in medicine and dentistry have been evident in the studies published in the past few years. 1 AI has mostly been involved in the diagnostic aspects of these fields, approaching the level of expertise of the clinicians. 10 dentistry, CNNs can detect and segment anatomical structures and S-61  conditions on radiographic images. 1 makes it easy for clinicians to distinguish the positions and characteristics of teeth and any associated pathology as part of their preliminary clinical examination. 2In general, CNNs have performed well in recognising and classifying teeth on both twodimensional (2D) and 3D radiographs. 1 They are being studied as adjuncts in the identification of dental pathology, including caries, periapical lesions, periodontal bone loss, vertical root fractures, osteoporosis, cancers of the head and neck region, as well as for working length determination in endodontics, etc. 6,[12][13][14][15][16][17][18] Theoretically, adequately-trained CNN architectures can perform as well as clinicians, with the potential to surpass the level of their performance and diagnostic abilities.The aim is to simplify the provision of dental care, make it cost-effective, and concurrently eliminate human error in diagnoses and treatment plans devised by clinicians. 2

Challenges and Prospects
The applications of AI in dentistry is lagging behind by a few years compared to that of medicine. 12This is largely due to the cost of acquisition and annotation of big data required for sufficient training of AI algorithms, which is needed to ensure robustness of the architecture. 12e ability of the algorithm to make predictions is where the 'generalisability' of an algorithm is important to consider. 1 The key to improving the generalisability of an algorithm is determined by the extensiveness of the dataset used to train the model.3 'Overfitting' of data is one of the main challenges in the applicability of AI, as an overfitted algorithm is not generalisable on unseen data.19 A method to reduce overfitting is via data augmentation, which includes flipping, translation, cropping, rotating and applying filters on the existing images in a dataset.8 This ensures the training of the algorithm on every possible presentation of the training data.8 An abundance of labelled data is desirable but rarely available in medical and dental imaging, and, hence, another method called 'transfer learning' can be applied.20 Transfer learning is a technique where a neural network that is pre-trained to predict a 'general feature', like a car, in an image is then utilised on a different dataset with seemingly incompatible features, like a tooth, for the architecture to recognise after minimal preliminary training on the relevant features. Employing this technique, however, significantly reduces training time and computational resources while producing quality architectures.20 The lack of open-access, standardised and annotated datasets to train AI algorithms is the main impediment in the exponential growth of AI in dentistry.12 The annotation of medical images adds to the cost of time as well as the cost of subject specialists needed for accurate annotation of datasets.The provision of standard quality datasets for training algorithms will allow for their consistent clinical implementation.The limitations of computing power has largely been overcome, but can be improved quantum computation which is faster and makes it a valuable platform that should be explored to cause further advancement in AI. 12 There is a need for better understanding of the processes involved in the AI as well as overcoming its vulnerabilities to allow for its applicability in crucial, patient-related diagnostic problems.12 The ultimate goal is to train a machine to think like humans and perform tasks with more accuracy and greater speed, lowering resource utilisation in the long run.

Conclusion
AI has several potential applications in the fields of medicine and dentistry and there is a need for clinicians to be cognizant of all the past and current advancements.Being aware of the key concepts, workings, strengths as well as limitations of AI in clinical practice is crucial for clinicians to work towards further improving diagnoses and patient care.

Figure- 3 :
Figure-3: The three basic steps of developing an Artificial Intelligence (AI) algorithm.
The various computer vision tasks performed by CNNs include Object Classification and Localisation which entailes assigning labels to a single object in an image and localising their positions via a bounding box (Figure-5A); Object Detection which includes assigning labels to every object in an image belonging to a particular group via bounding boxes (Figure-5B); Semantic Segmentation in which individual pixels are labelled into groups via fluid margins (Figure-5C); and Instance Segmentation which employs Object Detection as well as Semantic Segmentation, labelling each pixel in every group via fluid margins (Figure-5D). 9