Automatic Blob Detection for Dental Caries

Abstract

Dental Caries are one of the most prevalent chronic diseases around the globe. Detecting carious lesions is a challenging task. Conventional computer aided diagnosis and detection methods in the past have heavily relied on the visual inspection of teeth. These methods are only effective on large and clearly visible caries on affected teeth. Conventional methods have been limited in performance due to the complex visual characteristics of dental caries images, which consist of hidden or inaccessible lesions. The early detection of dental caries is an important determinant for treatment and benefits much from the introduction of new tools, such as dental radiography. In this paper, we propose a deep learning-based technique for dental caries detection namely: blob detection. The proposed technique automatically detects hidden and inaccessible dental caries lesions in bitewing radio-graphs. The approach employs data augmentation to increase the number of images in the data set to have a total of 11,114 dental images. Image pre-processing on the data set was through the use of Gaussian blur filters. Image segmentation was handled through thresholding, erosion and dilation morphology, while image boundary detection was achieved through active contours method. Furthermore, the deep learning based network through the sequential model in Keras extracts features from the images through blob detection. Finally, a convexity threshold value of $0.9$ is introduced to aid in the classification of caries as either present or not present. The process of detection and classifying dental caries achieved the results of $97\%$ and $96\%$ for the precision and recall values, respectively.

1. Introduction

Caries is the most widespread chronic disease in the world. While there has been a decline recently in rates of large cavity lesions, there are still early lesions that can be identified in most people [1]. Most conventional caries detection methods rely on inspecting teeth visually, and this is effective for large and clearly visible carious lesions as well as those that are partially visible but can be viewed by a handheld mirror [2]. The introduction of dental radiography is to detect hidden or inaccessible lesions that could not be seen through conventional methods. The early detection of dental caries lesions is an important determinant treatment measure and, therefore, a beneficiary of the introduction of new tools [3].

The fastest growing sector in the health care industry is dental services. This involves the prevention, treatment and diagnosis of oral cavity diseases [4]. Most dentists use bite-wing radiographs to aid in finding the location of dental caries. They rely on information from radiographs together with their patient’s medical history. Locating dental caries is a challenging task, and sometimes even experienced dentists miss carious lesions when presented with bite-wing radiographs [5]. Traditionally, the detection of dental caries has relied on visual-tactile methods [6]. The sensitivity of visual-tactile methods is limited especially when performed on posterior proximal tooth surfaces. Radiographic methods tend to have high sensitivity but require ionizing radiation [7].

Most of the people who are at risk of dental caries are low-income minorities, socially and economically less privileged, and uneducated people. Furthermore, clinical studies have shown that dental caries are caused or related to dietal changes and risk factors for cardiovascular disease [8]. Some retention and restoration methods have been introduced, proposed and improved to treat dental caries in the past. However, the diagnosis of dental caries has been difficult due to the various anatomical morphologies and different shapes of teeth restorations.

In some teeth, it is difficult to detect caries at an early stage. Many of the carious lesions are detected at an advanced stage. Dental radiography is regarded as the most reliable diagnostic tool for detecting dental caries; however, most of its screening relies on empirical evidence results. The introduction of convolution neural networks in medical imaging analysis has shown excellent performance results. Medical imaging analysis has led to medical segmentation and diagnosis as one of the most important fields for pattern recognition and image processing.

Some deep learning CNN models are used for detection and classification of skin cancer, knee cartilage, diabetic retinopathy, pulmonary tuberculosis and brain tumours among others. These models have shown very high performance accuracy, efficiency and promising clinical applications in various fields. However, there been very few studies that investigate detection and diagnosis in dentistry based on deep CNN model architectures.

Furthermore, the research on detecting and diagnosis of dental caries is even more limited [9]. Accordingly, our approach presents a technique that evaluates the efficacy of deep learning methods for the detection of dental caries in bite-wing radiographs. We introduce the use of blob detection on bite-wing radiographs to detect dental caries. Blob detection is a mathematical method that detects specific regions in digital or radiographic images. Blobs are regions with notable differences with those in their neighbourhoods.

Blobs are also referred to as regions that are either brighter or darker than their neighbourhood. Blob detectors can be classified as differential methods that are derivative functions based on their position. They can also be classified as the local maxima and minima of the differential method’s derivative function. Blobs provide more information about various regions of interest for use for further image processing. With increased interest in medical imaging processing [10] presented a survey of several blob detection methods. We used blob detection to automate dental caries detection, and this will further increase the efficiency and reliability in medical care standards. Details of our approach will be discussed in depth in one section of this research.

Haghanifar et al. [11] presented an automatic diagnosis system for detecting dental cavities in panoramic images. This is done via transfer learning of pre-trained deep learning models to extract relevant features from x-rays and also use a capsule network to draw prediction results. Duong et al. [12] proposed a computational algorithm that automates the recognition of carious lesions on occlusal tooth surfaces in smartphone images through the international caries detection and assessment system (ICDAS).

Jusman et al. [13] presented a study that analyses feature extraction performance of dental caries images using the Gray Level Co-occurrence Matrix (GLCM) algorithm. This method is used to determine pixel and quantization values of the GLCM that is used for an automated classification system for dental caries types. Prados-Privado et al. [14] proposed data extraction on dental images using two reviewers. Most of the included studies employed periapical and bitewing radiography. Image databases ranged from 87–3000 images, with a mean of 669 images. Each study included used a different neural network and different outcome metrics.

Paqué et al. [15] investigated the potential of salivary bacterial and protein markers for evaluating the disease status in healthy individuals with caries. Saliva samples from caries and gingivitis-free individuals (n = 18), patients with deep caries lesions (n = 38) was collected and analysed for 44 candidate biomarkers namely: selected oral bacteria, growth factors, chemokines and proteolytic enzymes among others. Research by Lee et al. [16] presented a CNN model using a U-shaped deep CNN (U-Net) for dental caries detection on bitewing radiographs and how the model can improve clinician performance.

2. Methods and Techniques

The proposed approach is broken down as shown in Figure 1 and Algorithm 1, and discussed in the subsections that follow.

2.1. Pre-Processing of Dental Images

The input images used by this system were 120 dental radiographs, which were then augmented to produce 11,114 images sufficient for the deep learning process. The augmentation process involved the rotation, scaling and resizing of the images. Additionally, these images were pre-processed, which included contrast and brightness adjustment and scaling to compensate for the non-uniformity from image illumination. These sub processes were performed via pre-processing techniques that included gray image scaling and blurring as seen in Figure 2.

2.2. Segmentation of Images

Segmentation involves image thresholding to convert blurred source images to several binary images and masks. Thereafter, erosion and dilation morphology was used to diminish foreground features while accentuating background features, respectively, as shown in Figure 2. Additionally, the eight-way kernel was used to combine darker or brighter pixel regions on the image into a final caries lesion mask. The detection of these darker or brighter regions was influenced by the (LoG) Laplacian of Gaussian approach, which is a convolution kernel of the form:

$$LoG=\frac{x^2+y^2-2{\sigma }^2}{{\sigma }^4}{e}^{-\frac{x^2=y^2}{2{\sigma }^2}}$$

(1)

where $\sigma$ is the width of the kernel. The masking procedure increases segmentation accuracy. Further, image edge segmentation is handled via active contours method to detect tooth edge boundaries. This aids easier isolation of the caries lesion region from the surrounding normal skin. From the segmented caries lesion, there still exists other blobs that are not carious. To overcome this issue, we found only the biggest blobs in the segmented image, and these are dental caries, as in Figure 3.

2.3. Feature Extraction

2.3.1. Noise Reduction

Before the process of feature extraction starts, it is important to remove noise and retain edge information on images. This post-processing step is achieved by the use of edge-preventing smoothing Gaussian filters [17], that has a Gaussian distribution with mean $\mu$ and standard deviation $\sigma$. The Gaussian filter values are calculated by averaging centre’s of neighbourhood pixels with a certain range $[\mu -2\sigma ,\mu +2\sigma ]$. The M number of the centre’s neighbour pixels within the given range is then calculated.

The centre pixel’s intensity value is replaced by the calculated filter values when the number M of the centre’s neighbour pixels within the range is greater than a specific threshold K, or the average value of the neighbour pixel when the above condition is not met. Some edge information is removed if a high value of threshold K value is used. In our experiments, K the value was set at 3 with a $3\times 3$ kernel to remove unwanted noise in our masked images before extracting the needed features.

2.3.2. Caries Candidate Detection

Geometric features are the most important extraction features to look for when detecting caries on dental radiographs. Geometric features on images are enhanced through the adoption of Hessian analysis [18]. Hessian analysis technique computes the second derivatives along 3D directions by convoluting the given image with derivatives of the Gaussian kernel [19]. The Hessian analysis matrix is defined as:

$$H_{\sigma }(x,y,z)=\left(\begin{array}{ccc}{I}_{xx}& I_{xy}& I_{xz}\\ I_{yx}& I_{yy}& I_{yz}\\ I_{zx}& I_{zy}& I_{zz}\end{array}\right)$$

(2)

where $\sigma$ is the standard deviation of the Gaussian distribution and $I_{ij}$ is the second deviation along $i$th and $j$th direction. The standard deviation $\sigma$ could be used to control the radius of blob-like structures. Eigen values that indicate three orthogonal directions, and Eigen vectors that indicate degrees of curvature along the given directions can be used to solve the hessian matrix at each pixel. Therefore, the likelihood of blobs together with the magnitude of pixel’s Eigen values can be formulated as:

$$R_B=\frac{|{\lambda }_1|}{\sqrt{|{\lambda }_2\left|\right|{\lambda }_3|}}$$

(3)

$$M=\sqrt{{\sum }_{n=1}^3{{\lambda }^2}_n}$$

(4)

where $R_B$ and M represent the likelihood of blobs and the magnitude of pixel’s Eigen values, respectively. Generally, the magnitude of Eigen values of objects are larger than that of the background. Blobs can be enhanced further using the formula:

$$B_{\sigma }\left({\lambda }_p\right)={\{}_{0,otherwise}^{(1-e-\frac{R_B}{2{\alpha }^2}),if{\lambda }_1>0,{\lambda }_2>0,and{\lambda }_3>0}\}$$

(5)

where ${\lambda }_p$ indicates Eigen values at position $p=(x,y,z)$ and $\alpha$ and $\beta$ refer to sensitivity parameters of $R_B$ and, M, respectively. In our approach, the sensitivity parameter was set at 0.3 to control the number of blobs detected in the darker regions. The condition ${\lambda }_1>0,{\lambda }_2>0,and{\lambda }_3>0$ is used to enhance only dark objects. Furthermore, carious lesions of different sizes can be detected, by adjusting the formula above as:

$$Blobness\left({\lambda }_p\right)=maxB\sigma {\left({\lambda }_p\right)}_{{\sigma }_{min}\le \left(\sigma \right)\le {\sigma }_{max}}$$

(6)

where ${\sigma }_{min}$ is the minimum scale and ${\sigma }_{max}$ the maximum scale of carious lesions.

2.3.3. Caries Candidate Selection

From setting the blobness sensitivity parameters, many redundant non-caries candidates and actual caries candidates were detected. If all the caries candidates were used to extract features and estimate the likelihood of caries, processing would take a long time. Additionally, a long processing time leads to poor performance of the CAD system, and this is because of the high frequency of false positives (FPs). Therefore, caries candidate selection is adopted to eliminate FPs from carious candidates. High blobness values indicate the candidate has caries, while those with lower blobness values are non-caries candidates. Therefore, maximum and mean blobness values of caries candidates were used to eliminate false positives. The formulas are defined as:

$$Blobnes{s}_{mean}\left(T\right)=\frac{{\Sigma }_{p\epsilon T}Blobness\left({\lambda }_p\right)}{N_T}$$

(7)

$$Blobnes{s}_{max}\left(T\right)=ma{x}_{p\epsilon T}Blobness\left({\lambda }_p\right)$$

(8)

where T refers to a caries candidate with $N_T$ voxels and p a voxel that belongs to the caries candidate. The size of blobs was also used as a feature in selecting caries candidates. Size feature is used because the caries candidate detection based on Hessian analysis is sensitive to intensity variations and leads it to detect many small regions including edge spaces of teeth as caries candidates. These edge spaces and small regions are not part of true caries lesions and their size can be used to eliminate many false positives from among the caries candidates as seen in Figure 3. The linear regression model [20] is applied to eliminate false positives from among caries candidates. The caries selection function $L_s\left(T\right)$ is defined as:

$$Z\left(T\right)={\beta }_0+\sum _{i=1}^{N_f}{\beta }_i{x}_i$$

(9)

$$L_s\left(T\right)=\frac{1}{1+e^{-Z\left(T\right)}}$$

(10)

where $N_f$ is the number of features, $x_i$ the feature value, ${\beta }_0$ is the constant coefficient, and ${\beta }_i$ is the corresponding coefficient estimated by the linear regression model. Lastly, the criteria used for caries selection is expressed as:

$$L_s\left(T\right)\ge THresh$$

(11)

where $THresh$ is the threshold used to determine the remaining caries candidates after eliminating false positives. The detected regions remain as dental caries when values estimated by the caries selection model was equal or greater than threshold. Those values less than the threshold given were removed. In our experiments, the convexity threshold value was set at 0.9 to detect dental caries. After the caries candidate selection, false positives were eliminated, and  only carious candidates remain, which are now classified as “Dental Caries Detected”.

Algorithm 1 Proposed Model for Dental Image Segmentation.

1: procedureEncoder(Y)              ▹$Y:y_i$ is an input image with dimension (L, H). 2:     Initialization: ${\mathrm{Feature}}_i=y_0$; 3:     for ($i=0:N-1)$do 4:         ${\mathrm{AugmentedFeature}}_i=\mathrm{DataAugmentation}\left({\mathrm{Feature}}_i\right)$; 5:         ${\mathrm{GrayScalingFeature}}_i=\mathrm{GrayScaling}\left({\mathrm{AugmentedFeature}}_i\right)$; 6:         ${\mathrm{FilteringFeature}}_i=\mathrm{GuassianBlurFiltering}\left({\mathrm{GrayScalingFeature}}_i\right)$; 7:     end for 8: end procedure 9: procedureProcedure Enhancement(FilteringFeature ${}_i$)                 ▹ It is the output of the preprocessing procedure 10:     for ($j=0:M-1)$ do 11:         ${\mathrm{ErisonFeature}}_i=\mathrm{ErisonMorophology}\left({\mathrm{FilteringFeature}}_i\right)$; 12:         ${\mathrm{DialtionFeature}}_i=\mathrm{DialtionMoropology}\left({\mathrm{ErisonFeature}}_i\right)$; 13:     end for 14: end procedure 15: procedureProcedure Segmentation(DialtionFeature ${}_i$)                    ▹ It is the output of the enhancement procedure 16:     for ($s=0:L-1)$ do 17:         ${\mathrm{ThresholdingFeature}}_i=\mathrm{Thresholding}\left({\mathrm{DialtionFeature}}_i\right)$; 18:         ${\mathrm{ConnectedFeature}}_i=\mathrm{ConnectedComponentFeature}\left({\mathrm{ThresholdingFeature}}_i\right)$; 19:     end for 20: end procedure 21: procedureProcedure FeatureExtraction(ConnectedFeature ${}_i$)              ▹ It is the output of the segmentation procedure 22:     for ($r=0:K-1)$ do 23:         ${\mathrm{Feature}}_i=\mathrm{FeatureExtraction}\left({\mathrm{ConnectedFeature}}_i\right)$; 24:         Segmented Image Feature= ${\mathrm{Feature}}_i$;         ▹ Final Segmented Output Display. 25:     end for 26: end procedure

3. Results and Discussion

Experimental results from the proposed approach are based on the analysis of the caries candidate detection process. Experiments were carried out on 11,114 bitewing X-ray images of the augmented dental dataset. The dataset was split into 10,090 training set images and 1024 test images. In order to process the data set, several individual stages were used to analyse the dataset, as have been discussed in the previous sections of this work. Figure 1 shows an overview of the processing stages that include: preprocessing, segmentation and feature extraction.

Additionally, performance evaluation of the proposed approach was based on its ability to detect and locate dental caries on test data images. A Gaussian filter was used to reduce noise on the images before introducing a convexity threshold value to eliminate false positives regions from actual detected caries candidates. Figure 4 shows the caries detection algorithm’s identification of caries locations on Gaussian filtered images before false positives are eliminated. Figure 5, shows the identification of dental caries locations remaining after introducing a convexity threshold value of 1.0 that eliminates false positives.

The results comparison was performed against different caries detection diagnostic methods to determine if the results fell within acceptable limits or not. The caries detection diagnostic methods compared against include textural classification as discussed by [21], dental classification for periapical radiographs by [22] and caries detection through a supervised learning model proposed by [23] for panoramic images. The comparison of our proposed method with other detection techniques is shown in

Table 1. Comparison of various caries detection methods.

Detection Methods	Dataset Images	Accuracy
Caries detection in panoramic images [23]	1392	$98.0\%$
Dental classification for periapical images [22]	78	$82.5\%$
Textural classification of digital images [21]	64	$96.88\%$
PaxNet [11]	470	$86.05\%$
ICDAS [12]	620	$92.37\%$
GLCM [13]	240	$80\%$
Deep learning [16]	304	$64.14\%$
Proposed approach	11,114	$\mathbf{97}.\mathbf{0}\%$

.

Our proposed method achieved better accuracy than the others with the introduction of blob detection on dental radiographs. Despite our results falling within the acceptable range of a novel approach, [23] enjoyed marginally higher results, and this is attributed to various reasons. Some reasons include differences in the database analysed, where our approach used bitewing radiographs while [23] used panoramic radiographs. Another reason is attributed to the errors experienced when analysing bitewing radiographs, namely exposure errors and image augmentation anomalies.

Limitations

Most of the existing systems dwell on the segmentation of caries and not on caries detection. The reluctance from healthcare experts to invest in computer aided systems (CAD) systems is due to the high number of false positive results. There is a need to improve existing systems, and one way to do so is by the introduction of automatic blob detection technique. Blob detection has been used in other fields of medical imaging but has not seen substantive use in the field of dental imaging.

4. Conclusions

There is a huge potential for use of dental radiography and, in particular, for work focused on caries detection. Blob detection was used in this research to detect all caries candidates, which also included false positives. The use of image analysis techniques was encouraged to filter out detected unwanted regions, such as edges from the caries regions. A threshold property value was then introduced to aid in the elimination of false positives from the selected regions to leave the detected caries candidates. The proposed caries detection method was able to administer favourable results compared to other detection models with an accuracy of $97\%$.

The introduction of blob detection offers a more robust and automatic way to detect caries on dental radiographs compared to other detection methods. The proposed approach presents an avenue for future research using unsupervised models from its impressive success rate with supervised models. More dental radiographic images are needed to improve the training and learning process to produce even better results. Figure 6 shows both the training and validation accuracy graphs of the proposed method. Weight regularization methods are also encouraged to reduce overfitting.

5. Future Works

From our survey, various techniques, methods and approaches were discussed that point towards the segmentation and detection of dental images. Various works that stem from the industry and academia have been mentioned and discussed regarding existing algorithms, segmentation and detection methods and also various protocols for evaluating performance. There are issues and encouraging future perspectives of study that have resulted from our discussions, which are highlighted as:

• Data availability and reliability. Deep learning networks require large amount of data to achieve meaningful and effective performance results. Due to the nature of dental images, there is need for hybrid datasets to aid good performance of the networks. There is need of public available datasets for dental images to make deep learning in the field possible. Data augmentation is also a good method to obtain more images for dental datasets.
• Data standardization. Many methods discussed are handling the pre-processing step through manual methods, such as cropping the region of interest on an image. These methods contribute to the loss of some key details from the images. Some networks end up dividing a whole image into subregions, and this slows down the learning process of one subregion after the other. There are methods like down-sampling, which might lead to the deletion of important details and this seem to have been due to limitations in computational power. Deep learning approaches have seen increased learning on whole images rather than manual manipulation of images at the pre-processing stage, in order to obtain more general and accurate results.
• Weight Regularization Methods. Deep learning networks can also be improved by introducing weight regularization to improve their performance. The regularization of weights involves optimization of model hyper-parameters, such as the learning rate and the dropout rate. Basically, weight regularization methods are introduced into networks for parameter optimization.
• Hybrid approaches. Deep networks can also be enhanced by combining several models or methods to form hybrid networks that will improve overall evaluation performance. The combination can be in any stage of the model, for instance combining two pre-processing techniques to come with a single one to enhance image quality. This combination can also be handled by joining various attributes of different models to form one hybrid model that will enhance the training, extraction, detection and classification of objects.