Transfer Learning Based Automatic Human Identification using Dental Traits- An Aid to Forensic Odontology

doi:10.1016/j.jflm.2020.102066

Journal of Forensic and Legal Medicine

Volume 76, November 2020, 102066

https://doi.org/10.1016/j.jflm.2020.102066 Get rights and content

Highlights

•
A Novel method of Human Identification by using Deep Learning approach is proposed.
•
Transfer learning using a deep CNN architecture namely AlexNet is employed in three stages for automatic feature extraction, tooth classification and numbering.
•
Data Augmentation was performed in each stage, to provide necessary data to train the network.
•
The proposed methodology is found to have higher accuracy than other state-of-the-art methods and also Hit rate is higher.

Abstract

Forensic Odontology deals with identifying humans based on their dental traits because of their robust nature. Classical methods of human identification require more manual effort and are difficult to use for large number of Images. A Novel way of automating the process of human identification by using deep learning approaches is proposed in this paper. Transfer learning using AlexNet is applied in three stages: In the first stage, the features of the query tooth image are extracted and its location is identified as either in the upper or lower Jaw. In the second stage of transfer learning, the tooth is then classified into any of the four classes namely Molar, Premolar, Canine or Incisor. In the last stage, the classified tooth is then numbered according to the universal numbering system and finally the candidate identification is made by using distance as metrics. These three stage transfer learning approach proposed in this work helps in reducing the search space in the process of candidate matching. Also, instead of making the network classify all the 32 teeth into 32 different classes, this approach reduces the number of classes assigned to the classification layer in each stage thereby increasing the performance of the network. This work outperforms the classical approaches in terms of both accuracy and precision. The hit rate in human identification is also higher compared to the other state-of-art methods.

Introduction

Identification of Humans can be done based on several anatomical and observable characteristics such as Face, Fingerprint, Iris, DNA, Signature, Voice, etc.¹ In case of disasters involving large number of victims these traits may be lost and hence cannot be used for identification. Around 46.2% victims, of the Indian Ocean Tsunami Disaster in Thailand, were identified based on Dental records, in 2005. The dental traits were mostly preferred because of its longevity, distinguishability and resistivity to heat. Images taken when the candidate is alive are called Ante-Mortem (AM) images and those of the deceased taken post disaster are called Post-Mortem (PM) Images. Forensic Odontologists are those, who manually compare and find a match between the PM and AM dental records to identify the victims.² Such a Manual Process of comparison is time-consuming and may lead to inaccuracies.

To avoid this, Automatic Dental based Human Identification is introduced. It is a three-step process which involves i) Image Enhancement and Teeth isolation ii) Teeth Classification and Numbering iii) Candidate Matching and Identification as shown in Fig. 1.

Broadly, dental radiographs can be classified into two types intra-oral and extra-oral. In which Bitewing and Periapical falls under the former and Panoramic falls under the latter type of radiographs, and is shown in Fig. 2.

For our study, panoramic type of radiograph which includes the entire mouth area is used. The works in the literature discussed in the section below uses several mathematical approaches for shape and feature extraction which is highly difficult to use for larger data.

Deep Learning uses dense neural network which enables it to work on large data and extract features from the images automatically with its layers in hierarchy, without need for manual feature extraction. The accuracy of various deep learning architectures has proven to outsmart the classical methods. Hence, a popular architecture of CNN namely AlexNet is used with Transfer Learning (TL) for this application of Human identification based on Dental radiographs.

The paper is organized as follows: Section 2 presents an overview of various methods available in the literature which involves human identification using dental radiographs. Section 3 illustrates the proposed methodology. The experimental results are evaluated in section 4. The results of the proposed methodology are compared with the state-of-art methods in section 5 and section 6 mentions a brief conclusion and future work.

Section snippets

Literature Survey

There are few works in the literature related to Human Identification based on various features extracted from Dental Images and those are discussed in this section. Jain et al.³ used intensity projection technique to partition the tooth separately. The shape of both the root and the crown was extracted which was then compared with the query image to find the best match. But it failed for occluded and blurred images. The teeth matching were done by calculating bidirectional Hausdorff distance

Proposed Methodology

In this work, a three stage transfer learning approach is proposed to identify human, based on features automatically extracted from panoramic type of radiographic images using AlexNet with TL. The First stage identifies whether the Query tooth Image or the PM image, belongs to the upper or the lower Jaw. In the second stage, the tooth is further classified into Molar, Premolar, Canine or Incisor for both the maxilla and mandible region. In the third stage the correct tooth number is identified

Results and Discussion

In this section, the results of the proposed algorithm for all the three stages are presented separately. The dataset used for this work was collected from PSG Institute of Medical Sciences & Research, Coimbatore (PSG-IMSR) and the Institutional Human Ethics Committee has approved this study.

Comparison with state–of–art methods

The performance of tooth classification using the proposed methodology is compared with few state-of-the-art methods in the literature. Oktay et al.²⁸ proposed tooth classification for panoramic images using CNN with a sliding window approach and classified the tooth into 3 categories. A certain level of preprocessing was implemented for the input data. Miki et al.²⁴ used AlexNet for classification of CBCT images and classified it into seven categories. Zhi Li et al.²⁹ employed seven layer DCNN

Conclusion

Identification of victim using dental characteristics in the field of forensic odontology is a still challenging task because of the complexity involved in it. A novel way of human identification using transfer learning was proposed to reduce the complexity. A three stage transfer learning approach was used i) to identify the query tooth location ii) to classify the tooth and iii) to number the tooth. Finally, the decision of human identification is made using distance metrics. To our

CRediT author contribution statement

Sathya B: Conceptualization, Methodology, Software, Writing - original draft, preparation. Neelaveni R: Visualization, Investigation, Supervision, Validation, Writing - review & editing.

References (30)

J.D. Zhou et al.
A content-based system for human identification based on bitewing dental X-ray images
Pattern Recogn
(2005)
O. Nomir et al.
A system for human identification from X-ray dental radiographs
Pattern Recogn
(August 2005)
M. Mahoor et al.
Classification and numbering of teeth in bitewing dental images
Pattern Recogn
(April 2005)
Z. Li et al.
Teeth category classification via seven-layer deep convolutional neural network with max pooling and global average pooling
Int J Imag Syst Technol
(2019)
A. Jain et al.
Biometrics-Personal Identification in Networked Society
(1999)
P. Stimson et al.
Forensic Dentistry
(1997)
K. Jain et al.
Matching of dental x-ray images for human identification
Pattern Recogn
(2004)
O. Nomir et al.
Hierarchical Dental X-Ray Radiographs Matching
(2006)
O. Nomir et al.
Fusion of matching algorithms for human identification using dental X-ray radiographs
IEEE Trans Inf Forensics Secur
(2008)
H. Chen et al.
Dental biometrics

G. Fahmy et al.

Towards an automated dental identification system (ADIS)

H.H. Ammar et al.

A neural network based system for dental radiograph comparison

E. Haj Said et al.

Teeth segmentation in digitized dental X-Ray films using mathematical morphology

IEEE Trans Inf Forensics Secur

(Jun. 2006)

D. Frejlichowski et al.

Application of the laplacian pyramid decomposition to the enhancement of digital dental radiographic images for the automatic person identification

Vijayakumari pushparaj et al.

Dental radiographs and photographs in human forensic identification

IET Biom

(2013)

Cited by (12)

A comprehensive survey of deep learning algorithms and applications in dental radiograph analysis
2023, Healthcare Analytics
The Integration of machine learning and traditional image processing in dentistry has resulted in many applications like automatic teeth identification and numbering, caries, anomaly, disease detection, and dental treatment prediction. They have a broad scope in different applications observed in the dentistry literature review. This study reviews the literature on deep learning and dental radiograph analysis. We present an overview of machine learning algorithms in different areas of dentistry: tooth identification and numbering, Dental disease detection, and dental predictive treatment models. The methods under each area are briefly discussed. The dental radiograph data set required for performing experiments is summarized from the available literature. The study concludes by discussing new research opportunities and initiatives in this field. This paper offers a comprehensive overview of this innovative, challenging, and growing area in dentistry.
Collaborative deep learning model for tooth segmentation and identification using panoramic radiographs
2022, Computers in Biology and Medicine
Citation Excerpt :
Moriyama et al. [50] increased accuracy by examining pocket locations using radiographs and blood test data in conjunction with concurrent training. Sathya et al. [45] examined dataset expansion, the use of contemporary CNNs, and advanced augmentation approaches. Krois et al. [35] proposed using a seven-layer deep CNN with global average and max pooling to categorize teeth into four categories: molar, premolar, canine, and incisor.
Panoramic radiographs are an integral part of effective dental treatment planning, supporting dentists in identifying impacted teeth, infections, malignancies, and other dental issues. However, screening for anomalies solely based on a dentist's assessment may result in diagnostic inconsistency, posing difficulties in developing a successful treatment plan. Recent advancements in deep learning-based segmentation and object detection algorithms have enabled the provision of predictable and practical identification to assist in the evaluation of a patient's mineralized oral health, enabling dentists to construct a more successful treatment plan. However, there has been a lack of efforts to develop collaborative models that enhance learning performance by leveraging individual models. The article describes a novel technique for enabling collaborative learning by incorporating tooth segmentation and identification models created independently from panoramic radiographs. This collaborative technique permits the aggregation of tooth segmentation and identification to produce enhanced results by recognizing and numbering existing teeth (up to 32 teeth). The experimental findings indicate that the proposed collaborative model is significantly more effective than individual learning models (e.g., 98.77% vs. 96% and 98.44% vs.91% for tooth segmentation and recognition, respectively). Additionally, our models outperform the state-of-the-art segmentation and identification research. We demonstrated the effectiveness of collaborative learning in detecting and segmenting teeth in a variety of complex situations, including healthy dentition, missing teeth, orthodontic treatment in progress, and dentition with dental implants.
A fine-grained network for human identification using panoramic dental images
2022, Patterns
Citation Excerpt :
Lai et al.6 presented a network that incorporated a channel attention module to selectively emphasize channel information. Sathya and Neelaveni7 used transfer learning and classified teeth into four classes, thereby increasing the performance of the model. Wu et al.8 proposed a multisupervision network with an attention-based mechanism to obtain an attention mask for human identification.
When accidents occur, panoramic dental images play a significant role in identifying unknown bodies. In recent years, deep neural networks have been applied to address this task. However, while tooth contours are significant in classical methods, few studies using deep learning methods devise an architecture specifically to introduce tooth contours into their models. Since fine-grained image identification aims to distinguish subordinate categories by specific parts, we devise a fine-grained human identification model that leverages the distribution of tooth masks to distinguish different individuals with local and subtle differences in their teeth. First, a bilateral branched architecture is designed, of which one branch was designed as the image feature extractor, while the other was the mask feature extractor. In this step, the mask feature interacts with the extracted image feature to perform elementwise reweighting. Additionally, an improved attention mechanism was used to make our model concentrate more on informative positions. Furthermore, we improved the ArcFace loss by adding a learnable parameter to increase the loss of those hard samples, thereby exploiting the potential of our loss function. Our model was tested on a large dataset consisting of 23,715 panoramic X-ray dental images with tooth masks from 10,113 patients, achieving an average rank-1 accuracy of 88.62% and rank-10 accuracy of 96.16%.
3D superimposition of human dentition contours in personal identification: A preliminary study
2024, Journal of Forensic Sciences
Deep learning IoT platform for dental disease detection
2023, Internet of Things in Modern Computing: Theory and Applications
Survey on the investigation of forensic crime scene evidence
2022, International Journal of Modeling, Simulation, and Scientific Computing

View all citing articles on Scopus

View full text

Transfer Learning Based Automatic Human Identification using Dental Traits- An Aid to Forensic Odontology

Highlights

Abstract

Introduction

Section snippets

Literature Survey

Proposed Methodology

Results and Discussion

Comparison with state–of–art methods

Conclusion

CRediT author contribution statement

Pattern Recogn

Pattern Recogn

Pattern Recogn

Int J Imag Syst Technol

Biometrics-Personal Identification in Networked Society

Forensic Dentistry

Matching of dental x-ray images for human identification

Pattern Recogn

Hierarchical Dental X-Ray Radiographs Matching

Fusion of matching algorithms for human identification using dental X-ray radiographs

IEEE Trans Inf Forensics Secur

Dental biometrics

Towards an automated dental identification system (ADIS)

A neural network based system for dental radiograph comparison

Teeth segmentation in digitized dental X-Ray films using mathematical morphology

IEEE Trans Inf Forensics Secur

Application of the laplacian pyramid decomposition to the enhancement of digital dental radiographic images for the automatic person identification

Dental radiographs and photographs in human forensic identification

IET Biom