Linear measurements of facial morphology using automatic aproach

Abstract Introduction Clinical extraoral examination prior to orthodontic treatment includes face analysis (front and profile). Development of computer technology has increased efficacy and simplified this process through automating several steps of the analysis. The aim of this paper was to examine the possibility of automatic determining of linear measurements based on the facial image of a patient. Material and Methods Based on the set of 20 patients in NHP (Natural Head Position) position, three sets of measurements were conducted. Trained orthodontist performed positioning of predefined points on the image of the patient two times with one week apart, after which the points were automatically determined using customized computer software. Based on the position of the points, measurements for bizygomatic distance, upper and lower facial height and full facial height were computed. Three sets of measurements were compared and statistically analyzed. Results showed that computer software produced measurements comparable to measurements obtained by a trained orthodontist. Statistical analysis included calculating mean values and standard deviations, as well as paired two‑tailed T‑test. Differences between measurements ranged from 0.03% to 0.6% suggesting that automatic method can be successfully used. Conclusions The results of this research suggest that it is possible to ease, accelerate and automate work of the orthodontist on the image analysis using suitable software without significant differences in measured values.


INTRODUCTION
Human face analysis represents both art and science. In order to assess the characteristics of human face, various measurements are used, such as: anthropometry, cephalometry and photogrammetry. Application of photogrammetry in orthodontics was first suggested by Stoner who compared profiles before and after the orthodontic treatment with ideal profiles [1]. Taking into consideration ethnic characteristics, gender, and age, it becomes clear that one culture's concept of beautiful and acceptable can be seen differently in other cultures [2,3,4]. By measuring facial soft tissues, as well as dentofacial ratios, standard reference values for different populations were determined [5,6].
A number of orthodontic patients is interested in orthodontic therapy due to aesthetic disharmony. Such problems are easiest to observe and analyze on photographs [7]. Photographs represent auxiliary diagnostic method in orthodontics. They are used for proper diagnosis, treatment planning and treatment progress, tracking changes during growth and development, as well as an aid in communication with the patient. Photographs are also used for planning surgical treatment in orthodontics. Photographs have to be taken in standardized conditions, including predefined distance from lens to the patient, as well as standardized head position during imaging [8]. Photographs can also be used for educational purposes and for research. Digital technology has become an important element of clinical activities in orthodontic documentation. Digital extraoral and intraoral photographs can be imported into the software and presented together on the screen [9]. Photographs are part of proper documentation and visual reference for tracking changes in growth period. They are more understandable and selfexplanatory to patients compared to radiographic images. Making measurements from photographs is less invasive procedure for the patient and long lasting record that can be accessed at a later time [10,11].
Determining positions of anthropological points and angles is necessary for precise determining deviation of normal. Anthropological analysis allows obtaining valuable information of the face characteristics. Proper choice and precisely determined reference points, regardless of method of analysis (manually or by a software) is of utmost importance for obtaining the exact measurements of angular and linear values [12].
The aim of this paper was to assess usability of computer software for automated quantifying of linear measurements of frontal face photography of the patient.

MATERIAL AND METHODS
Measurements were performed on the set of 20 frontal face images in NHP (Natural Head Position) collected from various sources: 4 photographs from FEI set [13], 7 photographs of students of the University of Banja Luka and 9 publically accessible photographs of the models (in total 8 female and 12 male images). In order to make conditions more realistic for the computer software, dimensions of the photographs, position and relative face size were not the same on all photos.
Measurement points were chosen to fulfill two criteria: 1) most often used points in orthodontic analysis and 2) those that could be determined on two-dimensional image. Measurement procedure and data analysis were conducted based on the analysis performed by Bland and Douglas [14].
Reference measurement was done as described: positions of the following points on every image were determined by an orthodontist: N (Nasion), Sn (Subnasale), Gn (Gnation) and left and right Zg (Zigion). Measurements were repeated after one week. Based on those two measurements, the average position of every point was calculated and used as a third set of data. Figure 1 demonstrates the work window of used computer program.
After the initial manual measurements were done, the same points' positions were determined automatically by custom-made computer program. The program uses OpenCV (Open Computer Vision) [15] library for face detection, dlib [16] library and the predictor [17] that enables efficient detection of 68 reference points shown on the Figure 2. Based on those points, it is possible to determine positions of dependent points as well as required measurements. As all measurements were made in pixels, which real size varies between photographs, the program also calculates normalized values by dividing measured vertical distances (upper and lower facial height and total facial height) with bizygomatic distance (facial width). This approach required establishing correct position of Zg points first and whether the calculated values of bizygomatic distance were close to individually measured.  Obtained values were statistically analyzed and mean values and standard deviations were calculated for direct and normalized values. Paired two-tailed T-test was used and conclusions were made. Table 1 shows summary overview of measured values with mean values, standard deviations and p values of paired two-tailed T-test of average values measured by the orthodontist and computer. Differences of mean values for directly measured values ranged from 0.03% for bizygomatic distance, over 0.17% for facial height, 0.37% for upper facial height and 0.6% for lower facial height. Differences of means for normalized values ranged from 0.08% for upper facial height, over 0.22% for facial height to 0.48% for lower facial height. P values strongly suggested association between manually and automatically obtained measurements for every parameter. Table 2 shows differences of mean values obtained for every measured parameters using manual (average of two measurements) and computer approach. Mean values for 4 parameters were smaller for automatic measurements, while values for 3 parameters were smaller for manual measurements. Figures 3 to 6 give graphical representation of measured values as follows: blue line represents the mean values of two manual measurements, marks "+" and "x" represent individual manual measurements and red circles represent automatic measurements. There is no qualitative difference between automatic and manual measurements for observed set of samples, which is in agreement with values in Tables 1 and 2. There is a high level of correlation of measured values for bizygomatic distance obtained manually and automatically ( Figure 3). Figure 4 shows measurements for facial height acquired manually and by the computer software. Differences in obtained values are minimal. Based on the Figure 5 that represents measurements for lower facial height, it can be seen that the differences are small while relatively larger discrepancies are present only with extreme values. Also, differences between manually and automatically obtained    values for upper facial height are minimal, except for extreme values of measured parameters ( Figure 6).

DISCUSSION
In clinical practice, but also in significant number of papers, one can observe the trend of automation of extraoral examination through the use of digital imaging and software that allows automatic calculation of various values based on reference points determined by an orthodontist [18][19][20]. Moshkelgosha et al. performed statistical analysis based on 27 points and 43 calculated values on the sample of 110 patients [18]. Aksu et al. analyzed the reliability of reference points in photogrammetry based on 9 measurements taken directly on 100 patients, as well as measurements determined by computer using facial images. After statistical analysis they concluded that reliability depends on gender and measured parameters [19]. Milutinović et al. compared standard and computer aided method of analysis of profile cephalogram on the sample of 32 patients. They found no statistically significant difference between the measurements obtained by the two methods with significant advantages in efficacy with computerized method [20].
On the other side, there is a widespread use of 3D scanners that enable generating and analysis of three-dimensional models of the patient's face with the possibility of automatic determining of the reference points for further analysis [21].
Over the course of multiple years of research Deli et al. identified problems related to automatic determining of the facial indicators. When reference indicators were projected on the faces or structured light approach used, the biggest problem has been shown to be low contrast between indicators and the skin of the patient [22]. Therefore, they concluded that the use of photogrammetry and multiple standard cameras were better solution compared to other approaches such as laser scanning and structured light [23]. The whole system included five precisely positioned digital cameras, predefined background with coded markers and light system. Differences from reference measurements were under 1% that was considered acceptable.
As indicator positioning directly on face is not an optimal approach, Loconsole et al. developed a method based on Microsoft Kinect device and custom made software for automatic determining of 11 points on the face [24]. During the research they used three methods of measurements -manually with digital two-pronged orthodontic caliper, automatic measurement based on the single recording and automatic measurement based on 100 successive recordings of the same face by averaging the values. Based on performed analysis they concluded that proposed method provided good results for nasion, subansale and left and right chelion. They identified problems with automatic measurements of the points that included palpation in manual measurements as well as the points that could not be precisely positioned in two-dimensional image (e.g. tip of the nose).
The proposed method for automatic determining of linear measurements of the face provided results comparable to measurements generated by standard manual method with no statistically significant difference. Therefore, it can be concluded that the use of automation in this case did not deteriorate the exactness of measurements, where on the other hand computer program was much faster in measuring than an orthodontist. The software is capable to position the points, calculate dependant parameters and analyze them for the time that human operator need just to select the image for analysis -about one second. Also, taking digital images is far cheaper and faster than generating 3D model of the face. Also patients are more familiar with digital imaging than 3D scanning.
Keeping in mind rapid advances in information technologies and mobile computing systems, it is expected in the near future to enable the smartphone or tablet for rapid determining of facial parameters of the patient and analysis of obtained data with improvements in interaction and cooperation with the patient.

CONCLUSION
The results of this research suggest that it is possible to ease, accelerate and automate the work of an orthodontist in the image analysis by using adequate supporting software without the significant differences in measured values.