Accurate prediction of saw blade thicknesses from false start measurements.

BACKGROUND
False start analysis is the examination of incomplete saw marks created on bone in an effort to establish information on the saw that created them. The present study aims to use quantitative data from micro-CT cross-sections to predict the thickness of the saw blade used to create the mark. Random forest statistical models are utilised for prediction to present a methodology that is useful to both forensic researchers and practitioners.


METHOD
340 false starts were created on 32 fleshed cadaveric leg bones by 38 saws of various classes. False starts were micro-CT scanned and seven measurements taken digitally. A regression random forest model was produced from the measurement data of all saws to predict the saw blade thickness from false starts with an unknown class. A further model was created, consisting of three random forests, to predict the saw blade thickness when the class of the saw is known. The predictive capability of the models was tested using a second sample of data, consisting of measurements taken from a further 17 false starts created randomly selected saws from the 38 in the experiment.


RESULTS
Random forest models were able to accurately predict up to 100% of saw blade thicknesses for both samples of false starts.


CONCLUSION
This study demonstrates the applicability of random forest statistical regression models for reliable prediction of saw blade thicknesses from false start data. The methodology proposed enables prediction of saw blade thickness from empirical data and offers a significant step towards reduced subjectivity and database formation in false start analysis. Application of this methodology to false start analysis, with a more complete database, will allow complementary results to current analysis techniques to provide more information on the saw used in dismemberment casework.


Introduction
Disciplines of forensic science are under scrutiny by both governing bodies [1,2] and other researchers in the field [3][4][5]. In particular, those forensic sciences which rely upon pattern-based evaluation, such as toolmark evaluation, are questioned [3,5,6]. Subjectivity is inherent in any technique which requires human interpretation and experience. Hence, increasing quantitation within the analysis improves objectivity and increases confidence in reliability and validity. Pelletti and colleagues state that, in order for a novel method to be validated, a technique must have proven accuracy, precision and inter-rater reliability [7].
In dismemberment cases, the study of saw marks is essential in attempting to identify the weapon used [8,9]. Current casework methodologies focus on casting and comparing striation detail of saw marks [10] allowing determination of similar qualities between experimental saw marks and casework saw marks using human evaluation. Previous research studies have aimed to improve the information that can be gathered from this evidence, for example the type of saw used and the blade thickness [7,11]. Previous research has described in detail how profile shapes of false starts can be used to establish the teeth set of the saw used [11,12].
The knowledge and characterisation of false starts, defined as incomplete saw marks, is invaluable in dismemberment investigation to provide critical information on the saw that created it [9,10,13]. Research into false starts has involved the use of a multitude of experimental materials. Bones from animals, including pig [14], deer [15], and cow [16], have been previously used as representative of human tissue due to greater availability. However, Nogueira and colleagues call into question the suitability of these materials due to differences in false starts on dry pig bones and human bones [12]. Further to this, research has strongly recommend that fleshed, human remains be the focus of toolmark on bone research experiments suggesting dry bones are unsuitable for experimental development [11].
Micro-Computed Tomography (CT) is an imaging technique that has been applied to many areas of forensic science in recent years, including gunshot residues [17,18], bloodstains [19], pathology [20][21][22][23][24] and forensic anthropology [11,25]. Micro-CT works by capturing thousands of 2D radiographs from the full rotation of the sample which are then reconstructed to form a full 3D digital model [26]. This is advantageous as the technique is nondestructive and also allows for cross-sections and measurements to be taken at any point. The field of toolmark analysis is one area that has benefitted from this technology through visualisation and measurement of toolmark cross-sections [7,11,27].
Previously, Norman and colleagues [11] introduced a univariate linear regression method for predicting saw blade thickness of false starts from micro-CT data. The research concluded that a fleshed, free-saw action is likely to provide false start data similar to casework. The researchers were able to accurately predict the tool thickness in 94% of cases across all methodological conditions, although this reduced significantly when predicting from only fleshed, free-saw toolmarks [11].
This study presents a more objective, robust, and repeatable random forest method for saw blade thickness predictions, which can be complementary to current human evaluation techniques. Random forests methodologies for saw marks have been previously proposed and described by Love and colleagues [28]. These researchers used traditionally accepted saw mark properties, determined through microscopy analysis, to predict the blade type of the saw used [28]. Love and colleagues [28] demonstrated the applicability of this technique to forensic toolmark analysis. Norman and colleagues also suggest the use of decision trees, which is the foundation of a random forest, for the analysis of false starts [11].
Random forests are an ensemble learning technique consisting of many decision trees grown from a dataset. Using this uncorrelated forest of trees, instead of a singular decision tree, provides a more accurate prediction and prevents overfitting.
Random forests offer a trainable, robust method suitable for datasets with missing values and possible outliers due to the bagging process, making this technique ideal for prediction with multiple variables [29].
Hence, the use of regression random forest in the prediction of saw blade thickness, from micro-CT cross-sections, is proposed in this study with the aim of introducing a method which improves overall accuracy and sensitivity of micro-CT saw mark analysis. In this study, micro-CT measurements of false starts created on fleshed human cadavers and statistical analysis is used to produce random forest models to predict the thickness of the saw used to create a mark.

Materials
A total of 38 newly purchased saws, Fig. 1, were selected based on market research establishing the best-selling and most popular models of saw. The saws also were chosen to reflect the variety of profile shapes which have been described by previous authors [11,12,30]. Materials were purchased from four well known hardware suppliers in the United Kingdom, and one online marketplace. The final saws used consisted of seven reciprocating (power) saws, eleven hacksaws, and twenty hand powered saws over five sub classes, shown in Table 1. Hacksaws are separated from hand saws in this study due to the difference in false start properties exhibited by these marks [11,30]. The saw blade thicknesses were established by taking 30 measurements, with digital callipers, over the length of the blade and establishing the mean and standard deviation, as with previous studies [11].

False start creation (experimental)
Sixteen non-pathological human legs, from eight cadavers (5 male, 3 female) with a mean age 82, were sourced following full ethical approval by the first author's institution and following standard Human Tissue Authority guidelines. Femurs and tibias were chosen due to the prevalence of false starts in this area during dismemberment [11,31]. The precedent set by Norman et al. to use fleshed, cadaveric limbs and a free-saw action to create the marks, was followed in this study [11]. The location of the false starts was randomised across femur/tibia, donor, bone side, and location on the bone. One individual attempted ten marks per saw to ensure statistical reliability, as established by the effect size analysis of data carried in previous studies [11]. Some attempted false starts were not measurable resulting in 8-10 marks per saw, this was due to some attempted false starts not reaching the bone, or lightly scratching the surface. Each bone was defleshed manually after false start creation for ease of transportation and scanning. False starts were created in two sessions on the same day and under the same conditions, as described above. Sample 1 consisted of 340 false starts, of randomised location, created by all 38 saws on leg bones from seven cadavers. Sample 1 measurements were used to create all statistical models. Sample 2 consisted of one false start per saw, to be used to establish the predictive capability of the statistical models. Due to the experimental variability described above, only 17 false starts were measurable in Sample 2.

Micro-CT scanning
A Nikon XT H 225/320 LC micro-CT scanner (Nikon Metrology, UK) was used to image the false starts. Scan parameters were 160 kV, 20 W, no filter and 500 ms exposure which resulted in resolutions of approximately 55 mm with a total scan time of 140 min per bone. The 3D micro-CT data were reconstructed from the 3142 x-ray projections using Nikon's proprietary software, CT Pro and then exported to VGStudio MAX 2.2 (VolumeGraphics, Heidelberg, Germany). A calibrated work-piece was included in each scan to allow voxel rescaling and improve measurement accuracy [32,33].

Quantitative data collection
Using VGStudio MAX 2.2, a cross-section image was obtained from the digital models of each false start. The cross-section location was obtained parallel to the floor of the false start and halfway through the mark. In cases where the false start floor was not visible at this point, the nearest section showing a full floor was selected. Repeatability of the cross-section measurement procedure was tested through inter-and intra-operator assessment using Intra-class Correlation Coefficient (ICC). Three operators were evaluated using a sub-sample of randomly selected crosssections from different saws. The operators consisted of one individual with prior experience of the procedure, and two individuals with no prior training or experience in toolmark analysis. Intra-operator agreement was measured through three different measurement sessions of Operator 1.
This study builds on a study conducted by Norman and colleagues who proposed five measurements to assess micro-CT cross-sections of false starts [11]. The researchers proposed the use of: minimum toolmark width at floor, wall angle, trough height, trough angle deep, and trough angle shallow. The latter two measurements were only suggested for convex profiles, those exhibiting a 'W' profile shape (Fig. 2). However, in the current study it is proposed that both angles be taken on all profiles but standardised to avoid directionality misinterpretations, hence this study utilises angle mean (Am) and angle difference (AD), Fig. 2(a) and (b). Trough height (h) and minimum toolmark width at floor (W 0 ) are measured, see Fig. 2(d); h is measured as 0 for non-convex profiles. Furthermore, this study proposes the use of toolmark width at 50% height (W 50 ), toolmark width at maximum (W 100 ), and internal angle (g) to analyze false start profiles, Fig. 2(c) and (d). Post-hoc multivariate analysis of data from the previous study suggested wall angle measurements to not be useful in predicting saw blade thickness, so this measurement has been omitted from the analysis [11].

Statistical model creation
Using in-built MATLAB (The MathWorks, Inc., MA, USA) functions, the random forest models were produced. The holdout validation method was used to reduce overfitting of the model, with 15% held out [34]. The random forest models were tested through prediction of blade thicknesses from Sample 1 data which was used to build the model.
Predictive capabilities of the models were established through prediction of blade thicknesses from Sample 2 false starts. This was used to confirm the models were representative of the saws and the practicability of the models for simulated casework prediction. To test the general applicability of the model to casework a third set of measurement data was used to predict blade thickness, known as Sample 3. Sample 3 used false starts collected from a previous studys [11], with the addition of the extra measurements proposed in the current study, to predict blade thickness of saws not in the original data. Two random forest model approaches are proposed. The Unknown Class Model (UCM) is suitable for all false starts without prior knowledge of the saw class, and hence is a standalone method for saw blade thickness prediction. A second model, the Known Class Model (KCM) is applicable if the overarching saw class handsaw, hacksaw or power sawis known prior to prediction. The KCM approach requires some prior knowledge of the class and therefore must be applied in combination with another classification technique. While the UCM is built on one regression random forest, the KCM consists of three random forest models separated by class.

Inter-and intra-operator agreement
Intra-operator reliability was proven through ICC analysis on all measurements, Table 2. Agreement was shown with ICC values >0.75 for all measurements, indicative of 'excellent' agreement on the ICC agreement level scale [35].
Inter-operator reliability also showed an 'excellent' agreement over all measurements, Table 2. As such, Operator 1 0 s measurement data on all cross-sections were solely used for statistical analysis going forward.

Prediction of saw blade thicknesses
The prediction of saw blade thicknesses through the random forest models is laid out in Table 3. Each prediction value against  the accepted blade thickness range is shown in Figs. 2 and 3, and the variability of the predicted value against the blade thickness mean is shown in Tables 4 and 5. The KCM consistently predicts saw blade thickness with a greater accuracy than the UCM. The applicability of the model, when the class is known is shown through the prediction of Sample 3 by the KCM, with accuracy up to 100%.

Discussion and conclusions
Following many researcher and governmental guidelines [1][2][3][4][5][6]36], this study aims to present a reduced subjectivity methodology. The current study introduces a random forest prediction approach to false start profile analysis in combination with an improved cross-section measurement methodology from micro-CT data, building on work from previous researchers [11,37]. As such, features used in previous studies which require human determination (e.g. profile shape, toolmark shape [11]) were omitted. Measured saw blade thickness in this study experimentally followed previous studies [11] by taking thirty thickness measurements with digital callipers down the length of the blade. Experimental error and blade variability has not been studied and hence is a limitation of this work.
The random forest methodology introduced in this study is aimed as a tool to complement current methodologies. Striation analysis and profile shape determination are important techniques in toolmark examination. Where striation analysis and profile shape can inform on the TPI and tooth set respectively of the saw used [11,38], the random forest method proposed may inform on the thickness of the saw used. In combination, these techniques should enable elimination of some suspect saws and possible indication of the type of saw used.

Micro-CT
Micro-CT as a technique for analysing forensically relevant toolmark evidence has been shown by previous studies to provide valuable data and methodology for both research and casework [7,11,37,39,40]. This study further shows the applicability of micro-CT technology to these forms of evidence. Non-destructive crosssection data for false starts is unparalleled and is proven to provide the necessary data for new methodologies.

Saw blade thickness prediction
The results of this study present the possibility for accurately, within 2 standard deviations, predicting saw blade thicknesses from false start data, particularly for saws within the model. This high accuracy for Sample 2 prediction offers insight into the practicability of the methodology to casework. Accurately predicting the saw blade thickness from a false start will give a clear indication of the tool used to create the mark, and importantly allow elimination of saws unlikely to have created the mark. Assuming a representative database, the saw blade thickness prediction allows up to accuracy.
Previous studies have shown the practicability of statistical analysis for complementary saw mark analysis techniques [7,11,16,37]. Univariate regression models have been employed Table 3 Overall correct prediction results of the two random forest models: UCM and KCM, as applied to Sample 2 and Sample 3 false start measurements. by Bailey and colleagues [16] showing the effectivity of saw mark measurements in elimination of incorrect saw classes. Statistical evaluation of micro-CT data has also been shown, by Giraudo and colleagues [37], to enable a good discrimination between false starts and is suggested as complementary to currently employed methodologies. Previously, a univariate approach to saw blade thickness prediction has been proposed [11]. The random forest approach introduced in the current study builds on these previous studies to introduce a complementary methodology with a generalised applicability. A random forest approach improves overall accuracy and sensitivity of this analysis technique to allow practicability of the technique within casework.

Limitations and future work
Due to the nature of scientific donation, human cadavers for research purposes are often older than the general population, as seen in this study with a mean age 82. It is worth noting that without casework validation, the applicability of the proposed method is unknown. The nature of false start analysis exclusively studies toolmarks in cortical bone, indeed those extending into the trabecular are excluded from the study, and previous, studies [11,13]. Age related factors are unlikely to alter the mechanical properties of the cortical bone [41] but further research into applicability is necessary. The KCM, particularly for Sample 3 data, show a much higher predictive accuracy than the UCM. Hence, features of the saws, such as tooth set or TPI, have an impact on the prediction of saw blade thicknesses. This highlights the necessity for complementary analysis with current techniques to establish the class. Furthermore, in order to be implementable to casework, a more comprehensive database of saw false starts would be needed. However, the practicability of creating a comprehensive database is reliant on the resources available.
Reducing subjectivity is a prime aim of this study. Nevertheless, human error is still likely to be introduced in the manual process. Extraction of the cross-sections from 3D digital models is a probable introduction of human error. As 3D models are manually oriented in order to obtain the cross-section image, it is possible error is introduced. Though it is expected that any cross-sections across the false start should still be representative of the weapon, production of a more objective automatic process should eliminate the introduction of error at this point. Automation of cross-section extraction will also standardise this area which has currently not been studied for error. With further experimental data it is envisioned this tool may be extremely useful in a more objective, and data driven, method for false start analysis.

Conclusions
The methodology proposed in this study shows the applicability of regression random forest models in the accurate prediction of saw blade thicknesses from quantitative false start data. Practically, this method should be complementary to current analysis techniques to provide useful information on the saw used in dismemberment casework. With the methodology presented it is proposed, with a thorough database of false start measurements from representative saws, a digital tool for identification of probable saws used in a dismemberment.

Declaration of Competing Interest
The authors declare no conflicts of interest.