Evaluation and comparison of methods for forensic glass source conclusions

https://doi.org/10.1016/j.forsciint.2019.110003Get rights and content

Highlights

  • We use a large database of LA-ICP-MS glass analyses to simulate forensic comparisons.

  • We apply a machine learning method to the forensic glass source conclusions.

  • We compare the machine learning method to the standard interval-based method.

  • In this large database, the machine learning method has much lower error rates.

  • We investigate the two methods to uncover why the standard method fails more often.

Abstract

Float glass, a common type of glass used in windows and doors, can be important evidence in the investigation of a crime. If fragments are optically indistinguishable, they may be distinguishable in their chemical compositions, which can be measured using inductively coupled mass spectrometry with a laser add-on (LA-ICP-MS) [14]. If the measurements are “similar enough” then the recovered fragment is indistinguishable from the crime scene glass [1]. Recently, Park and Carriquiry [10] proposed using machine learning methods to establish probabilistic source conclusions for glass, and found that these methods have lower classification error than traditional methods. Using an experimental database of glass elemental concentrations to simulate different forensic scenarios, we examine the results from two different classifiers to understand why learning algorithms appear to outperform traditional methods when making source conclusions for forensic float glass questions. By analyzing each step in the recommended ASTM decision process, we conclude that the standard ASTM method is not the optimum and that more data are required to determine a better comparison rule for source conclusions based on the chemical makeup of float glass.

Section snippets

Background

Float glass, a common type of glass used in windows and doors, can be important evidence in the investigation of a crime. Because this type of glass is so prevalent, understanding its properties is crucial to forensic investigations. For example, consider a burglary of a jewelry store. There may be a great deal of broken float glass at the crime scene from the windows, doors, and display cases in the store, and determining the various sources of broken glass may be key to apprehending the

Understanding the ASTM method

To investigate the ASTM method described in the background, we designed a “forensic scenario” sampling scheme to sample observations from the database and construct theoretical forensic glass problems. First, we sample one fragment from one pane to serve as the questioned source sample GQ. Then, to adhere to the ASTM standard [2], we randomly pick three fragments from a single pane in the database to comprise the known source fragments GK. We repeated this sampling 30 times for each possible

“Stress Testing” the ASTM standard

We conclude our examination of the ASTM method by examining its behavior under “extreme” conditions. First, we compare the ASTM standard to the RF method by their final classification results: TP, TN, FP, FN. Then we analyze the results for all methods when comparing closely manufactured panes, which were made by the same manufacturer within a two-week period.

Table 5 contains a summary of classification results from both methods applied to all 69,120 comparisons. The results are grouped

Discussion

By using a large database of glass concentrations, we have explored methods for making forensic glass source conclusions. Following the guideline defined in ASTM-E2330-12 [1], we performed over 69,000 simulated forensic glass source conclusions and uncovered some informative properties of the ASTM guideline, and compared it to a proposed machine learning method. By looking at the distribution of measurements for each element, we learned that the mean fragment concentrations do not follow the

References (16)

  • Tom Fawcett

    An introduction to Roc analysis

    Pattern Recognit. Lett.

    (2006)
  • ASTM-E2330-12

    Standard Test Method for Determination of Concentrations of Elements in Glass Samples Using Inductively Coupled Plasma Mass Spectrometry (ICP-MS) for Forensic Comparisons.” Retrieved from 10.1520/E2330-12

    (2012)
  • ASTM-E2927-16

    Standard Test Method for Determination of Trace Elements in Soda-Lime Glass Samples Using Laser Ablation Inductively Coupled Plasma Mass Spectrometry for Forensic Comparisons.” ASTM International Retrieved from 10.1520/E2927-16

    (2016)
  • Shirly Berends-Montero et al.

    Forensic analysis of float glass using laser ablation inductively coupled plasma mass spectrometry (La-Icp-Ms): validation of a method

    J. Anal. Atom. Spectr.

    (2006)
  • Leo Breiman

    Random Forests

    Mach. Learn.

    (2001)
  • Hendrik Dorn et al.

    Discrimination of float glass by LA-ICP-MS: assessment of exclusion criteria using casework samples

    Can. Soc. Forensic Sci. J.

    (2015)
  • Robert D. Koons et al.

    Interpretation of glass composition measurements: the effects of match criteria on discrimination capability

    J. Forensic Sci.

    (2002)
  • Robert D. Koons et al.

    Classification and discrimination of sheet and container glasses by inductively coupled plasma-atomic emission spectrometry and pattern recognition

    J. Forensic Sci.

    (1988)
There are more references available in the full text version of this article.

Cited by (13)

  • Inter-laboratory workflow for forensic applications: Classification of car glass fragments

    2022, Forensic Science International
    Citation Excerpt :

    One such approach is based on the usage of Machine Learning (ML) algorithms. Several models derived by such algorithms were reported in the literature and used to estimate the similarity between a pair of glass fragments in order to determine whether they originate from the same source [13,14] as well as to classify glass fragments into one of pre-defined classes [15]. These and other studies were summarized in several review papers [16,17].

  • Homogeneity assessment of the elemental composition of windshield glass by µ-XRF, LIBS and LA-ICP-MS analysis

    2022, Forensic Chemistry
    Citation Excerpt :

    For instance, a generally accepted assumption within the forensic trace community is that multiple known sources originating from a single glass pane will have very similar elemental compositions. The homogeneity of single sources of glass has been demonstrated using LA-ICP-MS [5,7,38]. It was reported that glass is homogeneous even at the micro-range level and that the variation (%RSD) within the fragments from a same source was below 5–10% [5,7,38].

  • PIXE based, Machine-Learning (PIXEL) supported workflow for glass fragments classification

    2021, Talanta
    Citation Excerpt :

    More recently, Park and Carriquiry [23] have demonstrated that ML algorithms (RF and Bayesian Additive Regression Trees) perform better than traditional, interval-based algorithms in determining whether two glass specimens come from the same or from different sources. In this work the glass fragments were characterized by the concentrations of 18 elements obtained by LA-ICP-MS. In a subsequent paper, Park and Tyner [24] attributed the better results obtained with the ML methods to the ability of these methods to correctly handle non-uniformly distributed data as well as correlated data. ML-based methods for glass engineering were reviewed by Krauss and Drass [25] and by Liu et al. [26].

  • Adjusted binary classification (ABC) model in forensic science: An example on sex classification from handprint dimensions

    2021, Forensic Science International
    Citation Excerpt :

    If the probability is greater for the second group, the specimen will be classified in group 2 [9,10]. The alternative approach, more frequent in other forensic fields (e.g., [11–14]), is the empirical thresholding using Receiver Operating Characteristic (ROC) curves. This way, selected thresholds allow researchers to minimize both false positive and false negative results [15], or only one of them depending on the research question [13].

  • A database of elemental compositions of architectural float glass samples measured by LA-ICP-MS

    2020, Data in Brief
    Citation Excerpt :

    In some cases, a fragment may have fewer than five replicate measurements if the ablation spot was not optimal (e.g., hitting a fracture). Fig. 2 in Park and Tyner (2019) [4] shows an example scheme of how we get measurements from one pane. Analytical procedures to carry out the measurements followed the protocols recommended by ENFSI [1] and ASTM ([2]).

View all citing articles on Scopus
View full text