Comparison of supervised machine learning algorithms for waterborne pathogen detection using mobile phone fluorescence microscopy

1 Introduction

Giardia lamblia is a waterborne pathogen that affects millions of people each year worldwide and causes health problems such as diarrhea, not only in developing countries but also in developed countries; it is one of the most common infectious parasites in contaminated drinking water [1], [2], [3]. Traditional treatment methods such as chlorination are not as effective to remove the cysts of the parasite because the cyst’s thick cell wall makes it moderately resistant to chlorine [1], [2]. Therefore, timely detection of the presence of the cysts in water samples is extremely important to treat and prevent the spread of the disease, especially in field settings [4], [5], [6], [7], [8], [9], [10].

Toward this goal, we previously developed a mobile phone-based waterborne pathogen detection platform [11]. This field-portable and cost-effective platform (Figure 1) uses the machine learning-based analysis of fluorescence images acquired by a mobile phone microscope and does not require a benchtop device for sample preparation or a microbiology expert for the analysis of results. This mobile phone-based platform achieved an average cyst capture efficiency of 79% and a limit of detection of 12 cysts per 10 ml of water sample [11]. In this paper, we provide a survey of supervised machine learning classifiers to further improve the same mobile platform’s cyst detection sensitivity and specificity, together with several performance improvements on the sample preparation side. For this study, we created several prototypes of our mobile phone-based pathogen detection platform to evaluate device to device variations and prototyping errors. Furthermore, a cyst training database containing >30,000 fluorescent images of individual Giardia cysts, each labeled by an expert, was created at different image file formats including joint photographic experts group (jpeg) and digital negative (DNG) formats. We further improved our training image database for raw images by increasing the number of dust particles (i.e. other fluorescent objects that are not cysts) to ~155,000 to improve the specificity of the platform. On the basis of this significantly enlarged database of images, we then evaluated the overall classification accuracies of several supervised machine learning classifiers [12], [13], [14], [15], [16] [including, e.g. support vector machine (SVM) [15], decision trees [14], and ensemble [16] methods] and demonstrated that a bagging classifier model gives the highest accuracy levels and the largest area under the receiver operating characteristics (ROC) [17] curve. Our results also demonstrated that raw image format (DNG) is better than other file formats in terms of detection and classification accuracy and the area under the ROC curve.

Figure 1:

(A) Schematics of our smartphone-based pathogen detection platform with its illumination path, and (B) the contents of a prefiltration unit and a disposable sample processing cassette. (C, D) Photographs of the platform and three prototypes used throughout our experiments. These mobile phone-based sensing devices follow our previous design reported in [11], and in this work, we quantified the impact of device-to-device, phone-to-phone as well as image file format-related variations on the performance of our mobile sensor platform using various water samples, including, e.g. tap water, non-potable water, and pond water.

We tested this machine learning enabled pathogen detection platform using various types of water samples (e.g. tap water, pond water, non-potable water, and reagent grade water) that are screened using our pre-filtration device which removes unwanted particles/debris prior to introducing Giardia-specific staining into water samples. Our cyst counting results provide very good agreement with expert counting results obtained with a benchtop fluorescence microscope as well as with our dilution tests using flow-cytometer counted samples. This waterborne pathogen detection platform with its rapid, sensitive, and automated cyst detection and counting interface can be a useful tool for monitoring of water quality in field settings and resource-limited environments.

2 Experimental section

2.2 Sample preparation

Sample preparation steps are described in our previous article [11] with the exception of using a syringe barrel instead of a 15 ml falcon tube as a sample container and a pre-filtration unit (Figure 2 [11]). In summary, we first modified the surface of a syringe barrel using 0.01% Tween 20 solution. For this purpose, 10 ml of Tween 20 solution was collected into the barrel and stored at room temperature for 20 min to reduce nonspecific binding of Giardia cysts. After emptying the syringe barrel, we collected 10 ml of water under test (e.g. tap water, pond water, non-potable water, or reagent grade water) into the barrel while a pre-filtration unit was coupled to the syringe barrel. After the collection of the water sample into the barrel, the pre-filtration unit was removed and we added 200 μl of 1× diluted Giardia-specific stain into the sample and closed it using a Luer cap. After shaking the mixture gently for about 10 s, a small piece of aluminum foil was wrapped around the barrel and stored at room temperature for 30 min for labeling. The water sample was then dispensed onto a black membrane in the disposable sample processing cassette. The barrel was refilled with 5 ml of reagent water from its piston end and shaken vigorously for 15 s to wash out the remaining sample in the barrel. This step was repeated twice. Two hundred microliters of counterstain (diluted in 3:1 ratio) was dispensed onto the filter membrane, with a waiting time of about 1 min. Finally, we dispensed a few droplets of mounting medium to cover the black membrane and decrease the fading of the fluorescent stain. The cassette containing the fluorescently labeled Giardia cysts was then coupled to the attachment unit and an image of the filter membrane was captured using our waterborne pathogen detection platform through our custom-developed Windows-based application. We prepared Giardia-spiked water samples using 10 ml of reagent grade water for the population of training data, while we used water samples from other sources (such as potable water) to independently validate the performance of the mobile device.

Figure 2:

Sample preparation, image processing, and machine learning workflow.

(A) Steps for populating a training image database and choosing a classifier model [11]. Following a generally similar approach compared to [11], the sample preparation steps outlined in (A) have been optimized and refined to handle realistic water samples, e.g. tap water and pond water. These optimized sample preparation steps are highlighted with bold letters. The number of steps to prepare a water sample for testing is reduced by using a syringe barrel as a sample collection tube and the total time required for filtration is shortened to 21 min. (B) Steps for automated detection and classification of Giardia cysts using the trained model in (A).

2.6 GiardiaAnalyzer: our custom-developed smartphone application

We developed a Windows-based smartphone application, named as GiardiaAnalyzer, to automatically detect and count Giardia cysts using our waterborne pathogen detection platform (Figures 2B and 3). The application allows the user to capture an image, select an existing image in the directory of the phone, upload an image to our servers for digital image processing and search for Giardia cysts, and view the history of measurements. The user is also guided on how to use the smartphone microscope with on-screen instructions. After selecting an image to be uploaded to our servers, the GPS location and date/time settings of the smartphone are attached to the image automatically, and the user can assign a custom job name to each image to see the results (e.g. cyst count, GPS location, and date/time) in the history. The application enables processing of more than one image on the servers and updates the user about the status of each image (e.g. uploading, processing, etc.).

Figure 3:

(A) Following our previous design reported in [11], graphical user interface of our smartphone application, GiardiaAnalyzer, is shown. The new version of the application has several additional features: (1) it enables parallel processing of the acquired images in the server, saving processing time; (2) it provides continuous feedback to the user about the status of the image analysis; and (3) it enables assigning a job name for each image to be uploaded to server. (B) Screenshots of the application for different stages of the image analysis. (B-i,ii,iii) Photographs demonstrating the feedback to the user about the image analysis and processing of the results.

For each water sample that is imaged, digital image processing and automated classification of fluorescent particle candidates captured on the filter membrane are done at our servers. Our training data contain more than 30,000 fluorescent images of Giardia cysts and, after selecting a test image for uploading to the servers, it takes ≤90 s to get the detection/counting results back on the screen of the smartphone.

3 Results and discussion

Our smartphone-based microscope has a large field of view (FOV) that covers the entire area of the filter membrane, i.e. ~0.8 cm², and has no moving parts. This handheld, field-portable and cost-effective platform weighs only ~205 g (excluding the phone, but including the sample cassette, which weighs ~25 g) and is battery-powered, i.e. does not require stable electricity to operate. Sample preparation is easy and the entire platform can be operated by a minimally trained user. There is no need for a pump or any benchtop device for sample preparation and optical detection. Also, there is no need for a microbiology expert to label and count the cysts that are captured since both of these steps are automated in our mobile platform using machine learning.

We used supervised learning algorithms to classify the detected particles on our fluorescence microscope images and be able to specifically count Giardia cysts in the sample. In supervised machine learning, a known set of input data and known responses to this data are used to form and train a model, which is then used to predict the labels of the newly generated data, which in our case are fluorescence images captured using our mobile microscope. To increase the detection and classification accuracy of our platform, we first populated training data and evaluated the detection performance of our image processing algorithms with the images captured using GiardiaAnalyzer (i.e. JPG-GiardiaAnalyzer) as well as the regular camera application of the phone (i.e. JPG and DNG file formats). In these experiments, we combined the image data corresponding to five mobile device configurations (i.e. 1A, 1B, 1C, 2B, and 3C – see Section 2.3) for each image file format. In our database formed with these images, the total number of fluorescent particles labeled as Giardia cysts, also confirmed with a benchtop fluorescence microscope, was >30,000. For example, the training dataset populated using raw DNG images is composed of 30,674 cyst images and 104,699 images of other fluorescent objects, including, e.g. dust particles and background noise. Similar numbers of images are used to form training datasets in other file formats.

After populating the training data for each image file format, we assigned “1” as the label for a single cyst or a cluster of cysts and “0” as the label for other fluorescent objects (non-cysts) in our mobile phone images to have binary classification of fluorescent objects. We chose 5% of this populated data for each image format as our independent test data and 95% as our training data to compare the performances of different prediction models. In addition to the classification accuracy, the area under ROC curve (AUROC) is also another measure that we used to demonstrate how well the selected algorithm performs; for example, the accuracy of a test is classified as “excellent” if its AUROC is >0.90. In our comparisons, we tested several supervised learning algorithms such as SVMs [18], nearest neighbors [13], [19], and ensemble methods [20]. Figure 4 summarizes the effect of the image file format on the overall classification accuracy and the specificity of different classifier models. The classification accuracy of all the supervised learning methods that we tested are >81%, while the AUROC values are >0.7 regardless of the image file format. The best performance is obtained with the raw image format (i.e. DNG images), which is due to the fact that it preserves spatial details, retaining more information in the extracted features of each fluorescent object or cyst candidate. Lumia Camera JPG images resulted in a lower classification accuracy compared to that of JPG images captured using our GiardiaAnalyzer application since Lumia Camera JPG images are significantly more compressed, lowering the image quality. As shown in Figure 4, the best cyst predictive accuracy of ~95% and an AUROC value of 0.97 are obtained using bagged trees (number of trees, N_tree=400) as a classifier model using DNG images. Fine and cubic k-nearest-neighbor classifiers [18], [21], [22] provided fast fitting speeds but their predictive accuracy values are relatively poor, especially for JPG images. On the other hand, SVM and bagged ensemble classifiers are very good at their prediction accuracy values, while their training speeds are slower compared to other methods.

Figure 4:

The effect of the image file format on the classification accuracy and specificity of various classifier models.

DNG images resulted in much better classification accuracy compared to other file formats.

Next, we tested the performance of the best prediction model, i.e. bagging trees with N_tree=400, for the detection of Giardia cysts in different types of water including, e.g. reagent grade water, non-potable water, pond water, and tap water samples using the training database populated with raw format DNG images. Using 71 spatial features and ~104,000 images of non-cyst fluorescent objects in our database, our mobile platform measured 4.00±2.16, 2.67±2.49, 3.00±2.16, and 3.00±2.16 cysts for 10 ml of Giardia-free non-potable water, pond water, reagent grade water, and tap water, respectively (see Figure 5A). When we used a training database containing more features per fluorescent particle (e.g. 96 spatial features instead of 71), false positive cyst counts per 10 ml of clean water sample decreased slightly for all the control samples. Some of the spatial features or image structures in these 25 extra features are in fact common for both Giardia cysts and other non-specific fluorescent particles. This introduces a sampling bias in the training set and results in our machine learning algorithm to have more false negatives as compared to the training database with 71 features and 104,000 dust particles as part of our database. When we used a training database containing more non-cyst fluorescent objects, e.g. ~155,000 images of fluorescent dust particles, we observed a further improvement on the standard deviations and cyst counts per 10 ml of sample (i.e. 2.00±0.00 and 1.00±0.81) corresponding to non-potable water and pond water samples, respectively.

Figure 5:

Predicted cyst levels corresponding to 10 ml of (A) Giardia-free samples from various water sources, (B) Giardia-free and Giardia-spiked water samples, (C) low concentrations of Giardia-spiked samples using different training databases for a bagged tree classifier with 400 trees.

In addition to these control samples and false positive detection rates reported above, Figure 5 also reports predicted cyst count values for 10 ml of Giardia-spiked water samples against total cyst counts obtained using a benchtop fluorescence microscope. Giardia cyst count predictions trained with an image database containing 71 spatial features per object and ~104,000 images of non-cyst objects were in very good agreement (R²=0.9807) with the cyst counts obtained using our gold standard method, especially for high concentrations of Giardia cysts in water samples (Figure 5B). When we used a training image database containing a larger number of non-cyst fluorescent objects, the predicted cyst count values were much better for lower concentrations (e.g. <120 cysts per 10 ml) – see Figure 5C. Therefore, a combination of both of these training databases to separately handle low and high cyst concentration ranges would provide better overall performance. These results summarized in Figure 5 demonstrate that our machine learning-based mobile microscopy platform can automatically detect, and specifically and sensitively classify Giardia cysts from different water sources.

There are several optical features that contribute positively to the sensing performance of our waterborne detection platform. The first feature is the wide FOV of our design. Our waterborne pathogen detection platform with its wide FOV (~0.8 cm²) enables capturing an image of the entire sample filter membrane all at once, whereas, for comparison, several tens of images should be taken using, e.g. a 10× objective lens and stitched together to have the whole image of the same filter membrane using a benchtop fluorescence microscope. In our platform, the time that is needed to capture an image of this filter membrane is only 4 s without the need for any mechanical scanning or digital image alignment/stitching. Another important feature of our design is the use of a long-pass filter rather than a single-bandpass filter. As a part of our sample preparation protocol, Giardia cysts are labeled with fluorescein (490 nm/525 nm) conjugated antibodies for specific detection of the cysts in water samples. There may be some other fluorescent particles (e.g. dust particles) captured on the sample filter membrane, yielding fluorescence both in the green and red regions of the spectrum. Our platform utilizes a long-pass filter (i.e. transmitting >515 nm) to enhance the detection and classification performance of our supervised machine learning classifier, because some of the features included in the training database are based on intensity values of the detected particles at green and/or red image channels. We should emphasize that we used the green channel of an image acquired using our platform to detect possible fluorescent particles that are captured on the membrane. The blue channel of the acquired image does not provide any useful information since our long-pass filter blocks the blue region of the spectrum, while the red channel of the image contains information about dust particles only. This way, our machine learning framework has additional degrees of freedom to differentiate true positives (cysts) from non-specific fluorescent objects/particles.

4 Conclusions

We demonstrated a hand-held and cost-effective platform for automated detection and counting of Giardia cysts in large volumes of water samples using machine learning. This platform includes a mobile phone-based fluorescence microscope to capture an image of fluorescently labeled Giardia cysts captured over a filter membrane that has a large FOV of ~0.8 cm². This fluorescence image of the sample filter membrane, captured using a custom-developed application, is processed at our servers for automated detection and enumeration of Giardia cysts using a training image dataset, containing 71 spatial features per object and labels of >30,000 images of cysts and >100,000 images of non-cyst objects. The total time of this analysis per 10 ml water sample under test, from sample preparation to automated cyst counting, is less than an hour. By comparing several supervised machine learning classification models, we demonstrated that this mobile platform achieves the best performance for prediction of cyst counts using a bagging classifier with a classification accuracy of ~95%. We also demonstrated that this platform can be used for automated detection and counting of Giardia cysts in water samples taken from different sources (e.g. tap water, non-potable water, reagent grade water, and pond water) with a detection limit of ~12 cysts per 10 ml. This portable and cost-effective platform provides a powerful tool for rapid and sensitive detection of waterborne pathogens in water samples, even in resource-limited settings with minimal training of the users.