Abstract
Digital holography is an imaging process that encodes the 3D information of objects into a single intensity image. In recent years, this technology has been used to detect and count various microscopic objects and has been applied in submersible equipment to monitor the in situ distribution of plankton. To count and classify plankton, conventional methods require a holographic reconstruction step to decode the hologram before identifying the objects. However, this iterative and time-consuming step must be performed at each frame of a video, which makes it difficult to support real-time processing. We propose a real-time object detection based approach that simultaneously performs the detection, classification and counting of all plankton within videos of raw holograms. Experiments show that our pipeline based on YOLOv5 and SORT is fast (44 FPS) and can accurately detect and identify the plankton among 13 classes (97.6% mAP@0.5, 92% MOTA). Our method can be implemented to detect and count other microscopic objects in raw holograms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bernardin, K., Stiefelhagen, R.: Evaluating multiple object tracking performance: the CLEAR MOT metrics. Eurasip J. Image Video Process. 2008 (2008). https://doi.org/10.1155/2008/246309
Bewley, A., Ge, Z., Ott, L., Ramos, F., Upcroft, B.: Simple online and realtime tracking. In: Proceedings - International Conference on Image Processing, ICIP 2016-August, pp. 3464–3468 (2016). https://doi.org/10.1109/ICIP.2016.7533003
Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: optimal speed and accuracy of object detection. ArXiv:abs/2004.10934 (2020)
Lam, H.S., Tsang, P.W.: Invariant classification of holograms of deformable objects based on deep learning. In: IEEE International Symposium on Industrial Electronics 2019-June, pp. 2392–2396 (2019). https://doi.org/10.1109/ISIE.2019.8781149
Latychevskaia, T.: Iterative phase retrieval in coherent diffractive imaging: practical issues. Appl. Opt. 57(25), 7187 (2018). https://doi.org/10.1364/ao.57.007187
Latychevskaia, T., Fink, H.W.: Practical algorithms for simulation and reconstruction of digital in-line holograms. Appl. Opt. 54(9), 2424 (2015)
Lee, S.J., Yoon, G.Y., Go, T.: Deep learning-based accurate and rapid tracking of 3D positional information of microparticles using digital holographic microscopy. Exper. Fluids 60(11) (2019). https://doi.org/10.1007/s00348-019-2818-y
Liu, X., Liu, X., Zhang, H., Fan, Y., Meng, H.: Research progress of digital holography in deep-sea in situ detection. Seventh Symposium on Novel Photoelectronic Detection Technology and Applications, vol. 11763, pp. 1760–1766 (2021)
Memmolo, P., et al.: On the holographic 3d tracking of in vitro cells characterized by a highly-morphological change. Opt. Express 20(27), 28485–28493 (2012). https://doi.org/10.1364/OE.20.028485
Memmolo, P., et al.: Recent advances in holographic 3D particle tracking. Adv. Opt. Photon. 7(4), 713 (2015). https://doi.org/10.1364/aop.7.000713
Mudanyali, O., Tseng, D., Oh, C.: Compact, light-weight and cost-effective microscope based on lensless incoherent holography for telemedicine applications. Lab. Chip 10(11), 1417–1428 (2010). https://doi.org/10.1039/c000453g.Compact
Padilla, R., Passos, W.L., Dias, T.L., Netto, S.L., Da Silva, E.A.: A comparative analysis of object detection metrics with a companion open-source toolkit. Electronics (Switzerland) 10(3), 1–28 (2021)
Persson, J., Mlder, A., Sven-Göran Pettersson, P., Alm, K.: Cell motility studies using digital holographic microscopy. Microsc. Sci Technol. Appl. Edu 4 (2010)
Picart, P., Montresor, S.: Digital Holography. Elsevier Inc. (2019). https://doi.org/10.1016/B978-0-12-815467-0.00005-0
Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2016-Decem, pp. 779–788 (2016)
Rivenson, Y., Zhang, Y., Günaydin, H., Teng, D., Ozcan, A.: Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl. 7(2), 17141 (2018)
Scholz, G., et al.: Continuous live-cell culture imaging and single-cell tracking by computational lensfree LED microscopy. Sensors (Switzerland) 19(5), 1–13 (2019). https://doi.org/10.3390/s19051234
Shimobaba, T., Kakue, T., Ito, T.: Convolutional Neural Network-Based Regression for Depth Prediction in Digital Holography. In: IEEE International Symposium on Industrial Electronics 2018-June, pp. 1323–1326 (2018)
Trujillo, C., Garcia-Sucerquia, J.: Automatic detection and counting of phase objects in raw holograms of digital holographic microscopy via deep learning. Opt. Lasers Eng. 120, 13–20 (2019)
Wu, Y., et al.: Label-free bioaerosol sensing using mobile microscopy and deep learning. ACS Photon. 5(11), 4617–4627 (2018). https://doi.org/10.1021/acsphotonics.8b01109
Wu, Y., Rivenson, Y., Zhang, Y., Günaydin, H., Lin, X., Ozcan, A.: Extended depth - of - field in holographic image reconstruction using deep learning based auto - focusing and phase - recovery. Optica 5, 704–710 (2018)
Zhang, Y., Lu, Y., Wang, H., Chen, P., Liang, R.: Automatic classification of marine plankton with digital holography using convolutional neural network. Opt. Laser Technol. 139(January), 106979 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Scherrer, R. et al. (2022). Real-Time Automatic Plankton Detection, Tracking and Classification on Raw Hologram. In: Chicco, D., et al. Computational Intelligence Methods for Bioinformatics and Biostatistics. CIBB 2021. Lecture Notes in Computer Science(), vol 13483. Springer, Cham. https://doi.org/10.1007/978-3-031-20837-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-20837-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20836-2
Online ISBN: 978-3-031-20837-9
eBook Packages: Computer ScienceComputer Science (R0)