Abstract
Medical imaging is a critical component of clinical decision-making, patient diagnosis, treatment planning, intervention, and therapy. However, due to the shortage of qualified radiologists, there is an increasing burden on healthcare practitioners, which underscores the need to develop reliable automated methods. Despite the development of novel computational techniques, interpreting medical images remains challenging due to noise and varying acquisition conditions. One promising solution to improve the reliability and accuracy of automated medical image analysis is Interactive Machine Learning (IML), which integrates human expertise into the model training process. Active learning (AL) is an important IML technique that can iteratively query for informative samples to be labeled by humans, leading to more data-efficient learning. To fully leverage the potential of active learning, however, it is crucial to understand the optimal setup for different components of an AL system. This paper presents an evaluation of the effectiveness of different combinations of data representation, model capacity, and query strategy for active learning systems designed for medical image classification tasks. The results of this evaluation show that employing raw image representations as input, in conjunction with a ResNet50 model and margin-based queries, yields more reliable and accurate automated methods for medical image analysis.
This publication has emanated from research conducted with the financial support of Science Foundation Ireland under Grant number 18/CRT/6183. For the purpose of Open Access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
ResNet50 model pre-trained on the Imagenet dataset. Available at: https://pytorch.org/vision/main/models/generated/torchvision.models.resnet50.html.
References
Aghdam, H.H., et al.: Active learning for deep detection neural networks. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3672–3680 (2019)
Amershi, S., et al.: Power to the people: the role of humans in interactive machine learning. AI Mag. 350(4), 105–120 (2014)
Angluin, D.: Queries and concept learning. Mach. Learn. 2, 319–342 (1988)
Arrieta, A.B., et al.: Explainable artificial intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI. Inf. Fusion 58, 82–115 (2020)
Atlas, L., Cohn, D., Ladner, R.: Training connectionist networks with queries and selective sampling. In: Advances in Neural Information Processing Systems, vol. 2 (1989)
Baum, E.B.: Neural net algorithms that learn in polynomial time from examples and queries. IEEE Trans. Neural Netw. 20(1), 5–19 (1991)
Belkin, M., et al.: Reconciling modern machine-learning practice and the classical bias-variance trade-off. In: Proceedings of the National Academy of Sciences, vol. 1160. no. (32), pp. 15849–15854 (2019)
Berg, S., et al.: Ilastik: interactive machine learning for (bio) image analysis. Nat. Methods 160(12), 1226–1232 (2019)
Branson, S., Perona, P., Belongie, S.: Strong supervision from weak annotation: Interactive training of deformable part models. In: 2011 International Conference on Computer Vision, pp. 1832–1839. IEEE (2011)
Budd, S., Robinson, E.C., Kainz, B.: A survey on active learning and human-in-the-loop deep learning for medical image analysis. Med. Image Anal. 71, 102062 (2021)
Cho, J.W., et al.: MCDAL: maximum classifier discrepancy for active learning. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Chung, M.-H., et al.: Interactive machine learning for data exfiltration detection: Active learning with human expertise. In: 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 280–287. IEEE (2020)
Cohn, D., Atlas, L., Ladner, R.: Improving generalization with active learning. Mach. Learn. 15, 201–221 (1994)
Dagan, I., Engelson, S.P.: Committee-based sampling for training probabilistic classifiers. In: Machine Learning Proceedings 1995, pp. 150–157. Elsevier (1995)
Dasgupta, S., Kalai, A.T., Tauman, A.: Analysis of perceptron-based active learning. J. Mach. Learn. Res. 100(2) 2009
Du, X., Zhong, D., Shao, H.: Building an active palmprint recognition system. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1685–1689. IEEE (2019)
Dudley, J.J., Kristensson, P.O.: A review of user interface design for interactive machine learning. ACM Trans. Interact. Intell. Syst. (TiiS) 80(2), 1–37 (2018)
Fails, J.A., Olsen Jr, D.R.: Interactive machine learning. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, pp. 39–45 (2003)
Goyal, P., et al.: Accurate, large minibatch SGD: training imageNet in 1 hour. arXiv preprint arXiv:1706.02677 (2017)
He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Ho, Y., Wookey, S.: The real-world-weight cross-entropy loss function: modeling the costs of mislabeling. IEEE Access 8, 4806–4813 (2019)
Holzinger, A.: Interactive machine learning for health informatics: when do we need the human-in-the-loop? Brain Inform. 30(2), 119–131 (2016)
Holzinger, A., Plass, M., Holzinger, K., Crişan, G.C., Pintea, C.-M., Palade, V.: Towards interactive machine learning (iML): applying ant colony algorithms to solve the traveling salesman problem with the human-in-the-loop approach. In: Buccafurri, F., Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds.) CD-ARES 2016. LNCS, vol. 9817, pp. 81–95. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45507-5_6
Hu, R., Namee, B.M., Delany, S.J.: Sweetening the dataset: using active learning to label unlabelled datasets. In: Proceedings of AICS, vol. 8, pp. 53–62 (2008)
Jiang, L., Liu, S., Chen, C., Recent research advances on interactive machine learning: Recent research advances on interactive machine learning. J. Vis. 22, 401–417 (2019)
Kelly, D., et al.: Methods for evaluating interactive information retrieval systems with users. Found. Trends® Inf. Retrieval, 30(1–2), 1–224 (2009)
Kim, M., et al.: Topiclens: efficient multi-level visual topic exploration of large-scale document collections. IEEE Trans. Vis. Comput. Graph. 230(1), 151–160 (2016)
King, R.D., et al.: The automation of science. Science 3240(5923), 85–89 (2009)
Kose, I., Gokturk, M., Kilic, K.: An interactive machine-learning-based electronic fraud and abuse detection system in healthcare insurance. Appl. Soft Comput. 36, 283–299 (2015)
Lewis, D.D., Catlett, J.: Heterogeneous uncertainty sampling for supervised learning. In: Machine Learning Proceedings 1994, pp. 148–156. Elsevier (1994)
Lin, X., Parikh, D.: Active learning for visual question answering: an empirical study. arXiv preprint arXiv:1711.01732 (2017)
Liu, Z., et al.: Patterns and sequences: Interactive exploration of clickstreams to understand common visitor paths. IEEE Trans. Vis. Comput. Graph. 230(1), 321–330 (2016)
McCallum, A., Nigam, K., et al.: Employing EM and pool-based active learning for text classification. In: ICML, vol. 98, pp. 350–358. Citeseer (1998)
Rawat, S., et al.: How useful is image-based active learning for plant organ segmentation? Plant Phenomics, 2022 (2022)
Sabato, S., Hess, T.: Interactive algorithms: pool, stream and precognitive stream. J. Mach. Learn. Res. 18, 1–39 (2017)
Scheffer, T., Decomain, C., Wrobel, S.: Active hidden Markov models for information extraction. In: Hoffmann, F., Hand, D.J., Adams, N., Fisher, D., Guimaraes, G. (eds.) IDA 2001. LNCS, vol. 2189, pp. 309–318. Springer, Heidelberg (2001). https://doi.org/10.1007/3-540-44816-0_31
Schumann, R., Rehbein, I.: Active learning via membership query synthesis for semi-supervised sentence classification. In: Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pp. 472–481 (2019)
Settles, B.: Active learning literature survey (2009)
Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 270(3), 379–423 (1948)
Smith, S.L., et al.: Don’t decay the learning rate, increase the batch size. arXiv preprint arXiv:1711.00489 (2017)
Teso, S., Kersting, K.: Explanatory interactive machine learning. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 239–245 (2019)
The Royal College of Radiologists. Rcr clinical radiology census report 2021 (2021). https://www.rcr.ac.uk/clinical-radiology/rcr-clinical-radiology-census-report-2021 . Accessed 27 Feb 2023
Wang, Y., et al.: Efficient DNN training with knowledge-guided layer freezing. arXiv preprint arXiv:2201.06227 (2022)
Yang, J., et al.: MedMNIST v2-a large-scale lightweight benchmark for 2D and 3D biomedical image classification. Sci. Data 100(1), 41 (2023)
Betül Yüce, A., Yaslan, Y.: A disagreement based co-active learning method for sleep stage classification. In: 2016 International Conference on Systems, Signals and Image Processing (IWSSIP), pp. 1–4. IEEE (2016)
Zhan, X., et al.: A comparative survey of deep active learning. arXiv preprint arXiv:2203.13450 (2022)
Zhang, Y., Lease, M., Wallace, B.: Active discriminative text representation learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 31 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Parise, A., Mac Namee, B. (2023). Exploring Optimal Configurations in Active Learning for Medical Imaging. In: Bramer, M., Stahl, F. (eds) Artificial Intelligence XL. SGAI 2023. Lecture Notes in Computer Science(), vol 14381. Springer, Cham. https://doi.org/10.1007/978-3-031-47994-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-031-47994-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-47993-9
Online ISBN: 978-3-031-47994-6
eBook Packages: Computer ScienceComputer Science (R0)