Abstract
This paper presents an innovative approach to curriculum learning, which is a technique used to train learning models. Curriculum learning is inspired by the way humans learn, starting with simple examples and gradually progressing to more challenging ones. There are currently two main types of curriculum learning: fixed curriculum generated by transfer learning, and self-paced learning based on loss functions. However, these methods have limitations that can hinder their effectiveness. To overcome these limitations, this article proposes a new approach called Dynamic Curriculum Learning via In-Domain Uncertainty (DCLU), which is derived from uncertainty estimation. The proposed approach utilizes a Dirichlet distribution classifier to obtain prediction and uncertainty estimates from the network, which can be used as a metric to quantify the difficulty level of the data. An uncertainty-aware sampling pacing function is also introduced to adapt the curriculum according to the difficulty metric. This new approach has been evaluated on two medical image datasets, and the results show that it outperforms other curriculum learning methods. The source code for this approach will be released at https://github.com/Joey2117/DCLU.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Appalaraju, S., Chaoji, V.: Image similarity using deep CNN and curriculum learning. arXiv preprint arXiv:1709.08761 (2017)
Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 41–48 (2009)
Cohen, J.P., Morrison, P., Dao, L.: Covid-19 image data collection. arXiv preprint arXiv:2003.11597 (2020)
Gal, Y., Ghahramani, Z.: Dropout as a bayesian approximation: representing model uncertainty in deep learning. In: International Conference on Machine Learning, pp. 1050–1059. PMLR (2016)
Ganesh, M.R., Corso, J.J.: Rethinking curriculum learning with incremental labels and adaptive compensation. arXiv preprint arXiv:2001.04529 (2020)
Gawlikowski, J., et al.: A survey of uncertainty in deep neural networks. arXiv preprint arXiv:2107.03342 (2021)
Hacohen, G., Weinshall, D.: On the power of curriculum learning in training deep networks. In: International Conference on Machine Learning, pp. 2535–2544. PMLR (2019)
Hardie, R.C., Ali, R., De Silva, M.S., Kebede, T.M.: Skin lesion segmentation and classification for ISIC 2018 using traditional classifiers with hand-crafted features. arXiv preprint arXiv:1807.07001 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hemmer, P., Kühl, N., Schöffer, J.: Deal: Deep evidential active learning for image classification. Deep Learn. Appl. 3, 171–192 (2022)
Jiang, L., Meng, D., Zhao, Q., Shan, S., Hauptmann, A.G.: Self-paced curriculum learning. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Kong, Y., Liu, L., Wang, J., Tao, D.: Adaptive curriculum learning. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 5067–5076 (2021)
Kotz, S., Balakrishnan, N., Johnson, N.L.: Continuous multivariate distributions. In: Models and applications, vol. 1. John Wiley & Sons (2004)
Kumar, M., Packer, B., Koller, D.: Self-paced learning for latent variable models. Adv. Neural Inf. Process. Syst. 23 (2010)
Lakshminarayanan, B., Pritzel, A., Blundell, C.: Simple and scalable predictive uncertainty estimation using deep ensembles. Adv. Neural Inf. Process. Syst. 30 (2017)
Morerio, P., Cavazza, J., Volpi, R., Vidal, R., Murino, V.: Curriculum dropout. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3544–3552 (2017)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: ICML (2010)
Ozturk, T., Talo, M., Yildirim, E.A., Baloglu, U.B., Yildirim, O., Acharya, U.R.: Automated detection of covid-19 cases using deep neural networks with x-ray images. Comput. Biol. Med. 121, 103792 (2020)
Platanios, E.A., Stretcu, O., Neubig, G., Poczos, B., Mitchell, T.M.: Competence-based curriculum learning for neural machine translation. arXiv preprint arXiv:1903.09848 (2019)
Qu, M., Tang, J., Han, J.: Curriculum learning for heterogeneous star network embedding via deep reinforcement learning. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 468–476 (2018)
Ruiter, D., España-Bonet, C., van Genabith, J.: Self-induced curriculum learning in neural machine translation (2019)
Sensoy, M., Kaplan, L., Kandemir, M.: Evidential deep learning to quantify classification uncertainty. Adv. Neural Inf. Process. Syst. 31 (2018)
Tschandl, P., Rosendahl, C., Kittler, H.: The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5(1), 1–9 (2018)
Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2097–2106 (2017)
Wang, X., Chen, Y., Zhu, W.: A survey on curriculum learning. IEEE Trans. Pattern Anal. Mach. Intell. (2021)
Weinshall, D., Cohen, G., Amir, D.: Curriculum learning by transfer learning: theory and experiments with deep networks. In: International Conference on Machine Learning, pp. 5238–5246. PMLR (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, C., Li, M., Peng, C., Lovell, B.C. (2023). Dynamic Curriculum Learning via In-Domain Uncertainty for Medical Image Classification. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14224. Springer, Cham. https://doi.org/10.1007/978-3-031-43904-9_72
Download citation
DOI: https://doi.org/10.1007/978-3-031-43904-9_72
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43903-2
Online ISBN: 978-3-031-43904-9
eBook Packages: Computer ScienceComputer Science (R0)
