Abstract
Metainformation is a common companion to biomedical images. However, this potentially powerful additional source of signal from image acquisition has had limited use in deep learning methods, for semantic segmentation in particular. Here, we incorporate metadata by employing a channel modulation mechanism in convolutional networks and study its effect on semantic segmentation tasks. We demonstrate that metadata as additional input to a convolutional network can improve segmentation results while being inexpensive in implementation as a nimble add-on to popular models. We hypothesize that this benefit of metadata can be attributed to facilitating multitask switching. This aspect of metadata-driven systems is explored and discussed in detail.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ali, M.A.S., et al.: ArtSeg: rapid artifact segmentation and removal in brightfield cell microscopy images, January 2022
Ali, M.A.S., Misko, O., Salumaa, S.O., Papkov, M., Palo, K., Fishman, D., Parts, L.: Evaluating very deep convolutional neural networks for nucleus segmentation from brightfield cell microscopy images. SLAS Discov. 26(9), 1125–1137 (2021)
Amer, A., Ye, X., Zolgharni, M., Janan, F.: ResDUnet: residual dilated UNet for left ventricle segmentation from echocardiographic images. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2020, 2019–2022 (2020)
Beucher, S.: Use of watersheds in contour detection. In: Proceedings of International Workshop on Image Processing, Sept. 1979, pp. 17–21 (1979)
Bhalgat, Y., Shah, M., Awate, S.: Annotation-cost minimization for medical image segmentation using suggestive mixed supervision fully convolutional networks, December 2018
Brocal, G.M., Peeters, G.: Conditioned-U-Net: introducing a control mechanism in the U-Net for multiple source separations. In: Proceedings of the 20th International Society for Music Information Retrieval Conference. Zenodo (2019)
De Vries, H., Strub, F., Mary, J., Larochelle, H., Pietquin, O., Courville, A.C.: Modulating early visual processing by language. Adv. Neural Inf. Process. Syst. 30 (2017)
Georgiou, T., Liu, Y., Chen, W., Lew, M.: A survey of traditional and deep learning-based feature descriptors for high dimensional data in computer vision. Int. J. Multimed. Inf. Retrieval 9(3), 135–170 (2020)
Gessert, N., Nielsen, M., Shaikh, M., Werner, R., Schlaefer, A.: Skin lesion classification using ensembles of multi-resolution EfficientNets with meta data. MethodsX 7, 100864 (2020)
Greenspan, H., van Ginneken, B., Summers, R.M.: Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Trans. Med. Imaging 35(5), 1153–1159 (2016)
Gros, C., Lemay, A., Vincent, O., Rouhier, L., Bourget, M.H., Bucquet, A., Cohen, J., Cohen-Adad, J.: Ivadomed: a medical imaging deep learning toolbox. J. Open Source Softw. 6(58), 2868 (2021)
Heller, N., et al.: The state of the art in kidney and kidney tumor segmentation in contrast-enhanced CT imaging: results of the KiTS19 challenge. Med. Image Anal. 67, 101821 (2021)
Hu, J., Shen, L., Albanie, S., Sun, G., Wu, E.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 2011–2023 (2020)
Kawahara, J., Daneshvar, S., Argenziano, G., Hamarneh, G.: 7-point checklist and skin lesion classification using Multi-Task Multi-Modal neural nets. IEEE J. Biomed. Health Inform., April 2018
Khan, S., Sajjad, M., Hussain, T., Ullah, A., Imran, A.S.: A review on traditional machine learning and deep learning models for WBCs classification in blood smear images. IEEE Access 9, 10657–10673 (2021)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization, December 2014
Lemay, A., Gros, C., Vincent, O., Liu, Y., Cohen, J.P., Cohen-Adad, J.: Benefits of linear conditioning for segmentation using metadata. In: Heinrich, M., Dou, Q., de Bruijne, M., Lellmann, J., Schläfer, A., Ernst, F. (eds.) Proceedings of the Fourth Conference on Medical Imaging with Deep Learning. Proceedings of Machine Learning Research, vol. 143, pp. 416–430. PMLR (2021)
Litjens, G., et al.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)
Otsu, N.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)
Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Advances in Neural Information Processing Systems, pp. 8026–8037 (2019)
Perez, E., Strub, F., de Vries, H., Dumoulin, V., Courville, A.: FiLM: Visual reasoning with a general conditioning layer. AAAI 32(1), April 2018
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roy, S.K., Dubey, S.R., Chatterjee, S., Chaudhuri, B.B.: FuSENet: fused squeeze-and-excitation network for spectral-spatial hyperspectral image classification (2020)
Smith, L.N.: Cyclical learning rates for training neural networks. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 464–472, March 2017
Van der Walt, S., Schönberger, J.L., Nunez-Iglesias, J., Boulogne, F., Warner, J.D., Yager, N., Gouillart, E., Yu, T.: scikit-image: image processing in python. PeerJ 2, e453 (2014)
Woo, S., Park, J., Lee, J.-Y., Kweon, I.S.: CBAM: convolutional block attention module. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 3–19. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_1
Acknowledgements
This work was funded by Revvity, Inc. (previously known as PerkinElmer Inc., VLTAT19682), and Wellcome Trust (206194). We thank High Performance Computing Center of the Institute of Computer Science at the University of Tartu for the provided computing power.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Plutenko, I., Papkov, M., Palo, K., Parts, L., Fishman, D. (2024). Metadata Improves Segmentation Through Multitasking Elicitation. In: Koch, L., et al. Domain Adaptation and Representation Transfer. DART 2023. Lecture Notes in Computer Science, vol 14293. Springer, Cham. https://doi.org/10.1007/978-3-031-45857-6_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-45857-6_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45856-9
Online ISBN: 978-3-031-45857-6
eBook Packages: Computer ScienceComputer Science (R0)
