Abstract
Classification of emphysema patterns is believed to be useful for improved diagnosis and prognosis of chronic obstructive pulmonary disease. Emphysema patterns can be assessed visually on lung CT scans. Visual assessment is a complex and time-consuming task performed by experts, making it unsuitable for obtaining large amounts of labeled data. We investigate if visual assessment of emphysema can be framed as an image similarity task that does not require expert. Substituting untrained annotators for experts makes it possible to label data sets much faster and at a lower cost. We use crowd annotators to gather similarity triplets and use t-distributed stochastic triplet embedding to learn an embedding. The quality of the embedding is evaluated by predicting expert assessed emphysema patterns. We find that although performance varies due to low quality triplets and randomness in the embedding, we still achieve a median \(F_1\) score of 0.58 for prediction of four patterns.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., Navab, N.: Aggnet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE TMI 35(5), 1313–1321 (2016)
Cheplygina, V., Perez-Rovira, A., Kuo, W., Tiddens, H.A.W.M., de Bruijne, M.: Early experiences with crowdsourcing airway annotations in chest CT. In: Carneiro, G., Mateus, D., Peter, L., Bradley, A., Tavares, J.M.R.S., Belagiannis, V., Papa, J.P., Nascimento, J.C., Loog, M., Lu, Z., Cardoso, J.S., Cornebise, J. (eds.) LABELS/DLMIA 2016. LNCS, vol. 10008, pp. 209–218. Springer, Cham (2016). doi:10.1007/978-3-319-46976-8_22
From the Global Strategy for the Diagnosis, Management and Prevention of COPD, Global Initiative for Chronic Obstructive Lung Disease (GOLD) (2015)
Kovashka, A., Russakovsky, O., Fei-Fei, L., Grauman, K.: Crowdsourcing in computer vision. Found. Trends. Comput. Graph. Vis. 10(3), 177–243 (2016)
Lo, P., Sporring, J., Ashraf, H., Pedersen, J.J.H.: Vessel-guided airway tree segmentation: a voxel classification approach. Med. Image Anal. 14(4), 527–538 (2010)
Maier-Hein, L., Mersmann, S., Kondermann, D., Bodenstedt, S., Sanchez, A., Stock, C., Kenngott, H.G., Eisenmann, M., Speidel, S.: Can masses of non-experts train highly accurate image classifiers? In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) MICCAI 2014. LNCS, vol. 8674, pp. 438–445. Springer, Cham (2014). doi:10.1007/978-3-319-10470-6_55
McFee, B., Lanckriet, G.: Learning multi-modal similarity. J. Mach. Learn. Res. 12, 491–523 (2011)
Mitry, D., Zutis, K., Dhillon, B., Peto, T., Hayat, S., Khaw, K.-T., Morgan, J.E., Moncur, W., Trucco, E., Foster, P.J.: The accuracy and reliability of crowdsource annotations of digital retinal images. Trans. Vis. Sci. Technol. 5(5), 6–6 (2016)
Nishio, M., Nakane, K., Kubo, T., Yakami, M., Emoto, Y., Nishio, M., Togashi, K.: Automated prediction of emphysema visual score using homology-based quantification of low-attenuation lung region. PLOS ONE 12(5), 1–12 (2017)
Pedersen, J.H., Ashraf, H., Dirksen, A., Bach, K., Hansen, H., Toennesen, P., Thorsen, H., Brodersen, J., Skov, B.G., Døssing, M., Mortensen, J., Richter, K., Clementsen, P., Seersholm, N.: The Danish randomized lung cancer CT screening trial-overall design and results of the prevalence round. J. Thorac. Oncol. 4(5), 608–614 (2009)
Nyboe Ørting, S., Petersen, J., Wille, M., Thomsen, L., de Bruijne, M.: Quantifying emphysema extent from weakly labeled CT scans of the lungs using label proportions learning. In: MICCAI PIA, pp. 31–42. CreateSpace Independent Publishing Platform (2016)
van der Maaten, L., Weinberger, K.: Stochastic triplet embedding. In: IEEE MLSP, pp. 1–6 (2012)
Wah, C., Van Horn, G., Branson, S., Maji, S., Perona, P., Belongie, S.: Similarity comparisons for interactive fine-grained categorization. In: IEEE CVPR, pp. 859–866 (2014)
Wille, M.M., Thomsen, L.H., Dirksen, A., Petersen, J., Pedersen, J.H., Shaker, S.B.: Emphysema progression is visually detectable in low-dose CT in continuous but not in former smokers. Eur. Radiol. 24(11), 2692–2699 (2014)
Acknowledgments
We would like to thank family, friends and coworkers at the University of Copenhagen, Erasmus MC - University Medical Center Rotterdam, Eindhoven University of Technology, and the start-up understand.ai for their help in testing prototype versions of the crowdsourcing tasks. This study was financially supported by the Danish Council for Independent Research (DFF) and the Netherlands Organization for Scientific Research (NWO).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ørting, S.N., Cheplygina, V., Petersen, J., Thomsen, L.H., Wille, M.M.W., de Bruijne, M. (2017). Crowdsourced Emphysema Assessment. In: Cardoso, M., et al. Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis. LABELS STENT CVII 2017 2017 2017. Lecture Notes in Computer Science(), vol 10552. Springer, Cham. https://doi.org/10.1007/978-3-319-67534-3_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-67534-3_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-67533-6
Online ISBN: 978-3-319-67534-3
eBook Packages: Computer ScienceComputer Science (R0)