Francisco Maria Calisto
Nuno Nunes
Jacinto C. Nascimento
ABSTRACT
This paper describes the field research, design and comparative deployment of a multimodal medical imaging user interface for breast screening. The main contributions described here are threefold: 1) The design of an advanced visual interface for multimodal diagnosis of breast cancer (BreastScreening); 2) Insights from the field comparison of Single-Modality vs Multi-Modality screening of breast cancer diagnosis with 31 clinicians and 566 images; and 3) The visualization of the two main types of breast lesions in the following image modalities: (i) MammoGraphy (MG) in both Craniocaudal (CC) and Mediolateral oblique (MLO) views; (ii) UltraSound (US); and (iii) Magnetic Resonance Imaging (MRI). We summarize our work with recommendations from the radiologists for guiding the future design of medical imaging interfaces.
- Faranak Aghaei, Seyedehnafiseh Mirniaharikandehei, Alan B Hollingsworth, Rebecca G Stoug, Melanie Pearce, Hong Liu, and Bin Zheng. 2018. Association between background parenchymal enhancement of breast MRI and BIRADS rating change in the subsequent screening. In Medical Imaging 2018: Imaging Informatics for Healthcare, Research, and Applications, Vol. 10579. International Society for Optics and Photonics, SPIE, Houston, Texas, United States, 105790R.Google Scholar
- Fabrizio Balducci and Paolo Buono. 2018. Building a Qualified Annotation Dataset for Skin Lesion Analysis Trough Gamification (AVI '18). ACM, New York, NY, USA, Article 36, 5 pages. https://doi.org/10.1145/3206505.3206555Google ScholarDigital Library
- Duncan P. Brumby, Ann Blandford, Anna L. Cox, Sandy J. J. Gould, and Paul Marshall. 2017. Understanding People: A Course on Qualitative and Quantitative HCI Research Methods (CHI EA '17). Association for Computing Machinery, New York, NY, USA, 1170--1173. https://doi.org/10.1145/3027063.3027103Google ScholarDigital Library
- Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "Hello AI: Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article Article 104 (Nov. 2019), 24 pages. https://doi.org/10.1145/3359206Google ScholarDigital Library
- Francisco Maria Calisto. 2017. Medical Imaging Multimodality Breast Cancer Diagnosis User Interface. Master's thesis. Instituto Superior Técnico, Avenida Rovisco Pais 1, 1049--001 Lisboa - Portugal (EU). A Medical Imaging study for Multimodality of Breast Cancer Diagnosis on a novel User Interface.Google Scholar
- Francisco Maria Calisto. 2019. Assistant Introduction: User Testing Guide For A Comparison Between Multi-Modality and AI-Assisted Systems. Technical Report. Madeira Interactive Technologies Institute. https://doi.org/10.13140/RG.2.2.16566.14403/1Google Scholar
- Francisco M. Calisto, Alfredo Ferreira, Jacinto C. Nascimento, and Daniel Gonçalves. 2017. Towards Touch-Based Medical Image Diagnosis Annotation (ISS '17). ACM, New York, NY, USA, 390--395. https://doi.org/10.1145/3132272.3134111Google ScholarDigital Library
- Nora Castner, Solveig Klepper, Lena Kopnarski, Fabian Hüttig, Constanze Keutel, Katharina Scheiter, Juliane Richter, Thérése Eder, and Enkelejda Kasneci. 2016. Overlooking: The Nature of Gaze Behavior and Anomaly Detection in Expert Dentists (MCPMD '18). ACM, New York, NY, USA, Article 8, 6 pages. https://doi.org/10.1145/3279810.3279845Google ScholarDigital Library
- Andy Cockburn and Carl Gutwin. 2019. Anchoring Effects and Troublesome Asymmetric Transfer in Subjective Ratings (CHI '19). Association for Computing Machinery, New York, NY, USA, Article Paper 362, 12 pages. https://doi.org/10.1145/3290605.3300592Google ScholarDigital Library
- Luigi Gallo, Giuseppe De Pietro, Antonio Coronato, and Ivana Marra. 2008. Toward a Natural Interface to Virtual Medical Imaging Environments (AVI '08). Association for Computing Machinery, New York, NY, USA, 429--432. https://doi.org/10.1145/1385569.1385651Google ScholarDigital Library
- Michael J. Gonzales, Vanice C. Cheung, and Laurel D. Riek. 2015. Designing Collaborative Healthcare Technology for the Acute Care Workflow (PervasiveHealth '15). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), Brussels, BEL, 145--152.Google Scholar
- Emilie L Henriksen, Jonathan F Carlsen, Ilse MM Vejborg, Michael B Nielsen, and Carsten A Lauridsen. 2018. The efficacy of using computer-aided detection (CAD) for detection of breast cancer in mammography screening: a systematic review. Acta Radiologica 60, 13--18 (2018), 0284185118770917.Google Scholar
- Takeo Igarashi, Naoyuki Shono, Taichi Kin, and Toki Saito. 2016. Interactive Volume Segmentation with Threshold Field Painting (UIST '16). ACM, New York, NY, USA, 403--413. https://doi.org/10.1145/2984511.2984537Google ScholarDigital Library
- Sébastien Jodogne. 2018. The Orthanc Ecosystem for Medical Imaging. Journal of Digital Imaging 31, 3 (01 Jun 2018), 341--352. https://doi.org/10.1007/s10278-018-0082-yGoogle ScholarCross Ref
- Panayiotis Koutsabasis and Chris K. Domouzis. 2016. Mid-Air Browsing and Selection in Image Collections (AVI '16). Association for Computing Machinery, New York, NY, USA, 21--27. https://doi.org/10.1145/2909132.2909248Google ScholarDigital Library
- Bongshin Lee, Arjun Srinivasan, John Stasko, Melanie Tory, and Vidya Setlur. 2018. Multimodal Interaction for Data Visualization (AVI '18). Association for Computing Machinery, New York, NY, USA, Article Article 11, 3 pages. https://doi.org/10.1145/3206505.3206602Google ScholarDigital Library
- Gabriel Maicas, Andrew P. Bradley, Jacinto C. Nascimento, Ian Reid, and Gustavo Carneiro. 2018. Training Medical Image Analysis Systems like Radiologists. In Medical Image Computing and Computer Assisted Intervention - MICCAI 2018, Alejandro F. Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-López, and Gabor Fichtinger (Eds.). Springer Intern. Publishing, Cham, 546--554.Google ScholarDigital Library
- Gabriel Maicas, Andrew P. Bradley, Jacinto C. Nascimento, Ian Reid, and Gustavo Carneiro. 2019. Deep Reinforcement Learning for Detecting Breast Lesions from DCE-MRI. Springer Intern. Publishing, Cham, 163--178. https://doi.org/10.1007/978-3-030-13969-8_8Google Scholar
- D. O. Medley, C. Santiago, and J. C. Nascimento. 2019. Segmenting The Left Ventricle In Cardiac In Cardiac MRI: From Handcrafted To Deep Region Based Descriptors. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, Venice, Italy, 644--648.Google Scholar
- J. C. Nascimento and G. Carneiro. 2019. One shot segmentation: unifying rigid detection and non-rigid segmentation using elastic regularization. IEEE Transactions on Pattern Analysis and Machine Intelligence 1, 2922959 (2019), 1--1.Google Scholar
- Kwan Hoong Ng, Jeannie Hsiu Ding Wong, and Sock Keow Tan. 2017. Technical Specifications of Medical Imaging Equipment. In Defining the Medical Imaging Requirements for a Rural Health Center. Springer, Singapore, 49--74.Google Scholar
- Louise Oram, Karon MacLean, Philippe Kruchten, and Bruce Forster. 2014. Crafting Diversity in Radiology Image Stack Scrolling: Control and Annotations (DIS '14). ACM, New York, NY, USA, 567--576. https://doi.org/10.1145/2598510.2598585Google ScholarDigital Library
- Soraia Figueiredo Paulo, Filipe Relvas, Hugo Nicolau, Yosra Rekik, Vanessa Machado, João Botelho, José João Mendes, Laurent Grisoni, Joaquim Jorge, and Daniel Simões Lopes. 2019. Touchless interaction with medical images based on 3D hand cursors supported by single-foot input: A case study in dentistry. Journal of Biomedical Informatics 100 (2019), 103316. https://doi.org/10.1016/j.jbi.2019.103316Google ScholarDigital Library
- Marco Porta. 2006. Browsing Large Collections of Images through Unconventional Visualization Techniques (AVI '06). Association for Computing Machinery, New York, NY, USA, 440--444. https://doi.org/10.1145/1133265.1133354Google ScholarDigital Library
- Luís Rosado, Maria João M. Vasconcelos, Fernando Correia, and Nuno Costa. 2015. A Novel Framework for Supervised Mobile Assessment and Risk Triage of Skin Lesions (PervasiveHealth '15). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium, Belgium, 266--267. http://dl.acm.org/citation.cfm?id=2826165.2826213Google Scholar
- C. Santiago, J. C. Nascimento, and J. S. Marques. 2017. Fast and accurate segmentation of the LV in MR volumes using a deformable model with dynamic programming. In 2017 IEEE International Conference on Image Processing (ICIP). IEEE, Beijing, China, 1747--1751.Google Scholar
- Carlos Santiago, Jacinto C. Nascimento, and Jorge S. Marques. 2018. Fast segmentation of the left ventricle in cardiac MRI using dynamic programming. Computer Methods and Programs in Biomedicine 154 (2018), 9--23. https://doi.org/10.1016/j.cmpb.2017.10.028Google ScholarCross Ref
- Maurício Sousa, Daniel Mendes, Soraia Paulo, Nuno Matela, Joaquim Jorge, and Daniel Simões Lopes. 2017. VRRRRoom: Virtual Reality for Radiologists in the Reading Room (CHI '17). ACM, New York, NY, USA, 4057--4062. https://doi.org/10.1145/3025453.3025566Google ScholarDigital Library
- D.A. Spak, J.S. Plaxco, L. Santiago, M.J. Dryden, and B.E. Dogan. 2017. BI-RADS® fifth edition: A summary of changes. Diagnostic and Interventional Imaging 98, 3 (2017), 179--190. https://doi.org/10.1016/j.diii.2017.01.001Google ScholarCross Ref
- preethi Srinivas. 2015. Modeling Clinical Workflow in Daily ICU Rounds to Support Task-Based Patient Monitoring and Care (CSCW'15 Companion). Association for Computing Machinery, New York, NY, USA, 105--108. https://doi.org/10.1145/2685553.2699332Google ScholarDigital Library
- Mari Tyllinen, Johanna Kaipio, Tinja Lääveri, and Marko H.T. Nieminen. 2016. We Need Numbers!: Heuristic Evaluation During Demonstrations (HED) for Measuring Usability in IT System Procurement (CHI '16). ACM, New York, NY, USA, 4129--4141. https://doi.org/10.1145/2858036.2858570Google ScholarDigital Library
- Trinity Urban, Erik Ziegler, Rob Lewis, Chris Hafey, Cheryl Sadow, Annick D Van den Abbeele, and Gordon J Harris. 2017. LesionTracker: Extensible open-source zero-footprint web viewer for cancer imaging research and clinical trials. Cancer research 77, 21 (2017), e119-e122.Google Scholar
- Boris W. van Schooten, Elisabeth M. A. G. van Dijk, Elena Zudilova-Seinstra, Avan Suinesiaputra, and Johan H. C. Reiber. 2010. The Effect of Stereoscopy and Motion Cues on 3D Interpretation Task Performance (AVI '10). Association for Computing Machinery, New York, NY, USA, 167--170. https://doi.org/10.1145/1842993.1843023Google ScholarDigital Library
- Harini Veeraraghavan, Brittany Z. Dashevsky, Natsuko Onishi, Meredith Sadinski, Elizabeth Morris, Joseph O. Deasy, and Elizabeth J. Sutton. 2018. Appearance Constrained Semi-Automatic Segmentation from DCE-MRI is Reproducible and Feasible for Breast Cancer Radiomics: A Feasibility Study. Scientific Reports 8, 1 (2018), 4838. https://doi.org/10.1038/s41598-018-22980-9Google ScholarCross Ref
- Lauren Wilcox, Dan Morris, Desney Tan, and Justin Gatewood. 2010. Designing Patient-centric Information Displays for Hospitals (CHI '10). ACM, New York, NY, USA, 2123--2132. https://doi.org/10.1145/1753326.1753650Google ScholarDigital Library
- Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures (CHI '11). ACM, New York, NY, USA, 143--146. https://doi.org/10.1145/1978942.1978963Google ScholarDigital Library
- Paweł Woundefinedniak, Andrzej Romanowski, Asim Evren Yantaç, and Morten Fjeld. 2014. Notes from the Front Lines: Lessons Learnt from Designing for Improving Medical Imaging Data Sharing (NordiCHI '14). Association for Computing Machinery, New York, NY, USA, 381--390. https://doi.org/10.1145/2639189.2639256Google ScholarDigital Library
- Zizhao Zhang, Lin Yang, and Yefeng Zheng. 2018. Translating and Segmenting Multimodal Medical Volumes With Cycle- and Shape-Consistency Generative Adversarial Network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Salt Lake City, Utah, USA, 9242--9251.Google Scholar
Index Terms
- BreastScreening: On the Use of Multi-Modality in Medical Imaging Diagnosis
Recommendations
Digital Breast Tomosynthesis Reconstruction Techniques in Healthcare Systems: A Review
Bioinformatics and Biomedical EngineeringAbstractDigital Breast Tomosynthesis (DBT) images are widely used to increase breast cancer detection and reduce recall rates in healthcare systems for breast cancer detection. In the field of medical imaging, computer-aided diagnosis (CAD) systems are ...
Enhancing Clinical Support for Breast Cancer with Deep Learning Models Using Synthetic Correlated Diffusion Imaging
Applications of Medical Artificial IntelligenceAbstractBreast cancer is the second most common type of cancer in women in Canada and the United States, representing over 25% of all new female cancer cases. As such, there has been immense research and progress on improving screening and clinical ...
Markov random field driven region-based active contour model (MaRACel): application to medical image segmentation
MICCAI'10: Proceedings of the 13th international conference on Medical image computing and computer-assisted intervention: Part IIIIn this paper we present a Markov random field (MRF) driven region-based active contour model (MaRACel) for medical image segmentation. State-of-the-art region-based active contour (RAC) models assume that every spatial location in the image is ...
Comments