Original ArticleEvaluating Artificial Intelligence Systems to Guide Purchasing Decisions
Introduction
Considering the purchase of an artificial intelligence (AI) solution is similar to any health technology solution. The process starts with defining the problem at hand and evaluating the environment surrounding the problem including workflow and cost. For example, if critical conditions such as pulmonary emboli or stroke are not being diagnosed quickly enough for optimal patient care, an AI solution that detects these entities and effectively accelerates diagnosis could improve patient outcomes. On the contrary, purchasing an AI solution simply because AI is a hot topic, or because one generally thinks AI will improve the practice of radiology, is not recommended because it may consume resources without benefit to clinical practice. If a tangible problem is identified and AI tools targeting that problem are commercially available, then one can begin to evaluate whether purchasing an AI tool makes sense. We outline a stepwise process for evaluating AI tool purchases.
Section snippets
Initial Evaluation
Once an AI tool is identified to address a tangible problem, one of the first things to consider is how well this tool might work in practice—its performance characteristics. Performance is often reported using a receiver operating characteristic curve in addition to sensitivity and specificity [1]. As with any research study, the quality of evidence should be evaluated. Performance measurements from the clinical setting are more valuable than highly controlled (and potentially biased) research
Infrastructure Choices
AI purchasers face a decision between two strategies to AI implementation: best of breed and platform. Simply treating each AI algorithm as a stand-alone application gives the radiology practice ultimate flexibility to choose whatever vendors or tools they would like to work with, sometimes referred to as best of breed. However, this approach is least scalable because each vendor must go through a separate demand management, planning, security evaluation, and system integration each time a tool
Financial Considerations
There are several different business models for deploying AI, ranging from large capital purchases to ongoing subscription fees. To date, most AI vendors have their fee structure tied to either radiologist consumption in a per-click model or to examination volume. Flat rates or perpetual licenses are less prevalent. Implementers should also negotiate with vendors on whether future versions or improvements to the AI tool will be included or require additional purchases in the future. The best
Quality and Safety
Successful application of AI to our patients requires not only a thoughtful purchasing process but also designing appropriate monitoring based on the effect of the AI model in question. Partnership between informatics and quality and safety leaders is crucial because monitoring requires expertise from both subspecialties.
As described previously, an important aspect of safety is consideration of whether AI reinforces existing health care disparities, which can be insidious and difficult to
Conclusion
Purchasing AI systems requires close coordination with many stakeholder groups and consideration of system performance, validation, IT requirements, cost, as well as quality and safety. A summary checklist for practices evaluating purchasing AI is available in Figure 3.
Take-Home Points
- ▪
AI implementations should address a well-defined problem in the radiology practice.
- ▪
Ease of use and workflow integration quality should be assessed before and after implementation.
- ▪
AI models should be monitored for patient safety, including unintended bias and especially the potential for reinforcing health care disparities.
- ▪
Impact on IT infrastructure and cost should be included in return-on-investment calculations.
References (10)
- et al.
ROC, LROC, FROC, AFROC: an alphabet soup
J Am Coll Radiol
(2009) - et al.
Inconsistent performance of deep learning models on mammogram classification
J Am Coll Radiol
(2020) Why deep-learning AIs are so easy to fool
Nature
(2019)Amazon scraps secret AI recruiting tool that showed bias against women
Reuters
(2018)- et al.
The case for user-centered artificial intelligence in radiology
Radiol Artif Intell
(2020)
Cited by (18)
Artificial Intelligence in Radiology: Opportunities and Challenges
2024, Seminars in Ultrasound, CT and MRIKeeping Patient Data Secure in the Age of Radiology Artificial Intelligence: Cybersecurity Considerations and Future Directions
2023, Journal of the American College of RadiologyAcademic Radiology Departments Should Lead Artificial Intelligence Initiatives
2023, Academic RadiologyCitation Excerpt :Because clinical use of AI is in its infancy and continually evolving, evaluation of AI solutions for potential purchase and deployment, as well as governance of AI algorithms in clinical deployment, is challenging, as AI algorithms frequently cross imaging modalities and/or radiology sections. Regarding our first point, several frameworks have been proposed to guide academic radiology departments in evaluating potential AI algorithms for purchase and clinical deployment, including guidance on external validation of AI solutions on institutional data, technology requirements of AI solutions, and cost-benefit ratio of these solutions (18–20). Regarding our second point, each institution and department will need to consider pre-existing models of governance for imaging information technology (IT) and abide by these conventions.
Updates in Artificial Intelligence for Breast Imaging
2022, Seminars in RoentgenologyCitation Excerpt :The algorithm may be integrated into practice via a platform through which other AI vendors could also deploy their tools or integrated into practice as a standalone application, which offers maximum flexibility but is less scalable.66 With regard to financial considerations, purchase of an AI tool may require a one-time fee, pay-per-use, or an ongoing subscription.66,67 Additional fees may be incurred for installation, maintenance, future updates, and training of new users.
Evaluation and Real-World Performance Monitoring of Artificial Intelligence Models in Clinical Practice: Try It, Buy It, Check It
2021, Journal of the American College of RadiologyCitation Excerpt :Review articles outlining the functionality and availability of FDA-cleared AI models for clinical use in radiology will certainly be useful [2,4] but will likely become rapidly dated as the number of models cleared for clinical use escalates. Other authors have developed checklists for evaluation of AI models that can be used to inform AI purchase decisions, and these will all be valuable to end users as well [5,6]. These checklists emphasize the importance of good machine learning practices in the training process, an assessment of the demographics of the patient population used for training, the representativeness of the validation data used to define the expected clinical performance of the model, and the mechanisms for longitudinal performance monitoring [6].
Artificially Practical in Every Way
2020, Journal of the American College of Radiology
Dr Filice reports other from BunkerHill, Inc, grants from Nvidia, Inc, outside the submitted work. Dr Mongan reports grants and personal fees from GE, personal fees from Siemens, other from Nuance, grants from Enlitic, other from University of California San Francisco Radiology and Biomedical Imaging, outside the submitted work. Dr Kohli reports nonfinancial support from Society of Imaging Informatics in Medicine, personal fees from Gilead, personal fees from Honor Health, nonfinancial support from Dy Patil University, nonfinancial support from RSNA. Dr Kohli and Dr Mongan are employed by the University of California San Francisco as nonpartner employees. Dr Filice is a MedStar Health nonpartner employee.