Abstract
Ideally, the outcome of any CAD performance assessment should predict how well the system would work if used clinically. In principle, if the selection process draws cases that are “representative” of the general patient population, the study design will be unbiased. In this study we explored the effect of stratified sampling on stand-alone and radiologists’ performance using data from an observer study. Although our database was relatively small, 50 cancer cases, no meaningful difference in performance was measured among different stratified sampling schemes or against the whole dataset nor was there any difference in the variance in the measured performance metrics. These results cast doubts on the usefulness of requiring stratified sampling, whose added cost does not seem to be justifiable without empirical evidence. We believe that it is more important to specify how cases should be collected than try to define the range and frequency of the characteristics of patients and cancers to be included the dataset, which we suspect to be prone to actually produce unrealistic samples.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Efron, B., Tibshirani, R.J.: An Introduction to the Bootstrap. Chapman and Hall, New York (1993)
Gilbert, F.J., Astley, S.M., McGee, M.A., Gillan, M.G., Boggis, C.R., Griffiths, P.M., Duffy, S.W.: Single reading with computer-aided detection and double reading of screening mammograms in the United Kingdom National Breast Screening Program. Radiology 241, 47–53 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nishikawa, R.M., Pesce, L.L. (2010). Stratified Sampling for Case Selection Criteria for Evaluating CAD. In: Martí, J., Oliver, A., Freixenet, J., Martí, R. (eds) Digital Mammography. IWDM 2010. Lecture Notes in Computer Science, vol 6136. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13666-5_72
Download citation
DOI: https://doi.org/10.1007/978-3-642-13666-5_72
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13665-8
Online ISBN: 978-3-642-13666-5
eBook Packages: Computer ScienceComputer Science (R0)