Abstract
Purpose
Many neurosurgical planning tasks rely on identifying points of interest in volumetric images. Often, these points require significant expertise to identify correctly as, in some cases, they are not visible but instead inferred by the clinician. This leads to a high degree of variability between annotators selecting these points. In particular, errors of type are when the experts fundamentally select different points rather than the same point with some inaccuracy. This complicates research as their mean may not reflect any of the experts’ intentions nor the ground truth.
Methods
We present a regularised Bayesian model for measuring errors of type in pointing tasks. This model is reference-free; in that it does not require a priori knowledge of the ground truth point but instead works on the basis of the level of consensus between multiple annotators. We apply this model to simulated data and clinical data from transcranial magnetic stimulation for chronic pain.
Results
Our model estimates the probabilities of selecting the correct point in the range of 82.6\(-\)88.6% with uncertainties in the range of 2.8\(-\)4.0%. This agrees with the literature where ground truth points are known. The uncertainty has not previously been explored in the literature and gives an indication of the dataset’s strength.
Conclusions
Our reference-free Bayesian framework easily models errors of type in pointing tasks. It allows for clinical studies to be performed with a limited number of annotators where the ground truth is not immediately known, which can be applied widely for better understanding human errors in neurosurgical planning.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of data and materials
Data are available for this study at https://github.com/JSHBaxter/bayes_error_of_type.
References
Fitzpatrick JM, West JB (2001) The distribution of target registration error in rigid-body point-based registration. IEEE Trans Med Imaging 20(9):917–927
Wiles AD, Likholyot A, Frantz DD, Peters TM (2008) A statistical model for point-based target registration error with anisotropic fiducial localizer error. IEEE Trans Med Imaging 27(3):378–390
Luo J, Frisken S, Machado I, Zhang M, Pieper S, Golland P, Toews M, Unadkat P, Sedghi A, Zhou H, Mehrtash A, Preiswerk F, Cheng C-C, Golby A, Sugiyama M, Wells WM (2018) Using the variogram for vector outlier screening: application to feature-based image registration. Int J Comput Assist Radiol Surg 13:1871–1880
Bardosi Z, Freysinger W (2016) Estimating \({FLE}_{image}\) distributions of manual fiducial localization in CT images. Int J Comput Assist Radiol Surg 11(6):1043–1049
Francel PC, Jackson TR, Kamiryo T, Laws ER (1999) Optimizing accuracy in magnetic resonance imaging-guided stereotaxis: a technique with validation based on the anterior commissure-posterior commissure line. J Neurosurg 90(1):94–100
Prakash KB, Hu Q, Aziz A, Nowinski WL (2006) Rapid and automatic localization of the anterior and posterior commissure point landmarks in mr volumetric neuroimages1. Acad Radiol 13(1):36–54
Eckstein MP, Abbey CK, Bochud FO (2000) Visual signal detection in structured backgrounds. iv. figures of merit for model performance in multiple-alternative forced-choice detection tasks with correlated responses. JOSA A 17(2):206–217
Elangovan P, Mackenzie A, Dance DR, Young KC, Cooke V, Wilkinson L, Given-Wilson RM, Wallis MG, Wells K (2017) Design and validation of realistic breast models for use in multiple alternative forced choice virtual clinical trials. Phys Med Biol 62(7):2778
Roxin A (2019) Drift-diffusion models for multiple-alternative forced-choice decision making. J Math Neurosci 9(1):1–23
Lefaucheur J-P, André-Obadia N, Antal A, Ayache SS, Baeken C, Benninger DH, Cantello RM, Cincotta M, de Carvalho M, De Ridder D, Devanne H, Di Lazzaro V, Filipović SR, Hummel FC, Jääskeläinen SK, Kimiskidis VK, Koch G, Langguth B, Nyffeler T, Oliviero A, Garcia-Larrea L (2014) Evidence-based guidelines on the therapeutic use of repetitive transcranial magnetic stimulation (rtms). Clin Neurophysiol 125(11):2150–2206
Kim WJ, Min YS, Yang EJ, Paik N-J (2014) Neuronavigated vs. conventional repetitive transcranial magnetic stimulation method for virtual lesioning on the broca’s area. Neuromodulation: Technology at the Neural Interface 17(1), 16–21
Mylius V, Ayache S, Ahdab R, Farhat W, Zouari H, Belke M, Brugières P, Wehrmann E, Krakow K, Timmesfeld N, Schmidt S, Oertel WH, Knake S, Lefaucheur JP (2013) Definition of dlpfc and m1 according to anatomical landmarks for navigated brain stimulation: inter-rater reliability, accuracy, and influence of gender and age. Neuroimage 78:224–232
Baxter JSH, Bui QA, Maguet E, Croci S, Delmas A, Lefaucheur J-P, Bredoux L, Jannin P (2021) Automatic cortical target point localisation in mri for transcranial magnetic stimulation via a multi-resolution convolutional neural network. Int J Comput Assist Radiol Surg 16(7):1077–1087
Vakharia VN, Sparks R, Pérez-García F, Granados A, Miserocchi A, McEvoy A, Ourselin S, Duncan JS (2019) Machine learning for stereotactic neurosurgery: A prospective implementation and validation. Hugh Cairns Prize Essay
Baxter JSH, Croci S, Delmas A, Bredoux L, Lefaucheur J-P, Jannin P (2022) Errors of type or errors of degree? cortical point targeting in transcranial magnetic stimulation. In: Medical imaging 2022: image-Guided Procedures, Robotic Interventions, and Modeling, vol. 12034, p. 1203403. SPIE
Warfield SK, Zou KH, Wells WM (2004) Simultaneous truth and performance level estimation (staple): an algorithm for the validation of image segmentation. IEEE Trans Med Imaging 23(7):903–921
Kulis B, Jordan MI (2012) Revisiting k-means: new algorithms via bayesian nonparametrics. In: Proceedings of the 29th international coference on international conference on machine learning, pp. 1131–1138
Allen K, Shelhamer E, Shin H, Tenenbaum J (2019) Infinite mixture prototypes for few-shot learning. In: International conference on machine learning, pp. 232–241. PMLR
Acknowledgements
The authors would like to thank J.-P. N’Guyen and H. Hodaj for their assistance in annotating the chronic pain treatment points along with J.-P. Lefaucheur.
Funding
No funding was received to assist in the preparation of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
S. Croci, A. Delmas, and L. Bredoux are employees of SYNEIKA. J.S.H. Baxter, J.-P. Lefaucheur, and P. Jannin have no financial or non-financial conflicts of interest.
Ethical approval
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Code availability
Code is available for this study at https://github.com/JSHBaxter/bayes_error_of_type.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Baxter, J.S.H., Croci, S., Delmas, A. et al. Reference-free Bayesian model for pointing errors of typein neurosurgical planning. Int J CARS 18, 1269–1277 (2023). https://doi.org/10.1007/s11548-023-02943-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-023-02943-w