Skip to main content
SpringerLink
Log in

Automated universal fractures detection in X-ray images based on deep learning approach

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

At present, bone fracture is a common clinical disease, while the missed diagnosis or misdiagnosis of fracture is harmful to the recovery of patients. Fracture diagnosis often needs the X-ray image as an assistive tool and many fracture detection CAD systems on X-ray images have been explored. However, the majority of existing works mainly focus on detecting fractures in a specific human body part. It’s desirable and feasible to propose a more practical system that can detect various anatomical region fractures ideally due to their similar general fracture characteristics. In this paper, a universal fracture detection CAD system has been developed by us on X-ray images based on the deep learning method. Firstly, we design an image preprocessing method to improve the poor quality of these X-ray images and employ several data augmentation strategies to enlarge the used dataset. Secondly, based on our modified Ada-ResNeSt backbone network and the AC-BiFPN detection method, we propose our automatic fracture detection system. Finally, we establish a private universal fracture detection dataset MURA-D based on the public dataset MURA. As demonstrated by our comprehensive experiments, compared with other popular detectors, our method achieved a higher detection AP of 68.4% with an acceptable inference speed of 122 ms per image on the MURA-D test set, achieving promising results among the state-of-the-art detectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Adams M, Chen W, Holcdorf D, McCusker MW, Howe PD, Gaillard F (2019) Computer vs human: deep learning versus perceptual training for the detection of neck of femur fractures. Journal of medical imaging and radiation oncology 63(1):27–32

    Article  Google Scholar 

  2. Al-Ayyoub M, Hmeidi I, Rababah H (2013) Detecting Hand Bone Fractures in X-Ray Images. Journal of Multimedia Processing and Technologies 4:155–168

  3. Vishnu V, Prakash J, Rengasamy Swathika, Sharmila T et al (2015). Detection and classification of long bone fractures. International Journal of Applied Engineering Research 10:18315–18320

  4. Beyaz S, Açıcı K, Sümer E (2020) Femoral neck fracture detection in X-ray images using deep learning and genetic algorithm approaches. Joint Diseases and Related Surgery 31(2):175–183

    Article  Google Scholar 

  5. Bhalke JS, Wyawahare MV (2014) Survey on classifiers: implemented for fracture detection. International Journal of Computer Applications 89(9):47–51

    Article  Google Scholar 

  6. Cai Z, Vasconcelos N (2018) Cascade R-CNN: delving into high quality object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 6154–6162

  7. Cao J, Cholakkal H, Anwer RM, Khan FS, Pang Y, Shao L (2020) D2Det: towards high quality object detection and instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 11485–11494

  8. Chung SW, Han SS, Lee JW, Oh KS, Kim NR, Yoon JP, Kim JY, Moon SH, Kwon J, Lee HJ, Noh YM, Kim Y (2018) Automated detection and classification of the proximal humerus fracture by using deep learning algorithm. Acta Orthop 89(4):468–473

    Article  Google Scholar 

  9. Dai J, Qi H, Xiong Y, Li Y, Zhang G, Hu H, Wei Y (2017) Deformable convolutional networks. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 764–773

  10. Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88:303–338. https://doi.org/10.1007/s11263-009-0275-4

    Article  Google Scholar 

  11. Gale W, Oakden-Rayner L, Carneiro G, Bradley AP, Palmer LJ (2017) Detecting hip fractures with radiologist-level performance using deep neural networks. arXiv preprint arXiv:1711.06504

  12. Gan K, Xu D, Lin Y, Shen Y, Zhang T, Hu K, Zhou K, Bi M, Pan L, Wu W, Liu Y (2019) Artificial intelligence detection of distal radius fractures: a comparison between the convolutional neural network and professional assessments. Acta Orthop 90(4):394–400

    Article  Google Scholar 

  13. Ghiasi G, Lin TY, Le QV (2019) NAS-FPN: learning scalable feature pyramid architecture for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognitio. IEEE, pp 7036–7045

  14. Guan B, Yao J, Zhang G, Wang X (2019) Thigh fracture detection using deep learning method based on new dilated convolutional feature pyramid network. Pattern Recogn Lett 125:521–526

    Article  Google Scholar 

  15. Guan B, Zhang G, Yao J, Wang X, Wang M (2020) Arm fracture detection in X-rays based on improved deep convolutional neural network. Computers & Electrical Engineering 81:106530

    Article  Google Scholar 

  16. Guo C, Fan B, Zhang Q, Xiang S, Pan C (2020) AugFPN: improving multi-scale feature learning for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 12595–12604

  17. He K, Gkioxari G, Dollár P, Girshick R (2017) Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 2961–2969

  18. Al-Ayyoub M, Hmeidi I, Rababah H (2013) Detecting Hand Bone Fractures in X-Ray Images. Journal of Multimedia Processing and Technologies 4:155–168

  19. Kim DH, MacKinnon T (2018) Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks. Clin Radiol 73(5):439–445

    Article  Google Scholar 

  20. Donnelley M, Knowles G, Hearn T (2008) A CAD system for long-bone segmentation and fracture detection. In: International Conference on Image and Signal Processing. IEEE, pp 153–162

  21. Lim SE, Xing Y, Chen Y, Leow WK, Howe TS, Png MA (2004) Detection of femur and radius fractures in x-ray images. In: Proceeding of the 2nd Interational Conference on Advances in Medical Signal and Information Processing 65

  22. Lin TY, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 2980–2988

  23. Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 2117–2125

  24. Lindsey R, Daluiski A, Chopra S et al (2018) Deep neural network improves fracture detection by clinicians. Proc Nat Acad Sci 115(45):11591–111596

  25. Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. In: European conference on computer vision. IEEE, pp 21–37

  26. Liu S, Qi L, Qin H, Shi J, Jia J (2018) Path aggregation network for instance segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 8759–8768

  27. Mahendran SK, Baboo SS (2012) Ensemble systems for automatic fracture detection. International Journal of Engineering and Technology 4(1):7–10

    Article  Google Scholar 

  28. Mayne R (2013) How to identify fracture types. https://allthingsaafs.com/tag/how-to-identify-fracture-types. Accessed 7 Apr 2019

  29. Myint WW, Tun KS, Tun HM, Myint H (2018) Analysis on leg bone fracture detection and classification using X-ray images. Machine Learning Research 3(3):49

    Article  Google Scholar 

  30. Pang J, Chen K, Shi J, Feng H, Ouyang W, Lin D (2019) Libra R-CNN: towards balanced learning for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 821–830

  31. Pranata YD, Wang KC, Wang JC, Idram I, Lai JY, Liu JW, Hsieh IH (2019) Deep learning and SURF for automated classification and detection of calcaneus fractures in CT images. Comput Methods Prog Biomed 171:27–37

    Article  Google Scholar 

  32. Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 779–788

  33. Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems. IEEE, pp. 91–99

  34. Shao Q, Gong L, Ma K, Liu H, Zheng Y (2019) Attentive CT lesion detection using deep pyramid inference with multi-scale booster. In: International conference on medical image computing and computer-assisted intervention. IEEE, pp 301–309

  35. Tan M, Le QV (2019) EfficientNet: rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946

  36. Tan M, Pang R, Le QV (2020) EfficientDet: scalable and efficient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 10781–10790

  37. Tian Z, Shen C, Chen H, He T (2019) FCOS: fully convolutional one-stage object detection. In: Proceedings of the IEEE international conference on computer vision. IEEE, pp 9627–9636

  38. Tomita N, Cheung YY, Hassanpour S (2018) Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans. Comput Biol Med 98:8–15

    Article  Google Scholar 

  39. Urakawa T, Tanaka Y, Goto S, Matsuzawa H, Watanabe K, Endo N (2019) Detecting intertrochanteric hip fractures with orthopedist-level accuracy using a deep convolutional neural network. Skelet Radiol 48(2):239–244

    Article  Google Scholar 

  40. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 1492–1500

  41. Yan K, Wang X, Lu L, Summers RM (2018) DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. Journal of medical imaging 5(3):036501

    Article  Google Scholar 

  42. Yang F, Lv C, Guo Y, Latecki LJ, Ling H (2019) Dually supervised feature pyramid for object detection and segmentation. arXiv preprint arXiv:1912.03730

  43. Zhang C, Kim J (2019) Object detection with location-aware deformable convolution and backward attention filtering. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 9452–9461

  44. Zhang H, Wu C, Zhang Z, Zhu Y et al (2020) ResNeSt: Split-attention networks. arXiv preprint arXiv:2004.08955

  45. Zhu X, Hu H, Lin S, Dai J (2019) Deformable ConvNets v2: more deformable, better results. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, pp 9308–9316

  46. Improved RetinaNet:Zlocha M, Dou Q, Glocker B (2019) Improving RetinaNet for CT lesion detection with dense masks from weak RECIST labels. In: International conference on medical image computing and computer-assisted intervention. IEEE, pp 402–410

  47. Zoph B, Cubuk ED, Ghiasi G, Lin TY, Shlens J, Le QV (2019) Learning data augmentation strategies for object detection. arXiv preprint arXiv:1906.11172

Download references

Acknowledgments

This work is supported by the Innovation Capacity Construction Project of Jilin Province Development and Reform Commission(2021FGWCXNLJSSZ10),the National Key Research and Development Program of China (No. 2020YFA0714103) and the Science & Technology Development Project of Jilin Province, China (20190302117GX),the Fundamental Research Funds for the Central Universities, JLU.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Shuzhen Lu, Shengsheng Wang & Guangyao Wang

  2. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Shuzhen Lu, Shengsheng Wang & Guangyao Wang

Authors
  1. Shuzhen Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shengsheng Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangyao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shengsheng Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, S., Wang, S. & Wang, G. Automated universal fractures detection in X-ray images based on deep learning approach. Multimed Tools Appl 81, 44487–44503 (2022). https://doi.org/10.1007/s11042-022-13287-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13287-z

Keywords

Search

Navigation