Skip to main content
SpringerLink
Log in

Intraoperative Guidance Using 3D Scene Reconstruction from Endoscopic Images

  • Chapter
  • First Online:
Intraoperative Imaging and Image-Guided Therapy

  • 3660 Accesses

Abstract

Thus far, endoscopes are still mainly used as a keyhole to look inside the human body. Of course, endoscopes can be integrated into standard navigation systems, allowing to fuse the acquired images with other imaging modalities or 3-dimensional models derived from them. Such approaches usually rely on tracking external markers, allowing the visualization of the endoscopic probe in coordinate systems, which can be fixed either to the patient on the operating table or to some preoperatively acquired images. This technology has been used both with rigid endoscopes like in ventriculostomy and, when relying on magnetic tracking methods requiring no direct visibility of the markers, also with flexible devices like in image registered gastroscopic ultrasound procedures. Nevertheless, existing endoscopes can be enhanced by appropriate computer vision tools to a full imaging device allowing taking quantitative measurements and can therefore be used even standalone in an image-guided interventional systems, without incorporating any other imaging modalities. In this chapter, we concentrate on the corresponding approaches and describe the main technological components of such Quantitative endoscopy systems, while also illustrating their value as intraoperative imaging device by some prototypical applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Note, that even the very recent HD cameras rely on interlaced frame acquisition.

  2. 2.

    Distortion corrected endoscopes now exist and are clearly preferable for computer vision tasks.

References

  1. Aron M, Simon G, Berger M. Handling uncertain sensor data in vision-based camera tracking. In: Proceedings of the IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR). Los Alamitos: IEEE Computer Society; 2004. p. 58–67.

    Chapter  Google Scholar 

  2. Bauer M, Schlegel M, Pustka D, Navab N, Klinker G. Predicting and estimating the accuracy of vision-based optical tracking systems. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Los Alamitos: IEEE Computer Society; 2006.

    Google Scholar 

  3. Baumberg A. Reliable feature matching across widely separated views. In: IEEE Computer Society, editor. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos: IEEE Computer Society; 2000. p. 774–81.

    Google Scholar 

  4. Bay H, Tuytelaars T, Gool LV. SURF: speeded up robust features. In: Proceedings of the European Conference on Computer Vision (ECCV). Berlin/New York: Springer; 2006. p. 404–17.

    Google Scholar 

  5. Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. Technical report, UC Berkeley. 2001.

    Google Scholar 

  6. Besl PJ, McKay ND. A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell (PAMI). 1992;14(2):239–56.

    Article  Google Scholar 

  7. Bianchi G, Wengert C. Camera-marker alignment framework and comparison with hand-eye calibration for augmented reality applications. In: Proceedings of the IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR). Los Alamitos: IEEE Computer Society; 2005. p. 188–9.

    Google Scholar 

  8. Bockholt G, Bisler U, Becker A. Augmented reality for enhancement of endoscopic interventions. In: IEEE virtual reality proceedings. Los Alamitos: IEEE Computer Society; 2003. p. 97–101.

    Google Scholar 

  9. Bouguet J. Camera calibration toolbox for MATLAB. 2007. Internet: http://www.vision.caltech.edu/bouguetj/calib_doc/. Last visit: 19 Aug 2013.

  10. Bretzner L, Lindeberg T. Feature tracking with automatic selection of spatial scales. Comput Vis Image Underst (CVIU). 1998;71(3):385–92.

    Article  Google Scholar 

  11. Bricault I, Ferretti G, Cinquin P. Registration of real and CT-derived virtual bronchoscopic images to assist transbronchial biopsy. IEEE Trans Med Imaging. 1998;17(5):703–14.

    Article  CAS  PubMed  Google Scholar 

  12. Brohan AM, Rudolph T, Amstutz CA, Kowal JH. Real-time multimodal retinal image registration for a computer assisted laser photocoagulation system. Trans Biomed Eng. 2010;1:1–8.

    Google Scholar 

  13. Brown DC. Close-range camera calibration. Photogrammetric Eng. 1971;37:855–66.

    Google Scholar 

  14. Burschka D, Li M, Ishii M, Taylor R, Hager G. Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery. Med Image Anal (MIA). 2005;9(5):413–26.

    Article  Google Scholar 

  15. Caban JJ, Seales WB. Reconstruction and enhancement in monocular laparoscopic imagery. Stud Health Technol Inform. 2004;98:37–9.

    PubMed  Google Scholar 

  16. Can A, Stewart CV, Roysam B, Tanenbaum HL. A feature-based technique for joint, linear estimation of high-order image-to-mosaic transformations: mosaicing the curved human retina. IEEE Trans Pattern Anal Mach Intell. 2002;24:412–9.

    Article  Google Scholar 

  17. Cash D, Miga M, Glasgow S, Dawant B, Clements L, Cao Z, Galloway R, Chapman W. Concepts and preliminary data toward the realization of image-guided liver surgery. J Gastrointest Surg. 2007;11(7):844–59.

    Article  PubMed  Google Scholar 

  18. Cattin PC, Bay H, Gool LV, Szekely G. Retina mosaicing using local features. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin/New York: Springer; 2006.

    Google Scholar 

  19. Cech P, Andronache A, Wang L, Székely G, Cattin P. Piecewise rigid multimodal spine registration. In: Bildverarbeitung für die Medizin. Berlin: Springer; 2006. p. 211–5.

    Google Scholar 

  20. Cheng C-L, Van Ness JW. Statistical regression with measurement error, Kendall’s Library of statistics, vol. 6. London: A Hodder Arnold Publication; 1999.

    Google Scholar 

  21. Chesi G. A simple technique for improving camera displacement estimation in Eye-in-hand visual servoing. IEEE Trans Pattern Anal Mach Intell (PAMI). 2004;26(9):1239–42.

    Article  Google Scholar 

  22. Chou J, Kamel M. Quaternions approach to solve the kinematic equation of rotation of a sensor-mounted robotic manipulator. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Pasadena: IEEE Robotics and Automation Society; 1988. p. 656–62.

    Google Scholar 

  23. Chum O, Matas J. Matching with PROSAC — progressive sample consensus. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR05. 2005. 1(I):220–6.

    Google Scholar 

  24. Daniilidis K. Hand-eye calibration using dual quaternions. Int J Robot Res. 1999;18(3):286–98.

    Article  Google Scholar 

  25. Deguchi KK, Kawamata D, Mizutani K, Hontani H, Wakabayashi K. ‘3D fundus shape re- construction and display from stereo fundus images’, IEICE Transactions on Information and Systems 2000. Vol. E83-D No.7 p.1408–14.

    Google Scholar 

  26. Deguchi K, Okatani T. Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center. Comput Vis Image Underst (CVIU). 1996;66(2):119–31.

    Google Scholar 

  27. Delaunay BN. Sur la sphere vide. Bull Acad Sci USSR. 1934;6:793–800.

    Google Scholar 

  28. Deng L, Janabi-Sharifi F, Wilson WJ. Stability and robustness of visual servoing methods. In: IEEE Society, editor. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Pasadena: IEEE Robotics and Automation Society; 2002. p. 1604–9.

    Google Scholar 

  29. Dey D, Gobbi D, Slomka P, Surry K, Peters T. Automatic fusion of freehand endoscopic brain images to three-dimensional surfaces: creating stereoscopic panoramas. IEEE Trans Med Imaging. 2002;21(1):23–30.

    Article  PubMed  Google Scholar 

  30. Dey T, Zhao W. Approximate medial axis as a voronoi subcomplex. In: Proceedings of the seventh ACM symposium on solid modeling and applications. New York: ACM Press; 2002. p. 356–66.

    Chapter  Google Scholar 

  31. Doignon C, Nageotte F, de Mathelin M. Segmentation and guidance of multiple rigid objects for intra-operative endoscopic vision. In: Dynamical Vision. Berlin/New York: Springer; 2007.

    Google Scholar 

  32. Doignon C, Nageotte F, Maurin B, Krupa A. Model-based 3-D pose estimation and feature tracking for robot assisted surgery with medical imaging. In Kragic D, editor. From features to actions – unifying perspectives in computational and robot vision, Workshop at the IEEE International Conference on Robotics and Automation (ICRA). Pasadena: IEEE Robotics and Automation Society; 2007.

    Google Scholar 

  33. Doignon C, Graebling P, Mathelin M. Real-time segmentation of surgical instruments inside the abdominal cavity using a joint hue saturation color feature. Real Time Imaging. 2005;11:429–42.

    Article  Google Scholar 

  34. Edelsbrunner H, Mücke E. Three-dimensional alpha shapes. ACM Trans Graph. 1994;13(1):43–72.

    Article  Google Scholar 

  35. Espiau B. Effect of camera calibration errors on visual servoing in robotics. In: Experimental Robotics III Lecture Notes in Control and Information Sciences Volume 200, Berlin/New York: Springer; 1994. p. 182–92.

    Google Scholar 

  36. Faltin P, Behrens A. Projective 3D-reconstruction of uncalibrated endoscopic images. Acta Polytechnica J Adv Eng. 2010;50(4):29–34.

    Google Scholar 

  37. Farenzena M, Fusiello A, Dovier A. Reconstruction with interval constraints propagation. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos: IEEE Computer Society; 2006. p. 1185–90.

    Google Scholar 

  38. Feuerstein M, Wildhirt S, Bauernschmitt R, Navab N. Automatic patient registration for port placement in minimally invasive endoscopic surgery. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin/NewYork: Springer; 2005. p. 287–94.

    Google Scholar 

  39. Fischler M, Bolles R. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM. 1981;24(6):381–95.

    Article  Google Scholar 

  40. Fitzgibbon A, Pilu M, Fisher R. Direct least squares fitting of ellipses. In: Proceedings of the IEEE International Conference on Pattern Recognition (ICPR). Los Alamitos: IEEE Computer Society; 1999. p. 253.

    Google Scholar 

  41. Fitzgibbon A, Zisserman A. Multibody structure and motion: 3-D reconstruction of independently moving objects. In: Proceedings of the European Conference on Computer Vision (ECCV). Berlin/New York: Springer; 2000. p. 891–906.

    Google Scholar 

  42. Fitzpatrick JM, West JB, Maurer CR. Predicting error in rigid-body point-based registration. IEEE Trans Med Imaging. 1998;17(5):694–702.

    Article  CAS  PubMed  Google Scholar 

  43. Florou G, Mohr R. What accuracy for 3D measurements with cameras? In: Proceedings of the IEEE International Conference on Pattern Recognition (ICPR). Los Alamitos: IEEE Computer Society; 1996. p. 354.

    Google Scholar 

  44. Forster Q, Tozzi C. Towards 3D reconstruction of endoscope images using shape from shading. In: Proceedings of the XIII Brazilian Symposium on Computer Graphics and Image Processing. Los Alamitos: IEEE Computer Society; 2000. p. 90–6.

    Google Scholar 

  45. Freeman WT, Adelson EH. The design and use of steerable filters. IEEE Trans Pattern Anal Mach Intell (PAMI). 1991;13(9):891–906.

    Article  Google Scholar 

  46. Fujimoto T, Nomura Y, Zhang D. Theoretical error analysis with camera parameter calibration. Mach Vis Optomechatronic Appl. 2004;5603(1):182–90.

    Article  Google Scholar 

  47. Gelfand N, Ikemoto L, Rusinkiewicz S, Levoy M. Geometrically stable sampling for the ICP algorithm. In: Proceedings of the international conference on 3D digital imaging and modeling. Los Alamitos: IEEE Computer Society; 2003. p. 260–7.

    Google Scholar 

  48. Gomez G, Sucar L, Gillies D. The pq-histogram as a navigation clue. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Pasadena: IEEE Robotics and Automation Society; 2002. p. 3362–7.

    Google Scholar 

  49. Gomez JF, Simon G, Berger M. Calibration errors in augmented reality: a practical study. In: Proceedings of the IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR). Los Alamitos: IEEE Computer Society; 2005. p. 154–63.

    Google Scholar 

  50. Guendouz B, Eswaran C, Muniandy SV. Error propagation and accurate calibration for camera model. In: IEEE Society, editor. Proceedings of the IEEE International Conference on Engineering of Intelligent Systems. Piscataway: IEEE; 2006. p. 1–5.

    Google Scholar 

  51. Han M, Kanade T. Reconstruction of a scene with multiple linearly moving objects. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). 2000. p. 542–9.

    Google Scholar 

  52. Hargreaves G. Interval analysis in MATLAB (416). Technical report, Manchester Centre for Computational Mathematics, Department of Mathematics, University of Manchester. 2002.

    Google Scholar 

  53. Harris C, Stephens M. A combined corner and edge detection. In: Proceedings of the fourth Alvey Vision conference. Manchester: Britisch Machibe Vision Association; 1988. p. 147–51.

    Google Scholar 

  54. Hartley R, Sturm P. Triangulation. Comput Vis Image Underst (CVIU). 1997;68(2):146–57.

    Article  Google Scholar 

  55. Hartley R, Zisserman A. Multiple view geometry in computer vision. Cambridge: Cambridge University Press; 2000.

    Google Scholar 

  56. Hasegawa K, Sato Y. Endoscope system for high-speed 3D measurement. Syst Comput Jpn. 2001;32(8):30–9.

    Article  Google Scholar 

  57. Heikkilä J, Silven O. A four-step camera calibration procedure with implicit image correction. In: Proceedings of the IEEE conference on conference on computer vision and pattern recognition (CVPR). Los Alamitos: IEEE Computer Society; 1997. p. 1106–12.

    Chapter  Google Scholar 

  58. Heikkilä M, Pietikäinen M, Schmid C. Description of interest regions with center-symmetric local binary patterns. In: 5th Indian Conference on Computer Vision Graphics and Image Processing ICVGIP. 2006;2(3):58–69.

    Google Scholar 

  59. Helferty JP, Sherbondy AJ, Kiraly AP, Higgins WE. Computer-based system for the virtual endoscopic guidance of bronchoscopy. Comput Vis Image Underst. 2007;108:171–87.

    Article  PubMed Central  PubMed  Google Scholar 

  60. Holloway R. Registration errors in augmented reality systems. PhD thesis, University of North Carolina at Chapel Hill. 1995.

    Google Scholar 

  61. Hu M, Penney G, Figl M, Edwards P, Bello F, Casula R, Rueckert D, Hawkes D. Reconstruction of a 3D surface from video that is robust to missing data and outliers: application to minimally invasive surgery using stereo and mono endoscopes. Med Image Anal. 2012;16:597–611.

    Article  PubMed  Google Scholar 

  62. i-Logic™ System for bronchoscopy, superDimension Ltd, http://www.superdimension.com/index.cfm/go/Products.iLogic.

  63. Kadir T, Zisserman A, Brady M. An affine invariant salient region detector. In: Proceedings of the European Conference on Computer Vision (ECCV). Berlin/New York: Springer; 2004.

    Google Scholar 

  64. Ke Y, Sukthankar R. PCA-SIFT: a more distinctive representation for local image descriptors. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos: IEEE Computer Society; 2004. p. 506–13.

    Google Scholar 

  65. Kearfott RB. Interval computations: introduction, uses, and resources. Euromath Bull. 1994;1(2).

    Google Scholar 

  66. Kim K-H, Jun H. Realistic 3D reconstruction from an image sequence. In: IEEE Society, editor. Proceedings of the 7th Korea-Russia International Symposium on Science and Technology. Piscataway: IEEE; 2003. p. 125–8.

    Google Scholar 

  67. Kolar A, Romain O, Ayoub J, Faura D, Viateur S, Granado B, Graba T. A system for an accurate 3D reconstruction in video endoscopy capsule. EURASIP J Embedded Syst. 2009. Article 12 p. 1–15, doi: 0.1155/2009/716317.

  68. Koppel D, Wang Y-F. Image-based rendering and modeling in video-endoscopy. In: Proceedings of the IEEE International Symposium on Biomedical Imaging: Macro to Nano. Los Alamitos: IEEE Computer Society; 2004. p. 269–72.

    Google Scholar 

  69. Kowal J, Amstutz C, Ioppolo J, Nolte LP, Styner M. Fast automatic bone contour extraction in ultrasound images for intraoperative registration. Technical Report, M.E. Müller Institute of Biomechanics, University of Bern. 2002.

    Google Scholar 

  70. Krupa A, et al. Automatic 3-D positioning of surgical instruments during robotized laparoscopic surgery using automatic visual feedback. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). 2002. p. 9–16.

    Google Scholar 

  71. Kwoh C, Khan G, Gillies D. Automated endoscopic navigation and advisory system from medical image. In: Proceedings of SPIE Medical Imaging: Physiology and Function from Multidimensional Images. 1999. p. 214–24.

    Google Scholar 

  72. Lange R, Seitz P. Solid-state time-of-flight range camera. IEEE J Quantum Electron. 2001;37(3):390–7.

    Article  CAS  Google Scholar 

  73. Lazebnik S, Schmid C, Ponce J. Sparse texture representation using affine-invariant neighborhoods. IEEE Trans Pattern Anal Mach Intell (PAMI). 2003;27(8):1265–78.

    Article  Google Scholar 

  74. Lindeberg T, Garding J. Shape-adapted smoothing in estimation of 3-D shape cues from affine deformations of local 2-D brightness structure. Image Vis Comput. 1997;15(6):415–34.

    Article  Google Scholar 

  75. Liu X, Kanungo T, Haralick R. On the use of error propagation for statistical validation of computer vision software. IEEE Trans Pattern Anal Mach Intell (PAMI). 2005;27(10):1603–14.

    Article  Google Scholar 

  76. Reeff M, Gerhard F, Cattin PC, Szekely G. Mosaicing of endoscopic placenta images. In: Proc Informatik 2006. Informatik für Menschen. Berlin: Springer; 2006. p. 467–74.

    Google Scholar 

  77. Matas J, Chum O, Urban M, Pajdla T. Robust wide baseline stereo from maximally stable extremal regions. In: Rosin PL, Marshall D, editors. Proceedings of the British machine vision conference. Manchester BMVA; 2002. p. 384–93.

    Google Scholar 

  78. Mavroidis C, Dubowsky S, Drouet P, Hintersteiner J, Flanz J. A systematic error analysis of robotic manipulators: application to a high performance medical robot. In: IEEE Society, editor. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). Pasadena: IEEE Robotics and Automation Society; 1997, p. 980–5.

    Google Scholar 

  79. Mikolajczyk K, Schmid C. A performance evaluation of local descriptors. IEEE Trans Pattern Anal Mach Intell (PAMI). 2005;27(10):1615–30.

    Article  Google Scholar 

  80. Mikolajczyk K, Schmid C. Scale & affine invariant interest point detectors. Int J Comput Vis (IJCV). 2004;60(1):63–86.

    Article  Google Scholar 

  81. Mikolajczyk K, Tuytelaars T, Schmid C, Zisserman A, Matas J, Schaffalitzky F, Kadir T, Van Gool L. A comparison of affine region detectors. Int J Comput Vis (IJCV). 2005;65(1–2):43–72.

    Article  Google Scholar 

  82. Moravec H. Obstacle avoidance and navigation in the real world by a seeing robot rover. Technical report, Robotics Institute, Carnegie Mellon University. 1980.

    Google Scholar 

  83. Mori K, Deguchi D, Akiyama K, Kitasaka T, Maurer C, Suenaga Y, Takabatake H, Mori M, Natori H. Hybrid bronchoscope tracking using a magnetic tracking sensor and image registration. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin/New York: Springer; 2005. p. 543–50.

    Google Scholar 

  84. Mountney P, Lo P, Thiemjarus S, Stoyanov D, Yang GZ. A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). 2007. p. 34–41.

    Google Scholar 

  85. Mourgues F, Devemay F, Coste-Maniere E. 3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery. In: Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISMAR). 2001. p. 191.

    Google Scholar 

  86. Moyung T. Incremental 3D reconstruction using stereo image sequences. In: Proceedings of the International Conference on Image Processing (ICIP). Los Alamitos: IEEE Computer Society; 2000. p. 752–5.

    Google Scholar 

  87. Muacevic A, Muller A. Image-guided endoscopic ventriculostomy with a New frameless armless neuronavigation system. Comput Aided Surg (CAS). 1999;4(2):87–92.

    Article  CAS  Google Scholar 

  88. Muja M, Lowe D. FLANN – Fast Library for Approximate Nearest Neighbors user manual, Writing. 2009.

    Google Scholar 

  89. Neo M, Matsushita M, Iwashita Y, Yasuda T, Sakamoto T, Nakamura T. Atlantoaxial transarticular screw fixation for a high-riding vertebral artery. Spine. 2003;28(7):666–70.

    PubMed  Google Scholar 

  90. Nicolaou M, James A, Lo B, Darzi A, Guang-Zhong Y. Invisible shadow for navigation and planning in minimal invasive surgery. In: Medical Image Computing and Computer Assisted Intervention (MICCAI). Berlin/New York: Springer; 2005. p. 25–32.

    Google Scholar 

  91. Nister D, Stewenius H. Scalable recognition with a vocabulary tree. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society; 2006. Volume 2 CVPR06 2(c), 2161–8.

    Google Scholar 

  92. Noonan DP, Mountney P, Elson DS, Darzi A, Yang G-Z. A stereoscopic fibroscope for camera motion and 3D depth recovery during minimally invasive surgery. In: Proceedings of the 2009 IEEE international conference on robotics and automation. Piscataway: IEEE Press; 2009. p. 3274–9.

    Google Scholar 

  93. Obstein KL, Estépar RSJ, Jayender J, Patil VD, Spofford IS, Ryan MB, Lengyel BI, Shams R, Vosburgh KG, Thompson CC. Image registered gastroscopic ultrasound (IRGUS) in human subjects: a pilot study to assess feasibility. Endoscopy. 2011;43(5):394–9.

    Article  CAS  PubMed  Google Scholar 

  94. Park F, Martin B. Robot sensor calibration: solving AX=XB on the Euclidean group. IEEE Trans Robot Autom. 1994;10(5):717–21.

    Article  Google Scholar 

  95. Penne J, Häller K, Stürmer M, Schrauder T, Schneider A, Engelbrecht R, Feussner H, Schmauss B, Hornegger J. Time-of-flight 3-D endoscopy. Med Image Comput Comput Assist Interv. 2009;12(Pt 1):467–74.

    PubMed  Google Scholar 

  96. Pezzementi Z, Voros S, Hager GD. Articulated object tracking by rendering consistent appearance parts. In: Proc. IEEE Int. Conf. on Robotics and Automation. 2009. p. 3940–47.

    Google Scholar 

  97. Pollefeys M, Koch R, Van Gool L. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). 1998. p. 90–5.

    Google Scholar 

  98. Miranda-Luna R, Ch. Daul, W. B. Y. H.-M. D. W, Guillemin F. Mosaicing of bladder endoscopic image sequences. Pasadena: IEEE Robotics and Automation Society; Distortion calibration and registration algorithm. In: IEEE Transactions on Biomedical Engineering. 2008;55(2) p. 541–53.

    Google Scholar 

  99. Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. In: Proceedings of the Third International Conference on 3D Digital Imaging and Modeling. Los Alamitos: IEEE Computer Society; 2001. p. 145–52.

    Google Scholar 

  100. Sato Y, Nakamoto M, Tamaki Y, Sasama T, Sakita I, Nakajima Y, Monden M, Tamura S. Image guidance of breast cancer surgery using 3-D ultrasound images and augmented reality visualization. IEEE Trans Med Imaging. 1998;17(5):681–93.

    Article  CAS  PubMed  Google Scholar 

  101. Sato Y, Sasama T, Sugano N, Nakahodo K, Nishii T, Ozono, K, Yonenobu K, Ochi T, Tamura S. Intraoperative simulation and planning using a Combined Acetabular and Femoral (CAF) navigation system for total hip replacement. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin/New York: Springer; 2000. p. 1114–25.

    Google Scholar 

  102. Sauer F, Khamene A, Vogt S. An augmented reality navigation system with a single-camera tracker: system design and needle biopsy phantom trial. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin/New York: Springer; 2002. p. 116–24.

    Google Scholar 

  103. Schwarz Y, Mehta AC, Ernst A, Herth F, Engel A, Besser D, Becker HD. Electromagnetic navigation during flexible bronchoscopy. Respiration. 2003;70(5):516–22.

    Article  PubMed  Google Scholar 

  104. Shi J, Tomasi C. Good features to track. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (CVPR). Los Alamitos: IEEE Computer Society; 1994. p. 593–600.

    Google Scholar 

  105. Sinha TK, Dawant BM, Duay V, Cash DM, Weil RJ, Thompson RC, Weaver KD, Miga MI. A method to track cortical surface deformations using a laser range scanner. IEEE Trans Med Imaging. 2005;24(6):767–81.

    Article  PubMed  Google Scholar 

  106. Slama C. Manual of photogrammetry. Falls Church: American Society of Photogrammetry; 1980.

    Google Scholar 

  107. Smith SM, Brady JM. SUSAN – a new approach to low level image processing (TR95SMS1c). Technical report, Chertsey, Surrey. 1995.

    Google Scholar 

  108. Stehle T, Truhn D, Aach T, Trautwein C, Tischendorf J. Camera calibration for fish-eye lenses in endoscopy with an application to 3D reconstruction. In: Proceedings IEEE International Symposium on Biomedical Imaging (ISBI). Los Alamitos: IEEE Computer Society; 2007.

    Google Scholar 

  109. Stoyanov D, Scarzanella M, Pratt P, Yang G-Z. Real-time stereo reconstruction in robotically assisted minimally invasive surgery. In: Jiang T, Navab N, Pluim J, Viergever M, editors. Medical image computing and computer-assisted intervention MICCAI 2010. Berlin/Heidelberg: Springer; 2010. p. 275–82.

    Chapter  Google Scholar 

  110. Sugano N, Sasama T, Sato Y, Nakajima Y, Nishii T, Yonenobu K, Tamura S, Ochi T. Accuracy evaluation of surface-based registration methods in a computer navigation system for hip surgery performed through a posterolateral approach. Comput Aided Surg (CAS). 2001;6(4):195–203.

    Article  CAS  Google Scholar 

  111. Szeliski R. Image mosaicing for tele-reality applications. In: Proceedings of the Second IEEE Workshop on Applications of Computer Vision. Los Alamitos: IEEE Computer Society; 1994.

    Google Scholar 

  112. Telle B, Aldon MJ, Ramdani N. Camera calibration and 3D reconstruction using interval analysis. In: Proceedings of 12th International Conference on Image Analysis and Processing. Los Alamitos: IEEE Computer Society; 2003. p. 374–9.

    Google Scholar 

  113. Tellinghuisen J. Statistical error propagation. J Phys Chem. 2001;105(15):3917–21.

    Article  CAS  Google Scholar 

  114. Thoranaghatte RU, Zheng G, Langlotz F, Nolte LP. Endoscope based hybrid-navigation system for minimally invasive ventral-spine surgeries. Comput Aided Surg (CAS). 2005;10(5–6):351–6.

    Google Scholar 

  115. Tjoa M, Krishnan M, Zheng S. A novel endoscopic image analysis approach using deformable region model to aid in clinical diagnosis. In: Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society. Piscataway: IEEE Engineering in Medicine & Biology Society; 2003. p. 710–3.

    Google Scholar 

  116. Tomasi C, Kanade T. Shape and motion from image streams under orthography: a factorization method. Int J Comput Vis. 1992;9(2):137–54.

    Article  Google Scholar 

  117. Tomasi C, Kanade T. Detection and tracking of point features (CMU-CS-91-132). Technical report, Carnegie Mellon University. 1991.

    Google Scholar 

  118. Torr PHS. Robust parameterization and computation of the trifocal tensor. Image Vis Comput. 1997;15(8):591–605.

    Article  Google Scholar 

  119. Tsai R. A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE Trans Robot Autom. 1987;3(4):323–44.

    Google Scholar 

  120. Tsai R, Lenz R. A new technique for fully autonomous and efficient 3D robotics hand eye calibration. IEEE Trans Robot Autom. 1989;5(3):345–58.

    Article  CAS  Google Scholar 

  121. Tuytelaars T, Van Gool L. Matching widely separated views based on affine invariant regions. Int J Comput Vis (IJCV). 2004;59(1):61–85.

    Article  Google Scholar 

  122. Van Gool L, Moons T, Ungureanu D. Affine/photometric invariants for planar intensity patterns. In: Proceedings of the European Conference on Computer Vision (ECCV). Berlin/New York: Springer; 1996. p. 642–51.

    Google Scholar 

  123. Voros S, Long J-A, Cinquin P. Automatic detection of instruments in laparoscopic images: a first step towards high-level command of robotic endoscopic holders. Int J Robot Res. 2007;26:1173–90.

    Article  Google Scholar 

  124. Wei G-Q, et al. Automatic tracking of laparoscopic instruments by color coding. In: Proceedings of the First Joint Conference on Computer Vision, Virtual Reality and Robotics in Medicine and Medial Robotics and Computer-Assisted Surgery. Berlin/New York: Springer; 1997. p. 357–66.

    Google Scholar 

  125. Weise T, Leibe B, Gool LV. Fast 3D Scanning with automatic motion compensation. IEEE Conf Comput Vis Pattern Recognit. 2007;07:1–8.

    Google Scholar 

  126. Weng J, Cohen P, Herniou M. Camera calibration with distortion models and accuracy evaluation. IEEE Trans Pattern Anal Mach Intell (PAMI). 1992;14(10):965–80.

    Article  Google Scholar 

  127. Wengert C, Cattin P, Duff J, Székely G. Markerless endoscopic registration and referencing. In: Larsen R, Nielsen M, Sporring J, editors. Medical Image Computing and Computer-Assisted Intervention (MICCAI). Berlin/New York: Springer; 2006. p. 816–23.

    Google Scholar 

  128. Wengert C, Reeff M, Cattin P, Székely G. Fully automatic endoscope calibration for intraoperative use. In: Bildverarbeitung für die Medizin. Berlin: Springer; 2006. p. 419–23.

    Google Scholar 

  129. Wigfield C, Bolger C. A technique for frameless stereotaxy and placement of transarticular screws for atlanto-axial instability in rheumatoid arthritis. Eur Spine J. 2001;10(3):264–8.

    Article  CAS  PubMed Central  PubMed  Google Scholar 

  130. Xiao J, Chai J-X, Kanade T. A closed-form solution to non-rigid shape and motion. Robotics. 2007. Int. J. Computer Vision 67(2). 2006. p. 233–46.

    Google Scholar 

  131. Yang CC, Marefat MM, Ciarallo FW. Error analysis and planning accuracy for dimensional measurement in active vision inspection. IEEE Trans Robot Autom. 1998;14(3):476–87.

    Article  Google Scholar 

  132. Zhang Z. Flexible camera calibration by viewing a plane from unknown orientations. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Los Alamitos: IEEE Computer Society; 1999. p. 666–73.

    Google Scholar 

  133. Lowe D. Distinctive image features from scale-invariant keypoints. Int J Comput Vis (IJCV). 2004;60:91–110.

    Article  Google Scholar 

  134. Scholz M, Konen W, Tombrock S, Fricke B, Adams L, von Düring M, Hentsch A, Harders LHA. Development of an endoscopic navigation system based on digital image processing. Comput Aided Surg. 1998;3:134–43.

    Article  CAS  PubMed  Google Scholar 

  135. De Ipiña D, Mendonça P, Hopper A. TRIP: a Low-cost vision-based location system for ubiquitous computing. Pers Ubiquitous Comput. 2002;2002(6):206–19.

    Google Scholar 

  136. Chung A, Deligianni F, Shah P, Wells, A, Yang G-Z. VIS-a-VE: visual augmentation for virtual environments in surgical training.In: Proceedings of the 7th Joint Visualization Symposium of the Eurographics Association and the Visualization and Computer Graphics Technical Committee (VGTC). Geneva: Eurographics Association; 2005. p. 101–8.

    Google Scholar 

  137. Schaffalitzky F, Zisserman A. Multi-view matching for unordered image sets, or “how do I organize my holiday snaps?”. In: Proceedings of the European Conference on Computer Vision (ECCV), vol 1. Berlin/New York: Springer; 2002. p. 414–31.

    Google Scholar 

  138. West J, Fitzpatrick M. The distribution of target registration error in rigid-body, point-based registration. In: Proceedings of the 16th International Conference on Information Processing in Medical Imaging (IPMI). Berlin/New York: Springer; vol. 1613. 1999. p. 460–65.

    Google Scholar 

  139. Kyrki V, Kragic D, Christensen HI. Measurement errors in visual servoing. Robot Auton Sys. 2006;2006(54):815–27.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Vision Laboratory, ETH Zürich, Zürich, Switzerland

    Christian Wengert PhD & Gábor Székely PhD

Authors
  1. Christian Wengert PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gábor Székely PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gábor Székely PhD .

Editor information

Editors and Affiliations

  1. National Center for Image Guided Therapy, Department of Radiology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA

    Ferenc A. Jolesz

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer Science+Business Media New York

About this chapter

Cite this chapter

Wengert, C., Székely, G. (2014). Intraoperative Guidance Using 3D Scene Reconstruction from Endoscopic Images. In: Jolesz, F. (eds) Intraoperative Imaging and Image-Guided Therapy. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7657-3_30

Download citation

  • DOI: https://doi.org/10.1007/978-1-4614-7657-3_30

  • Published:

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4614-7656-6

  • Online ISBN: 978-1-4614-7657-3

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation