Skip to main content

Advertisement

SpringerLink
Log in

Real-time image composition of bladder mosaics in fluorescence endoscopy

  • Special Issue Paper
  • Published:
Computer Science - Research and Development

Abstract

Today, photodynamic diagnostics is commonly used in endoscopic intervention of the urinary bladder. Excited by a narrow band illumination, fluorescence markers enhance the visual contrast between benign and malignant tissue. Since in this modality the endoscope must be moved close to the bladder wall to provide sufficiently exposed images, the field of view (FOV) of the endoscope is very limited. This impedes the navigation and the re-identifying of multi-focal tumors for the physician. Thus, an image providing a larger FOV, composed from single images is highly desired during the intervention for surgery assistance. Since endoscopic mosaicking in real-time is still an open issue, we introduce a new feature-based image mosaicking algorithm for fluorescence endoscopy. Using a multi-threaded software design, the extraction of SURF features, the matching and the image stitching are separated in single processing threads. In an optimization step we discuss the trade-off between feature repeatability and processing time. After adjusting an optimal thread synchronization, the optimal workload of each thread results in a fast and real-time capable computation of image mosaics. On a standard hardware platform our algorithm performs within the RealTimeFrame framework with an update rate of 8.17 frames per second at full input image resolution (780×576). Providing a fast growing image with an extended FOV during the intervention, the mosaic displayed on a second monitor promises high potential for surgery assistance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. American Cancer Society: Cancer facts and figures 2009

  2. Bagley DH, Huffman JL, Lyon ES (1985) Urologic endoscopy: a manual and atlas. Lippincott Williams and Wilkins, Philadelphia

    Google Scholar 

  3. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Surf: Speeded up robust features. Comput Vis Image Underst (CVIU) 110(3):346–359

    Article  Google Scholar 

  4. Behrens A (2008) Creating panoramic images for bladder fluorescence endoscopy. Acta Polytech J Adv Eng 48(3):50–54

    MathSciNet  Google Scholar 

  5. Behrens A, Bommes M, Stehle T, Gross S, Leonhardt S, Aach T (2010) Mosaickingalgorithmus zur schnellen Panoramabilderstellung in der Fluoreszenzendoskopie. In: Bildverarbeitung für die Medizin (BVM), vol 574. Springer, Berlin, pp 41–45

    Google Scholar 

  6. Behrens A, Bommes M, Stehle T, Gross S, Leonhardt S, Aach T (2010) A multi-threaded mosaicking algorithm for fast image composition of fluorescence bladder images. In: Wong KH, Miga MI (eds) Proc. SPIE medical imaging 2010: visualization, image-guided procedures, and modeling (San Diego, USA), vol 7625. SPIE, Bellingham, p 76252S

    Google Scholar 

  7. Behrens A, Stehle T, Gross S, Aach T (2009) Local and global panoramic imaging for fluorescence bladder endoscopy. In: Engineering in medicine and biology society, EMBC 2009. 31th annu int conf of the IEEE, Minneapolis. IEEE, New York, pp 6990–6993

    Chapter  Google Scholar 

  8. Hamadou A Ben, Soussen C, Blondel W, Daul C, Wolf D (2009) Comparative study of image registration techniques for bladder video-endoscopy. In: SPIE European conference on biomedical optics—novel optical instrumentation for biomedical applications IV, vol 7371. SPIE, Bellingham

    Google Scholar 

  9. Bergen T, Ruthotto S, Munzenmayer C, Rupp S, Paulus D, Winter C (2009) Feature-based real-time endoscopic mosaicking. In: Proc 6th int symp image and signal processing and analysis (ISPA), pp 695–700

  10. Brown M, Szeliski R, Winder S (2005) Multi-image matching using multi-scale oriented patches. In: Proc IEEE computer society conference on computer vision and pattern recognition (CVPR), vol 1, pp 510–517

  11. Cao CG, Milgram P (2000) Disorientation in minimal access surgery: A case study. In: Proc IEA 2000/HFES 2000 congress, vol 4, pp 169–172

  12. Cheng Z, Devarajan D, Radke RJ (2007) Determining vision graphs for distributed camera networks using feature digests. EURASIP J Appl Signal Process 2007(1):220–220

    Google Scholar 

  13. Fischler MA, Bolles RC (1981) Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395

    Article  MathSciNet  Google Scholar 

  14. Gross S, Behrens A, Stehle T (2009) Rapid development of video processing algorithms with RealTimeFrame. In: Proc biomedica, pp 217–220

  15. Gross S, Stehle T (2008) RealTimeFrame—a real time processing framework for medical video sequences. Acta Polytech J Adv Eng 48(3):15–19

    Google Scholar 

  16. Halim I, Tavakkolizadeh A (2008) Notes: The next surgical revolution? Int J Surg 6(4):273–276

    Article  Google Scholar 

  17. Hartley R, Kang SB (2007) Parameter-free radial distortion correction with center of distortion estimation. IEEE Trans Pattern Anal Mach Intell 29(8):1309–1321

    Article  Google Scholar 

  18. Höller K, Penne J, Schneider A, Jahn J, Boronat JG, Wittenberg T, Feußner H, Hornegger J (2009) Endoscopic orientation correction. In: Medical image computing and computer-assisted intervention (MICCAI). LNCS, vol 5761. Springer, Berlin, pp 459–466

    Google Scholar 

  19. Hudson MA, Herr HW (1995) Carcinoma in situ of the bladder. J Urol 153:564–572

    Google Scholar 

  20. Hungerhuber E, Stepp H, Kriegmair M, Stief C, Hofstetter A, Hartmann A, Knuechel R, Karl A, Tritschler S, Zaak D (2007) Seven years’ experience with 5-aminolevulinic acid in detection of transitional cell carcinoma of the bladder. Urology 69(2):260–264

    Article  Google Scholar 

  21. Kassouf W, Swanson D, Kamat AM, Leibovici D, Siefker-Radtke A, Munsell MF, Grossman HB, Dinney CP (2006) Partial cystectomy for muscle invasive urothelial carcinoma of the bladder: A contemporary review of the m. d. Anderson cancer center experience. J Urol 175(6):2058–2062

    Article  Google Scholar 

  22. Konen W, Breiderhoff B, Scholz M (2007) Real-time image mosaic for endoscopic video sequences. In: Bildverarbeitung für die Medizin (BVM). Springer, Berlin

    Google Scholar 

  23. Kourogi M, Kurata T, Hoshino J, Muraoka Y (1999) Real-time image mosaicing from a video sequence. In: Proc IEEE international conference on image processing (ICIP), vol 4, pp 133–137

  24. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110

    Article  Google Scholar 

  25. Mann S, Picard R (1994) Virtual bellows: constructing high quality stills from video. In: Proc IEEE international conference image processing (ICIP), vol 1, pp 363–367

  26. Matouschek E (1987) Urologisch-endoskopische Operationen. Schattauer, Stuttgart

    Google Scholar 

  27. Miranda-Luna R, Daul C, Blondel W, Hernandez-Mier Y, Wolf D, Guillemin F (2008) Mosaicing of bladder endoscopic image sequences: Distortion calibration and registration algorithm. IEEE Trans Biomed Eng 55(2):541–553

    Google Scholar 

  28. Miranda-Luna R, Hernandez-Mier Y, Daul C, Blondel W, Wolf D (2004) Mosaicing of medical video-endoscopic images: data quality improvement and algorithm testing. In: 1st int conf electrical and electronics engineering (ICEEE), pp 530–535

  29. Mountney P, Yang GZ (2008) Soft tissue tracking for minimally invasive surgery: Learning local deformation online. In: Medical image computing and computer-assisted intervention (MICCAI). LNCS, vol 5242. Springer, Berlin, pp 364–372

    Google Scholar 

  30. Mountney P, Yang GZ (2009) Dynamic view expansion for minimal invasive surgery using simultaneous localization and mapping. In: Engineering in medicine and biology society, EMBC 2009. 31th annu int conf of the IEEE, pp 1184–1187

  31. Olijnyk S, Mier YH, Blondel WM, Daul C, Wolf D, Bourg-Heckly G (2007) Combination of panoramic and fluorescence endoscopic images to obtain tumor spatial distribution information useful for bladder cancer detection. In: SPIE European conference on biomedical optics—novel optical instrumentation for biomedical applications III, vol 6631. SPIE, Bellingham, p 66310X

    Google Scholar 

  32. Orozco RE, Martin AA, Murphy WM (1994) Carcinoma in situ of the urinary bladder clues to host involvement in human carcinogenesis. Cancer 74(1):115–122

    Article  Google Scholar 

  33. Rattner D, Kalloo A (2006) Asge/sages working group on natural orifice translumenal endoscopic surgery. Surg Endosc 20(2):329–333

    Article  Google Scholar 

  34. Seshamani S, Lau WW, Hager GD (2006) Real-time endoscopic mosaicking. In: Medical image computing and computer-assisted intervention (MICCAI). LNCS, vol 4190. Springer, Berlin, pp 355–363

    Google Scholar 

  35. Stehle T, Behrens A, Aach T (2008) Enhancement of visual contrast in fluorescence endoscopy. In: IEEE international conference on multimedia and expo

  36. Stehle T, Behrens A, Aach T (2008) Visual enhancement of facial tissue in endoscopy. In: SPIE medical imaging 2008: visualization, image guided procedures, and modeling, vol 6918. SPIE, Bellingham

    Google Scholar 

  37. Stehle T, Wulff J, Behrens A, Gross S, Aach T (2009) Denoising fluorescence endoscopy: A motion-compensated temporal recursive video filter with an optimal minimum mean square error parametrization. In: Proceedings of IEEE international symposium on biomedical imaging (ISBI), pp 314–317

  38. Szeliski R (1996) Video mosaics for virtual environments. IEEE Comput Graph Appl 16(2):22–30

    Article  Google Scholar 

  39. Szeliski R (2006) Image alignment and stitching: A tutorial. Tech rep MSR-TR-2004-92, Microsoft Research

  40. Thormählen T, Broszio H, Meier PN (2002) Three-dimensional endoscopy. In: Falk symposium, medical imaging in gastroenterology and hepatology. Kluwer Academic, Dordrecht, p 124

    Google Scholar 

  41. Tomasi C, Kanade T (1991) Detection and tracking of point features. Tech rep CMU-CS-91-132. Carnegie Mellon University

  42. Triggs B, Mclauchlan PF, Hartley RI, Fitzgibbon AW (2000) Bundle adjustment—a modern synthesis. In: Vision algorithms: theory and practice. LCNS, vol 1883. Springer, Berlin, pp 298–375

    Chapter  Google Scholar 

  43. Wald D, Reeff M, Székely G, Cattin P, Paulus D (2005) Fließende Überblendung von Endoskopiebildern für die Erstellung eines Mosaiks. In: Bildverarbeitung für die Medizin (BVM). Springer, Berlin, pp 287–291

    Chapter  Google Scholar 

  44. Wickham JEA (1987) The new surgery. Br Med J 295(6613):1581–1582

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Imaging & Computer Vision, RWTH Aachen University, Aachen, Germany

    Alexander Behrens, Michael Bommes, Thomas Stehle, Sebastian Gross & Til Aach

  2. Philips Chair for Medical Information Technology, Helmholtz-Institute, RWTH Aachen University, Aachen, Germany

    Steffen Leonhardt

Authors
  1. Alexander Behrens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Bommes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Stehle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sebastian Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Steffen Leonhardt
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Til Aach
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Behrens.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Behrens, A., Bommes, M., Stehle, T. et al. Real-time image composition of bladder mosaics in fluorescence endoscopy. Comput Sci Res Dev 26, 51–64 (2011). https://doi.org/10.1007/s00450-010-0135-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00450-010-0135-z

Keywords

Search

Navigation