Abstract
In this paper, we present a parallel 4D vessel reconstruction algorithm that simultaneously recovers 3D structure, shape, and motion based on multiple views of X-ray angiograms. The fundamental goal is to assist the analysis and diagnosis of interventional surgery in the most efficient way towards interactive and accurate performance. We start with a fully parallelized algorithm to extract vessels as well as their skeletons and topologies from dynamic image sequences. Then, instead of resorting to registration, we present an algorithm to formulate the reconstruction problem as an energy minimization problem with color, coherence, and topology constraints to reconstruct the 3D vessel initially, which is robust to combat noise and incomplete information in images. Next, we incorporate temporal information into our energy optimization framework to track and reconstruct 4D kinematics of the dynamic vessels, which is also capable of recovering previous incomplete and misleading shapes acquired from static images otherwise. We demonstrate our system in coronary arteries reconstruction and movement tracking for percutaneous coronary intervention surgery to help medical practitioners learn about the 3D shapes and their motions of the coronary arteries of specific patient. We envision that our system would be of high assistance for diagnosis and therapy to treat vessel-related diseases in a clinical setting in the near future.
Similar content being viewed by others
References
Sarry, L., Boire, J.-Y.: Three-dimensional tracking of coronary arteries from biplane angiographic sequences using parametrically deformable models. IEEE Trans. Med. Imaging 20(12), 1341–1351 (2001)
Cañero, C., Vilariño, F., Mauri, J., Radeva, P.: Predictive (un) distortion model and 3-d reconstruction by biplane snakes. IEEE Trans. Med. Imaging 21(9), 1188–1201 (2002)
Dumay, A.C., Gerbrands, J.J., Reiber, J.H.: Automated extraction, labelling and analysis of the coronary vasculature from arteriograms. Int. J. Card. Imaging 10(3), 205–215 (1994)
Aylward, S.R., Bullitt, E.: Initialization, noise, singularities, and scale in height ridge traversal for tubular object centerline extraction. IEEE Trans. Med. Imaging 21(2), 61–75 (2002)
Sen, A., Lan, L., Doi, K., Hoffmann, K.R.: Quantitative evaluation of vessel tracking techniques on coronary angiograms. Med. Phys. 26(5), 698–706 (1999)
Kirbas, C., Quek, F.: A review of vessel extraction techniques and algorithms. ACM Comput. Surv. (CSUR) 36(2), 81–121 (2004)
Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19(3), 203–210 (2000)
Li, Q., You, J., Zhang, D.: Vessel segmentation and width estimation in retinal images using multiscale production of matched filter responses. Expert Syst. Appl. 39(9), 7600–7610 (2012)
Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: MICCAI’98, Springer. pp. 130–137 (1998)
Condurache, A.-P., Aach, T.: Vessel segmentation in angiograms using hysteresis thresholding. In: IAPR Conference on Machine Vision Applications, pp. 269–272 (2005)
Zhang, B., Zhang, L., Zhang, L., Karray, F.: Retinal vessel extraction by matched filter with first-order derivative of gaussian. Comput. Biol. Med. 40(4), 438–445 (2010)
Van Uitert, R., Bitter, I.: Subvoxel precise skeletons of volumetric data based on fast marching methods. Med. Phys. 34(2), 627–638 (2007)
Hassouna, M., Farag, A.: Multistencils fast marching methods: a highly accurate solution to the eikonal equation on cartesian domains. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1563–1574 (2007). doi:10.1109/TPAMI.2007.1154
Zhang, T.Y., Suen, C.Y.: A fast parallel algorithm for thinning digital patterns. Commun. ACM 27(3), 236–239 (1984)
Wellnhofer, E., Wahle, A., Mugaragu, I., Gross, J., Oswald, H., Fleck, E.: Validation of an accurate method for three-dimensional reconstruction and quantitative assessment of volumes, lengths and diameters of coronary vascular branches and segments from biplane angiographic projections. Int. J. Card. Imaging 15(5), 339–353 (1999)
Messenger, J.C., Chen, S.J., Carroll, J.D., Burchenal, J., Kioussopoulos, K., Groves, B.M.: 3d coronary reconstruction from routine single-plane coronary angiograms: clinical validation and quantitative analysis of the right coronary artery in 100 patients. Int. J. Card. Imaging 16(6), 413–427 (2000)
Gollapudi, R.R., Valencia, R., Lee, S.S., Wong, G.B., Teirstein, P.S., Price, M.J.: Utility of three-dimensional reconstruction of coronary angiography to guide percutaneous coronary intervention. Catheter. Cardiovasc. Interv. 69(4), 479–482 (2007)
Movassaghi, B., Rasche, V., Grass, M., Viergever, M.A., Niessen, W.J.: A quantitative analysis of 3-d coronary modeling from two or more projection images. IEEE Trans. Med. Imaging 23(12), 1517–1531 (2004)
Sprague, K., Drangova, M., Lehmann, G., Slomka, P., Levin, D., Chow, B., et al.: Coronary x-ray angiographic reconstruction and image orientation. Med. Phys. 33, 707–718 (2006)
Hansis, E., Schäfer, D., Dössel, O., Grass, M.: Projection-based motion compensation for gated coronary artery reconstruction from rotational x-ray angiograms. Phys. Med. Biol. 53(14), 3807–3820 (2008)
Nguyen, T.V., Sklansky, J.: Reconstructing the 3-d medial axes of coronary arteries in single-view cineangiograms. IEEE Trans. Med. Imaging 13(1), 61–73 (1994)
Fessler, J.A., Macovski, A.: Object-based 3-d reconstruction of arterial trees from magnetic resonance angiograms. IEEE Trans. Med. Imaging 10, 25–39 (1991)
Liu, I., Sun, Y.: Fully automated reconstruction of three-dimensional vascular tree structures from two orthogonal views using computational algorithms and productionrules. Opt. Eng. 31(10), 2197–2207 (1992)
Weng, J., Ahuja, N., Huang, T.S.: Optimal motion and structure estimation. IEEE Trans. Pattern Anal. Mach. Intell. 15(9), 864–884 (1993)
Chen, S.-Y.J., Hoffmann, K.R., Carroll, J.D.: Three-dimensional reconstruction of coronary arterial tree based on biplane angiograms. Proc. SPIE Med. Imag. Image Process. 2710, 103–114 (1996)
Chen, S.Y.J., Metz, C.E.: Improved determination of biplane imaging geometry from two projection images and its application to 3-d reconstruction of coronary arterial trees. Med. Phys. 24, 633–654 (1997)
Ruan, S., Bruno, A., Coatrieux, J.-L.: Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography. Image Vis. Comput. 12(10), 683–689 (1994)
Puentes, J., Roux, C., Garreau, M., Coatrieux, J.-L.: Dynamic feature extraction of coronary artery motion using dsa image sequences. IEEE Trans. Med. Imaging 17(6), 857–871 (1998)
Ingrassia, C., Windyga, P., Shah, M.: Segmentation and tracking of coronary arteries. In: BMES/EMBS Conference, vol. 1, IEEE, pp. 203–203 (1999)
Chen, S.-Y., Carroll, J.D.: Kinematic and deformation analysis of 4-d coronary arterial trees reconstructed from cine angiograms. IEEE Trans. Med. Imaging 22(6), 710–721 (2003)
Naegel, B., Passat, N., Ronse, C.: Grey-level hit-or-miss transforms-Part I: unified theory. Pattern Recognit. 40(2), 635–647 (2007)
Shechter, G., Devernay, F., Coste-Manière, E., Quyyumi, A., McVeigh, E.R.: Three-dimensional motion tracking of coronary arteries in biplane cineangiograms. IEEE Trans. Med. Imaging 22(4), 493–503 (2003)
Shechter, G., Resar, J.R., McVeigh, E.R.: Displacement and velocity of the coronary arteries: cardiac and respiratory motion. IEEE Trans. Med. Imaging 25(3), 369–375 (2006)
Blondel, C., Malandain, G., Vaillant, R., Ayache, N.: Reconstruction of coronary arteries from a single rotational x-ray projection sequence. IEEE Trans. Med. Imaging 25(5), 653–663 (2006)
Bouattour, S., Arndt, R., Paulus, D.: 4D reconstruction of coronary arteries from monoplane angiograms. In: Computer Analysis of Images and Patterns, Springer. pp. 724–731 (2005)
Schoonenberg, G., Florent, R., Lelong, P., Wink, O., Ruijters, D., Carroll, J., ter Haar, B.: Projection-based motion compensation and reconstruction of coronary segments and cardiac implantable devices using rotational x-ray angiography. Med. Image Anal. 13(5), 785–792 (2009)
Geman, S., Geman, D.: Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)
Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp. 282–289 (2001)
Pearl, J.: Reverend bayes on inference engines: a distributed hierarchical approach. AAAI. pp. 133–136 (1982)
Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Proceedings of the Tenth IEEE International Conference on Computer Vision, ICCV ’05, IEEE Computer Society, pp. 428–435 (2005)
Murphy, K.P., Weiss, Y., Jordan, M.I.: Loopy belief propagation for approximate inference: An empirical study. In: Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc. pp. 467–475 (1999)
Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)
Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical mrf parameters. In: Ninth IEEE International Conference on Computer Vision, IEEE, pp. 900–906 (2003)
Potetz, B., Lee, T.S.: Efficient belief propagation for higher-order cliques using linear constraint nodes. Comput. Vis. Image Underst. 112(1), 39–54 (2008)
Brunton, A., Shu, C., Roth, G.: Belief propagation on the gpu for stereo vision. In: The 3rd Canadian Conference on Computer and Robot Vision, pp. 76–76 (2006)
Coughlan, J., Shen, H.: Dynamic quantization for belief propagation in sparse spaces. Comput. Vis. Image Underst. 106(1), 47–58 (2007)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70(1), 41–54 (2006)
Boykov, Y.Y., Jolly, M.-P.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: Eighth IEEE International Conference on Computer Vision, vol. 1, pp. 105–112 (2001)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)
Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)
Vezhnevets, V., Konouchine, V.: Growcut: Interactive multi-label nd image segmentation by cellular automata. In: Proc. of Graphicon, pp. 150–156 (2005)
Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Gr. (TOG) ACM. 23, 309–314 (2004)
Acknowledgments
This work was supported in part by National Natural Science Foundation of China (Grant No. 61190120, 61190121, 61190125, 61300068, 61300067), National Science Foundation of USA (Grant No. IIS-0949467, IIS-1047715, and IIS-1049448), the National High Technology Research and Development Program (863 Program) of China (Grant No. 012AA011503), Postdoctoral Science Foundation of China (Grant No. 2013M530512).
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1 (MP4 85,218 kb)
Rights and permissions
About this article
Cite this article
Liu, X., Hou, F., Hao, A. et al. A parallelized 4D reconstruction algorithm for vascular structures and motions based on energy optimization. Vis Comput 31, 1431–1446 (2015). https://doi.org/10.1007/s00371-014-1024-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-014-1024-4