Abstract
Many vision applications require high-accuracy dense disparity maps in real-time and online. Due to time constraint, most real-time stereo applications rely on local winner-takes-all optimization in the disparity computation process. These local approaches are generally outperformed by offline global optimization based algorithms. However, recent research shows that, through carefully selecting and aggregating the matching costs of neighboring pixels, the disparity maps produced by a local approach can be more accurate than those generated by many global optimization techniques. We are therefore motivated to investigate whether these cost aggregation approaches can be adopted in real-time stereo applications and, if so, how well they perform under the real-time constraint. The evaluation is conducted on a real-time stereo platform, which utilizes the processing power of programmable graphics hardware. Six recent cost aggregation approaches are implemented and optimized for graphics hardware so that real-time speed can be achieved. The performances of these aggregation approaches in terms of both processing speed and result quality are reported.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Birchfield, S. and Tomasi, C. 1998. A pixel dissimilarity measure that is insensitive to image sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(4):401–406.
Bobick, A.F. and Intille, S.S. 1999. Large occlusion stereo. International Journal of Computer Vision, 33(3):181–200.
Boykov, Y., Veksler, O., and Zabih, R. 1998. A variable window approach to early vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1283–1294.
Forstmann, S., Ohya, J., Kanou, Y., Schmitt, A., and Thuering S. 2004. Real-time stereo by using dynamic programming. In Proc. CVPR Workshop on Real-time 3D Sensors and Their Use, Washington, DC, USA, pp. 29–36.
Fusiello, A., Roberto, V., and Trucco, E. 1997. Efficient stereo with multiple windowing. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Puerto Rico, pp. 858–863.
Gong, M. and Yang, R. 2005. Image-gradient-guided real-time stereo on graphics hardware. In Proc. International Conference on 3-D Digital Imaging and Modeling, Ottawa, ON, Canada, pp. 548–555.
Gong, M. and Yang, Y.-H. 2005. Near real-time reliable stereo matching using programmable graphics hardware. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, pp. 924–931.
Harwood, D., Subbarao, M., Hakalahti, H., and Davis, L. 1987. A new class of edge-preserving smoothing filters. Pattern Recognition Letters, 6:155–162.
Hirschmuller, H., Innocent P.R., and Garibaldi, J. 2002. Real-time correlation-based stereo vision with reduced border errors. International Journal of Computer Vision, 47:1–3.
Kanade, T. and Okutomi, M. 1994. Stereo matching algorithm with an adaptive window: theory and experiment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(9):920–932.
Kim, J.-C., Lee, K.M., Choi, B.-T., and Lee, S. U. 2005. A dense stereo matching using two-pass dynamic programming with generalized ground control points. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, 1075–1082.
Scharstein, D. and Szeliski, R. 2002. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1–3):7–42.
Scharstein, D. and Szeliski, R. 2003. High-accuracy stereo depth maps using structured light. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Madison, Wisconsin, pp. 195–202.
Tao, H. and Sawhney, H.S. 2000. Global matching criterion and color segmentation based stereo. In Proc. Workshop on the Application of Computer Vision, pp. 246–253.
Veksler, O. 2003. Fast variable window for stereo correspondence using integral images. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 556–561.
Wang, L. Gong, M., Gong, M., and Yang, R. 2006. How far can we go with local optimization in real-time stereo matching: A performance study on different cost aggregation approaches. In Proc. International Symposium on 3D Data Processing, Visualization and Transmission, Chapel Hill, NC, USA.
Wang, L., Kang, S.B., Shum H.-Y., and Xu, G. 2004. Cooperative segmentation and stereo using perspective space search. In Proc. Asian Conference on Computer Vision, Jeju Island, Korea, pp. 366–371.
Wang, L., Liao, M., Gong, M., Yang, R., and Nister, D. 2006. High quality real-time stereo using adaptive cost aggregation and dynamic programming. In Proc. International Symposium on 3D Data Processing, Visualization and Transmission, Chapel Hill, NC, USA.
Woodfill, J. and Von Herzen, B. 1997. Real-time stereo vision on the PARTS reconfigurable computer. In Proc. IEEE Symposium on FPGAs for Custom Computing Machines, pp. 201–210.
Yang, Q., Wang, L., Yang, R., Wang, S., Liao, M., and Nister, D. 2006. Real-time global stereo matching using hierarchical belief propagation. In Proc. British Machine Vision Conference, Edinburgh, UK.
Yang R. and Pollefeys, M. 2003. Multi-resolution real-time stereo on commodity graphics hardware. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp. 211–220.
Yang, R., Pollefeys, M., and Li, S. 2004. Improved real-time stereo on commodity graphics hardware. In Proc. CVPR Workshop on Real-time 3D Sensors and Their Use, Washington, DC, USA.
Yoon, K.-J. and Kweon, I.-S. 2005. Locally adaptive support-weight approach for visual correspondence search. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, pp. 924–931.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Gong, M., Yang, R., Wang, L. et al. A Performance Study on Different Cost Aggregation Approaches Used in Real-Time Stereo Matching. Int J Comput Vis 75, 283–296 (2007). https://doi.org/10.1007/s11263-006-0032-x
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-006-0032-x