Skip to main content
SpringerLink
Log in

Virtual Volumetric Graphics on Commodity Displays Using 3D Viewer Tracking

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Three dimensional (3D) displays typically rely on stereo disparity, requiring specialized hardware to be worn or embedded in the display. We present a novel 3D graphics display system for volumetric scene visualization using only standard 2D display hardware and a pair of calibrated web cameras. Our computer vision-based system requires no worn or other special hardware. Rather than producing the depth illusion through disparity, we deliver a full volumetric 3D visualization—enabling users to interactively explore 3D scenes by varying their viewing position and angle according to the tracked 3D position of their face and eyes. We incorporate a novel wand-based calibration that allows the cameras to be placed at arbitrary positions and orientations relative to the display. The resulting system operates at real-time speeds (∼25 fps) with low latency (120–225 ms) delivering a compelling natural user interface and immersive experience for 3D viewing. In addition to objective evaluation of display stability and responsiveness, we report on user trials comparing users’ timings on a spatial orientation task.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. A video demo of the system is available in the Supplementary Material.

References

  • Alnowami, M., Alnwaimi, B., Copland, M., & Wells, K. (2011). A quantitative assessment of using the Kinect for Xbox 360 for respiratory surface motion tracking. In Proc. SPIE medical imaging.

    Google Scholar 

  • Brar, L., Sexton, I., Surman, P., Bates, R., Lee, W., Hopf, K., Neumann, F., Day, S., & Williman, E. (2010). Laser-based head-tracked 3D display research. Journal of Display Technology, 6(10), 531–543.

    Article  Google Scholar 

  • Chen, C., Huang, Y., Chuang, S., Wu, C., Shieh, H., Mphepo, W., Hsieh, C., & Hsu, S. (2009). Liquid crystal panel for high efficiency barrier type autostereoscopic 3D displays. Applied Optics, 48(18), 3446–3454.

    Article  Google Scholar 

  • Dang, T., Hoffmann, C., & Stiller, C. (2009). Continuous stereo self-calibration by camera parameter tracking. IEEE Transactions on Image Processing, 18(7), 1536–1549.

    Article  MathSciNet  Google Scholar 

  • Dodgson, N. (2004). Variation and extrema of human interpupillary distance. Proceedings of SPIE, 5291, 36–46.

    Article  Google Scholar 

  • Ellis, S. R., Wolfram, A., & Adelstein, B. D. (2002). Three dimensional tracking in augmented environments: user performance trade-offs between system latency and update rate. Proceedings of the Human Factors and Ergonomics Society annual meeting, 46(26), 2149–2153.

    Article  Google Scholar 

  • Erden, E., Kishore, V., Urey, H., Baghsiahi, H., Willman, E., Day, S., Selviah, D., Fernandez, F., & Surman, P. (2009). Laser scanning based autostereoscopic 3D display with pupil tracking. In Proc. IEEE photonics (pp. 10–11).

    Google Scholar 

  • Ezra, D., Woodgate, G., Omar, B., Holliman, N., Harrold, J., & Shapiro, L. (1995). New autostereoscopic display system. In Proc. SPIE Intl. Society of Optical Engineering (pp. 31–40).

    Google Scholar 

  • Free2C (2010). The free2c desktop display (Technical report). Heinrich Hertz Institute.

  • Freund, Y., & Schapire, R. (1999). A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence, 14(7), 771–780.

    Google Scholar 

  • Lee, J. (2008). Head tracking for desktop VR displays using the WiiRemote (Technical report). Carnegie Mellon University.

  • Malleson, C., & Collomosse, J. (2011). Volumetric 3D graphics on commodity displays using active gaze tracking. In Proc. ICCV workshop on human computer interaction.

    Google Scholar 

  • Nishimura, H., Abe, T., Yamamoto, H., Hayasaki, Y., Nagai, Y., Shimizu, Y., & Nishida, N. (2007). Development of a 140-inch autostereoscopic display by use of full-color LED panel. Proceedings of SPIE, the International Society for Optical Engineering, 6486, 64861B.

    Article  Google Scholar 

  • OpenCV. Open source computer vision library. Accessed July 2011.

  • Perlin, K., Poultney, C., Kollin, J., Kristjansson, D., & Paxia, S. (2001). Recent advances in the NYU autostereoscopic display. Proceedings of SPIE, the International Society for Optical Engineering, 4297, 196–203.

    Article  Google Scholar 

  • Sandin, D., Margolis, T., Dawe, G., Leigh, J., & DeFanti, T. (2001). Varrier autostereographic display. Proceedings of SPIE, the International Society for Optical Engineering, 4297, 204–211.

    Article  Google Scholar 

  • Schwartz, A. (1985). Head tracking stereoscopic display. In Proc. IEEE intl. conf. on display research (pp. 141–144).

    Google Scholar 

  • Sorensen, S., Hansen, P., & Sorensen, N. (2004). Method for recording and viewing stereoscopic images in color using monochrome filters. U.S. Patent 6687003.

  • Surman, P., Sexton, I., Hopf, K., Lee, W., Buckley, E., Jones, G., & Bates, R. (2008a). European research into head tracked autostereoscopic displays. In Proc. conf on 3DTV (pp. 161–164).

    Google Scholar 

  • Surman, P., Sexton, I., Hopf, K., Lee, W., Neumann, F., Buckley, E., Jones, G., Corbett, A., Bates, R., & Talukdar, S. (2008b). European research into head tracked autostereoscopic displays. Journal of the Society for Information Display, 16, 743–753.

    Article  Google Scholar 

  • Takaki, Y. (2006). High-density directional display for generating natural 3D images. Proceedings of the IEEE, 94(3), 654–663.

    Article  Google Scholar 

  • Tetsutani, N., Ichinose, S., & Ishibashi, M. (1989). 3D-TV projection display system with head-tracking. In Japan Display (pp. 56–59).

    Google Scholar 

  • Tetsutani, N., Omura, K., & Kishino, F. (1994). Study on a stereoscopic display system employing eye-position tracking for multi-viewers. Proceedings of SPIE, the International Society for Optical Engineering, 2177, 135.

    Article  Google Scholar 

  • Thacker, N. A. (1992). Online calibration of a 4 DOF stereo head. In Proc. British machine vision conference (BMVC) (pp. 528–537).

    Google Scholar 

  • Tsai, R., Tsai, C., Lee, K., Wu, C., Lin, L., Huang, K., Hsu, W., Wu, C., Lu, C., Yang, J., & Chen, Y. (2009). Challenge of 3D LCD displays. Proceedings of SPIE, the International Society for Optical Engineering, 7329, 732903.

    Article  Google Scholar 

  • Urey, H., & Erden, E. (2011). State of the art in stereoscopic and autostereoscopic displays. Proceedings of the IEEE, 99(4), 544–555.

    Article  Google Scholar 

  • Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proc. computer vision and pattern recognition.

    Google Scholar 

  • Welch, B. L. (1947). The generalization of student’s problem when several different population variances are involved. Biometrika, 34(1–2), 28–35. doi:10.1093/biomet/34.1-2.28.

    MathSciNet  MATH  Google Scholar 

  • Woodgate, G., Ezra, D., Harrold, J., Holliman, N., Jones, G., & Moseley, R. (1997). Observer-tracking autostereoscopic 3D display systems. Proceedings of SPIE, the International Society for Optical Engineering, 3012, 187–198.

    Article  Google Scholar 

  • Woodgate, G., Harrold, J., Jacobs, A., Mosely, R., & Ezra, D. (2000). Flat-panel autostereoscopic displays: characterization and enhancement. Proceedings of SPIE, the International Society for Optical Engineering, 3957, 153–164.

    Article  Google Scholar 

  • Woods, A. (2009). 3D displays in the home. Information Display, 7, 8–12.

    Google Scholar 

  • Yamamoto, H., Kouno, M., Muguruma, S., Hayasaki, Y., Nagai, Y., Shimizu, Y., & Nishida, N. (2002). Enlargement of viewing area of stereoscopic full-color LED display using parallax barrier combined with polarizer. Applied Optics, 41(32), 6907–6919.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Centre for Vision Speech and Signal Processing, University of Surrey, Guildford, Surrey, UK

    Charles Malleson & John Collomosse

Authors
  1. Charles Malleson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John Collomosse
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Collomosse.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

(AVI 6.2 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Malleson, C., Collomosse, J. Virtual Volumetric Graphics on Commodity Displays Using 3D Viewer Tracking. Int J Comput Vis 101, 519–532 (2013). https://doi.org/10.1007/s11263-012-0533-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0533-8

Keywords

Search

Navigation