Abstract
This paper presents a comparison of latest software and hardware methods for rapid reconstruction of real humans using as an input RGB or RGB-D images, and base on this comparison is introduced the pipeline that produces high realistic reconstructions in a reasonable amount of time, suitable for real-time Virtual Reality (VR), Augmented Reality (AR) as well as Holographic Mixed Reality (HMR). In this work, we also present and compare usage of latest VR and AR Head Mounted displays (HDMs), which are Microsoft Hololens, Oculus Rift and mobile AR. Specifically, we compare the immersion experience, interaction system, field of view and level of presence that each of these technologies provide. We demonstrate our results at Asinou church, a UNESCO Cultural Heritage monument located in Cyprus. Our reconstructed virtual narrator is the real priest of Asinou church which gives a virtual tour in the church. This interactive virtual narrator supports a range of different capabilities like performing gestures, speech and lip synchronization.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Microsoft Hololens. www.microsoft.com/hololens. Accessed 12 May 2017
Mixamo. www.mixamo.com. Accessed 12 May 2017
Arnold, D., Day, A., Glauert, J., Haegler, S., Jennings, V., Kevelham, B., Laycock, R., Magnenat-Thalmann, N., Maïm, J., Maupu, D., Papagiannakis, G., Thalmann, D., Yersin, B., Rodriguez-Echavarria, K.: Tools for populating cultural heritage environments with interactive virtual humans. In: Open Digital Cultural Heritage Systems, EPOCH Final Event 2008, February 2008
Georgopoulos, A.: Data acquisition for the geometric documentation of cultural heritage. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds.) Mixed Reality and Gamification for Cultural Heritage, pp. 29–73. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49607-8_2
Hildenbrand, D.: Foundations of Geometric Algebra Computing, vol. 8. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-31794-1
Kateros, S., Georgiou, S., Papaefthymiou, M., Papagiannakis, G., Tsioumas, M.: A comparison of gamified, immersive VR curation methods for enhanced presence and human-computer interaction in digital humanities. Int. J. Heritage Digital Era 4(2), 221–233 (2015). Also presented in The 1st International Workshop on ICT for the Preservation and Transmission of Intangible Cultural Heritage, EUROMED2014
Papaefthymiou, M., Feng, A., Shapiro, A., Papagiannakis, G.: A fast and robust pipeline for populating mobile AR scenes with gamified virtual characters. In: SIGGRAPH Asia 2015 Mobile Graphics and Interactive Applications, SA 2015, pp. 22:1–22:8. ACM, New York (2015)
Papaefthymiou, M., Kateros, S., Georgiou, S., Lydatakis, N., Zikas, P., Bachlitzanakis, V., Papagiannakis, G.: Gamified AR/VR character rendering and animation-enabling technologies. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds.) Mixed Reality and Gamification for Cultural Heritage. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49607-8_13
Papagiannakis, G., Schertenleib, S., O’Kennedy, B., Arevalo-Poizat, M., Magnenat-Thalmann, N., Stoddart, A., Thalmann, D.: Mixing virtual and real scenes in the site of ancient pompeii: research articles. Comput. Animat. Virtual Worlds 16(1), 11–24 (2005)
Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 2001, pp. 497–500. ACM, New York (2001)
Shapiro, A., Feng, A., Wang, R., Li, H., Bolas, M., Medioni, G., Suma, E.: Rapid avatar capture and simulation using commodity depth sensors. Comput. Anim. Virtual Worlds 25(3–4), 201–211 (2014)
Vacchetti, L., Lepetit, V., Ponder, M., Papagiannakis, G., Fua, P., Thalmann, D., Thalmann, N.M.: A stable real-time AR framework for training and planning in industrial environments. In: Ong, S.K., Nee, A.Y.C. (eds.) Virtual and Augmented Reality Applications in Manufacturing, pp. 129–145. Springer, London (2004). https://doi.org/10.1007/978-1-4471-3873-0_8
Verbiest, F., Proesmans, M., Van Gool, L.: Autonomous mapping of the Priscilla Catacombs. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds.) Mixed Reality and Gamification for Cultural Heritage, pp. 75–98. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-49607-8_3
Acknowledgments
The research leading to these results was partially funded by the European Union People Programme (FP7-PEOPLE-2013-ITN) under grant agreement 608013 and was partially funded by the Virtual Multimodal Museum (ViMM), a Coordination and Support Action (CSA), funded under the EU Horizon 2020 programme (CULT-COOP-8-2016).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Papaefthymiou, M., Kanakis, M.E., Geronikolakis, E., Nochos, A., Zikas, P., Papagiannakis, G. (2018). Rapid Reconstruction and Simulation of Real Characters in Mixed Reality Environments. In: Ioannides, M. (eds) Digital Cultural Heritage. Lecture Notes in Computer Science(), vol 10605. Springer, Cham. https://doi.org/10.1007/978-3-319-75826-8_22
Download citation
DOI: https://doi.org/10.1007/978-3-319-75826-8_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-75825-1
Online ISBN: 978-3-319-75826-8
eBook Packages: Computer ScienceComputer Science (R0)