A review on making things see: Augmented reality for futuristic virtual educator

Abstract In the past few years many choreographers have focused upon implementation of computer technology to enhance their artistic skills. Computer vision technology presents new methods for learning, instructing, developing, and assessing physical movements as well as provides scope to expand dance resources and rediscover the learning process. This work reviews the study done over AR based learning technologies for the development of inter personal skills. This paper further elaborates the literature done till date within the scope of AR based training for educational aspects. The review focuses on the techniques categorized according to the type of dance learning method which can further be enhanced and addressed by means of novel AR based technology. The authors aim to provide an overview for learning standards based on AR Kinect sensors. In addition the future work is toward exploring the latest version of Kinect V2 for dance training that could become the next futuristic virtual educator.


PUBLIC INTEREST STATEMENT
Dance learning through television or pre-recorded video leads to improper training. The reason for ineffective dance learning based on existing methods is the inability to provide timely feedback and lack of interactivity, where the dancers have limited options to control the media. However in this modern technology era, technological assisted approaches have been innovated to aid people. In this paper, we reviewed the techniques categorized according to the type of dance learning methods which can further be enhanced by means of modern technologies. A brief background is supported by data showing learning phases, articles according to learning stages, and technology-based learning. As a case study, we selected India as a country that practices various types of dance and have one of the highest populations for motivational dance training skills. The paper then provides the benefits and suggestions of utilizing futuristic technologies to expertise in dance training skills.

Dance learning technology
Learning dance and mastering in a particular dance style can turn out to be the most challenging task for new learners. Although this will need a choreographer or professional training sessions, technology has replaced such traditional way of learning. People who consider dance as their hobby or who hardly find time to look for a trained teacher or attend any dance classes; will mostly go for self-learning or in-house practice. Technology has made available numerous ways to search for information or video to make self-learning easier. But the major drawback is that the online websites do not provide any interactive feedback and inspiration to the learner by which the learning process becomes less interesting and ineffective.
Recently, there have been many advances in movement training, self-paced dance learning and choreographic demonstrations. The different learning activities can be adopted depending upon the learning method and content (Guyon, Athitsos, Jangyodsuk, & Escalante, 2014;Parrish, 2007). The emergence of dance video games and fitness training with gaming console and tracking technology such as Microsoft Kinect, motion sensors, augmented and virtual reality environments, and visualization methods have made movement skill better but the focus remains mainly on entertainment. For human beings, facial expressions occupy an important position in interpersonal and non-verbal communication. They are influenced by changes of internal emotion states and generated by the movements of facial muscles. Over the last two decades, numerous researches have focused on the analysis of these facial motions and the recognition of related emotions which can be noted from Table 1. Till date, those tasks are still a challenging issue in both psychology and CV. The existing methodologies in dance technology have focused upon just training a single person with the limited number of data-sets.

AR applications
Augmented reality has numerous applications in the education sector. We can classify application areas as follows: (1) Knowledge acquisition, (2) Social development, (3) Education and training, (4) Navigation and construction. Figure 1 represents the significant transformation and the learning shift from traditional aspects toward multimedia services. AR also has made its impact on the navigation and construction industry. The last decade has seen a significant transformation in learning shift where AR has entered into wide variety of domains. The dance and music learning has also become more interesting with the employment of AR technology and AR handheld devices.
The state of art that has been dealt with the authors in the literature in the past decades, includes many applications such as helping the students to understand the X-ray view of a human body in medical sector as described by authors in Kandikonda (2011) allowing the architect students to not only see the structure of a building but also to get insights into the underlying infrastructure. In the field of navigation, AR has been employed in handheld devices and user interfaces for providing location information, traffic updates, route finding, data about new places of interests, and many varied applications. The entertainment and attractiveness of AR have created a pathway for evolution of AR theme park in South Korea. More information about this AR theme park is provided at http://www.thelivepark.com/blog/en/archives/category/about. The educational perspective has made its major transformation from classroom training toward computer visional augmented world as described in Table 2. This significant shift has made many students to opt for AR enabled learning rather than traditional methods. In this paper, the authors have reviewed the AR based dance learning methods and the impact of CV and AR in educational sector.

Advancement in AR technology
Learning is said to be a life-long process. But educating an individual is tailored to the learner's level of understanding, competence, and abilities. Our objectives are to review the AR based learning strategies till date and discuss the existing education and information-seeking processes as described in Table 3. Motivation must be considered as one of the important factor because it is directly linked to learning achievements. Therefore, AR applications, which are interactively and visually understandable, seems more attractive and motivating than traditional tools to the learner's perspective. On the other hand, games for learning are valued for their inspirational power and user-oriented features . Generally, the concreteness of game-based actions tends to involve pragmatism, if not enthusiasm, or trial and error. This learning phenomenon offers enormous directions for training, based on strong personal interest and inspiration.
CV has a major impact on the younger generation due to advancements in technology and the modern era of cyberspace world. The authors till date have presented different theories of enhanced learning through the discoveries made and developed in the last two decades (Hall et al., 2002;Kaufmann & Schmalstieg, 2003;Schall, 2009). Learning through AR helps the students with external amplification, internal rewards, challenge, and increased self-confidence (Chen, 2006). Therefore our survey deals with the reviewing of how the existing learning technology in AR influences the learner's insights and performance. AR based education is deemed to provide constructivism which is considered to be the fundamental theory of motivation and self-inspiration in educational sector as depicted in Figure 2. Eventually the aim of this review is to provide perspectives for AR based education, for specializing in a particular aspect, with interesting features, lower costs, easier understandability and reusability of content, easier integration with existing and future applications.

Different types of dance in India
India has been selected as our case study since various forms of dance are practiced and learnt by people of all ages varying from young children to adults.
According to Wikipedia, "Dance is the art of movement of the body, usually rhythmically and to music, using prescribed or improvised steps and gestures." Indian classical dance is one of the oldest dance traditions associated with worlds majority of religious culture and heritage. In Indian culture, dance and music play a vital role for religious worship and spirituality. From the historical aspects, it is evident that lord Shiva has his imprint on the importance of dance and music. The presence of dance acts as a way of reacting and reaching out spiritually to the holy power of divinity in Hinduism. People in India also engage in dance and singing very frequently on a variety of occasions such as festivals, cultural programs, social events, and dance reality shows as depicted in Figure 3. In India, Bollywood, Kollywood, and Tollywood films are also deeply rooted with dance and music numbers that makes them unique when compared with Hollywood movies. Dance is vitally essential to Indian identity and plays key role as an inherit part of expression of Indianness.
It would appear that by the time of earliest civilizations in Indian subcontinent, dance had achieved a considerable way of expressing physical gestures. Indian classical dance is a gesture of all body parts each class of dance has its own way of learning and expression. The main drawback of existing solutions is that during dance performance due to obstructions it becomes difficult to capture all the gestures with the help of existing technology (Samanta & Chanda, 2013). In Saha,  Ghosh, Konar, and Janarthanan (2013), authors have introduced an algorithm for identification of dance video by recognizing posture from each frame for the purpose of e-learning by taking Indian classical dance oddisi as the input. The researchers in Majumdar (2012) have illustrated a method to develop a digital bharathanatiyam interaction. The existing solutions have focused only upon specific dance forms and hence cannot be applied to other dance movements. Therefore this study  deals with reviewing various existing dance learning technology for the different types of dances in India. Figure 4 and Table 4 show different types of dances in India according to region.

From traditional learning toward computer vision
The process of educating an individual has emerged from classroom teaching, motor learning, elearning toward CV. During the 1990s the advancements of technology gave way to computer-based training. The following decades eventually have witnessed the proliferation of information sharing, embedded systems, artificial intelligence, neural networks, and cloud computing. The authors in Han, Shao, Xu, and Shotton (2013) have presented a comprehensive resent Kinect based CV algorithms and applications. The reviewed approaches are classified according to vision problems and include topics such as preprocessing, object tracking, object recognition, human activity analysis, hand gesture analysis, and indoor 3D mapping. The preliminary explanation of AR technology adaption for teaching support and learning enhancement have been identified, summarized, and analyzed using interactive marker AR technology based models. The researchers in Wang et al. (2014) have presented an AR navigation system with automatic marker free image registration using 3D image overlay and stereo tracking for dental surgery. Image registration is performed by patient tracking and real-time 3D contour matching without requiring any fiducial and reference markers(fiducial is an object placed in the field of view of an imaging system which appears in the image produced, for use as a point of reference or a measure). Although there are many educational prototypes in the current literatures, the research contribution in Santos et al. (2014) urge that only a few prototypes are developed by interdisciplinary groups and base their work on learning theory. Thus their work enumerates the affordance of AR for leaning and discusses learning theories relevant to future AR educational content. The authors have outlined a comparative study of various AR software development kits (SDk's) to create AR applications. The paper (Amin & Govilkar, 2015) provides a comprehensive description on how AR is different from Virtual reality, working scenarios of AR, and various types of tracking employed in AR technology. Last few decades have shown a significant transformation from traditional learning toward CV in educational sector.

AR in education and training
Augmented reality as defined earlier is a technology that makes use of CV techniques to collaborate computer generated virtual objects with real-time environment in order to increase or to enhance what can be visualized by the human user (Hall et al., 2002). In educational sector AR can be used in dance education, Chemistry, Biology, Astronomy, Mathematics, Computer graphics, etc. AR in chemistry education is used for the exploration of physical models of amino acids (Fjeld & Voegtli, 2002). In biology AR based learning system is used to get insights into the interior of human organs on a detailed basis with easier understanding (Gillet, Sanner, Stoffler, Goodsell, & Olson, 2004). In astronomy AR technology is applied for better understanding of seasonal changes in light and temperature, rotation/revolution of Sun and Earth (Shelton & Hedley, 2002). AR is also used as a visualization tool in computer graphics laboratories and in computer aided design lectures as described in Kaufmann and Meyer (2008) and Chen (2006). In mathematics AR is used for teaching calculus and algorithms (Kaufmann & Schmalstieg, 2003). This research throws light on various benefits of AR applications in educational environment. The authors have eventually concluded that improved learning curve and increased motivation account for more than 20% of all the advantages of AR in educational aspects mentioned in literature till date (Diegmann, Schmidt-kraepelin, Van Den Eynden, & Basten, 2015). In the field of dance education Kinect sensors are employed to train the students in physical movements as well as master them in their skills.

Fun filled AR based dance learning using Kinect
Computer vision and motion sensing technologies have enabled the users to actively, physically, and mechanically interact with the digital environment in varied ways. The hybrid combinations of traditional art forms and advanced CV techniques have made the authors in the last decade to drive out to AR based dance leaning systems. The researchers in Guyon et al. (2014) have described Cha learn gesture data-set that is user dependent, small vocabulary, and one-shot learning using Kinect camera. In Silva et al. (2013), the authors have developed a prototype to monitor fall risk while playing a game using smart phone accelerometer. The experimenters in Fan, Xu, and Geng (2012) have introduced a novel method for synthesizing dance motion that follows the emotions and the contents of a piece of music. The research contribution in Yang, Leung, Yue, and Deng (2012) had presented an automatic dance lesson generation system which is suitable in a learning-by-mimicking scenario where the learning objects can be represented as multi-attribute time series data. The authors in Chan, Leung, Tang, and Komura (2011) proposed a new dance training system based on motion capture and virtual reality technologies. The research in Golshani, Vissicaro, and Park (2004) had presented multimedia information repository for cross cultural dance studies such as East Indian dance with the use of two 3D Vicon motion capture systems. The researchers in Eichner and Ferrari (2012) proposed techniques for novel Human Pose Co-estimation for joint pose estimation over multiple persons in a common, but unknown, pose. In Kuramoto, Nishimura, Yamamoto, Shibuya, and Tsujino (2013), a visualization method of velocity and acceleration of teacher's motion for the learner to understand more clearly and easily has been proposed. This research provides an updated interactive performance system for floor and aerial dance that controls visual and sonic aspects of the presentation through Microsoft Kinect camera. Improvised gesture recognition and tracking system called Action Graph (AG) is described in this article, which has the capability of capturing incoming gestures in an unsupervised way & enables mapping between input gestures to desired rendering functionalities (Wang, 2015). The authors of Anderson, Grossman, Matejka, and Fitzmaurice (2013) have discussed a novel system YouMove that allows users to record and learn physical movement sequences. The Kinect-based recording system is designed to be simple, allowing anyone to create and share training content, some of the screen shots are shown in Figure 5. The corresponding training system uses recorded data to train the user using a large-scale AR mirror. The system trains the user through a series of stages that gradually reduce the user's reliance on guidance and feedback. This also discusses the design and implementation of YouMove and its interactive mirror. The authors have presented a user study in which YouMove was shown to improve learning and short-term retention by a factor of 2 compared to a traditional video demonstration. While the presented implementation uses a half-silvered mirror as a display, the software could also run as a traditional videobased AR system. The Kinect has difficulty tracking movements that cause large amounts of occlusions. This would be more accessible to users, but does not provide the real-time feedback that the mirror does. It would be interesting to better understand any learning difference between a mirror and video-based system on various devices (large screen, small screen, etc.). The addition of social features and richer inclusions of gaming technologies could also greatly help YouMove. One can imagine online yoga, dance or martial arts classes, with competition from online peer groups, but more work is needed to achieve this. http://dx.doi.org/10.1080/2331186X.2017.1287392

Feature comparison of Kinect V2 with Kinect V1
The features of Kinect V1 and Kinect V2 are tabulated in    Figure 6 represents cyclic taxonomy view of this review where AR based learning technologies were reviewed for dance education that have employed Kinect sensors. There are numerous advantages of AR in educational sector some of which are interactive, easier to understand, and attractive way for learning and teaching according to past studies.

Benefits of AR in educational sector
Other benefits include stimulation of conceptual thinking, constructivism, receiving greater sense of 3D space, theoretical understanding, perception enhancement, and fun-based interaction as depicted in Figure 7. The features of AR based learning include robust, cheaper, user friendly, animated, timely, easily understandable, responsive, seamless integration with other media, interesting, reliable, efficient, and portable (Carmigniani et al., 2011;Dong & Kamat, 2013;Jee, Lim, Youn, & Lee, 2014).

Conclusion
Augmented reality has made distinctive contributions toward learning experiences. The developments in CV technology have led researchers to enhance and assess AR learning aspects. In the course of this review there are many interesting facts that can be derived cyclic taxonomy of which is illustrated in Figure 6. The evolution of learning paradigm from traditional methods toward CV, relative to the theory of education technology has showed a significant transformation and has eventually led researchers and technologists to adopt AR as one of the promising direction for multimedia learning technology. The insights for AR technology in navigation and construction industries provide an essential need for comprehensive systems, integration of multiple platforms, user friendly interfaces, defect detection, and seamless integration at an affordable cost. The AR technology can be used for group as well as individual learning which motivates the learner in every possible way. The advantage of AR in educational sector includes opportunity to visualize digital information, observe the finer details of subjects and possibilities to examine the virtual information perceptively as many times as needed. Hence AR has made its major contribution toward knowledge acquisition in educational sector during the last few decades. There has also been emergence of AR based dance learning technology with tracking capabilities such as Microsoft Kinect. This technology has been employed for fun-filled dance learning with responsive interactions and self-motivating feedback. It can further be concluded that the traditional dances can be explored and the cultural heritage can be preserved with CV technology. The Kinect V2 sensor is seen as a future direction for AR based motion recognition and gesture detection, for which the work is underway.