Augmented reality support for self-service automated systems

Self-service automated systems can be found in almost all aspects of our lives. They can be found in the form of vending machines, ATMs, rent-a-bike systems, automatic car wash, etc. (Castro, Atkinson & Ezell, 2010). Self-service automated systems provide their services or products to customers 24/7 without requiring human presence to work. In this way, labor costs are reduced (Lin et al., 2011). According to the European Vending Association, there are around 4 million vending machines in Europe that sell about 80 million items a day. The total turnover of vending machines for 2018 is € 15 trillion (EVA, 2020). Due to the increasing use of self-service automated systems, there is a constant tendency to improve them. One form of improvement is the expansion of functionality and the introduction of new technologies. However, it is important not only to provide services but also a way of providing services. The service must be affordable, consistent and reliable. Otherwise, customers will be dissatisfied and may no longer use the service provided by the automation system or may use a third-party automated system (Wachter, 2002) and (Vicini, Sanna & Bellini, 2012). This is especially true for the new services and technologies being introduced. The cost-effectiveness of a self-service automated system depends on its productivity and the quality of service it provides (Dilijonas & Sakalauskas, 2011).


Introduction
Self-service automated systems can be found in almost all aspects of our lives. They can be found in the form of vending machines, ATMs, rent-a-bike systems, automatic car wash, etc. (Castro, Atkinson & Ezell, 2010). Self-service automated systems provide their services or products to customers 24/7 without requiring human presence to work. In this way, labor costs are reduced (Lin et al., 2011). According to the European Vending Association, there are around 4 million vending machines in Europe that sell about 80 million items a day. The total turnover of vending machines for 2018 is € 15 trillion (EVA, 2020). Due to the increasing use of self-service automated systems, there is a constant tendency to improve them. One form of improvement is the expansion of functionality and the introduction of new tech-nologies. However, it is important not only to provide services but also a way of providing services. The service must be affordable, consistent and reliable. Otherwise, customers will be dissatisfied and may no longer use the service provided by the automation system or may use a third-party automated system (Wachter, 2002) and (Vicini, Sanna & Bellini, 2012). This is especially true for the new services and technologies being introduced. The cost-effectiveness of a self-service automated system depends on its productivity and the quality of service it provides (Dilijonas & Sakalauskas, 2011).
One way for the improvement of the service and user experience can be by introducing augmented reality. Augmented reality is a way of looking at the real environment in which the computer-generated environment, such as still graphics, audio, or videos, is located.

Gordana Ostojić Sabolč Horvat Tegeltija Srđan
Augmented reality is not a new technology, it was invented in 1968 when Ivan Sutherland's develop the first head-mounted display (HMD) system (Billinghurst, Clark & Lee, 2015), (Azuma et al., 2001). HDM device enabled users to see computer-generated images combined with the real world. Although the augmented reality was an attractive concept, it never experienced widespread commercial use due to technological constraints. The hardware needed for augmented reality, such as computers, cameras, and displays, was too big to make it for commercial use. Some well-known AR headsets are the Microsoft HoloLens, the Meta 2, and the Magic Leap even though they are available for purchase none of them is truly a mass consumer device just yet. With the expansion of smartphones, augmented reality gained its importance. Development of the software development kit, such as ARCore, ARKit, Vuforia, augmented reality it is now accessible to the average developer. This practically means that all devices on iOS and Android platforms became AR-compatible devices. It is easier for the developers to build an AR experience using a variety of platforms. Although mobile devices provide low-end AR experiences, they currently cover the largest market segment for AR (Chatzopoulos et al., 2017), (Wagner & Schmalstieg, 2009). Smartphones can be used as an entry point into augmented reality. To gain a better AR experience AR headset can be used. Even though they are not yet accessible to an average user in the future this will change. Many companies such as Google, Sony, HTC and Samsung are investing heavily in the development of this field. With the development of the technology augmented reality will become more and more available to average users. AR market prediction on global AR users by Tomi Ahonen (2014) is shown in Figure 1 (Billinghurst, Clark & Lee, 2015).
Expansion of the augmented reality and self-service automated systems raises the question is how augmented reality can improve not only user experience but also the way self-service automated system work.
» Figure 1: AR market prediction on global AR users (Billinghurst, Clark & Lee, 2015) Classification of augmented reality As mentioned earlier, AR is a computer-generated content placed on top of real-world content. To be able to place and register 3D objects onto the real world, some form of tracking (positioning and orientation) of the device with respect to some coordinate system in the real world is required.
According to (Rabbi & Ullah, 2016), (Zhou, Duh & Billinghurst, 2008), (Kim et al., 2017) and (Wang, Ong & Nee, 2016), the tracking technologies for augmented reality can be classified into sensor-based, vision-based, and hybrid approaches. Sensor-based systems use data obtained from the sensors to determine the position and orientation of the device. They can be classified onto acoustic, magnetic, optical and inertial tracking. The vision-based tracking uses a camera and image recognition to detect and recognize the object and also to calculate the position and orientation of the object. There can be marker-based tracking and markerless tracking.
Marker-based tracking uses a specifically designed marker that is placed in the real world, while the markerless tracking does not require them and uses natural features of the environment to calculate position and orientation of the object, and thus make in it more flexible. Even though markerless tracking is easier for the user, the marker-based approach is currently more popular because it is easier to implement. The vision-based tracking needs only to recognize the marker and not to process the entire image received. Markerless tracking can suffer from the shape and texture of the object as well as the lighting and reflection. The hybrid-based methods combine the advantages of each method by combining them together, and thus the accuracy is enhanced.
To make augmented reality applications to be functional user interface is required. Placing objects into the real world without the ability for users to interact with them is not practical. One of the challenges is how to utilize user input without constraining the user's movements and ability to interact with the physical world. In mobile applications, the touch screen can be used. But using a touch screen is restricting users because one hand is required for holding the device and the other hand is used for manipulating (Seo & Lee, 2013). In recent years, there is an attempt to provide intuitive and natural AR interaction through tangible interfaces. Tangible interactions can be paddle-based, glove-based, and direct hand-based interactions (Seo & Lee, 2013) and (Wang, Ong & Nee, 2016). The paddle-based interaction means that some marker is used for the interaction.
Glove-based interaction means that the user wears special gloves with or without sensors. In both previous cases, the user requires to have additional hardware. Paddle-based interaction does not provide natural and intuitive movement and can be used for simple manipulations. Wearing gloves with attached markers and optional additional sensors can be inconvenient for the users. Hand-based interaction is most natural as users use their hands to interact with the environment. However, detecting hands and gestures is difficult since the detection is influenced by light and illumination conditions. To augmented reality become fully functional an easy and intuitive user interaction is required.
The research and technology progress in the future will enable better object tracking and a natural and intuitive user interface with the augmented objects.

Supporting the self-service automated systemsby augmented reality
The world's increasing demand for quick and personalized service has lead to an increase in using self-service automated systems in different areas. With the concept of the Internet of Things (IoT) data are collected, analyzed, and are available for further use. IoT extends the connectivity between things with minimal need for human intervention. Data are collected and analyzed to increase the efficiency and productivity of the self-service automated system (Stankovski, Ostojić & Zhang, 2016) and . Self-service automated systems become digitalized, allowing them not only to provide the service but also to have a marketing service. Smart-systems that are more service-oriented cloud solutions, interconnected and intelligent systems enable retailers to provide a better shopping experience to customers (Demirkan & Spohrer, 2014) and (Nemet et al., 2019).Understanding and prediction of customer's behaviorare very important characteristics/attributes of the intelligent/smartself-service automated systems.  and (Ostojić et al., 2015). Edge computing is a computer architecture that addresses some of these problems and enables well synergy with Cloud computing (ECC & AII, 2017;Stankovski et al., 2020a;Stankovski et al., 2020b). Also, these architectures are suitable for the implementation of Big Data technologies and the development of hybrid databases (Zečević et al., 2018) The design of a self-service automated system must be attractive to catch the attention of customers. Further increasing user experience can be with augmented reality. User experience is a significant quality factor of interactive products and services. As more and more services emerge, the user experience will be a dominant factor for the success of interactive products and services (Irshad & Rambli, 2014). Augmented reality can have a positive and significant impact on user experience (Irshad & Rambli, 2014;Poushneh & Vasquez-Parraga, 2017).
Current studies - Pantano & Servidio, 2012;Domina, Lee, & MacGillivray, 2012;Demirkan & Spohrer, 2014, Barlow, Siddiqu & Mannion, 2004Spreer & Kallweit, 2014-indi-cate that the introduction of immersive environments in shopping experience can positively affect consumer buying behavior. The shopping experience must be as easy to use, useful while providing enjoyment. Enjoyment of the customers has the most powerful influence on consumer's towards the innovative shopping experience. Retailers are turning around to augmented reality to improve their product information and engage customers to try them. The goal is for each customer to be personalized, and based on his habits to suggest products that are interesting for him. The authors of this paper are part of the team that developed smart store showed in Figure 2. This smart shop will work in conjunction with a salesman to allow the sale of products that would require specific vending machines (Stankovski et al., 2020c). Collecting the data from the self-service automated system with the introduction of business intelligence and knowledge management services can allow personalized augmented messages to be presented to the customers in order to attract them to use the self-service automated system. The usage of mobile augmented reality applications in retailing has proven to engage user experience (Dacko, 2017). Users can try on products and get more information about them. One example is IKEA that has created the AR app where users can try out their products in their apartments. On the other hand, virtual information kiosks can be found in public places. They allow the customers to access information like product details, price, and availability through the virtual interface which is projected in front of the AR glass (Rajendran, 2018). For example, clothes retailer Topshop has a fitting room with AR support. Standing in front of augmented reality kiosks, customers can see a digital reflection of themselves without need to change the clothes (Kovics, 2020). Another example is the LEGO AR kiosk that provides a visual 3D animated model of what contents inside the box.
» Figure 2: Providing an augmented experience to the user's Navigating through the building and finding self-service systems can also be done using augmented reality. GPS system is not precise enough to navigate the users through buildings. Augmented reality can detect a real 3D object and based on it directly superimpose directions on a real-world view, therefore making the directions easier to perceive. Studies showed that AR could support accurate localization and improve user experience (Rehman & Cao, 2017). Providing directions inside the building can be personalized based on a user's habits and suggest directions. Users can find easier the self-service automated system that they prefer and thus improve their user experience.
Servicing the system can also be much easier with the augmented support for servicers. Augmented reality has proven in maintenance applications. Using the AR technology and interactive guidance can have multiple advantages and can lead to improvements in a shorter duration of the procedure of disassembly and faster component condition analysis and diagnosis (Tegeltija et al., 2016) and (Rentzos et al., 2013). Also, remote maintenance using AR can easier provide information and analytics to the servicers to resolve any issues that may arise during maintenance (Masoni et al., 2017). Servicers with the head-mounted displays can have real-time information on the go, and guidance on how to resolve problems of the self-service automated systems. Interactive tutorials can be provided with the augmented reality. Providing 3D virtual information on how to use the self-service system can make it easier for people to use it. The educational experience using augmented reality includes the interaction between real and virtual environments and powerful contextual learning experiences (Bower et al., 2014;Lee, 2012).

Conclusions
Self-service automated systems can benefit using augmented reality improving user experience while providing entertainment. User experience is shown to be one of the important criteria for the usage of self-service systems. Usage of the mobile augmented reality is now a reality and has found a way to improve user experience in retailing, but there is still much work to be done until we can use the full potential of the augmented reality.
Studies had shown that detecting hand gestures can be tricky due to light and illumination conditions. Also wearing the head-mounted device can affect user attention outdoor. Users are distracted with information that is seeing and has a reduced field of view. Information that is shown through AR glass is affected by the sun, and sometimes they are not visible enough. Using a mobile phone for the augmented reality is accessible to the users but provide a limited experience. Users have to use one hand to hold the device and constantly point out to the device in the direction of the interest. Due to the size of the mobile phone, they are providing only a small window to the augmented reality. The first results from the implementation of the smart shop show user satisfaction with augment reality for personalized user experiences.