AUGMENTED REALITY APPLICATION DEVELOPMENT USING UNITY AND VUFORIA

This article presents a method to develop an application based on Augmented Reality technology in a Unity development environment. The article presents two test scenarios that give an insight into the possibilities of augmented reality applications. The first scenario is a presentation of a simple game based on a graphic marker and its generated content in the form of a maze. By physically moving the marker in front of the camera, the maze and the ball inside is moved. The second scenario includes a simulation environment of a collaborative robot in the form of an augmented reality application that offers the possibility of planning the robot’s trajectories. The possibilities of this technology are endless and cover a large segment of application development from educational to industrial applications. Marker based Augmented Reality using predefined visual markers embedded within the system, physical world objects are detected for superimposition of virtual elements. The test application was designed using a combination of Vuforia, Unity 3D platforms, and the mobile phone used was an Android based Xiaomi Redmi 10 pro phone. This article focuses on the implementation challenges faced whilst designing Augmented Reality applications on mobile platforms.


INTRODUCTION
Augmented reality (AR) is a kind of virtual (apparent) extension of reality, when, for example, virtual elements can be projected into the real environment by looking around with a mobile phone camera or using glasses created for this purpose [1]. The horizon of meaning and appearance of augmented reality is very wide, but all of its forms have common features. The most important of these is that virtual objects are integrated into the context of the material world in real time. The process, which is definitely a part of some kind of mediatized communication, is inseparable from the technology that creates augmented reality, as it requires optics that detect the outside world (and other sensors), as well as a display that meets the requirement of natural fidelity. With the help of applications, information about the material world becomes interactive and digitized, thus becoming storable and easier to access, while being superimposed on the real world as an information layer. Augmented reality is therefore device-dependent, technically determined and convergent: it appears in the context of smartphones, films and books, and even medical visual diagnostics. This work explores the use of Unity 3D, the integrated game development tool for multiple platforms. It uses the extension of the software development kit developed by Qualcomm for mobile devices. People can experience AR, which is also known as hybrid reality in the Unity 3D environment. The application of AR technology in Unity 3D can be reflected through the design of a game and the basic use of reality enhancement can be explored.

RELATED WORK
The research discussed by Sarosa et. al. is about developing the AR system that will improve the learning process of character education by helping the teacher to provide the new interactive tools for teaching the students [2]. Desiertot et. al. aims to apply the augmented reality (AR) technology in storytelling to enhance and give a more motivating and fun reading experience to children. The primary objective of this study is to develop a bilingual AR children storybook mobile application called GoonAR [3]. Nazar et. al. are developing and measuring the usability of the Augmented Reality App used for learning Chemistry, focusing on the concept of molecular geometry. AGILE model of development was employed to step by step development, starting from the analysis of the curriculum and need assessment, followed by design, development, deployment and evaluation [4]. The work done by Koca et. al. developed an augmented reality-based education application for preschool children using Unity 3D Platform and Vuforia SDK. In the application developed for Android phones, when users display the card of one of the different animal pictures defined in the Vuforia database in the phone camera, they see the 3D character defined for that photo and hear the character's voice [5]. Subhashini et. al. presented a remarkable arrangement that utilizes augmented reality to make the learning measure more interactive and fascinating. The application when focused on text or image shows significant 3-dimensional (3D) model or video on the smart phone screen [6].

RESULTS AND DISCUSSION
AR applications can be developed using a variety of tools and frameworks, including Unity and Vuforia. Unity is a popular game engine and development platform that can be used to create AR applications. It provides a wide range of features, including a powerful editor, a scripting API, and a large community of developers. Unity supports a variety of platforms, including iOS, Android, and Windows, and can be used to create AR applications for smartphones, tablets, and head-mounted displays (HMDs) such as the Hololens [7]. Vuforia is an AR development platform that can be integrated with Unity to create AR applications [8]. It provides a variety of features, including image and object recognition, and support for 3D models and animations. Vuforia also includes an SDK (Software Development Kit) for Unity, which allows developers to easily integrate AR functionality into their Unity projects [9]. The data stream of the AR SDK is divided into four separated modules: camera input, object database, object tracking and matching and display output as depicted in Figure 1.
As shown in Figure 1 the first step is to design and import the target image in any image editing software. This can be a logo or a digital ID in the form of a QR code [10]. The next step is to design the AR experience elements such as the 3D object with or without animation using any CAD software for 3D modelling. Table 1 shows a smartphone specification used for AR application testing.

TEST SCENARIO 1 -LABYRINTH GAME
The aim of this test scenario is to create an augmented reality game in which the user have to deliver a ball through a maze to the exit, as if the virtual ball were affected by gravity. This labyrinth is actually a 2D image printed on an A4 sheet, which can be transformed into a 3D model with the help of AR, Figures 2-4. To create the project, the Unity 3D development environment with the Vuforia plugin can be used [11]. The former is responsible for creating the 3D environment, program functions and models, while the latter is responsible for placing Vuforia Online's target manager must be used to create the image target, which will be the maze image file itself. After the image import, these components can be placed in the space. It should be placed neither too close nor too far away. Then a Plane can be created, which can be placed in the plane of the image target, or a little below it. The black walls can be built in the space from cubes (first the horizontal ones, then the vertical ones), and then the ball can be placed also. An 'Ice' material can be added to everything to ensure sliding between the virtual surfaces and color the ball red and the walls black. A gravity parameter can be defined for the Ball. After that, an empty Cube can be placed, it will be the spawn point of the ball. A script can be also added to the ball so that if it falls below the plane of the Plane, it will automatically return to the spawn point.
The ready-made Plane can be assigned and the ball to the public variables of the Script. All the Android plug-ins can be easily downloaded and can already build the application in .apk form.   The Unity video game engine has won several awards since its release. In 2010, it won the Technology Innovation Award in the software category, which is presented every year by the American newspaper The Wall Street Journal. In 2006, at the Apple Design Awards, the game engine won second place in the category recognizing the best graphics software that can be run on Mac OS X systems [12]. One of its biggest advantages is that it is completely free for personal use, at the hobby level. As a result, there is a large user community and thus a lot of help and documentation can be found. Due to its multiplatform support, programs can be developed even for the following systems: Windows, iOS, Android, OSX, Linux, Nintendo switch, Xbox, Playstation and other supported platforms.

TEST SCENARIO 2 -AR ROBOT CONTROL
The UR5 robot arm by Universal Robots is a popular industrial robot that can be used in a variety of applications, including manufacturing, assembly, and material handling. One way to enhance the capabilities of the UR5 is by using it in conjunction with trajectory planning and simulation software, Figures 5-7. Trajectory planning software can be used to create optimal and safe paths for the UR5 to follow during its operation [10]. The software can take into account the robot's kinematic constraints, such as joint limits, and can also take into account the environment and other objects that the robot may come into contact with. Simulation software can be used to test and validate the trajectory plans created by the trajectory planning software. The simulation software can simulate the UR5's motion, and can also simulate the behavior of the environment and other objects that the robot may come into contact with. This allows users to test and validate the trajectory plans in a virtual environment before they are implemented on the real robot. Additionally, the simulation software can be used to test and optimize the performance of the UR5, such as cycle time, payload, and repeatability, under different scenarios and conditions. This can help to identify any issues or constraints that may arise during the operation of the robot and allows users to make adjustments before deployment [5]. The UR5 robot model and the corresponding script are taken from github. The downloaded file can be extracted and then imported into the newly created Unity project. It included the model of the robot with the individual segments, as well as their control using a slider. A Reset button can be added as well and a Scale option to the project. Scale can be used to set the size of the robot to a maximum of 4 times the initial value. The control UR5Controller.cs is written in C#.  It can be seen that the individual segments form a parent-child relationship on the Hierarchy tab. At the top level is the Controller. To this, a script can be assigned, in which the inputs are defined on the Inspector tab, these are the public variable game objects in the script. In addition, a Rotation empty object can be created, which contains only one coordinate system, so that the robot returns to its starting position at reset.
Universal Robots UR5 collaborative robotic arms can be used in virtually any industry and any process, and can be used by any worker. The goal of Universal Robots is to make collaborative robot technology accessible to all businesses, regardless of their size.
The difficulty of industrial robot trajectory planning is that in the case of a general six-jointed industrial robot, the state space is also six-dimensional. Randomized pathfinding algorithms are best able to cope with this challenge. Among these, the Rapidly-exploring Random Trees (RRT) and Probabilistic Roadmaps (PRM) procedures are the most significant. The former executes all trajectory planning requests without preprocessing according to the current cell state (so-called single-query procedure), while the latter does this with the help of a previously prepared collision-free map (multi-query approach). To implement the algorithms, a selfmaintained environment is created from the ground up in the form of AR application, which contains the necessary geometric and kinematic models, collision detection and trajectory planning algorithms, as well as the visualization of the UR5 used for planning and the calculated robot movements. Tracks designed using the completed program library are tested in a simulated environment and in real life with the help of UR5 collaborative industrial robots, and the experiences gathered in this way are also displayed. As a case study, the automation of two real industrial applications, a pick-and-place and an assembly operation, which fully utilize the implemented feature, was tested. In summary, trajectory planning and simulation software can be used to enhance the capabilities of the UR5 robot arm by Universal Robots. They can be used to create optimal and safe paths for the robot to follow, test and validate these paths in a virtual environment, and optimize the robot's performance before deployment, which can improve the safety, efficiency, and overall performance of the robot.

CONCLUSIONS
Virtualization systems allow manufacturers to study the virtual elements of the system, enabling them to analyse and design where real-world changes are needed. Virtual reality reduces unnecessary design by giving the engineer the opportunity to test changes before the final solution is created. Virtual reality training programs can simulate realistic and risky scenarios in a manufacturing environment (such as chemical spills, dangerous machinery, and noisy environments) without putting workers in actual danger. If the inevitable does happen, employees will have usable experience and are more likely to respond appropriately to the situation. Perhaps one of the most significant indicators of the industrial potential of augmented and virtual reality can be seen in the change in recruitment by major engineering companies. Lately, companies have been extremely open and actively recruiting people with degrees in game design. These young engineers are adept at virtual reality and Android and mobile devices, helping to make Industry 4.0 and Internet of Things solutions tangible. As a future work development of a virtual manufacturing assembly simulation system for industry 4.0 platform is considered.