Augmented Reality Indoor Navigation Based on Wi-Fi Trilateration

— Location based systems have become a vital part of our daily life. While outdoor localization has been around for years, indoor localization is still under heavy research due to the limitations of GPS satellite. In this paper, we present an indoor localization method based on existing Wi-Fi access points and the trilateration approach. In addition, we present an augmented reality navigation solution that enables the users to augment their real word with virtual characters to guide them to the desired destinations instead of traditional routes and arrows. The system has been implemented and tested and showed promising results.


INTRODUCTION
Throughout life, men have been in continuous need of localization for many daily aspects.Localization in its basic approach was used even without exceptional instrumentations in order to recognize some location in space.Sailors, for example, had means for sea locations thousands of years before, besides other tools like the chronometer, astrolabe, compass, detailed maritime charts and maps, etc. that were primitives in the localization field.The 1920s witnessed the beginning of the first radio signal localization where it was applied for sea-based navigation, then in 1960s, the military applications raised by the U.S. Department of Defense which lead them to develop a satellite-based localization system known as Global Positioning System (GPS).Since then and starting 1983, GPS has moved to be more publicly used serving an accuracy of 100 meters in the 1990s.In 2000, GPS started to be of 10-meter precision after the deactivation of the so-called selective availability (SA) noticing its useful commercial and civil applications.Outdoor localization using GPS is typically feasible, however indoor cannot be achieved due to the difficulty of satellite signal to go through buildings and ceilings.Other methods are used to solve this, such as indoor Wi-Fi signals for positioning.Holding the fact that Wi-Fi works anywhere an access point is available, many applications depend on it in localization like mobile robotics that assign its own location in an indoor environment, using no extra hardware.Also, a person's location indoors can be found using a device connected to Wi-Fi access point which furthermore can be used in various applications for touristic aims such as museums, university campuses, restaurants, malls, shops, transportations, and nonetheless tracking malicious users of some Wi-Fi network [1] With the growth of location services, indoor positioning systems is in increased global evolution where companies and organizations are going toward developing new user experiences, profits, and advantageous brands of mobile applications applying location-based services, and those who haven't yet improved these services are on their way to in the coming two years in purpose of attracting more customers by enhancing their experience and utility of their services beside compensating sales via proximity marketing, making use of locations, and taking much care of their brands served to users.Nowadays 38% of organizations and companies surveyed have already applied indoor positioning systems with the lead for US with 43%, UK (33%), and Asia (33%).With the evolution of mobiles as a necessity in everyone's life, and the truth that most of the peoples' time is spent indoors, internal localization becomes more dependent on the use of smartphones to engage clients indoors and is branched into many types: Wi-Fi, Bluetooth Low Energy beacons(BLE), and geomagnetic indoor positioning systems [2].
AR (Augmented Reality) is the addition of computergenerated objects to the realistic world that the user is visualizing in real time and in an interactive manner [3].Nearly in 2005, AR was the predicted technology to arise significantly in the future of four to five years [4].According to a review research done in the conference of AR in 2008, this technology was expected to be trending in the period of 2008 to 2012, also a report of 2011 mentioned that apps implementing AR will show a 1.4 billion downloads all over the world near 2015.Additionally, AR is growing more specifically in marketing and entertainment, more beyond location-based apps into games, education, every day's life, and medication [5].
As mentioned before, GPS is weak for indoor positioning, it is inapplicable in the domain of indoor navigation that needs penetration of signals inside buildings, so many problems can be recorded that will lead to adoption of new method for indoor localization: • Need of high accuracy and scalability of positioning indoor which is hard to be applied using the available apps of outdoor positioning or methods of indoor that depend on Bluetooth beacons where developers have concerns in implementing it concerning scalability and expense, as well as the geomagnetic that is still not well known technology and can't be risked to be used in applications that need high precision.
• Holding the truth that technologies other than Wi-Fi used in indoor positioning are of expensive and more complicated manners like the ultrasound or Laser, and need extra hardware to be tested and implemented [6] • Obstructed human experience while being inside touristic places, where localization and navigation has turned into a necessity compelled by the penetration of smartphones in our life as obligation in order to ease his discovery of nearby places as shops or transportation stations faster and more feasibly.
• Indoor positioning systems are still in relative infancy, however our research of this domain has revealed a huge demand, utilization, and growth of indoor positioning system services [1].• Significant demand of Wi-Fi localization in many fields of healthcare, proximity marketing, retail, travel and big estate offices that are difficult to navigate where people get lost when heading to a meeting which spends a lot of their work time, so deployments of indoor positioning services is increasing accordingly in the coming few years.
• Human senses don't have the ability to visualize information directly in most of life applications like medical visualization aid for surgeries, manufacturing and repair, annotation and visualization; consequently performing real-world tasks holds some complexity.Augmented reality view in applications enhances this ability of visualizations for users [5].
• Deaf and blind users are in need for visual and audio services that aid them in their daily life; augmented reality features sensory cues that substitute their weak senses [6].
Applications of tracking and localization combined with techniques like augmented reality give the user advantageous performance that provide simpler visual navigation especially in indoors where the effects of outside nature as lightning are absent [4].
In this work, we intend to develop a real-time Android based mobile application for indoor navigation using Wi-Fi trilateration as an adopted tool for localization where it is more dominant in most buildings and removes the requirement for additional infrastructure.Wi-Fi localization involves many methods that depend on wireless access points to locate signals from a device like smartphone using the strength of the signals received.One of these methods is Wi-Fi trilateration which is as the name mentions a "tri" based technique that uses three access points to locate, it estimates the distance between the receiver and three transmitters by generating circles of radius around each access point as the distance from it, and their intersection points will define the area where the receiver is [7].Another method for localization is fingerprinting, in this technique there is matching between previously calculated RSSI values with the real-time measured data [8].Added to this, the augmented reality feature that enhances the user's insight and interaction with the real world, when AR was used in approximate of six domains like medical, maintenance and repair, annotation, robot path planning, and surely entertainment, it was highly recommended to be a part of this location based navigation service [9].The mobile application that the user downloads will be able to spot his location in some building, use it to accurately guide him in locations found inside buildings by projecting a virtual character on top of the camera such that the user can visualize both the real word and the character guiding him/her to the desired location.As a result, the problem of indoor penetration of satellite signals is answered and the complication in using available tools and applications to compensate human's need in various daily life aspects for localization is satisfied.
The rest of the paper is organized as follows: section 2 discusses the related work and the background of this project, while section 3 presents the system design and requirements.In section 4, the implementation details of this work are revealed and in section 5, the simulation environment and testing results are presented.Section 6 concludes the paper.

II. RELATED WORK
As mentioned before, the creation of systems and solution for indoor positioning and navigation topic has been widely studied lately due to its significance in future industry among various aspects.During the research of this high momentous topic, it was found that large companies are employing systems of indoor localization in most of its planned technologies, architectural designs, and trends, beside those that have already updated their platforms, mobile applications and APIs to engage this approach as part of their services.These companies include also large and important ones such as Apple, Google, Cisco, and Alibaba.Wi-Fi triangulation is the most popular method used among most of the systems of indoor positioning due to its high accuracy compared with other techniques, this precision reached as a maximum of 1 meter.This section introduces some of the most popular applications of indoor navigation.
Apple's project cost 20$ million to be done and used Wi-Fi and sensors as micro-electromechanical systems (MEMS) for indoor localization [10].The system uses Wi-Fi signals, GPS, cell tower trilateration, and fingerprinting to track accurately positions mainly when applied indoors.Besides, the sensors were used to acquire locations and data without the need to save them as old methods used to do.
WIFISLAM [11] makes use of pattern recognition and machine learning and correlate them with the sensors' gathered information with the intervention of trilateration at the end to create an adequate location.Its main aim was to provide indoor positioning to everyone that is in need to it.
Google Maps address outdoor and indoor positioning too.For open areas GPS and cellular networks is used as the tool for localization and navigation, but concerning indoor locations, google is only available at select locations providing the user with the indoor map without any navigation or indoor positioning information.
In 2015, "Eddystone" beacons were released by Google as a private beacon standard that will provide high accuracy, usability and scalability for indoor maps in the coming future [12].Also, Google developed a new project "Project Tango" that should be a capable tool of evaluating distances without the need of GPS for indoor navigation [13].
IndoorAtlas is an SDK for iOS and android that makes use of magnetometer sensors found in the phones and mapping that is cloud-based in order to locate indoors within 2 meters of accuracy.It depends on magnetic fields emitted by movements of bodies which allow locating them within an environment; it also uses Wi-Fi and Bluetooth to enhance accuracy [14].
QPS is a platform created by Qubulus.It depends on fingerprinting algorithm for indoor and outdoor positioning.QPS is considered as a recording tool for data of WiFi, 3G, 2G of a user's location and saves it as compressed version in FPD file for later use as Geo-coordinates displayed on Google Maps or Bing Maps.This company provides customers with APIs that they can as developers use in terms of indoor positioning features in their applications with shelf level of precision of 3 and 5 meters.
Nokia has developed an indoor navigator that precisely localizes users with no need of GPS and with a high accuracy indoor positioning (HAIP).This method uses Bluetooth 4.0 and Wi-Fi trilateration for localization and recommends strengthening of Bluetooth beacons in each location, since its architecture includes positioning server in the top, then beacons in numerous locations, and the global positioning database.Knowing that there is 0.3 meters of accuracy in this system but it is still not released yet and has no newly published reports about it [15].
Place Lab uses radio beacons which physically send radio signals in a periodic manner by wireless LAN access points, Bluetooth beacons, and GSM towers.It reaches an accuracy of 10-13 meters [16].The work in [17] is a prototype of tracking a physical direction and location.It is a projection of virtual world from a projector to an indoor real environment, and it allows reaction with an augmented reality added to real world.It uses Raspberry Pi computer, and Pico handheld battery powered projector beside a smartphone, they all should be handheld simply by the users.It was an attempt to develop the technology that can be used in AR applications later and to test the efficiency and usability as well as functionality of such technology.
Several systems have been implemented in the field of indoor navigation, some of them provide APIs for developers in this domain, and others are by themselves applications for indoor positioning where they may include navigation features or only providing on map locations, they could provide information of places and statistical properties of studied areas, and some are still in the infancy stage to be applied in the near future.Most of the surveyed applications are considering Wi-Fi trilateration as their main method to achieve indoor positioning and navigation consequently, but they were always lacking some feature in solving this significant problem of localization inside which lead them to be of low efficiency and made this field still in the stage of experimentations.Appending indoor localization real environment with virtual objects gives users an opportunity to experience augmented reality feature in their navigation where they can react and visualize real world overlapped or amplified with virtual characters , so bearing all of these truths in mind we are going in this project to address indoor positioning system using Wi-Fi as the best effectual tool, in purpose of fulfilling a complete system of navigation inside buildings with statistical and context information of users' movements, density and personal information as well as geometry of the place and its fingerprinting in addition to real time reaction with the user achieved by augmented reality implementation.

III. SYSTEM DESIGN
In this section, we will discuss two approaches: trilateration and fingerprinting.Both approaches depend on RSSI measurements which consist of received signal strength indication that is the power level sent from an RF device (Wi-Fi Access point).In order to calculate the location of a user, the distance from three access points computed using RSSI is evaluated.

A. Wi-Fi Trilateration
The distance is calculated from each access point to the user using the following equation defined as free space path loss in decibels: (1) where d is the distance in meters and f is the frequency in MHz.
After the distance from the three access points is calculated, the location of a user can be estimated using Wi-Fi trilateration method that generates a circle around each transmitter (access point).Solving for the intersection of the three circles using 3 equations result in the centroid of the triangle.The localization (x, y) can be computed as follows, given the coordinates (xi, yi) for each access point are already known.The approach is depicted in Figure 1.

B. Fingerprinting
Fingerprinting is a two-stage approach.The process starts by an offline acquisition followed by a localization approach based on interpolation.

1) Offline acquisition:
In this stage, the indoor map is divided into sub grids and fixed points are places all over the map as the "fingerprints" separated by x meters (2 meters were used in this work).The RSSI values received on each point from three APs along with the MAC address and SSID are recorded, where RSSI is measured by dB.These values are then stored in a database for later use at the next stage.

2) Localization:
This is the phase of determining the position and is done in real-time simulation.The current RSSI is measured from the three access points.Only two values are chosen which have the closest values to those pre-measured in in phase 1, and consequently are the nearest to the AP.The next step consists of measuring the midpoint between these two points, which will result in three points that form a triangle including the area of the location.Calculating the centroid of this triangle gives the required location

C. System Architecture
The system is designed in a client-server form consisting of three tiers: the client that is supposed to be a Wi-Fi enabled device (mobile android device), network of three Wi-Fi access points, and a server.The system architecture is depicted in figure 2.

1) The client:
The client consists of a wifi-powered mobile handheld device.The client's task is to scan available Wi-fi signals in the range and search for specific SSIDs.The SSIDs can be either coded into the application or can be chosen by the user.They can be also pushed from the server upon request.Once the SSIDs are chosen, the application can start calculating the RSSI from each of the three access points and either the fingerprinting or the trilateration computations can take place to calculate the coordinates of the mobile device.
2) The Access points: The access points that are used in the system use the IEEE 802.11n standard that establish wireless connection in an easy way with high speed and coverage compared to those of 11g standards.Featuring transmission speed of 300Mbps allows less data loss in case of long distances and blocked paths with obstacles like in offices and apartments.Most importantly, the signal can be effortlessly detected over distances rather than difficulty of this in other standards and that is what we care for in our system that needs the signal's strength from these access points that helps in the calculation of the distances as well as the location of the user away from them.
3) The server: In this project, the server may serve two jobs.First, it can hold the list of SSIDs that are present in the vicinity.This list can be updated by an administrator based on the company's request.Second, the server holds the locations (coordinates) of the destinations.For example, it can hold the coordinates of the stores in a mall, the gates in an airport or the merchandise in a clothing or grocery store.These coordinates can be either fetches upon user request to navigate to a destination or downloaded and cached in the application each time the user opens it for later and faster use.The administrator can update and edit coordinates on the server and push them to the application.The above three sections are tied up in the sequence diagram depicted in figure 3. The above sequence diagram shows the flow of the system procedure, where as a start the user triggers the application in his mobile, thus requesting the server to get locations that are already predefined, the server extracts the updated locations from the database and forwards them to the mobile application.At the mobile application, the user has to choose which location he wishes to go to in order to be guided.

International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181 http://www.ijert.orgAfter that the mobile application starts scanning the area looking for signals that belong to access points that are also predefined.These access points shall supply the mobile device with the signal strength level and SSID.Using this information, the application will do the necessary computations to first determine the location of the user using an algorithm, then to prepare the phase for guiding the user.Finally, augmented objects will be displayed on the mobile app that shall help the user to reach his chosen destination step by step.The state diagram in figure 4 describes the states of the mobile application; first the user starts the mobile application, which in turn automatically requests predefined location data from the server.The server extracts the info from the database and returns it to the requesting side, the mobile app.At this case the user will choose the location he wants to go to, if he does, a camera activity will be initiated.At the background of the camera activity, the scanning process will start which scans the signal strength RSSI from the predefined access points, these measurements will be used in the process of localization, where the location of the mobile device will be computed using an algorithm.When done, on the camera activity augmented objects will appear that guide the user to the destination he wanted to reach.If he succeeds in reaching the desired destination the process would have achieved its task otherwise it will still renter the camera activity and continue the flow till task accomplishment.

A. Distance calculation
The implementation of this work is done in two-tiers: trilateration and fingerprinting.Note however, that distance calculation for both approaches, whether it's offline or realtime, follow the same algorithm presented below.The reason behind initiating several scans from the same SSID lies in the fact that RSSI measurements are noisy and hence one measurement might dictate a totally wrong distance.For this reason, we propose to scan 16, 20 or 25 times, depending on the situation.After obtaining all these values, noisy values are usually either much lower or much higher than the real values and that's why, sorting is employed.Once sorted, the lower M values and the higher N values can be discarded.M and N can be chosen to be the same amount.Next, the remaining values are averaged.In all cases, three access points are employed that can have the same or different SSIDs.For this purpose, we have given the user the ability to choose the SSIDs as well as control some of the settings of the algorithm such as the number of scans and the time between each scan.This is depicted in figure 5.The testing environment is presented in figure 6.This map was specifically chosen to test the performance of the algorithm in the presence of walls, hallways, and interference which will affect the signal strength and play a role in path loss and fading.The origin of the map (0,0) was chosen to be the lower left corner.For the offline acquisition phase in fingerprinting, the map is depicted in figure 7. The "x" marks are the predefined fingerprints while "AP" denotes Access Point.

B. Augmented Reality integration
After successfully determining the user (phone) indoor position, the coordinates are passed to the second module of the system consisting of projecting a 3D or 2D object into the screen in order to guide the user to the desired location.For that purpose, Google ARCore was employed.ARCore is Google's platform for building augmented reality experiences.Using different APIs, ARCore enables your phone to sense its environment, understand the world and interact with information.Some of the APIs are available across Android and iOS to enable shared AR experiences.ARCore uses three key capabilities to integrate virtual content with the real world as seen through your phone's camera: • Motion tracking allows the phone to understand and track its position relative to the world.• Environmental understanding allows the phone to detect the size and location of all type of surfaces: horizontal, vertical and angled surfaces like the ground, a coffee table or walls.• Light estimation allows the phone to estimate the environment's current lighting conditions.
ARCore's motion tracking technology uses the phone's camera to identify interesting points, called features, and tracks how those points move over time.With a combination of the movement of these points and readings from the phone's inertial sensors, ARCore determines both the position and orientation of the phone as it moves through space.
In addition to identifying key points, ARCore can detect flat surfaces, like a table or the floor, and can also estimate the average lighting in the area around it.These capabilities combine to enable ARCore to build its own understanding of the world around it.An example is depicted in figure 8.The simulation of this work is two-fold.First, the accuracy of the localization is tested for two scenarios: fingerprinting and trilateration.Second, the integration of the localization with the augmented reality module was tested to assess the reliability of the developed app in real-time scenarios.

A. Localication
To test the localization accuracy, the two approaches outlined above, namely the fingerprinting and the trilateration are compared.The simulation environment is outlined in figure XX.The origin (0,0) is set to the lower left corner.The

International Journal of Engineering Research & Technology (IJERT)
ISSN: 2278-0181 http://www.ijert.orglocations of the three access points are also known a priori.Table XX outlines the different values obtained given the two methods.The first two columns are the real coordinates (x,y) of the smart phone, while the third and fourth column are the coordinates (x',y') obtained using equation XX and fifth and sixth column are the coordinates (x'',y'') obtained using the fingerprinting interpolation method.The average error in distance between the real location of the device and the computed coordinates was found to be 3.8 meters for Wi-Fi trilateration method and 4.062 meters for the fingerprinting method.While both methods are considered comparable, the trilateration method was chosen due to the lower error rate and the fact that no offline acquisition is required.
On the other hand, the number of scans required per location computation is tested.As outlined in the algorithm above, x number of scans are initiated per access points.All RSSI values are gathered, ordered in ascending order and averaging y median values to overcome noisy and faulty measurements.Decreasing the value of x, and consequently y, is crucial for real-time performance.In addition, the amount of time between successive scans is also tested in order to find the

B. Augmented reality
After tuning the algorithm parameters as detailed in the previous section, it is then plugged within the augmented reality module.When launching the application, the user is presented with a number of destinations.In the current prototype, these destinations are stores in a mall as seen in figure 14.After clicking on the one of the options, the coordinates that are pre-saved in a database are fetched from the server into the main thread of the application.Note that, when a location is fetched from the server, it's saved into the local storage of the application so that when the user chooses that same destination, the coordinated are locally fetched to speed up the process and control the bandwidth consumption.Also note that the coordinates of the hallway beginnings and endings are also fetched and stored.This is important for navigation as they will be used to guide the user right or left.Once this is done, the current location of the user is computed as per the previous sections and a routing path is decided.The routing path consists of 4 possible values: forward, left, right, and wrong.Forward means the user has to walk forward.Left and right entitle the user to take a left or right turn respectively, while wrong means the user is walking in the wrong direction and must turn.For example, if the destination is at location (4,4) while the current location is (2,3) and the hallway ending is at (2,4), the routing path is formed such that the user must go forward until he/she reaches (2,4) then make a right turn and move forward until coordinates (4,4).A virtual character appears on the screen and laid on top of the front facing camera.The user can now follow the character to the destination.A snapshot of the application is shown in figure 15.

Algorithm 1 .
Distance calculation Input: RSSI measurements Output: (X,Y) coordinates 1: For each SSID i DO Scan 16, 20, or 25 times to get the RSSI value.Sort the measurements in ascending order.Choose the median 8, 10 or 15 value by removing the lower and higher values.Compute the average value of the remaining measurements.Get di by applying equation (1) END DO 2: Apply equations (2-5) to get (X,Y)

Fig 5 .
Fig 5. SSID input in the application

Fig
Fig 9. X coordinates minimum amount of time that can result in different RSSI values.Figures 11, 12 and 13 depict the different results when 25, 16 and 10 scans are chosen respectively.In each figure, different delays are employed.

Fig
Fig 11. 25 scans Fig 14.Destination choice in the app

Fig 15 .
Fig 15.Testing for augmented reality navigation VI.CONCLUSION This paper presents an indoor navigation solution based on Wi-Fi trilateration coupled with augmented reality for interactive user experiences.The algorithm tends to reject noisy values and the testing showed acceptable performance and real-time results.Future enhancements include but are not limited to improving accuracy, filtering noisy measurements and adding fingerprinting for more robustness.

TABLE I .
. Trilateration versus fingerprinting results Figures 9 and 10 depict the difference between the two approaches.