Augmented Reality Uses in Interior Design

Increasingly, interior design retailers rely on online stores to reach their customers as more and more people are connected to the internet. One of the issue in online shopping is that customers cannot visualize how new furniture pieces would blend in their home, but that is not the case anymore as Augmented Reality (AR) becomes more mainstream. This paper analyses several AR applications used in the interior design and highlights their pros and cons. Based on this analysis, we propose an AR application that solves most of the current issues of the interior spaces design.


Introduction
Augmented reality (AR) represents a live modified view of the real world by adding some computer-generated objects in the environment rendered by the computer. These changes can be visual, auditory or olfactory [1]. The first ever-functional AR system has been developed in the beginning of 1990 by Armstrong laboratories inside the U.S. Air Force. This system was capable of replacing human arms by robotic ones. For a long time after this first breakthrough, AR systems were only implemented with help from some specials instruments, but as computers became more and more powerful and with a smaller footprint, these systems were starting to be used in everyday life. Today we live in a period in which the interest for this technology and its uses is on the rise because of the large adoption of smartphones.  [4] The first product that made AR widely popular was the game Pokemon Go, which used the phone camera during the game to add different characters in the surrounding environment (see Figure 1). The interesting part is that the game used both GPS and camera, so the players had to move in the real world to reach some meeting points were they could find new characters. Until recently the problem with this kind of applications was 1 that the placement of objects in the virtual world wasn't too good, but as the interest in AR rose, big tech companies started to invest more and more in this technologies and create libraries with which developers are able to create great applications, not just simple, fun, games. One area in which AR could have a big impact is interior design. Everyday life is full of choices that we have to make regarding our living space; from the color of the walls, to the couch we choose for our living room. This makes room for many possible mistakes, from incorrectly measuring the space in the corner for that dream desk to not correctly assessing how two new pieces of furniture would blend. We could measure the new furniture, but we can never see how it looks in a place until we actually place it there, at least that was the case until recently. AR applications have a lot a potential and we are going to expand this subject in section 3. The paper analyses several applications based on AR technology, to see their options for the users who use them for interior design, and come with an idea to improve their functionality by using the latest smartphone technological improvements in a way that makes possible for anyone to design a room from ground up.

Literature review
Augmented Reality helps the user to perceive the environment and the products. It can help in developing and having test in any phase of the building up of the product to be tested, or to make the user participate to some phases of the product life, like helping in the decision process during the buying step. In other words, augmented reality is considered an application that is designed inside your tablet or phone that is similar to the live view of physical world.

Augmented reality evolution
The research methodology for this concept was based on achieving five steps: selecting journals and articles, reviewing selected papers, defining the relevant categories in order to classify the articles and classifying the defined categories. Review the articles and define categories were iterated until the results were achieved [8]. AR has an important role in improving relation or interaction between the people involved in the different phase of the product life cycle and the artefact itself. Although this concept sounds futuristic, a similar form of AR occurred in the 1960s. The concept of computer-generated information was turning into augmented and virtual reality concept and the one who started all that is considered Ivan Sutherland [7]. In 1965, he left a famous quote on his essay about immersive displays and anticipation of AR: The user of one of today's visual displays can easily make solid object transparent -he can "see through matter!" A few years later after Sutherland studied the possibilities of what this technology can offer, he invented with the help of his assistants the first VR system [7]. In 1968 he finished the first head-mounter display capable of generating graphics to show users simple wireframe drawings. The display only showed the users crude outlines of a virtual environment. The invention was nicknamed "Sword of Damocles" and had the capacity to see through optics and head tracking. Between 1980s and 1990s AR concept had become an independent field of research and had fast evolution. Myron Krueger, Dan Sandin, Scott Fisher and others used concepts of mixing human interaction with computergenerated overlays [7] on video for interactive art experiences. Around 1974 Krueger created the video-place, an artificial reality's laboratory who combined projectors with video cameras capable of emitting onscreen silhouettes, creating an interactive environment for users. Soon scientists were trying to combine video-place systems with data gloves and tactile interfaces. Augmented reality had a fast evolution and a lot of scientists developed new toolkits which resolved the people different needs. Although this technology increased in recent years they are still in the research stage, they still have limitations. Moreover, trend analysis shows that web-based and wireless network technologies became so popular in recent years and has many applications in people's daily lives. It has a large impact on society by giving free access for the public, making them interesting in the technology. More and more people are talking about it and downloading applications on their smartphones that makes their lives easier. Many manufacturing companies and computer scientists have been studying this technology as a new human machine interface with the development of VR technologies. Virtual reality was used to analyze dynamic and static behavior of system or product. Most of software require the perfect 3D models of whole product or the whole system in order to represent the target system. Many manufacturing companies studied AR technology as an interface between man and machine. Once the AR system was accessible from smartphones, many applications for interior design appeared. This system was able to overlays the virtual furniture in a physical environment. Markers were used for tracking purpose to define the scale and coordinate system of the room. The principle is simple, the user can select virtual furniture on the application screen and place it into the space he wants. In this case, AR scope is to integrate the virtual furniture selected by user in the physical space and to create the illusion that the objects placed are real there. The principal component requested by AR applications is the smartphone's camera. More that placing furniture and changing layout of the room, some applications are so performant that offer the possibility to interact with the virtual objects using tangible augmented reality in real time.

Software design for augmented reality
The basic properties of AR are represented by combining real and virtual world, providing interaction in real time and registration objects in 3D. The first property influences the model of the real world by using sensors to measure individual properties of that. Applications require the synthesis of high-level commands and interactions from the raw tracking data by a series of processing steps [9].
An integrated software design is based on following components: pipes and filters architecture in order to implement a data flow approach to manipulate tracking data, high level multi-dispatch design for scene graph in order to combine semantic and dependent implementation structures in the scene graph, a data driven three-tier architecture for providing a scalable data management approach, a generic session and space management system for distributed AR applications.

Software frameworks for augmented reality
Coterie system is based on Modula3 and can support scene graph, graphics programming and abstractions for tracking devices. Later extension simplified the implementation of distributed AR applications [9]. ARToolkit is a library considered a minimal AR framework. It provides a tracking solution based on fiducial markers and rendering simple graphics using OpenGL on top of the video stream used for tracking [9]. The template of applications is based on a main loop iterating over the tracking and rendering functions. Although the library is considered simple, it was adopted by a large number of people who developed many AR applications. The ARToolKit tracking works as follows [10]: 1. First, the camera captures video of the real world and after that send it to the computer 2. Software on the computer searches any square shapes through each video 3. If a square is found then the software uses some mathematics to calculate the position of the camera relative to the black square. 4. Once the position of the camera was known, computer graphics model is drawn from the same position 5. This model is drawn on the top of the video of the real world and it appears like it's stuck on the square marker 6. The final output is shown back in the handheld display, so the user can see graphics overlaid on the real world. This domain will continue to growth in the use of internet and web-based devices and also in the use of mobile and portable multi-user AR devices. Collaborative, ubiquitous and interned-based AR systems enable users to update and synchronize the information from a remote location.  IKEA had introduced the "smart store" concept for its customers utilizing the augmented reality technology. They launched IKEA place application that represents the latest effort towards this ambition to perhaps change the way people buy furniture forever. IKEA application is a great example of an application that helps with furniture placement. If the room scan process is done correctly, the results have a margin of error of a few centimeters, as seen in Figure 2. where an AR generated plant support is placed next to a real one. The application can also scan an object and show you something similar from IKEA catalog, making the searching for certain objects incredible easy. But the application isn't without it's problems, placing sometimes objects in an incorrect scale or on top of other objects. Houzz implementation of the same concept is a bit more refined, because it correctly detects flat surfaces on which furniture pieces can be placed and it correctly puts objects behind others when this is the case (see chair leg in Figure 3.). This implementation isn't without it's faults, especially the fact that not all furniture available can be placed with AR and it's more like a picture on picture setup. Also, the application lacks a search function which is incredibly useful.

Interior design application models
One of the best implementations is done by Homestyler Interior Design (see Figure 4.) which for the best possible up to scale furniture double scans the floor. Also, if the user want, it permits object resizing and it displays in real time the new furniture piece dimensions which is great when we take into account customizability. Besides that, the app has the option to choose from a selection of empty rooms and decorate it as you please; the only drawback that we encountered is the fact that it is missing a complete room scan and redesign. All of the presented applications are useful and help you orient on how to place new furniture, but they cannot completely redesign a room. For example, there are many cases when people buy an old apartment or house and then they want to make everything suit their style. Normally, this requires a person trained in this area, which has to come to see the location, talk with the owner, understand what he likes and dislikes and afterwards to draw a sketch. Then the designer presents the sketch to the client, asks for feedback and changes the plan accordingly. All of this back and forth (especially the drive to the location) translates in time loss and money spent. From an objective perspective, we searched people's opinions about Ikea place and Houzz applications in the last month with IBM Watson Social Media tool and discover this way that Houzz application is more popular than Ikea place application (see Figure 5). Also, we made an interest comparison between the three applications presented in this paper over the last 12 months (see Figure  6) and discovered that the interest for Houzz app is maintained over time comparing with Homestyler which is almost constant and IKEA place which had a strong growth in September 2017 and after that the interest for that application suddenly decreased. Fig. 6. Interest comparison about IKEA place, Houzz and Homestyler applications in the last 12 months by web search criteria, retrieved from Google Trends Analytics [11] We chose to pay attention for comparison more on IKEA and Houzz app because the popularity of Homestyler is very low. Those few users are satisfied about 3D models offered by this application but they've complained about user experience and like IKEA app, the limited product selection. According to IBM Analytics tool, consumers using Houzz application are 11 times more likely to purchase when engaged with its augmented reality features. More than that, people considered that Houzz app is in top 8 best AR applications and in the last 12 months, by comparing with the other two, this application had the biggest level of interest in Canada, United States, Brasilia and Australia according to Google Trends tool.
Although the usability of AR app is increased and a lot of people are satisfied about their features, almost 10% had complained about Houzz app that it's frequently keep freezing, glitches across devices and not so user experience. The percentage of negativity is greater at Houzz than at IKEA app, but the positivity of Houzz app is greater than IKEA app for its design inspiration and over 5 million high-res home images (see Figure 7). Fig. 7. Sentiment opinion about IKEA and Houzz app retrieved from IBM Watson Analytics [12] People are satisfied about 3D models offered by IKEA app and for the great user experience, but according with IBM Watson tool 5% of them are complaining about the limited product selection and only for their store. Now that we have a perspective from our point of view and an objective perspective about those three AR applications, we will present our idea of improvement the AR products. Our solution is represented by an application that scans the surrounding environment and has the ability to remove certain objects, or everything, in case we want to design the room from scratch. The ability to remove all objects from a room represents the novelty, as this feature, currently cannot be found on any smartphone application. We consider that this function would considerably reduce the hassle of choosing the room design as you do not need to physically remove objects in order to see how an AR projected one would look in that spot.
To make this possible using the actual technology, we propose an app based on the following algorithm: 1. Scan the room from corner to corner (we will have to point the smartphone camera in each corner) so we can determine the exact room shape and area.

2.
After the room is scanned, the application will generate multiple points in the room. For each point do: I. Walk to the point that is indicated on the smartphone II.
Scan the room with phone camera 360O degree (this will enable the application to accurately determine the area and shape of the room) 3. After the room is scanned and the furniture identified, if some of the objects shape is not correctly identified, the application will ask for a point-to-point scan of that object.
4. Now that everything is scanned, give the user the possibility to remove certain objects and place AR generated one in their place.
For a better understanding please take a look at Figure 8. As seen there, for a complete scan, the user would have to physically move to every corner of the room and afterwards, if the application cannot determine the area exactly it will require some additional scans. The algorithm described above will make possible for the application to determine where the room walls and furniture exactly located are. Based on this, the user will be able to remove certain objects, or everything, if he wants. Based on the room space, we can apply certain room templates or themes (for example show how an "urban" design would fit). Also, the user can send the room plan (the scan made with the phone) to a specialist which can design everything from the comfort of his home. In this way, if for example we want a Japanese inspired room we can get in touch with a real Japanese. Using the application, the specialist can design the room and send the sketch to the client who will have the ability to see the room design proposal using AR directly on his phone. We consider that current AR interior design applications are headed in the right direction with more and more features added each year, but still lacking a common request of completely redesign. If the algorithm described above would be included in any of the presented applications, the benefit would grow exponentially, especially if the company has the possibility to provide customer support for customized design.

Conclusions
The smartphone has changed a lot how we see and interact with the world, and Augmented Reality makes no exception from this. The market for AR applications has evolved a lot in the last 2 years, and many companies started offering services that we could not imagine 6 years ago. Interior design is another area of our life that is going to be impacted in a good way by all of those changes, helping us in taking better, more informed decisions. As described, AR could cut prices on interior design by reducing the time and money put in every project, eventually making good design available to a larger group of people. The next step in this direction would be a combination of AR and VR (Virtual Reality) so we would have the ability to walk around the dream house unconstrained by the phone screen.