Using augmented reality to inform consumer choice and lower carbon footprints

Consumers who wish to consider product attributes like carbon footprints in their purchasing decisions are often blocked from meaningful action by a lack of information. We conducted a randomized controlled trial at a grocery store to evaluate the effects of providing such product attribute and carbon footprint information via augmented reality (AR) displays on bottled water and breakfast cereal, two frequently purchased goods. Using an AR smartphone app that combines comparative and detailed product information into personalized data and recommendations, a 23% reduction in carbon footprint was found for bottled water, and non-significant reductions for breakfast cereal. However, AR informed choice lead to healthier cereal purchases with an average of 32% less sugar, 15% less fat, and 9.8% less sodium. This research suggests that AR techniques can help facilitate complex decision-making and lead to better choices.


Introduction
In the United States, the indirect CO 2 emissions from consumer purchases are double the direct emissions from home energy use and personal travel (Bin and Dowlatabadi 2005). Consumers who wish to consider carbon footprints in their product purchases are often blocked from meaningful action by a lack of usable information about the emissions embodied in products they are considering. In this experiment, we explore the effects of providing such carbon footprint information via augmented reality displays on products at the point of purchase. We designed an augmented reality smartphone app that incorporates carbon footprint information into personalized data and recommendations. We then tested the app in a supermarket and found large carbon reductions for bottled water purchases but only small changes for cereal purchases. Participants expressed high levels of satisfaction with the augmented reality shopping experience and a willingness to use a similar device for more informed shopping.
Individuals often lack the information, cognitive ability or time necessary to evaluate every option and arrive at the best choice (Simon 1955). On the other hand, providing too much information and data can impede the ability to make decisions. Research has shown that decision-makers can suffer from 'information overload' wherein they are less satisfied with their choice as the number and complexity of options increases (Reutskaja andHogarth 2009, Lurie 2004) and that time pressure can hinder optimal decisionmaking (Haynes 2009, Lurie 2004). Adding carbon footprint information via eco-labels (Shewmake et al 2015, Vandenbergh et al 2011 can lead to more environmentally sustainable choices (Min et al 2014, Davis and Metcalf 2014, Ward et al 2011, Murray and Mills 2011, but necessarily increases choice complexity. While the heuristics people use to make decisions often lead to reasonably good outcomes, new technologies, particularly advances in mobile computing, hold promise for reducing cognitive, information, and time barriers, potentially improving decision-making and allowing new attributes like carbon footprints to be included. Augmented reality (AR) is one such technology and allows users to view the real world alongside super-imposed or composited virtual objects Original content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence.
Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. (Azuma 1997) so that they receive relevant and personalized information about their surroundings in real time and in a form that facilitates decision-making. We call this 'real-time informed choice' (RTIC) and it can be applied to a wide range of consumer-related activities.
RTIC techniques are particularly applicable to environmental product attributes, an important but often missing piece of information in many consumer purchasing decisions. Literature suggests that many individuals have strong environmental preferences that can be expressed in purchasing decisions (Roe et al 2001, Teisl et al 2002. To date, AR tools have been developed but not tested on actual shoppers (Zhu et al 2004, Espinosa Ceniceros et al 2014 and experiments have been conducted providing real-time information via non-AR technologies (Kourouthanassis et al 2007). ElSayed et al (2016) test AR's ability to help individuals filter, find, and rank alternatives. To our knowledge, this research is the first to show the effectiveness of AR in actual consumer choice situations.

Methods
To test this new technology and its effectiveness in aiding decision-making, we developed a personalized AR decision support tool and conducted a randomized controlled trial in a commercial grocery store examining consumer purchasing behavior of cereal and bottled water. The experiment incorporated carbon footprint information with other relevant product details to aid consumer choice. Figure 1 shows the app in operation and the use of personalized letterbased recommendations on distant products and more detailed information on nearby products.
We will first describe how the AR app was designed and then discuss the details of the experiment.

App design
When confronted with many options, individuals often employ a two-stage decision making process (Beach 1993, Payne et al 1993. In stage one, individuals identify a set of promising alternatives and in stage two they examine those candidates in more detail. To accommodate this, the AR app shows large letter grades from a distance and then detailed information on physically nearby items. The letter grades serve to help individuals identify a subset of preferred items for further inspection while the detailed information aids in the more thorough comparison. We used Edwards and Barron's (1994) SMARTER technique to personalize the AR display for each participant. The results were used to determine what letter grades to show on distant products, and what information to display on nearby products. SMARTER works by breaking a product up into attributes, asking participants questions to determine the shape of each attribute-level utility function, then combining the normalized attribute-level utilities using weights inferred from a user-supplied ranking of the attributes. Srivastava et al (1995) found that the SMARTER elicitation procedure resulted in recommendations that correlated reasonably well with actual choices by test subjects. Normally, the attributes are determined by the researcher and are constant across all participants. Given the number of attributes in our study, we added an initial step where participants selected what attributes were important to them. They could select a single attribute or all attributes if they desired.
SMARTER was implemented using a three stage survey illustrated in figure 2. The first screen showed the entire list of attributes available (for water or cereal, depending on a participant's randomization), with the order of attributes randomized for each participant. The middle screen asked for additional information on each of the selected attributes. Listed first were categorical attributes with more than two options (see 'Grain' in figure 2). Participants first had to rank the different options from most favorite to least favorite and then indicate via the sliders the importance of each option. For continuous attributes, individuals indicated whether they wanted a higher or lower value (see 'Sugar' in figure 2). For binary attributes, individuals simply indicated whether they preferred it or not (see 'Whole Grain' in figure 2). Table 1 lists all the attributes used in the experiment along with their default values.
Finally, individuals ranked their chosen attributes from most to least preferred, with the initial list randomized. Carbon was added to list of attributes for every participant, but individuals could rank it however they wanted.
Equation (1) is the final utility function where x i is the level for attribute i. The first term sums over all attributes with discrete levels (like whether an individual likes whole grain, or their preferences over different grain types) and f is a function that maps the attribute level to a utility. This mapping is obtained directly from the participant and ranges from zero (least preferred attribute level) to one (most preferred attribute level).
The second term sums over all continuous attributes where more is better, and the third where less is better. The 'w' weights are subscripted with user-supplied rank for attribute i, r i . The x i,min and x i,max terms refer to the maximum and minimum attribute levels for attribute i. See the online supplementary information for more details.
For the near view display (see the left side of figure 1), the product attributes selected in the preference elicitation survey were shown in the order they were ranked by the participant, with the price per serving displayed at the top for all rank order displays. The colors used to represent the remaining attributes were derived from their scores relative to all other products. For instance, if a participant stated a preferences for a high calorie cereal, the calorie label and value would be presented in green for products in the top third of calorie values, yellow for the middle third, and red for the lowest third. The color scheme was reversed for those who wanted a low calorie cereal. For the carbon footprint data, prior research has shown that people do not understand data shown in units of mass, such as grams (Waygood and Avineri 2011), so we converted grams to the equivalent number of 'miles of driving.' A screen in the app explained to participants that this was based on a standard US sedan.
Carbon footprints for cereal and bottled water were calculated using a process based lifecycle cost analysis (LCA) methodology (Finkbeiner et al 2006). For bottled water, the carbon footprint is driven largely by producing the bottle and transporting it (Gleick and Cooley 2009). All the plastic bottles in this study were made from polyethylene terephthalate (PET) which has a carbon intensity of 3.1 gCO 2 /gPET (Boustead 2005). For transportation, we assumed that bottles traveled by sea (if needed) and then by truck to Denver, with either Long Beach or Savannah as the port of entry. Carbon intensities per ton-mile were taken from the EPA's Emission Factors Hub (EPA 2014).
The cereal carbon footprint calculations were considerably more involved. Products had between 4 and 25 ingredients, each in theory requiring an extensive LCA analysis. To simplify the analysis, only the top five ingredients were considered. Due to lack of  A (À) indicates that 'less is better' was the default, a (þ) that 'more is better,' and other values in parentheses indicate options for categorical attributes, with the first option being the default in any dropdown menu.
Environ. Res. Lett. 12 (2017) 064002 data, all transportation related aspects were ignored. The ingredient nutrition data and the nutrition data for the entire product were used to solve for the ingredient ratios. This was complicated by the FDA's rounding rules for nutrition information (FDA 2013) so we used a Monte Carlo method to find a set of ingredient ratios that fell within the plausible final product nutrition bounds. The LCA values for raw ingredients were obtained primarily from SimaPro (Goedkoop et al 2016). Cereal footprints ranged from 22.5 to 73.2 grams per serving and water from 97.4 to 1160 grams per liter. The AR app was created using the Vuforia 4.0 library for image recognition and position estimation and OpenGL ES2.0 for rendering information on screen. Two different methods of enabling AR were used in this research. For cereal, the actual box images were recognized by the app. Images and text were overlaid on the product and rendered to appear as if actually written on the box, even if viewed from an angle or upside down. For bottled water, the curved surfaces and relatively small images made this impossible given our tools. Instead, black markers with a unique series of white dots were used as reference points. The AR app would recognize the markers and draw the appropriate information. This information was rendered to appear over the bottles and in the same vertical plane as the product shelves. Images of the app in-use for bottled water can be found in the online supplemental. The app generally performed well in displaying the correct information. However, it would sometimes display incorrect information for visually similar boxes. Individuals could compare the overlaid name to the actual product to see if the app was working correctly. Even with these limitations, participants still reported very positive experiences (see the summary of the post-experiment survey in the results section).

Experimental design
The experimental design consisted of two parallel randomized controlled trials and was conducted at a major grocery store in the greater Denver metropolitan area for two weeks in July of 2015. The grocery store was a major retail chain, not a specialty goods store. Two weeks allowed for 126 people to successfully complete the experiment, with 63 using the AR app for bottled water and 62 for cereal. Individuals were approached at a booth setup at the store's entrance and no advertising was done to recruit particular demographics. The only requirement to take part in the research was that individuals already own and know how to use a smartphone. The average age was 40 and the oldest individual was 75. Staff members were always available to answer any questions or trouble shoot equipment but were trained to instruct participants that they could take as long as they wanted to make choices and to keep a distance during the choice phases of the experiment.
Once individuals agreed to participate and informed consent was obtained, they were randomized by alternating assignment to using the AR app to choose a box of breakfast cereal or a bottle of water. However, all participants bought both products. Participants were provided with a smartphone for the research. The group that received the app for buying cereal bought water without the app and served as a control group for those using the app to purchase water and vice versa. Each group made the non-AR product choice first before moving on to the next section of the grocery store and using the app to make a selection. Table 2 lists the demographics by treatment status for the experiment.
We chose to experiment on bottled water and breakfast cereal because they represent relatively simple vs. complex products respectively. Bottled water has few distinguishing attributes compared to breakfast cereal. For cereal, participants could choose from 41 different brands while for bottled water they could choose from 18. The grocery store actually stocked over 150 unique brands of cereal, and so participants were shown the section that contained the cereals available to them. For bottled water, individuals could choose any item from among the bottles available for individual sale, excluding flavored beverages other than a single lemon lime sparkling mineral water.
After using the app to make a choice, participants were asked to complete a 25 question survey after which they were given a $20 gift card to the grocery store and instructed to purchase their items. Participants were informed about the $20 payment and requirement to purchase their chosen items before enrolling in the experiment. We did not enforce this requirement but random checks on exit did not find anybody that didn't purchase the items they selected. The study was approved by MRIGlobal Institutional Review Board for Human Studies.

Results
Post-experiment survey responses indicate that individuals enjoyed using the app and found it useful. In addition, 88% of individuals reported that they would use the app if it were available today and 83% thought they made a more informed choice because of the app. Further survey details can be found in the supplemental material. While the AR app was well received by participants, our primary research question was whether the real-time provision of personalized information via AR led to changes in purchasing behavior. An analysis of the purchased items between treatment and control groups shows that indeed this was the case. Carbon footprint was a significant differentiator in purchase choice of bottled water, showing a decrease of 23% (95% CI 5.4% to 39%) relative to the control group, with RTIC information, whereas cereal purchases were not sensitive to carbon footprint information (though see the online supplemental for an important discussion of outliers). Confidence intervals for percent changes are estimated using the Fieller method as implemented in the R mratio package (Djira et al 2012). Tables 3 and 4 summarize the experimental results for bottled water and cereal purchases, respectively.
While the water purchase results show a statistically significant effect of the AR intervention on carbon footprints and no other attributes, the cereal purchase results are quite different. Carbon footprint had a non-significant decrease of 3.2%, but numerous health related attributes experienced significant changes. The results are summarized in table 4.
Sugar decreased by 32% (95% CI 16% to 46%), fat by 15% (95% CI 2.4% to 26%), sodium by 9.8% (95% CI 0.7% to 18%), and the number of ingredients listed on the box by 15% (95% CI 3.5% to 26%) while fiber increased by 47% (95% CI 9.3% to 89%). The calories per serving also decreased, but at a marginal significance level. It is noted that there are many outcomes being analyzed here, and so the chance of a false positive is higher than if only a single outcome variable were studied. The supplemental information contains results that adjust the estimates for the covariates listed in table 2, and complements the t-tests with non-parametric permutation tests. With covariate adjustment, the change in sodium becomes marginally significant. No other significance tests are altered.

Discussion
In this study, we attempted to determine if the realtime provision of personalized information via AR would lead to changes in purchasing behavior. In a field test conducted in a grocery store, individuals were provided with a personalized AR decision support tool to help them choose either bottled water or breakfast cereal. Their choices were compared to an equivalent group presented with the same choice situation but without the aid of the AR tool. The results indicate that the AR tool caused individuals to select, on average, lower carbon footprint bottles of water and healthier cereals.
These nutrition results are uncommon in the literature on product information programs, which is replete with findings of no effect (Mayer et al 1989, Russo and Leclerc 1991, Escaron et al 2013, Russo et al 1986. The 23% reduction in the carbon content of bottled water would likely be difficult to achieve through technical means. The production costs depend largely on transportation and bottle manufacturing issues, both of which are already under intense economic pressure with respect to minimizing fuel and raw bottle material usage respectively. However, by better informing the consumer-potentially a low cost intervention-large decreases in a market's carbon footprint may be possible. In 2007, U.S. consumption of bottled water accounted for between 32 and 54 million barrels of oil (Gleick and Cooley 2009). Reducing this by 20% would save roughly 8.6 million barrels. This is a small fraction of total U.S. energy consumption, but bottled water is only one of thousands of consumer goods to which RTIC techniques could be applied.
However, our results come with several limitations. First, the experiment was conducted at a single grocery store in the greater Denver area. While a major chain and not a specialty goods store, the sampled population is not representative of the U.S. in terms of Table 3. Bottled water results. Selected refers to the percent of individuals choosing that attribute in the preference elicitation step, AR and Control are the average attribute values of the chosen water for those with and without the AR app.

Attribute
Selected ( The effect size is the difference in average values between AR and control and the P value is from an unpaired t-test with unequal variances. The categorical outcome variables were analyzed using a x 2 approximation to test for equality of proportions, as implemented in the R functions prop.test and chisq.test (R Core Team 2014) Environ. Res. Lett. 12 (2017) 064002 age, income, political orientation, openness to new technologies, and other variables that may affect the outcome. Further, this research involved just two products, bottled water and cereal, and given the disparity in outcomes between the two it is likely that the effect will be heterogeneous across product types. We also paid people to participate, and while the payment was incentive compatible in that they could keep the money they did not spend, some people may have still considered the purchases free and opted for higher quality products than they otherwise would have. Our experimental design consisted of two randomized controlled trials run in parallel. Each participant made two choices, one as a member of the control group for one product, and one as a member of the treatment group for the other product. This is a non-standard setup and does provide for the chance of contamination of the control group, either from the initial survey altering a participant's control group choice, or the participant's control group choice influencing their treatment group choice. We think the effect of any such contamination is small, and include a more thorough discussion in the supplemental information.
In addition to the location-specific limitations, our use of SMARTER to generate the personalized scores limits the generalizability of our results. Other methods could have been chosen, and these would likely result in different outcomes. SMARTER was chosen for its speed and simplicity and comes with some limitations that other methods may not. For instance, it assumes that a product's utility can be decomposed into additive components, with no interactions between attributes. And while SMARTER does allow for non-linear attribute utility functions (e.g. an individual may like some sugar, but not too much), we did not allow participants to express such preferences. We did not attempt to validate the resulting utility functions as part of this research other than to ask participants how useful they found the letter grade information, to which 81% found them useful or very useful. Further, aside from carbon footprint information, the attributes provided to participants were those already readily available, either from the nutrition and ingredient labels, the store shelf itself (e.g. price), or obvious from inspection (e.g. bottle size). Other research on choice often starts with a preliminary elicitation of the attributes that are important to the choice of a product or service (Hensher et al 2005, Lancsar andLouviere 2008). In this case, for design simplicity, we did not conduct pilot testing and we selected the set of attributes that we believe to be of interest without that initial pilot testing. We are cognizant that participants may value other attributes and that is a limitation of our study that future work could address.
Prior research has shown habit formation and brand to be important aspects associated with consumption decisions (Chaudhuri and Holbrook 2001, Wood and Neal 2009, Zhen et al 2011. In this study, we studied only a single choice and so cannot adequately address such factors. This limits the interpretation of our findings. Our results do show that participants with the app made very different choices than those without it. However, we cannot attribute our results to the app itself versus the novelty effect of using the app, and we cannot conclude that those choices would be repeated in future purchases. That provides an interesting set of future research questions that is outside of the scope of the current study. Finally, carbon information was included in all participants' attribute sets. We were particularly interested in how people would respond to carbon footprint data so we made sure all participants were exposed to it (however, they could give it whatever rank they wanted). This could have resulted in a demand effect (Orne 1962) whereby the participants attempted to be 'good subjects' and over-weighted carbon in their decisions. If present, this effect was not detected in the cereal group but could account for some or all of the effect in the bottled water group.

Conclusion
This research demonstrates that personalized information, provided in real-time at the point of purchase, can significantly alter an individual's decisions. We found a statistically significant 23% decrease in the average carbon footprint of bottled water, and statistically significant improvements in numerous nutrition outcomes including reductions of 32% in sugar, 15% in ingredient count, 9.8% in sodium, 15% in fat, and a 47% increase in fiber content. Future research should focus on more carbon intensive consumer products, like meat and dairy (Eshel et al 2014), and non-food related purchases like appliances and consumer electronics. While bottled water and cereal are not representative of the thousands of consumer products available in today's marketplace, they do represent products at opposite ends of one dimension; the number of product attributes. Bottled water and cereal were chosen to explore the technology's effectiveness along this scale. And while the impact of real-time informed choice on other product types is as yet unknown, it is unlikely that water and cereal are the only two products that exhibit strong behavioral responses to this technique. This research opens the door to consumer-focused product information. Rather than being limited to the information producers and regulators have decided should be on the box, consumers can overlay their own, personally relevant information. While the AR experience used in this research proved effective, there are likely ways of further reducing cognitive burden and information overload that will yield even stronger results.