Study design and participants
This was a crossover randomized controlled trial conducted via an online grocery store in Singapore. Participants were recruited online via Facebook advertisements from January to April 2019. Prospective participants were directed from recruitment advertisements to the study website (https://nusmart.duke-nus.edu.sg/DIET) and asked to complete an online screener to determine their eligibility. Potential participants were eligible to participate if they were Singapore residents 21 years of age or above, and the primary grocery shopper for their household.
Potential participants who were both interested and eligible were then asked to complete: 1) a registration form containing name, delivery address, mobile number and email address; 2) an online consent form; and 3) the baseline questionnaire. Upon completion of the three forms, the website created the participant account and unique identification number for use throughout the study. Participants then received an automated email with their unique login details and were asked to logon to the NUSMart online grocery store to complete the first of three shopping tasks.
The study was conducted according to the guidelines laid down in the Declaration of Helsinki and all procedures were approved by the National University of Singapore Institutional Review Board Reference Code: S-18-073. Informed consent was obtained from all subjects. The trial was registered on the American Economic Association’s registry for randomized controlled trials, RCT ID: AEARCTR- 0002883; Registered July 09, 2018. The protocol was not amended following commencement of the trial and is available online.
NUSMart Online Grocery Store
NUSMart is an online experimental grocery store developed by the study team and used to run the present trial (https://nusmart.duke-nus.edu.sg/DIET). At the time of the trial, NUSMart contained over 4,000 food and beverage products commonly purchased at local supermarkets in Singapore. The web store was designed to mirror actual web-based grocery stores in Singapore, such as FairPrice Online (https://fairprice.com.sg), in both look and feel. It contained products across 26 major food and beverage categories, further subset into 111 sub-categories for easier shopping. All products include pictures of the items, current retail price and product descriptions. NIP and product information are available on click-through. NUSMart operates similar to other online grocery stores in that participants fill a cart with products as they shop and have the ability to add and remove products, and review purchases before hitting the checkout button.
Randomisation and masking
Using a within-person crossover design, participants were randomly assigned to one of six intervention sequences, which included the order of the three shops and when the actual purchase would take place, via random permuted blocks of size three with equal allocation for the six sequences (see Additional File 1 Table A1) by a computer program. Participants were blinded to intervention allocation, which was allocated via the NUSMart system. Allocation results were recorded within NUSMart and all investigators, including the data analyst, were blinded to group allocation.
Procedures
Arm 1 was the Control condition, which did not display FOP labels on any products. Arm 2 (termed HCS-only) displayed the HCS on eligible products, crossed referenced via the Health Promotion Board’s HCS database (https://www.hpb.gov.sg/food-beverage/healthier-choice-symbol). Out of the 4,177 products available on NUSMart, 311 (7·45%) carried the HCS. This was comprised of 150 foods and 161 beverages. Arm 3 displayed the HCS on eligible products as in Arm 2 and the PAE label on all products (termed HCS + PAE). PAE was calculated as the minutes required to burn off the calories of a single serving for a 73 kg person jogging at 8 km per hour.
For the study, we designed a simple PAE logo (Additional File 1 Figure A1) and added a description encoded into the NUSMart user interface to ensure that participants would understand the contents of the label. Participants saw the following description whenever their cursor hovered over the PAE label: “The Physical Activity Equivalent (PAE) refers to the number of minutes that a typical adult would need to jog to burn off the calories associated with one serving of the product.” Previous studies have shown this labelling approach to be effective (6, 7).
The labels were displayed at the bottom of the product images. Figure 2 shows examples of what participants saw in each arm for the same product.
All participants were exposed once to each of the three shopping conditions (1xControl, 1xHCS-only, 1xHCS + PAE) in random order. Participants were asked to shop once a week over a three-week period and were told on enrollment that they would need to purchase at least one and up to all three of their grocery orders. Following each shopping task, participants completed a brief survey to assess their mood and hunger level. ‘Mood’ took the values 1–5 where 1 was ‘very happy’ and 5 was ‘very unhappy’. ‘Hunger’ took the values 1–10, where 1 was ‘not at all hungry’ and 10 was ‘extremely hungry’. We used this information to determine if being either unhappy or hungry at the time of shopping moderated the effect of the labels. After completing the survey, participants spun a “Wheel of Purchase” to determine if they had to purchase their order. This was to ensure that there was a positive probability of having to purchase and receive the chosen products, thereby increasing the chance that product selections were an accurate reflection of participants’ actual shopping behavior.
For each shop, there was a minimum and maximum spend requirement of SGD50 and SGD250 respectively. This was to ensure that participants completed a typical weekly grocery order and to make the study more manageable given that the orders, when necessary, were repurchased by the study team using an external online grocery store, RedMart (https://redmart.lazada.sg). Participants who completed all study elements were rewarded with SGD75 worth of electronic vouchers from an online marketplace.
Outcomes
The primary outcome is the average calories per serving purchased (kcal per serving). We also assessed the following secondary outcomes:
- Proportion of HCS labelled products purchased (or would have been if not in control arm);
- Total Calories per shopping trip (kcal);
- Diet quality per shopping trip as measured by the Grocery Purchase Quality Index-2016 (GQPI-2016) and weighted average Nutri-Score;
- Sugar (g), Sodium (mg), and Saturated Fat (g) per serving;
- Calories per dollar (kcal per dollar) spent.
The Grocery Purchase Quality Index-2016 (GPQI-2016) contains 11 different food components with eight components scored based on adequacy and three moderation components. We followed the standard GPQI-2016 scoring methods by mapping NUSMart’s subcategories to USDA food plan categories and then to the GPQI components (8). Each component was scored based on the deviation of the observed expenditure share of each component and the expected expenditure share, and the scores were totaled up to generate the final GPQI-2016 score for each participant’s weekly grocery order.
We applied the standard Nutri-Score algorithm to assign a grade to each product (9–11). This algorithm assigns a score of A to E based on nutritional quality, which we recoded to 5 to 1 and then calculated an average score for each participant’s weekly grocery order, weighted by the number of servings of each product.
Prior to conducting the analysis, we standardized the serving size by using the mean serving size within each subcategory. This standardization ensures that similar products are compared equally as serving sizes can be arbitrarily set by the manufacturers. Missing values for nutritional information were imputed using the median value of the non-missing nutrients within the product subcategory. The median was used to avoid the imputed value being susceptible to outliers, which is possible for some subcategories that contain a small number of products.
Statistical Analysis
Sample Size
The sample size was estimated based on an the ability to detect a standardized effect size of 0·3 in calories per serving between any two arms with 80% power, 5% (two-sided) level of significance, and a correlation of 0·5 in purchases across the three shops. Accounting for a 20% attrition rate based on prior studies using NUSMart (12), we estimated that the required sample size was 108 participants. Lower attrition would allow for a smaller sample size.
Model
To test our hypotheses, a first difference regression model was used with coefficients estimated via Ordinary Least Squares (OLS) with errors clustered at the individual level to account for correlation within individuals across shops. Therefore, each participant generates two observations, with each dependent variable being the difference in the outcome for each treatment condition (HCS or HCS + PAE) relative to the Control condition. This difference was calculated by subtracting the control arm outcome from the treatment arm outcome. We tested the effects of the labels by applying the following first-differenced model separately for each outcome of interest:
which exploits the repeated observations of individuals by differencing out time invariant heterogeneity (e.g., age, health consciousness etc.) within individuals. The constant term α represents the incremental effect of the HCS condition relative to control. PAEit is a dummy variable that is set to one when the difference in outcome is between the HCS + PAE condition and control condition. βA represents the incremental effect of the HCS + PAE condition relative to HCS only. εit is the error term for each individual, i, and treatment condition, t. To account for the potential impact of hunger and mood on the primary outcome, the following model was employed:
where the additional Moderatorit term is a binary variable equal to one when the participants are not happy or hungry. We defined hungry and unhappy participants as those who had scores above the median. Separate analyses were run for each moderator. All analyses were run in Stata Version 15.2 (Stata Corp LP, College Station, TX).