eNutri app

The study relates to the eNutri app [9], in which participants complete an online FFQ and a report with personalised food-based dietary recommendations is automatically generated and presented on the screen. The report comprises an introductory message (e.g. “Hi John, this is your personalised report. The following messages present the most important diet changes recommended for you.”), followed by messages highlighting three dietary changes that the participant is recommended to consider. The messages for the participant are automatically selected by the eNutri app based on the three m-AHEI components with the lowest scores [11].

The report also shows the participant’s m-AHEI score, referred to in the report with a more user-friendly name of “Healthy Eating Score”, presented as a percentage and a coloured bar (Fig 1). The colour of the bar denotes how closely the user complied with the AHEI, with red representing a score in the lowest third, yellow in the middle third, and green in the highest third. Intakes were scored as a percentage of the target as recommended by the m-AHEI. The participant’s score on each of the 11 m-AHEI components are also shown, their scores for ‘recommended foods’ (vegetables, fruits, whole grains, dairy products, nuts & legumes, healthy fats, oily fish) were presented first, followed by their scores for ‘foods to limit’ (sugars, red and processed meat, salt and alcohol). The bars for the ‘Recommended Foods’ used the same traffic light system as the overall “Healthy Eating Score” score. Although the ‘Foods to Limit’ had the same style of bars, the colours were inverted (e.g. red colour if the score exceeded two-thirds of the recommended maximum intake).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. eNutri presenting the ‘Healthy Eating Score’, ‘Recommended foods’ and ‘Foods to limit’.

Reprinted from [10] under a CC BY license, with permission from Rodrigo Zenun Franco, 2019.

https://doi.org/10.1371/journal.pone.0214931.g001

A user could see a weight range bar and recommendations based on their body mass index (BMI). The report did not present the BMI explicitly, but used the healthy BMI range (18.5-25kg/m²) to calculate the healthy weight range for the user’s height, and this range was presented in green. An arrow on the bar indicated the user’s current weight [11]. The physical activity levels are defined based on the Baecke Questionnaire [15], which is a questionnaire of habitual physical activity. It has been validated and found to be repeatable [16] (Fig 2).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. eNutri presenting the weight and physical activity ranges.

Reprinted from [10] under a CC BY license, with permission from Rodrigo Zenun Franco, 2019.

https://doi.org/10.1371/journal.pone.0214931.g002

Formative study

For a fuller understanding of the study described in the next sections involving evaluation of the eNutri app by nutrition professionals, this section describes some of the background to and rationale behind the design of the personalised diet report. Using an initial version of the app described in [11], a formative study with 20 participants, without nutrition expertise, was conducted. The participants’ characteristics of the formative study have been described by Franco and colleagues in a publication that presented the evaluation of the FFQ (9) but not the evaluation of the nutrition report. Here, we report the latter which was evaluated via a semi-structured face-to-face interview (S1 File) and addressed the first aim of the study to i) evaluate and optimise the personalised nutrition report and app delivery. The interview included questions regarding participants’ understanding of the content and terms used in the diet messages, and also the visual representation of the status bars [11]. The interviews also explored questions related to the perceived effectiveness of the report and ideas for potential improvements and new features for the system. An interview protocol, available as supporting information (S1 File), was used by the researcher. Positive comments, negative comments and suggestions were grouped and counted. The results of this formative study and how they influenced the design of the nutrition report are summarized in the following paragraphs.

The most frequently displayed components (according to the participants’ FFQ results) were “nuts and legumes” (27%), “red and processed meat” (27%) and “salt” (15%). The 20 participants indicated that they understood all the content and terms used in the advice messages. Two participants clicked on the small superscript help icons (Fig 1). Only one participant clicked on the “Your Progress” tab (Fig 1), which was the second option on the menu tab, while for the remaining 19 participants, the researcher had to point out the second tab.

The 2 and 3-colour progress bars used for weight and physical activity [11] received satisfactory responses, confirming participants’ understanding of these visual representations. One participant stated clearly that “I am thinking as a traffic light system”. Regarding the single colour status bars used for the food-based scores (Fig 1), the participants provided very good explanations for the traffic light representation for both categories (e.g. red colour in ‘Recommended Foods’ means to eat more and in ‘Foods to Limit’ means to decrease consumption). When asked, they were also able to compare the components (e.g. the ‘Vegetables intake’ is better than the ‘Fruits intake’ in Fig 1) for both categories, although they were not sure of the meaning when the score was 100% (i.e. full bar). One participant stated: “I understand the message, but I am not sure about the percent of what”. For the ‘Recommended Foods’, responses including “the higher, the better”, “I am eating 75% of what I should” were given, but some clarified that they were not sure if they should be eating exactly 100% and not more than that. For the ‘Foods to Limit’, the responses were much less satisfactory compared with responses for the ‘Recommended Foods’, in the context of what the bars were designed to represent, and it seemed that understanding of the messages were based on the traffic light colours only. In other words, the status bar was not increasing the user’s understanding.

Based on the participants’ feedback, a number of improvements were applied to the eNutri report based on the ‘main suggestions’ of the non-professional group and taken forward into the study with the NP. These are summarised below:

1. The two tabs (“Advice for you” and “Your progress”) of the report (Fig 1) were merged into a single tab and the progress report was placed below the advice messages;
2. Clarification of which individual food items had the greatest contribution to the recommended foods or foods to limit (e.g. which items were the top contributors to “red and processed meat”) (Fig 3);
3. Explanatory subheadings of the two component groups were included in parenthesis: Recommended Foods (The higher the better) and Foods to Limit (The lower the better).
4. Each main message block (Fig 3) was revised to a standard format: the first paragraph focused on the current status of the specific component; the second provided advice (i.e. “call to action”) and the final paragraph described the health benefit potentially obtained from that specific behaviour change.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Advice on the biggest contributors to a component as main advice and component details.

Reprinted from [10] under a CC BY license, with permission from Rodrigo Zenun Franco, 2019.

https://doi.org/10.1371/journal.pone.0214931.g003

Participants

The revised design of the reports were evaluated with RD and RN with more than 2 years experience of providing individual dietary advice. RDs were recruited via the 'Freelance/Private Practice Dietitian's' Facebook Group, which included 945 members at the time of the recruitment post. A total of 28 RD responded to the request for study volunteers within 12 hours of the post, after which time the researcher [RF] advised the group that the recruitment target had been met. RD eligibility was verified according to participants’ Health and Care Professions Council (HCPC) registration number (http://www.hcpc-uk.co.uk).

RN were recruited from the Association for Nutrition (AfN) website (www.associationfornutrition.org) via the 'search the register' function. Individuals registered as a Nutritionist on the AfN voluntary register who declared that they 'accepted clients' were contacted (n = 85). The study was also advertised on the researcher’s Twitter account and re-tweeted by the AfN. In total, 21 RN responded to the email and 2 to the tweet. For both RD and RN, the first 16 participants who responded were included in the study.

Study design

Recruited NP (n = 32) were invited to complete an online survey evaluating the eNutri app, via JotForm (jotform.com, San Francisco, CA). Following online screening and consent, each participant was sent two pre-defined scenarios, comprising dietary intake analysis of FFQ data (portion, frequency and weight per food item, and energy and nutrient intake per day) and individual characteristics (age, sex, weight, height and BMI). A total of 16 scenarios were designed to mimic outputs from Nutritics Professional Analysis Software (Nutritics Limited, Dublin, Ireland) (S2 File) and included the Dietary Reference Values (DRVs) provided by the Committee on Medical Aspects of Food and Nutrition Policy (COMA) and the Scientific Advisory Committee for Nutrition (SACN) in the UK (year 2015) [17]. NPs were randomly assigned two of the 16 scenarios and asked to consider these scenarios using their professional judgment and provide the three most relevant nutrition recommendations, via JotForm (Q1). Participants were asked to provide responses via audio recording or free-text boxes.

Following this stage, NPs were asked to access the nutrition advice (i.e. not including the weight and physical activity blocks shown in Fig 2) generated by eNutri for each scenario (via a password-protected link) and rate their level of agreement, considering its appropriateness, relevance and suitability, via 5-star scales (5 denoted complete agreement and 1 no agreement). NPs were also asked to comment (via audio or free-text) on the eNutri recommendations (Q2), scores generated (Q3) and overall impression (Q4), according to each scenario presented. Finally, participants were asked to provide any final comments or feedback on the eNutri app, irrespective of the individual scenarios evaluated.

Scenarios.

Each of the 16 scenarios (where a scenario comprised a FFQ completed by an individual) was independently evaluated by four NP (2 RD and 2 RN). With this approach, each scenario received recommendations and evaluations by 4 different NPs. The scenarios were designed to represent a diversity of diets and individuals. The diet variable defined was the m-AHEI with selection of individuals also based on sex, age and BMI. These four variables at two levels each (m-AHEI: <40 or >60, sex: M or F, age: <40 or >40 and BMI: Ideal or Overweight) were used to generate 16 distinct combinations as shown in Table 1. The FFQ scenarios (n = 16) were drawn from a pool of FFQ responses from actual users, and so have good ecological validity. In total, 64 analyses were undertaken (16 scenarios x 4 analyses per scenario).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of subjects presented to nutrition professionals according to scenario.

https://doi.org/10.1371/journal.pone.0214931.t001

The characteristics presented in Table 1 and their associated FFQs were entered into the app database in order to simulate the nutrition reports for the scenarios (S2 File). The NPs were able to browse the reports online and their screenshots are presented in S3 File.

Data analysis

Participant characteristics

Participant characteristics according to profession (RD, n = 16; RN, n = 16) are shown in Table 2. 90% of NPs were female. The mean age was 37 years (SD 10) and reported years since graduation was 10.9 (SD 8.2). No significant differences were observed between the professions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Participant characteristics according to profession (RD, n = 16; RN, n = 16)^a.

https://doi.org/10.1371/journal.pone.0214931.t002

Professional recommendations

The option to provide recommendations via audio recording was not used by any NP. The primary nutritional targets (n = 3) listed by NPs following evaluation of the scenarios are shown in Fig 4 and detailed in the S1 Table; a total of twelve targets are listed per scenario (4 NP x 3 recommendations), although this analysis does not illustrate the agreement at the level of the individual scenarios. In total, 43 distinct targets were selected, which included nutrients (e.g. fibre), food items (e.g. red meat), eating occasions (e.g. breakfast) or ‘ratios’ of different nutrients/composition of macronutrients in the diet (e.g. ratio of saturated to unsaturated fats). Targets were not set on five occasions (scenarios 8, 14 and 16), whereby the NPs described needing further information in order to ascribe a recommendation. For scenario 2, one NP stated that the individual’s diet was “ok”/required no improvement. The greatest agreement in targets selected by NP, according to scenario, (selections per scenario > 3) was observed for energy, macronutrient targets (e.g. saturated fatty acids (SFA), fibre, alcohol) and fruit. Less agreement was seen for micronutrients (e.g. phosphorus, iron, vitamin A, selenium), which NP selected less frequently and accounted for < 10% of the total targets. The top five items targeted by NP, according to overall frequency, were SFA (9.4%), salt (8.3%), fibre (7.8%), sugar (7.8%) and vitamin D (7.8%).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Number of times each target recommendation was selected by nutrition professionals and by the eNutri app for all scenarios combined.

https://doi.org/10.1371/journal.pone.0214931.g004

In the majority of cases a nutrient was the first target mentioned by the NP, although this was frequently followed up with food-based advice; for example: “Increase intake of dietary fibre by increasing whole grain food instead of white bread, non wholegrain cereals” (RN16) and “reduce sugars: reducing intake of sweet biscuits, chocolate, rich cakes, buns, muffins, pastries (…)” (RD04). In some cases, NPs also mentioned the reason and associated health benefits of implementing the suggested changes; for example: “It is important that you start your day with a nutrient rich breakfast as this will not only help to keep your energy levels steady, but it will also support cognitive function helping to keep you focused and alert throughout the morning” (RN05) and “Reducing his intake of alcohol could help to reduce his weight” (RD03). Some NP also listed likely improvements to other nutrients, in addition to that targeted, for example: “try to include 2 portions of oily fish per week (…) This would also boost vitamin D and iron intake” (RD15). Alternatively, some described the consequences of current intake/habit: “Deficiency of vitamin A can cause fatigue, increase risk of infection, poor vision and dryness of the eyes, skin and hair” (RN06).

Nutrition recommendations were typically framed according to “increase”/”choose”/”consume more” or “reduce”/”decrease”/”limit”/”stop”/”cut down”, and less frequently as substitutions or food-swaps; for example: “Swap jams/marmalade/chocolate and nut spreads for low-fat cheese spreads” (RN10) and “(…) reducing intake of biscuits, chocolate and fizzy drinks. In place of these try fruit and natural yoghurt for puddings (…)” (RD02).

Overall, the detail provided by the NPs for each recommendation varied (range 12–176 words), with the greatest quantity of text provided for a “calorie deficit” recommendation; in this case, the NP described the weight loss target, energy requirements and deficit, and a list of food based swaps and portion size recommendations to facilitate a calorie reduction (e.g. “grill fish rather than fry”) (RN10). NPs referred to individuals’ characteristics, and particularly BMI for overweight cases, where relevant; recommendations to reduce energy were also more frequent in scenarios with overweight individuals (3–4, 7–8, 11–12, 15–16) (see Table 1). Females of childbearing age (“females of birthing age”, “(…) she may be wishing to conceive”) were also highlighted in relation to recommendations to increase iodine and iron (scenario 9), folate (scenario 12) and reduce protein by “(…) reducing number of oily fish portions per week (no more than 2 per week recommended” (scenario 10, RN10).

Vitamin D was targeted by at least one NP in the majority of scenarios; follow-up recommendations included the consumption of vitamin D rich food/fortified products: “Increase intake of vitamin D foods e.g. dairy produce, oily fish, eggs, fortified products” (RD16) or supplementation during the autumn/winter months; for example “Depending on time spent outside in the summer, take a vitamin D supplement to ensure (…) intake averages out to 10ug/day” (RN07).

Evaluation of the eNutri feedback

The diet targets automatically selected by the eNutri app and the professional evaluation of the advice generated, according to each scenario, are shown in Table 3. The most frequently selected items, according to overall frequency, were legumes (20.8%), whole grain (16.7%), and red meat, oily fish, healthy fats and sugar (all 10.4%). The mean scores for the appropriateness, relevance and suitability of the eNutri diet messages were 3.5, 3.3 and 3.3 respectively (maximum score, 5). Two scenarios (1 and 3) scored > 4 on all three aspects and the lowest mean scores (2.5) were observed for scenarios 15 and 16, which were both overweight females, aged > 40 years.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Professional evaluation of eNutri automated personalized nutrition advice according to scenarios (n = 4 responses per scenario).

https://doi.org/10.1371/journal.pone.0214931.t003

Recommendations.

The NPs comments regarding the eNutri recommendations predominantly related to whether the NP agreed with the three food components selected by the eNutri app. Positive quotes included: “All advice given was very appropriate” (RD16), “I definitely agree with the recommendations provided and would say they are more suitable than the ones I have provided! Great practical advice provided in the report” (RD15) and “Good advice relevant to diet” (RN03). However, considering the differences between the NP and eNutri recommendations (see Tables 3 and 4), preferential targets were frequently mentioned: “Give good information regarding the topics, although I feel some of the areas chosen by the software were not the most important aspect to change in terms of the clients diet” (RN08). Particular components perceived to be important and omitted from the eNutri app advice included saturated fat, vitamin D and micronutrients. NP also queried the prioritisation of components such as whole grain when dietary fibre was adequate; and dairy when calcium and protein were sufficient.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Quotes identifying positive and negative aspects of the eNutri automated personalized nutrition app scoring system identified by nutrition professionals.

https://doi.org/10.1371/journal.pone.0214931.t004

Themes largely mapped to those identified in the analysis of NP recommendations, including message framing, client context (e.g. allergies) and reason for implementing suggested recommendation. There were contradictory opinions regarding the quantity of information presented to the individual: “recommendations are fine on but worrying that it is limited to 3?” (RN16) versus “It felt too overwhelming (…)” (RN04). NPs commented that client context, including allergies, intolerances, religious beliefs and food preferences, should be considered further to increase the ‘personalisation’ of the recommendations. For example, it was noted that in certain scenarios, individuals reported 0g of red meat, fish and dairy intake, yet advice included the consumption of more oily fish; thus, it was highlighted by a NP the “Need to discuss alternatives to oily fish for vegetarians or none [sic] fish eaters” (RD14). A NP also commented that: “It is not a tool that would help a professional especially if they were looking to help with health issues as it does not look at the individual—it is based on general dietary advice but what happens if that person has an allergy […]” (RN02).

The inclusion of descriptions relating to the benefit of implementing a dietary change were perceived as positive: “I liked that the advice included the reasoning for the recommendations, and it was great that it mentioned preventing the non-communicable diseases”, although it was suggested that more relevant conditions could be mentioned according to the specific case: “The participant was a 25 year old male, who may not yet be concerned about Heart Disease etc. It may have been more appropriate to also mention about other benefits of the recommended foods, such as increasing satiety.” (RN04). Message framing was also praised: “(…) the positive reinforcement preceding the recommendation is an excellent way to introduce this change.” (RD12).

Scores.

Quotes relating to NP appraisal of the eNutri scoring system are summarised in Table 5. In general, NP described ‘agreement’ with the scoring system (i.e. components and scales) and scores in the context of the scenarios’ nutritional analysis, for example: “Appears to reflect the individuals food choices” (RN05), “I felt the scores were all accurate and suitable for this lady” (RD16) and “the healthy eating score is reasonable” (RN09). Although, a few NP stated, “The healthy eating score is very low; it should be higher” (RN09) and “82% seems a bit low given that the diet is pretty good” (RN07). Perceived benefits of the scoring system included the presentation of the data according to the traffic light system and focusing the client to an area of diet that needs improving.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Positive aspects and areas for improvement to the eNutri automated personalized nutrition advice identified by nutrition professionals.

https://doi.org/10.1371/journal.pone.0214931.t005

However, concerns were raised regarding the benefit of a scoring approach: “I’m not convinced about the scoring system. It could appeal to those with competitive edge, but many of my clients would not be interested (…)” (RN04), including whether it would be understood: “Scores are useful but may be confusing to someone without a nutritional background” (RD16) or useful: “most people know if their diet is not healthy–the difficulty is how to change (…) scoring them could just be another indicator that they are not capable of getting their life in order” (RN02). It was also noted that a predominance of ‘red’/’negative’ scores might be demotivating.

Within the comments relating to scores, NP also highlighted particular components that would benefit from refinement; for example, the combination of red and processed meat in the context of achieving sufficient iron intake was raised: “(it’s) risky to be advising intakes (of red and processed meat) should be ‘as low as possible’ when female, and clients intakes are low” (RN15). The absence of a component on total and saturated fat was also raised and a NP commented that feedback “Should consider fruit carbohydrate sugars in the sugar recommendation or provide comment” (RD14).

In general, opinions on the scoring system differed, as illustrated in Table 4, however a consistent theme was the desire for greater understanding of the scoring system in terms of an absolute amount (e.g. users intake marked as 2 portions/160g of vegetables), as opposed to percentage of an unknown maximum.

Overall impressions and suggestions

Positive aspects and areas for improvement to the eNutri app identified by NP are summarised in Table 5. Positive comments were generally shorter in length compared with negative comments/ recommended improvements. NP were positive about the ‘food-based’ approach used in the eNutri app: “(…) It was useful that individual food groups were recommended, rather that macronutrients as this may appeal to individuals who are less educated in nutrition” (RN04); although, several NPs commented on the need for specific advice on vitamin D intake and supplementation. One NP also perceived a lack of information about saturated fat as “a big gap” (RD10). The consideration of ‘foods to add’ in addition to ‘foods to limit/cut out’ was also considered positive, as was the framing of foods that should be eaten less frequently (i.e. foods to limit): “(…) I also like the fact that it describes food to “limit” rather that totally avoid which is a healthier message to spread” (RN04).

Feedback relating to the visual presentation of the data (e.g. key messages, scores, use of traffic light system) was largely positive, for example: “the use of a scoring system is a helpful motivating measure for participants to make recommended changes” (RD12), “quite impressed with the report generated” (RD13), “Good to provide swap ideas” (RD01) and “really practical recommendations highlighting several important changes which could be made” (RD03). However, contrary opinions were also noted in relation to specific scenarios and, when comparing reports, a NP commented: “I am less happy with these recommendations. They are very difficult to put into practice” (RN11). In addition, NP raised concerns with the intake of certain foods, such as nuts and low-fat dairy products, being described in terms of ‘the more the better’.

Of particular note was feedback relating to overweight participants, and NP who reviewed these scenarios commented on the need for a greater focus on energy balance, nutrient density, motivation and weight loss: “Does not consider the need to lose weight and have a healthy BMI” (RD01); “too basic and general for someone who clearly needs nutritional help–they are overweight so the chances are they know diets inside out already” (RN02); “I don’t think the advice was correct for this patient as it did not take into account the fact she was overweight” (RD05). In relation to this, NP suggested that messages should include content on energy reduction and portion sizes.

Overall, NP considered that the system and associated advice was ‘basic’, provided “Good advice for general healthy eating feedback” (RD07) and was “(…) ok for someone who just wants to see where his diet needs improving (…)” (RN02). However, the advice was also described as: “too basic and generalised” (RN02) and that the advice “could be more personalised e.g. inclusion of (fibre-containing) nuts/oats in the participants' smoothies which would be in line with their existing intake and thereby making it more meaningful and easier to incorporate” (RD12) and “still far from a personalised advice that takes into consideration the preferences, adverse reactions to foods, body composition, etc.” (RN09). Whilst it was acknowledged that an app might not be capable of providing the same ‘level’ of personalisation as a NP, several areas for improvement were noted, including: relating advice to a user’s actual intake, providing links to recipes, ability for the user to interact with recommendations (e.g. select those they feel most able to implement) and including a message on the overall diet.

NP also commented on the need to consider the wider ‘context of the client’ and lack of ‘big picture’ when providing dietary advice, for example: “Need to understand background such as cholesterol and blood pressure” (RN02), “consideration of body composition” (RN09), “consideration of personal preferences/ cooking abilities” (RD11), noting that the system “does not take into context wider health and lifestyle issues” (RN15). A suggested improvement was that wider health issues be considered within the app: “I think it would be good to get an indicator of levels of activity and lifestyle choices e.g. smoking as you have mentioned alcohol but also I think as HCP we should also be encouraging healthier lifestyles in regards to alcohol, smoking and exercise” (RD09).

Specific improvements relating to the scores were mentioned, as there was some confusion regarding the percentage system; NP suggested that the absolute amount corresponding to the maximum score be provided for each component (e.g. a score of 100% for vegetables equals 3 portions per day) and that potential improvements in scores (i.e. if changes were made) might be visually presented to the user to enhance motivation. Contradictory comments were made in relation to the framing of diet messages, which some NP commented were ‘slightly judgemental’/’harsh’ and others described as ‘quite positive’, for example “The inclusion of positive reinforcement with words such as 'Good Job' also lends a personal, positive touch” (RD12).

Limitations

The eNutri app relies on data from a FFQ, which has some limitations. Identification of specific foods consumed is more limited compared with other methods, such as diet diaries, which may reduce its ability to identify changes in consumption of some foods. In addition, FFQs are not necessarily the nutrition assessment method that NP would utilise during nutrition consultations; 24-hour recall and food diaries are most frequently selected. This study is subjective and qualitative in nature making it more difficult to apply comparative statistics. Furthermore, the food intake on which the eNutri personalised advice was based was translated to food group equivalents by the app, however this information was not provided to the NP. In some cases, this could have contributed to the different dietary targets identified by the eNutri app and the NP. The number of female NP was much higher than male NP, which may be considered as a potential limitation. However this is reflective of the gender distribution of these professions; in the present study 16% of RNutr identified were male and in a recent international survey of RD, 4% of UK respondants were male [2].