The “Healthy Meals” web app for the assessment of nutritional content and food allergens in restaurant meals: Development,evaluation and validation

Abstract

Objective

The development and the evaluation of the Healthy Meals web app designed for professionals from different disciplines related to food, aimed to assess the nutritional and food allergen content of restaurant meals, was described.

Methods

App evaluation concerned: (1) usability, scored on a 7-point scale by 6 restaurateurs and 10 nutritionists through the Computer System Usability Questionnaire; (2) quality, scored on a 5-point scale by 10 nutritionists through the Mobile App Rating Scale; (3) validation, by two nutritionists through differences in entered nutrient contents. Ratings reliability was assessed by the interclass correlation coefficient.

Results

Users agreed with the web app usability (mean 5.6/7 points, SD 0.9), with moderate reliability among ratings (interclass correlation coefficient = 0.57; 95% CI, 0.18 to 0.82). The web app showed good objective quality (mean 4.0/5 points, SD 0.4), with excellent reliability among nutritionists (interclass correlation coefficient = 0.91; 95% CI, 0.85 to 0.96). For web app validation, no significant differences were observed between the two nutritionists’ data, with excellent reliability (interclass correlation coefficient = 0.98; 95% CI, 0.97 to 0.99). App data entry was identified as a point to improve.

Conclusions

The Healthy Meals web app designed for professionals related to food, such as restaurateurs, demonstrated to be usable, of good quality and valid for dishes nutritional assessment and food allergen identification. Points to improve were identified, while app effectiveness should be tested in trials.

Keywords

Nutrition assessment food allergy food intolerance allergens mobile applications mobile health restaurants

Introduction

The assessment of the nutritional contents of meals and the identification of food allergens is necessary to guarantee the availability of heathier meals and allergen-free dishes to consumers.¹ Nutritional information on the menus of food services and restaurants could help customers make healthier food choices while increasing their awareness about the nutritional composition of meals.^2,3 In particular, previous nutritional content assessments of restaurant and fast food offerings have pointed out the high energy content and poor nutritional quality of the offered dishes.^4,5 Thus, more support of restaurants is required for the establishment of healthier menus that are developed according to the recommended nutritional guidelines.⁶

Nutritional content analysis and dietary intake assessment can be performed using different food composition databases,^7,8 which are valid cost-effective alternatives to chemical analysis.⁹ However, restaurateurs are demanding better digital tools that could provide more consistent nutritional data and related food information,¹⁰ including suitability for particular dietary patterns such as veganism/vegetarianism or the presence of food allergens. In Europe, the vegetarian and vegan population is growing, and in Spain, the prevalence of vegetarians and vegans reached 1.5% and 0.5%, respectively, in 2019.¹¹ Similarly, food allergen detection by restaurants represents another important assessment to allow for correct allergen management and the avoidance of severe reactions by consumers with allergies,¹² whose incidence is growing worldwide.¹³

Apart from the European Union (EU) legislative regulation that requires food producers to declare the 14 groups of foods recognized as allergenic, reliable methods are needed to help restaurants identify these food allergens in their menu offerings.¹⁴ From recent surveys, it was found that restaurant managers and staff still have important gaps in food allergy knowledge, such as believing that consuming small amounts of allergenic food is not harmful for allergic people¹⁵ or that removing the allergen from a finished meal provides a safe meal.¹⁶ To fill this allergy knowledge gap, aside from ensuring adequate training to the staff with the support of health professionals,¹⁷ the US Food and Drug Administration recommends that restaurants implement computer technology for the identification of food allergens present in menu items, which could also help cook in the preparation of allergen-free meals.^18,19

Thus, eHealth technology in the form of a web app represents a potential strategy for food composition analysis and allergen identification due to its feasibility, accessibility,²⁰ customization and engagement.²¹ However, existing diet- and nutrition-related eHealth systems for nonprofessional use still lack data on validity and reliability.²²

The present study describes the development of the Healthy Meals web app with the proposal of (1) assessing the nutritional content of dishes according to the Guideline Daily Amounts (GDA) for food servings, expressed in the form of a Multiple Traffic Light (MTL) system label, and (2) detecting the presence of the 14 recognized food allergens in restaurant meals.

In particular, the Healthy Meals web app could increase awareness of the nutritional composition of restaurant meals, encouraging the improvement of meal healthfulness and food allergen identification by restaurateurs.

Thus, the present study aims to describe the following regarding the Healthy Meals web app designed for professionals from different disciplines related to food, such as restaurateurs: (1) its development and (2) its evaluation, including the assessment of usability, quality and validation.

Methods

The Healthy Meals web app was created in the framework of a EU funded project, called PECT-TurisTIC in Family, for use in the Healthy Meals Randomized Controlled Trial, of which one of the aims was to improve the nutritional content and food allergen management of meals offered by full-service restaurants in the province of Tarragona (Catalonia, Spain).

Development of the Healthy Meals web app

The development of the Healthy Meals web app required four steps: (1) the analysis of the scientific evidence regarding food labelling systems providing nutritional information, (2) design of the web app interface and features, (3) development of the first prototype in the Spanish language and (4) revision of the web app by researchers and developers to verify its functionalities and identify required adjustments.

Development step 1: analysis of the scientific evidence regarding food labelling systems providing nutritional information

The first step of web app development consisted of a literature search to determine a more convenient and comprehensible labelling system to communicate the nutritional information of the assessed meals to restaurateurs. The MTL labelling system was identified as the most readable label for the interpretation of nutrient content.²³ Along with the need to facilitate the nutritional assessment of meals, restaurateurs need to facilitate the identification of food allergens,¹⁹ as well as suitable options for individuals with particular dietary patterns such as vegetarianism and veganism.²⁴ Thus, these features were then considered for the development of the web app.

Development step 2: design of the web app

For a single plate portion, the Healthy Meals web app requested the entry of the following data:

name, description and photo (optional) of the plate, and

ingredients, selected from a food database together with their detailed information such as (a) weight in kg/g/mg (net or brute); (b) type of cooking (no cooking, fried, stir-fried, boiled, floured, battered), and according to the selected options, the web app calculates the percentage of oil/egg/flour absorbed during cooking based on the food net weight; and (c) type of oil (olive, sunflower, palm) or flour (wheat, rice, oat, barley, rye, chickpeas, soy, corn) if required for cooking.

From these data entered in the web app, the nutritional composition of the dish is shown in the form of a technical sheet (Figure 1), which summarizes the ingredients, weights, cooking methods, oil used, and whether the plate is a vegetarian/vegan option.

Figure 1.

Example of a technical sheet created by the Healthy Meals web app for a seafood paella recipe, per single portion serving. Note: The web app generated a technical sheet for the seafood paella recipe (translated to English by researchers). Starting from the far-left column, the following information is provided: The list of ingredients, the ingredients’ weight, the type of cooking, the type of oil used and the allergens within the dish.

Furthermore, the web app identifies and indicates which of the 14 common allergens were certainly (red-marked) or potentially (orange-marked) present, according to the ingredients and the cooking method used (Figure 1). When appropriate measures have been taken to avoid allergen cross-contamination during meal preparation (for instance, if the cook has used a clean pan and oil for frying a food instead of using the common fryer), the user could also deselect the orange-marked allergens that the web app had automatically indicated for precaution.

From the nutrient content analysis, an MTL label including energy (kcal), protein (g), carbohydrates (g), sugar (g), fat (g), saturated fat (g), sodium (mg) and fibre (g) content was created (Figure 2). In the MTL label, nutrient content is rated as one of three colours according to (a) high (red-coloured), (b) medium (orange-coloured) or (c) low (green-coloured) content, in agreement with the cut-off of the UK Food Standards Agency and the GDA for an adult's healthy diet of 2000 kcal recommended by the EU.^25,26

Figure 2.

Example of the MTL label derived from the nutritional assessment of the seafood paella recipe generated by the Healthy Meals web app. Note: Using the Healthy Meals web app, an MTL label was created after entering data from the seafood paella recipe (translated to English by researchers). This MLT shows the content of energy in kcal, protein in grams, carbohydrates in grams, sugar in grams, total fat in grams, saturated fat in grams, salt as sodium in milligrams, fibre in grams and the GDA for each nutrient as a percentage. Nutrients are classified into three colour codes according to their high (red), medium (orange) or low (green) content. In the seafood paella dish, energy, protein, carbohydrates, fat, saturated fat, and sodium are coloured red, while sugar is green and fibre is yellow. Moreover, the Health Meals web app provided a summary of potential and certain allergens present in dishes in the box above the MTL.

Specifically, a single plate portion was marked in green if it contained <7.5% of the GDA, orange when it contained between 7.5% and 20% of the GDA, and red when it contained >20% of the GDA.

Development step 3: Healthy Meals web app prototype

The Healthy Meals web app prototype was developed in cooperation with professional software developers. Data from different public food composition databases^27–31 and commercial food products were inserted into the Healthy Meals web app as a source of nutritional information for nutrient assessment and allergen identification.

Additionally, the possibility of further inclusion of nutritional data was included, allowing for the update and enrichment of the food database by nutrition professionals and for the consideration of users’ requests, especially concerning food products not present yet in the database, such as specific commercial food brands.

Development step 4: revision of the Healthy Meals web app

The initial prototype of the Healthy Meals web app was reviewed step by step by developers under the supervision of researchers to verify functionalities and detect points to be adjusted. Examples of data entry and nutritional calculations were performed to test the functionality of the web app and evaluate its design.

Evaluation of the Healthy Meals web app

The evaluation of the final prototype of the Healthy Meals web app included the assessment of (1) web app usability to assess the usability, functionalities and features; (2) web app quality to assess its objective and subjective quality; and (3) web app validation to measure the accuracy of recipe data entry (Figure 3).

Figure 3.

Description of the healthy meals web app evaluation process including usability, quality and validation.

Evaluation of web app usability by restaurateurs and nutritionists

For the web app usability evaluation, a minimum number of 16 users comprising restaurateurs and nutritionists were recruited as an appropriate sample for identifying web app usability problems, in line with Alroobaea et al.³² Users were asked to enter a recipe sheet into the web app with detailed information about the cooking steps and the ingredients and their quantities for one serving. Following the data entry of the recipe sheet, web app usability was evaluated through the Computer System Usability Questionnaire (CSUQ, version 3), which was validated in the Spanish language³³ and administered online. The CSUQ was derived from the Post-Study System Usability Questionnaire, and its reliability and validity have previously been demonstrated.³⁴ The CSUQ consists of 16 items rated on a 7-point Likert scale: 1: strongly disagree, 2: disagree, 3: somewhat disagree, 4: neither disagree nor agree, 5: somewhat agree, 6: agree, and 7: strongly agree.

The CSUQ evaluates four different factors: (1) the system's usability (questions 1–6), (2) information quality related to the web app usability (questions 7–12), (3) interface quality including the set of features provided by the web app to users (questions 13–15), and (4) overall usability (question 16).

Moreover, reliability among the 16 users evaluating web app usability was assessed to measure the replicability of their ratings and was considered (a) moderate (interclass correlation coefficient (ICC) 0.50 to 0.75), (b) good (ICC 0.75 to 0.90), or (c) excellent (ICC >0.90).³⁵

Moreover, to identify the web app's points for improvement, users were asked two further questions in Spanish by researchers, and these results were expressed as percentages:

Have you encountered difficulties entering the recipe data? If so, at what point? Please, select from the following options using a cross: (a) Finding ingredients (the system does not work/lack of ingredient availability/lack of specific food brands/other problems), (b) entering food weight (I do not know the exact weight/I know quantities in pieces or spoons or pinches/other problems), and (c) selecting type of cooking (It is not available/there are more types of cooking/I use a different type of oil or flour or egg/other problems).

Do you think the web app should be improved? If so, in which aspects? Please, select from the following options using a cross: (a) Design, (b) Functionality, (c) Comprehensibility, (d) Data entry, (e) Data analysis, (f) Data presentation, and (g) User support.

Evaluation of web app quality by nutritionists

Web app quality evaluation was performed using the Mobile App Rating Scale (MARS) tool among the same panel of 10 nutritionists who also evaluated web app usability, since there is no evidence regarding the minimum sample size required for web app quality assessments. Nutritionists were recruited for the quality evaluation since they are confident and competent to verify app components such as the information content and functionalities.³⁶ Nutritionists were asked to enter a recipe sheet into the web app and to assess web app usability and web app quality through the MARS tool, which was provided online.

The MARS consists of objective and subjective quality assessments and has been recently adapted and validated in Spanish.³⁷ The objective quality comprises four sections with 19 total items. Specifically, the sections are (1) engagement, (2) functionality, (3) aesthetics, and (4) information. On the other hand, the subjective quality assessment comprised four items, with a total of 23 questions about web app quality.³⁸

Furthermore, to collect information about the perceived impact of the web app on the user, six items belonging to the optional app-specific MARS section were asked.³⁸

Each MARS item was scored on a 5-point scale: 1: inadequate, 2: poor, 3: acceptable, 4: good, and 5: excellent. Total mean scores were calculated for each section. In particular, regarding the objective quality sections, for the engagement section, the maximum score was 25 points; for the functionality section, it was 20 points; for the aesthetics section, it was 15 points; and for the information section, it was 35 points, for a total of 95 points. For subjective quality, the maximum score was 20 points, and for app-specific section, it was 30 points.

Reliability among nutritionists evaluating the web app was assessed to measure the replicability of their ratings and was rated as moderate (ICC 0.50 to 0.75), good (ICC 0.75 to 0.90), or excellent (ICC >0.90).³⁵

Evaluation of web app validation by nutritionists

Finally, the Healthy Meals web app was validated according to the accuracy of data entry by two different nutritionists who did not participate in the usability and quality assessment since there is no evidence about the minimum sample size required for web app validations. Furthermore, the Healthy Meals web app was validated by nutritionists because they are experts in the field of nutrition, which allows them to evaluate the appropriateness of the web app for providing nutritional content data.³⁶ Nutritionists were asked to enter the same 10 recipe sheets into the web app. From the resulting nutritional assessment, differences in nutrient contents expressed as the mean ± standard deviation (SD) were compared between the two nutritionists to evaluate web app validity.

Reliability between the two nutritionists validating the web app was assessed to measure the replicability of their ratings and was rated as moderate (ICC 0.50 to 0.75), good (ICC 0.75 to 0.90), or excellent (ICC >0.90).³⁵ Furthermore, percent agreement was measured for the resulting MTL labels from nutritional composition analysis to evaluate the nutrient colour rating agreement between nutritionists.³⁹

Statistical analysis

Continuous variables are presented as the mean (SD), while categorical variables are presented as percentages.

Interrater reliability among nutritionists and restaurateurs evaluating and validating the web app was measured as percent agreement for categorical variables³⁹ and according to the ICC for continuous variables, with 95% confidence intervals based on mean rating (k = 2), consistency and a two-way random-effects model.³⁵

Bonferroni tests were conducted to assess the differences among MARS sections and CSUQ factor scores. Student's t-test or Mann–Whitney nonparametric test for independent samples was used for the assessment of data entry accuracy according to the difference among nutrient contents assessed by the two nutritionists in the validation phase. Statistical analysis was performed using SPSS software (version 25), and the significance level was fixed at P ≤ 0.05.

Results

Evaluation of web app usability by nutritionists and restaurateurs

Web app usability was evaluated using the CSUQ by 16 users (n = 10 nutritionists and n = 6 restaurateurs). The mean total score (SD) for the four CSUQ factors assessed by the 16 users was 5.6/7 points (SD 0.9), meaning that users found the web app usable (Table 1).

Table 1.

Healthy Meals web app usability evaluation using the CSUQ.

CSUQ factors	Total^aMean (SD)	Total nutritionistsMean (SD)	Total restaurateursMean (SD)
System usability^{b, g}	5.7 (1.0)	5.8 (0.7)	5.4 (1.4)
Information quality^{c, g}	5.5 (1.2)	5.9 (0.9)	5.0 (1.5)
Interface quality^{d, g}	5.6 (1.2)	5.9 (0.7)	5.1 (1.8)
Overall usability^{e, g} (user satisfaction)	5.7 (1.4)	6.1 (0.6)	5.2 (2.1)
Total CSUQ usability ^f	5.6 (0.9)	5.9 (0.1)	5.2 (0.2)

Total mean (SD) for the 16 users, on 7 maximum points.

Including 6 items scored on 7 maximum points.

Including 3 items scored on 7 maximum points.

Including 1 item scored on 7 maximum points.

Total mean of CSUQ factors comprising system usability, information quality, interface quality and overall usability.

Significant P-value <0.05.

However, reliability among users evaluating web app usability using the CSUQ was moderate (ICC = 0.57; 95% CI, 0.18 to 0.82).

For each of the four CSUQ factors, scores obtained by the 16 users were as follows (Table 1):

For the system usability factor (questions 1–6), the web app obtained a total mean score of 5.7/7 points (SD 1.0), which means that users agreed that the web app system was usable. Regarding each question item of the system usability factor, lower agreement scores were obtained for the agreement of users on the speed with which they could complete the activity (mean 5.5/7 points, SD 1.4) and become comfortable with its use (mean 5.5/7 points, SD 1.3 points).

For the information quality factor, related to web app usability (questions 7–12), users gave a total mean score of 5.5/7 points (SD 1.2). Regarding the single question items of the information quality factor, lower agreement scores were found regarding the provision of clear error messages to solve problems in the web app (mean 5.2/7 points, SD 1.7), effective instructions to perform an activity (mean 5.2/7 points, SD 1.5), and the web app's simple and fast problem solving (mean 5.4/7 points, SD 1.7). Despite these lower results, users’ responses were higher than half of the total points.

For the interface quality factor (questions 13–15), the web app obtained a total mean score of 5.6/7 points (SD 1.2). Regarding the single question items of the interface quality factor, the lowest agreement score was regarding the statement that the web app had all the expected tools (mean 5.0/7 points, SD 1.6).

Finally, regarding the overall usability factor in the context of the user's satisfaction with the web app (question 16), the web app received a total mean score of 5.7/7 points (SD 1.4), which indicates that users agreed that they were satisfied overall with the web app.

Regarding the differences in results between nutritionists and restaurateurs, nutritionists gave higher scores for the usability of the web app; however, the differences were not significant (P-value = 0.23) (Table 1).

On the other hand, differences among the four CSUQ factors for the 16 users were statistically significant (Table 1). Specifically, the system usability and the overall usability factors were significantly higher with respect to the information quality (P-value = 0.01; P-value = 0.02, respectively) and interface quality factors (P-value = 0.02; P-value = 0.00, respectively). Thus, the usability of the information provided by the web app, as well as the availability of the features accessible to the user, should be improved in the Healthy Meals web app to obtain higher CSUQ usability scores for information and interface factors.

Regarding the web app points for improvement and suggestions, two further questions were asked to the 16 users (n = 10 nutritionists and n = 6 restaurateurs) during the usability evaluation step (Table 2).

Table 2.

Web app points for improvements identified by nutritionists and restaurateurs.

	N = 16% (n)
Have you encountered difficulties entering the recipe data? Yes No	81.3 (13) 18.7 (3)
if so, at what point? a) Finding ingredients^a The system does not work Lack of ingredient availability Lack of specific food brands Other problems Not answered	0.0 (0) 83.3 (10) 25 (3) 41.7 (5) 25 (4)
b) Entering food weight^a I do not know the exact weight I know quantities in pieces or spoons or pinches Other problems Not answered	25 (3) 25 (3) 58.3 (7) 25 (4)
c) Selecting type of cooking^a It is not available There are more types of cooking I use a different type of oil or flour or egg Other problems Not answered	44.4 (4) 66.7 (6) 11.1 (1) 33.3 (3) 43.8 (7)
Do you think the web app should be improved? Yes No Not answered	87.5 (14) 0.0 (0) 12.5 (2)
if so, in which aspects? Design Functionality Comprehensibility Data entry Data analysis Data presentation User support	6.3 (1) 0.0 (0) 0.0 (0) 87.5 (14) 0.0 (0) 0.0 (0) 6.3 (1)

Total percentage of respondents is higher than 100% due to the multiple-option responses.

In particular, most of the users (n = 13/16, 81.3%) encountered difficulties entering the recipe data due to (a) a lack of ingredient availability in the food database (n = 10/12, 83.3%), (b) a lack of specific food product brands (n = 3/12, 25%) or (c) other unspecified problems (n = 5/12, 41.7%) (Table 2).

Furthermore, other participants encountered problems related to the entry of ingredients’ weights (n = 7/12, 58.3%) because (a) they did not know the exact weight of the ingredients (n = 3/12, 25%) as in the case of seasonings, spices or salt, for instance, or (b) they could not convert in gram weights expressed in the recipe sheet into household measures such as pieces, spoons or pinches (n = 3/12, 25%) (Table 2).

Difficulties were also experienced during the selection of the cooking methods due to (a) the need to include more types of cooking for the same ingredient (n = 6/9, 66.7%) or (b) because a specific cooking method was not available (n = 4/9, 44.4%) (Table 2).

Thus, all the responding users reported that the web app should be improved (n = 14/14, 87.5%), especially regarding where data entry is concerned (n = 14/16, 87.5%) (Table 2).

Evaluation of web app quality by nutritionists

Web app quality was assessed by the same 10 nutritionists who evaluated web app usability. Nutritionists’ MARS evaluation of the web app objective quality resulted in a total mean score of 4.0/5 points (SD 0.4), which indicates a good evaluation of web app quality (Table 3).

Table 3.

Results from the nutritionists’ MARS evaluation of the Healthy Meals web app quality.

MARS sections	Total^a Mean (SD)
Engagement^b	3.4 (0.7)
Functionality^{c, i}	4.5 (0.6)
Aesthetics^{d, i}	4.2 (0.5)
Information^{e, i}	3.9 (0.6)
Total objective quality^{f, i}	4.0 (0.4)
Subjective quality^{g, i}	3.1 (0.4)
App specific section^h	4.3 (0.7)

Total mean (SD) score of the 10 nutritionists, on 5 maximum points.

Including 5 items scored on 5 maximum points.

Including 4 items scored on 5 maximum points.

Including 3 items scored on 5 maximum points.

Including 7 items scored on 5 maximum points.

Sum of engagement + functionality + aesthetics + information sections, including 19 items scored on 5 maximum points.

Including 4 items scored on 5 maximum points.

Including 6 items scored on 5 maximum points.

ⁱ

Significant P-value <0.05.

Reliability among the ten nutritionists evaluating web app quality through the MARS was excellent (ICC = 0.91; 95% CI, 0.85 to 0.96).

Specifically, regarding the four objective quality sections, good scores (≥4 points) were given to the web app's functionality (mean 4.5/5 points, SD 0.6), aesthetics (mean 4.2/5 points, SD 0.5), and information (mean 3.9/5 points, SD 0.6), while an acceptable evaluation (≥3 points) was given for engagement (mean 3.5/5 points, SD 0.7) (Table 3).

The subjective quality section scored 3.1/5 points (SD 0.4), which represents an acceptable evaluation. On the other hand, the app-specific section related to the web app impact on the user obtained a mean score of 4.3/5 points (SD 0.7), representing a good evaluation (Table 3).

Differences among the MARS sections showed that the total objective quality and the aesthetics sections were significantly higher with respect to the engagement (P-value = 0.04; P-value = 0.04, respectively) and information sections (P-value = 0.02; P-value = 0.02, respectively), meaning that the web app should be improved in regard to the provision of information and the entertainment of the user. Moreover, web app functionality was significantly higher than that in the app-specific section (P-value = 0.02), and the total objective quality was higher than that in the subjective quality sections (P-value = 0.048) (Table 3). Thus, although the overall quality of the web app is good, the positive impact the web app gives to the user, as well as the subjective quality, should be improved, perhaps by enhancing the web app's information and engagement sections.

Evaluation of web app validation by nutritionists

Web app validity was assessed by two different nutritionists. After the entry of 10 recipe sheets by nutritionists, differences between nutritional contents were assessed according to nutrients’ mean values (Table 4).

Table 4.

Results from the nutritionists’ validation of the Healthy Meals web app.

Nutrients	Nutritionist 1 Mean (SD)^a	Nutritionist 2 Mean (SD)^a	P-value
Energy (kcal)	761.1 (564.9)	731.4 (570.7)	0.74^b
Proteins (g)	31.4 (22.1)	30.7 (23.8)	0.68^b
Carbohydrates (g)	50.9 (41.1)	52.7 (41.5)	0.92^c
Sugar (g)	9.5 (15.8)	10.2 (17.1)	0.68^b
Fats (g)	46.7 (56.6)	43.3 (57.0)	0.63^b
Saturated fats (g)	8.2 (6.2)	7.7 (6.0)	0.85^c
Sodium (mg)	706.8 (370.2)	955.1 (485.2)	0.21^c
Fibre (g)	6.1 (8.6)	5.7 (7.0)	1.00^b

Nutrients content differences between the two nutritionists are expressed in mean (SD).

Mann–Whitney test for not normally distributed values.

Student's t-test for normally distributed values.

No statistically significant difference was observed between the two nutritionists’ data (P > 0.05), indicating the validity of the nutritional content assessment of dishes (Table 4). Furthermore, the reliability between the two nutritionists was excellent (ICC = 0.98; 95% CI, 0.97 to 0.99).

From the analysis of the MTL labels, the total percent agreement between the two nutritionists for the eight assessed nutrients was equal to 93.75%. A lower agreement in the MTL labels between the two nutritionists was found for fibre (80%), while for protein, sugar, and saturated fat, agreement reached 90%. MTL labels for energy, carbohydrates, total fat and sodium were equal between the two nutritionists.

Discussion

The usability and quality of the Healthy Meals web app was evaluated by a panel of nutritionists and restaurateurs and then validated by two different nutritionists, confirming it to be a usable, high-quality, valid tool for restaurants to improve dishes’ nutritional content and identify food allergens in their menu.

Evaluating the usability of apps is an essential step to determine the ease by which the app could be used to achieve a specific goal.⁴⁰ In this regard, it is important to evaluate the usability of a web app before it is released to a wider population, and the evaluation should be performed through appropriate and validated usability criteria,⁴¹ such as the CSUQ, which is a highly reliable and valid tool.³⁴

Specifically, in the present usability evaluation, nutritionists and restaurateurs agreed about the usability of the Healthy Meals web app (mean 5.6/7 points, SD 0.9) according to the CSUQ, meaning that a positive evaluation concerning user satisfaction was given about the usability of the four assessed factors, which are the web app system, interface, information and overall usability. Therefore, the Healthy Meals web app was demonstrated to be a simple and user-friendly tool that requires minimal effort from the users to conduct the expected activity.⁴² Furthermore, the good usability of the Healthy Meals web app also represents its acceptability to users, who are more encouraged to use a tool that is easy to understand and use.⁴¹

The moderate reliability observed in the usability evaluation among restaurateurs and nutritionists was due to lower ratings given by restaurateurs, who do not have experience performing nutritional compositional analysis and may need more time than nutritionists to become confident with it. However, as observed by Condrasky et al., in a nutrition knowledge survey, chefs are willing to acquire more competences regarding food composition and recipe modification principles to cook healthier meals for consumers.⁴³ However, to achieve this, chefs need appropriate education through innovative tools that fit with their busy schedule, such as digital tools.⁴³ Thus, the Healthy Meals clearly represents an opportunity for restaurateurs to learn more about food composition and increase cooking competences over time through the easy-to-read MTL label provided by the web app after the food composition analysis. Furthermore, in the present study, the interface and information usability were the CSUQ factors with lower scores. For instance, the provision of clear, easy and concise instructions and information in the web app interface to guide restaurateurs could increase the usability and confidence of restaurateurs with the Healthy Meals web app.

Regarding the web app points for improvement identified through two open-ended questions during the usability evaluation, 81.3% of nutritionists and restaurateurs encountered difficulties during recipe sheet data entry. Some of the problems faced were related to the lack of ingredient availability in the food database. Gaps in food composition databases are due to the constant production of new food products according to the needs of consumers, so a steady effort must be made to update these food composition databases.⁴⁴ However, as the Healthy Meals web app has an open food ingredient database, ingredients could be easily introduced when requested by restaurants. Other encountered difficulties were related to the entering of the ingredients’ weights, with users not being able to estimate portions or convert household measures into grams. Accuracy in the estimation of portion sizes using hand or household measures is difficult and could be subjected to considerable errors.⁴⁵ A cross-sectional survey about adults’ perceptions of household measuring utensils demonstrated that perceived sizes of household measurements could be under- or overestimated by participants, leading to an incorrect evaluation of the ingredient's weight.⁴⁶ In this regard, visual aids such as photographs or atlases for the identification of portion size ranges could be a valuable tool⁴⁷ to implement in the Healthy Meals web app.

The quality evaluation of web apps is another important step to assess to ensure high-quality content and functionalities for users, especially when these are related to health aspects, as in the case of the Healthy Meals web app. Actually, it is highly recommended to involve nutrition experts in the design and development of nutrition-based apps to safeguard users.⁴⁸ For this reason, the quality evaluation was assessed by a panel of nutrition professionals who were able to better verify the content and functionality of the web app and were more competent in regard to nutritional data.

Specifically, the MARS total objective quality received a mean score of 4.0 (SD 0.4) of 5 maximum points, which indicates a good quality evaluation of the Healthy Meals web app by nutritionists. Furthermore, excellent reliability was observed, reinforcing the reproducibility of the results.

In particular, the functionality and aesthetics MARS sections obtained the highest scores in the quality evaluation, meaning that the Healthy Meals web app appeared aesthetically pleasant and practical to nutritionists. This represents a favourable point for the Healthy Meals web app, since users generally tend to prefer apps that are enjoyable and have a positive visual impact⁴⁹ and good functionality to motivate use by people with little familiarity with technology.⁵⁰

Nonetheless, similar to what was observed in the usability evaluation, the engagement and information quality could be improved through the introduction of more ingredients in the food database, as well as through the provision of different cooking methods and aids in the entry of ingredient weights. Therefore, through the provision of more inputs, user engagement would also increase, resulting in long-term use and better experience of the web app.⁵¹

Last, validation of eHealth app contents by health professionals is very important to ensure that the information generated by the app is valid and correct for use by end-users.⁵² For instance, assessing the accuracy in data in the Healthy Meals web app by nutritionists, who are experts in food nutritional composition, helped us to demonstrate the appropriateness and relevance of the web app for wider population use, such as restaurateurs.³⁶ Furthermore, their participation throughout the entire process of the web app evaluation maximized the identification of the web app points for improvement by providing a more professional and qualified viewpoint.⁵³

Specifically, the results from the Healthy Meals web app validation showed that there were similar results between nutritionists entering the data, with 93.75% agreement in the MTL labels, meaning that the nutrient labels obtained from the two nutritionists were exactly the same for most of the assessed micro- and macronutrients. Excellent reliability values between nutritionists were observed, which further supports the replicability of their ratings.

Thus, the Healthy Meals web app represents a valid tool for restaurants to improve their offering of healthier meals, as well as for the identification of food allergens, for which restaurant staff still have many gaps.^54,55

Unlike the previous experimental apps and web apps for nutritional content assessment and food allergen identification, the Healthy Meals web app exhibits several differences, demonstrating it to be an innovative web app. For instance, the Foodtracker app was developed as a system for fast food recommendations to help consumers make their food choices by providing them with the caloric content of restaurant menu offerings in the form of an MTL label.⁵⁶ However, this system only focuses on nudging individual behaviour dietary changes, not considering restaurateurs as the first step for enacting healthy changes, as does the Healthy Meals web app. On the other hand, leading nutrition-tracking apps aimed at assessing consumer nutritional intake underestimate nutrient contents, indicating that commercial apps still lack appropriate evaluation and validation.⁵⁷

Furthermore, the Healthy Meals web app represents an important food allergen database that could support restaurateurs in the preparation of allergen-adapted meals for allergic and intolerant consumers and in the identification of the 14 most common food allergens in menu offerings to communicate to customers. Thus, the Healthy Meals web app also presents several differences and advantages with respect to other existing technologies for food allergen detection. For instance, Lizuka et al.⁵⁸ developed a “Food Menu Selection Support” system, available from mobile and computer devices, to help users make safe food choices according to different dietary constraints (food allergies, dialysis, old age, recovery after hospital stay, etc.). This type of system considers individual personal information, which should be entered to create a user profile and generate a list of tailored food products and menus adapted for the user.⁵⁸ Similarly, the “Personal Mobile Restaurant” system was designed to approach the strict dietary needs of consumers, such as food allergies, health conditions, dislikes, diets, and religious or culture aspects, by identifying suitable restaurants and creating a personalized list of adapted meals to order.⁵⁹ However, these systems are centred only on consumers’ personal use and needs, rather than on both restaurateurs and consumers, and do not integrate food allergen identification with nutritional content assessment, as does the Healthy Meals web app. Furthermore, the Healthy Meals web app has been evaluated and validated by a panel of nutrition professionals.

Limitations

Several limitations arose during the development and validation of the Healthy Meals web app. First, because of the COVID-19 pandemic, the participation of restaurants in the evaluation was limited due to the difficult situation of the catering sector and the lack of restaurant personnel. Second, the lack of similar web apps prevented the comparison of our web app with other existing apps and limited the availability of information about evaluation methods for this kind of web app. Finally, recipe sheets contained rough amounts of ingredients (spoons, glasses, pinches, etc.), so the lack of knowledge of users regarding the exact amounts may have limited the data entry and affected the nutritional analysis.

Suggestions for future improvements

According to the two open-ended questions asked to nutritionists and restaurateurs during the usability evaluation and personal suggestions received by some restaurateurs after participation in the web app evaluation, considerable suggestions for future improvements are provided. These future improvements could increase both usability and quality aspects assessed in the present study by nutritionists and restaurateurs. In particular, the next advances should include the following:

Introduction of more cooking methods (i.e. grilled, roasted, pre-processed, precooked, etc.), with the possibility of inserting more cooking methods for the same ingredient.

Addition of more household units of measurement for the ingredients (i.e. mL, pieces, spoons, pinches, etc.), to help users enter data of ingredient weights. For instance, the implementation of visual aids such as food portion photographs could help restaurateurs enter the correct weights when corresponding grams are unknown.

Implementation of a food database with more product brands and varieties.

Improvement of the overall data entry by improving the ingredient search.

Creation of an observation field for additional comments about the recipe preparation or content.

Addition of new functionalities to help restaurateurs in the design of the menu, in the management of the warehouse, in the preparation of the food shopping list, etc.

Inclusion of alerts to remind restaurateurs to insert any after-preparation dressing.

Conclusion

The Healthy Meals web app designed for professionals related to food, such as restaurateurs, demonstrated to be usable, of good quality and a valid tool for the nutritional assessment and food allergen identification of dishes. Points to improve were identified, while the effectiveness of the app should be tested in scientific trials.

Footnotes

Acknowledgements

This publication is co-funded by the European Regional Development Fund (ERDF) of the European Union within the framework of the ERDF operative programme of Catalonia 2014–2020 aimed at an objective of investment in growth and employment. This publication is framed within the initiative of coordinated PECT TurisTIC en familia, Operation 12: “Healthy Meals.” The NFOC-Salut group is a consolidated research group of Generalitat de Catalunya, Spain (2017 SGR522).

Conflict of interest

The authors declare that there is no conflict of interest.

Contributorship

All the authors were involved in the study design, in the web app design and development, in the performance of the web app evaluation and in data analysis. All the authors participated in the drafting of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Ethical approval

No ethical approval was needed for this type of study.

Funding

This publication has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 713679. This publication has been possible with the support of the Universitat Rovira i Virgili (URV) and Banco Santander.

Guarantor

EL.

ORCID iD

Elisabet Llauradó

References

Elmadfa

Meyer

. Importance of food composition data to nutrition and public health. Eur J Clin Nutr 2010; 64: 4–7.

Mills

Thomas

. Assessing customer expectations of information provided on restaurant menus: a confirmatory factor analysis approach. J Hosp Tour Res 2008; 32: 62–88.

Gallicano

Blomme

Van Rheede

. Consumer response to nutrition information menu labeling in full-service restaurants: making the healthy choice®. Adv Hosp Leis 2012; 8: 109–125.

Roberts

Das

Suen

VMM

, et al. Measured energy content of frequently purchased restaurant meals: multi-country cross sectional study. Br Med J 2018; 363: k4864.

Robinson

Jones

Whitelock

, et al. (Over)eating out at major UK restaurant chains: observational study of energy content of main meals. Br Med J 2018; 363: k4982.

Williamson

. Synthesis report No 2 : the different uses of food composition databases. 2006.

EuroFIR. EuroFIR, European Food Information Resource [Internet]. [cited 2021 Mar 18]. Available from: http://ebasis.eurofir.org/Default.asp.

INFOODS. International Network of Food Data Systems (INFOODS) [Internet]. [cited 2021 Mar 7]. Available from: http://www.fao.org/infoods/infoods/en/.

Church

. The importance of food composition data in recipe analysis. Nutr Bull 2015; 40: 40–44.

10.

Kapsokefalou

Roe

Turrini

, et al. Food composition at present: new challenges. Nutrients 2019; 11: 1714.

11.

Medina

. Spain: Spanish Consumers Grow Interest in Free From Functional Foods [Internet]. Madrid, Spain; 2019. Report No.: SP1941. Available from: https://apps.fas.usda.gov/.

12.

Mills

ENC

Valovirta

Madsen

, et al.

Information provision for allergic consumers - where are we going with food allergen labelling?

Allergy Eur J Allergy Clin Immunol 2004; 59: 1262–1268.

13.

Warren

Jiang

Gupta

. Epidemiology and burden of food allergy. Curr Allergy Asthma Rep 2020; 20: 1–9.

14.

Cucu

Jacxsens

De Meulenaer

. Analysis to support allergen risk management: which way to go? J Agric Food Chem. 2013; 61: 5624–5633.

15.

Radke

Brown

Hoover

, et al. Food allergy knowledge and attitudes of restaurant managers and staff: an EHS-Net study. J Food Prot 2016; 79: 1588–1598.

16.

Sogut

Kavut

Kartal

, et al. Food allergy knowledge and attitude of restaurant personnel in Turkey. Int Forum Allergy Rhinol 2015; 5: 157–161.

17.

Bailey

Albardiaz

Frew

, et al. Restaurant staff’s knowledge of anaphylaxis and dietary care of people with allergies. Clin Exp Allergy 2011; 41: 713–717.

18.

Carter

Pistiner

Wang

, et al. Food allergy in restaurants work group report. J Allergy Clin Immunol Pract 2020; 8: 70–74. DOI: 10.1016/j.jaip.2019.09.013.

19.

US Public Health Service, US Food and Drug Administration (FDA). Food Code [Internet]. College Park, MD; 2017. Available from: https://www.fda.gov/Food/ResourcesForYou/Consumers/ucm239035.htm.

20.

Koroušić Seljak

. Web-based eHealth applications with reference to food composition data. Eur J Clin Nutr. 2010; 64: 121–127.

21.

Ritterband

Thorndike

. Internet interventions or patient education Web sites? J Med Internet Res 2006; 8: e18.

22.

Eysenbach

Powell

. Empirical studies assessing the quality of health information for consumers A systematic review. JAMA 2002; 287: 2691–2700.

23.

Emrich

Lou

, et al. Traffic-light labels could reduce population intakes of calories, total fat, saturated fat, and sodium. PLoS One 2017; 12: e0171188.

24.

>Rivera

Shani

. Attitudes and orientation toward vegetarian food in the restaurant industry: an operator’s perspective. Int J Contemp Hosp Manag 2013; 25: 1049–1065.

25.

Federación Española de Industrias de Alimentación y Bebidas (FIAB); Confederación Europea de Industrias de Alimentación y Bebidas (CIAA). Recomendación CIAA para un esquema común de etiquetado nutricional. In: CIAA Recommendation for a Common Nutrition Labeling Scheme, 2008. Madrid: FIAB.

26.

Food Standards Agency, Department of Health, Scotland Northern Ireland and Wales Governments. Guide to creating a front of pack (FoP) nutrition label for pre-packed products sold through retail outlets [Internet]. London, UK.; 2016. Available from: https://www.food.gov.uk/sites/default/files/media/document/fop-guidance_0.pdf

27.

RedBedca

AESAN

. Base de Datos Española de Composición de Alimentos-BEDCA (“Spanish Database of Food Composition-BEDCA”) [Internet]. 2006 [cited 2019 Jan 16]. Available from: https://www.bedca.net/.

28.

MCGRAW-HILL

. TABLAS DE COMPOSICION DE ALIMENTOS DEL CESNID (“CESNID Food composition tables”). Madrid, Spain: INTERAMERICANA DE ESPAÑA; 2003.

29.

Verdú

. TABLA DE COMPOSICIÓN DE ALIMENTOS (“Food composition table”). 4th ed. University of Granada, editor. Granada, Spain; 2003. 560 p.

30.

Favier

Ireland-Ripert

Toque

, et al. Répertoire général des aliments: table de composition (“general food directory: composition table”). Paris, France: Technique & Documentation, editor, 1995.

31.

Moreiras

Carbajal

Cabrera

, et al. Tablas de composición de alimentos (“food composition tables”). 16th ed. Madrid, Spain: Pirámide, editor, 2013.

32.

Alroobaea

Mayhew

. How many participants are really enough for usability studies? In: Science and Information Conference 2014, editor. Proceedings of 2014 Science and Information Conference, SAI 2014. London, UK; 2014. p. 48–56.

33.

Hedlefs Aguilar

De la Garza González

Sánchez Miranda

, et al. Adaptación al español del cuestionario de usabilidad de sistemas informáticos CSUQ (“spanish language adaptation of the computer systems usability questionnaire CSUQ”). Rev Iberoam las Ciencias Comput e Informática 2016; 4: 84.

34.

Sauro

Lewis

. Quantifying the user experience- practical statistics for user research. In: Morgan Kaufmann Publishers Inc. (ed) Library of congress cataloging-in-publication data, British library cataloguing-in-publication data. 2nd ed. Cambridge, United States: Elsevier, 2016, pp.1–354.

35.

Koo

. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016; 15: 155–163.

36.

Fallaize

Franco

Hwang

, et al. Evaluation of the eNutri automated personalised nutrition advice by users and nutrition professionals in the UK. PLoS One 2019; 14: e0214931.

37.

Martin Payo

Fernandez Álvarez

Blanco Díaz

, et al. Spanish adaptation and validation of the Mobile application rating scale questionnaire. Int J Med Inf 2019; 129: 95–99.

38.

Stoyanov

Hides

Kavanagh

, et al. Mobile App rating scale: a New tool for assessing the quality of health Mobile apps. JMIR mHealth UHealth 2015; 3: e27.

39.

McHugh

. Interrater reliability: the kappa statistic. Biochem Medica [Internet] 2012; 22: 276–282. Available from: https://hrcak.srce.hr/89395.

40.

Swaid

Suid

. Usability heuristics for M-commerce apps. In: Ahram

Falcão

(eds) Advances in usability, user experience and assistive technology [internet]. Basingstoke, UK: Springer International Publishing, 2019, pp.79–88. Available from: http://link.springer.com/10.1007/978-3-319-94947-5.

41.

Abrahão

Cachero

Matera

. Web usability and accessibility. J of Web Eng (JWE) 2008; 7: 257.

42.

Mathews

McShea

Hanley

, et al. Digital health: a path to validation. npj Digit Med 2019; 2: 38.

43.

Condrasky

Hegler

Sharp

, et al. Opinions, knowledge, and current practices of culinary arts instructors and professionals in regards to healthy food techniques. J Culin Sci Technol 2015; 13: 287–302.

44.

Ocké

Westenbrink

van Rossum

CTM

, et al. The essential role of food composition databases for public health nutrition – experiences from the Netherlands. J Food Compos Anal 2021; 101: 103967.

45.

Gibson

Hsu

MSH

Rangan

, et al. Accuracy of hands v. Household measures as portion size estimation AIDS. J Nutr Sci 2016; 5: e29.

46.

Nnyepi

Mthombeni

Thekiso

, et al. Perceptions of household measuring utensils amongst mmopane community in Botswana. African J Food, Agric Nutr Dev 2019; 19: 14261–14276.

47.

Lucas

Niravong

Villeminot

, et al. Estimation of food portion size using photographs: validity, strengths, weaknesses and recommendations. J Hum Nutr Diet 1995; 8: 65–74.

48.

Braz

Lopes

MHBDM

. Evaluation of mobile applications related to nutrition. Public Health Nutr 2019; 22: 1209–1214.

49.

Armstrong

. Which app should I use? Patients and doctors are making increasing use of health apps, but there is little guidance about how well they work. Br J Sports Med. 2017; 51: 1237–1239.

50.

Boulos

MNK

Wheeler

Tavares

, et al. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online [Internet] 2011; 10: 24. Available from: http://www.biomedical-engineering-online.com/content/10/1/24.

51.

Lalmas

O’Brien

Yom-Tov

. Measuring user engagement. Synt Lect Inf Concept Retriev Serv 2014; 6: 1–132.

52.

Becker

Miron-Shatz

Schumacher

, et al. Mhealth 2.0: experiences, possibilities, and perspectives. JMIR mHealth UHealth 2014; 2: e24.

53.

Harricharan

Gemen

Celemín

, et al. Integrating mobile technology with routine dietetic practice: the case of myPace for weight management. Proc Nutr Soc 2015; 74: 125–129.

54.

Loerbroks

Tolksdorf

Wagenmann

, et al. Food allergy knowledge, attitudes and their determinants among restaurant staff: a cross-sectional study. PLoS One 2019; 14(4): e0214625. https://dx.doi.org/ 10.1371/journal.pone.0214625 .

55.

Lefèvre

Abitan

Goetz

, et al. Multicenter survey of restaurant staff’s knowledge of food allergy in eastern France. Allergo J Int 2019; 28: 57–62.

56.

Johnson

Vergara

Doll

, et al. A Mobile Food Recommendation System Based on The Traffic Light Diet [Internet]. 2014. Available from: http://arxiv.org/abs/1409.0296.

57.

Griffiths

Harnack

Pereira

. Assessment of the accuracy of nutrient calculations of five popular nutrition tracking applications. Public Health Nutr 2018; 21: 1495–1502.

58.

Iizuka

Okawada

Matsuyama

, et al. Food menu selection support system: Considering constraint conditions for safe dietary life. In: ACM, editor. CEA 2012 - Proceedings of the 2012 ACM Workshop on Multimedia for Cooking and Eating Activities, Co-located with ACM Multimedia 2012. Nara, Japan.; 2012. p. 43–8.

59.

Daraghmi

Yuan

. PMR: Personalized Mobile Restaurant system. 2013 5th Int Conf Comput Sci Inf Technol CSIT 2013 - Proc. 2013;275–82.