Abstract
Previous multiple-choice food-based food frequency questionnaires (FFQs) were not validated against weighed dietary records (WDRs) in Iran. This study investigated the validity and reproducibility of a multiple-choice semi-quantitative food frequency questionnaire (SQ-FFQ) in adults living in central Iran. Patients with diabetes and their spouses were asked to complete 3 SQ-FFQs by interview, and nine 3-day WDRs, over 9 months. They provided 2 blood samples to assess serum calcium, magnesium, zinc, and vitamin C levels. The Pearson and intraclass correlation coefficients were calculated to assess reproducibility and validity. The degree of misclassification was explored using a contingency table of quartiles which compare the information between third FFQ and WDRs. The method of triads was incorporated to assess validity coefficients between estimated intakes using third FFQ, WDRs, and biochemical markers and assumed true intakes. A total of 180 participants aged 48.9 ± 8.4 years completed the study. Compared to WDRs, FFQs overestimated all nutrient intakes except for iron. The median intraclass correlation between FFQs was 0.56. The median de-attenuated, age, sex, and education adjusted partial correlation coefficients for validity were 0.17 and 0.26 for FFQ1-WDRs and FFQ3-WDRs, respectively. The FFQ3 validity coefficients for vitamin C, calcium, magnesium, and zinc were 0.13, 0.62, 0.89, and 0.66, respectively, using the triads method. The median exact agreement and complete disagreement between FFQ3 and WDRs were 33% and 6%, respectively. The SQ-FFQ seems to be an acceptable tool to assess the long-term dietary intake for future large-scale studies in this population.
Introduction
Diet plays an important role in the development and control of chronic diseases. 1 -3 Valid and reliable dietary assessment methods developed for each population are needed to find out the dietary determinants of health. 2 Several methods such as 24-hour recall, dietary record (DR), and food frequency questionnaire (FFQ) have been used to assess dietary intake in different studies. 4 All of these methods have some individual limitations in estimating the dietary food intake; therefore, they are selected by researchers to be used based on the aim and the design of each study. 5,6 The semi-quantitative food frequency questionnaire (SQ-FFQ) is a widely used method to assess long-term dietary intake in population-based studies 7 ; because it is inexpensive and easy to complete in large populations. 4 The questionnaires are mostly composed of a food list and a frequency response section. Some SQ-FFQs also ask about the usual portion size of each consumed food item.
Food frequency questionnaires are prone to bias and misclassification because they rely on memory, and miss some food items that are high in the targeted nutrients, frequently consumed in the population, and are differently used by individuals living in the target population. 4 Therefore, FFQs should be validated using the other dietary assessment methods which are not prone to the same sources of bias. 4 Biological markers are likely to be able to improve the estimations of dietary intake assessment because random errors are independent of other dietary assessment tools. 4,8 Nevertheless, the majority of biomarkers are expensive and it is not desirable to replace the other methods of dietary assessment as part of a large epidemiological study. 8,9 Furthermore, the reproducibility of the FFQs should be assessed because they are designed to provide data on long-term intakes. 4 Any changes in the design of FFQs might lead to a change in the performance of the questionnaire. 10 In addition, these questionnaires are culture-specific which means that the dietary culture and foods consumed are highly variable between populations even in the same country. Therefore, the validity and reproducibility of a questionnaire are needed to be measured for any population. 4,6
Several FFQs are developed in Iran. 11 -14 For instance, Tehran Lipid and Glucose Study (TLGS) 13 and Golestan cohort study 12 have used open-ended FFQs which were designed to be used in Tehran and Golestan provinces, respectively. Furthermore, the FFQ used in the TLGS asked about the frequency and portion size of the intake of 168 food items. 13 The participants’ food consumption frequency over the previous year was asked on a daily (eg, bread), weekly (eg, rice, meat), or monthly (eg, fish) basis. Likewise, a 150-item FFQ was designed for the Golestan cohort study and the frequency of consumption was recorded as times per day, week, month, year, and never. For 51 food items, pictures of different portion sizes were used to increase the precisions of estimations. 12 Both FFQs were validated using 24-hour recalls as a reference method to validate the dietary intakes and showed good validity and reproducibility. 12,13
To the best of our knowledge, no study has tried to develop and validate a multiple-choice food-based FFQ in Iran using weighed dietary records (WDRs). People residing in Yazd province, central Iran, are living in arid and semi-arid areas that have different dietary habits and food items. Therefore, we designed a semi-quantitative food-based FFQ with 178 food items to assess the habitual food and nutrient intake of adults living in Yazd to be used in large-scale epidemiologic studies. The current FFQ used a multiple-choice approach; therefore it is easy to perform for participants and interviewers to fill the questionnaire. 4 The present study aimed to assess the validity and reproducibility of the semi-quantitative multiple-choice food frequency questionnaire using WDRs biochemical markers in a sample of adults living in Yazd city, who were attended a clinical trial.
Materials and Methods
Study Design
The present study was conducted in the context of a clinical trial. The study protocol has been described in detail, elsewhere. 15 In brief, a triple-blind randomized 3-way crossover clinical trial which aimed to compare the effect of the replacement of the regular consumed oils with 3 edible oils (canola, sesame, and sesame-canola oils) on cardiovascular risk factors in adults with diabetes and their spouse, who were recruited from the diabetes research center, Yazd, Iran. A total of 102 adults with diabetes (50 males, 52 females) and 101 spouses (50 males, 51 females), aged between 18 and 60 years old entered the original clinical trial. Informed consent was taken from all study participants. The protocol of the parent clinical trial was registered in the Iranian Registry clinical trials (IRCT) on November 14, 2016 (registration ID: IRCT2016091312571N6). The methodology of the current study was also ethically approved by the research ethics committee of Shahid Sadoughi University of Medical Sciences (approval code: IR.SSU.SPH.REC.1396.155).
Each participant entered three 9-week intervention periods and randomly received all intervention oils. The intervention periods were separated by 4 weeks of washout periods. The participants were visited at baseline, in the middle, and at the end of each intervention period. Therefore, each study attendant was visited 9 times for 35 weeks (about 9 months) in the clinical trial between August 2016 and May 2017. The participants were asked to provide 3 weighed dietary food records (2 weeks and 1 weekend day) for each visit; therefore, each participant would provide data on dietary intake for 27 days during the study period. For the current investigation, 3 FFQs separated by 3 months were recruited to address the reproducibility. The first FFQ was filled at the sixth month of the clinical trial (visit 6), the second one was administered at the end of the study (9 months from baseline, visit 9) and the participants were invited to fill the third FFQ 3 months after the end of the clinical trial. The study flow diagram is provided in Figure 1.
The study flow diagram. BV indicates baseline visit; BS, blood sample.
Food Frequency Questionnaire
The semi-quantitative FFQ used for the present study was a modified version of an open-ended 168-item FFQ which was validated and used in the TLGS. 13 In the current version, we added 10 food items that were typically consumed in Yazd. Besides, compared to the TLGS FFQ, the current questionnaire was designed to be a multiple-choice questionnaire. Altogether, 178 food items were included in the questionnaire: breads and grains (n = 23); beans (n = 7); meats, fish, and shellfish (n = 19); milks and dairy products (n = 17); vegetables (n = 26); fruits (n = 40); fats and nuts (n = 13); beverages (n = 5); and snacks and sweets (n = 28). The study participants had to answer 2 questions by the interview regarding each food item: (1) the frequency of consumption and (2) the portion size. The frequency responses for each food item were as follows: never or less than 1 month, 1 to 3 times per month, 1 time per week, 2 to 4 times per week, 5 to 6 times per week, 1 time per day, 2 to 4 times per day, 5 to 7 times per day, 7 to 9 time per day, and 10 times and more per day.
The portion sizes were estimated by natural units (eg, one banana) or standard quantities (eg, one spoon of olive oil). The portion size of all the items which were listed in the questionnaire was asked in separate questions. Furthermore, a separate section was considered to estimate the supplements’ consumption: fish oil (or omega-3), calcium, vitamin D, folic acid, iron, and multivitamin-mineral supplements. The reported frequencies of each item were converted to the number of intakes per day and multiplied by the indicated portion size to convert the reported intakes to gram/day.
Dietary Food Records
Participants were asked to provide 3-day (2 weekdays and 1 weekend day) WDRs for each visit (data on 27 days were recruited for each participant). A digital kitchen scale (model Electronic kitchen scale, SF-400) was provided for the study attendees to help the participants record their dietary intake with maximum accuracy. They were trained how to fill dietary food records by a nutritionist. The participants were asked to write down the consumed foods and beverages with their weight; in addition, they were asked to describe all supplements and medications which were consumed each day. Prior to starting the study, a protocol for coding WDRs was prepared by a research supervisor (ASA). Based on the protocol, a trained nutritionist collected the WDRs and asked the participants to clarify unclear descriptions, errors, and doubtful entries. Trained nutritionists checked all completed WDRs for accuracy. All food items were converted to grams and the energy and nutrients’ intakes were calculated using Nutritionist IV software (version 3.5.2, Axxya Systems) which was modified for Iranian foods. In general, we used the United States Department of Agriculture’s (USDA) food composition table (FCT) 16 to calculate the energy and nutrient intake from either FFQ or WDR for most items except those were available in the Iranian FCT such as types of bread, pepper green, mint, wild plum, and sweet canned cherry. 17
Biochemical Markers
Blood samples from participants were taken after an overnight fast (10-12 hours) and stored at −80 °C in DNase and RNase-free micro tubes until analysis. The average of blood samples recruited at visits 6 and 9, before the time of FFQ 1 and FFQ 2, respectively, were used for the current analysis. Serum calcium, magnesium, and zinc levels were measured by using an auto-analyzer (Alpha-classic, model: AT++) using Pars-azmoon (for serum calcium and magnesium levels) and Biorex Fars (for serum zinc levels) standard kits. The inter- and intra-assay coefficients of variability were 2.4% and 1.2% for serum calcium, 1.4% and 1.1% for serum phosphorus, 1.3% and 0.8% for serum magnesium, and 1.7% and 1.2% for serum zinc assessment, respectively. Moreover, serum vitamin C levels were determined by an Enzyme-linked immunosorbent assay (ELISA) kit (Zellbio standard kit). The inter- and intra-assay for serum vitamin C measurements were 4.7% and 3.5%, respectively.
Sample Size
The sample size for the parent clinical trial was calculated based on a formula suggested for crossover studies 18 [n = [(z 1−α/2+z 1−β)2 · s 2]/2Δ2] assuming type one error of 5% and type 2 error of 10% (power of 90%), and serum glucose as the key variable. Using this formula, a minimum of 34 participants with type 2 diabetes was calculated as the required sample size. Investigators targeted to enter 50 men and 50 women with the eligibility criteria taking the attrition and stratified analyses based on gender into account. As the spouses of the participants attended the original clinical trial, in total, 203 participants were included. 15 Conducting validation studies in the context of clinical trials was done in previous investigations, too. It is mentioned that a reasonable sample size for a validation study thus seems to be about 100 to 200 persons and the sample size needed for a validation study decreases with increasing the number of replicates for the daily intake of participants. As dietary food records were administered for 27 days for each participant in the current study, the sample size seems to be reasonable for this investigation. 4
Statistical Analysis
All nutrients and energy intake values were log-transformed (Log10) prior to analysis to optimize the normality of distribution. Energy adjustment was performed by computing the residual method using the linear regression model in which the nutrient intakes were defined as dependent variables and the energy intakes as an independent variable. 4,19 To compare the absolute nutrient intakes from FFQs and 27-day WDRs, the reported mean values and their corresponding standard deviations were calculated and compared using the generalized linear model repeated measures.
The Pearson correlation coefficient and intraclass correlation coefficient (ICC) were calculated to assess the reproducibility of FFQs for the assessment of dietary nutrient intakes. The validity of FFQs was checked by assessing the Pearson correlation, partial correlation (adjusted for age, sex, and education), and the intraclass correlation between the intake of nutrients assessed by FFQs (the first and the third FFQs) and 27-day WDRs. To correct the coefficients for the within-individual measurement error of WDRs, we multiplied the observed correlation coefficients for the association between the intakes from DRs and the intakes from FFQs by the de-attenuation factor [(1 + (σw2 / σw2) / n)1/2], where σw2 is the within-individual variance, σb2 is the between-individual variance, and n is the number of replicate measurements (here n = 27). 4
The misclassification of questionnaires was assessed by classifying the participant’s nutrient intakes measured by FFQs and 27-day WDRs into quartiles and evaluating the degree of agreement between the third FFQ and WDRs using contingency tables. The Bland-Altman analysis was used to graphically check the agreement between the 2 methods (FFQs and WDRs) for log-transformed macronutrients intakes. This method shows the differences between the 2 methods (FFQs-WDRs) against the mean intake of the 2 measurements ([FFQs + WDRs]/2). 20
The method of triads was used for calcium, magnesium, zinc, and vitamin C to evaluate the validity coefficient between assumed true intake and estimated intakes from the third FFQ, WDR, and biochemical markers (Figure 2). 21 We considered the validity coefficient of FFQ (ρ QI) as the upper limit and the correlation coefficient of FFQ and biomarker (r QB) as the lower limit of the validity coefficient between FFQ and the true intake. 22 We considered the validity coefficients as weak (ρ < 0.2), moderate (0.2 ≤ ρ ≤ 0.6), and high (ρ > 0.6). 21 If we observed the validity coefficient of greater than one for any of the assessment methods, which is known as the Heywood case, 23 we truncated it to one and the validity coefficient of the method was considered as the upper limit and the correlation coefficient of the method and biomarker as the lower limit of the validity coefficient. 22,24 All analyses were performed using a statistical package for social sciences (SPSS), version 21 (SPSS Inc). P values < .05 were considered statistically significant.
Triangular comparison between 3 dietary exposure measurements (triads method). R: reference method (WDR), Q: food frequency questionnaire, B: biomarker, I: true intake, r QR: correlation between food frequency questionnaire and reference method, r BR: correlation between biomarker and reference method, r QB: correlation between food frequency questionnaire and biomarker, ρ QI: validity coefficient for food frequency questionnaire, ρ BI: validity coefficient for biomarker, and ρ RI: validity coefficient for reference method. WDR indicates weighed dietary record.
Results
A total of 180 subjects (89%) aged 48.9 ± 8.4 years after excluding 22 participants who did not administer 3 FFQs were included in the current analysis (50.2% female and 49.8% male). The general characteristics of the study participants are provided in Table 1.
Baseline Characteristics of 180 Participants Who Were Included in the Study.
The mean daily nutrient intake based on nine 3-day WDRs and FFQs are shown in Table 2. Compared to WDRs, the mean daily intakes calculated from FFQs tended to overestimate the energy and nutrients intake except for the iron intake. The FFQs were not significantly different in estimating the dietary nutrients intake except for the dietary protein, fat, cholesterol, niacin, folate, beta-carotene, vitamin E, zinc, copper, selenium, and manganese. The sex-stratified mean daily nutrient intakes estimated by using the three FFQs and WDRs are also described in Supplementary Tables 1 and 2. The analysis based on sex revealed that Food frequency questionnaires provided higher estimations for the intakes of dietary nutrients compared to WDRs in both sexes except for iron.
The Mean Daily Intake of Energy and Nutrients Estimated by Three FFQs and Nine 3-Day Weighed Dietary Records (WDR).*
Abbreviations: SD, standard deviation; RE, retinol equivalent.
* Mean values with dissimilar superscripts were statistically different (P < .05).
Table 3 presents the reproducibility of 3 FFQs (FFQ1 vs FFQ2, FFQ1 vs FFQ3, and FFQ2 vs FFQ3) which is calculated by Pearson correlation and ICC. The median Pearson correlation value was 0.31, 0.44, and 0.38 for FFQ1 versus FFQ2, FFQ1 versus FFQ3, and FFQ2 versus FFQ3, respectively. Moreover, the ICC ranged from 0.43 to 0.73 and was mostly above 0.50 (median: 0.56). The highest and lowest ICC was shown to be for vitamin D and thiamin, respectively (P < .001). The sex-stratified analyses are shown in Supplementary Tables 3 and 4. For men, the median: Pearson correlation value was 0.31, 0.33, 0.40 for FFQ1 versus FFQ2, FFQ1 versus FFQ3, and FFQ2 versus FFQ3, respectively. Also, the ICC ranged from 0.36 for thiamin to 0.76 for vitamin D and were mostly above 0.50 (median: 0.59). For women, the median Pearson correlation value was 0.32 for FFQ1 versus FFQ2, FFQ1 versus FFQ3, and FFQ2 versus FFQ3. Furthermore, the ICC vary from 0.34 for pyridoxine to 0.72 for sucrose and vitamin D and were mostly above 0.50 (median: 0.58).
Pearson Correlation Coefficient Between Energy and Nutrient Intake in Three Food Frequency Questionnaires (FFQs).a
Abbreviations: CC, correlation coefficient; ICC, intraclass correlation.
a Energy and nutrients values were log-transformed (Log10) to optimize normality.
b Energy adjusted.
Correlation coefficients for the validity of FFQs compared to WDRs are displayed in Table 4. The Pearson correlation coefficients between FFQ1-WDR and FFQ3-WDR for nutrients varied from 0.05 for vitamin A (P = .498) to 0.35 for cholesterol (P < .001) and −0.03 for vitamin A (P = .757) to 0.45 for thiamin (P < .001; median: 0.23 and 0.35), respectively. The partial correlations were 0.01 for vitamin A (P = .917) to 0.35 for vitamin D (P < .001) and −0.04 for vitamin A (P = .646) to 0.40 for manganese (P < .001; median: 0.14 and 0.25) between FFQ1-WDR and FFQ3-WDR, respectively. The correlation coefficient values did not change remarkably when de-attenuation factors were considered 0.01 for vitamin A to 0.40 for vitamin B12 and −0.05 for vitamin A to 0.41 for manganese (median: 0.17 and 0.26) for FFQ1-WDR and FFQ3-WDR, respectively. Furthermore, the ICC for FFQ1-WDR association and FFQ3-WDR association ranged from 0.09 for vitamin A (P = .261) to 0.49 for cholesterol (P < .001) and −0.05 for vitamin A (P = .621) to 0.58 for selenium (P < .001) and manganese (P < .001; median: 0.30 and 0.46), respectively. The sex-stratified analyses are also provided in Supplementary Tables 5 and 6.
Correlation Coefficients for the Association Between Energy and Nutrient Intake Measured by the Nine 3-Day Weighed Dietary Food Records (WDRs) and the Food Frequency Questionnaires (FFQs).a
Abbreviations: ICC, intraclass correlation; CC, correlation coefficient.
a To correct the coefficients for the within-individual measurement error of WDRs, we multiplied the observed correlation coefficients for the association between the intakes from dietary records and the intakes from FFQs by the de-attenuation factor.
b Adjusted for age, sex, and education level.
When considering if the methods agreed for individuals using Bland-Altman method, the differences in log-transformed nutrient intake between the FFQs and the 27-day food records were plotted against the log-transformed mean nutrient intakes of the two methods for energy, protein, carbohydrates, and fats (Bland-Altman plots; Figure 3). The points are scattered above zero in most plots, suggesting that the FFQ provides a higher intake compared with the food record. In addition, there was a trend of decreasing accuracy with increasing protein and fat intake, as the scatter plots show overdispersion at higher intakes.
Bland-Altman plots representing the relative validity of the semi-quantitative food frequency questionnaires (FFQs) for estimating the daily intake of energy, protein, carbohydrate, and fat intake. For each participant, the difference in intake between the log-transformed average of the 3 FFQs and the log-transformed mean of the 27-day weighted dietary records (WDRs) is plotted against the mean intake from the 2 methods for: (A) energy; (B) protein; (C) carbohydrate; and (D) fat. Horizontal lines represent the mean difference and the 95% limits of agreement.
The contingency tables in which the quartiles of the dietary intakes of nutrients assessed using WDRs and the third FFQ are simultaneously provided against each other, are described in Table 5. The analyses revealed that 21.3% to 42.1% of individuals were classified in the same quartiles for potassium and selenium, respectively (median: 32.3%); and 34.6% to 48.2% for α-tocopherol and fat intake were categorized in adjacent quartiles, respectively (median: 42%). Except for vitamin A (11.9%), β-carotene (13.4%), α-tocopherol (12.2%), and iron (10.4%) the extreme misclassification was lower than 10% for other nutrients (median: 6%).
The correlation coefficients obtained from the triads method, calculated using correlation coefficients between FFQ, WDRs, and the biochemical measurements are demonstrated in Table 6. The validity coefficients between different measurement methods and the calculated true intake were considered as weak for FFQ (ρ QI: 0.13) and biochemical measurement (ρ BI: 0.16) and high for WDRs (ρ RI: 0.9) for vitamin C. The validity coefficients for calcium were considered as high for the questionnaire (ρ QI: 0.62), and biochemical measurement (ρ RI: 0.62), and moderate for WDRs (ρ BI: 0.21). The validity coefficients for magnesium were evaluated to be high for the questionnaire (ρ QI: 0.89) and moderate for biochemical assessment (ρ RI: 0.31) and WDRs (ρ BI: 0.22). Also, for zinc, the validity coefficients were considered as moderate for FFQ (ρ QI: 0.56) and WDRs (ρ BI: 0.32), and high for biochemical assessment (ρ RI: 0.66). The validity coefficient for FFQ and the correlation between questionnaire and biomarker were considered as the upper and the lower limit of the validity coefficient between FFQ and the true intakes, respectively. Therefore, the lower and upper limits ranged from 0.02 to 0.13, 0.13 to 0.62, 0.20 to 0.89, and 0.21 to 0.56 for vitamin C, calcium, magnesium, and zinc, respectively.
Agreement Proportion in Quartile Distribution of Energy and Nutrients Intake Between the Third Food Frequency Questionnaire and Nine 3-Day Weighed Dietary Records.
Validity Coefficients Between the Third FFQ, Nine 3-day WDRs, and Biomarkers (Vitamin C, Calcium, Magnesium, and Zinc) as Calculated by the Triads Method.
Abbreviations: FFQ, food frequency questionnaires; WDR, weighed dietary records.
a All the values > 1.0 were truncated as this is the highest possible value, ρ QI: validity coefficient for the food frequency questionnaire, ρ BI: validity coefficient for biomarkers, and ρ RI: validity coefficient for WDRs.
b The lower limit for the FFQ and the biomarker is the correlation between the FFQ and biomarker, and the lower limit for the WDR is the correlation between the biomarker and the WDR, and the lower limit for the biomarker is the correlation between the FFQ and the WDR, and the upper limit is calculated by the method of triads.
Discussion
In the present study, we examined the validity and reproducibility of a 178-item multiple-choice SQ-FFQ which was assessed in a long-term clinical trial. The present results demonstrated a reasonable relative validity concerning WDRs for energy and all nutrients, except for vitamin A and β-carotene (median 0.46). The agreement between these 2 methods was reasonably acceptable (median 76.2%) and the median correlation between FFQs was 0.56 for all nutrient intakes. The present study tried to include a reasonable number of participants. Furthermore, to reduce the random error due to within-individual variation, both energy-adjusted and de-attenuated correlation coefficients were calculated. In addition, 2 blood samples were collected with 3-months intervals to reduce the influence of measurement errors.
It is proposed that measuring the dietary intakes using multiple DRs that are not dependent on memory and has a great specificity in describing foods is a suitable choice to be used as a reference method in validation studies. 4,25 Biochemical markers are also used in epidemiological studies to measure the participants’ status regarding specific nutrients or dietary compounds. 26,27 Previous studies indicate high correlations between dietary intake and some biochemical markers. 28,29 It should be noted that disease and homeostatic regulations might affect biomarkers’ status; furthermore, biomarkers should be assessed several times to show the long-term dietary intakes. These problems might reduce the applicability of biochemical markers to be used as the sole indicator of dietary intakes. 29 It is suggested that validation studies would provide a better insight if they compare FFQs with both DRs and biomarkers. 21 Therefore, we used 27-days WDRs which have the least correlated error, 4 as a reference to compare the energy and nutrients intakes from the questionnaire and biochemical markers.
We observed a general overestimation of nutrient intake using FFQs in comparison with WDRs. It is probably due to the seasonal availability of food items like fruits and vegetables, the misconception of portion size, and a long list of food items. In line with our results, other validation studies also reported that the FFQs, as compared with food record or 24-hour recall, overestimate the nutrient and energy intake. 30 -32 Likewise, Considering that types of bread and rice are stapled foods, the overestimation of carbohydrates intake was found in another validation study in Iran. 13 It is proposed that compared with reference methods, FFQs estimate higher intakes for most of the nutrients particularly when FFQ exceeds 100 food items. 33,34 We also observed the mean daily intake of nutrients is higher in men compared with women (Supplementary Tables 1 and 2). Sex differences in reporting energy intake exist and women were more likely to underreport energy intake. 35 Furthermore, according to sex differences in the food portion size, the sex-specific typical portion weights are recommended to be used instead of standard portion size. 36,37
The range of reproducibility of our questionnaire was 0.43 for thiamin to 0.73 for vitamin D for adjusted data which is comparable to other validation studies. 7,12,13,30,31 According to the reports of a comprehensive review, the time interval in the validation studies varied from 2 hours to 15 years. 38 We chose 3-month intervals between the FFQs and tried to administer them at the same time as blood sample collection to diminish the difficulties for participants. The participants were asked not to change their diet during the study period.
Although the mean daily nutrient intake estimates between FFQs were not significantly different for the majority of the nutrients and energy intakes, the third FFQ showed a better correlation with WDRs, perhaps because of the learning bias that can result from participants learned how to answer the questions in the same way as previous questionnaires or WDRs, or change in participant’s diet. 39 Moreover, FFQ3, administered at the end of the study could comprise all WDRs in the period of the study which might explain the better correlation. In addition, we observed the higher median ICC in men between nutrients assessed by FFQs (0.59 for men and 0.58 for females; Supplementary Table 2) or between FFQs and WDRs (0.27 for men and 0.24 for women; Supplementary Table 3), which is in line with other reports. 40,41 As men tend to be unconcerned about their daily diets, it might have been easier for men to complete the FFQ, which requires simplified dietary habits. 41
We expected that the random error correction for within-individual variation increases the correlation values. However, similar to the finding from other studies, the de-attenuation correlations were not substantially different from noncorrected estimates. 13,30 A large number of DRs (27 days) or the low within-individual variation compared to between-individual variation might explain this similarity. 30
Energy adjustment appears to improve correlation coefficients and diminish the measurement errors in the FFQ instrument. 4 However, along with the finding of other studies, 13,42,43 using energy adjustment in our study, the median correlation coefficient of nutrients tends to lower the correlation values. It seems that the low between-individual variation in nutrients’ intakes measured by WDRs has led to lower correlation coefficients after adjustment. 44
The FFQs are mainly used to rank individuals based on their dietary intake and this is important in obtaining correct risk estimates of diseases. 4,5,45 The present study demonstrated that about 33% of participants were classified in the same quartiles using FFQs and WDRs. Furthermore, above 70% of participants were classified to the same or adjacent quartiles which are in agreement with other validation studies. Furthermore, the present study demonstrated that the proportion of complete disagreement was in the range of 3% to 13.4% (median 6%). These results were in line with other studies that used quartiles to classify their participants and were conducted in Asian adults. 31,46
As biomarkers represent the quantitative measurements and not rely on subjects’ memory which is the main source of bias in dietary assessment methods, 21 we also used the serum biomarkers in our validation study. Using the method of triads, the correlation between estimated nutrient intakes using third FFQ and WDRs and measured biomarkers was calculated. Although the validity coefficients of a nutrient are not common to compare between studies because of differences in sample size, duration of studies, the number of food items, food consumption which is culture-specific, and intrinsic variability of biomarkers (bioavailability and metabolism of nutrients), 4,47 the FFQ validity coefficients for all biomarkers except for vitamin C (0.13 for vitamin C, 0.62 for calcium, 0.89 for magnesium, and 0.56 for zinc) were considered as moderate and high which is similar to findings from Mc Naughten et al (0.50, 0.63, 0.45, 0.62), 22 and Andersen et al (0.58, 0.51). 48 In addition, Mirmiran et al 13 found that the range of validity coefficient (ρ QI) was 0.21 to 0.95 (TLGS) which is in line with our results (0.02-0.89).
The realistic correlation coefficients of validation studies tend to be in a range of 0.5 to 0.7. 4 In the TLGS, the mean Pearson correlation coefficient and the mean intraclass correlation coefficient between twelve 24-hour dietary recalls and FFQ for men were 0.53 and 0.59, and for women were 0.39 and 0.60 in energy-adjusted values. 13 In the Golestan cohort study, the correlations coefficient between twelve 24-hour dietary recalls and the mean of four FFQs ranged from 0.49 to 0.82 and the intraclass correlations were between four FFQs varied from 0.66 to 0.89. 12 The validity correlation coefficients were reported to be lower in the present study compared to the previous Iranian studies. We observed that the median Pearson correlation coefficient and median intraclass correlation coefficient between 27-day WDRs and SQ-FFQ were 0.35 and 0.46. It should be noted that the previous investigations had used 24-hour dietary recalls for examining the validity which both rely on memory and this might increase the correlation coefficients by error. 4 This is while our study used 27-day WDRs for validity assessment in the Iranian population for the first time which is not the same in the sources of bias. 4 The range of correlation coefficients in our study was similar to studies previously conducted in Asia 31,49 -53 which compared FFQs with DRs. They found that the range of correlation coefficients of nutrient intakes between FFQs and WDRS was 0.06 to 0.81 and the median ranged between 0.3 and 0.5. It should be noted that serving sizes and foods consumed in Asian regions are different; furthermore, meals are served as family-style and the family members share their foods. Thus, it might lead to a low perception of portion size when reporting their dietary intake using FFQs. 31
The present study has some limitations that should be considered. First, the same portion size was used for both sexes which may result in substantial errors in the estimation of nutrient intakes. Second, as no complete Iranian FCT exists, we used the USDA FCT to calculate the energy and nutrient intakes for the majority of foods. This point might not affect the correlation coefficients and the assessment of misclassifications, however, might lead to biased absolute intakes. Furthermore, the same FCT was used to calculate the dietary intakes reported using FFQs and WDRs. This might lead to higher correlation coefficients. Moreover, our study was conducted on patients with diabetes and their spouses; therefore, the generalization of our findings to the general population should be done with caution. As the present study was conducted in the context of a clinical trial aimed to examine the effect of different plant oils on cardio-metabolic outcomes, the reproducibility and validity of FFQs might be prone to bias for dietary fatty acids. Therefore, we removed the validity and reproducibility statistics for different dietary fatty acids.
Conclusion
In summary, this study found that the present 178 item SQ-FFQ has overall acceptable levels of validity for assessing the dietary nutrients intake except for Vitamin A and β-carotene. Furthermore, the level of reproducibility for all nutrients was acceptable. Thus, the SQ-FFQ used in this study seems to be a useful instrument to measure the dietary nutrients in epidemiological studies conducted in Yazd province, central Iran.
Supplemental Material
Supplemental Material, sj-pdf-1-fnb-10.1177_03795721221078353 - Validity and Reproducibility of a Semiquantitative Multiple-Choice Food Frequency Questionnaire in Iranian Adults
Supplemental Material, sj-pdf-1-fnb-10.1177_03795721221078353 for Validity and Reproducibility of a Semiquantitative Multiple-Choice Food Frequency Questionnaire in Iranian Adults by Alireza Zimorovat, Fatemeh Moghtaderi, Mojgan Amiri, Hamidreza Raeisi-Dehkordi, Matin Mohyadini, Mohammad Mohammadi, Sadegh Zarei, Elham Karimi-Nazari, Masoud Mirzaei, Azadeh Nadjarzadeh and Amin Salehi-Abargouei in Food and Nutrition Bulletin
Footnotes
Authors’ Note
The data of the present study will be available from the corresponding author. The data used for the current study are already published in individual papers. The study participants were given verbal and written information about the study and signed informed consent before participation. The present study was conducted in accordance with the declaration of Helsinki and the methodology of the current study was ethically approved by the research ethics committee of Shahid Sadoughi University of Medical Sciences (approval code: IR.SSU.SPH.REC.1396.155). Consent for publication: No individual detail is presented in this manuscript.
Acknowledgments
The authors wish to thank all participants who voluntarily participated in the study. The authors also appreciate Shahid Sadoughi University of Medical Sciences. Furthermore, we are grateful to the research council of Nutrition and Food Security Research Center for their scientific support. The authors express appreciation for the executive collaboration of the diabetes research center of Shahid Sadoughi University of Medical Sciences, Yazd.
Authors’ Contribution
ASA contributed to the study concept and supervision. ASA and AZ designed the study protocol. The recruitment of the study participants was carried out by MA and FM. AZ, FM, MA, and HRD had a role in data collecting. SZ, EKN, and AZ had responsibility for laboratory analyses. The data entry was carried out by AZ, FM, MM, MA, HRD, and MM. ASA provided counseling for statistical analysis. ASA, AN, and MM played a role in counseling throughout the study. ASA and AZ wrote the manuscript. All authors read and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The study was funded by Shahid Sadoughi University of Medical Sciences, Yazd, Iran.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
