Abstract
Background:
The Anterior Cruciate Ligament–Quality of Life (ACL-QoL) questionnaire is a valid and reliable injury-specific instrument to assess the impact of an ACL rupture on the daily life of a patient.
Purpose:
To translate the ACL-QoL into Dutch (ACL-QoL-NL) and to study its psychometric properties in patients with an ACL rupture.
Study Design:
Cohort study (diagnosis); Level of evidence, 2.
Methods:
The original ACL-QoL questionnaire was translated from English to Dutch according to international guidelines. The study population consisted of 122 patients with a confirmed ACL rupture who completed the ACL-QoL-NL as well as 4 questionnaires: the International Knee Documentation Committee subjective score, Knee injury and Osteoarthritis Outcome Score, 36-item Short Form Health Survey, and Lysholm Knee Score. We determined the internal consistency, construct validity, and prevalence of floor and ceiling effects of the ACL-QoL-NL total score as well as its 5 subscales. To determine test-retest reliability, we analyzed the data of 68 patients who had a stable knee condition and who completed the ACL-QoL-NL a second time after a 2-week interval.
Results:
The confirmatory factor analysis showed that the original structure of the ACL-QoL-NL was confirmed, except for 4 of 31 items. Internal consistency of the total scale and all 5 subscales was good. More than 75% of the predefined hypotheses on the correlations between the ACL-QoL-NL and the 4 questionnaires were met, indicating good construct validity. No significant floor or ceiling effects were observed. Test-retest reliability was good, and no systematic bias between test and retest was found. Standard error of measurement for the total score was 4.8 points out of 100, and the smallest detectable changes at the group and individual levels were 1.6 and 13.2 points, respectively.
Conclusion:
The ACL-QoL-NL questionnaire was successfully translated from English to Dutch and demonstrated good internal consistency, validity, and reliability, with no presence of floor or ceiling effects.
Anterior cruciate ligament (ACL) rupture is a common injury of the knee in athletes, causing instability, reduced level of performance in sports, and even severe impairment in daily functioning. 18,19 Nonoperative treatment options consist of rehabilitation with physical therapy and exercises. A surgical intervention is recommended for competitive elite athletes or when rehabilitation fails with persistent symptoms of instability. 17 To evaluate recovery after ACL rupture, several objective and functional outcome measures are available to measure pain, function, and instability of the knee. However, an important outcome measure that is not always considered in the evaluation of recovery after ACL rupture is patient-perceived quality of life (QoL). Two systematic reviews 9,10 demonstrated that, irrespective of surgical or nonsurgical management, individuals with ACL ruptures are at risk of long-term QoL impairments.
QoL, as defined by the World Health Organization, is “an individual’s perceptions of their position in life taken in the context of the culture and value systems where they live and in relation to their goals, expectations, standards and concerns.” 23 Several aspects of QoL can be assessed, varying from generic health to specific disease. 2 QoL is an individual construct, 5 and individuals with an ACL rupture form a specific population of mainly young and active patients experiencing long-term problems. As such, an injury-specific QoL measure is necessary to determine the impact of an ACL rupture on an individual’s daily life.
The Anterior Cruciate Ligament–Quality of Life (ACL-QoL) questionnaire was developed in English by Mohtadi 14 in 1998 to measure QoL in young and active individuals, and its reliability, validity, and responsiveness were assessed. 13,14 Turkish and Brazilian versions are currently available, and their psychometric properties have been tested, 12,20 although a Dutch version is still lacking. To use the ACL-QoL questionnaire in Dutch-speaking patients, a valid and reliable Dutch version of the questionnaire is needed. Hence, the aim of this study was to translate the ACL-QoL questionnaire into Dutch (ACL-QoL-NL) and study its psychometric properties in individuals after an ACL rupture.
Methods
This study was divided into 2 stages, each with a study protocol. In the first stage, the ACL-QoL questionnaire was translated into Dutch, and in the second stage, the psychometric properties of the translated version were tested. Both study protocols were approved by the ethics committee of our institution.
Translation Procedure
The ACL-QoL questionnaire was translated according to the guidelines for cross-cultural adaptation of self-report measures.
3
The translation procedure consisted of 5 stages: Translation of the ACL-QoL questionnaire into Dutch by 2 independent bilingual translators with Dutch as their native language. Both forward translations were discussed, and 1 forward translation was obtained by merging both (differences were resolved by consensus) Back-translation into English by 2 native English speakers who were fluent in Dutch. An expert committee consisting of 2 human-movement scientists (among whom A.J.dV.), an orthopaedic surgeon (R.W.B.), and a general physician agreed on a prefinal version of the translated ACL-QoL questionnaire. The final draft version was tested in a sample of 13 patients with an ACL rupture who visited the outpatient clinic.
Patients were asked about the comprehension of the questionnaire; after which, the final version of the ACL-QoL-NL questionnaire was realized (Supplemental Material).
Participants
Participants were recruited between January 2018 and March 2020 at the Orthopaedic Department of Martini Hospital, Groningen, and the Centre for Orthopaedic Surgery and Sports Medicine OCON, Hengelo, the Netherlands. Eligible patients had a primary ACL rupture, were older than 18 years, and were on a waiting list for ACL reconstruction. Exclusion criteria were complete and/or surgically treated collateral ligament ruptures, posterior cruciate ligament ruptures, multiligament injuries, revision surgery, and inability to read and understand the Dutch language. Baseline characteristics were recorded for all participants and for all patients who were eligible but did not want to participate (nonrespondents).
Procedures
Participants were contacted by mail. The purpose and procedures of the study were explained in an information letter. To assess the validity of the ACL-QoL-NL questionnaire, participants were asked to fill out 5 questionnaires: ACL-QoL-NL, 36-item Short Form Health Survey (SF36), International Knee Documentation Committee 2000 Subjective Knee Form (IKDC subjective), Lysholm Knee Score, and Knee injury and Osteoarthritis Outcome Score (KOOS). The opt-out procedure was used in this study, where completion of the questionnaires was considered implied consent.
To evaluate test-retest reliability, the ACL-QoL-NL questionnaire was sent to all participants 2 weeks after they filled it out the first time. A global rating of change (GRC) question was added to detect a possible clinical change between these administrations. The GRC question had a 6-point Likert scale ranging from “full recovery” to “worse than ever” and was used to rate the general condition of a patient’s injured knee as compared with 2 weeks before. Hence, patients who reported having “full recovery,” “much improved,” “much worsened,” or “worse than ever” on the GRC were excluded from the reliability analyses, leaving patients whose condition remained the same or had slightly improved/worsened.
Questionnaires
ACL-QoL
The ACL-QoL questionnaire is a patient-based, injury-specific subjective outcome measure to assess QoL in patients with ACL deficiency. This questionnaire consists of 32 items that are divided into 5 subscales: Physical Complaints, Work-Related Concerns, Recreational Activities and Sports, Lifestyle, and Social and Emotional Concerns. A 100-mm visual analog scale is used as the response format, resulting in a score between 0 and 100 for each item. Scores for the subscales as well as total score are calculated by summing the scores of individual items and transforming them to a 100-point scale, with a higher score indicating higher disease-specific QoL. 14
SF36
The SF36 is a generic questionnaire assessing the health status of an individual and health-related QoL. It consists of 36 questions that are divided into 8 subscales: Physical Functioning, Social Functioning, Role Limitations as a Result of Physical Problems (Role Physical), Bodily Pain, General Mental Health, Role Limitations as a Result of Emotional Problems (Role Emotional), Vitality, and General Health. For all subscales, a score between 0 and 100 is calculated, where a higher score indicates better health. The validity and reliability of the Dutch version have been shown to be good. 1
IKDC Subjective
The IKDC subjective questionnaire consists of 18 items to measure the presence of pain, symptoms, function, and sports activity. A sum score was calculated, which was transformed to a 0-100 scale, where a higher score indicates better functioning in daily life and sports activities and fewer symptoms. The IKDC subjective questionnaire has been translated into Dutch and has good validity. 11
Lysholm Knee Score
The Lysholm questionnaire consists of 8 items and is commonly used in patients after ACL ruptures to score knee instability. The total score ranges between 0 and 100, where a higher score indicates fewer symptoms. The psychometric performance of the Dutch version of this scale is acceptable. 8
KOOS
The KOOS questionnaire evaluates symptoms and limitations of the knee. It consists of 42 items that are divided into 5 subscales: Pain, Symptoms, function in Activities of Daily Living, Sport and Recreation functioning, and knee-related QoL. Answers are scored on a 5-point Likert scale. For all subscales, a score between 0 and 100 is calculated, where a higher score indicates fewer knee symptoms. The Dutch version has good validity. 6
Statistical Analysis
For sample size, this study followed the recommendations of the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) checklist. 15 The sample size should consist of at least 100 respondents for studies on validity and at least 50 respondents for test-retest reliability. Descriptive statistics were used to present the patient characteristics and outcomes of the questionnaire. In cases of missing data, patients were contacted by phone to obtain the missing information; if they remained missing, they were handled according to the guidelines of the different questionnaires. Data were analyzed using SPSS Version 20 for Windows (IMB Corporation), and P values <.05 were considered statistically significant.
Internal Consistency
Internal consistency reflects to what extent the different items of a questionnaire scale/subscale are correlated. 22 The predefined subscale structure of the ACL-QoL-NL was assessed using a confirmatory factor analysis with a forced 5-factor structure and direct oblimin rotation. 22 Internal consistency was determined for the total scale and all subscales using Cronbach alpha, which should be between .7 and .95. 22
Validity
The construct validity of the ACL-QoL-NL questionnaire was determined using predefined hypotheses concerning the measured constructs because of the absence of a gold standard. In total, 17 predefined hypotheses were formulated about the expected size of the correlation between the ACL-QoL-NL (total score and subscales) and the SF36 subscales, the IKDC subjective score, the Lysholm score, and the KOOS subscales. The Pearson correlation coefficient (r) was used to determine the actual correlation, which was interpreted as high if >0.6, moderate if 0.3 to 0.6, and low if <0.3. 22 When >75% of the arbitrarily set number of hypotheses is confirmed, construct validity is considered good. 22
Floor and Ceiling Effects
Floor and ceiling effects were evaluated since they may have a negative influence on the validity and reliability of the measurement instrument. Floor and ceiling effects are considered present when ≥15% of participants report the minimum or maximum score. 22 We additionally evaluated the percentage of patients who scored 0 to 5 points and those who scored 95 to 100 points, and we used the same cutoff percentage of 15%.
Reliability
Reliability refers to the ability of an instrument to obtain identical measurements in participants with a stable condition. Test-retest reliability was determined by calculating the intraclass correlation coefficient (ICC; 2-way random, absolute agreement) with 95% CIs between the data from the first and second administrations of the ACL-QoL-NL questionnaire. An ICC >0.7 is generally considered good. 22 The standard error of measurement (SEM) was calculated by dividing the standard deviation of the mean difference of the test and retest measurements (SDdiff) by √2. 7 The smallest detectable change (SDC) on the individual and group levels was calculated using the following formulas: SDCind = 1.96 × √2 × SEM and SDCgroup = SDCind/√n. Finally, the absolute agreement for the total score of the ACL-QoL-NL questionnaire as well as the subscales was tested using a Bland-Altman analysis. 4 The mean difference and 95% CI between the first and second administrations were calculated. When zero lies within the 95% CI of the mean difference, the criterion for absolute agreement is met, and when zero lies outside the 95% CI, it means that there is a measurement bias. Also presented in the Bland-Altman plots were the 95% limits of agreement, calculated using the following formula: mean difference ± 1.96 × SDdiff.
Results
Of 269 eligible patients, 122 (45%) were included for the validity analysis. The test-retest reliability analysis was performed in 68 patients who filled out the ACL-QoL-NL questionnaire a second time after an interval of 2 weeks (Figure 1).
Flowchart of inclusion of participants. ACL-QOL-NL, Anterior Cruciate Ligament–Quality of Life, Dutch version; GRC, global rating of change.
Descriptive Statistics
A comparison of baseline characteristics between the respondents and nonrespondents is presented in Table 1. No differences in baseline characteristics between groups were found. Only the interval between ACL rupture and the date when the questionnaire was sent to the participant was significantly longer in the respondent group (6 vs 4 months).
Patient Characteristics a
a Data are presented as No. (%) or mean ± SD (range) unless noted otherwise. Differences between groups were assessed using an independent-samples t test (age and body mass index), Mann-Whitney U test (interval from ACL rupture to ACL-QoL questionnaire), and chi-square test (sex, affected side, and concomitant injury). Bold P value indicates statistically significant between-group difference (P < .05). ACL, anterior cruciate ligament; ACL-QoL, Anterior Cruciate Ligament–Quality of Life.
b Based on magnetic resonance imaging.
c Median [interquartile range; range].
Internal Consistency
The confirmatory factor analysis largely confirmed the structure of the original subscales. Of 31 items, 4 loaded higher on a different subscale than what was originally constructed (Appendix Table A1): 2 items from the Physical Complaints subscale loaded highest on the Work-Related Concerns subscale; 1 item from the Recreational Activities and Sports subscale loaded on the Lifestyle subscale; and 1 item from the Lifestyle subscale loaded the highest on the Social and Emotional Concerns subscale.
The Cronbach alpha was .92 for the total ACL-QoL-NL score, .82 for Physical Complaints, .80 for Work-Related Concerns, .87 for Recreational Activities and Sports, .79 for Lifestyle, and .83 for Social and Emotional Concerns. These values indicated good internal consistency for the total score as well as for all the individual subscales.
Validity
Of the 17 predefined hypotheses on the measured constructs of the questionnaires and their subscales, 13 (76.5%) were confirmed (Table 2).
The 17 Predefined Hypotheses With Expected and Actual Pearson Correlation Coefficients a
a ACL-QoL-NL, Anterior Cruciate Ligament–Quality of Life, Dutch version; IKDC, International Knee Documentation Committee; KOOS, Knee injury and Osteoarthritis Outcome Score; PF, Physical Functioning subscale; QoL, Quality of Life; SF, Social Functioning subscale; SF36, 36-item Short Form Health Survey. Dashes indicate not applicable.
b Plus sign indicates that the predefined hypothesis was met; minus sign indicates that the predefined hypothesis was rejected.
Floor and Ceiling Effects
No significant prevalence of floor or ceiling effects was found for the ACL-QoL-NL total score and all subscales (Table 3). The Recreational Activities and Sports subscale had the largest number of patients (n = 18; 14.8%) with a score in the lowest 0% to 5%.
ACL-QoL-NL Subscale and Total Scores of All Study Patients (N = 122) a
a Data are presented as mean ± SD or No. (%) of patients. ACL-QoL-NL, Anterior Cruciate Ligament–Quality of Life, Dutch version.
Reliability
Reliability measures of the ACL-QoL-NL questionnaire are shown in Table 4. All ICC values were >0.7, indicating good test-retest reliability. The range for the SEM value was 4.4 to 13.3 points, and ranges for the SDC value were 12.1 to 36.8 points (individual level) and 1.5 to 4.5 points (group level). Bland-Altman analyses showed no systematic bias between first and second measurements (Figure 2).
Test-Retest Reliability of the ACL-QoL-NL Subscales and Total Score (n = 68 Patients) a
a ACL-QoL-NL, Anterior Cruciate Ligament–Quality of Life, Dutch version; ICC, intraclass correlation coefficient; Ind, individual; MD, mean difference; SDC, smallest detectable change; SEM, standard error of measurement.
Bland-Altman graphs show the mean difference (95% CI) between the first and second ACL-QoL-NL test administrations. When zero lies within the 95% CI of the mean difference, the criterion for absolute agreement is met, and when zero lies outside the 95% CI, it means that there is a measurement bias. The 95% limit of agreement (LOA) is also shown. (A) Total score and (B-F) subscales: Physical Complaints, Work-Related Concerns, Recreational Activities and Sports, Lifestyle, and Social and Emotional Concerns. ACL-QoL-NL, Anterior Cruciate Ligament–Quality of Life, Dutch version.
Discussion
This study aimed to determine the psychometric properties of the ACL-QoL-NL in patients with an ACL rupture. The results show that the Dutch version of the ACL-QoL has good internal consistency, validity, and reliability with no floor or ceiling effects.
The present study is the first to our knowledge to assess the factor structure of the ACL-QoL questionnaire, and the confirmatory factor analysis showed that 4 of 31 items do not load according to the original 5-factor structure defined by Mohtadi. 14 The finding that not all items fit in the original structure is not surprising: Mohtadi based the structure of the questionnaire on a literature review, an expert panel, and patient input and did so in an iterative way, but no factor analysis was performed to confirm this. Even though this development process of the ACL-QoL questionnaire seems to be thorough, the COSMIN guidelines now recommend a factor analysis to confirm the structure of the instrument. 22
Internal consistency values of the ACL-QoL-NL total score and subscales were good, with Cronbach alphas of .92 for the total score and .79 to .87 for the subscales. These values are in line with the Turkish and Brazilian versions 12,20 and the English version, even though only the internal consistency of the total score was assessed in the latter. 13
Currently there are no other valid and reliable injury-specific questionnaires available that measure the QoL of patients with an ACL rupture. Therefore, in this study, the validity of the different constructs of the ACL-QoL-NL was assessed by determining the correlation with the SF36, Lysholm Knee Score, IKDC subjective score, and KOOS questionnaire. Out of 17 predefined hypotheses, 13 (>75%) were confirmed, indicating good construct validity. 7 Hypotheses 7 and 10 were borderline rejected with a 0.001 deviation from the predefined hypotheses, which can be considered a negligible—and not clinically relevant—difference. Hypothesis 8, on the correlation between the IKDC subjective score and the ACL-QoL-NL Recreational Activities and Sports subscale, was also rejected. The finding that their correlation was lower than predefined can be retrospectively explained by the fact that the subscale of the ACL-QoL-NL refers to worries and fears during specific activities, while the IKDC subjective score measures ability and physical functioning without considering the mental aspect. Hence, the expected correlation of >0.6 was probably set too high, even though a correlation of 0.67 was found for the Brazilian version. 20
Hypothesis 16, on the relation between the SF36 General Health domain and the ACL-QoL-NL total score (r < 0.3), was rejected. This correlation of 0.33 was higher than expected and higher than that in the Brazilian version (r = 0.24) but still in the same order of magnitude, confirming that the relation between these constructs is relatively weak. Since all correlations between the Physical Complaints subscale and other questionnaires assessing symptoms or pain were high, as expected, it can be concluded that the ACL-QoL-NL subscale measures physical complaints in a valid way. High correlations were found between the Recreational Activities and Lifestyle subscales and their related KOOS subscales, indicating that similar constructs were measured. For the Social and Emotional Concerns subscale as well as the Work-Related Concerns subscale, however, only low to moderate correlations were found with the other questionnaires, indicating that these constructs could not be measured by the other questionnaires in this patient group. This shows the importance of having an injury-specific QoL questionnaire that assesses the physical and mental aspects determining QoL.
No floor or ceiling effects were observed for the total score or any subscale of the ACL-QoL-NL. By contrast, Kinikli et al 12 reported that with the Turkish version of the ACL-QoL, 22.9% of patients had a maximum score on the Work-Related Concerns domain (ie, had no problems in the work domain). The difference with the present study might be explained by the fact that in their study the mean interval between ACL rupture and assessment of the ACL-QoL was relatively long (2.8 years), while in the present study the mean interval was 1.2 years (median, 6 months). Lafave et al 13 in 2017 investigated the total ACL-QoL score at different time points (presurgery and postoperative 6, 12, and 24 months) and noted that the proportion of patients reaching the top 15% of the score increased to 42% at 24 months, demonstrating that time influences the presence of floor and ceiling effects. However, no floor and ceiling effects were reported for the Brazilian version of the ACL-QoL, 20 even though all patients were included postoperatively (range, 3 months–12 years). This indicates that other—unidentified—aspects play a role.
The test-retest reliability of the ACL-QoL-NL was considered good for the total score and all subscales. The Brazilian version of the ACL-QoL had comparable but slightly lower ICC values as compared with the ACL-QoL-NL, except for the Physical Complaints subscale, which had an ICC <0.7. 20 Other versions of the ACL-QoL were not comparable because of the use of a small convenience sample, where a measure to check if the condition changed between assessments was lacking, 13 or because a 10-point check box was used instead of the 100-mm visual analog scale used in the current study. 12
The SEM values of the ACL-QoL-NL total score (4.8 points) and subscales (4.4-13.3 points) were comparable with the Brazilian version (total score, 4.7 points; subscales, 6.4-10.4 points). 20 The same holds for the SDC at the individual level. The total value for the SDC individual for the ACL-QoL-NL was 13.2 points, with a subscale range between 12.1 points for Recreational Activities and Sports and 36.8 points for Work-Related Concerns. To confirm that a real change has occurred on an individual level, the difference between measurements should be higher than the SDC individual value. As these SDC individual values were relatively large, the ACL-QoL-NL is probably less appropriate to measure QoL over time in individual patients. The results of the SDC at the group level—with 1.6 points for the total score and 1.5 to 4.5 points for the subscales—indicate that the ACL-QoL-NL is suitable to use for group comparisons, since relatively low values are needed to detect change. Important to note, however, is that only values higher than the SEM can be distinguished from the measurement error. Furthermore, it should be considered that the SEM and SDC values are distribution-based methods and that no information is obtained about the change that is actually important for the patient. To determine whether the difference between groups is clinically important, the minimally important change (also referred to as the minimal clinically important difference) should be determined as well. 22
Last, this study is the first to assess the absolute agreement of the ACL-QoL questionnaire. No systematic bias was found for the total score or individual subscales, which means that no systematic difference between subsequent measurements can be expected. 4
Limitations
A limitation of this study might be that we included only patients who visited the outpatient clinic and planned to undergo ACL reconstruction surgery. The patients who planned to be treated nonoperatively were, for practical reasons, not included. Nevertheless, our final study population comprised a relative broad group of patients, considering that the median duration between the ACL rupture and the questionnaire was 6 months, with a range of 1 to 144 months. In addition, the majority of patients had ≥1 concomitant injury, which may have caused extra variation in the outcomes. As concomitant injuries are common in ACL ruptures, 16,17,21 one could argue that the outcomes of this study would reflect the actual population of ACL-ruptured individuals.
Conclusion
The ACL-QoL-NL questionnaire was successfully translated from English to Dutch and demonstrated good internal consistency, validity, and reliability with no presence of floor or ceiling effects. This study demonstrated that, to get a complete picture of the status and recovery of these patients, the ACL-QoL-NL can be used in clinical practice to measure the QoL in patients with an ACL rupture in addition to other objective and more functional outcome measures.
Supplemental material for this article is available at http://journals.sagepub.com/doi/suppl/10.1177/23259671221123297.
Supplemental Material
Supplemental Material, sj-pdf-1-ojs-10.1177_23259671221123297 - Translation, Validity, and Reliability of the Dutch Anterior Cruciate Ligament–Quality of Life Questionnaire
Supplemental Material, sj-pdf-1-ojs-10.1177_23259671221123297 for Translation, Validity, and Reliability of the Dutch Anterior Cruciate Ligament–Quality of Life Questionnaire by Astrid J. de Vries, Reinoud W. Brouwer, Rianne Huis in t’ Veld, Wybren A. van der Wal, Inge H.F. Reininga and Roy A.G. Hoogeslag in Orthopaedic Journal of Sports Medicine
Footnotes
Acknowledgment
The authors thank L. de Vries for her practical work on this project.
Final revision submitted January 3, 2022; accepted June 14, 2022.
The authors declared that there are no conflicts of interest in the authorship and publication of this contribution. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from Martini Hospital (Nos. 2017-058 and 2017-072).
APPENDIX
Pattern Matrix Confirmatory Factor Analysis: 5-Factor Structure (Direct Oblimin Rotation) a
| Item | Factor 1 | Factor 2 | Factor 3 | Factor 4 | Factor 5 |
|---|---|---|---|---|---|
| Physical Complaints | |||||
| 1a | –0.822 | ||||
| 1b | –0.752 | ||||
| 2 | 0.590 | –0.339 | |||
| 3 | 0.604 | –0.239 | |||
| 4 | –0.528 | ||||
| Work-Related Concerns | |||||
| 5 | 0.551 | ||||
| 6 | 0.693 | ||||
| 7 | 0.677 | ||||
| 8 | 0.591 | ||||
| Recreational Activities and Sports | |||||
| 9 | 0.572 | ||||
| 10 | 0.470 | ||||
| 11 | 0.621 | ||||
| 12 | 0.568 | ||||
| 13 | 0.691 | ||||
| 14 | 0.529 | ||||
| 15 | 0.715 | ||||
| 16 | 0.576 | ||||
| 17 | 0.773 | ||||
| 18 | 0.703 | ||||
| 19 | 0.513 | ||||
| 20 | 0.146 | –0.682 | |||
| Lifestyle | |||||
| 21 | –0.737 | ||||
| 22 | –0.532 | ||||
| 23 | 0.567 | –0.413 | |||
| 24 | –0.305 | ||||
| 25 | –0.598 | ||||
| 26 | –0.604 | ||||
| Social and Emotional Concerns | |||||
| 27 | 0.662 | ||||
| 28 | 0.828 | ||||
| 29 | 0.635 | ||||
| 30 | 0.402 | ||||
| 31 | 0.490 |
a Gray value indicates that the item loaded higher on a different subscale than what was originally constructed.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
