Abstract
BACKGROUND:
The activities and participation component of the International Classification of Functioning, Disability and Health (ICF) has gained increasing recognition in rehabilitation field. The vestibular activities and participation measure (VAP) was the first instrument using the ICF to evaluate activities and participation outcomes for vestibular research and clinical practice.
OBJECTIVES:
This study aimed to cross-culturally adapt the VAP into Chinese language and to examine the psychometric properties of the Chinese version of VAP (VAP-C).
METHODS:
A standard “forward-backward” translation procedure was followed and the results were presented to the panel of experts and consensus was sought. Internal consistency was assessed using Cronbach’s alpha and test-retest reliability examined by intraclass correlation coefficient (ICC). Convergent validity was determined by calculating the Pearson’s correlation coefficient between the VAP-C and the Visual Analog Scale (VAS) and the Dizziness Handicap Inventory (DHI). Discriminative validity for test subjects versus control subjects was evaluated by Receiver Operating Characteristic curve (ROC) analysis.
RESULTS:
Minor revisions were made during translation to ensure semantic equivalence and to suit Chinese culture. A total of 121 patients complaining of dizziness or vertigo due to peripheral vestibular disorders (test subjects) and 41 control subjects were enrolled. Cronbach’s alpha was 0.94 for the VAP-C total scale and greater than 0.80 for the two subscales. ICC for the VAP-C total scale was 0.78, and for subscale one was 0.77 and subscale two 0.76. The VAP-C had strong correlations (r = 0.85–0.88) with the VAS and fair to moderate correlations (r = 0.23–0.53) with DHI. Through ROC analysis, we found the VAP-C discriminated significantly between patients and control subjects. Based on the estimate of Youden J, the optimal cut-off values for VAP-C total and the two subscales were established.
CONCLUSIONS:
The VAP-C shows evidence of reliability and validity, which can be applied for evaluating the impact of the vestibular disorder on patients’ activities and participation in Chinese language populations.
Introduction
Among the components of the International Classification of Functioning, Disability and Health (ICF) from the World Health Organization, the Activities and Participation has gained increasing recognition. Activities and participation outcome measures developed based on the clearly defined concepts included in the ICF would help professionals in rehabilitation field to better understand the patients’ outcome [3, 10]. Vestibular disease is a common condition which can profoundly interfere with individuals’ physical and psychological functioning and overall quality of life [26, 27]. Patients suffering from vestibular disorders tend to avoid a wide range of activities, limit their movement and avoid specific circumstances because of fear of falling or provoking symptoms [5, 15]. In order to identify functioning and disability in this group of patients, a battery of valid and reliable measures have been developed; however, none of those specifically focused on activities and participation aspect [1]. To date, the full extent and specific nature of limitations and restrictions in activities and participation among patients with vestibular disorders remain unknown. One of the reasons might be the lack of validated measurement.
To bridge the existing gap, Alghwiri et al. developed the Vestibular Activities and Participation Measure (VAP) based on the ICF framework [2]. According to the ICF, activity is defined as “execution of a task or action by an individual” and participation is defined as “involvement in life situations”. The VAP was the first instrument using the ICF to introduce activities and participation outcomes for vestibular research and clinical practice. Utilizing expert consensus and an iterative Delphi process, the VAP was established and then validated in an ambulatory care setting [2]. The original VAP consists of 34 items, evaluating the effect of dizziness and/or balance problems on the ability to perform activity and participation tasks, with a 5-point ordinal response indicating the level of difficulty: 0 indicating none difficulty and 4 representing unable to do.
Acknowledging that 34 items might be too comprehensive and less amenable for clinical routine, recently, a shortened version of VAP with 12 items has been developed and culturally validated among four countries including Germany, United States, Jordan, and Saudi Arabia [16]. Based on the results of Rasch analysis, two scales were included in the shortened version of VAP, and the ordinal scoring was transformed to an interval-scaled metric score, making the scoring of VAP available for parametric analysis [16]. Sound psychometric properties for both the original VAP and the shortened version have been established [1, 16]. Owing to its shortness and cross-cultural validity, the shortened form of VAP is considered as a suitable measurement which can be used for investigating the impact of vertigo and dizziness on activity limitation and participation restrictions.
As yet, no measurements in Chinese language that quantifies activities and participation exist. Therefore, the aims of this study were (1) to translate and cross-culturally adapt the shortened VAP into Chinese (VAP-C) and (2) to evaluate the psychometric properties of the VAP-C including the: internal consistency, test-retest reliability, convergent validity, and the discriminant validity. We hypothesized that translation and validation of the VAP into Chinese would provide a measure instruments with sound psychometric properties.
Methods
This study was nested in a large randomized controlled trial aimed to explore effective approach for managing dizziness (China clinical trial center registry identifier, ChiCTR1800017328), in which the change of VAP score pre and post intervention serves as the primary outcome. The study protocol was approved by the Institutional Review Board. All participants provided a written informed consent prior to participation.
The study was conducted in two phases: Phase 1 involved translation and cross-cultural adaptation of the VAP into Chinese, including the pre-testing of the translated Chinese versions; and Phase 2 – investigation of the psychometric properties of VAP-C.
Translation and cross-cultural adaptation
Permission to translate the VAP was granted by the developer (Dr. Susan Whitney). Translation of the VAP into Chinese followed the standard guidelines for translation and cross-cultural adaptation of patient-reported outcome measures [4, 14]. The translation process described as following. First, forward translations were carried out by two native Chinese speakers (one certified physiotherapist, one English teacher with no medical background), they independently translated the original English versions of the VAP to Chinese, resulting in 2 versions: T1 and T2. Subsequently, a synthesis (T3) of these two versions was created following discussion and consensus among the two translators and the principal investigator. This was followed by a back translation of T3 into English by two independent bilingual persons who were native English speakers and raised in Chinese speaking families. Both of whom were blind to the original VAP, either of full version or short form, thus resulting in T4 and T5. Inconsistencies between T4 and T5 were discussed among the two back translators and the principal investigator, through which a single synthesized version (T6) was produced.
After that, an expert panel, consisting of all the translators, the authors, one methodologist experienced in clinical research, was convened to resolve discrepancies in the translations that did not reflect the original English version. Questionable words or phrases in the Chinese version were reworded with alternative expression which the panel considered to be “natural” and maintained the meaning of the English version instead of a direct literal translation. During this procedure, one change was made regarding item #11Using transportation (traveling using private or public transportation being a passenger). Because using transportation sounds ‘funny’ in Chinese language context, the panelists proposed to delete using transportation from the item. Another change was made for item #12 Operating a vehicle: driving a car or riding a bicycle, since operating a vehicle seems strange for most Chinese people; moreover, driving a car or riding a bicycle can sufficiently capture the essence of this item. Therefore, “operating a vehicle” was removed from the item. Similarly, the literal translation of item # 6 sports (engaging in competitive and formal or informal organized games, performed alone or in a group) turned to be inappropriate. We used the word “organized” to substitute “formal”, since “formal” usually refers to official issue in Chinese culture, which is obviously inappropriate and inaccurate in this scenario. For the item # 8 Climbing (up and down stairs, elevator, escalator), the panelists felt confused about the expression of climbing elevator/escalator, since elevator/escalator is electricity-driven, climbing these facilities is unlikely to happen in real life. Therefore, we revised the item as Climbing (up and down stairs, steep slope). After these modifications, a consensus was reached among the panelists. A pre-final Chinese version of VAP was created and was then tested among 30 patients presenting with dizziness due to vestibular disorders; patients found no difficulty in understanding and replying to the questions. No further rewording was needed following that pretesting. Table 1 presents the final VAP Chinese version (VAP-C).
Chinese version of VAP
Chinese version of VAP
Participants
Participants were recruited from the ENT Department of Eye & Ear Nose Throat Hospital of Fudan University during a 1-year period between November 2017 and September 2018. Eligible participants included both outpatients and inpatients with vertigo and/or dizziness persisting for longer than one week and have a physician-confirmed vestibular disorder diagnosis. Exclusion criteria were vertigo and/or dizziness due to cardio-vascular disease or neurological origin. Participants were also excluded if they were unable to read and understand Mandarin, and if they were less than 18 and or over 80 years of age.
A group of participants free from any vestibular signs and symptoms, mainly family members of patients and friends of health care professionals, were included after giving written consent. They served as a control group to examine the sensitivity and specificity of VAP-C in distinguishing activities and participation differences among patients with vestibular disorders and control subjects.
Procedure
Participants completed three self-administered questionnaires including Chinese version of VAP (VAP-C), Dizziness Handicap Inventory (DHI), and a Visual Analog Scale (VAS). Participants had the options of answering the questionnaires either on paper copy or on a web-based survey via smartphone which contains the same content. Medical variables were retrieved by a research assistant using electronic medical history system in the study setting. Participants who consented to the test-retest study were instructed to complete a second VAP-C on the same day with a 4-hour interval due to the episode nature of peripheral vestibular disorders. As the participants may have remembered their previous answers and would try to answer accordingly, we change the order of the VAP-C in the retest randomly so as to minimize recall bias.
Measures
2.2.3.1 Dizziness Handicap Inventory The DHI has been widely used as a reliable tool for evaluating the self-perceived handicap effects imposed by vestibular disease [11, 17]. It is composed of 25 questions covering three domains: physical, emotional, and functional. Total scores of DHI range from 0 to 100 with higher scores indicating greater perception of handicap because of dizziness. The DHI has been translated and validated in Chinese samples and has been widely used in vestibular research in China [7]. Thus, this instrument was chosen to investigate the convergent validity of the VAP-C even though these two scales are not based on same theoretical models.
2.2.3.2 Visual analog scale A 10-cm visual analog scale (VAS) was used to assess the severity of dizziness/vertigo symptoms. A vertically oriented 10 cm line was drawn on a blanket paper. Patients were instructed to place a mark on the 10-cm vertical line according to their dizziness/vertigo severity, with “no symptoms” corresponds to the bottom of the line and “the worst symptoms” corresponds to the top of the line. This measurement has been found psychometrically sound in evaluating vertigo and dizziness in patients with vestibular disorder [6, 23]. Given its user-friendliness, rapidity, and easy understandability, we chose VAS as another measure for evaluating the convergent validity of the VAP-C.
Statistical analysis
Statistical analyses were performed using IBM SPSS Statistics for Windows (SPSS V.22.0). Floor or ceiling effects of VAP-C were considered as present if more than 15% of the respondents achieve the lowest or highest possible score [17, 25]. Internal consistency was estimated using Cronbach’s alpha and test-retest reliability examined by intraclass correlation coefficients (ICC). Convergent validity was assessed by exploring correlations of VAP-C with DHI and VAS. Receiver operating characteristic curve (ROC) analysis was performed to test the discriminant validity of VAP-C. The area under the curve (AUC) generated by plotting sensitivity against the rate of false-positive (1- sensitivity) for discriminating of activities and participation data among patients with vestibular problems and control subjects was tested against a null hypothesis of AUC equal to 0.50, which refers no discriminative value [24]. The optimal cutoff score was determined based on Youden’s J (J = sensitivity +specificity-1) [9]. The sample size was based on rule of thumb noting that at least 1zarticipants for each scale item, i.e., an ideal ratio of respondents to items is 10:1 [19]. Thus, the minimal sample is to be 120 given that there are12 items in VAP questionnaire.
Results
One hundred thirty-four patients were approached, 11 showed no interest and four gave incomplete demographical data, leaving 121 patients in final analysis. Of those, 51 participated the test-retest study. The characteristics of total sample and retest sample are shown in Table 2. Fifteen participants (12%) did not answer the driving question. Eighteen participants (15%) did not answer the sport question. Those missing items were considered as missing completely at random and were deleted listwise in the data analysis.
Demographic and clinical characteristics of the study sample
Demographic and clinical characteristics of the study sample
A total of 41 people with no vestibular or balance problems represented the control group; the mean age was 48.85 years (SD = 14.61), 19 (46.3%) were male. Comparisons between test group and control group revealed no statistical significance regarding age and sex.
Table 3 gives distribution profile of the VAP-C in the total sample. Normality of the VAP-C total and subscale scores were tested using Shapiro-Wilk test, as well as visual inspections of the histogram, indicating that the VAP-C total and subscale scores were normally distributed (Shapiro-Wilk test, p > 0.05) and there was no evidence of outliers. No floor or ceiling effects were observed as only two of the respondents achieved the lowest score (that was zero in total score) and none of the respondents achieved the highest score.
Distribution profile of the VAP-C in the total sample (N = 121)
The Cronbach’s alpha coefficient for internal consistency was 0.94 for the VAP-C total scale and 0.89, 0.91 for the subscale one and subscale two, respectively, suggesting excellent internal consistency of the VAP-C.
Test-retest reliability
A total of 51 patients completed the VAP-C questionnaire twice with an interval of 4–8 hours (mean = 4.56 hours). The two-way mixed model ICC was used here because raters (subjects) are considered as fixed effects and the application based on a single measurement [23]. ICC for the total score of VAP-C was 0.78 (95% confidence interval: 0.70, 0.78), and for subscale one was 0.77 (95% confidence interval: 0.75, 0.78) and subscale two 0.76 (95% confidence interval: 0.73, 0.78), thus demonstrating excellent reproducibility.
Convergent validity
Pearson’s correlations were performed to investigate the association between the VAP-C and DHI subcategories and total score, as well as with the VAS. Significant strong correlations were observed between the VAP-C and the VAS (r = 0.85–0.88). Fair to moderate correlations were found between the VAP-C and the DHI subcategories and total score (r = 0.23–0.53). Detailed correlation coefficients are presented in Table 4.
Convergent validity testing of the VAP-C with DHI and VAS (Pearson correlation coefficient)
Convergent validity testing of the VAP-C with DHI and VAS (Pearson correlation coefficient)
All correlations are significant (P < 0.001). Abbreviations: DHI-E/F/P/T, dizziness handicap inventory-emotional subscale/functional subscale/physical subscale /total score.
The ROC curves are shown in Fig. 1. The AUC for the VAP-C total score was 0.91, and the two subscales 0.90 and 0.88, respectively. The AUCs for both the two subscales and the total score of VAP-C were significant compared with the null AUC of 0.50 (P < 0.01). Based on Youden’s J, optimal cut-off of ≥10.0 for total score and ≥4.5, ≥5.5 for subscale 1 and 2, respectively, were found to be discriminatory between patients with vestibular disorders and subjects without vestibular problems. Table 5 gives detailed ROC analysis results.
ROC curves for VAP-C total score and subscale scores between test group versus control group. The blue curved line indicates VAP-C total score; the green curved line indicates VAP-C subscale 1; the red curved line indicates VAP-C subscale 2. The straight line indicates nondiscriminating characteristics of the test.
ROC analysis of VAP-C
**indicates P < 0.0001.
The purposes of this study were to translate the original VAP questionnaire from English to Chinese, to adapt it for the circumstances in China, and to test psychometric properties of the VAP-C. In order to maintain the content validity of an instrument at a conceptual level across different languages and cultures, the instrument must be adapted culturally in addition to being translated linguistically. During the VAP translation stage, items number # 6, #8, # 11 and #12 were refined or modified to fit Chinese context. In the validation phase, all patients completed the questionnaire unaided, with no difficulty or report of a lack of clarity.
It is worthy of note that no response rate was relatively high with respect to items of sport and driving, which is partially in line with the Alghwiri et al.’s work [2]. Using the original version of VAP, they found that 9 out of 34 activities showed no responses, most of unresponsive cases were related to physical activities and occupational roles. They stated that no response was due to a relatively larger proportion of subjects of senior age; therefore, more younger participants with vestibular disorders would provide greater information about such activities. In present study, explanations may also include that driving and sporting are relatively uncommon for Chinese people who aged over 50, which is the average age of our sample. Similar result was found in a study by Ricci et al. [19], showing that driving was not performed by elderly people in São Paulo, Brazil, where their research was conducted. Explanations for these results could be that both Shanghai and São Paulo are big cities and have heavy traffic, which may be the barrier keeping patients with vestibular disorders from driving. Nevertheless, we decided to retain those items in the scale because they are important to help healthcare professionals better understand the level of activities that the patient performs.
This study demonstrated that the VAP-C has excellent internal consistency as measured by Cronbach’s alpha coefficients, which were comparable to that of the original version, in which a person separation index (PSI) was used to quantify internal consistency and a PSI value of 0.7 was reported [16]. This finding suggests that, generally, all items of VAP-C contributed to measuring the theoretically ICF-based construct of activity and participation. The results indicated that the Chinese version of the VAP is reliable and thus merits continued use, as hypothesized.
For the test-retest reliability, unlike commonly assumed one- or two-week-time interval, 4 hours’ interval was chosen in our study to minimize the possibility of substantial changes in the symptoms which may interfere with the result of studies of this kind [2]. The test-retest reliability, quantified by ICCs, shows that the ICCs of the two subscales and the total score of VAP-C were higher than 0.75, indicating “excellent” reliability. However, this ICCs value was considerably lower than that in original version, in which the ICC was found to be 0.95 for the total scale. Notably lower ICCs in the present study may be due to that different etiology of included sample. We excluded participants whose vestibular symptoms were due to central origin. Patients with sudden deafness with vertigo and benign paroxysmal positional vertigo occupied a large percentage of our sample. Given different pathophysiology, dizziness or imbalance due to periphery vestibular origin, such as BPPV, is typically transient and unpredictable. While in the original VAP study, both periphery and central vestibulopathy were included, thereby leading to a better temporal stability [1].
Convergent validity was determined by calculating the Pearson’s correlation for the VAP-C with the DHI as well as with the VAS. In the vestibular population, various studies have used the VAS to measure different vestibular symptoms, such as the severity of dizziness [8, 20], perception of unsteadiness [12], as well as the impact of oscillopsia while walking [22]. In this study we found strong correlation between VAP-C and VAS, suggesting good convergent validity of VAP-C.
However, we observed weak to moderate correlations between VAP-C and the DHI (Pearson’s r: 0.23–0.49). This finding differs from the previous studies [1, 16], in which the VAP was found to be highly correlated with the DHI. Although culture and etiological features of the recruited participants are the easiest explanations for these discrepancies, they are both unlikely. Rather, we speculate that it is more likely due to the inherent dimensional inconsistency of the DHI than to a bad translation in present study. Some studies [13, 17] pointed out that the original subscale structure of DHI could not be supported. A previous study investigating the association of DHI items with the VAP scale found that only 12 of the 25 items of the DHI showed strong and moderate correlations with the total VAP score (r = 0.48–0.68, p < 0.05), while 8 items showed weak correlation (r = 0.34–0.44, p < 0.05), and 5 items showed no correlation (r = 0.27–0.29, p > 0.05). Thus, less than half the 25 questions of DHI reflect activity limitations or participation restriction in patients with vestibular disorder. In another activity scale, the Vestibular Disorders Activities of Daily Living Scale (VADL), a plateau in the DHI score was achieved while the VADL score continued to increase, indicating that the DHI scale weakly correlates with VADL [21]. It is important to highlight that such discrepancy exists because of different dimensions evaluated by different scales. The VAP brings more information to the therapist about what aspect should be addressed specifically when planning the treatment, while the DHI helps understand some emotional problems and situations that provoke symptoms. Thus, their results could complement each other to obtain a comprehensive assessment.
To test the discriminative property of the VAP-C, ROC analyses were conducted and AUC was calculated for each of ROC analyses. An AUC is a reflection of how well the test discriminates between two groups of individuals. An AUC of 0.9 or greater is considered as excellent accuracy, while 0.7–0.9 indicates moderate accuracy, 0.5–0.7 low accuracy, and 0.5 a chance result [19]. The ROC analysis demonstrated AUCs were close or greater than 0.9 for VAP-C total score or two subscale scores, meaning that these scores could accurately discriminate activity and participation restriction between individuals with vestibular disorder and those without. Moreover, based on the estimate of Youden J, we identified the optimal cut-off value which will help healthcare professionals detect clinically significant activity and participation impairment in the sample of individuals with vestibular disorder. From vestibular rehabilitation’s point of view, established cutoff score will assist physiotherapist in planning physical therapy interventions accordingly.
Study limitations
Although the present study has important clinical implication, some limitations must be acknowledged. First, this study was conducted in mainland China where people speak and read simplified Chinese, therefore, this version of translation can only represent part of the Chinese population. Second, selection bias may exist because of the use of a convenience sample from a tertiary ENT specialized hospital that might not be representative of the entire population especially those who are seeking treatment for their dizziness or imbalance problems in general hospital. Furthermore, only patients with periphery vestibular disorders were included, the results may not be generalized to patients with dizziness or imbalance due to a broad spectrum of etiological reasons. Second, for discriminative validity assessing, ideally a variable widely agree upon as gold standard should be used as reference. Unfortunately, such a variable does not exist in our case. Therefore, we choose using the participants’ medical diagnosis, that is, participants diagnosed with vestibular disorders and without, that appears to be logically and predictably associated with activity and participation. However, it is possible that the so-called heathy group had undiagnosed vestibular problems. Third, we assumed two subscale structure of VAP-C as it was in the original study, however, it may not be the case when it was adapted in Chinses context. Future studies for the VAP-C should include factor analysis to verify its structure. Also, the responsiveness of the VAP-C to changes in subject status over the course of vestibular rehabilitation need to be investigated.
Conclusions
The present study supports the VAP-C as a reliable and valid tool for assessing activity and participation in persons with vestibular disorder. With established cutoff scores, the VAP-C can identify, with excellent accuracy, clinically significant impairment of activity and participation. Consequently, the VAP-C can be applied as a specific activity and participation assessing instrument for clinical and research studies in Chinese language populations, however further research is necessary in larger samples to clarify the factor structure and to assess the responsiveness of the VAP-C.
Grant support
This study was supported by Science and Technology Commission of Shanghai Municipality (grant number.184119551900) and the Department of Otorhinolaryngology at Eye & ENT Hospital of Fudan University.