Abstract
Study Design
A single-center validation study.
Objective
To translate and cross-culturally adapt the AO Spine PROST (Patient Reported Outcome Spine Trauma) into German, and to test its psychometric properties among German-speaking Swiss spine trauma patients.
Methods
Patients were recruited from a level-1 Swiss trauma center. Next to the AO Spine PROST, the EQ-5D-3L questionnaire was used for concurrent validity. Questionnaires were filled out at two-time points for test-retest reliability. Patient characteristics were analyzed using descriptive statistics. For content validity, floor, and ceiling effects, as well as any irrelevant and missing questions were analyzed. Construct validity of the AO Spine PROST questionnaire to the EQ-5D-3L was tested using Spearman correlation tests.
Results
The AOSpine PROST was translated and adapted into German using established guidelines. We included 179 patients. The floor effect for all items was well within the optimal range (below 15%), while the ceiling effect of seven items was within the optimal range. None of the items displayed a problematic floor or ceiling effect. The overall test-retest reliability of the total PROST score was excellent, with an ICC of .83 (95% CI .69-.91). The Spearman correlation coefficient between the total PROST summary score and EQ-5D-3 L was ρ = .63.
Conclusions
The German version of the AO Spine PROST questionnaire demonstrated very good validity and reliability results.
Introduction
Measuring the individuals’ health-related quality of life (HRQoL) after spine trauma is highly relevant. These injuries - often caused by high-velocity accidents, with the thoracolumbar spine (TL) being the most affected region - can lead to severe impairments of the patients and disabilities. From a social and economic point of view, this leads to a global health concern.1,2 Specific outcome measurements provide the opportunity to compare outcomes of different treatment strategies in a certain patient population. The results can contribute to the development and adaptation of patient-specific treatment plans. However, until the development of the spine-specific AO Spine Patient Reported Outcome Spine Trauma (PROST) questionnaire, only non-specific questionnaires were used. The questionnaires that are among the most used to assess overall health include the EQ-5D instruments and the Short Form 36 (SF-36) questionnaire. In addition, there are numerous other questionnaires for assessing patient-reported physical disability due to low back pain, such as the Roland-Morris Disability Questionnaire or the Oswestry Disability Index. However, these instruments were developed to assess the quality of life of patients with chronic low back pain, degenerative disease, or disability in low back pain. Their use as an instrument for clinical outcomes, quality of life, and functional limitations due to spine trauma is limited.3-6
In 2017, the AO Spine Knowledge Forum (KF) Trauma developed a disease-specific outcome measure for patients suffering from spine trauma: The PROST questionnaire, initially developed in Dutch. This questionnaire contains 19 items related to different areas of functional aspects in everyday life. 7 Patients with spinal injuries leading to complete paralysis or suffering from multiple traumatic injuries (Injury Severity Score >15) were excluded from the developmental process of the AO Spine PROST. This aimed to identify limited life quality solely related to spinal injury and exclude the overwhelming factors of paralysis and polytrauma. The recent validation studies of the Dutch, English, and Nepali versions of the PROST showed very good results.7-9 However, a psychometrically validated outcome measure for analyzing the trauma-related quality of life limitations and treatment success of German-speaking patients with spinal cord injuries is missing so far. 6
This study aimed to (1) translate the AO Spine PROST into German, including a cross-cultural adaption, and (2) test its psychometric properties in German-speaking patients suffering from an acute traumatic spinal injury.
Materials and Methods
Translation and Cross-Cultural Adaptation
Translation and cross-cultural adaption were performed using established guidelines. 10 Due to the missing analysis of clinical data, the institutional review board decided no explicit ethics approval was necessary; a waiver was granted. All patients signed the informed consent to be included in the study. The Dutch version of the AO Spine PROST was used and translated into German according to the method described earlier7,10: (1) Forward translations, (2) synthesis, (3) back translations, and (4) expert committee review, followed by validation.
The first stage comprised the initial translation. Two native bilingual German and Dutch speakers did the forward translation of the AO Spine PROST from Dutch into German independently. In stage 2, the two independent translators summarized their findings into one synthesis. In stage 3, two separate, uninformed bilingual translators with Dutch as their native language did the back-translation from German to Dutch. Both were blinded to the concepts under investigation. In the next phase, all translations and written reports were reviewed by an expert committee to achieve equivalence between the four domains, semantics, idiomaticity, experience, and conceptualism, developing the final German version of the AO Spine PROST questionnaire. In the fifth stage, the preliminary version was tested on 30 patients treated in our spine trauma outpatient clinic. Patients were asked to fill out the EQ-5D-3L questionnaire. Additionally, they were questioned whether and how they would change or improve the questionnaire. This pre-testing provided information on how patients interpreted the items of the questionnaire and ensured that the translated version retained its equivalence in an administered situation. After the validation process, no further changes were performed to the German version of the AO Spine PROST.
Patients and Study Procedures
All patients in our clinic (level-1 trauma center) were prospectively screened for eligibility. Inclusion criteria were history of spinal column injury as the primary diagnosis, age >18 years, and capability to understand and adequately fill out the questionnaires. The inclusion and exclusion criteria were in line with those in the original AO Spine PROST. Exclusion criteria were multiple injuries (Injury Severity Score [ISS] > 15) or an American Spinal Cord Injury Association [ASIA] impairment grade A or B at discharge or transfer from the hospital.
Instruments
The questionnaires were given to the patients in paper form. Each item of the PROST questionnaire is accompanied by a numerical rating scale ranging from 0 to 100. Here, 0 indicates no function at all, and 100 is the level of function before the injury. Smileys support this scale at each end of the ruler (supplementary material). For validity purposes, the AO Spine PROST was compared with the EQ-5D-3L, which is a commonly used, generic outcome instrument for assessing HRQoL (scale from 0 to 1, where 1 indicates perfect health). The EQ-5D-3L consists of two pages, the EQ-5D description system and the EQ-5D visual analog scale (EQ VAS).11,12
Statistical Analysis
The total PROST score was computed as the sum of all scores divided by the number of completed items, as previously described. 8 It can range from 0 (worst) to 100 (best functional status). The responses to the EQ-5D-3L questionnaire items were combined and converted into a single utility estimate using the EU scoring algorithm and value set. 13 At the same time, the EQ-5D-3L was reported as pure patient-based ratings of the HRQoL.
Missing data were analyzed for each of the 19 PROST items and the total score was obtained by dividing the number of missing values by the number of respondents. Floor and ceiling effects were assessed by calculating the percentage of respondents who reported minimum and maximum possible scores, respectively, on individual PROST items and the overall score. If floor and ceiling effects occur in many patients, a questionnaire may not be able to detect deterioration or improvement in the patient’s health status. Therefore, following established rules, we considered floor and ceiling effects larger than 70% to be problematic and effects smaller than 15% to be optimal. 14
We also investigated the construct validity of the PROST questionnaire by assessing the degree to which the overall PROST scores relate to EQ-5D-3L utility scores. EQ-5D and PROST summary scores were not normally distributed, in which case it is best to use a non-parametric measure. Therefore, Spearman’s Rho (ρ) corrected for ties was used to assess these correlations, whereby ρ > .80 was interpreted as an excellent outcome, .61-.80 very good, .41-.60 good, .21-.40 fair, and .00-.20 poor. 15
Finally, we assessed the test-retest reliability of the PROST questionnaire to explore the questionnaire’s stability over time by comparing PROST results at two points in time. We calculated intra-class correlation coefficients (ICC) and their 95% confidence intervals for each PROST item. ICC can occupy values between .0 and 1.0, with values of .6-.8 generally indicating good reliability and values above .8 indicating excellent test-retest reliability. 16
Results
Cross-Cultural Adaptation Process
The first stage involved the initial translation, which was performed according to the guidelines developed by Beaton et al 10 In the fifth step, the preliminary version was tested on 26 patients. This pretest provided information on how patients interpreted the items of each question. During the validation phase of the cross-culturally adapted and translated questionnaire, the first group of 26 patients was asked whether the questions were clearly worded and if the questions were difficult to understand. Only one patient added two comments. Question 3, recreation, and leisure (Freizeit-Aktivitäten (Ausführen von Sport, Hobbies): patient’s comment ‘A more detailed formulation would be better here, eg ‘I can perform these same leisure activities as before the accident’. Question 4, Social life (Sozialleben (Pflegen von Kontakten mit Familie, Freunden und Bekannten): patient’s comment ‘Is it meant whether physical changes have an influence on the social life or also whether the accident has brought other changes?‘ However, these comments did not lead to an adaptation of the questionnaire, since it is explained in the questionnaire, that patients should compare their ‘function NOW to BEFORE the accident’.
Validation of the Cross-Cultural Adapted PROST Questionnaire
Patient Characteristics and Demographic Data.
aData are mean ± standard deviation with percentages in parentheses.
Missing Values, Score Distribution, Floor, and Ceiling Effects
Missing Data, Ceiling and Floor Effects of All Patients Included (n = 179).
Construct Validity: Correlation Between PROST Summary Score and EQ-5D-3L
Spearman correlation coefficient between the PROST summary score and EQ-5D was ρ = .63, and the correlation between the PROST summary score and EQ VAS was ρ = .69, both P < .001. This is considered to be a very good outcome.
Test-Retest Reliability
Internal Consistency and Test-Retest Reliability Results. Internal Consistency is Sown for Each AOSpine PROST Item, Test-Retest Reliability was Assessed by Using ICC Along With its 95% CI.
Discussion
This single-center validation study of the German version of the AO Spine PROST questionnaire investigated its psychometric properties after initial translation and cross-cultural adaption. The German PROST questionnaire demonstrated very good construct validity and excellent test-retest reliability amongst German-speaking patients.
The PROST is a unique questionnaire and the first of its kind to focus specifically on spinal trauma patients.4,6 One of the unique features is that the patient is not asked only about his current condition as in many other HRQoL questionnaires, but to compare his current health status with that before the accident. PROST is designed to support trauma spine surgeons in the aftercare of their patients and to simplify spine research by allowing researchers to compare patient groups from multiple sites or across different regions of the globe. For the latter, the PROST must be available in many different languages. According to Beaton et al, the translation process by native speakers in Dutch and German guarantees a structured and well-adapted translation of the original Dutch version into German. The initial evaluation phase showed a very good understanding of all questions with only two comments from one patient which did not lead to any further changes to the questionnaire.
After cross-cultural adaptation was completed, we proceeded to test the German PROST in our patient cohort. During the analysis, we found that questions 2 (work/study ‘Arbeit/Studium’ and 13 (sexual function ‘Geschlechtsverkehr’) were not answered by many patients (30.2% and 17.9%, respectively). The PROST is designed in such a way that question 2 can be skipped if the patient did not work or study before the accident (this is also indicated in the questionnaire). In contrast, question 13 (sexual function) was skipped by many patients without being ‘permitted’ in the questionnaire (missing value of 17.9%). In this case, the total was calculated from 17 questions, even though this was not intended in the original version of the PROST. 17 Another important factor determining the instrument’s acceptability (ie, completeness of questionnaires) is the way the information is collected. When the questionnaire is completed by patients on paper, a higher rate of missing values is to be expected, whereas this can be largely avoided with electronic data collection. 18 This problem has not been discussed in the studies published so far; however, both the Dutch and the English validation of the questionnaires were performed with an online system, so this problem probably did not occur at all.
Furthermore, we analyzed the ceiling and floor effects of the German PROST. These can be observed when a significant number of patients reach either the maximum (ceiling) or minimum (floor) score. As a result, the true extent of their health status cannot be determined. The reported PROST scores were generally very high, suggesting that most patients included in the study had a quite mild disease or a very good outcome. In particular, the two items on urination and defecation had exceptionally high values. This is probably also because we had excluded ASIA-A, -B, and polytrauma patients. The feasibility of using PROST in patients with complete spinal cord injury was investigated separately. 19 As a result, the number of patients with a maximum score (ceiling effect) was high in most items, although not problematic in any of the items. This might also be related to the long time point after trauma. The floor effect for all items was well within the optimal range (below 15%). These results are comparable to the adapted version of the AO Spine PROST in English and the original Dutch version.7,17
Since the PROST is the only validated questionnaire that specifically collects data from spine trauma patients, we needed another questionnaire for comparison. While other studies used the SF-36, we chose the EQ-5D-3L because it is one of the most common questionnaires for HRQoL and is routinely collected in our clinic.7,17 The Spearman correlation coefficient between the PROST sum score and the EQ-5D-3 L was ρ = .63, demonstrating a consistent assessment of psychometric outcome analysis in spinal trauma patients.
This study has some limitations. The test-retest analysis was performed on all participants who completed the second questionnaire within 60 days of the first questionnaire (the median was 28 days). This is a relatively long interval (usually 2 weeks is targeted), which may have resulted in lower reliability compared with that observed with a shorter interval. However, the ICC of most items was reassuringly high despite the long interval. Another limitation of the study is the small sample size for the test-retest analysis (n = 35). Based on this sample, it was not possible to calculate the SEM and MDC 95% to determine the minimum detectable change. Larger validation studies are needed for this type of analysis. Another limitation could be the long period between trauma and completion of the PROST questionnaire in some patients. In half of the patients, this period was 5.5 months or longer. Whether or not this is a problem depends mainly on the primary purpose of the PROST instrument in the study used. Our research shows that the PROST questionnaire provides reliable and consistent results for both long- and short-to medium-term assessments after spinal trauma. However, the long-term reliability and validity could be confirmed in a recently published study. 19 A final limitation is that we excluded patients with severe neurological impairment (AISA A and B) after spinal trauma. The original intention in developing the PROST questionnaire was to assess the outcomes of patients suffering from only mild to minor neurological impairments.6,17 However, the applicability of the PROST in patients with severe spinal cord injury was confirmed in a recently published study. 19
In conclusion, the German version of the AO Spine PROST is a reliable and valid instrument for measuring HRQoL in German-speaking patients after spinal trauma, which can be used in clinical practice as well as in research.
The PROST questionnaires are available free of charge through the AO Spine website. 20
Footnotes
Acknowledgments
The authors thank Maike Niewkamp for the questionnaire translation and Manuel Ottinger for his contribution to patient recruitment. We thank Lukas Staub of Talus Research Consulting (talus.com.au) for the methodological support, for performing the statistical analyses, and for helping with results reporting.
Author Contributions
MCD, LMB, SS, FCO, and SH set up the study protocol. KO, LK, LAB, and SH selected the data. Finally, SH, KO, LK, and MCD prepared the manuscript. All authors approved the final manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was organized and funded by AO Spine through the AO Spine Knowledge Forum Trauma, a focused group of international Trauma experts. AO Spine is a clinical division of the AO Foundation, which is an independent medically-guided not-for-profit organization. Study support was provided directly through AO Network Clinical Research.
Informed Consent
This research does not include patient-specific data. Therefore, ethical consent was not necessary according to local regulations.
Data Availability
The datasets generated and analyzed during the current study are available from the corresponding author upon reasonable request.
