Abstract
Background:
No questionnaire is currently available for use in patients with anterior cruciate ligament (ACL) injuries in an Indonesian population. The most-used questionnaire in clinical research for these patients is the International Knee Documentation Committee (IKDC) Subjective Knee Form, as its psychometric properties are considered to be excellent.
Purpose:
To translate the IKDC into Indonesian and assess its validity for use in Indonesian-speaking patients with ACL injuries.
Study Design:
Cohort study (diagnosis); Level of evidence, 2.
Methods:
After a forward-and-backward translation procedure and cross-cultural adaptation, the validity and reliability of the questionnaire were investigated. The responses of ACL injury patients on 3 questionnaires, the Indonesian-IKDC (I-IKDC), 36-Item Short Form Health Survey, and Kujala Anterior Knee Pain Scale, were compared. Following consensus-based standards for the selection of health measurement instruments guidelines, construct validity, test-retest reliability, internal consistency, floor and ceiling effects, and measurement error were determined. The Bland-Altman method was used to explore absolute agreement.
Results:
Of 253 ACL injury patients, 106 (42%) responded to the invitation. Construct validity was considered good, as all predefined hypotheses on correlations between the I-IKDC and other scores were confirmed. Reliability proved excellent, with a high test-retest correlation (intraclass correlation coefficient = 0.99). Bland-Altman analyses showed no systematic bias between test and retest. Internal consistency was good (Cronbach α = .90). There were no floor or ceiling effects. Standard error of measurement was 2.1, and the minimal detectable change was 5.8 at the individual level and 0.7 at the group level.
Conclusion:
The I-IKDC, as developed, appeared to be a good evaluation instrument for Indonesian patients with ACL injuries.
The overall age- and sex-adjusted annual incidence of anterior cruciate ligament (ACL) tears is 74.6 per 100,000 person-years. 22 Although exact data are lacking about this incidence in Indonesia, the number of ACL implants used in ACL surgery increased by 42% in 2019 as compared with 2018 in Indonesia (1575 implants in 2018 vs 2236 in 2019, based on data from the 3 main ACL-implant companies in Indonesia; R. Deviandri, MD, unpublished data, 2021). The goal in treating ACL injuries is to regain knee function, preserve long-term knee-related health, return to the activities of daily living, and return to work and sporting activities. 21,23,24 One way to improve management of ACL injuries is to evaluate treatment outcomes from a patient’s perspective. Therefore, reliable and valid patient-reported outcome measures (PROMs) are needed.
In 1987, specialists of the AOSSM and the European Society of Sports Traumatology, Knee Surgery and Arthroscopy compiled the International Knee Documentation Committee (IKDC) Subjective Knee Form to develop a standardized international knee documentation system to evaluate the outcome of randomized controlled trials in patients with knee injuries. In 1998, the IKDC was revised, and that version was published in 2001. 11 Tanner et al 25 reported that the IKDC is the questionnaire that contains the most important questions on symptoms and disabilities of patients with ACL, meniscal, and degenerative knee pathology. Since then, the IKDC has been used widely and translated into many languages in different cultural settings, but a version for Indonesian-speaking patients has yet to be developed and validated.
The aim of this study was to translate and adapt cross-culturally the English version of the IKDC Subjective Knee Form into an Indonesian version (I-IKDC) and to analyze the validity and reliability of this questionnaire for individuals with ACL injuries. We hypothesized that the I-IKDC would be both valid and reliable for use in the Indonesian-speaking ACL population.
Methods
The study protocol was approved by an institutional review board, and patients provided informed consent before participating in this study. The translation procedure was approved by the AOSSM scientific board and the developers of the IKDC.
Translation Procedure
The I-IKDC was developed according to the forward-and-backward translation guidelines originally proposed by Guillemin et al 9 and later revised by Beaton et al. 1 This procedure consists of 6 stages: initial translation, translation synthesis, back translation, committee review, pretesting, and submission and appraisal of all written reports to the committee. Initially, 2 independent Indonesian individuals with a good command of the English language completed a conceptual and literal translation of the IKDC into Indonesian (T1 and T2). Next a synthesized version (T12) was produced, based on the comments of the 2 initial translations, and back-translated into English by 2 independent professional translators (BT1 and BT2) to check for inconsistencies with the original English version. The expert committee, consisting of 3 sports medicine orthopaedic surgeons, 1 methodological expert, and 1 translator, also reviewed this Indonesian translation. After being reviewed by the committee, the questionnaire was edited to its prefinal version by one of the authors (R.D.). This prefinal translated version was then distributed to 10 patients with ACL injuries. One of the researchers (R.D.) documented any difficulties that patients experienced while completing the questionnaire. The documentations were reviewed and used to modify the questionnaire into the final version of the I-IKDC.
Patients and Procedure
Between April and July 2020, a total of 253 randomly selected patients were invited by email to participate in the validity and reliability study. All participants were Indonesian patients who had been treated for ACL rupture with ACL reconstruction or structured rehabilitation between January 2015 and March 2020 at a hospital in Indonesia and had provided informed consent to participate. Next, patients were sent information and 2 sets of questionnaires (parts A and B) via email. Part A was sent in the first week and included the I-IKDC and 2 complementary questionnaires, the Medical Outcomes Study 36-Item Short Form Health Survey (SF-36) and the Kujala Anterior Knee Pain Scale. To explore test-retest reliability, participants were asked to complete the I-IKDC again after 1 week (part B). Patients were instructed to fill out each set of questionnaires and reply immediately to send them back. To determine whether health status and knee function remained indeed stable between the completion of parts A and B, patients were explicitly asked at the start of the part B questionnaire, “Has your status changed since filling out the initial questionnaire?” The 3 possible responses were (1) no; (2) yes, the problem changed for the better; and (3) yes, the problem changed for the worse. Only patients indicating no change in their knee function were included in the test-retest analysis; patients who returned both questionnaires on the same day or more than 1 month apart were excluded.
Patient-Reported Outcome Measures
The IKDC is a region-specific PROM. It includes 18 questions and was designed to measure symptoms, function, and sports activity in patients with a variety of knee conditions, including ligamental and meniscal injuries, articular cartilage lesions, and patellofemoral pain. The form is scored by summing the scores for the individual items and then transforming the score to a scale that ranges from 0 to 100, with 100 indicating the absence of symptoms and higher levels of functioning. 1
The Kujala Anterior Knee Pain Scale is a disease-specific outcome measure of anterior knee pain that documents responses to questions about 6 activities specifically associated with anterior knee pain: whether there is pain while walking, stairs, squatting, running, jumping, or during prolonged sitting with the knee in flexion; whether there is limping, swelling, or subluxation of the patella; and whether there is a need for a walking aid. There are also questions about amount of atrophy in the quadriceps muscle, flexion deficiency, and pain. The Kujala scale consists of 13 questions, with a total score ranging from 0 to 100 and the highest value indicating the best score. 14
The SF-36 is a generic score used to establish a health profile. It consists of 8 scaled scores, where each subscale is transformed directly into a scale from 0 to 100 to identify the patient’s physical and mental state: physical functioning (PF), physical role functioning (PR), bodily pain (BP), general health perceptions (GH), vitality (VT), social function (SF), emotional role functioning (ER), and mental health (MH). In addition, the sum of the PF, PR, BP, and GH subscales generates a physical component summary score (PCS), and the sum of the VT, SF, ER, and MH subscales generates a mental component summary score (MCS). Standardized scores range from 0 to 100, with higher scores indicating better health status. 27
Validity
Validity is the extent to which a score measures what it is supposed to measure: that is, whether it has the intended interpretation. The construct validity of the I-IKDC was assessed by determining the correlation with the Kujala score plus physical and mental domains of the SF-36.
According to the consensus-based standards for the selection of health measurement instruments (COSMIN) guidelines, 18 predefined hypotheses were formulated about the magnitude of the associations between the I-IKDC and subscales of the SF-36 and Kujala score. As both I-IKDC and Kujala scores were knee scales and more specific for abnormalities of the knee, we hypothesized that the I-IKDC would correlate better with the Kujala score than the SF-36. The correlation between the Turkish version of the IKDC and the Kujala score was 0.89. 8 Based on the correlations found in that study, correlations of 0.8 or higher between the I-IKDC and the Kujala score were hypothesized. Irrgang et al 11 showed a correlation of 0.66 between the original IKDC and the PCS subscale of SF-36 and 0.16 between the original IKDC and the MCS subscale of SF-36. Metsavaht et al 17 showed a correlation of 0.51 between the Brazilian Portuguese version of the IKDC and MCS subscale of SF-36. Based on this, it was hypothesized that the correlations of 0.6 or higher between the I-IKDC and the PCS of SF-36 and those SF-36 subscales that measure BP and physical function (PF, PR). Further, it was hypothesized that the correlation between the I-IKDC and the MCS of SF-36 would be lower than 0.6 as well as between the I-IKDC and all SF-36 subscales that measure general health and social and mental function (GH, VT, SF, ER, MH). The IKDC was developed to measure PF rather than social and/or emotional aspects; therefore, lower correlations were expected between the I-IKDC and the MCS than between the I-IKDC and the PCS. Construct validity can be considered good when at least 75% of the hypotheses are confirmed. 26
Floor and Ceiling Effects
The occurrence of floor and ceiling effects was assessed. These effects are considered to be present if more than 15% of respondents achieve the lowest or highest possible score. 26
Reliability
Reliability represents the extent to which individuals can be distinguished from each other despite measurement errors. Following the COSMIN guidelines, 18 reliability was assessed in terms of internal consistency, test-retest reliability, and measurement error. Internal consistency refers to the extent to which subscales of a questionnaire are related, test-retest reliability concerns the extent to which patients’ scores are the same for repeated measurements, and measurement error is a measure of systematic error of a patient’s score that is not caused by actual changes in the measured construct. The Bland-Altman method was used to explore absolute agreement, which reflects the amount of agreement in repeated measurements. 2
Statistical Analysis
The characteristics of the study population and scores on the questionnaires are described using means and standard deviations or frequencies and percentages. To determine construct validity, as variables showed normal distribution, the Pearson correlation coefficient (
The intraclass correlation coefficient (ICC) between test and retest I-IKDC scores was calculated to assess test-retest reliability.
18
Values <0.50, 0.50 to 0.75, 0.76 to 0.90, and >0.90 are indicative of poor, moderate, good, and excellent reliability, respectively.
13
Standard error of measurement (SEM) and minimal detectable change (MDC) were calculated to assess measurement error. SEM was calculated by multiplying the pooled standard deviation by Ö(1 –
Absolute reliability was assessed using Bland-Altman plots: When 0 was in the 95% CI of the mean difference between the first and second administration of the IKDC, no systematic bias was present. The 95% limits of agreement (LOA) were calculated with the formula mean difference ± 1.96 × SDdiff, where SDdiff is the standard deviation of the mean difference between the first and second administration of the IKDC. 2 Statistical analyses were performed using SPSS Statistics Version 26.0 (IBM), with level of significance set at 5%.
Results
Translation and Cross-Cultural Adaptation
Some cultural adaptations were made during the translation procedure. The word “skiing” was changed to “badminton” in questions 1, 5, 7, and 8, as the former is not a typical sport in Indonesia. The word “squatting” was changed to “squat while prayer,” which is quite common among the Indonesian population.
Patient Characteristics
Of the 253 patients, 106 (42%) responded to the invitation; 75 (70.8%) of these 106 patients filled in and returned 2 complete sets of questionnaires. The remaining 31 patients (29.2%) returned only part A. No patients reported a better or worse function of the operated knee when they filled out the second questionnaire. None of the patients were excluded because of missing data. Data of 106 patients were used for validity analyses and internal consistency, and data of 75 patients were used for determining test-retest reliability. Most of the patients were nonathlete men with a mean age of 30.5 ± 9.2 years. Mean scores of the first and second assessments of the I-IKDC were 66.4 ± 18.8 and 66.8 ± 19.1, respectively. Demographic patient characteristics are described in detail in Table 1. Scores from the PROMs can be found in Table 2.
Demographic Patient Characteristics (N = 106)
Patient-Reported Outcome Measures
Validity
All predefined hypotheses on the magnitude of associations between the I-IKDC and either the Kujala score or the SF-36 were confirmed. The I-IKDC score showed strong correlations with the Kujala score (
Pearson Correlation Coefficients Between I-IKDC, Kujala, and SF-36 Subscales
Internal Consistency, Test-Retest Reliability, and Measurement Error
The Cronbach α was .90, indicating good internal consistency. The ICC had a value of 0.99 (

Bland-Altman plot of the mean difference between the first and second measurements of the Indonesian translation of International Knee Documentation Committee (I-IKDC). The open circles represent data points, the solid line represents the mean difference, the dotted lines represent the 95% CIs, and the dashed lines represent the limits of agreement.
Discussion
From this study, we were able to confirm all predefined hypotheses on correlations between the I-IKDC and other scores, implicating good construct validity: An ICC value of 0.99 implicated good test-retest reliability; a Cronbach α of .90 implicated good internal consistency; and no floor or ceiling effects were found. In addition, the SEM was 2.1, and the MDC was 5.8 at the individual level and 0.7 at the group level. Based on these results, the I-IKDC can be considered a valid and reliable questionnaire for use in patients after ACL injury.
The construct validity of the I-IKDC was considered good, with all predefined hypotheses confirmed. As hypothesized, the I-IKDC score showed a strong correlation (>0.6) with the PCS and the PF subscale of the SF-36 (
Correlations Between Different Versions of the IKDC and the SF-36 Subscales
No floor or ceiling effects were observed. Theoretically, a ceiling effect would have occurred, especially in the longer term after treatment of an ACL injury, but this was not the case in the original version of the IKDC or the Dutch, Brazilian, or Turkish versions. 3,10,11,19 Hence, this effect was also expected to be absent in the Indonesian version of this form.
Internal consistency of I-IKDC was good, with a Cronbach α of .90. Higher values are not desirable, as this may indicate a redundancy of questionnaire items. Good internal consistency also was shown in the original version (0.92) and the Italian (0.91), Dutch (0.92), Thai (0.92), and Korean (0.91) versions. 10 –12,16,20
Test-retest reliability of the I-IKDC was found to be excellent (ICC = 0.99). This is comparable with the original English version (ICC = 0.95) and the Dutch (ICC = 0.96), Thai (ICC = 0.92), Brazilian (ICC = 0.99), and Turkish (ICC = 0.91) versions (Table 5). 3,10,11,16,17 The Bland-Altman plots showed adequate agreement, reporting only a small difference between the mean scores of the first and second assessments of the I-IKDC. No systematic bias was present because the value of zero was within the 95% CI of the mean difference between the test and retest score of the I-IKDC.
Psychometric Properties of the Different Versions of the IKDC
For the I-IKDC, the SEM was 2.1, MDCind was 5.8, and MDCgrp 0.7. These values are comparable with those presented in the original English version (SEM, 4.6) as well as the Brazilian (SEM, 2.4; MDC, 6.7) and Turkish (SEM, 6.0; MDC, 16.4) versions. 3,11,17 Low MDC values at the group level indicate that the I-IKDC can be used for group comparisons, as only low values are needed to detect change. However, only values higher than the SEM can be distinguished from the measurement error; therefore, to detect a statistically significant change in scores on the I-IKDC, the difference should be higher than the SEM. To distinguish from a measurement error and confirm that a real change occurred, the difference between 2 measurements should be greater than the MDCind value. It is therefore questionable whether the I-IKDC is an appropriate tool for monitoring individual patients over time.
The current study has some limitations. The response rate was not high (42%), yet the total number of participants can be considered sufficient, because according to the general recommendations for comparing measurement properties, at least 100 patients are needed to study validity and 50 patients to study test-retest reliability. 26 Another potential limitation is that we compared the I-IKDC scores with the SF-36 and only 1 other disease-specific scoring system (Kujala score). However, as discussed, no other validated PROM in the Indonesian language is available to evaluate knee pain or function in addition to these scores.
Next to the psychometric properties that have been examined in this study, future research should investigate the minimal important change (MIC) as a measure of responsiveness of the I-IKDC. The MIC should be determined to assess whether a found difference is also clinically important as perceived by the patient. 9 The MIC of the English version of the IKDC was reported to be 10.9. 19 For the German version, an MIC of 6.8 is reported, which is smaller than the MDC value. 15 In addition, effect sizes are reported: 1.13 in the English version, 8 2.09 in the Turkish version, 3 and 1.04 in the German version. 15 These findings are, however, not transferable to the Indonesian version, and hence, the MIC of the I-IKDC should be determined in future research. Again, these results cannot be transferred to the I-IKDC. Knowledge about responsiveness and capability of the I-IKDC to detect change over time is needed for the I-IKDC to be used in longitudinal research and in practice to follow Indonesian-speaking patients over time.
Conclusion
The I-IKDC can be considered to be both valid and reliable; therefore, it is applicable for patients with an ACL injury in the Indonesian population as a PROM. Now that this questionnaire is available, we can measure symptoms, function, and sports activity in the Indonesian ACL population. More research is needed regarding MIC and responsiveness.
Footnotes
Acknowledgment
The authors extend their gratitude to Bruce Reider, MD, for facilitating the translation and validation of the IKDC Subjective Knee Form score into the Indonesian language. They also thank Professor Ron Diercks for initiating this research and all IOSSMA members who participated in the study.
Final revision submitted April 1, 2021; accepted May 19, 2021.
The authors declared that there are no conflicts of interest in the authorship and publication of this contribution. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from the ethical review board of Universitas Riau (B/102/UN 19.5.1.1.8/UEPKK/2020).
