Cross-Cultural Adaptation and Measurement Properties of the Indonesian Kujala Score for Anterior Cruciate Ligament Tear Patients

Introduction: There is an overwhelming need for worldwide applicable subjective grading systems for patients' anterior cruciate ligament (ACL) tear. The scoring system should be validated for their use in advance related measurements. For the Indonesian language speaking population, only the Kujala Patellofemoral Score (KPS) questionnaire has been translated and validated into the Indonesian language for diseases related to knee pain, but none for ACL tears. The present study aims at cross-cultural adaption to measure the validity and reliability of the Kujala patellofemoral score Indonesian version (KPS-I) specifically for ACL tear patients. Material and methods: The responses of 106 ACL tear patients on two questionnaires comprising the KPS-I and Short Form (SF)–36 were examined by determining the validity and reliability. We conducted the validity construct and content, so the reliability was evaluated by test-retest reliability, internal consistency and measurement error. In addition, the research utilised the Bland and Altman method to explore absolute agreement. Results: The construct and content validity were good, where all hypotheses were confirmed, and the floor or ceiling effect did not occur. The reliability proved excellent between test and retest (ICC=0.99). An internal consistency showed a good Cronbach α of 0.86. The standard error of measurement (SEM), minimal detectable change at the individual (MDCind), and minimal detectable change at the group (MDCgrp) were determined to be 2.1, 5.8, and 0.7, respectively. The application of the Bland and Altman plot method revealed no bias in this study. Conclusion: The validation procedure shows that the KPS-I is a good evaluation instrument for Indonesian patients with ACL tear. However, it is suggested that this score be used for follow-up of patients after ACL reconstruction procedure, especially with anterior knee pain related to the original objective of the Kujala score.


INTRODUCTION
ACL tear rate has increased gradually in the recent decade. The annual global age and sex-adjusted ACL tears incidence were 74.6 per 100,000 person-years 1 . To recover the function of the knee, some treatment must be conducted, such as ACL reconstruction (ACLR) surgery. The treatment also has an impact on the patient to do sports again 2,3 . To evaluate and achieve this goal, patient-reported outcome measures (PROMs) or questionnaires were needed.
There is a great need for applicable subjective grading systems for patients' anterior cruciate ligament (ACL) tear. The scoring system should be validated for their use in advance related measurements to be performed. For Indonesian speaking patients, only the Kujala Patellofemoral Score (KPS) questionnaire has been translated and validated into the Indonesian language for knee pain-related disease, but not for ACL tear cases.
The questionnaire is an essential tool in orthopaedic surgery to evaluate the impact of any surgical procedure on patients' daily lives. Several health-related qualities of life (HR-QoL) questionnaires have been developed; some are generic, others are specific, or are joint-specific in the case of musculoskeletal issues. The Kujala Patellofemoral Score (KPS) is frequently used PROMs to measure knee-specific disease 4 . The tool is also intended to determine the patient's functional status with patellofemoral pain 5 . They have been translated and used in the Indonesian language 6 . However, to use this KPS-I for ACL tear patients, this questionnaire should be validated. Growing body of evidence showed a positive correlation between KPS and some ACL-specific PROMs. Celik et al found an excellent correlation between KPS and International Knee Documentation Committee (IKDC) for ACL tear patients 7 . Furthermore, Mustamsir et al explained that KPS could evaluate ACL tear patients followed by anterior knee pain6. Nevertheless, the literature relating to the reliability and validity of KPS-I for patients with ACL tears is still unavailable.
Thus, the purpose of the study was to adopt a cross-cultural approach, and assess the validity and reliability of the KPS-I questionnaire in patients with an ACL tear.

MATERIALS AND METHODS
Informed consent from patients was obtained for participation in this research as per the principles of the Helsinki declaration. We obtained approval for the study from the institutional review board. We used KPS-I and SF-36 questionnaires for this study. The KPS-I is a diseasespecific questionnaire of anterior knee pain. It consists of six activities associated explicitly with anterior knee pain. The 13 questions' KPS-I has a total score ranging from 0 to the maximum of 100 5 . The SF-36 generic questionnaire consists of eight sections, and the scores range from 0 to 100. The better health condition is reflected by the higher scores 8 .
KPS-I was prepared originally from the literature6, then distributed to ten patients with ACL tears during follow-up in the outpatient knee clinic of one hospital in Indonesia. ACL reconstruction. The researcher documented any difficulties experienced among patients during the completing of the questionnaire. The documentations were reviewed, and some cultural adaptation was made during the preparation. The word "squatting" in KPS-I was extended to "squat during prayer," which is quite common among the Indonesian population. The word "atrophy" was not familiar to some patients, so we added the explanation "decreasing the size of thigh muscle" to make this statement clearer.
Between April until September 2020, a total of 253 randomly selected participants were recruited to contribute to the research. All subjects were Indonesian patients who underwent reconstruction for their ACL tears between January 2015 until March 2020 at one of the hospitals in Indonesia. We included patients following ACL reconstruction for ACL tears, either isolated or multiligament injury, either acute or chronic, with or without concomitant meniscus injury. Patients were then sent information about this study and with two sets of questionnaires (Parts 1 and 2) by email. Part 1 was sent in the first week and included the KPS-I and SF-36. Part 2 was sent in the second week with only the KPS-I. We instructed patients to complete and submit each set of questionnaires and forwarded them to the researcher's address immediately. The patients were provided a global rating of change (GRC) question to be asked at the beginning of Part 2 about the subject's condition between the interval of the two times of completing the questionnaire. This step is needed to establish that the health and knee function persist stably between the recurrent examinations. We included patients with unchanging knee functions within the time interval of the test-retest analysis. In contrast, patients who re-sent each of the questionnaires on the same day or more than one month apart were excluded.
The KPS-I construct validity was tested by correlating it to the SF-36's physical domains (physical component summary, physical function, and physical role functioning). The content validity was evaluated by the ceiling and floor effects' distribution and occurrence. It means that the ceiling and floor effects were considered to be present if more than 15% of participants reached the lowest or highest possible score 9 .
Predefined hypotheses about the relationship's size between the KPS-I and subscales of the SF-36 were formulated according to the COSMIN guidelines 10 . The KPS-I correlates with physical functioning better than social and emotional aspects. Therefore, stronger correlations were expected between the KPS-I and physical component summary ( On the other hand, the correlation between the Thai version-KPS and either social functioning or emotional role functioning subscale of SF-36 showed 0.25 and 0.47, respectively. Based on the correlations found in that study, it was hypothesised to find correlations of 0.5 or higher between the KPS-I and the PCS, physical functioning, and physical role functioning subscale of SF-36. Despite this, it was hypothesised to find correlations of 0.5 or lower between the KPS-I and the MCS, social functioning, and emotional role functioning subscale of SF-36. All predefined hypotheses are shown in Table I.      In addition, to explore the absolute agreement, the Bland and Altman technique was used. The result reflected the amount of agreement in recurring measurements 13 . The characteristics of the study population and scores on the questionnaires were described using means and standard deviations (SD), or frequencies and percentages. Pearson correlation coefficients were calculated between the scores on the KPS-I and the other questionnaires to determine to construct validity. The values less than 0.10, 0.10-0.39, 0.40-0.69, 0.70-0.89, and greater than 0.90 indicate a negligible, weak, moderate, strong, and robust correlation 14 .
To determine internal consistency, Cronbach-α was calculated 15 . The ranges value 0.70-0.95 indicated a decent internal consistency 9 . The intraclass correlation coefficient (ICC) between test and retest KPS-I scores was calculated to assess the test-retest reliability 10 . Values less than 0.5, 0.5-0.75, 0.75-0.9, and greater than 0.90 indicate poor, moderate, reasonable, and excellent reliability, respectively 16 . Measurement error was assessed with SEM and MDC. We calculated SEM by multiplying the pooled standard deviation by √(1-r), where r is the ICC 17 . We calculated MDCind using the formula 1.96 × SEM × √2 and at the group level, MDCgrp was divided by √n 9 .
Absolute reliability was assessed using Altman-Bland plots; when 0 was in the 95% confidence interval (CI) of the mean difference between the test and retest score, no consistent bias is present. Then, we calculated the 95% limits of agreement (LoA) using the formula mean difference ± 1.96 × SDdiff (SDdiff = SD of the mean difference between the first and second administration of the KPS-I) 13 .
The data analyses and interpretation were performed using the SPSS Statistics version 26.0 (IBM) by significance level at 5%.

RESULTS
A total of the 253 patients were recruited to participate; 106 patients (42%) responded and returned the first mailing of questionnaires. Of 106 patients, 75 patients (70.8%) filled in and returned both Part 1 and Part 2 of the questionnaires. The remaining 31 patients (29.2%) returned only Part 1. No patients reported a changed function of the operated knee when they filled out Part 2 of the questionnaire. Hence, the material part of the analysis was conducted on 106 patients' data and test-retest examination on 75 patients' data. Fig. 1 depicts the framework of this study. The demographic characteristic of patients and the main results of the questionnaire are shown in Tables II and III.
The validity test showed that the KPS-I was strongly related to the physical domains of the SF-36 (physical functionality, role physical functioning) than to its cognitive domains (emotional role functioning, social functioning) as mentioned in (Table IV) (0.52, 0.50, 0.45, and 0.33, respectively). Furthermore, the KPS-I was strongly related to the PCS than the MCS subscale of SF-36 (0.60 and 0.50, respectively). The study confirmed all predefined hypotheses on the extent of associations between the KPS-I and the SF-36. The floor and ceiling effects were less than 15% (12%).
From the reliability test, the Cronbach-α showed 0.86 and 0.82 in the test and retest assessment, respectively. The ICC of KPS-I had a value of 0.99 (p< 0.001), with the 95% CI ranging from 0.99 to 1.0. The measurement error test found that the SEM, MDCind, and MDCgr were determined to be 1.57, 4.34, and 0,7, respectively. The Altman-Bland approach revealed a mean difference of the two assessments of 0.1 with 95% CI= -4.9-5.1, 95% LoA= -30.73-30.86 (Figure 2). Nonconsistent bias was presented because the value of 0 was in the 95% CI of the mean difference between the test and retest score.

DISCUSSION
This study aimed to get the validity and reliability of the KPS-I questionnaire in patients with an ACL tear. With the increasing number of ACL tear cases, international questionnaires for outcomes assessment are needed. The Kujala questionnaire is frequently used in anterior knee pain cases, and being a knee-specific questionnaire, it could also be used in other knee diseases. From this study, we found that KPS-I was valid and reliable for ACL tear cases.
The construct validity was assessed by outlining seven hypotheses about the magnitude of the association between the KPS-I score and SF-36. All hypotheses confirmed are the indication of good construct validity. The KPS-I score showed a stronger correlation with the physical component summary, the physical functioning, and the physical role functioning of the SF-36 (p<0.001). In this case, KPS-I can be indicated as a tool of examination in function and sports activities since the validity showed a good result. Although KPS-I also showed a moderate correlation to the MCS subscale of the SF-36, this correlation was weaker than its PCS subscale.
Further affirming the level of validity of the KPS-I, acceptable levels of ceiling and floor effects were found of less than 15% (12%). These are higher than those reported in previous studies, such as the Thai version 10 that did not find ceiling and floor effects. The ceiling and floor effect in this study may be due to the peculiarity of the sample selected (age and type of sports activity of the sample).
The KPS-I test-retest reliability result showed outstanding reliability (ICC 0.99). The result can be compared (Table V) 25 .
The time interval used between the test-retest reliability is essential. In general, the time interval between repeat administration for a patient-reported outcome measurement (PROM) should be relatively brief (three to seven days) when the condition being measured is expected to change rapidly 7 . Furthermore, Terwee et al stated that the time interval between the test-retest reliability should be long enough to prevent recall, though short enough to ensure that clinical change has not occurred. Often, one or two weeks will be appropriate 9 . In this study, we used a one-week time interval to ensure the individual's condition was stable between the test-retest reliability, as used in some studies for ACL tear patients 7 . The Altman-Bland plots showed a satisfactory agreement, reporting only a slight variance between the averages. It was shown that no consistent bias was presented because the value of zero was in the 95% CI of the mean difference between the test and retest score.
There are some limitations in this study. First, the response rate is relatively low (42%). We supposed this is because some patients were unfamiliar with email and an online questionnaire, typically among the Indonesian population.
In future studies, we suggest trying another technique to increase this rate, such as by direct contact in the outpatient clinic. However, at least 100 patients were needed for the validity test, and 50 patients were needed to conduct testretest reliability based on COSMIN guidelines 10 .
Second, the patients were examined by themselves. We could not control treatments of the patients during the testretest, which may have altered their symptoms. Thus, we asked the patients to explain their initial symptoms and conducted the test again. However, this study is the first to evaluate the validity and reliability of KPS-I among Indonesian patients with ACL. Third, some psychometric properties have not been measured. The capability of this KPS-I to detect change over time has not been examined. The responsiveness of KPS-I should also be determined in future research.

CONCLUSION
The KPS-I is a valid and reliable instrument to assess the functional outcome of ACL tear cases for the Indonesianspeaking population. However, it is suggested to use this score to follow-up the patient after the ACL reconstruction procedure, especially with anterior knee pain related to the Kujala score's original purpose. In addition, further study is required into the minimal important change and responsiveness of Kujala score for ACL tear patients.