Abstract
Objectives:
Katakori is a Japanese term for non-specific symptoms including discomfort or dull pain caused by muscle stiffness around the neck through the shoulders and is one of the most frequently reported symptoms in Japan. However, there has been no standardized patient-reported outcome measure to evaluate Katakori severity. This study aimed to investigate the reporting level on validity and reliability of patient-reported outcome measures of Katakori severity.
Method:
A systematic search in ICHUSHI, MEDLINE, EMBASE and PubMed was undertaken from inception to April 2017 without language limitations. Two authors independently undertook screening by inspecting the title and abstract. Inclusion criteria were as follows: (1) participants with Katakori symptoms, (2) reporting reliability or validity of questionnaire evaluating Katakori severity and (3) published journal articles. Studies that either of the authors retained through the screening process were inspected with full text by the two authors independently to examine eligibility of the study. Any disagreement on eligibility after full-text inspection was resolved by discussion between them. Methodological quality was rated with the COnsensus-based Standards for the selection of health Measurement INstruments checklist. Subsequently, the evidence level of each measurement property was assessed for each questionnaire. The two authors extracted data independently. Any disagreement was resolved by discussion between them.
Results:
Five questionnaires were identified in five studies. The Shoulder Pain and Disability Index and Scale for Measuring Felt Shoulder Stiffness had the highest level of methodological quality. However, excellent measurement properties were found in only two out of nine criteria. Furthermore, in particular, content validity was not investigated in any measure.
Conclusion:
There is preliminary evidence for the reliability and validity of the Shoulder Pain and Disability Index and Scale for Measuring Felt Shoulder Stiffness; however, much further research is required. Identifying or developing a patient-reported outcome measure with content validity would be a future research agenda.
Introduction
‘Katakori’ is a Japanese term indicating a non-specific symptom including discomfort or dull pain caused by muscle stiffness around the occiput through the cervical spine to the acromion and scapular area. 1 Katakori significantly impacts on Japanese society, being the most frequent symptom (12.5%) in Japanese men and the second most frequent symptom (6.0%) in Japanese women. 2
More than 550 papers have been published on Katakori in Japanese-language journals from 2011 to 2016. 3 However, there has been no standardized patient-reported outcome measure (PROM) developed to evaluate Katakori severity. Thus, it is necessary to comprehensively investigate the validity and reliability of PROM used to evaluate symptoms of Katakori. However, no systematic review has been undertaken.
Furthermore, developing a patient-individualized measure is one of the priorities in musculoskeletal research. 4 For a PROM, the patient-individualized measure includes a fully individualized questionnaire and a semi-individualized questionnaire, but not a fully structured questionnaire. A merit of the fully structured questionnaire is the ability to compare scores between patients as all patients answer all items. However, the importance of each item is different between patients. It is known that fully individualized questionnaires, where items were originally generated by each patient, have greater responsiveness than fully structured questionnaires.5,6 Nevertheless, it is not possible to compare scores between respondents using fully individualized questionnaires. As a consequence, there has been research on the development of semi-individualized questionnaires, which allow the comparison of scores between patients by reflecting patients’ differences.7,8 In the semi-individualized questionnaire, all patients answer all the same items and provide weight for each item. Therefore, it is also important to understand how items are described and whether the questionnaire is a structured questionnaire, semi-individualized questionnaire or a fully individualized questionnaire.
The primary purpose of this review was to evaluate the reporting level on validity and reliability of PROMs of Katakori severity in published studies. The secondary purpose was to understand the structure of the questionnaire.
Methods
Design
This systematic review was undertaken in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines 9 and updated method guidelines for Cochrane Musculoskeletal Group Systematic Reviews and Metaanalyses. 10 For comprehensive evaluation of validity and reliability of PROMs, we used the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist. 11 This study was registered in the International prospective register of systematic reviews (CRD42018081104).
Identification and selection of studies
A systematic search was performed in MEDLINE, EMBASE and PubMed (Supplemental Appendix 1) from inception to April 2017 without language limitations. Katakori has been well studied in Japan and therefore a Japanese database, ICHUSHI, was also systematically searched in Japanese (Supplemental Appendix 2) from inception to April 2017. The database search was undertaken by one author (H.T.).
Two authors (K.A. and H.T.) independently undertook screening by inspecting the title and abstract of potential papers. Inclusion criteria were as follows: (1) participants with Katakori symptoms including discomfort or dull pain caused by muscle stiffness around the back of the head through the shoulders and/or shoulder blades and (2) published journal articles reporting reliability or validity of a questionnaire to understand Katakori severity. The full text of studies that either of the authors retained through the screening process was inspected by the two authors (K.A. and H.T.) independently to examine eligibility of the study. Studies for which design was not suitable to investigate reliability or validity of a questionnaire for the population with Katakori symptoms (e.g. mixing participants with other symptoms/diagnoses) were not considered eligible. Conference proceedings were also not considered eligible. Any disagreement for eligibility after full-text inspection was resolved by discussion between the two authors (K.A. and H.T.). During full-text inspection, a hand search of the reference list from the identified studies was undertaken to identify relevant work.
Quality assessment and data synthesis
Methodological quality of each study and level of evidence for a measurement property for each questionnaire were evaluated similarly to a previous systematic review. 12 The evaluations were undertaken by the two authors (K.A. and H.T.) with assessor blinding. Any disagreement was resolved by discussion between the two authors (K.A. and H.T.).
The COSMIN checklist 11 was used as the risk of bias tool for methodological quality in nine measurement properties. The COSMIN checklist has nine measurement properties (internal consistency, reliability, measurement error, content validity, structural validity, hypothesis testing, cross-cultural validity, criterion validity and responsiveness). In each checklist item, methodological quality was rated on a four-point scale (excellent, good, fair, poor). Each measurement quality was determined by the lowest score of the four-point scale in the checklist item. Unreported items were rated as not reported (NR). The evaluation was undertaken following a training session with five articles irrelevant to this study13–17 in order to standardize skills for assessment of methodological quality.
Evidence levels of the nine measurement properties were assessed with the criteria modified from that developed by Elbers et al. 12 (Table 1). We added ‘no study’ to the unknown level.
Brief summary of the level of evidence for nine measurement properties, which was modified from that developed by Elbers et al. 12
Data analysis
Agreement in the assessment of the methodological quality with five categories (excellent, good, fair, poor, NR) between the two authors (K.A. and H.T.) was evaluated with percent agreement as NR did not allow the use of an ordinal scale.
The following data were extracted from each study and summarized in a table format: (1) measure, (2) participants, (3) domains, (4) response option, (5) score range, (6) item descriptions possibly relevant to Katakori and (7) structure of the questionnaire (fully individualized, semi-individualized or structured). Two authors (K.A. and H.T.) extracted data independently. Any disagreement was resolved by discussion with the two authors (K.A. and H.T.).
Results
Flow of studies through the review
Figure 1 presents a flow of the study. Five studies were finally included in this review18–22 and the following five questionnaires were identified: (1) Questionnaire on Musculoskeletal Disorders of the Japanese Committee on Occupational Cervico-branchial Disorders (QMD-JCOCD), (2) Shoulder Pain and Disability Index (SPDI), (3) Scale for Measuring Felt Shoulder Stiffness (SMFS), (4) Questionnaire on Stiff Shoulders with Activities of Daily Living (QSSADL) and (5) 12-item QSSADL.

Flow of the literature inclusion.
Table 2 summarizes the methodological quality and evidence level of validity and reliability of the five studies. Percent agreement in the assessment of the methodological quality was 84.44%. There was no study investigating measurement error, content validity, cross-cultural validity and responsiveness.
Summary for methodological quality and evidence level of measurement properties in five studies and questionnaires.
QMD-JCOCD: Questionnaire on Musculoskeletal Disorders of the Japanese Committee on Occupational Cervico-branchial Disorders; SPDI: Shoulder Pain and Disability Index; SMFS: Scale for Measuring Felt Shoulder Stiffness; QSSADL: Questionnaire on Stiff Shoulders with Activities of Daily Living; NR: not reported.
Methodological quality (excellent, good, fair, poor, NR)/evidence level of measurement properties (strong, moderate, limited, conflicting, unknown).
The SPDI and the SMFS had the greatest number of strong evidence for measurement properties, but for only two out of a maximum nine. Table 3 summaries the five studies. All questionnaires were identified as structured in design.
Summary of five studies.
QMD-JCOCD: Questionnaire on Musculoskeletal Disorders of the Japanese Committee on Occupational Cervico-branchial Disorders; SPDI: Shoulder Pain and Disability Index; SMFS: Scale for Measuring Felt Shoulder Stiffness; QSSADL: Questionnaire on Stiff Shoulders with Activities of Daily Living; ICC: intra-class correlation coefficient; RDQ: Roland–Morris Disability Questionnaire.
Discussion and conclusion
This study systematically searched the literature for studies reporting on the reliability or validity of Katakori severity. This study provides research agenda in relation to Katakori evaluation.
Generally, the overall quality of evidence for measurement properties in five questionnaires was not strong. The SPDI and SMFS had the largest number of properties with strong evidence but for only two out of nine properties. Furthermore, there was no study investigating measurement error, content validity, cross-cultural validity and responsiveness. It is important to note that content validity, which is considered a principal measurement property, 23 was not established in the current literature. These findings indicate a need to investigate content validity of the existing measures in the population who report Katakori, and a need to develop a new questionnaire fully satisfying the nine measurement properties when there is no content validity in the existing measures.
In the current literature, all questionnaires were structured measures. Developing a patient-individualized measure is one of the priorities in musculoskeletal research. 4 Comparing scores between individuals is not possible with fully individualized measure, but it is possible with semi-individualized measures, allowing reflection on individual differences between patients. Therefore, a semi-individualized measure would be ideal when a new tool is investigated.
Potential future research direction
Content validity is conventionally assessed with the Content Validity Index and multirater kappa coefficient of agreement among an expert panel. 24 However, there is uncertainty whether the expert panel fully understands symptoms and disabilities associated with Katakori. Therefore, it would be prudent to first fully understand symptoms and disabilities due to Katakori from a patient’s point of view. Problem elicitation technique (PET) which identifies problems that are most important to the individual patient has been used for investigation of content validity in previous studies25,26 and would appear a useful tool to investigate Katakori symptoms from the patient’s perspective.
Study limitations
This study was limited to investigations on patients with definite Katakori symptoms. However, there is no clear English translation for Katakori. Therefore, there may be questionnaires that were actually used for patients with Katakori symptoms, but patient’s symptoms were not specific enough to be included in this study (e.g. patients with neck pain). A further limitation is that full investigation of the gray literature was not undertaken in this review.
Conclusion
The SPDI and SMFS were the most fully investigated measures with respect to reliability and validity based on the literature review. However, content validity was not investigated in measures for Katakori severity and should be investigated in the future. Furthermore, it was found that all questionnaires regarding Katakori severity in the current literature were structured PROMs, not a patient-individualized PROM.
Supplemental Material
supplemental – Supplemental material for Reporting on the level of validity and reliability of questionnaires measuring Katakori severity: A systematic review
Supplemental material, supplemental for Reporting on the level of validity and reliability of questionnaires measuring Katakori severity: A systematic review by Kaori Aoki, Toby Hall and Hiroshi Takasaki in SAGE Open Medicine
Supplemental Material
supplemental_2 – Supplemental material for Reporting on the level of validity and reliability of questionnaires measuring Katakori severity: A systematic review
Supplemental material, supplemental_2 for Reporting on the level of validity and reliability of questionnaires measuring Katakori severity: A systematic review by Kaori Aoki, Toby Hall and Hiroshi Takasaki in SAGE Open Medicine
Footnotes
Acknowledgements
The authors acknowledge Mr Georg Supp for triple checking the content of a study written in German.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
