Abstract
Background:
High-quality clinical practice guidelines are necessary for effective use of resources both at an individual patient- and national-level. Nordic clinical practice guidelines recommendations for orthotic treatment of knee osteoarthritis vary and little is known about their quality.
Objectives:
The aim of the study was to critically evaluate the quality of clinical practice guidelines in orthotic management of knee osteoarthritis in the Nordic countries.
Study Design:
Systematic review.
Methods:
Four national clinical practice guidelines for treatment of knee osteoarthritis were assessed for methodological rigour and transparency by four independent assessors using the AGREE II instrument. Summary domain scores and inter-rater agreement (Kendall’s W) were calculated.
Results:
Domain scores indicate that many guidelines have not sufficiently addressed stakeholder involvement (average score: 55%), applicability (20%) and editorial independence (33%) in the development process. Inter-rater agreement for assessors indicated ‘good’ agreement for clinical practice guidelines from Finland, Norway and Sweden (W = 0.653, p < 0.001; W = 0.512, p = 0.003 and W = 0.532, p = 0.002, respectively) and ‘strong’ agreement for the clinical practice guideline from Denmark (W = 0.800, p < 0.001).
Conclusion:
Quality of clinical practice guidelines for orthotic treatment of knee osteoarthritis in the Nordic region is variable. Future guideline development should focus on improving methodology by involving relevant stakeholders (e.g. certified prosthetist/orthotists (CPOs)), specifying conflicts of interest and providing guidance for implementation.
Clinical relevance
The current review suggests that, for the Nordic region, there are areas of improvement which can be addressed, which ensure clinical practice guidelines are developed under stringent conditions and based on sound methods. These improvements would ensure knee osteoarthritis patients are receiving orthotic interventions based on appropriate guidance from published guidelines.
Background
Osteoarthritis (OA) is one of the most common causes of long-term disability with a prevalence estimated at 5% of the global population, 1 with knee OA representing the largest proportion at approximately 4%. The prevalence of hip and knee OA in the Nordic region has increased by 43% between 1990 and 2015. 2 The age-standardized prevalence (% (95% CI)) of OA for men and women in the Nordic region in 2015 was 8.4% (6.6–10.2) and 8.1% (6.3–9.8), respectively, with high prevalence in Sweden (11.9% (8.3–15.8)) and low in Finland (4.8% (1.0–8.7)). Although there remain gaps in understanding of the pathogenesis of the disease, it is generally accepted that biomechanical factors (knee alignment and/or external moments acting upon the knee) play a role in the initiation 3 and disease progression of OA4–6, irrespective of sex or ethnicity. 4
Recent systematic reviews have highlighted a lack of consensus in the efficacy of the use of foot and knee orthoses in the treatment of knee OA.7–13 These orthoses’ clinical objectivities are to reduce or stabilize deformity of the knee to reduce pain and improve functioning of the person. 14 Some researchers have concluded that the use of a prefabricated knee orthosis can reduce pain7,8,10–12,15 and can reduce the external knee adduction moment,9,11 while other researchers have concluded that the evidence is inconclusive for the effect of knee orthoses on pain, stiffness, function and quality of life.13,16 Weak evidence suggests that lateral wedged foot orthoses compared to no orthoses reduce pain. 13
This lack of consensus on the efficacy of orthotic treatment of OA of the knee seems to be further synthesized with varying recommendations in internationally published clinical practice guidelines (CPGs)17–20 and in the national CPGs of the Nordic countries. One country (Finland) recommends the use of knee orthoses, 21 two countries (Sweden and Denmark) recommend not using knee orthoses as a treatment option22,23 and the fourth country (Norway) does not mention knee orthoses. 24 Nordic consensus is also lacking in the recommendation for provision of foot orthoses in the treatment of knee OA, with the Finnish and Danish recommendations positive at least to certain types of foot orthoses,21,22 the Swedish recommendations discouraging use of foot orthoses and the Norwegian not mentioning foot orthoses as a treatment option.23,24 Furthermore, the CPGs differ in their purpose, conclusions, recommendation and intended users surrounding orthotic treatment of knee OA. These variations in CPGs are not reasonably due to differences in population or healthcare structure. The similarities between the Nordic countries, including expansive social safety net and relatively high tax rates to fund social programmes such as national healthcare systems, make analyses of these countries particularly interesting given the similar baseline conditions. Their geographic vicinity and shared cultural history establish a common platform in which meaningful comparisons can be made for many societal factors, including health care. The national healthcare systems of the Nordic countries share similar structures and challenges including the following: being publicly funded healthcare systems with national levels equal to or above the mean level of general domestic product for countries that report this data 25 and an aging population with increased demands on funding resources. 26 As the guidelines from the Nordic countries are often published in the official language(s) of the country of origin, they are not included in systematic reviews, which often have publication in English as an inclusion criterion. This means that these CPGs have not been subject to the types of external critical review which are necessary to provide the highest quality possible.
The aim of this systematic review is to critically evaluate the quality of CPGs in the orthotic management of knee OA in the Nordic countries. A second aim is to provide recommendations on areas of improvement in the quality of these CPGs.
Methods
Literature search
The literature search was conducted in the Medline and CINAHL databases during January 2018. The utilized Medline search string was ((MH ‘Practice Guideline’) OR (MH ‘Practice Guidelines as Topic’) OR clinical practice guideline*) AND ((MH ‘Osteoarthritis’) OR (arthritis OR osteoarthritis OR rheumatoid arthritis)). In addition to the database searches, contact was made with groups representing national orthopaedic associations, national prosthetic and orthotic associations, national boards of health and welfare and national patient groups for OA and rheumatoid arthritis.
The search was limited to the five official languages of the Nordic countries (Finnish, Swedish, Norwegian, Danish and Icelandic). No restrictions were placed on publication date (<31 December 2017). Those CPGs deemed appropriate for inclusion were all guideline documents – irrespective of source (evidence-based research, consensus documents or expert opinion) – providing recommendations regarding orthotic treatment of knee OA, which composed in whole, or part, of the target population. Some CPGs were written as a single document (Denmark and Norway), while others were web-based with the contents having multiple online locations (Finland and Sweden). A criterion for supplementary documents to be included, such as user manuals, needs to be referred to in the main CPG document and not as part of historical versions of the document. User manuals were required to be mandated components of the guideline development process for their inclusion of the quality assessment. Following identification of potential guidelines, four assessors with certification as CPOs (D.F.R., U.H.T., G.J. and E.N.) (1) completed a joint introductory educational module, as suggested by the AGREE II developers, as a preliminary calibration of the assessors, and then (2) independently determined the eligibility of the guidelines for inclusion (Figure 1). Group consensus determined eligibility in all cases.

Flowchart outlining CPGs inclusion selection process.
Analysis
Data extraction
The quality of the individual CPGs was evaluated utilizing the AGREE II instrument, 27 which was accessed in February 2018. 28 The AGREE II instrument has been used extensively elsewhere and functions as a quality assessment tool of CPGs. The AGREE II employs a 7-point Likert-type scale ranging from 1 (‘strongly disagree’) to 7 (‘strongly agree’) to assess quality in six domains: Scope and purpose, Stakeholder involvement, Rigour of development, Clarity of presentation, Applicability and Editorial independence. 29 Prior to data extraction, the four assessors met and participated in a mandatory joint educational session as per the AGREE II instruments recommendations.30,31
Domain scores were calculated and presented as outlined in the AGREE II guidelines 28 utilizing individual item-scores. Strength of the domain scores were interpreted using previously established levels,32–34 where a score greater than 60% is established as being effectively addressed. If four or more of the domain scores were above 60%, the guidelines were classified as highly recommended. 34
Statistical analysis
As a Shapiro–Wilks test indicated violations of normality, statistical analyses were conducted using non-parametric methods. Inter-rater agreement was assessed using Kendall’s coefficient of concordance (W) based on 25 items (23 individual items and the 2 overall items for overall score and recommendation) in the AGREE II instrument from each of the four raters for each of the national guidelines. Kendall’s W produces a coefficient between 0 and 1, where 0 indicates no agreement and 1 indicates total agreement.
Results
CPG summary
Four individual guidelines from Finland, Sweden, Denmark and Norway were determined to fit the inclusion criteria and subsequently included in the analyses. No guideline from Iceland was found. Summary characteristics are included as Table 1. The Finnish guideline is directed towards professionals in primary or specialist healthcare involved in the treatment of individuals who are affected by OA of the hip or knee. 21 The Swedish and Danish guidelines suggest the intended user of the guidelines can include individual providers, professionals, patients and organizational managers,22,58 and in contrast, the Norwegian guideline is intended as a support document for organizational resource distribution within healthcare. 24
Summary characteristics of the reviewed CPGs.
CPG: clinical practice guideline; OA: osteoarthritis.
‘Format’ refers to the distribution format of the CPG, either as a printed document or its availability as an online resource. ‘# Web Pages’ and ‘# Printed Documents’ refer to the total number of individual locations or documents utilized in the review. ‘Year of Publication’ refers to the most recent update of the reviewed CPG. ‘Intended User’ is the explicitly stated target group for the CPG. ‘Orthotic Interventions Outlined’ is the classification if orthosis was included in the CPG (these included only foot orthoses and/or knee orthoses). ‘Main recommendations’ are the explicitly expressed recommendations of the CPG regarding provision of orthotic treatment of OA of the knee. The recommendations are translated by the authors and the original recommendation in the applicable language is available in the CPGs (see references). ‘References utilized in recommendation’ are the declared references which were utilized by the guideline developers to establish the recommendation of the guideline.
CPG quality – domain scores
All assessors were able to complete the review of all documents with no missing or omitted data. Three of the guidelines had domain scores which place them in the highly recommended category, with at least four domains having scores greater than 60% (Finland, Sweden and Denmark) (Table 2). 34 The remaining guideline (Norway) had summary domain scores which placed it in the ‘not recommended’ category, with more than four domains less than 30%. 34 Across the four CPGs, the Scope and purpose, Rigour of development and Clarity of presentation domains received the highest scores (average score range: 62%–69%), while the Stakeholder involvement, Applicability and Editorial independence domains received lower scores (average score range: 20%–55%).
Summary for the domain scores (%) of the AGREE II instrument for each guideline.
⩾60% indicates that a domain is highly recommended and effectively addressed. Scores <60% (‘not recommended’) 34 are written in bold text.
Scope and purpose
This domain had the highest average score, 69%, with only the Norwegian guideline not reaching the threshold of 60%. Except for the Norwegian guideline, the CPG’s were clear on the objectives of the guideline, health questions covered and target population.
Stakeholder involvement
The average score in this domain was 55%, and two out of four CPGs reached 60% or more (Finland and Denmark). All working groups authoring the guidelines included orthopaedic surgeons and three groups included physiotherapists (Sweden, Denmark and Finland). Some working groups also included other professionals, such as physicians of different specialities, nurses and health economists, but no working group included a CPO. The involvement of patient groups was generally low and their role unclear.
Rigour of development
This domain had the second highest average score, 66%, and all guidelines but the Norwegian reached the 60% cut-off. The Norwegian guideline received 12% in this domain because it did not account for the methodology used, such as methods for searching and selecting evidence and methods for formulating recommendations.
Clarity of presentation
This domain received an average score of 62% and three of the guidelines reached 60% (Finland, Sweden and Denmark). The Norwegian guideline achieved 17% because it did not provide unambiguous recommendations for treatment of knee OA and did not account for different treatment options for managing the condition.
Applicability
This domain had the lowest average score, 20%, and only the Swedish guideline reached the 60% cut-off. In general, the guidelines did not describe facilitators and barriers to implement the CPG nor did the CPGs give advice or tools for putting the recommendations into practice. Consideration for resource implications of applying the recommendations or criteria for monitoring and auditing were lacking.
Editorial independence
This domain had the second lowest average score, 33%, and only the Finnish CPG reached the 60% cut-off. In general, it was unclear about potential influence of the funding body and competing interest of the members of the working groups.
Inter-rater agreement
The four assessors agreed in their assessments for each of the CPG (Finland: W = 0.653, p < 0.001; Sweden: W = 0.532, p = 0.002; Denmark: W = 0.800, p < 0.001; and Norway: W = 0.512, p = 0.003). These results indicate ‘good’ agreement for the CPG from Finland, Sweden and Norway and ‘strong’ agreement for the CPG from Denmark. 59
Interpretation of Findings
The primary aim of this systematic review was to critically evaluate the quality of CPGs in the orthotic management of knee OA in the Nordic countries. A second aim was to provide recommendations on areas of improvement in the quality of these clinical guidelines. The results suggest a variation in the quality of clinical guidelines published in the Nordic region. There are areas of insufficient quality common to many of the reviewed guidelines and simple interventions, as stated below, that could greatly improve their quality and usefulness to intended users.
None of the published guidelines attained satisfactory domain scores (⩾60%) 34 for all domains. The domains which most frequently scored <60% were Applicability and Editorial independence. As the Applicability domain relates to the users’ ability to implement the recommendations, there is a need for authors of CPGs to facilitate this process by discussing potential facilitators and barriers to implement the CPG, providing tools and advice for how to put the CPG into practice and discussing resource implications as limitations of resources may be a common barrier to implementing CPGs. 60 In the Editorial independence domain, simple clarifications in the text of the guideline can adequately address the two items in this domain: first, how the funder has influenced the guideline (if at all) and second, clarify how competing interests have been addressed in the development group. In the first point, a reader would like to assume that the funder has not unduly influenced the results, although without stating this explicitly in the text, it is not possible to assure that the developers have indeed addressed this important aspect of guideline development.
An additional domain which was insufficiently addressed in two of the guidelines (Sweden and Norway) was Stakeholder involvement. This domain deals with the relevant groups (including – but not limited to – clinicians and patients) being included in the development process and review of the published guidelines. The authors note that none of the development groups of the reviewed guidelines included a prosthetist/orthotist. These individuals are highly involved with the clinical treatment of these patients and, as such, it is methodologically unsound to omit this professional group from the development process.
An additional point of interest is that although not all CPGs reviewed have the same intended users, there are users which were common to a majority of the CPGs reviewed. ‘Professionals’ is stated in three (Finland, Sweden and Denmark) and ‘patients’ in two (Sweden and Denmark) of the CPGs reviewed. As these CPGs have common intended users and the body of literature available to base the conclusions of the guidelines is similar (publications 2012–2014 for Finland, Sweden and Denmark), it is reasonable to expect that there would be similar recommendations for the implementation of orthotic interventions in the treatment of OA of the knee. As this is not the case, it is curious to consider the reasons for this variation. One reason could be that the studies that provided the basis for the recommendations only partially overlapped between the CPGs (Table 1). Another reason could be that when quality of evidence is low, as is the case for orthotic treatment of knee OA, the likelihood that developers arrive at dissimilar conclusions is greater.
There are certain limitations that should be acknowledged. This review has assessed the quality of the clinical guidelines, but direct comparison between the guidelines is not possible. The reason for this is that they are dependent of national conditions and differ in scope, purpose and intended users. Each national guideline has been developed to address these goals which are specific to each national health care system. Given the scope of this study, it would therefore be inappropriate for the authors of this review to say whether one guideline is better than any other. Although each of these guidelines made use of a user manual, the publication of the manual did not guarantee its inclusion as review material. The manual needs to be a required component of the guideline development. For three of the guidelines, this requisite was fulfilled. The exception was Norway. The user manual published was not mandated to be followed by developers, and it was a suggested reference for development. As the authors of this review cannot establish if this manual was implemented during development, it was excluded from analysis, thus the lower score in the assessment of the Norwegian guideline. As this review was focused on orthotic interventions for knee OA, it is not possible to generalize these results to additional guidelines published by national healthcare institutions in each of the countries, although the current results indicate that for this specific subject area, there is significant area of improvement available to developers. One should also be cognisant of the fact that a low score with the AGREE instrument does not, in itself, ensure low quality of the CPG. It may simply be a measure of a developers poor reporting. Although it is impossible to establish this, developers should take all efforts to ensure necessary steps in development are not only taken, but also documented, to provide transparency of the development process. In addition, a high score with the AGREE instrument does not guarantee high quality of the CPG, even if the CPGs are developed and reported appropriately, as the strength of the evidence underlying the recommendations may be weak. As such, further diligence is needed in identifying areas where evidence is weak or inconclusive in the orthotic treatment of knee OA to assure that potentially limited research resources are focused on the most relevant areas. Only with both high-level quality research forming the basis for recommendations and high-level quality of reporting of the development process, users of CPGs can be assured they are making use of high-quality, relevant CPGs for orthotic treatment of knee OA.
This review has indicated that many common methodological steps which are suggested to maintain rigorous, high-quality CPGs are omitted. These steps can be easily implemented and stand to improve the quality to which users expect in a tool they will utilize and for patients who stand to benefit the most from any improvements. These improvements could ensure that guidelines, developed in the Nordic countries, are always developed under stringent conditions and based on sound methods, providing an evidence base for healthcare interventions and policies affecting patients.
Conclusion
There is variable quality in the CPGs published for orthotic interventions for treatment of knee OA in the Nordic region. Of the reviewed CPGs, low quality was found in the following domains: Applicability, Editorial independence and Stakeholder involvement. Methodological improvements should be made during the development process, which will improve the published CPGs. In addition, more high-quality studies on orthotic treatment of knee OA are needed to provide a stronger evidence base for CPGs.
Footnotes
Acknowledgements
The authors wish to thank Göran Sigblad and Claes Esplund.
Author contributions
D.F.R., G.J., U.H.T., E.N. and A.J. conceived and designed the study; D.F.R. performed the literature search; D.F.R., G.J., U.H.T. and E.N. assessed the guidelines; D.F.R. performed the statistical analysis; and D.F.R., G.J., U.H.T., E.N. and A.J. wrote the paper. All the authors have approved the final version of the manuscript.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship and/or publication of this article: G.J. is a consultant for Novo Nordisk; A.J. is an employee of Össur Clinics Scandinavia; and E.N. is an employee of Ottobock, Medical Care. The remaining authors have no conflicts of interest to disclose.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: The study was partially funded by the Swedish Prosthetics & Orthotics Industry Association (Ortopedtekniska Branschrådet) and Research Enabling an Active Life (REAL) at Örebro University.
