The Rare Knowledge Mining Methodological Framework for the Development of Practice Guidelines and Knowledge Translation Tools for Rare Diseases

Abstract

Rare diseases bring on a heavy health, social and economic burden that impacts patients’ lives and puts pressure on the healthcare system. Furthermore, they are often associated with limited published studies to inform multidisciplinary clinical practice thus limiting evidence-based practice. Moreover, the development of knowledge translation products including clinical care guidelines are often very challenging based on the current available methodological frameworks relying mostly on critical appraisal of the published research evidence where randomized clinical trial design is considered as the gold standard. To overcome this barrier, we proposed the Rare Knowledge Mining Methodological Framework (RKMMF). The RKMMF is one possible answer to improve the development of knowledge translation products for rare diseases. This framework includes other sources of evidence including registry information and qualitative studies and the involvement of expert patients. This article documents the RKMMF structure and its application is exemplified through knowledge translation products developed for a neuromuscular population.

BACKGROUND

Currently, there are more than 6000 rare diseases reported worldwide [1]. In a recent review, Ferreira [2] reported that the definition of a rare disease differs according to the prevalence, varying from 1/1500 to 1/2500. Although each disease is rare, there were globally 263 to 446 million people affected in 2017 [3].

Most rare diseases have a genetic etiology, resulting in several chronic and progressive signs and symptoms. They lead to important disabilities that contribute to patients’ distress, reduce quality of life and shorten life expectancy [4]. Although the needs of all patients with chronic diseases are undeniable, those with rare diseases report among the worst experience in terms of medical care [5, 6]. They face multiple challenges regarding the diagnosis, including waiting many years before having a diagnosis, seeing several medical specialists to get a diagnosis and often receiving an incorrect diagnosis at first [7]. They also face lack of information about the disease, prognosis, and treatment options, including their availability and efficacy. They also lack access to coordinated care from tertiary or expert centers, rehabilitation or community services (occupational therapy, social work, and so on) [8]. The diagnostic delay and inappropriate treatments are frequently due to healthcare professionals’ limited knowledge about any rare disease [9].

Evidence-based practice (EBP) is a gold standard that is now included in practice frameworks of healthcare professional associations. Sackett [10] described it as “  ...   the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of patients ...  ”, which includes professional expertise, research evidence, and the patient’s preferences and values [10]. Clinical practice guidelines are largely used as knowledge translation (KT) products by healthcare professionals to achieve EBP. Several professional organizations (e.g., American Academy of Neurology [11], etc.) have proposed a process for developing clinical practice guidelines (CPG). They usually involve synthesizing and assessing the quality of scientific evidence available [12]. However, in rare diseases, a huge barrier is encountered in the production of CPG. Indeed, knowledge is often rare and robust scientific evidence such as those issued from randomized controlled trials (RCTs), almost inexistent [8, 13–15]. Rare diseases have gathered more attention in the past decades with international network initiatives, national plans for rare diseases in many countries, and patient advocacy organizations raising public awareness [8]. Despite these efforts, rare diseases are still much less studied than common diseases. As an example, the results of a PubMed search on strokes yields 18,166 papers in 2018 (between 01/01/2018 and 31/12/2018; human; last accessed December 2020), of which 288 are related to occupational therapy and 635 to physiotherapy. In comparison, 123 papers on myotonic dystrophy type 1 (the most common type of muscular dystrophy in adults) were retrieved with four articles in occupational therapy and five in physiotherapy. This highlights how healthcare professionals are likely to struggle to improve their knowledge or practice based solely on published research evidence with mostly low robustness studies [8]. Furthermore, high quality evidence (RTCs) is not the best design for several areas related to clinical practice. Some questions such as “How long will my child be able to walk?” are better answered by longitudinal or observational studies. In addition, the burden of the disease is often best captured by qualitative studies.

A systematic review on CPG development methodological framework for common and rare diseases (neuromuscular disorders as an example of rare disease) covering the 2002–2014 period have been previously performed by the Institut national d’excellence en santé et en services sociaux with the main author of the article (C.G.) [16]. Two search strategies were conducted to document methodology used in developing CPG for rare diseases: 1) identify CPG in neuromuscular disorders and 2) identify methodological framework to develop CPG for rare diseases. The two most common methods for CPG development were systematic literature review and experts’ consensus. Although CPG mostly used a comprehensive process to gather experts’ consensus, the methodology varied considerably notably regarding patients’ participation and external validation. The search for methodological framework for CPG development for rare diseases yields very few results as found by others [16, 17]. Only one methodological framework was extracted from grey literature. It was a generic methodological framework that was systematically used for rare diseases from a national agency (Haute Autorité de Santé) [18].

The systematic review highlighted the need to improve or adapt existing methodological frameworks to develop CPG for rare diseases. More specifically, it evidenced the needs 1) to acknowledge the experience, values and preferences of patients to develop clinical recommendations; 2) to recognize the value of other sources of knowledge including qualitative research to gather information from experts and patients in the context of the paucity of evidence; 3) to identify and propose consulting and participating methods to involve patients in the production of CPG; 4) to develop a method to assess the quality of evidence and strength of recommendations adapted for rare diseases. Furthermore, no methodological framework was found for developing other KT products such as video, leaflet, lay summary, etc.

This is supported by the recent work of Pai et al. (2019) who have tested extended development strategies (i.e. qualitative research, systematic observation forms, registry data, etc.) to develop CPG for rare diseases [17]. They concluded that a more refined methodology is needed to produce CPG that are systematic, dynamic, and responsive to the paucity of high quality evidence in rare diseases.

OBJECTIVE

This paper aims to present the Rare Knowledge Mining Methodological Framework (RKMMF), which has been developed and refined by our team since 2012 (Fig. 1 for the English version; Fig. 2 for the French version). It might be one possible answer to improve the development of knowledge translation products (KT products) in rare diseases, including CPG. This methodological framework differs from previous ones as it aims to provide a methodological framework to develop KT products including CPG, to provide a standardized method to include other sources of knowledge including qualitative research and registry information, and to improve the quality/reproducibility of the expert level of evidence. Besides, it is consistent with an EBP perspective, which includes professional and patient expertise [10].

Fig. 1

Summary of the phases and associated methods of the Rare Knowledge Mining Methological Framework— English version.

Fig. 2

Summary of the phases and associated methods of the Rare Knowledge Mining Methodological Framework— French version.

METHOD

Development of the rare knowledge mining methodological framework

The RKMMF follows the recent definition suggested by McMeekin and Wu (19) where a methodological framework is a tool to guide the developer through a sequence of steps to complete a procedure. It provides a standardized and thorough process to develop KT products for rare diseases and thrive to increase the quality, accessibility, applicability, and acceptability of expert knowledge in the context of EBP. The development of the RKMMF followed several suggestions of McMeekin and Wu (19) when developing a methodological framework.

Identification of evidence to inform the methodological framework: A systematic review of the literature of clinical guidelines development frameworks was conducted. One hundred twenty-four documents were consulted and assessed including the Cochrane Handbook for Systematic Reviews of Interventions [20], the 2017 Edition Clinical Practice Guideline Process Manual of the American Association of Neurology [21] and the methodological guide for the Haute Autorité de Santé (France) [18]. The Knowledge-to-Action Framework (KTA), the most widely used KT model [22] adopted by the Canadian Institutes of Health Research (CIHR) was also taken into consideration [23]. Finally, a survey about clinical practice improvement was also carried out in 2009 among allied healthcare professionals working in six specialized neuromuscular disease clinics (Québec, Canada) to better understand the challenges of accessing knowledge in the context of rare diseases [24]. The main concerns highlighted were difficulties (more likely inability) finding and accessing high quality and relevant research evidence and then applying it to clinical contexts. The most important source of information was often experienced colleagues. The survey highlighted the needs to provide specific strategies to access tacit knowledge held by experienced colleagues working in rare diseases and to share it widely, in particular when striving for quality and ethical use of knowledge.

Development of the methodological framework: It was an iterative process involving the development team. The framework was developed over a 5-yr period using case studies to refine it through several iterations.

Evaluate and refine the methodological framework: the case studies presented in this article were used to evaluate and refine the framework using the participants’ feedback to improve it.

The RKMMF includes a flow of specific steps based on well-documented methods (Risk of bias, Delphi method, focus groups, in-depth interviews, etc.) and processes to access scientific and expert knowledge in a structured and replicable way in the context of rare diseases. The RKMMF includes four phases described below (see Fig. 1): 1) Knowledge mining; 2) group sharing phases; 3) external validation; 4) knowledge translation. For each phase, the RKMMF is thereafter documented in relation to the Knowledge-to-Action Framework [22] items to ease translation. A complete description of each phase is provided including the development of the KT products when relevant and suggested methods and steps associated with each phase.

Phases of the rare knowledge mining methodological framework

A knowledge broker should ideally coordinate the development of KT products using the RKMMF. They are intermediaries between knowledge producers and end users within and across organizations or sectors. Their role is to support the achievement of the different phases of KT activities. [25, 26], which will be further described in the knowledge translation phase section. (Knowledge to Action Framework (KTA): Monitor knowledge use/Evaluate outcomes/Sustain knowledge use).

Preliminary phase — The entry point: Identify needs

Producing KT products involves important resources (human, financial, time). It is essential that the decision to develop such products meets well-defined needs and the expectations of end users. The need (topic identification) may come from knowledge sources (PUSH: researchers, experts) or from the field (PULL: end users) [27]. Discussions with both groups regarding the identified need are necessary to increase relevance, applicability, and quality of any products developed. Once the need is identified (content), the end-users must be consulted to ensure that the product (or products) will be applicable in their context (format and content). Iteration along the process is another key to make sure the products stay focused, relevant and timely. The needs (topics) must be identified before using the RKMMF and the process could include informal discussions, needs’ assessments, group discussions and formal requests from official organizations. (KTA: Identify problem/Determine the know-do-gap/Adapt knowledge to local context).

Phase 1: Knowledge mining

Knowledge mining involves two steps: 1) extracting theoretical and scientific knowledge, and 2) extracting expert knowledge (see a detailed description below). Both processes are closely intertwined and bidirectional. On the one hand, usefulness and applicability of theoretical and scientific knowledge is validated through experts and end users. On the other hand, it is necessary to “try” to find theoretical and scientific knowledge that supports the expert knowledge. The word “try” is used as rare diseases mean rare knowledge. Based on our experience of KT products development, it may be challenging to find the theoretical/scientific knowledge to validate a specific element brought by an expert.

Theoretical and scientific knowledge

The literature review should address three aspects. First, as rare diseases are often unknown to healthcare professionals, it is important to provide a short disease clinical presentation, whatever the specific topics of the KT product might be. Second, specific information regarding the KT product topics should be extracted from the literature review. Third, additional knowledge brought from the experts regarding clinical practice (specific issues such as how hypersomnolence and apathy can modulate clinical care in myotonic dystrophy type 1) should be further validated with theoretical and scientific knowledge when possible.

Most of the time, quality of the studies will be low, or risk of bias high, due to methodological issues (RCT are very rare, number of participants is often low, etc.). It might be necessary to include in the literature search diseases presenting similar symptoms and etiology, or to search for specific symptoms among more common diseases, as specific rare diseases are often poorly documented. (KTA: Identify, review, select knowledge/Knowledge inquiry/Knowledge synthesis).

How: The theoretical and scientific knowledge search should be conducted by using recognized methods to identify scientific literature, extract and analyze information, and assess quality and risk of bias of studies (e.g., systematic literature review, scoping review, grey literature). PICO terms (population, intervention, comparison, outcome) could be used. Adaptation is often necessary to use knowledge that is somewhat outside of the very specific PICO question but nonetheless very useful to develop the KT products. An example would be to extend the search to all recessive ataxia (population) instead of limiting it to recessive spastic ataxia of Charlevoix-Saguenay.

Expert knowledge

In rare diseases, expert knowledge might be the only source of information. To overcome the challenges related to the quality and risk of bias of expert knowledge, the RKMMF provides a set of methods to rigorously extract expert knowledge and then assess if it is supported by scientific evidence regarding the specific disease or other related diseases. Although it is difficult to identify the adequate number of experts, different consultation processes of the RKMMF involve a set of experts at each phase. This iterative process improves the quality of results by not relying on “just one expert opinion” (e.g., when you ask a question to a colleague in the context of clinical practice).

The first step is to define what is an expert, and then to identify who are the experts who could provide answers related to the KT topic. As this can be particularly challenging in the context of rare diseases due to the paucity of experts, we suggest a general definition of an expert as “someone expert in the content, recognized for competence and experience in the studied field.” [28] The objective is to access and extract all potentially relevant knowledge, “experts” often being the key. Therefore, several types of experts and methods should be considered and tailored accordingly. In the RKMMF, we specified five types of experts for any given KT topic. First, the disease-specific expert refers to healthcare professionals who are familiar with the disease, symptoms’ presentation and progression, and treatment options. They often work in specialized clinics and have a long-standing clinical practice with this population and should be included in the development of any KT products in their specific field of expertise. However, in the context of rare diseases, it may be difficult to have access to disease-specific experts. For example, finding a physiotherapist who developed expertise in fall prevention for a specific neuromuscular disorder is often a challenge. Therefore, the expert might be someone who has worked with a patient who has a specific rare disease only a few times in its practice. The second type of experts refers to professionals who have knowledge about the KT topic in one rare disease that can be applicable to another “similar” disease. The third type of experts is related to professional practice or discipline. They are solicited to generate discussion on the best practice based on their professional domain and to validate the applicability of the KT product in their respective care settings. Depending on the topic of the KT product, a fourth type of experts could be included, namely context-related experts. These could be teachers, employers, and community services that may offer very specific and contextual information. Fifth, patients and family members are also disease experts and they should be included in a meaningful way. Their experiential knowledge of living with the disease and its impacts represents an important source of information, especially in rare diseases. Their inclusion is important to ensure that KT products focus on their concerns, experiences and expectations. There is a growing interest to include patient-partners in research settings. Patient-partners are “people with a relevant disease who operate as active research team members on an equal basis with professional researchers, adding the benefit of their experiential knowledge to any phase of the project” [29]. They can provide a highly valuable contribution in research with varying levels of involvement such as decision-making, asking delicate questions in interviews, creating or modifying research tools intended to patients in an accessible manner, validating questionnaires’ content and participating to data interpretation. Accordingly, patient-partners should be integrated in the development and adaptation of KT products, their implication varying depending on the tasks.

How: There is a variety of recognized methods from qualitative research to extract expert knowledge: interviews [30, 31], content analysis, case studies, explicit interviews, task analysis/observation [32], Delphi [33–35], case scenarios, etc. The selected method should be tailored to the context, experts and available resources (human, time, material, funding). Based on our experience, in-depth interviews are a key method to extract such knowledge.

KT product: This phase leads to a first preliminary version of the KT product (content only) integrating theoretical, scientific and experts’ knowledge, that will be the base of the group sharing phase.

Phase 2: Group sharing

This phase aims to enhance and validate results from the Knowledge mining phase. It decreases biases based on one’s experience and increases validity, as a number of people are challenging each other’s ideas and try to reach a consensus. It also has a positive impact on the transparency and reliability of the whole process. Aside from reaching a consensus, group sharing fosters a collective thinking effort that draws from the diverse experience that group members bring to the table. During the activity, knowledge is shared, and different practices are compared. Each participant’s opinion is solicited and considered. Participants also learn from these interactions, bringing home new knowledge to build on and new ideas to reflect upon. This will enhance the next iteration of the group sharing activity.

Discussions should be directed towards validating several aspects including accuracy, plausibility, relevance, applicability, clarity, lack of information, unanswered question, etc. It is important to allow enough time for experts to interact. Real-time interactions between participants rapidly improved and enriched the outcome of group discussions. When the contact is indirect, it might be more difficult to get fruitful interactions, but participants might offer a more in-depth analysis as they have more time to think and react. (KTA: Identify, review, select knowledge/Knowledge inquiry/Knowledge tools and products/Tailoring knowledge/Select, tailor, implement interventions).

How: Results of the Knowledge mining phase should be available for the group sharing activity. The Group sharing phase may be done in synchronous or asynchronous modes, considering availability of experts, resources and context. Synchronous modalities involve participation at the same time, such as focus groups [34], TRIAGE working groups [28, 36] or special interest groups. Asynchronous modalities allow participants to interact at their own convenience and include collaborative web sites, online discussions, webinars, and Delphi method [33–35, 38]. Iterations of the group sharing activities are important to improve the overall quality of the KT product, as they increase the effects of group sharing. A minimum of two to three iterations is recommended, with an improved version of the KT product available before each iteration. Creating an active working group that combines synchronous and asynchronous modalities is one way to foster in-depth communication and fruitful iterations over time.

Direct interactions are appreciated by the members from our previous experience but may be time-consuming for certain experts who have limited available time. Therefore, it is essential for these interactions to be as effective as possible. The discussion leader should have previous experience to direct the dialogue towards the improvement or acceptance of the KT product content (and format, where applicable).

The success of this phase requires a transparent process and a competent discussion leader to allow participants to share their knowledge with openness and confidence. Whether an activity takes place in real time or not, the discussion leader should be neutral, open-minded and responsive (stimulate discussion based on participants’ inputs), and involve remote participants (videoconference, phone).

KT product: At this phase, results from the group sharing activity might be in plain text only or presented as a first complete version of the KT product, including style and format if participants are also the end users (time effective).

Phase 3: External validation

External Validation refers to the validation by experts and end users who have not participated in the development of the KT product. Ideally, it requires validation from different types of experts, as defined in Phase 1. It could potentially identify situations in which a referral to another healthcare professional is appropriate or scenarios in which working in partnership would lead to the best possible outcomes. Finally, external validation confirms that the KT product meets the end user’s needs (content and format) in order to optimize applicability.

Involving a wide range of experts’ and end users’ profiles for external validation leads to; a more diversified opinion and should reduce bias; maximizes coverage of a variety of situations, and reduces the likelihood of overlooking important considerations. (KTA: Knowledge inquiry/Knowledge synthesis/Tailoring knowledge/Review and select knowledge).

KT product: It takes the form of plain text (content only) or a “final” version (content, format and style) if external validation includes end users.

Phase 4: Knowledge translation

The Knowledge translation phase aims to finalize the adaptation of the content and format of KT products to the target end users, contexts, barriers and facilitators. As specified in Phase 1, the end users should be involved from the beginning. Involving them as soon as possible is time efficient, and it gives them a chance to confirm that the KT product meets their needs and expectations in terms of content, style, format, level of information, applicability, usefulness, vocabulary, and so on. A minimum of two iterations should be done to achieve a final KT product. (KTA: Knowledge tools and products/Adapt knowledge to local context/Assess barriers, facilitators to knowledge use/Select, tailor, implement interventions/Sustain knowledge use)

How: The final version of the KT product including content, format and style is achieved through individual or group sharing, as described in phase 1, 2 and 3. It also includes reality test, including monitoring and evaluation by end users directly on the field to ensure the KT product is relevant and adapted for the particular context of ones’ professional practice.

KT product: At this point, a first “final” version (content, style, format) should be achieved.

TRANSVERSE ACTIVITY: KNOWLEDGE BROKERING

The role of knowledge brokers is to reduce the burden on the experts by facilitating dialogue and to (literally) mine for rare knowledge, to identify and respond to a need, to facilitate the dialogue and interactions between stakeholders, and to ensure a simple form of collaboration during each stage for everyone involved. This is particularly important, as the best available evidence in rare diseases often comes from experts (clinicians, researchers, patients) who really want to share their expertise but who have limited time, capacities or resources to participate in a somewhat time-consuming activity. Hence, the knowledge broker should ensure that the process remains time restricted, efficient, smooth, and profitable for experts while they are actively involved. Knowledge brokers are also responsible for the transparency and quality of the process, including documentation of every step taken to develop a KT product, for example: Who are the experts of what? Why do they qualify as experts in this context? How, when and by whom was the knowledge mining conducted, and why were these methods selected? Knowledge brokers are also responsible for documentation, meaning that the methods used can be described in a transparent manner, which increases trust in the content or could allow for a process review if required. (KTA: Monitor knowledge use/Evaluate outcomes/Sustain knowledge use).

APPLICATION OF THE RARE KNOWLEDGE MINING METHODOLOGICAL FRAMEWORK

We detailed an example of the RKMMF application [39] (Fig. 3). As this is an adaptable approach, three other examples are provided for different neuromuscular diseases, healthcare professionals, or topics (Supplementary material): 1) General practitioner guideline-Myotonic dystrophy type 1 (Supplementary File 1) [40], 2) Practice guideline for occupational therapists (OTs)— Sexuality in adults living with neuromuscular disorders (NMDs) (Supplementary File 2) [41] 3) Practice guideline for occupational therapist- Charcot-Marie-Tooth diseases [42] (Supplementary File 3; see also [43]). Each scenario required defining a course of actions in relation to each phase since RKMMF does not provide one standard path. Instead, it helps to determine how to meet the knowledge needs in the best possible way given the context of rare diseases.

Fig. 3

Development of a practice guideline for occupational therapist in primary care for myotonic dystrophy type 1 using the RKMMF.

Myotonic Dystrophy Type 1 Practice Guideline for Occupational Therapist in Primary Care [39] (Fig. 3).

Preliminary phase— Entry point— Identify need

In April 2015, our team published a comprehensive practice guideline for occupational therapists (OT) working in rehabilitation centers or specialized neuromuscular disease clinics. Thirty OTs from primary or community care settings contacted our team to know if a simplified version was available, as the comprehensive guideline was not adapted to their practice. They had a hard time using the comprehensive guideline in primary care for many reasons, including the duration of appointments, the focus on very specific patient’s needs, the time-consuming process of searching in the comprehensive guideline, and the information that was not relevant to primary care practice. Therefore, the “IDENTIFY NEED” was to adapt and tailor the comprehensive guideline to answer the needs of OTs working in primary care/community practice.

Phase 1— Knowledge mining

In this specific example, the theoretical and scientific knowledge base was readily available from a previous activity. The knowledge mining rather focused on identifying relevant OT primary care activities regarding myotonic dystrophy type 1 (DM1), through two independent actions:

Relevant activities for OT in primary care: a list of potential OT activities derived from the Canadian Occupational Performance Measure (COPM) [44] and the Quebec Classification Disability Creation Process (QCDCP) [45] was sent to six primary care OTs. They indicated if the activities were relevant/neutral/irrelevant to their practice, without any reference to a specific diagnosis.

Priority of interventions in DM1: two OTs with a high level of expertise regarding DM1 analyzed the comprehensive guideline and identified elements to consider in primary care practice.

Based on the results from both groups, the team leader and knowledge broker extracted information from the comprehensive guideline and combined it to make it relevant for the primary care context. Result: a text-only evidence-based practice for DM1 tailored for primary/community care settings.

Phase 2— Group sharing

The group sharing involved four OTs from tertiary practice and experts of DM1, and two OTs from primary care with DM1 basic knowledge. At iteration 1, they commented the text, identifying red flags for that specific disease and the need to contextualize to improve OTs’ understanding of recommendations. At iteration 2, a new version was used in the focus group, whose goal was to discuss contradictions and inconsistencies and to improve contextualization and examples given. At iteration 3, these OTs analyzed the first “official” version of the short guideline (including style and format). This version took into account the suggestions from the focus group. Result: a practice guideline in DM1 adapted to the context of primary care OTs.

Phase 3: External validation of the comprehensive guideline

A range of healthcare professionals revised the comprehensive guideline based on the Formal Consensus Method [46]. This method includes experts from all disciplines of interest, and considers scope of practice, timely references to other professionals, interdisciplinary practice, and so on. Twenty-one professionals participated in this stage, either to revise the whole guideline or specific sections.

Phase 4: Knowledge translation

The validation of the official practice guideline was conducted through a focus group involving five primary care OTs (three with no previous knowledge of DM1 and two with advanced practice with DM1). Three asynchronous iterations followed to improve and tailor the guideline for primary care. OTs with no knowledge of DM1 considered the contextualization a key component of the short guidelines: “It helps to understand why this intervention is important with this patient, even if it differs from what I would normally do, or sometimes it helps to explain the situation to the patient.” Other key components were simple identification of information related to the patient’s need, “on the spot”; simplicity of information (bullet point); logic of organization (based on a standard clinical framework widely used in practice); and the one-page summary of challenges in DM1.

DISCUSSION-CONCLUSION

The scarcity of rare disease knowledge is a challenge for professionals who want to provide evidence-based healthcare to their patients who suffer from rare diseases. To fill this gap, we created a methodological framework to develop KT tools in the context of rare diseases and rare knowledge, including the development of practice guidelines. The RKMMF aims to increase the quality of recommendations based mainly on expert knowledge. Moreover, the model respects the philosophy underlying the widely used Knowledge-to-Action Framework [22], adapted for the particular context of rare diseases.

To our knowledge, the only recent initiative focusing on rare diseases and the development of KT products is the work of Pai et al. (2019) [17]. Based on previous recommendations from the RARE-Best practice group [40], the authors tested the RARE-Best practices Working Group’s pilot framework in three different CPG. They faced limitations with the application of the framework to integrate expert-based evidence in their practice guidelines. As a result, they did not take into consideration the main outcomes identified by their expert panel in the development of a practice guideline for the sickle cell disease due to scarcity of data [17]. The RKMF proposes a thorough approach to access the expert-based evidence to develop and adapt practice guidelines in rare knowledge context.

The RKMMF is a systematic, dynamic and responsive process that addresses the concern Pai et al. (2019) [17] regarding the need for a refined method to develop practice guidelines in rare diseases. The main strength of the RKMMF is its flexibility and adaptability, making it possible to implement it in multiple contexts. It offers the possibility to develop CPG but also many other KT products according to the end users’ needs. Another key element is that the process decreases the risk of bias based solely on expert opinion, as it includes tips on how to adapt literature search to the context and gives a central place to the group sharing practice. It therefore provides a valuable source of information in the absence of a rigorous experimental study to provide accurate recommendations. Because every step is documented in a transparent way throughout the process, it provides the end users to assess the potential strength of the recommendations given in the KT products. We must also highlight the central role of the knowledge broker, who supports the achievement of every phase, empowers stakeholders, fosters timely participation and brings into focus the quality of the process and its documentation.

The RKMMF also has some limitations. First, it does not include specifications about the implementation process of the KT products developed from the RKMMF, which can be particularly challenging when, for example, well-established CPG are changed. The process may also be very time consuming for the knowledge broker, thus requiring financial and human resources. In addition, as the RKMMF may involve several methods to access and validate knowledge, the knowledge broker must be highly qualified with a broad spectrum of competencies, a resource that could be hard to find. Finally, the RKMMF has never been tested with rare diseases other than neuromuscular disorders. Although it has been successfully tested with several neuromuscular disorders, it must be validated with other types of rare diseases.

In summary, the RKMMF could meet the needs of the medical/allied health professionals and research communities for a rigorous and flexible process that can be adapted to the specific context of the many existing rare diseases. Although the RKMMF has limitations, the process has proven to be an effective method to develop all kinds of KT products, as evidenced by the variety of KT products our team developed. Finally, it increases the quality of the expert level of evidence [47] and is consistent with the Knowledge-to-Action Framework [22].