Abstract
BACKGROUND:
In 2017, 24 European Reference Networks (ERNs) have been recognized in order to improve rare diseases’ management. However, a lack of practical information persists regarding the most adequate ways to fulfil ERNs’ quality criteria.
OBJECTIVE:
Identify the most efficient strategies to improve ERNs’ quality.
METHODS:
67 strategies, related to 8 activity domains (Quality Management System, healthcare coordination, clinicians’ training, inter-ERNs’ members collaborations, clinical data management, ERNs’ external collaborations/supports, information provided to patients, and ERNs’ financing) were selected from peer-reviewed scientific publications. Their usefulness, feasibility and adequacy was evaluated by 17 ERNs-related clinicians and 10 Belgian health policy makers.
RESULTS:
Results showed that the most efficient strategies are the participation of quality controls, satisfaction surveys among patients and external physicians, diffusion of information about the hospitals’ research activities, specific trainings addressed to clinicians, exchange of medical samples between hospitals, and development of adequate clinical databases. The poor practical feasibility of 7 highly-useful strategies was also highlighted, such as major clinicians’ concerns regarding legal/ethical frameworks for the reimbursement of cross-country care and exchange of patients’ data.
CONCLUSIONS:
This study enabled identification of efficient strategies to improve and validate ERNs’ quality, expertise and functioning, as well as weaknesses that should be addressed.
Keywords
Abbreviations
External Quality Assessment
European Reference Network
Next Generation Sequencing
Odds Ratio
Research And Development Corporation/University of California, Los Angeles
Introduction
Hospitals have to continuously evaluate their strengths and weaknesses in order to validate their expertise, improve healthcare’s quality and fulfill the health policy requirements [1–4]. This continuous improvement is supported by national and international political authorities. For instance, in 2016, the European Commission opened a Call for application for the recognition of “European Reference Networks (ERNs)”. These networks have to be composed of at least 10 Members (healthcare professionals), across 8 European Member States [5]. Their missions are to provide highly specialized healthcare for rare or complex diseases (diagnosis, treatment, follow-up), and to share knowledge [5–7]. Twenty-four ERNs, involving more than 300 hospitals, were officially recognized in March 2017 [8]. Their selection was the continuation of a strategy that started in 2007 with the recognition and funding of a limited number of pilot-ERNs [9]. The European Commission frequently publishes recommendations in order to harmonize ERNs’ duties and quality/recognition criteria (cf. Table 1 [6, 7]).
Operational criteria and conditions for the recognition of European Reference Networks
Operational criteria and conditions for the recognition of European Reference Networks
These criteria and conditions were initially published in the COMMISSION DELEGATED DECISION of 10 March 2014 (2014/286/EU) [22].
However, in-depth analysis reveals that some of these criteria remain vague and lack practical information in order to advise hospitals and clinicians on the most adequate ways to adapt their activities in order to fulfill the recognition criteria defined for ERN’s members and, thereby, to be able to valorize their quality and expertise [10].
In this context, we conducted a literature screening of peer-reviewed scientific publications in order to identify possible interesting strategies for the improvement of various processes within the Quality Management System, healthcare coordination, clinicians’ training, collaborations between ERNs’ members and data management within reference hospitals. Then we conducted two surveys: a first one in order to ask the opinions of experts of the field regarding the usefulness and feasibility of inventoried strategies. The second one aimed to investigate the adequacy of collaborations between ERNs and external actors, of different types of logistic supports for ERNs, of information provided to patients, and several sources of funding for the activities of ERNs’ members.
In a first instance, a screening of the scientific literature was performed in order to identify potentially efficient strategies. This initial step took place between December 2015 and March 2016, and was performed by a single investigator. Strategies that were retained for the second step of the study were only selected from peer-reviewed scientific papers that could demonstrate their relevance and benefits in well-defined practical applications, frameworks or models (for more details, please see the bibliographic references mentioned in Table 2).
Sixty-seven strategies or aspects investigated during the study
Sixty-seven strategies or aspects investigated during the study
A short description and/or definition of highly specific or complex activities has been mentioned in the third column. The fourth column contains the bibliographic sources that support the relevance of each strategy/aspect investigated and provide practical examples of the use of these strategy. Abbreviations: DOC: Document; EQA: External Quality Assessment; ERN: European Reference Network; SOP: Standard Operating Procedure.
Secondly, the opinions of healthcare professionals about inventoried strategies were collected in May 2016 using two surveys. On one hand, the opinions of 17 physicians related to European academic hospitals, who applied to the European call for the selection of ERN’s (2016) and who thereafter obtained a recognition as members of different ERN’s in 2017, were collected in order to evaluate the strategies’ usefulness and practical feasibility, as well as the current adequacy of different types of ERNs’ collaborations and logistic supports. Categorical variables (usefulness [useful; unuseful; no opinion]; practical feasibility [high or normal; poor; no opinion]) were used for that purpose and the results expressed in rates (percentages) of participants who reported each category. In a secondary analysis, strategies’ usefulness and feasibility were investigated for participants stratified based on a prior membership versus no prior membership to pilot-ERNs. On the other hand, a second panel, composed of 10 physicians related to different Belgian health policy institutions (Federal Public Service Health, Food Chain Safety and Environment; Belgian Observatory of chronic diseases; National Institute for Health and Disability Insurance; Fund for Rare Diseases and Orphan Drugs; RaDiOrg.be), was requested to provide an opinion regarding the adequacy (using different categorical variables [excellent; moderately good; insufficient; woefully insufficient or non-existent; no opinion]) of different types of information provided to patients, and the most-adapted sources of funding for ERN members’ activities.
Each survey was accompanied by a cover letter specifying the context and objectives of the study, as well as ensuring that collected data would be stored and treated in complete confidentiality and anonymity. Participants had over one month to complete the surveys.
Figure SM1 (supplementary material) illustrates the study design.
Strategies’ identification
Sixty-seven potentially efficient strategies/aspects (summarized in Table 2) were identified during the literature screening. Aspects related to the same activities were grouped in order to define eight areas for improvement: (i) Quality Management System, (ii) healthcare coordination, (iii) clinicians’ training, (iv) inter-ERNs’ members collaborations, (v) clinical data management, (vi) ERNs’ external collaborations and supports, (vii) information provided to patients, (viii) financing of ERN members’ activities.
Strategies’ usefulness and feasibility
Table 3 shows the opinions of the physicians coming from ERNs (for the global population [n = 17], and for each stratum of physicians [prior membership (n = 8) versus no prior membership to pilot-ERNs (n = 9)]) about the usefulness and practical feasibility of strategies investigated in order to improve the Quality Management System, healthcare coordination, the training of clinicians, collaborations between ERN’s members, and management of different types of clinical data (cf. the five first areas investigated in this study and mentioned in Table 2). The proportion (%) of participants who reported each strategy as “useful” is shown in the second column. The third column mentions the rates of participants (%) who reported the strategies as “highly” or “normally” feasible.
Reported usefulness and feasibility
Reported usefulness and feasibility
The second column of Table 3 mentions the global proportion (%) of participants who reported each strategy as useful. The third column mentions the global proportion (%) of participants who reported each strategy as highly or normally easy to develop. *: Results are presented as global mean scores for all participants (without distinction between those having a prior membership to pilot ERNs and those who don’t) and as means scores after participants’ stratification (based on a prior membership versus no prior membership to pilot ERNs). Each mean score is accompanied by the number of participants who reported the categories “useful” or “high or normal feasibility” on the total number of respondents (between brackets). Abbreviations: ERN: European Reference Network; EQA: External Quality Assessment; NGS: Next Generation Sequencing.
Regarding the improvement of the Quality Management System, results show that 81 to 88% of participants reported the participation to ring tests (please see the definition mentioned in Table 2) and External Quality Assessment programs (EQAs), the establishment of multidisciplinary quality manuals focused on specific groups of diseases, and the financing of quality coordinators as useful. However, participants’ opinions about the feasibility of these strategies were globally more negative, especially for the two latest (<60% of participants reported a high or normal feasibility). Of note, a trend suggests that clinicians with a prior membership to pilot-ERNs reported the participation to ring tests as more difficult to develop versus those without previous membership.
Concerning the improvement of healthcare coordination, the (i) use of surveys in order to evaluate patients’ satisfaction and understanding of services/information provided by university hospitals, (ii) financing of healthcare coordinators to ensure patients’ follow-up between medical departments or between the hospital and the general practitioner/external hospitals, (iii) participation to weekly multidisciplinary discussions of clinical cases, (iv) hospitals’ reorganization in “technical poles” (e.g. laboratories, medical imagery, etc.), and (v) the optimization of patients’ flows seem to be efficient (reported as useful by 64–100% of participants and as highly or normally easy to develop by 54–100% of participants). The other strategies (hospital’s reorganization in “poles of care” [by diseases’ types]; financing of coordinators to ensure the communication between the hospital and patients’ organizations) were considered as useful by the majority (56–86%) of participants. However, their implementation seems to be fastidious (high/normal feasibility reported by ≤40% of participants). Of note, participants with a prior membership to pilot-ERN’s reported a lower usefulness for the financing of communication coordinators than those without prior membership.
Regarding clinicians’ training, the study enabled to assess the efficiency of (i) the dissemination of information about the hospital’s research activities, (ii) an easier access to medico-scientific databases/guidelines/informatics tools, (iii) trainings focused on diagnostic/therapeutic guidelines and genetic counseling, (iv) the financing of the participation to international scientific conferences, (v) the development of healthcare algorithms for some specific diseases (usefulness and high/normal feasibility reported by >60% of participants). The financing of clinicians long-term stays in foreign hospitals seem to be less efficient and only 54% of all participants reported it as highly/normally feasible.
As regards the improvement of inter-ERNs’ members collaborations, the exchange of medical samples is very efficient (reported as useful and feasible by respectively 93% and 79% of participants). To a lesser extent, the development of satisfaction surveys among external clinicians who refer patients to the hospital was also reported as useful and feasible by respectively 71% and 83% of participants. Some strategies (exchange of medical/epidemiological data, collaborative research projects, sharing patients, telemedicine) have been reported as useful by the 80 to 100% of participants but quite fastidious to perform. Lastly, the pooling of medico-scientific equipment and funding were reported as poorly useful and difficult to perform. Of note, a higher feasibility for sharing patients between hospitals was expressed by physicians with a prior membership to pilot-ERNs compared to those without a prior membership.
Finally, the development of a good management and storage system for the different types of patients’ data mentioned in Table 3 was considered as useful by 100% of participants. However, physicians with a prior membership to pilot-ERNs seem to be more pessimistic concerning the ease of management of patients and family medical histories, reports of medical consultations/exams, and of analyses of clinical pathology and anatomical pathology. Regarding the feasibility management of results of Next Generation Sequencing (NGS) analyses, we have to notice that a high rate (47%) of participants expressed no opinion. However, even if no statistically significant correlation could be obtained, it seems that the management of results of NGS analyses is considered as easier by physicians coming from pilot-ERNs vs. those without any prior membership. Curiously, the financing of data managers was only considered as useful and highly/normally feasible by respectively 71 and 65% of all participants, with participants with a prior membership to pilot-ERN’s reporting a lower usefulness and feasibility than those without prior membership to ERNs. This may partly be explained by the fact that the current data management issues aren’t related to a lack of staff dedicated to this activity but maybe more to inadequate tools and databases.
Figure 1 summarizes the global efficiency of the strategies based on their mean rates of usefulness (x axis) and high/normal practical feasibility (y axis) reported in Table 3. It appears that the strategies selected from the literature in order to improve the Quality Management System and the management of clinical data are really useful but their feasibility is heterogeneous. Two strategies presenting a low feasibility (<50%) but high usefulness (>90%) should receive more attention and means: the financing of quality coordinators and the development of adequate storage platforms, especially for NGS data.
Efficiency of the strategies selected from the literature based on their combined usefulness and feasibility. The global efficiency of strategies selected from peer-reviewed scientific publications in order to improve the Quality Management System, management of clinical data, healthcare coordination, collaborations between ERNs’ members, and healthcare professionals’ training has been determined based on their means scores of feasibility (y axis) and usefulness (x axis) reported in (Table 3). Strategies are plotted by activity domains using open circle symbols. The dashed black lines represent 50% of participants reporting a high/normal feasibility and usefulness. Stars within the circles represent strategies for which the reported feasibility is higher than the reported usefulness (data plotted above the feasibility/usefulness correlation line [solid black line; correlation coefficient = 1]).
Regarding the strategies proposed to improve the healthcare coordination and collaboration between ERNs’ members, we observe a broad dispersion of the efficiency profiles. This tends to suggest that literature’s proposals are not yet totally optimal, maybe because of the novelty of the activities and proposals for improvements in these domains.
Finally, even if all strategies selected to improve healthcare professionals’ training have been reported as useful and quite feasible, a gradient of efficiency is observed among them and helps to privilege the most efficient ones.
Of note, for some useful strategies, the rate of reported high/normal feasibility is higher than the rate of reported usefulness (*; data plotted above the feasibility/usefulness correlation line). This suggests that these strategies (development of satisfaction surveys among patients and external collaborators, hospitals’ organization in technical poles, development of diagnostic/therapeutic guidelines, clinicians participation to international scientific conferences and a broader diffusion of information about hospitals’ research activities in order to valorize their expertise) might be priority areas.
Figure 2 shows that 75 to 100% of participants reported excellent or good collaborations between ERNs’ members and the ERNs’ coordinator and between ERNs and patients’ organizations, second-line hospitals, research laboratories and pharmaceutical companies. It seems that the adequacy of the interactions with the European Commission and national healthcare authorities remain quite heterogeneous (excellent/good [50 to 62.5%]; significantly insufficient/non-existent [37.5%]; no opinion [0 to 12.5%]).
Evaluation of the adequacy of different types of collaborations and supports of European Reference Networks (ERNs). The horizontal stacked bar charts represent the rates (expressed in percentages) of participants who reported each of the categories used to evaluate the adequacy of different types of collaborations and supports. The level of adequacy was evaluated in a categorical way (excellent [white]; moderately good [light grey]; insufficient [grey hatched area]; woefully insufficient or non-existent [black]; no opinion [white hatched area]).
Concerning logistic support, the sufficiency of medical staff was reported as excellent/good by 75% of participants. However, the time that clinicians related to ERNs can dedicate to their research activities, the European legal frameworks for the exchanges of clinical data, reimbursement of cross-country care and recognition of clinicians’ expertise, as well as the funding of clinicians’ additional tasks related to their membership to ERNs were reported as significantly insufficient by 37.5 to 75% of participants.
Patients’ satisfaction considerably varies depending on the type of information received at the hospital. As shown in (Fig. 3A), information about diseases’ symptoms/evolution, drugs-related side effects/adverse events, available psychological support and interpretation of medical data were considered as good or excellent by >70% of participants. Information about compassionate drug use and existing patients’ organizations should be improved (reported as good by <60% of participants). Intriguingly enough, results showed a significant insufficiency of information about ethical or financial aspects, namely the exchange/retention of patients’ data, the reimbursement of cross-country care, and associated legal aspects (reported as insufficient by 42 to 86% of participants).
Evaluation of the adequacy of information provided to patients and sources of funding for the activities of the European Reference Networks (ERNs). (A) The horizontal stacked bar charts represent the rates (expressed in percentages) of Belgian health policy makers who reported each category used to evaluate the adequacy of the different types of information considered. The level of adequacy was evaluated in a categorical way (excellent [white]; moderately good [light grey]; insufficient [grey hatched area]; woefully insufficient or non-existent [black]; no opinion [white hatched area]). (B) The horizontal stacked bar charts represent the rates (expressed in percentages) of participants who reported each one of the sources of funding proposed in the literature as adequate for the financing of ERNs’ activities.
As shown in (Fig. 3B), the financing of ERN’s members activities should be supported by different actors.
Federal/regional public funding was the most often reported source for the financing of clinical data management, medical equipment, quality management, fundamental research, genetic counseling and training of healthcare professionals. Other sources of funding proposed to a much lesser extent for these activities were the European Commission (8–43%), hospitals’ own resources or universities (0–33%), specific research grants (0–25%), pharmaceutical companies (proposed by 10% of participants for medical equipment) or patient’s organizations (proposed by 8% of participants to fund data management and genetic counseling sessions).
Information provided to patients should also be funded using different sources, including patient’s organizations. The European Commission was the main funding source proposed for inter-hospitals communication. Finally, clinical research’s costs should be covered by academic hospitals themselves (or related universities), the European Commission and national/regional healthcare authorities.
Discussion
Choosing the best ways to improve healthcare’s quality, clinicians’ expertise, hospitals’ collaborations and financing remains a big challenge, especially for highly-qualified institutions, which already face budgetary shortfalls [11, 12].
To the best of our knowledge, this study is the first one that investigated experts’ opinions in order to deal with these combined issues. This work presents five main strengths: (i) focused on strategies described in peer-reviewed papers, (ii) its bottom-up approach (based on experts’ opinions), (iii) its large scope, (iv) the involvement of both clinicians and representatives of health policy institutions, (v) its methodology, validated by the Donabedian model and RAND/UCLA appropriateness method, two well-known approaches used to evaluate healthcare’s quality and appropriateness [13–15]. Its limitations are (i) the limited number of participants and the fact that sometimes participants decided to provide no opinion. This may affect results’ power, especially for ERNs-related clinicians (n = 17), even if this number represents a significant proportion of the total number of ERNs recognized in 2017 (n = 24) because participants are related to different ERNs. This enables to have a good representation of the opinion of most ERNs, (ii) the use of only categorical variables to report participants’ opinions and, (iii) the fact that health policy makers came only from Belgium and not from different European states. Moreover, we have to take into account that results are based on participants’ opinions and remain thereby quite subjective. This study should thus be considered as an initial and exploratory step in the improvement process of the quality of ERNs.
The results strengthen the importance of quality assessment and of involving both clinicians and patients in the improvement of healthcare coordination. Indeed, participative management is a key factor of hospitals’ quality, efficiency, and patients’ satisfaction [16]. To this aim, patients’ opinions should also be discussed during clinicians’ multidisciplinary meetings and adapted communication tools developed to this aim [17, 18].
This study also highlighted some clinicians’ concerns (e.g. lack of information or trainings in human genetics or telemedicine). Indeed, only a few existing training programs cover, for instance, all aspects of genetic counseling [19, 20]. Awareness campaigns should therefore be performed and adequate tools and data storage platforms developed at the national or international levels for genetic data. The study also enabled to highlight the imbalance between physicians’ clinical and research activities that affects their contribution to scientific knowledge [5, 21].
The very positive results obtained during this study regarding the collaborations between the ERNs recognized in 2017 and other actors from the public health sector (patients’ organizations, second-line hospitals, research laboratories, pharmaceutical companies), and between the ERNs and their respective coordinators, demonstrate the robustness and efficiency of the concept of network as improvement process for patients management. However, it seems that the partnership between ERNs and European or national healthcare authorities is still lagging when it comes to support ERNs’ work, especially financially. This can be explained by a lack of budgetary resources at the European and/or national level(s). Healthcare authorities from the different European states may also consider that they are not competent enough to evaluate the quality of the activities performed by the network, especially in other countries or to have an accurate idea of ERNs’ needs. Closer and frequent discussions should therefore be organized between ERNs and European or national political powers in order to improve this aspect and guarantee ERNs’ sustainability.
A priority should be given to the establishment of European legal frameworks for the reimbursement of cross-country care, share of patients’ data and international recognition of clinicians’ expertise.
The dissemination of adapted information to patients requires a clear definition of funding sources and a close collaboration between all healthcare actors, including political instances. This aspect is crucial in terms of quality of patients’ management and well-being. However, it seems that the quality level differs among the different types of delivered information considered. This means that there is still room for improvement for this ERNs’ duty. ERNs’ members and healthcare authorities should therefore encourage the reflection on an harmonized list of quality criteria that ERNs should fulfilled with accurate subcategories (e.g. different types of information provided to the patients) for specific activities with outcomes that can be assessed objectively.
In conclusion, the set-up of panels of multidisciplinary and international clinicians and of health policy makers enables to identify efficient strategies to deal with ERNs’ challenges. It also enables to balance unjustified concerns and to highlight possible threats.
Footnotes
Acknowledgments
The author thanks all participants for their contribution to this work, as well as Professor Frédéric Thys for his advices during the study design and Dr. Wim Coucke for his pertinent suggestions during the redaction of the article. NMV is a scientific collaborator from the Sciensano Institute, previously called Scientific Institute of Public Health, and is supported by the Belgian National Institute for Health and Disability Insurance (RIZIV-INAMI; grant W4043.0100.8).
