Abstract
BACKGROUND:
Medical devices (MDs) represent the backbone of the modern healthcare system. Considering their importance in daily medical practice, the process of manufacturing, marketing and usage has to be regulated at all levels. Harmonized evidence-based conformity assessment of MDs during PMS relying on traceability of medical device measurements can contribute to higher reliability of MD performance and consequently to higher reliability of diagnosis and treatments.
OBJECTIVE:
This paper discusses issues within MD post-market surveillance (PMS) mechanisms in order to set a path to harmonization of MD PMS.
METHODS:
Medline (1980–2021), EBSCO (1991–2021), and PubMed (1980–2021) as well as national and international legislation and standard databases along with reference lists of eligible articles and guidelines of relevant regulatory authorities such as the European Commission and the Food and Drug Administration were searched for relevant information. Journal articles that contain information regarding PMS methodologies concerning stand-alone medical devices and relevant national and international legislation, standards and guidelines concerning the topic were included in the review.
RESULTS:
The search strategy resulted in 2282 papers. Out of those only 24 articles satisfied the eligibility criteria and were finally included in the review. Papers were grouped per categories: medical device registry, medical device adverse event reporting, and medical device performance evaluation. In addition to journal articles, national and international legislation, standards, and guidelines were reviewed to assess the state of PMS in different regions of the world.
CONCLUSION:
Although the regulatory framework prescribes PMS of medical devices, the process itself is not harmonized with international standards. Particularly, conformity assessment of MDs, as an important part of PMS, is not measured and managed in a traceable, evidence-based manner. The lack of harmonization within PMS results in an environment of increased adverse events involving MDs and overall mistrust in medical device diagnosis and treatment results.
Keywords
Introduction
Medical devices (MDs) prove to be the backbone of the healthcare system nowadays. Given their high significance in maintaining human health, the processes of manufacturing, marketing and usage have been regulated globally and nationally. According to the definitions and requirements stated in directives and regulations, as well as international standards, activities related to MDs can be divided into pre-market processes and post-market surveillance (PMS).
For instance, in the European Union (EU), pre-market and PMS processes have been defined by Medical Device Directives (MDD) [1, 2, 3] since 1992 and recently updated by Medical Device Regulation (MDR) [4, 5, 6]. The directives and regulations define all aspects from ideation, design and development phase, up to testing, approval and certification before the production process, production itself and PMS of medical devices (Fig. 1). To support the implementation of directives and regulations international standards have been developed. For instance, according to the ISO 13485 standard, which manufacturers are obliged to comply with, during the pre-market phase, the manufacturer who produces MDs needs to demonstrate its ability to provide MDs and related services that consistently meet customer and applicable regulatory requirements [7]. During the development phase, MD functionality, safety and performance needs to be designed to satisfy multiple requirements defined in a variety of international standards such as basic safety standards [8], electro compatibility standards [9], biocompatibility standards [10], environmental protection standards [11], etc. Certification process confirms conformity of the producer and the device itself to these requirements. In the EU CE mark is introduced [12] and serves as an administrative marking that indicates conformity with health, safety, and environmental protection standards for products sold within the European Economic Area (EEA). It is issued by the European Notified Bodies [13], organizations appointed by EU countries to assess the compliance of products before they are placed on the market. Therefore, as it can be seen from Fig. 1, regulators authorize independent notified bodies to do the conformity assessment in order to prove the manufacturer’s and product’s compliance with directives and regulations. Similar methodology is used in other regions and countries worldwide.

In the United States of America (USA), pre-market and PMS processes have been defined by the Food and Drug Administration (FDA) [14]. Similarly, to EU legislation, FDA also defines all aspects from ideation to PMS and checks compliance with regulation. In the United States, equivalent to Europe’s CE marking is UL (Underwriters Laboratories) marking [15].
MD regulations for Middle East and North Africa (MENA) countries are heterogeneous as MENA region countries derive their regulations from the EU or USA regulations. Following that practice, in the MENA region, compliance with regulatory requirements for production and marketing of MDs is proven by ISO 13485 certificate. As the most developed countries in this region, Saudi Arabia and the United Arab Emirates (UAE) have very good medical device regulations. Saudi Arabia Food and Drug Authority (SFDA) is responsible for regulating the medical device market and is doing so through Medical Devices Interim Regulations (MDIR) from 2008. The responsible authority for medical devices in the UAE is the Ministry of Health whose legal basis founded upon the International Medical Device Regulators Forum (IMDRF) as well as EU Medical Devices Directives [16].
Generally, in the countries worldwide, MDs undergo a distinctive process from ideation, design and development phase, up to testing, approval and certification before the production, the production itself and PMS of MDs. Administrative marking that indicates conformity with health, safety, and environmental protection standards is introduced differently from country to country. For example, in China, CCC (China Compulsory Certification) [17] marking is used for labeling of approved devices. Japan uses PSE (Product Safety Electrical Appliance & Material) marking [18]. In Australia, there are various types of CE markings, or their equivalents used [19]. Since leaving the EU, the United Kingdom (UK) has introduced the new mark UKCA (United Kingdom Conformity Assessed) which is enforced from January 1st, 2021 [20]. Canada enforces its own CSA (Canadian Standards Association) marking [21]. In all regions, the manufacturer governs the overall process from ideation to market, whereas independent authorized third parties check for compliance, give approval for commercialization and certification before placing the device on the market. So, it is evident that MDs are produced in a strict process harmonized with international standards, and put on the market after authorization and marking.
After market placement a PMS mechanism is initiated. PMS is a collection of processes and activities used to monitor the performance of a medical device once it is placed in healthcare institutions. These activities are designed to generate information regarding the utilization of the device, to expediently identify device design and/or usage problems and accurately characterize the device behavior in practice [22]. Despite the fact that premarket processes are very well defined and standardized and that PMS is conducted by manufacturers or distributors as defined within directives and regulations, it is not unlikely for MDs to cause an error during their usage resulting in patient injury or in the worst-case scenario resulting in patient death [23]. The fact is that there is no difference in medical devices between countries however their performance differs which is noticeable by the number of incidents reported. So, if medical devices are the same the only cause of such state can be found in different approaches to management including preventive service and surveillance. This indicates that current PMS processes possibly have issues.
This paper discusses issues within MD PMS mechanisms recognized through a research review of scientific articles, as well as national and international legislation and standard databases.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology was used for the study [24]. Two researchers (LSB and AD), who were blinded to the articles’ author information, conducted the study inclusion, data extraction, and assessment of the risk of bias independently. Other authors (LGP and AB) reviewed the extracted data and were consulted in case of disagreements.
Search strategy
Medline (1980–2021) [25], EBSCO (1991–2021) [26], and PubMed (1980–2021) [27] as well as national and international legislation and standard databases along with reference lists of eligible articles and guidelines of referent regulatory authorities such as European Commission, Food and Drug Administration and national regulatory bodies were screened in the time period from August to October of 2021 with an aim of identifying data sources that have evaluated the importance of MD PMS.
The following search criteria for scientific article databases was used to limit the research:
Journal articles that contain keywords: standard, post-market medical device, FDA, EU; Journal articles that concern stand-alone devices (i.e. not implantable devices); Language of publication: English; Publication time: unlimited; Full-text journal publications;
The reference lists of the included studies were checked for other relevant studies and authors were contacted if necessary.
Duplicates from the initial search results were removed using EndNote 20X (Clarivate analytics) and additionally persisting duplicates were manually removed. The sifting of the articles was performed in three stages: by title, by abstract, and finally by full text against the exclusion criteria. Finally, LSB and AD independently screened the remaining studies firstly by title, then keywords and finally abstract. After this step, the full texts of the studies were selected, and the content was compared against the inclusion and exclusion criteria. The main consideration for inclusion of a study was mentioning PMS in the body of the article in a reflective manner.
After the search was finalized, two authors have extracted the vital information from the selected studies in the following manner:
Literature data (first author, publication date); Region of the world; The problem of PMS evaluated by the study; Information regarding the standard or guideline used in the study (if any).
The authors have independently grouped the studies into three groups of interest: (1) medical device registry, (2) adverse event reporting, and (3) performance evaluation. Figure 2 represents the PRISMA guided flow of the review.
PRISMA flowchart of literature search: Included/excluded titles, abstracts, and full papers.
The initial search has returned a total of 2282 articles. After the removal of duplicates and applying the inclusion criteria, a total of 24 papers(Supplementary Table 1.) were included in the meta-analysis. Out of the 24 articles included in the review, 3 of them are dealing with the topic of medical device registry, 3 with adverse event reporting, and 18 with performance evaluation. Recognized limitations of the study can be seen in the selection and exclusion of articles.
COVID-19 is the latest in a series of cases to show that raising the global awareness and knowledge about the importance of respecting the essential requirements is a precondition for assuring the appropriate quality, performance, and safety of medical products, especially in case of high demand. The most prominent example of consequences brought upon by the lack of PMS are mechanical ventilators that have been placed on the market without being tested in a clinical setting [28]. All these devices have successfully passed all steps necessary for their placement on the market, however, they have been proven to be faulty in clinical settings. In countries where PMS has not been properly regulated, such devices have been found in healthcare facilities and were actively used on patients. Today, therefore, lawsuits are being filed in many countries against those responsible for the deaths of many patients [29, 30, 31]. As regulators are becoming much more focused on MD PMS and risk management, this review presents the result of the analysis of PMS globally and provides insight into the future work that needs to be done in addressing the found issues within PMS mechanisms. This is important to address because the healthcare sector is one of the most important aspects for the functionality of each country. As such, the healthcare system directly affects the life of each individual and takes a lot of national resources to govern.
The Global Harmonization Task Force (GHTF) [32] discussed PMS and defined conformity assessment principles for MDs with a scope to present and introduce a problem of non-overseen medical devices. They suggested procedures that could be performed by manufacturers to maintain a proper function of their devices on the market. Their system was based on establishing a quality management system (QMS) before the device is even placed on the market, to ensure that the manufacturer will in fact proceed with conformity assessments even after the device is marketed. This proposed system included handling of the complaints by customers, handling the device malfunction reports and providing preventive and corrective actions for the device maintenance. Conclusions were drafted in a report, which defined the specific tasks needed for PMS in the industry and discussed how the requirements for each task could be harmonized across regulatory environments. This report was later updated by the International Medical Device Regulators Forum (IMDRF) [33] to elaborate on reporting guidelines for adverse events. IMDRF has provided a document with a scope to guide their members when to exchange the information about the device, what procedures to follow during this phase and which forms to use to fill a medical device report.
In 2020, the World Health Organization (WHO) published a draft guidance of a proposed PMS plan to be done by the manufacturers [34]. Their proposed plan includes information about how and when to do PMS and what to do based on the data obtained. Their plan consists of following points:
Manufacturer must state which types of their medical devices are undergoing PMS; Manufacturer should state the exact method of data collection through the device’s lifecycle; Manufacturer must state the exact method of data analysis on the obtained information; Manufacturer must use the obtained results in a way to improve its development phase and decrease possible risks associated with.
The recommendations of these international organizations have been taken into consideration by regulators since PMS is included as part of legislation concerning MDs globally. Supplementary Table 2 presents the list of legislation for MD PMS worldwide. As it can be seen from the Supplementary Table 2, existing regulations on PMS generally oblige stakeholders to monitor the quality, performance, and safety of a device throughout the product lifecycle and to apply corrective or preventive actions when necessary. PMS generally relies on manufacturers, clinicians, and patients to report incidents including medical devices used to assess experience gained from medical devices that have been placed on the market, as well as to determine the need for any action. The PMS strategy generally includes medical device registry, adverse event reporting and medical device performance evaluation. This has been found out in the review of scientific articles in this field. The results of the search in PubMed, EBSCOhost and Medline databases regarding these topics are summarized in Supplementary Table 1. General comments are presented below for each category.
Medical device registry
Generally, countries worldwide have successfully implemented medical device registry through national authority that is usually within the jurisdiction of a body called “Medicine Agency”. As it can be concluded from the review, professionals in the field agree that establishing a national medical device registry is the first step towards creating a PMS [35]. However, the steps towards achieving a functional PMS system go beyond just a creation of a comprehensive database containing all MD manufacturers, distributors, and MDs on the market. For achieving an effective MD PMS establishing a framework of operation is necessary [35, 36, 37]. In their research, Bisdas et al. [36] concluded that patient registries are preferred over medical device registries because they offer more information with respect to risk stratification. Kramer et al. [37], have pointed out that MD registries streamline the procedure of surveillance and diminish the risk of faulty medical devices. Based on this, one can conclude that both patient and medical device registries are equally important to risk stratification in the healthcare sector to ensure safety and quality of care provided to patients.
The analysis of regional/national legislation [38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48] suggests that authorized regulatory bodies, such as the Medicines Agency, maintain medical devices and medicines registries. According to the legislation, manufacturers/distributors are obliged to make a registration of their products within the registry upon import and market placement. In this process national regulatory bodies actually do the initial registration and take the results of initial product conformity assessment made by independent third-party bodies. This registration is a prerequisite for placement of MD on the national market, and the placement itself is prerequisite for all other PMS strategies which follow MD on the market. In some countries, who adopted different schemes of PMS, as will be shown in part c), different medical device registries exist [49, 50, 51]. Gurbeta et al. [50] explain how an independent inspection body performing PMS developed digital medical device registries for MD safety and performance inspection management. This registry, in particular, contains only information on 11 different groups of medical devices, but the data on safety and performance serves as good input for artificial intelligence tools which is seen as the next step in MD management globally. This opens the potential of enhancing medical device registries in the future.
Adverse event reporting
Reporting adverse events, especially in the case of high-risk MDs is a responsibility that medical professionals owe to their patients. This contributes to MD registries as it enriches the pool of information engineers can use to further improve products and PMS mechanisms [52]. Legislation worldwide foresees clinicians in their duty to report any adverse events, or near adverse events. Manufacturers are also required to do a follow up of these reports [53]. Manufacturer and User Facility Device Experience (MAUDE) database regulated by the FDA and European Databank on Medical Devices (EUDAMED) database provide information regarding reported adverse events in healthcare institutions.
Our review reports a study conducted by Kvanagh et al. [54] which showed that 96.6% of adverse events in the US are reported by the manufacturers themselves while the patients and their families have been found to submit adverse event reports five times more often than physicians even though physicians should be the ones governing adverse-event reporting. By comparing results for US and UK the Kvanagh et al. [54] concluded that the regulatory mechanism for adverse event reporting is better in the UK. Based on this, one can conclude that an unambiguous regulatory framework is needed for effective PMS. Also, to prevent discrepancies among countries harmonization within methodology of adverse event reporting is needed. GHTF, IMDRF and WHO set a very good reference for PMS principles including adverse event reporting that now need to be harmonized globally to reach its full effect. On the other hand, this adverse event reporting part of PMS unfortunately means that some unfortunate event involving MD had taken place and that possibly a harm to a patient has been done. To minimize the rate of these events, MD performance should be monitored through their usage in healthcare institutions.
Performance evaluation
Collection of information regarding the utilization of the device and the device behavior in practice is the third key component of a PMS mechanism. Kumar et al. [55] highlight the importance of extracting data and signals related to the performance of medical devices on the market. According to the current regional/national legislation this part of the PMS mechanism is a formal obligation of the manufacturer/distributor. The study by Ross et al. [56] reports that out of 47 manufacturers required to approach performance inspection of their MDs, only 22 responded which is less than 50%. This report is a sound for alarm as it shows that manufacturers are not so consistent in MD performance evaluation once the MDs are placed in healthcare institutions. The performance evaluation is done during the preventive maintenance, assuming that the healthcare institution is not doing that ‘in-house’ or during corrective maintenance when some damage potentially has already been done. Also, given that manufacturers have different distributors in different countries the collection of information regarding the utilization of the device and the device behavior in practice is aggravated. This study highlights an important issue in the PMS mechanism, and that is the lack of harmonized evidence-based methodology that will allow collection of MD performance data once they are used in healthcare institutions on a regular basis in order to assess the device behavior in practice and in different environments. This has already been recognized by the professional community. For instance, Tarricone et al. [57] conducted a review of methods used for MD PMS and concluded that well developed strategies for PMS would significantly reduce the risks of medical devices.
This is seen by the results of existing research focusing on MD performance evaluation. For instance, an analysis of results of PMS in the US, conducted in 2012, found that, most commonly, recalls on the US market were due to software failures in medical devices [58]. In the study by Badnjevic et al. [59] it has been reported that 30% of tested mechanical ventilators and infant incubators are not operating properly and should be serviced, recalibrated and/or removed from daily application. Gurbeta et al. [60] reported that 13.84% of tested anesthesia machines and 14.91% of defibrillators device performance is not in accordance with requirements and should either have the device removed from use or scheduled for corrective maintenance. On the other hand, for dialysis machines [61], the results show that 12.6% of inspected devices do not meet electrical safety requirements or have performance outside the specifications. Specifically, 2% of tested devices did not pass the safety inspection in accordance with IEC 60601. For approximately 22.64% of devices that do not meet the performance specification, malfunction of heating systems was detected. Additionally, 11.32% of devices from this group had performance that was not in accordance with device specifications although malfunction was not reported.
These specific studies on performance evaluation of MDs which have been used and managed within healthcare institutions best show how much the devices actually deviate from the reference values and how important it is to develop harmonized evidence-based methodology that will allow collection of MD performance data once they are used in healthcare institutions on a regular basis in order to assess the device behavior in practice and in different environments and prevent adverse events.
The conducted review study resulted in following PMS gaps recognized:
Limited function of existing medical device registries as currently they have administrative purpose – only facilitating registration and import of medical devices; Adverse event reporting mechanism is managed differently globally with varying response of stakeholders; MD performance evaluation reveals significant variation in MD performance; Utilization of MDs and their performance evaluation currently cannot be assessed with great certainty since data collection and performance evaluation is not done in the same manner through healthcare institutions.
The PMS mechanism is therefore the whitespace for further improvements but with direct impact on quality of care in healthcare institutions and patient safety.
Comparison of regional/national regulations in the field of PMS worldwide depicts a significant level of heterogeneity and fragmentation when it comes to MD performance evaluation. This review suggests that MD PMS mechanism, unlike pre-market, lacks harmonization. Crucial components of PMS being medical device registry, adverse event reporting and post-market performance evaluation should be harmonized, with the same approach as it has been done for MD pre-market processes, in order to achieve specific goal of PMS and that is increased quality of care to patients.
White-space for standardization and harmonization of PMS framework.
Figure 3 presents the current PMS mechanisms depicting key stakeholders. The most noticeable feature is that under current PMS MD performance evaluation is done by the manufacturer, as mentioned before. The fact is that standard ISO 13485:2016 [7] requires manufacturers to maintain a PMS system regardless of how their medical device is classified. However, this approach has its limitations which can be seen through variability of MD performance as reported in studies mentioned above [59, 60, 61, 62, 63, 64], as well as by the number of reported incidents of the patient injuries and deaths including medical devices [65, 66, 67]. Given these facts, one can conclude that current status of PMS results in duplication of resources and human effort, lack of strategic coordination, mistrust in medical performance and reliability as well as poor responses to global challenges such as the COVID-19 pandemic which only emphasizes the need for harmonization of MD performance evaluation during PMS.
What is interesting to see is that some countries in the world have recognized this issue and have taken steps toward answering the recognized issues. For instance, in China, aside from reports on MDs made by manufacturers, new State Council Order Number 739 [68] enforces more stringent PMS which resulted in more field inspectors in different levels from manufacturing sites to research and development sites, distribution sites and the sites of device usage (healthcare institutions) to see if requirements for every specific device have been met. Additionally, China has developed MD testing institutes on various levels to ensure PMS of MDs [69]. Japan’s regulatory body in charge of development and implementation of safety standards for drugs and medical devices prescribes the PMS which relies on adverse event reporting, information storing and analysis with the aim of ensuring the safety of MDs in healthcare institutions. What is interesting to see is that in Japan novel methodologies such as data mining and sentinel medical institution networks have been considered as a way to enable effective track of MDs, providing prompt response to potential problems and performing preventive activities to ensure safety of all medical devices [70, 71].
Some countries have bridged the identified issues regarding PMS and MD performance evaluation by expanding the regulatory framework and introducing MDs within legal metrology [64]. This approach has been implemented in Spain, Portugal, Saudi Arabia, Republic of Serbia, Bosnia and Herzegovina, Chezch Republic, etc. [72]. According to this legal metrology framework, an independent authorized third body measures MD performance through periodic inspections and the results of inspection are reported in the medical device registry developed for this purpose. Particularly, in Republic of Serbia and Bosnia and Herzegovina independent inspection bodies perform performance evaluation of 12 types of MDs: ECG devices, defibrillators, patient monitors, infant incubators, therapeutic ultrasounds, dialysis machines, anesthesia machines, mechanical ventilators, infusomats, perfusion pumps, high-frequency surgical units and blood pressure devices [59, 60, 61, 62, 63]. The results of these inspections are reported in medical device registry [49, 50, 51] and are used for MD surveillance. Similarly to the intentions of regulators in Japan, the MD performance evaluation data in this registry is considered as a way to enable effective track of MD performance providing prompt response to potential problems as it is shift to evidence-based surveillance of a specific device without generalization to model or manufacturer taking into account the environment in which MD is used. So far, the analysis of these databases [59, 60, 61, 62, 63] for individual medical devices led to a conclusion that MD performance deteriorates over time and that MDs tend to be out of the accuracy limits while still in use which in some cases cannot be recognized by MD users, but it is risky for the patients who are being treated with such devices.
As is known, standards are suggested practices produced by subject matter experts with the intention of describing the best feasible way to achieve a specific end goal, increase efficiency, and eliminate product failures. Therefore, to undoubtedly ensure MD safety through their usage in healthcare institutions harmonization of their performance evaluation during PMS is needed. Harmonized evidence-based conformity assessment of MDs during PMS relying on traceability of medical device measurements will contribute to higher reliability of MD performance and consequently to higher reliability of diagnosis and treatments (Fig. 4).
Suggested methodology for post-market surveillance of medical devices.
This is not a novel approach since this has been done for a wide range of products in industry. For those devices, for instance, reliability in safety and performance is ensured by periodic inspections or calibration by independent third-party bodies according to the ISO 17020 or ISO 17025 [73, 74].
This method, along with existing preventive service provided by manufacturers would lead to increase in medical device reliability resulting in decrease in adverse events such as injuries and death. It is important to emphasize that mentioned harmonization of PMS would not exclude manufacturers and distributors from doing the PMS. In fact, according to the ISO 17020 an inspection body can be classified in three categories: (1)
Based on the conducted review study, it can be seen that MD registry, adverse event reporting and performance evaluation can be improved by strategic work. Harmonization of PMS by adopting evidence-based conformity assessment of MDs during PMS relying on traceability of medical device measurements will not only contribute to higher reliability of MD performance and consequently to higher reliability of diagnosis and treatments but will also open opportunities to improve the function of MD registry by adoption of novel data mining techniques. Some countries have already undertaken steps in this way and adopted PMS regulations that require notified bodies to perform inspection according to international standards for bodies performing testing and inspection.
Given the rising challenges in the healthcare sector MD PMS should be considered more significantly. Strategic collaboration between the professional community and regulatory authorities is needed in order to respond to these challenges. International organizations such as International Federation on Medical and Biological Engineering (IFMBE), European Alliance on Medical and Biological Engineering and Science (EAMBES), American College of Clinical Engineers (ACCE) and others have the expert know-how from the practice that is invaluable for regulators. Such strategic cooperation can result in an environment where MD surveillance is done in a standardized, impartial, evidence-based way. That would ensure that every MDs conformity assessment is derived upon the same parameters regardless of who is performing the evaluation or where, just as is done in pre-market approval of MDs. IEEE has begun to work on harmonization of MD PMS performance evaluation within the Medical Device with Measuring Function (MDMF) group whose first result is the P2727 – Standard for General Vocabulary for Conformity Assessment of Medical Devices with Measuring Function [82, 83].
Footnotes
Conflict of interest
None to report.
Supplementary data
The supplementary files are available to download from http://dx.doi.org/10.3233/THC-220284.
