Improving quality management of newborn screening in southwest China

Abstract

Objective

Newborn screening (NBS) programs benefit tens of millions of infants worldwide each year. However, the extremely large screening populations and number of laboratories involved pose great challenges to maintaining high screening quality. To achieve continuous quality improvement, we established a comprehensive quality management system (CQMS) in southwest China.

Methods

External quality assessment (EQA) and internal quality control were carried out for basic quality management. We used 16 quality indicators (QIs) to monitor the entire screening process, with external supervision from the China National Accreditation Service for Conformity Assessment. All retrospective data for quality assessment were collected consecutively from laboratory management and patient follow-up systems.

Results

From 2015 to 2019, satisfactory EQA performance was achieved, with an average score greater than 97 for each screening item. QI monitoring showed that NBS quality improved continuously. The rate of health education provision increased from 90.9% to 100% and the recall rate after a positive primary screening increased from 85.4% to 99.2%. The unsatisfactory specimen rate and rate of newborns lost to follow-up decreased to 0.38% and 0.08%, respectively.

Conclusions

Implementing a CQMS and monitoring the whole screening process using QIs may yield continuous quality improvement of NBS.

Introduction

First started in the 1960s, newborn screening (NBS) is a highly systematic program that aims to prevent mental retardation, premature death, and other adverse outcomes in the early stage of an infant’s life. ¹ NBS in China began with a pilot study in 1981 and was extended to 31 provinces during the following two decades. ² Currently, four inherited congenital disorders, namely, hyperphenylalaninemia (HPA), congenital hypothyroidism (CH), congenital adrenocortical hyperplasia (CAH), and glucose-6-phosphate dehydrogenase deficiency (G6PDd), are included in the NBS program in most southern provinces of China. ³ Usually, at least one screening center (or laboratory) is authorized by the local government (provincial or county) to hold the responsibility for NBS. Dried blood spot (DBS) samples of newborns are collected from maternity hospitals and sent for multichannel screening tests provided by the screening center. Positive screening results are reported to both the parents and the corresponding maternity hospital; further confirmation is performed within the following few days. When any suspected disease is identified, the newborn is treated and followed up by specialists from both the screening center and the maternity hospital. To ensure the quality of screening, the National Center of Clinical Laboratory (NCCL) of China requires all NBS laboratories to participate in the Newborn Screening Quality Assurance Program (NSQAP). ⁴ , ⁵ By providing plans for internal quality control (IQC) and external quality assessment (EQA), the NSQAP ensures testing accuracy in all screening laboratories.

NBS is a complex system comprising more than only laboratory testing. ⁶ Accurate screening results are a basic benefit for all newborns, and highly qualified follow-up and treatment are also important for those with identified diseases. The processes of sample collection and transport, personnel factors involving nurses and general practitioners, as well as material conditions including filter paper cards, testing reagents, and equipment, are all closely related to the quality of screening. ⁶ , ⁷ Deficiencies in any part of the screening process may result in decreased screening quality and inadequate follow-up in an at-risk population. The Association of Public Health Laboratories of the United States developed the Newborn Screening Technical Assistance and Evaluation Program (NewSTEPs) in 2012, to support NBS laboratory and follow-up systems in achieving quality improvement. ⁸ NewSTEPs has been proven to be successful in supporting data-driven outcome assessment and performance tracking ⁹ , ¹⁰ and provides a model for other countries in improving NBS quality.

China initiated a similar study in 2015, implementing a comprehensive quality management system (CQMS) of NBS in southwest China. From 2015 to 2019, we tested more than 650,000 newborns and successfully performed basic quality management of NBS. In a pilot study, our laboratory set 16 quality indicators (QIs) to monitor the entire screening process, from DBS sample collection to patient follow-up, with the aim to achieve continuous quality improvement. As a requirement for CQM of NBS, we applied external assessment to ensure our screening process meets a high standard. We first received accreditation by the China National Accreditation Service for Conformity Assessment (CNASCA) in 2015. ¹¹ Herein, we describe the CQMS and screening process monitoring using QIs, for quality improvement of NBS.

Materials and methods

Study design and materials

We performed quality assurance by participating in the NSQAP, to maintain basic quality management of NBS. To achieve continuous quality improvement, 16 QIs for monitoring the entire screening process were introduced to the quality management system and an external supervision plan was implemented. The retrospective data collection in this study started in 2015 and was carried out by the NBS center of Chongqing Medical University, which holds the screening responsibility for a newborn population of 120,000 each year in southwest China.

Newborn heel prick blood was collected by nurses in a maternity hospital, dried on filter paper cards (Whatman® 903), prepared as DBS samples, then sent to the Chongqing NBS center for testing. Newborn DBS samples were consecutively collected from January 2015 to December 2019. Signed consent was obtained from the parents of newborns before blood collection. Qualified commercial kits were used for screening of HPA (Neonatal PKU; FENGHUA, Guangzhou, China), CH (Neonatal TSH; FENGHUA), CAH (GSP® 17OHP; Neonatal PerkinElmer, Waltham, MA, USA), and G6PDd (GSP® G6PD; Neonatal PerkinElmer). IQC samples were provided in the testing kits and EQA materials were provided by the NCCL in May, August, and October, with three panels a year. Documents needed for laboratory accreditation, including the annual plan, procedures of quality assurance, and QI assessments, were prepared according to the requirements of Quality and Competence (ISO 15189: 2012) and China Laboratory Accreditation Rules (CNAS-RL01:2015). ¹¹ , ¹² The QI monitoring quality improvement initiative was not powered to detect statistical differences. Monthly quality improvement data were converted to annual metrics for analysis. This retrospective study was approved by the Institutional Review Committee of Children’s Hospital of Chongqing Medical University (2019-GP-16) on 10 October 2019, and complies with Quality Improvement Reporting Excellence 2.0. ¹³

Basic quality management of NBS

Internal quality control (IQC) of NBS

IQC samples were tested daily together with the newborn samples. The mean and standard derivation (SD) of the quality control samples were determined using measurements from the first 20 days or testing batches. ¹⁴ Westgard rules criteria were used to decide whether an analytical batch was in control or out of control in our laboratory. ¹⁵ The 1-2SD rule refers to the control rule commonly used with a Levey–Jennings chart when the control limits are set as the mean ± 2 SD. The 1-2SD rule is used as a warning to trigger careful inspection of the control data according to the following rejection rules. A batch is rejected when a single control measurement exceeds the mean + 3 SD or the mean − 3 SD limit, which is the 1-3SD control rule. ¹⁶ The 2-2SD rule is used to reject a batch when two consecutive control measurements exceed the mean ± 2 SD control limit. When any batch breaks the rules, the screening process will be stopped and reassessed.^17,18

External quality assessment (EQA) of NBS

Our laboratory participates in EQA activities three times a year. Each panel of EQA has five replicates for each kind of analyte with different concentrations, which are prepared in the dried blood matrix. ¹⁹ EQA materials are tested simultaneously with the newborn samples. All measurements of the analytes and the results of classification (positive or not) based on the reference interval are submitted to NCCL’s online public system by the quality director of our NBS center within 7 days after testing. The NCCL will release a report of our laboratory’s performance within 2 weeks after the closure of this data reporting system. The report shows all false-positive or false-negative classified measurements and the satisfactory classification rate of each analyte. ²⁰ Laboratory performance is assessed by scoring: 100 points are given when all five replicates are classified correctly; 80 points or more is considered to be acceptable performance whereas less than 80 points is considered unacceptable. An annual summary of EQA performance of the four kinds of analytes is released in December, and laboratories that achieve acceptable performance receive a certificate.

Continuous quality improvement of NBS

Quality indicators (QIs) for monitoring the entire process of NBS

To achieve comprehensive quality improvement of NBS, we selected sixteen QIs to monitor the entire screening process (Figure 1), some of which are also recommended by the NBS System Performance Evaluation Assessment Scheme (PEAS) of the United States and other research groups. ²¹ , ²² For the pretesting procedures, four QIs were chosen to track newborn health education provision, DBS sample quality, information integrity on the cards, and transport time of samples outside the laboratory (Table 1). For the testing process, we used eight QIs to monitor the screening assay accuracy, cutoff value efficiency, and the quality of tests inside the laboratory. We used another two QIs to evaluate the performance of recall. For post-testing procedures, two QIs were selected to track the performance of patient follow-up and the annual screening rate. QIs were calculated and evaluated monthly or yearly. Since 2015, the data from Chongqing are required to be submitted to an online system by the NCCL as Chongqing is the first province in China to begin using QIs in the NBS program. As a pilot laboratory, our submitted data were summarized and analyzed to support data-driven outcome assessment and performance tracking, to improve the quality of NBS in China.

Figure 1.

Quality indicator monitoring for the entire process of newborn screening (NBS). The 16 quality indicators are shown in different screening processes (pre-testing, testing, and post-testing) and are organized according to the main responsible institutes.

Table 1.

Interpretation of quality indicators.

Quality indicators (%)	Measuring method based on counts	Interpretation
Newborn health education provision	Newborn families that received health education/Total newborn families per year	Education
Unsatisfactory DBS samples	Unsatisfactory DBS samples/Total samples per month	Sample collection
Cards missing essential information	Sample collecting cards missing essential newborn information/Total samples per month	Sample tracking
DBS samples with acceptable transport time	DBS samples with acceptable transport time/Total samples per month	Sample transport
Test reports with an acceptable time from specimen receipt to reporting of results	Test reports delivered within an acceptable turnaround time/Total reports per month	Reporting time limits
Test items running IQC	Test items running IQC/Total test items per year	Quality control
Unsatisfactory testing results in IQC	Tests with unsatisfactory results in IQC/Total tests per month	Quality control
Newborns with positive results in primary screening	Newborns with positive results in primary screening/Total newborns screened per month	Kits, sample quality, and cutoff value appropriateness
Newborns recalled after positive primary screening	Newborns recalled after positive primary screening/Total newborns that need recalling per month	Recall
Newborns with positive results in secondary screening	Newborns with positive results in secondary screening/Total newborns screened per month	Screening method and cutoff value appropriateness
Newborns recalled after positive secondary screening	Newborns recalled after positive secondary screening/Total newborns that need secondary recalling per month	Recall
Newborns with false-negative screening results	Newborns with false-negative screening results/Total newborns confirmed with a specific screened disease per year	Assay accuracy monitoring
Positive predictive value	Newborns confirmed with a screened disease/Total newborns with positive secondary screening per year	Assay accuracy and cutoff value efficiency monitoring
Annual disease incidence	Newborns confirmed with a screened disease/Total newborns screened per year	NBS program performance evaluation
Newborns that received NBS	Newborns that received NBS/Total newborns born alive per year	NBS program performance evaluation
Newborns lost to follow-up	Newborns lost to follow-up/Total newborns needing follow-up per year	Follow-up

IQC, internal quality control; DBS, dried blood spot; NBS, newborn screening.

Performing laboratory accreditation for NBS

To ensure continuous quality improvement of NBS, we established a CQMS (Figure 2). The procedures of quality assurance and QI assessments were documented according to the requirements for quality and competence (ISO 15189: 2012). First, we set our quality policy and objectives (QPO), which were integrated with the following processes required for a high screening quality. Then, we prepared CQM documents, including the system procedures and the supporting documents. These documents included a quality manual describing our QPO; the procedural files required by NBS laboratory management; the standard operating procedures used to ensure effective planning, operation, and control of the screening processes; and the charts used to monitor all testing processes, daily quality control activities, and the NBS quality performance achieved in each panel. All actions taken were following the annual plan, and continuous quality improvement of NBS was made during this process. Our laboratory also applied for accreditation by the CNASCA for external supervision.

Figure 2.

The Comprehensive Quality Management System (CQMS). Inside the triangle: supporting documents for CQM. Outside the triangle: working procedures of the CQMS; purple arrow illustrates actions carried out for quality management and red arrow shows how outcomes are used to improve screening quality.

Results

Outcomes for basic quality management of NBS

A total of 652,000 newborn DBS samples were collected. We tested approximately 600 samples each weekday. We usually conducted two batches of testing for each analyte in 1 day and used one standard curve in each batch to calculate the concentrations of 356 newborn samples and 8 quality controls. As a result, 1826 batches of tests were carried out. All tests for thyroid stimulating hormone were in control. Only one batch was out of control for phenylalanine tests, and a low out-of-control rate was achieved for G6PD tests and 17α-hydroxyprogesterone tests (0.3% [5/1826] and 0.4% [7/1826] respectively). The comparison using recommended values of IQC samples and laboratory-determined values for drawing the Levey–Jennings chart showed that the systematic error was reduced (Figure 3a).

Figure 3.

Performance of internal quality control (IQC) and external quality assessment (EQA). (a) Comparison using kit-recommended values of IQC samples and laboratory-determined values for drawing the Levey–Jennings chart (IQC data collected from 1 October to 31 October 2019). (b) Five-year performance of EQA. Annual EQA performance for each item assessed using the average score of the three panels; 20 points are given for each of the five proficiency testing material replicates when classified correctly (See Appendix A).

In EQA assessment, satisfactory performance of EQA was achieved from 2015 (Figure 3b). In total, 300 EQA samples were measured, and we obtained a minimum score of 86 for all analytes in each year (Appendix A). In 2016, we reported a false-negative classification of the analyte phenylalanine from the five replicates, the measurement of which was close to the cutoff value. We finally found that the source of error was unstable temperature control in an incubator; we therefore switched to a new constant-temperature incubator. The EQA samples and previously tested newborn samples with measurements near the cutoff value were then retested. By comparing the measurements before and after retesting, we could ensure the elimination of recurring error.

Outcomes for continuous quality improvement of NBS

To achieve continuous improvement in the screening quality, we set an expected performance for each QI and evaluated the QIs monthly or yearly. From 2015 to 2019, newborn health education provision increased to 100% whereas the percentage of unsatisfactory specimens decreased to 0.38% (Figure 4a, 4b). In the post-testing process, the number of infants who received NBS increased by 12% and the rate of loss to follow-up decreased to 0.08% (Figure 4c, 4d). For the testing process, the recall rate after positive primary screening also increased by 12%, on average (Figure 4e). Performance of all QIs showed continuous improvement in NBS quality in southwest China. Results of 5-year QI monitoring are shown in Appendix B.

Figure 4.

Quality improvement of newborn screening with monitoring using quality indicators. (a) Performance of newborn health education shown as the percentage of “Newborn health education”. (b) Performance of sample collection shown as the percentage of “Unsatisfactory DBS samples”. (c-e) Performance of primary screening, recall, and follow-up.

Our laboratory applied to be evaluated by the CNASCA in 2015 and became the first NBS laboratory to be accredited in China. The CNASCA provided strict supervision of laboratory management throughout the entire process of NBS and carried out reassessments every 2 years. We passed the reassessments in 2017 and 2019, which ensured that all our screening work still met the requirements. This external supervision process contributed greatly to continuous quality improvement of NBS in our laboratory.

Discussion

In the past decades, China has carried out one of the world’s largest NBS programs. Different to most Western countries, NBS in China is usually performed in a local government authorized center or laboratory rather than in the hospital where a baby is born. Centralized screening is efficient for the testing process, making quality control much easier and clearly reducing the cost for testing. However, it can be challenging for NBS centers to control the outside processes, such as sample collection, communicating screening results, patient recall, and follow-up. ²³ The Newborn Screening Center of Chongqing implemented a CQM system in southwest China. From 2015 to 2019, we successfully performed basic quality management by participating in the NSQAP. We used 16 QIs to monitor the entire screening process and achieved continuous improvement in NBS quality. We also applied for external assessment and were accredited by the CNASCA, which helped us to maintain a high quality of screening.

As a basic quality management process, all NBS laboratories were required to participate in the quality assurance program. IQC and EQA samples were submitted to the NCCL for further analysis. The NCCL did not assess the IQA measurements but analyzed the mean and SD of the IQC measurements according to the kits and testing equipment used in different laboratories. Laboratories used the mean ± 2 SD of each analyte to evaluate their performance for each batch in IQC. To reduce systematic error in routine screening tests, we used the laboratory-determined IQC values for drawing the Levey–Jennings chart. In EQA assessment, if any misclassification occurred, our laboratory received a notification from the NCCL so that immediate action could be taken to determine the source of error and eliminate the risk of recurrence. We recommend using the cumulative mean value of the IQC materials from the same batch of testing kits or third parties for analysis.

QI monitoring gradually improved the quality management of NBS, especially the external laboratory processes. Participants in the NBS system, including nurses, administrative staff, technicians, and doctors, were required to fulfill their responsibilities in the corresponding phases, and their work was assessed each year in a performance appraisal program. These measures contributed to a good outcome of QI monitoring and improvement in the pre-testing and post-testing processes, which were mainly carried out in the maternity hospital (Appendix B). Special circumstances may also exist in different laboratories; the recall rate of G6PDd was slightly lower than that of other diseases, probably because Chongqing has a high incidence of G6PDd and the false-positive primary screening rate tends to increase in the hot summer. Therefore, we started to use a cold chain for specimen transport from late 2017, and the false-positive primary screening rate of G6PDd declined significantly. The positive predictive value was also improved; however, until now we have not achieved the expected performance.

According to our experience, three steps might be necessary to achieve improvement in the CQM of NBS: participation in the NSQAP, using QIs to monitor the entire screening process, and receiving continuous external supervision after obtaining accreditation. Currently, QI assessment is accepted in China as a necessary process for recording, analyzing, and sharing experiences among different NBS laboratories. ²⁴ More laboratories are now preparing for CNASCA accreditation. The limitation of the present study was that more QIs were used for quality management monitoring, in comparison with recent studies, which may increase the workload of program participants. ⁹ , ²⁵ Therefore, further assessment should be performed to select the most efficient QIs. Current EQA performance scoring is based on quantitative evaluation of the submitted measurements; qualitative evaluation may also need to be included for screening tests.

Conclusion

NBS is a systematic program that includes the processes of sample collection, testing, and patient follow-up. A successful screening program must provide all newborns with not only accurate test results but also timely treatment and regular follow-up. Monitoring each step of the screening process using QIs may lead to continuous improvement in the screening quality and enable NBS laboratories to detect possible errors and make corrections in a timely manner. The present study describes the 5-year experience of quality improvement for NBS in southwest China. Our findings will be valuable for quality improvement in other NBS laboratories.