Analysis: Investigating the quality of POCT devices for HbA1c,what are our next steps?

Abstract

There are a growing number of publications evaluating the performance of HbA1c point-of-care testing (POCT) devices when compared to routine laboratory instruments, but is this what we need from future studies? Here we describe the current understanding of the performance of POCT for HbA1c, which areas need further studies, and the key requirements for future publications based on performance evaluations of these devices. These include studies in clinical settings, performance measured against internationally standardized reference methods, and the need to evaluate new to the market devices that do not currently have a detailed performance history. In addition we highlight the need for external quality assessment schemes that are designed to support POCT in a wide range of clinical settings.

Regular monitoring of HbA1c in people with diabetes is a vital step in diabetes management. However, there are situations where it is not possible to analyze HbA1c as often as recommended in the American Diabetes Association (ADA) guidelines that might, as a consequence, lead in a delay in the corresponding treatment modification and eventually to the development of complications.^1,2 Point-of-care testing (POCT) has the potential to offer an opportunity, for timely monitoring and ultimately lead to an improvement in diabetes care.

As described in a recent article in Journal of Diabetes Science and Technology, Nathan and coauthors performed a laboratory and clinical assessment of an HbA1c POCT analyzer (Alere Afinion™ AS100, Abbott Park, IL, USA). ³ The conclusion was that the POCT performed acceptablycompared to the laboratory assay under realistic clinical conditions. While the evaluation was conducted using an acceptable protocol, the question arises as to whether we should continue to publish results of analyzer evaluations that have already accrued a number of similar publications as well as comprehensive studies using data from external quality assessment (EQA)schemes.^4-21

The Afinion AS100 analyzer and the Afinion HbA1c assay were CE-marked and launched in the Nordic countries in 2005. In 2006 Food and Drug Administration (FDA) cleared this POCT analyzer as a Clinical Laboratory Improvement Amendments (CLIA) waived assay with the intended use to monitor metabolic control in patients with diabetes mellitus. In 2017 the successor of the AS100 was launched (Afinion 2 analyzer). Both POCT instruments achieved moderate complexity clearance from the FDA with the intended use to diagnose patients with diabetes in addition to monitoring patients at risk of developing diabetes.

It is worth noting that FDA moderate complexity clearance for the diagnosis of diabetes mandates proficiency testing (PT) and in CLIA-waived settings PT is not mandated.

Assessing Performance and Addressing Manufacturer Bias

There are different ways to assess the analytical performance of a POCT analyzer:

1. CE marking for the European market

2. FDA clearance for the American market

3. IFCC manufacturer certification

4. NGSP manufacturers certification

5. Laboratory assessment performed in a routine laboratory

6. Clinical assessment performed in a health care setting

7. Laboratory assessment performed in a reference laboratory

8. EQA undertaken by the intended clinical users

The first four are generally performed at the manufacturers’ sites, under ideal circumstances and are not necessarily independent. The two key advantages of International Federation of Clinical Chemistry (IFCC) manufacturer certification above National Glycohemoglobin Standardization Program (NGSP) manufacturer certification are that, unlike the NGSP system, the IFCC is a true standardization scheme with full traceability to the Primary Reference Measurement Procedure (PRMP) and primary reference material. ²² Second, the values assigned to the samples in this certification program are assigned using 21 certified IFCC network laboratories using the IFCC PRMP, while the values assigned to the samples in the NGSP manufacturer certification are only from one certified NGSP secondary reference laboratory using a certified secondary reference method. ²³ The uncertainty in the value assignment in the IFCC manufacturer certification is much smaller than in the NGSP manufacturer certification. In simple terms, the IFCC certification is through standardization, not harmonization and is required for the use of HbA1c for diagnosis for both laboratory and POCT devices.^24,25

Laboratory assessment performed in a routine laboratory has added value for the laboratory if they also want to use the POCT instrument in their clinic or at the physician’s site. This allows the scientists and clinicians using the analyzers to understand the relationship between them in order to draw conclusions about changes in HbA1c and patient care.

Analytical performance studies undertaken in a diabetes center, as per Nathan et al, give valuable information about the analytical performance of a POCT instrument in the hands of end users. Many analytical performance studies of POCT devices have been performed by trained laboratory staff and my not always reflect “real-world” use. However, a systematic review and meta-analysis of the performance of POCT HbA1c devices concluded that there was no difference in bias between clinical or laboratory operators in two devices. ¹⁴

Laboratory assessment done in a reference laboratory has the advantage of the availability of reference methods and reference materials to compare with instead of a routine method. The European Reference Laboratory for Glycohemoglobin (ERL) has seven different reference methods with different measurement principles which are calibrated with IFCC secondary reference material. These methods are one step higher in the traceability chain than routine HbA1c methods. ²² By taking the mean of at least three reference methods, the uncertainty is very low. Despite the common myth that a comparator method is a gold-standard method, there is in fact no “gold-standard” routine laboratory method for HbA1c as mentioned in Nathan et al.

The improved precision and accuracy of the commercially available HbA1c assays in recent years is a combined effort of the IFCC and the NGSP systems working together to improve the quality of HbA1c measurement. With many of the secondary reference methods of the NGSP actually calibrated using IFCC calibrators, the two systems are working in synergy. The recent EurA1c trial clearly demonstrates that the introduction of IFCC standardization has resulted in generally, very low method bias. ¹⁶ Manufacturers have worked hard to improve the quality of their methods and HbA1c is a prime example of how a collaborative approach to quality improvement can have significant impact.

In the consensus statement editors were recommended to publish in both SI and NGSP units. ²⁴ Unfortunately in the paper of Nathan and coauthors only NGSP units were shown.

Quality Frameworks for POCT Are Key to Patient Care

EQA schemes is one of the most informative ways to assess the analytical performance of an HbA1c method. Participating in EQA schemes is one part of a quality framework which should be in place before one starts to use a POCT instrument (Figure 1). Comparing the results of methods in an EQA scheme like the CAP or the EurA1c survey gives an objective overview of bias and imprecision of methods. With single method comparisons between the routine laboratory instrument and a new POCT it is common to assume that the laboratory method gives the correct result while the POCT device may actually be out performing the routine laboratory device. In the paper of Nathan et al, the POCT instrument had a negative bias compared to the laboratory method. The 2018 results of the CAP survey shows that the Premier had on average a positive bias of +0.16% and the Afinion had on average a mean negative bias of –0.07% which is in line with the results found in the EurA1c paper (Premier +0.10% and Afinion –0.06%). This highlights the importance of EQA schemes in understanding the true performance of a device.^16,21

Figure 1.

Key factors to consider when introducing a new POCT device.

What Do We Need in the Future?

Approximately 90% of the HbA1c POCT market is currently held by the Afinion and the DCA Vantage (Siemens Medical Solutions Diagnostics, Tarrytown, NY, USA). These two devices have proven their quality. ⁹ However, many new HbA1c POCT instruments are entering the market, the quality of which is unknown. To obtain a comprehensive appraisal of performance of these devices, it is important that future evaluations focus on using IFCC standardized comparison methods and address the performance in the intended clinical setting, and most importantly, new devices are registered and actively monitored through appropriate EQA schemes, which should be mandated for all clinical decision settings.