Internal quality control in point-of-care testing: where’s the evidence?

Abstract

ISO 22870 standards require protocols for performance of internal quality control for all point-of-care testing devices and training of users in its theory and practice. However, the unique setting of point-of-care testing (i.e. processes conducted by non-scientific users) means that laboratory internal quality control programmes do not easily translate to point-of-care testing. In addition, while the evidence base for internal quality control within the laboratory has been increasing, the equivalent literature surrounding point-of-care testing is very limited. This has led to wide variation in what is considered acceptable practice for internal quality control at the point of care. Indeed, it has been demonstrated that internal quality control is an area of deficiency in point-of-care testing. Internal quality control protocols used at point-of-care testing should be defined based on risk management. The protocol will therefore be dependent on analyser complexity and availability of inbuilt system checks, the risk associated with release of an incorrect patient result as well as frequency of use. The emphasis should be on designing an effective internal quality control protocol as opposed to the inherent tendency of introducing high-frequency quality control. Typically a simple pass or fail criterion is used for internal quality control in point-of-care testing based on whether internal quality control results fall within assigned ranges. While simply taught, such criteria can require broad internal quality control ranges to decrease the probability of false rejection (also reducing the probability of error detection). Customized internal quality control ranges, two-tier acceptance systems and assay-specific internal quality control can be used to improve error detection rates.

Introduction

Internal quality control, clinical governance and risk management

In the context of point-of-care testing (POCT), clinical governance aims to ensure that the patient comes to no harm as a result of the use of POCT processes. Key elements of clinical governance for POCT include the principles of best practice that promote good clinical outcomes, safety, reliability, suitability and efficiency. Internal quality control (IQC) in POCT is a crucial part of these processes.

IQC is used to assess the day-to-day consistency of assay performance, providing assurance of the quality of patient results. However, activities relating to quality control are among the 10 most cited deficiencies in POCT. ¹ These deficiencies include failures in performing and documenting quality control testing and in taking appropriate action for control outliers.

The Clinical Laboratory and Standards Institute (CLSI) has produced documents relating to development of quality control plans (QCP), including IQC protocols, based on risk management.^2,3 The CLSI standards EP23A require a risk assessment of each stage of patient testing followed by definition of a QCP that is individualized to the identified risks. ³ A review of Table 1 and the differences in the expected capabilities of users POCT compared to trained laboratory staff should highlight the need for IQC protocols that are specific to the POCT environment.

Table 1.

Comparison of the conduct of IQC within the laboratory against POCT.

Laboratory IQC	POCT IQC
Users aware of risk of analytical error, and hence need for IQC	Users may lack appreciation of potential for error – view analysers as infallible
IQC accepted as part of day-to-day procedures within the laboratory	IQC viewed as additional workload.
Trained in pipetting/used to using different vials for IQC material	Not trained in pipetting. Process of loading IQC does not resemble patient samples
Users trained to identify random or systematic errors	Users taught only basic interpretation – acceptable or failed
IQC material should be independent from instrument manufacturer	Manufacturer typically provides quality control material
Typically complex, multi-analyte analysers	Devices range from disposable kits to multi-analyte analysers
Generally predictable test volumes, therefore IQC frequency easily established	Frequency of testing can vary widely depending on location of POCT device. IQC frequency may need to vary accordingly.
Utilization of IQC: Patient testing ratio typically low	Utilization of IQC: patient testing ratio typically much higher than laboratory.

IQC: internal quality control; POCT: point-of-care testing.

We attempt here to provide guidance to POCT teams involved in the implementation of IQC in POCT, taking account of the risks that this unique environment poses.

Conduct of IQC

All POCT devices, from disposable pregnancy test kits to laboratory-type blood gas analysers, should have a defined IQC protocol. ⁴ Common to all devices is the requirement to conduct IQC when: (i) a new lot or new delivery of consumables is received; (ii) a patient result is queried; (iii) after maintenance procedures and (iv) if the user suspects there has been a physical insult to the device (e.g. exposure to high temperatures or if the device is dropped). ⁵ The frequency of further IQC testing will be analyser-dependent and should take into account the following:

Analyser complexity and availability of inbuilt system checks.

Frequency of patient testing.

Risk associated with release of an incorrect patient result.

The number of users (and maintenance of competency).

The added risk of a missed IQC.

The cost of IQC material invariably has an impact on protocol setting. However, while difficult to quantify, the cost of not conducting IQC in the event of an error affecting patient results and potentially leading to patient harm must also be considered.

The Medicines and Healthcare Products Regulatory Agency (MHRA) 2013 guidelines describe acceptance testing on receipt of a new batch of consumables as ‘an invaluable quality control measure before dispatch to test sites’ suggesting that this IQC test should be conducted by the POCT team. ⁶ However, beyond the initial acceptance testing, IQC should be processed by the POCT device operator in order to provide assurance of the whole testing process. Below, we have set out what should constitute the minimum IQC protocol required of the operator, based on analyser complexity.

Type of POCT device and IQC requirement

Disposable kits, strip technology e.g. pregnancy test kits

As a minimum requirement, strip-based devices should be subject to IQC testing at least monthly unless the manufacturer suggests more frequent analysis. ⁵ Two levels of IQC, containing (1) normal and (2) clinically relevant abnormal concentrations, should be tested on a representative sample of kits (e.g. two from each testing site). Such IQC provides assurance that functionality is not affected by environmental factors to which all kits have been exposed and also assesses operator competence. Inbuilt controls, for example of pregnancy test strips, provide additional assurance of the correct function of individual kits.

Training, as well as audit of IQC testing, is vital to encourage regular IQC. There is no potential to prevent use of disposable kits following poor/missed IQC. The interval between IQC testing also rapidly accumulates if the minimum of monthly IQC is missed. Setting a recurrent day on which IQC should be processed plus reminder emails may help maintain conformance to the IQC protocol.

Strip-based system with automatic reader e.g. glucose meters

IQC assesses the function of both the reader and disposable strip components of devices such as glucose meters. Strip-based devices generally lack an inbuilt electronic check that assesses the functionality of the reader in isolation of liquid IQC. Regular IQC is therefore required to limit the potential for a faulty reader to generate poor results for weeks before identification.

We suggest IQC analysis is conducted every day of device use. ⁷ In areas of low use, IQC should be conducted weekly even when patient samples are not processed to identify faults prior to impacting on processing of patient samples. Such regular testing of IQC material also maintains user competence.

Single-use cartridge systems e.g. Abbott i-STAT and Siemens DCA Vantage

Single-use cartridge systems typically employ a combination of liquid IQC with electronic IQC to assess the functionality of the whole device. Electronic IQC uses a simulator in place of the cartridge to assess the function of the reader and should be performed each day of patient use. Liquid IQC is essential to assess the analytical component (the disposable cartridge), and two levels of IQC should be tested at least monthly. ⁵ Testing should be conducted at each analyser site to monitor differences in storage conditions and operator performance. Liquid IQC should be conducted every day of patient use for analysers that lack an independent electronic check.

Multiple-use cartridge systems e.g. blood gas analysers

Multi-use cartridge technology forms the basis of many multi-analyte instruments. These more complex analysers require daily multilevel IQC, equivalent to laboratory based systems. However, the conduct of IQC is user independent for latest generation analysers. Rather, IQC is run automatically at predefined time points and assesses the fluidics path of a patient sample as well as the measurement system.

Manufacturers have implemented a number of strategies, which complement IQC, to identify analyser faults early. Systems such as the Instrumentation Laboratory GEM™ Premier analysers with intelligent quality management (iQM) include continuous integrated IQC which checks the quality of the signal, and sample integrity, every time a patient sample is run. Inbuilt IQC does not, however, assess the normal patient sampling procedure or user performance. Unfortunately, manually processed IQC is also limited in this respect if the material does not mimic patient specimens. For example, such IQC for blood gas analysers is provided in snap top ampoules, requiring vigorous shaking prior to analysis – neither resembling a whole blood specimen nor normal loading procedure. Comparative patient sampling can be useful to monitor the full analyser functionality, but is largely reserved for troubleshooting applications.

Training for end users

ISO 22870 requires that users of POCT devices should be trained in the theory and practice of IQC. ⁴ In addition to the basic principles of IQC, information should be provided regarding storage requirements, preparation of QC material, frequency of testing, documentation requirements and basic troubleshooting as appropriate. The required knowledge to successfully process and interpret IQC should be subsequently tested within the competency assessment of users.

Training sessions should also be used as tool to encourage users to participate. There can be a general underestimation of the risk of analytical error by the POCT user and therefore a lack of appreciation of the need for quality assurance. Running IQC will be seen as a burden unless its role is made clear. Confirmation that there is shared responsibility for IQC should also be monitored via annual audit of user ID and conduct of QC.

IQC material

In contrast to the laboratory setting, there is a reliance on the manufacturer both to provide material and to assign acceptable ranges for IQC in POCT. Indeed, some manufacturers state that their own IQC material should be used rather than third-party material. ⁸ However, we recommend that third-party IQC material is used to provide an independent check of the whole testing system where possible.

IQC material should contain analytes at clinically relevant concentrations, guided by the clinical setting of the POCT instrument. For example, IQC materials containing higher levels of bilirubin may be necessary for neonatal units compared to other clinical settings. Both abnormal and normal range IQC should be available. The material should be provided ready for use – with any manipulation of the material (e.g. reconstitution) completed prior to distribution. ⁹ It should also ideally be stable at ambient temperature to minimize storage problems.

If commercial IQC material is unavailable, patient samples can be used as a substitute for IQC material and a comparison made against the central laboratory. However, this only provides an assessment of accuracy, rather than precision, and therefore does not truly constitute IQC. The result of the comparison must also be interpreted with knowledge of the analytical error of both the POCT and laboratory analyser for any meaningful conclusion to be made. Logistically, such use of patient samples is limited to POCT sites located within a hospital, where central laboratory testing is more easily achieved.

Definition of IQC ranges

We have not identified any POCT guideline that suggests in-house definition of IQC acceptance ranges prior to processing patient samples. Statistical definition of IQC ranges at the acceptance testing stage is achievable, but only when the laboratory has an extensive input into the quality control of devices. In addition, for single-use cartridge-based assays it may be impractical (and rather expensive) to assign IQC acceptance ranges for each new lot of IQC material, as per laboratory guidelines, prior to distribution. However, if we are to rely on manufacturer ranges then we need to address analytes for which the assigned ranges appear excessively wide. For example, the assigned ranges for Total CO₂ for levels 1 to 3 on a recent lot of Chem 8+ IQC for the iSTAT were 8–26 mmol/L, 15–28 mmol/L and 28–40 mmol/L, respectively, despite a stated precision of 3.6% at TCO₂ 17 mmol/L.^10–13

If new lots of IQC material are initially introduced with manufacturer-assigned ranges, definition of the true standard deviation (SD) values could be undertaken as the IQC data accumulate. Sufficient data for such statistical range assignment would accumulate within three weeks for medium to high complexity POCT systems such as glucose meters and blood gas analyser. The emphasis should then be on minimizing the frequency of lot changes.

Documentation and review of IQC results

One of the major challenges in POCT is achieving proper documentation. All IQC values obtained by POCT must be recorded along with the date/time, user ID and consumable lot numbers. The user decision to accept/reject IQC, plus actions as appropriate, must also be documented. It must also be possible to identify the patient results associated with each QC result. Maintaining manual records (where necessary) of all this information may feel time consuming but can pay dividends during long-term IQC review/troubleshooting.

Traditionally in POCT a simple pass or fail criterion is implemented based on whether results fall within assigned ranges. This protocol is simply taught but can require broad IQC ranges to decrease the probability of false rejection (thus also reducing the probability of error detection). Local assignment of QC ranges, as described above, will allow tighter quality control but at the risk of high false rejection rates when a single 2SD cut-off is used.

For quantitative tests requiring manual recording, a two-tier acceptance/rejection criteria would improve error detection rates. A flow chart based on the system used within the Australian Government’s point of care testing in general practice trial is provided in Figure 1(b). ¹⁴ This system classifies the results of high and low IQC as green (within 2 SD of mean), amber (between 2 and 3SD) or red (greater than 3SD). The combination of the high and low QC colour code is then used to decide if the QC has passed or failed. ‘Red’ results should immediately initiate review by the POCT team via an alert from the end user, while the number of green/amber results should be reviewed by the POCT team on a regular basis, e.g. monthly. Standardized flow charts plus IQC documentation forms, such as those shown in Figure 1, could be used across analysers to aid correct interpretation and initiate appropriate action.

Figure 1.

Example of (a) a form, for manual IQC documentation plus (b) associated flow chart for user interpretation of results. Figure 1(b) is modified from the quality control action sheet provided to users in the Australian Government’s point of care testing in general practice trial. ¹⁴

For many POCT devices, the immediate decision of whether IQC is accepted is automatic, and lock-out features may be available to prevent patient testing. However, it is important that the POCT team understands the capacity and limitations of any inbuilt IQC features and customize protocols as required. For example, the i-STAT 1 hand-held lock-out feature only prevents processing patient samples if the allowed time between IQC has passed, not following a failed IQC. ¹⁵ We advise against the use of STAT features which allow patient samples to be processed without conduct of IQC within the required time frame. If the POCT service is fully risk managed, including performance of adequate IQC, then there can be no argument for using devices with expired IQC. The feature could also encourage poor compliance with regular IQC testing.

Automatic IQC review can be limited by wide manufacturer-assigned acceptance ranges, and acceptance rules which fail to detect early deterioration in performance. Locally assigned ranges in combination with analyte-specific rules could be used to maximize error detection of poor performing assays and minimize the probability of false rejection of well performing assays. Such an approach has begun to be accepted in laboratory settings and it could be applied within POCT provided there is the ability to customize IQC by analyte and that interpretation is automated.

Implementation of assay-specific rules requires definition of assay quality (the sigma metric) followed by application of IQC rules appropriate to the assay quality. The sigma metric is calculated as follows:

$\text{Sigma}{\text{metric}=(\text{TE}}_{\text{a}}-{\text{Bias}}_{\text{meas}})/{\text{CV}}_{\text{meas}}$

where TE_a is the percentage total allowable error, Bias_meas is the percentage bias of the assay and CV_meas is the coefficient of variation (precision) of the assay. Both bias (compared to the central laboratory) and precision should be available from the initial analyser evaluation. Total allowable error values can be obtained from tables of TE_a based on biological variation, hosted on the Westgard website (www.westgard.com, last updated 2014). ¹⁶ However, definition of TE_a by clinical requirement is more appropriate for some analytes, for example cholesterol.

Using the sigma metric definition, it has been shown that the quality of assays varies significantly between POCT analysers and the analyte tested. ¹⁷ For example, the sigma scores for analytes measured on the Vitros DT60 range from negative values for cholesterol (i.e. the bias alone exceed the total allowable error) to excellent (i.e. >6) for triglyceride. ¹⁷ Once calculated, the sigma metric can be used to inform the quality of the assay and the amount of IQC required for an analyte, as shown in Table 2. ¹⁸

Table 2.

Summary of the recommendations made by the 2010 convocation of experts on laboratory quality on the use of six sigma to initiate IQC design. ¹⁸

Sigma score	Performance	IQC design 18
>6 σ	Excellent	Once per day
		One level per day (alternating levels)
		13.5s rule
4σ to 6σ	Suited to purpose	Once per day
		Two levels per day
		Single IQC rule e.g. 13s
3σ to 4σ	Poor performers	Twice per day
		Two levels of IQC per day
		Multi-rule system
<3σ	Problematic	Three times a day
		Three levels
		Maximum IQC rules

IQC: internal quality control.

Maintaining an overview of IQC

It is the POCT co-ordinator’s responsibility to maintain an overview of IQC results. A monthly review should be conducted to identify persistent failures and perform trend analysis, a process made significantly simpler if all IQC results are held centrally, i.e. via IT link. ¹⁹

IT connectivity can automate IQC result collection, streamline IQC review and enable assay/user lock-out. A centralized IT system further allows comparison of coefficient of variation values (CV) and compliance rates across sites, enabling early identification of users/sites requiring retraining. Ideally, the IT connection should be a two-way link, permitting customization of IQC from a single point.

There are a limited number of vendor-neutral IT systems enabling connectivity of POCT systems from different manufacturers. Data managers developed specifically for POCT include those from Conworx Medical IT Ltd. (POCcelerator™) and Alere Ltd (AegisPOC™Data Management System). The major advantage of a single central IT system is the potential cost reduction of connecting a number of different devices and a reduced training burden on the POCT team using the system.

Troubleshooting

POCT systems are largely closed systems and manufacturer intervention will usually be required when deterioration in the fluidics, reagents or sensor is suspected. Nevertheless, there should be a defined protocol detailing the expected actions of a user following a failed IQC.

In our experience, manufacturers suggest an immediate action of repeating the IQC as the first stage of a troubleshooting protocol, and continuing use of the analyser if the repeat IQC is within range. On a system with disposable strips/single-use cartridge, this action is acceptable (as a component of the device has been changed); however, persistent failures should be investigated. A repeat test should not be advised as an action for multi-use cartridge devices.

Any troubleshooting protocol should be developed with knowledge of the most common errors and expected user capability. A local audit of glucose meter IQC failures over a four-month period found that 77% of 118 IQC failures were likely due to transposed high- and low-IQC materials. The troubleshooting protocol therefore requests the user checks that IQC material is assayed in the correct order as a first step. The protocol also requires the user to check expiry of consumables (reagent/cartridge/strip) and storage conditions of consumables prior to calling the POCT team for assistance. Regular engagement of the POCT team with users will also encourage early contact in the event of a possible IQC non-conformance.

Summary

The paucity of literature for IQC in POCT means that an evidence-based guideline cannot be provided at this time. It is also not possible to distinguish what should constitute best practice for the community versus acute care settings. The evidence base for IQC in the laboratory is increasing but this has not yet been matched by a corresponding evidence base for IQC in POCT.

In the absence of an evidence base, our suggestions reflect minimum levels of IQC, and individual users may decide to do more frequent testing. However, our aim is to promote more effective, and not necessarily more frequent, IQC. We hope our article will encourage debate between laboratory scientists plus engagement with manufactures so that an entirely evidenced-based practice guideline for IQC in POCT can be written in the future.