Audit of internal quality control practice and processes in the south-east of England and suggested regional standards

Abstract

Abstract

Background

Internal quality control (IQC) has a long and well-established role in clinical biochemistry laboratories. However, despite the duration of use, and the publication of several articles detailing best practice, the implementation and use of IQC vary significantly between institutions. Consequently, the North Thames Audit and Quality Assurance Group undertook a region-wide audit of current IQC practice in 2006.

Methods

On aspects of IQC testing, interpretation and laboratory processes, 54 laboratories in the region were audited.

Results

Audit data showed significant variability in all aspects of practice, including IQC frequency, use of appropriate material, statistical processing and grades of staff involved.

Conclusions

Some of the variation in practice may affect the effectiveness of laboratory IQC, and thus the adequacy of a laboratory to monitor system performance. Consequently, a set of proposed regional standards have been developed and disseminated, prior to re-audit at a future date.

Introduction

Internal quality control (IQC) is considered to play a major role for the majority of staff working in diagnostic laboratories. The use of IQC has a long history, ¹ and has been progressively modernized. ^2,3 However, as noted by Westgard, ³ despite this longstanding use of IQC procedures, actual practice is often far from ideal. The central role of IQC is to detect clinically important errors in the analytical process. Since a significant, but perhaps insufficient proportion of laboratory resource (both financial and human) is allotted to quality control procedures, greater effort should be made to ensure that IQC is efficient, fit for purpose and both clinically and cost-effective.

Anecdotally, it was felt that IQC procedures within the region varied widely in numerous aspects of practice, and this was surveyed as a ‘regional’ audit undertaken by the North Thames Audit and Quality Assurance Group.

Methods

The audit questionnaire was circulated to 54 laboratories in the North and South Thames and Eastern Regions. The questionnaire is not reproduced due to length (but can be obtained from the corresponding author if required), but in general it aimed to establish:

The frequency of IQC assessment and whether this is at set time intervals or after a fixed number of patients' samples;

Variation in IQC practice between standard working hours and on-call/out-of-hours periods;

IQC and use of appropriate matrices for non-serum based analytes;

Number of analyte concentrations in a standard IQC assessment;

Number of samples used to assign values to unassayed IQC material;

How target concentrations are chosen;

Use of patient daily means;

Use of Westgard multirules;

Grades of staff who accept/reject IQC runs;

Reasons for accepting failed IQC;

Grade of staff monitoring overall IQC performance;

Use of Six Sigma and frequency of IQC meetings.

All questions asked were in relation to analytes provided from a laboratory's main chemistry analytical platform, unless otherwise stated. Returned questionnaires were analysed and standards were produced after open discussion at a regional audit meeting (May 2006). Standards were based on Westgard 2003 ³ and Clinical and Laboratory Standards Institute (CLSI, formerly NCCLS) standards. ⁴

Results

Twenty-nine returns (54% return rate) were received from 18 District General, eight Teaching and three Tertiary NHS Trusts.

As opposed to seven sites processing IQC at fixed times of the day (1 at daily set-up/maintenance; 1 at 08:00 h and 16:00 h; 1 at 3 or 4 hourly intervals [day time only] and 1 at 24 h intervals), 19 sites indicated that they process IQC at fixed time intervals (range 1–24 h) (Figure 1). Three sites indicated that IQC material is processed after a fixed number of samples (1 after every 30 samples for all analytes, 1 after every 50 samples for all analytes and 1 after every 100 samples for electrolytes and albumin). Seventy percent of labs who responded had no policy detailing IQC practices during on-call/out-of-hours periods, and of these, eight labs specifically detailed how IQC practice differs between the routine working day and out-of-hours periods (Table 1).

Figure 1

Frequency of internal quality control (IQC) assessment by analytical system manufacturer. Each bar represents a separate laboratory

Table 1

Variation in frequency of internal quality control (IQC) processing between routine working hours and ‘out-of-hours’ periods

Frequency of IQC during routine working day	Frequency of IQC during on-call periods/outside of routine working day
Every 50 samples	Every batch
Every hour	2 hourly until midnight then at BMS discretion
Every hour	Concurrent samples
2–4 hourly	Regularly at BMS discretion
3–4 hourly	No IQC between 17:00 and 09:00 h
4 hourly	Midnight and 08:00 h
4–5 hourly	19:00, 23:00 and 06:00 h
2–6 hourly	None for general chemistry/CRP. TDM as required

BMS = Biomedical Scientist; TDM = therapeutic drug monitoring

Based on the frequency of IQC and the typical sample throughput, laboratories were asked to estimate the number of sodium measurements on patients' samples that would be made between IQC evaluations (Figure 2). Numbers processed between IQC samples ranged from 30–1000 (367 ± 63 [mean ± standard error of the mean]), with significant variation between users of the same analytical platform (n = 24).

Figure 2

Laboratories were asked to estimate the number of patients' sample sodium measurements made between formal internal quality control (IQC) assessments. The calculation was based on the length of time between IQC evaluations and the laboratory sample throughput. Each bar represents a separate laboratory

Laboratories were asked to provide information on whether they considered that they used appropriate matrices for IQC material. For the following urine analytes, all of the sites that responded reported using an IQC material based on an appropriate matrix: electrolytes, calcium, urine albumin (microalbumin) and urea/creatinine. However, for a range of other analytes, this was less frequently the case (Table 2). The main reasons quoted for failure to use an appropriate matrix were availability and cost. Additionally, several analytes were listed that never undergo IQC procedures (Table 3). Evaluation of how IQC target values are assigned within laboratories showed:

Thirty-five percent utilize assayed IQC material for at least some assays, without re-assigning values locally;

Sixty-six percent use an unassayed material and re-assign values locally;

Seventy-two percent of laboratories use assayed IQC material and re-assign values based on local evaluation. However, there was considerable variation in the number of samples, and nature of investigations used to re-assign target values to IQC material (Table 4).

Table 2

Percentage of laboratories that consider they utilize internal quality control (IQC) material based on an appropriate matrix for a selection of non-serum based analytes

Analyte	Proportion (%) of laboratories using appropriate matrix for IQC material
	Yes	No	No answer
Urine protein	94	3	3
Urine albumin (microalbumin)	90	–	10
CSF protein	45	52	3
CSF glucose	24	76	–
Fluid LDH	3	88	9
Fluid amylase	9	91	–

CSF = cerebrospinal fluid; LDH = lactate dehydrogenase. The term fluid applies to any material other than serum, plasma or urine

Table 3

Analytes in use that are never subject to internal quality control procedures

Analytes	Number of labs
Urine amylase	2
Urine and fluid amylase	1
Fluid glucose	1
Fluid amylase, albumin, LDH and CSF glucose	1
Fluid LDH and amylase	1
Urine amylase and myoglobin; CSF LDH and xanthochromia	1
Fluid triglyceride	1

CSF = cerebrospinal fluid; LDH = lactate dehydrogenase

Table 4

Number of internal quality control (IQC) samples analysed prior to re-assigning target and range values

Number of IQC samples analysed	Number of laboratories
Provisionally 3, then re-assessed at 15	1
10–15	2
20*	11
20–25	1
>30	1
At least 50	1
Adjusted every month (no number stated)	1

*One lab with minimum of one month run in; one lab re-assessed after 1000 measurements

As a form of regular IQC, 24% of laboratories use patient daily means, predominantly for selected analytes such as electrolytes, calcium and albumin. One laboratory has adopted Six Sigma processes into routine laboratory practice.

Eighteen laboratories (62%) reported using Westgard multirules as their primary form of IQC data analysis. The remainder used various rules including single 2 standard deviation (SD) rules (6 labs) and single 1.5 SD rules (1 lab). To all analytes at all concentrations, 22 labs (76%) apply the same multirules, and only one lab reported using analyte-specific rules.

A wide range of different staff grades were reported as being permitted to reject IQC data in realtime (i.e. confirm a method is outside its performance criteria) (Table 5). Likewise, there was significant diversity in what grades of staff can accept failed IQC and continue processing/authorizing patients' samples (Biomedical Scientist [BMS] 1 = 8 labs; BMS2 = 14 labs; BMS3 = 3 labs; BMS4/clinical scientist = 1 lab) and only three labs reported that failed IQC would not be accepted. The frequency with which ‘non-ideal’ IQC is accepted was reported as:

Rarely (n = 10);

Occasionally (n = 3);

Quarterly (n = 1);

Weekly (n = 2);

Two to three times per week (n = 1);

Daily (n = 1);

Only if within 3 SD and subsequently investigated (n = 1).

Table 5

Grades of staff permitted to reject internal quality control (IQC) in realtime

Staff grade	Laboratories allowing staff to reject IQC (%)	Laboratories not allowing staff to reject IQC (%)	No answer (%)
Trainee grades	7	42	51
Qualified but unregistered staff	3	71	26
MTO	10	59	31
BMS 1	90	0	10
BMS 2	93	0	7
BMS 3	86	0	14
BMS 4	48	24	28
Clinical Scientist (state-registered)	45	21	34
Consultant Clinical Scientist	35	34	31
SpR	3	34	63
Consultant Chemical Pathologist	7	31	62

BMS = Biomedical Scientist; MTO = Medical Technical Officer;

SpR = Specialist Registrar (Chemical Pathology)

The reasons stated for why failed IQC is accepted in allowing patients' specimens to be analysed were:

‘Only if not clinically significant’ (7 labs);

‘Lack of reagent prevents change’ (1 lab);

‘Lack of staff and pressure to reduce turnaround times’ (1 lab);

‘Checked with External Quality Assessment ’ (1 lab);

‘If IQC is very close to limit, if QC deteriorates before new QC is ready (e.g. prior to new range being established’ (1 lab);

‘Only if all attempts to correct problems have failed’ (1 lab);

‘Only if no reason can be found and only one QC is out’ (1 lab);

‘If lactate dehydrogenase IQC is out, reagent is changed the next day’ (1 lab).

When IQC has failed the respective rules in use, various systems are in place to prevent patients' samples from being inappropriately reported:

• Chemistry disabled on analytical system = 20 labs;

• Data not transferred to Laboratory Information System (LIS) = 20;

• Results not validated by LIS/data manager = 7;

• Operators do not process specimens = 20.

At a range of frequencies from fortnightly to every two months, 20 laboratories (69%) hold regular designated quality control meetings. Twenty-one laboratories (72%) also discuss quality control at management and clinical governance meetings. However, one laboratory did not discuss quality control at any meeting.

Discussion

IQC is primarily utilized in routine practice to monitor system performance in realtime (i.e. make comparisons to what is expected under stable conditions), and allow analytical failures that affect performance to be detected. However, many of the variations in practice reported by laboratories could have the potential to adversely affect the ability of laboratories to detect these errors. Variation in IQC practice may also be incompatible with the future of pathology modernization and greater consideration should be given to the harmonization of IQC effectiveness. We believe that the responses received were representative of IQC practice within the region surveyed, and consequently, the North Thames Audit and QA Group have developed standards as discussed below to help improve IQC planning and implementation within the region covered by this audit, although they may also form a useful basis upon which other local/national guidance is based. While no definitive evidence exists to define best practice in IQC, expert opinion literature was used to define these standards. ^2–4

Significant variation existed for the frequency of IQC and run length (number of samples processed) between IQC evaluations, which may presumably, at least in part be a reflection of a combination of poor or lack of IQC planning and confidence in methodology. More significantly however, many laboratory policies only detailed IQC practice for the traditional standard working day (usually 09:00–17:00 h, Monday to Friday), leaving ‘out-of-hours’ IQC to either the discretion of on-call or shift staff, or in many cases not processing IQC at all. The length of time separating patients' sample analyses and IQC testing should not be dependent on the time of day and IQC materials should be tested in the same manner as patients' samples. ⁵ Standard 1 therefore states:

STANDARD 1: IQC policies should cover 24 h period. If laboratories run shift or on-call services, the policies for these times should also clearly state what IQC is required. Performance of IQC during on-call periods should not be on a discretionary basis, and all staff should adhere to the same minimum standards, discretionary testing should be confined to problem solving only.

IQC practice for some non-serum based analyses was also less robust than for the equivalent analyte in serum. Availability and prohibitive cost were commonly listed as causes for not using an appropriate matrix, but a significant number of urine and fluid analytes were not subject to any IQC assessment. No standard was specifically developed for this, but it should be highlighted as an area of concern.

Other than the correct application of statistical rules, the ability of IQC procedures to detect error relies on the target values and SD ranges being appropriately assigned. The audit showed significant variation, ranging from simple adoption of manufacturer assigned values through to wide ranging opinion on appropriate studies required prior to assigning new ranges to IQC material. Since manufacturer assigned ranges on IQC material are of greater relevance to inter–laboratory rather than intra-laboratory situations, they are too wide for application as control limits. Therefore, standard 2 reads:

STANDARD 2: Target values and ranges for IQC material should be assigned locally for each instrument. Manufacturer target levels and ranges should not be used.

Additionally, it is also suggested that EP5-A2⁴ standards from the CLSI should be used as the criteria upon which practice for defining ranges is based (e.g. how many samples, how many analytical runs, how many days, etc.). As noted above, the nature and source of IQC material utilized was diverse, including poor use of appropriate matrices, and use of IQC materials supplied by the manufacturer of the analytical methodology. Standards 3 and 4 therefore state:

• STANDARD 3: Labs should use IQC material from a third party source.

• STANDARD 4: Use EP5-A2 or equivalent to determine IQC values within each individual laboratory. ⁴

Several laboratories suggested that their policy for establishing IQC ranges differed to the above standard, and in particular they used single runs of samples that would adversely affect the estimate, if the run did not truly reflect normal operating parameters. Single observations are also highly dependent on the number of days, which they are run.

The majority of laboratories use multirule (‘Westgard’) analysis of IQC data, although significant numbers continue to use other rules such as single 2 SD. These single rules are problematic, and may have cost implications for laboratories, as for example, if three levels of IQC are processed using single 2 SD rules, 18% of runs could be falsely rejected for statistical reasons alone.

With respect to statistical analysis, the most common solution in this audit is to apply the same rules to all analytes, and not to consider the quality requirements of each test separately. Consequently, it has been proposed that laboratories define the quality requirement for a test, assess the method precision and bias and then define an appropriate IQC strategy. ³ Standards 5 and 6 therefore state that:

STANDARD 5: Encourage method specific rules (single and multi) based on the required quality of the assay, and known method bias and coefficient of variation.

STANDARD 6: Discourage inappropriate use of single rules, but when used ensure they have required degree of error detection.

Only 10% of laboratories use failed IQC to indicate that a true failure of the process has occurred and cease processing patients' samples. If IQC is planned and implemented correctly, and assuming that the only variation being measured is that within the method itself, and statistical rules with the appropriate error detection are used, then clinically significant errors should be detected and false rejections should be minimized. Thus many of the examples detailing why laboratories continue to process patients' samples should be invalid. It is of concern however, that at least one laboratory reported staffing pressures and attempts to reduce turnaround times had been allowed to compromise IQC policy.

Additionally, a very wide range of staff, including training and non-state-registered grades were routinely making decisions on when to accept and reject IQC data and continue processing patients' samples. Although no definitive evidence exists, it has previously been shown that quality of laboratory performance can be influenced by the experience of the staff reviewing IQC data. Additionally, separating analytical and IQC acceptance duties (usually by a senior member of staff) has been noted to be beneficial. ⁶ Standards 7 and 8 therefore state:

STANDARD 7: Review grades of staff accepting and rejecting IQC. The use of trainee and other staff that are not state-registered to accept and reject QC should be discouraged or supervised.

STANDARD 8: If IQC policies are robust, and the correct IQC with appropriate error detection is used, failed IQC should not be accepted (e.g. to continue processing patients' samples). Procedures for accepting IQC because the ‘errors are not clinically significant’ should not be used as IQC failure should be viewed as highlighting significant problems if undertaken correctly.

The increasing use of analytical systems comprising multiple modules and IQC practice was not specifically questioned, but after discussion, the final standard was suggested as:

• STANDARD 9: For analysers that have multiple modules, each module should be assessed individually with IQC on each assessment.

The aim of these standards is to promote laboratories to adopt a total IQC programme that maximizes the cost effectiveness of IQC, while establishing IQC procedures that minimize the number of false rejections and maximize the detection of medically important errors. Laboratories should be encouraged to establish and define control limits based on their own data and then to reject out of control runs and eliminate problems prior to proceeding/continuing with the analysis of patients' samples. The use of modern IQC planning techniques, and a move toward IQC based on the quality requirements of a test is also encouraged. Following implementation of these standards, re-audit, perhaps on a wider basis will be required to complete the audit cycle, and establish that improvement in practice has occurred.