Sage Journals: Discover world-class research

Abstract

A review of the literature pertinent to interpretation of biochemistry data and quality control (QC) and proficiency testing data from 2 biochemistry analyzers was used to determine clinical quality requirements for biochemistry assays, characterize the performance of and calculate sigma metrics for the analytes run on the 2 analyzers, and perform QC validation in order to determine the needs for statistical QC for each analyzer. Quality requirements suitable for the analytes based on the needs of the authors' laboratory are presented. These requirements may or may not be appropriate for other laboratories, depending on the needs of the clients, species, and equipment performance capability. The majority of the analytes were easily controlled using the 1_3s control rule, with a sigma metric approaching or exceeding 6 and with a high probability of error detection and a low probability of false rejection. Some analytes could not be controlled using the 1_3s rule, and additional control rules with a greater number of control data points were required. There were differences between performances of the 2 analyzers. The findings in the present study emphasize the need for QC specific for the analyte and the clinical decision level and the need for separate QC validation on every instrument.

Keywords

Quality control quality control validation sigma metrics veterinary laboratory

Introduction

Quality planning in veterinary laboratories includes defining quality standards as the foundation for quality laboratory processes, quality control (QC), quality assessment (QA), and quality improvement. The definition of quality requirements for various laboratory tests is an important aspect of veterinary quality planning as regulatory requirements applied to human laboratories do not exist for veterinary laboratories. The quality requirements, expressed as total allowable error (TE_a), should indicate the degree of change that needs to be detected in an analyte for a clinically important decision to be made with regard to further investigation or treatment. For example, the reference interval for canine albumin used in the authors' laboratory is 25–41 g/l. A decrease in albumin from 25 to 24 g/l (a 4% change) is unlikely to stimulate further investigation, so a change as small as this does not need to be detected. However, a change from 25 to 22.5 g/l (a 10% change) is more likely to be clinically significant. 16 Therefore, the quality requirement determined for this test is TE_a = 10%.

Quality requirement varies greatly between analytes. For example, serum electrolyte levels are strictly regulated physiologically; therefore, small changes are likely to be clinically significant. In contrast, liver enzymatic activities show much greater variability; therefore, much larger increases are required to cause a clinically significant change that warrants further investigation or treatment. The quality requirements selected are likely to differ between laboratories based on the clientele, species of interest, and use of the laboratory data.

Once quality requirements have been established, the requirements can be analyzed with reference to actual performance within the laboratory in order to

determine if this level of quality is achievable given the actual performance of the analyzer(s) used;

determine what QC is needed to detect variation that affects the clinical interpretation of the results; and

determine the need for quality improvement.

Quality control validation is used to determine the statistical QC procedures that are appropriate for detecting errors that may be important in clinical interpretation of the test and to determine the QC rules that can be applied to each test in order to have a high probability of error detection (P_ed) and low probability of false rejection (P_fr) of test results. Quality control validation is the final step in validation of methods and instruments, and it ensures that appropriate statistical QC is being applied. 4

Sigma metrics are tools that are rapidly gaining popularity in laboratory applications (Riebling N, Tria L: 2005, Six Sigma project reduced analytical errors in an automated lab. Available at http://www.mlo-online.com/articles/0605/0605clinical_issues.pdf. Accessed January 9, 2008; Westgard S: 2003, From method validation to Six Sigma: translating method performance claims to into sigma metrics. Available at: http://www.westgard.com/lesson78.htm. Accessed April 4, 2008; Westgard JO: 2002, AACC expert access: a Six Sigma primer. Available at: http://www.westgard.com/sixsigprimer.htm. Accessed April 4, 2008). 5,8,9,12–15 Six Sigma is a process quality measurement and improvement program developed by Motorola in the early 1980s. Sigma methodology can be applied wherever an outcome of a process can be measured. A poor outcome is counted as an error or defect. This is quantified as defects per million (DPM). Sigma (σ) is the mathematical symbol for standard deviation (SD).

Approximately 99.73% of all results from a normal population (i.e., results that are equally distributed above and below the mean) fall within 3 SDs of the mean. Six Sigma focuses on controlling a process to 6 SDs, which equates to 3.4 DPM opportunities. Achievement of Six Sigma quality is considered to be a standard of excellence. 12 Performance at the 3-sigma level is considered the minimum acceptable quality for a production process. 12 In simpler terms, a higher sigma metric means the systematic error that must be detected to help ensure accurate results by the use of statistical QC is large and should be more easily detected. A lower sigma metric means QC must detect smaller systematic errors, which is more difficult. Six Sigma philosophy purports that there is a direct relationship between the number of product defects, wasted operating costs, and levels of customer satisfaction. Therefore, as sigma increases, the reliability of the test improves, operating costs are reduced, and customer satisfaction is increased:

This Equation has been deleted and replaced with an image

where TE_a is the total allowable error; Bias_obs is the bias, or accuracy, of the test; and S_obs is the coefficient of variation (CV), or precision, of the test. All are expressed as percentages.

Laboratory performance can be evaluated using Six Sigma. The process involves the following:

•

Defining the quality requirements for the test (expressed as TE_a);

•

Quantifying the performance of the test using standard statistical QC methods;

•

Analyzing the data and calculating a Six Sigma value;

•

Improving and optimizing the process based on the results of analysis; and

•

Continuing the measurement and control of the test.

The current report describes the authors' experience of sigma metrics and QC validation for 2 biochemistry analyzers (Olympus AU640 and AU2700) a used in a veterinary reference laboratory. The objectives of the study were to

determine clinical quality requirements for biochemistry assays based on a review of the literature and current interpretation of clinical biochemistry profiles for common domestic species seen in the authors' laboratory (dog, cat, and horse);

characterize the performance of, and calculate sigma metrics for, the tests run on the 2 analyzers to determine if significant differences in performance are present, and the best approach for optimizing the statistical QC for the clinical biochemistry assays (both analyzers were introduced into the laboratory at the same time and were new instruments at the time of installation); and

perform QC validation to determine the needs for statistical QC for each analyzer.

These objectives would enable the authors to understand the performance of the 2 analyzers in use in the laboratory, determine if significant differences are present, and provide the best approach for statistical QC. Reporting these findings may encourage and help other veterinary investigators and diagnosticians to analyze the performance of Olympus or other biochemistry analyzers in their own laboratories.

Materials and methods

Laboratory materials

Internal statistical QC data was extracted from the Olympus AU2700 and AU640 biochemistry analyzer records for the previous 6 months. Control materials were run daily. The number and levels of controls were selected on the basis of traditional internal QC practice, traditional availability, and cost. 4 Both low and high QC material were assayed at the beginning of each day, and then within each run, QC materials, alternating high and low materials, were assayed. b There were no more than 60 patient samples between QC materials.

Operation was considered stable during this time based on traditionally used, commercially available QC materials, and using an arbitrary choice of 3 SDs above and below the mean as acceptable performance (1_3s rule). No significant problems in performance were identified. Calibrations were done according to manufacturer's recommendations. The reference intervals used in the study are the authors' laboratory-established reference intervals for adult animals.

Chemistry analytes included albumin, alkaline phosphatase, alanine aminotransferase, amylase, aspartate amino-transferase, bile acids, total bilirubin, total calcium, chloride, cholesterol, creatine kinase, creatinine, gamma glutamyl transferase, glucose, lipase, inorganic phosphorus, potassium, sodium, total protein, and urea. The methodology used to measure each analyte is summarized in Table 1.

Quality requirements for analytes

The quality requirements for various analytes have not been previously defined. Defining quality requirements has been recommended as a starting point for quality planning by Westgard 13 and the National Committee for Clinical Laboratory Standards 1 and is implied by International Organization of Standardization requirements for “fitness for use.” 6

Analytic quality requirements were chosen based on review of the veterinary literature 2,3,7,10,11,16 and routine clinical use of data for medical interpretations based on internal clinical pathology discussions within the authors' laboratory. Although the Clinical Laboratories Improvement Act (CLIA) guidelines exist for human laboratories in the United States, there are no clinical quality requirements for veterinary laboratories in any country. Quality requirements were determined for cats, dogs, and horses, as these are the species most commonly encountered in the authors' laboratory. Quality requirements were determined for high, low, and/or reference values for each analyte. The quality requirements eventually used in the study were all either high or low. The level at which a change should be detectable by the assay is called the clinical decision level. The strictest of these quality requirements was used as the starting point for QC validation. For example, for creatine kinase (reference interval: 20–225 U/l) there was a need to detect a change in creatine kinase activity at approximately 340 U/l on exercise tolerance tests in horses with suspected rhabdomyolysis (clinical decision level of 340 U/l). The requirement in cats and dogs was considered to be less strict than this; therefore, the equine quality requirement was chosen as the most stringent for the study. The calculation for quality requirement is shown.

This Equation has been deleted and replaced with an image

Example using creatine kinase activity in the horse (reference interval: 20–225 U/l):

This Equation has been deleted and replaced with an image

Performance data

For QC validation, internal quality control (IQC) data were used to determine the CV. External quality assurance (EQA) data b,c were used to determine bias for each analyte. The QC materials currently used in the authors' laboratory had not been specifically chosen based on the levels of clinical significance but on factors such as cost and commercial availability. The level of control material or QA data point closest to the level of clinical significance with the most stringent quality requirement was used to determine the CV and bias (Westgard S: 2003, From method validation to Six Sigma).

The CV was estimated by calculating the mean from 6 months of IQC data. The CV for the control material most closely approximating the level of clinical significance with the most stringent quality requirement was used:

This Equation has been deleted and replaced with an image

The bias was determined by calculating the mean bias from EQA reports for 6 months for the control material most closely approximating the level of clinical significance with the most stringent quality requirement:

This Equation has been deleted and replaced with an image

Analysis

Westgard rules are a commonly used system of multiple rule QC developed by Westgard and colleagues in the 1980s. 14 The rules can be used to decide when a control result should be rejected, necessitating additional investigation to determine if accurate results are obtained. The Westgard rules are abbreviated to a shorthand form for simplicity. The number of control data points to be considered is indicated by an Arabic numeral. The acceptable variation (in SD[s]) from the mean is indicated by a subscript. For example, the 1_3s rule: when a single result is more than 3 SDs above or below the mean (expected value for the control), the 1_3s rule has been violated. Similarly, the 2_2s rule is violated when 2 consecutive controls are 2 SDs above or below the mean. There are many variations on these rules, and the rules can be added together to create multi-rules (e.g., 1_3s/2_2s/R_4s). Multi-rules are applied in sequence and are triggered by a violation of the 1_3s rule.

When applying any criteria (including Westgard rules) for acceptability of control data, a high probability of error detection and a low probability of false rejection are desirable. Westgard's recommendation for >90% P_ed and <5% P_fr as desirable criteria for biochemistry applications was used in the present study.

The EZ Rules d statistics package was used for analysis of the results, and a sigma value was produced for each analyte on each machine. The data are entered into the EZ Rules software using a “startup” design as the most stringent statistical QC requirement. The program is arranged in a series of self-prompting steps. For the section on expected test instability, the <2% option was selected as excellent stability from the authors' experience with the instruments and reagents. Two control materials were indicated to be the preferred option for number of control materials.

An Opspecs (operating specifications) charte is then produced automatically by the program, which shows the operating point for the assay. The sigma metric result, selected QC rule, probability of false rejection (P_fr), and the probability of error detection (P_ed) are displayed. The sigma metric result is calculated using Equation 1. A P_ed of >90% and a P_fr <5% were considered desirable for the startup QC (Westgard JO: 2006, Good laboratory practices (GLPs) for statistical quality control. Part II: QC limits and limitations. Available at: http://www.westgard.com/essay103.htm. Accessed April 4, 2008).

Results

The quality requirements for the biochemistry analytes are presented in Table 2. The most stringent requirements based on review of standard texts for dogs, cats, and horses were chosen. 1,3,4,6,9,11 A summary of the QC validation information for both analyzers is presented in Table 3.

The 1_3s rule was applied as the starting rule for all assays because it was applicable with >90% P_ed and <5% P_fr for 24 out of 29 analytes on the AU640 analyzer, and 21 out of 29 analytes on the AU2700 analyzer. To achieve a P_ed of ≥90% and a P_fr of ≤5% for those analytes highlighted in Table 3, different control rules and/or numbers of control material data points would be needed (see Tables 4 and 5).

Discussion

The choice of clinical quality requirements for each analyte, expressed as TE_a, was based on a review of the literature and current interpretation of clinical biochemistry profiles for common domestic species seen in the authors' laboratory (dog, cat, and horse). The requirement was selected based on the change in the analyte that would need to be detected in order to make a clinical decision based on that change. For many of the analytes (for example, all of the enzymes), the change that would affect a clinical decision is relatively large (up to 50% for alanine aminotransferase and lipase activities). However, for some of the analytes, a relatively small change will affect the clinical decision in the management of the case. For example very small changes (as little as 5% for sodium concentration) in electrolytes must be detected because they are tightly controlled physiologically. Unless the precision and bias are low, smaller amounts of error are difficult to detect and may require more rigorous IQC using multi-rule(s) and/or larger numbers of control material data points.

Some of the quality requirements decided at the beginning of the study, based on the literature review and desired clinical application, either did not have a statistical QC solution or required extensive QC that was considered too laborious or expensive to be of practical use. For these analytes, the clinical quality requirement needed to be altered. For example, the desired glucose quality requirement at the lower end of the reference interval was the most stringent requirement for glucose, with a total allowable error of 10%. The CLIA guidelines also recommend a total allowable error of 10% in humans. When the instrument performance was determined for the glucose assay over 6 months, the sigma value was extremely low, and P_fr and P_ed unacceptable using this total allowable error. Repeating the calculations using different rules was unrewarding, so the decision was made to alter the clinical decision requirement to 20% for this analyte. For similar reasons, the sodium quality requirement was increased from 3% to 5%, total bilirubin was increased from 32% to 50%, and the calcium requirement was increased from 10% to 14–16%. These alterations were considered acceptable in order to achieve a high P_ed, low P_fr, and balance with regard to cost and number of control materials.

Knowledge of the achievable clinical quality based on individual instrument performance is important for clinical pathologists and/or clinicians interpreting results. It is important to realize the error that may be present due to bias and/or imprecision, as well as the amount of error that can be detected by the statistical QC used for that analyte.

Analytes that were easily controlled, with a sigma value approaching or exceeding 6, a high probability of error detection, and a low probability of false rejection included alkaline phosphatase, alanine aminotransferase, amylase, aspartate aminotransferase, bile acids, total bilirubin, calcium, cholesterol, creatine kinase, creatinine, gamma glutamyl transferase, lipase, potassium, and sodium (on the AU640 analyzer and for the high result on the AU2700 analyzer). Additionally, although the probability of error detection was slightly less than desired for albumin (AU640 analyzer), chloride (low level), and glucose (high level), it was felt that these results allowed strong confidence in the results obtained when a 1_3s rule was applied.

Analytes that were difficult to control (based on a low sigma value [<5] and P_ed of <95%) included albumin (AU2700 analyzer), low glucose (both analyzers), high glucose (AU640 analyzer), low inorganic phosphorus (AU2700 analyzer), low sodium (AU2700 analyzer), low total protein (both analyzers), and high total protein (AU2700 analyzer). The control rules and number of data points needed to achieve improved P_ed and P_fr are presented in Tables 3 and 4. These require multi-rules for QC. This application of multi-rules may require additional technician training and may not be easily flagged by instrument QC software.

Table 1.

Summary of the methodologies used to measure selected analytes

Analyte	Methodology
Albumin	At pH 4.0, bromocresol green (BCG) reacts with albumin to form an intense green complex. The absorbance of the albumin-BCG complex is measured bichromatically (600/800 nm) and is proportional to the albumin concentration in the sample.
Alkaline phosphatase (ALKP)	The rate in change in absorbance due to the formation of para-nitrophenol in the presence of 2-amino-2-methyl-1-propanol (AMP) is measured bichromatically at 410/480 nm and is directly proportional to the ALKP activity in the sample. Temperature: 37°C
Alanine aminotransferase (ALT)	ALT-catalyzed conversion of L-alanine + 2-oxoglutarate to L-glutamate + pyruvate, with subsequent lactate dehydrogenase (LDH)-catalyzed conversion of pyruvate + nicotinamide adenine dinucleotide (NADH[H⁺]) to L-lactate + nicotinamide adenine dinucleotide (NAD⁺). The decrease in absorbance due to the consumption of NADH is measured at 340/660 nm and is proportional to the ALT activity in the sample. Temperature: 37°C
Amylase	The procedure uses 4,6-ethylidene (G₇)-p-nitrophenyl (G₁)-α-D-maltoheptaoside (ethylidene-G₇PNP) as substrate. This substrate reacts directly with alpha-amylase and the fragments with α-glucosidase to give 100% release of p-nitrophenol (PNP). The resulting increase in absorbance at 410 nm is directly proportional to the α-amylase activity in the sample. Temperature: 37°C
Aspartate aminotransferase (AST)	AST-catalyzed transamination of aspartate and α-oxoglutarate, forming L-glutamate and oxaloacetate. The oxaloacetate is then reduced to L-malate by malate dehydrogenase (MDH), while NADH is simultaneously converted to NAD⁺. The decrease in absorbance due to the consumption of NADH is measured at 340/660 nm and is proportional to the AST activity in the sample. Temperature: 37°C
Bile acids	Bile acid (3-α-hydroxysteroid) is oxidized specifically by 3-α-hydroxysteroid dehydrogenase (3-α-HSD) and the oxidized form of β-thionicotinamide adenine dinucleotide (Thio-NAD) to produce 3-ketosteroid and reduced β-nicotinamide adenine dinucleotide (Thio-NADH). The products then undergo a cycling reaction; 3-ketosteroid is reduced to 3-α-hydroxysteroid, and NADH is oxidized in the presence of 3-α-HSD, amplifying the amount of bile acid. The bile acid level is directly proportional to the increase in absorbance measured at 410 nm. Temperature: 37°C.
Total bilirubin	Bilirubin in serum or plasma when mixed with buffer absorbs light at 450–470 nm with secondary readings at 600 660 nm. Adding an oxidant oxidizes the bilirubin to biliverdin by the presence of ferricyanide ions. The difference in absorption is measured using the analyzer's self-blanking facility. Temperature: 37°C
Total calcium	Calcium ions react in a chelating reaction with Arsenazo III in an acid medium to form a purple complex. The absorbance of the complex is proportional to the calcium concentration in the complex.
Chloride	Ion selective electrode measures the electrical potential of chloride ions in solution and compares it against a stable reference electrode of constant potential. The potential difference between the 2 electrodes will depend on the activity of the specific ion in solution; this activity is directly related to the concentration of that specific ion.
Cholesterol	Cholesteryl esters are hydrolyzed at C-3 to form free cholesterol and the subsequent oxidation step is to produce hydrogen peroxide (H₂O₂), which is quantified by the formation of a colored oxidation product. Cholesterol esters in serum are hydrolyzed by cholesterol esterase (CHE). The free cholesterol produced is oxidized by cholesterol oxidase to cholestene-3-one with the simultaneous production of H₂O₂, which oxidatively couples with 4-aminoantopyrine and phenol in the presence of peroxidase to yield a red chromophore (quinoneimine). The cholesterol level is directly proportional to the increase in absorbance measured at 540/600 nm. Temperature: 37°C.
Creatine kinase (CK)	CK reversibly catalyses the transfer of a phosphate group from creatine phosphate to adenosine diphosphate (ADP) to give creatine and adenosine triphosphate (ATP) as products. The ATP is used to produce glucose-6-phosphate and ADP from glucose (catalyzed by hexokinase, which requires Mg²⁺ ions for maximum activity). The glucose-6-phosphate (G6P) is oxidized by the action of the enzyme glucose-6-phosphate dehydrogenase (G6P-DH) with simultaneous reduction of the coenzyme NADP+ to give NADPH and 6-phosphogluconate. The rate of increase of absorbance at 340/380 nm, due to the formation of NADH is directly proportional to the activity of CK in the sample. Temperature: 37°C
Creatinine	The assay uses a kinetic modification of the Jaffe procedure, in which creatinine reacts with picric acid at alkaline pH to form a yellow-orange complex. The rate of change of absorbance at 520/800 nm is directly proportional to creatinine concentration in the sample. Temperature: 37°C
Gamma glutamyl transferase (GGT)	GGT catalyses the transfer of the gamma-glutamyl group from the substrate gamma-glutamyl-3-carboxy-4-nitroanilide to glycylglycine, yielding 5-amino-2-nitrobenzoate. The change in absorbance at 410/480 nm is due to the formation of 5-amino-2-nitrobenzoate and is directly proportional to the GGT activity in the sample. Temperature: 37°C
Glucose	Glucose is phosphorylated by hexokinase (HK) in the presence of adenosine triphosphate (ATP) and magnesium ions, to produce glucose-6-phosphate (G6P) and adenosine diphosphate (ADP). Glucose-6-phosphate dehydrogenase (G6P-DH) specifically oxidizes G6P to 6-phosphogluconate with the concurrent reduction of NAD⁺ to NADH. The change in absorbance at 340/380 nm is proportional to the amount of glucose present in the sample. Temperature: 37°C
Lipase	Pancreatic lipase hydrolyzes esters of long-chain fatty acids from their triglycerides. This requires the presence of co-lipase. Pancreatic specific 1,2-diglyceride is hydrolyzed to 2-monoglyceride and fatty acid. The 2-monoglyceride is measured by coupled enzyme reactions. Temperature: 37°C
Inorganic phosphorus	The method is based on a modification of the reaction of phosphate with molybdate. This reaction gives a colorless phosphomolybdate complex that can be measured directly by ultraviolet absorption at 340 nm. The absorbance at 340/380 nm is directly proportional to the inorganic phosphorus level in the sample. Temperature: 37°C
Potassium	Ion-selective electrode measures the electrical potential of potassium ions in solution and compares it against a stable reference electrode of constant potential. The potential difference between the 2 electrodes will depend on the activity of the specific ion in solution; this activity is directly related to the concentration of that specific ion.
Sodium	Ion-selective electrode measures the electrical potential of sodium ions in solution and compares it against a stable reference electrode of constant potential. The potential difference between the 2 electrodes will depend on the activity of the specific ion in solution; this activity is directly related to the concentration of that specific ion.
Total protein	Cupric ions in an alkaline solution react with proteins and polypeptides containing at least 2 peptide bonds to produce a violet-colored complex. The absorbance of the complex at 540/660 nm is directly proportional to the concentration of protein in the sample. Temperature: 37°C.
Urea	Urea is hydrolyzed enzymatically by urease to yield ammonia and carbon dioxide. The ammonia and α-oxoglutarate are converted to glutamate in a reaction catalyzed by L-glutamate dehydrogenase. Simultaneously, a molar equivalent of reduced NADH is oxidized. The rate of change in absorbance at 340/380 nm, due to the disappearance of NADH, is directly proportional to the urea concentration in the sample. Temperature: 37°C

Table 2.

Summary of clinical decision levels and calculated quality requirements

Analyte	Species with most stringent requirement	IDEXX reference interval	Clinical decision level (most stringent)	Calculated quality requirement (total allowable error; %)
Albumin (g/l)	Dog	25-U	Low: 22.5	11
Alkaline phosphatase (U/l)	Cat	<60.0	High: 80	25
Alanine aminotransferase (U/l)	Dog, cat	5-60	High: 120	50
Amylase (U/l)	Dog, cat	100-1,200	High: 1,800	33
Aspartate aminotransferase (U/l)	Horse	100-370	High: 500	26
Bile acids (μmol/l)	Cat	0.1-10	High: 13	23
Total bilirubin (μmol/l)	Dog, cat	0.1-5.1	High: 9.3	50
Total calcium (mmol/l)	Dog	2.45-3.10	Low: 2.1	16
			High: 3.6	14
Chloride (mmol/l)	Dog	100-116	Low: 95	5
			High: 122	5
Cholesterol (mmol/l)	Dog	3.2-6.2	Low: 1.9	68
			High: 10.0	38
Creatine kinase (U/l)	Horse	20-225	High: 340	34
Creatinine (μmol/l)	Dog	20-150	High: 180	17
Glucose (mmol/l)	Dog, cat	3.3-5.8	Low: 2.75	20
			High: 7.25	20
Gamma glutamyl transferase (U/l)	Horse	10-45	High: 60	25
Lipase (U/l)	Dog	0.1-200	High: 400	50
Inorganic phosphorus (mmol/l)	Dog	0.8-1.6	Low: 0.6	33
			High: 2.0	20
Potassium (mmol/l)	Dog	3.6-5.6	Low: 3.2	12.5
			High: 6.2	10
Sodium (mmol/l)	Dog	135-155	Low: 128	5
			High: 163	5
Total protein (g/l)	Dog	55-75	Low: 50	10
			High: 83	10
Urea (μmol/l)	Dog	2.5-6.7	Low: 2.0	25
			High: 8.0	16

Table 3.

Summary of quality control validation information on the Olympus AU640 and AU2700 analyzers.*

	Mean CV at decision limit		Mean bias at decision limit (%)		Calculated sigma value		P_ed		(%)P_fr (%)		Control rule
Analyte	AU640	AU2700	AU640	AU2700	AU640	AU2700	AU640	AU2700	AU640	AU2700	AU640	AU2700
Alb (L)	1.50	1.82	2.62	3.65	5.59	4.04	97	50	0.00	0.00	1_3s	1_3s
ALKP (H)	1.85	2.42	-2.52	-0.13	12.15	10.28	>98	>98	0.00	0.00	1_3s	1_3s
ALT (H)	1.70	2.50	-1.95	-1.45	28.26	19.42	>98	>98	0.00	0.00	1_3s1_3s
Amy (h)	1.28	1.87	-6.53	-1.30	20.68	16.95	>98	>98	0.00	0.00	1_3s	1_3s
AST (H)	1.70	2.63	1.06	0.07	14.67	9.86	>98	>98	0.00	0.00	1_3s	1_3s
BA (H)	4.55	4.78	-0.23	0.67	5.00	4.67	86	76	0.00	0.00	1_3s	1_3s
Ca (L)	1.78	1.87	2.75	1.80	7.59	7.06	>98	>98	0.00	0.00	1_3s	1_3s
Ca (H)	1.65	1.63	1.86	2.10	7.36	7.30	>98	>98	0.00	0.00	1_3s	1_3s
Cl (L)	0.73	0.93	-1.50	-0.42	4.79	4.92	80	84	0.00	0.00	1_3s	1_3s
Cl (H)	0.62	0.80	-98	-0.52	6.48	5.60	>98	97	0.00	0.00	1_3s	1_3s
Chol (L)	2.02	2.63	0.37	-1.40	33.48	25.32	>98	>98	0.00	0.00	1_3s	1_3s
Chol (H)	1.68	2.00	1.70	0.65	21.61	18.67	>98	>98	0.00	0.00	1_3s	1_3s
CK (H)	1.25	2.22	-4.65	-3.23	23.48	13.86	>98	>0.98	0.00	0.00	1_3s	1_3s
Crea (H)	1.67	2.45	-2.75	0.65	8.53	6.67	>98	>98	0.00	0.00	1_3s1_3s
Gluc (L)	1.95	1.93	12.50	13.77	3.85	3.23	41	14	0.0013	0.00	1_3s	1_3s
Gluc (H)	1.65	1.83	11.77	9.72	4.99	5.62	86	97	0.00	0.00	1_3s	1_3s
GGT (H)	1.53	1.17	-4.2	-3.82	13.59	18.10	98	98	0.00	0.00	1_3s	1_3s
IP (L)	2.90	6.80	-3.23	5.57	10.27	4.03	98	50	0.00	0.00	1_3s	1_3s
IP (H)	1.90	2.80	-1.83	1.70	9.56	6.54	>98	>98	0.00	0.00	1_3s	1_3s
Lipa (H)	3.45	3.02	-6.38	-7.40	12.64	14.11	>98	>98	0.00	0.00	1_3s	1_3s
K (L)	0.92	1.63	-1.85	-0.28	11.58	7.50	>98	>98	0.00	0.00	1_3s	1_3s
K (H)	1.05	1.15	-1.03	-1.03	8.54	7.80	>98	>98	0.00	0.00	1_3s	1_3s
Na (L)	0.70	1.03	-1.00	-0.40	5.71	4.47	>98	69	0.00	0.00	1_3s	1_3s
Na (H)	0.65	0.68	-1.22	-1.07	5.82	5.78	>98	>98	0.00	0.00	1_3s	1_3s
Tbil (H)	3.78	3.05	29.13	24.62	5.52	8.32	96	>98	0.00	0.00	1_3s	1_3s
TP (L)	1.63	2.47	-2.22	-1.35	4.77	3.50	79	26	0.00	0.00	1_3s	1_3s
TP (H)	1.35	2.23	-0.35	0.47	7.15	4.27	>98	61	0.00	0.00	1_3s	1_3s
Ure a (L)	2.10	3.23	1.68	3.27	11.10	6.73	>98	>98	0.00	0.00	1_3s1_3s
Urea (h)	1.97	2.23	1.75	3.40	7.23	5.65	>98	>98	0.00	0.00	1_3s	1_3s

P_ed = probability of error detection. P_ed of <90% are highlighted in boldface. CV = coefficient of variation; P_fr = probability of false rejection; Alb = albumin; ALKP = alkaline phosphatase; ALT = alanine aminotransferase, Amy = amylase, AST = aspartate aminotransferase; BA = bile acids; Ca = total calcium; Cl = chloride; Chol = cholesterol; CK = creatine kinase; Crea = creatinine; Gluc = glucose; GGT = gamma glutamyl transferase; IP = inorganic phosphorus; Lipa = lipase; K = potassium; Na = sodium; Tbil = total bilirubin; TP = total protein; H = high decision level; L = low decision level.

When comparing the performance and sigma metrics of the analyzers, the 1_3s rule was acceptable for 24 of 29 analytes on the AU640 machine and 21 of 29 analytes on the AU2700 machine. Sigma metrics for these tests ranged from 5.52 to 33.48 and, as expected, these tests were easily controlled with statistical QC using a single rule. For a small number of analytes (albumin, total protein, and inorganic phosphorus), the performance differed significantly between the 2 analyzers. Based on identification of significant performance differences between the analyzers for measurement of inorganic phosphorus, and subsequent identification of high bias on external QC data for the AU 2700 analyzer, a factor change was introduced. The lamp was also replaced in the AU2700 machine because it had come to the end of its recommended use period. Weekly lamp checks were performed as part of the routine maintenance, so it is unlikely that the aging lamp had any significant impact on the results. Despite excellent technical follow-up, there was a persistent difference in performance between the AU640 and the AU2700 for inorganic phosphorus. No technical reasons for the difference in performance of the 2 analyzers could be identified, even though the same reagents and methods were used for both machines.

Table 4.

Summary of the control rule and number of data points required to achieve an improved probability of error detection (P_ed) and probability of false rejection (P_fr) for selected analytes on the Olympus AU640 biochemistry analyzer.

Analyte	P_ed with 1_3s rule	P_fr with 1_3s rule	Rule*	P_ed with new rule	P_fr with new rule
Low chloride	0.80	0.00	1_3s (4)	0.96	0.01
			1_3s/2_2s/R_4s (2)	0.90	0.01
High bile acids	0.86	0.00	1_3s/2_2s/R_4s (2)	0.94	0.01
Low glucose	0.41	0.00	1_3s/2_2s/R_4s (2)	0.52	0.01
			1_3s/2_2s/R_4s (4)	0.76	0.03
High glucose	0.86	0.00	1_3s/2_2s/R_4s (2)	0.94	0.01
Low total protein	0.79	0.00	1_3s/2_2s/R_4s (2)	0.90	0.01

The number of control data points need to achieve improved P_ed and/or P_fr are in parentheses. The 1_3s rule: when a single result is more than 3 SDs above or below the mean (expected value for the control), the 1_3s rule has been violated. The 2_2s rule is violated when 2 consecutive controls are 2 SDs above or below the mean. The R_4x rule violation indicates that consecutive results have deviated more than 4 SDs from one another (i.e., one result is 2 SDs below the mean and the next is 2 SDs above the mean).

Table 5.

Analyte	P_ed with 1_3s rule	P_fr with 1_3s rule	Rule	P_ed with new rule	P_fr with new rule
Low albumin	0.50	0.00	1_3s/2_2s/R_4s (2)	0.61	0.01
			1_3s/2_2s/R_4s (4)	0.83	0.03
Low chloride	0.84	0.00	1_3s/2_2s/R_4s (2)	0.93	0.01
High bile acids	0.76	0.00	1_3s/2_2s/R_4s (2)	0.87	0.01
			1_3s/2_2s/R_4s (4)	0.98	0.03
Low glucose	0.41	0.00	1_3s/2_2s/R_4s (4)	0.41	0.03
			1_3s/2_2s/R_4s/4_1s/8_x (2)	0.73	0.02
Low inorganic	0.50	0.00	1_3s/2_2s/R_4s (4)	0.83	0.03
phosphorus			1_3s/2_2s/R_4s/4_1s/8_x (2)	0.97	0.02
Low sodium	0.69	0.00	1_3s/2_2s/R_4s (2)	0.81	0.01
			1_3s/2_2s/R_4s (4)	0.94	0.03
Low total protein	0.26	0.00	1_3s/2_2s/R_4s/4_1s/8_x (2)	0.73	0.02
High total protein	0.61	0.00	1_3s/2_2s/R_4s (4)	0.900.03

The number of control data points need to achieve improved P_ed and/or P_fr are in parentheses. The 1_3s rule: when a single result is more than 3 SDs above or below the mean (expected value for the control), the 1_3s rule has been violated. The 2_2s rule is violated when 2 consecutive controls are 2 SDs above or below the mean. The R_4x rule violation indicates that consecutive results have deviated over 4 SDs from one another (i.e., one result is 2 SDs below the mean and the next is 2 SDs above the mean). The 4_1s rule is violated when 4 consecutive controls are 1 SD above or below the mean. The 8_x rule is violated when 8 consecutive results are on the same side of the laboratory mean.

When an analyte is identified as being poorly controlled, the following mitigating steps may be taken.

Limit the assay to the machine on which the statistical control is easier: This option is impractical in the authors' laboratory

Repeat low phosphorus results from the AU2700 on the AU640 machine: This option is also not practical in the authors' laboratory based on a need for rapid turnaround times for biochemistry assays and high volume batch analyses.

Consider the limitations of the assay when providing an interpretation for the result: This option is the most practical for the authors' laboratory. This option and the need for vigilance in interpretation would not have been identified had the study of analyzer performance not been undertaken.

For those analytes highlighted in Tables 3 and 4 (desired P_fr and P_ed not achievable), additional nonstatistical QC may be of benefit in increasing confidence in the accuracy of results. Examples of nonstatistical QC include the following:

•

Correlation with clinical signs that may lead to performing a repeat test if not consistent with the results produced.

•

Correlation with other analytes that provide support for the result. For example, where a single electrolyte abnormality is detected, additional data such as other electrolytes, albumin, and calculation of an anion gap may provide further evidence regarding the disease process.

•

Performance of additional diagnostic tests to confirm the result. For example, in an apparently hypercalcemic patient, further tests could include parathyroid hormone (PTH), PTH-related protein, ionized calcium, and diagnostic imaging.

Further characterization of performance using QC materials or reference materials with values closer to levels of clinical significance than the current control materials may also be helpful. Custom-designed control materials can be obtained that are closer to the clinical decision levels of interest are available from some sources. b

The findings in the current study helped to illustrate a need for separate QC validation on every instrument; a need for QC that is specific for the analyte and the clinical decision level; and variation in performance of individual instruments despite uniformity of manufacturer, reagents, preventative maintenance, and age of the instrument. Additional benefits included raising the clinical pathologists' awareness of the actual performance capability of the biochemistry analyzers and level at which statistical QC can detect a change in instrument performance, raising awareness of the difference between the performances of the 2 machines, and increasing confidence in statistical QC.

Acknowledgements

The authors wish to thank Mr. L. Roberts, Mrs. S. Boylan, and Mr. D. Fox of IDEXX Laboratories, Wetherby, UK for their input in this study.

Footnotes

a.

Olympus AU640 and AU2700 clinical chemistry analyzers, Olympus America, Center Valley, PA.

b.

Randox Laboratories Ltd., Crumlin, County Antrim, Northern Ireland.

c.

Technopath, Plassey, Limerick, Ireland.

d.

EZ Rules statistics package version 3, Westgard QC, Madison, WI.

References

Clinical and Laboratory Standards Institute (CLSI), National Committee for Clinical Laboratory Standards: 1999, C24-A2 statistical quality control for quantitative measurements: principles and definitions; Approved guideline, 2nd ed. CLSI, Wayne, PA.

Ettinger

Feldman

: 2000. Textbook of veterinary internal medicine, 5th ed. WB Saunders, St. Louis, MO.

Feldman

Nelson

: 2004. Canine and feline endocrinology and reproduction, 3rd ed. WB Saunders, St. Louis, MO.

Freeman

Gruenwaldt

: 1999, Quality control validation in veterinary laboratories. Vet Clin Pathol 28:150–155.

Hinckley

: 1997, Defining the best quality control systems by design and inspection. Clin Chem 43:873–879.

International Organization for Standardization (ISO)/Inter-national Electrotechnical Commission (IEC) 17025: 2005: 2005-5-12, General requirements for the competence of testing and calibration laboratories, 2nd ed., ed. ISO Committee on conformity assessment. ISO, Geneva, Switzerland.

Jensen

Kjelgaard-Hansen

: 2006, Method comparison in the clinical laboratory. Vet Clin Pathol 35:276–286.

Koch

Oryall

Quam

: 1990, Selection of medically useful quality-control procedures for individual tests done in a multitest analytical system. Clin Chem 36:230–233.

Nevalainen

Berte

Kraft

: 2000, Evaluating laboratory performance on quality indicators with six sigma scale. Arch Pathol Lab Med 124:516–519.

10.

Reed

Bayley

: 1998, Equine internal medicine. WB Saunders, Philadelphia, PA.

11.

Stockham

Scott

: 2002, Fundamentals of veterinary clinical pathology. Iowa State Press, Ames, IA.

12.

Westgard

: 2006, Quality control. How labs can apply six sigma principles to quality control planning. Clin Lab News 32:10–12.

13.

Westgard

: 2000, Basic planning for quality. Training in analytical quality management for healthcare laboratories. Westgard QC, Madison, WI.

14.

Westgard

Barry

Hunt

Groth

: 1981, A multirule Shewhart chart for quality control in clinical chemistry. Clin Chem 27:493–501.

15.

Wiegers

: 2003, Valid methods: The quality assurance of test method development, validation, approval, and transfer for veterinary testing laboratories. J Vet Diagn Invest 15:303–310.

16.

Willard

Tvedten

: 2004, Small animal clinical diagnosis by laboratory methods, 4th ed. WB Saunders, St. Louis, MO.

Quality Control Validation,Application of Sigma Metrics,and Performance Comparison between Two Biochemistry Analyzers in a Commercial Veterinary Laboratory

Abstract

Keywords

Introduction

Materials and methods

Laboratory materials

Quality requirements for analytes

Performance data

Analysis

Results

Discussion

Acknowledgements

Footnotes

a.

b.

c.

d.

References