Abstract
Background:
Point-of-care glucometer (PoCG) devices play a significant role in self-monitoring of the blood sugar level, particularly in the follow-up of high blood sugar therapeutic response. The aim of this study was to evaluate blood glucose test results performed with four randomly selected glucometers on diabetes and control subjects versus standard wet chemistry (hexokinase) methods in Addis Ababa, Ethiopia.
Method:
A prospective cross-sectional study was conducted on randomly selected 200 study participants (100 participants with diabetes and 100 healthy controls). Four randomly selected PoCG devices (CareSens N, DIAVUE Prudential, On Call Extra, i-QARE DS-W) were evaluated against hexokinase method and ISO 15197:2003 and ISO 15197:2013 standards.
Results:
The minimum and maximum blood sugar values were recorded by CareSens N (21 mg/dl) and hexokinase method (498.8 mg/dl), respectively. The mean sugar values of all PoCG devices except On Call Extra showed significant differences compared with the reference hexokinase method. Meanwhile, all four PoCG devices had strong positive relationship (>80%) with the reference method (hexokinase). On the other hand, none of the four PoCG devices fulfilled the minimum accuracy measurement set by ISO 15197:2003 and ISO 15197:2013 standards. In addition, the linear regression analysis revealed that all four selected PoCG overestimated the glucose concentrations.
Conclusions:
The overall evaluation of the selected four PoCG measurements were poorly correlated with standard reference method. Therefore, before introducing PoCG devices to the market, there should be a standardized evaluation platform for validation. Further similar large-scale studies on other PoCG devices also need to be undertaken.
Since the introduction of the first point-of-care glucometer (PoCG) device by Clark and Lyons in 1960s, 1 thousands of PoCG devices are available in the market nowadays. PoCG testing have an advantage like reduced turnaround time of blood sugar diagnostic testing, reduced preanalytic and postanalytic testing errors, rapid data availability, self-contained and user-friendly instruments, shorter patient length of stay, and use of small sample volume for a given test. 2 The majority of PoCG devices utilize glucose oxidase method, based on advanced electromechanical technology that is specific for β-D glucose measurement specifically catalyzes glucose and reduces interferences. In addition, PoCG devices measure blood glucose level within a range of 20-600 mg/dl, and work at an optimum temperature of 10-40°C. Most PoCG devices store the last 500 results in their memory.3,4
On the other hand, the Cobas 6000 c501 (Roche Diagnostics GmbH, Mannheim, Germany) clinical chemistry analyzer utilizes the hexokinase method in the measurement of glucose concentrations. According to the test principle, hexokinase enzyme catalyzes the phosphorylation of glucose to glucose-6-phosphate by ATP, and rate of phosphorylation is directly proportional with the concentration of glucose in the sample. There is 2-point end method assay and the absorbance of the reaction read at sub and main wavelength ranges of 700 and 340 nm. The concentration of glucose is expressed in mg/dl. The hexokinase method of glucose measurement is considered as a gold standard method, and considered as a reference method for most glucose measurement evolution tests. 5
Despite the wide variety and availability of PoCG devices and their clinical applicability, there is a lot of ambiguity and suspicion regarding their use for clinical decision making. 6 A study conducted in Germany showed that out of 34 BG systems assessed, only 18 (52.9%) fulfilled the minimal accuracy requirements of International Organization for Standardization (ISO) 15197 standards, 7 whereas another evaluation study published in 2010 showed that more than 40% of the systems investigated did not fulfill the minimum accuracy criteria of the ISO standards. 8
In Ethiopia, a study indicated that different PoCG devices were available on the market. 9 Of them, based on informal interviews with DM patients and assessment of commercially available glucometers in Addis Ababa, i-QARE DS-W blood glucose monitoring system (BGMS; Alliance International Co LTD, Taiwan), CareSens N (Gwangun-ro, Nowon-gu, Seoul, Korea), On Call Extra (ACON Laboratories, Inc, San Diego, CA, USA), and DIAVUE Prudential (Biocare Corporation, Taoyuan City, Taiwan) were arbitrarily selected, and accuracy of the machines was evaluated against the standard reference wet chemistry hexokinase methods, as well as evaluated their performance according to ISO 15197:2003 and ISO 15197:2013 standards.
Methods
The study was conducted from March to June 2017 at the Department of Medical Laboratory Sciences (DMLS), and the endocrinology and metabolism unit of Tikur Anbessa Specialized Hospital (TASH), College of Health Sciences, Addis Ababa University. The study protocol was approved by DMLS and the Department of Internal Medicine, TASH ethics committees. Informed consent forms were signed by all participants. Adult patients (≥18 years old) with type 1 and type 2 diabetes as well as control subjects without diabetes were included.
In the study, the mean glucose measurements of four PoCG devices, namely CareSens N, i-QARE DS-W, On Call Extra, and DIAVUE Prudential, were compared with the reference hexokinase method done on Cobas 600 c501 clinical chemistry machine.
Exclusion criteria included pregnancy or lactation period for female subjects and severe acute or chronic diseases that are considered to endanger the subject’s health or compromise the medical care as a result of involvement in the study. Interruption criteria for individual subjects were retraction of written informed consent and incidences or adverse events interfering with the study continuation.
Two hundred study participants were included for each system tested. The tests were performed by clinical personnel trained according to inserted manufacturer manuals of the PoCG devices, safety practices, and the test protocol. The tests were performed in a hospital and laboratory setting with controlled room temperature (23 ± 5°C).
Data Collection
After having received clear information on the aim, risk, and confidentiality of the study, participants have signed the informed consent and participated in the study. Two hundred (100 DM patients, and 100 Non DM controls) participants were enrolled. After an overnight fasting (8-12 hours), blood samples were collected first from capillaries, and sugar content measured by each of the four selected PoCG devices at the spot. Then within 5 minutes, an additional blood sample was collected from ante cubital vein, by using 5 ml serum separator (SS) tube by applying tourniquet for less than one minute, and sites of blood collection cleaned by 70% alcohol. Then the blood sample was centrifuged at 500 g for 5 minutes to obtain the serum/plasma. The serum separated into 2 ml epindroof tubes, and stored temporarily between 2-6°C for a maximum of 90 minutes, and was transported to the Ethiopian Public Health Institute (EPHI), clinical chemistry unit (Ethiopian National Accreditation Agency–accredited laboratory, which is recognized by ISO) for Cobas 6000 c501 clinical chemistry machine, hexokinase method serum glucose measurement. All measurements were done according to the manufacturers’ instructions.
Accuracy and Quality of Reference Hexokinase Method
The reference hexokinase method was done on a Cobas 6000 c501 clinical chemistry machine at the EPHI clinical chemistry laboratory, which is an ISO 15189:2012 accredited laboratory. For the tests on the machine, every day, every 8 hours two level internal quality control tests have been done. Moreover, the EPHI clinical chemistry laboratory participated in the “one world accuracy” external quality program from Canada.
Accuracy Evaluation
Accuracy was evaluated using ISO 15197:2003 and ISO 15197:2013 requirements by calculating the percentage of meter results falling within ±5%, ±10%, ±15%, and ±20% of the reference value for glucose concentrations ≥75 mg/dl and ≥100 mg/dl and within ±5, ±10, ±15, and ±20 mg/dl of the reference value for glucose concentrations <75 mg/dl and <100 mg/dl. The minimum acceptable accuracy for results produced by PoCG according to ISO 15197:2003 is ≥95% of the individual glucose results shall fall within ±15 mg/dl of the results of the manufacturer’s measurement procedure at glucose concentrations <75 mg/dl and within ±20% at glucose concentrations ≥75 mg/dl and according to ISO 15197:2013, is ≥95% of the individual glucose results shall fall within ±15 mg/dl of the results of the manufacturer’s measurement procedure at glucose concentrations <100 mg/dl and within ±15% at glucose concentrations ≥100 mg/dl.10,11 In addition, the Bland-Altman plot was used to estimate the difference (bias) limits containing 95% of data because normally distributed differences were needed. 12
Data Analysis
The data were checked for cleanness and completeness before analysis. Then, the complete data were entered in to spread sheet of Excel and imported for analysis to SPSS version 20 (IBM Statistics, USA). The Bland-Altman analysis was used to assess the agreement of four PoCG with reference spectrophotometric hexokinase method in measuring blood glucose concentration. Correlation coefficient and regression line were used to observe the degree of association of the PoCG with the reference method. A t-test was also used to compare the glucose concentration among varies groups and categories of participants. A P value < .05 was considered to be statistically significant at a 95% confidence interval (CI).
Results
In the present study, from all glucose meters and the reference hexokinase tested the minimum glucose concentration measurements were observed by CareSens N (21 mg/dl), followed by i-QARE DS-W (24 mg/dl) methods. Maximum measurements were observed by hexokinase followed by CareSens N, with values of 498.8 mg/dl and 498.0 mg/dl, respectively.
On the mean sugar value assessment, except the On Call Extra glucometer measurement, all other three glucometers (CareSens N, i-QARE DS-W, and DIAVUE Prudential) had high mean sugar values as compared with the reference hexokinase method (that is >117.74 mg/dl, which was mean value of the reference method), and this mean sugar value differences are statistically significant (P < .005). In terms of linear correlation evaluations, all calculated Pearson correlation coefficients for the selected four PoCG were greater than 0.8 (80%), which indicated all glucose meter methods measurements show a strong positive relationship with the reference hexokinase method (Table 1).
Mean, Minimum, Maximum, Mean Difference, and Correlation Coefficient Values of Four Selected PoCG Device Blood Sugar Measurements Versus Reference Wet Chemistry Hexokinase Method, Addis Ababa, 2017.
On the accuracy assessment of selected PoCG according to ISO 15197 standards, none of the four glucometer sugar values fall in ≥95% of the measurements, within ±20, ±15, ±10, and ±5 mg/dl of the reference results at blood glucose concentrations <75 and <100 mg/dl; and within ±20%, ±15%, ±10%, and ±5% of the reference results at blood glucose concentrations ≥75 and ≥100 mg/dl. This indicated that the selected PoCG did not fulfill the minimum requirements set by ISO 15197:2003 and ISO 15197:2013 (Table 2).
Accuracy Results for Four Selected PoCG Devices Compared to Reference Cobas 6000 c501 Clinical Chemistry Hexokinase Method, Addis Ababa, 2017.
On the Bland-Altman analysis, the reference hexokinase method was compared to selected four PoCG glucose measurements. Accordingly, CareSens N, i-QARE DS-W, and DIAVUE Prudential PoCG results showed higher sugar measurement values than the reference method (Figures 1A-1C). On the contrary, On Call Extra glucose meter results were lower than reference (Figure 1D).
The bias plot (Bland-Altman plot) of glucose measurements results done by selected four PoCG devices versus reference wet chemistry hexokinase method, Addis Ababa, 2017. (A) CareSens N versus Hexokinase; (B) DIAVUE Versus Hexokinase; (C) i-QARE DS-W Versus Hexokinase; (D) On Call Extra Versus Hexokinase.
In the linear regression graph analysis, the slopes of the regression line for reference hexokinase method versus CareSens N, i-QARE DS-W, DIAVUE Prudential, and On Call Extra were with a positive intercept and all values were below 1, except for i-QARE DS-W, which was above 1 and had positive intercepts of 19.848 mg/dl, 32.305 mg/dl, 30.012 mg/dl, and 10.005 mg/dl, respectively (Figure 2).
Linear regression graph analysis. Slope of the regression line for reference hexokinase method versus CareSens N (A), i-QARE DS-W (B), DIAVUE Prudential (C), and On Call Extra (D) glucose meters, Addis Ababa, 2017.
Discussion
In developing countries, where medical laboratory services encounter frequent challenges due to shortage of skilled manpower, electricity, and other hurdles, PoCG devices are a good alternate solution for the problem. However, as a variety of PoC machines are available on the market for given tests, the designed working environment, including temperature and humidity, the sensitivities and specificities of the machines, as well as procedures differ from device to device. Therefore, before introduction of PoC devices for personal use and laboratory services (because PoCGs are not mainly for diagnostic purpose), they should be evaluated by comparing with the reference methods, and standards set by accredited quality standardizing organizations such as the ISO.
In the present study, four randomly selected PoCG devices (CareSens N, i-QARE DS-W, DIAVUE Prudential, and On Call Extra) were assessed against the reference hexokinase method by Cobas 6000 c501 clinical chemistry machine. Accordingly, we demonstrated that sugar measurements done by CareSens N, i-QARE DS-W, and DIAVUE Prudential glucometers were significantly higher than the reference hexokinase blood sugar measurements; meanwhile the mean sugar results done by On Call Extra were relatively lower than that of the standard hexokinase (reference) method, although the differences were not significant. On the other hand, none of four PoCG fulfills the minimum accuracy measurement set by ISO 15197:2003 and ISO 15197:2013 standards. In the linear regression analysis, all four selected PoCG results overestimated the glucose concentrations.
In the present study, the high sugar values recorded by the four evaluated PoCG were further strengthened in the linear regression analysis, where the selected PoCG results have overestimated the glucose concentrations. The present findings were opposite to those of other studies done on other PoCGs in Ethiopia 13 and elsewhere, where PoCG values less than those of the standard clinical chemistry reference method were reported.14,15 This can be justified by the type of PoCG assessed (SensoCard, Accu-Chek, and others, which were not included in this study) and the reference standard clinical method used in the studies (such as glucose oxidase by Humastar 80 machine), which are different from our reference hexokinase method performed on Cobas 6000 c 501 Chemistry analyzer. On the other hand the mean sugar results measured by On Call Extra were lower than those of the standard hexokinase (reference) method. This may suggest that there is a need for a standard platform in each country to evaluate PoCG in a uniform manner before introducing them to the country’s market.
On the other hand, none of four PoCG fulfills the minimum accuracy measurement set by ISO 15197:2003 and ISO 15197:2013 standards. Similar findings were seen on a research conducted in other studies where the tested PoCG did not fulfill the ISO 15197 standards either all or some of them.6,7,13 In this regard, manufacturers of PoCG devices should assess the performance of their devises in a different working environment before releasing their products in to the markets, unless they specify the countries for which the PoCG devices are validated. In the meantime also, each country and individuals who use PoCG devices should maintain the standards by authorized bodies on the quality of products before introducing them for respective customers.
In general the four selected PoCG blood sugar measurements poorly correlated with the standard wet chemistry hexokinase method test results. These inappropriately elevated test results pose huge challenges to patients with diabetes and health care providers’ efforts in achieving the good glycemic control that is fundamental to curb the acute and chronic complications resulting from poor blood glucose control. These incorrect results may create loss of confidence by clients including DM patients, physicians, medical laboratory professionals, and others who use them. Therefore, as PoCG are becoming part of a routine blood glucose measurement and monitoring devises both at the patients’ self-care level and at health care facilities, before introducing them to the market, the products need attention from all stakeholders, including the Ministry of Health, physicians, patients, lab professionals, and quality and standard authorities.
Conclusions
The present study was the first of its kind in using more than one PoCG device at the same time, comparing their performances versus with the standard reference hexokinase method using Cobas 6000 c501 clinical chemistry machine in Ethiopia. The overall findings showed that the four PoCG did not fulfil the minimum accuracy requirements set by ISO 15197:2003 and 15197:2013.
Therefore, as a country we need to have a standard platform to evaluate PoCG devices according to similar standards before intruding them into the Ethiopian market. The regulatory authority also needs to exercise inspections and validations of PoCG devices to ensure that they fulfill the expected standard requirements in the Ethiopian context. Moreover, the Ministry of Health and its offices at different levels in the country as well as concerned health care professionals should raise the awareness of the society on the risks associated with the use of nonvalidated PoCG devices either for personal use or at diagnostic centers. Finally, further similar studies using different study centers, more sample sizes, and different PoCG devices available in the market need to be undertaken.
Footnotes
Acknowledgements
We gratefully acknowledge the study participants and the support provided by AAU, CHS, the Department of Internal Medicine, and the diabetic center staffs.
Abbreviations
BGMS, blood glucose monitoring system; CI, confidence interval; DM, diabetes mellitus; DMLS, Department of Medical Laboratory Sciences; EPHI, Ethiopian Public Health Institute; ISO, International Organization for Standardization; PoCG, point-of-care glucometers; SS, serum separator; TASH, Tikur Anbessa Specialized Hospital.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by Addis Ababa University, Ethiopia through adoptive research fund.