Dose Accuracy and Reliability of a Connected Insulin Pen System

Abstract

For people with diabetes on insulin injection therapy, insulin pen dose accuracy and reliability are key features. Dose accuracy of the Tempo Pen^TM with attached Tempo Smart Button^TM [Module] (the system) was tested for three doses under standard atmospheric conditions. Reliability and the ability of the Module to detect, store, and transmit dose-related data were also tested. The system met the International Organization for Standardization 11608-1:2014 requirements for dose accuracy at all doses. Mean (standard deviation) doses were 0.010 mL (0.002), 0.299 mL (0.002), and 0.597 mL (0.004) for the 0.010-mL, 0.300-mL, and 0.600-mL doses, respectively. The Module also met requirements for data transfer after every injection. The system accurately delivered doses and reliably captured and stored insulin dosing information.

Introduction

Insulin treatment optimization is essential to assist individuals in reaching target blood glucose levels. ¹ However, people undergoing insulin therapy are faced with challenges regarding dosing decisions, dose documentation, and glucose monitoring.^2-8 A variety of digital health tools and automated insulin delivery systems such as closed-loop systems are increasingly used in diabetes management.^9-12 In addition, technological advancements in the connected health care environment now enable communication between devices. Use of such systems can ensure insulin dose adjustments are based on contemporary and accurate data.

The Tempo Pen^TM (Eli Lilly and Company, Indianapolis, IN) is a disposable, modified KwikPen with an integrated component that works with the Tempo Smart Button (Eli Lilly and Company; CE marked). The Smart Button^TM (Module) is a data transfer module that enables the automatic transfer of insulin dose-related data to a mobile application (Figure 1). The Pen is available with insulin lispro (Humalog; Eli Lilly and Company), insulin glargine (Basaglar/Abasaglar; Eli Lilly and Company), and insulin lispro-aabc (Lyumjev; Eli Lilly and Company) U-100 formulations.

Figure 1.

Personalized diabetes management system. (a) Humalog Pen injector operational/mechanical characteristics. (b) Smart Button. (c) Connected care environment.

We report results from a study evaluating the technical performance, including dose accuracy and reliability, of the Pen and Module system.

Methods

Humalog, Basaglar/Abasaglar, and Lyumjev Pens were previously tested. This report focuses on the combined Pen and Module (the system).

Dose Accuracy

Sterile saline was used as a surrogate for insulin for all testing due to similarities in fluid properties. Sixty Pens and 60 Modules were preconditioned, visually and functionally inspected, subjected to standard atmospheric conditions (23°C ± 5°C at 50% ± 25% humidity) for ≥4 hours, primed, and inserted into the test stand.

Each Pen was dialed to a 0.010-mL, 0.300-mL, or 0.600-mL dose size. A linear actuator ejected the dose into a catch-cup on a precision balance, and mass was converted to volume using test fluid density. This was repeated until all three dose sizes were tested. A new Pen with a new Module attached was then inserted into the test stand and the dosing sequence repeated.

Glide Force

Glide force was evaluated during dose accuracy testing with the 0.600-mL dose. Glide force characteristics were assessed at delivery rates of nine insulin units per second.

System Performance Under Different Atmospheric Conditions

The dialed-dose function and Pen color identification by the Module were further evaluated under standard, cool, and warm conditions. Each test was performed using 119 Modules and 12 Pens. Modules were subjected to the following conditions for ≥4 hours and the dosing procedure was repeated for “at-condition” testing: 23°C ± 5°C and 50% ± 25% relative humidity (standard); 5°C ± 3°C (cool); and 40°C ± 5°C and 50% ± 10% relative humidity (warm).

Modules were then allowed to equilibrate to standard room conditions, and the dosing procedure was repeated for the posttest assessment.

Accelerated Aging

This test was performed to evaluate the Module’s accuracy and reliability at end-of-life (ie, two years for the Module and seven years for the Pen). Altogether, 59 Modules and 59 Humalog, Lyumjev, and Basaglar/Abasaglar Pens each were used. Accelerated aging was performed using an accelerated aging factor according to the American Society for Testing and Materials 2011 guide (ASTM F1980-07). ¹³ The Modules were subjected to 62°C ± 2°C and 20% ± 10% relative humidity for ≥61 days (equivalent to two years at 21°C). The Pens were subjected to real-time aging in addition to accelerated aging at 62°C ± 2°C to represent the end-of-life conditions of the Pen. The dosing procedure as previously described was then followed.

Reliability Testing

The ability of the Module to detect, store, and transmit dose-related data, including the color corresponding to product information, time, and angular rotation of the magnet during each dose, were tested over the life of the Module. The test was performed on 59 Modules in three groups (20, 20, and 19 samples). The Module was attached to a Pen, and priming and dosing events were simulated accounting for four priming and four dosing events per day in an accelerated test that represented a 1-year product life (1460 priming events and 1460 dosing events for a total of 2920 dose records).

Statistical Analyses

Attributes and variables data were collected for analyses. Attributes data were summarized to evaluate against the acceptance criteria. For variables data, analyses were performed using SAS JMP version 15+ (SAS Institute, Cary, NC). The distribution function in JMP was used to determine normality and to calculate the mean and standard deviation of the dose delivered and glide force. Normal data (goodness-of-fit P > 0.05 using the Shapiro-Wilk test) were fit with a normal distribution curve and a capability analysis performed to generate a Process Performance Index (Ppk). Nonnormal data (P < 0.05) were fit with a smooth curve, a capability analysis was performed, and a new Ppk was generated from the capability analysis. To determine the calculated k-value, the Ppk was multiplied by 3. This was then compared with the target k-value to determine if the results met the acceptance criteria. The Supplementary Appendix provides more detailed methods.

Results

Dose Accuracy and Glide Force

All Pen and Module systems met the acceptance criteria for glide force and dose accuracy testing, with all mean doses within International Organization for Standardization (ISO) specified ranges (Table 1; Figure 2; Table S1).

Table 1.

Dose Accuracy and Glide Force.

Specification requirement	Number of samples	Mean (mL/lbf)	Std Dev (mL/lbf)	LSL (mL)	USL (mL/lbf)	Observed k-value	Target k-value	Test result
0.010 mL ± 0.010 mL	60	0.0100	0.0016	0.000	0.020	3.354	2.670	Pass
0.300 mL ± 5%	60	0.2987	0.0022	0.285	0.315	6.276	2.670	Pass
0.600 mL ± 5%	60	0.5965	0.0044	0.570	0.630	5.967	2.670	Pass
Glide force sample avg ≤5.5 lbs. Individual glide force ≤8.0 lbs. (0.600-mL dose only)	60	3.8832	0.2849	N/A	8.0	14.454	2.022	Pass

LSL and USL indicate the International Organization for Standardization specification limits. The test result was “Pass” if the observed k-value was greater than or equal to the target k-value.

Abbreviations: avg, average; LSL, lower specification limit; N/A, not applicable; Std Dev, standard deviation; USL, upper specification limit.

Figure 2.

Distribution curves of dose accuracy analysis for (a) 0.010 mL, (b) 0.300 mL, and (c) 0.600 mL. The blue lines indicate the International Organization for Standardization specification limits.

System Performance Under Different Atmospheric Conditions

The Module met specifications after conditioning according to the ISO 11608-1 standard, cool, and warm criteria (Table 2). There were four observations under “cool” and five observations under “warm” conditions. In each instance of reporting an error after the Pen was replaced, the test was repeated, and the Module correctly recorded dose-related data.

Table 2.

System Performance Under Different Atmospheric Conditions and Accelerated Aging.

Test description	Number of observations	Allowable observations	Number of defects	Allowable defects	Test result
Standard (238 doses)	0	Accept on 7, reject on 8	0	Accept on 0, reject on 1	Pass
Cool (357 doses)	4	Accept on 7, reject on 8	0	Accept on 0, reject on 1	Pass
Warm (357 doses)	5	Accept on 7, reject on 8	0	Accept on 0, reject on 1	Pass
Accelerated aging (177 doses)	1	Accept on 2, reject on 3	0	Accept on 0, reject on 1	Pass

If one or more dose records on an individual Module indicated an error, the Module was identified as an “observation” on initial execution. The affected Pen was changed, and the functional assessment was repeated with a replacement Pen. If the observation (one or more doses with an error) occurred during the repeated step, the Module would be identified as a “defect” on the re-execution. For the standard, cool, and warm condition tests, the test result was “Pass” if there were no more than seven “observations” on initial execution of the test (accept on seven, reject on eight) and no more than zero “defects” on the final visual/functional assessment (accept on zero, reject on one). For the accelerated aging test, the test result was “Pass” if there were no more than two “observations” on initial execution of the test (accept on two, reject on three) and no more than zero “defects” on the final visual/functional assessment.

Accelerated Aging

The Module met specifications after accelerated aging conditioning (Table 2). One observation was noted for a dosing error using a Basaglar Pen during the initial functional test. The Pen was replaced with an alternate aged device and the test repeated per protocol; the Module correctly reported the dose-related data. No defects were identified.

Reliability Testing

Modules passed the reliability test and were functional during their final cycle, meeting requirements for Module product life (Table 3). No incorrect angular rotation readings, incorrect Pen button color identifications, incorrect time stamps, or missing doses were observed (Table 3). The Module-observed time shift was 0.0 to 5.0 seconds for a maximum observed clock error of 5.0 seconds out of 927 535 seconds test time or 0.47 seconds per day; the 700-ppm allowable clock drift was 649.3 seconds for the test duration. No clock drift defects were observed.

Table 3.

Summary of Results From Module Product Life and Module Reliability Tests.

Samples	Test	Number of defects a	Allowable defects	Test result b
59 (172 280 doses)	Product life	0	Accept on 0, reject on 1	Pass
59 (172 280 doses)	Module reliability	0	Accept on 0, reject on 1	Pass

a

For Module reliability, defects assessed as the number of Modules with greater than 55 anomalies or a time shift defect. An anomaly was defined as an incorrect dose reading (incorrect measurement of angular rotation), incorrect button color identification, incorrect time stamp (time of dose), or missing dose record.

b

For product life, the test result was “Pass” if there were zero nonfunctional devices at the end of the test. For Module reliability, the test result was “Pass” if zero incorrect angular rotation readings, incorrect Pen button color identifications, incorrect time stamps, or missing doses were observed.

Modules met the acceptance criteria for the maximum angular measurement accuracy (Table 4; Figure 3a). Data were normally distributed and indicated with 95% confidence and 96.5% probability content that the angular measurement from the Module would enable reliable calculation of the number of units dosed.

Table 4.

Summary of Results From Angular Measurement Accuracy Test.

Samples	Test	Mean (degrees)	Std Dev (degrees)	Upper specification limit (degrees)	Targetk-value	Observed calculated k-value	Test result
59 (172 280 doses)	Maximum angle deviation from nominal (all doses)	4.00	1.52	9	2.218	3.297	Pass
59	Maximum angle deviation from nominal (first dose excluded)	2.53	0.63	9	2.218	10.335	Pass

Modules met the acceptance criteria for the maximum angular measurement accuracy if the data were normally distributed and indicated with 95% confidence and 96.5% probability content that the angular measurement from the Module would enable reliable calculation of the number of units dosed.

Abbreviation: Std Dev, standard deviation.

Figure 3.

Histogram showing the data for maximum angle deviation from nominal (a) including all doses and (b) excluding the first dose. The blue line indicates the International Organization for Standardization specification limit.

Further review revealed that a test machine bias was present in the first prime dose operation on each test sample. Data were re-analyzed after removing the first prime dose, and results indicated a k-value of 10.335, demonstrating high capability to the specification (Table 4; Figure 3b).

Human Factors Validation

A human factors validation study with 30 participants was conducted (data not shown). All participants completed multiple tasks including dialing and administering a dose without dosing errors, with all doses accurately shown in the mobile application.

Discussion

This is the first study describing the technical performance of our Pen and Module system. The Module met all acceptance criteria for dose accuracy and reliability in detecting, storing, and transferring insulin dose-related data. This indicates accurate dosing across the dose range under different atmospheric conditions intended to simulate real-world use. Testing confirmed that the Module does not impact the Pen’s compliance to specifications for injection force and dose accuracy, nor does it impact the Pen’s compliance with ISO 11608-1. Furthermore, the Module met requirements for data transfer after every injection and completed data transfer to a paired mobile application upon user demand.

The Pen is currently the only disposable pen designed with a built-in magnet to work with the Module as a system to provide accurate and reliable capture and transmission of dose-related data. In this study, dose accuracy and reliability testing were conducted using a test mobile application. However, the system can work with other mobile applications such as TempoSmart^TM (part of the system platform in the United States) and independent existing partner mobile applications outside the United States.

Limitations include simulated use in human factors rather than clinical use studies. However, the methods employed conform to consensus standards, regulatory guidance, and industry best practices. To complement these data, future prospective studies could be employed to evaluate the impact of the system on quality of life and clinical outcomes. In addition, summative human factor validation studies could enhance our understanding of the device’s usability.

Conclusions

The Smart button, when connected to the Pen, provided accurate and reliable insulin dose-related data during engineering testing. Findings from this study could stimulate more research and reporting on this and other important diabetes technology that help people with diabetes and their health care providers optimize insulin therapy.

Supplemental Material

sj-docx-1-dst-10.1177_19322968241301760 – Supplemental material for Dose Accuracy and Reliability of a Connected Insulin Pen System

Supplemental material, sj-docx-1-dst-10.1177_19322968241301760 for Dose Accuracy and Reliability of a Connected Insulin Pen System by Matthew Backfish, Kimberly Kovalchick, Robert Albert, Myriam Rosilio, Farai Chigutsa and Birong Liao in Journal of Diabetes Science and Technology