Demonstration of the Use of Remote Temperature Monitoring Devices in Vaccine Refrigerators in Haiti

Abstract

After the 2010 earthquake, Haiti committed to introducing 4 new antigens into its routine immunization schedule, which required improving its cold chain (ie, temperature-controlled supply chain) and increasing vaccine storage capacity by installing new refrigerators. We tested the feasibility of using remote temperature monitoring devices (RTMDs) in Haiti in a sample of vaccine refrigerators fueled by solar panels, propane gas, or electricity. We analyzed data from 16 RTMDs monitoring 24 refrigerators in 15 sites from March through August 2014. Although 5 of the 16 RTMDs exhibited intermittent data gaps, we identified typical temperature patterns consistent with refrigerator door opening and closing, propane depletion, thermostat insufficiency, and overstocking. Actual start-up, annual maintenance, and annual electricity costs for using RTMDs were $686, $179, and $9 per refrigerator, respectively. In Haiti, RTMD use was feasible. RTMDs could be prioritized for use with existing refrigerators with high volumes of vaccines and new refrigerators to certify their functionality before use. Vaccine vial monitors could provide additional useful information about cumulative heat exposure and possible vaccine denaturation.

Worldwide vaccination coverage among infants has increased substantially since the World Health Organization (WHO) established the Expanded Program on Immunization (EPI) in 1974. Globally, vaccination coverage with the third of 3 required doses of DTP3 vaccine (diphtheria, tetanus, pertussis) reached 86% in 2015. ¹ To ensure that potent vaccines reach their target population, the 2011-2020 Global Vaccine Action Plan recommended the development of innovative vaccine supply chain technologies. ² These technologies will become increasingly important as the number of new vaccines and the storage requirements for these vaccines continue to expand.

All vaccines require a system of storage and transportation to ensure that they remain within recommended temperature ranges and thus potent. This system is achieved through a cold chain, a temperature-controlled series of refrigerated production, storage, and distribution activities, with associated equipment and logistics. At the country level, the cold chain begins when vaccine is received from the manufacturer at the airport; it continues through storage at central, intermediate, and peripheral locations; and it ends where the vaccine is administered.

The recommended storage temperature to ensure potency of most vaccines at peripheral locations is between 2°C and 8°C, although some vaccines are kept at temperatures as low as –25° at central and intermediate locations. Periods of time outside the recommended temperature range are called excursions. ³ Vaccine cumulative heat exposure (up to 37°C) can be monitored on vaccine vial monitors (vial labels that give visual indications of whether vaccines have been kept at temperatures that preserve their potency). However, to more accurately monitor excursions in cold-chain equipment, WHO recommends the use of continuous electronic temperature monitoring devices, backed up with stem thermometers. ⁴ These devices include an electronic temperature logger, such as Fridge-tag (Berlinger), that measures and records data on temperatures digitally for 30 days. Logging the temperature allows users to read the daily maximum and minimum logged temperatures and the duration of any excursions. ⁵ More recently, the expansion of cellular networks and internet capacity globally has provided an opportunity for remote monitoring of temperatures in cold-chain equipment through the use of programmable remote temperature monitoring devices (RTMDs). These devices have sensors that read the temperature at a location in or around cold-chain equipment; they also contain a logger that records and communicates data wirelessly. These devices allow temperature alarm thresholds to be set so that in the event of temperature excursions, the system can notify recipients via short message service (ie, text message). ⁶

Occupying the western third of the island of Hispaniola, Haiti is home to >10 million people. ⁷ Haiti exemplifies the challenges associated with expanding the cold chain to enable the introduction of new vaccines. As the poorest country in the Western Hemisphere, Haiti depends on international economic assistance and therefore has limited resources to maintain, monitor, and upgrade its cold-chain equipment. After the devastating 2010 earthquake, global partners collaborated with the Haitian EPI (Direction du Programme Élargi de Vaccination in French) on the goal of adding 4 new antigens (Haemophilus influenzae type b, hepatitis B, rotavirus, and pneumococcus) to the routine immunization schedule in Haiti by 2015. ⁸ Doing so would require a 7.5-fold increase in vaccine storage capacity per fully immunized child and reliable cold-chain equipment monitoring, because all of the vaccines scheduled for introduction in Haiti (pentavalent [diphtheria, tetanus, whooping cough, hepatitis B, and Haemophilus influenzae type b], rotavirus, and pneumococcal conjugate vaccines) can be damaged by freezing. ⁴ However, a cold-chain assessment of Haiti in 2011 found insufficient vaccine storage capacity at national and intermediate locations, chronic shortages of the propane needed to fuel most refrigerators at intermediate and peripheral locations, and aging equipment. ⁹ Furthermore, monitoring internal refrigerator temperatures has been done only with stem thermometers, which often do not detect vaccine exposures to extreme heat or cold because they cannot provide continuous data on temperatures or generate alarms in the event of excursions. ¹⁰ In addition, vaccines used in Haiti have historically lacked vaccine vial monitors to help identify heat exposure because, while WHO and the United Nations Children’s Fund (from which many countries procure vaccines) promote the use of vaccine vial monitors, ¹¹ the Pan American Health Organization through its regional procurement mechanism neither requires nor provides vaccine vials with these monitors. ¹²

To address the need for more storage capacity for new vaccines, the Haiti EPI added cold rooms at its central storage facilities. In hard-to-reach peripheral locations, 153 solar refrigerators were installed in 2011 and 2012 and equipped with Fridge-tags so that data on temperatures could be collected and the functionality of the refrigerators could be validated. However, because temperatures were not reported, validation of the refrigerators for vaccine storage was delayed by >2 years. We hypothesized that the use of RTMDs to provide continuous data on refrigerator temperatures at remote locations could facilitate the evaluation and validation of new cold-chain equipment and identify when repairs are needed.

The Haitian EPI, the Centers for Disease Control and Prevention, the United Nations Children’s Fund, and ScaleStation, a US information technology company, endeavored to demonstrate the feasibility of using RTMDs to monitor refrigerators at intermediate and peripheral locations of the vaccine cold chain in Haiti.

Methods

Test Phase

In October 2013, ScaleStation installed RTMDs in 1 solar refrigerator and 2 propane refrigerators, which were located in 3 health facilities (1 urban, 2 rural) in Haiti. Installing the refrigerators allowed us to test data collection and transmission and observe correlations between the temperatures recorded by the device and the actual conditions at the sites (ie, propane supply, open-closed door status, interior packing of the refrigerator).

Demonstration Phase

From October 2013 through April 2014, Haitian national and department cold-chain staff members helped install 13 additional RTMDs, creating a total of 16 devices, all located in 2 of Haiti’s 10 administrative divisions (ie, departments), Ouest and Artibonite. These RTMDs were placed at 15 sites (1 site had 2 RTMDs), consisting of 10 peripheral locations (5 urban and 5 rural health facilities) and 5 intermediate storage locations, with the purpose of monitoring 24 refrigerators (8 propane, 15 solar, 1 electric). The sample size of 16 RTMDs was limited by the funds available for their purchase and installation. We selected demonstration sites based on (1) convenient access to them from Port-au-Prince and (2) whether they contributed to a representative mix of subnational settings (ie, department-level facility, rural peripheral facility, and urban peripheral facility).

The RTMD we used was the FridgeFone (Beyond Wireless Technology Ltd). The RTMD received input from 4 sensors: 2 sensors (1 on top shelf and 1 on bottom shelf) recorded refrigerator interior temperatures; 1 sensor (attached to door jamb) monitored the open-closed door status; and 1 sensor (on wall outside refrigerator) monitored the ambient room temperature. The FridgeFone sampled data every 10 minutes and uploaded them automatically via mobile networks to an internet-based server; we were able to access live and historical data through a graphical user interface. Although each FridgeFone could receive input from up to 16 sensors, making it capable of monitoring 4 adjacent refrigerators at a single site, the sites involved in this demonstration had only 1 or 2 refrigerators.

Each FridgeFone used 90 to 240 V of alternating current and 0.1 kilowatt hour per day of energy (a 100-W incandescent light bulb uses 2.4 kilowatt hours per day). A rechargeable lead-acid battery (that would provide up to 240 hours of power) served as a backup power source in case of outages, and it required 3 to 5 hours of cumulative power to recharge. We expected this power requirement to be feasible because, whereas many vaccination sites lacked enough power to constantly run electric refrigerators, most had enough electricity to run lights. As such, we expected that they would also have enough power to run the RTMD and/or recharge the battery.

We established contracts with the 2 local cellular data companies that provided service throughout Haiti. We made agreements for prepaid monthly data (without telephone) service to maintain a minimum data threshold and prevent interruption of data transmission. To prevent an excessive number of alarms, we set alarm thresholds at a level that was slightly less stringent than that recommended by the WHO Performance, Quality and Safety guidelines. We set the upper alarm threshold at >10°C for >10 hours continuously and the lower alarm threshold for <0.5°C for >2 hours continuously. ^6,13 When thresholds were breached, the device sent a short message service message about the excursion to local immunization and departmental cold-chain staff members in Haiti. However, we did not monitor or evaluate the local responses to excursions.

Analysis of Temperature Patterns

We collected data produced by the 16 RTMDs at 15 sites from March 1, 2014, through August 16, 2014. We analyzed the data pertaining to temperature excursions (<2°C, >8°C, site, time of day, day of week, duration, and frequency), ambient room temperatures, and open-closed door status. Based on correlations that we observed during the test phase between reported temperatures and actual conditions, we inferred the causes of temperature excursions that occurred during the demonstration phase. We emailed weekly reports on temperature data and analysis to Haitian national and departmental cold-chain staff members.

System Costs

We documented costs of the demonstration, including start-up costs (ie, all hardware and shipping; excluding installation), and we estimated annual maintenance costs (ie, data management, cellular connectivity) and electricity costs. We calculated start-up costs, annual maintenance costs, and annual electricity costs per refrigerator used in the demonstration, and we calculated hypothetical costs per refrigerator if each RTMD monitored either 4 refrigerators (4 sensors each) or 8 refrigerators (2 sensors each). We calculated electricity costs based on the RTMD manufacturer’s estimate of 0.1 kilowatt hour used per RTMD unit per day, using the average price of 1 kilowatt hour of electricity at the public government rate ($0.37). ¹⁴ Because we were unable to determine how often the batteries were needed and how often they required recharging, we did not include an estimate of the electricity cost for the lead acid batteries in our estimates. In this setting where power outages occur frequently, the batteries likely needed to be replaced every 2 years. However, these batteries required minimal electricity to recharge, were inexpensive, and were readily available locally.

Postdemonstration Activities

At the end of the demonstration, we trained Haitian cold-chain staff members on RTMD installation, deinstallation, and maintenance. We customized the graphical user interface to a Haitian setting and trained staff members on data accession and analysis.

Outcomes

Data Transmission and Temperature Patterns

We received consistent data from 14 of 16 RTMDs, pertaining to 21 of the 24 refrigerators. Of the 2 RTMDs that did not transmit consistent data, 1 was located in a rural health dispensary, and the other was in an intermediate storage depot.

Although 5 devices had intermittent gaps in data, we identified several interior refrigerator patterns. During the test phase, temperature patterns were consistent with depletion of propane, insufficiency of the thermostat, door opening and closing, or overstocking the interior of the refrigerator, which resulted in inadequate air circulation (Figure). During the demonstration phase, after propane depletion, interior refrigerator temperatures typically increased to the level of ambient room temperatures during a 12-hour period. Door monitors were prone to fall off or become disconnected, resulting in gaps in data. Door openings and closings were associated with a sharp rise and fall of interior temperatures.

Figure.

Temperatures recorded by 2 interior sensors (one on the top shelf and one on the bottom shelf) in the interior of a propane-fueled vaccine storage refrigerator at a facility in the Ouest (West) administrative department, Haiti, August 4, 2014, at 4 pm, through August 9, 2014, at 4 pm.

System Costs

Start-up costs—including receivers, sensors, power cables, rechargeable backup batteries (1 per RTMD), installation hardware, and shipping—were $686 per refrigerator in the demonstration phase (Table); these costs would be $257 and $129 per refrigerator if each RTMD were to monitor 4 or 8 refrigerators, respectively. Recurrent costs of contracts for data transmission and cellular telephone connectivity for 1 year (of a 3-year contract) were $179 per refrigerator in the demonstration phase; these costs would be $67 and $34 per refrigerator if each RTMD were to monitor 4 or 8 refrigerators, respectively.

Table.

Actual start-up and estimated annual maintenance and electricity costs (in US dollars) for RTMDs used for vaccine refrigerators in Haiti, 2014

	Cost per Monitored Refrigerator, $
Category	Actuala	Estimated (1 RTMD per 4 Refrigerators)b	Estimated (1 RTMD per 8 Refrigerators)c
Start-upd	686	257	129
Annual maintenancee	179	67	34
Annual electricityf	9	4	1
Total	874	328	164

Abbreviation: RTMD, remote temperature monitoring device.

^aBased on 16 RTMDs monitoring 24 refrigerators during the demonstration.

^bBased on 1 RTMD monitoring 4 refrigerators (ie, 4 sensors per refrigerator).

^cBased on 1 RTMD monitoring 8 refrigerators (ie, 2 sensors per refrigerator).

^dStart-up costs included RTMDs (each requiring 1 receiver @ $733; 4 sensors @ $33; power cables, battery, miscellaneous hardware @ $37; and shipping @ $127). Start-up costs excluded installation, customs clearance, and any import duties or taxes.

^eAnnual maintenance costs included annual contracts for data management and cellular telephone connectivity ($235 and $33, respectively, per RTMD).

^fAnnual electricity costs based on RTMD manufacturer’s estimate of 0.1 kilowatt hour used per RTMD per day and with an average price of 1 kilowatt hour of electricity at the public government rate ($0.37), which resulted in an annual electricity cost of $13.50 to run each RTMD. ¹⁴

Annual electricity costs to run the RTMD were estimated to be $9 per refrigerator used in the demonstration phase; these costs would be $4 and $1 per refrigerator if each RTMD were to monitor 4 or 8 refrigerators, respectively (Table). The annual electricity cost was estimated to be $13.50 per RTMD.

Lessons Learned and Recommendations

It was feasible to obtain data from RTMDs in rural and urban locations in Haiti; only a small proportion of the devices were limited by insufficient electrical power or unreliable cellular telephone coverage. Interior refrigerator readings provided valuable data on low propane supply, thermostat malfunction, and improper stocking of the refrigerator, whereas ambient room temperature and open-closed door readings did not provide helpful information. We based our observations on the monitoring of 24 refrigerators in 2 departments; thus, the results cannot be generalized to the nearly 700 vaccine refrigerators in all 10 of Haiti’s departments. ¹⁵

Given the human resource and financial constraints in Haiti, the cost of RTMDs would be prohibitive if used in every vaccine refrigerator. However, a small number of RTMDs could be effectively used in 3 contexts: (1) routine monitoring of existing equipment at storage sites with high vaccine volume, (2) monitoring of new refrigerators for a defined period immediately after installation to certify proper functioning before being used to store vaccines, and (3) spot checking of existing refrigerators, with priority given to those with a history of excursions. Vaccine vial monitors could then be used as accessories to identify instances of cumulative heat exposure up to 37°C and possible vaccine denaturation.

In many immunization settings, continuous electronic temperature monitoring devices have been shown to be more useful than stem thermometers, in part because they provide data that can detect cold-chain equipment needing service or replacement. ^16,17 The data provided by these devices have also been used to document staff member training and financial resource needs (eg, cold-chain equipment fuel, repairs, or replacements) and to advocate for those needs. ¹⁸ Nevertheless, many immunization programs worldwide use only stem thermometers. ^10,19 In addition, even with remote access to data on continuous temperatures, the lack of response to documented excursions has been common, not only in low-resource settings where shortages of propane, transportation, and spare parts are commonplace, but also in high-resource areas. ^10,19,20 These findings highlight the need to develop local infrastructures to coordinate resources and ensure accountability. ^17,18

Finally, key factors in the effective use of RTMDs include training staff members on device maintenance and data interpretation and developing standard procedures for responding to alarms. Without the underpinning of appropriately trained and resourced cold-chain staff members, RTMD use alone will not be sufficient to ensure that vaccines are stored under the correct conditions to preserve potency.