Abstract
There are few tried and tested mobile technology applications to enhance and standardize the quality of health care by frontline rural health providers in low-resource settings. We developed a media-rich, mobile phone–based clinical guidance system for management of fevers, diarrhoeas and respiratory problems by rural health providers. Using a randomized control design, we field tested this application with 16 rural health providers and 128 patients at two rural/tribal sites in Tamil Nadu, Southern India. Protocol compliance for both groups, phone usability, acceptability and patient feedback for the experimental group were evaluated. Linear mixed-model analyses showed statistically significant improvements in protocol compliance in the experimental group. Usability and acceptability among patients and rural health providers were very high. Our results indicate that mobile phone–based, media-rich procedural guidance applications have significant potential for achieving consistently standardized quality of care by diverse frontline rural health providers, with patient acceptance.
Keywords
Introduction
In developing countries, several types of primary health workers with varying levels of literacy and training provide health care at first contact to the vast majority of populations, 1 but they may lack essential health information on how to diagnose and manage common diseases according to existing protocols and guidelines.2,3 Health-care providers’ lack of adherence to standardized evidence-based clinical guidelines is widely regarded as a barrier to improve health outcomes. 4 The World Health Organization’s 5 (WHO) Integrated Management of Childhood Illnesses (IMCI) is one example of standardized guidelines available for primary health-care workers. However, even with standard guidelines in place, primary health-care workers may lack the training and motivation necessary to successfully apply these guidelines. 6
The need for innovation is crucial, especially in lower and middle-income countries, in order to reduce disparities in health-care access, quality and resource distribution. 7 Mobile health (mHealth) has already shown the potential to overcome many of these obstacles; 8 it is now a widely accepted approach to transform health-care delivery in resource-poor countries.9,10 Smartphones are one of the most powerful tools that can be used to enhance access to quality health care in rural settings. 11 However, most smartphones depend on Internet connectivity to be ‘smart’, and a system that is independent of Internet connectivity would be preferable for rural settings. 12 A clinical practice guideline when presented in a structured algorithmic fashion improves standardization of care. 13 Hence, it appears that smartphones in conjunction with clinical guidelines like the IMCI presented as media-rich algorithms can be potent tools for frontline primary health-care workers. Here, ‘media-rich’ refers to presenting information with audio (speech), images and videos in order to overcome literacy and training deficits among both health workers and the patients they treat. For brevity, we shall refer to such instantiations of clinical guidelines as mobile media-rich interactive guidelines (mMRIGs) in the remainder of this article.
One such mHealth project studied the use of clinical practice guidelines presented in an interactive, structured, step-by-step guidance form, accompanied by instructions in audio, text and images/video on mobile phones to community health workers (CHWs) in a laboratory setting in Colombia. 14 Use of mMRIGs improved the performance of the CHWs by decreasing errors in the performance of diagnostic/treatment medical tasks, and increasing protocol adherence, leading to high acceptance and intention to use. This system was shown to enable incorporation of principles of persuasive technology.15–17 Use of systems incorporating persuasive technologies, especially in the context of rural health, can greatly enhance health workers’ self-efficacy and the confidence needed to provide high-quality client counselling on essential health-related knowledge, attitudes and practices.18,19
This study was set up to field test such a system in a real-time field setting in two rural locations in Southern India. The study objectives were to (a) use a randomized control design to measure changes in protocol compliance (PC) by health workers in their everyday work settings, (b) assess the usability and acceptability of the mobile application with health workers in the experimental group and (c) obtain patient feedback on health workers’ use of the mobile system during treatment.
Methods
Study settings and samples
A total of 16 willing and consenting rural health providers (RHPs) were drawn into the study from two field sites in two neighbouring districts, Sivagangai and Dharmapuri, in rural Tamil Nadu, a South Indian state. At one of these sites – Thirupathur in district Sivagangai – the RHPs consisted of eight independent informal health-care providers based in villages (all male). These providers had been part of a previous technology-enabled distance learning project developed by one of the co-authors. The other group of RHPs consisted of eight female community health auxiliaries linked with a non-governmental hospital (Tribal Health Initiative, or THI, hospital) in a tribal area called Sittilingi in the district of Dharmapuri. While the informal providers in Thirupathur had been practising for more than 10 years on average, THI’s community health auxiliaries had been recruited less than a year prior to this study and all were undergoing training by the hospital. These two different groups of RHPs – one, a group of independent practitioners, and another, a group that was directly linked to a formal hospital set up – were selected because we assumed that they would provide us with valuable additional lessons on the contribution of local contexts in the way our application was finally used and its sustainability and success.
In each location, four RHPs were randomly assigned to the experimental group and four to the control group; thus, there were eight RHPs in each group. All 16 RHPs underwent a 2-day training programme in guideline-based care and in use of the mHealth system by physicians who spoke the local language. We assessed all RHPs on their understanding of guideline-based care using a simple assessment instrument developed by the research team, on which every RHP received a pre- and post-training test score (PTTS). This PTTS initially was to be used as a covariate, but was not, because mixed-model and regression analyses showed that the PTTS was not significantly (
Technology and phone
A system called guideVue® was used for developing mMRIGs and deploying them on cell phones. Details of the system can be found elsewhere.14,15 This system enables development and deployment of mMRIGs without programming. Figure 1 shows a sample screen from an mMRIG developed for our study.
Rich-media guidelines (IMCI) in Tamil language on Windows Mobile 6.5 cell phone.
Development of the mMRIGs for field testing
Our team of three physicians and a public health specialist adapted two validated clinical guidelines for implementation as mMRIGs, deployed on Windows Mobile 6.5 cell phones. These were the WHO’s well-known IMCI and Integrated Management of Adult Illnesses (IMAI). With a focus on three conditions – fever, diarrhoea and respiratory problems – the available WHO’s guidelines were customized to the local Indian settings by our team of senior and experienced physicians and then converted into interactive, structured, rich-media clinical algorithms in Tamil language using the guideVue system (see Figure 1).
Field test and data collection
Two months after the first training, all RHPs were evaluated in their field settings using several methods and instruments. All study instruments were administered by four trained study fieldworkers, recruited with the help of a local partner organisation. All fieldworkers had a social work background and prior experience of working on community health–related projects. The field research team was supervised by two field managers, qualified in social work and nursing, respectively, and with prior experience of qualitative field research methods in health services research, including patient–provider observations. The principal investigator (first author (M.G.)), along with the supervisors, first trained the field investigators in a 2-day classroom training, and thereafter, the supervisors provided further supervised practice in the field, before the research observations were formally initiated.
Data relating to PC, acceptability and usability of the phone and patient feedback were collected as described in the following. Usability and acceptability of the mHealth system by the RHPs was assessed by means of a structured questionnaire, with some open-ended questions (only with the experimental group of RHPs). PC was evaluated by direct observations of the RHPs’ performance as they were treating the patients. Trained field investigators observed each RHP managing eight patients comprising an ideal mix of four paediatric and four adult patients (a total of 128 across the 16 RHPs). Patients were recruited as they presented spontaneously at the RHP clinics and consented to participate in the study. Patients’ reactions to their RHP’s use of the mobile system was assessed by means of an exit interview questionnaire.
Analysis
All data collected were entered into Microsoft Excel spreadsheets. Basic data analysis including simple statistics and histograms were developed using Microsoft Excel. The SPSS 19 MIXED procedure was used to perform multilevel linear model and repeated measures analyses.
PC was measured by the following simple formula that measures the percentage of guideline steps that were actually performed, to the number of steps that are specified in the guideline and are relevant for that patient
The study design was reviewed by a local ethical committee comprising two medical doctors from the All India Institute of Medical Sciences and a social scientist in Delhi. Ethical approval was obtained on 16 March 2011 from this committee.
Results
This study included 16 RHPs divided equally between 8 male practitioners (in Thirupathur, Sivagangai) and 8 female health auxiliaries (in Sittilingi, Dharmapuri). The male practitioners were older, with a mean age of 48.75 years. They had been working as independent health-care providers for more than 10 years out of solo clinics in different villages. All had studied up to class 10 or more. Two held diplomas in laboratory technology and one in pharmacology; the rest had learned mainly as doctors’ apprentices. The women auxiliaries attached to the tribal hospital were younger, with a mean age of 24.75 years. Five had studied up to class 10 or more, and three had completed secondary school. Except for two senior RHPs who had been working with the hospital for several years, the remaining six had been recruited about a year ago and had been undergoing training by the hospital staff over the last 9 months prior to the study.
Pre–post training scores
At the end of the 2-day training on use of the mobile phone to understand protocol-based health-care delivery, all RHPs showed an increase in knowledge as measured through the pre–post training questionnaire. However, while there was no difference in the mean pre-training scores of the two groups, the control group scored significantly higher than the experimental group (mean scores: control group, 13.68; experimental group, 9.51;
Pre- and post-training scores obtained by the control and experimental groups.
SD: standard deviation.
Usability of the mobile system
The first three histograms below relate to RHPs’ reactions to use of the system (Figures 2 to 4). More than 85 per cent RHPs said they were comfortable with the system and found it easy to use; 75 per cent said it helped them follow standardized guidelines better and more consistently than before. Table 5 summarizes RHPs’ responses to other usability-related questions. We note that all of the RHPs would like to have more conditions presented in the system and also that without the system, they would not be able to remember and perform all steps. This last result indicates that without this kind of job-aid, RHPs felt they could not be fully protocol compliant. The majority (87.5%) of RHPs also agreed that they would recommend the system to friends and colleagues, continue using it, and that the combination of rich media was indeed useful. Figures 5 and 6 depict the patients’ reactions to the use of the mobile phone–based system by their providers. The majority of patients said it increased their confidence and respect in their RHPs. This is significant for future adoption of mMRIG systems in the rural health context. If, on the contrary, patients had reported decreased respect and confidence, RHPs would not continue use of mHealth tools.
RHPs’ level of comfort using the mHealth system.
RHPs’ ease of use with the mHealth system.
Use of mHealth system improves RHPs’ ability to deliver guideline-based care.
Patients confidence in their RHP enhanced by RHPs’ use of mHealth system.
Patients reported increased respect of their RHP due to RHPs’ use of mHealth System.
Protocol compliance
To study PC, trained field investigators observed all 16 RHPs as they treated 126 adult and paediatric patients. Although the target was 128 patients, finally 126 patients were recruited for reasons beyond control, and it was not always possible to get a perfect mix of four adult and four paediatric patients per RHP. Thus, out of 126 patients seen by all 16 RHPs, there were 57 paediatric patients (45.2%) and 69 (54.8%) adult patients. A paediatric patient was defined as someone whose age was less than or equal to 10 years. With respect to patients’ gender, 45 (35.7%) were male and 81 (64.3%) were female.
Tables 2 and 3 show the further breakdown of patients categorized by age and sex in the experimental and control groups. While patients were more or less equally distributed across the two groups with respect to age, with respect to gender, there were more male patients in the experimental group (60%) than in the control group (40%). Female patients were distributed more equally.
Patients’ gender categories across the experimental and control groups.
Patients’ age categories across the rural health provider groups.
Mean PC was calculated for the experimental and control groups across a variety of stratifications of interest, including patients’ gender and age categories (paediatric vs adult) and study locations
Figures 7 and 8 and Table 4 show that experimental RHPs’ mean PC initially exceeded control RHPs’ by 26.80 percentage points for males (
Estimated protocol compliance means for the significant treatment by patient number interaction for males.
Estimated protocol compliance means for the significant treatment by patient number interaction for females.
Estimates of provider protocol compliance fixed predictor effect.
Estimated protocol compliance means for the significant treatment by sex interaction.
From RHPs’ comments on usability of the system, we also identified a few problematic areas of use. These included difficulties in navigating through the largely English phone menu (before entering into the Tamil mMRIG application), difficulties with some letters of the available Tamil font that was used in the system, the time required for navigating through some of the screens which had heavier visual files and the time required for entering patient data on the first screen. These are problem areas that will need to be refined in the next version of the system.
Discussion
This article describes a randomized control study to evaluate the practical application of an mHealth system by village-based RHPs at two sites in rural Southern India. PC was calculated across a mix of adult and paediatric cases – a total of eight per RHP, for both the experimental and control groups. Even though the control group scored significantly higher immediately after the 2-day training on the mHealth system and content, 2 months later, the experimental groups at both sites exhibited higher PC, significantly so for initial male patients.
The overall emerging picture was that, statistically controlling for all other predictors, experimental group RHPs’ PC exceeded that of control RHPs, significantly so (by 26.8 percentage points) for initial male patients. The experimental group RHPs’ PC exhibited significantly greater consistency and at a higher level across patient order and across sexes than control RHPs’. After training, experimental RHPs started at high PC levels and exhibited consistent improvements. Their initially sizable PC superiority for both male and female patients diminished, as RHPs continued to serve patients. Finally, RHP PC for adult patients substantially exceeded that for children.
The increase in PC across patient order exhibited by control RHPs could be the result of having opportunities to intermingle with and observe the ways of the experimental group, as was the case in Sittilingi. As it was a real-time field setting, both groups were at work in the hospital’s outreach clinic, and it was difficult to isolate one group from the other. In Thirupathur, however, this could not have been the case as the practitioners were all independent clinic-based providers, quite far from each other. Here, the reason may be attributed to an observation bias that led to the control group providers becoming more and more conscious of the observers and attempting to perform better and better perhaps by consulting the paper guidelines more frequently. Florez-Arango et al. 14 have also reported a similar effect of patient order on control providers’ PC. Even though their study was based on human patient simulators, their RHPs did not have a chance to mingle or receive feedback during observations, similar to our Thirupathur setting.
Another similarity with the Florez-Arango study was the ease of use reported by a majority of providers. It took us by surprise to see how well the women from Sittilingi adapted to the phone during the 2-day training and even more so in the field. In fact, during our in-depth analysis, we found that one of the women auxiliaries with low literacy skills and a very high no-response rate in the pre–post training evaluation had scored the highest PC scores in the field test. We also saw that contrary to our earlier apprehensions, providers were able to easily accommodate the ear phones to hear the instructions (usually in one ear) and carry on with their treatment procedures. They also found (Table 5) the audio instructions to be one of the most useful features of the phone. Another study similarly found that use of audio instructions significantly improved effectiveness of counselling sessions. 20
Health workers’ responses to usability questions.
There were a few practical difficulties such as navigating through the English phone menu and the time delays caused by the heavier visuals on some screens. We considered that this may come in the way of regular use of the system in providers’ daily work flows, so we sought providers’ views on removing the visuals, but they were unanimously against this step. They said that they frequently used the system as a memory aid also, as well as a training tool, and would like to retain the pictures and videos. One option may be then for us to develop two sets of applications in future – one with just audio (and some line drawings perhaps) and text for use in regular patient encounters, and another visually embellished for refreshing providers’ knowledge. Pictures and videos are essential for demonstrating certain conditions (e.g. lethargy in a dehydrated child) and procedures (e.g. skin pinch test for dehydration), so it would not be possible to do away with them.
The small sample size of our study was its main limitation. Despite this, valuable lessons were learned that can be applied and generalized to other larger studies.
This study supports the implication that mMRIGs comprise a feasible and effective solution for standardizing and enhancing quality of care delivered by millions of primary health workers with varying levels of training and literacy.21,22 The use of mobile technology has the potential to transform the way health service is delivered across the globe. 7 The practice of medicine is an information-intensive field, and the principles of informatics have been an integral part of the medical practice. 23 Facilitating the storage and delivery of vast amount of clinical guidelines to practising health workers in an organized media-rich format is essential for success of any health program.18,24 The benefits of any new technology or self-help tools should be applied in a bottom-up strategy to reach even those in the lowest strata of society, only then can we make true progress in terms of empowerment and independence.25,26 The mMRIG technology is an important tool in the success of this system because it allows for easy customization of a variety of clinical guidelines, by local interdisciplinary teams even without any programming expertise. Other benefits can include the ability to document patients seen and the care they received in electronic form on the mobile device. These data can subsequently be uploaded to an electronic health record. Such mHealth-enabled capabilities can greatly improve documentation of care as well as reduce data quality errors. 27 Future work will be related to scaling up this application and including more patient-level data in study designs, such as treatment adherence and health outcomes. As Internet access and, in particular, participation in social networks become more widespread even in developing countries, an interesting aspect would be to integrate mMRIG systems with social media at least for the benefit of RHPs. As shown in Miron-Shatz et al., 28 social media have the potential to greatly improve and enhance medical knowledge and practice. In particular, in the context of RHPs, these social media could help them develop mMRIGs for patient education and care collaboratively.
Footnotes
Acknowledgements
The authors would like to thank Drs Regi and Lalitha George of the Tribal Health Initiative, Sittilingi, Tamil Nadu, India, for their support and cooperation.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The second author (M.S.I.) is part owner of a company incorporated in the United States for commercialization of the GV technology.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was carried out with financial support provided by a small grant from the Information Society Innovation Fund (ISIF) Asia allocated in a competitive process to Garhwal Community Development and Welfare Society (GCDWS) in 2010. ISIF Asia operates through a partnership between the International Development Research Centre of Canada, the Swedish International Development Agency – SIDA and the Asia Pacific Network Information Centre (APNIC), with sponsorship from the Internet Society and the Dot Asia Organization.