Abstract
eHealth is an opportunity cost, competing for limited available funds with other health priorities such as clinics, vaccinations, medicines and even salaries. As such, it should be appraised for probable impact prior to allocation of funds. This is especially pertinent as recognition grows for the role of eHealth in attaining Universal Health Coverage. Despite optimism about eHealth’s potential role, in Africa there remain insufficient data and skills for adequate economic appraisals to select optimal investments from numerous competing initiatives. The aim of this review is to identify eHealth investment appraisal approaches and tools that have been used in African countries, describe their characteristics and make recommendations regarding African eHealth investment appraisal in the face of limited data and expertise. Methods: Literature on eHealth investment appraisals conducted in African countries and published between January 1, 2010 and June 30, 2020 was reviewed. Selected papers’ investment appraisal characteristics were assessed using the Joanna Briggs Institute checklist for economic evaluations and a newly developed Five-Case Model for Digital Health (FCM-DH) checklist for investment appraisal. 5 papers met inclusion criteria. Their assessments revealed important appraisal gaps. In particular, none of the papers addressed risk exposure, affordability, adjustment for optimism bias, clear delivery milestones, practical plans for implementation, change management or procurement, and only 1 paper described plans for building partnerships. Discussion: Using this insight, an extended 5-Case Model is proposed as the foundation of an African eHealth investment appraisal framework. This, combined with building local eHealth appraisal capabilities, may promote optimal eHealth investment decisions, strengthen implementations and improve the number and quality of related publications.
Economic assessments are challenging in settings constrained by insufficient data and economics expertise, such as in many African countries, where investment appraisal of digital health initiatives is inadequate. A novel approach for evaluating investment appraisals is proposed that utilises the Joanna Briggs Institute checklist for economic evaluations and a Five-Case Model for Digital Health investment appraisal checklist newly developed by the authors. The combination of the JBI checklist for economic evaluations and the Five-Case Model for Digital Health investment appraisal checklist provides a tool for assessing digital health investments and contributes components towards the development of an African eHealth investment framework which, combined with modest building of eHealth appraisal capabilities, may stimulate meaningful improvement in investment decisions and complement specialist health economics expertise, which is in short supply.Highlights
What do we already know about this topic?
How does your research contribute to the field?
What are your research’s implications towards theory, practice or policy?
Introduction
Ehealth is an opportunity cost that will utilise public funds that could otherwise have been used to fund alternate health priorities such as building clinics, rolling out vaccination programmes or hiring more healthcare staff. Since healthcare funds are limited, the case for investing in eHealth should be justified with evidence that the investment will produce sufficient net benefits over time, to satisfy its stakeholders. Consequently, eHealth should not attract public investment until its probable impacts, financial and otherwise, have been appraised. 1 This applies especially in lower- and middle-income countries (LMICs) such as those in sub-Saharan Africa, where annual health expenditure per capita ($83,25 in 2018) is less than 1 10th of the global average ($1111.08 in 2018), 2 and as recognition grows for the role of digital health in attaining Universal Health Coverage (UHC) in Africa.3,4 The World Health Organization (WHO) continues to encourage member states to adopt digital health 5 across a wide range of applications, 6 expanding more than a decade of effort underway through the World Health Assembly 7 and the WHO Regional Office for Africa.8,9 The principle remains relevant whether the term is eHealth or digital health 10 and despite any perceived differences in their meanings. 11
Countries strive to bridge the digital divide, expand connectivity and grow their digital economies, 12 now heightened by COVID-19 experiences,12,13 and highlighted by whether or not people have access to advanced technologies such as smartphones 14 and the internet-of-things. 15 Access to digital tools has begun to function as a health determinant 16 and there is an increased opportunity for eHealth to expand its role in health systems’ strengthening. 17 While economic appraisal can estimate cost-effectiveness and distinguish between more or less favourable innovations, 18 there is a need for investment appraisal to go beyond this to help identify what is needed for a potentially cost-effective solution to provide the most value. 19
Experience during the COVID-19 pandemic has shown rapid uptake of digital health13,17 without clarity on whether there has been appraisal of the impact of these investments. Numerous digital initiatives are underway to help contain the virus, including tools for remote consultation,20,21 contact tracing 22 and patient management. 23 While there is recognition that, ideally, these require investment appraisal, 1 the reality of the pressures of the pandemic needs agile and innovative responses. Nonetheless, responsible research and innovation must select the most appropriate digital health initiatives and requires investment appraisal techniques that can be applied with available but incomplete data and limited economic and investment appraisal skills. This need for pragmatism is consistent with conditions in many African countries.
Given above, it is more crucial than ever to understand and employ appraisal options to enable favourable investment decisions. Investment appraisal is a process ‘to evaluate the worth, significance or status of 24 a financial investment. It has been described as ‘a collection of techniques used to identify the attractiveness of an investment’, with its goals being to assess the likelihood of achieving identified objectives and to support the production of a viable business case. 25 Methods include average rate of return, discounted cash flow techniques such as internal rate of return or net present value and payback period. 26 eHealth investment appraisal is therefore a process to evaluate which information and communication technology investment in health produces optimal net benefits. To arrive at these benefits it should consider outcomes across the health and social contexts in which eHealth is implemented, acknowledging the different stakeholders it is intended to benefit, each of which may have a different perspective on value and benefits.27,28
Recent initiatives such as the Digital Investment Implementation Guide (DIIG) 29 exemplify the feeling of urgency regarding understanding the eHealth investment process. However limitations to current tools exist, including DIIG, in terms of the breadth of issues considered, the depth of appraisal of each issue and the applicability and utility of tools for non-economist decision makers.
Two commonly recommended digital health appraisal techniques are Cost-effectiveness Analysis (CEA) and Cost-benefit Analysis (CBA). The data and expertise required to perform them are demanding, and in short supply in LMICs, particularly at sub-national locales.30,31 Data issues include ‘relevant data are not local; are incomplete, unreliable or imprecise; are challenged by experts or are completely absent and skill-related issues include the lack of ‘high quality analytical capacity’. 32 Consequently, tools such as CBA and CEA may be impractical for general and regular use as decision making tools in resource constrained LMIC settings.
Few African eHealth initiatives undergo economic assessment33,34 with some countries reporting little prior appraisal of any kind for eHealth projects.35,36 It is not clear whether lack of analytical capacity is the main barrier, though it is likely to be an important factor. Nevertheless, it is unlikely that this expertise will become readily available across Africa’s extensive health service delivery landscape in the short term. In the meantime, health-strengthening capabilities of digital health are no longer disputed and are deemed necessary to meet UHC objectives. 3 An eHealth investment appraisal approach is required that can be readily used by officials who have limited economics training or experience in identifying optimal investments, especially related to eHealth.
Internationally recognised metrics for eHealth investment appraisal have been proposed and used to demonstrate the applicability of the 5-Case Model (FCM) to African countries. 1 The aim of this scoping review is to identify and assess eHealth investment appraisal approaches and tools that have been used in African countries, describe their characteristics and assess their utility. Based on the findings, recommendations regarding eHealth investment appraisal are presented to inform development of an evidence-based ‘eHealth investment appraisal framework’ suitable for African settings.
Methods
Three electronic databases ‘suitable for use as principal search systems’ and ‘well-suited to evidence synthesis’ 37 (PubMed, Scopus and Science Direct) were searched for articles published between January 1, 2010 and June 30, 2020, since reviews had noted that there were few studies of good quality identified prior to that.38-40 Inclusion criteria were that the paper addressed any form of eHealth or digital health investment appraisal in Africa and was published in English. Conference proceedings were excluded.
Search terms used MeSH indexing and free text terms ‘electronic health records’, ‘telemedicine’, ‘e-health’, ‘ehealth’, ‘digital health’, ‘developing country’, ‘Africa’, ‘economic’, ‘business case’ and ‘finance’. The PubMed search string was ((electronic health records [MeSH terms] OR telemedicine [MeSH terms]) AND (developing countries [MeSH terms] OR (developing [title/abstract] AND countries [title/abstract]) OR developing countries [title/abstract] OR Africa [MeSH terms] OR Africa [title/abstract]) AND (economics [subheading] OR health care economics and organizations [MeSH Terms] OR business case [title/abstract] OR finance [title/abstract])).
One author (SB) conducted the searches and removed duplicates. Thereafter, all authors screened the abstracts and titles independently to determine inclusion status. Discrepancies were discussed, and consensus reached. Full texts of selected resources were retrieved and reviewed by 3 authors (SB, MM and RES) using the inclusion criteria. Discrepancies were again discussed and consensus reached to determine final selection of papers.
Economic Evaluations Checklists.
A three-point Likert scale (yes, unclear or no) was used to indicate the level of agreement with each checklist question. While higher numbers of Likert scale response options can improve reliability and validity of psychometric studies, 44 the three-point scale was deemed appropriate for this study to identify whether or not each appraisal characteristic was utilised by an African eHealth investment appraisal. The assessment was conducted by 3 authors (SB, MM and RES) and discrepancies discussed until consensus was reached. A narrative approach was used to summarise findings and comment on the included studies’ overall relevance to eHealth investment appraisal in Africa and any implications for developing an eHealth investment appraisal framework.
Results
The search process identified 488 articles, reduced to 468 after removing duplicates and articles not in English. Screening of abstracts found 22 articles of which, after full paper review, 5 met inclusion criteria (Figure 1). They are listed and categorised in Table 2. Flow diagram of search process. List and Characteristics of Selected Papers.
Characteristics of Investment Appraisal Checklists.
All 5 papers provided a well-defined research question (JBI1). Four papers provided a case for change (FCM-DH1) and strategic fit between health strategy and the digital health initiative (FCM-DH2).45-48 The role of digital health in extending access to healthcare services was emphasised in 4 of the 5 studies.45-48 4 studies reported on mobile health implementations; 1 was an initiative supporting AIDS care 49 and the other 3 varying aspects of Reproductive, Maternal, Neonatal and Child health (RMNCH).45,47,48 The fifth study dealt with costing a hypothetical teleradiology initative. 46 While all but 1 study achieved the JBI criterion of a description of alternatives (JBI2), including comparison between an intervention and a comparator, 46 only 1 study met the more stringent criterion of analysis of intervention alternatives (FCM-DH3). 47
The investment analyses reported were simple costings46,49; 1 break-even analysis, 47 1 CEA 45 and 1 cost-outcome analysis (COA). 48 All 5 papers identified costs. Three papers identified costs and benefits (JBI3, FCM-DH4),45,48,49 2 of which identified all important and relevant costs and outcomes for each alternative and measured costs and outcomes accurately (JBI5),45,48 for example, using 4 different costing tools to collect costs in 20 facilities. 45 2 papers measured costs and consequences in their control groups.45,49
Costs and outcomes were valued credibly (JBI6) in the only paper to establish clinical effectiveness (JBI4). 45 1 paper adjusted costs and outcomes for differential timing (JBI7), 48 no paper conducted an incremental analysis of costs and outcomes (JBI8) and 4 studies conducted a sensitivity analysis (JBI9, FCM-DH5) to investigate the uncertainty of estimates for costs and consequences.45,47-49
No paper addressed risk exposure (FCM-DH6) or adjusted for optimism bias (FCM-DH7). Neither did they provide clear practical plans for implementation (FCM-DH9) and delivery milestones (FCM-DH10) nor change management (FCM-DH11) or procurement (FCM-DH12). None of the papers addressed availability of data. One paper described plans for building partnerships (FCM-DH13). 47
Two papers reported results regarded as generalisable for the setting of interest in the review (JBI11),46,47 and 1 addressed generalisability by substantially describing 4 different cost-recovery scenarios and using uncertainty analysis to demonstrate that break-even points were driven by user volume rather than variations in programme costs. 47 No paper assessed all issues identified as of concern to users (JBI10).
The range of scale of the reported projects varied from 20 facilities to 125,320 users. Affordability challenges for poor users were noted45,47 and none of the papers addressed affordability adequately (FCM-DH8).
One paper recognised connectivity as a pre-requisite for digital health and the need for minimum functional bandwidth to achieve large scale implementations. 46 Cloud computing was a suggested approach.
Collaboration with international non-governmental organisations was identified as helpful, citing WHO, International Atomic Energy Agency, International Telecommunicatuions Union and policy statements such as the Smart Africa Manifesto. 46 The importance of the global digital health community helping to create successful and socially desirable business models was also noted. 47
Discussion
This scoping review identifies the general absence of adequate investment appraisal of African digital health initiatives in studies published to June 2020, something specifically noted in 3 of the 5 papers, and emphasises the importance of the work presented here.45,47,48 While they may have been fit for their original purpose, the 5 papers were consistent in their absence of important investment appraisal characteristics but inconsistent in their approaches, using simple costings, break-even analysis, CEA or COA. The choice of these assessment approaches may be explained by the predominance of mobile health initiatives (4 of the 5 studies identified)45,47-49 since CEA and COA are well suited for appraisal of mobile health’s relatively well-defined business cases and value chains. The relative simplicity of these investment appraisal techniques, which are easier to apply for users without advanced health economics expertise, may further explain the predominance of mobile health initiatives in the selected studies. More complex initiatives such as electronic patient records, hospital information systems and routine health information systems require more sophisticated approaches such as CBA.
The predominance of mobile health projects (4 out of 5) is not surprising. The opportunity for mobile phones to contribute to advancing UHC has been recognised by WHO 50 and WHO Guidelines on Digital Health provide extensive mobile health use cases. 51 Furthermore, the WHO guidelines emphasise the role of mHealth in targeted client communication specifically addressing the use of mobile phones to support communication for sexual, reproductive, maternal, newborn, child and adolescent health interventions. This role of mHealth to support RMNCH is also well established52-54 and consistent with the focus on maternal and child health strengthening between 1990 and 2015 in the Millennium Development Goals, 55 and now addressed in goals focussing on good health and well-being, and gender equality, in the Sustainable Development Goals. 56 This may explain the high proportion of papers on RMNCH projects (3 out of 5).
Experience and research has emphasised the importance of needs-based interventions that respond to identified health priorities. Thus, the momentum initiative in Europe noted ‘Identify a compelling need’ as a critical success factor for telemedicine interventions, 57 and prior research noted the need to find an ‘optimal solution to the most pressing (existing or anticipated) health-related problems’ for successful eHealth strategies. 58 The included papers align with health priorities with all but 1 study 49 articulating a case for change and strategic fit between the health strategy and digital health initiative. The issue is no longer whether a digital health initiative is aligned to health-strengthening needs, but rather which – from a number of options – offers the optimal solution and what investment appraisal approach should be used to make the case.
There was a disjunction between the recognition of the importance of appraisal techniques, and the actual application of these techniques in the reviewed papers. For example, whilst 3 studies identified costs and benefts,45,48,49 only 2 measured them,45,48 and only 1 of those valued them credibly. 45 Similarly, while 4 studies conducted sensitivity analysis, the absence of credible values assigned to costs and benefits made these analyses difficult to interpret.45,47-49
Noting that 40% of sub-Saharan Africans live on less than $1.90 a day, 59 sustainable digital health investments in Africa will need to be affordable and support other cost-contained strategies enabled by digital transformation. 46 This reinforces the important role of investment appraisal to help select optimal investments from numerous competing initiatives.
The role of partnerships to reduce communications cost was emphasised, 47 as well as ethical issues arising from certain cost reduction strategies such as data mining and advertising, which need further exploration. One paper 49 noted how an initiative could be ‘potentially affordable’ pending addressing a number of considerations such as task shifting. The opportunity to transfer costs to users was noted, as well as the concerning potential exclusion of those who are ‘too poor to pay’. 49 This highlights the need for a suitable business model, particularly to realise scale. 47 The inconsistent scale of the 5 studies’ initiatives made further comparison of scale impractical.
The importance of making informed decisions about optimal allocation of scarce resources when considering investment in eHealth applications is clear. How to perform this task in a straightforward manner in resource-constrained local and national settings is not provided, 29 particularly given data availability and quality challenges and limited training and skills in economic analyses. Whilst ‘appraisal checklists’ or other reporting guideline tools are available to guide reviews of available evidence,41,60-64 their applicability for digital health investment appraisal is uncertain. Some are narrow in their scope of application and can be complex, or require specific training or experience to apply correctly.65,66 Most are intended to enhance the quality of reported research. To address data and human capability challenges, it was necessary to find a compromise between sophistication and practical utility. To this end, the JBI tool was adopted, together with the adapted FCM tool.
The FCM-DH checklist was newly developed for this review and no evidence could be found of the JBI checklist being used in the manner described. Certainly, the 2 checklists applied have commonalities and differences, and when used in conjunction, they provided a novel, structured approach to the analysis of the investment appraisals reviewed. The FCM-DH checklist addresses characteristics not considered in the JBI checklist, such as exposure to risk, adjusting for optimism bias, having a practical plan for delivery, providing delivery milestones, addressing change management and building partnerships. Similarly, the JBI checklist addresses characteristics not considered by the FCM-DH checklist, such as a more thorough exploration of the extent to which costs and outcomes are evaluated, whether clinical effectiveness was established, whether the appraisals included all issues of concern to users and whether results are generalisable. Using them in conjunction may be an appropriate approach to compensate for the frequent absence of reliable data and lack of skills and ability when performing traditional economic assessments. 32
Neither checklist addresses 2 important factors, usability and sustainability, both necessary to secure eHealth’s benefits. 67 None of the 5 papers addressed usability or the related aspects of user acceptance, nor the technical knowledge and support required for productive use of the digital health tools. These contribute to sustainability, as does the existence of a business model, which was also absent from the papers. The lack of business models limits sustainability, increasing reliance on additional funding streams, including donor funding, to ensure continued service.
If either checklist had been used alone to select economic evaluations, all 5 articles would have been excluded. Therefore, despite their individual limitations, inclusion of aspects from both would provide a useful foundation for the development of a strong African-specific eHealth investment framework. The FCM-DH checklist identified the most gaps, and its modification could address missing aspects such as more robust values for costs and benefits, whether a study establishes clinical effectiveness, application usability and sustainable business models. This will provide the foundation for an eHealth investment framework that is accessible to those without high levels of health economics expertise, to guide Africa’s eHealth investment decisions.
Reports on the subsequent progress of the 5 initiatives could help to demonstrate the role, if any, that their investment appraisals played in their further development. While we could not find any formal evidence of further development, all 4 mobile health initiatives appear to have continued supported by grant funding, with information about their progress available on related project websites. This raises a sustainability issue that is beyond the scope of our paper; projects that are not able to transition from grant funding to a more sustainable funding model are at risk of not becoming sustainable interventions in the long term. We could not find any evidence of further development of the hypothetical teleradiology initiative described in the fifth paper.
Conclusion
The aim of this scoping review was to identify, characterise and assess eHealth investment appraisal methods used in African countries. The absence of appropriate methods for assessing this in resource-constrained settings led to selection of the JBI checklist and development of the FCM-DH checklist. Their application revealed key missing elements from the few publications found, such as affordability, risk, usability, sustainable business models, credible valuation of costs and benefits, planning and change management and addressing implementation issues.
While neither checklist was sufficient on its own, a combined JBI and FCM-DH checklist could form the foundation of an African eHealth investment appraisal framework. This, combined with modest building of eHealth appraisal capabilities among a cadre of African implementers and decision makers, may stimulate meaningful improvement of investment decisions and complement specialist health economics expertise, which is in short supply. It is unlikely that these concerns are confined to Africa and the findings may be pertinent to lower- and middle-income countries generally.
The findings of this study will facilitate the further development of an African eHealth investment framework.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclose receipt of the following financial support for the research, authorship and publication of this article: This work was supported by the Fogarty International Center of the National Institutes of Health [Award Number D43TW007004-13]. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
