Sage Journals: Discover world-class research

Abstract

Objective. The purpose of this study was to explore the research on the delivery and evaluation of pediatric health services by non-governmental organizations in low-and middle-income countries to better understand how they contribute to positive and sustainable health outcomes. Methods. A scoping review was completed using a 2-step study selection procedure. Results. Of the 5742 studies, 17 met criteria, including quantitative and mixed method designs, representing 10 different non-governmental organizations with programs in 33 low-and middle-income countries. Health outcomes were reported 89 times across the studies. A total of 56 different outcomes were identified in total, of which 24 were positive, 27 were negative, and 5 were unchanged. Conclusions. Widespread variation between non-governmental organizations exist, however, comprehensive pediatric health outcome evaluation is growing. Further emphasis should be given to adolescent specific research and robust measurement of quality of life.

Keywords

pediatric(s)non-governmental organizations low-and middle-income countries health outcomes evaluation

Introduction

Health inequities affect millions of people on a global scale, however, children and adolescents living in low-and middle-income countries (LMICs), especially those living in remote and poor areas, bear the burden of being on the bottom of the socioeconomic scale and social gradient of health.^1,2 Statistics from the early 2000’s indicated that globally, over 9 million children per year died before their fifth birthday and almost all of these deaths occurred in LMICs.¹ Appropriately, the set of 8 Millennium Development Goals (MDGs) established by the United Nations to be achieved by 2015, included the goal for a two-thirds reduction in the under-5 year old child mortality rate.^1,3 Reports from these goals have shown that between the years 1990 and 2015, global under-5 child mortality declined by 48%.⁴ This rapid mortality decline is associated with evidence-based health interventions such as skilled birth attendance, increased coverage of immunizations, and multi-sectoral socioeconomic improvements.⁴

Although the MDGs have contributed to the prioritization of pediatric health in the last 20 years, the emphasis has been on children 5 years and younger. As a result, much of the published literature has focused on programs and interventions that address maternal, newborn and child health issues,^5,6 leaving an incomplete understanding of health outcomes of older children and adolescents. The newly established Sustainable Development Goals (SDGs) with targets for the year 2030 are intended to build on the MDGs and continue to achieve sustainable development of nations.⁷ Goal number 3—to ensure healthy lives and promote well-being for all at all ages - is essential and continued efforts are required to meet this goal.⁷ This goal targets achievements such as ending epidemics like AIDS, tuberculosis, and malaria, as well as promoting mental health, prevention and treatment of substance abuse, and decreasing global deaths and injuries from traffic accidents.⁷ These targets are more inclusive of health issues for all population ages, including older children and adolescents, rather than specifically targeting children 5 years and younger. The priorities placed on all ages, in combination with the shift in recent years toward evaluation of more positive health outcomes (eg, improved health status, functional ability, and perceived quality of life) rather than death, disease, and disability, further emphasize the need for increased focus on the entire pediatric age spectrum and the broadening of health evaluation beyond mortality rates and negative health indicators.⁸

Non-governmental organizations (NGOs) have a significant presence within health systems in LMICs and provide services that are increasingly utilized to fill gaps in government health programs.^9-11 There are an estimated 10 million NGOs worldwide, including more than 40 000 international NGOs and thousands of local NGOs participating in health and development work in any single country.^12,13 While the benefits of NGO and government collaborations for sustainability of programs and access to care have been identified, these collaborations are contingent on a variety of factors and may not always occur.^14,15 In an era where evidence-based practice and financial accountability are drivers of healthcare, it is appropriate to examine whether NGOs operate in a way that transparently links positive and sustainable health outcomes with program implementation and spending. Because there are no mandatory governing bodies monitoring their agendas, it is imperative that NGOs are accountable not only financially to their donors, but also to the populations they serve for the impact and outcomes of their work.

Merton’s theory of unanticipated consequences of purposive action proposes that all social interventions have unintended consequences, and that interventions should be thoroughly investigated with attention to context and evidence before implementation.^16,17 Global health is filled with illustrations of unintended and often harmful consequences of programs, as good intentions in humanitarian focused environments can lead to the misconception that any healthcare is good healthcare, regardless of the quality of services provided.^18,19 This theory informed the proposed study, as it questions the interests and values of stakeholders and funders of NGOs and suggests that NGOs should thoroughly address the components of their programs and assess the potential consequences of their well-intended actions.

NGOs can face many challenges for monitoring of outcomes and evaluation of programs due to small operating budgets, limited resources, and unskilled staff.²⁰ When evaluations are feasible, the focus is generally on reporting metrics (eg, number of patients treated) rather than outcomes demonstrating effectiveness (eg, improved viral suppression rates in HIV positive individuals) and are completed to appease the program requirements mandated by donors.^20,21 Since the activities of each NGO vary by their country of operation, services provided, and populations served, standardization of these processes is a challenge. Additionally, NGO involvement in LMIC pediatric health initiatives is in the early stages from a research perspective.²² Studies exist that reveal a vast array of NGOs providing surgical care in LMICs, and highlight that short-term missions are the method most frequently utilized for delivery of surgical services.^23,24 While the studies contribute to the knowledge of pediatric surgical service delivery, there remains a gap in the amount of research known to be available, as well as any synthesis of pediatric medical services. In addition to this gap, there is no clear understanding of whether a standardized process exists for NGOs providing pediatric medical or surgical health services to evaluate their programs for positive and sustainable health outcomes.

In summary, there are significant knowledge gaps concerning NGO pediatric health outcome evaluation in LMICs, especially for NGOs operating independently of government programs. Variation in NGO practice coupled with outcome evaluation challenges make it difficult to identify what interventions are effective or ineffective. Synthesizing what it currently known in the academic, peer-reviewed, published research on NGO pediatric health delivery and outcome evaluation will help identify what strategies are currently being utilized, identify other knowledge gaps and areas for research on this topic, and further contribute to the growing body of pediatric NGO research in LMICs. Understanding how healthcare NGOs contribute to positive health outcomes in children and adolescents who live in LMICs is instrumental in ensuring the delivery of quality healthcare that is evidence-based, financially accountable, and falls in line with the goals and objectives for global health in the coming years. The specific objectives for this study were: 1) to identify and describe existing types of programs and services NGOs are providing to children and adolescents in LMICs, 2) to identify and describe the types of pediatric health outcomes NGOs are reporting, 3) to identify and describe NGO’s outcomes related to sustainability, and 4) to determine the pattern of outcome reporting over time.

Methods

Design

This was a scoping review as articulated by Arksey and O’Malley.²⁵ A scoping review is useful for addressing exploratory research questions by identifying all relevant literature on a specific topic, regardless of study design.^25,26 Considering the limited availability of empirical pediatric NGO research and the heterogenous nature of NGO programs, a scoping review was most appropriate to assess the potential size and scope of the research; summarize and analyze research findings; and identify gaps in the existing literature.^25,27

Stage 1: Identifying the research question

The research question and eligibility criteria were formulated using patient, intervention, comparison, outcome, and time frame (PICOT), a mnemonic used to guide the construction of evidence-based practice questions for health research.²⁸ Since the research question for this study was exploratory in nature and did not include a comparison or specific time frame, the “C” and “T” sections of the tool were not used. The PICOT strategy is outlined in Table 1, and each concept is defined to ensure a clear understanding of the components of the research question.

Table 1.

PICOT Strategy.

PICOT	Formulation	Definition of terms	Inclusion criteria	Exclusion criteria
P (Population)	Pediatric patients living in LMIC(s)	Pediatrics: Age group from birth to adolescence. The term pediatric is referred to as a specialty within healthcare stated to be concerned with maintaining health and providing medical care to children.²⁹LMIC(s): Countries with a GNI per capita of $12,055 or less. Includes countries classified by low-income (GNI per capita = < $1025), lower-middle income (GNI per capita = $1026-$3995) and upper-middle income economies (GNI per capita = $3996-$12 375).³⁰	• Age range of participants from birth-18 years of age • Age of participants > 18 years of age but identified as adolescents by individual study definitions • Research conducted in a LMIC	• Adult participants only • No separation between pediatric data and adult data when both populations included • Specific to maternal health, including pregnancy • Research/interventions conducted in high-income countries
I (Intervention)	Programs/service(s) delivered by healthcare NGO(s)	Healthcare NGO(s): Any non-profit, non-public, voluntary organization outside direct state control, pursuing activities related to health promotion, disease prevention, service delivery, health advocacy, education or training^31,32Healthcare services: The range of medical, surgical, social or psychological services, nursing care, health education or training, immunizations and disease prevention that primarily focus on improving health,³³ including the spectrum of care from promotion and prevention, diagnostic, rehabilitation and palliative care, self-care, home care, community care, primary care, long-term care and hospital care.³⁴*Healthcare intervention will be used interchangeably with healthcare service	Program/intervention by an organization identified as: • Non-governmental • Non-profit • Charity • Faith-based • Health service delivery relevant to any/all of the following: • Program implementation • Health outcomes or impacts • Sustainability • Capacity building • Evaluation • Monitoring • Performance • Health prevention • (eg, immunizations)	• Health promotion (eg, education) • Nutrition or nutritional interventions • WHO recommendations/policies implemented or tested in government or private healthcare contexts • Government and NGO collaborations • Epidemiology studies focused on only trends • NGO programs used as the comparison for an intervention
O (Outcome)	Health outcomes	Health outcomes: A change in the health status of an individual, group or population as a result of an intervention.³⁵Positive health outcome: The result of an intervention contributing to good or improved health.^35,36Negative health outcome: The result of an intervention contributing to worsened or ill health.^35,36	• Positive health outcomes (eg, clinical improvement, improved school attendance, cure) • Negative health outcomes (eg, death, complications)	• Either outcome(s), impact or sustainability not stated in aim/purpose of the research or within the research question

Abbreviations: LMIC, Low-and middle-income country; GNI, Gross national income; NGO, Non-governmental organizations; WHO, World Health Organization.

Research Question: What is known in the academic, published, peer reviewed literature about the delivery and evaluation of pediatric healthcare services by NGOs operating independently of government collaborations in LMICs for positive and sustainable health outcomes in children and adolescents?

Stage 2: Identifying relevant studies

The search strategy was developed in consultation with a library scientist to identify all relevant research on the topic. The initial MEDLINE search (Supplemental Appendix A) was peer reviewed using Peer Review of Electronic Search Strategies (PRESS)³⁷ prior to translation into CINAHL, CAB Direct, Nursing & Allied Health (via ProQuest), and PAIS Index (via ProQuest) databases. Key search terms included pediatrics/children/adolescents, low-and middle-income countries/developing countries/LMIC, and non-governmental organizations/non-profit organizations/voluntary agency/charity. The search was limited to English-language studies; however, no other limits were placed on the strategy. The initial search was conducted on February 27, 2019 and updated on September 21, 2020, during which no new studies were identified that met inclusion/exclusion criteria. Reference lists of each included study were further reviewed for any other relevant studies. One additional study that was not captured in the search, but met inclusion criteria through a hand search, was also included. Although Arksey & O’Malley’s framework indicates that the use of both published and unpublished studies helps ensure a comprehensive review,²⁵ newer scoping review protocols suggest that the inclusion of grey literature is dependent on the research question and objectives.^38,39 Given that NGO involvement in LMIC pediatric health initiatives is in the early stages from a research perspective, our purpose was to highlight and summarize what is known in published, academic, peer-reviewed work.²² Therefore, grey literature was not included.

Stage 3: Study selection

A 2-step study selection procedure was used, with 2 reviewers independently applying the inclusion and exclusion criteria to all citations obtained from the systematic literature search.⁴⁰ Step 1 was conducted at the title and abstract level. The citations selected from step 1 then underwent step 2, which involved full text review.²⁵ Disagreements between reviewers were resolved by a third member of the research team. The web-based software platform Covidence, was used to support both citation screening and full text screening. Only primary research studies (qualitative, quantitative, or mixed-methods study designs) published in English were included. As per the iterative nature of the scoping review framework, certain exclusion criteria were developed as more familiarity with the literature occurred.²⁵ To ensure that the scope of the review was not too broad, and that focus remained on the catchment of children that had not been extensively researched before, studies were excluded if they were specific to nutrition or maternal health, including pregnancy, as considerable research is already available on these topics. Studies were also excluded if they focused on government and NGO collaborations, as these did not answer the research question specific to NGO programs and interventions alone.

The remaining inclusion and exclusion criteria are outlined in Table 1. See the PRISMA (Figure 1) for the number of citations identified, screened, excluded, assessed for eligibility and included for evaluation.⁴¹

Figure 1.

PRISMA flow diagram.

Stage 4: Charting the data

Data extraction was done using an extraction table created for the review. The table was completed by the primary investigator and independently checked by a second reviewer for accuracy. The following components were extracted for each study (see Supplemental Appendix B); author(s), year of publication, country, NGO, study design, research question/purpose, population/sample, data collection, main results/findings specific to pediatric health outcomes. A quality assessment of studies was completed to determine the strengths and weaknesses of each study using a framework by Caldwell et al⁴² This framework provides a list of criteria for evaluating qualitative, quantitative and mixed methods research.⁴² The results of the application of this framework are reported in the last column in Supplemental Appendix B.

Stage 5: Collating, summarizing, and reporting the results

The results were organized according to the objectives of the scoping review and data extraction table fields. The collated data for the following study characteristics were summarized and reported on, including; date published, study design, categories of age groups, representation of LMICs by location and income status, representation of NGOs by individual organization, types of health services or health programs delivered, and data collection methods used. Health outcomes were extracted across studies and classified as positive, negative or unchanged outcomes and then grouped into categories of similar outcome types. Data analysis involved descriptive quantitative measures including counts, frequencies and percentages. The pattern of outcome reporting over time was determined by graphing the total number of outcomes reported throughout the period of published works collected as well as graphing the number of outcome types reported by year of study publication.

Ethical Approval and Informed Consent

Ethics approval and informed consent was not required for this scoping review study.

Results

Study Details

A total of 17 studies published between 1999 and 2016 were included in the review. They comprised both quantitative studies (n = 15, 88%) and mixed methods studies (n = 2, 12%). Of the 15 quantitative studies, the predominate study design was cross-sectional (n = 8, 47%), followed by retrospective record (eg, chart) reviews (n = 4, 24%), and cohort studies (group of individuals studied with data collected at different intervals over time²⁸) (n = 3, 18%). Categories of age groups included children (n = 6, 35%), adolescents (n = 1, 6%), children and adolescents (n = 6, 35%), and all ages (n = 4, 24%) (data for children and adolescents reported separately from adults).

Representation of LMICs

Fifteen studies reported on NGO interventions/programs delivered in 33 different LMIC’s. The mapped global distribution of these countries is shown in Figure 2. The remaining 2 studies reported on data collected in an additional 36 LMIC’s, but these countries were not listed. The majority of the identified countries were located on the continents of Africa (n = 19, 58%) and Asia (n = 13, 39%), with only 1 country located on the continent of South America (n = 1, 3%). In terms of economic status, 52% (n = 17) of the identified countries were classified as lower-middle income, 42%, (n = 14) as low-income, and 6% (n = 2) as upper-middle income.

Figure 2.

Global distribution of countries represented.

Representation of NGOs

The interventions/programs of 10 different NGOs were evaluated across the 17 studies. One study did not identify a specific organization but instead referred to a network of NGOs that contributed to a health program being examined.⁴³ Interventions by Médecins Sans Frontières were most commonly studied (n = 4, 24%), followed by Operation Smile (n = 3, 18%) and Walk for Life (n = 2, 12%).

Representation of Health Services

Several types of health services were represented, including medicine (n = 7 studies, 41%), surgery (n = 7 studies, 41%), mental health (n = 1 study, 6%), program implementation (n = 1 study, 6%), and mobility device provision (n = 1 study, 6%) (Table 2). The most common health interventions were cleft lip/palate repair (n = 4 studies, 24%), clubfoot treatment (n = 2 studies, 12%) and anti-retroviral therapy (n = 2 studies, 12%) (Table 2).

Table 2.

Summary of Health Service Provision.

Health service type	n (%)	NGO	References
Medical	7 (41%)
Ponseti method for clubfoot deformity	2	Walk for Life	Evans et al⁴⁴
Ponseti method for clubfoot deformity	2	Walk for Life	Perveen et al⁴⁵
Anti-retroviral therapy	2	Maryknoll	Myung et al⁴⁶
Anti-retroviral therapy	2	Médecins sans Frontières	Sauvageot et al⁴⁷
TB treatment	1	Médecins sans Frontières	Isaakidis et al⁴⁸
SAFE intervention for trachoma	1	Unnamed NGO network	Ngondi et al⁴³
Dermatology	1	Doctors in Aid of Children with Skin Diseases in Africa	Schmeller and Dzikus⁴⁹
Surgical	7 (41%)
Cleft lip & palate repair	4	Project Harar	Fell et al⁵⁰
		Operation Smile	Fisher et al⁵¹
		Operation Smile	McQueen et al⁵²
		Operation Smile	Sharp et al⁵³
Caustic stricture dilatation	1	Emergency NGO	Contini et al⁵⁴
Cardiac surgery	1	Chaine de l’Espoire	Mirabel et al⁵⁵
General surgical interventions	1	Médecins sans Frontières	Trudeau et al⁵⁶
Mental health	1 (6%)
Brief trauma focused therapy		Médecins sans Frontières	Lokuge et al⁵⁷
Program implementation	1 (6%)
Multi-sectoral development		Save the Children US	Gonzales et al⁵⁸
Mobility device provision	1(6%)
Wheelchair provision		United Cerebral Palsy Wheels for Humanity	Toro et al⁵⁹

Abbreviations: TB, tuberculosis; SAFE, surgery, antibiotics, facial cleanliness and environmental change

Data Collection Methods

Data used to report on the study outcomes were collected using a range of methods. Open ended or semi-structured interviews were used for data collection in 24% (n = 4) of the studies.^43-45,50,53 A survey or questionnaire was used for data collection in 24% (n = 4) of the studies.^43,58 Two studies (12%) used study specific, data collection tools, both evaluating outcomes related to surgical interventions within a peri-operative setting.^51,56 Direct observations or examinations by a clinician were used in 35% (n = 6) of the studies, either as a single method of data collection or in combination with other methods.^{43,44,49,50,57} Two studies (n = 2, 12%) had an unclear description of data collection methods, including 1 study that did not report any method of data collection, and another which only specified that data were collected using patient monitoring data (n = 1, 6%).^47,54 For the chart review studies, the authors extracted data pertinent to the outcome of interest directly from the health record and/or chart reviewed (n = 4, 27%).^46,52,55,57

Health Outcomes

The studies reported a relatively equal mix of time points at which outcomes were measured, including short-term (eg, immediately post-intervention) (n = 5 studies), medium-term (eg, 6 months to 2-years post-intervention) (n = 7 studies) and long-term outcomes (eg, >2 years post-intervention) (n = 5 studies). Six studies reported both positive and negative health outcomes, 5 studies reported only negative health outcomes, and 2 studies reported only positive health outcomes. Two studies reported positive, negative, and unchanged outcomes and 2 studies reported positive and unchanged outcomes. Outcomes were reported 89 times across the 17 studies. A total of 56 different outcomes were identified in total, of which 24 were positive, 27 were negative, and 5 were unchanged.

Six categories of positive health outcomes were identified, including; (1) successful intervention/cure rates; (2) reduction of symptoms or improving clinical signs (eg, improving CD4 count); (3) satisfaction with care or results of treatment; (4) physical improvement (eg, improved mobility, facial appearance, increase in weight, improved speech, etc.); (5) social improvement (eg, ability to play with friends unimpeded); and (6) improved prevalence rate after treatment. Four categories of negative health outcomes were identified, including; (1) death; (2) complications/adverse events (eg, return to OR, infection post-op, GI intolerance, etc.); (3) default from treatment or loss to follow-up; and (4) relapse or recurrence (eg, stricture recurrence). The most common category of positive health outcome reported across studies was physical improvement (n = 5, 29% of studies) (Table 3). The most common category of negative health outcome reported across studies were death (n = 9, 53% of studies) and complications/adverse events (n = 9, 53% of studies) (Table 3).

Table 3.

Summary of Outcome Categories Reported by Study.

Category	n (%) (studies)	References
Negative outcomes
Death	9 (53%)	Contini et al⁵⁴, Fisher et al⁵¹, Gonzales et al⁵⁸, Isaakidis et al⁴⁸, McQueen et al⁵²Mirabel et al⁵⁵, Myung et al⁴⁶Sauvageot et al⁴⁷, Trudeau et al⁵⁶
Complications/Adverse events	9 (53%)	Contini et al⁵⁴, Fell et al⁵⁰, Fisher et al⁵¹, Isaakidis et al⁴⁸, McQueen et al⁵², Mirabel et al⁵⁵, Myung et al⁴⁶, Sauvageot et al⁴⁷, Sharp et al⁵³
Default from treatment/ Loss to follow-up	5 (29%)	Evans et al⁴⁴, Isaakidis et al⁴⁸, Lokuge et al⁵⁷, Mirabel et al⁵⁵, Sauvageot et al⁴⁷
Relapse/Recurrence	2 (12%)	Contini et al⁵⁴, Mirabel et al⁵⁵
Positive outcomes
Success/cure rates	3 (18%)	Contini et al⁵⁴, Fell et al⁵⁰, Isaakidis et al⁴⁸
Reduction of symptoms/Improving clinical signs	3 (18%)	Isaakidis et al⁴⁸, Lokuge et al⁵⁷, Myung et al⁴⁶
Satisfaction	3 (18%)	Evans et al⁴⁴, Perveen et al⁴⁵, Toro et al⁵⁹
Physical improvement (eg, mobility, ability to speak, facial appearance, weight gain etc.)	5 (29%)	Evans et al⁴⁴, Fell et al⁵⁰, Gonzales et al⁵⁸, Perveen et al⁴⁵, Sharp et al⁵³
Social improvement (eg, playing with friends, school attendance)	2 (12%)	Fell et al⁵⁰, Sharp et al⁵³
Improved prevalence rate post treatment	2 (12%)	Ngondi et al⁴³, Schmeller and Dzikus⁴⁹
No change in outcome
Unchanged prevalence rate	2 (12%)	Gonzales et al⁵⁸, Schmeller and Dzikus⁴⁹
Unchanged school attendance	1 (6%)	Toro et al⁵⁹
No improvement in CD4 count	1 (6%)	Myung et al⁴⁶

Patient versus provider outcomes

Seven studies reported outcomes from the patient perspective compared to 10 studies that reported outcomes from the provider perspective alone. Outcomes reported by providers were specific to physical or physiological outcomes such as success or cure rate, complications, death, improving or worsening clinical signs etc. Outcomes reported from patients themselves incorporated a wider spectrum of outcome types including social (eg, ability to play with friends), financial (eg, cost of surgery), and emotional (eg, satisfaction) outcomes.

Outcome Evaluation Techniques

Most of the described outcomes were evaluated quantitatively and reported as frequencies, percentages, proportions, means or standard deviations. Inferential statistics were used across 15 studies primarily to draw comparisons between outcomes, groups and patient demographics, however, 3 studies used inferential statistics to report outcomes. The study by Gonzales et al⁵⁸ evaluated the point prevalence of diarrhea within 4 treatment groups using Chi-square analysis. The study by Ngondi et al⁴³ evaluated the change and percent reduction in active trachoma and unclean faces using 95% Confidence intervals. Lastly, the study by Toro et al⁵⁹ evaluated satisfaction with means of mobility between the treatment group and comparison group using Fisher’s exact test and Mann-Whitney U test.

The 2 mixed methods studies, both focused on evaluating NGO programs that repaired cleft lip and palates, reported positive outcomes which were identified within the themes of their qualitative analyses. These themes included improved appearance, improved school attendance, ability to play with friends unimpeded, improved speech, improved eating, improved social benefits (eg, less teasing) and simply, a repaired cleft.^50,53 Only 1 negative outcome, discussed as the report of negative features associated with surgery (eg, fear of pain or death, travel costs), was identified through qualitative analysis in the Sharp et al⁵³ study.

Three validated tools were used to assess outcomes across 3 studies. The Bangla Clubfoot Assessment tool⁶⁰ evaluated 6 positive outcomes in the Evans et al⁴⁴ and Perveen et al⁴⁵ studies including, brace use, ability to walk, run, squat and manage steps appropriately, and parental satisfaction. The Functional Mobility Assessment—Kids (FMA-Kids)^61,62 and Craig Handicap Assessment Recording Technique Short Form (CHART-SF)⁶³ assessed outcomes in the Toro et al⁵⁹ study, and resulted in the report of positive outcomes including increased satisfaction with mobility as well as a higher mobility domain for children with proxies who received a wheelchair, compared to children with proxies in the waitlist group. The Evans et al⁴⁴ study also utilized the Bhaskar tool⁶⁴ to evaluate relapse in children after clubfoot treatment, however, it is unclear if this tool has been validated.

Sustainability

Sustainability of interventions or programs was infrequently mentioned in the studies. Mirabel et al,⁵⁵ in their study on pediatric cardiac surgery programs in Cambodia and Mozambique, briefly suggested in their conclusion that capacity building aimed at improving program delivery, in addition to local political support, would assist in improving healthcare systems. Myung et al,⁴⁶ reported that further data needs to be collected to evaluate the long-term sustainability of their program for directly observed anti-retroviral therapy in HIV infected children. Perveen et al⁴⁵ stated that the purpose of their study was to evaluate the long-term outcomes in their clubfeet program to build on their assessment of project sustainability previously identified in their short-term results. However, except for the study by Gonzales et al,⁵⁸ which focused on health outcomes of a multi-sectoral development program, the studies included in this scoping review, despite the inclusion criteria, focused heavily on treatment and management of symptoms with immediate or sustained outcomes, and did not primarily focus on capacity building or improving infrastructure and current health systems in LMIC contexts.

Outcome Reporting Over Time

Over time, the overall reporting of outcomes by NGOs increased, however year-to-year variability was evident (Figures 3 and 4). Specifically, in the earliest publications (1999-2007), an average of n = 3 outcomes were reported per year and in the most recent publications (2008-2016), an average of n = 7 outcomes were reported per year. The evaluation of broader health outcomes beyond death, complications, physical or clinical improvement (eg, improved school attendance, improved ability to play with friends) were reported in the more recent years of publications (2008-2016) (Figure 4).

Figure 3.

Total number of outcomes reported over time by year of study publication.

Figure 4.

Outcome types reported over time by year of study publication.

Discussion

This scoping review sought to explore peer-reviewed, published research on the delivery of pediatric health services by NGOs in LMICs to further understand how healthcare NGOs are evaluating and contributing to positive and sustainable health outcomes in children and adolescents. Review of the included studies suggests that while there is very little consistency in data collection and outcome evaluation, this work is growing within the field of NGO practice in LMICs (Figures 3 and 4). The increased evaluation of health outcomes over time as well as the types of outcomes evaluated (eg, satisfaction of care, improved ability to play with friends etc.) indicates that NGOs are considering a broader spectrum of outcome types when evaluating their programs, rather than solely assessing and reporting on morbidity, mortality or numbers treated. These types of outcomes are useful not only for delineating between effective interventions, ineffective interventions and areas for improvement, but shows that NGOs are evaluating outcomes related to the complex health needs of children and adolescents in LMICs.

Interestingly, the majority of the included studies were published during the MDG era (2000-2015) as well as an increased number of outcomes reported in the latter years between 2008 and 2016. Studies published before the year 2000 or during the beginning on the MDG era focus on prevalence of disease and negative health indicators, while studies toward the middle and end of the MDG era have more comprehensive outcome evaluation using broader research methods. While this is a positive finding and may indicate that improved health outcome evaluation is due to the emphasis placed on achieving specific health goals during the MDG period, further research advancement is required to continue to advance health outcome evaluation and reporting, as well as increase the quality of research on this topic. The quality appraisal of the studies (Supplemental Appendix B) revealed a number of weaknesses within the body of research. These weaknesses highlight the need for more robust data collection methods, clearer and more thorough study designs, consistent inclusion of ethical considerations, and improved management of missing data. Additionally, although appropriate inferential statistics were used to answer many of the research questions, confounding variables were often not adjusted for and very few studies reported power calculations. These weaknesses coupled with the lack of overall generalizability of the results indicates the need for a stronger state of science. Lastly, all but 1 study conducted internal evaluations of their programs. The extent to which this may contribute to biased studies is unclear. Therefore, future evaluation research should include broader outcome types, improved study design and reporting of methods, and perhaps consider the use of external evaluations to improve research and evaluation quality.

Qualitative Research Methods for Outcome Evaluation

The under-representation of qualitative research methods in the included studies also emphasizes the need for the improvement of research methods in NGO outcome evaluation. Rich data on health outcomes were gathered through the research done by Fell et al⁵⁰ and Sharp et al,⁵³ and assisted in a deeper understanding of the experiences of the participants in the studies. Many authors and researchers proclaim the benefits of both mixed methods and qualitative research, as they contribute to an improved understanding of the human and social experience.^65-68 Polit and Beck²⁸ discuss that although outcomes research has been primarily focused on quantitative design approaches, the complexities of outcomes research is evolving to allow for more novel methodologic approaches in this area. Even relatively straightforward outcome research in LMICs could be considered complex due to context, and thus would benefit from a qualitative component. As previously discussed, multiple barriers exist for NGOs when evaluating program outcomes due to competing priorities, lack of financial capability, untrained staff, etc.²⁰ Perhaps further education on the benefits of qualitative methods for outcome evaluation would encourage NGOs to collect data in this manner, as these approaches allow for insights that quantitative approaches alone do not provide.⁶⁵ Qualitative research in this field, would help to further understand how the participants’ experiences relate or contribute to the achievement of positive, negative or unchanged outcomes in children and adolescents in LIMCs.

Evaluation of Quality of Life

Although the evaluation of broader health outcomes has increased over time and the types of health outcomes reported reflect some aspects of quality of life measurement, specific evaluation of quality of life was rarely conducted across the studies. Toro et al,⁵⁹ were the only researchers who stated an intended purpose to evaluate quality of life as a specific outcome in their study. Although they utilized a validated tool (World Health Organization Quality of Life-BREF scale⁶⁹) to measure this outcome, it was only used for quality of life assessment in adult participants, resulting in no actual measurement of quality of life for children or adolescents in the study.⁵⁹ The authors did measure outcomes that reflect components of quality of life (eg, improved mobility, improved participation in society) using other tools such as the FMA-Kids,^61,62 and CHART-SF⁶³ which resulted in statistically significant results for improvement in satisfaction with mobility for children with proxies (p<0.001, r = 0.50, ß = 0.99) and a decrease in mobility for children with proxies in the waitlist group in comparison to the wheelchair group. Although the results did not show any statistically significant results for improved participation in society, the study showed that wheelchair provision can improve certain aspects of quality of life in this population (mobility). Unfortunately, the FMA-Kids and CHART-SF tools did not completely measure or address the multiple domains included in a comprehensive quality of life assessment. At minimum, instruments measuring pediatric health related quality of life must include physical, mental and social health dimensions, and incorporate the role of school functioning.⁷⁰

The study by Sharp et al,⁵³ assessed the long-term outcomes of cleft lip and palate repair by Operation Smile’s programs using a study specific survey and questionnaire titled “Quality of Life After Cleft Palate Repair.”⁵³ The results included a report of positive outcomes experienced and perceived as important by children and adolescents such as improved speech, improved eating, improved social benefit and improved appearance.⁵³ Additionally, 3 other studies reported outcomes that suggested improvement in quality of life for either the child or family unit.^44,45,50,53 The 2 studies that evaluated the interventions provided by the NGO Walk for Life, reported outcomes of parental satisfaction in addition to improvements in physical functioning for the child such as the ability to walk, run and manage steps independently.^44,45 The study by Fell et al,⁵⁰ which assessed Project Harar’s impact of cleft lip and palate repair, reported qualitative themes that conveyed improvements in quality of life such as improvement in appearance, ability to play with friends unimpeded, as well as better school attendance. While these outcomes were certainly linked to or reflected improvement in quality of life, the studies within this review lacked robust validated measurement of quality of life as a primary outcome.

Further work is therefore needed on the comprehensive measurement of pediatric quality of life as an outcome of NGO programs/interventions, as health is not unidimensional. Pediatric quality of life measures assess the wider impact that health interventions have on a child or adolescent’s well-being and is seen as an essential health outcome.^70,71 Understanding how interventions impact pediatric quality of life is important, as the WHO endorses that health is not just the absence of disease but rather a state of complete physical, mental and social well-being.⁷² For NGOs, a more holistic approach to pediatric quality of life would improve performance measurement, inform quality improvement, and determine best areas for financial investment.⁷¹ Therefore, it is imperative that more robust assessments of quality of life are accounted for when determining whether NGO programs are positively impacting the populations they serve.

A study by Mabugu et al⁷¹ which assessed the methodological challenges for quality of life estimation in low-income countries for children, identified 8 different scales that measure pediatric quality of life. Although the study concluded that the use of these tools in low-income settings should be done cautiously with consideration of context-specific factors,⁷¹ the variety of tools identified reiterates that many are available for use. Unfortunately, it is unclear why any of these tools were not utilized for the pediatric participants in the Toro et al⁵⁹ study or why quality of life was not mentioned by any of the other researchers. After consideration of context-specific factors as, Mabugu et al⁷¹ suggests, these instruments could be useful to further report on the effectiveness of NGO interventions as well as add depth to the meaning of their outcome results. Proper measurement of quality of life could also be an indicator of long-term health improvements and sustainable outcomes of interventions which was identified as lacking within the included studies.

Adolescent Research

While a number of the studies included participants within the adolescent age bracket (children and adolescents, all ages), there was a lack of studies on adolescents alone. This was expected given that research priorities have been focused on maternal, newborn and under 5 years of age population health in recent years. The only study specific to adolescents was the Isaakidis et al⁴⁸ study, which presented the intermediate outcomes of HIV infected adolescents receiving anti-TB treatment by the NGO Médecins Sans Frontières in India.

According to UNAIDS 2019 estimates,⁷³ approximately 1.6 million adolescents between the ages of 10 to 19 are living with HIV globally. Yet, adolescents still lack adequate access to HIV treatment and other health services, even though they have been recognized as a critical age group requiring care through this time of development and transition into adulthood.⁷⁴ Issues highlighted within the Isaakidis et al⁴⁸ study mirror those discussed in the adolescent HIV literature, including higher rates of loss to follow-up, poor treatment adherence and increased need for psychosocial support compared to the adult HIV population in LMICs.^75,76 Barriers such as these contribute to undesirable outcomes such as death and loss from treatment, as reported in the Isaakidis et al⁴⁸ study, suggesting that research focused on adolescent populations needs to capture the effects of barriers to care for this age group and strategies to address them in order to improve adolescent health outcomes in LMICs. Armstrong et al⁷⁴ suggests that improved understanding of how to best provide care to the adolescent population is needed to inform innovative and targeted interventions to improve health outcomes. One of the priorities highlighted by the WHO and Collaborative Initiative for Pediatric HIV Education and Research is to gather further evidence on the management of coinfections among adolescents with HIV, especially those with advanced disease.⁷⁴ Isaakidis et al⁴⁸ suggest that active case-finding, direct referrals to treatment centers, targeted adherence counseling and social support around life events could improve the outcomes in their study, but also highlight that research on problem solving and improved decision making around interventions is also needed.

The 14 targets for adolescent health and development outlined in the sustainable development goals reiterate the message that further research, policy development and program modification is needed for this population.⁷⁷ The lack of studies focusing on outcomes in adolescents within this review supports the need for continued focus and effort among these targets. Healthcare professionals working in LMICs have a profound ability to be active participants in assisting adolescents with the management of their health needs and contribute to improved positive outcomes for those living with illness. Not only can healthcare professionals contribute to research initiatives, and policy development, they can be available to adolescents at the point of care to assist with effective initiatives such as active case finding, referral to supportive services, counseling, and social support.

Limitations

Despite contributing to a better understanding of the research by NGOs providing pediatric healthcare in LMICs, this scoping review has several limitations. The results are restricted solely to the literature that is published in academic, peer reviewed forums, as grey literature was not included in the review design. Therefore, the findings cannot be used as a complete representation of knowledge on this topic. Grey literature could further achieve the study objectives by contributing to a more detailed synthesis of reported health outcomes, sustainability, and types of programs and services NGOs are currently providing. Additionally, the pattern of outcome reporting over time may differ if grey literature reports were included. Although the purpose of the current study was to synthesize what is known based on published, academic, peer reviewed studies, future reviews could consider broadening the search strategy to include grey literature to provide a more comprehensive representation of knowledge on this topic.

Studies published in a language other than English may also further contribute to the findings but were not included in the review due to the language limitations placed on the search. One study that met criteria was included after a hand search of an NGO website was completed but was not identified via the systematic search. This was due to the fact that the MeSH term used to identify the organization in the study was ‘national health programs/economics’, which was not a keyword or MeSH heading determined appropriate in the search strategy used for this scoping review. Therefore, further studies may have been missed if they were inappropriately classified by the databases.

Conclusion

Findings from this scoping review highlight a number of gaps in research and practice. The results emphasize numerous variations between NGO practice for children and adolescents receiving care in LMICs. Not only is there a lack of research available regarding the work of pediatric NGOs globally, there is also widespread diversity between how outcomes are evaluated and reported. Progress has occurred toward an increased report of more comprehensive health outcomes beyond morbidity and mortality. However, there is still significant room for growth in this area, including the use of validated tools that capture data for a holistic quality of life measure. The use of qualitative or mixed methods research needs to be further utilized in NGOs’ research to better understand the complex context in which interventions are delivered, as well as processes and life experiences of children and adolescents. Finally, additional emphasis and priority needs to be given to research specific to adolescents due to the complex nature of their needs and barriers to achieving positive health outcomes in this population. Healthcare professionals are well situated within programs of many NGOs globally to advocate for and be involved in these changes to improve care and health outcomes in the pediatric population of LMICs.

Supplemental Material

sj-docx-1-gph-10.1177_2333794X21991011 – Supplemental material for Pediatric Health Outcome Evaluation in Low-and Middle-Income Countries: A Scoping Review of NGO Practice

Supplemental material, sj-docx-1-gph-10.1177_2333794X21991011 for Pediatric Health Outcome Evaluation in Low-and Middle-Income Countries: A Scoping Review of NGO Practice by Jennifer Taylor, Paula Forgeron, Amanda Vandyk, Allen Finley and Sophie Lightfoot in Global Pediatric Health

Supplemental Material

sj-heic-2-gph-10.1177_2333794X21991011 – Supplemental material for Pediatric Health Outcome Evaluation in Low-and Middle-Income Countries: A Scoping Review of NGO Practice

Supplemental material, sj-heic-2-gph-10.1177_2333794X21991011 for Pediatric Health Outcome Evaluation in Low-and Middle-Income Countries: A Scoping Review of NGO Practice by Jennifer Taylor, Paula Forgeron, Amanda Vandyk, Allen Finley and Sophie Lightfoot in Global Pediatric Health

Footnotes

Acknowledgements

The authors would like to acknowledge Marie-Cécile Domecq, Library specialist, for her expertise and peer review of the electronic search strategy. Her valuable contribution to the search has enriched this work.

Author Contributions

Jennifer Taylor: Contributed to conception and design; contributed to analysis; drafted the manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

Paula Forgeron: Contributed to conception and design; contributed to analysis; critically revised the manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

Amanda Vandyk: Contributed to conception and design; critically revised the manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

Allen Finley: Contributed to conception and design; critically revised the manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

Sophie Lightfoot: Contributed to analysis; critically revised the manuscript; gave final approval; agrees to be accountable for all aspects of work ensuring integrity and accuracy.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jennifer Taylor

Supplemental Material

Supplemental material for this article is available online.

References

Houweling

TAJ

Kunst

. Socio-economic inequalities in childhood mortality in low-and middle-income countries: a review of the international evidence. Br Med Bull. 2010;93:7-26. doi:10.1093/bmb/ldp048

Commission on Social Determinants of Health. Closing the Gap in a Generation: Health Equity through Action on the Social Determinants of Health. Final Report of the Commission on Social Determinants of Health. Geneva; 2008. doi:10.1080/17441692.2010.514617

World Health Organization. MDG 4: reduce child mortality. World Health Organization. https://www.who.int/topics/millennium_development_goals/child_mortality/en/. Published 2015. Accessed January 12, 2019.

Cohen

Murray

Jack

Arscott-mills

Verardi

. Impact of multisectoral health determinants on child mortality 1980 – 2010: an analysis by country baseline mortality. PLoS One. 2017;12:1-16. doi:10.1371/journal.pone.0188762

Bhutta

Black

. Global maternal, newborn, and child health — so near and yet so far. N Engl J Med. 2013;369:2226-2235. doi:10.1056/NEJMra1111853

Corsi

Subramanian

S V

. Association between coverage of maternal and child health interventions, and under-5 mortality: a repeated cross-sectional analysis of 35 sub-Saharan African countries. Glob Health Action. 2014;7:1-11. doi:10.3402/gha.v7.24765

United Nations. Sustainable development goals. https://sustainabledevelopment.un.org. Published 2018. Accessed January 13, 2019.

Mitchell

Ferketich

Jennings

, et al. Quality health outcomes model. J Nurs Scholarsh. 1997;30:43-46. doi:10.1111/j.1547-5069.1998.tb01234.x

Lurie

. Global health equity and advocacy: the roles of international non-governmental organizations. Heal Cult Soc. 2012;2:103-114. doi:10.5195/HCS.2012.41

10.

Lorgen

. Dancing with the state: the role of NGOs in health care and health policy. J Int Dev. 1998;10:323-339. doi:10.1002/(SICI)1099-1328(199805/06)10:3<323::AID-JID424>3.0.CO2-4

11.

Sarriot

Winch

Ryan

, et al. Qualitative research to make practical sense of sustainability in primary health care projects implemented by non-governmental organizations. Int J Health Plann Manage. 2004;19:3-22. doi:10.1002/hpm.743

12.

Weintraub

Talbot

Wachter

, et al. Strategic, value-based delivery in global health care: innovations at Harvard University and Brigham and women’s hospital. Mt Sinai J Med. 2011;78:458-469. doi:10.1002/MSJ

13.

Nonprofit Action. Facts and stats about NGOs worldwide. http://nonprofitaction.org/2015/09/facts-and-stats-about-ngos-worldwide/. Published 2015. Accessed January 12, 2020.

14.

Ullah

ANZ

Newell

Ahmed

Hyder

MKA

Islam

. Government-NGO collaboration: the case of tuberculosis control in Bangladesh. Health Policy Plan. 2006;21:143-155. doi:10.1093/heapol/czj014

15.

Nelson-Nuñez

Cartwright

. Getting along or going alone: understanding collaboration between local governments and NGOs in Bolivia. Lat Am Polit Soc. 2018;60:76-101. doi:10.1017/lap.2017.7

16.

Allen-Scott

Hatfield

McIntyre

. A scoping review of unintended harm associated with public health interventions: towards a typology and an understanding of underlying factors. Int J Public Health. 2014;59:3-14. doi:10.1007/s00038-013-0526-6

17.

Merton

. The unanticipated consequences of purposive social action. Am Sociol Rev. 1936;1:894-904.

18.

Kleinman

. Four social theories for global health. Lancet. 2010;375:1518-1519. doi:10.1016/S0140-6736(10)60646-0

19.

Maki

Qualls

White

Kleefield

Crone

. Health impact assessment and short-term medical missions: a methods study to evaluate quality of care. BMC Health Serv Res. 2008;8:1-8. doi:10.1186/1472-6963-8-121

20.

Bozzo

. Evaluation resources for nonprofit organizations: usefulness and applicability. Nonprofit Manag Leadersh. 2000;10:463-472.

21.

Palumbo

. Exploring the divide between output and outcome measures in health care. J Health Manag. 2017;19:537-552. doi:10.1177/0972063417727622

22.

Magalhães Costa

Silveira da Silva

Bonan

. Non-novernmental organizations in the area of child healthcare – review of the literature. SciELO Bras. 2011;16:3181-3196.

23.

Shrime

Sleemi

Ambereen

. Charitable platforms in global surgery: a systematic review of their effectiveness, cost-effectiveness, sustainability, and role training. World J Surg. 2015;39:10-20. doi:10.1007/s00268-014-2516-0

24.

Nguyen

Jacobs

Dearani

, et al. Survey of nongovernmental organizations providing pediatric cardiovascular care in low- and middle-income countries. World J Pediatr Congenit Hear Surg. 2014;5:248-255. doi:10.1177/2150135113514458

25.

Arksey

O’Malley

. Scoping studies: towards a methodological framework. Int J Soc Res Methodol Theory Pract. 2005;8:19-32. doi:10.1080/1364557032000119616

26.

Colquhoun

Levac

Brien

KKO

, et al. Scoping reviews: time for clarity in definition, methods, and reporting. J Clin Epidemiol. 2014;67:1291-1294. doi:10.1016/j.jclinepi.2014.03.013

27.

Grant

Booth

. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26:91-108. doi:10.1111/j.1471-1842.2009.00848.x

28.

Polit

Beck

. Nursing Research: Generating and Assessing Evidence for Nursing Practice. 9th ed. Lippincott Williams & Wilkins; 2012.

29.

Kastner

Wilczynski

Walker-dilks

Mckibbon

. Age-specific search strategies for medline. J Med Internet Res. 2006;8:e25. doi:10.2196/jmir.8.4.e25

30.

The World Bank. World bank country and lending groups. https://datahelpdesk.worldbank.org/knowledgebase/articles/906519. Published 2017. Accessed November 18, 2018.

31.

Piotrowicz

Cianciara

. The role of non-governmental organizations in the social and the health system. Przegl Epidemiol. 2013;67:69-74.

32.

Hecht

Tanzi

. The Role of Non-Governmental Organizations in the Delivery of Health Services in Developing Countries. World Development Report Background Papers. Washington, DC; 1994. Accessed February 20, 2018. http://documents.worldbank.org/curated/en/711461468125054585/The-role-of-non-governmental-organizations-in-the-delivery-of-health-services-in-developing-countries

33.

Chaudhry

. NGOs and health care: deliverance or dependence. Worldmark Global Health Med Issues 2016;2:439-445. http://link.galegroup.com/apps/doc/CX3628100065/GIC?u=albertak12&xid=d4a1d443

34.

World Health Organization. Health services. https://www.who.int/topics/health_services/en/. Published 2019. Accessed January 6, 2019.

35.

World Health Organization. Health Promotion Glossary. Geneva, Switzerland; 1998. Accessed January 6, 2019. https://www.who.int/healthpromotion/about/HPRGlossary1998.pdf?ua=1

36.

World Health Organization. Health impact assessment: glossary of terms used. https://www.who.int/publications/m/item/glossary-of-terms-used-for-health-impact-assessment-hia. Published 2020. Accessed February 11, 2020.

37.

Mcgowan

Sampson

Salzwedel

Cogo

Foerster

Lefebvre

. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40-46. doi:10.1016/j.jclinepi.2016.01.021

38.

Tricco

Lillie

Zarin

, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467-473. doi:10.7326/M18-0850

39.

Joanna Briggs Institute. Scoping reviews. In: Aromataris

Munn

, eds. JBI Manual for Evidence Synthesis. Joanna Briggs Institute; 2020. https://synthesismanual.jbi.global/

40.

Levac

Colquhoun

O’Brien

. Scoping studies: advancing the methodology. Implement Sci. 2010;5:1-9. doi:10.1186/1748-5908-5-69

41.

Moher

Liberati

Tetzlaff

Altman

. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:1-8. doi:10.1136/bmj.b2535

42.

Caldwell

Henshaw

Taylor

. Nurse education today developing a framework for critiquing health research: an early evaluation. Nurse Educ Today. 2011;31:e1-e7. doi:10.1016/j.nedt.2010.11.025

43.

Ngondi

Onsarigo

Matthews

, et al. Effect of 3 years of SAFE (surgery, antibiotics, facial cleanliness, and environmental change) strategy for trachoma control in southern Sudan: a cross-sectional study. Lancet. 2006;368:589-595. doi:10.1016/S0140-6736(06)69202-7

44.

Evans

Chowdhury

MMH

Kabir

Rahman

. Walk for life - the national clubfoot project of Bangladesh: the four-year outcomes of 150 congenital clubfoot cases following ponseti method. J Foot Ankle Res. 2016;9:1-10. doi:10.1186/s13047-016-0175-0

45.

Perveen

Evans

Ford-powell

, et al. The Bangladesh clubfoot project: audit of 2-year outcomes of ponseti treatment in 400 children. J Pediatr Orthop. 2014;34:720-725. doi:10.1097/BPO.0000000000000225

46.

Myung

Pugatch

Brady

, et al. Directly observed highly active antiretroviral therapy for HIV-infected children in Cambodia. Am J Public Health. 2007;97:974-977. doi:10.2105/AJPH.2005.083253

47.

Sauvageot

Schaefer

Olson

Pujades-Rodriguez

O’Brien

. Antiretroviral therapy outcomes in resource-limited settings for HIV-infected children <5 years of age. Pediatrics. 2010;125:1039-1047. doi:10.1542/peds.2009-1062

48.

Isaakidis

Paryani

Khan

Mansoor

Manglani

. Poor outcomes in a cohort of HIV-infected adolescents undergoing treatment for multidrug-resistant tuberculosis in Mumbai, India. PLoS One. 2013;8:e68869. doi:10.1371/journal.pone.0068869

49.

Schmeller

Dzikus

. Skin diseases in children in rural Kenya: long-term results of a dermatology project within the primary health care system. Br J Dermatol. 2001;144:118-124. doi:10.1111/j.1365-2133.2001.03962.x

50.

Fell

Hoyle

Abebe

, et al. The impact of a single surgical intervention for patients with a cleft lip living in rural Ethiopia. Br J Plast Surg. 2014;67:1194-1200. doi:10.1016/j.bjps.2014.05.019

51.

Fisher

Nichols

Stewart

Finley

Magee

Nelson

. Assessing pediatric anesthesia practices for volunteer medical services abroad. Anesth Analg. 2001;95:1315-1322.

52.

Mcqueen

KAK

Magee

Crabtree

Romano

Burkle

. Application of outcome measures in international humanitarian aid: comparing indices through retrospective analysis of corrective surgical care cases. Prehospital Disaster Med. 2009;24:39-46. doi:10.1017/S1049023X0000652X

53.

Sharp

Canady

Ligot

Hague

Gutierrez

. Caregiver and patient reported outcomes after repair of cleft lip and/or palate in the Philippines. Cleft Palate-Craniofacial J. 2008;45:163-171. doi:10.1597/06-230.1

54.

Contini

Garatti

Swarray-deen

Depetris

Cecchini

Scarpignato

. Corrosive oesophageal strictures in children: outcomes after timely or delayed dilatation. Dig Liver Dis. 2009;41:263-268. doi:10.1016/j.dld.2008.07.319

55.

Mirabel

Lachaud

Offredo

, et al. Cardiac surgery in low-income settings: 10 years of experience from two countries. Arch Cardiovasc Dis. 2016;110:82-90. doi:10.1016/j.acvd.2016.05.006

56.

Trudeau

MON

Baron

Hérard

, et al. Surgical care of pediatric patients in the humanitarian setting: the médecins sans frontières experience, 2012-2013. JAMA Surg. 2015;150:1080-1085. doi:10.1001/jamasurg.2015.1928

57.

Lokuge

Shah

Pintaldi

, et al. Mental health services for children exposed to armed conflict: médecins sans frontières ‘ experience in the Democratic Republic of Congo, Iraq and the occupied Palestinian territory. Paediatr Int Child Health. 2013;33:259-272. doi:10.1179/2046905513Y.0000000098

58.

Gonzales

Dearden

Jimenez

. Do multi-sectoral development programmes affect health? A Bolivian case study. Heal Policy Plan. 1999;14:400-408. doi:heapol/14.4.400

59.

Toro

Eke

Pearlman

. The impact of the World Health Organization 8-steps in wheelchair service provision in wheelchair users in a less resourced setting: a cohort study in Indonesia. BMC Health Serv Res. 2016;16:1-12. doi:10.1186/s12913-016-1268-y

60.

Evans

Perveen

Ford-Powell

Barker

. The Bangla clubfoot tool: a repeatability study. J Foot Ankle Res. 2014;7:1-6. doi:10.1186/1757-1146-7-27

61.

Rispin

Schein

Wee

. Modification of the functional mobility assessment for use with school children in Kenya. In: 29th international seating symposium. Nashville, TN, 2013.

62.

Kumar

Schmeler

Karmarkar

, et al. Test-retest reliability of the functional mobility assessment (FMA): a pilot study. Disabil Rehabil Assist Technol. 2013;8:213-219. doi:10.3109/17483107.2012.688240

63.

Hall

Dijkers

Whiteneck

Brooks

Stuart

. The Craig handicap assessment and reporting technique (CHART): metric properties and scoring. Top Spinal Cord Inj Rehabil. 1998;4:16-30.

64.

Bhaskar

Patni

. Classification of relapse pattern in clubfoot treated with Ponseti technique. Indian J Orthop. 2013;47:4-9.

65.

Doorenbos

. Mixed methods in nursing research: an overview and practical examples. Kango Kenkyu. 2014;47:207-217. http://www.ncbi.nlm.nih.gov/pubmed/25580032%0Ahttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4287271.

66.

Forman

Creswell

Damschroder

Kowalski

Krein

. Qualitative research methods: key features and insights gained from use in infection prevention research. Am J Infect Control. 2008;36:764-771. doi:10.1016/j.ajic.2008.03.010

67.

Sandelowski

. Using qualitative methods in intervention studies. Res Nurs Health. 1996;19:359-364. doi:10.1002/(sici)1098-240x(199608)19:4<359::aid-nur9>3.3.co;2-8

68.

Malterud

. The art and science of clinical knowledge: evidence beyond measures and numbers. Lancet. 2001;358:397-400. doi:10.1016/S0140-6736(01)05548-9

69.

Harper

Power

Orley

, et al. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychol Med. 1998;28:551-558. doi:10.1017/S0033291798006667

70.

Varni

Seid

Kurtin

. PedsQL^TM 4.0: reliability and validity of the Pediatric Quality of Life Inventory^TM version 4.0 generic core scales in healthy and patient populations. Med Care. 2001;39:800-812. doi:10.1097/00005650-200108000-00006

71.

Mabugu

Revill

van den Berg

. The methodological challenges for the estimation of quality of life in children for use in economic evaluation in low-income countries. Value Heal Reg Issues. 2013;2:231-239. doi:10.1016/j.vhri.2013.07.005

72.

World Health Organization. Constitution of the World Health Organization. WHO; 1948.

73.

UNAIDS. UNAIDS 2019 Estimates. UNAIDS; 2019. http://aidsinfo.unaids.org. Accessed November 20, 2019.

74.

Armstrong

Nagata

Vicari

, et al. A global research agenda for adolescents living with HIV. J Acquir Immune Defic Syndr. 2018;78:S16-S21. doi:10.1097/qai.0000000000001744

75.

Auld

Shiraishi

Wabwire-Mangen

, et al. Antiretroviral therapy enrollment characteristics and outcomes among HIV-infected adolescents and young adults compared with older adults-seven African countries, 2004-2013. Morb Mortal Wkly Rep. 2014;63:1097-1103.

76.

Hudelson

Cluver

. Factors associated with adherence to antiretroviral therapy among adolescents living with HIV/AIDS in low- and middle-income countries: a systematic review. AIDS Care - Psychol Socio-Medical Asp AIDS/HIV. 2015;27:805-816. doi:10.1080/09540121.2015.1011073

77.

United Nations. Every Woman, Every Child. Global Strategy for Women’s, Children’s and Adolescent’s Health (2016-2030): Survive, Thrive, Transform. United Nations; 2015. https://www.who.int/life-course/partners/global-strategy/globalstrategyreport2016-2030-lowres.pdf

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

1.06 MB