Sage Journals: Discover world-class research

Abstract

Objective

This study evaluated the implementation fidelity (IF) of HIV/AIDS Care and Support (C&S) services for orphans and vulnerable children (OVC) at the African Network for Prevention and Protection Children against Neglect (ANPPCAN) Project, Gondar City, 2023.

Method

A facility-based convergent parallel mixed-methods design was used to assess IF and identify factors influencing caregiver responsiveness. Quantitative data were collected from 424 OVC caregivers, 385 document reviews, 27 observations, and an inventory checklist. Qualitative data were obtained from five key informant interviews. Data were gathered concurrently, analyzed separately, and integrated during interpretation to provide a comprehensive view.

Result

The overall IF was 67.35%, with adherence at 81.5%, dosage at 45%, and caregiver responsiveness at 70.6%. Responsiveness was significantly associated with having separate waiting rooms, take-home materials, and friendly service delivery.

Conclusion

Although adherence was high, dosage was low and responsiveness moderate, indicating the need to strengthen service delivery components to improve implementation fidelity.

Plain Language Summary Title

How Well are HIV/AIDS Support Services for Orphaned and Vulnerable Children Working in Gondar, Ethiopia?

Plain Language Summary

Orphans and vulnerable children affected by HIV/AIDS often need special care and support. In Gondar City, Northwest Ethiopia, programs are in place to provide these services in a way that is focused on the needs of the child and their family. This study looked at how well these services are being carried out as planned. We collected information from caregivers, health workers, documents, and direct observations. We also interviewed key people involved in the program. The study found that the services are being followed quite well overall, but some parts—like how often the services are delivered—need improvement. Most caregivers responded positively to the services, especially when the environment was friendly, there were separate waiting rooms, and children received items to take home. In conclusion, while the services are on the right track, there is room to make them stronger and more consistent so that all children receive the full support they need.

Keywords

participant responsiveness adherence dosage implementation fidelity Gondar Ethiopia

Introduction

Globally, there is a growing concern for orphans and vulnerable children (OVC) under the age of 18, particularly in sub-Saharan Africa (SSA), where many have lost 1 or both parents to human immunodeficiency virus (HIV)/acquired immune deficiency syndrome (AIDS).¹ According to the United Nations Children’s Fund (UNICEF) and the United States Agency for International Development (USAID), a significant number of children are also responsible for caring for chronically ill parents. By 2020, an estimated 15.4 million children under 17 were expected to have lost 1 or both parents to AIDS. In 2020, about 1.7 million children aged under 14 were living with HIV, but only 54% were receiving treatment, and just 40% of those achieved viral suppression.²

In developing countries, surveys indicate that 1 in 5 children under 18 lives in extremely challenging conditions due to HIV/AIDS.³ The AIDS pandemic in sub-Saharan Africa has led to an unprecedented increase in the number of OVC. Notably, teenage females are disproportionately affected, with higher rates of HIV contraction compared to their male counterparts.²

In Ethiopia, over 12% of the entire child population is impacted by the loss of 1 or both parents due to HIV/AIDS. The situation is especially critical in Gondar City, which has an estimated 13 234 OVC.⁴ Several factors contribute to this high burden. Gondar is one of the largest and fastest-growing urban centers in northwest Ethiopia, attracting large numbers of rural migrants in search of employment, many of whom live in poor housing and under precarious economic conditions.^5,6 This urban migration is associated with increased risks of family disintegration, poverty, and limited access to social protection, all of which exacerbate the vulnerability of children. Moreover, the city has a relatively high HIV prevalence compared to national averages, partly due to its position as a regional hub with a mobile population, increasing the number of children orphaned by HIV/AIDS.⁷ Weak social support networks, limited access to basic health and education services for poor families, and high rates of child labor and neglect also contribute to the large number of OVC in the city.^8,9

Care and Support (C&S) services for OVC affected by HIV/AIDS are prioritized by key international agencies and donors working to enhance OVC wellbeing globally.^10,11 However, significant treatment gaps exist, with service coverage in many SSA countries ranging from 12% to 48%.¹² Community-based organizations, such as the African Network for Prevention and Protection Children against Neglect (ANPPCAN)-Ethiopia, play a crucial role in addressing these issues. ANPPCAN is a nonprofit organization focused on child protection, aiming to improve local HIV pandemic control through a comprehensive case management approach that enhances access to care and promotes economic stability and wellbeing for OVC.¹³

To ensure the effectiveness of OVC C&S programs, it is essential to implement them with high fidelity to their original design.^14,15 Implementation fidelity (IF) refers to the degree to which programs are delivered as intended, which is crucial for assessing the quality of implementation.^16,17 Without evaluating fidelity, stakeholders cannot determine the quality of service delivery, making monitoring an essential aspect of program success.¹⁴

Despite some progress in HIV testing, the global community has not met the United Nations target of detecting 90% of individuals living with HIV by 2020, particularly among pediatric and adolescent populations. In eastern and southern Africa, only 66% of young women and 50% of young men aged 15 to 24 were aware of their HIV status.¹ Early infant diagnostic testing is also inadequate, with only 52% of HIV-exposed infants tested by 2 months of age in high-burden countries.¹⁸

In resource-constrained settings, identifying individuals with undiagnosed HIV has become increasingly challenging.^19-21 Facility-based HIV testing strategies often overlook OVC and do not specifically target children and adolescents served by community programs. Home-based HIV testing has shown promising results, with higher positivity rates reported among OVC.^22-24 However, challenges remain, including stigma and confidentiality concerns.^25-27

Therefore, the objective of this study was to evaluate the IF of HIV/AIDS C&S services provided to OVC in Gondar City, Northwest Ethiopia. Specifically, the study aimed to assess the levels of adherence, dosage, and caregiver responsiveness, and to identify factors influencing these fidelity dimensions.

Evaluating fidelity and providing feedback to practitioners can improve IF.²⁸ Despite the importance of fidelity in HIV prevention interventions, limited data exists on the screening processes targeted at OVC in their home environments.²⁹ HIV/AIDS C&S service Logic Model presented in Figure 1.

Figure 1.

A Logic Model for IF of HIV/AIDS C&S Services in ANPPCAN, Gondar City, June 2023. Abbreviations: AIDS, Acquired Immune Deficiency Syndrome; ANPPCAN, African Network for Prevention and Protection Children Against Neglect; C&S, Care and Support; HIV, Human Immunodeficiency Virus; IF, Implementation Fidelity.

Evaluation Methods and Materials

Evaluation Area and Period

The study was conducted in Gondar City, Northwest Ethiopia, located approximately 750 km from Addis Ababa and 182 km from Bahir Dar. Gondar is a densely populated and historically significant urban center, with an estimated population of 360 600 in 2023, including 13 234 OVC for the past 2 years, the ANPPCAN-Ethiopia has been the sole nongovernmental organization providing family-centered HIV/AIDS C&S services to OVC in the city.

Inclusion and Exclusion Criteria

Program eligibility includes households meeting at least one of the following criteria: child-headed households, families caring for orphans, households with HIV-positive members, those with severely malnourished children, and households headed by sex workers. During enrollment, ANPPCAN-trained community caseworkers conduct HIV risk screenings for children under 18. Children who respond affirmatively to any screening question are referred to appropriate service delivery points (SDPs) for HIV testing and related care.

To achieve the evaluation objectives and ensure data quality, the study applied the inclusion criteria were as follows: caregivers of OVC aged 18 years or older; key informants who had worked at their respective SDPs for at least 6 months; C&S providers with a minimum of 6 months of experience at their SDPs; and officially registered OVC C&S documents maintained for at least 1 year. The exclusion criteria included caregivers who were critically ill during the data collection period and caregivers who visited SDPs multiple times during the data collection period.

An evaluability assessment was conducted from October 1 to 30, 2022, followed by the main evaluation, which took place from May 1 to June 1, 2023.

Evaluation Design

A facility-based convergent parallel mixed-methods design was employed to evaluate the IF of HIV/AIDS C&S services for OVC and to identify factors influencing caregiver responsiveness to these services. In this design, qualitative and quantitative data were collected concurrently, analyzed independently, and then integrated during the interpretation phase to provide a comprehensive understanding of the findings.

Evaluation Approach

A process evaluation documents and analyzes the early development and actual implementation of a program, assessing whether planned strategies were executed and whether intended outputs were achieved. It focuses on how a program is delivered, rather than on outcomes alone, and is essential for understanding the implementation context and identifying areas for improvement. Therefore, this evaluation adopted a formative approach, which is particularly useful for informing ongoing program improvements during the implementation phase rather than after completion. Formative and process evaluations are often used in combination to enhance implementation quality by providing timely feedback to program managers and stakeholders on barriers, facilitators, and fidelity to the intended model of delivery.^30,31 The primary purpose of this evaluation was to gather insights on program performance by identifying challenges in service delivery and assessing whether core program activities were executed as intended.

Focus on Evaluation and Dimensions

This evaluation focused on the IF of HIV/AIDS C&S services for OVC, with the goal of examining both the overall fidelity of implementation and the factors influencing caregiver responsiveness. IF was assessed using 3 core dimensions commonly cited in implementation science: adherence (the extent to which program components were delivered as planned), dosage (the quantity or intensity of delivered services), and participant responsiveness (PR) (the engagement and receptivity of caregivers receiving the services).^32-34 This multidimensional approach was used to determine the degree to which the HIV/AIDS C&S services were implemented in alignment with the program’s design and objectives

Sample Size Determination and Sampling Procedures

For Exit Interviewee Questionnaire

A single population proportion formula was used to determine the number of caregivers for OVC in order to assess the factors affecting their responsiveness to the program. The following assumptions were made: a target population of 13 234 OVC, a 95% confidence level, a 5% margin of error (to maximize precision), and a P value of 50%. A proportion of 50% was chosen to obtain the maximum sample size.

The sample size was calculated using the formula: $n = \frac{{(Z α /)}^{2} * P (1 - P)}{d^{2}} = \frac{{(1.96)}^{2} (0.5) (0.5)}{{(0.05)}^{2}} = 385$

Considering a 10% nonresponse rate, the final sample size was adjusted to 424 caregivers of OVC.

In Gondar City, ANPPCAN implements HIV/AIDS C&S services across six subcities, each with 1 SDPs: Fasil, Mehal Arada, Jantekel, Zobel, Maraki, and Azezo. From these, 3 SDPs (Fasil, Maraki, and Azezo) were randomly selected as clusters. Within each cluster, the strength of the HIV/AIDS C&S program, SDP readiness, and the competency of service providers were assessed.

The sample size was then proportionally allocated based on the population size of each SDP to ensure representative participation. Caregivers of OVC were selected using a systematic random sampling technique during all working hours throughout the data collection period. These caregivers served as respondents for exit interviews administered through questionnaires.

For Direct Observation

A total of 33 observations were conducted to evaluate the interaction between OVC and their care providers, as well as the clinical competency of healthcare providers (HPs) at 3 selected SDPs. The first OVC was conveniently selected for each observation to assess adherence to guidelines. To minimize the Hawthorne effect, the first 2 observations were excluded, resulting in 30 observations for the OVC–care provider interaction and clinical competency assessment. From these, 24 observations were considered suitable for analysis.

In addition, each SDP was evaluated for facility readiness, contributing an additional 3 observations. In total, 27 observations were used for the evaluation.

This data collection method aimed to assess C&S providers’ adherence to HIV/AIDS service guidelines through nonparticipatory direct observations. Consent was obtained prior to observations, which were conducted while the community HIV/AIDs linkage coordinator provided services to OVC. C&S providers were conveniently selected based on their availability during the data collection period.

For Retrospective Document Review

A total of 385 OVC documents were reviewed as part of the evaluation, covering the period from June 1, 2022 to June 1, 2023 (1 year) retrospectively.

For the key informant interviews, a purposive sampling technique was employed to select participants who could provide relevant and comprehensive information. The interviews included the project manager, an OVC specialist, community social workers, and community HIV/AIDS linkage coordinators.

Resource Inventory

The availability of program resources was assessed over the previous 3 months, focusing on referral slips, Information Education Communication/Behavioral Change Communication materials, registers, HIV diagnostic testing kits, supplies, guidelines, standard operating procedures, and protocols. A resource inventory checklist was used to evaluate these items at the selected SDPs.

For Key Informant Interview

Key informant interviews were conducted to evaluate the general service delivery structure, the training status of service providers, their experience, resource availability, and suggestions for improving HIV/AIDS C&S services for OVC. The key informants included the ANPPCAN project manager, an OVC specialist, community social workers, and community HIV/AIDS linkage coordinators. In total, 5 key informant interviews were conducted.

A purposive sampling technique was employed to select these informants, ensuring that relevant and sufficient information was gathered from individuals with direct knowledge and experience in the field.

Data Collection Tools and Sampling Techniques

This study employed a mixed-methods approach, integrating quantitative and qualitative data collection to comprehensively evaluate HIV/AIDS C&S services for OVC. The rationale for this approach was to capture both measurable service indicators and contextual insights, consistent with established mixed-methods research frameworks [Harvard Catalyst, Mixed Methods Research; https://catalyst.harvard.edu/community-engagement/mmr/]. The reporting of this study conforms to the Good Reporting of a Mixed Methods Study (GRAMMS) statement (O’Cathain et al, 2008) (Supplemental File 1), and the completed GRAMMS checklist is provided as a Supplemental file. Additionally, the questionnaire used in this study has been uploaded as a separate Supplemental file.

Exit Interviewee Questionnaire: Structured questionnaires were developed based on national OVC C&S service guidelines, the Yekokeb Berhan program evaluation report for highly vulnerable children in Ethiopia, and PEPFAR monitoring and evaluation reference guidelines.^5,28,29 The questionnaire comprised sections on sociodemographic characteristics of OVC caregivers, service-related variables (such as counseling received, testing experiences, perceived stigma, and satisfaction with friendly service provision), and facility-related factors (including availability of private counseling spaces and educational materials). Although the tools were not formally validated with statistical tests like Cronbach’s alpha, content validity was ensured through expert consultation and literature review. Pretesting was conducted to assess clarity, relevance, and cultural appropriateness.

Observation Checklist: A structured observation checklist, developed based on national HIV/AIDS C&S guidelines,¹⁰ was used to directly assess providers’ adherence to service protocols during care delivery. The checklist included indicators such as adherence to counseling procedures and confidentiality practices. To maintain consistency, the principal evaluator conducted all observations personally.

Document Review Checklist: A structured checklist guided the review of program documents, including HIV/AIDS C&S service registers, client intake forms, counseling logs, and monthly service reports. These documents were examined for completeness, accuracy, and consistency of recorded service delivery indicators (eg, number of clients served and counseling sessions conducted). A total of 512 registers and reports, covering the period from June 1, 2022 to June 1, 2023, were reviewed consecutively by trained data collectors to provide quantitative evidence on service implementation.

Resource Inventory Checklist: Based on national standard guidelines,¹⁰ the inventory checklist captured the availability of essential HIV/AIDS C&S supplies, reporting tools, logistics, and reference materials in the service facilities. The principal evaluator coordinated with facility staff to verify resource availability for the 3 months preceding data collection.

Key Informant Interviews: Conducted after completing other data collection phases, semistructured interviews with key informants (such as program managers and frontline providers) were guided by an interview protocol with open-ended questions and probing prompts. These explored facilitators and barriers to service delivery and contextual factors influencing program implementation. Consent was obtained for audio recording, and field notes were taken in Amharic. Interviews occurred at the ANPPCAN office and respective SDPs, scheduled according to informants’ availability.

This multimethod data collection approach allowed triangulation of findings to enhance validity and deepen understanding of the program’s implementation.

Data Quality Control

To ensure data quality, the principal evaluator carefully reviewed all questionnaires for completeness and internal consistency before data collection began. The pretesting of tools was conducted with 21 participants, representing 5% of the total sample size. These participants were selected from Mehal Arada subcity, which was not included in the actual study but shares similar demographic and service characteristics with the selected subcities. The purpose of pretesting was to assess clarity, cultural relevance, and comprehension of questions. Based on the pretest findings, minor revisions were made, particularly to the wording of questions to improve clarity and contextual appropriateness.

Prior to data collection, all data collectors received 1 day of training on the purpose and content of the questionnaires, use of Kobo Toolbox software, ethical considerations, and effective communication with respondents. Supervisors were also trained on the data management process, including oversight of data quality. During the actual data collection period, continuous supervision was conducted to ensure accuracy, completeness, and consistency, with real-time corrections provided on site when needed.

Variables and Measurements

Implementation fidelity: This term describes the degree to which HIV/AIDS C&S services for OVC are delivered as originally intended, which is critical for successfully translating evidence-based interventions into practice.¹⁴ In this evaluation, IF was measured across 3 key dimensions: adherence, dosage/exposure, and PR. An overall program implementation score of 75% or higher across these dimensions was considered high fidelity, scores between 50% and less than 75% were classified as medium fidelity, and scores of 50% or below indicated low fidelity.^14,27

Caregiver (participant) responsiveness: Caregiver responsiveness refers to the level of involvement and engagement of OVC and their caregivers in HIV/AIDS C&S services. In this study, it was assessed using 12 indicators, with scores of 75% or higher indicating high responsiveness, scores between 50% and less than 75% indicating medium responsiveness, and scores below 50% denoting low responsiveness.^14,27,32 A weight of 30% was assigned to this dimension following discussions with stakeholders.

PR, measured as the dependent variable, was assessed using 8 binary (yes/no) questions. A mean score of 5.06 was calculated to categorize responsiveness as high or low; caregivers who answered “yes” to more than the mean were classified as having high responsiveness, while those who answered “yes” to fewer than the mean were classified as having low responsiveness.^27,32

The independent variables (predictors) in this study were grouped into 3 main categories. Sociodemographic factors included sex, age, religion, residence, family size, and educational status. Service-related factors encompassed counseling, testing, receipt of test results, experiences of discrimination and stigma, and the provision of attractive and friendly services. Facility-related factors included the availability of separate waiting rooms, private counseling areas, separate entrances, educational posters, and materials to take home.

Adherence: This refers to the extent to which HIV/AIDS C&S services for OVC are delivered according to program guidelines. Adherence was measured using 20 indicators, with scores of 75% or higher indicating high adherence, scores between 50% and less than 75% indicating medium adherence, and scores below 50% indicating low adherence.^14,27,32 A 40% weight was assigned to the adherence dimension after stakeholder discussions.

Dosage/exposure: Dosage refers to the number of sessions or contacts, attendance, and the frequency and duration of participation in HIV/AIDS C&S services. In this study, dosage was measured using 9 indicators, with scores of 75% or higher considered high exposure, scores between 50% and less than 75% considered medium exposure, and scores below 50% considered low exposure.^14,27,32 A 30% weight was assigned to the dosage/exposure dimension after discussions with stakeholders.

Orphan child: An orphan is defined as a child under the age of 18 who has lost 1 or both parents, regardless of the reason for the loss. Orphans can be classified as maternal (having lost their mother), paternal (having lost their father), or double orphans (having lost both parents).^35,36

Vulnerable child: A vulnerable child is defined as a child under the age of 18 whose existence, care, protection, or development may be compromised due to certain conditions. This can include children with a chronically ill parent who has been very sick for 3 or more months, placing them in circumstances that hinder their ability to exercise their rights.^37,38

Data Management and Analysis

Quantitative data were collected using the Kobo Toolbox mobile data collection software. After completion, the data were checked for completeness and consistency, then exported to STATA version 14 for analysis. Data cleaning was conducted by the principal investigator after data exportation to STATA to identify coding errors and missing values. Errors that occurred during data collection were corrected, and the completeness of the data was verified daily. Multiple backups of the data were maintained in secure locations. For analysis, descriptive statistics were used to determine the frequency, mean, and proportions of the variables. Both bivariable and multivariable logistic regression models were employed, with a P value of <.05 and a 95% confidence interval used to identify factors affecting OVC caregiver responsiveness to HIV/AIDS C&S services. The chi-square assumption and model fit were assessed using the Hosmer-Lemeshow test and goodness-of-fit tests. Qualitative data were gathered through field notes and analyzed using thematic analysis. Each response obtained from interviews was coded, thematized, and categorized. The qualitative findings were then presented in narrative form.

Matrix of Analysis and Judgment

The evaluation of IF utilized 2 techniques: an average technique and a multiplicative technique. The average technique involved summing the IF dimension scores to calculate an average, while the multiplicative technique multiplied the percentage scores of the implementation components. Weights for each indicator were assigned by stakeholders before the evaluation, and indicator scores were calculated using the formula: $(Indicator score = \frac{Observed number \times 100}{Indicator weight})$ . The evaluation of IF focused on 3 key dimensions: adherence, dosage/exposure, and PR adherence was assessed using 20 indicators weighted at 40%, dosage/exposure was evaluated with 9 indicators weighted at 30%, and PR was measured through 12 indicators with a weight of 30%. In total, the evaluation encompassed 41 indicators with a cumulative weight of 100%. Scores for each dimension categorized IF into 3 levels: high (≥75%), medium (50%-74%), and low (<50%). This structured approach enabled a comprehensive assessment of the program’s fidelity across its various components, providing insights into effectiveness and identifying areas needing improvement.^13-15,27

Statistical Analysis

Quantitative data were initially collected using Kobo Toolbox mobile data collection software. Following data collection, the data were checked for completeness and consistency and then exported to STATA version 14 for further analysis. Data cleaning was performed by the principal investigator to identify and correct any coding errors or missing values. Descriptive statistics, including frequency distributions, means, and proportions, were calculated to summarize the characteristics of the study participants and key variables.

For inferential analysis, both bivariable and multivariable logistic regression models were applied to identify factors associated with OVC caregiver responsiveness to HIV/AIDS C&S services. Variables with a P value of less than .05 and a 95% confidence interval not crossing one were considered statistically significant. The chi-square assumption was assessed, and model fit was evaluated using the Hosmer-Lemeshow goodness-of-fit test.

Qualitative data were analyzed through thematic analysis. Field notes and interview responses were systematically coded, thematized, and categorized. The final themes were described and presented in narrative form to complement the quantitative findings.

Result

The study involved 415 OVC caregivers whose children had a mean age of 12.75 years. Most of these children were male (61.5%) and lived in urban areas (81.7%). The majority practiced Orthodox Christianity (80%), and nearly two-thirds (64.6%) were attending primary education, while about a third (34.2%) were in secondary education. Only a small proportion (1.2%) were unable to read and write or were just able to read and write. Regarding caregivers, 61.5% reported being involved in decision making about HIV/AIDS care and support services, 68.9% had a high level of overall involvement, and 55.9% served as peer educators.

Among the 24 healthcare and support service providers, most were male (62.5%), with the majority holding Bachelor of Science degrees (83.3%) and a smaller proportion (16.7%) holding diploma-level qualifications.

Additionally, 5 key informants (3 males and 2 females), including program managers, OVC specialists, and social workers, participated in qualitative interviews. All had more than 5 years of experience and relevant professional or managerial roles.

HPs Adherence to HIV/AIDS C&S Services

The adherence of C&S service providers to the national standard OVC C&S service guidelines was evaluated at 3 SDPs through document reviews and direct observations while providers delivered services to OVC.

According to the results from key informant interviews, essential tools such as HIV diagnostic testing kits, referral slip forms, Information, Education, and Communication (IEC)/Behavior Change Communication (BCC) materials, and risk assessment screening tools were available but not consistently found over the past 3 months. One program manager noted:

“Even though we had the HIV diagnostic testing kit, OVC register, referral slip form, IEC/BCC materials, and risk assessment screening tool, which are essential for our C&S services, there is an ‘on and off’ availability of these materials due to budget deficits.” [38-year-old Male, Program Manager]

Each of the 3 SDPs had national standard OVC C&S service guidelines to guide the providers. However, only 1 SDP had essential standard operating procedures (SOPs) in place. Most OVC C&S service providers had received training on HIV/AIDS C&S services and received supportive supervision. This finding aligns with the key informant interviews, as 1 OVC specialist remarked:

“C&S service providers are expected to conduct C&S services based on national standard OVC C&S guidelines, which were printed and donated to us by USAID.” [41-year-old Male, OVC Specialist]

Availability of HIV/AIDS C&S service supplies and equipment presented in Table 1.

Table 1.

Availability of HIV/AIDS Care and Support Service Supplies and Equipment at ANPPCAN, Gondar City, June 2023.

Resource inventory checked by observation (N = 3 SDP)
Sl. No.	Equipment	Yes observation
1	National Standard OVC C&S guideline	3/3
2	Essential SOP	1/3
2	HIV diagnostic testing kit (one step, first response and unigold)	3/3
3	Family HIV risk assessment and HIV status tracking tool	1/3
4	Referral slip form	2/3
5	IEC/BCC materials	2/3
6	Provider initiated testing and counseling register	3/3
7	VCT register	3/3
8	HTS log book	3/3
9	ART regimen register	1/1
10	ART follow-up register	1/1
11	Individual recording form	3/3
12	Staff training register	1/1
13	Review meeting agenda	1/1
14	Health education package register	3/3

Abbreviations: ANPPCAN, African Network for Prevention and Protection Children against Neglect; ART, antiretroviral therapy; C&S, Care and Support; HIV, human immunodeficiency virus; HTS, HIV testing services; IEC/BCC, Information, Education, and Communication/Behavior Change Communication; OVC, orphan and vulnerable children; SDPs, service delivery points; SOP, standard operating procedure; VCT, voluntary counseling and testing.

Characteristics of HIV/AIDS C&S Providers

A total of 24 HPs involved in OVC C&S services across 3 SDPs participated in the study. Among them, 15 (62.5%) were male, and 23 (95.83%) had received pre-service training in HIV/AIDS C&S services. In terms of professional qualifications, 20 (83.33%) held Bachelor of Science degrees, while 4 (16.67%) were clinical nurses with diploma-level education.

Eighteen providers (75%) referenced the National OVC C&S guidelines, and 18 (75%) received supervision from higher officials who utilized the national OVC C&S quality standard checklist. One OVC specialist stated:

“Almost all C&S providers received essential capacity development training to ensure adherence to the national guidelines. Only one SDP had essential standard operating procedures.” [41-year-old Male, OVC Specialist]

A program manager added:

“During the staff recruitment process, we consider the educational background and direct work experience of C&S service providers on OVC projects to ensure they are capable of effectively managing the project. When there are updates to the protocols, staff receive orientation and capacity-building training.” [38-year-old Male, Program Manager] (see Table 2)

Table 2.

Characteristics of the Care and Support Providers, ANPPCAN, Gondar City, June 2023.

Variables	Frequency	Percent
HIV/AIDS C&S provider educational level
Degree	20	83.33
Diploma	4	16.67
Trained on HIV/AIDS C&S
Yes	23	95.83
No	1	4.17
There were means of capacity building training provided to you
Yes	23	95.83
No	1	4.17
You have got supportive supervision
Yes	18	75
No	6	25

Abbreviations: AIDS, acquired immune deficiency syndrome; ANPPCAN, African Network for Prevention and Protection Children against Neglect; C&S, Care and Support; HIV, human immunodeficiency virus.

Document Review

A total of 385 OVC documents enrolled in the program from June 1, 2022, to June 1, 2023, were reviewed. Of these, 310 OVC were referred for HIV testing to the community HIV/AIDS linkage coordinator by social workers. However, subpopulation codes, the HIV status of mothers, and criteria for OVC HIV testing referrals were not fully registered. Among those referred for testing, 7 out of the 310 OVC documents did not have their respective referral slips attached.

Social and community-level workers were responsible for recording each OVC’s age, current HIV status, HIV status of the mother, and date of HIV testing. However, some workers failed to document the subpopulation code, attach referral slips, and note the criteria for OVC HIV testing referrals on the family HIV risk assessment and HIV status tracking tool (see Table 3). This finding contradicts insights from key informant interviews. One social worker emphasized:

“All social and community workers in the organization must document each OVC’s relevant characteristics accurately. If there are any missed variables, we are not seen as diligent workers. To maintain a good reputation in this organization, we always strive for thorough documentation.” [25-year-old Female Social Worker]

Table 3.

Documentation Review of HIV/AIDS Care and Support Service Providers at ANPPCAN, Gondar City, June 2023.

Checked by document review from June 1, 2022 to June 1, 2023
	Reviewed document (N = 385)	Yes	No
1	Age of OVC registered	385	0
2	Sub population code documented	369	16
3	Current HIV status documented	385	0
4	HIV status of mother registered	382	3
5	Criteria of OVC for HIV testing referral registered	371	14
6	Referrals for HIV testing attached	303	7
7	Date of referral accepted documented	310	0
8	Date of HIV testing documented	310	0
9	Date HIV test obtained registered	300	0
10	Final HIV status Documented	310	0

Abbreviations: AIDS, acquired immune deficiency syndrome; ANPPCAN, African Network for Prevention and Protection Children against Neglect; C&S, Care and Support; HIV, human immunodeficiency virus; OVC, orphan and vulnerable children.

Nonparticipatory Direct Observation

A total of 27 nonparticipatory direct observations were conducted to assess the interaction between OVC and C&S service providers, the readiness of SDPs, and the clinical competencies of C&S providers. The observation results indicated that 57.5% of C&S providers demonstrated good interaction with OVC while delivering services, and 80% exhibited strong clinical competence during service delivery. Additionally, each SDP met 72.2% of the standards necessary for effectively implementing the HIV/AIDS C&S service program. See the Table 4 for further details.

Table 4.

Nonparticipatory Observation of OVC and Care Provider Interaction at ANPPCAN, Gondar City, June 2023 (N = 27).

Observation of HIV/AIDS care and support service provider skills and procedures (N = 10)
Sl. no.	Skills and techniques	Yes, for Items observed Counseling required skill & Techniques	Frequency	Percentage
1	Establish a relationship	Greet the client (shake hands if appropriate)	8	80
		Offer the client a seat	7	70
		Introduce herself/himself to the client	0	0
		Lean forward when talking	5	50
		Make eye contact when talking to the client	9	90
		Show interest in the client	7	70
		Maintain professional conduct	4	40
2	Question	Ask questions relevant to the topic of discussion	10	100
		Ask open-ended questions	10	100
		Use closed-ended questions to get basic information such as demographic Data	10	100
		Use a questioning style that reflects interest, concern and care rather than Interrogation	8	80
3	Listen well	Look at the client	8	80
		Listen carefully and actively	7	70
		Use body language to indicate attention to the speaker	4	40
		Make eye contact to indicate interest and care	7	70
		Treat the client with respect and acceptance	10	100
		Use encouraging words such as “Yes” and “Okay”	8	80
		Occasionally sum up the client’s statements	2	20
		Notice the client’s verbal and nonverbal cues	5	50
		Wait after asking questions to allow the client to formulate responses or Questions	5	50
4	Empathies	Recognize and praise what the client is doing correctly	0	0
		Reflect the client’s statements to show she/he was understood	0	0
		Accept what the client thinks and feels	10	100
5	Provide information	Clearly communicate important HIV/AIDS related information based on the client’s:
		Knowledge, cultural values and beliefs	8	80
		Use simple language	6	60
		Give a little relevant information at a time	10	100
		Make 1 or 2 suggestions without giving commands	0	0
6	Clarify	Check what the client said to ensure correct understanding	0	0
		Use phrases such as, “Did I understand you correctly when you said	5	50
		Avoid words that sound judgmental	6	60
7	Find solutions	Suggest acceptable, affordable and feasible options	8	80
		Help the client find practical and realistic solutions	7	70
		Convince the client to implement solutions	7	70
		Help the client verbalize what other people may say about the suggested Solutions	7	70
8	Summarize	Summarize the information the client has shared	0	0
8	Summarize	Check whether the client understood the important concerns or information	2	20
		Praise and reaffirm things the clients are doing right	0	0
9	Follow up	Discuss appropriate follow up with the client?	10	100
9	Follow up	Encourage the client to adhere to the follow-up plan	10	100
Total			230	57.5
Observation to assess the SDPs readiness at ANPPCAN (N = 3)
	Sl. no.	Yes, for Items observed (n = 3)	Frequency	Percent
	1	Counseling area kept visual privacy	3	100
	2	The examination room kept auditory privacy	2	66.7
	3	Separate entrance to ensure OVC privacy	0	0
	4	Transparent and confidential system to submit OVCs’ comments	1	33.3
	5	Separate waiting room for OVC	3	100
	6	Adequate waiting room for OVC (open door, window, ventilation)	2	66.7
	7	Educational posters displayed	3	100
	8	Posters that describe clients’ rights	3	100
	9	Materials for OVC clients to take home	2	66.7
	10	Service provision attractive and friendly to OVC	2	66.7
	11	System in place to provide continuous support to OVC C&S services staff	3	100
	12	Is there Clear, written guidelines or SOPs existing in the OVC C&S services	2	66.7
	Total		26	72.2
	Observation to assess clinical competencies of C&S providers at ANPPCAN (N = 10)
		Yes, for Items observed (n = 10)	Frequency	Percent
	History taking	Does the health worker:
		Tell the client that he/she is going to ask some personal questions	7	70
		Reassure the client that all information provided will be kept confidential	9	90
		Give the client the time to explain why he/she thinks he/she could be infected with/ have HIV	10	10
		Ask the client if he/she has had unprotected sex in the last 72 h?	0	0
		Ask about symptoms associated with HIV infection?	10	100
		Ask about factors which increase the likelihood of infection with HIV?	10	100
		Ask about symptoms of STI syndromes?	8	80
	Assessment	Tell the patient that he is going examine him/her now?	10	100
		Check for signs associated with HIV infections?	6	60
		Check for signs of STI syndromes?	7	70
	Classification	Correctly classify the OVC as having possible HIV infection, at risk of HIV infection or unlikely to have HIV infection?	10	100
	Classification	Explain the classification?	8	80
	Management	Does the health worker:
	Management	Appropriately manage the OVC based on classification?	10	100
	Counseling and follow up	For a client offered an HIV test: Does the health worker:
		Appropriately explain	10	100
		Check the OVC understands?	5	50
	Total		120	80

Observation on HIV/AIDS C&S service program implementation in Figure 2.

Figure 2.

Observation on HIV/AIDS C&S Service Program Implementation at ANPPCAN, Gondar City, June 2023. Abbreviations: AIDS, Acquired Immune Deficiency Syndrome; ANPPCAN, African Network for Prevention and Protection Children Against Neglect; C&S, Care and Support; HIV, Human Immunodeficiency Virus.

The overall measurement of the adherence dimension was 81.5%, rated as high based on 20 indicators. This indicates that the HIV/AIDS C&S service providers at ANPPCAN, Gondar City, demonstrated strong compliance with established guidelines and practices. Out of a total of 155 indicators assessed, 135 met the required standards. The score of 81.5% reflects the effectiveness of the training and implementation of protocols aimed at enhancing the quality of care for OVC (see Table 5).

Table 5.

Summary of Adherence Dimension Indicators at ANPPCAN, Gondar City, June 2023.

Sl. no.	Indicators	E	O	W	S	Ach (%)	Judgment
1	Percentage of care provider who trained in OVC C&S service	24	23	2.5	2.4	96	High
2	Number of community social workers trained on comprehensive OVC services	14	13	2.5	2.3	92	High
3	Proportion of care providers trained on ART adherence counseling	24	22	2.5	2.3	92	High
4	Number of healthcare provides (HCP) who check OVC for signs associated with HIV infections	10	6	3.5	2.1	60	Medium
5	Number of HCP who correctly classify the OVC based on risk assessment	10	10	2.5	2.5	100	High
6	Number of HCP who explain the classification of OVC based on risk assessment	10	8	2.5	2	80	High
7	Number of HCP who appropriately manage the OVC based on classification	10	10	3.5	3.5	100	High
8	Number of HCP who appropriately explain to OVC/caregivers what an HIV test is	10	10	2.5	2.5	100	High
9	Number of HCP who tells that care and support services would be provided in case the test is positive	10	10	1.5	1.5	100	High
10	Proportion of SDPs whose counseling area kept visual privacy	3	3	1.5	1.5	100	High
11	Proportion of SDP whose examination room kept auditory privacy	3	2	1.5	1	66.7	High
12	Proportion of SDP having a separate entrance to ensure OVC privacy	3	0	1.5	0	0	Low
13	Proportion of SDP with a transparent and confidential system to submit OVCs’ comments	3	2	1.5	1	66.7	High
14	Proportion of SDP with a separate waiting room for OVC	3	3	1.5	1.5	100	High
15	Proportion of SDP with adequate waiting room for OVC	3	2	1.5	1	66.7	High
16	Proportion of SDP with educational posters displayed	3	3	1.5	1.5	100	High
17	Proportion of SDP with posters that describe clients’ rights	3	3	1.5	1.5	100	High
18	Proportion of SDP with materials for OVC clients to take home	3	2	1.5	1	66.7	High
19	Proportion of SDP having with standard operating procedure	3	1	1.5	0.5	33.3	Low
20	Proportion of SDP having OVC care and support standard guide lines	3	2	1.5	1	66.7	High
	Overall	155	135	40	32.6	81.5	High Adherence

Note: W = weighted, E = expected, O = observed, score = (observed *weight/expected), achieved (%) = score/weight*100 and judgment parameter ≥75%=high, 50% −74%=medium, < 50=low.

Abbreviations: ANPPCAN, African Network for Prevention and Protection Children against Neglect; ART, antiretroviral therapy; C&S, Care and Support; OVC, orphan and vulnerable children; SDPs, service delivery points.

Dosage/Exposure to HIV/AIDS C&S Services

The overall exposure assessment for HIV/AIDS C&S service providers at ANPPCAN, Gondar City, shows a low exposure rating. This evaluation, based on 9 indicators, yielded a total score of 30 out of 6031, resulting in an average score of 13.58. This low exposure rating indicates significant gaps in the timely initiation of HIV testing and antiretroviral therapy (ART) for OVC (Table 6).

Table 6.

Summary of Dosage/Exposure Indicators in HIV/AIDS C&S Service in ANPPCAN, Gondar City, June 2023.

Sl. no.	Indicators	E	W	O	S	Achi (%)	Judgment
1	Frequency of OVC aged under 13 years old counseled for HIV test per year	112	2.7	101	2.4	88.9	High
2	Frequency of OVC aged 14-18 clients counseled for HIV test per year	792	2.7	396	1.4	51.9	Medium
3	Frequency of OVC aged under 13 years old clients tested for HIV per year	112	2.7	101	2.4	88.9	High
4	Frequency of OVC aged 14-17 clients tested for HIV per year	792	2.7	594	2.03	75.1	Medium
5	percentage of OVC clients who are being seen for clinical follow-up visits every quarter	1024	2.7	987	2.59	95.9	High
6	percentage of OVC clients who pick up their ART drugs every quarter	869	2.7	867	2.69	99.6	High
7	Duration of time to start HIV test for OVC after HIV risk assessment done	310days	2.7	-1860Days	-16.2	-600	Low
8	Duration of time to start ART after confirmation test conducted	20days	2.7	-120Days	-16.2	-60	Low
9	Percentage of OVC ever tested for HIV	5434	2.7	4965	2.47	91.4	High
	Overall		30	6031	13.58	45	Low exposure

Notes: W = weighted, E = expected, O = observed, score = (observed *weight/expected), achieved (%) = score/weight*100 and judgment parameter ≥75%=high, 50%-74%= medium, < 50=low.

Abbreviations: AIDS, acquired immune deficiency syndrome; ANPPCAN, African Network for Prevention and Protection Children against Neglect; ART, antiretroviral therapy; C&S, Care and Support; HIV, human immunodeficiency virus; OVC, orphan and vulnerable children; SDPs, service delivery points.

Key informants supported this finding, stating:

“Most OVC caregivers do not properly attend their OVC’s follow-up visits. This is because many caregivers need cash or incentives beyond the services provided. When they do not receive these incentives, they lose track of follow-ups and only restart after being traced by community case workers.” [41 years old, Male, OVC Specialist]

From the 310 OVC referred for HIV testing, 112 were under age 13, expected to be tested for HIV once per year, while the remaining 198 OVC, aged above 13, were expected to be tested quarterly.

Caregivers of OVC Responsiveness to HIV/AIDS C&S Services

Sociodemographic Characteristics of OVC

Of the total 424 OVC caregivers, 415 (97.8%) completed the self-administered exit interview. The mean age of the OVC was 12.75 years, with 255 (61.45%) being male. In terms of residence, 339 (81.69%) lived in urban areas, and 332 (80%) identified as Orthodox Christians. Regarding educational status, 268 (64.58%) were attending primary education (Table 7).

Table 7.

Sociodemographic Characteristics of OVC at ANPPCAN, Gondar City, June 2023.

Variables		Frequency	Percentage
Sex	Male	255	61.45
	Female	160	38.55
Religion	Orthodox	332	80
	Muslim	78	18.8
	Other (Protestant and Catholic)	5	1.2
Residence	Urban	339	81.69
Residence	Rural	76	18.31
Work for money	Yes	227	54.70
Work for money	No	188	45.30
Educational status	Primary education	268	64.58
	Secondary education	142	34.22
	Un able to read and
	write	2	0.48
	Able to read and write	3	0.72

Abbreviations: ANPPCAN, African Network for Prevention and Protection Children against Neglect; OVC, orphan and vulnerable children.

Caregivers’ Level of Engagement to HIV/AIDS C&S Services

Two hundred fifty-five respondents (61.45%) were involved in decision making regarding HIV/AIDS C&S services, while 286 (68.92%) demonstrated a high overall level of engagement in providing these services. Additionally, 232 respondents (55.9%) served as peer educators in HIV/AIDS C&S services, and 292 (70.36%) were involved in the design and delivery of these services (Table 8).

Table 8.

OVC’s Caregiver Level of Engagement in the HIV/AIDS C&S Service, ANPPCAN, Gondar City, June 2023.

	Level of engagement (N = 415)
	High	Low
Engagement items	Frequency (%)	Frequency (%)
You were involved in decision making regarding the HIV/AIDS C&S services	255(61.45)	160(38.55)
Your overall level of involvement in the provision of HIV/AIDS C&S services was high	286(68.92)	129(31.08)
You have been involved as a peer educator in HIV/AIDS C&S services	232(55.9)	183(44.1)
You know any peer educator trained in HIV/AIDS C&S services	261(62.89)	154(37.11)
You are involved in decisions about how HIV/AIDS C&S services should be delivered to OVC	234(56.39)	181(43.61)
You are aware of OVC who are involved in decisions about how HIV/AIDS C&S services should be delivered to OVC	247(59.52)	168(40.48)
Community involvement in HIV/AIDS C&S services design, monitoring and evaluation was high	295(71.08)	120(28.92)
Involvements in HIV/AIDS C&S services service design and delivery were high	292(70.36)	123(29.64)

Note: High involvement is when OVC/caregivers/community participated in HIV/AIDS C&S service at least 3 times per year.

This finding aligns with insights from key informants, who noted:

“Participation from the community, particularly members of the community coalition committee selected from local administration and stakeholders, has helped ANPPCAN identify target beneficiaries and deliver appropriate services. Community engagement in implementing the project at the local level has fostered a sense of ownership, trust, and belonging, ensuring the sustainability of C&S services.” [38 years old, Male, Program Manager]

Overall, 283 OVC caregivers (68%) were highly responsive to HIV/AIDS C&S services, while the remaining 132 caregivers (32%) showed low responsiveness to these services. This data is illustrated in Figure 3.

Figure 3.

OVC’s Caregiver Overall Responsiveness to HIV/AIDS C&S Service in ANPPCAN, Gondar City, June 2023. Abbreviations: AIDS, Acquired Immune Deficiency Syndrome; ANPPCAN, African Network for Prevention and Protection Children Against Neglect; C&S, Care and Support; HIV, Human Immunodeficiency Virus; OVC, Orphan and Vulnerable Children.

Factors Affecting Caregiver’s Responsiveness

In total, 24 variables were considered for inclusion in the model. In the bivariable logistic regression, candidate variables for the multivariable analysis included religion, residence, whether children were asked about their problems, counseling area privacy, the availability of a separate waiting room, adequacy of the waiting room, educational posters describing clients’ rights, materials for OVC clients to take home, attractiveness and friendliness of services, continuous support for OVC C&S services, and the presence of clear, written guidelines or SOPs, all with a P value < .25.

In the multivariable analysis, the presence of a separate waiting room, the availability of materials for OVC clients to take home, and the attractiveness and friendliness of the services were significantly associated with OVC caregivers’ responsiveness to HIV/AIDS C&S services (P < .05). Specifically, caregivers served in SDPs without a separate waiting room were 95% less likely to be responsive compared to those with a separate waiting room (adjusted odds ratio [AOR] = 0.05, 95% confidence interval [CI]: 0.005-0.52). Caregivers who received readable materials to take home were 1.76 times more likely to be responsive (AOR = 1.76, 95% CI: 1.67-7.59), and those who experienced attractive and friendly services were 2.24 times more likely to be responsive compared to those who did not (AOR = 2.24, 95% CI: 1.08-4.64; see Table 9).

Table 9.

Bivariable and Multivariable Analysis Result of OVC’s Caregiver’s Responsiveness to HIV/AIDS C&S Services in ANPPCAN, Gondar City, June 2023 (N = 415).

Variables	PR		OR
	High PR (%)	Low PR (%)	COR (95% CI)	P value	AOR (95% CI)	P value
Religion
Orthodox	221(66.6)	111(33.4)	1		1
Muslim	59(75.6)	19(24.4)	1.55(0.88,2.74)	.12	1.62 (0.9,2.91)	.11
Other	3(60)	2(40)	0.753(0.12,4.57)	.76	0.84(0.13,5.35)	.86
Residence
Urban	236(69.6)	103(30.4)	1		1
Rural	47(61.8)	29(38.2)	0.7(0.42, 1.18)	.19	0.76 (0.44,1.33)	.34
Asked OVC problem
Yes	200(66.2)	102(33.8)	0.7(0.43, 1.14)	.16	0.86 (0.51,1.45)	.58
No	83(73.5)	30(26.5)	1		1
Counseling area visual privacy
Yes	197(66.3)	100(33.7)	0.73(0.45, 1.17)	.19	0.73(0.44, 1.2)	.22
No	86(72.9)	32(27.1)	1		1
Waiting room
Yes	273(68.8)	124(31.2)	1		1
No	10(55.6)	8(44.4)	0.57(0.21, 1.47)	.23	0.05(0.01,0.52)	.012**
Materials to take-home
Yes	244(66.5)	123(33.5)	1		1
No	39(81.2)	9(18.8)	2.18(1.02, 4.65)	.04	1.76(1.67,7.59)	.014**
Posted clients rights
Yes	223(66.8)	111(33.2)	0.66(0.38, 1.15)	.15	1.17(0.69,1.98)	.55
No	60(75)	20(25)	1		1
Adequate waiting room
Yes	204(70.3)	86(29.7)	1.38(0.88, 2.14)	.15	1.3(0.82,2.09)	.26
No	79(63.2)	46(36.8)	1		1
Friendly service
Yes	266(70)	114(30)	2.47(1.23, 4.97)	.01	2.24 (1.08, 4.64)	.03**
No	17(44.7)	18(55.3)	1		1
Support services
Yes	205(65.5)	108(34.5)	0.58(0.35, 0.98)	.04	0.69(0.4, 1.2)	.29
No	78(76.5)	24(23.5)	1		1
Guidelines existing
Yes	274(67.7)	131(32.3)	0.23(0.03, 1.85)	.17	0.46(0.05,4.21)	.49
No	9(90)	1(10)	1		1

Note: 1 = reference, I = confidence interval.

Abbreviations: AIDS, acquired immune deficiency syndrome; ANPPCAN, African Network for Prevention and Protection Children against Neglect; AOR, adjusted odds ratio; C&S, Care and Support; COR, crude odds ratio; HIV, human immunodeficiency virus; OR, odds ratio; OVC, orphan and vulnerable children; PR, participant responsiveness; SDPs, service delivery points.

The overall measurement of OVC caregivers’ responsiveness to the HIV/AIDS C&S program was 60.6%, categorized as medium, based on 12 indicators. PR was assessed through exit interviews and document reviews (Table 10).

Table 10.

Summary of PR Indicators in ANPPCAN, Gondar City, June 2023.

Sl. no.	Indicators	E	O	W	S	Achi (%).	Judgment
1	Proportion of OVC perceived community involvement in HIV/AIDS C&S service is high	415	295	2.5	1.8	72	Medium
2	Percentage of OVC living with HIV who know their status	987	869	2.5	2.2	88	High
3	Percentage of OVC’s caregivers participated on enhanced adherence counseling	250	135	2.5	1.4	56	Medium
4	Percentage of OVC’s caregiver accepting to pick up ART for their OVC living with HIV	987	985	2.5	2.49	99.6	High
5	proportion of OVC’s caregiver with high involvement to HIV/AIDS C&S services	415	286	2.5	1.7	68	Medium
6	Number of OVC on ART who experienced an interruption in treatment during any previous reporting period successfully restarted ART within the reporting period	4	3	2.5	1.9	76	High
7	proportion of OVC’s caregiver with high involvement in C&S service design	415	292	2.5	1.8	72	Medium
8	Number of newly diagnosed HIV-positive OVC who received testing for recent infection	11	4	2.5	0.9	36	Low
9	Number of OVC’s caregiver who know peer educator trained on HIV/AIDS C&S services	415	261	2.5	1.6	64	Medium
10	Number of OVC’s caregiver participated to VCT sensitization meeting	415	413	2.5	2.48	99.2	High
11	Number of OVC’s caregiver involved in decision-making regarding the HIV/AIDS C&S services	415	255	2.5	1.5	60	Medium
12	Number of OVC’s caregiver involved as a peer educator in C&S services	415	232	2.5	1.4	56	Medium
	Over all	5144	4030	30	21.17	70.6	Medium PR

Notes: W = weighted, E = expected, O = observed, score = (observed *weight/expected), achieved (%) = score/weight*100 and judgment parameter ≥75%= high, 50% -74%= medium, < 50= low.

The overall IF of the HIV/AIDS C&S service was 67.87%, indicating a medium level of implementation that requires improvement (Table 11).

Table 11.

Summary of Overall Performance Implementation of HIV/AIDS C&S Services for OVC in ANPPCAN, Gondar City, June 2023.

Sl. no.	Dimensions	Weight (W)	Score (S)	Achv.in % (S/W*100)	Judgment parameter
1	Adherence	40	32.6	81.5	High
2	Dosage/exposure	30	13.58	45	Low
3	PR	30	21.17	70.6	Medium
4	Overall HIV/AIDS C&S services IF level	100	67.87	67.35	50–74 = Medium IF (needs improvement)

Discussion

The IF of the HIV/AIDS C&S program was assessed across 3 dimensions: adherence, dosage/exposure, and PR. Overall, the implementation of HIV/AIDS C&S services was rated at 67.35%, indicating a medium level that suggests the program requires improvement.

The adherence of HPs to national HIV/AIDS C&S guidelines was high at 81.5%. This indicates that a majority of providers effectively follow the established protocols when delivering services to OVC. High adherence signifies that training and resources are effectively utilized, allowing providers to meet established standards. This is crucial for ensuring that OVC receive the necessary care aligned with national guidelines.

This finding aligns with other studies conducted in Ethiopia, which reported adherence levels ranging from 70% to 90% in similar contexts.³⁹ However, it remains lower than evaluations by the United States Department of Defense, which reported adherence rates of 100%.⁴⁰ Differences in adherence rates could be attributed to variations in healthcare infrastructures, availability of resources, and the economic context of the regions studied. In Ethiopia, factors such as limited training opportunities and resource constraints may impede adherence in some settings.^10,41

Maintaining high adherence is vital for the program’s success. Continuous training, supportive supervision, and monitoring are essential to sustain this level of adherence and ensure that all HPs are consistently following best practices.⁴²

The dosage or exposure to HIV/AIDS C&S services was notably low at 45%. This indicates that many OVC did not receive timely and necessary interventions. A low exposure score suggests significant gaps in service delivery, particularly in initiating and following up on necessary treatments such as HIV testing and ART. This can negatively impact health outcomes for OVC. The low dosage rating aligns with findings from the Integrated AIDS Program in Kenya, which reported that only 26% of OVC accessed necessary services.⁴¹ This emphasizes a common challenge in the region where service delivery remains inconsistent.

Factors contributing to low exposure may include economic barriers faced by families, lack of transportation, and insufficient incentives for caregivers to seek follow-up services. Additionally, cultural beliefs and stigma surrounding HIV may deter caregivers from utilizing available services.⁴³

Addressing the low exposure requires targeted interventions such as community outreach programs, financial incentives for caregivers, and enhanced follow-up mechanisms to ensure that OVC receive essential services promptly.⁴⁴

The responsiveness of OVC caregivers to HIV/AIDS C&S services was rated as medium at 70.6%. This indicates a moderate level of involvement from caregivers in the care process. Medium responsiveness suggests that while caregivers are engaged, there remains significant room for improvement. Factors influencing responsiveness include the availability of separate waiting rooms and the provision of materials for caregivers to take home. This finding is consistent with studies indicating that caregiver engagement is often moderate, with varying levels of involvement reported across different regions.^45,46 A study in South Africa found that caregiver participation significantly impacts service utilization.⁴⁷ Variations in responsiveness can stem from different community engagement strategies, cultural attitudes toward healthcare, and the availability of supportive resources. In Gondar, the lack of effective communication and motivation from service providers may also contribute to lower engagement levels. Improving caregiver responsiveness is crucial for enhancing the overall effectiveness of HIV/AIDS C&S services. Strategies should focus on creating a welcoming environment, enhancing communication, and providing educational resources that empower caregivers to engage more fully in the care process.⁴⁸

Caregivers who access HIV/AIDS C&S services for their OVC at SDPs without a separate waiting room are 95% less likely to be responsive compared to those at SDPs with a separate waiting room (AOR=0.05, 95% CI: 0.005, 0.52). This finding suggests that caregivers receiving HIV/AIDS C&S services in SDPs with a separate waiting room are 5% more likely to be highly responsive to the services they receive. This aligns with studies conducted in the sub-Saharan Africa, which indicated that OVC served in health facilities with separate waiting rooms demonstrated better retention among HIV-infected individuals.^49,50 Furthermore, this finding is consistent with the WHO quality assessment standards,^51,52 which state that the presence of a separate waiting room for C&S services enhances service delivery and improves responsiveness.

One reason for this increased responsiveness could be that waiting rooms for OVC and their caregivers are typically equipped with educational materials, such as visual aids, audio resources, leaflets, televisions, posters, and peer educators. This environment provides OVC and caregivers with better information about the available HIV/AIDS C&S services and guidance on how to approach service providers.^48,50 Additionally, separate waiting rooms offer visual and auditory privacy from adult clients, which may enhance the confidence of OVC and caregivers when seeking HIV/AIDS C&S services, ultimately contributing to higher responsiveness.^5-7 Moreover, the odds of OVC caregivers being responsive to HIV/AIDS C&S services are 2.24 times higher when they receive attractive and friendly service compared to those who do not (AOR=2.24, 95% CI: 1.08, 4.64). When OVC caregivers receive appealing and friendly service, they are likely to feel more motivated and satisfied with the care provided, which encourages them to visit the SDP more frequently for their children’s needs.

Additionally, the odds of OVC caregivers being responsive to HIV/AIDS C&S services increase by 1.76 times when they take readable materials home compared to those who do not (AOR=1.76, 95% CI: 1.67, 7.59). Key informants have highlighted the necessity of providing IEC/BCC readable materials to enhance HIV/AIDS knowledge for each OVC and their caregivers. This access to information directly impacts caregiver participation in C&S services, as it helps them understand what types of services are available at each SDP.

In summary, when OVC and their caregivers take readable materials home, their attitudes and awareness toward the services improve, enabling them to access the necessary services without interruption.

Strength and Limitations of Study

The study employed a triangulated qualitative design to explore the underlying reasons for the medium level of fidelity in the implementation of HIV/AIDS C&S services. It assessed fidelity from the perspectives of both OVC caregivers and C&S providers, ensuring that the providers’ viewpoints and direct observations were taken into account. Consequently, fidelity items required to be completed by the providers were included in the evaluation.

One challenge encountered was the potential for biased behavioral responses during observations of client–provider interactions, known as the Hawthorne effect. To mitigate this bias, the first and last 2 observations were excluded from the analysis. Another limitation of the study was that during document reviews, activities that had not been completed may have been incorrectly recorded as having been accomplished, potentially leading to inflated results. This issue was addressed by incorporating direct observations into the evaluation process. Additionally, it should be noted that this evaluation did not encompass all dimensions of implementation fidelity.

Conclusion

The IF of HIV/AIDS C&S services for OVC in ANPPCAN, Gondar City, was assessed as medium, indicating a need for improvement based on the agreed judgment criteria. The evaluation focused on 3 dimensions: adherence, PR, and dosage/exposure. Adherence to the national standard OVC C&S guidelines during the provision of HIV-related services was notably strong. Specifically, 57.5% of C&S providers demonstrated good interaction with OVC, while 80% exhibited good clinical competence in delivering these services. Additionally, SDPs fulfilled 72.2% of the established standards for implementing HIV/AIDS C&S services effectively. Training for C&S providers was conducted according to the established training plans; however, documentation practices by community social workers were found to be relatively poor.

In terms of dosage/exposure, the program was judged to be low because not all OVC caregivers received essential interventions such as counseling, ART pickup, testing, and follow up according to the established protocols.

The responsiveness of OVC caregivers to HIV/AIDS C&S services was rated as medium, reflecting their level of involvement from the program’s initiation to decision-making processes. Factors influencing caregiver responsiveness included the availability of a separate waiting room, attractive and friendly service, and take-home materials.

This study provides new evidence regarding the actual performance of the program concerning the IF of HIV/AIDS C&S services for OVC, which can serve as a foundation for strengthening these services. It is recommended that rigorous efforts be made to establish systems that align with the policies and principles of IF for HIV C&S services within nongovernmental organizations. Consequently, the government, partners, program managers, and implementers should prioritize improving the IF of these services.

Supplemental Material

sj-docx-1-jia-10.1177_23259582251372458 - Supplemental material for Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City, Northwest Ethiopia: A Mixed-Methods Evaluation

Supplemental material, sj-docx-1-jia-10.1177_23259582251372458 for Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City, Northwest Ethiopia: A Mixed-Methods Evaluation by Samuel Abebayehu and Endalkachew Mesfin Gebeyehu in Journal of the International Association of Providers of AIDS Care (JIAPAC)

Supplemental Material

sj-docx-2-jia-10.1177_23259582251372458 - Supplemental material for Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City, Northwest Ethiopia: A Mixed-Methods Evaluation

Supplemental material, sj-docx-2-jia-10.1177_23259582251372458 for Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City, Northwest Ethiopia: A Mixed-Methods Evaluation by Samuel Abebayehu and Endalkachew Mesfin Gebeyehu in Journal of the International Association of Providers of AIDS Care (JIAPAC)

Footnotes

Abbreviations

Acknowledgments

The authors would like to acknowledge the University of Gondar, the College of Medicine and Health Sciences, and the Institute of Public Health for their support and facilitation of this study. They also extend their gratitude to the data collectors, supervisors, and staff from the African Network for Prevention and Protection of Children Against Neglect for their cooperation and for providing the necessary documents and information that contributed to this work.

ORCID iD

Endalkachew Mesfin Gebeyehu

Ethics Approval and Consent to Participate

Ethical clearance for this study was obtained from the University of Gondar’s Institute of Public Health and the Ethical Review Committee, referenced under Ref No/IPH/2254/2014. The study adhered to ethical guidelines for research involving human subjects, ensuring the protection of participants’ rights and welfare. Official permission for data collection was granted by ANPPCAN and local authorities. Informed written consent was secured from each study participant. Given that the target group for this evaluation comprised individuals under 18 years of age, both assent from the minors and parental consent were obtained. Confidentiality was prioritized throughout the research process, ensuring anonymity and privacy to protect respondents’ rights. Participants were informed of their right to withdraw from the study at any time without any risk of harm. These ethical considerations were essential to uphold the integrity of the research and safeguard the welfare of all participants involved.

Authors’ Contributions

SA contributed to the conceptualization of the study, the design of the research methodology, data collection, and analysis. He was also involved in developing the research framework, interpreting the data, and synthesizing the findings. EMG played a key role in drafting and revising the manuscript, ensuring the accuracy of the data presented, and integrating feedback from co-authors. He significantly contributed to writing the discussion section and provided critical insights based on his extensive field experience in HIV/AIDS care and support services. Additionally, he reviewed the manuscript for clarity and coherence.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Availability of Data and Material

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Supplemental Material

Supplemental material for this article is available online.

References

Mahy

Sabin

Feizzadeh

Wanyeki

. Progress towards 2020 global HIV impact and treatment targets. J Int AIDS Soc. 2021;24:e25779.

Unicef . Special COVID-: UNICEF child protection update. 2021.

Carpenter

. USAID’s assured assistance: USAID’s humanitarian aid in Latin America and the Caribbean 2001-2019. Rev Int Cooper Desarrollo. 2020;7(2):163–176.

Mirkuzie

Ali

Abate

Worku

Misganaw

. Progress towards the 2020 fast track HIV/AIDS reduction targets across ages in Ethiopia as compared to neighboring countries using global burden of diseases 2017 data. BMC Public Health. 2021;2021(21):1–10.

Csace I . Ethiopia demographic and health survey 2016. Addis Ababa, Ethiopia, and Rockville. CSA and ICF; 2016, :1-551.

Mezgebo

. Urbanization and development in Ethiopia: policy issues, trends and prospects. State Ethiop Econ. 2020;2021:15–34.

Mulugeta

Baynesagn

. The development of child protection systems and practice in Ethiopia. In: The Development of Child Protection Systems and Practice in Low-to Middle-Income Countries, 2025, p. 101-19. Policy Press.

Zewude

Tadele

Engdawork

Assefa

. Exploring targeted policies and institutional frameworks to address the problem of child streetism in Ethiopia. Eth J Pediatr Child Health. 2024;19(2):193–218.

Bryant

Beard

Sabin

, et al. PEPFAR’s support for orphans and vulnerable children: some beneficial effects, but too little data, and programs spread thin. Health Aff. 2012;31(7):1508–1518.

10.

Organization WH . Consolidated guidelines on HIV prevention, testing, treatment, service delivery and monitoring: recommendations for a public health approach: World Health Organization; 2021.

11.

Lwandiko

. Children in orphanage centers: a call to serve. Available at SSRN 4594087. 2023.

12.

MENSAH

. Protection of children in the African Human Rights System: Framework and Institutions. IN Regional human rights systems.237.

13.

Carroll

Patterson

Wood

Booth

Rick

Balain

. A conceptual framework for implementation fidelity. Implement Sci. 2007;2:1–9.

14.

Century

Rudnick

Freeman

. A framework for measuring fidelity of implementation: a foundation for shared language and accumulation of knowledge. Am J Eval. 2010;31(2):199–218.

15.

Dusenbury

Brannigan

Falco

Hansen

. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ Res. 2003;18(2):237–256.

16.

Proctor

Silmere

Raghavan

, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administ Policy Mental Health Mental Health Serv Res. 2011;38:65–76.

17.

Ripamonti

Leon

. World AIDS day 2022: putting ourselves to the test: achieving equity to end HIV. Public Health Front. 2024;12:1478645.

18.

Jones

Taylor

Tieu

H-V

Wilkin

. CROI 2017: advances in antiretroviral therapy. Top Antivir Med. 2017:25(2):51.

19.

Dziva Chikwari

Simms

Busza

, et al. Community health worker support to improve HIV treatment outcomes for older children and adolescents in Zimbabwe: a process evaluation of the ZENITH trial. Implement Sci. 2018;13:1–9.

20.

Afrane

Goka

Renner

, et al. HIV virological non-suppression and its associated factors in children on antiretroviral therapy at a major treatment centre in southern Ghana: a cross-sectional study. BMC Infect Dis. 2021;21:1–11.

21.

Mathenjwa

Kim

H-Y

Zuma

, et al. Home-based intervention to test and start (HITS) protocol: a cluster-randomized controlled trial to reduce HIV-related mortality in men and HIV incidence in women through increased coverage of HIV treatment. BMC Public Health. 2019;19(1):969.

22.

Koris

Stewart

Ritchwood

, et al. Youth-friendly HIV self-testing: acceptability of campus-based oral HIV self-testing among young adult students in Zimbabwe. PLoS ONE. 2021;16(6):e0253745-e.

23.

Govindasamy

Meghij

Negussi

Baggaley

Ford

Kranzer

. Interventions to improve or facilitate linkage to or retention in pre-ART (HIV) care and initiation of ART in low- and middle-income settings—a systematic review. J Int AIDS Soc. 2014;17(1):19032–19.

24.

Tokar

Osborne

Hengeveld

Lazarus

Broerse

JEW

. ‘I don’t want anyone to know’: experiences of obtaining access to HIV testing by eastern European, non-European Union sex workers in Amsterdam, The Netherlands. PLoS ONE. 2020;15(7):e0234551-e.

25.

Abebe

Teferra

. Disclosure of diagnosis by parents and caregivers to children infected with HIV: prevalence associated factors and perceived barriers in Addis Ababa, Ethiopia. AIDS Care. 2012;24(9):1097–1102.

26.

Kyilyosudu

Msuya

Ngocho

Damian

. Factors associated with offer and uptake of provider-initiated HIV testing and counselling among men attending healthcare facilities in Moshi Municipality, Northern Tanzania Factors associated with offer and uptake of provider-initiated HIV testing and counselling among men attending healthcare facilities in Moshi Municipality, Northern Tanzania. PLoS ONE. 2023;18(9):e0291792.

27.

Durlak

DuPre

. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41:327–350.

28.

Medina-Marino

Bezuidenhout

Ngwepe

, et al. Acceptability and feasibility of leveraging community-based HIV counselling and testing platforms for same-day oral PrEP initiation among adolescent girls and young women in Eastern Cape, South Africa. J Int AIDS Soc. 2022;25(7):e25968.

29.

Research

. The effects of early social-emotional and relationship experience on the development of young orphanage children. Monogr Soc Res Child Dev. 2008;73(3):vii.

30.

Salmi

Noël

Saillour-Glénisson

. Decision-making process and evaluation of public health interventions. . Rev D’épidémiol Santé Publique. 2023;71(2):101384. -.

31.

Stetler

Legro

Wallace

, et al. The role of formative evaluation in implementation research and the QUERI experience. J General Internal Med: JGIM. 2006;21(S2):S1–S8.

32.

Carroll

Patterson

Wood

Booth

Rick

Balain

. A conceptual framework for implementation fidelity. Implement Sci: IS. 2007;2(1):40.

33.

Pérez

Van der Stuyft

MdC

Castro

Lefèvre

. A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implement Sci: IS. 2016;11(1):91.

34.

Breitenstein

Gross

Garvey

Hill

Fogg

Resnick

. Implementation fidelity in community-based interventions. Res Nurs Health. 2010;33(2):164–173.

35.

Assembly UG . The convention on the rights of the child. United Nations; 1989.

36.

Fayed

De Camargo

Kerr

, et al. Generic patient-reported outcomes in child health research: a review of conceptual content using World Health Organization definitions. Dev Med Child Neurol. 2012;54(12):1085–1095.

37.

AIDS WHOGPo . Immunization WHOEPo, United Nations Children’s F. Joint WHO/UNICEF statement on early immunization for HIV-infected children / World Health Organization in collaboration with United Nations Children’ s Fund. Geneva: World Health Organization; 1989.

38.

Goba

Wood

Nelson Mandela

. Care and support of orphaned and vulnerable children at school: helping teachers to respond. South African J Educ. 2011;31(2):275–290.

39.

Ababa

. HIV/AIDS in Ethiopia. 2014.

40.

Tariku

Worede

Belete

. Adherence to antiretroviral therapy and associated factors among human immunodeficiency-positive patients accessing treatment at health centers in East Gojjam Zone, Northwest Ethiopia, 2019: community-based cross-sectional study. Heliyon. 2023;9(7):e18279.

41.

Otara

Chebor

Raballah

. The influence of HIV/AIDS care and support services accessibility on utilization among HIV infected adults in Baringo county, Kenya. Evid-Based Nurs Res. 2024;6(2):16–28.

42.

Duffy

Sharer

Berhan

, et al. HIV risk screening for high-yield community testing services for orphans and vulnerable children: a literature review. Vulnerable Child Youth Stud. 2018;13(2):95–115.

43.

Radfar

Nematollahi

Arasteh

. Factors influencing access and use of care and treatment services among Iranian people living with HIV and AIDS: a qualitative study factors influencing access and use of care and treatment services among Iranian people living with HIV and AIDS: a qualitative study. Iran J Public Health. 2016;45(1):109–111.

44.

Lorenzetti

Sousa

Martinez

, et al. Assessing the effect of COVida orphans and vulnerable children support services on viral load coverage and suppression among children and adolescents living with HIV in four provinces in Mozambique. AIDS Care. 2024;36(8):1190–1198.

45.

Ssewamala

Nabunya

Mukasa

Ilic

Nattabi

. Integrating a mentorship component in programming for care and support of AIDS-orphaned and vulnerable children: lessons from the Suubi and bridges programs in sub-Saharan Africa. Global So Welfare: Res Policy & Practice. 2014;1(1):9–24.

46.

Ferreira

Paula

Kleinubing

Kinalski

DDF

Anversa

ETR

Padoin

. Evaluation of the quality of primary health care for children and adolescents with HIV: PCATool-Brasil. Rev Gaúcha Enferm. 2016;37(3):e61132-e.

47.

Molato

Moloko-Phiri

Koen

Matsipane

. Support provided by outreach team leaders to caregivers of HIV/AIDS orphans in the north-west province of South Africa. BMC Nurs. 2024;23(1):605.

48.

Assefa

Gilks

Dean

, et al. Towards achieving the fast-track targets and ending the epidemic of HIV/AIDS in Ethiopia: successes and challenges. Int J Infect Dis. 2019;78:57–64.

49.

Adeyemi

Lyons

Njim

, et al. Integration of non-communicable disease and HIV/AIDS management: a review of healthcare policies and plans in east Africa. BMJ Global Health. 2021;6(5):e004669.

50.

Rosen

Fox

. Retention in HIV care between testing and treatment in sub-Saharan Africa: a systematic review. PLoS Med. 2011;8(7):e1001056.

51.

McCree

Young

Henny

Cheever

McCray

. US Centers for disease control and prevention and health resources and services administration initiatives to address disparate rates of HIV infection in the south. AIDS Behav. 2019;23(Suppl 3):313–318.

52.

Organization WH . Consolidated guidelines on the use of antiretroviral drugs for treating and preventing HIV infection: recommendations for a public health approach. 2013.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.04 MB

Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City,Northwest Ethiopia: A Mixed-Methods Evaluation

Abstract

Objective

Method

Result

Conclusion

Plain Language Summary Title

Keywords

Introduction

Evaluation Methods and Materials

Evaluation Area and Period

Inclusion and Exclusion Criteria

Evaluation Design

Evaluation Approach

Focus on Evaluation and Dimensions

Sample Size Determination and Sampling Procedures

For Exit Interviewee Questionnaire

For Direct Observation

For Retrospective Document Review

Resource Inventory

For Key Informant Interview

Data Collection Tools and Sampling Techniques

Data Quality Control

Variables and Measurements

Data Management and Analysis

Matrix of Analysis and Judgment

Statistical Analysis

Result

HPs Adherence to HIV/AIDS C&S Services

Characteristics of HIV/AIDS C&S Providers

Document Review

Nonparticipatory Direct Observation

Dosage/Exposure to HIV/AIDS C&S Services

Caregivers of OVC Responsiveness to HIV/AIDS C&S Services

Sociodemographic Characteristics of OVC

Caregivers’ Level of Engagement to HIV/AIDS C&S Services

Factors Affecting Caregiver’s Responsiveness

Discussion

Strength and Limitations of Study

Conclusion

Supplemental Material

sj-docx-1-jia-10.1177_23259582251372458 - Supplemental material for Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City, Northwest Ethiopia: A Mixed-Methods Evaluation

Supplemental Material

sj-docx-2-jia-10.1177_23259582251372458 - Supplemental material for Implementation Fidelity of HIV/AIDS Care and Support Services for Orphan and Vulnerable Children in Gondar City, Northwest Ethiopia: A Mixed-Methods Evaluation

Footnotes

Abbreviations

Acknowledgments

ORCID iD

Ethics Approval and Consent to Participate

Authors’ Contributions

Funding

Declaration of Conflicting Interests

Availability of Data and Material

Supplemental Material

References

Supplementary Material