Abstract
Background
Youth living with HIV with perinatal infection spend a lifetime taking antiretroviral treatment (ART) to suppress the virus, and face significant challenges to successfully maintaining ART adherence. Tools to measure adherence include self-report, medication event monitoring system (MEMS) pill bottle caps, pill counts, and plasma or hair drug levels; however, the inter-rater agreement between child and caregiver self-report has not been validated in an African setting. This study aims to assess inter-rater agreement between child and caregiver self-reports, compared to reporting from MEMS pill bottle caps.
Methods
This was a secondary analysis of a cluster-randomized trial to evaluate an intervention for children living with HIV, conducted at the Academic Model Providing Access to Healthcare in western Kenya. We analyzed data from 285 child-caregiver dyads to compare adherence self-reported by children and their caregivers, and subsequently compared all self-reports to adherence reported by MEMS pill bottle caps to determine whether child or caregiver self-reports aligned more closely with adherence measured by MEMS.
Results
Children and their caregivers reported similar levels of adherence and numbers of missed doses in the past month, and both reports were similarly associated with adherence reported by MEMS pill bottle caps. Children with a caregiver that was not a biological parent were significantly more likely to report more missed doses than their caregiver. The correlation coefficient for the relationship between the child and caregiver self-reports was 0.71; for the relationship between child report and MEMS was 0.23; and for the relationship between caregiver report and MEMS was 0.20. Both children and caregivers under-reported non-adherence compared to MEMS data.
Conclusion
Children and caregiver self-reports were generally similar in reporting adherence and were not highly correlated with MEMS reports of adherence, with children and caregivers reporting higher level of adherence than the MEMS data. This may indicate that children and caregiver reports are similarly inaccurate or biased; however, further research with larger sample sizes is required to further understand the differences in these reports.
Plain Language Summary
The study aims to compare adherence between children and caregivers of Youth Living with HIV (YLWH) with perinatal infection, comparing data from 285 child-caregiver dyads and MEMS pill bottle caps. Results showed similar levels of adherence and missed doses in the past month, with a correlation coefficient of 0.71. However, children and caregivers reported higher levels of adherence than MEMS data. The study highlights the importance of understanding the reliability between self-reports and MEMS data in promoting adherence among YLWH.
Introduction
Among the world’s 1.75 million children and adolescents living with HIV, 90% of whom reside in African countries, adherence to antiretroviral treatment (ART) is essential for living a long, healthy life. 1 Youth living with HIV (YLWH) with perinatally acquired infection require lifelong ART, and often face challenges to adherence, which may depend on their age and social context. For both younger and older YLWH, poor adherence has been reported to be associated with forgetting doses, orphan status, and change to routine.2,3 Some studies have also reported adherence worsening over time from childhood to adolescent, with more adolescents reporting the influence of travel, the need for secrecy to avoid stigma, treatment fatigue, high pill burden, and living a considerable distance from the clinic as reasons for non-adherence.2–6
The impact of non-adherence, including viral failure, drug resistance, and increased morbidity and mortality, is exacerbated in resource-limited settings where second- and third-line ART regimens are limited, and the timely detection of treatment failure is not readily accessible.7,8 Comprehensive assessment of ART adherence is therefore a critical component in preventing these adverse outcomes.9–12
Previous work in Uganda, Kenya, South Africa, Thailand, and the Netherlands have used various combinations of caregiver self-reports of missed doses, electronic dose monitoring, pill counts, and plasma or hair drug levels to measure and report on ART adherence.5,13–15 Due to the high cost associated with its’ use, electronic dose monitoring is not feasible in every setting; however, it is considered to be a reliable measurement of medication adherence. 16 Medication event monitoring systems (MEMS®; Aardex Group Ltd., Seraign, Belgium, Switzerland) 17 pill bottle caps are commonly used for this purpose and record a time stamp of each time the pill bottle is opened in order report on how many doses are taken in a time period. A more feasible and low-cost measurement of adherence is child and caregiver self-reports, as they don’t require any additional equipment, blood draws or for YLWH to bring medication into clinic.13,14,18–20 Self-reports, however, introduce a risk of social desirability bias, in which individuals may over-report the number of doses taken out of a fear of reporting undesirable outcomes to researchers and clinicians. 21 While social desirability bias is somewhat intractable in research and clinical assessment, it may be mitigated by establishing trust and rapport with researchers or clinicians or using particular language in surveys and questionnaires, which may increase the validity of self-reports.22–24
Caregiver reports of child ART adherence have been validated against MEMS and viral load measurements in Kenyan and North American settings.20,25 The inter-rater agreement between child and caregiver reports of ART adherence; however, has not been validated in settings in African countries. This secondary analysis of data from a disclosure intervention study seeks to compare reports of ART adherence between child and caregiver self-reports and MEMS reports in a cohort of YLWH in western Kenya.
Methods
Study Design
This was secondary analysis of data from a cluster-randomized controlled trial to evaluate an intervention to support disclosure of HIV status to children living with HIV (CLWH) in western Kenya (Vreeman 1R01MH099747-01, ‘Patient-Centered Disclosure Intervention for HIV-Infected Children). 26 The disclosure intervention included intensive individual and group counseling sessions, culturally tailored pamphlets and videos designed locally (http://hiv-films.org), and clinical and psychosocial assessments. Assessments, including detailed evaluations of ART adherence, disclosure status, emotional health, stigma and other physical, emotional and social outcomes, were conducted at baseline and 6-month intervals thereafter for 2 years, for a total of five visits. This analysis uses only the results of the adherence evaluations. Randomization was done at the clinic level; intervention clinics received the disclosure intervention and control clinics received the standard of care for completing HIV disclosure. Details regarding the intervention have been discussed elsewhere.26,27
Setting and Participants
The original study was conducted at eight health facilities of the Academic Model Providing Access to Healthcare (AMPATH) in western Kenya. AMPATH is a long-standing partnership between Moi University School of Medicine, Moi Teaching and Referral Hospital, the Kenyan Ministry of Health, and a consortium of North American universities.28,29 Participants were selected using convenience sampling, and the inclusion criteria were (a) 10-14 years of age, (b) perinatally acquired HIV infection, and (c) enrolled in HIV care at one of the eight study clinics; the only inclusion criteria for caregivers was that they reported significant involvement in the child’s care. The child’s disclosure status was not an inclusion or exclusion criteria for the study. At the time of study and currently in Kenya, the local and national protocols state that CLWH should be informed of their HIV status by age 10 years, and WHO stages of disease are defined for both children and adolescents. 30
Ethics Approvals and Informed Consent
The parent study was approved by the Institutional Review Board at Indiana University School of Medicine in the United States (Protocol #1206009036) and the Institutional Research Ethics Committee at Moi University School of Medicine in Kenya (Study Number IREC/2012/162; Approval #000872). Written informed consent was received from caregiver participants, and assent was received from participants under 18. Participants received financial reimbursement for their time and travel.
Data Collection Tools
Self-reports of ART adherence were collected through questionnaires previously validated in western Kenya, with versions for both caregiver self-report of children’s adherence and child or youth self-report. 20 The questionnaires were administered orally by trained research assistants in either Swahili or English, depending on the preferences of the child and caregiver. Children and their caregivers were provided questionnaires during the same visit. Participants were provided with MEMS pill bottle caps to keep their ART medication in, 17 which record the time and date of bottle opening to track the number and frequency of doses removed from the bottle. Only one MEMS cap was issued per participant; if participants were prescribed more than one medication, preference was given to the medication taken twice per day. Additional data, including child’s school attendance status, orphan status, and whether the child has a sibling living with HIV, were obtained through review of the medical record.
Statistical Analysis
Demographic variables were reported by mean and standard deviation for continuous variables (age and % MEMS non-adherence) and as number and percentage for categorical variables (sex, relationship to caregiver, WHO stage, orphan status, having a sibling with HIV, and school attendance). We analyzed child and caregiver reports of adherence both as continuous (whole number of missed doses in the past 30 days) and binary variables (fully adherent [0 missed doses in the past 30 days] and any non-adherence [any missed doses in the past 30 days]), both assessed longitudinally over the 2-year study period at intervals of 6 months. Exploratory analyses were conducted first, using scatterplots comparing the number of missed doses in the past month as reported by children and caregivers. Mean number of missed doses, as reported by children and caregivers, were compared, and Spearman correlation coefficients were calculated to estimate the correlation between child and caregiver estimates. Correlation coefficients were also calculated to estimate the agreement between child and caregiver reports of the number of missed doses in the past month and percent ART adherence as measured by MEMS. Finally, we compared the average percentage non-adherence measured by MEMS in concordant child-caregiver dyads in which both child and caregiver reported either adherence or non-adherence, and in discordant dyads in which either child or caregiver reported adherence and the other reported non-adherence. All analyses were conducted in R version 4.0.5.
Results
Participant Characteristics
A total of 285 child-caregiver dyads were enrolled at baseline. Of these, 253 who had completed all four follow-up visits were included in the secondary analysis. The mean age of the children at enrollment was 12.31 years (range 9-15), and 51.4% were male (Table 1). Of note, participants were recruited outside of the original inclusion criteria—the resultant sample was appropriately representative of the clinical population. Caregivers were most often stepmothers (37.5%) or mothers (32.8%) of participants. Nearly all of the participants attended school (98.4%), nearly half were orphaned (49.2%), and less than a quarter reported having a sibling living with HIV (17.8%). Of participants who had WHO stage reported (n = 250), most were stage 1 (33.2%), stage 2 (32%) or stage 3 (31.6%). The average percentage ART non-adherence reported through MEMS caps over the entire study period was 29.0%.
Participant Demographics at Baseline.
Percent of sample who was ever non-adherent (missing ≥ 10% of doses).
MEMS: medication event monitoring system.
Mean Missed Doses Reported by Children and Caregivers
Over the entire study period, children reported a mean (SD) of 0.77 (2.93) missed doses in the past 30 days and caregivers reported 0.54 (3.01) missed doses (Table 2). Overall, children reported 0.22 more missed doses than their caregivers; however, this difference was not statistically significant (P = 0.16). Children generally reported more missed doses in the past month than their caregivers; however, none of these differences was statistically significant.
Average Number of Missed Doses Reported by Child and Caregiver.
Number of observations in each demographic group are not summative to total; variables abstracted from medical charts were only recorded once, and not every variable was reported on at every study visit.
Results of a Welch two-sample T-test.
Comparison of Child and Caregiver Reports of Adherence with MEMS
Overall child-caregiver dyads in which both parties reported no missed doses had an average of 19.7% non-adherence as reported by MEMS (Table 3). Dyads in which both child and caregiver reported any missed doses had an average of 33.8% non-adherence. There was a moderately weak positive correlation between child and caregiver reports of missed doses in the past 30 days, with a correlation coefficient of 0.38 (Table 4). The correlation coefficient comparing self-report of missed doses and MEMS percentage non-adherence for both children and caregivers was 0.21. There were slight differences in the correlations between child or caregiver report and MEMS percentage non-adherence among the demographic groups that were not significant, with the strongest correlation between child and caregiver being among children whose caregiver is their biological parent, and the weakest correlations among children who were orphans or had siblings living with HIV.
Average Percent MEMS Non-adherence for Dyads Reporting Full Adherence or Any Non-adherence.
MEMS: medication event monitoring system.
Correlations Between Child and Caregiver Reports of Missed Doses in the Past Month and MEMS Reports of Non-adherence.
MEMS: medication event monitoring system.
Discussion
This analysis followed child-caregiver dyads longitudinally for 2 years and found no significant differences between child and caregiver self-reports of the average number of missed doses in the past 30 days among any of the dyad sub-categories. Overall, there was moderate agreement between child and caregiver self-reports of missed doses. The correlation between child and caregiver self-reports and MEMS reports of non-adherence varied and were not significantly different.
These analyses highlight discrepancies in self-report and MEMS-reported adherence; even when both the child and caregiver reported no missed doses, MEMS data report nearly 20% of doses missed. These trends seem to suggest that children and caregivers are often in agreement about how many doses were missed in the past month, and self-reports by both groups are similarly inaccurate against the average report of non-adherence measured by MEMS. These results do not fully align with those presented in existing literature, which suggest that MEMS data have been correlated to self-reported adherence among CLWH both in the same Kenyan setting as well as in South Africa and Thailand. 14 Reasons behind this warrant further analysis and may be related to previous literature using a definition of a missed dose in the past 7 days as non-adherence, rather than a missed dose in the past month. 14 Discrepancies may also be due to differences in the use of the MEMS caps, in which participants may open the pill bottles outside of times when they take doses.
When dyads did not report the same levels of ART adherence, MEMS-reported adherence was more concordant with caregiver self-report. These findings continue to suggest that electronic dose monitoring is more of a “gold standard” than self-reports, in that it has a greater detection of non-adherence in this population.
These findings support reasonable concordance between the child and caregiver reports of adherence, even when both under-reported non-adherence as compared with electronic dose monitoring. This somewhat contradicts previous studies that have illustrated differing levels of concordance between child and caregiver reports of adherence.16,31–33 A study in South Africa found that adolescents self-reported slightly higher percentages of non-adherence (30% vs. 24%), with higher concordance in dyads where the caregiver is a biological parent. 31 Studies from cohorts of children perinatally infected with HIV in the United States have largely found low concordance between child and caregiver reports of missed and late doses, with higher levels of discordance in older groups.16,32,33
This study has several limitations that warrant discussion. First, the sample size was small, which prevented meaningful subgroup analysis by age and caregiver relationship, and limits the validity of interpretations of P-values. Second, the purpose of the parent study was to evaluate a disclosure intervention, and thus the secondary data analysis presented here is limited in that not all outcomes of interest for this analysis (eg, viral loads and pill counts) were recorded and not all variables were fully reported. However, 34% of participants in both arms of the study went through the disclosure process, and this study presented the opportunity to assess potential fluctuations in adherence over the course of 2 years of longitudinal follow-up whether or not they were disclosed. No analyses were conducted on the impact of the disclosure intervention on adherence. Additionally, the analyses did not take into consideration inter-participant measurements, despite there being data recorded across multiple time points, in order to maintain focus on the comparison of child and caregiver reports. Finally, the MEMS caps were only used for one ART medication, regardless of how many medications a participant was taking. While medications would typically be taken at the same time, this may have impacted results by under- or overestimating the number of doses taken. The data arising from the questionnaires may have also been subject to social desirability bias; however, the use of these questionnaires has been previously validated, suggesting the impact of this bias is minimal.
Conclusion
Child and caregiver self-reports of ART adherence overall were not found to be significantly different, and both participants similarly under-reported non-adherence compared to MEMS pill bottle data. Discrepancies between self-reports and MEMS data raise potential concerns about the accuracy of both measurements of adherence. It is critical to better understand the accuracy of how adherence is measured and reported, and to identify whether child and caregiver reports of adherence are accurate against one another and other quantifiable methods in order to identify the most accurate methods of adherence reporting. It is particularly important to confirm the accuracy of self-reports, as one of the most common and accessible forms of adherence reporting, to better understand the adherence of YLWH throughout childhood and adolescence and inform their HIV care. Therefore, further research with larger sample sizes is suggested to confirm these findings.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the National Institute for Mental Health, Bethesda, Maryland, USA through a grant entitled ‘Patient-Centered Disclosure Intervention for HIV-Infected Children’ (1R01MH099747-01).
