Sage Journals: Discover world-class research

Abstract

Background:

There are few validated tools to measure stigma, particularly among children living with HIV and their families.

Methods:

This study was nested within a larger study that followed 240 child–caregiver dyads (children aged 10-15 years) at 8 clinics in western Kenya. The stigma instrument was administered to all child–caregiver dyads at 2 time points 6 months apart. The primary end point was to construct validity assessed by comparison to criterion constructs using generalized estimating equation models.

Results:

Mean age of child participants was 12.3 years and 52% were female. Generally, caregivers reported experiencing higher levels of HIV stigma compared to their children. Children (9%) and caregivers (14%) reported that HIV stigma made them feel stressed, anxious, and depressed. Child and caregiver stigma items showed high construct validity by emotional and behavioral outcomes.

Conclusions:

The stigma instrument showed high validity when compared to emotional and behavioral outcomes.

Keywords

HIV/AIDS stigma children Kenya validation study

What Do We Already Know About This Topic?

HIV stigma is a major barrier to all aspects of HIV testing and treatment, but there are few validated tools to measure stigma among children living with HIV.

How Does Your Research Contribute to the Field?

Using a mixed-methods approach, we developed and tested a grounded stigma instrument among children living with HIV in western Kenya, which showed good construct validity with relevant emotional and behavioral outcomes.

What Are Your Research’s Implications Toward Theory, Practice, or Policy?

Our research demonstrates that it is possible to develop a simple, low-cost questionnaire to identify children experiencing HIV-related stigma in a clinical setting who can be targeted for additional intervention or services.

Introduction

HIV-related stigma is an important barrier to all aspects of HIV prevention and treatment, including delayed HIV testing and enrollment in care,^1
–3 increased risk of not being retained in care,^4
–6 nonadherence to antiretroviral therapy (ART),^7

–10 and higher likelihood of nondisclosure to partners.^11,12 The negative impact of HIV stigma on psychosocial health, such as depression, anxiety, and decreased quality of life (QoL), has also been noted.^13

–16 Unfortunately, the literature on HIV stigma has been heavily skewed toward adults living with HIV and has not adequately considered the impact of stigma on children and adolescents living with HIV, most of whom reside in sub-Saharan Africa.¹⁷

An estimated 1.8 million children younger than 15 years are living with HIV, with less than half accessing ART.¹⁸ Children living with HIV may be at substantial risk of HIV stigma and its deleterious effects on HIV treatment and psychosocial health in ways both similar and distinct from adults. For example, perinatally infected children are at higher risk for stunted growth and delayed bodily development, and these differences in appearance may be a source of anxiety and internalized stigma or increase risks for social isolation and discrimination. Studies suggest that HIV stigma encountered from peers at school in the form of taunting, gossiping, or bullying, either because of the child’s own status or the status of a family member, are formative experiences for HIV-infected and affected youth^19

–22 and may lead to problems in school attendance or accessing peer support networks.^23
–25 Moreover, children may be affected by HIV stigma experienced at the caregiver level, which has been shown to negatively impact HIV-infected children,^26
–28 including caregivers delaying giving children medicines or taking them to clinical care.^29
–31 Generally, HIV stigma has been hypothesized to exacerbate poverty, malnutrition, and access to services for HIV-affected families.^32
–34

As more children access treatment, survive into adolescence, and transition into adult care settings, they must learn about their HIV status. Parents and caregivers of children with HIV often report that disclosure to children will improve ART adherence,^{35

–44} increase the child’s long-term engagement in care,^{35,38

–42,44
–46} and equip the child to protect himself or herself and others from HIV.^39,43,47 On the other hand, the potential negative impacts of HIV stigma weigh heavily in decisions about disclosure. Studies frequently cite parents’ and caregivers’ fears that the child will subsequently disclose his or her HIV status to others,^{38

–41,44,46,48
–50} leading to potential stigma and social isolation for the child as well as the family.^{38,40,41,44,45,49}

A major barrier to investigating the prevalence and impacts of HIV stigma among children living with HIV and their families, as well as evaluating interventions to combat stigma, is the lack of reliable and valid stigma measures, particularly in resource-limited settings.^51
–53 Several authors of this paper (blinded) previously undertook a systematic review of measures of HIV stigma among HIV-infected and affected children.⁵⁴ Among 27 articles describing quantitative HIV stigma measures for HIV-infected and affected youth younger than 18 years, only a handful explicitly evaluated the validity of stigma measures using rigorous methods to ensure content validity, internal consistency, and construct validity, with most studies modifying existing measures such as the HIV Stigma Scale developed among adults in the United States. In this article, we report the results of the SAFI (Stigma in AIDS Family Inventory) validation study that aimed to systematically evaluate a culturally adapted, developmentally appropriate measure to assess HIV stigma experienced by children living with HIV and their caregivers in western Kenya.

Methods

Study Design

We conducted a multiphased validation study to develop and test a reliable, valid instrument to measure HIV stigma among children living with HIV and their families. In the first phase, we constructed a questionnaire for children and their caregivers. A potential set of questionnaire items were identified through a systematic review of existing questionnaires and other methods used for pediatric HIV stigma measurement.⁵⁴ We utilized a conceptual framework that assumes people living with HIV (or caring for those living with HIV) may experience 4 types of HIV stigma that operate in subtle but in distinctive ways. Perceived and enacted stigma are characterized by the fear that HIV stigma and discrimination may occur and the belief that it has occurred, while internalized stigma captures negative beliefs among oneself because of their HIV status.^55,56 A fourth type of stigma, courtesy stigma, describes stigma and discrimination directed at an individual who is associated with or attending to someone who is living with HIV (but not necessarily infected themselves), which has important consequences for families caring for infected children.⁵⁷ We incorporated these various dimensions of stigma into the stigma questionnaire.

Adaptation of stigma questionnaire items in the literature and the conceptual framework to the local setting and population were informed by focus groups with children living with HIV and caregivers of HIV-infected children (results published elsewhere), as well as cognitive interviews with children and caregivers to test and maximize face validity of items. The final set of stigma questionnaire items was translated from English to Kiswahili by a trained translator and then back-translated into English, repeating this process until items were consistent. Following recommendations for cross-cultural test adaptation,⁵⁸ a team of multicultural and multilingual experts reviewed the translations and assessed items for cultural sensitivity and intercultural communication. This team has extensive qualitative research methods and instrument validation and cross-cultural adaptation expertise in this setting, particularly related to pediatric adherence to treatment and disclosure of HIV status in which issues of stigma are prominent.^{31,44,59

–64}

The resultant questionnaire, known as the SAFI Stigma questionnaire, asks a series of questions regarding the stigma and discrimination that an individual experiences because of his/her own or someone in their families’ HIV status. Participants were instructed to respond to an experience as having “never happened” or “ever happened.” If “ever happened” was selected, the participant was then prompted to describe the time frame in which the experience happened, categorized as either in the “past 6 months” or “longer than 6 months ago.” Many of the question prompts are led by the statement, “Because I have HIV or someone else in my family has HIV…”

In the second phase, which are reported here, we tested the reliability and validity of the HIV stigma questionnaire items (child and/or caregiver stigma questionnaires provided on request). Reliability and validity testing was nested within a cluster, randomized, prospective cohort study to evaluate a disclosure counseling intervention. The study followed 240 families with HIV-infected children aged 10 to 15 years for 2 years at 8 clinics (∼35 children per clinic), who are part of the Academic Model Providing Access to Healthcare (AMPATH) in western Kenya, one of the largest HIV care providers in sub-Saharan Africa.^65,66 Informed consent was required from all of the child participants’ parents or guardians, and assent was also required from all child participants. Study assessments were conducted at baseline and every 6 months thereafter, which included clinical characteristics (WHO disease stage, CD4 counts, and viral load) and various questionnaires administered by research assistants assessing disclosure status, adherence to treatment, behavioral and mental health, and QoL. Child participants were also issued Medication Event Monitoring Systems (MEMS^® Cap, WestRock, Atlanta, USA) caps for electronic dose monitoring. Baseline characteristics of this cohort and study methods, including the various assessments and how they were interpreted, are published elsewhere.⁶⁷ The stigma questionnaire was administered (in English or Kiswahili, depending on participant preference) to all children and their caregivers at the final 2 routine study evaluations (month 18 and 24) alongside the other assessments. Due to the fact that the stigma questionnaire specifically references HIV, only children who were disclosed and knew their HIV status were administered the stigma questionnaire (all caregivers were administered the questionnaire). At month 18 and 24, 126 and 155 children were administered the stigma questionnaire, respectively. Data were collected from October 2014 to June 2015.

Ethical Statement

The study was approved by the institutional review board of Indiana University School of Medicine (Protocol #1206009036) in Indianapolis, Indiana, and the Moi University/Moi Teaching and Referral Hospital Institutional Research and Ethics Committee (Approval #000872) in Eldoret, Kenya. All study participants aged 18 years and older provided informed consent prior to enrollment. For children younger than 18 years, we obtained assent as well as informed consent from their guardian according to local laws and guidelines. Research participants were provided with a small stipend (about US$3) per study assessment to compensate them for their travel and time. To reduce the risk of accidental disclosure of HIV status to children, research assistants underwent additional training before consenting and assessing study participants, which included modules on disclosure assessment protocols, confidentiality, and privacy. Additionally, research assistants were trained to refer any participants whom they deemed to be demonstrating significant psychiatric distress. A specific mental health study referral form was created to document and refer participants for mental health services.

Analysis

Descriptive statistics and item analysis of the participants’ responses to stigma measurement items were conducted, including item response distribution, item discrimination indices, and item-scale correlation. Test–retest reliability was assessed for individual stigma items at the 2 time points for stigma items administered to children and separately for stigma items administered to caregivers. We also examined the agreement between child and caregiver responses to individual stigma items that were sufficiently similar (a total of 8 items), which was assessed using κ statistics. The primary end point of the validation study was construct validity, which was assessed by comparison to primary criterion constructs using generalized estimating equation (GEE) models. These GEE models allowed for the modeling of repeated measures across visits, gave the ability to accurately model the covariance structure to ensure the best fit, and allowed for the use of distributions within the exponential family, to model nonlinear data. Primary criterion constructs were adherence to treatment, QoL, behavioral health, depression, and clinical characteristics. Adherence was assessed using a previously validated questionnaire in this setting that asks about missed and late doses as well as barriers to adherence.^59,60 In addition, participants were issued electronic dose monitors (MEMS) that recorded a time and date stamp each time the bottle was opened. Quality of life was assessed using the multidomain Pediatric AIDS Clinical Trials Group General Health Assessment for Children (GHAC) QoL questionnaire developed by the Pediatric AIDS Clinical Trial Group. The GHAC was constructed through previously validated measures with good internal consistency, reliability, and validity. The modular, age-specific GHAC instrument for children measured the QoL on the following 5 domains: general health, physical functioning, symptoms, psychological well-being, and social and role functioning. Each domain of the GHAC is scored separately on a Likert scale, with scaled scores ranging from 0 to 100, with higher scores indicating better QoL.^68
–70 Behavioral health was assessed using the 25-item Strengths and Difficulties Questionnaire (SDQ)–Youth Version,^71,72 which consists of 5 scales that measure difficulties in 5 psychological domains: emotional symptoms, conduct problems, hyperactive-inattention, peer relationship, and prosocial behavior. Participants are asked to rank their behaviors as “not true,” “somewhat true,” or “certainly true.” Depression was assessed using the Patient Health Questionnaire–9-item (PHQ-9) depression instrument,⁷³ a 9-item module adapted from the full PHQ assessment and provides diagnosis of probable depression and assesses severity of symptoms. Participants are asked to report how often they have experienced depressive symptoms over the last 2 weeks, on a scale of 0 (not at all) to 3 (nearly every day). The PHQ-9 has demonstrated high internal reliability. Patient responses to the GHAC,^69,74,75 SDQ,^71,72 and PHQ-9⁷⁶ were scored and transformed consistent with the published literature. Linear (for continuous outcome variables) and logistic (for dichotomized variables) regression models were used for validating stigma items against criterion constructs, while accounting for key demographic variables. Demographic and clinical characteristics were extracted from the participants’ medical chart using a standardized clinical data extraction tool.

Results

Participants’ Characteristics

At study baseline, mean age of child participants was 12.3 years and 147 were female (52%), with the majority of caregiver participants being the biological mother of the child (54%), as well as a number of aunts/uncles (19%), biological fathers (17%), and grandparents (11%; Table 1). About half of children were total orphans (ie, both their mother and father were deceased) and a fifth has a sibling who was also living with HIV. Adherence monitoring using MEMS revealed only 28% of children were able to sustain >90% of doses taken over the follow-up period, with median adherence of 70% of disease taken.

Table 1.

Participants’ Characteristics.

Characteristics	N	Mean ± SD or Median (IQR) or n (%)
Age (years)	285	12.31 ± 1.50
Male	285	147 (51.6%)
CD4 (cells per mm³)	271	732.84 ± 504.26
CD4 percent	259	26.45 ± 10.38
WHO stage	282
1		90 (31.9%)
2		90 (31.9%)
3		92 (32.6%)
4		10 (3.6%)
Orphan status	285
Both parents deceased		141 (49.5%)
Mother deceased		89 (31.2%)
Father deceased		90 (31.6%)
Sibling has HIV	265	52 (19.6%)
Caregiver at assessment	285
Mother		155 (54.4%)
Father		48 (16.8%)
Sibling		14 (4.9%)
Grandparent		31 (10.9%)
Aunt/uncle		55 (19.3%)
Children’s home		5 (1.8%)
Others		8 (2.8%)
Attends school	279	275 (98.6%)
MEMS % doses taken	270	70.54 (46.4)
MEMS adherent (>90%)	270	76 (28.2%)

Abbreviations: IQR, interquartile region; MEMS, Medication Event Monitoring Systems; SD, standard deviation.

Responses to the Stigma Questionnaire

Child and caregiver participants’ responses to stigma items are provided in Table 2. A small proportion of children (ie, less than 10%) responded that they had lost friends due to their HIV status, had been bullied or insulted, experienced HIV-related discrimination, and had lost social support. More children, although still a minority, reported that their HIV status had made them feel stressed or anxious (9% at month 24) or depressed or sad (9% at month 24). Almost half of the children (45%) reported that it was important to keep their HIV status a secret (45%), but only 11% reported that they delayed taking their HIV medications so others would not see them. Caregivers reported slightly higher levels of stress or anxiety (13% at month 24) and depression or sadness (14% at month 24) due to their child’s HIV status. Moreover, caregivers were more likely to report other forms of stigma and discrimination that they experienced. For example, while 11% of caregivers reported being called names or bullied due to the child’s HIV status, only 3% of children reported these experiences, and 10% of caregivers reported experiencing discrimination in the neighborhood compared with only 2% of children. Caregivers were also much more likely to report thinking that people in the community think HIV is a dirty or immoral disease, with 21% reporting a few people think that and 19% reporting most people think that.

Table 2.

Responses to Stigma Items.

Stigma Items	Frequency Responding That Item “Ever Happened” (and %)
Stigma Items	Month 18	Month 24
Child stigma items	N = 126	N = 155
Q1. Lost friends due to HIV status	8 (6.4%)	3 (1.9%)
Q2. Called names/bullied/insulted due to HIV	9 (7.1%)	5 (3.3%)
Q3a. Experienced discrimination at home due to HIV	6 (4.8%)	6 (3.9%)
Q3b. Experienced discrimination in neighborhood due to HIV	8 (6.4%)	3 (2.0%)
Q3c. Experienced discrimination at church due to HIV	0 (0%)	1 (0.7%)
Q3d. Experienced discrimination at clinic due to HIV	0 (0%)	0 (0%)
Q3e. Experienced discrimination at school due to HIV	7 (5.6%)	4 (2.6%)
Q3f. Experienced discrimination at another place due to HIV	1 (0.8%)	0 (0%)
Q4a. Lost financial support/work due to HIV	4 (3.2%)	1 (0.7%)
Q4b. Lost social support due to HIV	2 (1.6%)	0 (0%)
Q5a. HIV stigma has made me feel stressed/anxious	11 (8.7%)	14 (9.2%)
Q5b. HIV stigma has made me feel depressed/saddened	9 (7.1%)	14 (9.2%)
Q6. Choose not to play with others or go places due to HIV	4 (3.2%)	9 (5.9%)
Q7. Feel it is important to keep HIV status secret	72 (57.1%)	69 (45.1%)
Q8. Caregiver feels it is important to keep my HIV status secret	70 (56.5%)	65 (42.8%)
Q9. Future and hopes have changed in a negative way due to HIV	8 (6.4%)	8 (5.2%)
Q10. Delayed taking meds so others do not see	17 (13.5%)	17 (11.1%)
Q11. People in community think HIV is dirty/immoral/shameful
No one thinks	82 (66.1%)	117 (78.5%)
A few people think	31 (25.0%)	18 (12.1%)
Most people think	11 (8.9%)	14 (9.4%)
Caregiver stigma items	N = 254	N = 251
Q1. Negatively judged due to child’s HIV	30 (11.9%)	25 (10.0%)
Q2. Isolated/avoided due to child’s HIV	23 (9.1%)	21 (8.4%)
Q3. Called names/insulted due to child’s HIV	31 (12.4%)	27 (10.8%)
Q4a. Experienced discrimination at home due to child’s HIV	19 (7.5%)	20 (8.0%)
Q4b. Experienced discrimination in neighborhood due to child’s HIV	23 (9.1%)	26 (10.4%)
Q4c. Experienced discrimination at church due to child’s HIV	4 (1.6%)	6 (2.4%)
Q4d. Experienced discrimination at clinic due to child’s HIV	1 (0.4%)	2 (0.8%)
Q4e. Experienced discrimination at work due to child’s HIV	5 (2.0%)	3 (1.2%)
Q4f. Experienced discrimination at another place due to child’s HIV	3 (1.2%)	3 (1.3%)
Q5a. Lost financial support/work due to child’s HIV	15 (5.9%)	11 (4.4%)
Q5b. Lost social support due to child’s HIV	17 (6.8%)	12 (4.8%)
Q6a. HIV stigma has made me feel stressed/anxious	31 (12.3%)	33 (13.2%)
Q6b. HIV stigma has made me feel depressed/saddened	34 (13.5%)	34 (13.6%)
Q7. Not allowed child to play with others or go places due to child’s HIV	10 (4.0%)	12 (4.8%)
Q8. People in community think HIV is dirty/immoral/shameful
No one thinks	143 (56.8%)	150 (60.5%)
A few people think	62 (24.6%)	52 (21.0%)
Most people think	47 (18.7%)	46 (18.5%)
Q9. Child is isolated/avoided due to HIV	17 (6.8%)	22 (8.8%)
Q10. Child has been called names/bullied/insulted	18 (7.1%)	23 (9.2%)
Q11a. Child has experienced discrimination at home	15 (5.9%)	17 (6.8%)
Q11b. Child has experienced discrimination in neighborhood	19 (7.5%)	15 (6.0%)
Q11c. Child has experienced discrimination at church	2 (0.8%)	4 (1.6%)
Q11d. Child has experienced discrimination at clinic	0 (0%)	2 (0.8%)
Q11e. Child has experienced discrimination at school	12 (4.8%)	8 (3.2%)
Q11f. Child has experienced discrimination at another place	0 (0%)	1 (0.4%)
Q12a. HIV discrimination affected child socially	11 (4.3%)	8 (3.2%)
Q12b. HIV discrimination affected child emotionally	13 (5.1%)	24 (9.6%)
Q12c. HIV discrimination affected child’s education	10 (4.0%)	14 (5.6%)
Q13. Child’s future and hopes changed in a negative way	13 (5.2%)	14 (5.6%)

Reliability and Construct Validity

Test–retest reliability was high; responses by both children and caregivers on individual stigma items were highly consistent from month 18 to month 24 (agreement was >90% for the majority of stigma items). Child and caregiver stigma items showed high construct validity, with the SDQ and caregiver stigma items showed high construct validity with the GHAC General Health (Table 3). Several caregiver stigma items also showed construct validity with MEMS adherence and viral load. No child or caregiver stigma items were significantly associated with child- or caregiver-reported adherence using the CAMP Adherence Questionnaire, depression using the PHQ-9 questionnaire, or any GHAC domains outside of General Health (ie, no items correlated with physical functioning, psychological functioning, social and role, HIV, and health care).

Table 3.

Association of Stigma Questionnaire Items with Relevant Constructs.^a

Stigma Items	Association with Criterion Constructs
Stigma Items	MEMS Adherence	SDQ	GHAC General Health	Viral Load
Child stigma items
Q3a		+
Q4b		+
Q5a		+
Q5b		+
Q9		+
Caregiver stigma items
Q1	+
Q3		+
Q4f	−
Q5a		+
Q5b				−
Q6b		+	−
Q8				+
Q9			−
Q11a			−
Q11e			−
Q12c			−
Q13		+

Abbreviations: GHAC, General Health Assessment for Children; MEMS, Medication Event Monitoring Systems; PHQ-9, Patient Health Questionnaire–9 item; SDQ, 25-item Strengths and Difficulties Questionnaire.

^a± indicates direction of the slope for stigma items significantly associated with construct (P < .05). No stigma items were significantly associated with child- or caregiver-reported adherence using the CAMP Adherence Questionnaire or depression using the PHQ-9 questionnaire. Several stigma items were associated with the GHAC but only within the General Health subdomain (ie, no significant association with the other domains—physical functioning, psychological functioning, social and role, HIV, and health care).

On the child questionnaire, stigma items that showed high construct validity with the SDQ included experiencing discrimination at home due to HIV, losing social support, feeling stressed/anxious due to HIV stigma, feeling depressed/sad due to HIV stigma, and hopes for the future changing in a negative way due to HIV. Similar but not identical stigma items on the caregiver questionnaire showed high construct validity with the SDQ, including being called names/insulted due to the child’s HIV status, losing financial support/work due to the child’s HIV status, feeling depressed/sad due to HIV stigma, and that the child’s future hopes had changed in a negative way. All child and caregiver stigma items showed high construct validity in the same “direction”—that is, child and caregiver reports of stigma/discrimination or its negative effects on the stigma questionnaire were associated with poorer total difficulties scores on the SDQ (poorer emotional and behavioral health). Five caregiver stigma items showed high construct validity, with the GHAC General Health domain in the same direction as the results above. These items were related to feeling depressed/sad due to HIV stigma (which also showed validity with the SDQ), the child being isolated due to their HIV status, the child experiencing discrimination at home or at school, and HIV discrimination affecting the child’s education.

Agreement Between Child and Caregiver Responses

There were 8 items on the child and caregiver questionnaires that asked about HIV stigma that the child had experienced with the items being sufficiently similar to compare child and caregiver responses. Agreement was fairly high (between 84% and 100%), but not all κ statistics were statistically significant (Table 4). Overall, caregivers were slightly more likely to report that their child had experienced stigma when compared to child self-reports. For example, at month 18, only 3% of children reported being called names or bullied due to their HIV status but 9% of caregivers reported their child was called names or bullied.

Table 4.

Agreement Between Child- and Caregiver-Reported Individual Stigma Items.

Stigma Item	Month 18			Month 24
Stigma Item	κ	P Value	% Agreement	κ	P Value	% Agreement
Child lost friends due to HIV	.03	.7670	85.7	−.03	.6098	90.2
Child called names/bullied/insulted due to HIV	.09	.2857	84.0	.13	.0488	87.4
Child experienced discrimination at home due to HIV	.34	<.0001	92.4	.18	.0190	90.7
Child experienced discrimination in neighborhood due to HIV	.05	.5546	87.9	.11	.0825	92.0
Child experienced discrimination at church due to HIV	0	n/a	99.2	−.01	.8860	97.3
Child experienced discrimination at clinic due to HIV	n/a	n/a	100	0	n/a	98.0
Child experienced discrimination at school due to HIV	0.39	<.0001	92.9	0.53	<.0001	96.0
Child experienced discrimination at another place due to HIV	0	n/a	99.2	0	n/a	97.8

n/a: Not applicable.

Discussion

A minority but not insignificant proportion of children and caregivers reported fearing or experiencing HIV stigma among this cohort of families in western Kenya. Generally, caregivers tended to report high levels of both child-level and caregiver-level stigma. Our results support that the phenomenon of courtesy stigma, whereby caregivers may fear or directly experience stigma due to the status of their child (rather than their own status), is an important characteristic of HIV stigma in this setting, which can affect child outcomes both clinically and psychosocially.⁷⁷ Up to 1 in 10 caregivers in our cohort reported being negatively judged or isolated/avoided due to the child’s HIV status. In their qualitative study of HIV-related sigma in the United States, Bogart and colleagues referred to this phenomenon as the “interconnectedness of stigma experiences in families living with HIV.”⁷⁸ Our study supports the need for instruments to be attentive to the ways in which HIV stigma can impact families caring for children living with HIV.

The lower levels of stigma reports from children versus their caregivers deserve further investigation in this setting. Given many of the children’s younger age—average age was 12 years—it is possible that many children, even though they knew about their HIV status, were too young to recognize or connect their feelings and experiences to HIV-related stigma. It is also possible that because only children who were disclosed to were administered the questionnaire but all caregivers responded, the children who were yet to be disclosed to were more at risk of HIV stigma. Studies report a close association between caregivers’ fears of HIV stigma and nondisclosure to children,⁶² which we also found in our own focus groups with caregivers in this setting. Finally, children in this setting may be reluctant to report their own internal or external experiences of HIV stigma; about half of children reported that they felt it was important to keep their HIV status a secret. This may extend to keeping their own feelings and experiences related to HIV stigma private. Among adolescents in high-income settings, staying silent and keeping secrets about their experiences of living with HIV in different social contexts (of which the clinic or research study is one) is an important adolescent practice as both a protective and self-care strategy.⁷⁹ How these silences may operate in the context of children and adolescents living with HIV in Kenya need to be further explored.

One of the most routinely reported outcomes by both children and caregivers related to HIV stigma was experiencing anxiety, stress, depression, and a negative outlook for the future. This is consistent with other studies that showed that HIV stigma is associated with increased risks for mental health issues in adults^80
–82 and adolescents.^83,84 Stigma items related to experiencing anxiety, depression, and negative outlook showed high construct validity with the SDQ, an instrument designed to measure emotional and behavioral symptoms. Surprisingly, no child or caregiver stigma items showed significant construct validity with the PHQ-9, a common depression screening instrument. Anecdotal evidence from the research team in Kenya that administered the questionnaires to children and their caregivers reported in debrief meetings their particular dissatisfaction with the PHQ-9 instrument and that participants were more likely to have difficulty understanding and responding to these items during study assessments.

The finding that caregiver stigma items showed high construct validity by the GHAC General Health domain but not child stigma items is interesting. The GHAC General Health domain asks about the child’s overall health, physical health, emotional health, and daily activities on a 10-point scale from very worst to very best. This finding may also be related to children’s unwillingness to disclose physical and mental problems related to their HIV status or to the lack of sensitivity of the particular measure. We did not find strong construct validity on stigma items by adherence to treatment in this study. Quantitative adherence monitoring using the gold standard of electronic dose monitoring (MEMS) revealed significant levels of missed doses (eg, only 28% of children achieved >90% doses taken during follow-up). Interesting, a study among Thai youths living with HIV found while stigma reported using the HIV Stigma Scale was not associated with adherence by quantitative assessment (visual analog scale), stigma was associated with poorer adherence and missed doses by qualitative assessment (in-depth interviews).⁸⁵ In qualitative work in this setting, we have also documented a strong association between HIV stigma and child adherence to HIV medication.

There are a number of important limitations of this study to consider. First, as noted earlier, one major difficulty of validating HIV stigma by its connection to mental and behavioral health outcomes is the lack of validated instruments for children and adolescents in sub-Saharan Africa.⁸⁶ We could not find any research that rigorously evaluated the validity of the PHQ-9 in East Africa among children or adolescents, although the PHQ-9 was evaluated among adults living with HIV in western Kenya and showed good validity and reliability in assessing Diagnostic and Statistical Manual of Mental Disorders, 4th Edition (DSM-IV) depressive disorders.⁸⁷ A study in Zambia found high rates of emotional problems among adolescents living with HIV,⁸⁸ but a study evaluating the validity of the SDQ among HIV orphaned children in South Africa showed valid clinical cutoffs for caregivers but not for children.⁸⁹ Second, while both the SDQ and PHQ-9 have been translated into more than 60 languages, validation of the instruments into many languages (eg, Kiswahili) has not been investigated.⁹⁰ This validation study was tested in only one country in sub-Saharan Africa, and there may be additional language and cultural content challenges if using it in another setting. Nonetheless, the Academic Model Providing Access to Healthcare program in Kenya is clinically representative of many in sub-Saharan Africa,^91,92 and the study sites included urban, peri-urban, and rural clinical sites. Moreover, while participants had the choice to have the questionnaire administered in English or Kiswahili, almost all participants chose Kiswahili, which prevented us from comparing the validity of the questionnaire when it was administered in English versus Kiswahili. Additional work adapting and retesting this stigma instrument is ongoing. Third, enrollment in this study was done via convenience sampling. It is possible that children or their caregivers who were experiencing higher levels of HIV stigma would have been less inclined to enroll in the study, which required participation in study assessments and keeping medication in MEMS bottles. Additionally, the instrument was only tested in disclosed children, as the questionnaire directly asks about HIV. Finally, the stigma instrument was delivered by 8 research assistants (1 per clinic). To improve standardization of instrument administration, extensive training sessions and bimonthly meetings with the senior research team to discuss study assessments were conducted.

Conclusion

We provide evidence for the reliability and validity of a culturally tailored and patient-informed instrument to measure HIV-related stigma among Kenyan children and families. The stigma instrument showed good reliability and construct validity was high on child- and caregiver-reported emotional and behavioral health, with higher levels of reported HIV stigma associated with poorer mental health. While additional testing of this instrument is required, our study provides an important contribution to the limited literature on instruments to measure stigma among HIV-infected children and their families in resource-limited settings.

Footnotes

Authors’ Note

The views expressed in this article are those of the authors and do not necessarily represent the view of Icahn School of Medicine at Mount Sinai, Indiana University School of Medicine, University of Massachusetts Boston, or the Moi University School of Medicine. The primary author had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Acknowledgments

First and foremost, the authors acknowledge the invaluable contribution of time from the cohort of families who participated in this study and the health-care workers involved in their routine care. The authors also acknowledge the diligent work by the study’s counselors and research assistants.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by a grant “Stigma in AIDS Family Inventory” (3R01MH099747-01S1) to Dr Rachel Vreeman from the National Institute for Mental Health, Bethesda, MD, USA.

ORCID iD

Rachel Christine Vreeman

References

Kalichman

Simbayi

. HIV testing attitudes, AIDS stigma, and voluntary HIV counselling and testing in a black township in Cape Town, South Africa. Sex Transm Infect. 2003;79(6):442–447.

Obermeyer

Osborn

. The utilization of testing and counseling for HIV: a review of the social and behavioral evidence. Am J Public Health. 2007;97(10):1762–1774.

Steward

Bharat

Ramakrishna

Heylen

Ekstrand

. Stigma is associated with delays in seeking care among HIV-infected people in India. J Int Assoc Provid AIDS Care. 2013;12(2):103–109.

Sayles

Wong

Kinsler

Martins

Cunningham

. The association of stigma with self-reported access to medical care and antiretroviral therapy adherence in persons living with HIV/AIDS. J Gen Intern Med. 2009;24(10):1101–1108.

Kinsler

Wong

Sayles

Davis

Cunningham

. The effect of perceived stigma from a health care provider on access to care among a low-income HIV-positive population. AIDS Patient Care STDS. 2007;21(8):584–592.

Lifson

Demissie

Tadesse

, et al. Barriers to retention in care as perceived by persons living with HIV in rural Ethiopia: focus group results and recommended strategies. J Int Assoc Provid AIDS Care. 2013;12(1):32–38.

Rintamaki

Davis

Skripkauskas

Bennett

Wolf

. Social stigma concerns and HIV medication adherence. AIDS Patient Care STDS. 2006;20(5):359–368.

Kumarasamy

Safren

Raminani

, et al. Barriers and facilitators to antiretroviral medication adherence among patients with HIV in Chennai, India: a qualitative study. AIDS Patient Care STDS. 2005;19(8):526–537.

Vanable

Carey

Blair

Littlewood

. Impact of HIV-related stigma on health behaviors and psychological adjustment among HIV-positive men and women. AIDS Behav. 2006;10(5):473–482.

10.

Dlamini

Wantland

Makoae

, et al. HIV stigma and missed medications in HIV-positive people in five African countries. AIDS Patient Care STDS. 2009;23(5):377–387.

11.

Simbayi

Kalichman

Strebel

Cloete

Henda

Mqeketo

. Disclosure of HIV status to sex partners and sexual risk behaviours among HIV-positive men and women, Cape Town, South Africa. Sex transmit infect. 2007;83(1):29–34.

12.

Peretti-Watel

Spire

Obadia

Moatti

. Discrimination against HIV-infected people and the spread of HIV: some evidence from France. PloS one. 2007;2(5):e411.

13.

Lee

Kochman

Sikkema

. Internalized stigma among people living with HIV-AIDS. AIDS Behav. 2002:309–319.

14.

Clark

Lindner

Armistead

Austin

. Stigma, disclosure, and psychological functioning among HIV-infected and non-infected African-American women. Women Health. 2003;38(4):57–71.

15.

Thomas

Rehman

Suryanarayanan

, et al. How stigmatizing is stigma in the life of people living with HIV: a study on HIV positive individuals from Chennai, South India. AIDS care. 2005;17(7):795–801.

16.

Greeff

Uys

Wantland

, et al. Perceived HIV stigma and life satisfaction among persons living with HIV infection in five African countries: a longitudinal study. Int J Nurs Stud. 2010;47(4):475–486.

17.

Deacon

Stephney

. HIV/AIDS, Stigma and Children: A Literature Review. Cape Town, South Africa: Human Sciences Research Council; 2007.

18.

Joint United Nations Programme on HIV/AIDS. Children and HIV Fact Sheet. Geneva, Switzerland: Joint United Nations Programme on HIV/AIDS; 2016.

19.

Chase

Aggleton

. Stigma, HIV/AIDS and Prevention of Mother-to-Child Transmission: A Pilot Study in Zambia, India, Ukraine and Burkina Faso. London, United Kingdom: United Nations Children’s Fund/Panos Institute; 2001.

20.

Baggaley

Sulwe

Chilala

Mashambe

. HIV stress in primary school teachers in Zambia. Bull World Health Organ. 1999;77(3):284–287.

21.

Mahati

Chandiwana

Munyati

, et al. A Qualitative Assessment of Orphans and Vulnerable Children in two Zimbabwean Districts. Cape Town, South Africa: HSRC Press; 2006.

22.

Alliance

StC

. Mitigation of HIV/AIDS Impact on Orphans, Vulnerable Children and Their Families. London: Save the Children Alliance and Hope for African Children Initiative; 2005.

23.

Dago-Akribi

Cacou Adjoua

. Psychosexual development among HIV-positive adolescents in Abidjan, Cote d’ivoire. Reproduc Health Matt. 2004;12(23):19–28.

24.

O’Hare

Venables

Nalubeg

Nakakeeto

Kibirige

Southall

. Home-based care for orphaned children infected with HIV/AIDS in Uganda. AIDS Care. 2005;17(4):443–450.

25.

Eisenberg

Fabes

Guthrie

. Coping with stress: the roles of regulation and development. In: Wolchik

Sandler

, eds. Handbook of Children’s Coping: Linking Theory and Intervention. New York, NY: Plenum Press; 1997.

26.

Rebhun

. Sexuality, color, and stigma among northeast Brazilian women. Med Anthropol Q. 2004;18(2):183–199.

27.

Mawn

Raising a child with HIV: an emerging phenomenon. Famil Syst Health. 1999;17(2):197–215.

28.

Poindexter

. ‘It don’t matter what people say as long as I love you’: experiencing stigma when raising an HIV-infected grandchild. J Mental Health Agin. 2002;8(4):331–348.

29.

Strode

Barrett-Grant

. The Role of Stigma and Discrimination in Increasing Vulnerability of Children and Youth Infected with and Affected with HIV/AIDS. London, United Kingdom: Save the Children; 2001.

30.

Robertson

Ensink

. The psychosocial impact of human immunodeficiency virus infections in children. J Child Adolesc Ment Health. 1992;4(2):46–49.

31.

Vreeman

Nyandiko

Ayaya

Walumbe

Marrero

Inui

. Factors sustaining pediatric adherence to antiretroviral therapy in western Kenya. Qual Health Res. 2009;19(12):1716–1729.

32.

Monasch

Boerma

. Orphanhood and childcare patterns in sub-Saharan Africa: an analysis of national surveys from 40 countries. AIDS. 2004;18(suppl 2):S55–S65.

33.

Giese

Meintjes

Croke

Chamberlain

. Health and Social Services to Address the Needs of Orphans and Other Vulnerable Children in the Context of HIV/AIDS: Research Report and Recommendations Report Submitted to HIV/AIDS Directorate, National Department of Health. Cape Town, South Africa: Children’s Institute, University of Cape Town; 2003.

34.

Davids

Letlape

Magome

, et al. A Situational Analysis of Orphans and Vulnerable Children in Four Districts of South Africa. Cape Town, South Africa: HSRC Press; 2006.

35.

Biadgilign

Deribew

Amberbir

Deribe

. Barriers and facilitators to antiretroviral medication adherence among HIV-infected paediatric patients in Ethiopia: a qualitative study. Sahara J. 2009;6(4):148–154.

36.

Bikaako-Kajura

Luyirika

Purcell

, et al. Disclosure of HIV status and adherence to daily drug regimens among HIV-infected children in Uganda. AIDS Behav. 2006;10(suppl 4):S85–S93.

37.

Blasini

Chantry

Cruz

, et al. Disclosure model for pediatric patients living with HIV in Puerto Rico: design, implementation, and evaluation. J Dev Behav Pediatr. 2004;25(3):181–189.

38.

Corneli

Vaz

Dulyx

Omba

Rennie

Behets

. The role of disclosure in relation to assent to participate in HIV-related research among HIV-infected youth: a formative study. J Int AIDS Soc. 2009;12(1):17.

39.

Kallem

Renner

Ghebremichael

Paintsil

. Prevalence and pattern of disclosure of HIV status in HIV-infected children in Ghana. AIDS Behav. 2011;15(6):1121–1127.

40.

Oberdorfer

Puthanakit

Louthrenoo

Charnsil

Sirisanthana

. Disclosure of HIV/AIDS diagnosis to HIV-infected children in Thailand. J Paediatr Child Health. 2006;42(5):283–288.

41.

Punpanich

Detels

Gorbach

Leowsrisook

. Understanding the psychosocial needs of HIV-infected children and families: a qualitative study. J Med Assoc Thai. 2008;91(suppl 3):S76–S84.

42.

Vaz

Corneli

Dulyx

, et al. The process of HIV status disclosure to HIV-positive youth in Kinshasa, Democratic Republic of the Congo. AIDS Care. 2008;20(7):842–852.

43.

Vaz

Eng

Maman

Tshikandu

Behets

. Telling children they have HIV: lessons learned from findings of a qualitative study in sub-Saharan Africa. AIDS Patient Care STDS. 2010;24(4):247–256.

44.

Blinded.

45.

Hejoaka

. Care and secrecy: being a mother of children living with HIV in Burkina Faso. Soc Sci Med. 2009;69(6):869–876.

46.

Moodley

Myer

. Paediatric HIV disclosure in South Africa—caregivers’ perspectives on discussing HIV with infected children. S Afr Med J 2006;96(3):201–204.

47.

Domek

. Debunking common barriers to pediatric HIV disclosure. J Trop Pediatr. 2010;56(6):440–442.

48.

Abadia-Barrero

Larusso

. The disclosure model versus a developmental illness experience model for children and adolescents living with HIV/AIDS in Sao Paulo, Brazil. AIDS Patient Care STDS. 2006;20(1):36–43.

49.

Kouyoumdjian

Meyers

Mtshizana

. Barriers to disclosure to children with HIV. J Trop Pediatr. 2005;51(5):285–287.

50.

Yeap

Hamilton

Charalambous

, et al. Factors influencing uptake of HIV care and treatment among children in South Africa—a qualitative study of caregivers and clinic staff. AIDS care. 2010;22(9):1101–1107.

51.

Nyblade

MacQuarrie

. Can We Measure HIV Stigma and Discrimination? Current Knowledge about Quantifying Stigma in Developing Countries. Washington, DC: Joint United Nations Programme on HIV/AIDS; 2006.

52.

Nyblade

. Measuring HIV stigma: existing knowledge and gaps. Psycholo Health Med. 2006;11(3):335–345.

53.

Sengupta

Banks

Jonas

Miles

Smith

. HIV interventions to reduce HIV/AIDS stigma: a systematic review. AIDS Behav. 2011;15(6):1075–1087.

54.

Blinded.

55.

Steward

Herek

Ramakrishna

, et al. HIV-related stigma: adapting a theoretical framework for use in India. Soc Sci Med. 2008;67(8):1225–1235.

56.

Earnshaw

Chaudoir

. From conceptualizing to measuring HIV stigma: a review of HIV stigma mechanism measures. AIDS Behav. 2009;13(6):1160–1177.

57.

Wight

Aneshensel

Murphy

Miller-Martinez

Beals

. Perceived HIV stigma in AIDS caregiving dyads. Soc Sci Med. 2006;62(2):444–456.

58.

Hambleton

Merenda

Spielberger

, eds. Adapting Educational and Psychological Tests for Cross-Cultural Assessment. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers; 2005.

59.

Vreeman

Nyandiko WM Liu

, et al. Measuring adherence to antiretroviral therapy in children and adolescents in western Kenya. J Int AIDS Soc. 2014;17:19227.

60.

Vreeman

Nyandiko

Liu

, et al. Comprehensive evaluation of caregiver-reported antiretroviral therapy adherence for HIV-infected children. AIDS and Behavior. 2015;19(4):626–634.

61.

Zeunik

Nyandiko

Inui

Ayaya

Merrero

Vreeman

. A qualitative assessment of barriers to antiretroviral therapy adherence among adolescents in western Kenya. Unpublished; 2011.

62.

Vreeman

Gramelspacher

Gisore

Scanlon

Nyandiko

. Disclosure of HIV status to children in resource-limited settings: a systematic review. J Int AIDS Soc. 2013;16(1):18466.

63.

Vreeman

Wiehe

Pearce

Nyandiko

. A systematic review of pediatric adherence to antiretroviral therapy in low-and middle-income countries. Ped Infect Dis J. 2008;27(8):686–691.

64.

Vreeman

Nyandiko

Ayaya

Walumbe

Inui

. Cognitive interviewing for cross-cultural adaptation of pediatric antiretroviral therapy adherence measurement items. Int J Behav Med. 2014;21(1):186–196.

65.

Einterz

Kimaiyo

Mengech

, et al. Responding to the HIV pandemic: the power of an academic medical partnership. Acad Med. 2007;82(8):812–818.

66.

Inui

Nyandiko

Kimaiyo

, et al. AMPATH: living proof that no one has to die from HIV. J of Gen Int Med. 2007;22(12):1745–1750.

67.

Vreeman

Scanlon

Marete

, et al. Characteristics of HIV-infected adolescents enrolled in a disclosure intervention trial in western Kenya. AIDS Care. 2015;27(Suppl 1):6–17.

68.

Gortmaker

Lenderkin

Clark

Development and use of a pediatric quality of life questionnaire in AIDS clinical trials: reliability and validity of the General Health Assessment for Children (GHAC). In: Drotar

, ed. Measuring Health-Related Quality of Life in Children and Adolescents: Implications for Research, Practice, and Policy. Nahwah, NJ: Lawrence Erlbaum Associates; 1998:219–235.

69.

Lee

Gortmaker

McIntosh

Hughes

Oleske

; Pediatric AIDS Clinical Trials Group Protocol 219C Team. Quality of life for children and adolescents: impact of HIV infection and antiretroviral treatment. Pediatrics. 2006;117(2):273–283.

70.

Lenderking

Testa

Katzenstein

Hammer

. Measuring quality of life in early HIV disease: the modular approach. Qual Life Res. 1997;6(6):515–530.

71.

Goodman

Meltzer

Bailey

. The Strengths and Difficulties Questionnaire: a pilot study on the validity of the self-report version. Eur Child Adolesc Psychiatry. 1998;7(3):125–130.

72.

Goodman

Renfrew

Mullick

. Predicting type of psychiatric disorder from Strengths and Difficulties (SDQ) scores in child mental health clinics in London and Dhaka. Eur Child Adolesc Psychiatry. 2000;9(2):129–134.

73.

Kroenke

Spitzer

Williams

. The PHQ-9: validity of a brief depression severity measure. J Gener Int med. 2001;16(9):606–613.

74.

Bele

Valsangkar

Bodhare

. Impairment of nutritional, educational status, and quality of life among children infected with and belonging to families affected by HIV/AIDS. Vulnerable Child Youth Stud. 2011;6:284–292.

75.

Butler

Williams

Howland

, et al. Impact of disclosure of HIV infection on health-related quality of life among children and adolescents with HIV infection. Pediatrics. 2009;123(3):935–943.

76.

Richardson

McCauley

Grossman

, et al. Evaluation of the patient health questionnaire-9 Item for detecting major depression among adolescents. Pediatrics. 2010;126(6):1117–1123.

77.

Sherr

Skeen

Hensels

Tomlinson

Macedo

. The effects of caregiver and household HIV on child development: a community-based longitudinal study of young children. Child Care Health Dev. 2016;42(6):890–899.

78.

Bogart

Cowgill

Kennedy

, et al. HIV-related stigma among people with HIV and their families: a qualitative analysis. AIDS and behavior. 2008;12(2):244–254.

79.

Fielden

Chapman

Cadell

. Managing stigma in adolescent HIV: silence, secrets and sanctioned spaces. Cult Health Sex. 2011;13(3):267–281.

80.

Charles

Jeyaseelan

Pandian

Sam

Thenmozhi

Jayaseelan

. Association between stigma, depression and quality of life of people living with HIV/AIDS (PLHA) in South India—a community based cross sectional study. BMC Public Health. 2012;12:463.

81.

Lee

Thammawijaya

Jiraphongsa

Rotheram-Borus

. Stigma, social support, and depression among people living with HIV in Thailand. AIDS care. 2009;21(8):1007–1013.

82.

Simbayi

Kalichman

Strebel

Cloete

Henda

Mqeketo

. Internalized stigma, discrimination, and depression among men and women living with HIV/AIDS in Cape Town, South Africa. Soc Sci Med. 2007;64(9):1823–1831.

83.

Cluver

Gardner

Operario

. Effects of stigma on the mental health of adolescents orphaned by AIDS. J Adolesc Health. 2008;42(4):410–417.

84.

Tanney

Naar-King

MacDonnel

; Adolescent Trials Network for HIVAIPT. Depression and stigma in high-risk youth living with HIV: a multi-site study. J Pediatr Health Care. 2012;26(4):300–305.

85.

Fongkaew

Viseskul

Suksatit

, et al. Verifying quantitative stigma and medication adherence scales using qualitative methods among Thai youth living with HIV/AIDS. J Int Assoc Provid AIDS Care. 2014;13(1):69–77.

86.

Blinded.

87.

Monahan

Shacham

Reece

, et al. Validity/reliability of PHQ-9 and PHQ-2 depression scales among adults living with HIV/AIDS in western Kenya. J Gene Int Med. 2009;24(2):189–197.

88.

Menon

Glazebrook

Campain

Ngoma

. Mental health and disclosure of HIV status in Zambian adolescents with HIV infection: implications for peer-support programs. J Acquir Immune Defic Syndr. 2007;46(3):349–354.

89.

Sharp

Venta

Marais

Skinner

Lenka

Serekoane

. First evaluation of a population-based screen to detect emotional-behavior disorders in orphaned children in sub-Saharan Africa. AIDS Behav. 2014;18(6):1174–1185.

90.

Stolk

Kaplan

Szwarc

. Review of the strengths and difficulties translated into languages spoken by children and adolescents of refugee background. Int J Methods Psychiatr Res. 2017. doi:10.1002/mpr.1568.

91.

International Epidemiologic Database to Evaluate AIDS. A survey of paediatric HIV programmatic and clinical management practices in Asia and sub-Saharan Africa—the international epidemiologic databases to evaluate AIDS (IeDEA). J Int AIDS Soc. 2013;16:17998.

92.

Leroy

Malateste

Rabie

, et al. Outcomes of antiretroviral therapy in children in Asia and Africa: a comparative analysis of the IeDEA Pediatric Multiregional Collaboration. J Acquir Immune Defic Syndr. 2013;62(2):208–219.

Validation of an HIV/AIDS Stigma Measure for Children Living with HIV and Their Families

Abstract

Background:

Methods:

Results:

Conclusions:

Keywords

What Do We Already Know About This Topic?

How Does Your Research Contribute to the Field?

What Are Your Research’s Implications Toward Theory, Practice, or Policy?

Introduction

Methods

Study Design

Ethical Statement

Analysis

Results

Participants’ Characteristics

Responses to the Stigma Questionnaire

Reliability and Construct Validity

Agreement Between Child and Caregiver Responses

Discussion

Conclusion

Footnotes

Authors’ Note

Acknowledgments

Declaration of Conflicting Interests

Funding

ORCID iD

References