Sage Journals: Discover world-class research

Abstract

A Patient-Reported Outcome (PRO) measure titled Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral (acronym CEAT-VIH) is currently available in paper-and-pencil and digital (online assessment) formats. Due to the advantages of online assessment, the main objective of this work was to evaluate psychometric properties of the online version, in an international sample, to accumulate evidence of its validity and provide score norms for the questionnaire. A psychometric study was performed with an international sample of 1,470 participants, from 30 countries, to accumulate evidence of CEAT-VIH validity regarding internal structure and related external criterion (e.g., viral load, number of pills, and AIDS-related symptoms). Descriptive statistics and normative data for scores are also presented. The majority of participants were men (72.4%), aged 15 to 78 years old (M = 39.3, SD = 12.6). A unidimensional model with five facets occurred as the observed variables converged, which presented a good model fit (comparative fit index [CFI] = 1.000; Tucker–Lewis index [TLI] = 0.999; standardized root mean square residual [SRMR] = 0.027; and root mean square error of approximation [RMSEA] [90% confidence interval, CI] = 0.009 [0.000, 0.038], p = .995). There was a weak invariance for the CEAT-VIH structure for language versions and countries. Cronbach’s alpha values for the instrument (17 items) were acceptable across language versions (.88-.96). Evidence of validity related to external criteria was achieved by associations (e.g., Spearman and Mann–Whitney) between CEAT-VIH scores and relevant clinical (e.g., CD4+ cells, viral load, number of pills, and AIDS-related symptoms) and sociodemographic (e.g., gender, age, employment status, education level, place of residence, and participation at local AIDS association) variables. In conclusion, the overall data on the evaluated psychometric properties allow recommendation of the use of this instrument in research and applied settings.

Keywords

HIV adherence antiretroviral therapy assessment psychometrics CEAT-VIH

Introduction

Adherence to HIV treatment is a complex and multifaceted behavior (Remor, 2013b), and can be understood as the extent to which one’s behavior coincides with the health recommendations he or she receives (Haynes, Taylor, & Sackett, 1979). However, associations between adherence levels and health outcomes are complex; many mediators may be implicated, including time in treatment, psychological variables, quality of the provided treatment, patient age, and viral load levels (Costa, Torres, Coelho, & Luz, 2018; Marks et al., 2015; Remor, Penedo, Shen, & Schneiderman, 2007). Psychosocial aspects should also be investigated, because they may act as barriers or facilitators to the adherence process (Biello et al., 2016; Costa et al., 2018; Dima, Schweitzer, Diaconiţ, Remor, & Wanless, 2013). The combination of drug treatments and psychosocial approaches allows for various vulnerabilities to which people with HIV are exposed to be addressed. Non-adherence or low adherence to treatment is a threat to treatment effectiveness, and they contribute to elevated viral load, the incidence of opportunistic diseases, and an increase in the number of hospital admissions (e.g., Foresto et al., 2017).

To contribute with an adequate assessment of adherence, a self-report measure titled Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral (CEAT-VIH, acronym in the original) (or “Assessment of Adherence to Antiretroviral Therapy Questionnaire” in a free translation) was developed to evaluate adherence to antiretroviral treatment for people with HIV. The original instrument was developed by Eduardo Remor between 1999 and 2001 in Spanish (Remor, 2002), and later adapted to other languages (e.g., Portuguese, Brazilian Portuguese, Romanian, and English; Dima et al., 2013; Reis, Lencastre, Guerra, & Remor, 2009; Remor, 2013a; Remor, Milner-Moskovics, & Preussler, 2007). Since the first publication (Remor, 2002), the instrument has been used in several studies (see Remor, 2013b, for a review). A review of the literature on the psychometric properties of the questionnaire, in its paper-and-pencil version across 20 independent studies in different countries, showed that the questionnaire achieved good psychometrics in terms of reliability, no floor and ceiling effects, evidence of criterion-related validity, responsiveness to intervention programs, sensitivity, specificity, and evidence of validity related to patterns of convergence and divergence (Remor, 2013b). Furthermore, subsequent studies applied the instrument and reported its usefulness for assessing adherence to antiretroviral treatment (e.g., Calvetti, Giovelli, Gauer, & Moraes, 2014; Foresto et al., 2017; Nogueda-Orozco et al., 2016; Salmanton-García et al., 2015; Silva, Reis, Nogueira, & Gir, 2014; Tello-Velásquez et al., 2015; Zuge et al., 2017). A recent meta-analysis (Costa et al., 2018) identified the CEAT-VIH as the second most common measure included in Latin American studies that address adherence behavior.

Given the extensive use of the instrument in its paper-and-pencil version, its predictive ability related to viral load levels, and ability to identify patients struggling with HIV treatment adherence (Remor, 2013b), the CEAT-VIH author (refers to Remor, 2013a) decided to develop an online version of the instrument to facilitate its use, namely, to provide accurate and automatic score results and graphical feedback to clinicians and patients (Remor, 2013a). In addition, the adaptation to an online version amplifies its potential to be used, and easily integrated, to assess interventions that focus on improving HIV adherence. One of the innovations of the digital version was to provide additional scores for subscales that underlie the pool of items, based on the theoretical development of the instrument (Dima et al., 2013; Remor, 2013a, 2013b). The subscales show scores on five major indicators (facets) that explain individual differences in adherence behavior: Compliance, Antecedents of Non-Adherence Behaviors, Doctor–Patient Communication, Personal Beliefs, and Expectations About Treatment and Treatment Satisfaction.

A few studies already utilized the online CEAT-VIH version. Table 1 summarizes the psychometric information identified in these publications (Conz, 2015; Herranz-Alvarez, Ríos-Maldonado, & Hernández, 2017; Neves, 2017; Padilla, 2016). Overall, the studies that evaluated the online version found an acceptable level of reliability (Cronbach’s alpha: α > .70; Urbina, 2004) and no floor and ceiling effects (Table 1). There was criterion-related evidence with viral load, presence of side effects, AIDS symptoms, and body mass index; score responsiveness after intervention or predicting patients’ abandonment were identified; and patterns of convergence and divergence with other instruments were observed (i.e., 20-item Self-Reporting Questionnaire [SRQ-20], HIV/AIDS-targeted quality of life [HAT-QoL], Beck Depression Inventory [BDI]; see Table 1 for details).

Table 1.

Summary of Psychometric Properties Assessed by Research Reports Using the Online Version of the CEAT-VIH.

Reference (year), Country, Sample size (N)	Age (M, range); Sex (n)	CEAT-VIH raw score, M (SD), minimum-maximum	Floor/ceiling effects	Reliability (Cronbach’s α)	Evidences of validity
Reference (year), Country, Sample size (N)	Age (M, range); Sex (n)	CEAT-VIH raw score, M (SD), minimum-maximum	Floor/ceiling effects	Reliability (Cronbach’s α)	Criterion-related (clinical)	Responsiveness	Patterns of convergence and divergence (r)
Conz (2015), Brazil, N = 92	42.1, 19-62; F (51), M (41)	59.7 (7.0), 40-72	0%/0%	.70	Viral load: r = −.33; p = .001Presence of side effects: r = −.21; p = .043	Abandonment of health care service (yes/no); U = 556.5; p = .002	SRQ-20 (−.41)HAT-QoL (.70)
Padilla (2016), Venezuela, N = 209	NE, 18-78; F (42), M (167)	74.7 (7.3), 31-84	0%/0%	.81	AIDS symptoms (Yes = lower adherence score/No = higher adherence score) t = 2.97; p = .003Viral load (≤3.000 copies/ml vs. ≥3.001 copies/ml) t = 2.45; p = .015	NE	NE
Neves (2017), Brazil, N = 10	32.8, 21-48; F (5), M (5)	^a	NE	NE	Viral load (BL): r_s = −.24; p > .05	Increase of regularity taking meds at pharmacy after feedback with CEAT-VIH (yes/no); U = .00, p = .016 (effect size = .75)	NE
Herranz-Alvarez, Ríos-Maldonado, and Hernández (2017), Venezuela, N = 100	38.9, 22-72, F (25), M (75)	73.2 (9.7), 50-85	0%/3.1%	.85	BMI: r = .28; p = .027	NE	BDI (–.59)

Note. CEAT-VIH = Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral; M = mean; n = number of respondents; F = female, M = male; SD = standard deviation; r = Pearson’s correlation; t = Student’s t test; NE = not evaluated; BL = baseline; r_s = Spearman’s correlation; U = Mann–Whitney U test; BMI = body mass index. Instruments: SRQ-20 = 20-item Self-Reporting Questionnaire; HAT-QoL = HIV/AIDS-targeted quality of life; BDI = Beck Depression Inventory.

Not reported.

The main objective of this article was to increase information about the psychometric properties of the instrument, in a larger international sample, by evaluating the internal structure of the online version of the CEAT–VIH questionnaire, and describe additional validity evidence (e.g., criterion related). To achieve these aims, a confirmatory factor analysis (CFA) and two multigroup confirmatory factor analyses (MGCFAs) were conducted to demonstrate the dimensionality of the instrument. In addition, to verify and accumulate validity evidence, hypotheses of association among the instrument and both relevant external clinical criteria and sociodemographic profiles were tested. Score standardization by language version and gender by country groups was also performed.

Method

Participants

The sample consisted of 1,470 participants from different countries (15% participating in local AIDS organizations). Ages ranged from 15 to 78 years (mean [M] = 39.3, standard deviation [SD] = 12.6, median [Mdn] = 38). Furthermore, 72.4% of participants were male, 64.6% were working, and 70.7% lived in large cities or capitals. The full characterization of the sample and percentage of participants by country are summarized in detail in Table 2. Regarding HIV infection treatment, the majority of the participants were taking one (25.8%) or three (31.8%) pills per day (M = 2.9, SD = 1.7, Mdn = 3), and 82.5% were free from AIDS-related symptoms (opportunistic diseases). Participants were HIV+ from less than one to 37 years (M = 10.6, SD = 6.6, Mdn = 9). The mean CD4+ lymphocyte cells count was 506.6 (SD = 355.8, Mdn = 465), and the average viral load was 12,372.3 copies/ml (SD = 64,260.6, Mdn = 100). Almost half (45.1%) of the participants had an undetectable viral load (less than 50 copies/ml).

Table 2.

Characteristics of the Participants in the Present Study.

Variable	N = 1,470	%
Gender
Male	1,064	72.4
Female	406	27.6
Response language
Spanish	955	65.0
Brazilian Portuguese	326	22.5
Portuguese	136	9.3
English	53	3.6
Marital status
Single	868	59.0
Married (or cohabitation)	423	28.8
Separated or divorced	98	6.7
Widowed	81	5.5
Occupation
Unemployed	384	26.1
Working	949	64.6
Studying	99	6.7
Working and studying	38	2.6
Education
None	48	3.3
Elementary school	302	20.5
High school	616	41.9
Undergraduate school	417	28.4
Graduate school	87	5.9
Residence
Large city or capital	1,039	70.7
Small town	376	25.6
Countryside (non-urban)	55	3.7
Perceived economic status
Something worse than others	285	19.4
Equal to others	1,019	69.3
Something better than others	140	9.5
Better than the others	26	1.8
Countries
Other^a	27	1.8
Argentina	14	1.0
Brazil	321	21.8
Chile	14	1.0
Colombia	16	1.1
Ecuador	64	4.4
Spain	14	1.0
The United States	6	0.4
Honduras	11	0.7
Mexico	573	39.0
Peru	41	2.8
Portugal	142	9.7
Turkey	7	0.5
Venezuela	220	15.0

Countries with less than five participants (i.e., Afghanistan, Belgium, Bolivia, Canada, Czech Republic, Philippines, France, Georgia, Guatemala, India, Ireland, Nigeria, Dominican Republic, Russia, South Africa, Taiwan, Vietnam).

Instruments

CEAT-VIH online version

The online version of the CEAT-VIH¹ is a patient-reported outcome (PRO) measure, brief and easy to answer, that assesses adherence to antiretroviral therapy from a multidimensional perspective (Remor, 2013a, 2013b). During the adaptation process of the paper-and-pencil version of the instrument to the online version, three items with a dichotomous response scale were excluded, based on the following considerations: (a) these items showed low item-total correlation in previous studies (e.g., Dima et al., 2013) and (b) their format made it difficult to score the instrument because they included qualitative responses.

Thus, the CEAT-VIH online version is composed of 17 items with a 5-point Likert-type scale for responses, distributed into five facets, according to the theoretical structure of the instrument: Compliance (three items) indicates the extent to which the behavior of the person reflects strict medication-taking adherence; Antecedents of Non-Adherence Behaviors (four items) indicates the extent to which personal or situational antecedents are related to non-adherence behavior; Doctor–Patient Communication (three items) indicates the perceived quality of the doctor–patient relationship as a motivational reinforcement to treatment adherence; Personal Beliefs/Expectancies About the Treatment (five items) indicates the extent to which the patient’s beliefs and expectations affect treatment adherence behavior; and Treatment Satisfaction (two items) indicates the person’s degree of satisfaction with treatment outcomes. In addition to the scores for each facet, the instrument also provides the summary adherence score that indicates the degree of overall adherence to antiretroviral treatment, which combines all five mentioned facets. Regarding the summary adherence score, the raw score can range from a minimum of 17 to a maximum of 85 points (the higher the score, the greater the treatment adherence).

The online interface allows automatic correction, and it generates standardized scores (from the raw scores) that range from 0 to 100 for each of the facets and the summary adherence score to facilitate users’ interpretation of their scores. The information is shown in a graphic format and can be sent by email. These features allow professionals and patients to map which aspects promote adherence and which are barriers that need to be ameliorated with therapeutic action and/or counseling.

Sociodemographic and clinical variables

Sociodemographic data, including gender, age, education, employment status, perceived socioeconomic status (SES; 1-4), country, marital status, place of residence (large city, small city, countryside [non-urban]), participation in local AIDS association (yes/no), and clinical data, such as time of treatment, number of CD4+ cells, viral load, number of pills, and presence of AIDS-related symptoms or opportunistic diseases (yes/no), were collected (see Table 2).

Procedures

The multilanguage CEAT-VIH online version is available at http://www.ceat-vih.info/ (compatible with mobile devices). The data used in the present study were collected through the website. Two basic types of access are observed: (a) responses directly from patients interested in their adherence self-evaluation (in some cases, patients access the CEAT-VIH website after recommendation from health professionals or non-governmental organization [NGO] personnel interested in how they are coping with adherence to medical therapy) or (b) researchers who use the instrument as a measure for adherence in their study. Researchers authorized for using the online version completed a form that explicitly granted permission for the data to be transferred to the author for psychometric analysis. Moreover, each researcher was responsible for submitting their study to their institution’s ethics and research committee.

Ethical Aspects

The study was conducted in compliance with the Code of Ethics of the World Medical Association (Declaration of Helsinki). The participants did not receive any payment for their participation. Before answering the instrument, an initial page with a digital consent form was presented, and consent to the research terms was provided by clicking the “I have read and understand the terms and conditions of this agreement” button. The study was approved by the Institutional Review Board of the Institute of Psychology, Universidade Federal do Rio Grande do Sul.

Data Analysis Plan

The statistical analyses were conducted using PASW Statistics for Windows, Version 18.0 software (SPSS, Inc., Chicago, IL, USA), for descriptive statistics, skewness and kurtosis, reliability (internal consistency was measured by Cronbach’s alpha), correlations, and statistical hypothesis tests. In addition, R Software (R Core Team, R Foundation, Vienna, Austria) was used for CFA and MGCFAs, and model fit indices.

No missing values were found in the questionnaire responses. The skewness (sk) and kurtosis (ku) of all individual items showed values considered to not be normally distributed: –2.4 ≤ sk ≤ −0.2 and −0.2 ≤ ku ≤ 5.6. To confirm the theoretical structure of the CEAT-VIH online version, CFA was employed (weighted least squares means and variance adjusted [WLSMV]; Brown, 2015). The considered model fit indices were chi-square (χ²) and degrees of freedom (df), incremental fit indices (comparative fit index [CFI] and Tucker–Lewis index [TLI]), and residual adjustment indices (standardized root mean square residual [SRMR] and root mean square error of approximation [RMSEA]). The models were evaluated based on CFI (>0.90), TLI (>0.90), and RMSEA (<0.08, with the 90% confidence interval [CI] not exceeding 0.10) fit indices. Measurement invariance was assessed based on the CFI difference values between the models (ΔCFI; Damásio & DeSousa, 2015). After fitting the model separately, in a second step, two MGCFAs were conducted that aimed to investigate the invariance of CEAT-VIH by the questionnaire language version and country of residence. This analysis requires at least 100 individuals per group per variable (Myers, Ahn, & Jin, 2011). To accomplish the MGCFA by country, a separate database was created with individuals only from Brazil, Mexico, Portugal, and Venezuela (n = 1,256), and for the language analysis, another database was created with individuals that responded to the questionnaire in Spanish, European Portuguese, or Brazilian Portuguese (n = 1,417). Measurement invariance was achieved if ΔCFI was <0.02 and ΔRMSEA was <0.03 for tests of metric invariance, and the traditional criteria of <–0.01 for both ΔCFI and ΔRMSEA were used for scalar invariance tests (Putnick & Bornstein, 2016; Rutkowski & Svetina, 2014). In addition, Cronbach’s alpha was used to evaluate internal consistency within the complete sample. Evidence for criterion-related validity was examined for the complete sample, including relevant clinically related variables and sociodemographic characteristics. Descriptive statistics and percentile norms by language version and gender by country are also presented.

Results

CFA to Assess Construct Validity and Dimensionality of the Instrument

The CFA was used to evaluate the goodness of fit of the theoretical model that supports the CEAT-VIH online version. The unidimensional model, composed of a single adherence factor with the five facets as observed variables, converged after 46 iterations and presented the following fit indices: χ² = 1,665.072 (df = 10, p < .001); CFI = 1.000; TLI = 0.999; RMSEA [90% CI] = 0.009 [0.000, 0.038], p = .995; SRMR = 0.027. All fit indices indicated that the final CEAT-VIH measurement model was appropriate for the data. Table 3 contains the full details of this analysis.

Table 3.

Results for the CFA of the Unifactorial Model (n = 1,470) and MGCFA by Countries (n = 1,256) and Language (n = 1,417) of CEAT-VIH Online Version.

Model	χ² (df)	CFI	ΔCFI	TLI	SRMR	RMSEA	ΔRMSEA	90% CI
Unifactorial model	1,665.072 (10)*	1.000		0.999	0.027	0.009		[0.000, 0.038]
Country measurement invariance
Brazil (n = 321)	299.092 (10)*	1.000		1.009	0.041	0.000		[0.000, 0.066]
Mexico (n = 573)	662.039 (10)*	0.997		0.994	0.038	0.026		[0.000, 0.068]
Portugal (n = 142)	277.079 (10)*	1.000		1.036	0.013	0.000		[0.000, 0.000]
Venezuela (n = 220)	182.447 (10)*	1.000		1.043	0.034	0.000		[0.000, 0.024]
Configural invariance	140.37 (20)*	0.961	—	—	—	0.138	—	—
Metric invariance	210.20 (32)*	0.942	0.019	—	—	0.133	0.005	—
Scalar invariance	409.67 (44)*	0.881	0.061	—	—	0.163	0.029	—
Language measurement invariance
Spanish (n = 955)	1,104.635 (10)*	0.999		0.998	0.031	0.017		[0.000, 0.050]
Brazilian Portuguese (n = 326)	311.960 (10)*	1.000		1.015	0.037	0.000		[0.000, 0.055]
International Portuguese (n = 136)	246.614 (10)*	1.000		1.041	0.014	0.000		[0.000, 0.000]
Configural invariance	163.32 (20)*	0.965	—	—	—	0.140	—	—
Metric invariance	205.08 (32)*	0.958	0.010	—	—	0.121	0.019	—
Scalar invariance	309.06 (44)*	0.935	0.023	—	—	0.128	−0.007	—

Note. CFA = confirmatory factor analysis; MGCFA = multigroup confirmatory factor analysis; CEAT-VIH = Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral; χ² (df) = chi-square (degrees of freedom); CFI = comparative fit index; ΔCFI= delta comparative fit index; TLI = Tucker–Lewis index; SRMR = standardized root mean square residual; RMSEA = root mean square error of approximation; ΔRMSEA = delta root mean square error of approximation; CI = confidence interval.

p < .001.

Multigroup CFA to Evaluate Invariance

With regard to MGCFA, the model demonstrated weak invariance, a finding that indicated the scale showed adequate configural and metric invariance (ΔCFI < 0.02, ΔRMSEA < 0.03) and also scalar non-invariance, once ΔCFI > –0.01 and ΔRMSEA > 0.01. According to the countries and language of the participants, it converged after 194 iterations, and for the questionnaire language, it converged after 165 iterations (Beaujean, 2014; Putnick & Bornstein, 2016; Rutkowski & Svetina, 2014). Table 3 shows fit indices; alternative fit indices (AFIs) for the one-factor model and invariance indices per country and per language are included in the analysis.

Reliability

Cronbach’s alpha for the summary adherence score and CEAT-VIH facets was computed for the total sample (n = 1,470; α = .91) and by language version: Spanish (n = 955; α = .88), Brazilian Portuguese (n = 326; α = .88), European Portuguese (n = 136; α = .95), and English (n = 53; α = .96), as indicated in Table 4. Internal consistency for the summary adherence score (all 17 items) indicated acceptable values. For a few facets individually, and depending on the language version (i.e., Spanish and Brazilian Portuguese), there was an alpha below expectations (<.70). It is important to clarify that the facets with low alphas contain two to five items (Cronbach’s alpha is calculated based on the number of items, and scales with fewer items are especially vulnerable to low alpha coefficients; Furr, 2011). On the contrary, some facets performed well, in terms of reliability, for participants who completed the European Portuguese- and English-language versions (see Table 4). The details and values for each facet and scores are presented in Table 4.

Table 4.

Reliability (Cronbach’s α) for the Summary Adherence Score and the CEAT-VIH Facets With the Total Sample and by Language Versions.

Name of facets and summary score (number of items)	Total Sample(n = 1,470)	Spanish(n = 955)	Brazilian Portuguese(n = 326)	Portuguese(n = 136)	English(n = 53)
Compliance (3)	.74	.67	.71	.86	.93
Antecedents of Non-Adherence Behaviors (4)	.82	.77	.82	.91	.91
Doctor–Patient Communication (3)	.61	.57	.48	.78	.91
Personal Beliefs/Expectancies About the Treatment (5)	.59	.59	.43	.73	.73
Treatment Satisfaction (2)	.65	.56	.64	.86	.81
Summary Adherence Score (17)	.91	.88	.88	.95	. 96

External Criterion-Related Validity: Association With Relevant Clinical-Related Variables

Clinical variables were correlated with CEAT-VIH scores to determine the evidence of clinical validity for the instrument. The hypotheses stated that the summary adherence score was expected to correlate positively with CD4+ cell count and negatively with time since diagnosis (due to treatment burden), viral load, and the number of pills prescribed to the patient. In addition, the absence of AIDS-related symptoms was expected to be associated with a higher adherence score.

The results indicated significant associations between summary adherence score and CD4+ cell count, log₁₀-transformed viral load, the number of pills prescribed, and the presence of AIDS-related symptoms. Time since HIV diagnosis was not significantly associated with adherence score. Details for all facets are presented in Table 5.

Table 5.

External Criterion-Related Validity of the CEAT-VIH Scores With Clinical-Related Variables and Sociodemographic Variables (n = 1,470).

	Name of facets and summary score (number of items)
	Compliance	Antecedents of Non-Adherence Behaviors	Doctor–Patient Communication	Personal Beliefs/Expectancies About the Treatment	Treatment Satisfaction	Summary Adherence Score
Clinical-related variables
Time since HIV diagnosis (r_s)	−.02, p = .337	−.02, p = .463	.01, p = .759	.04, p = .115	.01, p = .624	.01, p = .601
Number of CD4+ cells (r_s)	.10, p = .000	.13, p = .000	.09, p = .000	.05, p = .052	.08, p = .002	.11, p = .000
Viral load (log₁₀) (r_s)	−.10, p = .000	−.14, p = .000	−.03, p = .210	−.05, p = .056	−.08, p = .002	−.10, p = .000
Number of pills (r_s)	−.11, p = .000	−.12, p = .000	−.04, p =170	−.07, p = .006	−.02, p = 365	−.10, p = .000
AIDS-related symptoms (U)	[↑No]126,704.0, p = .000	[↑No]131,216.5, p = .000	[↑No]140,704.5, p = .002	[↑No]139,705.5, p = .001	[↑No]125,937.0, p = .000	[↑No]128,668.5, p = .000
Sociodemographic variables
Gender (U)	[↑Male]188,924.5, p = .000	[↑Male]183,862.5, p = .000	[↑Male]197,858.0, p = .002	[↑Male]167,788.5, p = .000	[↑Male]195,364.5, p = .003	[↑Male]175,444.5, p = .000
Age (r_s)	.17, p = .000	.18, p = .000	.10, p = .000	.19, p = .000	.17, p = .000	.19, p = .000
Marital status (r_s)	.03, p = .216	.01, p = .640	−.02, p = .341	.01, p = .603	−.00, p = .809	.01, p = .694
Perceived SES (r_s)	.01, p = .640	.02, p = .528	−.02, p = .490	.00, p = .977	.00, p = .995	.00, p = .992
Employment status (U)	[↑Unemployed]187,166.5, p = .002	199,566.0, p = .201	207,945.0, p = .936	196,915.5, p = .102	201,893.5, p = .339	195,161.0, p = .062
Education level (r_s)	.02, p = .423	−.02, p = .479	.08, p = .002	.06, p = .018	.05, p = .039	.07, p = .007
Place of residence (U)	121,878.0, p = .888	[↑city]209,072.5, p = .041	[↑city]198,292.5, p = .000	[↑city]174,793.0, p = .000	[↑city]207,613.0, p = .023	[↑city]193,468.0, p = .000
Participation at local AIDS association (U)	[↑No]123,109.0, p = .012	[↑No]113,140.5, p = .000	[↑No]109,373.0, p = .000	[↑No]114,292.5, p = .000	[↑No]125,576.0, p = .034	[↑No]115,439.5, p = .000

Note. CEAT-VIH = Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral; r_s = Spearman rank correlation coefficient; U = Mann–Whitney U test; ↑ = higher score; AIDS-related symptoms (Yes/No); Gender (male/female); Perceived socioeconomic status (SES) (something worse than others 1 to 4 Better than the others); Employment status (unemployed/working or studying or both); place of residence (countryside [non-urban]/city [urban]); participation at local AIDS association (Yes/No).

External Criterion-Related Validity: Association With Characteristics and Sociodemographic Profile

To test the hypothesis that some sociodemographic characteristics may explain certain levels of adherence to treatment (evidence of validity related to the external criterion), the CEAT-VIH scores were correlated with gender, age, marital status, perceived SES, employment status, education level, place of residence, and participation in local AIDS organizations.

The results indicated that the summary adherence score was significantly associated with gender (men scored higher), age, education level, place of residence (participants who lived in urban centers scored higher), and participation in local AIDS organizations (non-participants scored higher). The variables marital status, employment status, and perceived SES did not significantly associate with adherence scores. However, unemployed participants scored significantly higher on the compliance facet. The compliance facet indicates the extent to which the behavior of the person reflects strictly taking medication following time schedules and not forgetting doses. The unemployed apparently have fewer obstacles in terms of following time schedules or managing schedules to avoid omitting doses compared with those who work or study or combine work and study. The details for all facets are presented in Table 5.

Norms for the CEAT-VIH

To facilitate interpretation and classification of individuals evaluated in future research, the percentile norms for the CEAT-VIH were calculated. Descriptive results of the scores and percentile posts (position of the individual’s score in relation to the distribution of the sample scores; Field, 2013) are available as Supplementary Material.

Discussion

The present study achieved its aims, as it demonstrated new and relevant evidence of CEAT-VIH measurement properties. One novelty in the present study is the evidence regarding the internal structure of the instrument. The unidimensional structure that underlies the summary adherence score is a composite of five facets (i.e., Compliance, Antecedents of Non-Adherence Behaviors, Doctor–Patient Communication, Personal Beliefs/Expectancies About the Treatment, and Treatment Satisfaction) that explain individual differences in adherence behavior. Such a structure was confirmed through a CFA with goodness-of-fit indices in the acceptable range. Considering such evidence, it is recommended to use the summary adherence score as the main outcome for adherence and to evaluate or classify patients with HIV who receive treatment in terms of their adherence level (e.g., low, struggling, good, or high). The summary score can be used as a selection criterion or an outcome for interventions that address adherence improvement, and it may indicate which patients need intervention and/or counseling to overcome struggles with their antiretroviral treatment. Hence, the summary adherence score, in conjunction with the five facet scores, will help health professionals evaluate patients’ difficulties in more detail and choose the most appropriate method to advise for each specific patient.

The MGCFA results evidenced that CEAT-VIH showed weak invariance. Configural and metric (ΔCFI < 0.02, ΔRMSEA < 0.03) steps of invariance were achieved for country of residence and questionnaire language. However, the scalar step of invariance was not accomplished, once ΔCFI > –0.01 and ΔRMSEA > 0.01. Regarding these findings, it is necessary to consider political and social particularities in public policies and how each country’s public health systems are structured and function. In Brazil and Portugal, infectology appointments, antiretroviral medicines, complementary exams for diagnosis, and attendance and treatments directly related to HIV (AIDS) can be universally accessed. In Mexico and Venezuela, however, a certain population quota does not have consistent access to the same patterns of health care, factors that could explain the weak invariance of the construct (Biello et al., 2016; Calvetti et al., 2014; Costa et al., 2018; Padilla, 2016). In addition, adherence to HIV treatment is a complex and multifaceted behavior (Remor, 2013b), and interindividual, and consequently intercountry, differences are apparent in the construct.

Another important claim is that there is very little research on the accuracy of mean-level tests for partially invariant models, and thus, much more research is required to identify the statistical and conceptual consequences of partial metric and scalar invariance. In this way, although the results showed weak invariance, CEAT-VIH demonstrated a stable structure of the model form and item loadings on the factors across samples and is a reliable and valid instrument that can be employed in different application contexts, such as multicentre studies with the online versions (Damásio & DeSousa, 2015; Putnick & Bornstein, 2016).

Although some authors (e.g., Costa et al., 2018) claim that self-reports may overestimate adherence due to social desirability (i.e., the tendency of survey respondents to answer questions in a way that will be viewed favorably by others) and recall biases, other authors (e.g., Simoni et al., 2006; Thirumurthy et al., 2012) recognize that individual self-reports can be inexpensive, easy to administer, and accurately identify medication-taking behavior. Therefore, the availability of measures that are robust in terms of psychometric properties is very much needed to minimize limitations attributed to self-report instruments. Information about validity evidence related to relevant clinical criteria variables is especially recommended (e.g., in case of HIV infection, viral load would be the gold standard).

In the authors’ opinion, the CEAT-VIH overcame several limitations noted in the literature about self-report measures. For example, it had good construct validity, dimensionality invariance, acceptable reliability across samples (e.g., countries and language versions), and good criterion-related validity with relevant clinical outcomes (e.g., viral load or AIDS-related symptoms). Previous work reported responsiveness after intervention on adherence (Remor, 2013b) and convergent and divergent validity with other PRO measures (e.g., depression, anxiety, stress, psychopathology, social support, and quality of life; Remor, 2013b). Moreover, it has been used extensively in different studies and cultural contexts (36 studies published using the instrument across 12 countries) by independent researchers and always showed good performance. Regarding the online version, it is easy to apply and integrate in routine clinical settings, the questions do not exhibit comprehension problems, and it has an attractive and clean interface, an automatic scoring system, immediate graphical feedback, and the possibility of sending results by email.

Reliability coefficients for the instrument in the total sample were satisfactory (between .88 and .95 across the four language versions). However, reliability coefficients for the facets were variable across different language versions. Some facets showed reliability below expectations (<.70) in the current sample (see Table 4), meaning that these facets would not be recommended to be used separately from the conjunction of items in research. It is worth mentioning that reliability values are related to the test scores in a specific sample and are not a trait of the instrument (Urbina, 2004).

In the present study, adherence to antiretroviral treatment scores was associated with clinical variables, as expected based on the literature. Moreover, the results from the present study corroborate previous data (Remor, 2013b) regarding the relationships between external clinical variables and CEAT-VIH scores; the data highlight especially the association between viral load and AIDS-related symptoms and summary adherence score. This feature would allow researchers and clinicians working with limited resources, where biological markers are scarce, to use the instrument for follow-up for HIV infection control. Viral load control is essential to buffer HIV progression and prevent its spread (Marks et al., 2015). These results were expected along with the other associations such as CD4+ counts and the number of pills prescribed (similar results were described in Costa et al., 2018). Although these correlation coefficients were small, they appear consistently across studies and cannot be neglected.

The literature (e.g., Dunbar-Jacob & Mortimer-Stephens, 2001) is controversial about the prediction of adherence from sociodemographic variables, specifically to avoid patient stigmatization. Nevertheless, there is evidence (e.g., Rolnick, Pawloski, Hedblom, Asche, & Bruzek, 2013; World Health Organization, 2003) that some patient characteristics may indicate more risk to experience difficulties with treatment adherence. Our results indicated that age was positively associated with adherence level (similar results were reported by Biello et al., 2016), gender was associated with adherence scores (male scores were slightly higher than female scores; similar results were described in Costa et al., 2018), and education level was related to better doctor–patient communication (also reported in Pérez-Salgado et al., 2015). Patients with better formal education may connect more easily with their doctor or feel less distant or uncomfortable with health care personnel. There was an association of place of residence with personal beliefs/expectancies about the treatment, where urban participants had more positive beliefs and expectations about treatment compared with rural area individuals. In addition, individuals who participate in local AIDS organizations had slightly lower adherence scores; perhaps patients in an AIDS association are in a more vulnerable condition or are looking for help and support from the organization, potential factors that would explain their lower scores. On the contrary, marriage and occupational statuses were not related to adherence scores. In conjunction, these results may frame sociodemographic variables as risk factors (or barriers) that should be evaluated case by case to personalize health care attention. For the present study, these results support the construct validity of the CEAT-VIH, because all observed results are corroborated in the literature (e.g., Biello et al., 2016; Calvetti et al., 2014; Costa et al., 2018; Pérez-Salgado, Compean-Dardon, Staines-Orozco, & Ortiz-Hernandez, 2015), a fact that indicates the scores from the instrument behave in an expected manner.

The present study, however, is not free from limitations. The most significant limitation may be the use of a non-random sample, limited to patients or participants from countries with understanding of the languages in which the online version is available (Spanish, Portuguese, and English), with access to the Internet and the majority with a medium-to-good educational level. Further studies may be needed to test invariance across English-speaking countries or countries with other languages not included in the present study. Moreover, the design of the present study did not allow the researchers to conduct analyses of reliability test–retest or responsiveness to changes. Although previous works addressed that matter with the paper-and-pencil version (e.g., Remor, 2013b; Tafur-Valderrama, Ortiz-Alfaro, Garcıa-Jimenez, Faus-Dader, & Martinez-Martinez, 2012), it is recommended that future research address this same question with the online version.

Adherence to antiretroviral treatment is a focus of attention for multidisciplinary teams. It provides a great opportunity for psychosocial intervention that can promote self-care, adherence, healthy behaviors and lifestyles, and quality of life. The availability of reliable, valid, and culturally sensitive instruments are relevant to evaluate interventions with accuracy and precision. In conclusion, the results described in this work about the psychometric properties of the CEAT-VIH online version allow the recommendation of its use in research as a way to measure adherence to antiretroviral treatment in people with HIV infection.

Supplemental Material

Supplementary_material – Supplemental material for Evidence of Validity for the Online Version of the Assessment of Adherence to Antiretroviral Therapy Questionnaire

Supplemental material, Supplementary_material for Evidence of Validity for the Online Version of the Assessment of Adherence to Antiretroviral Therapy Questionnaire by Bruno de Brito Silva, Ariane de Brito, Erika Pizziolo Monteiro, Gabriela Pasa Mondelo and Eduardo Remor in SAGE Open

Footnotes

Acknowledgements

We would like to thank the researchers who used the Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral (CEAT-VIH) in their studies and shared their databases. We also thank the participants for their participation in our research.

Author Contributions

E.R. conceived the study, collected the data (website), supervised data analysis, interpreted the results, and wrote the manuscript. Students B.B.S., A.B., E.P.M., and G.P.M. performed data analysis and helped write the manuscript. All authors read and approved the final manuscript.

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: E.R. is the author of CEAT-VIH questionnaire. Authors B.B.S., A.B., E.P.M., and G.P.M. declare that they have no conflicts of interest.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the “Conselho Nacional de Desenvolvimento Científico e Tecnológico,” Brazil, under Brazilian research grants (Numbers 304616/2014-1 and 302850/2017-1) to E.R.

Ethics Approval and Consent to Participate

The study was approved by the Research Committee of the Institute of Psychology, Universidade Federal do Rio Grande do Sul (Number 28203). All participants provided their digital informed consent at the website before data were collected.

ORCID iD

Eduardo Remor

Availability of Data and Material

The data that support the findings of this study are available from the Cuestionario para la Evaluación de la Adhesión al Tratamiento Antiretroviral (CEAT-VIH) website, but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are, however, available from the authors upon reasonable request and with permission of the author (E.R.).

Supplemental Material

Supplemental material for this article is available online.

Notes

Author Biographies

Bruno de Brito Silva is a PhD student at the Universidade Federal do Rio Grande do Sul (Brazil).

Ariane de Brito is a PhD student at the Universidade Federal do Rio Grande do Sul (Brazil).

Erika Pizziolo Monteiro is a PhD student at the Universidade Federal do Rio Grande do Sul (Brazil).

Gabriela Pasa Mondelo received a Master’s degree from the Universidade Federal do Rio Grande do Sul (Brazil).

Eduardo Remor, PhD, is a professor at Institute of Psychology at Universidade Federal do Rio Grande do Sul (Brazil). His research interests are broadly in the area of health psychology and behavioral medicine, and he has published extensively on psychology-related issues and HIV infection.

References

Beaujean

A. A.

(2014). Latent variable modeling using R: A step-by-step guide. New York, NY: Routledge. doi:10.4324/9781315869780

Biello

K. B.

Oldenburg

C. E.

Safren

S. A.

Rosenberger

J. G.

Novak

D. S.

Mayer

K. H.

Mimiaga

M. J.

(2016). Multiple syndemic psychosocial factors are associated with reduced engagement in HIV care among a multinational, online sample of HIV-infected MSM in Latin America. AIDS Care, 28, 84-91. doi:10.1080/09540121.2016.1146205

Brown

T. A.

(2015). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.

Calvetti

P. Ü.

Giovelli

G. R. M.

Gauer

G. J. C.

Moraes

J. F. D.

(2014). Psychosocial factors associated with adherence to treatment and quality of life in people living with HIV/AIDS in Brazil. Jornal Brasileiro de Psiquiatria, 63(1), 8-15. doi:10.1590/0047-2085000000002

Conz

(2015). A não adesão ao tratamento da Aids: o abandono em questão [Non-adherence to AIDS treatment: The abandonment in question] (Unpublished master’s thesis). Pontifícia Universidade Católica do Rio Grande do Sul, Porto Alegre, Brasil.

Costa

J. M.

Torres

T. S.

Coelho

L. E.

Luz

P. M.

(2018). Adherence to antiretroviral therapy for HIV/AIDS in Latin America and the Caribbean: Systematic review and meta-analysis. Journal of the International AIDS Society, 21, e25066. doi:10.1002/jia2.25066

Damásio

B. F.

DeSousa

D. A.

(2015). Análise fatorial confirmatória multigrupo: Descrição e aplicação no software Mplus [Multigroup confirmatory factor analysis: Description and application in Mplus software]. In Silva

M. C. R.

Bartholomeu

Vendramini

C. M. M

Montiel

J. M.

(Orgs.), Aplicações de métodos estatísticos avançados à avaliação psicológica e educacional [Applications of statistical methods to advanced psychological assessment and educational] (pp. 175-192). São Paulo, Brazil: Vetor.

Dima

A. L.

Schweitzer

A. M.

Diaconiţ

Remor

Wanless

R. S.

(2013). Adherence to ARV medication in Romanian young adults: Self-reported behaviour and psychological barriers. Psychology, Health & Medicine, 18, 343-354. doi:10.1080/13548506.2012.722648

Dunbar-Jacob

Mortimer-Stephens

M. K.

(2001). Treatment adherence in chronic disease. Journal of Clinical Epidemiology, 54(Suppl. 1), 57-60.

10.

Field

(2013). Discovering statistics using IBM SPSS statistics (4th ed.). London, England: Sage.

11.

Foresto

J. S.

Melo

E. S.

Costa

C. R. B.

Antonini

Gir

Reis

R. K.

(2017). Adherence to antiretroviral therapy by people living with HIV/AIDS in a municipality of São Paulo. Revista Gaucha de Enfermagem, 38(1), 1-7. doi:10.1590/1983-1447.2017.01.63158

12.

Furr

R. M.

(2011). Scale construction and psychometrics for social and personality psychology. London, England: Sage. doi:10.4135/9781446287866

13.

Haynes

R. B.

Taylor

D. W.

Sackett

D. L.

(1979). Compliance in health care. Baltimore, MD: Johns Hopkins University Press.

14.

Herranz-Alvarez

Ríos-Maldonado

S. R.

Hernández

(2017). Caracterización de la población VIH-SIDA del hospital universitario de Caracas [Characterization of the HIV-AIDS population of the Caracas university hospital] (Unpublished thesis). Universidad Central de Venezuela, Caracas, Venezuela.

15.

Marks

Gardner

L. I.

Rose

C. E.

Zinski

Moore

R. D.

Holman

Giordano

T. P.

(2015). Time above 1500 copies: A viral load measure for assessing transmission risk of HIV-positive patients in care. AIDS, 29, 947-954. doi:10.1097/QAD.0000000000000640

16.

Myers

N. D.

Ahn

Jin

(2011). Sample size and power estimates for a confirmatory factor analytic model in exercise and sport: A Monte Carlo approach. Research Quarterly for Exercise and Sport, 82, 412-423. doi:10.1080/02701367.2011.10599773

17.

Neves

H. R.

(2017). O impacto do feedback do perfil de adesão ao tratamento antirretroviral sobre as percepções e o comportamento do paciente com infecção pelo HIV [The impact of antiretroviral treatment adherence profile feedback on the perceptions and behavior of patients with HIV infection] (Bachelor thesis, Universidade Federal do Rio Grande do Sul, Porto Alegre, Brasil). Retrieved from http://hdl.handle.net/10183/169469

18.

Nogueda-Orozco

Fresán-Orellana

Vite-Sierra

Sánchez-Sosa

Sierra-Madero

Robles-García

(2016). Escala de apreciación cognitiva del VIH/SIDA: adaptación y evaluación psicométrica en población mexicana [Scale of cognitive appreciation about HIV/AIDS: psychometric assessment and adaptation in the Mexican population]. Psicología Iberoamericana, 24, 55-60.

19.

Padilla

H. L. E.

(2016). Adherencia al tratamiento antirretroviral en los pacientes de la consulta de VIH del Iahula y su posible relación sobre el comportamiento de la epidemia del VIH/SIDA en el edo. Mérida durante el quinquenio 2011-2015 [Adherence to antiretroviral treatment in patients in the Iahula HIV clinic and its possible relationship on the behavior of the HIV/AIDS epidemic in the edo. Mérida during the five-year period 2011-2015] (Unpublished thesis). Universidad de Los Andes, Mérida, Venezuela.

20.

Pérez-Salgado

Compean-Dardon

M. S.

Staines-Orozco

M. G.

Ortiz-Hernandez

(2015). Satisfaction with healthcare services and adherence to antiretroviral therapy among patients with HIV attending two public institutions. Revista de Investigación Clinica, 67, 80-88.

21.

Putnick

D. L.

Bornstein

M. H.

(2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71-90. doi:10.1016/j.dr.2016.06.004

22.

Reis

A. C.

Lencastre

Guerra

M. P.

Remor

(2009). Adaptação portuguesa para a avaliação da adesão ao tratamento anti-retrovírico (CEAT-VIH) [Portuguese adaptation of the Questionnaire to assess adherence to antiretroviral treatment – HIV (CEAT-VIH)]. Psicologia, Saúde & Doenças, 10, 175-191.

23.

Remor

(2002). Valoración de la adhesión al tratamiento antirretroviral en pacientes VIH+ [Assessment of adherence to antiretroviral therapy in HIV+ patients]. Psicothema, 14, 262-267.

24.

Remor

(2013a). Release of an online self-reporting tool for assessing adherence to antiretroviral therapy (CEAT-VIH). Journal of Antivirals and Antiretrovirals, 5, 178-179. doi:10.4172/jaa.1000085

25.

Remor

(2013b). Systematic review of the psychometric properties of the questionnaire to evaluate the adherence to HIV therapy (CEAT-VIH). The Patient—Patient-Centered Outcomes Research, 6, 61-73. doi:10.1007/s40271-013-0009-0

26.

Remor

Milner-Moskovics

Preussler

(2007). Brazilian adaptation of the Assessment of Adherence to Antiretroviral Therapy Questionnaire. Revista de Saúde Publica, 41, 685-694.

27.

Remor

Penedo

F. J.

Shen

B. J.

Schneiderman

(2007). Perceived stress is associated with CD4+ cell decline in men and women living with HIV/AIDS in Spain. AIDS Care, 19, 215-219.

28.

Rolnick

S. J.

Pawloski

P. A.

Hedblom

B. D.

Asche

S. E.

Bruzek

R. J.

(2013). Patient characteristics associated with medication adherence. Clinical Medicine & Research, 11, 54-65. doi:10.3121/cmr.2013.1113

29.

Rutkowski

Svetina

(2014). Assessing the hypothesis of measurement invariance in the context of large-scale international surveys. Educational and Psychological Measurement, 74, 31-57. doi:10.1177/0013164413498257

30.

Salmanton-García

Herrador

Ruiz-Seco

Nzang-Esono

Bendomo

Bashmakovic

Aparicio

(2015). Self-reported adherence to antiretroviral therapy in HIV+ population from Bata, Equatorial Guinea. AIDS Care, 28, 543-553. doi:10.1080/09540121.2015.1124976

31.

Silva

A. C. O.

Reis

R. K.

Nogueira

J. A.

Gir

(2014). Quality of life, clinical characteristics and treatment adherence of people living with HIV/AIDS. Revista Latinoamericana de Enfermagem, 22, 994-1000. doi:10.1590/0104-1169.3534.2508

32.

Simoni

J. M.

Kurth

A. E.

Pearson

C. R.

Pantalone

D. W.

Merrill

J. O.

Frick

P. A.

(2006). Self-report measures of antiretroviral therapy adherence: A review with recommendations for HIV research and clinical management. AIDS and Behavior, 10, 227-245. doi:10.1007/s10461-006-9078-6

33.

Tafur-Valderrama

E. J.

Ortiz-Alfaro

Garcıa-Jimenez

Faus-Dader

M. J.

Martinez-Martinez

(2012). Pharmacist intervention in the improvement of adherence in HIV/AIDS patients with antiretroviral treatment in Lima (Peru). Pharmaceutical Care España, 14, 146-154.

34.

Tello-Velásquez

J. R.

Díaz-Llanes

B. E.

Mezones-Holguín

Rodríguez-Morales

A. J.

Huamaní

Hernández

A. V.

Arévalo-Abanto

(2015). La mala calidad de sueño se asocia a una menor adherencia al tratamiento antirretroviral de gran actividad en pacientes peruanos con infección por VIH/SIDA [Poor quality of sleep associated with low adherence to highly active antiretroviral therapy in Peruvian patients with HIV/AIDS]. Cadernos de Saúde Publica, 31, 989-1002. doi:10.1590/0102-311X00010014

35.

Thirumurthy

Siripong

Vreeman

R. C.

Pop-Eleches

Habyarimana

J. P.

Sidle

J. E.

Bangsberg

D. R.

(2012). Differences between self-reported and electronically monitored adherence among patients receiving antiretroviral therapy in a resource-limited setting. AIDS, 26, 2399-2403. doi:10.1097/QAD.0b013e328359aa68

36.

Urbina

(2004). Essentials of psychological testing. Hoboken, NJ: Wiley.

37.

World Health Organization. (2003). Adherence to long-term therapies: Evidence for action. Geneva, Switzerland: Author.

38.

Zuge

S. S.

Primeira

M. R.

Remor

Magnago

T. S. B. S.

Paula

C. C.

Padoin

S. M. M.

(2017). Fatores associados à adesão ao tratamento antirretroviral em adultos infectados pelo HIV: estudo transversal [Factors associated to the adherence to antiretroviral treatment in HIV-infected adults: Cross-sectional study]. Revista de Enfermagem da UFSM, 7, 577-589. doi:10.5902/2179769225657

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.53 MB