Abstract
Background:
Self-reports of medication adherence are subject to recall errors and social desirability bias. The visual analogue scale (VAS) is a brief measure and minimally burdensome. Its accuracy among substance users needs confirmation.
Methods:
We administered the VAS on an audio computer-assisted self-interview (ACASI) to 102 HIV-positive active substance users. Viral load (VL) was obtained via chart review. Cohen κ assessed the overall agreement between VL suppression and adherence, and a receiver–operator characteristic (ROC) curve assessed the diagnostic performance.
Results:
The VAS estimates of adherence correlated with VL. Various cutoffs for adherence were associated with VL suppression, demonstrating fair to moderate agreement through κ. Area under the curve (AUC) demonstrated that the ACASI-administered VAS strongly predicted VL suppression (AUC = 0.79).
Discussion:
The VAS administered by ACASI is viable for measuring highly active antiretroviral therapy (HAART) adherence among substance users. Its benefits as a screening instrument may make it useful for both clinical and research purposes.
Introduction
Adherence to highly active antiretroviral therapy (HAART) is crucial for achieving clinical benefits (such as survival and the prevention of opportunistic infections), 1 –3 but adherence remains a challenge in drug users. Reports suggest that drug users are less adherent to HAART than nondrug users. 4 Further, drug users often underreport negative behaviors, making the assessment of HAART adherence more difficult. Concordance among different measures of adherence, such as self-report and pharmacy refills, can be low among injection drug users. 5
The visual analogue scale (VAS) and audio computer-assisted self-interview (ACASI) provide potential benefits in assessing HAART adherence. The VAS for adherence requires a participant to choose a spot on a line that corresponds with the percentage of their prescribed medications taken. The measure is brief, often containing only 1 question, and is easy to use. 6,7 Patients may be better able to estimate their general adherence instead of reporting the specific number of doses taken. 8 Compared with in-person interviews, ACASI administration has been associated with a higher likelihood of reporting socially undesirable behaviors, such as drug use and poor medication adherence. 9,10
Some research has been conducted with the VAS as a HAART adherence assessment, with varied results. Kalichman et al 11 found concordance between the VAS administered on ACASI software and phone-based pill counts among HIV clinic patients. Studies of a Chinese sample 12 and HIV clinic outpatients 13 failed to find a relationship between the VAS and objective measures of adherence. A relationship was found between VAS for adherence and viral load (VL) suppression in HIV clinic outpatients in Italy 14 as well as among patients in Thailand 15 and with other measures of adherence among newly diagnosed patients just starting HAART. 16 Substance use, however, was not assessed in any of these studies. The best methods for assessing HAART adherence remain unclear, 17 especially in substance-using patients.
The accuracy of a single-item computerized assessment for HAART adherence in substance abusers remains unknown. We sought to determine the efficacy of the VAS administered on ACASI as an adherence measurement in substance-using patients. To do this, we compared adherence, as measured by the VAS administered on ACASI, to CD4 count and HIV RNA VL. We also sought to identify the best cut point for adherence based on self-report using CD4 count and VL.
Methods
We recruited participants from the waiting room of the Positive Health Program, an HIV clinic at the San Francisco General Hospital. All participants were HIV positive and prescribed HAART. For the purpose of this analysis, we focused on participants who reported use of heroin, cocaine, and/or amphetamines in the past 30 days. The procedures were approved by the Committee on Human Research at the University of California, San Francisco.
Participants reported their adherence to HAART in the past 30 days using the VAS on ACASI software. The VAS asks individuals to consider a specified time period and estimate along a continuum the percentage of medication doses (0%-100%) that they had taken as prescribed. The participant chooses a spot on a line, which represents the percentage of their medication they have taken. 6,18 Substance use was obtained with the Addiction Severity Index–Lite. 19 HIV RNA VL (copies/mL) and CD4 count (cells/mm3) were obtained via chart review of the clinic’s electronic medical records. Participants were determined to have achieved VL suppression if they had less than 40 copies/mL. We used a logarithmic transformation to express VL.
Data Analysis
Due to their nonnormal distributions, adherence and VL were compared through a Spearman correlation. Viral load suppression was then compared to dichotomous levels of adherence via chi-square. Cohen κ values were calculated to assess the overall agreement between these dichotomized VAS ratings of adherence and VL suppression. Then, we performed a receiver–operating characteristic (ROC) for self-reported adherence and CD4 count on VL suppression. First, we used the continuous measure of adherence. Then, we performed the ROC analysis on adherence at various cut points to determine which level of adherence matched most closely with viral suppression. The ROC curve offers a graphical illustration of the trade-offs between sensitivity and specificity at each value for any diagnostic test that uses a continuous variable. 20 All analyses were conducted with SPSS 17.0.
Results
Among participants, 102 provided adherence information, reported heroin, cocaine, and/or amphetamine use, and had CD4 count and VL results available by chart review. Specific information on these participants is included in Table 1. They were 70% male, mostly Caucasian (44%) and African American (38%), with a mean age of 46.2 years (standard deviation [SD] = 9.3). Mean self-reported adherence on VAS was 77% (SD = 24%). Regarding specific adherence cutoff points, 64 (63%) reported adherence at 75% or greater, while 51 (50%) reported 85% or greater adherence, and 33 (32%) reported 95% or greater adherence. Mean VL was 16 494 copies/mL (SD = 43 107), while the mean log VL was 2.56 (SD = 1.32). Over half (60; 59%) achieved VL suppression.
Demographics.
Abbreviations: SD, standard deviation; GED, general equivalency diploma.
a Stably housed was defined as owning or renting a house or apartment or living with someone who owns or rents their house or apartment.
The VAS estimates of adherence had a strong negative correlation with VL (r = −.52, P < .001). Individuals who demonstrated VL suppression demonstrated significantly higher self-reported adherence (88.1% versus 62.3%, Z = 4.99, P < .001).
When analyzing adherence at specific cutoffs, VL suppression was significantly associated with all dichotomized levels of adherence, including 75% or greater (χ2(1) = 22.32, P < .001), 85% or greater (χ2(1) = 10.36, P = .001), and 95% or greater (χ2(1) = 13.64, P < .001). Sensitivity was highest for 100% adherence (0.89), followed by 75% or greater adherence (0.82), and decreased at higher (≥85%: 0.63; ≥95%: 0.47) and lower cutoffs. Specificity was highest for lower adherence cutoffs (≥60%: 0.89; ≥65%: 0.88; ≥70%: 0.89), which decreased at higher adherence cutoffs (Table 2).
Agreement between Self-reported Adherence and Viral Load Suppression.
Abbreviations: CI, confidence interval; SD, standard deviation; ROC, receiver–operator characteristic; AUC, area under the curve.
We then calculated κ values for VL suppression and each dichotomous level of adherence. κ values for self-reported adherence were in the fair to moderate agreement ranges (Table 2). 21 The largest κ value was for the adherence cutoff of 70% or greater (κ = 0.55), while the agreement decreased with adherence cutoffs of 80% or greater.
Next, we performed the ROC analysis. The area under the curve (AUC) for adherence measured on a continuum compared to VL suppression was 0.79 (95% confidence interval [CI]: 0.70-0.88, P < .001; Figure 1). This was slightly better than the AUC with CD4 count and viral suppression (AUC = 0.74; 95% CI: 0.65-0.84, P < .001).
Receiver–operating characteristic for adherence measured by VAS on ACASI. VAS indicates visual analogue scale; ACASI, audio computer-assisted self-interview.
We further determined the AUC for adherence cutoffs to assess the efficacy of each cutoff compared to VL (Table 1). The greatest AUC was found for self-reported adherence ≥70% (AUC = 0.76, 95% CI: 0.66-0.86, P < .001). As the adherence cutoff increased, AUC decreased.
Discussion
Substance-using patients demonstrated low rates of HAART adherence and VL suppression, especially compared to other studies using the VAS. 6,14 –16 In our study, VAS estimates of adherence correlated with VL, and dichotomized adherence was related to VL suppression. Sensitivity decreased at higher cutoffs, while specificity increased. ROC analysis was comparable to CD4 and VL in this sample as well as to another study that compared CD4 on VL 22 and similar to adherence from pharmacy refills on VL. 23 These results suggest that the VAS administered on ACASI software is an acceptable and effective method of assessing HAART adherence in substance-using patients.
Our analysis demonstrated that 70% was the most effective cutoff of adherence in predicting VL suppression in this study. Previous research had suggested that the ideal level of HAART adherence for achieving its benefits was 95%, 24 but advances in HAART since the early 21st century may have changed this. Recent research suggests that HAART regimens containing a nonnucleoside reverse transcriptase inhibitor (NNRTI) may require less stringent adherence than non-NNRTI regimens. 25 Our research provides support for this finding.
Our study has several limitations. First, this is not a random sample, but a cohort of self-selecting individuals who responded to waiting room recruitment. Also, we did not have a more objective method of pill adherence, such as pill counts or electronic pill bottles, to corroborate the VAS estimates.
The VAS administered on ACASI software is a viable method for assessing HAART adherence in a substance-using population. The benefits of administering the VAS on ACASI over other methods of assessing adherence, such as surveys or pill counts, include that the reporting burden for patients is considerably less. This method can identify individuals who would benefit the most from interventions that increase adherence, and participants may find the VAS shorter and less complicated than other methods of measuring adherence.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the National Institute of Drug Abuse (F32 DA032446 and P50 DA09253) and the Roy and Marie Scola Endowed Chair in Psychiatry at the University of California, San Francisco.