Abstract
Background:
HIV medication nonadherence is a major problem, yet many providers lack the time and training to carefully ask patients about their adherence.
Objective:
To design and pilot a technology-assisted intervention, for use in clinical settings, to identify nonadherent patients.
Methods:
The intervention uses audio computer-assisted self-interview (ACASI) to improve the assessment of adherence and medication-related problems. Patients completed a touch screen computer ACASI which generated graphic clinician and patient reports for discussion during the clinical encounter.
Results:
72 patients and 11 providers participated in this study. The patients easily completed the ACASI. Adherence was 63% (3-day) and 47% (30-day). Using the ACASI, 22% of patients identified themselves as nonadherent, when their providers perceived them as adherent.
Conclusions:
This ACASI-based intervention is easy to use and helps identify nonadherence. The pilot test engendered enhancements including the addition of phone-based adherence counseling. A larger trial is underway to evaluate whether the intervention leads to improved HIV-related outcomes.
Introduction
In the United States, HIV infection is now widely regarded as a chronic illness. While combination antiretroviral therapies (ARTs) have dramatically improved patient outcomes, they generally require sustained adherence of at least 90% to 95% of doses taken, 1 –5 though newer regimens appear to be effective at lower adherence thresholds. 6 Without high levels of adherence there is risk of increasing or recrudescent viral replication, increased morbidity, the development of ART resistance, and increased mortality. 7 –9 Over the long term, that is, years and decades, consistently high adherence is challenging for most patients. Estimates suggest that ART adherence ranges from 48% to 83%, with most achieving about 70% adherence. 10 –17 Clinicians who care for HIV-infected patients can intervene to improve adherence if they can accurately identify nonadherent patients. Unfortunately, evidence indicates that doctors have difficulty identifying patients having adherence trouble or those at risk of becoming nonadherent. 18 –22 Patients overreport the number of doses they take, 23 providers lack time to review adherence in a thorough and standardized manner, and providers sometimes think that detailed questioning could harm the patient–provider relationship.
A wide variety of methods have been tried to improve medication adherence, including pill boxes, automated text-messaging reminder systems, and many others. 24 Audio computer-assisted self-interview (ACASI) may improve provider assessments of ART adherence. 25 The ACASI is essentially a computerized tablet questionnaire which includes branching logic to customize and streamline questions and provide patient feedback in the form of tailored informational and motivational messages, and automatic prepopulation of data from the electronic medical record (EMR). Evidence from ACASI use for a variety of health issues 26 –29 indicates that people more accurately report sensitive information through a computerized assessment than through face-to-face interviews. 30 –32
We developed an intervention, Medication for Chronic HIV Education and Collaboration (MedCHEC), to improve the assessment of patient medication adherence and provide targeted adherence counseling. While the intervention includes an ACASI device, we made several important improvements over earlier ACASI-based interventions. 25 –27 First, we adopted a user-centered design, including large font, uncluttered screens, and an audio component. 33,34 Second, we integrated the device and its use into the clinical setting. Third, we added a supportive component so that patients identified as nonadherent or at risk of nonadherence were contacted and counseled by an adherence care manager (ACM). Fourth, we created succinct, easy to understand provider and patient summary reports, which included graphic as well as numeric representations of nonadherence. 35,36 The design and implementation of this intervention were guided by the Information, Motivation, and Behavioral Skills (IMB) model. 37,38 The IMB has been applied to HIV adherence interventions to enhance patients’ behavioral skills in adhering to ART. Studies have shown that the combination of information, that is, providing patients with information about the importance of adherence and advice for surmounting barriers to adherence, and motivation, for example, nonjudgmental encouragement to improve their medication taking, improves patients’ ability to adhere to their medications. 39,40 In this article, we describe the development, piloting, and clinical implementation of MedCHEC.
Methods
System Description
Overview
The MedCHEC intervention includes ACASI, patient report, provider report, and phone-based adherence counseling. Figure 1 shows how the device is used in clinical settings and how it interacts with patients, clinicians, and ACMs. Using the ACASI, patients indicate their understanding of their medication regimen, describe their level of medication adherence, and, if not completely adherent, explain their reasons for missing ART doses. The development of the MedCHEC intervention occurred over 2 phases. The pilot study version of MedCHEC included the ACASI and provider report. Then, based on the findings from the pilot study we added the patient report and the ACM.

Integration of MedCHEC intervention into clinical care. Patient completes the ACASI in the waiting room using a touch screen computer. Provider and patient each receive a report prior to the clinical encounter, allowing them to discuss the report contents during the visit. Some patients receive phone counseling, after the visit, from an adherence care manager. ACASI indicates audio computer-assisted self-interview; MedCHEC, Medication for Chronic HIV Education and Collaboration.
Hardware, software, and data storage
The MedCHEC ACASI uses a tablet computer with touch screen capability. The ACASI data are stored in an SQL database system which is wholly contained behind secure firewall barriers consistent with those protecting health institutions’ EMR. The SQL database is both the repository for collected data and the conduit for real-time generation of the provider and patient reports. Administrative tools within the ACASI software allow export of data into standard analytic formats. The MedCHEC was implemented using the CASIC Builder software program from West Portal Software Corporation (San Francisco, California).
ACASI components
The MedCHEC ACASI components are regimen confirmation, adherence assessment, reasons for missing doses, depressive symptoms, alcohol use, and drug use. Patients answer questions and enter data using the touch screen computer interface, either with a stylus or with their finger. The ACASI is programmed using internal branching logic to present appropriate, tailored questions and to assess ART use and related behaviors while minimizing patient burden. If desired, patients wear headphones to hear the text spoken aloud. The ACASI interface is designed to be easy to use and interactive. Buttons and fonts are large, and the text is readable and accessible. Language is constructed to avoid blame or coercion. The MedCHEC ACASI is currently available in English only.
Patients can use the device in the clinic waiting area or in a private room, depending on the clinic configuration and wireless capability. Clinic staff (receptionist, clerk, or nurse) hands the patient the ACASI touch screen device, prepopulated with his or her HIV medication regimen, immediately after the patient has checked in. The patient completes the ACASI and returns the device to the clinic staff who prints out the provider and patient reports. The patient receives his or her report right then, while the provider report is delivered to the provider, just prior to the consultation, by the clinic staff who directs the patient to the examination room. The ACASI modules and reports are described below.
The Regimen Identification section assesses patient knowledge of their ARTs and prescribed regimens. High-quality photographs of ARTs are shown on the computer screen with brand name and generic name printed below the photograph. This “error check” process is analogous to a computer-implemented version of the recommended (but time-consuming, and often skipped) “brown-bag review” of patients’ medications. 41 The patient selects the medications he or she believes are current prescriptions. The patient then is asked how many pills of each medication he or she is supposed to take and the times of day taken (morning, afternoon, and evening). A summary page shows the full regimen selected, that is, picture of each pill, number of pills, and times of day, for the patient to confirm, or reject, and make corrections. The ACASI informs the patient about the correct dosing prior to going on to the adherence component.
The next section, Patient Reported Adherence, asks patients how accurately they are taking their ART medications. The first item is a global rating, “Thinking about the past 4 weeks on average how would you rate your ability to take ALL of your HIV medications as your doctor prescribed them?” with 6 response categories from very poor to excellent. 23 The patient is then shown a Visual Analog Scale ([VAS] 42 labeled from 0%-100% with each 10% increment labeled), and again asked, globally, about all their medications and then asked about each one individually.
If the patient reports imperfect adherence (<95%), the next section that appears is Adherence Barriers. The patient is asked to select all that apply among a list of 21 barriers. Examples include “I ran out of, or lost my pills,” “I felt down or depressed,” and “My lab tests were good.” The ACASI gives tailored advice responding to the barriers, for example, reporting sleeping through doses results in a message about using an alarm clock. Regardless of self-reported adherence, the final question asks whether a patient would like additional help with adherence. If a patient checks the “yes” box, his or her contact information is forwarded to the ACM who initiates phone contact with the patient and conducts counseling. This section ends with a picture of the patient’s prescribed regimen, which can be printed out by the patient.
The ACASI next covers mood and substance use, both known correlates of medication adherence. 43 A series of 9 questions, the Patient Health Questionnaire 9 (PHQ-9), 44 inquires about depressive symptoms, for example, “Over the last 2 weeks, how often have you been bothered by little interest or pleasure in doing things?” with responses of not at all, several days, more than half the days, and nearly every day. This is followed by alcohol use questions using the Alcohol Use Disorders Identification Test (AUDIT) instrument 45 which assesses symptoms of alcohol addiction and the effects of alcohol use on behavior and health. The last section assesses the patient’s substance use (other than alcohol) with 11 questions, adapted from the HCSUS study, 46 about any drugs that have been used in the past 4 weeks.
Two reports are created (in the initial pilot, described below, there was only a provider report). The patient report has 3 parts (Figure 2). “My Medications” (not shown) is a summary of the regimen the patient is prescribed, with photographs of medications, times of day to be taken, and number of pills. The “Tips for Staying on Track” section suggests ways to overcome specific barriers the patient reported. For this patient, who forgets to take his or her pills, the report suggests leaving reminder notes around the house and asking friends to help him or her. The “Taking Your HIV Medications” section uses graphics displays, based on patient answers to the VAS, to show how the patient is doing with “overall medications” and with a single “problem” medication (missed most often by the patient). The “thermometer” on the left shows 65% for overall medications, while the one on the right shows 60% for Truvada (emtricitabine and tenofovir (TDF); Truvada is manufactured by Gilead Sciences, Inc, Foster City, California). The patient report was not a part of the initial pilot study but rather was added to MedCHEC in response to the pilot study findings.

Patient report generated by the MedCHEC ACASI, showing sections on overcoming barriers and self-reported adherence. Actual reports given to patients in clinic have color “thermometer” bars that transition from green for the “safest zone” to red for the “danger zone.” ACASI indicates audio computer-assisted self-interview; MedCHEC, Medication for Chronic HIV Education and Collaboration.
The provider report is a single-page printout, with 4 sections, designed for quick review (Figure 3). It is given to the provider shortly before seeing the patient. The summary flags discrepancies between the patient's understanding of his or her regimen and what is in the medical record. It also indicates whether an ACM is recommended—based on an algorithm programmed into the ACASI. Large, red text starting with “Alert” is used for warnings (eg, patient suicidal thoughts). The next section, “Medication Use,” recapitulates the medications the patient reported taking and the medications listed in the medical record and indicates whether there were discrepancies between patient report and the medical record. Also for each medication, and for overall medications, it summarizes the patient’s reported adherence, as a rating, for example, “Good” and as the percentage of doses taken (eg, 60%). The next section, “Medication Barriers” summarizes which, if any, barriers the patient reported; and, the last section “Mood and Substances” provides summary scores, for the PHQ-9 and the AUDIT, along with their normal ranges, and indicates whether illicit drugs have been used.

Provider report generated by the MedCHEC ACASI. Top portion shows summary findings and recommendations in boldface. Next are details of HIV medications including whether the patient report of his or her medications matches the medical record medication list. Patient’s self-report of adherence and self-reported medication-taking barriers come next. The final portion summarizes depressive symptoms, alcohol use, and illicit drug use. ACASI indicates audio computer-assisted self-interview; MedCHEC, Medication for Chronic HIV Education and Collaboration.
Adherence Care Manager
The ACMs selected for MedCHEC intervention have expertise in HIV medications, with either clinical or pharmacy background, as well as motivational interviewing skills. He or she contacts the patients by phone within 1 week of their completion of the ACASI and initiates a series of up to 5 phone counseling sessions to discuss and overcome the adherence barriers. As a background for these phone sessions, the ACM has electronic access to the patient report and the provider report, including the patient responses to individual items in the ACASI. The ACM has a resource manual that guides his or her counseling of patients on a wide variety of adherence-related topics, from taking pills in the workplace to avoiding poor adherence on weekends when schedules change or when alcohol or other substances may be used to a greater extent. When the ACM talks to the patient, he or she uses a philosophy of patient-centered care. 47 This includes emphasis on active listening—eliciting patient concerns, wants, needs; minimal use of directive comments; and shared decision making. The ACM helps patients identify and eliminate barriers to HIV medication adherence. The ACM keeps a record of conversations with patients including duration of the call, narrative summary of main points, and whether the patient set an adherence-related goal to achieve before the next phone call. The ACM is an element of MedCHEC that was not a part of the pilot study.
Pilot Study Methods
We conducted a pilot of the MedCHEC intervention with HIV-positive patients at the San Diego VA Medical Center. The goal was to assess the feasibility of using the MedCHEC ACASI and intervention in HIV clinics and to gauge patient and provider perceptions. At the time of pilot testing, the intervention included the computerized patient questionnaire (regimen confirmation, adherence assessment, reasons for missing doses, depressive symptoms, alcohol use, and drug use), tips for patients of ways to overcome specific adherence-related barriers, and the provider report. It did not include either the patient report or the use of the ACM. These 2 components were added as a result of the findings from our pilot testing, and consistent with the findings from another research team, 48 that simply giving HIV clinicians more information about their patient’s suboptimal adherence did not increase patient adherence.
We invited 110 English-speaking patients who received care at the San Diego VA Medical Center, and were taking at least 2 ART medications to use the touch screen ACASI device. Seventy-two patients agreed to be in the study and completed the informed consent. Eleven of these patients’ providers (10 physicians and 1 nurse practitioner) completed a questionnaire describing their patients’ medication regimens and estimating missed doses over the past 3 days and past 30 days. At the end of the study, patients completed a written 7-question exit survey that asked (1) how easy it was to complete the ACASI, (2) how much they enjoyed completing it, (3) how much completing the ACASI made them think more about how they take their medications, (4) their willingness to complete a similar assessment in the future, (5) how important it was for their physician to see their answers, (6) how well they felt the assessment reflected how they took their medicines, and (7) their comfort in using a computer. Providers were interviewed at the end of the study by one of the authors (ALG), following a semi-structured interview guide to assess the acceptability of the ACASI. Topics covered experience participating in the pilot study, what it was like having their patients use the ACASI, what they thought of the provider report, whether they found it helpful in evaluating patient adherence, and how the ACASI might be improved and better integrated into clinical care. Statistical analyses included estimating frequencies and means and preparing crosstabulations of variables. We also fit bivariate logistic regression models to examine associations between provider missed nonadherence (ie, patient reported being nonadherent while provider assessed the patient as being adherent) and patient demographic characteristics. Our definition of adherence was based on a cutoff of ≥95% of HIV medication taken.
The patient report and the ACM were not part of the pilot study. They were added afterward as a response to the pilot study findings and to research that was published after the pilot study. 48
Results
Pilot Study Results
All patients completed the ACASI successfully. Ninety-one percent of the patients were able to complete the ACASI after brief instruction by the research assistant (RA), while 9% needed additional RA assistance. Less than 10% elected to use the headphones. Most reported that the ACASI was easy to complete (77% “very easy”; 21% “somewhat easy”). Similarly, 90% of patients selected the top 2 ratings for all other items—enjoy completing it; made them think more about how they take medications; willingness to complete a similar assessment in the future; importance of their physician seeing their answers; how accurately the assessment reflected how they took their medicines; and their comfort in using a computer.
Pilot participants were mostly men and 51% were 50 years of age or older (Table 1). Medication errors, depressive symptoms, and substance use were all common. More than a third (36%) made at least 1 error in reporting their medication dose instructions, including identifying medications not actually prescribed or failing to identify prescribed medications (17%); incorrectly identifying doses per day (32%); and incorrectly identifying number of pills per dose (32%). Using ≥95% cutoff, 63% of patients achieved 3-day adherence and 47% achieved 30-day adherence. Mean ACASI completion time was 9.2 minutes, ranging from 3.2 to 27.9 minutes.
Pilot Study Results (N = 72).
Abbreviations: ARV, antiretroviral; PI, protease inhibitor; NNRTI, nonnucleoside reverse transcriptase inhibitors; CESD, Center for Epidemiologic Studies Depression scale.
We compared provider judgments of their patients’ adherence with patients’ reported adherence (Table 2), using 95% as the adherence definition. Provider judgments differed from patients’ in 46% (33 of 72) of cases for 30-day adherence and 39% (28 of 72) for 3-day adherence. “Missed nonadherence” (provider judges that a patient is adherent, but the patient reports being nonadherent) is a particular clinical concern. Over 30 days, missed nonadherence was detected in 22% (16 of 72) of the cases, and over 3 days, in 14% (10 of 72) of the cases. Notably, missed nonadherence was more common in older patients (odds ratio [OR] 3.4, ≥50 years vs <35 years) and in college graduates (OR 9.0 vs high school), suggesting that provider biases may lead to greater risk that patient adherence problems could go unnoticed.
Most commonly reported reasons for nonadherence were forgetting, reported by 31%, being out of pills (14%), away from home (11%), busy with other things (9%), changed schedule or work routine (8%), and fell asleep or slept through dose (8%). The provider report included cues to use with his or her patient, customized to the pattern of responses given by the patient using the ACASI. The cues addressed behavioral issues prominently featured in provider counseling guidelines, for example, encouragement, linking patient to other resources, and empathy. 49 –51
Provider Estimate of Adherence Compared with Patient Report.
Abbreviations: ACASI, audio computer-assisted self-interview; MedCHEC, Medication for Chronic HIV Education and Collaboration.
a Current version of MedCHEC ACASI does not have 3-day adherence.
Providers reacted favorably to the ACASI. They believed patients disclosed more nonadherence, that it saved providers time that they would otherwise have spent probing patients’ adherence, and that it provided valuable additional information not collected in the medical record or during the clinical encounter. Medication Event Monitoring System (MEMS) caps, which help provide an objective measure of adherence, 52,53 were used without difficulty by 17 patients. Mean adherence as assessed by MEMS was 79% of doses taken (SD = 23%). Suboptimal adherence (<95% of doses) was 53% (9 of 17).
Initial Qualitative Findings from Randomized Trial
In our ongoing randomized trial of MedCHEC, we have encountered some predictable implementation challenges. At one facility, there have been occasional wireless Internet lapses and printer problems (eg, lack of ink and paper jams), which caused brief delays in intervention delivery. Also, HIV medication formularies change over time, necessitating changes in the ACASI. A new nevirapine (NVP) formulation caused a brief period during which the ACASI indicated a NVP reconciliation problem in a small number of patients, when in fact none existed. We updated the software to eliminate this problem.
Discussion
We developed an intervention to improve ART adherence that involves improved assessment of nonadherence, facilitation of patient–provider conversations about medication use, and phone-based counseling for patients identified as having adherence difficulties. The intervention incorporates touch screen technology to assess patient adherence, feedback of information to patients and providers through brief printed reports, and provision of targeted, phone-based adherence counseling. The patient and provider reports summarize patient understanding of their regimen, self-reported adherence, and self-reported barriers. Patient reports also include tips, tailored to the individual patient’s reported difficulties, for overcoming specific adherence barriers. Unlike previous ACASI-based HIV medication adherence interventions, 25 MedCHEC integrates counseling from an ACM. The ACM component was added because research has suggested that on their own, physicians may not respond adequately or effectively to information about their patients’ nonadherence, neglecting to counsel them, or at best to provide ineffective counseling. 48 With training and dedicated clinical time, we suggest that ACM support can help patients overcome barriers and improve adherence.
A substantial body of research shows that, across a variety of health conditions and classes of medications, providers often overestimate patient adherence to medications. 19,54,55 In HIV care, providers are often surprised by the nonadherence and medication-taking errors of their patients. Bangsberg et al found that patient’s use of CASI helped clinicians uncover potentially harmful medication-taking errors and gave them a fuller picture of patient adherence to multiple medications. 56 This kind of information is rarely discovered in typical HIV clinical care encounters, because of the demands on clinicians to address many HIV and non-HIV issues in a relatively short period of time. Our pilot findings concurred with other research, 18,54 demonstrating that patient medication errors were overlooked, and that providers overestimated how well their patients were adhering to ART. We found that 36% of patients made at least 1 error in describing their prescribed ART regimen, and provider assessments missed nonadherence 22% of the time for 30 days and 14% for 3 days.
In spite of the large, high-quality color photographs of the pills, many pilot study patients incorrectly identified their medications or incorrectly stated the number of doses per day or number of pills per dose in their regimen. This error rate may partially reflect the more complex regimens that the patients were on when they took part in the pilot study. In the past few years ART regimens have become simpler, requiring only 1 or 2 pills a day. Nevertheless, this finding is another reason for systematically evaluating patient understanding of how they are supposed to take their medications. Providers may need to regularly review medication instructions with their patients. Additionally, instructions for taking the most common HIV regimens could, in the future, be provided to patients in video format either on the ACASI or via the patient’s electronic personal health record.
Our pilot findings suggest that HIV clinicians operate with uncertainty about how well their patients understand their regimen and are adhering to medication—this uncertainty may lead to clinical inertia. 57 The MedCHEC intervention could help reduce this uncertainty. Hyman et al 58 recently reported on a randomized trial of a hypertension intervention in which physicians were offered tools to closely track blood pressure (BP), medication adherence, and lifestyle assessment and counseling. Of the patients whose adherence was monitored, 30% were nonadherent. Clinicians in the intervention arm, who received detailed information about medication adherence and about BP levels, were more aggressive about drug titration, and their patients had improvements in BP control.
The MedCHEC intervention capitalizes on the specialized counseling skills of the ACM, while relieving the HIV clinician of some of the adherence counseling task—a task for which some HIV doctors may have little time, training, or inclination. 48 MedCHEC may save clinicians’ time by reducing the need for them to use the clinical encounter for in-depth questioning about patient nonadherence and barriers to adherence. MedCHEC collects the detailed adherence information and risk factors for nonadherence in an unbiased, standardized manner. Clinicians have the opportunity to reinforce the importance of medication adherence by reviewing the provider report, discussing it with the patient, and encouraging him or her to use the ACM counseling.
A future improvement involves incorporating MedCHEC into Web-based personal health record systems so that the patients can complete the ACASI at home, online, prior to coming in for their HIV visit. Mobile phone versions are also feasible. In our pilot testing, a research assistant handed the touch screen device to the patient, retrieved it from them, and printed out the reports. These functions in most cases should not require additional full-time equivalent staff nor impose undue burdens on existing staff.
There are a number of potential problems that, in theory, could interfere with the success of MedCHEC in clinical settings. Patient self-reported adherence (as recorded by the ACASI) may “improve” as patients realize that doing so will avoid difficult conversations with their clinicians; similarly there is no guarantee that clinicians or patients will use their reports for enhanced discussion of adherence difficulties. And, one should not forget that ACASI, while having a number of advantages over direct clinician questions about adherence, is nonetheless a patient self-report and subject to biases related to recall and social desirability. Our current randomized-controlled trial, which uses MEMS with all participants, will help evaluate the extent of this bias.
The ACASI-type adherence interventions may not be suitable in all environments and with all patients. Some patients may be unwilling or unable to use computerized assessments, while some clinics and clinicians may find the intervention interferes with patient–provider relationships. Our pilot results, however, suggest that most patients are willing and able to use ACASI technology, and that patients and providers believe it provides substantial benefits for medication adherence. Similarly, ACM phone counseling may pose challenges in resource-constrained HIV clinics in terms of both the clinic’s ability to pay an ACM and the patients’ reliable access to a telephone. The proliferation of inexpensive cell phones, 59 however, will help minimize this barrier.
The rising use of technologies in society, in general, and in health care fields, in particular, is likely to increase the acceptability by patients and clinicians of tools such as the MedCHEC ACASI. 60,61 While rates of technology adoption are slower in certain groups, for example, older patients, lower socioeconomic patients, they are still on the increase; and surveys indicate that the presence of chronic health conditions (often more prevalent in older and lower socioeconomic status patients) is associated with greater use of Internet for health-related purposes. 62 As health systems and organizations continue making progress implementing electronic health records in response to federal incentives, 63 it will become easier for tools such as MedCHEC to link to electronic pharmacy databases 64 thus when new HIV medications become available, the ACASI will automatically update the names and photographs that are displayed to the users.
In summary, our pilot study indicated that clinical use of ACASI can help identify patients who may be making medication errors, are nonadherent, and who otherwise might go undetected by their providers. The MedCHEC intervention, developed from our pilot study, enhances a basic ACASI with important new elements, such as a patient report of medication adherence and barriers, and an ACM who conducts phone counseling. The design is intended to help patients and providers identify and understand problems with adherence, and then give patients the guidance and support to achieve and sustain high medication adherence rates. A controlled trial of MedCHEC is ongoing.
Footnotes
Acknowledgments
The authors thank Lee Ann Lowe for her invaluable management and organizational work to launch the MedCHEC study.
Authors’ Note
The views presented here do not reflect the views of the Veterans Administration or the US Government.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Coauthor Hofmann has a potential conflict of interest in his role as president of West Portal Software Corporation, whose software was used in the computer application described in this manuscript. We believe our manuscript provides a fair and balanced view of the software, including its weaknesses as well as strengths. His company’s software has been used in prior research studies that have been reported in peer-reviewed academic publications.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the National Institute of Mental Health (#1 R01 MH076911-01 A2); the Veterans Health Administration (VHA) Health Services Research and Developoment Service Quality Enhancement Research Initiative for HIV and Hepatitis (#HIV 98-001); and by the VHA Career Development Program (#CDA 09-016).
