Sage Journals: Discover world-class research

Abstract

The Diabetes Self-Management Education and Support (DSMES) program provides education and medical monitoring of diabetes to Veterans through the Virtual Medical Center (VMC). Qualitative interviews were conducted with 15 key stakeholders (4-DSMES VMC trainers, 5-clinical faculty, and 6-Veterans) from across Ohio urban and rural populations for up to 1h about their experiences using the program and suggestions for improvement. All the Veterans interviewed were able to access care within the DSMES VMC and reported a positive experience using the program, and improved diabetes self-management. Other stakeholders suggested more administrative and technical support for the DSMES VMC to increase awareness for VA staff and Veterans of the program to improve recruitment, and to shift to a web-based platform that is more easily accessible by clicking a link to reduce technical issues with downloading the program. These findings can inform future implementation efforts using technology to increase access to care allowing better health education for Veterans.

Keywords

Veteran diabetes virtual telehealth qualitative methods

Introduction

The importance of self-management of chronic illnesses such as diabetes is becoming increasingly important especially in the Veteran population where approximately 1 in 4 Veterans enrolled in the Veterans Health Administration (VA) are afflicted with diabetes mellitus.¹ Diabetes Self-Management Education and Support (DSMES) was developed in the VA as a collaboration between a diabetes team and Veteran to ensure the Veteran gains knowledge and skills to successfully self-manage their diabetes.² This program was based on the National Diabetes Self-Management Education and Support toolkit from the Center for Disease Control and Prevention (CDC) and achieved accreditation and recognition by meeting the 10 national standards of excellence by national diabetes groups such as the American Diabetes Association.³ DSMES participation can help improve quality of life through changes in health behaviors (eg, improved nutrition and medication adherence) and prevent diabetes complications such as heart disease and chronic kidney disease.⁴

Many VA sites offer in-person diabetes shared medical appointments for groups of Veterans to meet with a multidisciplinary team for diabetic care management and learn self-management.⁵ Unfortunately, few VAs provide DSMES services. Restrictions presented during the COVID-19 pandemic and travel burdens for rural Veterans halted the in-person DSMES program triggering the use of the VA Virtual Medical Center (VMC).

The VMC, developed in 2015, is a virtual environment for Veterans and their family members to access health information from home. The VMC serves as a platform that facilitates social connection for Veterans and offers virtual telehealth technologies and digital game-based learning. This approach provides Veterans with access to educational resources, diabetes management tools, staff support, and eliminating the need to travel to the nearest VA hospital.⁶ In March 2020, the COVID-19 pandemic prompted the VA to shift from in-person visits to pandemic-related safety protocols requiring virtual care delivery.⁷ For Veterans with diabetes the VMC can help reduce risks of comorbidities to diabetes such as heart disease, hypertension, obesity, and cancer.⁸

Digital game-based learning has been applied across diverse sectors including business entities, academic science,⁹ and education¹⁰ to achieve objectives and facilitate training programs. The VMC employs gamification, a design approach that incorporates gaming engagement and motivational elements to reinforce desired healthy behaviors like dieting and exercise.¹¹ When health education strategies integrate gamification, they transform the participant's intrinsic motivation from completing a mandatory task into a captivating and immersive experience.^12,13 Medical avatars are used within the VMC to make the virtual environment more visually compelling and adept at motivating Veterans to meet personal goals. The use of avatars resembling older participants in a 3D virtual reality intervention showed improvements to lifestyle changes such as physical activity.¹⁴ Although the VMC offers a virtual platform for Veterans to receive DSMES remotely, this approach also has challenges related to technical aspects. Veterans who live in rural areas face significant challenges accessing healthcare due to limited technological infrastructure such as digital literacy and broadband issues that hinder their ability to utilize telemedicine services.^15,16 The advanced age of many Veterans exacerbates the issue, as they may lack familiarity with modern technology, posing a barrier to effectively navigate and benefit from digital healthcare solutions.^15,17

Older individuals’ willingness to accept virtual healthcare is currently being investigated. Older Veterans may not have the technological devices or knowledge of how to successfully participate in the VMC.¹⁷ Veterans who used the DSMES VMC appreciated the chance to connect with peers who shared the same disease while they participated in interactive games such as virtual grocery shopping, virtual fitness center, and virtual cooking classes to learn about the effect of diabetes on their bodies, average blood sugar levels, and glucose metabolism. To identify factors that contribute to successful implementation and inform future implementations, a formative evaluation¹⁸ of the VA DSMES VMC program was conducted to learn more about the users’ experiences through semistructured interviews. A formative evaluation uses methods like a summative evaluation but emphasizes a goal of identifying ways to adapt and improve the implementation of the subject of the evaluation.¹⁹

Methods

A formative evaluation of the DSMES VMC was conducted using qualitative methods through semistructured interviews with healthcare providers and their patients. We asked providers about their experiences working in the VMC and to help manage their patients with diabetes mellitus; we asked patients about their experiences and managing their diabetes after attending DSMES VMC sessions. Our goal was to identify facilitators and barriers for improvement of the VMC to inform successful future implementation. Semistructured interview guides (see Appendix I) were created by combining the Technology/Task Fit and Unified Theory of Acceptance and Use of Technology (TTF-UTAUT) model.²⁰ This combined model was chosen because of the novelty of the VMC and the different types of VMC users. The interview guide assisted in providing elicit descriptions of stakeholders’ experiences with using the DSMES within the VMC, recommendations, and suggested improvements for the program. Telephone interviews were approximately 20-60 min each.

The VA Northeast Ohio Healthcare System's Research and Development Committee review and determination of the evaluation plan²¹ and deemed it nonresearch and quality improvement activity designed for internal VA purposes on August 30, 2020 (VHA Handbook 1200.21). The interview guide was approved by employee unions before administering. A DSMES program lead provided names of Veterans who participated in DSMES classes within the VMC, clinical faculty who conducted the courses and met with their patients within the VMC, and trainers involved in training the faculty to provide this education within the VMC. Clinical faculty were invited to participate in a 1h telephone interview by VA email. Veterans were invited through letters outlining the interview's purpose for program improvements, requirement for verbal consent, recording of information, and assurance of name removal for publication. Veterans were asked to respond to the letter if interested in participating in interview by calling the evaluation team for further clarifications. Invitations for participation were emailed to 17 clinical faculty and 6 trainers. Letters were mailed to 13 Veterans. Once the evaluation team received consent, an appointment was scheduled, and participant ID created.

Interviews were led by a team of interviewers with over 20 years of combined experience in qualitative data collection and analysis. The interviewers were trained in health services research and quality improvement implementation to inform, advise, and evaluate strategies designed to improve access to primary and specialty care. Training is completed when team members are onboarded. Interviewers followed a standard script explaining the interview process including the confidentiality and voluntary nature of the interview. Interviewers also adhered to the open-ended questions in semistructured interview guide adding only grounded probe questions (Appendix I) using the respondent's words throughout the interview.²² During the interview each participant consented to being audio-recorded. The data were anonymized, stored on a VHA secure server, transcribed verbatim, and quality checked by a member of the analysis team. To analyze the qualitative data a rapid matrix analysis^23,24 was completed using a priori neutral domains derived from the UTAUT model to code data from transcripts of interviews with Veterans (Appendix II) and clinical staff and trainers (Appendix III); adding inductive domains as identified. The matrixes were populated with data from each transcript by members of the qualitative analysis team. For each populated domain, analysts sought to identify barriers and facilitators to program implementation and identified targeted actionable steps to improve the program. Analysts met weekly to discuss and build consensus in team meetings where any coding discrepancies were identified and resolved often by refining code definitions to develop consistent coding of data within the matrix. Analysis was ongoing and iterative; analysts referred back to interview transcripts to ensure that the identification and assignment of codes were grounded in the data. The completed matrix was discussed by the entire team to validate results.²⁵ Results from this study could inform adaptation of healthcare delivery when a virtual modality is preferred or necessary.

Results

Fifteen (n = 15) key stakeholders (4 VMC trainers, 5 clinical faculty, and 6 Veterans) across Ohio urban and rural populations agreed to participate and were interviewed. Data collection was completed between February 2021 and June 2021. The focus was to gain knowledge of the experiences with training providers on how to use the VMC to provide care and receiving or providing virtual DSMES care within the VMC. Perceived facilitators and barriers of the VMC were also explored. Most trainers reported about their experiences using the program, providing care, and suggestions for improvement, while clinical faculty reported challenges with downloading the application. Despite these issues, Veterans who were able to access care within the VMC reported a positive experience using the program, better self-management, and easier access to diabetes care. Stakeholder's quotes can be found in Table 1.

Table 1.

Stakeholder Quotations.

Domain	Quotations
Veterans experience
Facilitators: Discussion/Review of Platform	“I would most definitely recommend the VMC to others.” (Vet_001) “I would recommend it's a way to continue to learn.” (Vet_004) “It keeps them (Veteran) from having to physically go talk, go see someone uh especially if it's a disabled Veteran who can’t get out as, as easily. I highly recommend it.” (Vet_003)
Facilitating Conditions (Ease of Use)	“Well just being able to get in there and not have to take a trip down to the VA itself and be able to do the classroom and the experience of going through the classroom virtually like that was really easy and kind of helpful cause you could do it right from your house, from your laptop.” (Vet_005) “I like it, it's easy to use.” (Vet_004)
Motivation	“I thought it would be interesting to do.” (Vet_005)
Behavioral Intention (Goals)	“When I was told about it and asked if I’d be interested in trying it, I didn’t have any specific goal I just wanted to see what it is and I guess my goal was to see how it could help me in what I could do to improve my health by using it. I just wanted to see how it could help me… like going to the meetings and stuff, doing this it saves me a lot of time. I don’t have to drive down to the VA, I don’t have to drive back home and I can do this any place I have an internet. So I don’t have to be at home. I can be on the road and log in to my diabetes.” (Vet_006) “I’d like to use it for all my appointments that don’t require me to sit down at the VA center.” (Vet_005)
Use Behavior (Results)	“I weighed 250 pounds with high sugar, I started caring more about my health, now I’m below 200 pounds and I walk 2-3 miles per day.” (Vet_003)
Benefits	“If you have questions about things that are going on with you, how to deal with stress, how to learn how to breathe, how to eat differently. You know, all of those things can be found in the VMC. I’m just saying if you want to better yourself with knowledge, go to the VMC.” (Vet_001) “If you don’t know enough about your disease you need to be in there and get the information.” (Vet_001) “Learn more about the disease and how to cope with it.” (Vet_002)
Resources	“Now I’d go back to that Medical Center [VMC], it's more targeted to what I’m looking for… On google you have to keep looking further into something.” (Vet _002)
Suggested Improvements	“I feel my experience has been pretty good and I look forward to the next time I use it.” (Vet_001) “I didn’t have a bad experience at all, except the amount of homework.” (Vet_002) “Can’t think of anything to improve.” (Vet_003) “Mostly if there had been more people. We just went from the main atrium to the classroom.” (Vet_004)
Other	“Addition of the VMC has been a knockout … I just hope it continues.” (Vet_001)
Use of Avatars Facilitating Conditions (Ease of Use)	“I’m not one of these people that like to be in a crowd or being around a whole bunch of people so sitting in classroom thing is not me so doing it virtually like we were with the avatar, and everything fit right into my style.” (Vet_005) “Well, it[VMC] allows us to have group meetings or one-on-one meetings for that matter and it's in a, well it's in a virtual space, in an office, in an auditorium. It's just like being there only you’re not there. Your avatar is there. Your avatar, you get to build to look whatever way you want it to look. I guess most of us try to look as much as we ourselves as we can. it's a normal presentation…it's interactive. I mean literally, it's just like being there.” (Vet_006) “Well, everybody involved has an avatar that is there in the meetings sitting there. It's a spacebar on your computer and well you can talk anyway. You can be heard. You can ask questions. There's a way to do messaging if you’re not comfortable talking, you can type in whatever message you want. Instant messaging” (Vet_006) “I really like the hands on, I really like the vision meetings, but it was okay. It took a little while to get used to it, all the nuance and different, sit down, stand up, walk, so it took a minute but once I got it, it was okay. It was kind of like a game.” (Vet_ 004)
Barriers: Burdens/Challenges: Technical Issues Facilitating Conditions (Ease of Use)	“The hardest part was setting it up. I had to get with a technician to set it up and we had, it took a while, I mean we worked it out but it took a while to get set up on my computer. But once I got it, it was real easy.” (Vet_004) “Following parameters and helped by educator or the help category. It was easier to ask the educator and they could get you right where you needed to be.” (Vet_001) “A tag team of providers working with us. It was nice to have two voices on the different modules, but they were wearing the same outfit.” (Vet_002) “Like I say, you really don’t have to be computer savvy. It’ll be great when it becomes Web-based so that anybody can get to it. Cause like I said, you need some technical support to download it. And, the technical support is excellent.” (Vet_006) “Once they told me, once I got instructed, instructions how to do it, it was cool. I just did it.” (Vet_004)
Burdens/Challenges: Attendance Facilitating Conditions (Ease of Use)	“A lot of homework, if there were more Veterans in there you couldn’t do two modules at a time. You wouldn’t get through them.” (Vet_002) “Everything was helping but there wasn’t enough people in the class.” (Vet_004) “I like to sit back and listen but because there weren’t a lot of us I had to be engaged.” (Vet_004)
Clinical Faculty Experience
Facilitators: Experience	“Once I heard about it, I thought it was definitely something I wanted to get involved with because it would definitely increase access to Veterans and the biggest problem with the DSMES in our country right now is that every VA has a Diabetes Self-Management Education and Support program and the ones that do have them, the class times are not available to every Veteran so, you know, the lack of access both what, you know, geographically where you’re at and the lack of access for the lack of availabilities for days and times that the classes or programs are available. So, the VMC will help with both of those and that's why I wanted to definitely get trained on it, it could be the future.” (Fac_005)
Social Influence/Conditions	“With some effective and efficient changes and more support administratively, it would be a pretty good platform.” (Fac_004) “I would tell (colleagues) that it is a platform for Veterans to have 24/7 education at their fingertips. It gives them the ability to connect with other Veterans and also provides education material on a wide variety of medical conditions and it's great for the person who wants to do their own research and learn at their own speed” (Fac_001)
Facilitating Conditions (Ease of Use)	“I think it's a great platform to bring people in one area so then way you can teach.” (Fac_004) “It's really got a lot of information that is in one area so that I think is very neat and good, that Veterans can kind of just go and explore at their own will.” (Fac_002)
Attitude	“I think you have to use it more and more to get comfortable with using it.” (Fac_003) “It was an okay experience.” (Fac_004)
Behavioral Intention (Goals)	“Education available 24/7.” (Fac_002) “Great for independent learning.” (Fac_003)
Use Behavior (Results)	“Better outcomes as in behavioral change, you know, meeting their personal goals with making changes to their diet or checking their blood sugars or coping with stress and then clinical outcomes would be like improvement in their A1c, improvement in their blood pressure, improvement in preventing a foot infection.” (Fac_005) “it's a unique platform and having access to education 24/7 is a benefit” (Fac_001)
Benefits	“There's more access to more education virtually than just paper you know, books and whatnot.” (Fac_003) “I think they’re [Veterans] excited to be able to be part of something any time they want so I really think that, you know, as VMC continues to grow, it can keep that, that connectiveness that Veterans are missing especially with COVID that they would be able to go and connect with one another in the public area and so, I think there's really some potential there for them to connect, especially with Veterans all, all over the United States.” (Fac_001)
Other	“Oh, they [Veterans] love it because they have those resources, you know, like I said, the videos, they have the library there, they can click on, and they can interact with each other. If they want to make their own support group, they can do that. I think there's more access to more education virtually than just paper you know, books and whatnot. I think it's a nice, a different medium for education.” (Fac_003)
Barriers: Burdens/Challenges: Technical barriers Facilitating Conditions (Ease of Use)	“So there's interest out there but it's, it's been a little bit difficult to have the Veterans actually understand the platform and have that technology at home today I think is one of the barriers too, so, I mean at this point it has not been available by tablet or smartphone, which is a big barrier because many Veterans just doesn’t have a desktop or a laptop to utilize the program.” (Fac_001) “Younger Veterans probably would utilize this [VMC] more than our older Veterans …our patients are elderly they don’t always want to do this type of technology.” (Fac_003) “It's been difficult to have the Veterans understand the platform… someone to walk them through setup as clinicians we don’t have that kind of time to spend the 15–20 min getting them signed up.” (Fac_001) “This is a great platform to use has some excellent information if they’re able to download the software and if the computer is able to have access to that software, if they’re able to have enough RAM.” (Fac_004) “Such a small band of Veterans that are able to do the VMC because of the lack of computer skills and the lack of their availability of a nicer computer to do it with.” (Fac_005)
Suggested Improvements	“So having more of our management support so that they’re encouraging Veterans to get signed up for VMC even if they’re with someone in our clinics who were at the front desk just notifying patients that we have even such a program available.” (Fac_001) “More support administratively and technically.” (Fac_004) “Need admin support to help Veteran learn how to access, move, gesture, chat, etc. Needs to be smaller and easier to use…make the clinic one room… put the posters closer together… no need to walk down hallways…make movements easier like a video game.” (Fac_005) “Need to be able to use a joystick or more like gaming technology.” (Fac_002)
Burdens/Challenges: Referrals Suggested improvements	“Definitely need more support and practice with the Veterans just exploring the VMC so that's been the most difficult part is getting Veterans signed up for the VMC.” (Fac 001) “Our fear is that you get someone in there and if it's a lot of work for them to do they’re not going to want to do it.” (Fac_005) “Figuring how we’re going to get Veterans recruited other than the ones we’re trying to do by ourselves our Veterans rather than other Veterans.” (Fac_001)
Trainer Experience
Facilitators: Benefits	“I think the biggest thing that was convenient on the VMC is having a place you can go to get information without having to Google and search and try to figure out what is reliable information or not.” (Trainer_001) “It [VMC] allows people to stay home and still be able to function in kind of getting information they need or even getting the coaching they need.” (Trainer_001) “We have a standardized curriculum, we have a standardized process and this [Education] can be delivered outside of the Virtual Medical Center.” (Trainer_002) “They [Veterans] can access 24/7 and its loaded with information that they can, they can use anytime.” (Trainer_003) “I enjoy it. I like the fact that you can still be in a remote environment working from home whether you’re a Veteran or provider working through an avatar and still be able to create the type of person, the type of avatar you want – still gives you a little bit of privacy and you’re not forced to, show your face on camera and things that, that you have to do when you’re doing virtual education so I like that you still have privacy because even, without people realizing that you’re looking in people homes and, and I really don’t like that to be honest with you so I like the privacy and then you’re able to access at your convenience.” (Trainer_004)
Barriers: Burdens/Challenges	“I could see that being a big barrier for someone that didn’t really have to, to access it or get to it so a lot of times uh just anecdotally Veterans, if it's a hassle they’re not going to do it regardless, even if it could save their lives, they’re not going to do because it's a hassle so I could see that access, getting access to it which again with the new web-based program it supposedly is not going to be that difficult. It's going to be like, you’re going to hit a link just like you would with any website and go directly there. So hopefully will eliminate some of those issues.” (Trainer_001) “We have Veterans that love it that are really interested but its hard sometimes to talk them into it initially and a lot of that has to do with loading that program and it's a very large and it's a very secure program and so often the Veterans have to actually disarm their firewall on their computer just to get it loaded so that's, you know, if they have any qualms about it, and its a good program but yeah you can usually load a program on your computer without disabling your firewall so I hate to ask the Veterans to do that but I’ll tell them, you know, the DoD and the VA is using this, the firewall goes right back up as soon as you load it but I don’t even know how to do that so I always have to refer them to somebody else to make that happen.” (Trainer_003)
Other	“I would like to see, my expectations is that patients will use this, that they’ll benefit, that we’ll see some improvement in any behavior they decide to choose. I also would like to see more engagements, maybe within their own, you know, classes, like have conversations with other participants. But definitely I would like to see some type of behavior change within this.” (Trainer_004)

The table above lists supporting quotations for results.

Veteran Experience: Facilitators and Use of Avatars

All the Veterans interviewed had previous diabetes education in-person with providers, but they agreed that they learned how to better manage their diabetes and implement healthy lifestyle changes through the VMC. The VMC gave Veterans the opportunity to join the program anonymously through the patient avatars instead of using a camera for live video feed. This virtual learning environment was preferred by some of the Veterans because of its accessibility with no requirement to travel to the VA and the materials available for diabetes management. One Veteran shared how it took some time to get use maneuvering the avatars around the virtual platform but “once you get the hang of it” it was like a game. One Veteran highlighted how the VMC was targeted for everything he wanted to learn whereas other resources like Google “you have to keep looking further into something.” All the Veterans reported that they enjoyed learning about their disease, hoped it continued, and would recommend the VMC to others.

Clinical Faculty Experience: Facilitators and Perspectives

The clinical faculty described many benefits of the VMC. For example, one clinical faculty shared that the VMC provided more health information for Veterans than a single in-person appointment. Another explained how the VMC helped the participating Veterans meet their personal goals, as well as better health outcomes such as improvements in blood pressure, diet, and blood sugar control. One clinical faculty also described how the Veterans connected with each other and had the option to create small groups in the VMC.

Barriers and Challenges of DSMES Within the VMC

Technical issues relating to downloading and initial set-up on personal home computer, long patient instructions, and duplicate avatars were barriers reported by most of the stakeholders interviewed and may account for the low number of VMC users. A Veteran mentioned how the number of avatar options was limited so multiple faculty in one class looked the same. Most Veterans reported satisfaction with the technical support given to use the VMC. One Veteran reported an easy log-in experience when using mailed instructional materials and receiving assistance from the help desk. A clinical faculty recognized the need for the VMC to be more user friendly. Sometimes teaching Veterans to use the program required an extensive time commitment. Another clinical faculty suggested making the VMC more user friendly with easier movements like a video game using a joystick. One clinical faculty reported on how Veterans struggled to load the desktop application due to the size of the program. While VA Office of Information Technology (IT) could assist Veterans with technology issues, faculty also needed computer support during and prior to DSMES sessions. A clinical faculty reported difficulties getting Veterans referred to use the VMC due to the lack of awareness about VMC. They also stated that there was a need for administrative and technical support to help Veterans explore the VMC. The clinical faculty needed other providers to refer Veterans to the classes offered through the VMC.

Trainer Experience: Involvement and Suggestions

One of the trainers interviewed explained how they became involved with the VMC because of the increase of access it provides Veterans and that they would recommend the VMC to others. Another trainer recognized the technical issues that some Veterans had while trying to access the VMC and suggested shifting to a web-based platform that is more easily accessible by clicking a link to reduce issues with downloading the program.

Differing Points of View

The clinical faculty and trainers expressed concerns about the VMC that were not uniformly shared by Veterans. All the clinical faculty conveyed concerns about technical barriers in the VMC, expressing apprehension that older Veterans might be deterred or lack the required equipment for active participation. Additionally, clinical faculty reported that the VMC lacked user-friendliness. To mitigate the technical issues associated with downloading the program it was suggested that the VMC needs to transition into a web-based platform to enhance accessibility. Some clinical faculty believed there was a low attendance of Veterans in VMC due to insufficient awareness and inadequate support for the program. However, most of the Veterans interviewed in this evaluation reported not having issues with the VMC. Veterans highlighted how they asked the clinical faculty or used the help category for assistance from VA Internet Technicians when needed in the VMC. A few Veterans explained that technical support was excellent and that once they received the instructions on how to access the VMC it was easy to do and “like a game.” The Veterans unanimously gave positive feedback about their experience with VMC, even remarking how they would recommend the program to others. One Veteran stated a preference for utilizing the VMC for all appointments to avoid the need to travel to the VA. Another finding from the Veteran interviews was the desire for increased attendance of Veterans for more interactions. If more Veterans participated in VMC there would be opportunities for Veterans to share experiences related to diabetes and medications. Overall, the Veterans found the program and platform relatively easy and enjoyable to access and use; the clinical faculty and trainers believed a few changes could make the virtual platform better.

Discussion

The purpose of this evaluation was to explore the experiences of VA staff involved with training others to provide diabetic education, VA providers involved with facilitating this education, and Veterans with diabetes who received DSMES within the VMC. Veterans who accessed DSMES within the VMC had positive experiences using the program reporting better self-management and easier access to diabetes education. VA clinical faculty and trainers reported advantages of the program for Veterans and VA staff, as well as the challenges in getting more technical support and Veteran referrals for the program. Our findings highlight the facilitators and barriers of utilizing the VMC via avatars and preferred virtual accessibility and suggests this is an important option for many Veterans, to improve health behaviors.

Virtual technology has been shown to increase access for preventive health and to resources that improve health such as recipes, exercise, stress and anxiety management.²⁶ The Veterans interviewed in this evaluation described the benefits of virtual learning from DSMES within the VMC, their ability to access it, and adopt healthy lifestyle changes. Studies have shown that technology can assist in helping Veterans shift to healthier habits.¹⁴ The Veterans interviewed were satisfied with the remote access to diabetes care, preferring to utilize the VMC rather than traveling to an in-person meeting. The VMC allowed Veterans to learn at their own pace in their home environment to reach personal health goals. Avatars within the VMC promoted an interactive experience. Previous studies have described the effectiveness of medical avatars for patient learning and health outcomes. One study reported how avatars with digital characteristics can serve as health coaches to foster health promotion and lifestyle changes in Veterans.¹⁴ Another provided evidence that showed psychosocial improvement for patients who used avatars in virtual healthcare environments.²⁶ Some Veterans referred to DSMES VMC as it being “like a game.” Gamification adds gaming elements such as rewards for participation or task completion and promotes motivation, engagement, and improvement for users.²⁷

One of the clinical faculty noted how older Veterans did not understand the technology for using the VMC. In previous studies, technology is known to offer abundant resources, but the absence in ease of use can become a barrier for elderly users.²⁷ Although the Veterans reported having technical issues, they also reported positive experiences receiving assistance which was “excellent” or “real easy.”

Clinical faculty stated that there was a lack of awareness for Veterans to know about what was offered in DSMES VMC. Enhancing administrative and technical support for the DSMES VMC could potentially improve Veteran referrals, thereby expanding program participation. A clinical faculty reported that more support was also needed to help the Veterans with exploring the VMC. Studies have suggested that elderly patients need support with virtual technology due to cognitive or physical abilities.²⁷ Participant barriers could be addressed by offering support to Veterans as they begin the program so that the clinical faculty have more time to focus on the DSMES education and less time teaching the programing basics (eg, walking, sitting, asking questions, etc). Based on our findings, future implementation of the VMC should consider packaging the platform into an application that is easy to download and install with technical support available for using the application.

Limitations

Our findings and recommendations are based on a small sample size due to the relatively low and limited number of VA providers, staff, and Veterans involved in the DSMES VMC at the time of this evaluation. Most respondents had not regularly used the VMC in a long time and in general said that they had “not a lot of experience” and that it “needs to be updated to meet Veteran needs, in order to be used.” Future studies that use mixed methods with a larger sample could offer further understanding of the faculty and Veteran satisfaction with DSMES sessions using the VMC platform.

Conclusion

Although the stakeholders interviewed noted some challenges in using the VMC DSMES platform, overall, respondents felt that the program increased access to resources on diabetes self-management. Veterans who were able to access care within the VMC reported a positive experience using the program reporting better self-management and easier access to diabetes education. VA clinical faculty and trainers reported the advantages of the program for Veterans and VA staff as well as the challenges and offer some suggestion for improvement. The VMC provided a virtual opportunity for Veterans to share experiences and learn to live a healthier lifestyle, especially during the COVID-19 pandemic when there was a disruption in care.

Footnotes

Acknowledgments

We are grateful to the Veterans Health Administration staff and Veterans who generously gave their time and shared their experiences with our team. We would like to acknowledge Michael Heh from the Employee Education Service department of Veterans Affairs.

Declaration of Conflicting Interests

The authors declared that there are no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical Approval

This evaluation was reviewed and declared to be nonresearch by the VA Northeast Ohio Healthcare System Research and Development Committee and did not require Institutional Review Board oversight.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Memorandum of Understanding with the Dayton VA Medical Center through the DOD/VA Joint Incentive Fund (JIF) National Defense Authorization Act 2003 Section 721, amended Section 8111 of Title 38 of the United States Code.

ORCID iDs

Tai-Lyn Wilkerson

Soumya Subramaniam

Appendix I. Semistructured Interview Guides.

Veteran Experience

Grounded probes/prompts: Grounded prompts are used to elicit detailed responses and should be used verbatim.

What do you mean by ____________?

Tell me more about___________.

Give me an example of ____________.

Tell me about a time when ____________.

Who ____________?

Where ____________?

What, if anything, was helpful about ________________?

What, if anything, was not helpful about ________________?

What, if anything, made ________________ difficult?

What, if anything, made ________________ easier?

Tell me about your experiences using the DMSES within the VMC.

How often to you access the system?

How did you find out about the system?

How did you decide to use the system?

Effort and Facilitating: Ease of use.

Tell me about times you used the system.

Probe: What components, if any, did you find helpful?

Probe: What components, if any, did you unhelpful?

Tell me about other on-line resources, if any, you have used to help you manage your health.

Performance: Results

What, if any, are your goals in using this system?

Have you met your goals?

Probes

To what extent, if at all, has your ability to manage your health changed?

Performance: Behavior change

Have you made any changes to your lifestyle since participating in this platform or using these resources?

Motivation

Under what circumstances do you expect to access the system again?

Would you recommend this system to another Veteran?

What would you tell Veterans about it?

Social Support

What, if anything, about this system have you discussed with your VA providers?

What, if anything, about this system have you discussed with your family, friends, and/or caregiver(s)?

What, if anything, could have improved your experience?

Is there anything else you would like us to know?

VA Trainers and Clinical Faculty Experience

Grounded probes/prompts: Grounded prompts are used to elicit detailed responses and should be used verbatim.

What do you mean by ____________?

Tell me more about___________.

Give me an example of ____________.

Tell me about a time when ____________.

Who ____________?

Where ____________?

What, if anything, was helpful about ________________?

What, if anything, was not helpful about ________________?

What, if anything, made ________________ difficult?

What, if anything, made ________________ easier?

What was the impact of ________________?

Tell me about your role at the VA.

Tell me about how you first learned about using the VMC.

Tell me about how you decided to use the VMC.

Training

Tell me about your experience using the VMC.

How often to you access the VMC?

Training

Tell me about any user training you may have provided to Veterans (For Faculty/CDEs/trainees) OR

To Faculty/CDEs (For Trainers)

Tell me about the training you participated in to learn how to use the VMC. (For Faculty/CDEs/trainee)

[probes]

Effort and Facilitating: Ease of Use. (Trainers and Faculty/Trainees)

Walk me through how you use the VMC.

Probes: What components, if any, did you find helpful? (Trainees/Faculty)

What components, if any, did you find unhelpful?

Performance: Results

What are your goals in using the VMC? (Trainers and Faculty/Trainees)

Have you met your goals?

Tell me about managing your Veterans using the VMC. (Trainees/faculty)

Performance: Results (Trainers)

What are your goals in training others to use the VMC?

Have you met your goals? Tell me about your expectations for use of the VMC as part of Veteran management.

Performance: Behavior Change

In what ways, if any, has using the VMC influenced how you interact with your Veterans? (trainees/Faculty)

What changes, if any, have you noticed in your Veterans who have been using the VMC? (trainees)

[IF NO EXPERIENCE USING VMC WITH VETERANS] Social

Under what circumstances do you expect to use the VMC? (Faculty)

At this point, would you recommend the VMC to a VA colleague or co-worker? (Faculty)

What would you tell them about it?

What, if anything, could have improved your learning/training experience? (Trainers and Faculty)

[EXPERIENCE USING VMC WITH VETERANS] Social

Under what circumstances do you expect to use the VMC again? (Faculty/trainees)

Would you recommend the VMC to a VA colleague or co-worker? (Faculty/trainees)

What would you tell them about it?

What, if anything, could have improved your experience? (Trainers and Faculty)

Other DMSM resources

Tell me about other online resources, if any, you have used to help you manage your Veterans’ health. (Trainers and Faculty).

Demographics

How long have you worked at the VA?

Appendix II. Neutral Domains and Interview Questions for Veterans

Appendix III. Domain Matrix for Clinical Faculty and Trainers.

Domain	Response to Interview Question
Experience	Tell me about your role at the VA. Tell me about how you first learned about using the VMC. Tell me about how you decided to use the VMC. Tell me about your experience using the VMC.
Training	Tell me about any user training you may have provided to Veterans (For Faculty/CDEs/trainees) OR to Faculty/CDEs (For Trainers) Tell me about the training you participated in to learn how to use the VMC. (For Faculty/CDEs/trainee)
Social Influence/Conditions	Under what circumstances do you expect to use the VMC again?
Facilitating Conditions (Ease of Use)	What components, if any, did you find helpful? Walk me through how you use the VMC.
Attitude	In what ways, if any, has using the VMC influenced how you interact with your Veterans?
Behavioral Intention (Goals)	What are your goals in using the VMC? What are your goals in training others to use the VMC?
Use Behavior (Results)	Have you met your goals? Tell me about your expectations for use of the VMC as part of Veteran management.
Benefits	What components, if any, did you find helpful? What changes, if any, have you noticed in your Veterans who have been using the VMC?
Burdens/Challenges	What components, if any, did you find unhelpful?
Suggested Improvements	What, if anything, could have improved your experience? Would you recommend the VMC to a VA colleague or co-worker?
Resources	Tell me about other online resources, if any, you have used to help you manage your Veterans’ health.
Other	Other notable points that did not fit the domains or fit multiple domains

References

Liu

Sayam

Xiaonan

, et al. Prevalence of and trends in diabetes among veterans, United States, 2005-2014. Prev Chronic Dis. 2017;14(E135):1-5. doi: https://doi.org/10.5888/pcd14.170230

Funnell

Brown

Childs

, et al. National standards of diabetes self-management education. Diabetes Care. 2010;33(1):89-96. doi: https://doi.org/10.2337/dc10-S089

Cross

Kurmas

Byrne

, et al. Diabetes self-management education support healthy teaching kitchen cooking classes delivered using a telehealth technology. Sci Diabetes Self Manag Care. 2021;47(3):199-206. doi: https://doi.org/10.1177/26350106211004887

Centers for Disease Control and Prevention. Diabetes self-management education and support (DSMES) toolkit. Published July 18, 2023. Accessed December 19, 2023. https://www.cdc.gov/diabetes/dsmes-toolkit/background/benefits.html.

Siple

Harris

Morey

, et al. Experiences of veterans with diabetes from shared medical appointments. Fed Pract. 2015;32(5):40-5.

Ingmundson

Scott

Gallimore

, et al. The VA virtual medical center: implementing a vision for a virtual health campus for our Veterans. https://www.researchgate.net/project/VA-Virtual-Medical-Center accessed on November 2, 2022. 2015.

Der-Martirosian

Wyte-Lake

Balut

, et al. Implementation of telehealth services at the US Department of Veteran Affairs during the COVID-19 pandemic: mixed methods study. JMIR Form Res. 2021;5(9):1-11. doi: https://doi.org/10.2196/29429

DiNardo

Phares

Jones

, et al. Veterans’ experiences with diabetes: a qualitative analysis. Diabetes Educ. 2020;46(6):607-16. doi: https://doi.org/10.1177/0145721720965498

Kim

Castelli

. Effects of gamification on behavioral change in education: a meta-analysis. Int J Environ Res Public Health. 2021;18(7):1-13. doi: https://doi.org/10.3390/ijerph18073550

10.

Bonde

Makransky

Wandall

, et al. Improving biotech education though gamified laboratory simulations. Nat Biotechnol. 2014;32(7):694-7. doi: https://doi.org/10.1038/nbt.2955

11.

Koivisto

Malik

. Gamification for older adults: a systematic literature review. Gerontologist. 2020;61(7):345-57. doi: https://doi.org/10.1093/geront/gnaa047

12.

Anam

Andrade

Ruiz

. Promoting lifestyle change through medical avatars. Encyclop e-Health Telemed IGI Global. 2016;1:316-30. doi: https://doi.org/10.4018/978-1-4666-9978-6.ch026

13.

Koivisto

Juho

. The rise of motivational information systems: a review of gamification research. IJIM. 2019;45:191-210.

14.

Ruiz

Andrade

Anam

, et al. Using anthropomorphic avatars resembling sedentary older individuals as models to enhance self-efficacy and adherence to physical activity: psychophysiological correlates. MMVR. 2012;19:405-11. doi:https://doi.org/10.3233/978-1-61499-022-2-405

15.

O’Shea

AMJ

Gibson

Merchant

, et al. Understanding rural-urban differences in veterans’ internet access, use and patient preferences for telemedicine. J Rural Health. 2023:1-8. doi: https://doi.org/10.1111/jrh.12805

16.

Drake

Zhang

Chaiyachati

Polsky

. The limitations of poor broadband internet access for telemedicine use in rural America: an observational study. Ann Intern Med. 2019;171(5):382-4. doi: https://doi.org/10.7326/M19-0283

17.

Dryden

Kennedy

Conti

, et al. Perceived benefits of geriatric specialty telemedicine among rural veterans and caregivers. Health Serv Res. 2022;58(Suppl 1):26-35. doi: https://doi.org/10.1111/1475-6773.14055

18.

Stetler

Legro

Wallace

, et al. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med. 2006;21 Suppl 2(Suppl 2):S1-8. doi: https://doi.org/10.1111/j.1525-1497.2006.00355.x. PMID: 16637954; PMCID: PMC2557128.

19.

Bauer

Damschroder

Hagedorn

Smith

Kilbourne

. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):32. doi: https://doi.org/10.1186/s40359-015-0089-9. PMID: 26376626; PMCID: PMC4573926.

20.

Venkatesh

Morris

Davis

, et al. User acceptance of information technology: toward a unified view. MIS Q. 2003;27(3):425-78. https://doi.org/10.2307/30036540

21.

Veterans Health Administration. VHA Operations activities that may constitute research, VHA handbook 1200.21. U.S. Department of Veterans Affairs; 2019.

22.

Elo

Kääriäinen

Kanste

, et al. Qualitative content analysis: a focus on trustworthiness. Sage Open. 2014;4(1):1-10. https://doi.org/10.1177/2158244014522633

23.

Averill

. Matrix analysis as a complementary analytic strategy in qualitative inquiry. Qual Health Res. 2002;12(6):855-66.

24.

Hamilton

. Qualitative methods in rapid turn-around health services research. VA HSR&D Cyberseminar. 2013. Available: www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780.

25.

Morse

Barrett

Mayan

Olson

Spiers

. Verification strategies for establishing reliability and validity in qualitative research. Int J Qual Methods. 2002;1(2):13-22.

26.

O’Connor

. Virtual reality and avatars in health care. Clin Nurs Res. 2019;28(5):523-8. https://doi.org/10.1177/1054773819845824

27.

Shaked

. Avatars and virtual agents–relationship interfaces for the elderly. Healthc Technol Lett. 2017;4(3):83-7. https://doi.org/10.1049/htl.2017.0009

A Formative Evaluation of Diabetes Self-Management Education and Support Within the Virtual Medical Center