Abstract
OBJECTIVES
To reduce health inequities for lesbian, gay, bisexual, transgender, queer, intersex, asexual, and all sexually and gender diverse (LGBTQIA+) people, healthcare professionals need increased access to education and training resources on LGBTQIA + health. Web-based, asynchronous, electronic learning (e-learning) resources are critical for expanding the availability of LGBTQIA + health programs. This article presents the design and utilization outcomes of a novel e-learning platform for engaging healthcare professionals in LGBTQIA + health online continuing education.
METHODS
As of December 2022, the e-learning platform consisted of 293 resources within 17 topic domains. Modalities included: learning modules, recorded webinars, publications, videos, and toolkits. We conducted a descriptive analysis of the e-learning platform's website traffic and user engagement data. Google Universal Analytics and event tracking were used to measure website traffic, user locations, and publication downloads. Learning module and webinar completions were exported from the learning management system and run as frequencies.
RESULTS
Between January 1, 2020, and December 31, 2022, over 650,000 people from all U.S. states, 182 countries, and 31 territories visited the website. Platform users downloaded publications 66,225 times, and completed 29,351 learning modules and 24,654 webinars.
CONCLUSION
The broad reach and high user engagement of the e-learning platform indicate acceptability of web-based, asynchronous online continuing education in LGBTQIA + health, and suggest that this platform is filling a need in health professional education. Remote, online learning opportunities may be especially important in jurisdictions with bans on medical care for transgender and gender diverse youth. Future growth of the platform, paired with in-person and other online learning opportunities, has the potential to reduce gaps in LGBTQIA + health training, and mitigate LGBTQIA + health inequities.
Introduction
Lesbian, gay, bisexual, transgender, queer, intersex, asexual, and all sexually and gender diverse (LGBTQIA+) people face significant health disparities in the U.S. and across the globe.1-7 Gaps in LGBTQIA + health education and cultural sensitivity training for healthcare professionals may contribute to these disparities.8-15 According to a 2022 nationally representative survey of LGBTQIA + adults in the U.S., about 21% of respondents delayed or avoided medical care in the past year due to mistreatment by providers, and about 30% of transgender and gender diverse respondents had to teach their healthcare provider about transgender and gender diverse people to receive appropriate care. 16 Similarly, two in three intersex people had to educate their providers about intersex people to ensure appropriate care. 16 LGBTQIA + Black, Hispanic, Asian, and multiracial respondents reported a higher burden of negative experiences with healthcare providers. 16
To reduce health inequities among LGBTQIA + people, clinicians of all disciplines, as well as healthcare administrators and support staff, need increased access to high-quality, evidence-based training, resources, and materials that are responsive to the health priorities of LGBTQIA + communities.17–20 Currently, however, most medical, allied, and other health professional schools do not sufficiently prepare students for LGBTQIA + healthcare,9-11,14,15,21 making it necessary for healthcare professionals to seek out training and continuing education (CE) to fill gaps in knowledge and skills. 19
Web-based electronic learning (e-learning) platforms can help expand global access to LGBTQIA + health education by reducing the costs, travel time, and other resources associated with in-person programs. Studies suggest that online educational programs produce similar learning outcomes to in-person programs.22-25 Furthermore, e-learning that is asynchronous (ie, not occurring in real time) offers certain benefits over synchronous online programs, including more flexibility for the learner to decide what, when, and how to learn, and increased capacity for the learner to re-access and review materials at any point in time. 24 Finally, it is increasingly important to provide remote LGBTQIA + health learning opportunities as more U.S. states pass legislation to restrict gender-affirming medical care for transgender and gender diverse youth, 26 with laws that have been interpreted to include a ban on teaching as well as practice.
In this article, we describe the design, development, content, reach, and user engagement of a novel, web-based, e-learning platform consisting of comprehensive resources on LGBTQIA + health and intended for educating the full range of clinical and non-clinical staff within healthcare organizations in the U.S. and internationally. The purpose was to assess the acceptability of the platform and to inform the development and enhancements of other remote educational opportunities on LGBTQIA + health.
Materials and Methods
Purpose and audience of the e-learning platform
Launched in 2012, the National LGBTQIA + Health Education Center e-learning platform was created to increase the capacity of healthcare organizations to provide evidence-based, culturally responsive care and services for the LGBTQIA + people in their communities. The platform is produced by the National LGBTQIA + Health Education Center, and is housed within the organization's main website, in a section called “Learning Resources” (Figure 1
Screenshot of the National LGBTQIA + Health Education Center's e-learning platform, where the resources are filtered by modality type (“publication”) and topic domain (“Introduction to LGBTQIA + Health”).
Many of the platform resources are developed under a cooperative agreement with the Health Resources and Services Administration (HRSA) Bureau of Primary Health Care to deliver comprehensive LGBTQIA + health training and technical assistance to nearly 1400 HRSA-funded health centers located in all U.S. states, commonwealths, and territories. 27 Health centers provide primary medical care and behavioral healthcare, as well as supportive and wrap-around services, including dental care, pharmacy, case management, HIV specialty care, and optometry.
While the platform resources are designed to be relevant for health centers, they also are applicable for healthcare organizations of all types, including academic medical centers, community hospitals, student health clinics, specialty care practices, and public health departments. As such, the platform resources aim to educate the full range of clinical and non-clinical healthcare staff, including but not limited to providers of all disciplines, medical support staff, executives, paraprofessionals, registration and front desk staff, and financial administrators.
Modalities
To meet the needs of diverse learning styles and preferences, the platform currently offers 5 types of asynchronous modalities: 1) learning modules: interactive, self-paced programs with embedded knowledge tests and video clips; 2) webinars: recordings of synchronous presentations, with slides, audio and/or video, and question-and-answer sessions; 3) publications: clinical guides, toolkits, patient brochures, policy briefs, and fact sheets in PDF format; 4) videos: scripted dramatizations of best practices for clinical and non-clinical staff; interviews with clinical experts; interviews with LGBTQIA + people discussing their experiences with healthcare; and 5) toolkits: curated collections of resources on a specific topic. Learning modules take about 20 to 30 min to complete. Most webinars run for 60 min; videos range from 30 s to 15 min; and publications range from 2 to 88 pages.
Curriculum development process
To develop resources, the National LGBTQIA + Health Education Center staff collaborate with researchers and clinicians who have expertise in relevant aspects of LGBTQIA + healthcare. The teaching faculty includes skilled clinicians at Fenway Health, a health center recognized as a model for LGBTQIA + healthcare.28-30 To identify additional faculty, we reach out to our professional networks for recommendations and aim to engage racially and ethnically diverse LGBTQIA + faculty. Staff work closely with faculty members to develop learning objectives that follow the adult learning principles of practicality, relevance to work, and problem- and goal-orientation. Additionally, we encourage faculty to create resources that present a balanced mix of current evidence, clinical guidelines, best and promising clinical practices, case studies, and innovative models of care. For many programs, we also feature the voices of LGBTQIA + patients and community leaders who share their perspectives and lived experiences. Depending on the subject matter, we either design a resource for all healthcare staff roles, or we tailor to specific roles (eg, primary care clinicians, behavioral health clinicians, support staff, administrative leaders). The program director and an additional staff medical clinician review all resources for clinical accuracy and relevance.
Topic domains
As of December 31, 2022, the e-learning platform consisted of 17 topic domains covering a broad range of LGBTQIA + medical and behavioral health topics, as well as topics focused on organizational change
Topic domains and subtopics for the National LGBTQIA + Health Education Center's e-learning platform.
Abbreviations: LGBTQIA+: lesbian, gay, bisexual, transgender, queer, intersex, asexual, and all sexually and gender diverse people; STI: sexually transmitted infections; PrEP: pre-exposure prophylaxis; PEP: post-exposure prophylaxis.
Platform functionality
Users of the e-learning platform can search for resources by typing keywords into a search bar, or by filtering content by modality type, topic domain, or both. To access learning modules and recorded webinars, platform users must create a free Learning Profile account by registering with a username and password. Once registered, account holders can log into their Learning Profile to monitor their progress in completing learning modules and webinars.
Continuing education (CE) credits
Learning Profile account holders can receive prescribed CE credits from the American Academy of Family Physicians (AAFP) after completing learning modules and webinars. AAFP credits are accepted by the American Medical Association and many nursing professional societies. Account holders may also receive credits toward the training requirements for the Human Rights Campaign Foundation's Healthcare Equality Index (HEI). The HEI is a biannual survey that evaluates more than 2220 US healthcare organizations on their level of equity and inclusion for LGBTQIA + patients, visitors, and employees. To become eligible for HEI top performance status, organizations must have their staff complete a certain number of hours of LGBTQIA + health training.
Platform dissemination
The National LGBTQIA + Health Education Center website and e-learning platform can be found by potential users from a variety of sources, including: Internet searches for LGBTQIA + health resources; direct links on partner sites such as the HEI; and dissemination newsletters and other dissemination channels overseen by HRSA. We also publicize our resources at meetings, conferences, and during our synchronous in-person and online programs. To build awareness of upcoming and recently released resources and programs, staff send out an e-newsletter to subscribers. Any user who creates a Learning Profile automatically becomes subscribed to the newsletter, with the option to unsubscribe.
Ethical approval
The data for this study were originally collected as program evaluation data rather than intended for generalizable human research or publication. The Fenway Health Institutional Review Board (IRB) granted the authors a memorandum stating that if the project had been prospectively reviewed, it would have been deemed not human subjects research and therefore excluded from participant consent and IRB review per 45 CFR Part 46.
Data collection and analysis
We conducted a descriptive analysis of the e-learning platform's website traffic and user engagement data. To measure website traffic, locations of website users, and publication downloads, we used Google Universal Analytics and event tracking. To assess video views, we gathered view-count data from Vimeo, which is the video platform we use to distribute our videos. Data related to Learning Profile accounts, including completions of learning modules and webinars, CE credits earned, and HEI affiliation, were exported from the learning management system (LearnDash), imported into Microsoft Excel for Windows, and run as frequencies. We did not include utilization and engagement outcomes for toolkits because the first such toolkit was added at the end of our measurement period.
We restricted our measurement period to the most recent two full years of website usage, January 1, 2020, to December 31, 2022, because a 2019 redesign of the website infrastructure precluded accurately combining or comparing with data before 2020. Due to constraints on Vimeo, however, video view counts, could only be measured from 2017 (the publication date of the first videos) through May 1, 2023 (the date we measured the video views).
Results
Website scope and reach
Between January 1, 2020, and December 31, 2022, 659,016 unique users visited the National LGBTQIA + Health Education Center website (Table 2). During those visits, the website pages had over 3.8 million views during 965,450 user sessions (ie, the discrete episodes during which a user interacts with a website). Nearly half of all users had 2 or more sessions.
Website traffic for the National LGBTQIA + Health Education Center's website, January 1, 2020, to December 31, 2022.
Users accessed the website from 182 countries and 31 territories. The U.S. accounted for about 91% of all users. The next 9 countries with the highest user frequencies (each with over 1000 users), in descending order, were: Canada, United Kingdom, Australia, India, Germany, China, Philippines, Mexico, and Spain. In the U.S., all 50 states were represented. The10 states with the highest proportion of the 659,016 total unique website users were: California (12.9%), New York (7.6%), Virginia (6.2%), Massachusetts (6.1%), Pennsylvania (5.4%), Texas (4.4%), Ohio (4.1%), Florida (3.7%), Illinois (3.2%), and Washington (3.2%).
The website acquired over half of its users through a direct link (57.2%), meaning users either typed the link directly into a browser, connected to the site from a bookmark, or clicked on a link from a non-web file or messaging service. Most of the remaining users linked to the site through either organic (unpaid) searches from a search engine (24.3%), or through a referral website (19.9%). The most common referrals originated from non-governmental partnering organization websites, such as the Human Rights Campaign Foundation and the American Academy of Family Physicians, or from government websites within the U.S. Department of Health and Human Services, such as HRSA, the Substance Abuse and Mental Health Services Administration, and the Centers for Disease Control and Prevention.
Learning profiles and CE credits
As of December 31, 2022, there were 229,885 Learning Profile accounts registered with the e-learning platform. Of these account holders, 100,345 (43.7%) had earned at least 1 CE credit. In total, account holders earned 233,828 CE credits. Additionally, 67,130 (28.7%) account holders had an HEI profile associated with their account.
Resource utilization and engagement
By December 31, 2022, the e-learning platform consisted of 293 resources (21 learning modules, 182 webinars, 68 publications, 21 videos, and 1 online toolkit). About 70% of the resources on the platform were created between 2020 and 2022; the remaining 30% were developed prior to 2020.
Learning modules and webinars
During the measurement period of January 1, 2020, to December 31, 2022, users completed 29,351 learning modules and 24,654 webinars (Table 3). Among the 5 most frequently completed learning modules (representing 55% of all completed modules), 4 focused on introductory and foundational topics, such as LGTBQIA + health concepts, terminology, and communication (Table 4). The fifth most completed module focused on caring for transgender and gender diverse people.
Resource engagement by modality, National LGBTQIA + Health Education Center's e-learning platform, January 1, 2020, to December 31, 2022.
Engagements refer to the number of completions for learning modules and webinars; number of downloads for publications; and number of views for videos.
Video views were counted from January 1, 2017, to May 1, 2023.
Resources with the highest usage, by modality, on the National LGBTQIA + Health Education Center's e-learning platform, January 1, 2020, to December 31, 2022.
Engagements refer to the number of completions for learning modules and webinars; number of downloads for publications; and number of views for videos.
Video views were counted from January 1, 2017, to May 1, 2023.
Among the 5 most frequently completed webinars, 4 focused on the introductory topics of addressing social determinants of health, achieving health equity, and addressing unconscious and implicit bias. The fifth most completed webinar focused on COVID-19 disparities among LGBTQIA + people of color. These 5 most accessed webinars represented only 14% of all webinars completed, meaning that user engagement was more evenly distributed across webinars than across learning modules.
Publications and videos
Publications were downloaded 66,225 times during the 2-year measurement period (Table 3). The most frequently accessed publication was the Glossary of LGBTQIA + Terms for Healthcare Providers, including a Spanish translation from the original English version (Table 4). The next 3 most downloaded publications were guides focused on: collecting patient data on sexual orientation and gender identity; providing affirmative frontline services for transgender and gender diverse people; and creating inclusive healthcare environments for LGBTQIA + people. These 5 publications represented 39% of all publication downloads.
Between January 1, 2017, and May 1, 2023, videos were viewed 492,613 times. The 5 most accessed videos, which represented 64% of all video views, were a variety of demonstration videos of best practices for collecting sexual orientation and gender identity data with patients.
Discussion
This article described the development, design, and recent three-year utilization outcomes of a novel, e-learning platform designed to increase global access to LGBTQIA + health CE programs and resources. While other organizations have developed online programs on LGBTQIA + health topics, 10 the National LGBTQIA + Health Education Center website, to our knowledge, is the only e-learning platform dedicated to all aspects of LGBTQIA + health. From our analysis, we observed very high usage of asynchronous resources on the e-learning platform, demonstrated by users completing nearly 30,000 learning modules and 25,000 webinars, viewing videos over 492,000 times, and downloading publications over 66,000 times. Moreover, the website was visited by people from all 50 U.S. states and from 182 countries. In sum, the broad reach and engagement of our platform suggest that this delivery model for LGBTQIA + health CE is acceptable to healthcare professionals nationally and globally. This finding aligns with evidence from survey research indicating that clinicians have a strong interest in learning more about LGBTQIA + health, and in using online educational formats for this purpose.21,34-36
Overall, we found that users accessed resources focused on introductory concepts more frequently than other topics. This usage pattern suggests a continued need for CE programs to develop and disseminate resources that build foundational knowledge and skills in LGBTQIA + health that apply to all healthcare staff, and not just specialized areas of care. Our analysis also indicated that learning modules had a greater number and proportion of completions than webinars. This finding suggests that shorter, interactive products are of greater interest and utility to healthcare students and professionals than longer, minimally interactive products like webinars. Similarly, the very high number of video views may indicate that learners appreciate short audiovisual demonstrations of best clinical practices. At the same time, the large number of publication downloads implies that printed text continues to appeal to healthcare professionals, despite the availability of interactive and audiovisual modalities. Finally, we observed that Florida, Ohio, and Texas were among the top 10 states that accessed our resources, suggesting that states with bans and restrictions on gender-affirming care have a demonstrated need for online LGBTQIA + health CE.
Overall, our findings reflect those of the medical education literature that has demonstrated a desire among adult learners for multimedia formats that are brief, case-based, visual, interactive, and/or print-based.37-39 Accordingly, we recommend that CE providers consider the need for and importance of developing online resources that teach foundational LGBTQIA + health concepts in digestible and diverse formats.
Limitations
This study has several limitations. First, because we did not systematically collect data on the professional roles or organizational types of users, we cannot be certain that our resources are reaching the intended audience of clinical and non-clinical healthcare staff. Nonetheless, we do know that over 67,000 users had an HEI profile, and that all people registered with HEI work for a US healthcare organization. In addition, over 100,000 users applied for and earned at least 1 CE credit from the AAFP: these credits are primarily applicable for physicians and nurses.
A second limitation relates to interpretation of user preferences based on usage patterns and frequencies. For example, our finding that publications and videos were more frequently accessed than learning modules and webinars may be partially explained by the differences in how we measured engagement across modalities. For learning modules and webinars, we were able to count completions of an entire resource; for publications and videos, we could only count clicks on a resource. The number of clicks on a resource is likely to be higher than the number of completions, because some users inevitably stop or click away from a resource before completion. Another reason that videos and publications may have had higher usage than learning modules and webinars is that videos and publications did not require a Learning Profile account for access, which could serve as a barrier for some users. Moreover, because resources were added at different time points across the three-year measurement period, older resources had more time to accumulate views, clicks, and completions. Because we did not adjust for follow-up time, we could not directly compare modality usage.
This study was also limited by not collecting data on access or usability issues encountered by e-platform users. Finally, this article did not report an evaluation of user satisfaction, learning outcomes, or practice change. Thus, we do not yet know if engagement with the resources is associated with improvements in knowledge, attitudes, or skills. These data will be collected, analyzed, and reported in future studies.
Implications for resource development and research
In the coming years, our staff and faculty will continue to develop and revise resources based on emerging evidence in LGBTQIA + health as well as changes in clinical guidelines and healthcare policies. We will also continue updating resources to reflect evolving LGBTQIA + terminology; for example, we launched a revised glossary of LGBTQIA + health terms in February 2024.
To further enhance engagement and learning, we will seek to incorporate new educational modality types, such as animations, gamification, and simulations using artificial intelligence. Additionally, to widen our reach outside the U.S., we will look to partner with governmental and LGBTQIA + health leaders based in other countries. Broader global outreach will likely necessitate not only linguistically appropriate and culturally congruent translation of concepts and terminology, but also responsiveness to regional variability in attitudes and clinical guidelines.
Future studies should, at minimum, assess changes in knowledge and attitudes based on pre- and post-completion surveys, and should collect and analyze data on the professional roles and organizational types of users. Finally, collecting more granular geographic data would help further illuminate the extent and dispersion of our reach into areas of the U.S. that are most in need, including those with limited educational options and bans and restrictions on gender-affirming care for adolescents.
Conclusion
LGBTQIA + healthcare educational resources are needed to increase health equity for this vulnerable population. Asynchronous e-learning platforms offer an opportunity to help fill gaps in LGBTQIA + healthcare knowledge. In this article, we described high usage and engagement of an e-learning platform that provides open access resources on culturally affirming healthcare for LGBTQIA + communities, suggesting acceptability and feasibility. More research is needed, however, to assess whether the e-learning material is reaching the full extent of its intended audience, and if engaging with the content is associated with improvements in knowledge, attitudes, or skills in caring for LGBTQIA + patients. Finally, while e-learning may be an important addition to LGBTQIA + health education, it alone cannot address gaps in knowledge and skills among healthcare professionals. Worldwide efforts are still necessary to achieve this goal, such as programs that expand mentorship opportunities and integrate LGBTQIA + health topics into existing health professional school curricula.
Footnotes
Acknowledgments
None
Contributorship Statement
Sanjana Srinivasan: conceptualization, writing - original draft, writing - review & editing; Hilary Goldhammer: visualization, writing - original draft, writing - review & editing; Brittany M. Charlton: conceptualization, methodology, formal analysis, writing - review & editing; Tess McKenney: conceptualization, data curation, software, formal analysis, writing - review & editing; Alex S. Keuroghlian: conceptualization, methodology, writing - review & editing, supervision, funding acquisition
Declaration of conflicting interests
A. S. Keuroghlian declares royalties as editor of a McGraw Hill textbook on transgender and gender diverse health care and an American Psychiatric Association Publishing textbook on gender-affirming psychiatric care. The other authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
This work was supported in part by Grant U30CS22742 from the Health Resources and Services Administration Bureau of Primary Health Care to Dr A. S. Keuroghlian, the Principal Investigator. The funder was not involved in the writing of the article or in the decision to submit the article for publication.
Health Resources and Services Administration, (grant number U30CS22742 ).
Data Sharing
The data underlying this article will be shared on reasonable request to the corresponding author.
Ethical Approval
The Fenway Community Health Institutional Review Board (IRB) granted the authors a memorandum stating that if the project had been prospectively reviewed, it would have been deemed not human subjects research and therefore excluded from participant consent and IRB review per 45 CFR Part 46.
