Abstract
Background
Reducing access to lethal means is one of the few empirically supported approaches for lowering suicide rates, and safe firearms storage practices have been associated with reduced risk of death by suicide. Although there is substantial opportunity for primary care to assist in addressing lethal means with veterans, approaches to intervention and educating staff are not well documented. We sought to 1) describe development of an education program for primary care teams to help them discuss firearms storage safety (FSS) with veterans during primary care visits; and 2) conduct a preliminary evaluation of the pilot education program.
Methods
We used an iterative process involving veterans and primary care staff stakeholders to develop program content, format, and supplemental materials. A grounded theory approach was used to analyze data from focus groups and individual interviews. Following piloting of the program with 71 staff members in two primary care clinics, we analyzed pre- and post-training participant surveys of program satisfaction and attitudes comfort related to firearms safety discussions.
Results
During the development phase, 68 veterans and 107 staff members participated in four veteran focus groups and four primary care focus groups, respectively, and/or individual interviews. The program that was developed, “‘Just in Case’: Discussing means safety with veterans at elevated risk for suicide,” addresses knowledge and skills learning objectives, and includes video demonstrations and skills practice. Survey data obtained just prior to the pilot training sessions showed low self-reported rates of discussing firearms safety with veterans who may be at elevated risk for suicide. Immediate post-training data showed generally high satisfaction with the program and significant improvements in participant self-reported ratings of the importance of, and comfort with FSS.
Conclusions
This interactive knowledge and skill-based means safety curriculum shows promise as a means for educating primary care staff to deliver messaging about firearms safety to veterans. Additional research is needed to refine and evaluate impacts of this or similar training programs on clinician and veteran behaviors over time.
Background
Veterans are 1.5 times more likely to die by suicide compared to non-veterans. 1 Veterans are also significantly more likely to use firearms as a means of suicide compared to the general population. 2 Reducing access to lethal means is one of the few empirically supported approaches for lowering suicide rates,3,4-7 and safe firearms storage practices are associated with reduced risk of death by suicide.5-7
Means safety counseling approaches have been developed in an effort to reduce deaths by firearms and other suicide methods.8,9 Means safety counseling rests on the premise that moments of increased suicide risk are often fleeting, and that 1) preventive steps can be taken to reduce the likelihood of making an attempt, and 2) reducing access to the most lethal methods of suicide (eg, firearms) increases likelihood that individuals who do attempt suicide will survive.10,11 Means safety counseling is considered a clinical best practice for individuals with identified suicide risk, 12 and increasingly supported by multiple national organizations.13-15 Promising models exist for means safety counseling, such as those incorporated into Counseling on Access to Lethal Means (CALM).8,16 Typically, means safety counseling consists of a conversation between a counselor and an at-risk individual, often in clinical settings, whereby the at-risk individual is counseled to reduce their access to firearms (often through removal) as well as other lethal means. Bryan 17 recently found that a training program for VHA-based clinical staff led to higher rates of clinician-reported counseling of veterans about firearms safety. Although the study participants included a small proportion of physicians and nurses, the majority of the sample was comprised of social workers and psychologists.
While means safety counseling can theoretically be delivered in any clinical setting, discussions about firearms occur infrequently in non-mental health settings, including primary care.18-20 Primary care clinicians are often uncomfortable discussing firearms with patients due to perceived barriers, such as lack of knowledge or personal experience with firearms, or concerns about negative impacts to the therapeutic alliance. 21 Traditional means safety counseling can be time intensive, and competing demands during brief appointments also likely limit the ability and willingness of primary care clinicians to discuss firearms.8,9
Yet, primary care is a critically important setting for identifying and engaging veterans at risk for suicide. Half of individuals, including veterans, who die by suicide are seen in primary care settings in the month prior to death.22,23 Most individuals who receive mental health treatment receive it only in the primary care setting. 24 Thus, many patients who might benefit from discussions about firearms storage safety (FSS) in particular will be missed if interventions are not developed specifically for primary care. FSS discussions specifically include 1) advising veterans to keep firearms locked and unloaded when not in use; and 2) identifying additional ways to reduce access to firearms (eg, removal of the firearm) in situations when suicide risk may increase, such as during a crisis. Over the past decade, many healthcare systems, including the Veterans Health Administration (VHA), have implemented standardized approaches to screening for mental health conditions, including depression and post-traumatic stress disorder (PTSD), as well as suicidal ideation. The screening process provides an important opportunity to implement FSS discussions into primary care, allowing for intervention with individuals who may be at higher risk for suicide, but before the individual develops a crisis.
Our research has demonstrated that both primary care teams and veterans support having FSS discussions during primary care visits, with certain caveats.25-27 The purpose of this manuscript is to 1) describe stakeholder-informed development of an education program to help primary care teams discuss FSS during primary care visits; and 2) present results of a preliminary evaluation of the pilot program.
Methods
Setting and Samples
The goal of the overall project was to develop an education program to facilitate delivery of FSS by VHA primary care teams. While we envision FSS as being deliverable to any Veteran, we specifically sought to help primary care teams focus on delivering FSS to individuals with positive depression or PTSD screens. The first phase of the project, focused on developing the program itself, was designated as a quality improvement activity by VA Portland Health Care System (VAPORHCS) on 6/15/2018 (no reference number). The second phase, which focused on evaluating the pilot program, was approved by the VAPORHCS Institutional Review Board (IRB) on 11/30/2018 (reference #4347); the IRB approved a waiver of written informed consent; all participants were provided an information sheet and gave verbal consent to participate. During the first phase, we worked with veteran and primary care teams to explore attitudes toward FSS, and to gather input on development of the training program. Three local veteran organizations were identified via word-of-mouth, and organization leadership was approached to set up focus group meetings. All members of these organizations were invited to respective group meetings to create a convenience sample of veterans. A fourth group of veteran consultants, identified via communications with leaders of the above three groups, was created to provide input into this specific project. We also conducted focus groups with primary care teams at three VA community-based outpatient clinics (CBOCs) and one hospital-based primary care clinic. Due to firearm ownership being a sensitive issue, we intentionally did not ask individuals participating in focus groups about their own firearms ownership. However, prior research indicates that approximately half of veterans own firearms, 28 and we presumed or observed that the majority of participants either owned firearms or had positive attitudes toward firearms ownership, as indicated by self-disclosure during sessions and the general absence of comments indicating negative attitudes toward gun ownership. Five large primary care clinics associated with VAPORHCS were also invited to participate in sessions; we met with the first four clinics that responded. Finally, six veterans and five primary care providers completed individual, semi-structured interviews following the focus group meetings. The total number of clinicians who participated in group and/or individual sessions was 107 and the total number of veterans who participated in group and/or individual sessions was 68.
Development of the Training Program
The methods and findings from the focus group meetings and individual interviews with veterans and staff have previously been published.25-27 Table 1 shows a summary of findings regarding attitudes and advice given by these stakeholders which specifically informed development of the training program.
Summary of themes from development Phase. 25
From this prior work, we learned that primary care staff would like to have examples of how to effectively speak about FSS with veterans. To this end, we extracted sample messages regarding FSS by reviewing existing messaging on FSS publicly available on-line. A set of messages were presented to veterans and clinicians during individual interviews to refine a set of sample scripts for staff to be able to use during visits. To identify potential messages, extensive searches were conducted in several databases, including PubMed, PsycINFO, ISI Web of Knowledge, and The Cochrane Library for the time period from the early 2000's to 2017. In addition, we evaluated content and artifacts (eg, brochures) that had been designed to limit access to lethal means available from organizations, campaigns, and other initiatives on lethal means safety.
Specifically, we evaluated materials developed for individual firearm owners and clinicians who may interact with patients who own firearms. We identified an initial set of 60 messages, scripts, or text from awareness campaigns for our dataset. We then grouped the messages in an effort to inform our ultimate aim, that is, development of scripts for use by primary care staff to facilitate FSS conversations. After several iterations of review (by KC and EK), messages were classified as being: 1) ice-breakers (eg, opening statements), 2) context (eg, why conversation is happening), 3) facts about firearms/suicide/risk, 4) safety tips, and 5) frequently asked questions (eg, will my mental health diagnosis prevent me from owning a firearm?).
Table 2 shows the messages we abstracted from our search and presented to veterans and clinicians, and the rating scales associated with each domain. Interviews lasted for approximately one-hour and were recorded and professionally transcribed. Within each domain, we first asked participants to rate each message using Likert-type scales, and asked them to explain why they thought a particular message was acceptable or not acceptable. Across veteran and clinician interviews, we calculated mean scores of ratings (because the sample size was small, these ratings served as non-parametric indicators of acceptability). We then used a descriptive qualitative analysis approach29-31 to identify a set of highly acceptable messages based on the expertise and experiences of the interviewee participants. After incorporating actionable qualitative suggestions (eg, wording changes or content changes), the study team combined highest rated messages in each domain to create a set of scripts organized by level of risk (ie, low risk for suicide vs. higher level of risk). For example, one script developed for lower risk patients was as follows, “Because rates of suicide by firearms are high among veterans, and depression and PTSD increase risk for suicide, I am talking to all of my patients who may have depression or PTSD about the safe storage of firearms in the home. Would it be okay if we talked about that for a few minutes?”
Message ranking exercise (based on means safety messages from public sources.
In addition to developing sample scripts for staff to use to guide firearms safety discussions, as was done in the initial analysis of staff and veteran attitudes,25-27 we used a grounded theory approach32-34 to identify candidate learning objectives for the training program and to guide program format.
We learned from focus group meetings that primary care teams and leadership preferred that training sessions be brief (preferably a maximum of 1.5 hours), as primary care staff frequently participate in trainings on competing high priority topics. Primary care teams had also expressed interest in having written materials available to help educate veterans and to normalize conversations about safe firearm storage. Suggestions included having a pocket-card which contained scripts they could examine prior to FSS discussions and patient-facing posters or brochures, which might be placed in waiting rooms or exam rooms. We therefore collaborated with VA's Office of Mental Health and Suicide Prevention (funder of this project) and obtained input from the National Shooting Sports Foundation (https://www.nssf.org/) to create a pocket card for staff 35 and a brochure for primary care patients. 36
Piloting of Program
Based on information from the focus groups and individual interviews, our team developed the interactive education session, “‘Just in Case’: Discussing Means Safety with Veterans at Elevated Risk for Suicide.” The program was designed to last 1.5 hours, include approximately 20–25 participants, and to be led by two local facilitators: a mental health clinician (in this case, a psychiatrist), and a primary care provider (in this case, an internist). We also recommend including a veteran firearm owner (in our case, a VHA employee) to offer perspective regarding firearms ownership and answer questions that might regard on how firearms work. Program knowledge and skills objectives and content are presented in Table 3. The PowerPoint® presentation (available on request from the authors) includes several video segments that serve as examples and are designed to prompt further discussion. The program is broken down into approximately 50 minutes of didactive and 20 minutes of skills practice. We developed the GROW Model to provide a framework to guide clinical team members in preparing for and structuring FSS discussions (Get ready; provide a Reason for the discussion; Offer brief advice; and communicate We’re here to help).
“‘Just in Case’: Discussing means safety with veterans at elevated risk for suicide” program summary.
Program materials provided in a binder for participants at the beginning of sessions included slide outlines, clinician pocket card, patient brochure, copies of several key references,7,37 suicide risk stratification table developed by the VA Rocky Mountain Mental Illness Research, Education and Clinical Center (MIRECC) for Suicide Prevention, 38 a brief compendium of how firearms work and firearms terminology, and a set of sample scripts to use in the skills portion of the education session and to guide discussions with patients (Table 4).
Final sample scripts for session participants to use for skills practice and for FSS discussions with veterans.
Program Evaluation
Methods
Between April and July 2019, we piloted the training program in two of the Medical Center's larger primary care clinics. Participants were asked to complete questionnaires just before, then immediately at the end of each session. Questionnaires included items to assess experiences and attitudes related to firearms and to discussing firearms during primary care visits. The post-session questionnaire also inquired about satisfaction with the educational program. Item responses were captured using 4-point Likert scales (4-point scale: excellent, good, fair, poor). T-tests and Chi square were used to compare pre-session to post-session responses.
Results
Together, 71 staff participated in the two pilot sessions held at two VHA clinics including all PACT members: licensed independent practitioners (physicians and nurse practitioners), registered nurses, licensed practical nurses, and medical staff assistants (MSAs). Forty-five percent of participants identified themselves as nurses, 18% as physicians, 3% as nurse practitioners, and 32% as MSAs/other support staff. Sixty-eight percent of participants were women. Eighty-six percent of staff members reported working in direct patient care 50% of the time or more. In the baseline questionnaires, 58% of participants reported ever having discussed firearms safety with a veteran, and 44% reported having offered a firearm cable lock to a veteran (note that cable locks are provided free to veterans as part of VHA's suicide prevention strategy). Across the two clinics, immediately post-session, 88% rated the quality of the program; 94% rated the usefulness of the information; 86% rated the pace of the session; 92% rated the structure of the session; and 94% rated the usefulness of written materials as “good” or “excellent”. Also, 72% responded “very likely” or “almost certain” to recommend the program to others. Free text feedback was also gathered, including what people liked most about the training, and what could be improved. The sample scripts emerged as the most liked element of the training, and suggestions for improvement included allowing more time for the training.
Participants also reported significant improvements compared to baseline in 1) the importance they placed on speaking with patients about firearms safety (3.27 vs. 2.91; p = .01); 2) ratings of perceived ability to effectively speak with patients about firearms safety (3.26 vs. 2.72; p < .0001); and 3) level of comfort speaking with patients about firearms safety (3.07 vs. 2.73; p = .034). Participants’ ratings of the extent they know enough about firearms to discuss firearm safety with patients did not change significantly post-course to pre-course (2.91 vs. 2.62; p = .085) for the 51 (72%) participants who completed pre- and post-course surveys.
Comments on Training Program Feasibility and Potential Implementation
Despite five larger clinics being invited to participate in the pilot training program, only two initially volunteered. A main obstacle noted by several clinic leaders was that there were many training needs for which they did not have time. Clinic managers also said that 1.5 hours would likely be the most time they could devote to the program if it were delivered in one session though the content and format were developed to be delivered in approximately two hours. In addition, despite our plan to present the program to smaller groups (ideally fewer than 20) to facilitate discussion and skills practice, the clinics preferred to have large groups participate in the sessions.
Discussion
This is the first project we have identified to use an iterative, stakeholder-informed process to develop and evaluate a program to teach staff to deliver firearms safety messaging in primary care settings. The education session we developed addresses knowledge and skills learning objectives and includes video demonstrations and skills practice; supplemental materials included a pocket card, which provides sample scripts to guide conversations, and a brochure to be used in primary care clinics. We piloted the program among 71 staff members, 58% of whom reported previously discussing firearms safety with veterans. An immediate post-training evaluation showed generally high satisfaction with the program, and that participants reported significant improvements in ratings of the importance of and comfort with FSS.
We identified a number of challenges to implementing the program, including competing demands on clinical leadership to find time for sessions, and their desires to have all staff participate at once. We had initially planned to deliver content in smaller group sessions to provide more time and support for skills development. We also confirmed that 1.5 hours was not enough time to cover all the material. Although one possibility would be to try to arrange for longer sessions, there may be value in separating the program into several sessions to allow for more question and answer time and post-session planning by care teams. We know from prior work that impacts of clinician training interventions can decay over time. 39 Delivering the program over several sessions or providing follow-up interactions might also help reinforce the material and any learned behaviors.
Research on messaging models such as the SBIRT (Screening, Brief Intervention, Referral and Treatment) for risky drinking,40,41 the “5 As” for smoking cessation, 42 and motivational interviewing43,44 suggest that when clinicians are trained to deliver brief sets of health messages, it can have positive impacts on patient behaviors. We found that, after delivery of this stakeholder-informed educational content, participants reported high satisfaction and an increase in comfort related to FSS. We note that in November 2020, VHA implemented a national mandatory video training session for clinical providers including primary care clinicians. While we see this as an important step and useful resource, based on the knowledge and skills gaps we learned about during our preliminary work with clinical teams, 25 we sought to create a more intensive training experience that included knowledge and skills components. Future research is needed to assess whether the participant-reported gains are sustained over time, and importantly, whether they 1) translate to changes in practice at the provider or clinic level, and 2) ultimately impact patient behaviors and suicide outcomes.
Next steps might include delivering a larger series of training sessions and evaluating clinician behaviors (both self-reported and observed) over time. Using standardized patients to gauge staff performance or identify a cohort of patients who are recipients of FSS messaging to evaluate their responses to the messaging could be helpful. It would also be important to ensure that training components align with national consensus guidelines for firearm injury education for medical professionals 45 as well as ongoing and increasing efforts within VHA to promote lethal means safety.
We utilized a stakeholder engagement to develop this course. We believe this approach is important given the potential sensitives of discussing firearm safety, particularly among veterans, and that including firearm owners with specialized knowledge in the process may help inform firearm storage safety recommendations. This approach could also potentially improve the credibility of course content and messengers, and allay concerns about the privacy implications of disclosing firearm access in healthcare settings. In support of this, a prior study among a national sample of firearm owners 46 found that, among 14 different potential messengers (eg, friends, law enforcement, hunting or outdoors magazines), military veterans were ranked as the third best messengers to teach firearm owners about safe firearm storage for the purposes of suicide prevention. Notably, in this study, “physicians or medical professionals” were ranked near the bottom. Importantly, this study assessed perceptions regarding which groups would be ‘good’ at counseling, and not whether clinician-delivered counseling was acceptable or appropriate. Multiple prior studies have clearly demonstrated that firearm owners, including veteran firearm owners, agree that clinicians have a role in firearm safety counseling among at-risk individuals.21,26,27,47
In addition to lack of information on clinician and veteran behaviors, there are other limitations to this work. Most of the data used to develop the course were derived from individuals living in urban or suburban areas in the Pacific Northwest. Veterans and clinicians from other parts of the country or rural areas may have differing perspectives; some of the scripts we developed may not be as acceptable or effective in other patient populations. While the course was designed for care teams working with Veterans, we expect that many of the messages and approaches would be applicable or adaptable for use with non-Veteran patient populations. It is possible that some of the clinicians and nurses who did not complete post-session evaluations were less satisfied with the course.
Conclusions
This education program shows promise as an approach toward helping move suicide prevention upstream into settings that often encounter patients at greater suicide risk and who may not interact with specialty mental health. More research is needed to refine and evaluate impacts of this or similar training programs over time.
Footnotes
List of Abbreviations
Acknowledgements
The authors gratefully acknowledge the veteran consultants, focus group members, other interviewees, and primary care staff who contributed time to this project and participated in training sessions. The authors also acknowledge and appreciate the efforts made by the Health Services Research and Development (HSR&D) Centralized Transcription Service Program (CTSP) to complete transcription of all interviews for this project.
Authors’ Contributions
SD and MG conceived of the project. SD, MG, KC, EK, JS, and JB designed and participated in delivery of the training program. EK and VE extracted and analyzed quantitative data for the program evaluation. SD and SN were major contributors to writing of the manuscript; all authors provided edits. All authors read and approved the final manuscript.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Availability of Data and Materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the United States Department of Veterans Affairs (VA), Veterans Health Administration, VA Office of Mental Health and Suicide Prevention, and the VA Health Services Research and Development Center to Improve Veteran Involvement in Care (CIVIC) (CIN 13-404). The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or United States government.
Ethical Approval
The evaluation for this project was considered a human subjects study and did have IRB approval: It was approved by the VAPORHCS Institutional Review Board (IRB) on 11/30/2018 (reference #4347); the IRB approved a waiver of written informed consent; all participants were provided an information sheet and gave verbal consent to participate.
Trial Registration
Not applicable, because this article does not contain any clinical trials.