Assessing a Revised Curriculum to Improve the Assessment and Management of Psychiatric Crises in an Outpatient Family Medicine Residency Clinic

Abstract

Background and Objectives:

There are few trainings for primary care physicians (PCPs) on how to navigate a psychiatric crisis (such as suicidal ideation, homicidal ideation, psychosis, and/or intoxication that impairs decision making capacity) in the outpatient setting. We previously published an approach to fill this gap and now report the results of our efforts to improve the curriculum and assess if confidence was maintained over time.

Methods:

Four, 1-h didactic sessions on screening and logistically managing patients in need of inpatient or outpatient psychiatric care were created. Objectives were based on the complexities of state law, institutional policy, and expert opinion. All 15 residents in a family residency program participated in the didactic series. Resident confidence to complete all aspects of these scenarios was measured by questionnaire before (Time 1), immediately after (Time 2), and 5 months after completion of the didactic series (Time 3).

Results:

From Time 1 to Time 3, resident confidence significantly increased for 10/17 (58%) aspects of assessing and managing psychiatric crises (all P < .017). There was no evidence that confidence for any measure decreased from Time 2 to Time 3.

Conclusions:

The revised curriculum increased family medicine resident confidence to assess and manage outpatient psychiatric crises 5 months after the training was completed.

Introduction

Primary care physicians’ (PCP) treatment of mental health in the outpatient setting continues to significantly increase, rising from 10.7% in 2006 to 2007 to 15.9% in 2016 to 2018. ¹ The Accreditation Council for Graduate Medical Education (ACGME) requires that family medicine residents develop a core competency to “diagnose, manage, and integrate care for common mental illness and behavioral issues.” ² Mental health concerns can include a psychiatric crisis, such as acute suicidal ideation, acute homicidal ideation, psychosis, or intoxication that interferes with an individual’s decision-making capacity. Despite the importance of assessing and managing acute psychiatric crises in primary care, the limited literature that does exist focuses solely on suicidal ideation and neglects the various other forms of crises that can present in primary care. This can help explain why there is no data on the prevalence rate of managing a range of psychiatric crises in the outpatient setting. Additionally, we were unable to find any training curricula to assist PCPs with assessment and management, which is critical. One of the few factors that has been found to increase rates of screening for suicidal ideation in a primary care setting is training on risk assessment. ³

To improve PCP capacity to help patients during psychiatric crises, we previously developed a brief training curriculum with point-of-care resources and implemented that curriculum with PCPs still training within a family medicine residency. We found that this educational intervention increased family medicine residents’ confidence in the assessment and management of various psychiatric crises in our outpatient clinic. ⁴ Although confidence increased in all domains studied, we found that the curriculum needed to be improved because many residents still reported lack of confidence in multiple domains of assessing and managing a psychiatric crisis after having completing the training.

Therefore, the aims of the current study were to (1) create a more robust and evidence-based curriculum to improve confidence in all domains of assessing and managing a psychiatric crisis in all residents and (2) address methodological limitations from our previous work (ie, match participant responses across time, add an additional hour of content, survey confidence several months post-didactic series). We planned to achieve this by updating the educational contentand methodology, implementing the didactic series, and examining family medicine residents’ confidence in the assessment and management of multiple psychiatric crises before, immediately after, and 5 months after the completion of the training.

Methods

The current study took place in a family medicine residency in Wisconsin that serves both rural and non-rural communities. It was hypothesized that changes to, and implementation of, the curriculum with point-of-care resources and as-needed consultation with the residency’s behavioral scientist would help to improve residents’ confidence in the assessment and management of various psychiatric crises.

Participants

All 15 residents enrolled in the program participated in the study, none of whom participated in the previous study. The residency is a 3-year program, with 5 residents per class. The protocol was reviewed by our organization’s Educational Research Committee, which is charged with ensuring education research is conducted ethically, and the Institutional Review Board.

Procedures

The original curriculum was created by the residency’s behavioral scientist faculty member, who is also a clinical psychologist embedded in the residency program and outpatient clinic. ⁴ The new curriculum was modified by the behavioral scientist and a psychiatrist who specializes in inpatient behavioral health. The new curriculum was refined to include 4, 1-h didactic sessions that were presented monthly over 4 contiguous months, with the first didactic occurring in October and the fourth in January. All residents were excused from clinical duties and required to attend. All residents attended the didactic sessions together as 1 group. Completion of the surveys was optional. Residents were offered no incentive to participate and were informed that no punitive action would be taken nor would their standing in the program be affected if they chose not to participate in the study. Residents were also informed that data would be deidentified and sent to the principal investigator by an employee in the residency that did not participate in any evaluation nor have any supervisory role of the residents.

The first didactic session taught how to screen for suicidal ideation and behaviors using the Columbia-Suicide Severity Rating Scale (C-SSRS), ⁵ following the same steps as the original curriculum. ⁴ The second didactic included 2 objectives: (1) learn how to assess homicidal ideation, hallucinations, delusions, and intoxication using validated and reliable assessment tools and direct questions, and (2) learn how to determine whether or not a psychiatric crisis was present. This didactic also included review of Wisconsin state laws, such as duty to warn, and the differences between navigating a crisis when a patient is in clinic, on the phone, or using direct patient messaging via the electronic medical record (EMR). The third didactic focused on discussing the criteria for both voluntary and involuntary admission for inpatient psychiatric hospitalization, including review of the state and county legal logistics for involuntary hospitalization. Additionally, the nuances of how to admit a patient to the inpatient unit (voluntarily or involuntary) directly from clinic or via the emergency department were discussed. Further, EMR resources to aid in documentation and a 1-page, point-of-care practice guideline were included. Lastly, the fourth didactic primarily reviewed the processes for outpatient management, including developing a crisis safety plan, discussing coping strategies, and providing appropriate referrals. Each session built upon and reviewed previous content. Materials are available upon request.

A 1-page practice guideline document was also modified that provides a brief overview of the content included in the curriculum (see Figure 1). More specifically, it includes a table offering what information to consider when determining if a patient meets criteria for an outpatient or inpatient level of care for the different psychiatric crises and when the patient wants to be admitted voluntarily. Of note, not all patients who are asking to be admitted to an inpatient level of care will be. Certain admission criteria must be met; thus, the information to consider when determining what level of care is appropriate for someone who wants to be admitted was included in the table along with the different psychiatric crises. Additionally, the guideline document includes detailed steps to take depending on if the patient is being voluntarily versus involuntarily admitted to the inpatient behavioral health unit or remaining outpatient. The documentation templates in the EMR that were created to aid in efficient and thorough documentation of the assessment and management of each psychiatric crisis were not changed. The behavioral scientist’s availability to resident PCPs also remained consistent: (a) co-precepting with a family physician in the clinic 12 h a week, (b) being available 20 h a week for consultation, and (c) having accessibility to provide independent review of the protocol with any resident as needed.

Figure 1.

One-page practice guideline document to assess and manage a psychiatric crisis.

Evaluation Methods

We measured resident confidence with a modified version of the previous survey. ³ Confidence was selected as the dependent variable because (a) we lacked the resources to measure behavior and (b) PCPs confidence to assess suicidality is directly correlated with their likelihood of screening.^6,7 The survey used included 18 total items that directly related to the content of the didactic series, which was developed to appropriately reflect the nuances of Wisconsin state law, institutional policies, and expert opinion; see Figure 2 for survey question areas. The first survey item asked if the participant had managed a psychiatric crisis in the family medicine residency clinic, which was answered on a binary scale of “yes” or “no.” The other 17 items were answered using a 5-point Likert scale (0 = not confident at all, 4 = extremely confident). The survey was administered via REDCap ⁸ at the beginning of the first didactic (Time 1), at the end of the final didactic (Time 2), and 5 months later (Time 3).

Figure 2.

The stacked bar graph indicates the percentage of residents with that response for 17 of the 18 survey questions at each timepoint. Item 1 was not included in the graph because it was measured on a binary scale.

Differences across time points among de-identified individual responses, that is, paired samples, were assessed using the Wilcoxon signed rank test. Analyses were conducted in JMP (version 18.0.1) with a type 1 error rate was set at 1.67% to adjust for 3 comparisons: Times 1 and 2, Times 1 and 3, and Times 2 and 3. Any statistical significant increase was considered a valuable improvement in confidence.

Results

The response rate was 100% (15/15) at Time 1, 67% (10/15) at Time 2, and 100% (15/15) at Time 3. The distribution of residents’ confidence handling the components of assessing and managing psychiatric crises across the 3 time points is presented in Figure 2. Compared to Time 1, at Time 3, we found significantly greater confidence with (a) assessing the severity and risk of hallucinations and delusions, (b) locating and using the point-of-care decision tree and EMR resources, (c) overall managing a patient that meets inpatient criteria, (d) differentiating whether an admission is voluntary or involuntary, (e) ensuring safe transportation to the emergency department, (f) initiating the process for a direct admission, (g) initiating the process for an involuntary admission, (h) collaboratively developing a safety plan with a patient that will not be admitted, (i) determining whether one has enough information to break confidentiality, and (j) using external resources to help manage a patient’s psychiatric crisis. There was not a significant change in confidence with (a) screening for suicidal and homicidal ideation, (b) assessing the severity or risk of a patient engaging in a suicidal or homicidal behavior, (c) screening for hallucinations and delusions, (d) determining if a patient required immediate inpatient psychiatric hospitalization, (e) managing a patient who is high risk for suicide or homicide but will remain at an outpatient level of care, (f) determining if a patient can be directly admitted or needs to go to the emergency department prior to admission, and (g) recognizing when to utilize integrated care staff to aid in managing a psychiatric crisis.

From Time 1 to Time 3, residents’ overall confidence to manage a patient that needs psychiatric hospitalization significantly increased (P = .001); however, the increased confidence of managing a patient that does not need to be hospitalized did not reach statistical significance (P = .047). Confidence scores for all items increased from Time 2 to Time 3.

Discussion

Our updated didactic series and related resources (eg, EMR tools and practice guideline document) related to improved confidence in multiple areas assessed. We sought to improve our previous curriculum because, although confidence in all areas significantly increased over the 3-h didactic series, most residents still reported not feeling confident in many areas at the last didactic. ⁴ In the previous study, we conceptualized confidence as either confident (reporting fairly to extremely confident) or not (reporting not at all to somewhat confident), and we aimed for all residents to feel confident in all areas measured. Our results build on our first study by measuring change in confidence over time for individual residents. This provided proxy information regarding degradation of perceived knowledge by retesting confidence several months following completion of the training. We had also hoped to strengthen the training with an additional hour to the didactic series, while recognizing that brevity is essential for implementation. Overall, we found that the revised curriculum significantly increased confidence in many important areas, but not all.

Adding a fourth didactic to the current curriculum dedicated more time to the assessment of homicidal ideation, hallucinations, delusions, and intoxication, using validated and reliable tools and direct questions, and review of state laws (ie, didactic 2). It also increased time in the other didactic hours to provide more detail on the various logistics/nuances of admitting a patient to an inpatient behavioral health unit, all of which were areas of low confidence from the first curriculum. ⁴ In the current study, after participating in the curriculum, participants demonstrated increased confidence in many of the areas where more time was dedicated. Although we cannot make a direct comparison between the previous and updated curriculum, we found that the updated curriculum more clearly demonstrated increased confidence to manage the various aspects of psychiatric crises in primary care.

Confidence did not significantly increase for the assessment and management of suicidal ideation; this is likely due to the majority of residents being fairly to extremely confident at baseline. This could reflect earlier spontaneous experiential learning, as residents often learn how to assess and manage suicidal ideation early in their training. Further, as we found with our literature review, suicidal ideation is the primary area studied and area in which curriculum has been created; therefore, it is possible that residents had exposure to the assessment and management of suicidal ideation prior to residency, but not other types of crises.

Additionally, confidence with recognizing when to use integrated care staff was high at baseline and did not significantly increase. It is possible that having a behavioral scientist consistently active in the residents’ training and readily available for any issue led to residents understanding the role more clearly and feeling confident about when to use the behavioral scientist’s services before the training. However, this does not negate that 30% of residents at Time 1 had no to little confidence in utilizing integrative care staff, whereas at Time 3, all residents were at least somewhat confident. This is worthy of mentioning because it highlights that knowledge of how to use integrated team members can be lacking, even though learners know the person in the integrative role. This can be critical, as Wellen et al ⁹ conclude that the combination of suicide screening and use of integrative behavioral health within primary care can appropriately assess risk of harm and lead to proper care.

Overall, an important takeaway from this study for other family medicine clinics and/or residencies is that teaching PCPs how to assess and manage a psychiatric crisis in their outpatient clinic does not require a high amount of time investment for the training (ie, 4 total hours). More specifically, the training can be spread out in 1 h increments across months, making it easily implemented into the busy schedules of PCPs, especially those in training. Another key takeaway is that building a system that maintains the knowledge gained from the didactic content is important. This system can include tools similar to the ones created and implemented for this study, such as a guideline document for use during a psychiatric crisis and EMR resources, or other systemic supports relevant to the specific clinic and residency program.

Limitations

A limitation is that it is unclear if the didactic itself is what improved resident confidence or if other factors contributed to the changes (eg, natural development of skill over time). It is also unclear which specific elements of the curriculum were the most useful and helpful, which is important for creating the most parsimonious curriculum that can be realistically implemented in other primary care settings. Notably, this curriculum was only studied within 1 residency and this residency has a full-time dedicated behavioral scientist; thus, the results may not generalize to other settings. Additionally, confidence is a subjective perception and not indicative of behavior. Deficits of the current study include lack of skill practice during the didactic (eg, role play), no qualitative information from the participants on why they continue to not feel confident in some areas after the didactic series, and no specific procedures implemented for those who were unable to attend a single didactic.

Future studies could address the limitations and deficits of the current study. For example, future study designs could incorporate measurement of physician behavior change. Additionally, future studies could examine this curriculum in other primary care residency programs, noting the specifics to the state and institution would need to be changed. There also could be benefit in implementing this curriculum in non-training settings to improve confidence for PCPs later in their career. Further, modifying or adding other important skills to the curriculum may be beneficial for PCPs when assessing and managing a psychiatric crisis, such as verbal de-escalation techniques ¹⁰ and skills related to emotional intelligence (eg, recognizing when others and/or own emotions are escalating and tools to help regulate distressing emotions). ¹¹ Including a role playing component would likely be helpful for skill practice prior to using the skills in the clinic with a patient in high distress. Adding a qualitative component to the last survey to better understand participant’s responses could aid in improvement of the curriculum.