Abstract
Objective:
To investigate clinical associations within Canadian psychiatric inpatient suicides.
Method:
We conducted a case-control study comparing 98 patients who died by suicide while in hospital and 196 similarly admitted living inpatient controls. All were admitted to an Ontario psychiatric bed between 2008 and 2015 inclusive and had data collected at admission using the Resident Assessment Instrument for Mental Health (RAI-MH). The data were analyzed with bivariate tests and logistic regression modeling.
Results:
The strongest associations with inpatient suicide were prior admission to any Ontario psychiatric bed within the previous 30 days (odds ratio [OR] = 6.13), self-harm assessed at prior admission to a psychiatric hospital other than the hospital of suicide (OR = 6.07), and prior admission to a psychiatric hospital other than the hospital of suicide in the previous year (OR = 5.38). A multivariate model using risk factors assessed at admission had an area under the curve (AUC) of 0.77. The model improved to (AUC) 0.81 using a retrospective search of all Ontario admissions to more accurately detect prior admissions. The risk model was optimized to (AUC) 0.83 when the model also included a “discrepancy” variable to denote records in which admission assessment data and retrospective search data did not agree regarding past month admissions.
Conclusions:
In addition to the well-known risks of suicide associated with previous suicide attempts and depressive conditions, our data suggest a particular risk of inpatient suicide associated with inpatient care in more than one hospital, particularly when the treating clinicians were unaware of recent previous admissions.
Around the world, current rates of suicide by psychiatric inpatients are alarmingly high. A recent meta-analysis looking back as far as 1945 found 1 in 676 admissions has on average ended in a suicide. However, as average lengths of stays have declined over time, the pooled rate of suicide has risen sharply within contemporary psychiatric hospital wards, now exceeding 600 per 100,000 person-years of inpatient care, 1 and is therefore about 50 times the global suicide rate. 2 Such high rates are understandable, given that inpatient mental health beds are a common final destination for people who are feared to be at risk for imminent suicide.
While it is true that decisions about who is admitted to a psychiatric bed are constrained by a wide variety of factors other than suicide risk, including the availability and suitability of community care and the relevant mental health law, 3 final decisions to admit or not to admit are largely heuristic in nature, being heavily influenced by clinician attitudes around suicide and self-harm. 4,5 As a result, many, if not most, inpatients are admitted because of perceived “danger to self” or acute “suicidality” or following an act of deliberate self-harm. 6,7
At the start of any admission within North America, protocols call for mental health patients to be assessed for suicide risk. 8 –12 In the United States, the 2015 Hospital National Patient Safety Goals set by The Joint Commission into Patient Safety recommend that all patients admitted to psychiatric hospitals should have a risk assessment that “identifies specific patient characteristics…that may increase or decrease the risk for suicide.” 13 Similarly in Canada, the 2017 update for “Required Organizational Practices” from Accreditation Canada mandate all health care providers to identify patients at risk of suicide and to track changes in risk so that “the immediate safety needs of clients identified as being at risk of suicide are addressed.” 14
Despite the obstacles posed to researchers by the rarity of suicide events, there is now a substantial body of research about risk factors for inpatient suicide. A 2011 meta-analysis of 28 controlled studies of inpatients’ suicide identified a wide range of risk factors, including any previous suicidal behavior, suicide attempt at the point of admission, depressed mood or an affective disorder, hopelessness, worthlessness, a family history of suicide, and suicidal ideation. 15 More recently, a meta-analysis of 18 studies found that high-risk categorization for suicide based on the presence of 2 or more clinical risk factors was associated with a pooled odds ratio of 7.1 and a meta-analytic area under the curve (AUC) of 0.83. 16 Despite the strength of this association, the authors urged caution about the application of this form of risk assessment given that its 53% sensitivity would leave about half of all future suicides judged “low risk.” Conversely, the very modest 0.4% positive predictive value would translate to fewer than 1 in 200 high-risk patients actually dying by suicide while in hospital.
The reasons for the limited performance of risk categorization in inpatient settings require further investigation. One possibility is that the strength of the association with the independent variable of suicide is limited by the accuracy in the measurement of the risk factors. Although there is almost no empirical evidence examining the effect of inaccurate assessment of risk factors on the strength of suicide risk categorization, this type of inaccuracy was identified by the majority of the 979 respondents to a recent survey of active clinical trial researchers. 17 Here, the survey respondents railed against the supposed validity of suicidality data and their responsibility for predicting suicide, citing their greatest challenge to be an inability to obtain accurate baseline histories of previous care and hospitalization. Concern around the accuracy of patient histories is further warranted given some findings around underreporting of inpatient suicides. In 1995, Blain and Donaldson 18 found evidence of missing inpatient suicide records, while a 2015 meta-analysis suggested more generally that studies in which suicide was ascertained using hospital records were associated with significantly lower inpatient suicide rates than those that relied on coronial data. 1
The advent of electronic medical records (EMRs) alongside standardized assessment tools hold potential via linkage to estimate the interrater reliability of traditional approaches to risk evaluation, and ultimately, guide meaningful refinements to current suicide prevention efforts. In Ontario specifically, concurrent with a gradual expansion of EMR use throughout the province, since 2008, all providers of mental health inpatient services have been mandated to assess each patient at admission, discharge, and in the case of extended stays quarterly, using the Resident Assessment Instrument for Mental Health (RAI-MH), which encompasses more than 300 data points, including demographics, symptom levels, Diagnostic and Statistical Manual of Mental Disorders diagnoses, and provided treatments (http://www.interrai.org/).
Pooled electronic data holdings allow not only novel examinations of many demographic, illness, and treatment factors immediately preceding suicidal admissions but also retrospective searches to examine all Ontario health care provided in the months and years prior to an inpatient suicide. The RAI-MH data available in Ontario have considerable advantages over other data sets used in the study of inpatient suicide because of the breadth of assessment data, all collected prospectively and stored electronically, thus combining not only the unbiased predeath demographic data commonly found in other administrative data sets used by inpatient suicide studies 19 –21 but also the clinically nuanced data that are usually found only in retrospective studies involving postmortem clinician questionnaires 22 or data extracted from medical records. 23,24 Moreover, a shared Ontario data repository is set up in such a way as to allow searches by licensed users via nonidentifiable encrypted patient numbers, thus to locate all clinical care data provided during admissions recorded at an earlier time and, possibly, at a different hospital from the final hospital in which suicide occurred.
Using the approach described above, we report the results of a case-controlled population-based study of all recorded inpatient suicides occurring within all psychiatric wards (i.e., both specialized and general hospitals) in the province of Ontario over an 8-year period. In addition to investigating risk factors for inpatient suicide in a Canadian setting, we tested whether a more accurate measure of independent variables associated with previous hospital care could enhance suicide risk assessment.
Methods
Study Data
The study data set was compiled from standardized assessment records required in Ontario for each inpatient psychiatric episode of care as recorded by clinicians at the point of admission and discharge, using the RAI-MH.
The deidentified case records of 98 individuals were obtained from the Ontario central health record repository (https://intellihealth.moh.gov.on.ca/). 25 Each involved an episode of care that began between January 1, 2008, and December 31, 2015, and ended before April 1, 2016, with “death due to suicide” coded as the “reasons for discharge.” The reliability of case ascertainment using this method was confirmed by the absence of any postdischarge health service activity in each case. The admission date of each case was used to sample 2 randomly selected controls with the same admission date, resulting in 196 living controls who were admitted in Ontario in the same time frame. In line with earlier studies of inpatient suicide, 19,22 and our being unaware of any such study using Canadian data, we elected no further matching of cases and controls, thereby collecting for the first time comparable estimates in Canada of suicide risk associated with basic demographic factors such as age and gender.
Risk Factors
The “Suicide” section of Kaplan and Sadock’s Comprehensive Textbook of Psychiatry (2009 edition) 26 (pp2717-2732) was first reviewed to identify in-patient risk factors contemporary psychiatrists would plausibly consider. Some preliminary modifications (e.g., dichotomization of education to “incomplete high school”) were made in line with current modes for reporting aggregated RAI-MH data back to hospital decision makers.
Embedded Scales
Given the paucity of data about the predictive properties of rating scales for suicide in inpatient settings, 16 2 dichotomous measures based on embedded RAI-MH scales were also included: (1) a score of greater than 0 on the Severity Of Self-Harm Scale (SOS) and (2) a score of greater than 2 on the Depression Severity Index (DSI).
Hospitalization History Variables
Two measures of hospitalization history were compared (Assessment and Administrative), along with a measure of disagreement (Discrepancy) between the 2 ways of recording past 30-day hospitalization. The Assessment measure of previous hospitalizations was based on a “Time Since Last Discharge” RAI-MH question required for answer at admission and based on the patient’s account and (when available) corroborated by family, friends, and existing hospital records. 27 The Administrative measure of previous hospitalizations was extracted from the online repository of all Ontario admissions using encrypted patient numbers to locate all “actual” prior hospitalizations in Ontario. A dichotomous Discrepancy variable was created to identify records in which RAI-MH data for past 30-day hospitalization (Assessment) did not align with a retrospective search of actual past 30-day hospitalization (Administrative).
Statistical Analysis
A descriptive analysis of the 98 suicide cases first explored factors such as prevalence and timing of suicide. Next, unadjusted risk factor rates in the suicide and nonsuicide groups were calculated. Significance testing was carrying out using chi-square tests (or Fisher’s exact test where any cell number was <5) or Mann-Whitney U test with a threshold of significance of <0.05. In some instances, continuous or interval variables were dichotomized ad hoc to calculate odds ratios (OR) consistent with previous research or standards for contemporary reporting. 15 The magnitude of un-adjusted effect sizes was classified as weak (OR between 1.5 and 2.5), moderate (OR between 2.5 and 4), strong (OR between 4 and 10), or very strong (OR greater than 10). 28 While no correction was made at this stage to significance testing because of multiple comparisons, a Bonferroni correction for an estimated 25 uncorrelated factors suggests a P value threshold of <0.002.
The reliability of clinically reported prior hospitalization (Assessment) compared with our retrospective calculation of prior hospitalization (Administrative) was estimated using sensitivity and specificity and Kappa statistics. The level of agreement defined by Kappa was classified according to a kappa of 0 to 0.2 poor, 0.2 to 0.4 fair, 0.4 to 0.6 moderate, 0.6 to 0.8 good, and 0.8 to 1.0 very good. 29
Logistic regression tests were constructed with Discharge Due to Suicide as the dependent variable of interest. Multiple collinearities were assessed prior to logistic regression using the variance inflation factor (with a test threshold of 5) for each independent variable in every combination by iterative testing against every independent variable. Independent variables were then included in a backward stepwise conditional logistic regression with an alpha of 0.05 at entry level, an exclusion level of 0.10, and overall decisions driven by the likelihood ratio statistic. The differing variables related to hospitalization history allowed final comparison of three models: (1) an Assessment-Derived Previous Admission Model using the “self-report” RAI-MH measure of past-year hospitalization, (2) an Administrative-Derived Previous Admission Model using our retrospective measure of past-year hospitalization determined from a search by encrypted patient numbers of all past-year hospitalization records, and (3) an Administrative-Derived Previous Admission + Discrepancy Model using both our aforementioned measure of “actual” past year hospitalization and a Discrepancy variable to denote records in which Assessment and Administrative past 30-day hospitalization data did not agree. Measures of performance by the final model were calculated using the constant and a probability cut point of 50%. The analyses were performed using SPSS for Mac, version 24.
The research was approved by the ethics review board of the principal investigator’s hospital (Orillia Soldiers’ Memorial Hospital).
Results
We located 98 Ontario inpatients plausibly discharged due to suicide in the 8 years between 2008 and 2015. Demographic majorities at admission were coded male, single, and unemployed. Median age at death was 43.5 years. Most had been admitted previously. Clinical majorities at admission had a mood disorder, threats of harm, and history of self-harm (Table 1).
Common suicide risk factors within Ontario mental health Inpatient episodes of care (2008-2015).
aNonparametric 2-sided Mann-Whitney U test used due to nonnormality.
bBased on Province of Ontario Administrative Records for All Inpatient Mental Health Care.
c“Within past year” includes “within past 30 days” cases.
dFisher’s exact test used due to small-n cells (<5).
*Bonferroni corrected P < 0.05 (i.e., corrected P < 0.002).
Assuming our data to be complete, an inpatient suicide occurred on average once every 4503 admissions. Further, there were a total 11,653,122 days of inpatient mental health care in the period January 1, 2008, to March 31, 2016; thus, our data measured 307.2 suicides per 100,000 person-years. Inpatient suicides occurred most often (73%) within acute-care or crisis programs. The length of inpatient stay prior to suicide ranged from 1 to 1116 days, with a 16-day median. Almost half (48%) occurred between 2 and 4 weeks after admission. Suicides occurred more often on Sundays and during the winter season (December 22 to March 21). Increasing amounts of daily precipitation were significantly associated with dates of inpatient suicide. (Based on data collected at the Barrie-Oro weather station in central Ontario. Approximately 80% of the provincial population lives within a 130-km radius of this station. Precipitation occurred on 56% of suicide days versus 47% of nonsuicide days. Average total daily precipitation was significantly greater on suicide days [3.8 mm vs. 2.4 mm], Mann Whitney Z = 1.81 [1-tailed, P = 0.035].) 30
Factors strongly associated with inpatient suicide by way of an OR greater than 4 included prior psychiatric admission to a hospital other than the hospital of the suicide within the past year, prior psychiatric admission to a hospital other than the hospital of the suicide within the past 30 days, self-harm reported at prior psychiatric admission to a hospital other than the hospital of suicide within the past year, an SOS score greater than 0, and self-harm with fatal intent reported at prior psychiatric admission within the past year (Table 1). Factors with a moderately strong association to inpatient suicide and an OR between 2.5 and 4 included prior psychiatric admission to any Ontario hospital in the past 30 days, prior psychiatric admission to any Ontario hospital in the past year, any history of self-harm with fatal intent, admission due to “threat to self,” any history of intentional self-harm, and primary diagnosis of unipolar depression. Factors with weak associations to inpatient suicide having an OR between 1.5 and 2.5 included male gender, not having an income, hopelessness, loss of interest, reduced social interaction, and having a mood diagnosis. Paradoxically, recent hazardous alcohol use (defined as the patient reporting drinking more than 1 drink per setting within 2 weeks of admission) was protective, albeit with borderline statistical significance (Table 1).
The RAI-MH–derived Assessment measure of any past-year admission had good agreement with our retrospective Administrative determination of any past-year admission (sensitivity = 0.88, specificity = 0.87, k = 0.71). The RAI-MH–derived Assessment measure of any past-month admission had moderate agreement with our retrospective Administrative determination of any past-month admission (sensitivity = 0.41, specificity = 0.99, k = 0.49). However, a final comparison of past-month admission to a facility other than the final facility (of suicide) showed little agreement between the Assessment and Administrative measures, albeit significantly better than would be expected by chance alone (sensitivity = 0.22, specificity = 0.96, k = 0.16).
Three backward stepwise logistic regression models were completed, fitting (1) only RAI-MH data, including the embedded RAI-MH Assessment measures of previous admission; (2) the same RAI-MH data except for our retrospective Administrative measures now accounting for previous admission and previously assessed self-harm; and (3) the same linkage of RAI-MH and our retrospective Administrative measures of previous admission and previously assessed self-harm, supplemented with the Discrepancy measure of past 30-day admission disagreement between the Assessment and Administrative values (Table 2; Figure 1).
Three Inpatient suicide risk models (backward stepwise logistic regression).
Note. BIC = Bayesian Information Criterion
Receiver-operating characteristic curves for 3 logistic models of inpatient suicide prediction.
Using only the RAI data, our first modeling found male gender, hazardous drinking, an SOS score greater than 0, a history of intentional self-harm, and a mood diagnosis all independently associated with suicide, accounting for 26% of the observed variance in suicide with an AUC of 0.77. Replacing the RAI-MH–derived Assessment measures of previous admissions with our retrospective Administrative measures significantly strengthened our second model, now accounting for 38% of the variance with an AUC of 0.81. The predictive properties of the Administrative model were further strengthened in our final modeling via the inclusion of our Discrepancy measure, accounting for 40% of the observed variance with an AUC of 0.83.
The above final Administrative-Derived Previous Admission + Discrepancy Model correctly identified 60 of the 98 suicides (sensitivity = 0.61) and 171 of the 196 nonsuicides (specificity = 0.87) and was associated with an OR of 10.8. A positive predictive value of a high-risk categorization was calculated using the base rate of 98 suicides per 441,297 admissions (0.000222 suicides per admission or 1 suicide per 4503 admissions) and the optimized sensitivity and specificity above, finding that one could expect 1.1 suicides per 1000 high-risk admissions.
Discussion
Our findings identified 2 potentially important new risk factors for inpatient suicide: recent treatment in another hospital and an inaccurate assessment of recent hospital care associated with self-reports of previous admission and self-harm. Several earlier studies have examined whether having a greater number of previous admissions poses a risk for inpatient suicide 22 –24 or the timing of any previous admission poses a risk. 20,31 Results have been inconsistent, leaving no clear meaning to prior hospitalizations. Only 1 small study examined the risk of suicide associated with admission to another facility, with the statistically nonsignificant but nonetheless worrisome finding that suicide was twice as common in those who were previously admitted to 2 or more different hospitals. 23 While our findings about previous hospitalization warrant more detailed examination of existing data and replication, the association between being admitted to 2 or more hospitals and suicide is certainly plausible because of discontinuity of care, more unwell patients seeking further treatment in other hospital settings, and/or unsatisfied patients seeking alternative care following disappointing inpatient experiences. 32 It can be reasonably assumed that suicide is more likely when an otherwise-skilled treating team is unaware of key elements of a patient’s health care history, as is very likely when earlier care was provided at another hospital. This is consistent with the second of our novel risk factors, characterized as a discrepancy between clinically assessed and administrative data. While this sort of discrepancy might be associated with suicide for a variety of factors, for example, more unwell patients or those who present without family providing an inaccurate history of inpatient treatment and thus spuriously resulting in increased risk of suicide, our results do not preclude the possibility that some level of concealment on the patient’s part or lack of assiduousness on the part of the assessor pose genuine risk for inpatient suicide, particularly where treatment planning software is prepopulated “under the hood” with administrative data (e.g., RAI-MH), leaving no processes in place for downstream quality checks on data triggers. Irrespective of cause(s), we found that the predictive potential of suicide risk assessment in inpatient settings can be enhanced both by accurate information and by knowing when clinically acquired information is inaccurate. These findings are timely given that real-time sharing of accurate hospitalization histories between hospitals is currently feasible. In Ontario, for example, the Health Level Seven International (HL-7) format is mandated for management of all health data, allowing communication between computers within and between all provincial health care facilities. 33
In most other respects, our results are consistent with previous studies, finding a similar range of factors for inpatient suicide, including depressed mood and deliberate self-harm, 2 of the most commonly named risk factors for inpatient suicide. 15,16 In line with findings from the 2011 meta-analysis, we also failed to find significant suicide risk associated with some plausibly meaningful factors. Thus, in contrast to the significant risk commonly identified within general populations, we found no significant associations between inpatient suicide and older age, being single, living alone, and being unemployed. On the other hand, we did find male gender to be significantly associated with inpatient suicide, confirming male associations reported in the 2 largest and most recent studies of inpatient suicide. 20,34
Our findings with respect to the potentially protective role of hazardous alcohol consumption are surprising because of the clear association between alcohol use and suicide in the general population. 35 Furthermore, we worry lest our honest reporting of increased drinking among nonsuiciding patients become dangerous through a reader errantly concluding alcohol could reduce risk of suicide. Explanations for a protective role of recent alcohol use are that patients with an alcohol use disorder might be at lower risk of suicide because of shorter than average periods of hospitalization and the ability of hospitals to prevent inpatients from drinking.
The findings of our logistic regression modeling are broadly similar to earlier studies. While we report a statistical model with a slightly higher odds of suicide among high-risk patients than was estimated by meta-analysis (OR = 10.8 vs. 7.1) or reported in most primary studies, 16 a markedly lower positive predictive value (0.11% vs. 0.43%) resulted from the possible reassuringly low base rate of inpatient suicide in Ontario.
Some limitations of study should be acknowledged. First, it is possible that in some cases patients received recent mental health inpatient care outside Ontario (e.g., in the United States or another province). We would have missed these recent hospitalizations using our Administrative-Derived method for detecting past admissions. On the other hand, most of the Ontario population is clustered around the Toronto area, where the closest border is with the United States, an hour’s drive to the south. Crossing to the United States requires passport and customs clearance and thus seems unlikely to have occurred very often if at all for most of the patients in our study. However, given this nonzero possibility, our hopes for future study would involve linkages to data outside Ontario.
A second limitation involves our reliance on discharge coding of death by suicide. Because of the deidentification process within the provincial data repository, we had less than optimal means for confirming death by suicide (i.e., no opportunity to link to mortality records), and we had no means at all to ensure we captured all cases of inpatient suicide. Thus, while we are reasonably certain the 98 suicide patients truly died given that each had no further records of health care use past the date of suicide, we have no certainty whether inpatient suicide cases were missed and, if so, what sort of biases might be associated with missing cases. While there has been a declining trend in rate of inpatient suicide per admission, our reported rate of 1 suicide per 4503 admissions is less than one-third the estimated rate of 1 in 1222 from papers published since 1999. 1 It could be, for example, that some Ontario suicides were not classified in the RAI-MH in order to protect the interests of the family or even the treating team. More generally, there is evidence that clinical records of suicide tend toward underreporting, as compared with coroner records. 36
A third limitation concerns our case-control design, which did not allow survival analysis. We cannot exclude the possibility that covariations between some risk factors and length of stay might have mediated the magnitude or significance testing of some risk factors. Thus, we once again caution readers around inferences drawn from the increased temperance noted in patient completing suicide. While sobriety might be necessary to successfully plan and carry out suicide on an inpatient unit stripped of any ready means for self-harm, one should not assume that alcohol use would in any way be protective for a long-term patient on weekend leave, depressed and alone in a room with a gun.
Additional study-specific limitations relate to the sample size of less than 100 suicides, our decision not to control for demographic variables, and even our decision to dichotomize some continuous and interval variables, might have reduced the power of the study to identify some potentially significant risk factors.
Finally, there are intrinsic limitations to observational data in this area. It is likely that hospitals provided significant additional personalized protection to patients who had blatantly obvious risk factors such as previous suicide attempts or command hallucinations. Thus, the most important risks could remain hidden or dampened within this analysis.
Conclusion
While there are doubts about the meaningful benefits of risk assessment as a method of suicide prevention in hospital, not the least because of the very low positive predictive value of suicide risk assessment, the importance of our study lies in our finding of 2 potentially modifiable risk factors for inpatient suicide. Whenever possible, psychiatric treatment should be provided rapidly by 1 hospital. When this has not occurred, subsequent clinicians need to have accurate information about earlier psychiatric treatment.
Clinical Implications
Past-year hospitalization in multiple facilities, and discrepancies in health records regarding past hospitalization might be incorporated into clinical risk assessment. Health services should monitor and attempt to minimize both hospitalizations in multiple settings and discrepancies between clinical and administrative data about past hospitalization. Inaccurate assessment of risk factors might contribute to the current statistical limits of suicide risk assessment in inpatient settings.
Limitations
Plausible risk factors such as impulsiveness, family history of suicide, and “suicidal” command hallucinations are not assessed in the RAI-MH and thus could not be considered in the logistic modeling. The study relied on a clinical definition of discharge due to suicide rather than an official mortality database. Patients identified as being at high risk by clinicians might have been protected by the treatment they received, lessening the extent of the observed associations.
Footnotes
Authors’ Note
A preliminary report of these findings was presented as a podium presentation at the 2017 Waypoint Research Conference, Barrie, Ontario, May 16, 2017.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.