Sage Journals: Discover world-class research

Abstract

Violence against healthcare staff, including a threat or an act of violence toward people during their work, poses a physical and psychological risk to workers internationally. Screening is an important strategy in preventing violence against healthcare professionals. The aim of this systematic review was to synthesize evidence on the predictive validity of risk assessment tools used to screen for violence and aggression risk toward healthcare workers in emergency and psychiatric departments (PD). Primary studies that examined the predictive validity of risk assessment tools for workplace violence were identified via a systematic search of Medline, PsycINFO, Embase, and the Cochrane databases. There were 62 eligible studies, ten of which had a lower risk of bias (RoB). Those studies with high RoB were primarily due to a failure to present calibration measures as part of the analysis. All included studies adopted a longitudinal design and were conducted in PDs. The ten highest-quality studies reported on eight different instruments, four of which showed acceptable to outstanding predictive performance. The Dynamic Appraisal of Situational Aggression and the Brøset Violence Checklist showed the best predictive performance; they were also validated in emergency departments and are best suited for short-term risk prediction. We recommend that the selection of a risk assessment tool should consider the following: (a) the target population, (b) the violence operationalization, and (c) the purpose of the monitoring. We note that the use of a screening tool should be a part of a multicomponent strategy to ensure staff safety.

Keywords

violence emergency department psychiatric department predictive validity risk assessment

Introduction

Violence against staff poses a risk to healthcare workers internationally (Grant et al., 2022; Hawkins & Ghaziri, 2022). Specifically, research showed consistently high rates of occurrence of violent incidents in emergency department (ED) and psychiatric department (PD) (Liu et al., 2019; Stowell et al., 2016; Vento et al., 2020). A recent meta-analysis showed that 77% to 84% of ED and 67% of PD healthcare workers experience workplace violence (Aljohani et al., 2021; Liu et al., 2019). Violence against healthcare workers increased over the recent decade (Nikathil et al., 2018; O’Brien et al., 2024), but particularly during the COVID-19 pandemic (McGuire et al., 2022; Odes et al., 2023; Ramzi et al., 2022). This increase in violence against healthcare workers has brought the issue of safe work environments to the forefront (Joyce et al., 2023).

Screening for violence risk is an important part of the strategy to prevent violence against staff. To improve the risk of violence screening, several structured instruments have been developed and validated (Cabilan & Johnston, 2019; Fricke et al., 2023). Most available risk assessment tools were first developed in either psychiatric (Hvidhjelm et al., 2023) or emergency (Cabilan et al., 2023) settings, but later validated in the other setting (Dugré et al., 2019). There were many attempts to provide evidence-based guidance on using risk assessment tools in healthcare. Earlier reviews were often focused on a specific healthcare setting (D’Ettorre et al., 2020; D’Ettorre & Pellicani, 2017; Hamrick et al., 2023; Lorettu et al., 2020; Mento et al., 2020; Ogonah et al., 2023; Sammut et al., 2023), tool (O’Shea & Dickens, 2014; O’Shea et al., 2013), or across general healthcare settings (Fricke et al., 2023; Ghosh et al., 2019). Given that the ED and PD are the two settings that have the highest rate of workplace violence, and organizations are seeking improved safety (Goldstein, 2022), consistency in the use of the assessment tools for risk of violence will be beneficial. Such an approach can enable (a) consistent observation of patients and (b) their (re)assessment if they transfer to a new setting (e.g., from ED to PD), which can be useful for early identification and prevention (ACSQHC, 2018; Holland et al., 2021; Munich et al., 2021).

This review identified evidence-based tools that may have practical utility in these environments. The present study reports the evidence on the predictive validity of risk assessment tools used to screen for violence and aggression risk toward healthcare workers in emergency and psychiatric departments. Our interpretation focuses on tools with the most reliable evidence of predictive performance.

Methods

Design

We conducted a systematic review, following the Cochrane methodology (Higgins et al., 2024) and report our findings according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Page et al., 2021). The protocol of the review was prospectively published at the Open Science Framework (https://osf.io/gw7an/).

Eligibility Criteria

Eligible studies were (a) primary research published in a peer-reviewed journal; (b) evaluations of the predictive validity or performance of instruments developed to identify the risk of violence against healthcare workers; and (c) tested in an emergency or psychiatric department. We excluded qualitative studies and studies that were conducted in forensic and outpatient settings, or when the outcome was assessed post-discharge (e.g., violence in the community).

Search Strategy

Four databases (Medline, PsycINFO, Embase, and the Cochrane Database of Systematic Reviews) were searched separately from inception until July 18, 2023. No limits, restrictions (including language), or search filters were used (see Supplemental materials, Table S1). We also hand-searched the reference lists of relevant systematic reviews and included studies for forward and backward referencing. The results from all searches were imported into Covidence, and duplicates were removed. Three authors (S. K., S. R. E. D., M. L.) conducted all screening, quality assessment, and data extraction. Two authors independently reviewed the titles and abstracts of all search results, and then the full text of relevant articles to determine eligibility. Consensus or discussion with a third reviewer was used if the two reviewers did not agree on eligibility.

Risk of Bias and Quality Assessment

All eligible studies were independently assessed by two authors for Risk of Bias (RoB) using the Prediction model Risk of Bias Assessment Tool (PROBAST) (Wolff et al., 2019). PROBAST is designed to assess the methodological quality of studies investigating diagnostic or prognostic accuracy, providing an estimation of RoB for each study, with low, unclear, or high RoB and concern (applicability) categorizations. PROBAST consists of four domains (participants, predictors, outcomes, and analysis), containing 20 signaling questions for RoB assessment. In addition, applicability concerns are deemed “high” when the population, predictors, or outcomes of the study do not match those in the review question.

Following the PROBAST tool workflow (Moons et al., 2019), evidence of external validity (applying the conclusions of a scientific study outside the context of that study) was prioritized when controlling for RoB. Consensus or discussion with a third reviewer was used if the two reviewers did not agree on RoB scoring.

Data Extraction

Two authors (S. K., S. R. E. D., or M. L.) independently completed data extraction. All inconsistencies were discussed and resolved via online meetings. All data were extracted and placed in Excel sheets (Microsoft Corp., Redmond, WA, USA) developed by the authors. Extracted data included information on the prediction tool and outcome, country, data collection process, sample characteristics, inclusion/exclusion criteria, diagnosis, previously reported violence, predictive validity (performance measures), and reliability (internal consistency measures using kappa and Cronbach’s alpha). Predictive validity data included sensitivity and specificity, positive and negative predictive values, area under the receiver operating characteristic curve (AUC), and odds ratios (the odds of a violent individual being identified as violent using the tool). When considering AUC, we used the following categorizations: ≤ 0.69 poor, 0.70 to 0.79 acceptable, 0.80 to 0.89 excellent, and 0.90 to 1.00 outstanding (Muller et al., 2005). The agreement between observed outcomes and predictions refers to calibration, demonstrated via graphical assessment (calibration plot or slope), the Hosmer–Lemeshow goodness-of-fit test, calibration-in-the-large (Hilden et al., 1978; Steyerberg et al., 2010). Overall performance measures quantify the distance between the predicted outcome and actual outcome, or the variance in the data that the model can explain (R², Brier score) (Steyerberg et al., 2010). This is usually presented in percentages ranging from 0% to 100%, where a value of 0% means that a model does not explain any of the variance in the data, and 100% indicates all of the variance.

Results

Search Outcome

Our search identified 2205 titles, from which we identified 62 eligible studies (Figure 1), 60 were published in English, one in Hungarian, and one in Polish. Most studies were conducted in psychiatric departments or wards (n = 58), four were in ED. Most studies used violent or aggressive incidents as the outcome (n = 53), nine studies used other outcomes to operationalize violence, including seclusion (n = 5), absconding (n = 4), physical restraint (n = 2), and restraint by administration of intramuscular sedative medication (n = 1). Several studies used multiple measures of outcome (n = 6). Studies were conducted in 22 countries. The majority of the studies were longitudinal (n = 54), seven studies were cross-sectional, and one was unclear.

Figure 1.

PRISMA diagram: Predictive validity of violence screening tools in emergency and psychiatric services: A systematic review.

Identified Violence Risk Assessment Tools

Table S1 presents 33 risk assessment tools reported in the 62 included studies (Supplemental Material). The majority were developed in psychiatric settings (n = 28). Seven tools were developed in correctional psychiatric settings and then adopted for general public application: Violence Risk Screening-10 (V-RISK-10), Structured Assessment of Protective Factors for violence risk, Historical Clinical Risk Management-20 (HCR-20), Novaco Anger Scale (NAS), Short-Term Assessment of Risk and Treatability (START), Measures of Criminal Attitudes and Associates Attitudes Towards Violence scale, Psychopathy Checklist-Revised (PCL-R). There were two instruments developed in ED settings, Queensland Occupational Violence Patient Risk Assessment Tool (QOVPRAO; Cabilan et al., 2022) and Brief Rating of Aggression by Children and Adolescents( BRACHA; Barzman et al., 2013), the latter was specifically developed in ED for the assessment of pediatric psychiatric admissions. Finally, the Aggressive Behaviour Risk Assessment Tool (ABRAT) was developed for other settings (medical and surgical units) but then validated in the ED (Kim et al., 2012, 2022). Four studies indicated a mixed population of children/adolescents and adults with age ranges: 13 to 77 years (Marques et al., 2015), 9 to 63 years (Miller et al., 2016), 16 to 92 years (Pujol et al., 2020), and 12 to 55 years (Yuniati et al., 2020). Five studies were focused specifically on children or adolescent populations (Barzman et al., 2013; De Beuf et al., 2023; Dutch & Patil, 2019; Roaldset et al., 2023; Stafford & Cornell, 2003) (see Table S1). Out of 62 studies, eight did not report any information about the age of the participants.

Quality of Included Studies

There were no studies that were low RoB with low applicability concerns (see Table S2 for quality assessment results). Figure 2 presents the summary of the quality assessment for RoB and acceptability; most studies were classified as high risk or unclear. Given no studies were classified for overall low RoB to receiving “yes” or “probably yes” across all four domains, we prioritized studies receiving “yes” or “probably yes” for at least three out of four RoB domains and at least two domains out of three for applicability. Ten studies met these quality assessment criteria and are described further (Table 1).

Figure 2.

Graphical presentation of quality assessment: Risk of bias and applicability.

Table 1.

Risk of Bias and Applicability: Included Studies Only.

Note. RoB = Risk of Bias; + (green) indicates low RoB/low concern, - (red) indicates high RoB/high concern, ? (yellow) indicates unclear RoB/unclear concern about applicability.

Data Synthesis

Table 2 presents the participant and study characteristics of the ten studies included following the quality assessment outlined above. All of them were conducted in mental health settings, with two targeting the child and adolescent population. The outcome in the ten included studies was assessed using the following four instruments: five studies used the Overt Aggression Scale (OAS), three the START Outcome Scale (SOS), the Staff Observation Scale Revised (SOAS-R), and the Report for Aggressive Episodes (REFA). Finally, one study used DASA-YV, an instrument developed to assess the likelihood of imminent aggression (within the next 24 hours) that contains both subscales for risk assessment and observed outcomes, allowing for day-to-day evaluation of aggression risk.

Table 2.

Population and Study Characteristics: Included Studies.

Participant Characteristics
Author, Year	Source of Data	Recruitment/Data Collection Period	Study Settings	Country	Sample Size (Missing)	Age	Gender (%)	Study Population (Inc and excl Criteria)	Health Complaints/Diagnosis	History of Violence
Adolescents
Dutch, 2019	Prospective cohort study	NI	MH (psychiatric hospitals)	The United States of America	103 enrolled, 83 analyzed (20)	Median age 14 years (IQR 13–15); range 6–18 years	54 (66.7%) females	Inclusion: hospitalized children and adolescents with psychiatric behavioral issues	NI	NI
Stafford, 2003	Prospective study	NI	MH (psychiatric hospital)	United States of America	72 (NI)	Mean 14.1 years (SD = 1.5); range 12–17 years	Males 37 (51.4%) and 35 (48.6%) females	Inclusion: hospitalization 14 + days; adolescent inpatients. Exclusion: a psychiatric diagnosis of current psychosis or mental retardation.	65% previously hospitalized; mood disorder 37 (51.4%); adjustment disorder and depressive mood 2; conduct disorder 10; oppositional disorder 6; ADHD 6; adjustment 5; substance use disorder 2; PTSD 1	43% had a history of aggression or fighting
Adults
Braithwaite, 2010	Longitudinal prospective	Over 2 years	MH (civil psychiatric hospital)	Canada	34 (NI)	Mean 37.91 (SD = 11.72)	79% (27) male, 21% (7) female	Continuous recruitment, participants included if they had 60 days of observations	Schizophrenia 30 (88%), substance abuse 27 (9%), personality disorder 7 (21%), intellectual disability 6 (18%), mood disorder 4 (12%)	NI
Eriksen, 2018	Naturalistic prospective observational study	March 21, 2012 to March 20, 2013	MH (acute psychiatric ward)	Norway	348 (180)	Mean: 41 years (range 18–83)	156 (45%) males and 192 (55%) females	Inclusion: all patients admitted to the acute psychiatric ward during recruitment period	Substance abuse 51 (15%), psychosis 102 (29%), bipolar disorder 44 (13%), depression 51 (15%), anxiety 47 (14%), personality disorders 27 (7.8%), other or no disorders 26 (7.5%)	NI
Jalil, 2019	Pseudo-prospective cohort study	April 2013 and May 2015	MH (Medium and low secure psychiatric wards)	The United Kingdom	76 (NI)	Mean 34.0 years (SD = 11.6)	38 (50%) male, 38 (50%) female	Inclusion: aged of 18 years or older, diagnosed with one or more mental disorders, assessed by two of the routinely completed risk vulnerability. Exclusion: patients with a diagnosis of a neurocognitive or neurodevelopmental disorder, lacked the capacity to consent, or who were not fluent in English.	Personality disorder (51.3%), Schizophrenia spectrum disorder (42.1%), Bipolar and related disorder (6.6%)	NI
Marriott, 2017	Pseudo-prospective cohort study	May 2011 to January 2014	MH (psychiatric inpatient care units)	The United Kingdom (England)	527 (0)	Mean 40.4 (SD = 15.8); range 18–90	388 (73.6%) males; 139 (26.4%) females	Inclusion: 18 years or older, admitted to MH ward during the recruitment period, stayed over 3 months, existing 1+ START. Exclusion: missing START item data in excess of pro-rating guidelines	Organic 139 (26.4%); psychosis 203 (38.5%); personality disorders 78 (14.8%); developmental disorder 41(7.8%); psychosis and personality disorder 46 (8.7%); psychosis and developmental disorder 20 (3.8%)	Mental health act: informal 57 (10.8%); civil section 234 (44.4%); forensic section 236 (44.8%); Security level: medium security 144 (27.3%) low security 374 (71%) open 9 (1.7%)
O’shea, 2015	Pseudo-prospective cohort design	May 1, 2011 to October 31, 2013	MH (Secure inpatient psychiatric department)	The United Kingdom (England)	827 (Unauthorized leave = 194; 23.4%)	Mean 38.5 years (SD = 16.7)	597 (72.2%) males and 230 females (27.8%)	Inclusion: all patients resident in the services during the recruitment period, 1+ START risk assessment completed, 3 months in service after initial assessment. Exclusion: START assessment had more than five missing strength or five missing vulnerability ratings.	Schizophrenia (n = 332); disorders of adult personality and behavior (n = 267); organic, including symptomatic, mental disorders (n = 215); disorders of psychological development (n = 151); mental retardation (n = 146); mental and behavioral disorders due to psychoactive substance use (n = 101); mood disorders (n = 56); behavioral and emotional disorders with onset usually occurring in childhood and adolescence (n = 50); neurotic, stress-related and somatoform disorders (n = 36); behavioral syndromes associated with physiological disturbances and physical factors (n = 12)	NI
Rechenmacher, 2014	Prospective cohort study	6 weeks (October 11, 2010 to November 23, 2010)	MH (acute psychiatric ward)	Austria	232 (0)	Mean 47.04 (SD = 17.39)	Males 115 (49.6%), Females 117 (50.4%)	Included: consecutively admitted patients of 18 years or older. Excluded: under 18 years old, patient transfers within the participating wards and other departments of the hospital.	Organic mental disorders 13 (5.6%); substance abuse 43 (18.5%); schizophrenia/psychotic 48 (20.7%); mood disorders 81 (34.9%); neurotic disorders 34 (14.7%); other diagnoses 13 (5.6%)	NI
Roaldset, 2012	Prospective, naturalistic observational study	March 7, 2006 to March 6, 2007	MH (psychiatric hospital)	Norway	238 (104)	Mean 42.8 years (NI)	Males 72 (48%)	Inclusion: patients accurately admitted during the period of recruitment.	Depressive disorders 45 (30%), anxiety 30 (20%), bipolar disorder 28 (19%), substance use 22 (15%), alcohol abuse 15 (10%), personality disorders 7 (4.7%)	54 (36%) had a history of previous violence
Sada, 2016	NI	NI	MH (psychiatric department)	Mexico	225 (NI)	Mean 34.4 years (SD = 11.1)	117 (52%)	Included: aged 18 years or over; diagnosed with a psychotic, affective or personality disorder. Excluded: patients with a concomitant medical or neurological illness.	Psychotic disorder (n = 92, 40.9%), mood disorder (n = 72, 32%), personality disorder (n = 61, 27.1%)	NI

Note. NI = No information; START = Short-Term Assessment of Risk and Treatability.

From the ten highest-quality studies, there were eight screening tools used; their estimates of predictive validity and reliability are presented in Table 3. Seven studies reported reliability measures, ranging from 0.74 to 1. Four instruments (HCR-20, V-RISK-10, DASA-YV, BVC-CH) showed acceptable to outstanding predictive performance for violence using suitable performance measures, while one (PCL-R) had acceptable sensitivity, specificity, negative and positive prediction values. Evidence for DASA-YV and PCL-R consisted of adolescent and youth participants (Dutch & Patil, 2019; Stafford & Cornell, 2003). None of the studies clearly presented calibration measures that report agreement between expected and observed probabilities. The characteristics of these tools are presented in Table 4.

Table 3.

Tool Characteristics of Included Studies.

Risk Assessment Tool/Author, Year	Outcome, Measure	Sample Size (Missing)	Modelling Method	Discrimination Performance Measures (C-, D-Statistics, AUC Graph)	Overall and Calibration Performance Measures	Reliability
Adolescents
DASA-YV (Dynamic Appraisal of Situational Aggression–Youth Version)/Dutch, 2019	Aggressive incidents, measured by DASA-YV	103 enrolled, 83 analyzed (20)	The area under the ROC curve; logistic regression	Any aggression : AUC = 0.90 (95% CI [0.86, 0.95]), p < .01 OR = 2.63 (95% CI [2.1, 3.3]), p < .01 Cutoff >= 3: AUC = 0.84 (95% CI [0.78, 0.90]), p < .01 Physical aggression against other people : AUC = 0.84 (95% CI [0.75, 0.93]), p < .01 AUC graph—yes	Calibration, Overall measures: NI	k = .79 (95%CI [0.70, 0.89]), p < .001; OR = 2.63 (95% CI [2.1, 3.3])
PCL-R (Psychopathology Checklist Screening -Revised)/Stafford, 2003	Violent incidents, measured using the Overt Aggression Scale (OAS)	72 (NI)	Regression analysis, discriminant function analysis	PLC-R predicted aggression median split V vs VN groups with 71% accuracy. Wilk’s lambda = 0.75, χ² = 19.75, p < .001 SN = 71% SP = 70%; PPV = 0.69; NPV = 0.72. AUC graph—No	Calibration measures: NI Overall measures: Total aggression: ΔR² = 0.14, F(1,66) = 17.87, p < .001; Peer aggression : ΔR² = 0.06, F(1,66) = 5.84, p < .05	NI
Adults
START (Short Term Assessment of Risk and Treatability)/Braithwaite, 2010	Aggressive incidents and unauthorized leave within 30 days after assessment done by START-Outcome Scale (SOS)	34 (NI)	The area under the ROC curve, logistic regression	Aggression toward others: Vulnerability scale: AUC = .66 [0.56, 0.75], p < .001 Strengths scale: AUC = .65 [0.56, 0.74], p < .05 Optimized vulnerability scale: AUC = .70 [0.61, 0.79], p < .001 The optimized strength scale: AUC = .65 [0.56, 0.75], p < .001 Unauthorized leave: Vulnerability scale: AUC = .65 [0.61, 0.79], p < .05 Strengths scale: AUC = .66 [0.53, 0.78], p < .05 Optimized vulnerability scale: AUC = 0.67 [0.56, 0.79], p < .05 Optimized strength scale: AUC = .70 [0.58, 0.82], p < .001. AUC graph—No	Calibration measures: NI Overall measures: Aggression toward others: Optimized vulnerability scale: OR = 1.23 (95% CI [1.03, 1.45]), p < .001 Optimized strengths scale: OR = 0.60 (95% CI [0.41, 0.88]), p < .05	NI
Jalil, 2019	Aggressive incidents, measured using the Overt Aggression Scale (OAS)	76 (NI)	The area under the ROC curve, logistic regression	Any aggression: Strengths total: AUC = .56 [0.43, 0.69], p = .34 Vulnerabilities total: AUC = 0.46 [0.33, 0.59], p = .58 Physical Aggression only: Strengths total: AUC = 0.69 [0.54, 0.83], p < .01, SN = 0.31, SP = 0.49 Vulnerabilities total: AUC = 0.56 [0.40, 0.71], p = .52 The model correctly classified 70% of cases. SN = 64.7%, SP = 73.8%, PPV = 66.7%, NPV = 72.1%.	Calibration measures: NI Overall measures: The model explained 21% of the variance and 30% of the variance. Physical aggression : OR = 0.95 (95% CI [0.78, 1.16]), n.s.	k = .74
Marriott, 2017	Aggressive incidents and absconding, measured using the START Outcomes Scale (SOS)	527 (0)	The area under the ROC curve; logistic regression	Physical aggression: Strength scale: AUC = 0.68 (99% CI [0.61, 0.74]), p < .001 Vulnerability scale: AUC = 0.65 (99% CI [0.59, 0.72]), p < .001; violence SRE: AUC = 0.68 (99% CI 0.61, 0.75) OR = 1.09 [1.05, 1.13], p < .001 Strength scale: OR = 1.11 [1.06, 1.15], p < .001 AUC graph—No	Calibration measures: NI Overall measures: Vulnerability scale: F(5, 521) = 5.59, p < .001;	k = 1
O’shea, 2015	Violent incidents, unauthorized leave, using the Start Outcome Scale (SOS)	827 (Unauthorized leave = 194; 23.4%)	The area under the ROC curve; t tests, and correlations.	Unauthorized leave: Vulnerability scale: AUC = .659 (95% CI [0.531, 0.786]), p < .05 PPV = 5.9%, NPV = 98.4%; SN and SP not reported. OR = 1.10 (95% CI [1.00, 1.20]), p < .05 AUC graph—No	Calibration, overall measures: NI	k = .857
HCR-20 (Historical, Clinical, Risk Management)/Jalil, 2019	Aggressive incidents, measured using the Overt Aggression Scale (OAS)	76 (NI)	The area under the ROC curve, logistic regression	Any aggression: Clinical: AUC = 0.72 [0.61, 0.84], p < .01; SN = 0.74, SP = 0.33 Risk Management: AUC = 0.63 [0.50, 0.76], p < .05, SN = 0.62, SP = 0.43 Physical Aggression only: Clinical: AUC = 0.81 [0.71, 0.92], p < .001, SN = 1.00, SP = 0.41 Risk Management: AUC = 0.66 [0.48, 0.84], p = 0.7 Any aggression:HCR-20 Clinical and HCR-20 Risk Management tool: PPV = 66.7%, NPV = 72.1%. Physical aggression:The model explained 30% of the variance and correctly classified 83% of cases. SN = 15.4%, SP = 97%, PPV = 50%, NPV = 85%. AUC graph—No	Calibration measures: NI Overall measures: The model explained 21% of the variance and 30% of the variance. Only, model 1 (HCR-20 Clinical Total and Strengths Total) was statistically significant v2 1⁄4 15.15, p < .01.	k = .74
Sada, 2016	Violent incidents, measured using the Overt Aggression Scale (OAS)	225 (NI)	ANOVA repeated measures model, the area under the ROC curve	AUC = 0.86 (95% CI [0.79, 0.92]) AUC graph—No	Calibration, overall measures: NI	k = .85; α = .83
V-RISK-10 (Violence Risk Acreening-10); V-RISK-EXT (Extended)/Eriksen, 2018	Violent incidents, measured using the Staff Observation Scale-Revised (SOAS-R)	348 (180)	The area under the ROC curve, logistic regression	V-RISK-10: AUC = .79 (95% CI [0.73, 0.85]), p < .001; SN = 0.86; SP = 0.58; PPV = 0.29; NPV = 0.96; NNA = 3.5; LR+ = 2.1; LR- = 0.24 OR (95%) = 1.3 [1.2, 1.4] V-RISK-EXT: AUC = .80 [0.74, 0.85], p < .001; SN = 0.88; SP = 0.61; PPV = 0.31; NPV = 0.96; NNA = 3.3; LR+ = 2.0; LR AUC graph—No	Calibration measures: NI Overall measures: V-RISK-10: Chi-squared = 51(1), p < .001; Cox & Snell R^2—Nagelkerke R² = 14–23% explained variance. V-RISK-EXT: Chi-square = 54 (4), p = .001; 14–24% explained variance	NI
Roaldset, 2012	Violent incidents, measured using the Report for Aggressive Episodes (REFA)	238 (104)	The area under the ROC curve, logistic regression	Any violence: V-RISK-10: AUC = 0.849 [0.77, 0.93], p < .001 V-RISK-10-EXT: AUC = 0.868 [0.79, 0.94], p < .001 Cutoff >= 6.5: SN = 0.79, SP = 0.71, PPV = 0.42, NPV = 0.92, NND = 2.4, LR+ = 2.7 [1.4, 5.4], LR- = 0.30 [0.15, 0.60] V-RISK-10: Cutoff >= 8.5: SN = 0.79, SP = 0.80, PPV = 0.51, NPV = 0.93, NND = 2.0, LR+ = 4.0 [1.7, 9.2], LR- = 0.26 [0.11, 0.60] Physical violence: V-RISK-10: AUC = 0.891 [0.80, 0.98], p < .00 V-RISK-10-EXT: AUC = 0.906 [0.82, 0.99], p < .001 AUC graph—No	Calibration measures: NI Overall measures: Univariate analysis: Age, gender, V-RISK-10, total cholesterol (TC) and SRS; for V-RISK-10: OR = 1.40, (95% CI [1.23, 1.56], p < .001). For violent acts, χ2 increase was 4.8 (df = 2, p = .093).	NI
NAS (Novaco Anger Scale)/Jalil, 2019	Aggressive incidents, measured using the Overt Aggression Scale (OAS)	76 (NI)	The area under the ROC curve, logistic regression	Any aggression: AUC = 0.64 [0.51, 0.77], p = .04, SN = 0.64, SP = 0.33 Physical Aggression only: AUC = 0.75 [0.60, 0.89], p < .001, SN = 0.85, SP = 0.40 The model correctly classified 70% of cases. SN = 64.7%, SP =73.8%, PPV = 66.7%, NPV = 72.1%. AUC graph—No	Calibration measures: NI Overall measures: Any aggression: B = 0.01 (95% CI [0.03, 0.05]), n.s. OR = 1.00 (95% CI [0.97, 1.04]), n.s. R² = .16(Cox & Snell); .21(Nagelkerke). χ² (2) = 13.00, p < .01. χ2(2)-χ2(1) = 0.12 p > .05. Physical aggression: B = 0.03 (95% CI [0.03, 0.10]), n.s. OR = 1.02 (95% CI [0.98, 1.07]), n.s. R² = .20(Cox & Snell), .33(Nagelkerke). Model χ2(2) = 16.81, p < .01. χ2(2)-χ2(1) = 1.66, p > .05	k = .74
BVC-CH (Extended Brøset Violence Checklist)/Rechenmacher, 2014	Aggressive incidents, measured the Staff Observation Scale-Revised (SOAS-R)	232 (0)	The area under the ROC curve	AUC =.931 (95% CI [0.882, 0.980]) Cutoff >= 7: SN = 58.8% (95% CI [34.900, 79.800]), SP = 96.8% (95% CI [95.600, 97.600]). The prevalence was 1.2%. PPV = 18.5% (95% CI [9.700, 31.800]), NPV 99.5% (95% CI [98.900, 99.800]). There were 51 false ratings registered (7 false-negative and 44 false-positive cases). For this cutoff, likelihood ratios (LRs) of LR+ = 18.10 (95% CI [11.100, 29.600]) and LR- = 50.43 (95% CI [0.240, 0.750]). Cutoff >= 6: SN = 64.7% (95% CI [41.300, 82.700]), SP =95.1% (95% CI [93.800, 96.200]). The prevalence = 1.2%. The PPV = 14.3% (95% CI [8.200, 23.800]); NPV = 99.5% (95% CI [99.000, 99.800]). The number of false ratings = 72 (6 false-negative and 66 false-positive cases). The likelihood ratios (LR+ = 13.10, 95% CI [8.700, 20.300]). AUC graph—yes	Calibration, overall measures: NI	NI
SRS (Self-report Risk Scale)/Roaldset, 2012	Violent incidents, measured using the Report for Aggressive Episodes (REFA)	238 (104)	The area under the ROC curve, logistic regression	Any violence: AUC = 0.64 (0.51–0.76, p = .025) AUC graph—No	Calibration measures: NI Overall measures: Univariate analysis: Age, gender, V-RISK-10, TC and SRS; for SRS: OR = 6.6 (95% CI [3.3, 13], p < .001]. For violent acts, χ2 increase was 4.8 (df = 2, p = .093).	NI

Note. Sensitivity (SN) = proportion of people who were violent and were correctly predicted to be violent; Specificity (SP) = proportion of people who were nonviolent and were correctly predicted to be nonviolent; FPR = false-positive rate (probability that a person who is predicted to be violent will be nonviolent); PPV = positive predictive power (accuracy of predictions that individuals will be violent); NPV = negative predictive power (accuracy of predictions that individuals will not be violent); OR = odds ratio (the odds of a person being violent who scored above the cutoff score are X times the odds of a person who scored below the cutoff score); NI = No information.

Table 4.

Risk of Violence Assessment Tools in Emergency and Mental Health Settings: Selected on Quality Assessment.

Violence Risk Tool/Included Study	Country of Development, Setting Used	Content	Scoring/Interpretation of Risk
Brøset Violence Checklist (BVC)/Rechenmacher, 2014	Developed in Norway in a high-secure mental health setting. Used in Australia, the United Kingdom, Switzerland, Austria, the Netherlands, Sweden, Norway, Indonesia, China, Turkey, Sri Lanka, Portugal in the settings of MH and ED.	Instrument for predicting inpatient violence within the next 24-hr period. 6 items: -confusion -irritability -boisterousness -physical threats -verbal threats -attacking objects	Each item scored 0 (absent) or 1 (present) Interpretation: 0 = low 1–2 = moderate ≥3 = high
Dynamic Appraisal of Situational Aggression (DASA)/Dutch, 2019	Developed in Australia in MH setting (secure mental health). Applied in the United States, Australia, and Canada in MH and ED settings. Alternative versions: DASA-IV (inpatient version), DASA-YV (children and youth)	7 items: -irritability -impulsivity -unwillingness to follow directions -sensitivity to perceived provocation -easily angered -negative attitudes -verbal threats	Each item scored 0 (normal for the patient) or 1 (increase in described behavior) Interpretation: 0–1 = low risk 2–3 = moderate risk >3 = high risk
Historical Clinical Risk Management–20 (HCR-20)/Jalil, 2019, Sada, 2016	Developed in Canada in forensic-psychiatric patients, applied to correctional/secure and general psychiatric settings; validated in Australia, the United Kingdom, Greece, the United States, Brazil, Mexica, Canada, and China	20 items (three domains of risk): -Historical (10 items; historical static variables), -Clinical (5 items: changeable concerns variables), -Risk Management (5 items: changeable future patient management considerations).	Each item has a three-level rating structure (low, moderate, or high). There are no assigned numerical values, nor are there any specified cutoff scores. The items are coded using a 3-point scale (0 = the risk factor is absent or does not apply, 1 = the risk factor may be present, 2 = the risk factor is being present).
Novaco anger scale (NAS)/Jalil, 2019	Suitable for use with both mentally disordered and normal populations, it was developed in mental health forensic patient setting in the United States. The NAS is a valid tool in a variety of populations, including as a predictor of violence in general, the clinical and forensic populations.	60-item measure comprising four subscales, each addressing an aspect of anger disposition related to: Cognition, Arousal, Behavior, and Regulation. Scales: Total—General inclination toward anger reactions, based on Cognitive, Arousal, and Behavior subscales. Cognitive—Anger justification, rumination, hostile attitude, and suspicion. Arousal—Anger intensity, duration, somatic tension, and irritability. Behavior—Impulsive reaction, verbal aggression, physical confrontation and indirect expression. Anger Regulation—Ability to regulate anger-engendering thoughts, effect self-calming, and engage in constructive behavior when provoked.	The response format for each item is a 3-point unipolar visual analog scale (1 = Never True, 2 = Sometimes True, and 3 = Always True) and scoring produces subscale scores and a total score based on 48 items across the Cognition, Arousal, and Behavior subscales. The Inconsistency Reporting Index is a check as part of the scoring method that is based on 16 selected item-pairs, to eliminate random or deliberately inconsistent reporting, which is indicated by a large number of dissimilarities between item-pair scores. Normative data are based on an age-stratified sample of 1,546 individuals, from 9 to 84 years of age. Separate norms are provided for preadolescents and adolescents (9–18) and adults (19 and older).
Psychopathy Checklist-Revised (PCL-R)/Stafford, 2003	PCL:SV is derived from PCL-R. Developed in Canadian offenders for use in mental health, correctional, and clinical settings. It has been validated in the United States, Australia, Greece, and China	20-items PCL-R: Glibness/superficial charm, grandiose sense of self-worth, Need for stimulation/proneness to boredom, Pathological lying, Conning/manipulative, Lack of remorse or guilt, Shallow affect, Callous/lack of empathy, Parasitic lifestyle, Poor behavioral controls, Promiscuous sexual behavior, Early behavior problems, Lack of realistic long-term goals, Impulsivity, Irresponsibility, Failure to accept responsibility for own actions, Many short-term marital relationships, Juvenile delinquency, Revocation of conditional release, Criminal versatility 12-item PCL:SV: Part 1: superficial, grandiose, deceitful, lacks remorse, lacks empathy, and does not accept responsibility. Part 2: impulsive, poor behavioral controls, lacks goals, irresponsible, adolescent antisocial behavior, and adult antisocial behavior.	The PCL-R total score ranges from 0 to 40; higher scores indicate a higher level of psychopathy. Each item is scored on a three-point scale from 0 to 2. 0 = doesn’t apply; 1 = not enough information/not persistent 2 = definitely applies PCL:SV: Six of the items assess the core personality traits of psychopathy (Part 1), and six items assess the socially deviant or antisocial behaviors of psychopathy (Part 2). The PCL:SV is structured to give a total score and two factor scores. Summing the scores for all the items gives the total score (ranging from 0 to 24), summing items 1–6 gives a Factor 1 score (ranging from 0 to 12), and summing items 7–12 gives a Factor 2 score (ranging from 0 to 12). A cutoff score of >18 -most likely psychopathic. 13–17 - further PCL-R assessment.
Self-Report scale (SRS)/Roaldset, 2012	Developed and validated in Norway in psychiatric (acute) settings	A measure patients’ own assessment of risk for violent behavior. Two of the questions (c, d) of the SRS address violent behavior: “What is your opinion of the risk that you will: (c) threaten to act violently toward, and (d) act violently against another person.	For each question the patient had one of seven response options: No risk (will definitely not happen), low risk (will hardly happen), moderate risk (may happen in certain situations); high risk (may happen in many situations), very high risk (almost permanent risk), don’t know the risk, and won’t answer about the risk.
Short-Term Assessment of Risk and Treatability (START)^a /Braithwaite, 2010, Jalil, 2019, Marriott, 2017, O’Shea, 2015	Developed in psychiatric forensic settings in Canada, and it is used in correctional, forensic, and community settings in Australia, Canada, the Netherlands, the United Kingdom. An alternative version was developed and validated for adolescents. Validated in the United Kingdom and Canada.	20 items that contain both risk and protective factors for the short-term assessment of risks/dynamic (changing) variables as a person’s strengths (e.g., social skills, relationships, coping, insight) and vulnerabilities (e.g., substance use, conduct, impulse control). It also allows for the recording of other unique, client-specific risks (called “case-specific” factors) based on the person’s recent level of functioning. Contains outcome measures: -Violence to others -Suicide -Self-harm -Self-neglect -Unauthorized absence (failure to return from a day pass) -Substance use -Risk of being victimized -General offending	Each item rated on a 3-point scale (absent, possibly present, present) both as a vulnerability and as a strength for that individual.
Violence Risk Screening - 10 (V-RISK-10)^b /Eriksen, 2018, Roaldset (2012)	V-RISK-10: Developed for use in forensic settings in Norway, but was then applied in acute and general psychiatry. It was validated in Norway, Iran, and China.	V-RISK-10: historical, clinical, future 10 items: previous/current violence, previous/current threats, previous/current substance abuse, previous/current major mental illness, personality disorder, lack of insight into illness/behavior, expresses suspicion, lack of empathy, future stress situations V-RISK-Y: 12 items, developed with particular emphasis on emergency settings. a new item was added, (11) Childhood adversities, previous or current. User (patient) participation has increasingly been emphasized and recommended in psychiatry, but this has been little emphasized for violence risk assessments. We found three articles concerning a patient’s own perception of violence risk, and all showed a positive correlation between user perception and later violence. In line with this, (12) Adolescents’/parents’ own assessment of risk was added as the last item in V-RISK-Y. V-RISK- EXT: biopsychological model that has 10 items + total cholesterol, high-density lipoprotein cholesterol	Each item has four options: 0 = No (does not apply); 1 = maybe/moderate; 2 = Yes (definitely present). Omit = Don’t know (not sufficiently valid). A sum score (range 0–20). For low, no more violence assessment needed; moderate, more assessment; and high, implementation of preventive measures.

Includes START:SV, an adolescent version.

V-RISK-10 modifications were included (V-RISK-Y, V-RISK-EXT).

Although four of 62 eligible studies used instruments developed and validated in ED (BRACHA, QOVPRAO, ABRAT, PANSS) (see Table S2), none of them met the quality assessment criteria to be included in the final selection. DASA is the only instrument from the high-quality studies included in the review that was also used in ED settings previously. Moreover, from the two included studies that had adolescent participants, this scale showed a good predictive validity and can be advised for application in this population (Dutch & Patil, 2019; see Tables 2 and 3).

Importantly, in our final selection of ten studies, two studies targeted adolescents and youth with median/mean age of 14 years (Dutch & Patil, 2019; Stafford & Cornell, 2003), while the remaining eight recruited adults of mean age 34 to 47 years. This may reflect the higher interest in the general population. Furthermore, six studies included a balanced representation of males and females, ranging between 50% and 55% of females, while three had a larger majority of males (21%–27% females) (Braithwaite et al., 2010; Marriott et al., 2017; O’Shea & Dickens, 2015) and only one study included 66.7% females (Dutch & Patil, 2019). We note that this representation was not linked to social identity but rather biological sex. Included studies did not report whether tool performance varied with sex, though according to the evidence, males were more inclined to commit violent or aggressive acts (especially physical) (e.g.,Gillespie et al., 2010; Nelson et al., 2024).

Included studies rarely presented information on the ethnicity or socioeconomic status of participants, which enabled the assessment of whether implicit biases exist within the tools that affect the performance of the tools based on these factors. Only one study included socioeconomic status in the prediction of aggression (Stafford & Cornell, 2003) and found it an insignificant predictor. Other studies recorded the ethnicity of the participants presented in baseline characteristics but did not include these variables in analyses (Braithwaite et al., 2010; Jalil et al., 2019; Marriott et al., 2017). This factor was not mentioned in most studies, while sometimes the findings related to it were redacted from Results and Discussion (O’Shea & Dickens, 2015). This is a significant limitation of the included research, given that ethnicity and socioeconomic status are shown to play a significant role in workplace violence in healthcare settings (Fujii et al., 2005; Sikstrom et al., 2023).

Discussion

This systematic review reports the predictive performance of risk assessment tools for violence in emergency and psychiatric settings. We identified 62 studies that reported the predictive validity of 33 risk assessment tools. Our results focus on the 10 studies of 8 tools for which there is higher quality evidence regarding their performance; all studies were conducted in psychiatric settings.

Of the eight tools for which there is highest-quality evidence, five (HCR-20, V-RISK-10/EXT, DASA-YV, BVC-CH, PCL-R) showed acceptable to outstanding predictive validity using AUC and other performance measures. All these tools were externally validated. Unfortunately, none of the studies presented sufficient information about calibration. Though none of the included studies were conducted in ED, one of the tools used in these studies —DASA—was also validated for use in ED settings. However, this study utilized a juvenile version of DASA and included adolescents.

The findings show a lack of high-quality studies, which may reflect difficulty in conducting such studies in EDs and PDs. We did not find any studies that met the highest standards of quality; most were classified as high or unclear risk. This is consistent with related reviews (Sammut et al., 2023). Furthermore, previous research often included forensic populations, which are inherently different to the general population (Dickens et al., 2020; Ogonah et al., 2023; Ramesh et al., 2018). This is the first systematic review to examine the predictive validity of violence screening tools on general populations in EDs and PDs.

Violence can be operationalized in many ways, which is reflected in the design of risk assessment tools as well as assessment of outcomes. Some discriminate between different types of violence, including verbal aggression, physical aggression against objects, physical aggression against self, and physical aggression against others (AOS; Yudofsky et al., 1986), or include scales for self-harm, suicide, substance use, victimization, absconding, and self-neglect (SOS; O’Shea & Dickens, 2015). Variability in the outcomes assessment in the included studies increased the heterogeneity of the findings. We attempted to overcome this by operationalizing our outcome as physical violence or any violence. We found that a more specific operationalization of the outcome enables better predictive validity (see Table 5). Indeed, “physical violence” showed higher predictive accuracy compared to “any violence” in START, HCR-20, V-RISK-10 in conjunction with either OAS or REFA (Jalil et al., 2019; Roaldset et al., 2012). Likewise, studies that used SOAR-S, a tool designed to measure the severity of physical assaults, showed better predictive validity than those non-specified violence (Eriksen et al., 2018; Rechenmacher et al., 2014).

Table 5.

Summary of the Review Findings and Their Implications for Practice, Policy and Research.

Finding	Implication
The methodological quality of the studies on predictive validity of risk assessment tools in ED and PD settings is not meeting high standards	Researchers should provide calibration measures when reporting the performance of the instrument
DASA and BVC, both are short-term assessment tools, showed the highest predictive validity, whereas DASA is applicable to be administered in both ED and PD settings	For short-term prediction of violence, DASA is a recommended screening tool in both settings to support consistency in monitoring the patients before and after transition across the settings
For long-term violence, V-RISK-10 showed excellent to outstanding prediction	For long-term prediction of violence, V-RISK-10 receives our evidence-based recommendation
Predictive validity of tools is sensitive to the specificity of outcome (violence) definition: severity of physical incidence and physical violence as outcomes shows higher predictive validity	Healthcare staff and researchers should carefully select the outcome measures, giving preference to more specific operationalization of violence (e.g., SOAS-R)
Instrument administration, staff skills in assessment, and communicating with patients can be factors capable of impacting the predictive validity of the screening tools	Healthcare providers need to ensure systemic implementation of the tools in ED and PD contexts to ensure the best outcomes in the reduction of patient-to-staff violence, ensuring safety to all members

We found that DASA-YV and BVC-CH showed the highest predictive validity compared to other instruments, where the former used the original outcomes, measured as a subscale of the tool, and the latter used the severity of physical assaults approach of SOAS-R (Dutch & Patil, 2019; Rechenmacher et al., 2014). Next, HCR-20 (Clinical Scale, a total score) and V-RISK-10-EXP showed excellent predictive validity for physical violence, whereas their performance was acceptable for any violence (Eriksen et al., 2018; Jalil et al., 2019; Sada et al., 2016).

The tool tested most often in included studies, START, had poorer predictive ability for any aggression, and physical aggression using SOS and AOS (Braithwaite et al., 2010; Jalil et al., 2019; Marriott et al., 2017; O’Shea & Dickens, 2015). PCL-R showed poor to acceptable predictive validity for both violence and physical violence (Jalil et al., 2019; Roaldset et al., 2012; Stafford & Cornell, 2003). The only instrument that measured unauthorized leaving (of the facility) was SOS, and its predictive ability was poor (Braithwaite et al., 2010; O’Shea & Dickens, 2015).

Our findings on predictive validity are consistent with earlier studies. A recent meta-analysis of the original BVC estimated a pooled area under the curve at 0.83 (95% CI [0.78, 0.87]) in a subset of 15 studies (Hvidhjelm et al., 2023). In line with our results, Kös et al. (2024) found that though both BVC and V-RISK-10 demonstrated excellent to outstanding prediction. The former had stronger predictive validity for short-term violence, while the latter was more predictive of long-term violence. Earlier tool-specific systematic research and meta-analysis indicated that the Clinical and HCR-20 total scores were good predictors of physical aggression, although their accuracy varied across different patient profiles (e.g., inpatient women) (O’Shea et al., 2013; Rossdale et al., 2020).

Our findings concluded that START, although being a popular tool, consistently showed poor predictive performance. In contrast, O’Shea and Dickens (2014) reported in their systematic review focused on START, its risk estimates demonstrated strong predictive validity for various aggressive outcomes. However, it can be argued that the study raises some methodological concerns, given that no standardized quality assessment tool for predictive models (e.g., PROBAST) was used and the authors did not adjust the findings by the quality of the studies.

Implications

The findings from this review have implications for research, policy, and practice (see Table 5). We recommend the DASA and BVC as short-term structured risk assessment tools with the best predictive validity based on the higher quality evidence. Of the two instruments, DASA has been used in PD and ED settings, so it is likely applicable to both contexts. However, there is a lack of quality studies to support its use in the adult population, so our recommendation applies to adolescents only. Most importantly, we recommend that the selection of a risk assessment tool should consider the following: (a) the target population (age, clinical factors), (b) the violence operationalization, and (c) the purpose of the monitoring (horizon of prediction—long term vs short term, evaluation frequency). As a measure of long-term risk of violence, we suggested V-RISK-10 as it was highly effective for less frequent assessment. Our findings also indicate that predictive performance is related to the specificity of the outcome; researchers should clearly define how violence is operationalized and provide justification for their definition.

The use of risk assessment tools by itself will likely not prevent or reduce violence; risk assessment needs to be part of an integrated strategy including staff training in terms of tool administration and clear risk management guidelines (Viljoen et al., 2018). We encourage workplaces to combine the use of the tool with appropriate staff training and education, as well as developing and implement the workforce values that support staff safety. Moreover, violence risk assessment requires staff training and climate to be implemented in combination with strategies that can effectively address and reduce potential or future harm (Florisse & Delespaul, 2020; Wand, 2012).

Limitations and Future Directions

This study has several strengths. An advantage of this study is its breadth and methodological rigor. Our searches identified a very large number of tools; however, we focused our analysis on those with the strongest supporting evidence. For quality assessment, this study adopted PROBAST, a robust RoB tool not yet used in the present application in healthcare settings. Moreover, the findings include application beyond a single setting, which can assist consistency in practice. However, a limited number of studies were of good quality based on the PROBAST tool, and none of the included studies contained calibration measures; other studies had issues with poor descriptions of research methodology, in particular study design. There are many measures and operationalizations of violence with varying specifications of violence toward the staff. This led to heterogeneity and complicated the synthesis. The small number of quality-based included studies and heterogeneity prevented us from conducting a meta-analysis.

Our study showed that studies in the field do not measure the calibration of prediction tools. The quality of information to guide decisions about the implementation of these tools in practice would be greatly improved by adding this methodological step. Therefore, future studies focused on predictive validity in the settings should adopt a higher level of methodological precision in research design, research coordination, analysis, and reporting to produce transparent and high-quality outputs. Future research should also record the ethnicity and socioeconomic status of participants and examine these factors as a predictor in the analysis, given that factors like ethnic background and socioeconomic status have been reported to cause bias in the perception of patient aggression (Fujii et al., 2005; Sikstrom et al., 2023).

Future measuring risk of violence needs to accurately capture the root of the risk without circularity (past-future behaviors) in reasoning (Mistler et al., 2024). This approach can improve the accuracy of prediction, usability, and interpretability of the assessment. In terms of future directions, recent studies have also incorporated technical developments using Artificial Intelligence methods such as machine learning. These include examining health records to predict aggression in psychiatric inpatients and have reported promising results (Tay et al., 2022). These methods may be particularly useful in settings with high patient volumes and reliable data capture systems.

Supplemental Material

sj-docx-1-tva-10.1177_15248380251358224 – Supplemental material for Predictive Validity of Violence Screening Tools in Emergency and Psychiatric Services: A Systematic Review

Supplemental material, sj-docx-1-tva-10.1177_15248380251358224 for Predictive Validity of Violence Screening Tools in Emergency and Psychiatric Services: A Systematic Review by Sviatlana Kamarova, Simon R. E. Davidson, Christopher M. Williams, Mariana Leite and Steven J. Kamper in Trauma, Violence, & Abuse

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical approval and informed consent statements

Not applicable.

ORCID iDs

Sviatlana Kamarova

Simon R. E. Davidson

Data availability statement

Not applicable.

Supplemental Material

Supplemental material for this article is available online.

Author Biographies

Sviatlana Kamarova, PhD, MSc, is an organizational and health psychologist whose research concerns Health Services and Implementation Science at the University of Sydney, embedded in Nepean Blue Mountains Local Health District. Dr Kamarova is an elected Fellow of Sydney Health Partners Implementation Academy with previous positions at the Health Economics and Data Analytics Centre, and the Future of Work Institute, Curtin University. She has a PhD in Psychology (Curtin University) and an MSc in Sport and Exercise Psychology (University of Jyväskylä).

Simon R. E. Davidson, PhD, is a physiotherapist with extensive clinical experience working in the public health system in Australia and the United Kingdom. He has a PhD in Public Health and Behavioural Science from the University of Newcastle. His research aims to improve health service provision, particularly for people with chronic diseases and persistent pain.

Christopher M. Williams, PhD, is an associate professor at University of Sydney and Research Development Manager at Mid North Coast Local Health District, Australia. He has broad experience in clinical trials, health service research, and Implementation Science. Chris has over 150 peer-reviewed publications and leads a program of research in pain, health, and lifestyle.

Mariana Leite, PhD, volunteers for PEDro, overseeing Portuguese translations and assessing clinical trials. Dr Laite has expertise in observational studies and systematic reviews and holds a PhD from the Universidade Cidade de São Paulo, Brazil. Her research is focused on musculoskeletal pain in children and adolescents.

Steven J. Kamper, PhD, is a Professor of Allied Health at the University of Sydney and Nepean Blue Mountains Local Health District, Australia. His role involves training and supporting clinicians to embed research into their clinical work and working with executives to design services that support of research activity. Steve leads a program of clinical research in chronic pain.

References

ACSQHC. (2018). Implementing the comprehensive care standard - Approaches to person-centred risk screening. Australian Commission on Safety and Quality in Health Care. https://www.safetyandquality.gov.au/publications-and-resources/resource-library/implementing-comprehensive-care-standard-approaches-person-centered-screening

Aljohani

Burkholder

Tran

Q. K.

Chen

Beisenova

Pourmand

(2021). Workplace violence in the emergency department: A systematic review and meta-analysis. Public Health, 196, 186–197. https://doi.org/10.1016/j.puhe.2021.02.009

Barzman

Mossman

Appel

Blom

T. J.

Strawn

J. R.

Ekhator

N. N.

Patel

DelBello

M. P.

Sorter

Klein

Geracioti

T. D.

(2013). The association between salivary hormone levels and children’s inpatient aggression: A pilot study. Psychiatric Quarterly, 84(4), 475–484. https://doi.org/10.1007/s11126-013-9260-8

Braithwaite

Charette

Crocker

A. G.

Reyes

(2010). The predictive validity of clinical ratings of the Short-Term Assessment of Risk and Treatability (START). The International Journal of Forensic Mental Health, 9(4), 271–281. https://dx.doi.org/10.1080/14999013.2010.534378

Cabilan

C. J.

Johnston

A. N.

(2019). Review article: Identifying occupational violence patient risk factors and risk assessment tools in the emergency department: A scoping review. Emergency Medicine Australasia : EMA, 31(5), 730–740. https://dx.doi.org/10.1111/1742-6723.13362

Cabilan

C. J.

McRae

Ganzon

Appo

Rogers

O’Sullivan

Eley

Snoswell

Johnston

(2023). Introducing a digital occupational violence risk assessment tool into an emergency department: A pilot implementation study. Journal of Emergency Nursing, 49(3), 360–370. https://dx.doi.org/10.1016/j.jen.2023.01.007

Cabilan

C. J.

McRae

Learmont

Taurima

Galbraith

Mason

Eley

Snoswell

Johnston Amy

N. B.

(2022). Validity and reliability of the novel three-item occupational violence patient risk assessment tool. Journal of Advanced Nursing, 78(4), 1176–1185. https://doi.org/10.1111/jan.15166

D’Ettorre

Caroli

Pellicani

Ceccarelli

(2020). Preliminary risk assessment of workplace violence in hospital emergency departments. Annali di igiene: Medicina preventiva e di comunita, 32(2), 99–108. https://dx.doi.org/10.7416/ai.2020.2334

D’Ettorre

Pellicani

(2017). Workplace violence toward mental healthcare workers employed in psychiatric wards. Safety and Health at Work, 8(4), 337–342. https://dx.doi.org/10.1016/j.shaw.2017.01.004

10.

De Beuf

T. L. F.

de Vogel

Broers

N. J.

de Ruiter

. (2023). Prospective field validation of the START:AV in a Dutch secure youth care sample. Assessment, 30(3), 633–650. https://doi.org/10.1177/10731911211063228

11.

Dickens

G. L.

O’Shea

L. E.

Christensen

(2020). Structured assessments for imminent aggression in mental health and correctional settings: Systematic review and meta-analysis. International Journal of Nursing Studies, 104, 103526. https://dx.doi.org/10.1016/j.ijnurstu.2020.103526

12.

Dugré

J. R.

Giguére

Percie

Sert

Potvin

Dumais

(2019). The psychometric properties of a short UPPS-P impulsive behavior scale among psychiatric patients evaluated in an emergency setting. Front Psychiatry, 10, 139. https://doi.org/10.3389/fpsyt.2019.00139

13.

Dutch

S. G.

Patil

(2019). Validating a measurement tool to predict aggressive behavior in hospitalized youth. Journal of the American Psychiatric Nurses Association, 25(5), 396–404. https://dx.doi.org/10.1177/1078390318809411

14.

Eriksen

B. M. S.

Faerden

Lockertsen

Bjorkly

Roaldset

J. O.

(2018). Predictive validity and gender differences in a biopsychosocial model of violence risk assessment in acute psychiatry. Psychiatry Research, 264, 270–280. https://doi.org/10.1016/j.psychres.2018.04.021

15.

Florisse

E. J. R.

Delespaul

P. A. E. G.

(2020). Monitoring risk assessment on an acute psychiatric ward: Effects on aggression, seclusion and nurse behaviour. PloS One, 15(10), e0240163. https://doi.org/10.1371/journal.pone.0240163

16.

Fricke

Siddique

S. M.

Douma

Ladak

Burchill

C. N.

Greysen

Mull

N. K.

(2023). Workplace violence in healthcare settings: A scoping review of guidelines and systematic reviews. Trauma Violence Abuse, 24(5), 3363–3383. https://doi.org/10.1177/15248380221126476

17.

Fujii

D. E.

Tokioka

A. B.

Lichton

A. I.

Hishinuma

(2005). Ethnic differences in prediction of violence risk with the HCR-20 among psychiatric inpatients. Psychiatric Services, 56(6), 711–716. https://doi.org/10.1176/appi.ps.56.6.711

18.

Ghosh

Twigg

Kutzer

Towell-Barnard

De Jong

Dodds

(2019). The validity and utility of violence risk assessment tools to predict patient violence in acute care settings: An integrative literature review. International Journal of Mental Health Nursing, 28(6), 1248–1267. https://doi.org/10.1111/inm.12645 (Comment in: International Journal of Mental Health Nursing 2020, 29(3), 544–545.)

19.

Gillespie

G. L.

Gates

D. M.

Miller

Howard

P. K.

(2010). Workplace violence in healthcare settings: Risk factors and protective strategies. Rehabilitation Nursing Journal, 35(5), 177–184. https://doi.org/10.1002/j.2048-7940.2010.tb00045.x

20.

Goldstein

M. K.

(2022). Care of patients with mental health issues in the emergency department: A quality improvement project. Emergency Nurse, 30(1), 12–17. https://doi.org/doi:10.7748/en.2021.e2092

21.

Grant

S. L.

Hartanto

Sivasubramaniam

Heritage

(2022). Occupational violence and aggression in urgent and critical care in rural health service settings: A systematic review of mixed studies. Health and Social Care in the Community, 30(6), e3696–e3715. https://doi.org/10.1111/hsc.14039

22.

Hamrick

van Hassel

Snyder

Stephens

(2023). Screening for behavioral health patient aggression in emergency departments to reduce workplace violence. Journal of Emergency Nursing, 49(3), 403–414. https://doi.org/10.1016/j.jen.2022.09.010

23.

Hawkins

Ghaziri

M. E.

(2022). Violence in health care: Trends and Disparities, Bureau of Labor Statistics Survey Data of Occupational Injuries and Illnesses, 2011–2017. Workplace Health & Safety, 70(3), 136–147. https://doi.org/10.1177/21650799221079045

24.

Higgins

J. P. T.

Thomas

Chandler

Cumpston

Page

M. J.

Welch

V. A.

(Eds.). (2024). Cochrane handbook for systematic reviews of interventions version 6.5 (updated February 2024). Cochrane. https://training.cochrane.org/handbook.

25.

Hilden

Habbema

J. D.

Bjerregaard

(1978). The measurement of performance in probabilistic diagnosis. II. Trustworthiness of the exact values of the diagnostic probabilities. Methods of Information in Medicine, 17(4), 227–237.

26.

Holland

K. M.

Jones

Vivolo-Kantor

A. M.

Idaikkadar

Zwald

Hoots

Yard

D’Inverno

Swedo

Chen

M. S.

Petrosky

Board

Martinez

Stone

D. M.

Law

Coletta

M. A.

Adjemian

Thomas

Puddy

R. W.

. . . Houry

(2021). Trends in US emergency department visits for mental health, overdose, and violence outcomes before and during the COVID-19 pandemic. JAMA Psychiatry, 78(4), 372–379. https://doi.org/10.1001/jamapsychiatry.2020.4402

27.

Hvidhjelm

Berring

L. L.

Whittington

Woods

Bak

Almvik

(2023). Short-term risk assessment in the long term: A scoping review and meta-analysis of the Brøset violence checklist. Journal of Psychiatric and Mental Health Nursing, 30(4), 637–648. https://doi.org/10.1111/jpm.12905

28.

Jalil

Huber

Sixsmith

Dickens

(2019). Incremental validity of patients’ self-reported anger beyond structured professional judgment tools in the prediction of inpatient aggression. The International Journal of Forensic Mental Health, 18(4), 365–375. https://doi.org/10.1080/14999013.2019.1588432

29.

Joyce

Pellatt

Ranse

Doumany

Hall

Sweeny

Keijzers

(2023). Occupational violence in a tertiary emergency department: A retrospective descriptive study. Australasian Emergency Care, 26(4), 346–351. https://doi.org/10.1016/j.auec.2023.05.003

30.

Kim

S. C.

Ideker

Todicheeney-Mannes

(2012). Usefulness of aggressive behaviour risk assessment tool for prospectively identifying violent patients in medical and surgical units. Journal of Advanced Nursing, 68(2), 349–357. https://doi.org/10.1111/j.1365-2648.2011.05744.x

31.

Kim

S. C.

Kaiser

Bulson

Hosford

Nurski

Sadat

Kalinowski

(2022). Multisite study of aggressive behavior risk assessment tool in emergency departments. Journal of the American College of Emergency Physicians Open, 3(2), e12693. https://doi.org/10.1002/emp2.12693

32.

Kös

Bräunig

Hausam

(2024). The predictive validity of the V-RISK-10 and BVC among involuntarily admitted patients. Frontiers in Psychiatry, 15, 1342445. https://doi.org/10.3389/fpsyt.2024.1342445

33.

Liu

Gan

Jiang

Dwyer

Yan

Sampson

Wang

Zhu

Chang

Yang

Chen

Song

(2019). Prevalence of workplace violence against healthcare workers: A systematic review and meta-analysis. Occupational and Environmental Medicine, 76(12), 927–937. https://doi.org/10.1136/oemed-2019-105849

34.

Lorettu

Nivoli

A. M. A.

Milia

Nivoli

(2020). Violence risk assessment in mental health. In B. Carpiniello, A. Vita, & C. Mencacci (Eds.), Violence and mental disorders, comprehensive approach to psychiatry (Vol. 1, pp. 231–252). Springer Nature. https://doi.org/10.1007/978-3-030-33188-7_13

35.

Marques

M. I.

Bessa

Santos

Carvalho

Neves

(2015). Assessment of the risk of violence in Portuguese psychiatric settings using the Brøset violence checklist. Sociology Study, 5, 360–370. https://doi.org/10.17265/2159-5526/2015.05.004

36.

Marriott

O’Shea

L. E.

Picchioni

M. M.

Dickens

G. L.

(2017). Predictive validity of the short-term assessment of risk and treatability (START) for multiple adverse outcomes: The effect of diagnosis. Psychiatry Research, 256, 435–443. https://doi.org/10.1016/j.psychres.2017.07.009

37.

McGuire

S. S.

Gazley

Majerus

A. C.

Mullan

A. F.

Clements

C. M.

(2022). Impact of the COVID-19 pandemic on workplace violence at an academic emergency department. American Journal of Emergency Medicine, 53, 285.e281–285.e285. https://doi.org/10.1016/j.ajem.2021.09.045

38.

Mento

Silvestri

M. C.

Bruno

Muscatello

M. R. A.

Cedro

Pandolfo

Zoccali

R. A.

(2020). Workplace violence against healthcare professionals: A systematic review. Aggression and Violent Behavior, 51, 101381.

39.

Miller

K. E.

Benden

M. E.

Shipp

E. M.

Pickens

A. W.

Wendel

M. L.

Pronovost

P. J.

Watts

B. V.

(2016). Predicting violent behavior in clinical settings: A case-control study of a mental health inpatient unit. Journal of Healthcare Protection Management: Publication of the International Association for Hospital Security, 32(1), 106–119. http://ovidsp.ovid.com/ovidweb.cgi?T=JS&PAGE=reference&D=med13&NEWS=N&AN=26978965

40.

Mistler

L. A.

Almvik

Fluttert

Lockertsen

Ø.

Hassiotis

Manton

Pekara

(2024). Violence risk assessment. In Hallett

Whittington

Richter

Eneje

(Eds.), Coercion and violence in mental health settings: Causes, consequences, management (pp. 125–148). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-61224-4_6

41.

Moons

K. G. M.

Wolff

R. F.

Riley

R. D.

Whiting

P. F.

Westwood

Collins

G. S.

Reitsma

J. B.

Kleijnen

Mallett

(2019). PROBAST: A tool to assess risk of bias and applicability of prediction model studies: Explanation and elaboration. Annals of Internal Medicine, 170(1), W1–W33. https://doi.org/10.7326/m18-1377

42.

Muller

M. P.

Tomlinson

Marrie

T. J.

Tang

McGeer

Low

D. E.

Detsky

A. S.

Gold

W. L.

(2005). Can routine laboratory tests discriminate between severe acute respiratory syndrome and other causes of community-acquired pneumonia? Clinical Infectious Diseases, 40(8), 1079–1086. https://doi.org/10.1086/428577

43.

Munich

Dennett

Swainson

Greenshaw

A. J.

Hayward

(2021). Impact of pandemics/epidemics on emergency department utilization for mental health and substance use: A rapid review. Frontiers in Psychiatry, 12. https://doi.org/10.3389/fpsyt.2021.615000

44.

Nelson

Ayaz

Baumann

A. L.

Dozois

(2024). A gender-based review of workplace violence amongst the global health workforce: A scoping review of the literature. PLoS Global Public Health, 4(7), e0003336. https://doi.org/10.1371/journal.pgph.0003336

45.

Nikathil

Olaussen

Symons

Gocentas

O’Reilly

Mitra

(2018). Increasing workplace violence in an Australian adult emergency department. EMA - Emergency Medicine Australasia, 30(2), 181–186. https://doi.org/10.1111/1742-6723.12872

46.

O’Brien

C. J.

van Zundert

A. A. J.

Barach

P. R.

(2024). The growing burden of workplace violence against healthcare workers: Trends in prevalence, risk factors, consequences, and prevention—A narrative review. eClinicalMedicine, 72, 102641. https://doi.org/10.1016/j.eclinm.2024.102641

47.

O’Shea

L. E.

Dickens

G. L.

(2014). Short-term assessment of risk and treatability (START): Systematic review and meta-analysis. Psychological Assessment, 26(3), 990–1002. https://doi.org/10.1037/a0036794

48.

O’Shea

L. E.

Dickens

G. L.

(2015). Predictive validity of the START for unauthorised leave and substance abuse in a secure mental health setting: A pseudo-prospective cohort study. International Journal of Nursing Studies, 52(5), 970–979. https://doi.org/10.1016/j.ijnurstu.2015.02.007

49.

O’Shea

L. E.

Mitchell

A. E.

Picchioni

M. M.

Dickens

G. L.

(2013). Moderators of the predictive efficacy of the historical, clinical and risk management-20 for aggression in psychiatric facilities: Systematic review and meta-analysis. Aggression and Violent Behavior, 18(2), 255–270. https://doi.org/10.1016/j.avb.2012.11.016

50.

Odes

Lee

S.-J.

Hong

Jun

(2023). The effect of COVID-19 on workplace violence in California’s hospitals: An interrupted time series analysis. Journal of Advanced Nursing, 79(6), 2337–2347. https://doi.org/10.1111/jan.15588

51.

Ogonah

M. G. T.

Seyedsalehi

Whiting

Fazel

(2023). Violence risk assessment instruments in forensic psychiatric populations: A systematic review and meta-analysis. Lancet Psychiatry, 10(10), 780–789. https://doi.org/10.1016/s2215-0366(23)00256-0

52.

Page

M. J.

McKenzie

J. E.

Bossuyt

P. M.

Boutron

Hoffmann

T. C.

Mulrow

C. D.

Shamseer

Tetzlaff

J. M.

Akl

E. A.

Brennan

S. E.

(2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. International Journal of Surgery, 88, 105906.

53.

Pujol

J. F.

Gras

M. E.

Calvó-Perxas

Vilalta-Franch

(2020). Scale for the evaluation of risk of aggressiveness in psychiatric intensive care units. Actas Espanolas de Psiquiatria, 48(1), 8–18.

54.

Ramesh

Igoumenou

Vazquez Montes

Fazel

(2018). Use of risk assessment instruments to predict violence in forensic psychiatric hospitals: A systematic review and meta-analysis. European Psychiatry: The Journal of the Association of European Psychiatrists, 52, 47–53. https://doi.org/10.1016/j.eurpsy.2018.02.007

55.

Ramzi

Z. S.

Fatah

P. W.

Dalvandi

(2022). Prevalence of workplace violence against healthcare workers during the COVID-19 pandemic: A systematic review and meta-analysis. Frontiers in Psychology, 13, 896156. https://doi.org/10.3389/fpsyg.2022.896156

56.

Rechenmacher

Müller

Abderhalden

Schulc

(2014). The diagnostic efficiency of the extended German Brøset violence checklist to assess the risk of violence. The Journal of Nursing Measurement, 22(2), 201–212. https://doi.org/10.1891/1061-3749.22.2.201

57.

Roaldset

J. O.

Gustavsen

C. C.

Lockertsen

Ø.

Landheim

Bjørkly

S. K.

(2023). Validation of a violence risk screening for youth in psychiatric inpatient care—A pilot study of V-RISK-Y. Frontiers in Psychiatry, 14. https://doi.org/10.3389/fpsyt.2023.1210871

58.

Roaldset

J. O.

Hartvig

Linaker

O. M.

Bjorkly

(2012). A multifaceted model for risk assessment of violent behaviour in acutely admitted psychiatric patients. Psychiatry Research, 200(2–3), 773–778. https://doi.org/10.1016/j.psychres.2012.04.038

59.

Rossdale

S.-V.

Tully

R. J.

Egan

(2020). The HCR-20 for predicting violence in adult females: A meta-analysis. Journal of Forensic Psychology Research and Practice, 20(1), 15–52. https://doi.org/10.1080/24732850.2019.1681875

60.

Sada

Robles-Garcia

Martinez-Lopez

Hernandez-Ramirez

Tovilla-Zarate

C.-A.

Lopez-Munguia

Suarez-Alvarez

Ayala

Fresan

(2016). Assessing the reliability, predictive and construct validity of historical, clinical and risk management-20 (HCR-20) in Mexican psychiatric inpatients. Nordic Journal of Psychiatry, 70(6), 456–461. https://doi.org/10.3109/08039488.2016.1159330

61.

Sammut

Hallett

Lees-Deutsch

Dickens

G. L.

(2023). A systematic review of violence risk assessment tools currently used in emergency care settings. Journal of Emergency Nursing, 49(3), 371–386.e375. https://doi.org/10.1016/j.jen.2022.11.006

62.

Sikstrom

Maslej

M. M.

Findlay

Strudwick

Hui

Zaheer

Hill

S. L.

Buchman

D. Z.

(2023). Predictive care: A protocol for a computational ethnographic approach to building fair models of inpatient violence in emergency psychiatry. BMJ Open, 13(4), e069255. https://doi.org/10.1136/bmjopen-2022-069255

63.

Stafford

Cornell

D. G.

(2003). Psychopathy scores predict adolescent inpatient aggression. Assessment, 10(1), 102–112. https://doi.org/10.1177/1073191102250341

64.

Steyerberg

E. W.

Vickers

A. J.

Cook

N. R.

Gerds

Gonen

Obuchowski

Pencina

M. J.

Kattan

M. W.

(2010). Assessing the performance of prediction models: A framework for traditional and novel measures. Epidemiology, 21(1), 128–138.

65.

Stowell

K. R.

Hughes

N. P.

Rozel

J. S.

(2016). Violence in the emergency department. Psychiatric Clinics of North America, 39(4), 557–566. https://doi.org/10.1016/j.psc.2016.07.003

66.

Tay

J. L.

Sim

(2022). Effectiveness of artificial intelligence methods in personalized aggression risk prediction within inpatient psychiatric treatment settings—A systematic review. Journal of Personalized Medicine, 12(9), 1470. https://doi.org/10.3390/jpm12091470

67.

Vento

Cainelli

Vallone

(2020). Violence against healthcare workers: A worldwide phenomenon with serious consequences. Frontiers in Public Health, 8, Article 570459. https://doi.org/10.3389/fpubh.2020.570459

68.

Viljoen

J. L.

Cochrane

D. M.

Jonnson

M. R.

(2018). Do risk assessment tools help manage and reduce risk of violence and reoffending? A systematic review. Law and Human Behavior, 42(3), 181–214. https://doi.org/10.1037/lhb0000280

69.

Wand

(2012). Investigating the evidence for the effectiveness of risk assessment in mental health care. Issues in Mental Health Nursing, 33(1), 2–7. https://doi.org/10.3109/01612840.2011.616984

70.

Wolff

R. F.

Moons

K. G. M.

Riley

R. D.

Whiting

P. F.

Westwood

Collins

G. S.

Reitsma

J. B.

Kleijnen

Mallett

(2019). PROBAST: A tool to assess the risk of bias and applicability of prediction model studies. Annals of Internal Medicine, 170(1), 51–58. https://doi.org/10.7326/m18-1376

71.

Yudofsky

S. C.

Silver

J. M.

Jackson

Endicott

Williams

(1986). The Overt Aggression Scale for the objective rating of verbal and physical aggression. The American Journal of Psychiatry, 143(1), 35–39. https://doi.org/10.1176/ajp.143.1.35

72.

Yuniati

Putra

K. R.

Widasmara

(2020). Brøset Violence Checklist (BVC) versus Dynamic Appraisal of Situational Aggression-Inpatient Version (DASA-IV): Their sensitivity, specificity and accuracy as a predictor of violence in 24 hours among schizophrenia patients. MEDISAINS, 18. http://doi.org/10.30595/medisains.v18i2.7454

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.19 MB