Abstract
See Review by Large et al., 2011, 45(8): 619–628
In their meta-analysis of predictors of suicide, Large et al. (2011) state:
‘using the values for sensitivity (proportion of suicides which are correctly categorized = 40%) and specificity (the proportion of non-suicides which are correctly categorized = 87%) derived by meta-analysis of high risk models for suicide after discharge and an estimate of 1% for the incidence of suicide within a year of discharge, we calculated that only 3% of patients categorized as high risk will commit suicide in the year after discharge.’
One could just as easily state for the analogous problem of predicting violence:
using the values for sensitivity (proportion of violent patients who are correctly categorized = 73%) and specificity (the proportion of non-violent patients who are correctly categorized = 63%) – derived, for example, from an analysis of the properties of a well-established violence risk assessment instrument (VRAG) (Buchanan, 2008) – and an estimate of 1% for the incidence of serious violence within a year of discharge, we can calculate that 2% of patients categorized as high risk will commit an act of significant violence in the year after discharge. Even if we were to consider less serious violence, occurring in say, 10% of the patient population, then 18% of the high-risk group will commit such an act.
In fact, using probably the best ever result for a risk instrument under research conditions (Monahan et al., 2001) – sensitivity = 75% and specificity = 75%, i.e. choosing a cut-off where the number of true positives and true negatives would be maximized – the percentage of high-risk patients who will commit an act of serious violence would be 3% for an incidence of violence of 1% in that population of patients. It is also important to note that the area under the curve (AUC) of the receiver operating characteristic curve in this study was 0.8, one of the highest ever achieved in this field; but it is just as noteworthy that even with a hugely statistically significant AUC, the value of a risk assessment instrument in predicting the outcome for an individual patient is extremely limited (Szmukler et al., 2011). [And, it might be noted that, as would be expected, on replication of the MacArthur study, the accuracy of the risk assessment instrument was reduced (Monahan et al., 2005).]
Rare events, such as suicide or serious violence – no matter how tragic they are or how much our society wishes us to prevent them – are impossible to predict with a degree of accuracy that is clinically meaningful.