Abstract
The Offender Group Reconviction Scale 2 (OGRS-2) excludes social and clinical factors that are known to be associated with offending risk in people with mental health problems, but shows similar predictive validity to structured professional judgment tools that include them. The aim of this study was to determine whether the predictive validity of OGRS-2 would be enhanced by the addition of social and clinical factors. A retrospective case note analysis was conducted on a national cohort of patients who were discharged from 35 UK medium secure between 01 April 1997 and 31 March 1998. Social factors, clinical factors and OGRS-2 probability scores were combined in a regression model using violent and any reconviction as outcomes across two-, four- and six year follow up periods. The OGRS-2 showed strong predictive validity for both any and violent reoffending, addition of social and clinical factors yielded no significant improvements. OGRS-2 may have reached an accuracy ceiling in predicting re-offending but remains effective for use alone to estimate risk of further convictions following discharge. OGRS-2 retains the well-established limitations of actuarial risk tools but can potentially aid in decision making around supervision and monitoring. OGRS-2 has potential to control for reconviction risk in reoffending outcome studies of medium secure service users.
Introduction
The relative merits of actuarial measures versus Structured Professional Judgment tools (SPJs) for estimating offending risk in psychiatric populations has been hotly contested and some have advocated complete abandonment of clinical judgment from risk assessment (Quinsey et al., 1998). Some evidence suggests that SPJs are ‘out-performed’ by more objective, actuarial methods (Gray et al., 2004; Singh et al., 2011), one such tool is the Offender Group Reconviction Scale 2 (OGRS-2: Copas & Marshall, 1998; Taylor, 1999), but the evidence is equivocal. OGRS-2 has been shown to be a valid prediction tool for both violent and any offending following treatment in secure psychiatric care (Snowden et al., 2007). There is however contradictory evidence that SPJs can out-perform actuarial methods and researchers have tended to conclude that an integrative approach using both types of measures in practice is optimal (Snowden et al., 2007). Despite this, the majority of UK secure services tend to rely on SPJs only (Khiroya et al., 2009; Lamont & Brunero, 2009; Tully, 2017). The OGRS-2 has gained no traction in clinical practice (Snowden et al., 2007) and has perhaps failed to influence psychiatric practice due to constrained face validity in terms of the absence of social and clinical factors that feature strongly in individual risk formulations and risk scenario planning in routine practice (Joint Commissioning Panel for Mental Health, 2013). Like most other actuarial measures, OGRS-2 is insensitive to changes in mental health related risk and the effects on risk of therapeutic engagement which are important considerations in practice.
It has been suggested that all measures of risk have reached a ceiling in predictive validity, as the literature consistently shows only moderate predictive validity for reconviction (Coid et al., 2011; Kennedy et al., 2019). Yet, there has been minimal research focused on the individual factors embedded in risk assessment measures to determine the scope for improvement (Coid et al., 2009). Evaluations of risk assessment measures have mainly been directed toward risk prediction of reoffending and have overlooked other elements of risk management in forensic practice. It would be meaningful, for example, to determine how well the HCR-20 maps onto other areas of risk assessment; including whether the presence of risk factors outlined in the HCR-20 significantly match those included in future risk scenarios drawn from detailed chain analysis of past offending behavior and observed risk behaviors of forensic service users. This however, will have to be a topic of future investigation.
The current analysis was designed to move away from simply comparing actuarial measures against SPJ’s (e.g., Singh et al., 2011) and determine instead whether criminogenic, social and clinical factors can act additively to improve estimation of reconviction risk in secure psychiatric service users. We conducted a retrospective case-note analysis from a national cohort of patients discharged from medium security to determine whether the predictive validity of the OGRS-2 could be improved with the addition of social (e.g. child adversity and drug misuse) and clinical factors (e.g. diagnosis, previous hospital admission, and medication) known to be associated with reoffending (Krona et al., 2017; Wolf et al., 2018).
The primary aim of the current study was to determine whether the predictive and face validity of OGRS-2 might be improved by the addition of more meaningful clinical and social characteristics; this may make actuarial measures (like OGRS-2) more acceptable and useful in practice. A second aim was to determine whether the OGSR-2 had adequate predictive validity for use in a second, sister paper (using the same data set), as a benchmark for evaluating the effects of medium secure care on subsequent levels of reconviction.
Method
Participants
Patients were discharged from 35 NHS and independent sector medium secure units across England and Wales between 01 April 1997 and 31 March 1998 (Maden et al., 2004). Clinical and forensic information were collected for 959 patients (836 male) by the original authors. The mean age was 25.5 at the time of admission and the average length of stay in medium security was 424 days (s.d = 495, range 7–3501). A total of 117 discharged patients had no recorded index offense, of these 90 were admitted under Section 3 of the Mental Health Act, 14 informally and 9 under Section 2, these admissions were therefore not triggered by a specific index offense, but most likely by indications of increased risk behavior. One hundred and fifty-four had no official criminal history prior to their index offense.
A total of 63.3% of discharged patients were of white ethnicity, 13.2% were black Caribbean, 4.7% were black African, 9.7% were of ‘other’ or mixed ethnicity, and 8.4% were of unknown ethnicity or had missing ethnicity data. The primary diagnoses of the discharged patients were collected based on healthcare records from the medium secure service. This ranged from a psychotic related diagnosis (63.8%), personality disorder (9.8%), and ‘other’ diagnosis (16.0%; mood disorder, substance abuse, intellectual difficulties). Approximately 1.6% of discharged patients had no diagnosis and 8.8% were unknown.
Measures
The Offender Group Reconviction Scale 2 (Taylor, 1999) is a modified version of the original OGRS which was validated with a probation sample (Copas & Marshall, 1998). OGRS-2 includes an additional four criminogenic variables, along with the original six criminogenic and demographic variables from the OGRS (see Table 1 for description of each OGRS-2 variable). The variables are combined using a statistical algorithm to calculate a total score that is translated into the probability of similarly profiled individuals being reconvicted within two years following release by comparison with the reconviction rates of the original validation sample. The OGRS-2 is unable to calculate scores for individuals with no or missing criminogenic history, and therefore, could not be applied to 247 discharged patients from this cohort.
OGRS-2 variables to calculate reconviction probability scores.
*Additional criminogenic items.
Design
The study was a retrospective case-note analysis that relied upon an existing database
Procedure
The original study was granted ethical approval by the London Multi-Centre Research Ethics Committee and did not require patient consent due to difficulties locating individuals following discharge (Maden et al., 2004). Clinicians from the medium secure units across England and Wales provided data related to the clinical and social history of each discharged patient. This information was sourced from healthcare records. Criminogenic history and reconviction data within two years following discharge were obtained from the Offenders Index managed at the time by the UK Government Home Office. Reconviction data to extend to a follow up period of six years was obtained from the Offenders Index during a subsequent study using the original cohort of discharged patients (Maden et al., 2004). Incomplete and missing offense data from the Offenders Index was, as far as possible, obtained from health care records held by the medium secure units. Different researchers calculated OGRS-2 scores and collected reconviction data. For this re-analysis of the raw, anonymised data from the original study, ethical approval was granted under the Cardiff Metropolitan University School of Sport and Health Sciences research ethics framework. The dataset was scrutinized and determined to be GDPR compliant.
Results
Although clinical and forensic information was collected for 959 patients by the original authors (Maden et al., 2004), only 616 discharged patients (543 male) had all available data for complete inclusion in the full model (OGRS-2 plus all social and clinical variables, see Figure 1).
Reduction in sample size based on each predictor variable added to adjusted OGRS-2.
Missing data was imputed via multiple imputation using chained equations with five imputations (Rubin, 2004) with all social, clinical and OGRS-2 predictor variables included in the model, along with the outcome variable (any reconviction in two years). Those added to the OGRS-2 variable were identified by pooling across imputations using binary logistic regression.
At follow up, 117 patients (19.00%) were reconvicted within two years. This rose to 169 patients (27.43%) after 4 years, and to 201 patients (32.63%) following six years post-discharge. Of those who were reconvicted, 53 were for violent offenses in the first two years, which rose to 101 after four years and 124 after six years.
Social and clinical factors associated with reconviction
We used binary logistic regression to evaluate which clinical and social predictor variables were significantly associated with the any reconviction outcome within two years following discharge. Those shown to be associated with the outcome variable were used to adjust OGRS-2.
Table 2 shows the hazard ratio of each predictor variable for any reconviction. A history of child abuse and drug related problems increased risk of any reconviction during the initial two years following discharge, whereas both having a history of self-harm, and medication on discharge were associated with reduced risk for any reconviction during the same period. Diagnosis and number of previous admissions were unrelated to any reconviction.
Associations between risk factors and all reconvictions after two years following discharge.
Adjusted alpha level =0.05.
Improvement in predictive validity of adjusted OGRS-2
The predictor variables shown to be significantly associated with any reconviction (Table 2) were added to create an adjusted OGRS-2 (bottom panel Table 3). ROC analysis was used to determine whether this manipulation enhanced predictive accuracy relative to OGRS-2 alone. An Area Under the Curve (AUC) above 0.5 shows a predictive power better than chance (Mossman, 1994).
Predictive power of models to predict any and violent reconvictions up to six years following discharge.
*Significant clinical and social variables from Table 2.
Table 3 shows that the OGRS-2 and adjusted OGRS-2, both showed significant predictive validity for any and violent reconviction across two-, four and six years following discharge. The Confidence Intervals (CI) show significant overlap between OGRS-2 alone and the adjusted model, addition of social and clinical variables therefore did not improve the predictive validity of OGRS-2.
Social and clinical factors associated with OGRS-2 probability scores
The overlap between the OGRS-2 and adjusted OGRS-2 may be a reflection that the OGRS-2 is already acting as a proxy-measure of underlying social and clinical factors (Copas & Marshall, 1998). Therefore, the potential impact of social and clinical factors may have been obscured by multi-collinearity. To our knowledge, there is no literature mapping specific social factors to OGRS-2 outcomes, and therefore a multiple regression was conducted to identify whether the clinical and social factors were already associated with OGRS-2 values. Table 4 shows that 2 social factors and only 1 clinical factor were significantly associated with OGRS-2 scores following Holm Bonferroni corrections (Holm, 1979).
Association between social and clinical factors with OGRS-2 probability scores.
Alpha was corrected using the Holm-Bonferroni method20.
Discussion
In this paper, we set out to determine whether there is scope for improvement in the OGRS-2 for predicting reconvictions in the forensic mental health population. Clinical and social factors shown previously to be associated with recidivism were added to this actuarial measure, but yielded only a nonsignificant improvement in the predictive validity of the OGRS-2. There may be a ceiling on the predictive validity of OGRS-2 so that it cannot be improved further, simply by expanding the inclusion of social or clinical variables (Coid et al., 2011; Gray et al., 2004).
Consistent with previous reports, the OGRS-2 showed favorable predictive validity for subsequent criminal behavior in a forensic mental health population, and it has been argued that this tool may be sufficient to consider alone when predicting the likelihood of future offending (Gray et al., 2004; Snowden et al., 2007). That being so, we do not suggest the OGRS-2 should replace SPJs already used in clinical practice. Instead, it simply shows that the OGRS-2 (or subsequent versions of the OGRS model) may be a quick and easy adjunctive measure to aid decision-making related to risk management planning following discharge from services. Whereas SPJs, such as the HCR-20 V3 allow individualized formulation of the imminence and severity of risk (Bonta & Andrews, 2007; Douglas et al., 2013), it is also prudent to include an indication of where the individual is located in relation to the population of offenders at risk of reconviction, and therefore how intensively resourced their risk management plan may need to be. Therefore, we argue that both types of risk measures have their place in forensic mental health services.
Copas and Marshall (1998) conceded that the OGRS criminogenic variables are likely to function as a proxy for underlying social factors. This may explain the overlap between the OGRS-2 and the adjusted OGRS-2, as drug related problems, history of self-harm, and history of child abuse were significantly associated with OGRS-2 probability scores in this national cohort of mentally disordered offenders. To our knowledge, the exact social factors that map onto each specific variable of the OGRS-2 are yet to be explored. Recent research shows that high numbers of adverse childhood experiences (ACEs) are strongly associated with both violence and incarceration (Krona et al., 2017). How ACEs relate to OGRS-2 scores in the prison population and secure psychiatric service users might therefore be a useful avenue for further research and might enhance the perceived value of actuarial tools in forensic mental health by increasing their face validity and reducing their reliance on proxies informing risk assessment.
Structured professional judgment tools (SPJs) have some advantages over actuarial measures for routine application in practice, principal amongst these is the scope for dynamic appraisal of risk as these tools encourage clinicians to consider the service user’s current presentation and context along with their historical risk indicators. In addition, SPJs can provide an opportunity for dialogue with service users about their current and future risk. In contrast, the OGRS-2 relies exclusively on historical factors such as the rate at which convictions have been acquired, so it is difficult for service users or clinicians to argue that there has been any appreciable reduction in risk, the only way in which a risk score can go down in OGRS-2 is as service users get older, their rate of reconviction (in the absence of fresh convictions) will decrease. The OGRS-2 does have one substantial advantage over other risk assessment tools however as it estimates the probability of a reconviction outcome as a percentage. This feature means that the scores of service users can be aggregated and OGRS-2 could be used to estimate the effects of services on reconviction in numerical terms, this will be the objective of the sister paper to this one.
Limitations
A number of limitations are present in this study. Firstly, the study relied on the Offenders Index to collect reconviction data. This source was likely to underestimate true reoffending rates, as many offenses fail to reach the threshold that warrants conviction. Clinical practice may be more fully informed by more complete indices of post-discharge criminal behavior. Ex-patients also tend to be supervised and monitored by mental health services following discharge which might result in diversion away from the criminal justice system when reoffending occurs (Gray et al., 2004). Collecting reoffending data from alternative sources, including healthcare records and ex-patients following discharge would help overcome this limitation.
Secondly, the use of ROC curves to estimate predictive validity has been criticized due to (1) the sensitivity/specificity play off and (2) the susceptibility to changes in base rates (Szmukler et al., 2012). There is concern that these issues lead to reduced predictive validity of instruments; especially if base rates change due to risk management strategies becoming more effective, or if economic and social factors influence the prevalence of offending. This suggests that the OGRS-2 may perform differently today than it did for this sample who were discharged from medium secure services in the late 1990s, when services were perhaps less well developed. This possibility prompts the potential usefulness of longitudinal analysis of AUCs to indirectly evaluate the influence of service changes on patient reconvictions over time, particularly the influence of diversion schemes. Alongside this, the OGRS-2 faces another challenge of neglecting a large sub-sample of the forensic mental health population. A number of individuals admitted to forensic mental health services have no criminogenic history, and therefore this measure would not be able to predict and aid decision making related to their risk following discharge.
OGRS-2 was perhaps not the best currently available risk assessment tool because OGRS has been updated with the subsequent release of OGRS-3 (Howard et al., 2009). This newer tool was developed to improve predictive validity and it could be argued it should have been preferred in the current analysis. OGRS-2 was used in this study because the sample used to validate OGRS-2 was released from prison contemporaneously with the cohort of service users reported here. Both the current service user cohort and the original OGRS validation sample would therefore have been released into the same context with respect to the UK criminal justice system and would have been exposed to risk for reconviction under optimally similar conditions. The OGRS-3 could also be preferred over OGRS-2 because it is designed to predict re-offending and not the highly time-lagged measure of reconviction, which may suggest superior temporal acuity. The extended follow up period in the current observations would however have gone some way to offset this sensitivity weakness of the OGRS-2.
The current field study relied on clinical measures that could not be validated in terms of their reliability. This is an inherent limitation of retrospective studies that rely on data collected from clinical settings, yet it may not be assumed that clinical information would have introduced any systematic bias in our analysis. Our findings reflected what was observed in routine clinical practice in the UK, it could be argued that using reliability analysis for diagnoses (for example) would have diminished the ecological validity of our observations.
Finally, simply including binary variables for our measures minimized detail of how they linked with recidivism. It would be more meaningful in the future to collect data related to the frequency, chronicity, severity and nature of the binary measures we employed, which applies to physical child abuse and drug related problems. This may highlight whether a specific sub-sample of these variables are influencing the overall predictive power.
Conclusion
We show that the OGRS-2 is a highly predictive measure of reconviction and does not benefit from the addition of clinical and/or social factors. This supports the notion that the OGRS-2 is a potentially useful adjunct in forensic mental health services for predicting the likelihood of reconviction and has potential to aid decision making related to discharge, including discharge pathways, aftercare, and supervision. OGRS-2 can generate valid estimates of expected rates of reconviction that can be contrasted with those observed amongst people discharged by secure services. Future research in this area should use more contemporary iterations of the OGRS model.
