Sage Journals: Discover world-class research

Abstract

There is a need to improve a) identification and monitoring of people with mental illness on probation and b) understanding of the impact of interventions on mental health outcomes for the probation population. If data were routinely collected using validated screening tools and shared between agencies, this could inform practice and commissioning decisions, and ultimately it could improve health outcomes for people under supervision. The literature was reviewed to identify brief screening tools and outcome measures that have been used in prevalence and outcome studies conducted with adults on probation in Europe. This paper shares findings from the UK-based studies in which 20 brief screening tools and measures were identified. Recommendations are made based on this literature regarding suitable tools for use in probation to routinely identify a need for contact with mental health and/or substance misuse services and to measure change in mental health outcomes.

Keywords

mental health mental illness probation criminal justice system screening assessment outcome measures

Introduction

Contact with a probation service can provide an opportunity for practitioners in the health and justice field to monitor and potentially help to improve the mental health of people that are often marginalised and are unlikely to access support until they are at crisis point. However, in the UK, a recent joint thematic inspection has highlighted numerous difficulties in relation to supporting people with mental health needs and disorders in the criminal justice system. These include failure to identify people with mental health needs throughout the criminal justice pathway, a need for a memorandum of understanding to improve data-sharing between agencies, a shortage of mental health services in England and Wales and long waiting lists for the services that are available (HM Inspectorate of Probation et al., 2021). Research into the prevalence of mental health needs and the efficacy of mental health interventions within the probation population and improvements to routine data collection around mental health needs in probation practice could begin to address these difficulties including through routine data and research findings being used to inform commissioning and service delivery decisions (Public Health England, 2020), ensuring that practice is based on the latest evidence-base and demonstrating the need for additional investment in mental health provision.

People on probation with mental illness are likely to also experience drug and/or alcohol misuse, unstable accommodation or homelessness, difficult family relationships, low levels of literacy and health literacy and a lack of access to and/or understanding of technology (Power, 2020; Revolving Doors Agency, 2017; Sirdifield et al., 2019). This complexity of need together with challenges like poor past experiences of service access and a lack of GP registration can form barriers to service access. Continuity of care as people progress through the criminal justice pathway is also problematic (HM Inspectorate of Probatiodn et al., 2021; HMIP, 2016). If probation staff understand an individual’s mental health needs and how they may relate to offending behaviour, they can affect positive change by signposting to relevant provision where it is available, including via Community Sentence Treatment Requirements if appropriate, and supporting continuity of care for people released from prison (HMPPS and NPS, 2019: p. 6).

Conversely, a lack of understanding of the mental health needs of those on probation, barriers to access and a lack of services that meet needs can result in poor health outcomes for this population and avoidable use of crisis care (Brooker et al., 2009; Public Health England, 2020; Revolving Doors Agency, 2017).

However, there are numerous ethical and practical challenges in improving our understanding of this population’s needs. ‘Gold standard’ screening tools for mental illness can be time-consuming to complete, and screening may need to be undertaken by a qualified clinician. Such resources are not always available to probation staff or researchers, so various methodological approaches have been employed to date in practice, research studies and evaluations.

Currently, data on mental health needs collected by probation staff in England and Wales are often based on simple self-report rather than validated screening tools. More work is required to ensure that data are collected consistently and in a research-informed way that is helpful to those commissioning and providing services (HM Inspectorate of Probation et al., 2021; Public Health England, 2020). By ensuring that the tools used to measure and assess mental health are valid and reliable, probation services can have greater confidence that the support they offer is effective and supported by accurate data.

In this paper, we report findings from a literature review that aimed to identify brief screening tools and outcome measures that have been used with adult probation populations to establish the prevalence of mental illness or symptoms of mental illness, or to investigate the effectiveness of interventions aimed at improving mental health outcomes for this population. The search encompassed studies conducted in European countries, but due to the volume of the literature and variations in the role of probation services in relation to mental health, our focus in this paper is primarily on UK studies. Those reported from other parts of Europe are briefly summarised for information. We report key findings from the UK studies to show how the measures have been used in practice and research and consider the meaning of scores on these measures and any information reported on the benefits or challenges of their use with probation populations for health and justice practitioners and for researchers.

Materials and methods

Although this is not a full systematic review, our approach was guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. We searched PsycINFO, MEDLINE, IBSS, CINAHL, AMED, ASSIS and Scopus (November 2021). To review as many potentially useful tools and measures as possible, the scope of the search was broad. No limit was set on the date of the included studies. The search strategy for PsycINFO is in Supplement Appendix 1 and was translated for the remaining databases. We also drew upon our existing knowledge of the field and a recent systematic review that aimed to identify the literature about approaches to improving mental health outcomes for adults on probation and/or the health needs of this group (Brooker et al., 2020).

To be included in the review, studies had to have used a brief screening tool or outcome measure to assess a common or serious mental illness/disorder (including substance use disorders and personality disorders) and to have applied the screening tool or outcome measure to adults on the caseload of probation services (including people on parole). Additionally, studies had to be published in English and conducted in a European country. Studies were excluded if they involved participants under the age of 18, were focussed on other settings/elements of the criminal justice system (e.g. prison), were not empirical studies or did not directly administer mental health screening tools.

Upon completing the search, the studies were further filtered by location. This paper presents findings from UK-based studies. Details of studies reported from other parts of Europe are in Supplement Appendix 2.

A list of brief screening tools and outcome measures was then compiled, and data were extracted from the identified studies together with associated publications around the validity of the measures into a bespoke data extraction form.

Results

A total of 15 UK-based papers or reports were included in the review, which employed 20 different brief screening tools and measures in total (see Figure 1 and Tables 1 and 2). This included four measures of substance misuse (AUDIT, CAGE, DAST and UNCOPE), 9 measures of mental state/symptom severity (GHQ-12, SF-36, geriatric depression scale [short form], CORE-34/OM, GAD-7, K6, K10 and PHQ-9), two tools for identifying ‘likely cases’ of mental illness (MINI and SAPAS) and individual measures of needs (CANFOR-S), service use (CSSRI-EU), self-efficacy (GSES), problem-solving ability (SPSI-R) and functional impairment (WSAS).

Figure 1.

PRISMA chart.

Table 1.

Measurement in mental health and probation studies.

Measure	What is measured	Validating paper	Validation with a mental health/alcohol/drug disorder	Benchmark scoring
AUDIT (10 items, short form AUDIT-C, 3 items)	Identifies people likely to have severe alcohol disorders	AUDIT: (Saunders et al., 1993)	Effective brief screen for heavy drinkers selected from general health clinics	AUDIT
				0 to 7: Low risk
				8 to 15: Increasing risk
				16 to 19: Higher risk
				20+: Possible dependence
		AUDIT-C: (Bush et al., 1998)		AUDIT-C
				A total of 5 or more is a positive screen
				0 to 4: Low risk
				5 to 7: Increasing risk
				8 to 10: Higher risk
				11 to 12: Possible dependence
CAGE (4 items)	Screens for alcohol misuse	(Chan et al., 1994)	Tested on patients currently receiving treatment for alcoholism, primary care outpatients and a general population group. When compared to the DSM-III-R criteria, the measure had good a specificity and sensitivity	A score of 2+ is considered to be clinically significant, that is, suggests problem drinking
CANFOR-S (25 domains)	The extent to which health and social needs are met	(Phelan et al., 1995)	Tested with people experiencing both serious mental illness and affective disorders	N/A
CORE – 34 (or OM, 34 items) (short form CORE-10, 10 items)	Measures psychological distress	CORE-OM: (Barkham et al., 2005)	Discriminant validity was high – differentiating between primary and secondary care clients	A higher score on the CORE measures, domains or individual items indicates a higher level of distress or symptom severity
		CORE-10: (Barkham et al., 2013)		Cut points of 21, 34, 51, 68 and 85+ on the CORE-34 can be interpreted as low, mild, moderate, moderate to severe and severe psychological distress, respectively (Callender, 2021)
		CORE-10: (Barkham et al., 2013)		The clinical cutoff for the CORE-OM is 1 (using the 0–4 scale) or 10 (using the 10–40 scale)
CSSRI-EU (around 20 min to complete)	A measure of mental health service use	(Chisholm et al., 2000)	Provides a standardised yet adaptable method for collating service receipt and associated data	N/A
DAST-20, 20 items short form DAST-10)	The degree of consequences related to drug abuse	(Gavin et al., 1989)	Correlated with demographic variables, psychiatric history and drug use. Showed very good concurrent and discriminant validity	5/6 best cutoff score for problematic drug use for the full DAST, 3 the equivalent for the DAST-10
GAD – 7 (7 items)	Measures the severity of anxiety	(Spitzer et al., 2006)	7-Item anxiety scale had good reliability and criterion, construct, factorial and procedural validity	10+ represents a reasonable cut point for identifying cases of GAD
GAD – 7 (7 items)	Measures the severity of anxiety	(Spitzer et al., 2006)		Cut points of 5, 10 and 15 might be interpreted as representing mild, moderate and severe levels of anxiety
Geriatric depression scale (short form, 15 items) (GDS-15)	Measures depressive symptoms in older people	(Nyunt et al., 2009)	A well validated tool for older people with acceptable levels of sensitivity and specificity	A cutoff point of 4 or 5 is used to define depression (maximum score 15)
GHQ-12 (12 items)	A measure of current mental health	(Goldberg et al., 1997)	Validated in 15 centres world-wide with the Composite International Diagnostic Instrument (CIDI)	Scores of over 5/6 indicate psychological distress
GSES (10 items)	Measures self-efficacy	(Jerusalem and Schwarzer, 1992)	Criterion-related validity is documented in numerous correlation studies where positive coefficients were found with favourable emotions	There are none
K6 (6 items) and K10 (10 items)	Global mental health screening tools	K6: (Kessler et al., 2002)	Strong discrimination between community cases and non-cases of DSM-IV/SCID disorders (K6 and K10)	The optimal cutoff scores are 24 (K10) and 14 (K6)
K6 (6 items) and K10 (10 items)	Global mental health screening tools	K10: (Cornelius et al., 2013)		The optimal cutoff scores are 24 (K10) and 14 (K6)
MINI (average of 15 min to complete)	The Mini International Neuropsychiatric Interview (MINI) is a short diagnostic interview	(Sheehan et al., 1998)	Validated with DSM-III-R	N/A
PHQ – 9 (9 items)	Depression	(Spitzer et al., 1999)	The PHQ has diagnostic validity comparable to the original clinician-administered PRIME-MD and is more efficient to use	Cut points of 5, 10, 15 and 20+ can be interpreted as mild, moderate, moderate to severe and severe depressive symptoms, respectively (Callender, 2021)
SAPAS (8 items)	Measures ‘likely’ personality disorder	(Pluck et al., 2012)	A valid screening tool for PD in probation practice and correlates well with SCID-II	A cutoff point of 4 is recommended as indicating the ‘likelihood of personality disorder’
36-Item short form (SF-36, 36 items)	Measures global	(Brazier et al., 1992)	More focussed on general well-being, usually used in healthy populations. Some validity studies, but again focussing on general public samples mainly	Each scale is directly transformed into a 0–100 scale on the assumption that each question carries equal weight. The lower the score, the more disability. A score of zero is equivalent to maximum disability, and a score of 100 is equivalent to no disability
36-Item short form (SF-36, 36 items)	Health status and has two dimensions – physical and mental health	(Ware, 2000)
SPSI-R: S (25 items)	Measures problem-solving ability	(D'Zurilla, 2002)	Validity established in numerous studies	There are none
UNCOPE (6 items)	Screens for substance use disorders	(Hoffmann et al., 2003)	Tested on a sample of recently incarcerated prisoners against the DSM-IV diagnostic interview. Showed high sensitivity, specificity and internal reliability	A score of 2+ indicates possible abuse or dependence
WSAS (5 items)	Measures functional impairment in different domains, for example, work	(Mundt et al., 2002)	A simple, reliable and valid measure of impaired functioning	Score above 20 appears to suggest moderately severe or worse psychopathology. Scores between 10 and 20 are associated with significant functional impairment but less severe clinical symptomatology. Scores below 10 appear to be associated with subclinical populations

Table 2.

Overview of included papers and measures used.

Paper	Type of study	Aim and sample	Key finding	Measures included
Brooker et al. (2009)	Health needs assessment	Based on a representative sample of 183 people on probation in two regions (response rate of 80%)	• ‘SF36 scores revealed that offenders’ subjective mental and physical health and functioning was significantly poorer than that of both the general population and manual social classes using comparative standardised data derived from the Third Oxford Healthy Life Survey’	CAGE
				SF-36
				UNCOPE
Brooker et al. (2011) and Brooker et al. (2012)	Prevalence	Identified the prevalence of drug and alcohol problems and mental illness in a stratified random sample of 173 people on probation in one region. The sample was representative of the wider probation caseload in terms of gender and ethnicity	• 39% had a current mental illness (MINI)	AUDIT
			• 49% had a past mental illness (MINI)	CANFOR-S
			• 47% screened positive for ‘likely personality disorder’ on SAPAS	CSSRI-EU
			• Of those with a current mental illness, and 72% also had an alcohol or drug problem, and 89% also screened positive for ‘likely personality disorder’	DAST-20
			• 56% score 8+ on AUDIT	MINI
			• 12% score 11+ on DAST-20	SAPAS
			• Low levels of access to mental health services	SAPAS
Brooker et al. (2021)	Prevalence/descriptive	Used secondary data to examine which variables are associated with suicidality amongst people on probation with a Kessler 6 (K6) score indicating a need for psychological intervention and who went on to accept that intervention. The final sample was 274 of 3700 people on probation in a Community Rehabilitation Company during the study period	• The sample was divided into three groups – those with current suicidal ideation, those with previous suicide attempts and the ‘no history’ group. ‘When those who had attempted suicide and those with suicidal thoughts were compared to the control group, a range of variables significantly differed including general anxiety, severe depression, severe functional impairment’ (p9)	GAD-7
				K6
				PHQ-9
				WSAS

Callender (2021)	Outcome	Examined health change following completion of a Mental Health Treatment Requirement using existing data for 493 people (208 had commenced the requirement, and 105 had completed it)	Statistically significant change was recorded on all three health measures pre–post intervention. Average pre- and post-scores were as follows	CORE-34
			• Global distress (sample n = 68, CORE-34) reduced from 65.6 (moderate) to 40.0 (mild)	GAD-7
			• Anxiety (sample n = 95, GAD-7) reduced from 13.5 (moderate) to 8.7 (mild)	PHQ-9
			• Depression (sample n = 47, PHQ-9) reduced from 14.9 (moderate) to 9.6 (mild)	PHQ-9
Fitton et al. (2018)	Prevalence	Identified the prevalence of drug and alcohol problems, mental illness in males aged 50+ years on probation in one region. A total of 726 men aged 50+ were identified and probation staff were asked to invite all of them to participate, resulting in 32 taking part	• 47% had a mental illness	AUDIT
			• 31% scored 8+ on AUDIT	DAST-10
			• 3% scored 3+ on DAST-10	GDS-15
			• 3% scored 3+ on DAST-10	MINI
Fowler et al. (2020)	Outcome	Follow-up to Long et al. (2018) pilot. Examines the impact of 1:1 sessions based on a CBT manual delivered by assistant psychologists to meet the mental health needs of people on probation through a third-sector provider based within probation premises. Based on pre- and post-treatment data from 75 people that were referred to the service and completed all 12 sessions during the study period	Statistically and clinically significant changes in pre- to post-therapy scores, with mean scores reducing as follows	GAD-7
			• K6: 22, reducing to 14.6	K6
			• PHQ-9: 17, reducing to 9.2	PHQ-9
			• GAD-7: 14.7, reducing to 8.4	WSAS
			• WSAS: 18.2, reducing to 11.8	WSAS
Hatfield et al. (2004)	Prevalence	Identified the prevalence of mental health problems amongst new residents of seven approved premises in one region during a 12-month period. To be eligible to participate, individuals needed to have been resident at the approved premise for at least seven nights. The GHQ was completed by the residents, and the GAF was completed by staff on a subgroup of residents identified as having mental health needs. The overall response rate was 88% (n = 533), although 66 of these individuals did not complete a GHQ	• 25.1% of residents recorded as having a known psychiatric diagnosis	GAF
			• The mean psychological distress (GHQ) score was 4.8 for those identified as having mental health needs, compared to 3.3 for other residents (p < 0.001)	GHQ
			• 29.9% of people that were not receiving mental health support had a GHQ score of 5+	GHQ
Long et al. (2018)	Outcome	Piloted an approach to improving the use of Mental Health Treatment Requirements and examined the impact of these requirements on a range of outcomes. Sample consisted of 76 consecutive clients of a probation area and in receipt of an MHTR during the study period, 48 of whom completed treatment during the study period. Clients referred to the service were those that scored 20+ on the K10 and had ‘primary care-level’ mental health problems. Those with more severe mental illness were referred to other services where needed	Measures were taken at baseline and 6-months	GAD-7
			• 88% screened positive for ‘likely PD’ on SAPAS	GSES
			• 48 people completed treatment, and there were significant reductions in scores between the time points as follows	K10
			o 81% scored <20 for psychological distress (K10) post-treatment	PHQ-9
			o Those scoring above the cutoff for depression (PHQ-9) reduced from 98% to 23%	SAPAS
			o Those scoring above the cutoff for anxiety (GAD-7) reduced from 96% to 12%	SPSI-R
			• Improvements were also reported in coping skills, self-efficacy, problem-solving and in work and social adjustment	WSAS
Newbury-Birch et al. (2009)	Prevalence	Identified the prevalence of alcohol use disorders (AUD) in criminal justice settings in the north east of England, including three probation areas. Compared identification of these disorders using AUDIT with the Offender Assessment System (OASys). Everyone on probation within the selected areas over a 1 month period was asked to participate. The probation response rate was 64% (n = 266)	• 69% of males and 53% of females on probation screened positive for an AUD	AUDIT
			• OASys under-estimates the prevalence of AUD – 41% of those identified as having an AUD on AUDIT ‘were not identified as needing some form of alcohol intervention by OASys and 10% of clients without an AUD issue were identified as having alcohol-related need using the OASys tool alone’ (p207)
			AUDIT should be used as a standard practice
			• OASys scores were not available for 27% of participants
Newbury-Birch et al. (2014)	Outcome	Randomised controlled trial that compared the use of a) feedback on screening outcomes and an information leaflet (control), vs b) this plus 5 min of structured brief advice, vs c) a and b plus 20 min of brief lifestyle counselling for reducing hazardous or harmful drinking in 525 adults on probation meeting inclusion criteria. Follow-up rate of 68% at 6 months and 60% at 12 months	• The mean AUDIT score at baseline was 16.07 (SD 8.57). 43% were drinking at harmful or dependent levels	AUDIT
Newbury-Birch et al. (2014)	Outcome		• There were no significant differences between groups at 6 months or 12 months – the proportion scoring <8 on AUDIT decreased in all three groups	AUDIT
Pluck et al. (2012)	Descriptive	Investigates the optimal cutoff score for identification of personality disorder in a probation population using SAPAS through comparison with the gold standard measure (Structured Clinical Interview for DSM-IV Personality Disorders [SCID-II]) based on a subsample of 40 consecutive cases from the Brooker et al. (2012) study	• Recommends a cutoff score of 3	SAPAS
Pluck et al. (2012)	Descriptive		• This gives a sensitivity of 0.73, specificity of 0.9 and overall accuracy of 78%	SAPAS
Pluck and Brooker (2014)	Prevalence	Investigates the lifetime and 1 month prevalence of self-harm in a probation population. Part of the wider Brooker et al. (2012) study	• 32.4% (95% CI = 25.6–40.0) of the overall sample had attempted suicide in their lifetime	MINI
			• 5.2% reported deliberate self-harm within the past month
			• 14.5% reported having ideas around self-harm during the past month
			• 8.7% (95% CI = 9.4–25.0) classified as ‘high’ suicidality risk
Pluck et al. (2015)	Descriptive	Examines the characteristics of individuals screening positive for probable personality disorder when compared to those without personality disorder	• Alcohol and drug misuse were the only variables found to be independently associated with the presence or absence of personality disorder	AUDIT
				DAST
				SAPAS
Shaw et al. (2012)	Prevalence and descriptive	Several aims including investigating ‘the relationship between the SAPAS and OASys PD screens in a probation community sample’ (p158), and reporting the number of people meeting the cutoff for each of these screens in a sample of 447 cases drawn from four boroughs in London, 62 of which were excluded due to missing data	• 40.3% met the cutoff for SAPAS (3+)	SAPAS
			• 15.1% met the cutoff for the OASys PD Screen (positive endorsement of 7+ items)
			• 56.9% of those identified as PD cases on OASys were not classified as likely PD on SAPAS
			• Recommends use of SAPAS alongside the OASys PD screen

Prevalence and descriptive studies

We identified six prevalence studies of substance misuse and/or mental illness in probation samples (Brooker et al., 2011, 2012; Fitton et al., 2018; Hatfield et al., 2004; Newbury-Birch et al., 2009; Pluck and Brooker, 2014) together with three papers focussing more on the use of particular measures (Pluck et al., 2012, 2015; Shaw et al., 2012) (see Table 2), a health needs assessment (Brooker et al., 2009) and a study providing insight into the prevalence of suicidal ideation and attempts in a probation population (Brooker et al., 2021). It is difficult to directly compare findings from these studies due to variation in the settings, approaches to sampling and screening tools/measures employed.

Alcohol and drug use

Four studies used AUDIT to identify hazardous and harmful alcohol use and possible dependence on alcohol (Table 2). This ‘gold standard’ tool is quick to complete, and the studies reported prevalence rates of harmful, hazardous or possibly dependent drinking of 56% (Brooker et al., 2012), 31% (Fitton et al., 2018) and 69% (males) and 53% (females) (Newbury-Birch et al., 2009). The latter study compared findings from AUDIT with those from the Offender Assessment System (OASys – the main assessment tool used by probation staff in England and Wales to identify risks and needs). The use of AUDIT in probation practice was recommended as OASys under-estimated the prevalence of alcohol use disorders.

In terms of drug misuse, several measures were identified, all of which are straightforward to use and have minimal administration times (see Table 1 for details of the number of items in each measure). Brooker et al. (2012) reported that 12.1% of participants scored above the cutoff score for problematic drug use on the DAST-20, whilst Fitton et al. (2018) reported that 3% of participants scored above equivalent cutoff score for the DAST-10. Finally, scores from UNCOPE and CAGE in a health needs assessment in two UK regions indicated that ‘almost half of the sample were identified at risk of alcohol abuse or dependence while 39 per cent of the sample was at risk of substance misuse’ (Brooker et al., 2009: p. 49).

Mental health

It is also possible to conclude that there is a high prevalence of mental illness and dual diagnosis within the probation population from these studies – those using the MINI reported overall prevalence rates for current mental illness of 39% (with 72% of these cases also having a drug or alcohol problem) (Brooker et al., 2012) and 47% (Fitton et al., 2018). In another linked study based on data from the MINI, Pluck and Brooker (2014) report on the lifetime and 1 month prevalence of deliberate self-harm, stating that 32.4% (95% CI = 25.6–40.0) of the overall study sample had attempted suicide in their lifetime (Pluck and Brooker, 2014 360), with 5.2% reporting deliberate self-harm within the past month, and 14.5% reporting having ideas around self-harm during that month. Overall, 8.7% (95% CI = 9.4–25.0) of participants were classified as ‘high’ suicidality risk.

Brooker et al. (2012) also explored ‘likely caseness’ of personality disorder using Standardised Assessment of Personality – Abbreviated Scale (SAPAS), reporting a prevalence of 47% based on scoring 3+ on this measure. The validation of this measure with a probation population and the characteristics of those with probable personality disorder are expanded upon in other papers produced from this study (Pluck et al., 2012, 2015), showing that although SAPAS is unable to distinguish individual types of personality disorder, it has potential as a brief screen for probable cases in probation practice. The optimal cutoff score for identifying cases of personality disorder using the SAPAS was established as 3 – with a sensitivity of 0.73 and specificity of 0.9 (Pluck et al., 2012). The use of SAPAS was also explored when compared to the OASys PD screen by Shaw et al. (2012). Here, 40.3% of the sample was classified as likely PD cases using SAPAS, whereas 15.1% screened positive using the OASys PD screen. SAPAS is recommended alongside the OASys PD screen as it is viewed as a useful tool but may have limitations when it comes to identifying the emotionally unstable antisocial personality disorder subgroup (Shaw et al., 2012: p. 164).

A health needs assessment conducted in two regions of the UK using various measures including version two of the short form 36 (SF-36) showed that the mental and physical health and functioning of people on probation ‘was significantly poorer than that of both the general population and manual social classes using comparative standardised data derived from the Third Oxford Healthy Life Survey’ (Brooker et al., 2009: p. 49).

Brooker et al. (2021) analysed data from 274 people on probation in a Community Rehabilitation Company who received a Kessler 6 score of 12+ indicating a need for psychological intervention during the study period to investigate variables that could be associated with suicidality. The sample was divided into a current suicidal ideation group (41%), a group with previous suicide attempts (36%) and a ‘no history’ group (23%). Those that had attempted suicide and those with suicidal thoughts were significantly more likely to have a higher score than the ‘no history’ group for psychological distress (K6 scores of 25+ for 38% of the attempt group, 36% of the ideation group and 15% of the no history group), generalised anxiety (GAD-7 scores of 15+ for 60% of the attempt group, 52% of the ideation group and 37% of the no history group), low mood (PHQ-9 scores of 20+ for 38% of the attempt group, 35% of the ideation group and 12% of the no history group) and functional impairment (WSAS score above 10 for 54% of the attempt group and 34% of those with no history).

Finally, in Hatfield et al.‘s study (2004), staff recorded 25.1% of approved premises (AP) residents as having a known psychiatric diagnosis. The GHQ-12 was used to measure psychological distress, and 29.9% of residents that were not receiving mental health support scored 5+ on this measure – indicating psychological distress.

Needs and service receipt

Findings from Brooker et al. (2011) also suggest that those with a current mental illness are likely to have a higher level of need (a mean score of 10.53 on the CANFOR-S) than those without (a mean score of 4.59). Results from the CSSRI-EU indicate that they are unlikely to access mental health services, with 60% of those with a current mood disorder, 59% of those with a current anxiety disorder, half of those with a current psychotic disorder and 55% of ‘likely cases’ of personality disorder not reporting accessing any mental health service (Brooker et al., 2011).

Outcome studies

In addition, we identified four outcome studies (Callender, 2021; Fowler et al., 2020; Long et al., 2018; Newbury-Birch et al., 2014). Three of these papers relate to the use of Mental Health Treatment Requirements (MHTRs), which can be recommended by the courts, but despite the high prevalence of mental illness in the probation population have rarely been used.

First, Long et al. (2018) piloted an approach to improving uptake of MHTRs in one probation area. Consecutive probation clients were screened for psychological distress using the K10, and eligibility was restricted to those scoring 20+ with ‘primary care-level’ mental health needs. Those with more severe needs were referred elsewhere if they were not already in contact with services. This was a small uncontrolled study, with just 48 people completing treatment during the study period, but these clients showed improvement across a range of mental health measures between baseline and 6-month follow-up, including significant reductions in those scoring above the threshold on measures of psychological distress, depression and anxiety (Table 2).

Second, Fowler et al. (2020) conducted a follow-up to this pilot after a full service had been commissioned. The K6 replaced the K10 as the initial screening tool, and the referral route was extended so that Probation Officers could directly refer those scoring 13+ to the service, rather than needing an MHTR to be recommended by the court. Those with symptoms of psychosis or high risk of suicide were not eligible for the service, which was delivered by a third sector provider based within probation. This study illustrates the difficulty in engaging people with this type of intervention – 569 people were referred to the service within the study timeframe, but just 75 completed all 12 one-to-one sessions, which were based on a cognitive-behavioural therapy manual that was tested in the pilot. Again, statistically and clinically significant reductions in scores were recorded across a range of measures (Table 2) with the numbers receiving a clinically significant score for general distress, anxiety, low mood and difficulties with social adjustability reducing by approximately 45%, 40%, 55% and 39%, respectively. Thirdly, Callender (2021) investigated changes in aspects of mental health after people completed an MHTR and similarly found reductions in average measures of global distress, anxiety and depression.

Newbury-Birch et al. (2014) reported findings from a randomised controlled trial comparing a control group that received feedback on screening for harmful or hazardous alcohol use and an information leaflet with a) a group receiving this plus 5 minutes of structured brief advice and b) a group receiving the feedback, leaflet, structured brief advice and 20 min of brief lifestyle counselling for reducing hazardous or harmful drinking. As there were no significant differences between groups in terms of the proportion scoring less than eight on AUDIT at six or 12 months, the study concludes that the addition of brief advice or lifestyle counselling did not produce any extra benefit in reducing harmful or hazardous drinking (Newbury-Birch et al., 2014: p. 540).

Discussion

Our interest in this topic arose when considering which measures to employ in a (currently ongoing) study aiming to introduce a research-informed approach to identifying health and social care needs amongst people on probation and increase understanding of the needs of the probation population and the extent to which they are being met by current service provision. When designing the study, we needed to consider which measures would be practical for use by probation staff and could potentially provide meaningful data for service commissioners and providers.

This review of mental health outcome measures used in the United Kingdom to assess the mental health of probationers is the first ever undertaken. We identified only 20 such measures which, in essence, measured substance use, mental health symptoms, likely mental illness/disorder and diagnosis, receipt of services and individual measures of needs. This small group of 20 outcome measures/measurement tools, laid out in Table 1, has been reported in 15 studies since 2004. This is an indication of how little research has been undertaken in the field of probation and mental health world-wide as these findings chime with the results of three systematic reviews of mental health, suicide and substance misuse recently reported (Brooker et al., 2020; Sirdifield et al., 2020a, 2020b).

The breakdown of the type of research where outcome measures are reported shows that just four studies examined outcomes as the result of an intervention; 10 reported prevalence and one report concerned a health needs assessment. Three of the four outcome studies used simple pre- and post-scores to measure the impact of referral for an MHTR and were uncontrolled. These studies would be classified as quasi-experimental pre- and post-test designs and are generally considered to be weak. Campbell et al. (1963) have classically described the problems associated with such studies as history, maturation effects, testing itself, regression to the mean and the loss of subjects over time. The fourth outcome study employed a randomised controlled trial (Newbury-Birch et al., 2014), a much stronger design, to examine the impact of providing advice to problem drinkers on probation. Although a high proportion (68%) of the sample was retained at 6-month follow-up, there were no significant changes in mean AUDIT scores following the intervention. Overall, it is important to note too that none of those four UK studies provided an intervention for people with a serious mental illness. This is in stark contrast to the literature in the United States where the role of the probation officer with seriously mentally ill people has been extensively reported (Epperson et al., 2014). This represents a significant and important gap in the UK literature.

Varying estimates of the frequency of serious mental illnesses were obtained in the prevalence studies, for example, Brooker et al. (2012) found that 11% experienced a psychosis and 14.5% a major depressive illness. Fitton et al. (2018), in a smaller sample of 33 older probationers (aged 50 and over), reported that 3% had a diagnosis of a psychosis although this figure rose to 19% when psychosis was considered over a lifetime. Both studies used the MINI to diagnose psychosis.

In a recent review of outcome measures used in the forensic mental health services, the primary objective was to identify ‘instruments which are clinically relevant, to increase applicability to real world settings’ (Ryland et al., 2021: p. 4). The review addressed all key criteria outlined by the COnsensus-based Standards for health Measurement INstruments (COSMIN) group which has developed taxonomy to define the various qualities of outcome measures (Mokkink et al., 2010). One key element of the COSMIN criteria is content validity which seeks to establish if the instrument measures what it is intended to measure. Furthermore, this should reflect those outcomes that are most important for stakeholders, including service users and relatives. Judged by this yardstick, the only measure that shows content validity of all those we review above in probation mental health studies is the CANFOR-S. This was tested with 60 forensic mental health service users, and all items were found to be at least moderately relevant (Thomas et al., 2008).

So, what does the review teach us about measures that probation staff might use routinely to assess need for mental health services in their populations? There are a small number of validated measures that have been successfully used in probation settings in the UK. It is important to note that these measures may not have been validated specifically with probation populations. This could potentially be an area for future research. However, the results of testing with other populations, including those within other areas of the criminal justice system are encouraging (see references in Table 1). Whilst our conclusions are based on a limited number of studies, these studies show the value of using these types of measures in probation practice and research. For the screening of primary care mental health, the K6 and K10 are obvious candidates to use in routine assessment, and maybe SAPAS if personality disorder is important. Indeed, SAPAS has been recommended for use alongside OASys in order ‘to enhance sensitivity to antisocial cases and incorporate information about risk’ (Shaw et al., 2012: p. 156).

However, better measures of the specific severity of depression are the PHQ-9 (and maybe the GDS-15 for older people) and for anxiety the GAD-7. In the case of these two latter measures, one would need to ensure that clinical levels of anxiety and depression are being reached. For the seriousness of substance misuse, the short forms of DAST and AUDIT would be useful. Newbury-Birch et al. (2009) found that OASys alone seriously underestimated the number of people on probation with an alcohol-related need.

To obtain a sense of what needs clients had in relation to their mental health the CANFOR-S, which has been service-user assessed, would be most suitable. This leaves open the question of measures that might be employed to screen for/assess serious mental illness. Here, one option would be a systematic review of probation research conducted in the United States. Another option might be to use the Psychosis Screening Questionnaire (PSQ) (Bebbington and Nayani, 1995). It consists of five items, and Coid and Ulrich (2011) demonstrated its use in a sample of remand and sentenced prisoners drawn from 131 prisons. Using cutoff points of 3+ for men and 2+ for women, it was established that the PSQ was a valid tool when compared with diagnostic interviews. However, further use of the tool in probation would need to be rigorously examined.

Conclusion

It is an unavoidable conclusion of the recent thematic review of mental health in the criminal justice system that more should be done to help probation staff recognise mental illness. We have reviewed the UK (and European) literature to identify measures that have been used in previous prevalence/descriptive and outcome studies. We recommend a suite of measures which probation staff could easily be trained to use to identify conditions which, in some cases, OASys under-reports. These include, inter alia, brief measures for mental health, psychosis and personality disorder screening, depression, anxiety and alcohol and drug consumption.

Supplemental Material

Supplemental Material – Identifying mental illness and monitoring mental health in probation service settings

Supplemental Material for Identifying mental illness and monitoring mental health in probation service settings by Charlie Brooker, Coral Sirdifield and Thomas Parkhouse in the European Journal of Probation.

Supplemental Material

Supplemental Material – Identifying mental illness and monitoring mental health in probation service settings

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This study/project is funded by the National Institute for Health Research (NIHR) [Research for Patient Benefit Programme (NIHR 201091). The views expressed are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

ORCID iDs

Charlie Brooker

Coral Sirdifield

Thomas Parkhouse

Supplemental Material

Supplementary material for this article is available online.

Author biographies

Charlie Brooker is Honorary Professor of Health and Justice at Royal Holloway, University of London. Along with Coral Sirdifield, he recently published the first book on probation and mental health with Routledge.

Coral Sirdifield is a Senior Research Associate at the University of Lincoln with an interest in health inequalities, quality improvement in healthcare and improving the health of people in the criminal justice system.

Thomas Parkhouse is a Research Assistant at the University of Lincoln with an interest in health and social care. He has a research background in forensic and cognitive psychology. His PhD focussed on the creative process of question generation in investigative interviewing.

References

Barkham

Bewick

Mullin

, et al. (2013) The CORE-10: a short measure of psychological distress for routine use in the psychological therapies. Counselling and Psychotherapy Research 13(1): 3–13. DOI: 10.1080/14733145.2012.729069.

Barkham

Gilbert

Connell

, et al. (2005) Suitability and utility of the CORE-OM and CORE-A for assessing severity of presenting problems in psychological therapy services based in primary and secondary care settings. The British Journal of Psychiatry: the Journal of Mental Science 186: 239–246.DOI: 10.1192/bjp.186.3.239.

Bebbington

Nayani

(1995) The psychosis screening questionnaire. International Journal of Methods in Psychiatric Research 5(1): 11–19.

Brazier

Harper

Jones

, et al. (1992) Validating the SF-36 health survey questionnaire: new outcome measure for primary care. Bmj: British Medical Journal 305(6846): 160–164. DOI: 10.1136/bmj.305.6846.160.

Brooker

Sirdifield

Blizard

, et al. (2012) Probation and mental illness. The Journal of Forensic Psychiatry & Psychology 23(4): 522–537. DOI: 10.1080/14789949.2012.704640.

Brooker

Sirdifield

Blizard

et al. (2011) An Investigation into the Prevalence of Mental Health Disorder and Patterns of Health Service Access in a Probation Population. Lincoln: University of Lincoln.

Brooker

Sirdifield

Marples

(2020) Mental health and probation: a systematic review of the literature. Forensic Science International: Mind and Law 1: 100003. DOI: 10.1016/j.fsiml.2019.100003.

Brooker

Syson-Nibbs

Barrett

, et al. (2009) Community managed offenders’ access to healthcare services: report of a pilot study. Probation Journal 56(1): 45–59. DOI: 10.1177/0264550509102401.

Brooker

Tocque

West

, et al. (2021) Suicide in probation: towards the ideation-to-action model. Probation Journal 0(0): 02645505211041581. DOI: 10.1177/02645505211041581.

10.

Bush

Kivlahan

McDonell

, et al. (1998) The Audit alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Ambulatory Care Quality Improvement Project (ACQUIP). Alcohol Use Disorders Identification Test. Archives of Internal Medicine 158(16): 1789–1795. DOI: 10.1001/archinte.158.16.1789.

11.

Callender

(2021) Community Sentence Treatment Requirements – Exploring Health Outcomes. Preliminary Findings Policy Brief. http://nectar.northampton.ac.uk/14812/1/CSTR_Exploring_Health_Outcomes_Policy_Brief_July_2021.pdf

12.

Campbell

Stanley

Gage

(1963) Experimental and Quasi-Experimental Designs for Research. Boston: Houghton, Mifflin and Company.

13.

Chan

Pristach

Welte

(1994) Detection by the CAGE of alcoholism or heavy drinking in primary care outpatients and the general population. Jornal of Substain Abuse 6(2): 123–135. DOI: 10.1016/s0899-3289(94)90157-0.

14.

Chisholm

Knapp

Knudsen

, et al. (2000) Client socio-demographic and service receipt inventory--european version: development of an instrument for international research. EPSILON Study 5. European psychiatric services: inputs linked to outcome domains and needs. The British Journal of Psychiatry: the Journal of Mental Science Suppl(39): s28–s33. DOI: 10.1192/bjp.177.39.s28.

15.

Coid

Ullrich

(2011) Prisoners with psychosis in England and Wales: diversion to psychiatric inpatient services? International Journal Law Psychiatry 34(2): 99–108. DOI: 10.1016/j.ijlp.2011.02.003.

16.

Cornelius

BLR

Groothoff

van der Klink

JJL

, et al. (2013) The performance of the K10, K6 and GHQ-12 to screen for present state DSM-IV disorders among disability claimants. BMC Public Health 13: 128–128. DOI: 10.1186/1471-2458-13-128.

17.

D’Zurilla

TJNAMM-OA

(2002) Social Problem-Solving Inventory-Rrevised (SPSI-R). North Tonawanda, NY: Multi-Health Systems.

18.

Epperson

Canada

Thompson

, et al. (2014) Walking the line: specialized and standard probation officer perspectives on supervising probationers with serious mental illnesses. International Jornal Law Psychiatry 37(5): 473–483. DOI: 10.1016/j.ijlp.2014.02.020.

19.

Fitton

Bates

Hayes

, et al. (2018) Psychiatric disorders, substance use, and executive functioning in older probationers. Criminal Behaviour and Mental Health: CBMH 28(6): 447–459. DOI: 10.1002/cbm.2094.

20.

Fowler

Price

Burger

, et al. (2020) Embedding third sector psychology services within the probation environment: an alternative to MHTRs. Journal of Criminal Psychology 10(1): 16–29. DOI: 10.1108/JCP-04-2019-0013.

21.

Gavin

Ross

Skinner

(1989) Diagnostic validity of the drug abuse screening test in the assessment of DSM-III drug disorders. British Journal of Addiction 84(3): 301–307. DOI: 10.1111/j.1360-0443.1989.tb03463.x.

22.

Goldberg

Gater

Sartorius

, et al. (1997) The validity of two versions of the GHQ in the WHO study of mental illness in general health care. Psychological Medicine 27(1): 191–197. DOI: 10.1017/S0033291796004242.

23.

Hatfield

Ryan

Pickering

, et al. (2004) The mental health of residents of approved premises in the greater manchester probation area: a cohort study. Probation Journal 51(2): 101–115. DOI: 10.1177/0264550504044169.

24.

HM Inspectorate of Probation; HM Crown Prosecution Service Inspectorate; HM Inspectorate of Constabulary and Fire and Rescue Services; Care Quality Commission; Healthcare Inspectorate Wales; HM Inspectorate of Prisons (2021) A Joint Thematic Inspection of the Criminal Justice Journey for Individuals with Mental Health Needs and Disorders. Manchester: Her Majesty’s Inspectorate of Probation.

25.

HMIP (2016) An Inspection of through the Gate Resettlement Services for Short-Term Prisoners: A Joint Inspection by HM Inspectorate of Probation and HM Inspectorate of Prisons. Manchester: Her Majesty’s Inspectorate of Probation.

26.

HMPPS; NPS (2019) National Probation Service Health and Social Care Strategy 2019-2022. London: National Probation Service.

27.

Hoffmann

Hunt

Rhodes

, et al. (2003) UNCOPE: a brief substance dependence screen for use with arrestees. Journal of Drug Issues 33(1): 29–44. DOI: 10.1177/002204260303300102.

28.

Jerusalem

Schwarzer

(1992) Self-efficacy as a resource factor in stress appraisal processes. Self-efficacy: Thought Control of Action. Washington DC: Hemisphere Publishing Corp, 195–213.

29.

Kessler

Andrews

Colpe

, et al. (2002) Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychological Medicine 32(6): 959–976. DOI: 10.1017/s0033291702006074.

30.

Long

Dolley

Hollin

(2018) The use of the mental health treatment requirement (MHTR): clinical outcomes at one year of a collaboration. Journal of Criminal Psychology 8(3): 215–233. DOI: 10.1108/JCP-01-2018-0003.

31.

Mokkink

Terwee

Patrick

, et al. (2010) The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiol 63(7): 737–745. DOI: 10.1016/j.jclinepi.2010.02.006.

32.

Mundt

Marks

Shear

, et al. (2002) The work and social adjustment scale: a simple measure of impairment in functioning. The British Journal of Psychiatry: the Journal of Mental Science 180: 461–464. DOI: 10.1192/bjp.180.5.461.

33.

Newbury-Birch

Coulton

Bland

, et al. (2014) Alcohol screening and brief interventions for offenders in the probation setting (SIPS Trial): a pragmatic multicentre cluster randomized controlled trial. Alcohol and Alcoholism 49(5): 540–548. DOI: 10.1093/alcalc/agu046.

34.

Newbury-Birch

Harrison

Brown

, et al. (2009) Sloshed and sentenced: a prevalence study of alcohol use disorders among offenders in the North East of England. International Journal of Prisoner Health 5(4): 201–211. DOI: 10.1080/17449200903343183.

35.

Nyunt

Fones

Niti

, et al. (2009) Criterion-based validity and reliability of the Geriatric Depression Screening Scale (GDS-15) in a large validation sample of community-living Asian older adults. Aging & Mental Health 13(3): 376–382. DOI: 10.1080/13607860902861027.

36.

Phelan

Slade

Thornicroft

, et al. (1995) The camberwell assessment of need: the validity and reliability of an instrument to assess the needs of people with severe mental illness. British Journal of Psychiatry 167(5): 589–595. DOI: 10.1192/bjp.167.5.589.

37.

Pluck

Brooker

(2014) Epidemiological survey of suicide ideation and acts and other deliberate self-harm among offenders in the community under supervision of the Probation Service in England and Wales. Criminal Behaviour and Mental Health: CBMH 24(5): 358–364. DOI: 10.1002/cbm.1909.

38.

Pluck

Brooker

Blizard

, et al. (2015) Personality disorder in a probation cohort: demographic, substance misuse and forensic characteristics. Criminal Behaviour and Mental Health: CBMH 25(5): 403–415. DOI: 10.1002/cbm.1938.

39.

Pluck

Sirdifield

Brooker

, et al. (2012) Screening for personality disorder in probationers: validation of the standardised assessment of personality—abbreviated scale (SAPAS). Personality and Mental Health 6(1): 61–68. DOI: 10.1002/pmh.177.

40.

Power

(2020) Engaging with mental health challenges in probation practice. Irish Probation Journal 17: 107–127.

41.

Public Health England (2020) Health and Social Care Needs Assessment of Adults under Probation Service Supervision in the Community. A Guidance Document. London: Public Health England.

42.

Revolving Doors Agency (2017) Rebalancing Act. A Resource for Directors of Public Health, Police and Crime Commissioners, the Police Service and Other Health and Justice Commissioners, Service Providers and Users. London: Revolving Doors Agency.

43.

Ryland

Cook

Yukhnenko

, et al. (2021) Outcome measures in forensic mental health services: a systematic review of instruments and qualitative evidence synthesis. European Psychiatry 64(1): e37. DOI: 10.1192/j.eurpsy.2021.32.

44.

Saunders

Aasland

Babor

, et al. (1993) Development of the alcohol use disorders identification test (AUDIT): WHO collaborative project on early detection of persons with harmful alcohol consumption-II. Addiction 88(6): 791–804. DOI: 10.1111/j.1360-0443.1993.tb02093.x.

45.

Shaw

Minoudis

Craissati

(2012) A comparison of the standardised assessment of personality – abbreviated scale and the offender assessment system personality disorder screen in a probation community sample. The Journal of Forensic Psychiatry & Psychology 23(2): 156–167. DOI: 10.1080/14789949.2012.655763.

46.

Sheehan

Lecrubier

Sheehan

, et al. (1998) The mini-international neuropsychiatric interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry 59(Suppl 20): 22–33. quiz 34-57.

47.

Sirdifield

Brooker

Marples

(2020a) Substance misuse and community supervision: a systematic review of the literature. Forensic Science International: Mind and Law 1: 100031. DOI: 10.1016/j.fsiml.2020.100031.

48.

Sirdifield

Brooker

Marples

(2020b) Suicide and probation: a systematic review of the literature. Forensic Science International: Mind and Law 1: 100012. DOI: 10.1016/j.fsiml.2020.100012.

49.

Sirdifield

Marples

Brooker

, et al. (2019) Probation Healthcare Commissioning Toolkit, A Resource for Commissioners and Practitioners in Health and Criminal Justice. Lincoln: University of Lincoln.

50.

Spitzer

Kroenke

Williams

(1999) Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary Care Evaluation of Mental Disorders. Patient Health Questionnaire. Jsls: Journal of the Society of Laparoendoscopic Surgeons 282(18): 1737–1744. DOI: 10.1001/jama.282.18.1737.

51.

Spitzer

Kroenke

Williams

, et al. (2006) A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 166(10): 1092–1097. DOI: 10.1001/archinte.166.10.1092.

52.

Thomas

SDM

Slade

Mccrone

, et al. (2008) The reliability and validity of the forensic camberwell assessment of need (CANFOR): a needs assessment for forensic mental health service users. International Journal of Methods in Psychiatric Research 17(2): 111–120. DOI: 10.1002/mpr.235

53.

Ware

Jr (2000) SF-36 health survey update. Spine (Phila Pa 1976) 25(24): 3130–3139. DOI: 10.1097/00007632-200012150-00008.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.20 MB

0.35 MB