Abstract
A number of studies have found that when probation officers, and others who supervise young people and adults on community based orders, have good intervention skills their clients are more likely to be engaged in supervision and to have low recidivism rates. The skills include, role clarification, pro-social modelling, problem solving, cognitive and relationship skills. Little research has been done, however, on the development of these skills across whole organisations. This study aimed to examine the extent to which training and coaching of probation officers, across two state youth justice departments in Australia, improved the use of workers’ skills. Audio-tapes of worker/client interviews were provided to research staff before and after training and coaching. Analysis of the audio-tapes found a significant increase in the overall use of worker skills following the training and coaching. However, the increases in the skills applied largely to role clarification, rather than pro-social modelling, problem solving and cognitive skills.
Introduction
This paper focuses on the extent to which the practice skills of youth justice workers can be enhanced through training and coaching in the use of those skills. Knowledge about this issue can contribute to the growing literature on ‘what works’ in criminal justice community supervision and potentially influence practices aimed at developing effective practice skills. The paper begins with a literature review on the topic and then outlines the methodology and results of the project. This is followed by limitations and discussion.
Literature review
An increasing number of studies and reviews of studies have found that community-based offender supervision can be effective in reducing recidivism rates. A number of literature reviews have suggested that when supervisors use particular skills clients' under their supervision re-offend significantly less often than clients' supervised by workers who do not use these skills (Chadwick et al., 2015; Dowden and Andrews, 2004; Trotter, 2013). Chadwick et al. (2015), for example, found that the average recidivism was a little over 36% for individuals supervised by trained officers compared to a little under 50% for the status quo, a 13% difference (or 28% lower for trained officers depending how the figures are interpreted). The argument presented by these authors is not that offender supervision always works, but that appropriate forms of intervention are effective if appropriate skills are offered by workers.
Effective practice skills
The Trotter (2013) and Chadwick et al. (2015) reviews identified similar supervision skills as being effective. These include pro-social modelling and re-inforcement, problem solving, cognitive techniques, relationship factors and role clarification.
Pro-social modelling and re-inforcement
Pro-social modelling and re-inforcement, has been shown to be effective in a number of studies in adult and juvenile settings (Andrews et al., 1979; Bourgon et al., 2010; Raynor et al., 2014; Trotter, 2012). It involves workers modelling pro-social behaviours (e.g. reliability and fairness), positively re-inforcing clients’ pro-social comments and behaviours (e.g. seeking work and accepting responsibility for offences) and challenging clients’ pro-criminal comments and behaviours (e.g. excuses for offending).
Problem solving
The studies suggest that effective interventions also address the issues which have led people to offend, sometimes referred to as criminogenic needs (Bonta and Andrews, 2017). Criminogenic needs may include employment, family relationships, drug use, peer group associations, finances or pro-criminal attitudes. A number of studies also suggest that working collaboratively with people who have offended, and focussing on the issues or problems which they themselves identify as problematic, leads to lower recidivism (Mcneill and Whyte, 2007; Trotter, 1996, 2013). There is also support for problem-solving approaches whereby workers canvass a wide range of client issues, reach agreement on problems to be addressed, set goals and develop strategies to achieve those goals (Dowden and Andrews, 2004; Trotter, 1996, 2013).
Cognitive behavioural techniques
Cognitive behavioural interventions, which help clients to address thinking patterns which relate to crime, have also been found to be effective. Dowden and Andrews (2004) in their meta-analysis of core practices, found that cognitive techniques may be the most influential of the skills. There are, however, varying definitions of cognitive techniques with problem solving and pro-social modelling sometimes included within the definition (Trotter, 2013).
Worker/client/relationship
A meta-analysis by Dowden and Andrews (2004) identified the client/worker relationship as one of the elements of core correctional practice. Relationship skills are also referred to in other reviews (e.g. Bonta et al., 2008; Bourgon et al., 2010; Trotter, 1996, 2013). These studies generally suggest that probation officers are more effective if their relationships with clients are characterised by empathy, openness, warmth, humour, enthusiasm and a positive view about the clients’ capacity to change.
Role clarification
Effective work in criminal justice may also involve what Rooney (2009) and Jones and Alcabes (1993) refer to as client socialisation or what others have referred to as role clarification (Trotter, 2015). One aspect of role clarification involves helping the client to accept that the worker can help with the client’s problems even though the worker has a social control role. Other aspects of role clarification involve exploring the client’s expectations, helping the client to understand what is negotiable, the limits of confidentiality and the nature of the worker’s authority. Research in mental health (Videka-Sherman, 1988) and child protection (Trotter, 2004) has explored the use of this skill in practice. Less work has been done in corrections settings although several studies (see Trotter, 2015 for a review) found that role clarification skills are part of a group of skills which are related to reduced re-offending by clients.
Developing skills across organisations
Despite the developing knowledge about effective practice, there is evidence to suggest that evidence-based practice skills are not used routinely in probation and parole supervision (Taxman and Sachwald, 2010) and that probation and parole supervision may have little overall impact (Hyatt and Barnes, 2014). While there is some evidence that practice skills may be increased by training and coaching (e.g. Young et al., 2013) this work has mostly been done with relatively small samples rather than considering the development of skills across whole organisations. Where the focus has been on whole organisations, such as the study by Raynor et al. (2014), the focus has often been on the relationship between skills and recidivism rather than on the impact of training on the use of specific skills (Trotter, 2013).
A study by Bonta et al. (2011), focused on a more widespread application of skills, found that training and coaching in effective practice skills led to increased use of skills by workers and lower recidivism by their clients compared to a control group which did not receive the training and coaching. However, a replication of this is in another jurisdiction in Canada did not to find the same results (Bonta et al., 2019). They found in audio recordings supplied by participants that there were inconsistent changes in officers’ skills and the trained officers did not have lower recidivism rates compared to the control group, although they did find that the use of cognitive techniques by workers was related to lower recidivism. They also found that by 10 months more than half of the probation officers had discontinued involvement in the professional development activities.
There remains some doubt therefore about the extent to which worker training, supervision and coaching can support the implementation of effective practice principles in probation and community supervision. Further the authors have been unable to locate studies, other than Bonta et al. (2019) and Raynor et al. (2014) which have examined attempts to implement evidence-based practices across whole organisations (as opposed to a specific sample of workers and clients). This is important because while it seems clear that supervisor skills make a difference to recidivism this is of little practical value if the skills can’t be developed across organisations. Bonta et al. (2019) research suggests that this widespread application of good practice may be problematic.
Also, previous studies have not attempted to examine the extent to which particular skills are impacted by training or coaching or other interventions This study aimed to help fill this gap through a project undertaken in Youth Justice, in two states in Australia, which aimed to increase the skill levels of youth probation officers through training and coaching and to consider the extent to which the different skills were impacted by the training and coaching.
Methodology
Aim
The aim of this study was to examine the extent to which the skills of youth justice workers are enhanced by training, coaching and other support mechanisms offered in two Australian Departments of Youth Justice. The project also aimed to examine which skills were most and least impacted by the training and coaching intervention.
Note that a previous publication (Trotter, 2019) provided some preliminary findings in relation to this project. This paper analyses updated data from two Australian states.
Design and procedure
The design of the study was a before/after comparison using quantitative methods (Bachman and Schutt, 2007). The limitations of this design are addressed in the discussion section of this paper.
The procedure was as follows: The effective practice skills were introduced to senior staff in the youth justice departments to gain their support for and understanding of the intervention. A series of 2-day training courses on effective practice skills were offered to all youth justice workers and staff supervisors in two states. The training was delivered by training staff within the organisation following train the trainer workshops.
Participants in the training courses were asked to provide an audio-tape of an interview they had recently conducted with a youth justice client. (Client consent was gained at the time of recording for the tape to be used for research purposes, consistent with university and youth justice departmental ethics). Staff participants in the training were invited to be involved in the research project. Following their informed consent, the tapes supplied to the training were used as the pre-tapes for the research project.
Following training, youth officers were coached by senior staff by reviewing and coding audio-tapes of interviews followed by feedback to workers about their use of the skills. The aim was for this to occur every 3 months for each worker. The supervisor would listen to the tape with the worker and help the worker identify when skills were used and when they could have been used but were not used. To assist with this the supervisor used a checklist of the key skills. The focus, consistent with the best practice skills, was on positive feedback to workers with careful challenging of poor use or absence of skills. The individual sessions were on some occasions supplemented by group sessions whereby a small group of workers would provide feedback to each other on the use of skills.
In a separate process, for the purposes of the research, researchers coded for skills evident in the pre-training audio-tapes and tapes recorded 2–6 months following training. The audio-tapes were coded using a coding form developed for this purpose in previous research (Trotter, 2012; Trotter and Evans, 2012). The data was then analysed to determine changes in use of skills between the pre-training tapes and the post training tapes. The framework for coding the skills involved research officers listening to the tapes and coding for each of the skills as they were identified. The research funding allowed for 1 hour for coding each tape and the researchers would often go back and forth within the tape to carefully examine whether particular skills were being used. After listening to the complete tape the research officers would then code each section and the global score. Much of the time of the research officers was spent making judgements about the skills and re-listening to different sections of the tapes as well as listening to each tape from beginning to end. The mean duration of the interviews was 25 min with the longest being 65 min and the shortest 5 min.
The coding manual
As mentioned the audiotapes were coded using a coding manual developed for a previous project (Trotter and Evans, 2012) and in consultation with others doing similar work, in particular James Bonta and colleagues in Canada and Peter Raynor and colleagues in UK (Bonta et al., 2011; Raynor et al., 2014).
The coding manual is divided into 15 sections: set-up of the interview; structure of the interview; role clarification; needs analysis; problem solving; developing strategies; relapse prevention/cognitive behavioural techniques; pro-social modelling and re-inforcement; nature of the relationship; empathy; confrontation; termination; use of referral/community resources; non-verbal cues; and incidental conversations. Each of the 15 sections contains a number of items which can be rated on a five-point scale. For example, the problem-solving section included: problem survey and identifying criminogenic needs; problem ranking; problem exploration; setting goals; timeframe; review; developing a contract; developing strategies; ongoing monitoring; and time spent conducting problem solving. For the skill to be rated highly it needed to be implemented in a way which was consistent with the research about good practice referred to in the literature review.
The researchers used the following scale to code an overall 10-point global score on the use of the effective practice skills: 1. The worker did not utilise any of the effective practice principles; 3. The worker used minimal effective practice skills; 5. The interview showed some use of the effective practice principals; 7. The worker used several of the effective practice principles in a deliberate manner; 10. The worker deliberately used the effective practice principles in an efficient and successful manner.
In rating the global score the researchers took into account each of the ratings of the individual skills and allocated a score on their judgement of the overall level of skills used by the worker in the interview. The global score correlated significantly with each of the measures of the extent to which workers used individual skills. For example, the correlation between use of pro-social modelling skills was 0.643 and with relationship skills 0.766. The correlation between the global score and client engagement was also high at 0.679.
Coders rated the pre and post tapes for engagement of clients in the interview using the following scale: 1. Client is not engaged in the interview, is non-responsive, giving monosyllabic responses; 3. Client partially engaged in the interview, thought responses still limited; 5. Client is occasionally engaged in the interview and is responsive to a proportion of what the worker is saying; 7. Client appears engaged in the interview although may appear distracted or disinterested at times; 10. Client is actively engaged in the interview, evidenced by taking notice of what worker is saying, that is, listening, responding to the worker, and actively learning.
The use of individual skills was also coded using the following scale regarding the time spent using the skill during the interview. High scores were given when the coder assessed the worker as using the skills correctly and frequently. 1. None 2. Not much 3. Some of the time 4. Quite often 5. A lot
Inter-rater reliability
Coding was undertaken by 3 different research staff members, each trained in the purpose and use of the coding manual. The coding manual scores demonstrated high levels of inter-rater reliability in earlier research (Trotter, 2012). For example, for 20 cross coded interviews the spearman rho correlation on the overall global skill score between first and second coders was 0.741 (n 40, p.000), for client overall engagement (0.424, p.031) for time spent discussing role clarification it was 0.548 (p.006), for time spent on problem solving it was 0.626 (p.002) and for pro-social modelling 0.561 (p.005). Note that a previous study (Trotter, 2012; Trotter and Evans, 2012) also found that scores, using the manual, allocated by research officers following analysis of audio-tapes of interviews were significantly associated with low re-offending and order completion. For example, interviews which were scored at 5 or more for skills were associated with 50% lower order completion rate compared to interviews scored at 4 or less on a 10 point scale.
Cross coding was also undertaken for 12 worker/client interviews for the current study. The spearman rho correlations were 0.413 (p.091) for the global score, 0.484 (p.056) for engagement 0.624 (p.015) for role clarification, 0.477 (p.059) for problem solving, 0.408 (p.106) for relationship skills, and 0.766 (p.003) for cognitive techniques. In other words, the cross coding revealed moderate to strong correlations across the two samples (Quirk, 2021). Coding is discussed further under the limitations of the study.
Sample
Sixty-two tapes, 32 pre-training (supplied at the first training course) and 30 post training (2 to 6 months after completing the training and during the period of the coaching) were available for coding. The tapes were supplied by 23 workers with each worker providing between 1 and 3 tapes. Nineteen workers provided both before and after tapes (38 of the 62 tapes). The remaining tapes were supplied by different workers. Approximately 200 direct practice staff were employed in two states (AIHW, 2018). The workers were employed as youth justice officers with responsibility for supervising young people placed on probation orders and supervised bonds. They also supervised young people on parole following their release from youth detention. The relatively small sample is discussed in the limitations section later in the paper.
Analysis
The tapes were coded and entered into SPSS and analysed to determine both the overall use of skills and the use of the individual skills. The use of skills was compared using comparisons of means and cross tabs in order to examine the differences between skill scores on the pre and post tapes.
Results
Overall use of skills
Global score out of 10.
p = 0.025 (one tailed test).
Global score of 5 or more given by researchers on pre and post tapes.
p.010 (one tailed test).
Was client engaged overall (out of 10).
Client engagement pre and post interviews (coding 5 or more).
p.09 (one tailed test).
Use of individual skills
The use of individual skills was also coded using the 5 point scale regarding the time spent using the skill during the interview. High scores were given when the coder assessed the worker as using the skills correctly and frequently.
Role clarification
Time spent discussing role clarification.
p = 0.012.
Individual aspects of the role clarification were also coded before and after the training and coaching. There was a significant increase in the skill of discussing the nature of the worker’s authority (1.4 to 2.2 on the 3 point scale p.049) and in discussions about the worker’s dual role as helper/investigator (1.2 to 2.3 p.004). On each of the other role clarification skills there was a non-significant increase in the score given by the coder with the exception of discussions about confidentiality where there was a minimal and non-significant decrease.
Relationship skills
Time spent using relationship skills.
p = 0.10.
Pro-social modelling
Time spent using pro-social modelling.
Problem solving
Time spent conducting problem solving.
There was also minimal use of individual problem-solving skills in the pre-tapes and the post tapes. Nevertheless, there were non-significant increases in the use of problem survey, problem exploration, and developing strategies.
Cognitive techniques
Time spent using cognitive techniques.
Discussion
The examination of the before and after tapes suggests that the training and coaching offered to youth justice workers in the study made a difference to the use of skills. The mean score given by the research officers following analysis of the pre and post tapes moved from minimal use of the skills to between minimal and some use. Only 23% of the interviews were given a global score of 5 or more before the training and coaching whereas 50% were given a score of 5 or more following training and coaching. The clients were also more engaged in the interviews which followed training and coaching although the differences do not reach conventional levels of statistical significance.
While the scores were relatively low even after training and coaching, particularly for problem solving and cognitive skills, this is consistent with previous studies in a number of jurisdictions which suggest that workers generally make minimal use of effective practice skills (Bonta et al., 2008; Trotter and Evans 2012) and that training may have an impact on the use of skills (Robinson et al., 2011; Trotter, 1996, 2013). Analysis of the tapes found that workers tend to spend a lot of their time doing specific tasks during supervision such as pre-sentence reports, workbooks and case plans. The coders found that in many of the interviews most of the time was spent on these tasks, limiting the use of skills such as problem solving. This could be an important finding and warrants further investigation. The focus on ‘what works’ in supervision and attempts to improve worker practices may be of little value if workers have only minimal opportunity to use these skills with their clients.
Limitations
This study has limitations. While the inter-rater reliability measures showed moderate to medium correlations many of them were not statistically significant. Although given that the inter-rater reliability was only undertaken on 12 audio-tapes this might explain the low levels of statistical significance. There were higher correlations and levels of significance in a previous studies study using the same form with a larger sample.
This study does not examine recidivism. An examination of recidivism would show both the relationship between the use of the skills and the impact of the skills on clients. On the other hand, numerous studies and reviews of studies referred to earlier in this paper have found a relationship between the use of skills and recidivism and the purpose of this study was to consider the level and nature of the impact of training and coaching on workers’ skills rather than to examine recidivism.
While the skills development project was undertaken across youth justice services in two states of Australia, with all direct practice staff being offered training and coaching, the sample is relatively small. It was dependent on volunteers and it was a convenience rather than a representative sample. Unfortunately, informed consent requirements make it difficult to gain large representative samples in correctional settings. Accessing data of this nature, even with limited numbers, requires high levels of co-operation and transparency from government departments and staff members and it could be argued that even gaining small numbers is an achievement. While it is acknowledged that the numbers limit the generalisability of the findings the numbers did prove to be sufficient to find some statistically significant differences between before and after interviews.
Another limitation was that while two youth justice services from two states were included in the study, one a large state with a population over 5 million people and the other a smaller state with a population less than 2 million the relatively small sample did not lend itself to comparisons between the two states.
Another potential limitation of the study is that staff are likely to tape record their best interviews and the level of skills displayed may not therefore be reflective of their general practice. Similarly, resistant staff may have poor skills and may not agree to being involved in the research. On the other hand, this study aimed to examine the skills pre and post training and coaching and there is no reason to believe that workers would have selected better interviews following, rather than before, training and coaching.
A final potential criticism is that successful implementation of any policy or practice is heavily dependent on organisational culture, management and staff involvement. This study does not capture these factors. In some ways, this is a fair criticism and there is evidence that these factors apply in corrections settings (Taxman and Sachwald, 2010). Further research might examine these factors in more detail.
Further research
The greatest changes in this study were seen in role clarification skills. This is difficult to explain. It may be that role clarification is a relatively new concept for many workers and they embraced it for this reason. There is little literature on role clarification in criminal justice work – the authors were able to find no subject references to role clarification or dual role in relation to criminal justice in a search of criminal justice abstracts (Criminal Justice Abstracts, 2010 to current) for the last 5 years. On the other hand, there is also little written about pro-social modelling. More research is certainly needed to explore why some worker skills may be more amenable to change than others.
Qualitative research which focuses on interviews or focus groups with workers, senior staff and other stakeholders including clients could also be valuable in explaining why workers seem more ready to change in relation to some skills rather than others. Given the potential benefits of improving practice skills among youth justice workers and others who work with people who have offended there is a need for more research on how this can be done in different jurisdictions.
Conclusion
This study suggests that training and coaching can have an impact on worker skills across organisations even though the changes in worker practices may be relatively moderate and not across all skills. It also suggests that some skills may be more resistant to change than others with little movement in problem solving, cognitive techniques and pro-social modelling. This is to some degree consistent with research in Canada by Bonta et al. (2019) which found that workers skills seem to dissipate over time following training – although this study found that some skills were only minimally present only a few months after training and coaching.
A considerable body of research suggests that workers skills are related to client engagement in supervision and to re-offending rates. It is likely therefore that the more worker skills can be improved the more benefits will flow for those under supervision and their victims. This study, along with a number of previous studies, suggests that there is value in putting resources into training and coaching justice workers in effective practice skills. Further research is needed, however, on how to develop the use of particular skills, including the style and duration of training, coaching methods, the nature of staff supervision, leadership issues and organisational culture.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Queensland and SA Departments of Youth Justice.
