Using Social Judgment Theory method to examine how experienced occupational therapy driver assessors use information to make fitness-to-drive recommendations

Abstract

Introduction

As people with a range of disabilities strive to increase their community mobility, occupational therapy driver assessors are increasingly required to make complex recommendations regarding fitness-to-drive. However, very little is known about how therapists use information to make decisions. The aim of this study was to model how experienced occupational therapy driver assessors weight and combine information when making fitness-to-drive recommendations and establish their level of decision agreement.

Method

Using Social Judgment Theory method, this study examined how 45 experienced occupational therapy driver assessors from the UK, Australia and New Zealand made fitness-to-drive recommendations for a series of 64 case scenarios. Participants completed the task on a dedicated website, and data were analysed using discriminant function analysis and an intraclass correlation coefficient.

Results

Accounting for 87% of the variance, the cues central to the fitness-to-drive recommendations made by assessors are the client’s physical skills, cognitive and perceptual skills, road law craft skills, vehicle handling skills and the number of driving instructor interventions. Agreement (consensus) between fitness-to-drive recommendations was very high: intraclass correlation coefficient = .97, 95% confidence interval .96–.98).

Conclusion

Findings can be used by both experienced and novice driver assessors to reflect on and strengthen the fitness-to-drive recommendations made to clients.

Keywords

Automobile driving decision-making cue use driver assessment

Introduction

Categorised as an instrumental activity of daily living (IADL), driving is an important and valued aspect of community mobility for many individuals. While driving affords independence, mobility and freedom (Korteling and Kaptein, 1996), health conditions and disability can impact on driving capacity. Many older people experience age-related health declines, increasing numbers of medical conditions and an increasing need for medication, all of which can impair driving ability. Many young adults with disabilities are also keen to enjoy the freedoms associated with driving and are increasingly seeking advice on whether they too can drive (Unsworth, Pallant, Russell, et al., 2011). In addition there are drivers with a newly acquired illness or disability, for example resulting from a stroke or deteriorating health condition such as multiple sclerosis, who also require occupational therapy driving assessment and rehabilitation in order to be able to continue driving safely or to know when they need to cease driving (DoH, 2012). Occupational therapists are paying increasing attention to this activity to ensure their clients have maximum opportunity to drive if this is a priority for the client, and to determine whether their clients have the necessary skills to perform this complex activity.

Assessment and advice to support safe driving is required to protect clients as well as other road users. Occupational therapists are well positioned to assess the fitness-to-drive of individuals, with health- or age-related disabilities, that have been medically cleared by a physician (Unsworth et al., 2011). These occupational therapists often have specialist training or qualifications and are referred to in the USA as certified driver rehabilitation specialists, or in Australia as occupational therapy driver assessors. The recommendations made by occupational therapy driver assessors concerning fitness-to-drive are crucial, given the role they may play in preventing road trauma, and the major implications that recommendations for licence cancellation or suspension have on a person’s lifestyle and possible need for family and community supports (Ralston et al., 2001). It is therefore not surprising that driving and community mobility have featured in several recent occupational therapy special issues, such as Canadian (Polgar, 2011), American (Classen, 2010) and Australian (Unsworth, 2012) journals.

When formulating a fitness-to-drive recommendation (and recommendations for the need for any future reassessments), occupational therapy driver assessors generally conduct an off-road (clinic) assessment as well as an on-road (in car) assessment with a driving instructor in a dual-controlled vehicle. Occupational therapy driver assessors may use a fully comprehensive off-road standardised assessment such as the OT-DORA Battery (Unsworth et al., 2011, 2012), or compile their own assessment and incorporate a variety of sub tests such as the Rookwood Battery (McKenna et al., 2004) or the Montreal Cognitive Assessment (Nasreddine et al., 2005). While recent research has led to a more standard approach to conducting on-road assessments (Kay et al., 2008), there is not, and may never be, a standardised on-road assessment. This is due to the unique constellation of variables that present each time an individual drives.

When an occupational therapy driver assessor uses and combines information to make a fitness-to-drive recommendation (which is then reported to the licensing authority), they are applying their own professional ‘judgment policy’ (Cooksey, 1996). Studies that try to identify these judgment policies, by examining how information is weighted in the decision-making process, are generally known as ‘policy capturing’ studies, and are conducted within the framework of Social Judgment Theory (SJT) (Hammond et al., 1980). It is not clear what judgment policies or recommendations occupational therapy driver assessors are using when making fitness-to-drive decisions, and whether these are optimal. Unsworth has begun to explore the process of making fitness-to-drive recommendations (Unsworth, 2007); however, a policy-capturing study is urgently required.

We know from over 40 years of judgment research in fields such as psychology and medicine, and our own work (Davies et al., 2011; Harries and Gilhooly, 2010; Unsworth, 2001), that decision-makers are inconsistent in their judgments, often have limited insight as to how they make judgments and disagree over judgments made (Shanteau et al., 1999). However, we also know it is possible to statistically model how decisions are made, identify the optimal judgment policies that produce these decisions and use these to improve decision-making capacity. For example, Harries et al. (2012) demonstrated that a decision training aid was successful in improving the capacity of novice occupational therapists when making decisions of referral priority. Harries et al. incorporated a previously developed expert model of decision-making in the decision training aid. They used a randomised controlled trial to demonstrate that students who used the decision training aid were more able to prioritise referrals post training, and sustain that capacity long term, compared with their untrained contemporaries.

SJT attempts to understand the relationship between the judge and the natural decision environment by the principle of probabilistic functionalism (Cooksey, 1996). SJT is ideal for use in decision-making studies where the optimal judgment is not known and where there are real-world consequences when errors are made. Hence, this approach is ideal in studying fitness-to-drive decisions as there are no ‘benchmarks’ or ‘gold standards’ to determine whether a correct decision has been made, yet errors in a clinician’s judgment could mean an individual is deprived of the independence driving affords, or may be involved in a crash. SJT is a quantitative approach that uses statistical methods to describe the relationship between the information available and an individual’s judgment. When occupational therapists make clinical decisions, they weight information, or ‘cues’, which they assign differing levels of importance. In SJT, these weights can be modelled by asking therapists to make a large number of decisions on a series of cases in which the cue values are varied. The weights are then determined statistically using such approaches as regression analysis or discriminant function analysis (DFA). The resulting decision-making model allows for the identification of individual differences in policies as well as help determine an overall decision policy (Cooksey, 1996). The findings of SJT studies provide a platform from which judgment policies can be examined and debated.

In addition to examining the fitness-to-drive decisions themselves, using an SJT approach also enables researchers to examine the level of agreement between judges. This indicates whether there is consensus in the field and has implications for equitable assessment of clients. For example, Harries and Gilhooly (2003) examined the agreement between 40 occupational therapists who made decisions about the referral needs of 90 clients with mental health problems. It was found that agreement between the decisions made by the therapists was rather low at .4 (using Kendal’s coefficient of concordance). Agreement was only slightly better between 13 teams making recommendations for the place where 50 clients would be discharged following rehabilitation using a seven-level decision (intra correlation coefficient (ICC) = .67, 95% confidence interval (CI) .58–.76) (Unsworth et al., 1995). This suggests that it is common for there to be disagreement between clinicians and teams on important decisions, although the types of decisions with limited numbers of decision options may reduce this.

Internationally, fitness-to-drive recommendations are being made on a daily basis by occupational therapy driver assessors, and we need to research these decisions to see how they are currently made. We can debate the optimal judgment policy needed to guide decision-making (that is, how information needs to be used and combined). Expert consensus standards for practice need to be identified in order to facilitate evidence-based education in driving assessment, share best practice internationally and build workforce capacity (Harries and Unsworth, 2013).

The study reported in this paper forms part of a research programme that aimed to model experienced occupational therapy driver assessors’ fitness-to-drive decisions, and develop and determine the effectiveness of a training package to teach these skills to novices. This paper reports results from the first phase of this programme. The research questions were: (1) what fitness-to-drive recommendations (fit, not fit) are made by a sample of experienced occupational therapy driver assessors for a series of written case scenarios? (2) How do occupational therapy driver assessors weight (determine the relative importance) different types of information when making fitness-to-drive recommendations for older people and people with disabilities? (3) Do experienced occupational therapy driver assessors agree (that is, have a good degree of consensus) about the fitness-to-drive decisions made?

Method

Sample

Occupational therapists were required to have experience, as an occupational therapy driver assessor, of assessing 30 or more clients in order to participate. It was calculated that 65 experienced occupational therapy driver assessors needed to be recruited to statistically model the judgment policies of the experienced occupational therapy driver assessors (Cooksey, 1996).

Recruitment

Occupational therapy driver assessors from the UK and Australasia were invited to participate, as these two locations have well developed approaches to the assessment of fitness-to-drive, as well as a substantial number of experienced occupational therapy driver assessors. It is acknowledged that North America also has well developed expertise in this field (Pellerito, 2006) and as such would be an appropriate location for recruitment in the future. The occupational therapy driver assessors were invited to participate from publically available lists, as well as driver education course lists from the researchers’ universities, where permission to recruit for research purposes had previously been granted by individual occupational therapy driver assessors. A total of 208 occupational therapy driver assessor names were compiled and an email invitation, along with a participant information sheet, was sent to potential participants.

Selection of informational cues and development of cue levels

In order to develop case scenarios to generate fitness-to-drive recommendations, the informational cues to be reported in the cases needed to be determined, and then levels for each cue generated. Fitness-to-drive is based on client skills and abilities, and the potential for these to improve or deteriorate. Therefore, the cases were designed to describe client functional status rather than diagnosis. However, the cases constructed can readily be seen to represent clients who have experienced stroke, head injury or an orthopaedic problem, or a person who has dementia. To maximise the validity of the cues selected for use in this research, they were drawn from Unsworth’s study (2007) on information use in fitness-to-drive decision-making in Australia. As part of this research, 56 occupational therapy driver assessors ranked the relative importance of 12 cues that had previously been identified through rigorous literature searches. The highest to lowest mean ranking of the cues was as follows: driving instructor intervention, driver behaviour, cognitive and perceptual skills, vehicle handling skills, road law/road craft knowledge, physical skills, sensory functions, medical prognosis, current driving needs, driving experience and history, residence and age. In a typical SJT study, the top key cues would be incorporated into the cases. In this study, these cues were not restricted in the first instance as Unsworth (2007) reported that there was a wide degree of variability between the occupational therapy driver assessors in terms of their subjective cue rankings.

These 12 cues were reviewed for international face validity by the research team and two occupational therapy driver assessors each from UK and Australia who were members of the project advisory panel. Based on review team feedback, a decision was taken to separate the cue ‘driving experience and history’ into the two distinct components of experience and history. Driving history, in terms of the number of accidents an individual has had in the last 12 months, is distinct from how long an individual has been driving as it may indicate a sudden decline in skills despite how long they have been a driver. It was also decided to remove the cue of ‘residence’, meaning the type of location they lived in, such as urban or rural, based on the fact that the cue levels could not be structured in a way that was meaningful to both UK and Australian occupational therapy driver assessors. Additionally, given the balanced cue presentation in the case sampling approach to be adopted in the research (a fractional factorial design) (Gunst and Mason, 2009), an equal proportion of clients living in rural areas versus urban would not seem realistic. The impact of not including ‘residence’ as a cue was viewed to be minimal given that it was given the lowest ranking of the 12 cues in research by Unsworth (2007). A written definition was produced for the final 12 cues, and three distinct levels of each cue were established to represent a range of case presentations, with level 3 being the most positive, representing the lowest impact on fitness-to-drive. These were again reviewed and revised by the research team and project advisory panel and agreed as valid. The final cues and their levels are presented in Table 1.

Table 1.

Cues and their levels for guiding the recommendation of an individual’s fitness-to-drive.

Cue	Description of cue	Cue levels
1. Age	Client’s chronological age.	1 = 80 years old 2 = 60 years old 3 = 40 years old
2. Driving experience	As well as number of years as a driver, driving experience should also take into account whether the client has had a recent gap in their driving career and why it occurred.	1 = Client has been driving for less than 3 years 2 = Client has been driving 3–7 years 3 = Client has been driving for more than 7 years
3. Driving history	Driving history can include the number of reported accidents, the types of vehicles driven and weather conditions driven in. Where recall is an issue, a family member has verified this information.	1 = Client has had a major accident in the last 12 months 2 = Client has had a few minor scrapes in the last 12 months 3 = Client has had no accidents in the last 12 months
4. Current driving needs	A client’s personal driving needs include where they tend to drive (for instance, locally on familiar roads, versus in unfamiliar areas). Driving needs can also refer to how often the client drives and the time of day, as well as if they tend to drive on their own or with others present.	1 = Client drives predominantly in unfamiliar areas 2 = Client drives predominantly in the local area with only occasional trips to unfamiliar areas 3 = Client drives predominantly in the local/familiar area
5. Physical skills	Includes the driver’s muscle strength, endurance, tone, grip strength and range of movement, and the driver’s psychomotor reaction time. Vehicle modifications or compensatory strategies may be possible to aid specific physical skills or alleviate symptoms of fatigue and pain. Dependent on the specific issues these could include changing to power assisted steering or using hand controls.	1 = Physical skills do not support safe driving (no vehicle modifications/compensatory strategies suitable) 2 = Minor problems with physical skills noted, even with vehicle modifications/compensatory strategies made 3 = Physical skills support safe driving
6. Cognitive and/ or perceptual skills	Cognitive skills include concentration, memory, planning and metacognitive ability such as insight into own limitations. Perceptual skills include visuospatial ability. This may be observed through failing to check mirror, not signalling before turning and poor negotiation of intersections/junctions. Vehicle modifications may be possible to aid specific perceptual skills. Dependent on the specific issues these could include adding a panoramic mirror to help the client to check the mirror more frequently.	1 = Cognitive and/or perceptual skills do not support safe driving and does not demonstrate capacity for learning and improvement 2 = Minor cognitive and/or perceptual problems identified but demonstrates capacity for learning and improvement 3 = Cognitive and/or perceptual skills support safe driving
7. Sensory functions	Includes tactile sensation, proprioception (awareness of position in space) and vision.	1 = Sensory functions do not support safe driving 2 = Some sensory problems noted but meets legal requirements 3 = Sensory functions support safe driving
8. Driver behaviour	This refers to psychosocial behaviour. Behaviours of concern include impulsivity, disinhibition, risk taking, aggression or poor frustration tolerance.	1 = Behaviour shown does not support safe driving 2 = Some behaviour problems identified 3 = Behaviour shown supports safe driving
9. Road law knowledge and/or road craft	Road law knowledge involves applying road laws while driving. Road craft includes an understanding of how the car will respond in specific situations (such as knowing that a car may skid if the wheels go from tarmac to gravel at speed). This also requires adjusting driving to match the demands of the situation (for instance, slowing down when driving in a residential area in case pedestrians walk into the road).	1 = Road law knowledge and/or road craft does not support safe driving 2 = Some problems with road law knowledge and/or road craft identified 3 = Road law knowledge and/or road craft support safe driving
10. Vehicle handling skills	These skills include managing the steering ability, braking speed and car controls such as pedal use.	1 = Vehicle handling does not support safe driving 2 = Some vehicle handling problems identified 3 = Vehicle handling supports safe driving
11. Driving instructor interventions	Verbal prompts may include a reminder to use the indicator, or to adjust the distance from the car in front. Physical interventions are as a result of serious driver errors requiring the driving instructor to take control of the vehicle to maintain safety. This could include braking or reaching over to steer the car.	1 = Driving instructor provides one physical intervention 2 = Driving instructor provides a verbal prompt 3 = No physical or verbal interventions made
12. Medical prognosis	Knowledge about whether the driver’s medical condition is stable or may deteriorate, as may occur with a diagnosis of dementia, multiple sclerosis, Huntington’s disease, Parkinson’s disease and motor neurone disease. The medical condition has been confirmed by the doctor.	1 = Deterioration expected, likely to impair safe driving 2 = Deterioration not expected but possible 3 = Medically stable

The dependent variable in this research was the judgment of whether the client is fit to drive or not. Unsworth (2007) outlined four possible recommendations that occupational therapy driver assessors in Victoria, Australia, can make in relation to an individual’s fitness-to-drive. These formed the basis of the outcome variables for each case referral as follows: ‘Fit-to-drive – Unrestricted licence’; ‘Fit-to-drive – With conditions’, for example, using an automatic car; ‘Not fit-to-drive – Driver rehabilitation to be completed’ (may require reassessment); and ‘Not fit-to-drive – Suspend or cancel licence’. Based on feedback from the project advisory panel, it was also decided that an additional dependent variable should be measured in response to each referral, this being the likelihood of future reassessment being required. Results for this aspect of the study are reported separately as it is beyond the scope of this paper to report these findings.

Development of case scenarios

The case scenarios used in this research were developed through fractional factorial design, which helps reduce the number of experimental conditions, as outlined by Cooksey (1996). A total of 81 original cases were developed, but it was established during piloting that once repeated cases had been added to measure consistency, the task became too labour intensive for participants due to the time it took to complete in full. As such it was necessary to reduce the case set. The case set was reviewed by two experienced occupational therapy driver assessors from the project advisory board, to remove the instances that were not reflective of what would reasonably be encountered in practice. Cases were removed resulting in a set of 64 cases. The research team also reviewed these cases and made a fitness-to-drive judgment for each to determine whether the method was likely to produce a range of outcomes. The research team rated eight cases as potentially ‘Fit-to-drive – Unrestricted licence’, 48 cases as ‘Fit-to-drive – With conditions’ or ‘Not fit-to-drive – Driver rehabilitation to be completed’ and eight cases potentially rated as ‘Not fit-to-drive – Suspend or cancel licence’. A case set of 64 is within the 1:5 to 1:10 range recommended by Cooksey (1996) when establishing the ratio of informational cues to case scenarios. The complete set of case scenarios was reviewed by the research team and project advisory panel, and agreed as a valid set of cases which could be presented in practice. The correlation between the 12 cues across the 64 cases was calculated using Kendall’s tau, and ranged from r = −.15 to r = .22. The set of case scenarios needed to be designed to minimise the intercue correlations so that the effect of the outcome decisions could be measured. This level of intercue correlations was sufficiently low to allow for the case scenarios to be used in the research.

A total of 22 repeated cases were also added to measure how consistent occupational therapy driver assessors were in their recommendations regarding fitness-to-drive. Cooksey (1996) recommends that at least one-third of case profiles be repeated in order to measure consistency, and so approximately every third of the base set of cases was selected for this purpose. A total of 86 cases was therefore provided to the participants. The case order was randomised for each participant, to counter any order effects. This number of case scenarios provided an appropriate balance between task demands and ensuring an adequate number of cases to enable stable data analysis. Since it is beyond the scope of this paper to report the results of the consistency analyses with these additional 22 cases, this paper presents the results for the 64 original cases. Figure 1 provides an example of a complete case scenario. The 86 case scenarios were then loaded onto a dedicated web platform for easy access by the occupational therapy driver assessor participants.

Figure 1.

Example of a case study to elicit experienced occupational therapy driver assessors’ recommendations for client fitness-to-drive.

Procedure

Ethical approval for the research was sought and granted from Brunel University London and La Trobe University. Email invitations were sent to all occupational therapy driver assessors on the lists generated through the universities and those publicly available. Occupational therapy driver assessors accepting the invitation were then sent passwords to access the case scenarios on the dedicated website. Consent was implied if participants chose to access and complete the task. Confidentiality was assured to protect both individual participants’ identity and their place of work. Participants took approximately 60 minutes to complete the task, and on completion were sent a £20/$30 Amazon gift voucher to thank them for their time. Data collection ran over a 3-month period from September 2013 to November 2013 inclusive.

Data analysis

Data were entered into SPSS 20 (IBM 2011). Data analysis was conducted on the individual case scenario judgments made by the participants (N = 2880; 64 × 45). Analysis of the repeated cases (n = 22) will be undertaken in future reports on judgment inconsistency. To answer the first question of what fitness-to-drive recommendations are made, descriptive analyses were undertaken to document the number of times each judgment recommendation (fitness-to-drive) was selected by the occupational therapy driver assessors. To explore the second question of how occupational therapy driver assessors weight different types of information, we undertook a direct entry DFA to develop an overall model of how experienced occupational therapy driver assessors use and weight information to arrive at fitness-to-drive recommendations. DFA was undertaken rather than multiple regression, as the dependent variable (recommendation of fitness-to-drive) was ordinal in nature. As part of the DFA, classification expected by chance was calculated to determine the extent to which the model then improved classification. To answer the final question of whether occupational therapy driver assessors agree concerning the decisions made, we examined the level of agreement between the fitness-to-drive recommendations made by the occupational therapy driver assessors (n = 45) using an ICC (Type 2,1) (Shrout and Fleiss, 1979).

Results

A total of 15 occupational therapy driver assessors from the UK and a further 40 from Australasia submitted data for analysis. A response rate for participating in the study was difficult to determine since many occupational therapy driver assessors who were sent an invitation to join the study were no longer working, on leave, had moved workplaces or had not yet conducted 30 assessments. Following exclusion of incomplete data sets, data from a total of 12 UK and 33 Australasian occupational therapy driver assessors were analysed. The mean age of the participants was 44 years (SD 7.5). The sample was predominantly female (n = 41, 91%). The mean number of years participants had worked as an occupational therapist was 21 (SD 8.5), with a mean of 11 (SD 7.2) years’ experience working as a driving assessor. Table 2 provides the recommendations for fitness-to-drive made by the 45 occupational therapy driver assessors across each of the 64 case scenarios. Most of the scenario clients were recommended to undergo rehabilitation (53%), with 27% able to resume driving.

Table 2.

Fitness-to-drive recommendations for 64 case study clients made by 45 experienced occupational therapy driver assessors, together with functions at group centroids from the discriminant function analysis (mean variate scores).

Fitness to drive recommendations		Functions at group centroids
	N (%)	Function 1^a	Function 2^b	Function 3^c
Not fit-to-drive – Suspend or cancel licence	569 (20%)	−1.075	−.282	.109
Not fit-to-drive – Driver rehabilitation to be completed	1529 (53%)	−.121	.230	−.038
Fit-to-drive – With conditions	415 (14%)	.619	−.431	−.187
Fit-to-drive – Unrestricted licence	367 (13%)	1.472	−.035	.200
Total (100%)	2880

Function 1: Discriminated clients who were Fit-to-drive from Not fit-to-drive.

Function 2: Discriminated clients who were Not fit-to-drive – Require rehab from the other three outcomes.

Function 3: Discriminated clients who were Fit-to-drive or Not fit-to-drive from the middle two outcomes of Fit-to-drive – With conditions, or Not fit-to-drive – Require rehab.

A DFA was performed to model how the occupational therapy driver assessors weighted information when formulating recommendations concerning fitness-to-drive. The DFA showed three discriminant functions. Function 1 explained 87.1% of the variance (canonical R²= .36), and therefore accounts for most of the variability. Functions 2 and 3 explained 10.9% (canonical R²= .07) and 2% (canonical R²= .01) of the variance, respectively. Each of the three functions was able to significantly differentiate fitness-to-drive recommendations: functions 1 to 3 Λ = .59, χ²(36) = 1525.47, p = < .001; functions 2 to 3 Λ = .92, χ²(22) = 234.37, p = < .001; and function 3 Λ = .99, χ²(10) = 37.82, p = < .001. Considering the model as a whole, by chance, an occupational therapy driver assessor would ‘correctly’ classify clients 36% of the time, based on the prior probabilities. The three functions improve this classification to 53%. This is considered acceptable for SJT studies (Cooksey, 2006) and stable and consistent within DFA (Tabachnick and Fidell, 1996).

The functions at group centroids from the DFA as presented in Table 2 provide a numerical descriptor of how the three functions distinguish the four fitness-to-drive recommendations. In Table 3, the cues that differentiate each function are highlighted, that is, which cues had the strongest influence. The minimal effect sizes of functions 2 and 3 mean that consideration of the most strongly correlated cues is more useful for interpretation than the strongest correlation between each of the 12 cues in turn and the respective functions.

Table 3.

The three functions produced by the discriminant function analysis, showing the correlations between cues and the fitness-to-drive recommendation (structure matrix).

	Functions
Cues	1 Discriminated clients who were Fit-to-drive from Not fit-to-drive	2 Discriminated clients who were Not fit-to-drive – Require rehabilitation from the other three outcomes	3 Discriminated clients who were Fit-to-drive or Not fit-to-drive from the middle two outcomes of Fit-to-drive – With conditions, or Not fit-to-drive – Require rehabilitation
Physical skills	.48^a	.43	.07
Instructor interventions	.44^a	−.38	−.42
Road law/road craft	.39^a	−.03	−.03
Cognitive and perceptual skills	.39^a	.37	−.09
Sensory functions	.33^a	.07	.20
Driving experience	.14^a	.05	.06
Medical prognosis	.12	.48^a	−.27
Driving need	.07	−.37^a	.10
Driving history	.04	−.19^a	.10
Vehicle handling skills	.39	−.29	.53^a
Age	.07	.15	.41^a
Driver behaviour	.32	−.08	−.38^a

Differentiation correlations highlighted.

Largest absolute correlation between each cue and any discriminant function.

The first function, explaining the greatest share of the variance, discriminated clients classified in either of the ‘Not fit-to-drive’ recommendation groups from those classified in either of the ‘Fit-to-drive’ recommendation groups. The ‘Not fit-to-drive’ recommendations were negatively weighted, whereas the ‘Fit-to-drive’ recommendations were positively weighted, therefore illustrating this judgment distinction (see Table 2). The loading matrix of the correlations between the informational cues and discriminant functions (see Table 3) shows that function 1 was differentiated by physical skills (r = .48), the number of driving instructor interventions (r = .44), cognitive and perceptual skills (r = .39), road law/craft skills (r = .39) and vehicle handling skills (r = .39).

The second function discriminated clients who were classified as ‘Not fit-to-drive – Rehabilitation to be completed’ (given a positive weighting) from the other driving recommendations which were negatively weighted (see Table 2). Function 2 was differentiated by medical prognosis (r = .48), physical skills (r = .43) and driving instructor interventions (r = –.38) (see Table 3).

Cue	Client cue level
Age	60 years old
Driving experience	Client has been driving 3–7 years
Driving history	Client has had a few minor scrapes in the last 12 months
Current driving needs	Client drives predominantly in the local/familiar area
Physical skills	Physical skills support safe driving
Cognitive and/or perceptual skills	Minor cognitive and/or perceptual problems identified but demonstrates capacity for learning and improvement
Sensory functions	Sensory functions support safe driving
Driver behaviour	Some behaviour problems identified
Road law knowledge and/or road craft	Road law knowledge and/or road craft support safe driving
Vehicle handling skills	Vehicle handling supports safe driving
Driving instructor interventions	Driving instructor provides one physical intervention
Medical prognosis	Medically stable

Lastly, the third function discriminated clients who were clearly either ‘Fit-to-drive’ or ‘Not fit-to-drive’ (given positive weightings) from the middling recommendation options, where clients either require rehabilitation (‘Not fit-to-drive – rehabilitation to be completed’) or were classified as fit-to-drive, but with licence conditions; these recommendations were negatively weighted (see Table 2). Function 3 was differentiated by vehicle handling skills (r = .53) and driving instructor interventions (r = −.42) (see Table 3).

Finally, agreement between the occupational therapy driver assessors’ recommendations for fitness-to-drive was calculated. Across the 45 participants, agreement was very high at ICC = .97 (95% CI .96–.98).

Discussion

Fitness-to-drive recommendations made by occupational therapy driver assessors

To answer the first question, Table 2 was constructed to show the range of fitness-to-drive (fit, not fit) recommendations made by a sample of 45 experienced occupational therapy driver assessors for 64 written case scenarios. Overall, 27% of drivers were recommended as fit-to-drive. This is a similar proportion to the number proposed by the research team as being fit-to-drive either with or without conditions. The most popular recommendation made was ‘Not fit-to-drive – Driver rehabilitation recommended’ (53%). While the most common form of rehabilitation may be for an occupational therapy driver assessor to develop an on-road training programme with a driver instructor, a recent systematic review (Unsworth and Baker, 2014) located very little evidence to support the effectiveness of driver rehabilitation activities such as off-road training or practice on a computer-based driving simulator. While anecdotal evidence suggests that remediation of many skills and behaviours may be possible through rehabilitation programmes, further research to demonstrate the efficacy of these types of interventions is urgently needed if these services are to be promulgated and funded.

Cues used by occupational therapy driver assessors to determine fitness-to-drive

To answer the second question, the DFA analysis enabled us to identify combinations of predictor cues for fitness-to-drive recommendations (the discriminant functions). We now know which cues can be used to separate clients who are not fit-to-drive from those who are fit-to-drive by understanding their capacities in relation to: physical skills, the number of driving instructor interventions, cognitive and perceptual skills, road law/road craft skills and vehicle handling skills. These cues were highlighted as influential in function 1, which accounts for most of the variance (87%), suggesting that most importance can be placed on these when modelling how recommendations are made. These five cues also match very closely to the top five cues that therapists believe to be most important when making fitness-to-drive decisions as identified in a previous study (Unsworth, 2007), with the exception of the cue ‘driver behaviour’. While therapists believed ‘driver behaviour’ to be important, and ranked it second highest (Unsworth, 2007), this was not borne out in the current study. This may be because driver behaviour is most likely to impact on fitness-to-drive only when it is an extreme problem (scored as level 1), and occupational therapy driver assessors may have considered this behaviour as something that was possible to change through rehabilitation. The fact that the cues used by occupational therapy driver assessors in this study matched so closely to those in Unsworth’s previous study (with a small portion of both samples believed to overlap) suggests that occupational therapy driver assessors are generally insightful about their use of information to formulate decisions. This is rather unusual as clinicians often lack this perception, as shown by Harries and Gilhooly (2003) who found clinician insight into their referral prioritisation for clients with mental health needs to be only moderate (around r = .6), and higher for cues that were most important and least important in formulating referrals.

When determining which clients would benefit from rehabilitation, three cues were given most attention in our study. The client’s medical prognosis was the key factor; if a client had a stable medical condition they were more likely to be offered rehabilitation, as opposed to those with a deteriorating condition. However, while prognosis may be a key consideration in whether or not clients can take advantage of rehabilitation, it is often acknowledged that predicting outcome for clients does require experience and expertise (O’Sullivan, 2014), so training novices to use this cue may present challenges for us in the future. Also considered to be important was the client’s physical capacity. Clients identified as having sufficient physical skills to support safe driving were more likely to be offered rehabilitation services than those who had limited skills, even when car adaptations and compensatory techniques had been trialled. The occupational therapy driver assessors were being asked to make a fitness-to-drive recommendation in cases where adaptations had not facilitated driving capacity, but it is acknowledged that, in practice, adaptations can be made to compensate for most physical disabilities (Rica, 2012). The third case feature given attention was related to whether the instructor had to intervene during the on-road assessment. If the instructor had intervened, the client was more likely to be offered rehabilitation services. We do know that in many licensing jurisdictions, occupational therapy driver assessors are required to fail clients if a physical on-road intervention is required. For example, in Victoria, Australia, this occurrence should produce an automatic fail decision as documented in the Competency Standards (OT Australia–Victoria, 1998). However, it is also known that this ruling is not always applied. For example, Caust (2010) reported, that in 10% of cases where an instructor intervention was required, occupational therapy driver assessors were still passing clients.

Finally, we can predict the driving outcomes for clients who were clearly either ‘Fit-to-drive’ or ‘Not fit-to-drive’ from the two middle recommendation options, where clients either require rehabilitation (‘Not fit-to-drive – Rehabilitation to be completed’) or who were classified as fit-to-drive, but with licence conditions, by knowing their level of vehicle handling skills and again, the presence of driving instructor interventions. It is interesting to consider the role of vehicle handling skills in the decision-making process. In this instance, where vehicle handling skills supported safe driving (for example, managing the steering ability, braking speed and car controls such as pedal use) occupational therapy driver assessors were more clearly able to determine whether licensing conditions or rehabilitation were appropriate. As vehicle handling skills affect driving capacity, it may be that performance can be improved by driving only in familiar areas, as well as driving during daylight hours and avoiding peak traffic. In Australasia, driving in specific times or locations can be recommended by occupational therapy driver assessors as a condition of the licence. These conditions have the benefit of allowing drivers to continue driving (thus maintaining their lifestyle), whereas in some countries, such as the UK, some of these conditions are not available. European law does allow these types of conditions to be used (European Union, 2006), and although some European countries have adopted them, the UK has not chosen to adopt most of them (CONSOL, 2013).

This means that some individuals would be allowed to drive with a conditional licence in areas of Europe, and yet would not be recommended as fit-to-drive in the UK, even though both are regulated by European law. This situation may limit an individual’s social inclusion and community mobility and perhaps needs revisiting in order to support optimal health and social outcomes for older drivers and those with disabilities. While the DFA reported in this research was conducted with the data set as a whole, future analyses will also be conducted to examine individual differences among occupational therapy driver assessors in the way they make recommendations and the consistency of these recommendations.

Agreement between the fitness-to-drive recommendations made by occupational therapy driver assessors

The final question related to whether experienced occupational therapy driver assessors could agree on fitness-to-drive recommendations for clients. We found that the agreement between the occupational therapy driver assessors was very high. This may have been due to the substantial level of experience held by the participant group. This bodes well for determining a consensus view of how fitness-to-drive recommendation should be undertaken. One reason the agreement was high may relate to the fact that there were only four choices for the outcome, rather than a visual analogue scale or a higher number of outcome choices. As reported in the introduction, studies using these types of outcomes have reported much lower levels of agreement among judges (Harries and Gilhooly, 2003; Unsworth et al., 1995). Further analysis of agreement between occupational therapy driver assessors’ recommendations for fitness-to-drive will need to be undertaken to determine whether there are any sub groups of occupational therapy driver assessors who used the cues in a similar way, but differently from others. Even if recommendations are well matched they may have arrived at their decisions using different cue sets. In addition the application of the judgment policy is susceptible to an individual’s cognitive control in terms of both linear and non-linear cue use (Cooksey, 1996). As mentioned, these investigations are planned and will be reported in due course.

Critical evaluation

Several limitations have already been described. In addition, it is also acknowledged that the sample of 45 occupational therapy driver assessors is small and the sample size could be increased in future studies. Nonetheless, a statistically stable DFA model was produced, and the high level of agreement suggests that our sample appears to have robust expertise in this domain. In addition, it must be noted that a DFA demands that all observations be independent of each other. In this research, while each observation was independent of others made by the same therapist, the observations may be viewed as clustered for each therapist. Future analyses may be conducted to tease out this issue further, possibly using a multi-level approach. The cues and their levels, as used in this research, were based on previous research in the area (Unsworth, 2007) and expert review by our project advisory panel. While this assures a high degree of validity, the reliability of the cue levels could be investigated in future research. Finally, further investigation into the effectiveness of driver rehabilitation programmes is urgently required. While driver rehabilitation continues to be a decision outcome for many clients, research is required to determine whether such programmes are sufficiently successful to warrant their use.

Implications for practice

Novice occupational therapy driver assessors will be able to incorporate the findings from this research in their daily practice. They can reflect on the most influential cues used by the experienced occupational therapy driver assessors when determining, in the first instance, whether a client is fit-to-drive or not, and then further consider the client’s medical prognosis when reflecting on whether driver rehabilitation might be beneficial to facilitate a return to driving. The research team will also debate the outcomes with the project advisory panel members to ensure that the consensus judgment policy follows broad practice guidelines, and then develop training materials for novice occupational therapists to provide clear guidance that can be used to train occupational therapy driver assessors how to differentiate and recommend that a client is or is not fit-to-drive. Our research team plans to test the effectiveness of these training materials in the future.

Conclusion

A total of 45 occupational therapy driver assessors, with an average of over 10 years’ experience in driver assessment, provided their fitness-to-drive recommendations for a large set of case scenario clients. Statistical modelling has shown that by attending to client performance in five key cues (physical skills, the number of driving instructor interventions, cognitive and perceptual skills, road law/craft skills and vehicle handling skills), distinctions between clients who are fit-to-drive and not fit-to-drive can potentially be determined. Rehabilitation is likely to be recommended if the client’s medical prognosis is relatively stable, there is minimal physical disability and the driving instructor intervened during testing. Recommendations around conditional licensing may also take account of any driving instructor interventions and the client’s vehicle handing skills. Recommendations made in this domain appear to demonstrate a strong consensus among occupational therapy driver assessors, which bodes well for providing a consensus judgment policy for use in training. Future research is planned to determine whether we can successfully educate novices to make fitness-to-drive recommendations in the same manner as experienced occupational therapy driver assessors, thus ensuring translation of this research into practice.

Key findings

We identified five key types of information that experienced occupational therapy driver assessors use when formulating fitness-to-drive recommendations.

What the study has added

Occupational therapy driving assessors are increasingly being asked to provide fitness-to-drive recommendations for clients with disabilities. This study provides evidence for how experienced occupational therapy driver assessors make fitness-to-drive recommendations, and this information will be used to develop training for novices to enhance their ability to make these important decisions.

Footnotes

Acknowledegments

We thank all the experienced occupational therapists who generously gave their time to participate in the study. We appreciate the expert assistance of Dr Chris Tomlinson, Imperial College London, in building the web platform to host the research task, and Dr Ken Gilhooly for expert statistical advice. Finally, many thanks to our project advisory board for their reviews and feedback on the research materials used in this study.

Research ethics

Ethics approval was provided by Brunel University London Ethics committee 13/10/STE/02 and La Trobe University Ethics Committee 12-105.

Declaration of conflicting interests

The authors declare that there is no conflict of interest.

Funding

This work was supported by the UK Occupational Therapy Research Foundation, Research Priority Grant scheme, 2012.

References

Caust S (2010) Results of OT Report Audit. VicRoads Medical Review Day Conference. Melbourne: author.

Classen

(2010) Editorial: Special issue on older driver safety and community mobility. American Journal of Occupational Therapy 64(2): 211–214.

CONSOL (2013) Driver licensing legislation. Available at: http://www.consolproject.eu/attachments/article/16/CONSOL%20Report_WP5.1_final.pdf (accessed 3 December 2013).

Cooksey

(1996) Judgement Analysis: Theory, Methods and Applications, London: Academic Press.

Davies

Harries

Cairns

(2011) Factors used in the detection of elder financial abuse: A judgement and decision-making study of social workers and their managers. International Social Work 54(3): 404–420.

DoH (2012) Allied Health Professions toolkits. Available at: http://www.networks.nhs.uk/nhs-networks/ahp-networks/ahp-qipp-toolkits (accessed 2 April 12).

European Union (2006) Directive 2006/126/EC. Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=CELEX:32006L0126:en:NOT (accessed 2 October 2013).

Gunst

Mason

(2009) Fractional factorial design. Wiley Interdisciplinary Reviews: Computational Statistics 1(2): 234–244.

Hammond

McClelland

Mumpower

(1980) Human Judgment and Decision Making: Theories, Methods and Procedures, New York: Praeger.

10.

Harries

Gilhooly

(2003) Generic and specialist occupational therapy casework in community mental health. British Journal of Occupational Therapy 66(3): 101–109.

11.

Harries

Gilhooly

(2010) Occupational therapists’ self-insight into their referral prioritization policies for clients with mental health needs. Australian Occupational Therapy Journal 57(6): 417–424.

12.

Harries

Unsworth

(2013) Editorial: Clinical assessment of fitness-to-drive: Positioning occupational therapy for workforce development. British Journal of Occupational Therapy 76(7): 299.

13.

Harries

Tomlinson

Notley

(2012) Effectiveness of a decision-training aid on referral prioritisation capacity: A randomized controlled trial. Journal of Medical Decision Making 32(6): 779–791.

14.

Kay

Bundy

Clemson

(2008) Validity and reliability of the on-road driving assessment with senior drivers. Accident Analysis and Prevention 40(2): 751–759.

15.

Korteling

Kaptein

(1996) Neuropsychological driving fitness tests for brain-damaged subjects. Archives of Physical Medicine and Rehabilitation 77(2): 138–146.

16.

McKenna

Jefferies

Dobson

(2004) The use of a cognitive battery to predict who will fail an on-road driving test. British Journal of Clinical Psychology 43(3): 325–336.

17.

Nasreddine

Phillips

Bedirian

(2005) The Montreal Cognitive Assessment (MOCA): A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society 53(4): 695–699.

18.

O’Sullivan

(2014) Clinical decision making. In: O’Sullivan

Schmitz

(eds) Physical Rehabilitation: Assessment and Treatment, 6th ed. Philadelphia: FA Davis1–29.

19.

Australia–Victoria

(1998) Competency Standards for Occupational Therapy Driver Assessors, Melbourne: OT Australia–Victoria.

20.

Pellerito

(2006) Driver Rehabilitation and Community Mobility, St Louis, MO: Elsevier Mosby.

21.

Polgar

(2011) Editorial: Enabling community mobility is an opportunity to practice social occupational therapy. Canadian Journal of Occupational Therapy 78(2): 67–69.

22.

Ralston

Bell

Mote

(2001) Giving up the car keys. Physical and Occupational Therapy in Geriatrics 19(4): 292–302.

23.

Rica (2012) Car Controls: A Guide for Older and Disabled People. Available at: www.rica.org.uk/node/496 (accessed 20 January 2014).

24.

Shanteau

Mellers

Schum

(1999) Decision Science and Technology: Reflections on the Contribution of Ward Edwards, Norwell, MA: Kluwer Academic Publishers.

25.

Shrout

Fleiss

(1979) Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 86(2): 420–428.

26.

Tabachnick

Fidell

(1996) Using Multivariate Statistics, 3rd ed. New York: HarperCollins.

27.

Unsworth

(2001) Selection for rehabilitation: Acute care discharge patterns of stroke and orthopaedic patients. International Journal of Rehabilitation Research 24(2): 103–114.

28.

Unsworth

(2007) Using social judgement theory to study occupational therapists’ use of information when making licensing recommendations to older and functionally impaired drivers. American Journal of Occupational Therapy 61(5): 493–502.

29.

Unsworth

(2012) Editorial: Community mobility promotes participation for people of all ages and ability. Australian Occupational Therapy Journal 59(1): 1.

30.

Unsworth

Baker

(2014) Driver rehabilitation: A systematic review of the types and effectiveness of interventions used by occupational therapists to improve on-road fitness-to-drive. Accident Analysis and Prevention 71: 106–114.

31.

Unsworth

Baker

Taitz

(2012) Development of a standardised Occupational Therapy Driver Off Road Assessment (OT-DORA) battery to assess older and/or functionally impaired drivers. Australian Occupational Therapy Journal 59(1): 23–36.

32.

Unsworth

Pallant

Russell

(2011) OT-DORA: Occupational Therapy Driver Off-Road Assessment Battery, Bethesda, MD: AOTA Press.

33.

Unsworth

Thomas

Greenwood

(1995) Rehabilitation team decisions concerning discharge housing for stroke patients. Archives of Physical Medicine and Rehabilitation 76(4): 331–340.