Utilizing nominal group technique to achieve consensus on a clinical decision-making tool for vocational rehabilitation at a grassroots level in South Africa

Abstract

Background

Barriers to the delivery of vocational rehabilitation have been reported as a lack of knowledge, skills and confidence in occupational therapists who deliver these services. A program aimed at overcoming these barriers was developed to provide training in the delivery of vocational rehabilitation services. The program includes three decision-making tools to facilitate not only knowledge translation but also practice changes. The tools are (1) Return-to-Work Decision Tree, (2) Performance-based Assessment Tool, (3) Job Match Tool.

Objective

The objective of the research was to obtain consensus on the acceptability and utility of the developed tools to facilitate clinical reasoning in the delivery of vocational rehabilitation services.

Methods

A nominal group technique was used to obtain consensus on the developed tools. During a second phase, participants were sent revised versions of the tools and asked to comment on the tools’ clinical utility using a four-point Likert scale.

Results

The Return-to-Work Decision Tree and the Performance-based Assessment Tool received a 100% agreement in terms of clinical utility with the Job Match Tool receiving a 90% agreement for clinical utility. Participants made valuable recommendations such as adding a user manual and a training workshop to facilitate implementation of the tools.

Conclusion

The three tools developed to support clinical reasoning and decision making during the vocational rehabilitation process were reported to be acceptable by the expert participants. Further development of the Job Match Tool is recommended.

Introduction

Work has been described as one of the most significant life roles of adults as it not only provides financial stability but also contributes to self-efficacy and meaning.^1,2 Therefore, when work participation is limited by injury or illness, many aspects of a person's life could be affected. Vocational rehabilitation (VR) is a service provided in the public health care sector in South Africa to address these limitations and to facilitate meaningful engagement in work activities. The public healthcare sector is government-funded and available to all South Africans, although it is mainly accessed by those who do not have private medical aid. This is estimated at more than 80% of the population since the latest statistics indicate that a mere 16.1% of South Africans have private medical aid coverage. ³ The public healthcare sector is divided into three levels: primary, secondary, and tertiary levels. Primary level hospitals are often community-based, and patients can receive basic medical care without referrals. ⁴ At secondary hospitals such as rehabilitation hospitals, patients are referred to providers with experience in treating a specific problem. ⁴ Tertiary hospitals are highly specialized, and patients can receive care from specialized health care providers who have access to various diagnostic and intervention resources. ⁴ VR services should be delivered at all levels, however, based on anecdotal information, clients in need of VR services are often referred to tertiary and some secondary facilities, leading to a bottleneck effect and long waiting times to access these services.

Gauteng Province has been reported to have the highest economic activity in South Africa, ³ and has a vast variety of industries. Historically, Gauteng's public health care settings had specialized units that focused on VR. At these units, occupational therapists provided support and advocacy to their clients to be employed in the labor markets of South Africa. This was well before the establishment of legislative support aimed at increased employment of people with disability. ⁵ Although the infrastructure of these units remained, redistribution of resources was necessary to meet the newly established needs of the population after the democracy was established in the 1990's. ⁵ This meant that designated VR posts were no longer feasible, and that VR services had to be absorbed into the caseloads of occupational therapists working in acute rehabilitation. ⁵

Abass and Soeker reported that between 2014 and 2020 the representation of persons with disability among those that were economically active in South Africa declined from 0.9% to 0.7%. ⁶ Given the extensive and affirmative legislation to promote and support increased employment of persons with disability in South Africa, this decline is concerning. ⁶ Furthermore, the lack of rehabilitation services beyond acute care has been identified as a barrier to economic empowerment and employment opportunities for people with disability in the National Development Plan, which is contained in the White Paper on the Rights of Persons with Disabilities (WPRPD). ⁷ Therefore, the need for more comprehensive VR services at a grassroots level in South Africa is evident. These services are vital in a country where economic and social barriers to work increasingly limit people with disabilities’ access to gainful employment.

Certain barriers to the delivery of VR services, especially at a grassroots level, have however been identified. Van Biljon ⁵ reported barriers that include low level of experience and skill as well as a lack of confidence of occupational therapists who deliver these services. ⁵ Furthermore, VR interventions have been reported as haphazard and ineffective due to the absence of defined procedures and protocols for the delivery of VR services. ⁸ The need is therefore evident for occupational therapists employed at a grassroots level in South Africa to employ increased clinical reasoning to deliver evidence-based, context specific VR services to clients across the continuum of care.

Using the logic model ⁹ and the Cultural-Historical Activity Theory (CHAT) ¹⁰ as a conceptual framework, a proposed program was developed to equip novice occupational therapists to provide VR services at a grassroots level in South Africa. This proposed program is aimed at empowering occupational therapists to consider and address work as an occupation at a grassroots level, and across the continuum of care. The focus is on clinical decision making and clinical reasoning, as opposed to determining work capacity as in a traditional functional capacity evaluation. The reason for this is that due to the structures of insurers and funders, these formal work assessments often occur only up to two years after illness or injury, during which time meaningful occupational therapy interventions could have contributed to regaining work ability. Logic models have been used extensively in health care to develop and evaluate programs. ¹¹ Generally, logic models include the following components: inputs, activities, outputs, outcomes and long-term impact. ¹¹ Logic models aim to link program inputs with expected outcomes, based on theoretical underpinnings. Figure 1 displays the logic model that was developed for the program, which includes training of novice occupational therapists to provide VR services at a grassroots level. Numerous authors have reported on the efficacy of provision of training on improving knowledge, skill and confidence of practitioners having to provide a specific service.^12,13 However, it has been found that training alone does not necessarily equip therapists to implement their newly acquired knowledge and skills to have changes in practice or service delivery. ¹² One of the assumptions of CHAT is that mediation occurs from certain tools or resources that are necessary to effect change in the activity system. ¹⁰ In other words, the provision of resources can aid in bringing on change in the activity of VR service delivery. Therefore, three distinct tools were developed to support clinical decision-making of novice occupational therapists having to deliver VR services at a grassroots level. These included a Return-to-Work Decision Tree, a Performance-based Assessment Tool, and a Job-Match Tool, to be used in the assessment and treatment of patients across the continuum of care, considering work as an area of occupation.

Figure 1.

Logic model.

As part of a larger research project done as part of a doctoral project at Boston University, the nominal group technique (NGT) was employed to achieve consensus on the proposed tools and to suggest revisions or modifications. NGT is a structured, in person method for group brainstorming aimed at obtaining consensus on a matter or idea. ¹⁴ It involves individuals generating ideas independently, followed by a round-robin sharing of ideas, group discussion for clarification, and individual ranking or voting to prioritize responses. ¹⁴ This technique is especially useful to obtain consensus in a manner that mitigates the effect of a dominant individual or lack of creativity when group discussions alone are used.

This paper presents the findings from the NGT, regarding the consensus on the acceptability of the three tools developed to support clinical decision-making in vocational rehabilitation service delivery.

Methods

The NGT was selected as the method to reach consensus of the three decision-making tools in VR. The advantages of using the NGT are that it aids in problem identification, supports the development of solutions, and helps establish priorities for action. Since the three tools are newly developed, the NGT assisted in the mutual clarification of issues. ¹⁴ The NGT is a structured method and guides the researcher through five steps to identify issues, provide suggestions for change and finally reach consensus. The first step is to introduce the topic and explain the process to the participants. The next step is the silent generation of ideas and ensuring that everyone has an opportunity to reflect without influence from others. The third step is sharing ideas in a round-robin format where participants take turns sharing one idea at a time, ensuring equal contribution. These ideas are listed on a visible medium for the group to see. Step four allows for group discussion and clarifying contributions from the participants and finally, step five is the voting and ranking of ideas and suggestions. ¹⁴

Fifteen (15) occupational therapists were purposively sampled to attend the nominal group technique based on the following inclusion criteria for the occupational therapist:

registered with the HPCSA.

at least 10 years of experience in occupational therapy, some of which should have included vocational rehabilitation.

completed a master's degree or published an article.

Of the 15 occupational therapists that were invited, four declined. Three therapists declined as they were not available on the day the data collection was to take place, and the fourth therapist declined since they had recently emigrated. Participants were provided with time to read through and sign the informed consent form with the principal investigator present to answer any questions or to provide any clarifications. They then proceeded to complete a demographics questionnaire.

In alignment with the NGT steps, the session started with an introduction and a presentation by the principal investigator on the problem to be addressed and the proposed program that was developed using the logic model. Core assumptions from CHAT were explained as well as the process of developing the three distinct tools. Hereafter each tool was described, and a handout of each tool was provided to the participants. The participants were given time to ask any questions on the tools for clarification purposes. From here, the NGT process was facilitated by an experienced facilitator (second author). The second step, silent generation of ideas, allowed for each participant to individually reflect on the posed instruction of critiquing the proposed tools as clinical reasoning tools to be used by novice occupational therapists at a grassroots level in South Africa. They were asked to write down their thoughts on small pieces of paper. During the third step, ideas were shared in a round-robin manner until all ideas had been shared and placed on a single board. Step four involved clarification and grouping of the ideas into different themes. During the final step, participants were asked to rank each of the 10 themes that emerged in order of priority. This was done by having each participant write a list of the themes and rank them in priority from 1–10, one being the highest priority. A point system was used where priority 1 was scored using 10 points, priority 2 using 9 points etc. for each priority for each participant. The rankings were transferred to an Excel spreadsheet, and the scores were added for each theme. Using the summed scores, each theme was provided with a rank, the first rank indicating the highest priority.

After the session, the principal investigator revised the proposed tools and elaborated on the necessary aspects in line with the ranked themes. The revisions made were based on suggestions made during the NGT process. Revisions with explanations of how the suggestions had been incorporated were sent electronically to all participants. Participants then answered specific questions related to the clinical utility of these tools using an online questionnaire developed using Microsoft Forms and a four-point Likert Scale.

Results

Eleven participants attended the workshop and engaged in collaborative discussions around the three presented tools. Two (n = 2) participants held bachelor's degrees, most participants (n = 6) held master's degrees, and three (n = 3) participants held doctorate degrees. All (n = 11) participants reported to work in the Gauteng province. Six (n = 6) participants worked in academia, four (n = 4) participants worked in the private sector, and one (n = 1) participant worked in the public sector. The years of experience in VR ranged from 10 years to 43 years and the average years of experience in VR was 16 years.

Ten themes emerged from the discussions that took place in steps two and three of the NGT process:

Competencies: The essential skills and knowledge that the participants felt novice occupational therapists would need before using these tools.

Essential considerations: These include considerations regarding the occupational profile of patients as well as facilitators and barriers that may not be as explicit during the return-to-work process which could either support or hinder successful return to work.

Suggestions to improve the Return-to-Work Decision Tree: Suggestions mainly focused on including the variable of unemployment prior to illness or injury, having more clearly defined steps and using consistent terminology.

Clarification of terms: Certain terms were perceived as unclear or ambiguous by the participants. It was advised that terms in line with the Occupational Therapy Practice Framework (OTPF-4) be used as far as possible. The use of legal terms was also cautioned against.

Pragmatic resources: Certain considerations were raised regarding the available resources required to use the tools, such as electronic devices and activities used for assessment.

Pragmatic format: Considerations were raised regarding the format. Some participants believed an electronic format may be more user-friendly, while others believed it would increase the resource strain.

Contextual considerations: Valuable feedback was provided in terms of including contextual considerations throughout the process and with all the tools. Specifically, how contextual factors would act as either facilitators or barriers to return to work.

Pragmatic process: Participants identified possible barriers in terms of the process, which is not always linear, and that the program and the tools should allow for variations in the process.

Suggestions to improve the Performance-based Assessment Tool: Suggestions were mainly centered around ease of use and assisting therapists to understand how to use the tool.

Suggestions to improve the Job Match Tool: Suggestions were made in terms of layout and terminology used in the tool, that would increase ease of use of the tool.

The themes were ranked individually in order of priority. The ranked results are presented in Table 1. Based on the ranked results, revisions were made to each of the tools during a two-month period after the NGT. During the NGT, the Return-to-Work Decision Tree received the most feedback, both positive feedback, and suggestions for change. Revisions made to the Return-to-Work Decision Tree included the following:

Including motivation at an earlier stage in the decision-tree and having a concluding result if motivation does not improve with intervention.

Adding consideration of individuals who may have been unemployed at the time of intervention as well as contextual factors that could either be barriers or facilitators during the return-to-work process.

Terminology was revised to ensure that terms were consistent with the OTPF-4 ¹⁵ and that legal terms were not included.

Table 1.

Ranked themes.

Option	1	2	3	4	5	6	7	8	9	10	11	TOTAL	RANK
Competencies	8	8	1	10	9	10	10	6	6	9	5	82	1
Essential considerations	7	9	9	9	10	9	9	10	7	2	1	82	1
Suggestions for decision tree	6	10	8	4	4	7	4	8	3	10	10	74	3
Taxonomy	9	7	10	5	1	1	6	9	8	3	4	63	4
Pragmatic resources	1	2	3	7	7	3	7	5	10	6	2	53	5
Pragmatic format	2	3	2	8	8	4	1	4	9	4	7	52	6
Context	10	5	7	1	5	8	5	2	4	1	3	51	7
Pragmatic process	3	1	4	6	6	2	8	3	5	5	8	51	7
Suggestions for Performance-based Assessment Tool	4	6	5	3	2	6	3	7	1	7	6	50	8
Suggestions for Job Match Tool	5	4	6	2	3	5	2	1	2	8	9	47	9

The Performance-based Assessment Tool received critique mainly regarding its ease of use, and that the instructions for using it were inadequate. Revisions included the following:

Clarification of terms and clear instructions were added to improve its ease of use and to reduce ambiguity.

A suggestion was made to have example activities that this tool could be used with. This will be included as a learning activity in the two-day workshop.

The Job Match Tool received the least feedback during the NGT. Feedback was mainly aimed at specific terminology, ordering ideas that could influence understanding and the length and complexity of the tool. Due to the limited feedback, fewer revisions were made during the revision phase. Moreover, revisions to this tool were ranked as the lowest priority during the first phase. Revisions included:

Moving concepts so that the user is not misled in terms of the layout of the tool. For example, in some instances it seemed like information placed vertically was grouped together instead of information that was placed horizontally, which was the intention.

Terms were clarified in the same manner as in the Return-to-Work Decision Tree.

Although the length and complexity received critique, it was not possible to reduce the information, and the length remained the same.

All participants (N = 11) provided feedback during the second round. Nine (n = 9) participants strongly agreed, and two participants (n = 2) agreed that the Return-to-Work Decision Tree is a clinically utile tool in the decision-making process of vocational rehabilitation. Figure 2 includes the revised Return-to-Work Decision Tree. Seven participants (n = 7) strongly agreed, and four participants (n = 4) agreed that the Performance-based Assessment Tool was clinically utile tool to facilitate decision making and reasoning in the intervention process. The Performance-based Assessment Tool is included in Figure 3. Ten participants responded to the question related to the clinical utility of the Job Match Tool, with four (n = 4) participants strongly agreeing and five (n = 5) participants agreeing that the Job Match Tool was clinically utile in the decision-making process during intervention. One participant (n = 1) disagreed that the Job Match Tool was clinically utile in facilitating decision making and reasoning in the intervention process. The Job Match Tool is not presented at this stage.

Figure 2.

Return-to-Work decision tree.

Figure 3.

Performance-based assessment tool.

Nine (n = 9) participants strongly agreed, and two participants (n = 2) agreed that the three tools would be supported by a training manual. Six (n = 6) participants strongly agreed, and four participants (n = 4) agreed that the tools are relevant for the South African context, with one participant (n = 1) disagreeing on this matter. Figure 4 summarizes the responses from participants. There was 100% agreement that the Return-to-Work Decision Tree and the Performance-based Assessment Tool were clinically utile and that the tools would be well supported by a training manual. There was 90% agreement that the Job Match Tool was clinically utile and that the tools were relevant to the South African context.

Figure 4.

Summary of second round responses.

Discussion

Certain barriers to the delivery of VR services at a grassroots level in South Africa had been reported.^5,8,16 While a large portion of the barriers are at a meso- and macro level related to institutional limitations, policies and organizational barriers, many of the micro-level barriers can be overcome by improved clinical reasoning. The three tools that were developed were developed specifically to support decision making and therefore clinical reasoning for novice occupational therapists. Studies have reported distinct differences between the reasoning process and content of novice and expert occupational therapists.^17,18 While it is recommended that reflexive learning and support from a mentor may be effective in developing these skills,^17,19 it remains difficult to achieve in environments where knowledge translation is limited by a high staff turnover. In two studies aimed at improving clinical reasoning skills for specific diagnostic populations, it was found that novice therapists’ clinical reasoning was supported by the use of tools.^18,20 Similarly, access to clinical practice guidelines was found to be a facilitator of the implementation of knowledge in practice. ²¹ Therefore, it is expected that the availability of the tools developed could improve clinical reasoning of novice occupational therapists during VR service delivery. The fact that expert occupational therapists found the tools to be useful and practical is expected to improve their inception into everyday practice. To the best of our knowledge, there are no other context specific tools aimed at supporting clinical decision making in the VR process.

The NGT process highlighted relevant considerations that contributed not only to the clinical utility but also the appropriateness of the tools developed specifically for the South African context. All participants agreed that the revised Return-to-Work Decision Tree was acceptable and clinically utile. Revisions included adding consideration of individuals who may have been unemployed at the time of intervention as well as contextual factors that could either be barriers or facilitators during the return-to-work process. The importance of contextual factors and participation in other life roles have been affirmed by a recent study that investigated the effects of contextual factors on return to work in a population of clients living with chronic pain. ²² Terminology was revised to ensure that terms were consistent with the OTPF-4 ¹⁵ and that legal terms were not included. The OTPF-4 has been reported to be included in the curricula of all eight universities of South Africa. ²³ It is therefore expected that the terminology included in the OTPF-4 would be commonly known in South Africa.

The Performance-based Assessment Tool was found acceptable and clinically utile by the expert participants. During the NGT this tool received critique mainly regarding its ease of use and that the instructions to use it were inadequate. Ease of use has been reported to be an important aspect of clinical utility. ²⁴ Therefore, revisions were made to clarify terms and possible ambiguity to improve its ease of use. In addition, the two-day workshop will include exercises where therapists can apply the tool using a variety of activities, in line with the prepositions of adult learning theory. ²⁵

The Job Match Tool was developed using personal experience, constructs from the Dictionary of Occupational Titles and the O*Net.^26,27 The Job Match Tool received the least feedback during the first round of consensus. Feedback was mainly aimed at specific terminology, ordering of ideas that could influence understanding and its length and complexity. Due to the limited feedback, fewer revisions were made during the revision phase. Moreover, revisions to this tool were ranked as the lowest priority during the first phase. However, this is the only of the three tools that did not receive 100% agreement in terms of clinical utility. Although a 90% agreement remains acceptable, this is indicative that further investigation and validation of this tool may be valuable. A possible reason for this tool to receive less feedback may include that other means of job analysis may be considered more effective. However, to the best of our knowledge, available tools are not necessarily context specific or are intended for specific diagnostic groups. Since job matching has been identified as a complex decision-making process, ²⁸ it is possible that the experts have become well versed in performing this task, and the importance of such a tool may not have been identified by the experts. Alternatively, the complexity and extent of content in this tool may have required additional time and discussions to truly evaluate the content and constructs of the tool. Discussions with a circle of advisors were held during the development of this tool and the initial feedback had all been positive. Job matching is a process that should be client centered, data driven and consistent, ²⁸ the use of a consistent data-driven tool may remain valuable. Therefore, further development in terms of sensitivity between the levels of job requirements, as well as its content specifically related to the cognitive and social demands of work is required. The tool is not considered ready for implementation and therefore not shown in this article, as further evaluation and development is warranted.

During the NGT participants provided feedback on the proposed program as well as practical aspects around the implementation of specific tools in addition to commenting on the tools themselves. The participants felt strongly that certain foundational knowledge had to be in place before the therapists could benefit from both the proposed program and the use of the tools. In response to this, the training manual was elaborated to include a summary of the foundational knowledge expected to be in place with resources where therapists could find more information should they require it. Aspects that were added include knowledge of legislation, applications for disability grants, and work classification. Most of these aspects have been identified as core competencies in the delivery of VR services. ²⁹ Additionally, participants felt that the program had to have specific focus on drawing from available resources in the community to refer to and source assistance from during the VR process. This is supported by one of the core beliefs of CHAT that activity occurs in a community and that interaction with the community mediates change. ¹⁰ Having an appreciation for how these interactions mediate the activity system will support therapists’ understanding of how to interact with the community of interest. Additionally, participants opined that clients’ performance in other life roles specifically as well as motivation to engage in work as occupation were important considerations. This is supported by numerous studies indicating the interrelatedness of successful participation in various areas of occupation as well as motivation as key considerations of work ability.^22,30,31 These were included in both the training manual and revision of the tools.

Although it was not a primary aim of the NGT, the general and specific feedback provided by participants further informed the structure and content of the two-day workshop and the content and schedule of the workshop was refined as a result. The training manual has also been developed further and elaborated on. In line with the preposition of CHAT that the provision of tools mediates change, it is anticipated that the use of the manual will facilitate practice change. This is expected to be supported by the anticipated increase in knowledge and confidence of occupational therapists after attending the two-day workshop.

Conclusion

VR is a multidimensional and complex process that requires occupational therapists to consider a variety of personal, occupational and environmental factors. The use of clinical reasoning and decision-making tools can be useful in supporting the delivery of complex services. The application of the NGT not only provided consensus on the acceptability of the three tools but also highlighted valuable areas of improvement and recommended revisions. As a result, two of the three tools are considered acceptable and clinically utile to support the decision-making process in VR service delivery at a grassroots level in South Africa.

Recommendations

Further evaluation of the need and content validity of the Job Match Tool is recommended. The efficacy of the entire program, i.e., the two-day workshop to improve the perceived knowledge and skills of occupational therapists as well as its translation to practice needs to be evaluated. This should include the practicality of the three tools and the training manual to support decision making after completion of the course.

Limitations

The authors acknowledge that the presented tools were developed for the South African context. Therefore, applying these tools in other contexts must be done with careful considerations of contextual differences that may affect the clinical utility and appropriateness of the tools. Specific limitations of the tools are further noted. Firstly, the level of detail included in the Return-to-Work Decision Tree may be considered overwhelming, however it is expected that revisions after clinical application could further refine the tool. Additionally, it should be emphasized that the purpose of these tools is not to make formal determinations of work capacity, but to inform clinical decision making to provide interventions aimed at work in diverse settings and across the continuum of care.