The effects of a mobile-based vocational skill building coaching technology intervention for people with cognitive disabilities: A pilot feasibility study

Abstract

We designed a feasibility study to evaluate a mobile-based vocational skill building coaching technology (aka Mobile Coach) intervention by using an ecological design approach. We compared the Mobile Coach to a standard job coach (no Mobile Coach technology) assistance in a facility that employs adults with significant cognitive disabilities (CDs). Twenty working-age adults with CDs were enrolled in this feasibility study and were asked to use the Vocational Mobile Coach Technology (on an iPad) to assist with their daily job functions. Project-specific usability and self-satisfaction survey was used to evaluate the user experience in performing the selected work assembly tasks with the Mobile Coach and without it. This report has the goal to describe our feasibility study design, methods, and results.

Keywords

Mobile technology assistive technology vocational cognitive disability cognitive impairment job coach ecological design workplace

Introduction

The World Health Organization (WHO) estimates that 15 percent of the world's population lives with some form of disability and almost 4 percent of them are 15-year-old and older requiring specialized services and care to function in our society.¹ The number of people with disabilities who experience significant difficulties in functioning is increasing, in part due to ageing populations as well as an increase in chronic health conditions worldwide.^1,2 Often, people with disability experience poorer health outcomes, have less access to education and work opportunities, and are more likely to live in poverty than those without a disability (WHO).¹ Therefore, the ability to work and succeed in maintaining employment status during working age is extremely important for people with disabilities.

In 2021, it will be the 31st Anniversary of the Americans with Disabilities Act (ADA) that prohibits employment discrimination against people with disabilities.³ Despite the signing of the ADA, people with disabilities in the United States continue to experience disproportionately lower rates of employment and significantly lower wages than people without disabilities and only one in three individuals with disabilities are employed.² Although the trends in the employment-population ratio and the labour force participation rate have overall improved in the United States for people with disabilities, these indicators continue to fall behind those for people without disabilities.⁴ In 2019, the employment-population ratio of Americans with disabilities was 19.3 percent, compared to 66.3 percent for people without disabilities.⁵ Moreover, when people with disability were employed, 82 percent were either under-employed or worked part-time.⁵ In 2020, due the COVID-19 health crisis, there was a significant increase in the unemployment rate for people with disabilities.⁶ These employment disparities are even worse for people with Cognitive Disabilities (CDs) who usually experience cognitive function challenges with tasks that required orientation, attention, learning, reading, memorizing, speaking, and social skills. Unfortunately, these tasks are critical for good workplace performance.⁷

For occupational activities that require planning, memorizing tasks, and locations, such as custodial duties, mail delivery, grounds maintenance, and assembly warehouse errands, the inability to memorize task order and places are the primary barrier for individuals with CDS to attain to a productive, safe, and enjoyable work environment.^8,9 Technology trends and demographics support the development of mobile technology applications for the working needs of adults with CDs.^8–10 Mobile technology can be described as any device with internet capability that is accessible from anywhere the user is, including smartphones and tablets.¹¹ In 2019 in U.S.A, over 90% of Americans own a mobile technology of some kind (i.e., smartphones) for emailing, instant-messaging, or information-seeking.¹² The benefits, ease of access, and ubiquity of online and mobile internet solutions make these mobile platforms attractive for the use and assistive technology needs of individuals with CDs.¹³

A Mobile Internet Device (MID) is described as a handheld device that is portable, can access the internet, and has down-graded computing capability compared to other portable computing devices like laptops.¹¹ MIDs have extended multimedia capabilities for handheld devices and are generally the size and shape of a typical tablet. MIDs are not used with the intention to replace mobile or smart phones and are designed to be used as companion devices.¹² In addition, handheld devices can be especially helpful for individuals with physical and mental frailty.^13,14

Mobile technology can be designed to be easily accessible, providing an opportunity to assist in occupational and training settings for individuals with CDs.¹⁵ Therefore, we designed a novel Mobile Skills Vocational Coach or Mobile Coach Technology (MCT) device aimed at assisting in job training for individuals with CDs.

The purpose of this pilot study was to evaluate the feasibility of the MCT in an ecological environment (warehouse) that employs adults with significant cognitive impairments. More specifically, we want to assess whether working-age adults with CDs who used the MCT (on an iPad) will show higher satisfaction in performing the selected work assembly tasks (i.e., easier and better to do the task) as compared to a non-MCT group.

Methods

This study is part of a series of development and research projects from the Rehabilitation Engineering Research Center for the Advancement of Cognitive Technologies (RERC-ACT).¹⁶ The engineering of the MCT device was based on a series of iterative design research processes prior to this feasibility pilot study.¹⁶ The iterative design was used to inform the protocol and consisted of several visits and observations in the real-world environment. Job coaches for people with CDs participated in the development of the protocol. Therefore, we are now reporting the results of the feasibility pilot study that used an ecological approach to evaluate the MCT for working age adults with CDs. The study protocol was approved by the Colorado Multiple Institutional Review Board Ethics Committee and followed the principles embodied in the Declaration of Helsinki.

Participants with cognitive disabilities

Working-age individuals with CDs from the Goodwill Industries Workday Program, ages 18–64, who met the criteria for Disability Benefits accordingly to the United States of America Social Security Administration^17,18 and by their state of residence,^18,19 were invited to participate in a single testing session of a technology-based mobile context-aware prompting system technology. The technology was designed to coach and aid individuals with Intellectual and Developmental Disabilities (IDD) in performing assembly tasks.^8,10 All participants, or their legalized authorized representative (LAR), signed an informed consent form and completed a demographics survey. The survey collected information about education level, disability type and living arrangements. For the purpose of this study, participants were only included in the study if they could express some sort of communication, either orally or by using a communication device and able to move objects (cases/boxes) from one room to another.

Instrumentation

The MCT is a prompting tool run on an iPad intended to assist individuals with cognitive impairments to work more effectively within jobs that require them to move from one location to another (mobile job). Mobile jobs are defined as jobs where the user is required to walk a distance to perform the job (i.e., warehouse workers; custodians; grounds keeper; etc.).^9,15 The MCT is a cloud-based application that prompts the users through a series of tasks required to complete a specific job, like stock a box on a shelf. The tasks for the mobile job required navigation, object recognition, object transportation, and task sequencing. Figure 1 represents the architecture for the MCT system.

Figure 1.

MCT model and architecture.

The MCT is a heuristic model that could assimilate a current activity, past work performed, and determine the next activity that should be performed. A Task Server received an incoming event and determined the next, best action to perform. The results were delivered to a Data Presentation Layer, the mobile worker’s prompting device (iPad) via the cloud. For this study, the Task Server was configured to recognize time events and gesture events. Navigational events were considered during the design; however, the performance and precision of the chosen technology was not sufficient to support this interface. The MCT has a touch-based screen (iPad) that has a virtual agent (job coach) that introduces instructions related to tasks. If the participant did not understand the instruction or forgot the step, the system would re-introduce the information when participants touch the screen.

Procedures

Ecological environment

The Mobile Coach Project team worked in conjunction with Goodwill Industries of Denver Job Day Program. Goodwill Industries has a contract with Century Link, where they supplement the packaging of residential telecommunication equipment (e.g., DSL modem). Goodwill Industries receives the equipment on a pallet. Depending on the type of residential telecommunication equipment, each pallet contains 12–24 cases, and each case contains 10–14 residential devices. Once delivered, these pallets are temporarily stored in the Goodwill Industries warehouse. The warehouse and the assembly area are adjoining rooms. The approximate distance from the pallets to the assembly pods range between 50 to 100 feet.

Intervention

Enrolled participants from the Goodwill Denver Vocational Training Day Program had the opportunity to volunteer or decline participation in the study. Consenting procedures were performed for each participant, and their understanding about the study was evaluated by asking specific questions about the study after consenting explanation, such as “what is this study about?” or “if you agree to participate what do you think that you will do in this study?” Demographic information was obtained from each participant. Enrolled participants’ level of comprehension and understanding was acquired by using the Mini Mental State Exam.²⁰

The intervention was based on one four-hour session. Participants were assigned to one of two groups, one that used the MCT or one that used a standard vocational coach. The MCT is a technology-based mobile context-aware prompting system (CAP) designed to job coach, train, and aid work performance in adults with significant cognitive impairments (Figure 1). The standard vocational intervention was job coaching performed by a human with no technology. Both groups were exposed to similar procedures while performing the assembly task job despite differences in the delivery of job coaching.

MCT device tasks

The job performed by the mobile worker included six high level tasks:

Unwrap new pallet;

Get new case/box;

Deliver new case/box to assembler (mobile task);

Retrieve completed case/box from assembler;

Place completed case/box on a pallet in the storage room (mobile task);

Secure completed pallet.

Completing each of these tasks required one or more prompts. The overall sets of prompting were a collection of sequential steps and dual path decision points. Sequential steps were incremented either via a timer event expiring or a gesture performed by the user on the MCT. All decision points required the user to perform the correct gesture.

User usability data

A project-specific survey was designed to evaluate usability and participants’ satisfaction with the intervention. The survey had questions related to their experience with the coaching intervention with a Likert scale response level of 1 to 5, 1 meaning “Not at all” to 5 meaning “Very Much”. A sample of how this scale was designed for adults with CDs can be seen in Figure 2.

Figure 2.

An example question on a Likert scale modified for adults with CDs.

Results

A total of 20 participants (5 females and 15 males) with CDs was consented and enrolled in the study. Participants’ disabilities ranged from a variety of cognitive impairments, including Down syndrome, Autism, and brain injury. The sample was comprised of 65% white, 20% Hispanic and 15% African American (Table 1).

Table 1.

Total sample characteristics.

Age	Gender	Cognitive dysfunction	Mini Mental State Exam (MMSE)	Diagnosis
59	Male	Cognitive impairment	24	Intellectual disability,Down syndrome
52	Male	Cognitive impairment	21	Intellectual disability
49	Male	Moderate cognitive impairment	24	Intellectual disability
55	Male	Moderate cognitive impairment	26	Intellectual disability
67	Female	Seizures, OCD cognitive impairment	27	Bipolar disease, intellectual disability
56	Female	Wheelchair user, seizures, epilepsy, cognitive impairment	25	Brain injury, Intellectual disability
56	Male	Moderate cognitive impairment	23	Intellectual disability
33	Male	Moderate Autism, moderate cognitive impairment	17	Intellectual disability
54	Male	Moderate cognitive impairment	23	Chronic schizophrenia, Intellectual disability
49	Male	Moderate cognitive impairment, gait problems	27	Intellectual disability
25	Male	Moderate cognitive impairment	25	Autism, intellectual disability
44	Female	Moderate Cognitive Impairment	23	Down syndrome, intellectual disability
44	Female	Moderate cognitive impairment, Raynaud’s	25	Down syndrome, Intellectual disability
37	Male	Moderate cognitive impairment, hyperactivity	23	Down syndrome, intellectual disability
24	Male	Moderate cognitive impairment, ADHD, OCD, depression	30	Intellectual disability
30	Male	Moderate cognitive impairment, cerebral edema, hydrocephalus	24	Spina bifida, intellectual disability
50	Male	Mild cognitive impairment	30	Autism Asperger syndrome, intellectual disability
34	Male	Developmental delay due to premature birth	27	Intellectual disability
39	Male	Cerebral Palsy, mental retardation, hydrocephalus, physical disability	23	Intellectual disability
48	Female	Down Syndrome, visually impaired, mild cognitive impairment	25	Intellectual disability

OCD: obsessive compulsive disorder; ADHD: attention deficit hyperactivity disorder.

The MCT group (N = 10) reported higher satisfaction with the intervention as compared to the non-MCT group. Approximately 45–100% of the MCT responses collapsed into the “Very Much” reporting level category on questions related to work enjoyment, ease of work, use of the technology, and if the MCT was helped in doing the job better. There were also questions related to adherence, such as “Would you like to use the MCT again?” (Table 2).

Table 2.

MCT user survey responses rate.

MCT user survey	Not at all (%)	Somewhat (%)	Very much (%)
Q1: Did you have fun piling the cases?	0	11	89
Q2: Were you good at piling the cases?	0	0	100
Q3: Was piling the cases easy?	11	22	67
Q4: Would you like to work at piling cases again?	11	11	78
Q5: Was it easy to remember where you were in the job?	11	22	67
Q6: Was it hard to use the mobile coach device?	33	56	11
Q7: Was it easy to understand the mobile coach instructions?	11	0	89
Q8: Did the mobile coach device help you do your job better?	0	55	45
Q9: Did the mobile coach device make it easy to get help?	0	55	45
Q10: Did you like the speed of the mobile coach device?	0	45	55
Q11: Was the talking face on the screen helpful?	0	22	78
Q12: Did the pictures on the screen help you?	0	11	89
Q13: Would you like to use the mobile coach device again?	0	22	78
Q14: Did the mobile coach device give you help when you needed it?	11	33	56
Q15: Would you rather have…	1) a person as a job coach (33%)	2) use the mobile coach device only (0%)	3) have a person and the mobile coach device (66.7%)

The standard human job coach group (N = 10; non-MCT) also expressed positive satisfaction with their standard job and in having the facility’s job coach assistance while performing their tasks. However, their response levels were lower when compared to the MCT group satisfaction at 35–75% responded “somewhat” to “very much” satisfaction with the human job coach.

The results of the survey indicate that there were differences in the responses between groups, and the integration of an MCT device was well received by the workers with CDs. Most of the participants were extremely excited and eager to participate in the study, as most of them wanted to use the technology. To accommodate the high motivation to use the technology, we provided time to “play” with the technology for the group that was not assigned to the MCT after they completed the experimental procedures. There were no differences in answers between the participants who were already competent with computers or iPads versus those with no previous experience with computer or tablet technology. In addition, 25% of the sample were illiterate, and literacy levels did not influence performance when using the MCT.

Discussion

MID technologies have improved over the years and are more affordable and easily available than years ago.^21,22 Smart technologies, particularly smartphones, are leading the market for mobile technologies and have the potential to be a critical assistive technology device to assist the employment needs of individuals with a variety of cognitive impairments.^8,23 This technology is designed to be easy to use, as it can be manipulated by touch or voice command.¹³ The ease of access and use of the mobile solution technologies make MIDs an attractive platform for helping individuals with cognitive impairments to perform tasks that require prompting assistance, such as an assembly job assignment.^9,10,21

The results of this pilot study support the notion that MIDs and related technologies can be easily engineered and adapted to suit the user needs. They can present an effective, accessible, and integrated system to meet the assistive occupational needs of individuals with CDs. These technologies should also be affordable and tailored accordingly to the person’s physical and cognitive abilities.²¹ Mobile technologies are recognized as flexible and powerful tools that can enhance human performance if the technology features are designed with the user in mind.²² For individuals with disabilities the habitual involvement with these devices may have a positive and lasting impact on users’ ability to think, remember, pay attention, regulate emotion, and communicate.²³ These technologies, if properly engineered, can have a powerful impact in the quality of life of a person with a disability by decreasing disability stigma and increasing socialization, societal engagement, and self-esteem by empowering the person’s abilities.²⁴

Although, the study results support the feasibility of the MCT as an innovative employment solution for people with CDs the study also had several limitations such as sporadic MCT malfunction during the experiment. However, this occurred only during one trial and we only had 2 episodes of network malfunctioning during the trial day. This technology malfunction was carefully recorded and after the system was restored and functioning, the timer and data collection system were re-started, and the trial was resumed. Another limitation was the short time that each participant was exposed and used the MCT (4 hours). Future studies could provide more detailed information related to the technology adoption and adherence if workers are exposed to the intervention longer. Another limitation of this study was the ecological environment. Since the intention of the study was to test the technology in a real-world setting, we had to deal with non-controllable factors that a real working warehouse presents, such as constant loud noise and other unexpected distractions. On two occasions, we had to postpone a planned visit to the facility due the absence. In addition, we had to deal with continuous stops during the experiment due to breaks and lunch times as part of the worker schedule. Although this study was designed as a feasibility study, the small sample size limits the generalizability of our results. After the study implementation phase, we also learned that the vocational coaches could have been important participants in the study. They are another population that can contribute with important design and such as the practical adaptation of the MCT system into their job coach program and work routine.

Conclusions

The findings of this feasibility study indicate that our MCT device has translational and pragmatic implications, particularly in settings that employ and provide vocational training for individuals with CDs. The results support that the MCT is feasible and that working-age adults with CDs can benefits of mobile technologies to improve work performance and enjoyment quality. Therefore, this technology system has the potential to assist workers with CDS in vocational training and job tasks that require memorizing steps that involve well-ordered action in conjunction with good spatial recognition. It is important to note that the program has great potential for commercialization and knowledge transfer. Despite the several limitations impacted the study’s generalizability, we were able to recruit a well-balanced and heterogeneous sample for comparisons between groups. In addition, due to the ecological element of the study, we used mix-methods appraisals (qualitative and quantitative) to evaluate the MCT feasibility. Further research is recommended to evaluate a longer duration experimental design in a larger sample for the successful adoption and adherence of the MCT for workers with CDs.

Supplemental Material

sj-pdf-1-jrt-10.1177_20556683211009731 - Supplemental material for The effects of a mobile-based vocational skill building coaching technology intervention for people with cognitive disabilities: A pilot feasibility study

Supplemental material, sj-pdf-1-jrt-10.1177_20556683211009731 for The effects of a mobile-based vocational skill building coaching technology intervention for people with cognitive disabilities: A pilot feasibility study by Patricia C Heyn, Aleaza Goldberg, Greg McGrew and Cathy Bodine in Journal of Rehabilitation and Assistive Technologies Engineering

Supplemental Material

sj-pdf-2-jrt-10.1177_20556683211009731 - Supplemental material for The effects of a mobile-based vocational skill building coaching technology intervention for people with cognitive disabilities: A pilot feasibility study

Supplemental material, sj-pdf-2-jrt-10.1177_20556683211009731 for The effects of a mobile-based vocational skill building coaching technology intervention for people with cognitive disabilities: A pilot feasibility study by Patricia C Heyn, Aleaza Goldberg, Greg McGrew and Cathy Bodine in Journal of Rehabilitation and Assistive Technologies Engineering

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by funding provided by the National Institute on Disability, Independent Living and Rehabilitation Research (NIDILRR) Grant #H133E090003 and the Coleman Institute for Cognitive Disabilities.

Guarantor

Patricia C. Heyn, PhD, FGSA, FACRM.

Contributorship

PCH and CB conceived the study. PCH developed the protocol and gained ethical approval. PCH and AG recruited and collected the data. PCH and GMG analyzed the data. PCH wrote the first draft of the manuscript. All authors edited and reviewed the manuscript.

Acknowledgments

We would also like to acknowledge Goodwill Industries of Denver Job Day Program for allowing us to use their facility and recruit their employees for this study.

ORCID iD

Patricia C Heyn

References

World Health Organization. WHO global disability action plan 2014–2021: better health for all people with disability. Geneva, Switzerland: World Health Organization, 2015.

Houtenville

Boege

2019 Annual report on people with disabilities in America. Durham: Institute on Disability, University of New Hampshire, 2020.

Tucker

BP.

The Americans with disabilities act of 1990: an overview. NML Rev 1992; 22: 13.

Friedman

The relationship between disability prejudice and disability employment rates. Work 2020; 65(3): 591–598. doi: 10.3233/WOR-203113. PMID: 32116277.

Guo

Kamar

Vaughan

, et al. Toward fairness in AI for people with disabilities: a research roadmap. arXiv 2019, arXiv:1907.02227.

Clifford Simplican

Politicizing disability in political science, COVID-19, and police violence. Politics Groups Ident 2021; 9: 387–388.

Schalock

Luckasson

Tasse

MJ.

Intellectual disability: definition, classification, and systems of supports. 12th ed. Washington, DC: American Association on Intellectual and Developmental Disabilities, 2021.

Sauer

Parks

Heyn

PC.

Assistive technology effects on the employment outcomes for people with cognitive disabilities: a systematic review. Disabil Rehabil Assist Technol 2010; 5: 377–391.

Carrieri

Sgaramella

Bortolon

, et al. Determinants of on-the-job-barriers in employed persons with multiple sclerosis: the role of disability severity and cognitive indices. Work 2014; 47: 509–520.

10.

Van Laarhoven

Carreon

Bonneau

, et al. Comparing mobile technologies for teaching vocational skills to individuals with autism spectrum disorders and/or intellectual disabilities using universally-designed prompting systems. J Autism Dev Disord 2018; 48: 2516–2529.

11.

Kumar

inventor. Mobile internet device with detachable wireless module. United States patent US 7,899,397, 2011.

12.

Carolus A, Binder JF, Muench R, Schmidt C, Schneider F, Buglass SL. Smartphones as digital companions: Characterizing the relationship between users and their phones. New Media & Society 2019; 21(4): 914–938. doi:10.1177/1461444818817074

13.

Darcy

Maxwell

Green

Disability citizenship and independence through mobile technology?

A study exploring adoption and use of a mobile technology platform. Disab Soc 2016; 31: 497–519.

14.

Stolz

Schulz

Krieger

, et al. A mobile app for social anxiety disorder: a three-arm randomized controlled trial comparing mobile and PC-based guided self-help interventions. J Consult Clin Psychol 2018; 86: 493–504.

15.

Setoguchi

Motruk

Frank

, et al. A review of mobile applications to enhance coaching in occupational therapy. Br J Occup Ther 2020; 83: 549–560.

16.

Heyn

Cassidy

Bodine

The rehabilitation engineering research center for the advancement of cognitive technologies. Am J Alzheimers Dis Other Demen 2015; 30: 6–12.

17.

Stith

Wasif

Nicholas

LH.

Effects of SNAP-associated work requirements on disability claiming. In: 9th Annual conference of the American Society of Health Economists, St. Louis, Missouri, 8 June, 2020. https://ashecon.confex.com/ashecon/2020/meetingapp.cgi/Paper/10035 (accessed March 29, 2021).

18.

Colorado Department of Human Services, https://www.colorado.gov/pacific/dds/about-dds (accessed 29 March 2021)

19.

Colorado's Definition of Developmental Disabilities Lakewood, CO: Developmental Disabilities Resource Center, https://ddrcco.com/accessing-services/developmental-disability-definition (2018, accessed 29 March 2021)

20.

Tombaugh

McIntyre

NJ.

The mini-mental state examination: a comprehensive review. J Am Geriatr Soc 1992; 40: 922–935.

21.

Kim

(eds). A service-based approach to developing android mobile internet device (MID) applications. In: 2009 IEEE international conference on service-oriented computing and applications (SOCA), Taipei, Taiwan, 2009, pp. 1–7, doi: 10.1109/SOCA.2009.5410278.

22.

Pew

Hemel

SV.

Technology for adaptive aging. Washington DC: National Academies Press, 2004.

23.

Khalifa

Sharif

Sultan

, et al. Workplace accommodations for adults with autism spectrum disorder: a scoping review. Disabil Rehabil 2020; 42: 1316–1331.

24.

Al-Dababneh

Al-Zboon

EK.

Using assistive technologies in the curriculum of children with specific learning disabilities served in inclusion settings: teachers’ beliefs and professionalism. Disab Rehab 2020; 15: 1–1.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.30 MB

0.61 MB