Sage Journals: Discover world-class research

Abstract

This study examined the feasibility of group interventions using the Direct Skills Teaching (DST) approach to teach interview preparedness skills (“Presenting Qualifications”) and basic work-related conversational skills (“Conversing with Others”) to individuals with psychiatric disabilities. This quasi-experimental pilot study included 87 participants in Study 1 and 55 participants in Study 2, all with various psychiatric diagnoses. Each group intervention consisted of four sessions delivered by graduate counseling students. Participants reported increased confidence in presenting qualifications at interviews in Study 1 and in conversational skills at work in Study 2. Participants in both studies reported high levels of satisfaction with the respective group intervention. The results of the study supported the feasibility of DST group interventions designed to teach work-related skills to individuals with psychiatric disabilities.

Keywords

psychiatric disabilities career/vocational work adjustment counseling

The advantages of employment for people, with or without disabilities, are well recognized. Employment provides financial means to purchase goods and services necessary for survival and general functioning. It also provides a sense of identity, fulfillment and self-worth, a valued social role, peer respect, community interactions, and one’s overall well-being (Angrave & Charlwood, 2015; Boreham et al., 2015; Julià et al., 2017). Furthermore, employment participation is correlated with decreases in health symptom distress including depression and anxiety, substance abuse, and emergency service utilization (Akihito et al., 2014; Bünnings et al., 2017; Nishi et al., 2017).

Conversely, unemployment, particularly long-term unemployment, can impede access to essential needs, health, and quality of life. Individuals who experience extended unemployment report increased use of/dependence upon public health services; amplified levels of stress regarding finances; decreased access to health care services; greater sleep loss or disturbance; increased risk of cardiovascular diseases; diminished sense of belonging and positive self-identity; and higher amounts of isolation, anxiety, and depression (Gonzalez et al., 2018; Hiswåls et al., 2017; Julià et al., 2017; Premji & Shakya, 2017; Sumner & Gallagher, 2017).

The high unemployment rate of individuals with serious mental illness (SMI) has a significant impact on the individuals and society. SMI is defined as any diagnosable mental, behavioral, or emotional disorder other than a developmental or substance use disorder that results in serious functional impairment, along with significant impacts to at least one major life activity (Substance Abuse and Mental Health Services Administration [SAMHSA], 2020). According to the 2018 National Survey on Drug Use and Health (NSDUH), approximately 47.6 million adults aged 18 years or older had any mental illness in the past year, including an estimated 11.4 million adults (4.6% of U.S. adult population) who had SMI (SAMHSA, 2020). Over 39% of persons with SMI experienced chronic poverty with yearly income of less than US$10,000 while unemployment rates can reach 80% to 90% (Luciano & Meara, 2014). Even employed people with SMI typically work part-time at entry-level jobs, earning low wages and experiencing short job tenure (Carmona et al., 2017; Evans-Lacko et al., 2013; Waghorn et al., 2012). Many continue to face difficulties earning a living wage or satisfying upward mobility, limiting their ability to exit public assistance programs, achieving economic independence, and full community integration (SAMHSA, 2020).

Around 3 million working-age adults, roughly 28% of all Social Security Disability Insurance (SSDI) recipients and 40% of all Supplemental Security Income (SSI) recipients, receive benefits due to a psychiatric disability (Salkever et al., 2014). Psychiatric disability is the primary disability for approximately 30% of people receiving disability benefits among adults age 50 and younger (Colpe et al., 2010; Drake et al., 2009; Mojtabal, 2011). Persons with psychiatric disabilities are less likely to be employed after age 49 compared with people with fewer mental illness symptoms and once people with psychiatric disabilities become beneficiaries of SSI and/or SSDI, they rarely attain economic self-sufficiency (Luciano & Meara, 2014; O’Leary et al., 2011).

Overall, unemployment among persons with mental illness has enormous economic and social costs (Banerjee et al., 2017). Persons living with SMI without effective employment services often do not obtain or retain competitive employment (J. A. Cook et al., 2005; Frey et al., 2011) and obtain low-wage jobs (Bond et al., 2012). Implementation of evidence-based supported employment services for persons with psychiatric disabilities has been proposed (Bond et al., 2020; Drake et al., 2016; Druss, 2014). Evidence-based supported employment directly addresses a client’s employment issues as they occur in the natural context of employment. Supported employment programs, however, do not have a structured skills training component for soft skills targeting job placement and retention.

Soft skills are broadly defined as non-technical interpersonal skills and have been used interchangeably with the term “employability skills” (Ju et al., 2012; Sherer & Eadie, 1987). They are increasingly recognized as important in maintaining employment, as more jobs require a higher level of soft skills and decreased levels of physical skill (Pierce, 2019). A lack of soft skills is often cited as a major reason people lose jobs and a significant barrier for employment for persons with disabilities (Duncan & Dunifon, 2012; Tulgan, 2015). Work-related soft skills accompany “hard” skills, or the practical skills of a job (Kyllonen, 2013). Furthermore, soft skills are associated with higher job satisfaction by enabling individuals with disabilities to effectively communicate and collaborate with coworkers and supervisors (Kyllonen, 2013).

Soft skills training can be effectively delivered via a Direct Skills Teaching (DST) approach, which integrates cognitive and behavioral techniques and allows participants a more active role (Cohen et al., 1985; Ellison et al., 2002; Shern et al., 2000). DST is a promising practice for individuals with psychiatric disabilities and implements a structured curriculum including detailed content outlines and lesson plans (Cohen et al., 1985; Rogers et al., 2006). The current study utilized two DST-based interventions for individuals with psychiatric disabilities to help build work-related soft skills and increase chances of positive employment outcomes.

This study implemented two curricula (“Presenting Qualifications” and “Conversing with Others”) among persons with psychiatric disorders who are receiving community mental health, vocational, or housing services. Both utilized DST as designed to help persons with psychiatric disabilities to communicate effectively to employers the skills that qualify them for a position (Oursler et al., 2019) or to engage in lunch time conversations with coworkers (Lu et al., 2020b). The study assessed the feasibility (i.e., acceptability and practicality) of the interventions in increasing perceived interview preparedness or in increasing perceived conversational ease with coworkers.

Method

Participants were adults living with psychiatric disabilities who were actively receiving mental health and/or vocational services from community-based agencies in the Northeast United States. These individuals were not only diagnosed with mental illnesses but also experienced major functional limitations to be eligible for receiving services. They needed mental health support and assistance to obtain or sustain employment. Prospective participants were advised of the opportunity to join groups facilitated by master’s degree–level counseling students completing internship requirements under the supervision of agency staff and university counseling faculty. For both studies, inclusion criteria were (a) age 18 years or over, (b) able to speak/understand English, (c) interested in finding employment, and (d) currently receiving mental health and/or vocational services at the participating agency. Both studies received Institutional Review Board (IRB) approval.

Participants

Study 1

A total of 87 individuals with psychiatric disabilities participated in Study 1, receiving the “Presenting Qualifications” intervention. The findings of 27 of the 87 participants were reported in an earlier study (Oursler et al., 2019). The earlier study reported data for people with various disabilities and did not provide analyses specifically for people with psychiatric disabilities. The participants were recruited from community mental health programs and vocational rehabilitation agencies. Participants were drawn from 25 groups, each led by graduate-level rehabilitation counseling students at the participating agencies. Participants in 2 groups (n = 14) participated in groups consisting of members with psychiatric disabilities and the groups took place in community mental health agencies that serve people with psychiatric disabilities. One group (n = 9) was conducted at a homeless shelter, while the rest of the participants (n = 64) came from 22 groups, which consisted of members with different disabilities and took place at agencies that served people with a wide range of disabilities. In Study 1, per agency chart, 19.5% of the participants had schizophrenia/schizoaffective disorders, 21.8% had major depression and other depressive disorders, 21.8% had bipolar disorders, 6.9% had co-occurring disorders, 6.9% had anxiety disorders, and 23.1% had other diagnoses.

Study 2

Participants in this study received the “Conversing with Others” intervention. A total of 55 adults with various psychiatric disabilities including schizophrenia spectrum, depressive, and bipolar-related disorders were enrolled in this study. Participants in Study 2 did not participate in Study 1 and vice versa. Some participants had disabilities (physical, developmental, co-occurring) in addition to their primary psychiatric disabilities. Data on these participants had not been published. There were seven groups, each led by graduate level rehabilitation counseling students during their internship year. Participants in five groups (n = 46) were from community mental health agencies serving people with psychiatric disabilities, while the rest of the participants (n = 9) comprised two groups consisting of members from supportive housing programs that serve people with psychiatric disabilities. In Study 2, per agency chart, 45.5% had schizophrenia/schizoaffective disorders, 23.6% had major depression and depressive disorders, 14.5% had bipolar disorders, 5.5% had trauma-related disorders, 5.5% had co-occurring disorders, 1.8% had anxiety disorders, and 3.5% had other diagnoses.

Measures

For both studies, demographic and disability information were collected from agency records. Participants also completed a 10-item questionnaire with a Likert scale ranging from “strongly disagree” to “strongly agree.” The content of these questionnaires varied based upon the intervention and skill being taught.

In Study 1 (“Presenting Qualifications”), the Perceived Interview Preparedness Questionnaire assessed participants’ confidence in their ability to get ready for an interview and to answer questions during an interview (Lu et al., 2019, 2021a, 2021b, 2020a). Sample questions included “I have a good idea about how to prepare for my job interview”; “I feel confident I can communicate my skills to an employer in a positive way”; and “I feel confident that I can answer difficult questions that an employer may ask on an interview.” Internal consistency reliability was .89 in this study. The concurrent and predictive validity for this scale is unknown.

In Study 2 (“Conversing with Others”), the Revised Perceived Conversation Easiness Questionnaire was a revised self-report measure (Lu et al., 2020b) used to obtain participants’ perceptions of their conversational skills (Oursler, 2020). Sample questions included “I know how to start a conversation with people I do not know well.”; “Part of having a good conversation is being a good listener”; and “There are some topics that are not good choices to talk about at work.” The internal consistency reliability was .67 in this study. The concurrent and predictive validity of this scale is currently unidentified.

Participants also completed post satisfaction surveys gauging personal satisfaction ratings with group experiences. The same questions were used for both studies and addressed the overall quality of the group, specifically: (a) What did you like about the group? (b) What would you change/improve about the group? (c) What will you do differently as a result of attending the group? (d) What other information would be helpful to include in the group? and (e) Any other comments?

Intervention

The DST approach

systematically incorporates basic educational and cognitive techniques so that providers can outline the knowledge needed to learn any relevant skill, develop a structured lesson plan to teach each component behavior and involve the person and individuals in the relevant environment in practice and generalization efforts. (Farkas & Anthony, 2010, p. 119)

It departs from Social Skills Training by (a) deciding on an overall rehabilitation goal (ORG), (b) teaching the critical skill(s) required to achieve and sustain the ORG, (c) adjusting the lessons in collaboration with client(s), and (d) integrating cognitive and behavioral techniques.

According to DST methodology, a skill is defined as knowledge followed by a corresponding behavior (Cohen et al., 1985). An individual skill is separated into behavioral stages that comprise essential skill components. As each skill is taught (knowledge) and practiced (behavior), the facilitator would offer feedback and enhancement suggestions to the participant(s). The desired outcome is that participant(s) will gain the knowledge and acquire the behavior required to perform the learned skill in a target environment. Teaching a new skill using a DST approach, involves (a) using a Content Outline to determine the skill dimensions, (b) developing a structured curriculum that is made up of lesson plans, and (c) cultivating an encouraging environment where participants can rehearse the skill in-session (Farkas & Anthony, 2010).

Both group interventions included four 60- to 90-min sessions. The structure of each session followed Review, Overview, Presentation, Exercises, and Summary (ROPES), which includes (a) Reviewing content from previous sessions or discussing participants’ prior knowledge of the skill in the initial session, (b) providing an Overview of the session agenda, (c) Presenting material to be used in the session, (d) implementing Exercises to practice and reinforce skills, and (e) giving a Summary of the session (Cohen et al., 1985). Each group session focused on one of the behaviors constituting the skill.

The “Presenting Qualifications” curriculum was used in Study 1. It is designed to assist individuals with effectively communicating their “qualifications” to a job to an interviewer in the context of the job specifications, navigating interview questions, and interview preparation. The content outline includes (a) a definition of the skill, (b) benefits of using the skill, (c) steps of the skill, and (d) identification of situations to use the skill. The identification of the required behaviors for the content outline encompasses contemplation of what is needed before, during, and after using the skill. The content outline becomes the foundation for a predetermined curriculum with session plans based on the ROPES method to teach the behaviors of the skill and then to combine the steps to use the skill as a whole.

The “Conversing with Others” curriculum was used in Study 2. The desired outcome of the program is for the individual to carry on an informal conversation about various topics other than oneself in an employment setting by performing these four behaviors: (a) identifying conversational topics, (b) displaying interest in another person’s conversation, (c) shifting topics during a conversation, and (d) ending a conversation. During the final group, participants engage in a role-play exercise with another group member and demonstrate a scenario of engaging in conversation with a co-worker, during a break time. This activity is designed to allow participants to demonstrate the four conversational behaviors learned in the group by demonstrating the “Conversing with Others” Skill.

To assess adherence to the intervention manuals, at the conclusion of each group session, the facilitators completed a form evaluating group adherence to the curriculum and provided a critical report of the session. The form included these open-ended questions: (a) session notes on modules covered and session length, (b) adaptations/modifications from the session outline, (c) overall impression of group including participant reactions, and (d) recommended adaptations/modifications for future groups. Completed adherence forms were submitted to internship instructors.

Analysis

Data from groups lead by different leaders were aggregated for analysis. Outliers and data integrity were examined through descriptive statistics prior to statistical analysis for each item. Wilcoxon signed-rank tests were conducted to examine pre- and postintervention differences on the questionnaires. This was due to the fact that most responses centered around “agree” to “strongly agree” in pre and post questionnaires, causing departure from a normal distribution in the dependent variables.

For open-ended questions, a modified version of conventional content analysis was used to analyze participant responses (Hsieh & Shannon, 2005). Two independent researchers reviewed responses for their content, coded their themes, and compared their themes. Differences and discrepancies were discussed and reconciled to clarify the coding scheme. Common themes and their occurrences were noted with percentages reported. The two independent researchers had no knowledge of the names of the instructors or the participants’ diagnoses.

Results

Study 1

There were 87 participants who ranged from 18 to 71 years, with a mean age of 43 years. Most participants were older than 36 years (69.5%), with others ranging from 25 to 35 years (16.8%) or 18 to 24 years (13.7%). The average level of education was 1 year post high school graduate/GED. The sample had slightly more male participants (56.3%) and the largest percentage of participants identified as Caucasian/European American (40.2%). The remaining participants identified as African American (39.1%), Hispanic American (14.9%), Asian American (2.3%), and Other (2.3%). The primary disability type for participants was psychiatric disorders, specifically: (a) Depressive disorders (21.8%), (b) Bipolar disorders (21.8%), (c) Schizophrenia spectrum disorders (19.5%), (d) Other/Unspecified psychiatric disorders (18.4%), (e) Co-Occurring/Dual disorders (6.9%), (f) Anxiety disorders (6.9%), (g) Obsessive Compulsive disorder (2.3%), and (h) Personality disorders (1.8%) (see Table 1).

Table 1.

Demographic and Diagnostic Characteristics of the Study Participants.

	Study 1: Presenting Qualifications (N = 87)		Study 2: Conversing with Others (N = 55)
Demographic Characteristics	n (%)		n (%)
Gender
Male	49 (56.3)		34 (61.8)
Female	38 (43.7)		20 (36.4)
Missing	0 (0.0)		1 (1.8)
Age range (years)
18–24	12 (13.7)		4 (7.2)
25–35	15 (16.8)		16 (29.1)
36+	60 (69.5)		35 (63.7)
Race/Ethnicity
European American	35 (40.2)		25 (45.5)
African American	34 (39.1)		18 (32.7)
Hispanic	13 (14.9)		8 (14.5)
Asian American	2 (2.3)		0 (0.0)
Other	2 (2.3)		2 (3.6)
Missing	1 (1.1)		2 (3.6)
Primary psychiatric diagnosis
Schizophrenia	17 (19.5)		25 (45.5)
Spectrum disorders	17 (19.5)		25 (45.5)
Depressive	19 (21.8)		13 (23.6)
Disorders	19 (21.8)		13 (23.6)
Bipolar disorders	19 (21.8)		8 (14.5)
Trauma related	0 (0.0)		3 (5.5)
Disorders	0 (0.0)		3 (5.5)
Co-occurring/dual	6 (6.9)		3 (5.5)
Disorders	6 (6.9)		3 (5.5)
Anxiety disorders	6 (6.9)		1 (1.8)
Obsessive	2 (2.3)		0 (0.0)
Compulsive
Disorder (OCD)
Personality	2 (2.3)		1 (1.8)
Disorders	2 (2.3)		1 (1.8)
Other/unspecified	16 (18.4)		1 (1.8)
Agency type
Community mental	14 (16.1)		39 (70.9)
Health
Non-profit	12 (13.8)
Vocational			11 (20.0)
Rehabilitation
Housing program	9 (10.3)		5 (9.1)
Government	52 (59.8)
Ever employed	60 (69.0)		37 (67.3)
Currently working	11 (12.6)		7 (12.7)
Years of competitive employment
0	9 (11.8)		8 (18.2)
1–5	17 (22.3)		16 (36.4)
6–10	19 (24.9)		10 (22.7)
10+	31 (40.6)		10 (22.7)
Number of sessions
1	11 (12.6)		10 (18.2)
2	15 (17.2)		6 (10.9)
3	7 (8.0)		8 (14.5)
4	54 (62.0)		31 (56.4)
Attended all sessions	54 (62.0)		31 (56.4)
	M (SD)	Range	M (SD)	Range
Age	43.0 (13.29)	18–71	41.44 (12.64)	21–69
Years of education	13.18 (2.5)	6–20	12.64 (1.89)	9–16
Years of employment	11.8 (10.2)	0–35	7.23 (6.96)	0–28

Note. OCD = Obsessive compulsive disorder.

A Wilcoxon signed-rank test supported statistically significant changes in participants’ perceived ability to present their qualifications; all 10 items achieved p-values of less than .001 (see Table 2). Post intervention, many participants’ responses changed to “agree” and “strongly agree” when asked about their self-assessed level of interview preparedness. The effect size for the total mean of all items was 0.80, indicating a large intervention effect (see Table 2).

Table 2.

Pre- and Post-Perceived Interview Preparedness Questionnaire for “Presenting Qualifications” Module (N = 87).

Questions	Pre		Post		Z	p	ES
Questions	M	SD	M	SD	Z	p	ES
1. I have a good idea about how to prepare for my job interview.	3.62	0.91	4.51	0.68	−5.27*	.00	0.57
2. I have a good idea about what kind of introductory questions are asked by employers.	3.59	0.90	4.39	0.60	−5.41*	.00	0.58
3. I have a good idea about the top skills employers are looking for when hiring someone.	3.63	0.90	4.33	0.68	−4.79*	.00	0.51
4. I have a good idea about what skills I have that match the skills employers are looking for.	3.69	0.84	4.41	0.67	−5.64*	.00	0.60
5. I feel confident that I can communicate my skills to an employer in a positive way.	3.66	0.97	4.48	0.63	−5.57*	.00	0.60
6. I have a good idea about questions employers might ask that can be difficult to answer.	3.49	1.00	4.32	0.70	−5.00*	.00	0.53
7. I feel confident that I can answer difficult questions that an employer may ask on an interview.	3.17	1.00	4.16	0.85	−5.61*	.00	0.60
8. I take a list of important information such as names of references to the interview.	3.86	0.84	4.46	0.70	−4.26*	.00	0.46
9. I can explain the reasons I left my last job to the interviewer.	3.70	0.92	4.41	0.75	−4.61*	.00	0.49
10. I have a good idea about how I should dress when going on an interview.	4.19	0.77	4.65	0.56	−4.35*	.00	0.47
11. Total scale (average of all items)	3.64	0.64	4.41	0.45	−6.56*	.00	0.80

Note. ES = effect size; 1 = “strongly disagree”; 2 = “disagree”; 3 = “neutral”; 4 = “agree”; 5 = “strongly agree.”

p ≤ .05.

Results also showed high intervention satisfaction among participants (see Table 3). Over 95% of participants endorsed “agree” or “strongly agree” to these statements: “The overall quality of the group and the instructor’s knowledge was good/excellent”; “I think the information I received will be helpful to me on the job”; “The group held my interest”; & “I feel confident that I can use the skill I learned in the group.” More than 70% of participants agreed or strongly agreed that the amount of sessions was about right. Over 85% of group members agreed that they would recommend this group to another person (see Table 3).

Table 3.

Satisfaction Survey for “Presenting Qualifications” Module (N = 87).

Satisfaction Questions	n	%
The overall quality of the group was good/excellent	64	95.5
The instructor’s knowledge was good/excellent	66	98.5
I think the information I received will be helpful to me on the job (agree/strongly agree)	65	97.0
The group held my interest (agree/strongly agree)	64	95.6
The number of sessions was about right	49	73.1
I feel confident that I can use the skill I learned in the group (agree/strongly agree)	62	92.6
I would recommend this group to a friend	57	86.4

Note. 17 (25.4%) thought the number of sessions was not enough. They recommended more sessions.

Group members also completed four open-ended questions. When asked, “What did you like about the group?,” several themes emerged: (a) learning new information about interviewing/job searching (n = 30; 33.1%); (b) positive group experiences including peer interactions (n = 14; 15.4%); (c) group dynamics, process, and structure (n = 10; 11.0%); and (d) positive view of facilitator (n = 8; 8.8%). Some participants (n = 25; 28.7%) did not respond or said “nothing” to this item.

Participants were also asked, “What would you change or improve about the group?.” These themes emerged: (a) more or longer sessions (n = 11; 12.1%); (b) changing group format/structure including environment (n = 7; 7.7%) (c) implementing ways to address attendance/participation issues (n = 4; 4.4%); and (d) additional skill information (n = 3; 3.3%). 62 (70.8%) participants listed “nothing/no changes” or left this item blank.

Common themes to the question, “What will you do differently as a result of attending the group?” included the following: (a) increasing interview preparation and practicing the skill (n = 26; 28.6%) and (b) working on increasing confidence and self-awareness (n = 13; 14.3%). Some (n = 42; 48.2%) of participants did not respond or stated “nothing” to this item. Also, some participants requested additional information or suggested changes to the group itself (n = 6; 6.6%), however their responses go beyond the scope of this question.

The penultimate question was “What other information would it be helpful to include in the group?.” Common themes included: (a) information about job applications and other relevant job skills (n = 13; 14.3%) and (b) additional time and information (n = 4; 4.4%). In addition, a majority (n = 70; 80.1%) of participants did not respond to this item or stated “nothing.”

Finally, there was a question for any other comments. Common themes included: (a) positive views of the group (n = 8; 8.9%); (b) favorable appraisal of group facilitator (n = 3; 3.3%); and (c) desiring additional information/more sessions (n = 2; 2.2%). A total of 74 participants (84.9%) reported having no other comments or did not respond to this item.

Study 2

There were 55 participants who ranged from 21 to 69 years of age with a mean age of 41 years. Participants were largely older than 36 years (63.7%), with others ranging between 25–35 years (29.1%) and 18–24 years (7.2%). The average educational level was high school graduate/GED. This sample consisted of more male participants (61.8%), and the largest percentage of participants was people who identified as Caucasian/European American (45.5%). The remaining participants identified as African American (32.7%), Hispanic American (14.5%), and Other (3.6%). The primary disability type for all participants was psychiatric disorders, specifically (a) Schizophrenia spectrum disorders (45.5%), (b) Depressive disorders (23.6%), (c) Bipolar disorders (14.5%), (d) Trauma related disorders (5.5%), (e) Co-Occurring/Dual disorders (5.5%), (f) Anxiety disorders (1.8%), (g) Personality disorders (1.8%), and (h) Other/Unspecified psychiatric disorders (1.8%) (see Table 1).

A Wilcoxon signed-rank test showed the intervention elicited statistically significant changes in participants’ perceived ability to successfully converse with others. Out of 10 questions, nine achieved p values of less than or equal to .01 on the Wilcoxon signed-rank test (see Table 4). The effect size for the total mean of all items was 0.74, indicating a large intervention effect (see Table 4).

Table 4.

Wilcoxon Signed Ranked Test for Pre- and Post-Perceived Conversation Easiness Questionnaire for “Conversing With Others” Module (N = 55).

Questions	Pre		Post		Z	p	ES
Questions	M	SD	M	SD	Z	p	ES
1. The people I work with want to talk about the same things I want to talk about^a	3.08	1.25	3.68	0.99	−2.93*	.00	0.40
2. I know how to start a conversation with people I do not know well	3.20	1.11	3.83	1.00	−3.51*	.00	0.47
3. I can research topics to talk about during lunchtime	3.20	1.28	3.71	0.90	−2.56*	.01	0.35
4. There are some topics that are not good choices to talk about at work	3.88	1.05	4.93	0.67	−3.11*	.00	0.42
5. I can talk about topics others than myself for 5 min	3.51	1.16	4.22	0.72	−3.43*	.00	0.46
6. It is important to make eye contact to show my interest in what the other person is saying	3.67	1.01	4.29	0.78	−3.54*	.00	0.48
7. I know how to change the conversation to a new topic	3.04	0.94	4.00	0.89	−4.57*	.00	0.62
8. When I don’t want to talk about a topic during lunch, I should get up and walk away in the middle of the conversation^a	2.61	1.10	2.15	1.22	−1.71	.09	0.23
9. I know how to end a conversation when it is time for me to go back to work	3.43	1.10	3.93	0.82	−2.49*	.01	0.34
10. Part of having a good conversation is being a good listener	3.94	1.05	4.46	0.64	−2.98*	.00	0.40
11. Total scale (average of all items)	3.42	0.49	3.90	0.49	−4.74*	.00	0.74

Note. ES = effect size. 1 = “strongly disagree”; 2 = “disagree”; 3 = “neutral”; 4 = “agree”; 5 = “strongly agree.”

Signifies reverse scored.

p ≤ .05.

Participants reported overwhelming satisfaction with the group intervention (see Table 5). More than 90% of group members endorsed “agree” or “strongly agree” to these statements: “The overall quality of the group and the instructor’s knowledge was good/excellent.” Over 80% “I think the information I received will be helpful to me on the job”; “The group held my interest”; and “I feel confident that I can use the skill I learned in the group.” A majority (53.5%) of group participants felt the amount of sessions was “about right,” however 34.9% believed there were not enough sessions. Finally, about 88% of participants would suggest this group to a friend (see Table 5).

Table 5.

Satisfaction Survey for “Conversing With Others” Module (N = 55).

Satisfaction Questions	n	%
The overall quality of the group was good/excellent	41	95.4
The instructor’s knowledge was good/excellent	40	93.0
I think the information I received will be helpful to me on the job (agree/strongly agree)	38	88.4
The group held my interest (agree/strongly agree)	37	86.1
The number of sessions was about right	23	53.5
I feel confident that I can use the skill I learned in the group (agree/strongly agree)	36	83.7
I would recommend this group to a friend	38	88.4

Note. 15 (34.9%) thought the number of sessions was not enough. They recommended more sessions.

Participants also completed four open-ended questions. When asked, “What did you like about the group?” several themes based on participants’ responses were (a) opportunity to enhance communication skills including building confidence (n = 19; 34.2%), (b) acquiring new/valuable information (n = 8; 14.4%), (c) format and structure of the group itself (n = 8; 14.4%), and (d) effective group facilitator (n = 4; 7.2%). Some participants (n = 16; 29%) left this item blank or reported disliking the group so the responses are based upon 45 participants comments.

Participants were also asked, “What would you change or improve about the group?” and these themes emerged: (a) more or longer sessions (n = 11; 19.8%), (b) nothing/no changes (n = 29; 52.7%), (c) fewer sessions or shorter sessions (n = 6; 10.8%), (d) changing the group format/structure (n = 5; 9.0%), and (e) adding other or more activities/information about the skill (n = 4; 7.2%).

Common themes to the question, “What will you do differently as a result of attending the group?” included the following: (a) engage with others and have more interactions/conversations (n = 16; 28.8%), (b) practice/utilize skill components (n = 15; 27.0%), and (c) nothing (n = 21; 38.1%). Some participants (n = 3; 5.4%) responded to this question in terms of changing their interactions within the group and general comments about the group activities which are beyond the scope of the question.

The penultimate question was “What other information would it be helpful to include in the group?” and common themes included: (a) other information relevant to the skill of making conversations (n = 7; 12.6%), (b) including more examples/group activities (n = 5; 9.0%), and (c) information about using the skill in different situations (n = 4; 7.2%). Some participants (n = 36; 65.4%) did not respond to this item, so they were recorded as “blank/nothing.” Also, some participants (n = 3; 5.4%) requested specific information about interpreting body language and non-verbal communication.

Finally, there was a question for any other comments. Common themes included the following: (a) positive views of the group overall (n = 8; 14.4%) and (b) positive appraisal of group facilitators (n = 2; 3.6%). Many participants (n = 44; 79.9%) did not respond to this item and one person (1.8%) reported disliking working with others as part of the group activities.

Discussion

Existing literature suggests that persons with psychiatric disabilities desire and attempt to engage in social roles such as employee and are capable of learning skills related to getting and keeping employment (L. L. Davis et al., 2012; M. Davis, 2003). Our findings suggest that participants with various psychiatric disorders, such as schizophrenia, major depression, and bipolar disorders, found the “Presenting Qualifications” and “Conversing with Others” interventions useful and helpful for their vocational needs.

Study 1 showed that individuals with psychiatric disabilities (i.e., schizophrenia/schizoaffective disorders, major depression, bipolar disorders, co-occurring disorders, anxiety disorders) can be instructed to use the skill, “Presenting Qualifications” in an employment interview. Participants provided positive feedback on the group intervention and the majority of individuals reported an increase in confidence and perceived preparedness to interview with a prospective employer.

Study 2 indicated that individuals with psychiatric disabilities (i.e., schizophrenia/schizoaffective disorders, major depression and depressive disorders, bipolar disorders, trauma-related disorders, co-occurring disorders, anxiety disorders) can be taught in a group setting to use the skill, “Conversing with Others” which involves teaching basic tenets of conversation. Overall, participants reported satisfaction with the group intervention and the acquired skill. Using the soft skill “Conversing with Others,” participants may develop their ability to chit chat/small talk at work, which has been shown to enhance employees’ daily positive social emotions, their organizational citizenship behaviors, and ultimately their work-related well-being in existing literature (Methot et al., 2020).

Overall, the studies provide initial evidence for the feasibility of using DST for work related soft skills among individuals with psychiatric disabilities. Although this study focused on two skills, using group methods to teach other work-related soft skills can result in greater confidence with using these skills in work environments. The DST approach focuses on teaching one critical skill per curriculum in a limited number of sessions (about four) that will enable employment success and satisfaction. Teaching participants one critical skill can be completed in 2 or 4 weeks depending on session frequency. This method is different from others that attempt to teach multiple skills over an extended time, for instance, the UCLA Workplace Fundamentals, which requires 5 to 6 months and biweekly 1.5-h-long sessions (Nuechterlein et al., 2020; Wallace & Tauber, 2004).

These results suggest that the group interventions “Presenting Qualifications” and “Conversing with Others,” delivered by graduate counseling students, have the potential to increase participants’ confidence when using work related soft skills. A focus on training in these employment-related skills for individuals with psychiatric disabilities is essential to the field because individuals with psychiatric disabilities strive to engage in valued life roles such as employee (M. Davis, 2003; L. L. Davis et al., 2012), but may experience challenges in gaining and keeping employment. Along with the aforementioned two skills, other work-related skills could be developed in employment training programs for this population (Lindsay et al., 2012). A recent randomized controlled trial found that supported employment combined with social skills training over 12 months, increased enrollment rates in educational programs and employment rates for those who experienced first episode psychosis (Nuechterlein et al., 2020).

This study offers promise for individuals with psychiatric disabilities as it provides introductory data for the feasibility of using soft skills training, focusing on one target skill, to enhance work-related soft skills. Such curricula can complement clients in need of such training who are receiving supported employment services. The curriculum emphasizes role playing to facilitate acquisition of the skill. When used as a complement in evidence based supported employment services such as individual placement supported employment model (individual placement and support [IPS]), this approach offers the advantage of supporting persons with psychiatric disabilities to practice skills in non-threatening environments. Future studies may examine if this approach could improve employment outcomes and job satisfaction for persons with psychiatric disabilities.

This study addresses another significant issue involving the approach practitioners use to assist individuals with psychiatric disabilities choose, get, and keep both employment and other valued life roles. Individuals may sometimes lack critical skills for succeeding in employment, so it may be beneficial and practical for practitioners to have a curriculum that is prepared specifically for a necessary skill that can be implemented with minimal training. The curriculum used in this study was developed to address this need. The graduate student facilitators learned how to organize, lead, implement, and evaluate the group interventions. The ability to implement this group intervention with minimal training potentially offers major advantages to agencies to provide this skill training and to maintain adherence to the content. All directions and materials for leading the groups are contained in the curriculum. A benefit of the approach used in this study appears to be that the curriculum can be successfully implemented with minimal training.

It is important to address the limitations of this research. First, there were no control groups to provide follow-up data to show if participants used the skills in the target environment of employment (i.e., acquiring or maintaining employment). Second, group facilitators included graduate students who were fulfilling internship requirements, which may not represent the same level of counseling skills that a seasoned clinician would provide. Furthermore, self-report data reported participant’s perception of interview preparedness skill or conversational skills, but follow-up on employment outcomes or other outcome variables were not examined in either study. Moreover, there was no indication that participants generalized the use of the skills taught outside of the sessions. So, there is a need for additional research specifically examining the impact of the intervention related to employment outcomes and generalized use. Also, the measures used to gauge perceived interview preparedness and conversation readiness have not yet demonstrated predictive validity, which will require further research. Finally, our adherence assessment was based on self-report and does not fully include other’s observation, which may present bias in measuring intervention delivery. Development of a robust fidelity measure to examine how consistent and accurately the intervention is delivered as intended would expand the generalizability of the findings. Fidelity is a critical factor that influences client outcomes and for evidence-based group treatments (e.g., Illness Management and Recovery (IMR) and group Cognitive Behavioral Therapy (CBT) fidelity helps maximize client outcomes (Barkowski et al., 2016; S. C. Cook et al., 2017; McLaughlin et al., 2019; Miles & Paquin, 2013).

Despite these limitations, both studies reinforce the feasibility of the DST approach for training entry level rehabilitation counselors to prepare individuals with psychiatric disabilities for employment at vocational agencies and other community programs. They also emphasize the utility of DST, specifically related to soft skills teaching in employment settings.

While findings were positive, a true experimental study of DST groups including control groups and longitudinal follow-up data regarding employment outcomes, job satisfaction, and collection of employer feedback will further inform the field. Future studies can also test whether the curriculum from this project can be adopted by other human services practitioners such as social workers or case managers. If so, this research could benefit many more practitioners, agencies, and most importantly, individuals with disabilities.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Akihito

Schaufeli

Kamiyama

Kawakami

(2014). Workaholism vs. work engagement: The two different predictors of future well-being and performance. International Journal of Behavioral Medicine, 22(1), 18–23. https://doi.org/10.1007/s12529-014-9410-x

Angrave

Charlwood

(2015). What is the relationship between long working hours, over-employment, under-employment and the subjective well-being of workers? Longitudinal evidence from the UK. Human Relations, 68(9), 1491–1515. https://doi.org/10.1177/0018726714559752

Banerjee

Chatterji

Lahiri

(2017). Effects of psychiatric disorders on labor market outcomes: A latent variable approach using multiple clinical indicators. Health Economics, 26(2), 184–205. https://doi.org/10.1002/hec.3286

Barkowski

Schwartze

Strauss

Burlingame

G. M.

Barth

Rosendahl

(2016). Efficacy of group psychotherapy for social anxiety disorder: A meta-analysis of randomized-controlled trials. Journal of Anxiety Disorders, 39, 44–64. https://doi.org/10.1016/j.janxdis.2016.02.005

Bond

G. R.

Drake

R. E.

Becker

D. R.

(2012). Generalizability of the Individual Placement and Support (IPS) model of supported employment outside the US. World Psychiatry, 11(1), 32–39. https://doi.org/10.1016/j.wpsyc.2012.01.005

Bond

G. R.

Drake

R. E.

Becker

D. R.

(2020). An update on Individual Placement and Support. World Psychiatry, 19(3), 390–391. https://doi.org/10.1002/wps.20784

Boreham

Povey

Tomaszewski

(2015). Work and social well-being: The impact of employment conditions on quality of life. The International Journal of Human Resource Management, 27(6), 593–611. https://doi.org/10.1080/09585192.2015.1027250

Bünnings

Kleibrink

Weßling

(2017). Fear of unemployment and its effect on the mental health of spouses. Health Economics, 26(1), 104–117. https://doi.org/10.1002/hec.3279

Carmona

V. R.

Gómez-Benito

Huedo-Medina

T. B.

Rojo

J. E.

(2017). Employment outcomes for people with schizophrenia spectrum disorder: A meta-analysis of randomized controlled trials. International Journal of Occupational Medicine and Environmental Health, 30(3), 345–366. https://doi.org/10.13075/ijomeh.1896.01074

10.

Cohen

Danley

Nemec

P. B.

(1985). Psychiatric rehabilitation practitioner package: Direct skills teaching. Boston University, Center for Psychiatric Rehabilitation.

11.

Colpe

L. J.

Barker

P. R.

Karg

R. S.

Batts

K. R.

Morton

K. B.

Gfroerer

J. C.

Stolzenberg

S. J.

Cunningham

D. B.

First

M. B.

Aldworth

(2010). The National Survey on Drug Use and Health Mental Health Surveillance Study: Calibration study design and field procedures. International Journal of Methods in Psychiatric Research, 19(Suppl. 1), 36–48. https://doi.org/10.1002/mpr.311

12.

Cook

J. A.

Leff

H. S.

Blyler

C. R.

Gold

P. B.

Goldberg

R. W.

Mueser

K. T.

Toprac

M. G.

McFarlane

W. R.

Shafer

M. S.

Blankertz

L. E.

Dudek

Razzano

L. A.

Grey

D. D.

Burke-Miller

(2005). Results of a multisite randomized trial of supported employment interventions for individuals with severe mental illness. Archives of General Psychiatry, 62(5), 505–512. https://doi.org/10.1001/archpsyc.62.5.505

13.

Cook

S. C.

Schwartz

A. C.

Kaslow

N. J.

(2017). Evidence-based psychotherapy: Advantages and challenges. Neurotherapeutics, 14(3), 537–545. https://doi.org/10.1007/s13311-017-0549-4

14.

Davis

L. L.

Leon

A. C.

Toscano

Drebing

C. E.

Parker

P. E.

Kashner

T. M.

Drake

R. E.

(2012). A randomized control trial of supported employment among veterans with posttraumatic stress disorder. Psychiatric Services, 63(5), 464–470. https://doi.org/10.1176/appi.ps.201100340

15.

Davis

(2003). Addressing the needs of youth in transition to adulthood. Administration and Policy in Mental Health, 30(6), 495–509. https://doi.org/10.1023/a:1025027117827

16.

Davis

Koroloff

Ellison

(2012). Between adolescence and adulthood: Rehabilitation research to improve services for youth and young adults. Psychiatric Rehabilitation Journal, 35(3), 167–170. https://doi.org/10.2975/35.3.2012.167.170

17.

Drake

R. E.

Bond

G. R.

Goldman

H. H.

Hogan

M. F.

Karakus

(2016). Individual placement and support services boost employment for people with serious mental illnesses, but funding is lacking. Health Affairs, 35(6), 1098–1105. https://doi.org/10.1377/hlthaff.2016.0001

18.

Drake

R. E.

Skinner

J. S.

Bond

G. R.

Goldman

H. H.

(2009). Social Security and mental illness: Reducing disability with supported employment. Health Affairs, 28, 761–770. https://doi.org/10.1377/hlthaff.28.3.761

19.

Druss

B. G.

(2014). Supported employment over the long term: From effectiveness to sustainability. The American Journal of Psychiatry, 171(11), 1142–1144. https://doi.org/10.1176/appi.ajp.2014.14070900

20.

Duncan

Dunifon

(2012). Soft skills and long-run labor market success. In Polachek

S. W.

Tatsiramos

(Ed.), 35th anniversary retrospective (Research in labor economics)(Vol. 35, pp. 313–339). Emerald Group. https://doi.org/10.1108/S0147-9121(2012)0000035036

21.

Ellison

Anthony

Sheets

Dodds

Barker

Massaro

Wewiorski

(2002). The integration of psychiatric rehabilitation services in behavioral health care structures: A state example. The Journal of Behavioral Health Services & Research, 29(4), 381–393. https://doi.org/10.1007/BF02287345

22.

Evans-Lacko

Henderson

Thornicroft

(2013). Public knowledge, attitudes and behaviour regarding people with mental illness in England 2009-2012. The British Journal of Psychiatry Supplement, 202(s55), s51–s57. https://doi.org/10.1192/bjp.bp.112.112979

23.

Farkas

Anthony

W. A.

(2010). Psychiatric rehabilitation interventions: A review. International Review of Psychiatry, 22(2), 114–129. https://doi.org/10.3109/09540261003730372

24.

Frey

W. D.

Drake

R. E.

Bond

G. R.

Miller

A. L.

Goldman

H. H.

Salkever

D. S.

Holsenbeck

(2011). Mental health treatment study: Final report. https://www.ssa.gov/disabilityresearch/documents/MHTS_Final_Report_508.pdf

25.

Gonzalez

Benitez

Martin-Martin

(2018). Long term unemployment, income, poverty, and social public expenditure, and their relationship with self-perceived health in Spain (2007-2011). BMC Public Health, 18, Article 133. https://doi.org/10.1186/s12889-017-5004-2

26.

Hiswåls

Marttila

Mälstam

Macassa

(2017). Experiences of unemployment and well-being after job loss during economic recession: Results of a qualitative study in east central Sweden. Journal of Public Health Research, 6(3), 995. https://doi.org/10.4081/jphr.2017.995

27.

Hsieh

H. F.

Shannon

S. E.

(2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687

28.

Zhang

Pacha

(2012). Employability skills valued by employers as important for entry-level employees with and without disabilities. Career Development and Transition for Exceptional Individuals, 35(1), 29–38. https://doi.org/10.1177/0885728811419167

29.

Julià

Vanroelen

Bosmans

Aerden

K. V.

Benach

(2017). Precarious employment and quality of employment in relation to health and well-being in Europe. International Journal of Health Services, 47(3), 389–409. https://doi.org/10.1177/0020731417707491

30.

Kyllonen

(2013). Soft skills for the workplace. Change: The Magazine of Higher Learning, 45(6), 16–23. https://doi.org/10.1080/00091383.2013.841516

31.

Lindsay

Adams

McDougall

Sanford

(2012). Skill development in an employment-training program for adolescents with disabilities. Disability and Rehabilitation, 34(3), 228–237. https://doi.org/10.3109/09638288.2011.603015

32.

Oursler

Gao

Herrick

Beninato

Hill

Durante

(2021a). Work-related interview skills training for persons served by Vocational Rehabilitation agencies in four states. Rehabilitation Research, Policy, and Education, 46(2), 146–160.

33.

Oursler

Herrick

Beninato

Gao

Brown

Durante

(2020a). Work-related interview skills training for persons with substance use disorders. Journal for Specialists in Group Work, 46, 146–160. https://doi.org/10.1080/01933922.2021.1900959

34.

Oursler

Herrick

Durante

Socha

Crisafulli

(2020b). Direct skills teaching of work-related conversational skills for individuals with disabilities. Rehabilitation Counseling Bulletin, 63(4), 235–244. https://doi.org/10.1177/0034355219878501

35.

Oursler

Herrick

Harris

(2019). Using direct skills teaching to improve job skills for persons with disabilities. Journal of Employment Counseling, 56(2), 69–84. https://doi.org/10.1002/joec.12113

36.

Oursler

Herrick

Socha

Zaher

Harris

Beninato

(2021b). A study of work-related interview skills training for people with visual impairments. Journal of Applied Rehabilitation Counseling. Advance online publication.

37.

Luciano

Meara

(2014). The employment status of people with mental illness: National survey data from 2009 and 2010. Psychiatric Services (Washington, D.C.), 65(10), 1201–1209. https://doi.org/10.1176/appi.ps.201300335

38.

McLaughlin

Barkowski

Burlingame

G. M.

Strauss

Rosendahl

(2019). Group psychotherapy for borderline personality disorder: A meta-analysis of randomized-controlled trials. Psychotherapy (Chicago, Ill.), 56(2), 260–273. https://doi.org/10.1037/pst0000211

39.

Methot

J. R.

Rosado-Solomon

E. H.

Downes

Gabriel

A. S.

(2020). Office chit-chat as a social ritual: The uplifting yet distracting effects of daily small talk at work. Academy of Management Journal, 64(5), 1445–1471. https://doi.org/10.5465/amj.2018.1474

40.

Miles

J. R.

Paquin

J. D.

(2013). Best practices in group counseling and psychotherapy research. In DeLucia-Waack

Kalodner

Riva

(Eds.), Handbook of group counseling and psychotherapy (2nd ed., pp. 178–192). SAGE. https://doi.org/10.4135/9781452229683.n10

41.

Mojtabal

(2011). National trends in mental health disability, 1997-2009. American Journal of Public Health, 101(11), 2156–2163. https://doi.org/10.2105/AJPH.2011.300258

42.

Nishi

Suzuki

Nishida

Mishima

Yamanouchi

(2017). Personal lifestyle as a resource for work engagement. Journal of Occupational Health, 59(1), 17–23. https://doi.org/10.1539/joh.16-0167-oa

43.

Nuechterlein

Subotnik

Ventura

Turner

Gitlin

Gretchen-Doorly

. . . Liberman

(2020). Enhancing return to work or school after a first episode of schizophrenia: The UCLA RCT of Individual Placement and Support and Workplace Fundamentals Module training. Psychological Medicine, 50(1), 20–28. https://doi.org/10.1017/S0033291718003860

44.

O’Leary

Livermore

G. A.

Stapleton

D. C.

(2011). Employment of individuals in the Social Security disability programs. Social Security Bulletin, 71(3), 1–10.

45.

Oursler

J. D.

(2020). Revised Perceived Conversation Easiness Questionnaire [Unpublished manuscript]. Rutgers University.

46.

Oursler

J. D.

Herrick

Harris

(2019). Using direct skills teaching to improve job skills for persons with disabilities. Journal of Employment Counseling, 56(2), 69–84. https://doi.org/10.1002/joec.12113

47.

Pierce

(2019). What employers want: Employers are looking to community colleges for help in teaching soft skills. Community College Journal, 89(3), 20–25.

48.

Premji

Shakya

(2017). Pathways between under/unemployment and health among racialized immigrant women in Toronto. Ethnicity & Health, 22(1), 17–35. https://doi.org/10.1080/13557858.2016.1180347

49.

Rogers

Anthony

Farkas

(2006). The choose-get-keep approach to psychiatric rehabilitation. Rehabilitation Psychology, 51(3), 247–256. https://doi.org/10.1037/0090-5550.51.3.247

50.

Salkever

D. S.

Gibbons

Drake

R. E.

Frey

W. D.

Hale

T. W.

Karakus

(2014). Increasing earnings of social security disability income beneficiaries with serious mental disorder. The Journal of Mental Health Policy and Economics, 17(2), 75–90.

51.

Sherer

Eadie

(1987). Employability skills: Key to success. Thrust, 17(2), 16–17.

52.

Shern

D. L.

Tsemberis

Anthony

Lovell

A. M.

Richmond

Felton

C. J.

Winarski

Cohen

(2000). Serving street-dwelling individuals with psychiatric disabilities: Outcomes of a psychiatric rehabilitation clinical trial. American Journal of Public Health, 90(12), 1873–1878. https://doi.org/10.2105/ajph.90.12.1873

53.

Substance Abuse and Mental Health Services Administration. (2020). Key substance use and mental health indicators in the United States: Results from the 2019 National Survey on Drug Use and Health (HHS Publication No. PEP20-07-01-001, NSDUH Series H-55). Center for Behavioral Health Statistics and Quality.

54.

Sumner

R. C.

Gallagher

(2017). Unemployment as a chronic stressor: A systematic review of cortisol studies. Psychology & Health, 32(3), 289–311. https://doi.org/10.1080/08870446.2016.1247841

55.

Tulgan

(2015). Bridging the soft skills gap: How to teach the missing basics to today’s young talent. Jossey-Bass.

56.

Waghorn

Childs

Hampton

(2012). Enhancing community mental health services through formal partnerships with supported employment services. American Journal of Psychiatric Rehabilitation, 15, 157–180. https://doi.org/10.1080/15487768.2012.679582

57.

Wallace

C. J.

Tauber

(2004). Supplementing supported employment with workplace skills training. Psychiatric Services, 55, 513–515. https://doi.org/10.1176/appi.ps.55.5.513

Feasibility of Work-Related Interviewing and Conversational Skills Trainings for Individuals With Psychiatric Disabilities

Abstract

Keywords

Method

Participants

Study 1

Study 2

Measures

Intervention

Analysis

Results

Study 1

Study 2

Discussion

Footnotes

Declaration of Conflicting Interests

Funding

References