Abstract
Multiple evidence-based practices (EBPs) were identified in education, but relatively little attention has been paid to implementation. Special education teachers in the Chinese mainland faced great challenges in terms of a lack of knowledge and skills regarding EBPs. Asynchronous online training has been successfully provided for pre-service professionals on instructional methods or strategies. The present study evaluated the effects of self-paced online training on pre-service special education teachers’ case application of visual schedules to support individuals with intellectual and developmental disabilities. A pre/post-test control group design was utilized. Seventy-seven undergraduate students (mean age 20.5 years) who majored in special education participated in this study. Results indicated that the self-paced online training was effective in improving pre-service special education teachers’ case application of the visual schedules strategy. In addition, participants were in agreement on the social acceptability of the online module of visual schedules. The results are discussed from a perspective of effective and socially valid online training, along with limitations and future implications.
Introduction
Intellectual and developmental disability (IDD) is characterized by individual limitations in reasoning, learning, problem-solving, language development, social interaction, emotion regulation, self-care, and behavioral regulation which not only affect a range of everyday social and practical skills of individuals but also severely impact the cognitive and physical abilities (Sun, 2022; Zheng & Genaro Motti, 2018). Even when attending schools or educational programs, individuals with IDD especially intellectual disability and autism spectrum disorder share some challenges in their education. Specifically, students with IDD are at risk of becoming prompt dependent, relying on teachers or instructors to provide cues or prompts about the next step or activity in a sequence, and completing tasks or transitioning from one activity or setting to another independently (Smith et al., 2015; Spriggs et al., 2017; Van Dijk & Gage, 2019). Also, such challenges in turn lead to challenging behaviors and lower academic and functional achievement. Supporting students with IDD in the school setting is a critical step. To this end, visual schedules have been used to support students with IDD. Researchers found that visual schedules can be an effective tool to improve a range of skills and behaviors across age groups and settings for individuals with IDD (Hume et al., 2021; Liang et al., 2023a, 2023b; Spriggs et al., 2017; Van Dijk & Gage, 2019).
Visual Schedules
Visual schedules (VS) are a series of images, pictures, photographs, or line drawings that illustrate a sequence of events with the goal of visually preparing individuals for the next action or step within an activity or chain of activities (Hume et al., 2021; Knight et al., 2015). Multiple empirical studies indicated positive effects of VS on individuals with IDD across a variety of skills and behaviors including daily living, navigation, vocational, recreation, and academic skills; increasing independence and on-task behaviors and/or decreasing challenging behaviors (Akers et al., 2016; Knight et al., 2015; Koyama & Wang, 2011; Lequia et al., 2012; Spriggs et al., 2017; Van Dijk & Gage, 2019). In addition, studies demonstrate that VS interventions can be used for individuals with IDD in a variety of settings (e.g., general education classrooms, self-contained classrooms, community, clinic, and home) and implemented by multiple intervention agents (e.g., teachers, caregivers, and paraprofessionals; Knight et al., 2015; Koyama & Wang, 2011; Spriggs et al., 2017; Van Dijk & Gage, 2019).
Based on a large number of studies, visual schedules are evaluated as one of the evidence-based practices (EBPs) that were widely used in different settings across age groups and a range of skills and behaviors for individuals with IDD (Hume et al., 2021; Knight et al., 2015; Van Dijk & Gage, 2019). Therefore, special education teachers can feel confident to use these methods to support their students with IDD. However, there are few peer-reviewed empirical studies on the implementation of EBPs and no known peer-reviewed empirical studies to evaluate the effects of visual schedules on individuals with IDD in the Chinese Mainland (Fu & Xiao, 2018; Ma et al., 2023).
Status of EBPs and Teacher Preparation in Chinese Mainland
Since EBPs were identified in education, relatively little attention has been paid to implementation (Cook & Odom, 2013). However, due to the lack of research evidence on EBPs and the lack of knowledge and skills of special education teachers regarding EBPs (Hume et al., 2021), the situation in mainland China became more urgent. The journey from evidence to application of evidences often lengthy (Clark, 2020), and dissemination of information without a systematic process is likely to be inefficient. Researchers suggested that teacher preparation may be one way to bridge the gap between research and practice related to EBPs and prepare pre-service teachers to implement VS in their future work (Golder et al., 2005; McLeskey & Billingsley, 2008; McLeskey et al., 2018). Although some universities and colleges offer some courses on EBPs in special education teacher preparation programs, most of these programs focus on teaching the concepts and theories of EBPs (Fu & Xiao, 2018; Ma et al., 2023). In order for these EBPs to be widely implemented, work must be done to translate research findings into everyday classroom teacher practice (Cook & Odom, 2013).
Online Training
Compared with face-to-face traditional instruction, online training allows professionals to deliver interventions remotely by using communication technology such as the web and mobile applications, which has the potential to improve traditional service models by increasing access to evidence-based interventions (Baggett et al., 2010). Asynchronous online training (e.g., self-paced telepractice) provides information via websites and encourages learners to independently review and learn the material without communicating with trained professionals in real-time, which could be more flexible to access, time-independent, learner-controlled, cost-effective, and globally accessible (Ingersoll et al., 2016; Johnson, 2004; McCoy & McNaughton, 2021). Asynchronous online training has been successfully provided for pre-service professionals on instructional methods or strategies such as the system of least prompts (McCoy & McNaughton, 2021), discrete trials teaching (Pollard et al., 2014), reciprocal imitation training (Wainer & Ingersoll, 2013).
Present Study
Researchers in a large, public university in Pennsylvania summarized the steps of the visual schedules strategy and delivered the strategy instruction through the self-paced online module. This self-paced online module will be available at the AAC Learning Center Moodle (Liang & McNaughton, 2022) as one of the freely available modules for current and future teachers, and speech-language pathologists, around the world. In 2023 the AAC Learning Center Moodle has been used by over 6000 students at over 100 colleges and universities in 6 different countries.
Given the solid evidence base of visual schedules, there is no known peer-reviewed empirical study on the use of self-paced online training to teach the visual schedule strategy to pre-service special education teachers in the Chinese Mainland. To that end, researchers in the current study created an online training module (Chinese version) on the use of visual schedules and evaluated its effectiveness on pre-service special education teachers’ case applications of visual schedules. In order to increase positive outcomes for individuals with IDD, pre-service professionals need to access systematic, evidence-based training on the visual schedule strategy. As such, the current study sought to find the preliminary answers to the questions: (a) What were the effects of online training on pre-service special education teachers’ application of key steps of making and using the visual schedules based on applied case examples? And (b) What was the reported social validity of the online training for visual schedules?
Method
Participants and Setting
Participants were recruited from a 12-week course at a public university in northern China. Undergraduate students who majored in special education were enrolled in the courses. Seventy-seven students provided informed consent to have their data included in the current study and are referred to as participants. All students were required to complete the VS module and assessments.
At the beginning of the study, all participants completed a demographic questionnaire. Participants ranged in age from 20 to 22 years old (mean age 20.5 years). Ten participants were male and sixty-seven were female. Overall, 59 participants had no experience with individuals with IDD. Additionally, 18 participants had short-term experiences with individuals with IDD during volunteer opportunities. When considering the coursework, 36 participants reported having prior coursework on visual schedule strategies. The remaining 41 participants reported no prior coursework on visual schedule strategies.
The study made use of an online learning management system (LMS) at this university. All instructional content on the visual schedule was delivered online.
Independent Variable
The independent variable was a self-paced online instructional module. Researchers summarized the key steps of making and using visual schedules based on the published literature reviews (e.g., Knight et al., 2015; Koyama & Wang, 2011; Lequia et al., 2012). Then researchers developed the steps of the visual schedules strategy by using the acronym “CHECK!”. The steps of the visual schedules strategy (“CHECK!”) are as follows: • Choose materials that are appropriate and support self-monitoring. • Help the learner with colorful images and text and provide a motivating arrangement. • Engage the learner using Model(M)/Guided Practice (GP)/Independent Practice (IP). • Collect and use data to adjust instructions. • Keep the schedule where it will be used. • ! Celebrate success.
Module Development and Training Materials
Researchers used LMS to create and deliver the training materials. All training materials were presented as a combination of written text, pictures, and videos within the LMS. By using the LMS, students were encouraged to actively engage in learning by themselves instead of only following the instructors’ requests. In addition, general guidelines from EBPs in e-learning design and instructional design were utilized to create the online training materials. First, the explanations in each section were short and succinct with relevant visuals and examples incorporated (Clark, 2020; Clark & Mayer, 2016). Second, for each section, there were frequent opportunities for participants to engage with the content (e.g., multiple choice questions and questions linked with case examples; Clark, 2020; Clark & Mayer, 2016). Third, instructional design and principles for multimedia (Clark, 2020; Clark & Mayer, 2016) were applied including (a) contiguity, presenting words and graphics near one another on-screen, and (b) coherence, limiting the quantity of information presented (i.e., less is more).
The main sections of this module included (a) introduction, (b) model and guided practice, and (c) independent practice. Within each section, some self-check practices were provided to help the participants learn the key points or principles of VS and the “CHECK!” strategy. Such activities were designed to be answered in different ways: (a) multiple-choice questions based on case examples and (b) short answer questions based on case examples. The applied case examples used in case application assessments, knowledge assessment, and self-check practices were created by the researchers based on real-life situations in different educational settings, and some of them were adapted from published peer-review studies (e.g., Cohen & Demchak, 2018). The details in each section of this self-paced online training module were as follows.
Introduction
The first section began with the introduction of the visual schedules (e.g., definition and key considerations) and a description of the “CHECK!” strategy. Descriptions were a combination of text and images. In addition, each step of the “CHECK!” strategy was explained separately, and multiple-choice questions were posed and required participants to answer prior to moving on to the subsequent section.
Model and Guided Practice
After the instruction of key concepts and instructional steps, the implementation of the “CHECK!” strategy was modeled. In the model section, a case example was shown along with a detailed description of the “CHECK!” strategy. Next, a new case example was introduced in the guided practice section and participants were provided with the opportunity to practice naming and describing the required instructional steps based on “CHECK!” (Kent-Walsh & Mcnaughton, 2005; McCoy & McNaughton, 2021).
Independent Practice
Independent practice activities were included at the end of the module. The independent practice was also based on a case example that was adapted from a published study. In this section, the participants were expected to use the newly acquired “CHECK!” strategy to provide short answers based on the given situation in the case example.
Procedures
Assessment Procedures
Based on the research design (see below), all the participants in both study groups (control and experimental) produced responses online for pre-probe (probe#1). Half of the participants in each study group were randomly assigned to Case One and the others were assigned to Case Two. Next, the participants in the experimental group started learning VS module. Finally, all participants completed the post-probe (probe#2) in the classroom. At this stage, the participants who completed Case One in pre-probe received Case Two, and vice versa.
Training Procedures
The online training in this study was self-paced and participants could complete the 40-min training at any time within a one-week period. The participants in the experimental group learned the contents of the self-paced online module section by section without additional feedback from the instructor.
Research Design
Research Design and Procedures.
Dependent Variables
The main dependent variable was the accuracy of the strategy steps as used in the applied case application of making and using the visual schedule. The case application assessment (See Appendix A and B for cases one and two along with participant instructions) consisted of two cases. In each case, detailed background information on individuals with IDD was provided. The individuals needed ongoing support (e.g., teachers or teaching aides) to complete academic tasks or routine activities independently in preschool and school settings and were also potential users of VS. Based on the background information, participants were required to provide a short-written description of the “CHECK!” strategy. The descriptions were evaluated by the correct steps of making and teaching VS as written on the answer sheet. The participants could also choose to draw a picture of the VS that they planned to make.
The researcher developed the case examples and questions after reviewing multiple sources on visual schedules (e.g., Bryan & Gast, 2000; Cohen & Demchak, 2018; Koyama & Wang, 2011; Pierce et al., 2013). Then the case examples and short-written questions were piloted with a small group (n = 7) of special education Ph.D. and master’s students. Furthermore, to strengthen the validity of the case application assessment questions, three faculty members with a minimum of 5 years of experience in intervention for individuals with IDD, especially all had the experience of implementing visual schedules, reviewed the case examples and questions and confirmed that the test items appropriately addressed the concept, procedure, and conditions of making and using visual schedules strategy for individuals with IDD.
Data Collection and Analysis
For scoring the response on the case application assessment (case response), researchers created a rubric (See Appendix C) based on the “CHECK!” strategy. The case response was evaluated by scoring the steps written in the responses (the full point is 10). SPSS software was used to complete the analysis. The researcher used an independent sample t test to compare the pre-and post-probe assessment between groups.
Training of Data Coder
A teaching faculty who had a master’s degree in special education worked as a coder. The training was provided individually and consisted of description, model, practice, and feedback. The researcher described and explained the rubric to the coder, and selected case responses randomly to model how to code. The researcher randomly selected 15 (20%) participants’ case responses from both groups. The researcher and coder coded case responses independently. Then, the researcher discussed with the coder on disagreement scores of certain steps in the case responses and asked the coder to practice until performance reached 100% agreement with the researcher.
Social Validity
Participants completed a social validity questionnaire (See Appendix D) developed by the researchers based on the framework of social validation by Schlosser (1999). Participants were asked to evaluate elements of social validity and to provide their perceptions on the importance of learning VS, the acceptability of the instructional methods used, and the impact of the instruction. Participants provided information by responding to the questions on a Likert scale of 1–5 (with 1: strongly disagree and 5: strongly agree) as well as one open response. The social validity assessment was part of the online module and all participants had to submit social validity information before the completion of learning the module.
Results
Means and SD of Participants by the Group.
Participant Social Validity Responses.
Case Application of Visual Schedules
At pre-probe assessment, the participants in the control group accurately implemented a mean of only 2.5 out of 10 steps, and participants in the experimental group accurately implemented a mean of only 1.8 out of 10 steps. Without training, participants completed a mean of less than 25% of the strategy steps. At post-probe assessment, the participants in the control group accurately implemented a mean of only 2.6 out of 10 steps, and participants in the experimental group accurately implemented a mean of 8.9 out of 10 steps. The experimental group (who completed the training) showed a gain of +7.1 (range +1 to +10) in steps completed accurately from the pre-probe to post-probe interactions. In comparison, the control group (who did not complete the training) demonstrated a gain of +.1 in steps completed accurately from the pre-probe to post-probe interactions.
An independent samples t test was run to confirm the difference between the gain scores of the experimental group and the control group. The results indicated no statistically significant effect for the pre-probe between the experimental and the control group indicating initial group equivalence. The gain scores were higher for the experimental group than the control group, with a statistically significant difference, t (1, 75) = 20.32, p < .001, d = 1.56. These data indicated an increase in the case application of visual schedules for participants in the experimental group after receiving the online training. Cohen’s d was calculated to determine the magnitude of the effect of the training. Results indicated a large effect (d = 1.56) since effect sizes greater than .8 are considered large (Cohen, 1988).
As shown in Figure 1, all the participants applied the step of helping learners with meaningful representations (100%). The majority of the participants applied the steps of choosing materials that are appropriate (80%) and support self-monitoring (90%); providing a motivating arrangement (85%); engaging the learner using the model (87%); guided practice (97%); independent practice (82%); collecting data and using data (85%); keeping the schedule where it will be used (92%) and celebrating success (95%). Case application of CHECK! strategy steps.
Social Validity
All the participants were provided with an opportunity to provide feedback about the online training goals, materials, and procedures anonymously. Most of the participants rated all social validity statements between a 5 (strongly agree or strongly improved or strongly recommended) and a 4 (agree or improved or recommended) on a 5-point scale. The percentage of participants’ ratings can be found in Table 3. The highest-rated questions indicated that participants felt the goals of learning the knowledge of making and using a VS were very important, the self-paced online learning methods were very helpful, and their skills had improved through the online training program on visual schedules. In addition, they felt more confident in supporting their students with IDD in the future.
In addition to rating four questions on a 5-point scale, participants were provided with one open-ended question about the suggestions for learning this module. Students suggested the following for improvement: adding more interactive and engaging activities, more independent practices, and more case examples in real-classroom settings.
Discussion
The current study sought to find the preliminary answers to the questions: (a) What were the effects of online training on pre-service special education teachers’ application of key steps of making and using the visual schedules based on applied case examples? And (b) What was the reported social validity of the online training for visual schedules?
Results indicated that the self-paced online training program was effective in increasing pre-service special education teachers’ case application of the visual schedules strategy. Furthermore, participants were in agreement on the social acceptability of this online module. The results are discussed further below, from a perspective of effective and socially valid online training, along with limitations and future implications.
Effective Online Training
There is an evident need for teachers or educators who work with individuals with disabilities to have effective training on evidence-based instructional strategies (Alexander et al., 2015). Previous research has demonstrated that asynchronous online training was effective in increasing pre-service professionals’ evidence-based instructional strategies (e.g., system of least prompts, discrete trial teaching, and reciprocal imitation training). The current study extended the research by utilizing self-paced online training to teach visual schedule strategies and accumulated evidence of the effectiveness of self-paced online training for pre-service professionals. Furthermore, the empirical study provided experience of how to translate research findings of EBPs into practice for researchers and teacher educators in China.
The question of how to support pre-service teachers in implementing EBPs in practice is a big challenge in teacher preparation. Generally, to solve this problem, researchers in this study first introduced the situations in which visual schedules can be helpful for individuals with IDD. Then, researchers introduced and demonstrated each step of the summarized strategy (CHECK!) based on real case examples. After that, learners were required to practice implementing the strategy in the applied case examples with a gradually decreasing level of prompting. Specifically, researchers in this study utilized multiple strategies to ensure the effectiveness of the online training program.
Researchers in this study organized and demonstrated the sections of the online training module in an effective way. As the first stage of this training module, the researchers utilized the video and the case example to help learners identify the importance and usefulness of making and using visual schedules for students with IDD. Then, the researchers described and demonstrated each step of the strategy with concise text and related images. Following this, researchers also used multiple stages of practice with gradually decreasing levels of prompting and feedback. Furthermore, general guidelines from EBPs in e-learning design were utilized when creating the online training module. Instruction in this module used short and succinct explanations with relevant visuals and examples that may support the learners to understand the key concepts and steps easily and clearly. The frequent follow-up activities for each section supported the mastery of learning concepts and steps and also engaged participants. The instructional design and principles for multimedia were also used in this online module, presenting words and graphics near one another on-screen and limiting the quantity of information presented, likely reducing the cognitive load associated with the task.
Socially Valid Training
Results from this study extended the work of recent research utilizing online instruction to teach specific EBPs, which have reported practitioner satisfaction with the training delivery format (e.g., online module; McCoy & McNaughton, 2021; Neely et al., 2016). In addition to the online training materials and delivery format, participants in the present study agreed with the training outcomes. Specifically, participants responded that: (a) online training improved their skills in VS, (b) they had learned an important skill for future work, and (c) online training increased participants’ confidence in supporting students with IDD. In addition, some participants responded to open-ended questions and some participants also suggested using different ways to practice, using more interactive activities, and providing more examples in the natural environment.
Future Implications
As the current study sought to explore the effects of self-paced online training on the pre-service special education teachers’ case application of visual schedules, future studies are necessary to find more effective ways of developing online training programs and delivering the training to support pre-service professionals as they learn how to use visual schedules or other EBPs for individuals with IDD.
Several recommendations could be drawn from the results of this empirical study for future research and practice. Future studies should evaluate the effects of online training programs for other groups of pre-service professionals for example pre-service general education teachers who would work with individuals with IDD or other disabilities in the inclusive classroom. Future online training modules should focus on practice or performance (e.g., how to make a real VS) by providing “engaging models”. To enhance generalization, researchers could use videos to model how to make a good VS, how to teach students to use the VS (e.g., M/GP/IP), and how to use the VS strategy to support certain students or clients instead of simply showing the strategy. By doing so, researchers could consider making real “products” of visual schedules via Microsoft Word and practicing the use of the VS strategy through role-playing.
In order to increase active learning, researchers could further consider using other EBPs in e-learning designs to increase active learning by (a) incorporating frequent behavioral activities that promote content transformation; (b) using questions throughout online training events to promote learning not only questioned content but adjunct content as well; (c) encouraging self-explanations of instructional content, including text explanations, video lessons, examples, and diagrams; (d) considering assignments that involve the transformation of textual information into visual-spatial representations such as flow charts or simple sketches; (e) assigning learners content resources to convert into lessons for others unfamiliar with the content. Researchers could also ask learners to provide explanations along with relevant drawings and assess the generalization of knowledge to real settings in schools and community settings.
Limitations
In terms of limitations in this study. First, and most importantly, the researcher in the current study did not purposely provide training for generalization and evaluate the generalization of the training effect. In addition, we could not know the accuracy of implementation when the participants of this training used the visual schedules strategy in terms of making a real visual schedule and using it for certain students. This is a limitation that the practical significance of this online training in the current study is unclear. Also, we could not know what differences the participants could make for the students with IDDs when using the visual schedules strategy in the classrooms.
It should be acknowledged that some participants did not demonstrate improvement in case application assessment. Future research should consider the evaluation of these participants through a three-tiered analysis (Light, 1999) to identify factors that may increase the likelihood that the participant will participate and respond to the online training. As such, it is important for researchers to not only discuss those participants who demonstrated improvement but also those for whom the training did not result in the desired outcome to create training that is effective for all participants. In addition, only one open-ended question was used in the social validity survey and the question was not required. Most participants did not provide responses to this question.
Lastly, there are some potential factors that may hinder the outcome of learning and performance on the post-probe assessment including that (a) the activities in the online training module were just focused on the exact steps of the “CHECK!” strategy instead of demonstrating the specific use of VS and (b) the instructions on the key steps in this module were mainly text-based description which may fail to motivate the learners.
Conclusion
To summarize, the present study added preliminary evidence that self-paced online training was effective in increasing pre-service special education teachers’ case application of visual schedules for individuals with IDD. In addition, this study adds to the extant literature by extending the effectiveness and social validity of online instruction when teaching evidence-based practice for pre-service special education teachers. This study is the first step in supporting pre-service teachers to implement evidence-based practices in teacher preparation programs in the Chinese mainland. Future research is needed to evaluate the effects of online training on the application and generalization of visual schedules for pre-service professionals.
Supplemental Material
Supplemental Material - An Evaluation of Online Training on Pre-service Special Education Teachers’ Case Application of Visual Schedules for Individuals With Intellectual and Developmental Disability in China
Supplemental Material for An Evaluation of Online Training on Pre-service Special Education Teachers’ Case Application of Visual Schedules for Individuals With Intellectual and Developmental Disability in China by Zhigao Liang, Juanjuan Zuo and David Lee in Journal of Special Education Technology
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the [Tangshan Normal University] under Grant [number 2022C36]; [Tangshan Normal University] under Grant [number 2022JG09]; [Education Department of Hebei] under Grant [number SZ2023194]; and [Tangshan Normal University] under Grant [number 2023XTTZ024].
Supplemental Material
Supplemental material for this article is available online.
Author Biographies
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
