Short- and medium-term impacts of the Just Like You disability awareness program: A quasi-experimental comparison of alternative forms of program delivery in New South Wales’ primary schools

Abstract

Given the demands for inclusive and more equitable education, this evaluation compared two versions of the Just Like You disability awareness program delivered in primary schools in New South Wales, Australia, by Cerebral Palsy Alliance. Participants included 297 students from three schools in the Hunter region in the established program and 327 students from four schools in the Sydney region in the newer version of the program. Compared with the established program, the new format comprised a single presenter with a disability, two shorter sessions (rather than one long session) delivered at least one week apart, revised content mapped to the Australian rather than the NSW curriculum and a non-compulsory ‘homework’ activity offered at the discretion of the presenter. Longitudinal data included three successive administrations (i.e. pre-test, post-test and delayed follow-up) of the Chedoke-McMaster Attitudes towards Children with Handicaps scale. Quasi-experimental comparisons produced intake-adjusted effect sizes favouring the newer program over the established program both in terms of post-test (d = 0.47) and delayed follow-up (d = 0.42) Chedoke-McMaster Attitudes towards Children with Handicaps scores. The program evaluation demonstrated the efficacy and sustainability of effects in this short-term focussed disability awareness intervention, with the newer version producing greater improvements in student attitudinal change towards people with a disability than the established version.

Keywords

Disability awareness inclusive education student attitudes primary school students intervention attitudinal change longitudinal study program evaluation

Introduction

As we move more firmly into the twenty-first century, it is time now for ideas and policy about inclusion to move forward once more, to fold around a range of matters concerning learning, community, identity and belonging. Inclusion has to be conceived with many surfaces – disability, certainly, and social justice, no less – but now other facets of life at school: community, social capital, equality and respect. (Thomas, 2013, p. 474)

Educational systems are seeking practical ways to address entrenched myths and misconceptions surrounding students with disabilities. Better understanding the needs of students with disabilities and having others appreciate their place and contribution in classrooms facilitates inclusive education (Beckett, 2009). Drawing on earlier work of Lee & Rodda (1994) and Stone & Colella (1996), Ison et al. (2010, p.361) makes the important point that it is the perceptions held about disability, whether they are accurate or not, that shape the way people with disabilities are treated. Thus interventions that include strategies to increase knowledge and break down stereotypes are likely to improve acceptance and attitudes towards people with disabilities (Lindsay & Edwards, 2013). Having fellow students and teachers understand the impacts of negative stereotyping has the potential to improve social and educational opportunities for students with disabilities (Milsom, 2006).

Research shows that negative attitudes towards children with a disability have an ongoing impact on the quality of life of children with a disability. Such attitudes affect their emotional, social, cognitive and physical wellbeing and have potential long-term implications, especially for their mental health (Lindsay, 2016; Lindsay & McPherson, 2012; Son, Parish, & Peterson, 2012). Disability awareness programs are often used to modify children’s attitudes to peers and others with a disability (see Tavares, 2011 for example).

Disability awareness programs focus on different participant ages and take many different forms and structures (Favazza & Odom, 1997; Rillotta & Nettelbeck, 2007). Common examples include programs for disability simulation, cognitive interventions, personal contacts with people with disabilities and information-based approaches. Some forms of intervention have been designed specifically for children, with the focus on the affective component of attitude, and less attention given to the cognitive aspects concerned with improving children’s knowledge of disability (Godeau et al., 2010).

Just Like You (JLY), a disability awareness program designed by Cerebral Palsy Alliance (CPA), aims to increase the awareness of the impact of disability on individuals and to demystify social stigmas associated with those who have a disability. The JLY program educates primary school students to develop understanding, acceptance and inclusive attitudes towards people with a disability, helping them recognise that people with a disability are ‘just like you’. The JLY program was delivered to over 25,000 primary school students in New South Wales (NSW) between 2006 and 2015. A group of consultants were recently engaged to refresh and extend the content of the JLY program across the primary years (for students aged 8–11, rather than those aged 10 and 11 only); two versions of the program (established and new) were subsequently delivered in different regions of NSW, Australia.

An earlier study undertaken when the JLY program was first developed (Ison et al., 2010), showed improvement in students’ knowledge and attitudes towards people with a disability, but concluded that further evaluation was required to determine whether these changes could be sustained over an extended period of time. Thus, the aims of the present study were to build on and extend beyond the earlier study by examining the longitudinal impact of the program, and to compare and contrast the two current delivery formats of the JLY program.

In general, there is a need to improve the attitudes of children towards persons with disabilities (Nowicki & Sandieson, 2002) and as Lindsay and McPherson (2012) identified, children with disabilities often feel excluded from activities at school and attribute much of this to a lack of understanding of their disability, thus signalling a need for effective disability awareness programs. Furthermore, it was identified at the time of the earlier study that there were no specific disability awareness programs within NSW which aligned with outcomes in the school curriculum, but that community-based providers may be able to fill this void (Ison et al., 2010). Hence, the earlier (pilot) study was undertaken to evaluate the JLY program designed, developed and delivered by CPA.

The primary aim of the JLY program (a short-term program) studied by Ison et al. was to improve the attitudes of children towards children with a disability, and results from the pilot study showed an improvement in the attitudes of the children who participated in the JLY program (Ison et al., 2010). The study however included three schools only, had no longitudinal components, and made no assessments about the potential role of gender. In another matched group study of 12- to 13-year-old French students, which evaluated the effectiveness of an intervention program aimed at improving students’ attitudes towards students with a disability, Godeau and colleagues (2010) found that over time, the intervention (a mandatory comprehensive educational project on disability) had no significant effect on Chedoke-McMaster Attitudes Towards Children with Handicaps (CATCH) scores. However, other researchers have reported that the duration of disability awareness programs does impact on program effectiveness (Brown, Ouellette-Kuntz, Lysaght, & Burge, 2011; Rillotta & Nettelbeck, 2007). Consequently, the aims of the current study were to establish the extent of the impact of the JLY program on attitudes of students towards students with a disability; determine the extent to which any changes were sustained over time; and, examine and compare the effectiveness of both delivery formats of the JLY program.

Methodology

This study set out to establish the type and degree of change associated with two versions of the short-term classroom-based JLY program (i.e. established and newer version), and to test the sustainability of any associated changes in the medium term (i.e. approximately three months post-intervention).

The classroom intervention programs

The two formats of the JLY program that were evaluated in this study are described in Table 1. The established program, an adaptation of that evaluated by Ison et al. (2010), which used a combined cognitive-behavioural approach (including contact with people with disabilities) and consisted of two sessions of 90 minutes each (p. 366). CPA changed to the new program in the Sydney region in 2015 as an improvement initiative. The JLY program was offered for the first time in Hunter region schools in 2015. However, the agreement with the sponsor (CPA) was to use the established program in the Hunter region schools and help fund the evaluation to establish the relative efficacy of the two delivery formats. Both programs in the current evaluation involved half the face-to-face contact time (one 90-minute or two 45-minute sessions) but used a cognitive-behavioural approach, similar to that of the original program previously evaluated by Ison et al. (2010). All programs had, for example students interacting with a person with a disability with time for questions and answers, discussion of different types of disabilities and discussions about appropriate communication/language towards people with a disability.

Table 1.

Comparison of the established and new delivery formats.

Aspect	Established program	New program
Length	One 90-minute session	Two 45-minute sessions
Presenter	Two trained presenters of whom at least one has a disability (co-presenting)	One trained presenter who has a disability
Content	Understanding the term ‘disability’The power of languageTypes of disabilityCommunication with a person with disability	Why uniqueness and difference should be valuedDifferent forms of disabilityImportance of inclusionEmpathising challenges people with disabilities faceUse of positive and inclusive language towards people with a disabilityOur role and contribution to positive and negative mindsetsKey principles we can learn from people who have overcome adversity
Curriculum linkage	Linkage to the NSW curriculum	Mapped to the Australian curriculum
Resources	Two fictional books donated to the school libraryMementos provided to students: two program booklets, pencil case with stationary and a reading list	Two fiction books donated to the school library
Cost	Provided free of charge to students and schools	Provided free of charge to students and schools

Source: Cerebral Palsy Alliance promotional materials.

In comparison with the established program, the new format comprised a single presenter with a disability, two shorter sessions (rather than one long session) delivered at least one week apart, revised session content mapped to the Australian (rather than the NSW curriculum) and a non-compulsory ‘homework’ activity offered at the discretion of the presenter. Booklets or mementoes were not provided. Presenters in both the Sydney and Hunter regions had received on the job training and were experienced in engaging and working with children. The same presenter delivered the program to all the Sydney schools. In contrast, in the Hunter region, pairs of trainers – one with a disability and one without a disability – delivered the program, whereby a different pair could deliver the program in different schools (see Table 1).

Recruitment and participants

Participants were recruited from seven government primary schools in the Sydney (four schools) and Hunter¹ (three schools) regions of NSW, Australia. These seven government schools were purposely selected from government schools that had already agreed to participate in the JLY program. Selection was based on class numbers, commencement date of the program and consent from the principal to participate in the study.

To maintain confidentiality throughout this article, the four Sydney region schools are referred to as S1, S2, S3 and S4, and the three Hunter region schools are referred to as H1, H2 and H3. Table 2 details the student population and Index of Community Socio-educational Advantage (ICSEA)² for each of the schools recruited.

Table 2.

Characteristics of schools in the study.

School code	Approximate school population (2015)^a	Approximate ICSEA (2015)^a	Number of classes^b	Sample size
S1	860–870	1105–1115	3	83
S2	380–390	1110–1120	4	107
S3	255–265	1145–1155	2	53
S4	370–380	1145–1155	3	84
H1	300–310	885–895	3	63
H2	245–255	905–915	3	71
H3	605–615	1070–1080	5	163
Sydney^c		M_S = 1127	12	327
Hunter		M_H = 995	11	297

Notes: ICSEA: index of community socio-educational advantage; M_S: mean of Sydney schools involved; M_H: mean of Hunter schools involved.

^aValues are reported in bands to maintain the confidentiality of schools.

^bNumber of classes from each school invited and participating in the study.

^cSydney and Hunter weighted average ICSEA for the schools in each program.

School S1 was the largest school from the Sydney region involved in the study and the largest overall. School H3 was the largest school in the Hunter region sample but considerably smaller than S1. The total sample size for each region, including all students that participated on at least one occasion, was similar between the two regions (Sydney schools, N = 327, 10% larger than Hunter schools N = 297). Twelve classes from the Sydney region were invited and participated in the study with similar average class sizes between regions (M_S = 27.25, standard deviation (SD_S) = 9.23), while 11 were from the Hunter region (M_H = 27.00, SD_H = 7.11). The average weighted ICSEA score for participating schools was 1127 for the Sydney sample (1.27 SDs above the national average ICSEA of 1000, SD = 100) and 995 for those from the Hunter region (0.05 SDs below the national average).

Participants were recruited from Stage 3³ in the above-mentioned schools. All students in Years 5 and 6 (generally 10- to 11-year-olds) at these schools were invited to participate in the evaluation, excluding S1 where participants were only from Year 5 (Year 6 students at S1 were not receiving the program at the same time as Year 5 or within the study period). There were no exclusionary factors. This age group was chosen in collaboration with CPA as the established format (Hunter) had been delivered to Stage 3 only, and to allow a fair comparison with the new delivery format, only those students from Stage 3 were invited to be involved.

This study was approved by the University’s Human Research Ethics Committee (HREC H2015-0130) and the State Education Research Applications Process (SERAP 2015265). Institutional consent for this study was obtained from the school principal prior to commencement of the study. Opt-out consent forms were provided to parents, since the school’s participation in the JLY program formed part of the regular classroom program. Students were also provided with a mechanism to opt-out on their own volition at the time of completion of the survey. Consent was also sought from the JLY program presenter(s) in each school.

Research design

In order to minimise the burden on and disruption to the delivery of the JLY program, a quasi-experimental design was chosen for the overall evaluation. The schools were assigned to one of two intervention groups by region (intervention Group 1 – Hunter schools; intervention Group 2 – Sydney schools) based on the delivery formats (i.e. two 45-minute session in the Sydney schools and one 90-minute session in the schools in the Hunter region) set by CPA.

The inclusion criteria consisted of participation in the JLY program in Term 3 or 4, 2015; institutional consent from the school principal to participate in the study (evaluation of the JLY program); sufficient student numbers in Stage 3 (minimum 25); and, location in either the Sydney region (where the new program was being delivered) or the Hunter region (where the established program was being received). The original design (see Table 1) included four schools in each of three groups, with Group 1 receiving the established program, Group 2 receiving the new program and Group 3 comprising two schools from each region, designated as the wait-list control group. However, difficulties with recruitment outside the researchers’ control forced a modification to the intended design, reducing the number of schools in the Hunter region to three and precluding the use of the wait-list control group.

Participants

Background and demographics

Table 3 presents background and demographic characteristics of the participants at baseline (N = 497). Overall, 53% (n = 263) of the sample were in Sydney schools and 47% (n = 234) in Hunter schools at baseline. Fifty-six percent of the overall sample was female and 44% male, which was relatively consistent across the two regions. The balance between year levels favoured Year 5 students overall (59%) but was more marked in the Sydney sample (65%) compared with the Hunter sample (52%). A higher percentage of the Hunter sample reported having a family member with a disability (46%) and a friend with a disability (52%) compared with the Sydney sample (29 and 33%, respectively).

Table 3.

Baseline characteristics of participants by location.

	Location
	Sydney		Hunter		Total
Characteristic	n	%	n	%	N
Sexa
Female	146	56	133	57	279
Male	117	44	101	43	218
Year levela
Year 5	171	65	121	52	292
Year 6	92	35	113	48	205
Knowing someone with a disability^a
Family member	76	29	107	46	183
Friend	87	33	122	52	169
Total^b	263	53	234	47	497

^aPercentage of regional sample.

^bPercentage of full sample analysed for primary outcome.

A total of 624 students responded to the survey on at least one or more occasions with a total of 497 usable surveys (a response rate of 79.6%). However, there was a relatively high incidence of missing data across occasions and the number of participants with valid data on the CATCH scales in both Sydney and the Hunter varied between occasions and across scales. The minimum number of respondents was 211 for overall CATCH on the follow-up administration in the Hunter and the maximum number of respondents was 262 for the affective and behavioural subscales on post-test administration in Sydney. These numbers were sufficient to establish statistical effects across the groups over time (Punch & Oancea, 2014).

Instruments and data collection

Assessments were conducted pre-intervention (baseline), post-intervention and at delayed follow-up. Baseline assessments occurred in the week prior to students receiving the JLY program, post-intervention assessments occurred within two weeks of receiving the program, and delayed follow-up approximately three months after the last JLY program session at each school. At all assessment sessions, an experienced research assistant read out the instructions and subsequently the questions to the students as they concurrently worked through the questions on a paper-based questionnaire.

The students completed a revised version of the internationally validated and reliable CATCH scales to measure their attitude towards children with a disability (Moore & Nettelbeck, 2013; Rosenbaum, Armstrong, & King, 1986; Vignes, Coley, Grandjean, Godeau, & Arnaud, 2008).

Permission was sought from and given by the author (Rosenbaum) to modify the wording of the original scales. The revisions included replacing some out-dated terminology (e.g. ‘retarded’ was replaced with ‘intellectual disability’, and ‘handicapped child’ with ‘child with a disability’).

To maintain consistency with the 1986 version of the CATCH scales (Rosenbaum et al., 1986), the final version consisted of 36 items with a 5-point Likert-type response format (strongly disagree, disagree, can’t decide, agree, and strongly agree) scored from 0 to 4. Negatively worded items were reverse-scored in the calculation of scale averages so that a higher total score indicated a more positive attitude towards people with a disability.

The overall CATCH score was calculated by summing the item scores, dividing by 36, and multiplying by 10, which obtained a score out of 40. This approach to scoring was used to ensure consistency with other studies that had used the CATCH questionnaire (Moore & Nettelbeck, 2013; Rosenbaum et al., 1986). These 36 questions consisted of 12 questions for each of the three CATCH sub-scales representing thethethe three components from Triandis’ attitudes model (1971), namely the affective component – related to feelings and emotional reactions (e.g. Item 15: I would be happy to have a child with a disability as a friend), the behavioural component – related to actual or intended behaviours (e.g. Item 7: I would stick up for a child with a disability who was being teased) and the cognitive component – related to knowledge, understandings and beliefs. (e.g. Item 19: Children with a disability know how to behave properly) (as cited in Vignes et al., 2008). In turn, each of the 12 questions in each sub-scale consisted of six positively and six negatively worded questions (Rosenbaum et al., 1986).

In this sample, the CATCH scales were found to have high internal consistency for the overall scale (at baseline, α = 0.90). Moore and Nettelbeck (2013) reported a similar internal consistency (α = 0.93) in their study. For each of the attitude components (affective, behavioural and cognitive) alpha reliability in the current study was 0.82, 0.83 and 0.66, respectively, at baseline. The convention for alpha reliability is that a scale has acceptable reliability if α > 0.7, although the acceptability of this figure remains contentious (Cortina, 1993; Shevlin, Miles, Davies, & Walker, 2000). Nevertheless, the lower reliability of the 12-item cognitive sub-scale flags the need for caution whenever its results are separately interpreted. De-emphasising the cognitive scale is consistent with findings of Bossaert and Petry (2013) who supported the use of a truncated version of the CATCH scales containing only affective and behavioural dimensions but with only seven items in total.

Background data of each student participant included school, class, year level and gender. Two additional questions asked the student participants if they had a family member with a disability (yes/no) and/or a friend with a disability (yes/no). In the post-intervention survey, students were also asked two open-ended questions related to the parts of the JLY program they liked most and liked least and these responses were coded thematically (Creswell, 2013).

As a measure of program fidelity or consistency, the presenters were asked to complete a very brief checklist either directly after each session, or shortly thereafter. The eight items on the checklist related to key concepts of the program (e.g. learning about inspiring Australians with disabilities and qualities of resilient people and challenging pre-conceived ideas about disabilities) with a tick box response option (not covered, incidental coverage, some direct coverage, detailed coverage, not applicable).

Analysis

Reliability analysis was undertaken using the baseline data. The Cronbach’s alpha (α) was used to assess internal consistency for the overall CATCH scale identified as the principal outcome of interest and the three CATCH sub-scales as component outcomes. The initial analysis considered differences in percentage agreement across the CATCH and CATCH sub-scales to help assess the general responsiveness of the student participants to the questions asked.

The main statistical tests were subsequently applied to identify differences between means for pre- and post-assessments for both intervention groups. The approach was to run a univariate analysis of covariance (ANCOVA) using SPSS software (IBM SPSS Statistics for Windows, 2015, version 23.0) to examine if there was a statistically significant difference between program type on post-test CATCH scores controlling for the respective pre-test CATCH scores, and delayed follow-up CATCH controlling for the respective pre-test CATCH scores (i.e. to statistically adjust for intake differences). Any effect associated with the intervention type that was dependent on prior attainment would thus have been ‘factored out’ during analysis. Student gender, having a friend with a disability and/or having a family member with a disability were included as additional fixed factors in the models.

Effect sizes in the current study were determined using Cohen’s d (d = (M₂ – M₁)/δ_p), where M₂ – M₁ is the mean difference between intake adjusted program scores and δ_p is the pooled SD of the intake adjusted scores). The meta-analysis of 213 school-based social and emotional learning interventions by Durlak, Weissberg, Dymnicki, Taylor, and Schellinger (2011) established average effect sizes of 0.23 for attitudes (including pro-social and attitudes about self) and 0.24 for positive social behaviours. Since the JLY program is a short classroom-based intervention (one 90-minute session in the Hunter program and two 45-minute sessions in the Sydney program) the expectation was that any significant differences would have an effect size in the range 0.16–0.20 (Cheung & Slavin, 2015; Slavin, 2013). Only students who completed the pre-intervention survey, participated in the program and completed the post-intervention or delayed follow-up survey were included in the analysis. In addition, any students that withdrew consent were excluded from the analysis. Statistical significance was set at p <.05 for all analyses.

Results

Percentage agreement (see Table 4) for the overall CATCH scales (representing the per cent scores in the ‘agree’ and ‘strongly agree’ range) was consistently higher for the Sydney program across the CATCH and CATCH sub-scale scores at pre-test, post-test and delayed follow-up compared with the Hunter program. Percentage agreement rates were in the range of 84–89% for the Sydney program and 70–74% for the Hunter program. The exception was for the cognitive sub-scale where percentage agreement rates were 46–66% for the Sydney program and 35–40% for the Hunter program.

Table 4.

CATCH percentage agreement across programs over time.

	Pre-test		Post-test		Delayed follow-up
Scale	Hunter	Sydney	Hunter	Sydney	Hunter	Sydney
CATCH overall	71	84	63	85	61	88
Affective	74	88	70	88	72	89
Behavioural	80	89	70	88	73	88
Cognitive	35	46	40	66	40	63

CATCH: Chedoke-McMaster Attitudes towards Children with Handicaps.

There were discernible differences in overall post-test CATCH scores across key segments of the data including having a family member with a disability, having a friend with a disability and being female. However, these differences proved not to be statistically significant after accounting for pre-test scores and therefore have not been reported in detail.

Comparison of program delivery format on the raw CATCH scores

Preliminary analysis also included inspection of the means and 95% confidence intervals of the raw CATCH scores at pre-test, post-test and delayed follow-up to assess the extent to which the raw CATCH results differed between the regions at different points in time (see Figure 1).

Figure 1.

Overall CATCH and CATCH sub-scale scores by region over time. School identifiers were obscured as an additional safeguard against breach of confidentiality. CATCH: Chedoke-McMaster Attitudes towards Children with Handicaps.

The scores in the Sydney program are observably higher than those in the Hunter program across all scales and points of assessment. It is also possible to compare the relative stability of the Hunter program scores over time compared with the upward trends between pre-test and post-test and pre-test and delayed follow-up in the Sydney program scores. Many of the comparisons between the pairs of raw means are statistically significant. However, these comparisons ignore the importance of comparing post-test scores after adjusting for pre-test differences (i.e. to assess the gains made after consideration of where the different groups started). For this reason, they are presented as indicative only and the final comparisons are based on intake-adjusted CATCH scores.

School-level differences were inspected in terms of raw pre-test overall CATCH scores to understand the level of between-school variation and further justify the need to use intake-adjusted comparisons (see Figure 2). A difference could be observed between the three Hunter schools with one having a higher average score and less dispersal compared with the first two Hunter schools, and being more similar to the three highest Sydney schools (F(6, 490) = 119.29, p < .000). The main effect for region was not significant after accounting for between-school differences.

Figure 2.

Pre-test overall CATCH scores by program area and school. CATCH: Chedoke-McMaster Attitudes towards Children with Handicaps.

Impact of program intervention type on the intake-adjusted CATCH scores

The post-test overall CATCH score was used as the principal outcome measure to assess the impact of program type, adjusted for pre-test scores to account for any achievement-level bias at baseline. This analysis was repeated with the CATCH sub-scales scores to provide supporting evidence. Univariate ANCOVA demonstrated statistically significant effects of program type on post-test overall CATCH and CATCH sub-scale scores after controlling for the respective CATCH pre-test score. The intake-adjusted mean scores for overall CATCH and the CATCH sub-scales (affective, behavioural and cognitive) were significantly higher for the Sydney program than those for the Hunter program (see Table 5). The adjusted differences favouring the Sydney program were small yet significant (1.41, 1.20, 1.03, 2.17 scores, respectively; p = .013) and associated values for Cohen’s d were small to moderate, indicating positive impacts favouring the newer Sydney delivery format (d values were 0.47, 0.31, 0.25 and 0.68, respectively).

Table 5.

Comparison of intake-adjusted CATCH and CATCH sub-scale scores.

CATCH scores	Hunter			Sydney			Mean difference
CATCH scores	M₁	SD₁	H₁	M₂	SD₂	H₂	M₂–M₁	95% CI	d	Significance
Post-test
Overall	27.60	2.99	182	29.00	2.98	220	1.41	[0.82, 2.00]	+.47	F(1, 401) = 21.95,p = .000
Affective	29.18	3.90	181	30.39	3.90	220	1.20	[0.43, 1.98]	+.31	F(1, 400) = 9.36,p = .002
Behavioural	29.57	4.09	182	30.60	4.09	220	1.03	[0.22, 1.83]	+.25	F(1, 401) = 6.23,p = .013
Cognitive	23.93	3.18	182	26.10	3.19	220	2.17	[1.54, 2.81]	+.68	F(1, 401) = 45.96,p = .000
Follow-up
Overall	27.48	3.50	182	28.93	3.51	220	1.46	[0.73, 2.18]	+.42	F(1, 372) = 15.83,p = .000
Affective	29.04	4.50	165	30.35	4.50	207	1.31	[0.38, 2.24]	+.29	F(1, 371) = 7.75,p = .006
Behavioural	29.41	4.51	166	30.80	4.52	207	1.40	[0.47, 2.32]	+.31	F(1, 372) = 8.80,p = .003
Cognitive	23.96	3.68	166	25.72	3.68	207	1.76	[1.01, 2.52]	+.48	F(1, 372) = 21.00,p = .000

CATCH: Chedoke-McMaster Attitudes towards Children with Handicaps; CI: confidence interval; SD: standard deviation.

Delayed follow-up CATCH scores were used to assess the longer term sustainability of impact of program type, which were also adjusted for pre-test scores to account for any achievement level bias at baseline. There was some variation in the timing of the delayed follow-up but in general this occurred 10–12 weeks after the classroom program delivery. The intake-adjusted Sydney program mean scores at delayed follow-up were significantly higher than the respective Hunter program mean scores for overall CATCH and the CATCH sub-scales (affective, behavioural and cognitive). The adjusted differences favouring the Sydney program at delayed follow-up were small yet significant (1.46, 1.31, 1.40, 1.76 scores, respectively; p = .006) and associated values for Cohen’s d were small to moderate, indicating positive impacts that also favoured the newer delivery format at delayed follow-up (d values were 0.42, 0.29, 0.31, and 0.48, respectively).

Three student background variables including gender, having a family member with a disability and having a friend with a disability were also fitted as additional fixed factors for analysis. However, none of these factors proved statistically significant in the ANCOVA models at post-test or delayed follow-up, after adjustment was made for pre-test scores. None of the interactions with program type were significant.

Process evaluation

Based on the post-intervention survey, 89% of students in Sydney indicated an aspect of the program they liked most compared to 85% in the Hunter. The three most popular aspects in Sydney were meeting and interacting with the presenter (20.8%), practicing sign language (16.5%) and the interactive or simulation activities (12.8%). The three most popular aspects in the Hunter region were meeting and interacting with the presenter (26.9%), interactive or simulation activities (21.4%) and the booklets (10.5%).

Students in the Hunter region, when asked what part of the program they liked least, 30% gave no response and 29.1% indicated that there was nothing they didn’t like. Of those students that did indicate parts of the program that they liked least, mostly related to the workbook activities/writing (8.8%) and length of time sitting (5.7%). In contrast students in Sydney, when asked what part of the program they liked least, 33.1% gave no response and 42.5% indicated that there was nothing they didn’t like. For those few in Sydney that did indicate parts that they liked least, the most common were learning sign language (9.5%) and seeing the challenges people with disabilities can face (4.3%).

Presenter checklists indicated that key concepts in Sydney schools had either some direct coverage (9.4%), detailed coverage (87.5%) or on one occasion ‘not applicable’ was ticked for a single item (investigating ways to seek help and support) when presenting at school S1. In the Hunter schools, overall it was reported key concepts received incidental coverage (25%), some direct coverage (41.7%) or detailed coverage (33.3%) during the program delivery. It was also noted that a short video clip which would normally be shown was not presented at one of the Hunter schools.

Discussion

The purpose of the research reported in this study was to examine the nature of student attitude change following the implementation of two versions of the JLY disability awareness program with children aged 10 or 11 years in the Sydney and Hunter regions of Australia. An additional aim was to assess whether any changes in attitude were maintained three months following the delivery of the program.

Notwithstanding some differences in the student and school characteristics of the Sydney and Hunter groups, the new version of the JLY program which involved two shorter sessions produced substantially greater improvements in CATCH scores overall and sub-scale attitude scores than the established version of the program. This difference was maintained at delayed follow-up. Intake-adjusted measures showed that the differences in attitude scores for the Hunter and Sydney versions of the program were modest but statistically significant.

There are a number of possible reasons for the differences described above. First, it may be that two 45-minute sessions delivered over a short period of time may be better suited to school students’ attention spans than a single, longer session. Additionally, two shorter sessions may have provided some opportunity for reinforcement of program content over time. Furthermore, students who participated in the version of the JLY program presented in Sydney were given ‘homework’; they were provided with a take-home sheet of the alphabet in AUSLAN and asked to learn how to sign their favourite food. This activity involved not only ‘homework’ but was an interactive, practical aspect identified in the post-intervention surveys as one of the things that students liked about the program.

A second possible reason for the attitudinal differences across programs may be that the Sydney-based program included more content material related to empathy, positive language and interaction with people with a disability than the Hunter-based program. The same presenter delivered the program to all the Sydney schools. In contrast, in the Hunter region, pairs of trainers – one with a disability and one without a disability – delivered the program, whereby a different pair could deliver the program in different schools (see Table 1).

Presenters were required to complete a checklist following each presentation, or shortly thereafter, as a measure of fidelity or consistency between program deliveries. Although a review of these did not suggest any major deviations from the program, there is still the possibility of a ‘teacher effect’. Plus, the presenter in Sydney may have been able to establish a greater rapport given there were two sessions compared to a one-off session. Although written responses in the post-intervention surveys showed that similar proportions of students in Sydney and the Hunter indicated that interacting with their presenter (a person with a disability) as one of the aspects they liked most. There are also other potential factors including exposure to and experience with people with a disability as noted below.

While this research points to the potentially beneficial aspects of the new program, the research has some limitations that need to be acknowledged. The quasi-experimental design of this study does not permit causal conclusions to be drawn. Resource and time constraints prevented the use of randomised matching of class groups across interventions and geographic location. For example it may be that the differences in the characteristics of the school students and the school communities across the Sydney and Hunter regions contributed to reported outcomes. Furthermore, the ICSEA index used to provide some comparative basis across schools does not report on a range of other potentially confounding aspects of school communities (e.g. school climate, number of students with a disability at the school, experience and disposition of school staff).

Review of the process evaluation data suggests there were differences in the content presented, or perhaps the way it was presented, in the Hunter region; for instance, some key aspects of the program were given incidental coverage yet direct coverage was given in other schools, while one school was not shown a video that was viewed by those at other schools. Nevertheless, the results of this research provide encouragement for further delivery and assessment of the shorter versions of the JLY program.

The literature suggests that an experiential component, such as ‘buddying’ activities, can have a powerful effect in improving school students’ attitudes to fellow students with a disability (Armstrong, Rosenbaum, & King, 1987; Rosenbaum, 2010). Such activities can be difficult to manufacture in the context of a short-term disability awareness program. However, providing meaningful opportunities for students with and without a disability to interact with each other should be part of the regular school curriculum and can be well integrated into targeted disability awareness programs. This issue is important because research on school students’ attitudes has shown that peer attitudes improve when students are given more opportunities to learn about disabilities (Boer, Pijl, Post, & Minnaert, 2013; Favazza & Odom, 1997; Godeau et al., 2010).

Conclusion

The findings of this evaluation demonstrate the efficacy and sustainability of effects from the participation of class groups in this short-term intervention. The new version of the JLY program was shown to enhance broader understanding of the value of uniqueness and difference, and position the personal and educational interests of students with disabilities in a more positive light, despite the limited period of participation in the program. Teachers and schools will find programs of this kind valuable as part of their broader policy response to the increasing demands for inclusive and more equitable educational provision.

Footnotes

Acknowledgements

We are grateful to the NSW Department of Education schools, students and parents/carers who participated in this project. We would also like to acknowledge the support of our students and research officers who supported our fieldwork, analysis and eporting: University of Newcastle Faculty of Education and Arts intern, Monica Waters-Neil, and School of Education intern, Joel Redman; and, research officers, Carly Berrigan, Alex Haschek, Hayley Maxfield, and Oriel Rose.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by Cerebral Palsy Alliance (CPA), NSW. The views and opinions expressed in this manuscript are those of the authors and do not necessarily reflect the views of CPA.

Notes

References

Armstrong

R. W.

Rosenbaum

P. L.

King

S. M.

(1987). A randomized controlled trial of a ‘buddy’ programme to improve children’s attitudes toward the disabled. Developmental Medicine & Child Neurology, 29, 327–336. doi:10.1111/j.1469-8749.1987.tb02486.x

Beckett

A. E.

(2009). ‘Challenging disabling attitudes, building an inclusive society’: Considering the role of education in encouraging non-disabled children to develop positive attitudes towards disabled people. British Journal of Sociology of Education, 30, 317–329. doi:10.1080/01425690902812596

Boer

Pijl

S. J.

Post

Minnaert

(2013). Peer acceptance and friendships of students with disabilities in general education: The role of child, peer, and classroom variables. Social Development, 22, 831–844. doi:10.1111/j.1467-9507.2012.00670.x

Bossaert

Petry

(2013). Factorial validity of the Chedoke-McMaster Attitudes towards Children with Handicaps Scale (CATCH). Research in Developmental Disability, 34, 1336–1345. doi:10.1016/j.ridd.2013.01.007

Brown

H. K.

Ouellette-Kuntz

Lysaght

Burge

(2011). Students’ behavioural intentions towards peers with disability. Journal of Applied Research in Intellectual Disabilities, 24, 322–332. doi:10.1111/j.1468-3148.2010.00616.x

Cheung

Slavin

R. E.

(2015). How methodological features affect effect sizes in education. Baltimore, MD: Johns Hopkins University, Center for Research and Reform in Education.

Cortina

J. M.

(1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78, 98–104. doi:10.1037/0021-9010.78.1.98

Creswell

J. W.

(2013). Research design: Qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: SAGE Publications.

Durlak

J. A.

Weissberg

R. P.

Dymnicki

A. B.

Taylor

R. D.

Schellinger

K. B.

(2011). The impact of enhancing students’ social and emotional learning: A meta‐analysis of school‐based universal interventions. Child Development, 82, 405–432. doi:10.1111/j.1467-8624.2010.01564.x

10.

Favazza

P. C.

Odom

S. L.

(1997). Promoting positive attitudes of kindergarten-age children toward people with disabilities. Exceptional Children, 63, 405–418. doi:10.1177/001440299706300308

11.

Godeau

Vignes

Sentenac

Ehlinger

Navarro

Grandjean

Arnaud

(2010). Improving attitudes towards children with disabilities in a school context: A cluster randomized intervention study. Developmental Medicine & Child Neurology, 52, e236–e242. doi:10.1111/j.1469-8749.2010.03731.x

12.

Ison

McIntyre

Rothery

Smithers-Sheedy

Goldsmith

Parsonage

Foy

(2010). ‘Just like you’: A disability awareness programme for children that enhanced knowledge, attitudes and acceptance: Pilot study findings. Developmental Neurorehabilitation, 13, 360–368. doi:10.3109/17518423.2010.496764

13.

Lee, T., & Rodda, M. (1994). Modification of attitudes toward people with disabilities. CanadianJournal of Rehabilitation, 7(4), 229.

14.

Lindsay

(2016). Child and youth experiences and perspectives of cerebral palsy: A qualitative systematic review. Child: Care Health Development, 42, 153–175. doi: 10.1111/cch.12309

15.

Lindsay

Edwards

(2013). A systematic review of disability awareness interventions for children and youth. Disability and Rehabilitation, 35, 623–646. doi:10.3109/09638288.2012.702850

16.

Lindsay

McPherson

A. C.

(2012). Experiences of social exclusion and bullying at school among children and youth with cerebral palsy. Disability and Rehabilitation, 34, 101–109. doi:10.3109/09638288.2011.587086

17.

Milsom

(2006). Creating positive school experiences for students with disabilities. Professional School Counseling, 10, 66–72.

18.

Moore

Nettelbeck

(2013). Effects of short-term disability awareness training on attitudes of adolescent schoolboys toward persons with a disability. Journal of Intellectual and Developmental Disability, 38, 223–231. doi:10.3109/13668250.2013.790532

19.

Nowicki

E. A.

Sandieson

(2002). A meta-analysis of school-age children’s attitudes towards persons with physical or intellectual disabilities. International Journal of Disability, Development and Education, 49, 243–265. doi: 10.1080/1034912022000007270

20.

Punch

K. F.

Oancea

(2014). Introduction to research methods in education (2nd ed.). London, UK: SAGE.

21.

Rillotta

Nettelbeck

(2007). Effects of an awareness program on attitudes of students without an intellectual disability towards persons with an intellectual disability. Journal of Intellectual and Developmental Disability, 32, 19–27. doi:10.1080/13668250701194042

22.

Rosenbaum

(2010). Improving attitudes towards children with disabilities in a school context [Commentary]. Developmental Medicine & Child Neurology, 52, 889–890. doi:10.1111/j.1469-8749.2010.03723.x

23.

Rosenbaum

P. L.

Armstrong

R. W.

King

S. M.

(1986). Children’s attitudes toward disabled peers: A self-report measure. Journal of Pediatric Psychology, 11, 517–530. doi:10.1093/jpepsy/11.4.517

24.

Shevlin

Miles

J. N. V.

Davies

M. N. O.

Walker

(2000). Coefficient alpha: A useful indicator of reliability? Personality and Individual Differences, 28, 229–237. doi:10.1016/S0191-8869(99)00093-8

25.

Slavin

(2013, January 9). Effect size matters in educational research [Web log post]. Retrieved from http://blogs.edweek.org/edweek/sputnik/2013/01/effect_size_matters_in_educational_research.html

26.

Son

Parish

S. L.

Peterson

N. A.

(2012). National prevalence of peer victimization among young children with disabilities in the United States. Children and Youth Services Review, 34, 1540–1545. doi: 10.1016/j.childyouth.2012.04.014

27.

Stone, D. L., & Colella, A. (1996). A Model of Factors Affecting the Treatment of Disabled Individuals in Organisations. The Academy of Management Review, 21(2), 352–401. doi: 10.2307/258666

28.

Tavares

(2011). An evaluation of the kids are kids disability awareness program: Increasing social inclusion among children with physical disabilities. Journal of Social Work in Disability & Rehabilitation, 10, 25–35. doi:10.1080/1536710x.2011.546296

29.

Thomas

(2013). A review of thinking and research about inclusive education policy, with suggestions for a new kind of inclusive thinking. British Educational Research Journal, 39, 473–490. doi:10.1080/01411926.2011.652070

30.

Triandis

H. C.

(1971). Attitude and attitude change. New York, NY: John Wiley & Sons.

31.

Vignes

Coley

Grandjean

Godeau

Arnaud

(2008). Measuring children’s attitudes towards peers with disabilities: A review of instruments. Developmental Medicine & Child Neurology, 50, 182–189. doi:10.1111/j.1469-8749.2008.02032.x