The role of transformational leadership and an experienced team in sustaining a technology enhanced learning intervention in schools

Abstract

Studies on school factors contributing to sustaining innovations hardly focused on differences across schools. We investigated school factors that are associated with the sustained implementation of a technology enhanced intervention across 59 Dutch elementary schools. Educational professionals of these schools (n = 551) rated various school factors and the sustained implementation of the intervention. Data were analyzed by two-level regression analyses separating within and between school relations. Based on perceptions of teachers within schools, we found significant correlations between sustained implementation and a school's social capital and absorptive capacity. However, these relationships are not confirmed on the school level. Differences between schools in sustaining the intervention were uniquely related to the school principal's transformational leadership and the team's experience.

Keywords

Sustained implementation innovation school factors transformational leadership team experience

Achieving sustained implementation of innovations in schools is a critical challenge for education researchers and practitioners (Krijgsman et al., 2022; Prenger et al., 2022). Sustained innovations are innovations that have become a routine and integral part of daily practice in schools (Prenger et al., 2022). Innovations that are sustained can create lasting changes in the school environment and might improve student outcomes. Also, for teachers it is important to see that innovations are prolonged. When innovations are stopped or slowly fade out, positive energy to change might be replaced with cynicism about the usefulness of innovations (McIntosh et al., 2018), whereas positive experiences with long-lasting innovations can stimulate teachers to take-up new innovations.

Despite the well-recognized importance to sustain educational innovations, few innovations are actually sustained in schools (Hargreaves and Goodson, 2006). In part this might be due to a lack of knowledge of the school factors that enhance sustained implementation of innovations (Prenger et al., 2022). Research on sustained implementation of innovations has mostly involved the development of theoretical frameworks (e.g. Aarons et al., 2011) and explorative and qualitative studies about school factors that might contribute to sustained implementation (see for an overview Prenger et al., 2022). Moreover, the relevant school factors are mostly based on individual perceptions of teachers within a school about what is important to sustain innovations. In none of the 44 studies included in the review by Prenger and colleagues (2022) it was examined which school factors contribute to differences between schools in sustaining innovations. In addition to the review of Prenger and colleagues (2022), we also examined research that focused specifically on the sustained implementation of the School-Wide Positive Behavioral Interventions and Supports (PBIS) based on the review of Fox and colleagues (2021). In this line of research, some quantitative studies focused on school factors that contribute to sustained implementation of PBIS, but these studies did not investigate differences between schools (e.g. Mathews et al., 2014; McIntosh et al., 2015) or only examined differences between schools with respect to general sociodemographic factors like number of students, district or educational level (McIntosh et al., 2018; Pas et al., 2015; Schaper et al., 2016) and less on school characteristics such as the schools’ implementation climate and the role of transformational leadership.

Investigating school factors on the school level related to the implementation climate is needed as interpreting school factors based on the perceptions of teachers within schools do not provide evidence as to whether these factors can explain between school differences in the extent to which innovations are sustained (Morin et al., 2014). Previous research, especially in the area of school effectiveness, has shown that it is important to measure school factors at the school level as this could yield different results from the results when only the within school level is considered (e.g. Lloyd et al., 2021; Morin et al., 2014; Ninković, 2023; Wong and Li, 2008).

To address this need, we investigated which school factors contribute to the sustained implementation of one intervention, Build! at the within school and the between school level. This intervention is a prevention program for young children at risk for reading difficulties and is widely implemented in elementary schools in the Netherlands. At the within level the focus is on the relation between differences in the (perception) of school factors and differences in the perception of the extent to which the implementation of the intervention has been sustained within schools. At the between level, the focus is on differences between schools with respect to relevant school factors.

Sustained implementation

In this study we focus on an open innovation. An open innovation is seen as a process in which external knowledge, or a technology developed outside the school, is integrated in the school (Pietsch et al., 2024). We focus specifically on the phase of sustaining an open innovation.

Prenger et al. (2022) define sustainability as: “the process of continuing and integrating the innovation's core aspects in organizational routines that are adaptive to ongoing work” (p. 14). This definition entails two important dimensions of sustained implementation. The first dimension describes whether the innovation has become a part of the organizational routines of an organization (Spillane et al., 2011). We refer to this dimension as the Organizational Routine dimension. If an innovation is part of the organizational routine, it is taken for granted, seen as a priority in the school and generally supported by the school team (McIntosh et al., 2013). In practice this would mean for example that an innovation is sustained when teachers agree to practice two times a week with the program Build! with students at risk for reading difficulties because they believe that this intervention is necessary for the progress of their students and these students would otherwise run the risk to lag behind in reading.

The second dimension of sustained implementation reflects the extent to which an innovation is monitored and adapted (McIntosh et al., 2013; Prenger et al., 2022). Monitoring an intervention can be done by evaluating how the use of the intervention can be optimized, and by routinely evaluating the quality of the intervention. In practice this would mean for example that the progress children make within the intervention Build! is monitored continuously and discussed in team meetings on a regular basis. This dimension will be further denoted as the Monitoring and Adaptation dimension.

School factors related to sustained implementation

There are various conceptual frameworks that describe determinants of implementation (Damschroder et al., 2009, 2022; Durlak and Dupre, 2008; Kirk et al., 2015) and sustained implementation in particular (Prenger et al., 2022). In this study we focused on the organizational level determinants. Based on these overviews we investigated the following school factors in relation to sustained implementation: transformational leadership in the school (Geijsel et al., 1999), social capital of a school (Nahapiet and Ghoshal, 1998), a school's openness to change (Mitchell and Shoho, 2017), a school's absorptive capacity (Cohen and Levinthal, 1990) and the number of years the team works in the same school. In what follows we consider how these school factors might relate to the sustained implementation of innovations in schools.

The type of leadership potentially influences the development, implementation, and maintenance of interventions (Durlak and Dupre, 2008; Geijsel et al., 2003; Moolenaar and Sleegers, 2010). Transformational leadership is defined as a leadership style that transforms employee attitudes, values, and goals so that work is performed for the collective good (Geijsel et al., 1999). Transformational leadership is known for its 4 Is, idealized influence, where leaders act as role models; inspirational motivation, where they articulate a compelling vision; intellectual stimulation, where they encourage innovation and critical thinking; and individualized consideration, where they provide personalized support (Geijsel et al., 1999). Transformational leadership motivates employees to do more than they intended to do and more than they thought possible because they positively stimulate colleagues and create a shared vision and ideas with the team (Geijsel et al., 2003). Previous research has found that transformational leadership is directly or indirectly associated with, to name a few, technology integration (Schmitz et al., 2023), use of digital learning materials (Vermeulen et al., 2015), innovative practices (Kılınç et al., 2022) and teacher work performance (Khan, 2023). A school leader's belief that an intervention is effective can have a positive impact on the extent to which the implementation of an intervention is sustained, acting as a role model who believes in the innovation having an idealized influence (Bean et al., 2015). Moreover, the intellectual stimulation of educational professionals is important for being able to be innovative as a team (Moolenaar et al., 2010; García-Morales et al., 2012). Transformational leadership is also important for achieving collective goals (Wang et al., 2011), being inspirational, motivating teachers, and creating a shared vision to sustain an innovation in teachers’ teaching practice and giving individual consideration where needed. Therefore, we suggest that transformational leadership is positively related to sustained implementation of an innovation.

Social capital refers to the actual and potential resources embedded in relationships among actors that allow professionals to share information, gain trust, and build a shared vision. Social capital is positively related to innovation in schools (Moolenaar, 2010), the implementation of new policies and reforms (Coburn, 2001), and the implementation of interventions (Auckland and Kilpatrick, 2018; Chen et al., 2016; Prenger et al., 2022). If professionals in a school work together on the same shared goals and visions, through interaction and reflection in a trustful environment (Nahapiet and Ghosal, 1998), this environment will possibly enable professionals to challenge taken-for-granted ways of working and bring innovation in their own educational practices (Prenger et al., 2022). Therefore, schools with a high degree of social capital will be more inclined to sustain an innovation.

Openness to change refers to a school's willingness to embrace new ideas and change, and act accordingly (Mitchell and Shoho, 2017). A school's openness to change has been positively associated with a teacher team's self-efficacy (Duygu et al., 2022) and computer use in elementary schools (Tondeur et al., 2008). Sustaining the implementing of an innovation requires a change in the daily practice and routines of the school. Being open to change can help schools to open their routines and embrace new ideas on how their practice can be strengthened. Especially when innovations are developed outside the own school context, this openness to new ideas could help to have an open mind towards new ways of working. Therefore, we hypothesize that openness to change positively relates to the sustained implementation of an innovation.

Absorptive capacity refers to the capacity of professionals in a school to value external knowledge and innovations and the capacity to translate this external knowledge into their own practice (Cohen and Levinthal, 1990; Da’as and Qadach, 2020). A higher degree of absorptive capacity enables an organization to develop new ideas to improve existing products, services, and processes (Leal-Rodríguez et al., 2014). Within the educational context, a high degree of absorptive capacity represents an ability to assimilate and transform externally acquired knowledge to improve the school's performance (Da'as and Qadach, 2020). Through the assimilation and transformation of knowledge, a higher level of absorptive capacity can also act as a buffer to integrate new knowledge and initiatives within existing practice, so that customization can be provided. This capacity is also seen as an enabler to integrate and adapt an innovation in the daily practice and routines of a school. Hence, we expect a positive relationship between a school's absorptive capacity and the ability to sustain an implementation (Da'as and Qadach, 2020).

Finally, we consider team experience as a relevant school factor, perceived as the average years of experience teachers have built up in the school. Current evidence about the relation between teacher work experience and attitude towards implementing innovative teaching practices is mixed. Some studies report a negative relationship (Ghaith and Yaghi, 1997; Yang and Huang, 2008), whereas others, especially with respect to the use of ICT in the classroom, did not find a relationship (Maican et al., 2019). However, these studies have not considered how long teachers have worked in the same school. We suggest that the amount of time that teachers have worked together as a team in the same school might affect the extent to which an innovation is sustained. One important reason in support of this contention is that staff turnover has been found to have a negative impact on sustained innovations (Saito et al., 2012) and can reduce staff commitment to implementation of innovations (Kirtman, 2002). Moreover, teachers who work together for a long time, can have a clearer perspective on their roles in the implementation process (Andreou et al., 2015) and have likely gained the expertise that is needed to adapt innovations to the local circumstances (Harn et al., 2013). We assume that an experienced team can build on their common teaching experiences and knowledge of the school, and, as a result, can more easily integrate the innovation in their daily practice.

The selection of school factors is based not only on their individual associations with sustained implementation, but also on their collective contribution to sustaining innovations. Openness to change makes schools receptive to externally developed innovations, while absorptive capacity ensures these innovations are adapted to the school's context. Social capital facilitates the integration of innovations into daily routines, and a transformational leader motivates staff to fully engage in these processes. Team experience further enhances the effective implementation and maintenance of innovations. This interplay is crucial, particularly when investigating interventions developed and managed outside the school, aligning with the concept of open innovation in education (Pietsch et al., 2024) and relates to the idea that in order to be able to absorb external knowledge, schools need to have a context in which leadership, social interactions and a link with the environment are present (Lenart-Gansiniec et al., 2022; Todorova and Durisin, 2007).

Present study

The research question of this study was about the relation between transformational leadership, social capital, openness to experience, and work experience, and the extent to which the implementation of an intervention for the prevention of reading difficulties was sustained by the school. We examined these relations both at the within and the between school level. At the within school level we enquired whether differences in work experience and the perception of transformational leadership, social capital, openness to change and absorptive capacity was associated with the perception of sustained implementation of the intervention. At the between level we examined whether mean differences in the perception of these school factors across schools as well as the overall experience of the team were related to mean differences between schools in the extent to which the implementation had been sustained.

The specific intervention was Build!, a computer-assisted early literacy intervention for children who are at risk of developing reading difficulties (Regtvoort et al., 2013; Zijlstra et al., 2020). The implementation of Build! in schools follows a structured process. A coordinator, often a remedial teacher or schoolteacher, is responsible for implementing Build! in the school. First, schools design or adopt a screening procedure to identify students at risk of reading difficulties. Eligible students receive Build! as a preventive intervention. The intervention starts in the second year of kindergarten and continues until mid-Grade 2. The intervention consists of lessons that cover pre-reading skills (letter knowledge and phonological awareness in kindergarten), beginning reading (decoding in first grade), and advanced reading (reading fluency from mid-first to mid-second grade) (Regtvoort et al., 2013; Zijlstra et al., 2014, 2020). The children practice with Build! outside their classroom with the help of a tutor. The tutor follows the on-screen instructions, reads them aloud, and offers feedback and emotional support when needed. Sessions occur three to four times per week, each lasting approximately 15 to 20 minutes. Progress is regularly monitored to ensure effectiveness, allowing for adjustments in frequency or intensity if needed. Through this structured and intensive approach, Build! aims to improve (early) literacy skills and thereby reduce reading difficulties.

The effectiveness of Build! has been supported in two randomized controlled trials (Regtvoort et al., 2013; Zijlstra et al., 2014). Build! is gradually implemented in schools from 2014 onwards and is currently used in over 80% of the primary schools in the Netherlands.

School professionals within schools were asked to rate a number of characteristics of the school as well as the extent to which the intervention was sustainably implemented on their school. School factors were the social capital of the school, the school's openness towards change, absorptive capacity and transformational leadership and team experience (see also Figure 1). The main teacher characteristic involved in the study was the number of years of experience in the school where they worked at the moment the data was collected.

Figure 1.

School factors related to the sustained implementation of Build!.

Method

Participants

Participants were 551 educational professionals working in 59 elementary schools in the Netherlands. All schools implemented the innovation under investigation. Participants had a mean age of 44.41 years (SD = 12.72) and a mean work experience of 9.42 years (SD = 9.45). In total, 39 participants were school principal and eight were adjunct principal. Another 39 participants were Build!- coordinator and 41 participants were the intermedial teacher in the school. Last, 362 participants were teachers, 46 participants had a teaching support role, and 16 participants indicated to have another role than specified.

Instruments

Openness to change

The scale Openness to Change measures the extent to which the school is open to change and willing to try out new things (Mitchell and Shoho, 2017). The original scale consists of 19 items with three subscales namely the degree to which the staff of a school perceives teachers being open to change with Cronbach's α of .95 (2) the principal being open to change, Cronbach's α being .87, and the extent to which a community is pressing for change with Cronbach's is α .87. In this study we used a Dutch translation of the scale enquiring about the first subscale referring to the openness of teachers. The choice for this subscale was driven by our specific interest in teachers’ openness to change. This scale consisted of 9 items. Each item consisted of a statement and participants were asked to indicate their agreement on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). An example of an item is “In this school, teachers are receptive to substantial changes.” In the present study, Cronbach's α of the scale was .85.

Absorptive capacity

The scale Absorptive Capacity measures the extent to which professionals in a school are able to translate innovations developed outside the school to their own practice. The original scale Absorptive Capacity developed by Flatten and colleagues (2011) consisted of 14 items referring to 4 subdimensions namely (1) Acquire knowledge with a Cronbach's α of .70, (2) Assimilate knowledge with a Cronbach's α of .80, (3) Transform knowledge with a Cronbach's α of .90 and Exploit knowledge with a Cronbach's α of .80. In this study, we created a Dutch translation and adapted version of this scale with all 4 subscales consisting of 10 items. Each item consisted of a statement. Participants were asked to indicate their agreement on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). An example of an item is “In our school it is easy to apply new knowledge in daily practice.” Cronbach's α of the scale was .84.

Social capital

Our scale was a Dutch translation of the cognitive dimension subscale of the Social Capital scale developed by Leana and Pil (2006). The cognitive dimensions refers to the extent to which there is a shared vision within the school with a Cronbach's α of .93, consisting of 6 items. In this study, each item consisted of a statement. Participants were asked to indicate their agreement on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). An example of an item is “Teachers share the same vision and ambition at this school.” The Cronbach's α of the scale in this study was .89.

Transformational leadership

We selected scales to reflect two aspects of transformational leadership. The first scale was intellectual stimulation (8 items) developed by Geijsel and colleagues (2003). The second scale, vision building (5 items), was developed by Moolenaar et al. (2010). Each item consisted of a statement. Participants were asked to indicate their agreement on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). An example of an item is “The school leader discusses the consequences of the school's vision for daily practice.” In line with the research of Moolenaar et al. (2010), who have used the same scales, we chose to combine them into a single overall scale. According to Moolenaar et al. (2010), this decision is consistent with previous research in which the components were found to be highly interrelated and together form a higher-order construct (Avolio et al., 1999; Bono and Judge, 2003; Jung and Sosik, 2002; Kark et al., 2003). In line with this research, we treated the two subscales as one scale. Cronbach's α of the scale in this study was .90.

Work experience in the school

Participants could indicate how many years they work in their current school.

Sustained implementation was measured by an adapted version of the Sustained Implementation scale by McIntosh et al. (2013). Based on a shortened version of this scale we distinguish two subscales, namely an Organizational Routine and an Adaptation and Monitoring subscale (see Theoretical Framework). The Organizational Routine scale was meant to measure the extent to which the intervention is considered as part of the daily routine of the school. It consisted of 9 items. Each item consisted of a statement. Participants were asked to indicate their agreement on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). An example of an item is “Build! is taken for granted at our school, it has become something “what you do at school.” Cronbach's α of the scale in this study was .91. The Adaptation and Monitoring scale aimed to measure the extent to which the intervention is regularly monitored and adapted. It was measured with 6 items. Each item consisted of a statement. Participants were asked to express their agreement on a 5-point Likert scale, ranging from 1 (completely disagree) to 5 (completely agree). An example of an item is “The use of Build! is evaluated at school at least once a year and adjusted if necessary.” Cronbach's α of the scale was .85.

The information about the scales is summarized in Table 1. For each scale, the score was the mean score across its items.

Table 1.

Overview of the scales used in the study.

Scale	No. items	Cronbach's α	Example item
Openness to Change	9	.85	“In this school, teachers are receptive to substantial changes.”
Absorptive Capacity	10	.84	“In our school, it is easy to apply new knowledge in daily practice.”
Social Capital	6	.89	“Teachers share the same vision and ambition at this school.”
Transformational Leadership	13	.90	“The school leader discusses the consequences of the school's vision for daily practice.”
Sustained Implementation—Organizational Routines	9	.91	“Build! is taken for granted at our school, it has become something ‘what you do at school’.”
Sustained Implementation—Adaptation and Monitoring	6	.85	“The use of Build! is evaluated at school at least once a year and adjusted if necessary.”

Procedure

The data was gathered in two periods through convenience sampling. The first period was in November of the schoolyear 2020 to 2021. One school district of 15 schools was contacted directly to participate in the study. In addition, a video (Deijle, 2021) in which the study was explained was distributed via various social media (WhatsApp, Facebook, Instagram, and LinkedIn). In this way, 61 schools were reached of which 25 schools (41%) agreed to participate. Of these schools, another five declined participations after receiving the online questionnaire, resulting in 20 schools and 212 participants. In second period, 45 school leaders from two school districts were approached by email by the first author of the paper with a request to participate in the survey in April of schoolyear 2020 to 2021. The two school districts were partner in a 3-year research project in which both the sustained implementation and the effectiveness of Build! was studied. Phase 2 resulted in 39 schools with a total of 339 respondents.

In the first period school leaders were asked to distribute an information letter among the teachers at their school containing the anonymous participation link to the online questionnaire and the consent form for participation. In the second period, the school leader asked permission to all colleagues to share their email address with the researchers. Then, an email with a direct link was send to 577 participants who had given consent to provide their email address.

In both phases 1 and 2, the online questionnaire was administered in Qualtrics. Before the start of the study, the respondents had to agree to participate in the study by means of an informed consent. Respondents were given 7 weeks to complete the questionnaire. Completion of the survey lasted approximately 15 minutes. Participants received the same online questionnaire, however, if participants indicated that they have a leadership role at the beginning of the questionnaire, items referring to transformational leadership were skipped automatically. After completion of the questionnaire, a digital book voucher was sent to the participants who filled in the questionnaire. After completion, the voucher was sent, and all email addresses were deleted. Before the start of the data collection, the research project was approved by the Ethical Board of the University of Amsterdam (project number 2019-CDE-11097).

Data analysis

To examine the unique contribution of various school-related factors (i.e. Social Capital, Transformational Leadership, Openness to Change, and Absorptive Capacity) in predicting the sustained implementation of the intervention Build!, we fitted a series of two-level regression analyses with a random intercept using Mplus version 8.4 (Muthén et al., 2017). This technique takes the clustering of educational professionals within schools into account by partitioning the variation in the sustained implementation of Build! into variance between and within schools (Snijders and Bosker, 1994). It also enables a split of the correlations among the variables into correlations within school and correlations across schools. This statistical method provides unbiased estimates of the standard errors associated with the regression coefficients. All fixed and random effects parameters in these models were based on maximum likelihood estimation with robust standard errors and a mean-adjusted chi-square test statistic (MLR). Predictors were centered around the grand mean to ease their interpretation. Work experience in the school was added as a covariate in all analyses.

Results

Preliminary analyses

Initially, we screened our data for accuracy of input, missing data, and outliers. The percentage of missing data across variables was very low, ranging from 0% to 8.7%, which was mainly due to principals or adjunct principals skipping the questionnaires about transformational leadership. All missing data patterns were treated using full information maximum likelihood estimation (FIML), after finding that they were missing completely at random, χ² (934) = 862.34, p = .954. Using a conservative cut-off value of ±3 SD, 11 outliers were detected (one for Social Capital, one for Openness, three for Transformational Leadership, and six for the Organizational Routines dimension of Sustained Implementation). These outliers were all substantially lower than average and were therefore excluded from further analysis. In this final dataset of N = 540 educational professionals, the distributions of all variables of interest were approximately normal, with low skewness and kurtosis values.

Prior to fitting the two-level regression models, we first calculated the intraclass correlations (ICC1) and the reliability of the aggregated variables at the school level (ICC2; Marsh et al., 2009; Raudenbush and Bryk, 2002). The ICC1 reflects the proportion of the total variance in the sustained implementation of Build! that is accounted for by the clustering of educational professionals within schools. This coefficient thus reflects the agreement between educational professionals within the same school. In this study, all ICC1 values were substantial, ranging from 0.13 to 0.38 (see Table 1). Next to ICC1 values, we evaluated the reliability of the aggregated measures at the school level. On average, about 9 educational professionals (range = 1–27) per school completed questionnaires about the sustained implementation of Build!. Based on the between- and within-level variance of the manifest variables, ICC2 values ranged between .57 and .85. According to the guidelines of Cicchetti (1994), ICC2 values can be considered good when they range between .60 and .74, and excellent when they range between .75 and 1.00.

Descriptive statistics and bivariate correlations at the within- and between-school level are summarized in Table 1. Correlations at the within-school level (below the diagonal) mainly reflected our expectations. All school-related factors were positively and moderately associated with the perceived sustained implementation of Build!. At the between-school level (above the diagonal), somewhat different patterns were found. Whereas Social Capital and Leadership were positively correlated with the Sustained Implementation factors, correlation coefficients for Openness and Absorptive Capacity were weaker and mainly non-significant. Hence, these descriptive findings underscore the relevance of investigating the relation of the school-related factors with the sustained implementation of Build! both at the within- and between-school level.

Two-level regression model of the sustained implementation of Build!

To investigate the unique associations of various school-related factors with the sustained implementation of Build!, we fitted a multivariate random-intercept model to the data for each dimension of sustained implementation (see Table 2). After controlling for the effects of Work Experience, our model only revealed statistically significant positive within-school (Level 1) associations of Social Capital and Absorptive Capacity with both the Organizational Routines and Adaptation and Monitoring dimension of Sustained Implementation. These uniquely significant parameters suggest that educational professionals who perceive that their school has a shared vision and perceive that their school has the capacity to translate external knowledge in their daily practice are more likely to implement Build! in a sustained way (Table 3).

Table 2.

Univariate higher-order moment descriptive statistics, bivariate correlations, reliabilities, and intraclass correlations.

	1.	2.	3.	4.	5.	6.	7.	ICC2
1. Experience	1.00	.05	.15	−.01	.08	.72**	.54**	-
2. Social Capital	.08	1.00	.62***	.60***	.64***	.35*	.37*	.69
3. Transformational Leadership	.04	.50***	1.00	.68***	.76***	.46**	.52***	.57
4. Openness to Change	.11**	.46***	.36***	1.00	.67***	−.03	.00	.72
5. Absorptive Capacity	.13**	.49***	.56***	.41***	1.00	.24	.39*	.59
6. Sustained Implementation (Organizational Routine)	.08	.27***	.20***	.25***	.29***	1.00	.85***	.82
7. Sustained Implementation (Adaptation and Monitoring)	.09	.32***	.30***	.27***	.39***	.50***	1.00	.85
Mean^a	9.42	4.15	3.54	3.81	3.75	4.04	3.49	-
Variance	90.25	0.32	0.25	0.31	0.30	0.39	0.56	-
ICC1	.04	.20	.13	.22	.14	.38	.37	-

Note: Level 1 (pooled teacher level) correlations are presented below the diagonal; Level 2 (school level) correlations are presented above the diagonal.

^a Range = 1–5. ***p < .001; **p < .01; *p < .05.

Table 3.

Two-level regression model for predicting the two dimensions of sustained implementation of Build! at level 1 (within school) and level 2 (between schools).

	Sustained implementation (organizational routines)		Sustained implementation (adaptation and monitoring)
Fixed effects	B (SE)	β	B (SE)	β
Intercept	4.05 (0.05)***		3.46 (0.06)***
Level 1:
Experience	0.00 (0.00)	0.04	0.00 (0.00)	0.03
Social Capital	0.14 (0.06)*	0.14	0.15 (0.06)*	0.13
Transformational Leadership	−0.01 (0.07)	−0.01	0.04 (0.07)	0.03
Openness to Change	0.11 (0.07)	0.11	0.11 (0.06)	0.09
Absorptive Capacity	0.17 (0.05)**	0.18	0.33 (0.08)***	0.29
Level 2:
Experience	0.12 (0.05)*	0.56	0.10 (0.04)*	0.40
Social Capital	0.51 (0.35)	0.31	0.49 (0.41)	0.26
Transformational Leadership	1.22 (0.41)**	0.53	1.31 (0.47)**	0.49
Openness to Change	−0.70 (0.30)*	−0.44	−0.93 (0.36)**	−0.50
Absorptive Capacity	−0.13 (0.47)	−0.06	0.45 (0.60)	0.18
Random Effects	Var	R ²	Var	R ²
Level 1	0.22	0.07	0.29	0.11
Level 2	0.02	0.90	0.06	0.74

Note: ***p < .001; **p < .01; *p < .05.

In contrast to the results at Level 1, we found a rather mixed pattern of associations between schools. After accounting for the effects of Experience, the associations of Social Capital and Absorptive Capacity did not differ significantly from zero. Instead, Transformational Leadership appeared to be positively associated with both implementation factors and, rather unexpectedly, Openness to Change showed a negative association. It seems as if Openness to Change functioned as a classical suppressor variable for the associations of other school-related factors, and Leadership in particular, with both Sustained Implementation variables. More specifically, Openness correlated substantially with other school-related factors (range = .60–.68, see Table 1), whereas the correlations with both outcome variables were close to zero. In the random-intercept model, however, the association between Openness and both Sustained Implementation variables was opposite in sign, and the coefficient of Leadership became somewhat stronger. When entered separately, the coefficients for Leadership remained relatively stable (β_routines = .50, p < .01; β_monitoring = .44, p < .05), and those of Openness decreased substantially (β_routines = −.38, p < .05; β_monitoring = −.43, p < .01). Notably, in the latter model without Leadership, the positive association of Social Capital with the Organizational Routines dimension of Sustained Implementation even reached the significance threshold (β_routines = .42, p < .05). This may indicate that Openness to Change had more in common with the school-related factors than with the sustained implementation of Build! at the school level (cf. Maassen and Bakker, 2001). Thus, by controlling for irrelevant variance that is shared with the school-related factors, but not with the sustained implementation of Build!, Openness may improve the associations of the other predictors with Sustained Implementation. The results are summarized in Figure 2.

Figure 2.

Standardized parameters of the multivariate random intercept model predicting the sustained implementation of Build!.

Discussion

Achieving sustained implementation of innovations, in which innovations have become a routine and integral part of daily practice and are adapted and monitored on a regular basis, is seen as a critical challenge for schools. This research aimed to get more insight into the relation between the school factors and the extent to which the implementation of an intervention has been sustained in schools. The measurement of the school factors as well as the extent to which the intervention, Build!, was sustained were based on the perception of professionals in schools, mostly teachers. Therefore, we made a distinction among relations of the various school factors with sustained implementation within and between (or across) schools.

The distinction between these two levels appeared to be important. Our results show that school factors accounting for differences across schools in sustaining innovations differ from the factors that explain differences among the perceptions of teachers within schools. At the within school level, we found that teachers who indicated that their school had more social capital and absorptive capacity also perceived the implementation of Build! as more sustained, both for the Organizational Routines and the Adaptation and Monitoring dimension. These results at the within school level are in line with (mostly qualitative) studies showing the important role of social capital within schools (see Prenger et al., 2022 for an overview) and is also in accordance with the central position of social capital and absorptive capacity in theoretical frameworks about implementation in organizations (Durlak and Dupre, 2008; CFIR; Damschroder et al., 2009; Kirk et al., 2015). However, our findings show that absorptive capacity and social capital are only associated with differences in teachers’ perception of sustained implementation of the intervention within schools and are unrelated with differences in sustained implementation between schools. This could indicate that, when it comes to differences between schools, the importance of these school factors might have been overestimated in previous research that focused solely on perceptions of educational professionals within schools, neglecting differences at the between school level.

At the between school level, our results show that schools with a transformational school leader and an experienced team, are more likely to implement an innovation such as Build! in a sustained way. The important role of transformational leadership is in line with previous research suggesting that transformational leadership is an important driver of innovation (Geijsel et al., 1999). A transformational leader stimulates employees to be creative, learn from mistakes and experiment with solutions without fear of failure and thereby increases the creative performance of employees (Wang et al., 2011). Transformational leadership is also seen as a catalysator of innovations by a team as transformational leaders stimulate teachers intellectually (Garcia-Morales et al., 2012; Moolenaar et al., 2010) and stimulate technology integration in the classroom (Schmitz et al., 2023). Thus, in addition to the known positive outcomes of transformational leadership and innovation, our results show that a school leader who stimulates and generates a shared vision within the school team, also helps a school team to sustain an innovation like Build!.

Our findings on the importance of transformational school leadership are also in line with more recent research on the specific role of technological leadership. Dexter and Richardson (2020), for example, found that for the integration of new technologies it is important that leaders, amongst other things, stimulate capacity building and consider professionals individualized needs. These capacities are in line with the characteristics of a transformational leader as investigated in our study. As our study shows that the role of the educational leader is substantially related to differences across schools in the sustained implementation of an intervention, future research could focus on other leadership characteristics as pointed out as important in the technology leadership literature (Dexter and Richardson, 2020; Flanagan and Jacobsen, 2003; McLeod and Richardson, 2011). Such future research should make a clear distinction between characteristics perceived as important by individuals within a school and technology leadership characteristics that account for differences across schools.

A striking finding of our study is that a team's experience of working in the same school substantially contributed to the sustained implementation of an innovation. Next to a transformational leader, a school with an experienced team is more likely to sustain a technology enhanced learning intervention. Possibly, the experience of a team is of particular importance in the phase that innovations have to be sustained. When innovations are sustained, they become part of the daily routine of the school. Possibly, an experienced team has more knowledge of established routines and might therefore be better able to adapt these routines, making it easier to integrate a new innovation like Build! in the school. An alternative explanation could be that team experience is related to team continuity, which is found to be positively related to the implementation of data-based decision-making interventions (van Geel et al., 2017). The importance of team experience may come as a surprise as teams with high levels of tenure are often perceived to rely on existing practices and routines, which can interfere with a teams’ openness to and engagement with school innovations (Muskat et al., 2022). Moreover, a recent review on team familiarity, a concept related to team experience, found that there is little to no evidence about the role of team familiarity in innovation and creativity processes (Muskat et al., 2022). The current finding is therefore unique and suggests that team experience might be important when it comes to sustaining innovations.

Limitations and future directions

Our research also has some limitations. One limitation is the type of innovation we have investigated. The prevention program Build! targets young students at risk for reading difficulties and is only used by a selected group of students which are supported by a tutor outside the classroom. To organize the implementation of Build! only one or two teachers are responsible. This differs from innovations that need to be implemented by all teachers. It is unclear to what extent these features of this intervention limit the generalizability of our findings to broader school innovations. For example, we cannot exclude the possibility that the absorptive capacity of the school and the social capital of the school could also explain differences between schools when an innovation concerns all children and teachers. However, as we do not see a reason why our main finding, the importance of transformational leadership and team experience for sustained implementation, would not be of importance for the sustainable implementation of other innovations.

As a further limitation we should mention that there are some school level factors that we did not include because of the unique way the implementation process of Build! is supported in most schools. For example, schools are explicitly advised to appoint a person to coordinate the implementation at every school. This is in line with an abundant amount of research that supports the idea that a person within the school should be responsible for the implementation of an innovation (Durlak and Dupre, 2008) and to play the important role of knowledge broker to transfer the external knowledge concerning Build! to the local context (Caduff et al., 2023). Because most schools followed this advice, we assumed there would be not enough variation in our sample to investigate this as a school level factor. But this feature could also be a critical success factor given the relatively high degree to which Build! is sustained within schools in the Netherlands. Research comparing the sustained implementation of an innovation with and without a person that is responsible for implementation and incorporating the external knowledge concerning the implementation of Build! in schools could give more insights of the unique contribution of the role of knowledge brokers (Caduff et al., 2023). Another potentially important factor not included in this study, is the amount of external support available to the school. For the current intervention, the educational software company that provides Build!, offers many opportunities for training, consultancy and support in the implementation process and most Build! coordinators followed these trainings. We believe that external support and a lot of training related to an innovation are probably also important to successfully sustain an implementation (e.g. Caduff et al., 2023; Durlak and Dupre, 2008). Another factor that was absent in our study, concerns the role of district leaders (Richardson and Sterrett, 2018). Although the implementation of Build! is also supported on the district level in our sample, due to the limited number of districts in our study, we were not able to investigate the possible impact of district leadership in the sustained implementation of Build!.

Another limitation of this study is the use of a doubly-manifest statistical model, which does not account for potential sampling and measurement errors. Although this approach simplifies the relationships between observed variables, it may lead to bias by not accounting for these errors. Future research could consider alternative modeling techniques, such as latent variable approaches, which can better control for both sampling and measurement inaccuracies, thereby providing a more precise understanding of the underlying constructs. Another limitation of our study is the unbalanced sample sizes, which could affect the estimation and interpretation of ICC values. As LeBreton and Senter (2008) highlight, best practices suggest computing multiple estimates to account for such imbalances, which was not done in this study. Additionally, missing data may have introduced self-selection bias (Grund et al., 2019; Newman and Sin, 2009), further complicating the reliability of group-level estimates (Lüdtke et al., 2011).

As a final limitation we acknowledge that the cross-sectional design of this study does not provide support for causal relationships. Given the current findings it would be interesting to examine the interplay between school factors and sustained implementation in a longitudinal study.

Future research directions

The current finding that schools with an experienced team and transformational leader are more likely to sustain innovations, raises further questions about the specific role of these factors in sustaining innovations. We can imagine that both the experienced team and the leader take crucial decisions in crucial phases of the implementation process, for example as to whether to sustain or not to sustain an innovation. The rationale behind these decisions could help to understand the role of school leaders and the team within these crucial phases of innovation. More qualitative research might help to understand how teams with experience in working together more easily include innovations in their daily routines and how transformational leaders affect this process. More research is needed to understand the relationship between the school leader and the experienced team of the school in the innovation process towards sustainability. The question also arises to what extent the experienced team acts as teacher leaders in the implementation process, as research indicates that transformational leadership stimulates teacher leadership in schools (Li and Liu, 2022). Further research is needed to understand how different forms of leadership like transformational leadership, teacher leadership and distributed leadership within a school interact to sustain innovations. Moreover, in this study, transformational leadership, and the other school factors were treated as single constructs. For future research, it may be valuable to investigate how the different subdimensions of these constructs relate to the sustained implementation of educational innovations.

Future longitudinal research could also focus on the different stages of an innovation process. Possibly different factors play a role in different phases of the innovation process. For example, in our results the culture of openness of a school towards innovation did not play a unique role in sustaining innovations. However, a culture of openness might be of crucial importance in the initiation or experimental phase of an innovation. For future studies it seems important to compare factors that influence the initial adoption and the sustainment of an innovation.

Practical implications

Our findings suggest three practical implications. First, school leaders need to be aware of their central and supportive role to sustain technology enhanced learning interventions. Transformational leadership in our study was characterized as school leaders who stimulate their staff intellectually. Intellectual stimulation involves providing opportunities for learning and professional development, and also challenging colleagues to experiment with new didactical approaches, engage in inspirational conversations about teachers’ professional goals and ambitions. Our study suggests that making time to invest in these types of informal conversations with a school team helps to sustain innovations.

Secondly, a transformational school leader was also operationalized as a leader who build up a shared vision. A school leader needs to think about how a technology enhanced learning intervention fits within the shared vision of the school. A transformational leader builds a shared vision by explicitly referring to the objectives of the school when important decisions are made. So, when an important decision is made to implement a new innovation, it helps when the school leader explains how this decision aligns with the vision and objectives of the school. Moreover, a transformational leader also discusses the consequences of the school's vision for daily practice and for the current and future challenges the school faces.

Third, an experienced team can help to sustain innovations. Sometimes managers or school leaders may wonder if implementing innovations in a school also requires new people, because change is often perceived as difficult by school teams. However, our study shows that it pays off when a team has build-up experiences together, as it seems that shared experiences, together with a transformational leader enhance sustained implementation of an innovation that aims to improve learning outcomes. A long-lasting positive impact on student outcomes (and especially helping children with reading difficulties) is a result that many schools, but also policy advisors and our society envision to strengthen the future of our children.

Footnotes

ORCID iDs

Bieke G M Schreurs

Peter F de Jong

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Nationaal Regieorgaan Onderwijsonderzoek, (grant number 40.5.18540.065).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Author biographies

Bieke G M Schreurs is assistant professor in the Department of Child Development and Education at the University of Amsterdam.

Marjolein Zee is associate professor of educational sciences at the Department of Psychology, Education, and Child Studies (DPECS), Erasmus University Rotterdam, Rotterdam, The Netherlands.

Frank Cornelissen is professor of innovation in education and leads the research group in Educational Sciences in the Department of Child Development and Education at the University of Amsterdam.

Peter F de Jong is emeritus professor of psychology and education in the Department of Child Development and Education at the University of Amsterdam. He is the former head of the research group Developmental Disorders and Special Education and of the Graduate School of Child Development and Education.

References

Aarons

Hurlburt

Horwitz

(2011) Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research 38: 4–23.

Andreou

McIntosh

Ross

, et al. (2015) Critical incidents in sustaining school-wide positive behavioral interventions and supports. The Journal of Special Education 49(3): 157–167.

Auckland

Kilpatrick

(2018) Leadership and social capital: engaging small Australian communities in development and implementation of learning plans. International Journal of Lifelong Education 37(4): 495–512.

Avolio

Bass

Jung

(1999) Re‐examining the components of transformational and transactional leadership using the Multifactor Leadership. Journal of Occupational and Organizational Psychology 72(4): 441–462.

Bean

Dole

Nelson

, et al. (2015) The sustainability of a national reading reform initiative in two states. Reading and Writing Quarterly 31(1): 30–55.

Bono

Judge

(2003) Self-concordance at work: toward understanding the motivational effects of transformational leaders. Academy of Management Journal 46(5): 554–571.

Caduff

Lockton

Daly

, et al. (2023) Beyond sharing knowledge: knowledge brokers’ strategies to build capacity in education systems. Journal of Professional Capital and Community 8(2): 109–124.

Chen

Zheng

Yang

, et al. (2016) Transformational leadership social capital and organizational innovation. Leadership and Organization Development Journal 37(7): 843–859.

Cicchetti

(1994) Guidelines criteria and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment 6(4): 284–290.

10.

Coburn

(2001) Collective sensemaking about reading: how teachers mediate reading policy in their professional communities. Educational Evaluation and Policy Analysis 23(2): 145–170.

11.

Cohen

Levinthal

(1990) Absorptive capacity: a new perspective on learning and innovation. Administrative Science Quarterly 35(1): 128–152.

12.

Da’as

Qadach

(2020) Examining organizational absorptive capacity construct: a validation study in the school context. Leadership and Policy in Schools 19(3): 327–345.

13.

Damschroder

Aron

Keith

, et al. (2009) Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science 4(1): 1–15.

14.

Damschroder

Reardon

Widerquist

, et al. (2022) The updated consolidated framework for implementation research based on user feedback. Implementation Science 17(1): 1–16.

15.

Deijle

(2021) Innoveren op basisscholen: de relatie tussen de verschillende dimensies van transformationeel leiderschap en de Absorptive Capacity van scholen . Master Thesis, Universiteit van Amsterdam. Scriptiedatabank. https://scriptiesubauvanl/search?id=c4556370.

16.

Dexter

Richardson

(2020) What does technology integration research tell us about the leadership of technology? Journal of Research on Technology in Education 52(1): 17–36.

17.

Durlak

Dupre

(2008) Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology 41(3): 327–350.

18.

Duygu

Yusuf

Hacı

(2022) Examining the relationship between the level of self-efficacy of teachers and the level of schools' openness to change. i-manager’s Journal on Educational Psychology 15(3): 11.

19.

Flanagan

Jacobsen

(2003) Technology leadership for the twenty-first century principal. Journal of Educational Administration 41(2): 124–142.

20.

Flatten

Engelen

Zahra

, et al. (2011) A measure of absorptive capacity: scale development and validation. European Management Journal 29(2): 98–116.

21.

Fox

Leif

Moore

, et al. (2022) A systematic review of the facilitators and barriers to the sustained implementation of school-wide positive behavioral interventions and supports. Education and Treatment of Children 45: 105–126.

22.

García-Morales

Jiménez-Barrionuevo

Gutiérrez-Gutiérrez

(2012) Transformational leadership influence on organizational performance through organizational learning and innovation. Journal of Business Research 65(7): 1040–1050.

23.

Geijsel

Sleegers

Leithwood

, et al. (2003) Transformational leadership effects on teachers’ commitment and effort toward school reform. Journal of Educational Administration 41(3): 228–256.

24.

Geijsel

Sleegers

van den Berg

(1999) Transformational leadership and the implementation of large-scale innovation programs. Journal of Educational Administration 37(4): 309–328.

25.

Ghaith

Yaghi

(1997) Relationships among experience teacher efficacy and attitudes toward the implementation of instructional innovation. Teaching and Teacher Education 13(4): 451–458.

26.

Grund

Lüdtke

Robitzsch

(2019) Missing data in multilevel research. In: Humphrey

LeBreton

(eds) The Handbook of Multilevel Theory, Measurement, and Analysis, pp.365–386.

27.

Hargreaves

Goodson

(2006) Educational change over time? The sustainability and nonsustainability of three decades of secondary school change and continuity. Educational Administration Quarterly 42(1): 3–41.

28.

Harn

Parisi

Stoolmiller

(2013) Balancing fidelity with flexibility and fit: what do we really know about fidelity of implementation in schools? Exceptional Children 79(2): 181–193.

29.

Jung

Sosik

(2002) Transformational leadership in work groups. Small Group Research 33(3): 313–336.

30.

Kark

Shamir

Chen

(2003) The two faces of transformational leadership: empowerment and dependency. Journal of Applied Psychology 88(2): 246–255.

31.

Khan

(2023) Transformational leadership and teacher work performance: mediating effect of job autonomy and trust in school principal—insights from senior secondary school data in India. Educational Management Administration and Leadership: 1–12. DOI: 101177/17411432231172359.

32.

Kirk

Kelley

Yankey

, et al. (2015) A systematic review of the use of the consolidated framework for implementation research. Implementation Science 11(1): 1–13.

33.

Kirtman

(2002) Policy and practice: restructuring teachers’ work. Education Policy Analyses Archives 10(25): 1–26.

34.

Kılınç

AÇ

Polatcan

Savaş

, et al. (2022) How transformational leadership influences teachers’ commitment and innovative practices: understanding the moderating role of trust in principal. Educational Management Administration and Leadership 52(2): 455–474.

35.

Krijgsman

Snoek

Abels

, et al. (2022) Duurzame innovaties in het onderwĳs: een systematische literatuurreview: de invloed van (gespreid) leiderschap en de samenhang met contextuele relationele en individuele condities. Eindhoven School of Education. Available at: https://wwwhvanl/binaries/content/assets/subsites/kc-oo/pdfs-bijlagen-bij-nieuwsberichten/krijgsman-et-al-2022_nro_main_documentpdf?1660124536137.

36.

Leal-Rodríguez

Roldán

Ariza-Montes

, et al. (2014) From potential absorptive capacity to innovation outcomes in project teams: the conditional mediating role of the realized absorptive capacity in a relational learning context. International Journal of Project Management 32(6): 894–907.

37.

Leana

Pil

(2006) Social capital and organizational performance: evidence from urban public schools. Organization Science 17(3): 353–366.

38.

LeBreton

Senter

(2008) Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods 11(4): 815–852.

39.

Lenart-Gansiniec

Czakon

Pellegrini

(2022) In search of virtuous learning circles: absorptive capacity and its antecedents in the education sector. Journal of Knowledge Management 26(11): 42–70.

40.

Liu

(2022) An integrated model of principal transformational leadership and teacher leadership that is related to teacher self-efficacy and student academic performance. Asia Pacific Journal of Education 42(4): 661–678.

41.

Lloyd

Schachner

(2021) School effects revisited: the size, stability, and persistence of middle schools’ effects on academic outcomes. American Educational Research Journal 58(4): 748–784.

42.

Lüdtke

Marsh

Robitzsch

Trautwein

(2011) A 2 × 2 taxonomy of multilevel latent contextual models: accuracy–bias trade-offs in full and partial error correction models.. Psychological Methods 16(4): 444–467.

43.

Maassen

Bakker

(2001) Suppressor variables in path models: definitions and interpretations. Sociological Methods and Research 30(2): 241–270.

44.

Maican

Cazan

Lixandroiu

, et al. (2019) A study on academic staff personality and technology acceptance: the case of communication and collaboration applications. Computers and Education 128: 113–131.

45.

Marsh

Lüdtke

Robitzsch

, et al. (2009) Doubly-latent models of school contextual effects: integrating multilevel and structural equation approaches to control measurement and sampling error. Multivariate Behavioral Research 44(6): 764–802.

46.

Mathews

McIntosh

Frank

, et al. (2014) Critical features predicting sustained implementation of school-wide positive behavioral interventions and supports. Journal of Positive Behavior Interventions 16(3): 168–178.

47.

McIntosh

Kim

Mercer

, et al. (2015) Variables associated with enhanced sustainability of school-wide positive behavioral interventions and supports. Assessment for Effective Intervention 40(3): 184–191.

48.

McIntosh

Mercer

Hume

, et al. (2013) Factors related to sustained implementation of schoolwide positive behavior support. Exceptional Children 79: 293–311.

49.

McIntosh

Mercer

Nese

RNT

, et al. (2018) Factors predicting sustained implementation of a universal behavior support framework. Educational Researcher 47(5): 307–316.

50.

McLeod

Richardson

(2011) The dearth of technology leadership coverage. Journal of School Leadership 21(2): 216–240.

51.

Mitchell

Shoho

(2017) High school teachers’ perceptions of school change and its implications for student achievement. Leadership and Policy in Schools 16(3): 431–454.

52.

Moolenaar

Sleegers

(2010) Social networks trust and innovation: the role of relationships in supporting an innovative climate in Dutch schools. In: Daly

(ed) Social Network Theory. Cambridge, MA: Harvard Education, pp.97–114.

53.

Moolenaar

(2010) Ties with potential: nature antecedents and consequences of social networks in school teams . Unpublished Doctoral Dissertation, University of Amsterdam. Handle https://hdlhandlenet/11245/1374804

54.

Moolenaar

Daly

Sleegers

PJC

(2010) Occupying the principal position: examining relationships between transformational leadership, social network position, and schools’ innovative climate. Educational Administration Quarterly 46(5): 623–670.

55.

Morin

Marsh

Nagengast

, et al. (2014) Doubly latent multilevel analyses of classroom climate: an illustration. The Journal of Experimental Education 82(2): 143–167.

56.

Muskat

Anand

Contessotto

, et al. (2022) Team familiarity—boon for routines bane for innovation? A review and future research agenda. Human Resource Management Review 32(4): 100892.

57.

Muthén

Asparouhov

(2017) Regression and Mediation Analysis Using Mplus. Los Angeles: Mplus.

58.

Nahapiet

Ghoshal

(1998) Social capital intellectual capital and the organizational advantage. Academy of Management Review 23(2): 242–266.

59.

Newman

Sin

H-P

(2009) How do missing data bias estimates of within-group agreement? Sensitivity of SD _WG, CV _WG, r _WG(J), r _WG(J) *, and ICC to systematic nonresponse. Organizational Research Methods 12(1): 113–147.

60.

Ninković

Knežević Florić

Momčilović

(2023) Multilevel analysis of the effects of principal support and innovative school climate on the integration of technology in learning activities. Computers & Education 202: 104833.

61.

Pas

Waasdorp

Bradshaw

(2015) Examining contextual influences on classroom-based implementation of positive behavior support strategies: findings from a randomized controlled effectiveness trial. Prevention Science 16(8): 1096–1106.

62.

Pietsch

Cramer

Brown

, et al. (2024) Open innovation in schools: a new imperative for organising innovation in education? Technology, Knowledge and Learning 29(2): 1051–1077.

63.

Prenger

Tappel

APM

Poortman

, et al. (2022) How can educational innovations become sustainable? A review of the empirical literature. Frontiers in Education 7: 970715.

64.

Raudenbush

Bryk

(2002) Hierarchical Linear Models: Applications and Data Analysis Methods, Vol. 1. Thousand Oaks, CA: Sage.

65.

Regtvoort

Zijlstra

van der Leij

(2013) The effectiveness of a 2-year supplementary tutor-assisted computerized intervention on the reading development of beginning readers at risk for reading difficulties: a randomized controlled trial. Dyslexia 19(4): 256–280.

66.

Richardson

Sterrett

(2018) District technology leadership then and now: a comparative study of district technology leadership from 2001 to 2014. Educational Administration Quarterly 54(4): 589–616.

67.

Saito

Khong

TDH

Tsukui

(2012) Why is school reform sustained even after a project? A case study of Bac Giang Province Vietnam. Journal of Educational Change 13: 259–287.

68.

Schaper

McIntosh

Hoselton

(2016) Within-year fidelity growth of SWPBIS during installation and initial implementation. School Psychology Quarterly 31(3): 358–368.

69.

Schmitz

Antonietti

Consoli

, et al. (2023) Transformational leadership for technology integration in schools: empowering teachers to use technology in a more demanding way. Computers and Education 204: 104880.

70.

Snijders

TAB

Bosker

(1994) Modeled variance in two-level models. Sociological Methods and Research 22: 342–363.

71.

Spillane

Parise

Sherer

(2011) Organizational routines as coupling mechanisms: policy school administration and the technical core. American Educational Research Journal 48(3): 586–619.

72.

Todorova

Durisin

(2007) Absorptive capacity: valuing a reconceptualization. Academy of Management Review 32(3): 774–786.

73.

Tondeur

Hermans

van Braak

, et al. (2008) Exploring the link between teachers’ educational belief profiles and different types of computer use in the classroom. Computers in Human Behavior 24(6): 2541–2553.

74.

van Geel

Visscher

Teunis

(2017) School characteristics influencing the implementation of a data-based decision making intervention. School Effectiveness and School Improvement 28(3): 443–462.

75.

Vermeulen

Van Acker

Kreijns

, et al. (2015) Does transformational leadership encourage teachers’ use of digital learning materials. Educational Management Administration and Leadership 43(6): 1006–1025.

76.

Wang

Courtright

, et al. (2011) Transformational leadership and performance across criteria and levels: a meta-analytic review of 25 years of research. Group and Organization Management 36(2): 223–270.

77.

Wong

(2008) Framing ICT implementation in a context of educational change: a multilevel analysis. School Effectiveness and School Improvement 19(1): 99–120.

78.

Yang

Huang

(2008) A study of high school English teachers’ behavior concerns and beliefs in integrating information technology into English instruction. Computers in Human Behavior 24: 1085–1103.

79.

Zijlstra

Koomen

HMY

Regtvoort

AGFM

van der Leij

DAV

(2014) Effects of quantitative and qualitative treatment fidelity of an individualized computer-supported early reading intervention delivered by non-professional tutors. Learning and Individual Differences 33: 55–62.

80.

Zijlstra

van Bergen

Regtvoort

, et al. (2020) Prevention of reading difficulties in children with and without familial risk: short- and long-term effects of an early intervention. Journal of Educational Psychology 113(2): 248–267.