Sage Journals: Discover world-class research

Abstract

Faced with accountability pressures and limited resources, education policymakers make difficult choices regarding school curricula. In recent years, studies have documented a decreased emphasis in arts and humanities instruction. One potential way for schools to fill this gap includes partnering with arts and cultural organizations to provide arts learning opportunities through arts integration, yet there is limited evidence on the effectiveness of such programs. We examine one such partnership by evaluating the efficacy of a program that infuses history content with theater. By randomly assigning school groups to participate in this program, we are able to draw causal conclusions about its effects. We find that students demonstrate increases in historical content knowledge, enthusiasm for learning about history, historical empathy, and interest in the performing arts as a result of this program. These findings suggest that there are valuable educational benefits from arts-integrated learning opportunities provided through school partnerships with arts organizations.

Keywords

arts education arts integration educational policy evaluation experimental design experimental research history learning processes/strategies motivation policy analysis

Introduction

The growth of test-based accountability systems in public schools in the United States has increased education administrators’ attention to standardized achievement, particularly “high-stakes” tested subjects, such as math and reading (Bassok et al., 2016). Beginning with the passage of the No Child Left Behind Act of 2001, schools have experienced significant declines in arts and humanities instructional resources (Gadsden, 2008; Yee, 2014). This emphasis has come at the expense of time and resources devoted to the arts and humanities (West, 2007).

One strategy for combatting these declines has been school partnerships with arts and cultural organizations. These partnerships typically consist of schools collaborating with art museums, performing arts centers, teaching artists, and arts integration specialists to supplement educational experiences and opportunities that schools struggle to provide on their own (Bowen & Kisida, 2017). Often the programming provided by arts organizations integrates content knowledge from formal curricular standards.

We evaluate one such program—the Digging Up Arkansas professionally staged performance produced by the Walton Arts Center (WAC) and Trike Theater. This show integrates Arkansas state history content standards into an interactive theater performance for elementary-aged students. Using an experimental methodology, we evaluate this program and identify the causal effects on participants. We find that compared with control group students, students in classrooms randomly assigned to participate in the Digging Up Arkansas program demonstrate increased historical content knowledge, historical empathy, interest in live theater, and interest in learning history. Our results indicate that schools searching for innovative ways to offer arts experiences while reinforcing learning in the humanities can benefit from partnering with arts and cultural organizations that offer arts-integrated programs.

Background and Previous Literature

Pressure on educational leaders to demonstrate performance through high-stakes testing corresponds with a decline in resources devoted to classroom instruction in the arts and humanities over the past decade (Bassok et al., 2016; Gadsden, 2008; West, 2007). Related evidence shows that these declines are primarily driven by decreases in arts access for historically underserved students (Chappell & Cahnmann-Taylor, 2013; Rabkin & Hedberg, 2011; Yee, 2014). Reductions in resources devoted to the arts corresponds to a similar decline in school visits to cultural organizations (Associated Press, 2012; Blair, 2008; Ellerson, 2010; Lewin, 2010; Mehta, 2007; Plummer, 2014). As a result, cultural organizations with a mission to provide educational opportunities to school-aged youth are increasingly under pressure to demonstrate their educational benefits.

Educational outreach is a key component of most cultural organizations’ missions. According to the United Nations Educational, Scientific and Cultural Organization’s (2001) Universal Declaration on Cultural Diversity, cultural organizations are those that are engaged in the study, preservation, and public education of a “set of distinctive spiritual, material, intellectual, and emotional features of society or a social group.” Cultural organizations receive more than 90 million student visits each year from K–12 school groups and spend more than $2 billion a year on educational activities (CNN, 2017). However, little is known about the effectiveness of educational programs developed by cultural organizations on educational outcomes. Research tends to be anecdotal, observational, or correlational. One exception is an experimental evaluation of a school partnership that randomly assigned students to participate in a major art museum’s educational program. In that study, researchers found that participating students experienced gains in historical empathy, tolerance, and art knowledge (Greene et al., 2014), critical thinking (Kisida et al., 2016), and an increased desire to be “cultural consumers” (Kisida et al., 2014). Despite this contribution, there is still a glaring need to better understand the impact of learning from cultural organizations’ educational programs, especially when there is a deliberate effort to infuse standards-based content.

Broadly speaking, arts integration refers to a method of instruction that “integrates content and skills from the arts with content and skills from other core subjects, toward increasing learning in both areas” (Goff & Ludwig, 2013, p. 1). The WAC’s Digging Up Arkansas is self-described as an arts integrative program for elementary school students. In this case, a theater performance that includes student interaction with actors and artifacts is used to deliver content knowledge of Arkansas history. Such a straightforward definition, however, misses the multiple nuanced types of arts integration within the field. Burnaford et al. (2007) describe three potential typologies—arts integration as learning through and with the arts, arts integration as a curricular connections process, and arts integration as collaborative engagement. Still, even these definitions fail to comprehensively describe the various examples of arts integration in practice. Digging Up Arkansas, for example, clearly incorporates the first and third definitions, as students both learn history through and with theater, and the program itself was created through a collaboration with teaching artists and traditional educators. It fails to satisfy the second definition, however, because no actual “art” content knowledge is introduced alongside history content. The art form (theater) is the vehicle by which students engage with history, and any learning about theater itself occurs passively.

While arts integration is difficult to define in theory, it is even more varied in practice. In some countries, such as Finland, “arts integration” occurs at scale throughout the national education system (Robinson, 2013). The federalist nature of the school system in the United States leads to more piecemeal approaches, with some of the largest examples involving networks of whole-school models, such as the multistate A+ network (Goff & Ludwig, 2013; Noblit et al., 2009). Other approaches involve coordinated networks of schools, teaching artists, and cultural organizations, such as Chicago’s Arts Partnerships in Education, Houston’s Arts Access Initiative, and the Guggenheim Museum’s Learning Through Arts Program (Bowen & Kisida, 2019; Goff & Ludwig, 2013). The delivery methods are varied, and include teaching artists working with schools, professional development opportunities for traditional classroom teachers, and coordinated integration strategies with cultural organizations to deliver content through the arts.

While research about the effectiveness of arts integration has generally pointed to positive benefits for students, the high variation in the types of programs delivered and methodological approaches employed have made it difficult to generalize the effects (Ludwig et al., 2016). The most rigorous research available includes a number of quasi-experimental studies that have explored the effects of arts integration and the potential mechanisms. Graham and Brouillette (2016) compared students who received arts-based physical science lessons that incorporated visual and performing arts in Grades 3 to 5 in a sample of schools in San Diego and found that it led to higher science achievement as measured by standardized tests. In another study examining science outcomes, Hardiman et al. (2014) found that an arts integration program focused on science learning for fifth graders led to higher knowledge retention for students at lower levels of baseline reading achievement. In a follow-up study, Hardiman et al. (2019) confirm these findings utilizing a larger sample of students and random assignment.

Additional studies have examined outcomes in English language arts and student engagement, particularly from strategies that used performing arts as the vehicle for arts integration. Peppler et al. (2014), for example, investigated a multiarts integration model delivered in partnership between a community arts organization and Los Angeles public schools and found that arts integration led to higher gains in English language arts for elementary students compared with a matched comparison group of schools. The program was particularly effective for English Language Learners (ELL) who began the study with lower test scores, suggesting that arts integration may be most effective for students who struggle in conventional school models. In another instance, Greenfader et al. (2015) used a randomized design to assess a yearlong drama and creative movement intervention among K–2 students and found that the impacts on oral language skills were highest for struggling ELL students. Similarly, Brouillette et al. (2014) examined an arts integration program in urban elementary schools that included teaching artists who co-taught dance and theater lessons with classroom teachers in schools with high concentrations of ELL students and found reductions in absences on days the artists visited. Inoa (2014) found that a program for sixth- and seventh-grade students that infused drama within a language arts curriculum was associated with higher language arts and math outcomes. Additionally, Walker et al. (2011) studied the effects of using arts-integrated drama techniques in fourth- and fifth-grade social studies and language arts classrooms. The findings included higher test scores in language arts, higher course grades in social studies, and more positive student attitudes about the arts.

Though the evidence base is still somewhat limited, these studies provide an empirical foundation that suggests that arts integration has the potential to improve academic performance in other subjects. While the specific mechanisms are not fully known, related theory and research provide important insights suggesting that programs that employ theater techniques allow students to develop a deeper connection with the academic content (Greene et al., 2018; Mehta, 2017). Importantly, performance and theater techniques often employ student participation. Walker et al. (2011) noted that classroom observations of theater interventions demonstrated that drama-related arts integration increases engagement and gives students a sense of “ownership” over their own learning. Additionally, arts integration through theater can increase social interaction, which provides an additional pathway for students to learn from each other and their teachers (Greenfader et al., 2015). Finally, theater and other performing arts interventions often involve movement and gesture. Hardiman et al. (2014) suggests that arts integration that involves gestures and movement may have larger benefits for struggling students because it provides the opportunity to interact with academic content through multiple pathways that promote retention.

Cook et al. (2007) add additional insights from experimental psychology. In their research, they found that experimentally incorporating gestures while learning a new concept increased students’ knowledge retention. They note that gesturing may give learners “an alternative, embodied way of representing new ideas” (p. 1047). Anderson (2018) illustrates a conceptual and theoretical approach to embodied learning using drama-based pedagogical tools and tableaux vivants, which are similar to an approach used in this study. Taken together, these insights suggest plausible mechanisms through which arts integration using theater techniques can increase student engagement, provide a sense of ownership, foster social interactions, and ultimately strengthen retention of course content.

Despite these promising developments, a number of systematic reviews and meta-analyses have attempted to make sense out of the wide variation in contexts and methodology in the arts integration research base, and a common conclusion is that rigorous research is still lacking (e.g., Goff & Ludwig, 2013; Podlozny, 2000; Robinson, 2013). This lack of a strong and coherent research base likely puts the arts at a disadvantage in terms of resources. Noting that the Every Student Succeeds Act of 2015 (ESSA) requires education agencies to cite evidence in order to qualify for funding streams that support arts integration. Ludwig et al.’s (2017) comprehensive systematic review of the arts integration literature focuses squarely on the quality of the evidence base. They consider ESSA’s model of evidence requirements by organizing research-based studies of arts integration into four “tiers.” As they note, ESSA’s Tiers I to III must “demonstrate a statistically significant effect on improving student outcomes or other relevant outcomes,” with Tiers I to III ranging from “strong evidence,” “moderate evidence,” to “promising evidence” (ESSA, 2015). At the upper end of the spectrum, a randomized controlled trial would constitute a Tier I study. Tier IV studies lack empirical evidence and are not sufficient to justify funding.

The main findings of the report corroborate and supplement findings from previous reviews of arts education research (McCarthy et al., 2004; Robinson, 2013). First, arts integration takes many forms across arts disciplines and instructional approaches, which can make it a difficult intervention to summarize, yet at the same time allows for many pathways through which it can align with ESSA’s goals. Second, through ESSA’s funding opportunities, there are at least 12 funding streams to support arts education through arts integration. Finally, the existing evidence base to support such funding and interventions is notably weak. In total, the review identified 44 interventions with evidence in Tiers I to IV, but 34 of those only met Tier IV status. Of the 10 studies that met Tier I to III standards, only one intervention and supporting study met the Tier I evidentiary threshold of strong evidence. The authors concluded that “researchers can help provide more Tier I evidence by using a randomized controlled trial study design” (Ludwig et al., 2017, p. 10). In the following sections, we describe just such an approach that we used to assess the effectiveness of integrating historical content standards into a live theater experience.

Method

The Program: Digging Up Arkansas

Established in 1992, WAC, a nonprofit regional arts center in Northwest Arkansas, serves nearly 350,000 people each year and is home to a regional symphony, a professional theater company, and a center for visual arts exploration (WAC, 2019b). WAC has a robust education program that offers professional development for classroom teachers and performing arts field trips for more than 36,000 students each year (WAC, 2019c). The center offers matinee field trips for students in a variety of art forms, including dance, theater, puppetry, and world music. These performances are opportunities for young people to explore world cultures, make connections to important works of literature, or to learn more about art forms. WAC education programmers make explicit efforts to link these performances to the core curriculum of Arkansas’s public schools (WAC, 2019a).

In an effort to assess the relevance of their offerings, WAC solicited feedback from local educators and identified a need to do more to engage students in the study of Arkansas history. WAC educators believed that live theater could make state history memorable and interesting to young people, and in response WAC produced an original play, Digging Up Arkansas, written by Arkansas playwright and educator Mike Thomas and developed, directed, and performed by Trike Theatre. The play tells the story of three writers from the Federal Writer’s Project who invite students to study and organize historic artifacts before President Roosevelt’s scheduled arrival to Arkansas in 1936. Topics investigated include Arkansas’s first people, the Arkansas Post, the Civil War, music, the important role of women in Arkansas history, and the introduction of electricity to rural Arkansas (WAC, 2012). The play incorporates third- and fourth-grade-level Arkansas history concepts and aligns with state curriculum goals. Since its debut performance in 2010 more than 50,000 children—representing each county across the state—have learned about Arkansas history through this production (Lane, 2017).

Digging Up Arkansas is not a typical theatrical production where actors perform on a stage to a passive audience. Rather, the actors invite students into the “Works Progress Administration research tent,” and their participation becomes part of the play’s action. Students are led into the tent by costumed ushers and actors in-character that actively engage with them. Throughout the performance, the actors ask the students to use their bodies to represent important ideas and concepts using physical gestures and posturing to represent ideas via the drama strategy of “tableau.” They are also encouraged to talk with their neighbor about their ideas regarding the content, and to use choreographed gestures to learn and then demonstrate factual knowledge and strengthen their understanding of key concepts (WAC, 2012). Research and theory suggest that such embodied learning techniques may increase content knowledge retention (Anderson, 2018; Hardiman et al., 2014).

By presenting historical events from the point of view of people who experienced it, Digging Up Arkansas aims to make facts about Arkansas’s past memorable, meaningful, and relevant (WAC, 2012). The play explains the significance of history and illustrates its relationship to contemporary life. As a result, it sets the stage for students to gain knowledge as well as form personal connections and empathize with historical Arkansans. By delivering history content through visual art, music, and drama, students are actively engaged in the learning process, and many different types of learners may develop an appreciation for the state’s history and retain more pertinent facts, including content mandated by Arkansas State History Standards.

Study Design

To best understand how experiencing the live performance affects learners, we designed an experimental evaluation focused on our hypothesized student learning outcomes. To recruit participants, we communicated notice of the Digging Up Arkansas field trip and associated study to educators working in the 16 school districts that comprised WAC’s service area through publication in WAC’s field trip guide and through direct emails to principals in surrounding districts.

The Digging Up Arkansas program was offered as one of 14 different field trip options available to school groups between mid-September 2013 and late-May 2014 at the WAC. Digging Up Arkansas, however, differed from other offerings because study participants were invited to participate at no cost (generally, schools pay $5 per student to attend performances at WAC). When teachers submitted an application for complimentary tickets, they were informed that they would randomly be assigned to bring their students in the current year or the following year, and they agreed that their students would participate in a brief study assessment at the end of the current school year. The researchers informed teachers that the collected data would be used to evaluate the impact of live performance fieldtrips on student learning and the potential benefits of out-of-classroom experiences.

Applicant groups were randomly assigned such that half of the groups were able to see the program Digging Up Arkansas during the current school year (the treatment group) while the other half were guaranteed a spot the following year (the control group). In some cases, individual classrooms applied, in other cases applicants were composed of entire school grades. Though it is often considered preferable to randomly assign at the student level for the purposes of study power and treatment-control comparability, the fact that the program was designed as a classroom-based program made student-level randomization impossible. Moreover, randomly assigning students at the group level has the added desirable feature of reducing the potential for treatment-control group contamination between subjects (Shaddish et al., 2002).

To ensure baseline equivalence between the treatment and control groups, the 28 eligible applicant groups were stratified into pairs prior to randomization. Twelve school group applicants (six pairs) were stratified based on pairing adjacent grades within the same schools. An additional 14 applicant groups were stratified based on being from the same district and same grade, but in different schools. We then used school percent eligible for free or reduced-price lunch as an initial screen and school percent minority as a second screen to create these seven pairs. Finally, two school groups in different districts within the same grade with similar demographics were used to make one additional treatment-control pairing.

Survey Measures and Data Collection

To evaluate the program, we took steps to understand the program theory of the intervention. Through conversations with the creators and producers of the show and prior experience working with educational arts programming we identified five key areas of likely impact. Two outcomes were self-evident: The show should teach historical content knowledge and generate an interest in theater and other performing arts. Additionally, program operators and educators believed that through the engaging medium of theater, students might show an increased interest in learning history. Finally, program operators and evaluators hypothesized that the re-creation of historical settings and events and the immersion through audience participation throughout the performance could promote historical empathy.

We developed a series of 13 multiple-choice questions to measure historical content knowledge. These questions were designed by us to assess Arkansas history content standards based on the state-mandated Social Studies Curriculum Frameworks for students in Grades 3 to 5 that were specific to Arkansas history (Arkansas Department of Education, 2007). To construct testable questions, we combed the curriculum framework for items that clearly defined content knowledge standards. In most cases, these were straightforward. For example, one standard directs educators to “discuss the meaning of the state motto of Arkansas,” while another standard mentions that students should be able to “recognize American Indian tribes of Arkansas” (Arkansas Department of Education, 2007). We constructed questions independent of any knowledge of the show, purposefully making sure not to rely on seeing the show or having access to the script. Without having tailored the assessment to particular elements from the show, this study more broadly addresses the question of whether or not students learned material listed in the state content standards—material that even control group classrooms engaging in “business as usual” should have learned. We also did not share this assessment with program operators prior to the completion of data collection to avoid the possibility of programmers and performers effectively “teaching to the test.” We provide the full list of the items used to assess content knowledge in the appendix. In addition to these items, the surveys also captured student demographic information and a measure of their previous cultural activities outside of school.

After data collection was completed, program operators reviewed the instrument and informed us that two (of the 13) questions were not connected to historical content covered in the production (one was a question about the state’s motto and the other was a question about the Mississippi River). Nevertheless, we kept all of the content questions in a summative outcome measure of the percentage of questions answered correctly that we used to evaluate the program. Tested separately, neither of these two questions showed a statistically significant treatment effect. The implications of this are twofold: First, this serves as further evidence of treatment-control baseline balance, because treatment and control group students equally knew something that was not directly related to the treatment. Second, the treatment effect is slightly underestimated, as the removal of these items leads to a slightly higher treatment group advantage on this summative outcome. That said, we believe the decision to include all the items is the best representation of the policy implications of an intervention intended to cover established state standards.

In addition to historical content knowledge and basic demographic information (gender, grade-level, ethnicity, and previous cultural activities), the surveys included Likert-type scale indicators of students’ attitudes about the arts, live theater and performing arts centers, attitudes toward learning about history, and historical empathy related to Arkansas history. Historical empathy “is the ability to understand and appreciate what life was like for people who lived in a different time and place” (Greene et al., 2014, p. 83). These survey items were adapted from prior studies measuring similar constructs in evaluations of arts-based educational interventions and reviewed by members of the research team for face validity (Greene et al., 2014). We provide the full list of items used to assess attitudinal outcomes in the appendix.

Interest in learning Arkansas history was measured using three questions with 4-point Likert-type scaled responses, and historical empathy was measured with two questions with 4-point Likert-type scaled responses (Cronbach’s α = .88 and .59, respectively). Interest in future theater and performing arts consumption was measured in two ways. A series of five Likert-type scaled questions probed students’ interest in returning to the WAC, while two separate questions asked about interest in seeing more performing arts generally (Cronbach’s α = .85 and .82, respectively). Each of these outcome domains were combined into scales and converted to standard deviation units, with a mean of 0 and standard deviation of 1. This conversion allows us to express our findings in terms of effect sizes as a percentage of standard deviation units.

Surveys were administered to treatment and control groups between 25 and 30 weeks after the treatment groups visited the WAC. Treatment groups visited the WAC and participated in the Digging Up Arkansas program during October and November 2013. Student surveys and instructions for implementing them were mailed to teachers in April 2014 and returned by May 2014. Teachers received a strict set of instructions to ensure careful and consistent administration. Teachers were instructed to make sure not to influence students’ responses and ensure that students could not copy from each other. Teachers were also instructed to tell students that the assessment was not for a grade, and that their answers were confidential. As a token of appreciation and to ensure buy-in, teachers were informed that if they returned student surveys by May 2, they would receive ticket vouchers to an upcoming WAC performance. Data were successfully collected from all but one of the school groups that agreed to participate. In this instance, the school seemed to lack the necessary organizational capacity to comply with data collection and may have misunderstood the terms of the agreement. To maintain internal validity, this school group and its matched control group were dropped from the study.

Analytic Approach

The random assignment of classrooms to treatment and control conditions achieved comparable balance across observable characteristics for the 1,892 Grade 3 to 5 students in our sample (Table 1). Students were balanced across measures of gender, ethnicity, grade level, the number of previous cultural activities they had participated in outside of school (e.g., music lessons, dance lessons), and school percent eligible for free or reduced-priced lunch. The majority of the students in the sample were White, with a sizable proportion of Hispanic students consistent with the demographics of the region.

Table 1

Baseline Characteristics of the Sample and Treatment/Control Balance

	Treatment Mean, n = 895	Control Mean, n = 997	Difference
Female	0.48	0.50	−0.02
White	0.56	0.59	−0.03
Hispanic	0.23	0.21	0.02
Black	0.05	0.05	0.00
Average grade level	3.68	3.85	−0.17
Previous cultural activities	0.94	0.90	0.04
School percent FRL	0.53	0.53	0.00
School percent minority	0.54	0.59	−0.05

Note. FRL = free/reduced-price lunch. Students who did not identify as White, Hispanic, or Black consisted of students whom self-identified as Asian/Pacific Islanders (3%), Native Americans (4%), Marshallese (3%), or an “Other” category (6%). Standard deviation for average grade level = 0.71. School percent FRL ranged from 0.25 to 0.88. School percent minority ranged from 0.21 to 0.90.

With comparable treatment and control groups, we can use fairly straightforward estimation techniques. To assess the impacts of the program, we estimate the following regression equation for student i in matched pair m and school s:

\begin{array}{l} O u t c o m e s_{i m s} = α_{0} + T r e a t_{s} β_{1} + M a t c h_{m} β_{2} + \\ X_{i} β_{3} + P o v e r t y_{s} β_{4} + ε_{i m s} \end{array}

The variable Treat_s is equal to 1 if the student’s school was in the treatment group randomly assigned to the Digging Up Arkansas program and 0 if in the control group. Because we stratified randomization within applicant group matched pairs, we include Match _m as a vector of dummy variables that generates estimates within matched pairs. To increase the precision of our estimates and adjust for any potential differences between the treatment and control groups, we include X _i, a vector of student characteristics including indicator variables for female, minority status, and grade level, as well as a sum of students’ reports of previous cultural activities (music, dance, art, or theater lessons/activity). Additionally, we include the school-level percentage of students eligible for free or reduced-priced lunch. Finally, ε_ims is a stochastic error term clustered at the applicant group level to account for the correlation between students nested within applicant groups.

We also test potential moderating variables by including interaction terms between the binary treatment indicator and the baseline characteristics of gender and minority status. This approach allows us to obtain separate estimates for these subgroups while also testing for any significant differences in the size of the effects between male and female or minority and nonminority treatment group students. We conduct this exploratory subgroup analysis because educational interventions commonly produce heterogeneous effects by student gender and race/ethnicity, including studies involving arts-based education. In terms of gender, male students have tended to show less enthusiasm for arts experiences than female students (Kisida et al., 2014). In terms of minority status, there is evidence that arts experiences may induce larger effects for a variety of reasons. First, due to socioeconomic conditions, they tend to have less access to arts experiences in schools and from their families. Starting from a lower base may mean they have more room to grow in their attitudes toward the arts. For academic outcomes, research involving arts integration has shown that arts experiences can provide a pathway for historically marginalized students who traditionally perform lower on standard measures of academic achievement. For example, Peppler et al. (2014) found that an arts integration program targeting literacy outcomes had larger positive effects for ELLs, perhaps because the experience helped them make more meaningful connections to the course content. Similarly, Hardiman et al. (2014) found that an arts integration program led to significantly better knowledge retention for students at lower levels of baseline reading achievement. They suggest that arts integration may have larger benefits for struggling students by providing the opportunity to learn content through alternative pathways.

Results

Results for the full sample of students show consistent and sizeable gains across all outcome measures (Table 2). In terms of historical content knowledge, the treatment group scored four percentile points higher than the control group, an effect size of 0.21 of a standard deviation (p = .04). Treatment group students also demonstrated a greater interest in learning Arkansas history (p = 0.008) and an increase in historical empathy (p = 0.003). In terms of generating interest in the performing arts, treatment group students were more likely than control group students to express an interest in performing arts generally (p < .001), as well as increased interest in visiting the WAC (p = .003).

Table 2

Treatment Effects for the Full Sample

Outcomes	Treatment Mean (SD)	Control Mean (SD)	Difference (SE)	Effect Size (SE)
Content knowledge	0.58 (0.19)	0.54 (0.18)	0.04** (0.02)	0.21** (0.09)
Interest in learning history	2.02 (0.91)	1.89 (0.92)	0.14** (0.03)	0.15*** (0.05)
Historical empathy	1.82 (0.83)	1.61 (0.86)	0.21*** (0.03)	0.25*** (0.07)
Interest in performing arts	2.40 (0.83)	2.15 (0.88)	0.22*** (0.03)	0.25*** (0.04)
Interest in WAC	2.11 (0.73)	1.96 (0.71)	0.15*** (0.05)	0.21*** (0.06)

Note. WAC = Walton Arts Center. Estimates are obtained from ordinary least squares regression models with robust standard errors clustered by applicant group. Effect sizes are in terms of standard deviation units. Models include controls for student gender, grade, minority status, school-percent free or reduced-price lunch, a count of baseline cultural activities, and an indicator variable for each matched pair. Treatment and control group means are from Likert-type scaled responses with four categories (0–3). Depending on item-response missing rates, valid n ranges from 1,874 to 1,882.

p < .10. **p < .05. ***p < .01.

To get a sense of whether the effects may be different for different populations of students, we also explored the potential for moderating effects by gender and race/ethnicity (Table 3). For every subgroup we examined, the results were positive and statistically significant across all outcome measures for all subgroups with the exception of the effect on interest in history for White students. By comparison, the significant effect size of 0.22 for treated minority students (p = .002) on this outcome is more than double in magnitude compared with White students. This corresponds to a similar pattern of larger effects for minority students on generating interest in the performing arts. This suggests that the intervention effect for interest in history and interest in performing arts depended, in part, on whether students were racially or ethnically from nondominant or minority groups. There are, however, no clear systematic differences between male and female students.

Table 3

Treatment Effect Sizes by Minority Status and Gender

Outcomes	White (SE)	Minority (SE)	Male (SE)	Female (SE)
Content knowledge	0.21** (0.09)	0.22* (0.13)	0.23** (0.10)	0.21** (0.09)
Interest in learning history	0.10 (0.07)	0.22*** (0.06)	0.14* (0.08)	0.16** (0.07)
Historical empathy	0.26*** (0.08)	0.23** (0.10)	0.26** (0.11)	0.23*** (0.08)
Interest in performing arts	0.19*** (0.06)	0.32*** (0.06)	0.29*** (0.07)	0.22*** (0.05)
Interest in WAC	0.12^a (0.07)	0.32***,^a (0.10)	0.19** (0.08)	0.24*** (0.08)

Note. WAC = Walton Arts Center. Estimates of effect sizes are in terms of standard deviation units and are obtained from ordinary least squares regression models with robust standard errors clustered by applicant group. Models include controls for student gender, grade, minority status, school-percent free or reduced-price lunch, a count of baseline cultural activities, and an indicator variable for each matched pair. Depending on item-response missing rates, valid n ranges from 1,874 to 1,882.

Indicates that an interaction term testing for differences between the two subgroups (e.g., males and females) was statistically significant at p < .10.

p < .10. **p < .05. ***p < .01.

Discussion

Employing a randomized controlled trial, we examined the effects for student groups randomly assigned to participate in a program that used an interactive live theater performance to engage students in history education. We find significant gains in content knowledge, increased interest in learning history, an increased sense of historical empathy, and higher interest in the performing arts as a result of the program. The experience increased student interest in both history and the arts, while effectively delivering content articulated in state curricular standards. The fact that we see gains across multiple important domains is especially promising—students gained content knowledge while also being exposed to live theater. This evidence suggests that arts integration has the potential to effectively address a variety of objectives seemingly without compromise. That said, we would caution against any notions that arts integration could serve as a replacement for the arts. Rather, as we demonstrate in this case, arts techniques may serve as an important vehicle for content delivery in other subjects.

When we disaggregate the data and examine whether the effects are moderated by gender and race/ethnicity, the results are remarkably consistent with some minor exceptions. The data suggest that minority students benefited more in terms of interest in history and interest in live theater. This is consistent with prior research, which has found that students from historically underserved groups are less likely to be exposed to culturally and academically enriching experiences, which in turn leaves them more room to grow when educational interventions activate their interests (Government Accountability Office, 2009; Kisida et al., 2014, Rabkin & Hedberg, 2011).

The evidence we present here, in conjunction with prior studies and theory, builds a case for the role of arts integration as a tool for enhancing learning. That said, arts integration is a broad term that covers many different disciplines and approaches, and to date we still have limited evidence about the effectiveness of all that occurs under the arts integration umbrella. In this particular case, an interactive theater experience improved content knowledge retention, attitudes toward learning history, historical empathy, and interest in theater. This builds nicely on prior theory and studies that suggest theater-based strategies improve retention, particularly through the use of interaction, participation, and gesture (Anderson, 2018; Cook et al., 2007; Hardiman et al., 2014). The fact that this theater experience includes these common elements builds a stronger case for these components of arts integration strategies. Moreover, the increases we see in interest in learning history as well as interest in theater are aligned with studies showing arts integration can increase student engagement and interest in art (Bowen & Kisida, 2019; Brouillette et al., 2014; Kisida et al., 2014; Walker et al., 2011).

A related and consistent finding in previous literature suggests that ELLs may particularly benefit from arts integration (Greenfader et al., 2015; Peppler et al., 2014). Though we lack an indicator variable for ELL status in our data, we similarly see evidence of stronger results for the minority students in our sample, roughly half of which identify as Hispanic. The evidence thus far suggests that arts integration may have particular benefits for historically marginalized groups and students who struggle within traditional educational settings. By opening up multiple pathways for alternative learning methods, arts integration promises to be an effective way to address persistent achievement gaps in our educational system.

Limitations

Though we apply a rigorous causal research design by randomly assigning student groups to participate in the program, there are several limitations to our study. We cannot be sure if the program itself is entirely responsible for the changes we observe. It could also be the case that the experience had positive spillover effects that motivated teachers to spend more time covering the content in their regular classroom setting. This “black box” question is impossible to determine with available data. Future qualitative or survey research that digs deeper into the precise mechanisms would be valuable, especially regarding the impacts of arts integration techniques on the practices of classroom teachers.

Moreover, while the experimental methodology gives this study high internal validity, it has limited external validity. In addition to not knowing how these effects might generalize to similar student populations in other areas, we can also not be sure how this particular program might be representative of similar programs that infuse academic content within an arts experience. It is also important to note that our sample is restricted to teachers who were motivated to seek out this educational experience. Similar results may not be achieved if educational administrators attempt to require such programs with teachers not interested in arts integration. Additionally, our attitudinal measures only capture short-term effects measured via survey assessment. We cannot say to what extent attitudinal changes in students will endure over time.

It is also important to note that this arts integration experience was created and delivered by professional artists working with an established cultural organization. It is possible, and perhaps probable, that the skills needed to deliver such a program are not common among typical classroom educators. Strategies that aim to enhance arts integration without partnering with professional artists may not be as effective. Finally, this program does not explicitly include teaching the arts. Rather, history is taught through an interactive arts experience. Arts integration that also includes explicit instruction in the arts remains an understudied area that is open for further research.

Conclusion

Despite these limitations, our findings have important implications for policy and practice. Our results suggest that efforts to convey history learning objectives can be effectively achieved through live interactive theater performances. According to state curricular standards, the historical content should have been introduced to students in the control group experiencing “business as usual.” Yet the treatment group outperformed the control group in terms of content knowledge while also gaining an increased interest in history and theater. Given the nascent state of research in arts integration, these findings are promising and should provide educators with additional tools through which to approach integrated learning strategies, as well as enable them to better leverage the resources available to support similar efforts through ESSA’s funding provisions. Educators can have confidence that standards-based learning is a viable outcome of arts-integrated learning settings and in some cases may be more effective than typical classroom learning environments. Educational leaders should explore partnerships with cultural institutions as a significant and efficacious tool for student learning across other subjects and arts experiences.

Footnotes

Appendix

Student Survey Items for Outcome Scales

Arkansas History Content Knowledge

• The state motto of Arkansas is “regnat populus.” What do you think this means? (Let the people rule)

• The name “Arkansas” comes from a word used by which people? (Quapaw tribe of native Americans)

• Which river runs along the eastern border of Arkansas? (Mississippi)

• Where in the United States is Arkansas located? (The South)

• On which side did Arkansas fight during the Civil War? (The Confederate states)

• What was the main issue that led to the Civil War? (Slavery)

• Who was president during the Civil War? (Abraham Lincoln)

• What happened because of the Civil War? (African American slaves received their freedom)

• What was the name of an early Arkansas settlement where French, Spanish, and American Indians did lots of trading? (Arkansas Post)

• What is the capital of Arkansas? (Little Rock)

• Which of these Native American tribes were important in Arkansas history? (Quapaw, Osage, and Caddo)

Interest in Arkansas History^a

• How interested are you in learning about Arkansas history?

• Arkansas history is interesting.

• I would enjoy learning more about Arkansas history.

Historical Empathy^b

• I can imagine what it was like to live in Arkansas 100 years ago.

• I have a good understanding of how early Arkansans lived.

Interest in Performing Arts^b

• How interested are you in going to see plays, musicals, concerts, and other shows?

• I enjoy seeing plays, musicals, concerts, and other shows.

Interest in Walton Arts Center^b

• How interested are you in visiting the Walton Arts Center?

• If your friends or family wanted to go to the Walton Arts Center, how interested would you be in going?

• I would tell my friends that they should visit the Walton Arts Center.

• Trips to the Walton Arts Center are fun.

• I plan to visit the Walton Arts Center when I am a grown up.

Four-category Likert-type response: not interested, a little interested, somewhat interested, and very interested.

Four-category Likert-type response: strongly disagree, somewhat disagree, somewhat agree, and strongly agree.

Authors

BRIAN KISIDA is an assistant professor in the Truman School of Public Affairs at the University of Missouri. His research examines the broad educational benefits of school partnerships with cultural institutions and community arts organizations, teacher diversity, and school integration.

LAURA GOODWIN, MFA Savannah College of Art and Design, is vice president of Learning & Engagement at Walton Arts Center in Northwest Arkansas. She leads the center’s learning initiatives and advocates for the increased role of arts field trips with education leaders and policymakers.

DANIEL H. BOWEN is an assistant professor with the Department of Educational Administration and Human Resource Development at Texas A&M University. His research employs experimental and quasi-experimental methods to investigate the broader impacts of culturally enriching, school-sponsored programs and policies.

References

Anderson

(2018). Creative engagement, embodied metaphor, the affective brain, and meaningful learning. Mind, Brain, and Education, 12(2), 72–81. https://doi.org/10.1111/mbe.12176

Arkansas Department of Education. (2007). Social Studies Cur-riculum Framework. http://dese.ade.arkansas.gov/public/userfiles/Learning_Services/Curriculum%20and%20Instruction/Frameworks/Social%20Studies/soc_studies_k_8_051308.pdf

Associated Press. (2012, October 13). School field trips in decline amid standardized testing. The Columbus Dispatch. http://www.dispatch.com/content/stories/local/2012/10/13/school-field-trips-in-decline-amid-standardized-testing.html

Bassok

Latham

Rorem

(2016). Is kindergarten the new first grade? AERA Open, 1(4), 1–31. https://doi.org/10.1177/2332858415616358

Blair

(2008, December 22). Museum field trips tailored to teach to the test. National Public Radio. http://www.npr.org/templates/story/story.php?storyId=98593843

Bowen

D. H.

Kisida

(2017). The art of partnerships: Communities collaborating to enhance arts education. Phi Delta Kappan, 98(7), 8–14. https://doi.org/10.1177/0031721717702624

Bowen

D. H.

Kisida

(2019). Investigating causal effects of arts education experiences: Experimental evidence from Houston’s Arts Access Initiative (Research Report, Vol. 7, Issue 4). Rice Kinder Institute for Urban Research. https://kinder.rice.edu/sites/g/files/bxs1676/f/documents/Investigating%20Causal%20Effects%20of%20Arts%20Education%20Experiences%20Final_0.pdf

Brouillette

Childress-Evans

Briana

Farkas

(2014). Increasing engagement and oral languages skills of ELLs through the arts in the primary grades. Journal for Learning Through the Arts, 10(1), 1–25. https://doi.org/10.21977/D910119300

Burnaford

Brown

Doherty

McLaughlin

H. J.

(2007). Arts integration frameworks, research & practice: A literature review. Arts Education Partnership. https://www.aep-arts.org/wp-content/uploads/Arts-Integration-Frameworks-Research-Practice_A-Literature-Review.pdf

10.

Chappell

S. V.

Cahnmann-Taylor

(2013). No child left with crayons: The imperative of arts-based education and research with language “minority” and other minoritized communities. Review of Research in Education, 37(1), 243–268. https://doi.org/10.3102/0091732X12461615

11.

CNN. (2017, July 12). Are museums still relevant? CNN. https://www.cnn.com/travel/article/are-museums-still-relevant/index.html

12.

Cook

S. W.

Mitchell

Goldin-Meadow

(2007). Gesturing makes learning last. Cognition, 106, 1047–1058. https://doi.org/10.1016/j.cognition.2007.04.010

13.

Ellerson

N. M.

(2010). A cliff hanger: How America’s public schools continue to feel the impact of the economic downturn. American Association of School Administrators. http://www.aasa.org/uploadedFiles/Policy_and_Advocacy/files/CliffHangerFINAL%281%29.pdf

14.

Every Student Succeeds Act, 20 U.S.C. § 6301 (2015). https://www.congress.gov/114/plaws/pubi95/PLAW-114publ95.pdf

15.

Gadsden

V. L.

(2008). The arts and education: Knowledge generation, pedagogy, and the discourse of learning. Review of Research in Education, 32(1), 29–61. https://doi.org/10.3102/0091732X07309691

16.

Goff

Ludwig

(2013). Teacher practice and student outcomes in arts-integrated learning settings: A review of literature. American Institutes for Research. http://www.air.org/resource/teacher-practice-and-student-outcomes-arts-integrated-learning-settings-review-literature

17.

Government Accountability Office. (2009, February). Access to arts education: Inclusion of additional questions in education’s planned research would help explain why instruction time has decreased for some students (GAO Report No. 09-286). https://www.gao.gov/new.items/d09286.pdf

18.

Graham

N. J.

Brouillette

(2016). Using arts integration to make science learning memorable in the upper elementary grades: A quasi-experimental study. Journal for Learning Through the Arts, 12(1), 1–17. https://doi.org/10.21977/D912133442

19.

Greene

J. P.

Erickson

H. H.

Watson

A. R.

Beck

M. I.

(2018). The play’s the thing: Experimentally examining the social and cognitive effects of school field trips to live theater performances. Educational Researcher, 47(4), 246–254. https://doi.org/10.3102/0013189X18761034

20.

Greene

J. P.

Kisida

Bowen

D. H.

(2014). The educational value of field trips. Education Next, 14(1), 78–86. https://www.educationnext.org/files/ednext_XIV_1_greene.pdf

21.

Greenfader

C. M.

Brouillette

Farkas

(2015). Effect of a performing arts program on the oral language skills of young English learners. Reading Research Quarterly, 50(2), 185–203. https://doi.org/10.1002/rrq.90

22.

Hardiman

M. M.

Johnbull

R. M.

Carran

D. T.

Shelton

(2019). The effects of arts integrated instruction on memory for science content. Trends in Neuroscience and Education, 14, 25–32. https://doi.org/10.1016/j.tine.2019.02.002

23.

Hardiman

Rinne

Yarmolinskaya

(2014). The effects of arts integration on long-term retention of academic content. Mind, Brain, and Education, 8(3), 144–148. https://doi.org/10.1111/mbe.12053

24.

Inoa

(2014). A study on the relationship between theater arts and student literacy and mathematics achievement. Journal for Learning Through the Arts, 10(1), 1–21. https://doi.org/10.21977/D910123495

25.

Kisida

Bowen

D. H.

Greene

J. P.

(2016). Measuring critical thinking: Results from an art museum field trip experiment. Journal of Research on Educational Effectiveness, 9(Supp1.), 171–187. https://doi.org/10.1080/19345747.2015.1086915

26.

Kisida

Greene

J. P.

Bowen

D. H.

(2014). Creating cultural consumers: The dynamics of cultural capital acquisition. Sociology of Education, 87(4), 281–295. https://doi.org/10.1177/0038040714549076

27.

Lane

P. B.

(2017, January 5). Walton Arts Center expands access to theater across Northwest Arkansas. Walton Family Foundation. https://www.waltonfamilyfoundation.org/stories/home-region/walton-arts-center-expands-access-to-theater

28.

Lewin

(2010, April 21). Museums take their lessons to the schools. The New York Times. http://www.nytimes.com/2010/04/22/us/22fieldtrips.html?pagewanted=all

29.

Ludwig

M. J.

Boyle

Lindsay

(2017). Review of evidence: Arts integration research through the lens of the Every Student Succeeds Act. American Institute for Research. https://www.wallacefoundation.org/knowledge-center/Documents/Arts-Integration-Research-Every-Student-Succeeds-Act-ESSA.pdf

30.

Ludwig

Marklein

M. B.

Song

(2016). Arts integration: A promising approach to improving early learning. American Institutes for Research. https://www.air.org/resource/arts-integration-promising-approach-improving-early-learning.

31.

McCarthy

K. F.

Ondaatje

E. H.

Zakaras

Brooks

(2004). Gifts of the muse: Reframing the debate about the benefits of the arts. RAND Corporation. https://doi.org/10.7249/MG218

32.

Mehta

(2017, February 8). Schools already have good learning, just not where you think. Education Week. http://blogs.edweek.org/edweek/learning_deeply/2017/02/why_schools_already_have_good_learning_just_not_where_you_think.html?qs=jal+mehta+periphery

33.

Noblit

G. W.

Corbett

Wilson

B. L.

McKinney

M. B.

(2009). Creating and sustaining arts-based school reform: The A+ Schools Program. Routledge. https://doi.org/10.4324/9780203887356

34.

Peppler

K. A.

Powell

C. W.

Thompson

Catterall

(2014). Positive impact of arts integration on student academic achievement in English language arts. Educational Forum, 78(4), 364–377. https://doi.org/10.1080/00131725.2014.941124

35.

Plummer

(2014, October 10). Long Beach museum looks to buck trend of declining student field trips. Southern California Public Radio. http://www.scpr.org/blogs/education/2014/10/10/17383/long-beach-museum-looks-to-buck-trend-of-declining/

36.

Podlozny

(2000). Strengthening verbal skills through the use of classroom drama: A clear link. Journal of Aesthetic Education, 34(3–4), 91–104. https://doi.org/10.2307/3333644

37.

Rabkin

Hedberg

E. C.

(2011). Arts education in America: What the declines mean for arts participation (Research report #52). National Endowment for the Arts.

38.

Robinson

H. A.

(2013). Arts integration and the success of disadvantaged students: A research evaluation. Arts Education Policy Review, 114(4), 191–204. https://doi.org/10.1080/10632913.2013.826050

39.

Shaddish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin.

40.

United Nations Educational Scientific, and Cultural Organization. (2001). UNESCO universal declaration on cultural diversity. http://portal.unesco.org/en/ev.php-URL_ID=13179&URL_DO=DO_TOPIC&URL_SECTION=201.html

41.

Walker

E. M.

McFadden

L. B.

Tabone

Finkelstein

(2011). Contribution of drama-based strategies. Youth Theatre Journal, 25, 3–15. https://doi.org/10.1080/08929092.2011.569471

42.

Walton Arts Center. (2012, March 23). Digging Up Arkansas [Video]. YouTube. https://www.youtube.com/watch?v=_hq2vu-2gYU

43.

Walton Arts Center. (2019a). Arts education & impact. https://waltonartscenter.org/about/arts-education-and-impact/

44.

Walton Arts Center. (2019b). Mission & history. https://waltonartscenter.org/about/mission-and-history/

45.

Walton Arts Center. (2019c). Vision & values. https://waltonartscenter.org/about/vision-and-values/

46.

West

(2007). Testing, learning, and teaching: The effects of test-based accountability on student achievement and instructional time in core academic subjects. In Finn

C. E.

Ravitch

(Eds.), Beyond the basics: Achieving a liberal education for all children (pp. 45–61). Fordham Institute.

47.

Yee

(2014, April 7). Arts education lacking in low-income areas of New York City, report says. The New York Times. http://www.nytimes.com/2014/04/07/nyregion/arts-education-lacking-in-low-income-areas-of-new-york-city-report-says.html?smid=tw-share

Teaching History Through Theater: The Effects of Arts Integration on Students’ Knowledge and Attitudes

Abstract

Keywords

Introduction

Background and Previous Literature

Method

The Program: Digging Up Arkansas

Study Design

Survey Measures and Data Collection

Analytic Approach

Results

Discussion

Limitations

Conclusion

Footnotes

Appendix

Authors

References