Abstract
Field trips to museums and performing arts centers are not a frequent activity for most students, so how do cultural organizations learn to make the most of these limited but potentially valuable “gateway experiences” toward increased arts participation? This cross-sectional study examines the engagement of students during a performance at a performing arts center with data collected over three seasons of performances across art forms, grade levels, and student populations (students who attend Title I schools, English learners, and students with disabilities) under the umbrella of Positive Youth Development theory. The findings of regression analyses and analyses of variance feature three aspects that may optimize positive engagement in the arts experience for students: prior attendance to performances, lessons in the art form, and preparation prior to the performance. However, when the data is disaggregated by priority populations, not all the factors predict higher positive engagement in a statistically significant way. The feature that may have a consistent incidence across all populations, and exclusively for students with disabilities, is preparation prior to the performance. These results present practical implications for audience recruitment, accessibility supports, and learning design for diverse learners.
Keywords
Introduction
The benefits of arts participation and engagement associated with youth development outcomes are numerous and broad in characteristics (for a useful review see Foster & Jenkins, 2017). A watershed publication about the development of arts participation (McCarthy et al., 2004) has argued for the value of both the intrinsic and instrumental outcomes of the arts. From this report we know that many active adult arts participants are able to identify an arts “gateway experience” in younger life that was a powerful factor in later participation in art-making and performance/museum attendance. The report identifies theoretical contributing factors that are relevant to this paper, particularly early positive engagement with quality arts experiences.
The key aspect of these initial experiences for future arts involvement is that the arts experience itself. . .engages the participant enough that he or she develops a positive attitude toward the arts and the possibility of future arts involvement. Whether this occurs may well depend on whether the initial experience is appropriate to the individual’s age, interests, and life experience (McCarthy et al., 2004, p. 55).
Recent arts education research has explored the impact of field trips to cultural institutions on varied student outcomes. This study focused on what factors before the field trip may help to optimize that experience. We selected engagement as our key outcome to be a consistent measure across a varied range of arts experiences, and because engagement plays a powerful role as an attentional gateway to further learning. This positive gateway experience is also relevant under the umbrella of the Positive Youth Development theory, which we introduce below.
Positive Youth Development and Arts Engagement
Echoing Martin et al. (2012), we consider the overarching theoretical framework of Positive Youth Development (PYD) as the context in which arts engagement and other factors (background, demographic, etc.) converge to promote positive development and outcomes in young ones. The PYD field developed in recent decades, focuses on the positive psychoemotional growth and development of youth, and on the strengths and resources required for a healthy physical, social, and emotional development. This model is a departure from the traditional approach of attempting to remediate deficits or problem behaviors.
The core concepts in the positive youth development field have been summarized in the following six principles (Benson et al., 2006, p. 896):
All youth have the inherent capacity for positive growth and development.
A positive developmental trajectory is enabled when youth are embedded in relationships, contexts, and ecologies that nurture their development.
The promotion of positive development is further enabled when youth participate in multiple, nutrient-rich relationships, contexts, and ecologies.
All youth benefit from these relationships, contexts, and ecologies. Support, empowerment, and engagement are, for example, important developmental assets for all youth, generalizing across race, ethnicity, gender, and family income. However, the strategies and tactics for promoting these developmental assets can vary considerably as a function of social location.
Community is a viable and critical “delivery system” for positive youth development.
Youth are major actors in their own development and are significant (and underutilized) resources for creating the kinds of relationships, contexts, ecologies, and communities that enable positive youth development.
The field of PYD is greatly shaped by the ecological theory of development, posed by Bronfenbrenner (1979), who argued that we must study young people in the developmental ecology in which they function, highlighting the “mutual accommodation between an active, growing human being and the changing properties of the immediate settings in which the developing person lives” (p. 21). This means that the different settings or systems that have an impact on development are not independent, but are usually nested and interactive, influencing, or modifying one another. What happens in one setting is impacted by multiple other systems, for instance, what happens in the family is impacted by what goes on in the classroom, and vice versa (Benson et al., 2006). Benson and Saito (2001) have described four main settings that are key in providing opportunities for positive youth development, which are (a) programs, (b) organizations, (c) socializing systems, and (d) community, with the last three being the most relevant settings for the current study and the ones we define below.
Organizations are those entities that provide a wide variety of activities and promote relationships, improving the well-being of youth through formal and informal developmental opportunities. Some examples are school-based extracurricular activities, parks, recreation centers, community centers, sports teams or leagues, religious or faith-based organizations, museums and, of course, performing arts centers. It is important to note that “youth in urban areas, particularly those whose families have lower incomes, have less access to (National Commission on Children, 1991) and lower participation in (National Center for Education Statistics, 1990) formal youth development organizations” (Benson & Saito, 2001, p. 130).
Socializing systems are a series of complex and universally present systems that enhance processes and outcomes consistent with youth development principles. These may include families, neighborhoods, schools, religious institutions, museums, cultural organizations, and libraries (Development Services Group, 2014).
Lastly, communities not only correspond to the geographic context where all the settings intersect, but it is also where the social norms, resources, and relationships that directly or indirectly inform human development are promoted or developed. In other words, community can be considered an incubator of positive development and a multifaceted setting where youth can exercise agency and inform the settings, people, places, and policies that will in turn impact their development (Benson et al., 2006).
This brings us to the definition proposed by Benson and Saito (2001): Youth development mobilizes programs, organizations, systems, and communities to build developmental strengths in order to promote health and well-being.
Following these authors’ (Benson & Saito, 2001) general youth development model and the antecedent posed by Martin et al. (2012) specifically regarding young audiences in performing arts, we hypothesize a similar expanded model including an extra level of outcomes, in which (a) background or demographic factors (priority populations: attending a Title I school, ELL student population, students with disabilities population) in confluence with (b) youth development inputs (arts background aspects) impact (c) psychological and educational youth outcomes (student audience engagement) that in turn will ultimately (and almost literally) set the stage for a rise in (d) broad positive health and well-being outcomes (this level is not part of this study).
Field Trips
A field trip is an excursion to a location outside of the classroom with educational intent, where students gain an experiential connection to ideas and subject matter (Tal & Morag, 2009). Davidson et al. (2010) argue that field trips “promote tenets of constructivist and experiential learning theories: connecting to students’ prior knowledge and experiences, tapping into students’ interests, and motivating students to learn more about the topics introduced at the site” (p. 124). In fact, students who participate in a field trip experience are more likely to generate a positive attitude about the subject of the experience (Behrendt & Franklin, 2014). All these aspects create authentic learning opportunities and contribute to student engagement, as students observe natural settings and create personally relevant meaning to the experience (Behrendt & Franklin, 2014). Furthermore, the complex nature of field trips encompasses a broader scope of learning experiences that go beyond cognitive processing and tap into affective and psychomotor domains. These stimulating environments can spark curiosity, develop appreciation, refine personal interests, and support identity development (Davidson et al., 2010).
Cultural field trips may produce significant benefits for students. Researchers suspect that being in the presence of real-time performers immerses students into a larger world that broadens their curiosity, perception, and acceptance (J. P. Greene et al., 2014, 2018). Even though field trips can be costly, time-sensitive, and challenging to coordinate, a single visit to an art museum may help students ask complex questions about museum content, accept multiple interpretations of artwork, think critically about material properties, and recall emotional reactions (Randi Korn & Associates Inc., 2018). School-based field trips tend to be the first encounter students have with museums and performances, and may become a gateway experience. These first encounters can be defining developmental moments that cultivate students’ future arts participation (McCarthy & Jinnett, 2001), particularly for students from disadvantaged backgrounds (high-poverty and rural areas).
There exist gaps in the literature on field trips which need to be addressed by researchers. Despite the promising field of researching informal learning experiences on students’ trips to different sites, studies done about school-organized trips to performing art centers are rare. The most studied sites for field trips include museums and galleries (e.g., Griffin, 2004; Kisiel, 2006; Wolins et al., 1992), zoos or wildland areas (e.g., Bixler et al., 1994; Davidson et al., 2010), and parks (e.g., Farmer et al., 2007; Nielsen et al., 2009). For student trips to performing art centers, research typically exists on the level of community extracurricular activities (e.g., Eccles & Barber, 1999), rather than school-organized performances. In addition, there is not yet enough focus on studying field trip experiences of students who attend Title I schools, students with disabilities, or students who are English learners. The present study provides initial context for addressing these more complex questions in the future.
Engagement
Student engagement is predominantly conceptualized as a typology of behavioral, cognitive, and emotional/affective dimensions (Appleton et al., 2008; Fredricks et al., 2004). Behavioral engagement can be broadly defined as participation and refers to student involvement in academic, social, or extracurricular activities. It also includes effort made to pay attention to a specific activity (de Vreede et al., 2019). Pendakur et al. (2020), argue that engagement is not only behavioral participation, but also requires students to understand concepts and experience emotions. Cognitive engagement typically focuses on students’ investments in the process of various academic tasks such as thinking, reasoning, and comprehension (Fredricks et al., 2004; Lawson & Lawson, 2013). Cognitive engagement includes connecting prior knowledge to new knowledge (B. A. Greene, 2015). It describes the ways in which students think deeply about ideas and concepts, how they make meaning of the material presented to them, and how they use self-regulating and metacognitive strategies to master academic content (Cleary & Zimmermann, 2012; Lam et al., 2012). Emotional engagement focuses on students’ range of attitudes or emotions in response to people, objects, and places. It includes students’ levels of interest, enjoyment, happiness, boredom, and anxiety during academic activity (Appleton et al., 2008; de Vreede et al., 2019; Pekrun & Linnenbrink-Garcia, 2012). The present study draws on each of the types of student engagement.
Contemporary research is highlighting the impact of emotions on student engagement by examining how different emotions shape engagement and the various ways students can manage their emotions to further their engagement (Pekrun & Linnenbrink-Garcia, 2012). Similarly, engagement is related to the personal connections students make with the arts performance. Lamborn et al. (1992) argue activities that supply real-world connections to students have been identified as significant contributors to student engagement. In today’s increasingly diverse society, students from different social and cultural backgrounds bring to school vastly different identities and perspectives, challenging their educators to find more effective ways to connect with their lives and personal experiences. Lawson and Lawson (2013) emphasize cultural relevance as a significant cognitive and affective indicator of student engagement. Cultural relevance refers to students’ emotional and cognitive reactions when an academic activity holds personal significance (Guthrie et al., 2012). Based on these insights from the literature, the degree of personal significance found within arts performances for students may impact their level of engagement and learning, as well as predict future arts participation. And beyond this, it may contribute to their positive development. Although the scope of this study does not include the examination of the broad positive developmental outcomes that may occur in the life of these young ones (last level of the model), there is research evidence that even the attendance to a single performance may impact these developmental outcomes, highlighting the importance of the exposure to the arts and audience/student engagement in performances. In a mixed-methods case study with middle school students who viewed a professional performance for young people, Omasta (2011) argued that viewing a single Theatre for Young Audiences production might affect the attitudes, values, and/or beliefs of adolescent spectators. Results suggested that it was highly likely that under certain circumstances viewing a single theatre production could influence the values of adolescent spectators. This could well be one of the gateway experiences described earlier.
Variables of Student Arts Engagement
Prior knowledge and experience
Identifying and activating students’ prior knowledge may foster engagement. Research in cognitive engagement and motivation suggests that students’ conscious decisions about whether to engage in instructional content often depends on their sense of control and ability to do the work (Headden & McKay, 2015). Characteristics of environments that create a balance between behavioral, cognitive, and affective engagement with regard to prior knowledge include students clearly identifying and activating their prior knowledge, as well as linking their prior knowledge to new learning (Almarode & Miller, 2013).
Martin et al. (2012) have reported increased theater and music performances attendance and participation by high-school students related to each of the following: (a) their current performing arts study, (b) prior drama and theater participation, (c) past attendance to school-based performing arts, (d) past involvement in drama class or group, and (e) family attendance to performing arts events (this last one being also associated to increased dance performances).
In this study, two of the variables measure prior experience and knowledge through items about lesson taking in the art form, and past experiences at similar performances.
Preparation
Behrendt and Franklin (2014) state that when organizing field trips, it is important to include an aspect of planning or preparation and student reflection afterwards to maximize the learning experience. In a study examining the effects of field trip pre-visit logistical and subject matter content preparation on elements of positive youth development and student outcomes, Lee et al. (2020) found that both logistical and subject matter pre-visit preparation were associated with more positive student outcomes (such as interest in learning, problem-solving skills, collaboration, identity, self-efficacy, school motivation, etc.). Pre-visit preparation can help students connect what they are learning or the experience they are acquiring on the field trip to prior experiences in the classroom, in a constructivist learning manner by fitting together new information with the information they have already assimilated and developing new knowledge (Bada, 2015; Lee et al., 2020).
Specifically, regarding performance experiences, Brown and Ratzkin (2011) argue that pre-performance preparation plays an important role in contextualizing audiences to build up their anticipation, increasing their engagement in the upcoming performance. Preparation materials that invite students to acquire prior knowledge about a particular arts performance before attending the show may optimize the experience, especially if the preparation materials are personally relevant to students.
Preparation is measured in this study by asking about whether discussions of the music, play, or dance occurred before the performance.
Considerations for Priority Populations
For students who are English learners, culturally relevant topics that are connected to their personal backgrounds, as well as culturally responsive teaching methods in class can significantly increase and transform their level of engagement (Feger, 2006; Gay, 2010). While there is evidence of non-English speaking students (especially Hispanics) being markedly less likely to participate in extracurricular and leisure activities (Darling, 2005) and, according to Martin et al. (2012), English-speaking students being more likely to attend theater and music performances (but not necessarily dance performances), some studies have found that English learner youth may benefit more than non-English learners from extracurricular activity (Randall & Bohnert, 2009). In a similar finding, Greenfader et al. (2015) concluded that when exposed to several sessions of drama and creative movement ELL students with lower English language speaking skills improved their oral language skills, however, higher language skilled students did not improve as much.
The same seems to be true for socioeconomically disadvantaged students, who are less likely to attend theater and music performances (Martin et al., 2012) however benefit as much or more than advantaged students (Marsh & Kleitman, 2002) in outcomes such as school grades, coursework selection, homework, educational and occupational aspirations, self-esteem, number of university applications, subsequent college enrollment, and highest educational level. For students who live in poverty, the profound impact of repetitive stressors (e.g., violence, grief, loss, instability, and hardship) that manifest from living in poverty can significantly impact the way students learn. Despite these circumstances, children’s brains are malleable and when they feel safe in a nurturing environment at school, they tend to have better academic outcomes (Cantor et al., 2019). J. P. Greene et al. (2014), argue that culturally enriching experiences for students living in poverty are important for their “critical thinking skills, historical empathy, tolerance, and becoming art consumers” (p. 85) compared to advantaged students. Another aspect to consider is the one proposed by Coppieters (1981, cited in Bennett, 1991) who concluded that a sense of community within an audience prior to the performance affected each audience member’s ability to engage and make meaning out of a theater performance. In other words, the audience is more likely to be affected by the performance if they feel being a part of the social makeup of the audience.
For students with disabilities, the literature provides a myriad of suggestions to motivate and engage students, including but not limited to, giving students choices, and providing activities that are relevant to students’ life experiences and interests (Weiser, 2014). Music-making experiences, according to Darrow (2014), can provide opportunities to practice important life skills that will benefit students’ social and emotional development. Furthermore, music engagement is said to contribute to a student’s well-being by promoting positive emotions, engagement with others, achievement, and self-awareness. Children who participate in music have a better quality of life and a deeper understanding of the world around them, including those students with disabilities. In a conclusion that greatly resonates with the positive youth development framework, Darrow (2014, p.31) states that:
When music educators structure their curriculum to enhance students’ affective education, they also increase the likelihood that these students will have a smoother and more successful transition to adulthood. Music education curriculum objectives for students with disabilities might wisely be focused on their emotional well-being.
Context
Over the 3-year time-span of the study, the urban performing arts center in which the study was conducted served 180,683 student audience members and their teachers with a range of day performances across multiple art forms offered to schools through the education division of the performing arts center. The education division plays various roles in commissioning, scheduling, recruiting schools, and designing educational supports and activities for these performances for young people. The season is a series of performances designed for young audiences across the art forms of theater, music, and dance. Each performance has a suggested grade-level band for the developmental level of the student audience. Each production also has a different development process and set of educational activities depending on the art form and program. Each registered school is sent a packet of paper performance guides for teachers and students to prepare for the performance experience.
Because school field trips may not be frequent due to resource limitations and scheduling, community arts and cultural partners need to design such “gateway” arts experiences to make the most out of these arts participation opportunities. To inform this design, the research team developed and administered a survey instrument to measure student engagement related to field trips to performances to be collected over three performance seasons. This paper examines student engagement in relation to prior experience in the art form, instruction in the art form, and preparation for the field trip, as factors that may play a part in the positive development of young audiences through engagement with the arts. The analyses are disaggregated by varied student populations so that we can predict what may work for whom under what conditions. The findings will be used to better understand what factors may optimize student arts engagement, and then to intentionally design performances for meaningful participation in learning across the whole experience for a diverse range of young people, in the context of positive youth development theory. Hopefully this data-informed investment may increase the likelihood of any of these high-quality arts experiences to be a “gateway” to further arts participation and, indirectly, to the positive development of youth.
Research questions
For this paper, we focused on the engagement scores across student populations, and then on what factors may account for some of the variance in those scores:
How much were students engaged in performances for young people by student populations?
How much does engagement vary within the student populations according to the arts background aspects of lesson taking in an art form, having prior experience attending a performance, or preparation?
Methods
Participants
A population size was drawn from participation data from the 2015 to 2016 season. In 2016 to 2017, a student survey was administered to a probability sample of students to get a balanced sample across art forms (music, theater, and dance), and grade-bands (elementary, middle, and high). Depending on the mode of administration, many instances required additional convenience sampling based on a set of criteria applied to the school list for the targeted performance date (i.e., match of grade-band; public/public charter; locale; Title I status; manageable group size). When performances were canceled or schools did not show up, performance dates were added to the sample until we approximated our targeted sample size. In 2017 to 2018 and 2018 to 2019, specific main offerings were selected in each art form and the same convenience sample strategy was applied.
Cross-sectional data was collected from 2,120 participants over three consecutive school years (2016–2017, 2017–2018, and 2018–2019). Participants were students between third and twelfth grade in the grade bands of elementary school (n = 892, 43.64%), middle school (n = 736, 36.01%), and high school (n = 416, 20.35%). Students were recruited from public schools in the districts within a school day round trip driving distance of the performing arts center located in the mid-Atlantic region of the United States. IntegReview of Austin, TX, an Independent Review Board, approved the study [2016-3.2] and informed consent was waived except for identified students with disabilities who required a written parental informed consent and a written student assent prior to the survey administration.
Three priority populations of students were disaggregated to examine variability in engagement, students who attended a Title I school (n = 1,030, 50.2%; NCES School Title I Eligibility). English learners (EL; n = 319, 15.6%; self-identified), and students with disabilities (SWD; n = 112, 5.8%; recruited; see Table 1.) There were also cases of co-occurrence of priority populations: 215 (10.9%) of a total of 1,973 participants were both from Title I schools and English learners; 22 (1.1%) out of 1,918 participants were English learners with a disability; 14 (0.8%) of 1,879 participants who were from Title I schools and had a disability; and 1 (0.1%) of 2,120 participants was from a Title I school and was an English learner with a disability. Data missingness was below 4% for all main variables except for the count of students with disabilities variable, which was 8.6%.
Demographics of Participants per Populations.
Note. N = 2,120. Co-occurrence of populations (EL × Title I, n = 215 (10.9%); EL × SWD, n = 22 (1.1%); SWD × Title I, n = 14 (0.7%)).
Missing data (n = 66).
Data gathered from https://nces.ed.gov/ccd/schoolsearch/
To include these populations, several strategies were employed. Students with disabilities were intentionally recruited from designated public and private schools that serve special populations. On a few occasions, pre-registered classes were identified as coming from these schools and coded in the dataset. English learners were identified through self-report on the survey asking them if they were learning English as an additional language. Students who may be living in poverty were coded using the Title I status of the school that they attended because we did not have student level data like FARM. School participation data was collected from school registration forms.
The study was designed to disaggregate data for these priority populations. These student groups were designated as students who are living in poverty and are served by schools with high concentrations of students living in poverty, English learners, and students with disabilities. In addition to being influenced by the data collection efforts of the United States Department of Education, Office of Civil Rights, the study design followed recommendations of the Universal Design for Evaluation Checklist (Sullivan Sulewski & Gothberg, 2013), and The Arts and Special Education: A Map for Research (Burnaford et al., 2017) for including diverse respondents.
All the data was gathered into an omnibus dataset, checking to make sure that there were no significant differences across the 3 years in terms of sampling and administration. The large and varied population across the different program art forms, types, and educational activities, makes for a large sample size with a lot of variability to explore. And by design, we also disaggregated multiple priority population groups to understand what factors may work better for whom.
Measurement
The student survey consisted of 11 items with a 6-item engagement subscale designed by the research team. The other items in the student survey measured students’ arts background and experience.
Engagement survey
The engagement scale was conceptually based on constructs of emotional, cognitive, and behavioral engagement, and examined students’ engagement in relation to their present-moment experience viewing of a live performance. Items designed to reflect each construct varied specifically by art form, the examples given below are related to a music performance, and each had four response options:
Emotional engagement:
a. “After seeing the music performance or rehearsal today, how much did you like it?” (1 = I really didn’t like it, 2 = I didn’t like it, 3 = I liked it, and 4 = I loved it) b. “Would you tell a friend to come to this music performance?” (1 = No, 2 = Probably not, 3 = Probably yes, and 4 = Yes, for sure)
Cognitive engagement:
a. “Did any piece of music you heard remind you of music you have heard before?” (1 = Not at all, 2 = No, not really, 3 = Yes, sort of, and 4 = Yes, for sure) b. “Was the trip to the performing arts center what you expected?” (1 = Not at all, 2 = No, not really, 3 = Yes, sort of, and 4 = Yes, for sure)
Behavioral engagement:
a. “How much were you paying attention to the music performance or rehearsal while it was happening?” (1 = Less than half of the time, 2 = About half of the time, 3 = Most of the time, and 4 = All the time) b. “Do you listen to music like this?” (1 = Not at all, 2 = No, not really, 3 = Yes, sort of, and 4 = Yes, for sure)
Performing arts engagement variables
As mentioned above, the additional questions of the student survey explored different background aspects of exposure to and participation in the performing arts. These included:
Prior experience attending a live performance: “Have you ever been to a show in a theater/concert hall?” (0 = No and 1 = Yes).
Students’ preparation in their classrooms before attending the show: “Did you and your class talk about the show before coming to the performing arts center today?” (1 = Not at all, 2 = No, not really, 3 = Yes, sort of, and 4 = Yes, for sure).
Students having taken lessons in acting, music, or dance: “Have you ever taken acting/music/dance lessons?” (1 = Never, 2 = In the past, but not now, and 3 = Yes, and I still do).
Survey administration
Survey data was gathered using a range of procedures across the 3 years including digital tablets in theater, paper surveys in school, paper surveys in theater, and on-your-own-device using a QR code/URL. Students had the choice of responding to the survey items in English or Spanish. Secondary data was collected from the performing arts center’s school performance brochure, registration data, and the National Center for Education Statistics (NCES) at the school-level.
To support all learners in responding accurately to the survey, efforts were made to make the administration flexible, and the content understandable to a range of students. The survey questions were assessed at a readability level for grade 4 (Flesch-Kincaid Grade Level 4.0) with English learners in mind (IELTS Level 4–5). Students were given the option to respond to the survey in English or Spanish. For students with disabilities, the performing arts center’s accessibility office liaised with the accompanying teachers to remove any physical or mobility barriers, and the Research and Evaluation team provided extra time, multiple options, and supports to respond to the survey including: paper or digital surveys using tablet computers; braille, text to speech software with headsets; and proctored administration with a data collector. One group opted to respond digitally to the survey back at the school site where the assistive technology was familiar to the students.
Analysis plan
Statistical analyses were conducted in two stages: data preparation and preliminary analyses and association analyses.
Stage 1: Data Preparation, Psychometrics, and Preliminary Analyses—This stage included preparing data for analysis, examination of missing data, factor analysis, reliability testing, descriptive analyses, and checks for normality.
Data preparation: Prior to beginning analyses, data was cleaned and consolidated into an omnibus dataset, including all years of data administration. Items were checked for the need of reverse coding and aggregated to obtain total scores for analysis. Aggregation was achieved by averaging item responses to the scale or subscale level, where applicable.
Missing data: Data was checked for missingness. Since data missingness did not exceed the 5% recommended threshold, there was no need to examine for patterns of missingness or any systematic associations between demographics or study variables and missingness.
Factor analysis: An exploratory factor analysis was conducted to understand the structural validity of the engagement scale. Once the initial analysis and resulting modifications to the scale are done, we performed a confirmatory factor analysis, with the a priori model including a single factor of audience engagement.
Reliability: Internal consistency reliability of the engagement scale was assessed using Cronbach’s alpha. Values for Cronbach’s alpha typically range from 0 to 1 with values at or above .70 indicating an acceptable level of internal consistency (Bland & Altman, 1997).
Descriptives: Basic descriptives (e.g., means, standard deviations, ranges, etc.) were assessed for all study variables, both as an overall sample and separated by priority population.
Normality: The assumption of normality was checked through visual examination of the distribution of the engagement score variable, and through examination of skewness and kurtosis values.
Stage 2: Association Analyses—Statistical analysis of association and prediction of study variables was accomplished through analysis of variance (ANOVA) and linear regression models.
The analysis of variance allowed us to compare the engagement scores of those students who had previously attended a performance and those who had not. The arts participation background item that captured whether a student had previously attended a performance was dichotomous and functioned as the independent variable. A composite variable had been created representing a mean of the items measuring engagement, which in this case was the dependent variable. We conducted these analyses for the overall population and for each priority population.
Linear regression analyses were used in a similar way, to predict engagement from the different levels of arts participation through lesson-taking in a performing art form and preparation prior to the performance. These analyses were run for the overall population and within each priority population.
Factor analysis
In order to validate the engagement scale psychometrically, factor analysis was performed with the original six questions and properties of reliability were examined. Before conducting a factor analysis, data was screened for missingness. About 65 cases out of 2,120 participants were found to be missing audience engagement data, which represented approximately 3% of all participants and was within the acceptable range of missingness for data to be considered missing at random. A final sample size of 2,055 participants was used for factor analysis, with a ratio of 342 cases per variable.
As a first approach, the research analyst conducted an exploratory factor analysis. As previously mentioned, the small number of items were conceptually designed to reflect three types of engagement (emotional, cognitive, and behavioral), however, the factor analysis indicated the engagement scale was unidimensional, reflective of a single general engagement factor. The factorizability of the six engagement items was examined through the criteria of Kaiser-Meier-Olkin test of sampling adequacy, which yielded a value of 0.781, well above the generally used threshold of 0.600, and close to the recommended value of 0.800, and Bartlett’s test of sphericity was significant (χ2 (15) = 2,290.903, p < .001). The diagonal values of the anti-image correlation matrix were all between .734 and .855. Given all of these criteria, factor analysis was considered to be suitable with the six items, for which both scree plot and Eigen-value criteria indicated that the scale had a single factor. This factor alone explained 42% of the variance in audience engagement.
After this initial analysis, one item (“Was the trip to the performing arts center what you expected?”) was eliminated because of doubts about its validity (the item seemed to be measuring something different than engagement, possibly satisfaction with the field trip), which was also reflected in the factor loading, substantially lower than the rest. In addition, this item had a positive skew effect, with 74% of participants reporting the trip to have been what they expected.
For the final stage, a confirmatory factor analysis of the remaining five items was conducted, with the model postulated a priori being that of a single factor of audience engagement. The goodness-of-fit criteria used in this case were Chi-square divided by the degrees of freedom (CMIN/DF; <2 or 3) root mean square error of approximation (RMSEA; <0.06–0.08), comparative fit index (CFI; >0.95), standardized root mean square error residual (SRMR; <0.08), and Tucker–Lewis Index (TLI; >0.95) with cut-off indices recommended by Schreiber et al. (2006). The resulting values were CMIN/DF = 11.239; RMSEA = 0.072; CFI = 0.974; SRMR = 0.035, and TLI = 0.947. Except for the proportion of Chi-square value and degrees of freedom, all the indices fit adequately, indicating that the hypothesized single factor model represented the best fit to the data. Factor loadings ranged from 0.389 to 0.618.
Internal consistency of the scale was examined using Cronbach’s alpha, which was acceptable (α = .703). A global score of engagement was created based on the mean of the items of this scale, with a range of 1 to 4. Higher scores indicated greater audience engagement with the performance. Descriptive analyses of this global score indicated the average was 2.997 in a scale that ranged from 1 to 4, with a standard deviation of 0.602 The skewness (−0.653) and kurtosis (0.294) were well within an acceptable range for the assumption of a normal distribution, and the examination of the histogram suggested that the distribution appeared normal.
Overall, these analyses indicated the unidimensionality of this audience engagement scale, which has shown moderate internal consistency. One of the original six items was eliminated, however the original factor structure of a single factor was maintained. A normal distribution was observed for the global score, thus the data were well suited for further parametric statistical analyses.
Results
The first research question asks how engagement varied by student populations. In order to examine this, we conducted several univariate analyses of variance (ANOVA) which did not show any significant differences in engagement derived from Title I status or being an English Language Learner on engagement. There was a significant difference in engagement depending on whether the student had a disability or not (F (1, 1,929) = 13.407; p = .000). Students with a disability appeared to be less engaged (M = 2.805, SD = 0.548, n = 109) than participants with no disability (M = 3.019, SD = 0.596, n = 1,822).
The second research question asks how the three aspects of performing arts background may explain or predict some of the variance in engagement scores. Statistical procedures followed for this purpose were analysis of variance in the case of previous experience attending performances, and regression analyses in the case of preparation prior to the performance and experience taking lessons in any form of performing arts. In Tables 2 and 3, for the overall population audience engagement was significantly and positively predicted by preparation prior to the performance (F (1, 2,033) = 91.942, p = .000; R2 = .043), although explaining only a small proportion (4%) of variance in engagement. Similarly, a 6% of the variance in student engagement (F (1, 2,035) = 130.084, p = .000; R2 = .060) was positively and significantly predicted by having taken prior lessons. Finally, there was a significant difference between those who had attended a prior performance and those who had not (F (1, 2,039) = 9.749; p = .002). Those who had been to a show reached a higher score on the engagement measure.
Descriptive Statistics for Population Groups.
Note. N = 2,120.
n = 2,055.
Analysis of Variance (ANOVA) and Regression of Study Variables on Engagement.
Note. n.a. = not applicable.
p < .05.
Although statistically significant results were found for all three factors, we wanted to see how these might hold or vary across our three priority population groups. For students attending a Title I school there was a significant and positive prediction of engagement by having prepared for the performance (F (1, 1,016) = 44.643, p = .000; R2 = .042; Table 3), with those who prepared more before the performance scoring higher in engagement, accounting for a 4% of variance in engagement. The prediction of having taken lessons over engagement was also significant for this population (F (1, 1,016) = 55.668, p = .000; R2 = .052), accounting for a variance of 5%, and in the same direction as the other populations. Having previously attended a performance or not made a significant difference on engagement for students from Title I schools (F (1, 1,017) = 5.501; p = .019). Having previously attended a show or performance was associated with greater engagement.
For the English learner (EL) population there was a significant and positive prediction of engagement (F (1, 313) = 26.375, p = .000; R2 = .078) by preparing more before seeing a performance, accounting for close to 8% of variance. Having taken previous lessons also significantly predicted a greater engagement for EL students (F (1, 313) = 19.564, p = .000; R2 = .059), and explained 6% of variance. Having previously attended a performance did not seem to matter for the audience engagement of EL students.
For students with disabilities there was a demonstrated statistically significant (F (1, 106) = 23.189; p = .000; R2 = .179) and positive association between engagement and prior preparation. Prior discussion and preparation accounted for 18% of variance in engagement scores. For students with disabilities, it appears that the engagement in the performance comes almost entirely from preparing prior to the performance. The effect of prior experience or lessons was not significant.
Discussion
Implications, Limitations, and Further Research
This study examined the engagement of students during a live performance associated with sociodemographic characteristics, and prior arts participation and exposure, under the Positive Youth Development framework. For this purpose, we used student audience engagement as a measure across a range of art forms, productions, and performances, as well as a range of student grade-levels and priority populations. Through ANOVAs and regression analyses we aimed to identify the performing arts background aspects that had a statistically significant effect on engagement for young audiences. In this discussion we investigate these aspects that predict and explain some of the variance in engagement and we disaggregate the findings by population groups to get insights into what may work better for whom. We discuss implications and limitations of the findings and suggest further research pathways.
Consistent with literature (Almarode & Miller, 2013; Brown & Ratzkin, 2011; Martin et al., 2012), we found the fact of having prior experience at a performance, taking lessons in the art form, and previous preparation each contributed to higher engagement for the overall sample population. In other words, the more interaction students experienced with the art form prior to attending the performance, the higher their engagement level. Going beyond this finding, a unique contribution of this study was to show which factors may work differently for which student population when the data was disaggregated. Another way to look at this is the following “what works for whom” summary:
Overall
It seems all three aspects of prior interaction with the art form played a significant role in the level of engagement of students. Higher student engagement was found for students who had prior experience attending performances, had previously taken lessons in a relevant art form, and received preparation prior to attending the performance, with each separately contributing to higher engagement at the time of the performance in the performing arts center. This is consistent with findings from Lee et al. (2020) in which pre-visit preparation was associated with positive student outcomes, some of which are considered forms of student engagement (i.e., interest in learning). Brown and Ratzkin (2011) have suggested that audience members will take more out of a theatrical experience if they have access to some background and contextualization. These conditions also seem to apply to the association between engagement and previous exposure and participation in the arts, as suggested by Martin et al. (2012). In their study, they found that students will attend more music and theater performances if they have had a greater exposure to the performing arts by taking classes in performing arts or attending performances. In Martin et al. (2012), performing arts attendance was taken as an indicator of arts participation, which is considered a form of arts engagement.
Students from a Title I School
Consistent with the overall finding, all three factors of interaction with the art form were associated with a greater engagement for students from Title I schools. Again, these factors are having experience attending performances, having previously taken lessons in a relevant art form, and having prepared prior to the performance. This result supports those findings indicating that students in socioeconomic disadvantage might benefit more than their advantaged counterparts from participating in the arts (J. P. Greene et al., 2014; Marsh & Kleitman, 2002).
English learners
For students who are English learners the fact of having previously taken lessons in a relevant art form and having prepared prior to the performance predicted greater engagement during the performance. Previous research (Greenfader et al., 2015; Randall & Bohnert, 2009) has suggested that English learners benefit greatly from extra-curricular, leisure activities, and arts performances, however they might be limited in accessing such activities and benefits, as reported in literature Darling, 2005; Martin et al., 2012). Interestingly, our results did not evidence previous attendance to performances as a relevant factor in audience engagement for ELL population, however, they did highlight the importance of preparation prior to a performance as a means of ensuring greater engagement during the performance. This is an encouraging result since preparation prior to the performance is not necessarily a costly or challenging way of enhancing the arts engagement and participation experience. Furthermore, our results also support the value of school-based (and extra-curricular) performing arts lessons, since this might be a unique opportunity for ELL students to take the most advantage out of a performance experience and engage more with the arts. This finding is relevant in informing policy makers looking to support arts education for all student populations and age levels.
Students with disabilities
For students with disabilities, the only aspect that seemed to relate to greater engagement came entirely from preparing prior to the performance. It is important to note that this is a low-cost approach that could make a big impact, not only for this population, but for all students. The benefits of engaging with the arts for students with disabilities have been well documented (Adamek & Darrow, 2012; Darrow, 2014).
Although the overall findings indicate prior experience of attending performances as an important factor for engagement, this seems to matter less for two of the priority populations. Taking lessons in the relevant art form seems to make a difference for English learners and for students from Title I schools.
Although the provision of arts lessons and the planning of out of school field trips are not the responsibility of cultural organizations, an implication for these findings could be to build relationships and strategically recruit schools. Further research is required to examine the importance of collaboration with schools in terms of recruitment of populations/grade-levels at strategic time-points (e.g., inviting fourth graders to an educational orchestral concert during the year that they choose to begin instrumental music lessons), and an alignment of in-school and out-of-school instructional goals, activities, and materials (e.g., reading and analyzing a text in school that has been turned into a theatrical play). This study did not collect in-school preparation data that could be accurately aligned with the student level data. A future study could be designed to match student level demographic and response data (i.e., emotional, cognitive, and behavioral engagement; learning assessments) to lesson taking and preparation activity types in and out of schools.
We do not know from the study data whether students with disabilities attend out of school performances more or less often than other students. We also do not know from the study data whether arts lessons are available to students with disabilities to the same degree as other students. Regardless of the availability of the arts lessons, we know from the study findings that preparation is an important factor for students with disabilities. In addition, preparation is also a factor that contributes to the engagement of all the other populations. We also know that students with disabilities are less positively engaged than their counterparts, suggesting that preparation strategies to increase engagement are critical to develop further. Designing and testing preparation materials for supporting students with disabilities would be a smart investment and may be helpful to other population groups. Enhanced performance guides based on good learning design may be a productive strategy here (e.g., clear learning goals, connections to background knowledge, culturally relevant examples, vocabulary/language supports, multi-modal representations of content, information chunking, alternative text, text-to-speech, captioning, and audio description; Glass et al., 2013; Meyer et al., 2014). There also may be other more robust strategies for preparing young people for a performance and the field trip experience (e.g., classroom pre-visits, videos with artists, social stories, and other emotional self-regulation strategies to manage anxiety, and provide expectations for a safe, non-distracting environment, etc.)
There are a few study limitations that should be considered when interpreting the data and findings. The sample was not randomized, and student groups were selected at the school level using a set of criteria and by convenience. Because we did not have access to student level demographic data, we relied on the Title I designation of the school as a proxy to identify students living in poverty, a self-report question to identify possible English learners, and direct recruitment strategy to include students with disabilities.
Specifically, regarding the lack of individual student data, we could not further categorize students with disabilities as having mobility or sensory disabilities that may require a 504 plan, nor do we know how many of the students have an intellectual or learning disability that would require an IEP. Students may also have both a 504 plan and an IEP. Each of these sub-groups may vary in engagement and the possible supports for improving that engagement. In addition, there may be students with disabilities represented in the overall sample that were not identified in the dataset. The national percentage of enrolled public-school students who receive services under the Individuals with Disabilities Education Act (IDEA) is 14% (National Center for Education Statistics, 2019), but we do not know how likely it is for students with disabilities to participate in field trips compared to the overall student population. A future research study should consider randomization and negotiating access with the district’s central office for student level data, and more intensive recruitment of schools that specialize in serving students with disabilities. An even larger sample size would allow for deeper analysis of co-occurrence and intersectionality across population groups. Also, we recommend the inclusion of student level demographic data to examine variability in engagement related to age and gender.
Costantoura (2000) has suggested that when it comes to arts engagement, voluntary participation in the arts is a better predictor or clearer indicator than school-based involvement with the arts. Future research should ask about the nature of previous experience and participation in the arts.
Conclusion
The results of this study contribute to a better understanding of the arts background factors that make an impact on arts engagement taking into consideration how this association varies within each priority population. The findings agree with prior research on the sociodemographic and arts background effects on engagement and support the theoretical framework of positive youth development. This is a correlational study that has examined the association of several background aspects with the engagement of students in out of school arts performances. The findings can be used to look at what factors may predict the likelihood of a young person being more positively engaged by high quality arts performances regardless of the art form. To get to more nuanced predictions, we also disaggregated the data by the priority populations of students who may be living in poverty, English learners, and students with disabilities. By doing so, we were able to identify the most critical factor that seems to matter for students with disabilities–preparation, which also matters for the other populations.
The study contributes a brief manageable survey instrument in English and Spanish with good readability statistics for diverse learners, confirmed validity for measuring engagement, and an acceptable level of reliability for consistent data collection. This paper also provides justification and strategies for including priority populations in the design of instruments, survey administration, and data analysis. Considering that preparation is the only positive engagement strategy for students with disabilities, and that this population is currently less positively engaged than the other population groups, it indicates a growth area for expanding pre-visit resources and activities, as well as considering differentiation strategies and accessibility supports to address variation in diverse learners. A solid investment in these resources that may be required for students with disabilities, may also be useful for all audiences to optimize potential “gateway experiences” to future arts participation. Or as universal design advocates say, if supports and options are, “essential for some, and useful for all,” then why not design for this diversity?
Footnotes
Acknowledgements
John F. Kennedy Center for the Performing Arts Research and Evaluation with contributions from Bina Ali, Stephanie McKeel, Gail Burnaford, Muna Shami, Reba Troxler, Charles Beekman, and Sarah Koo. Dr. Don Glass (deceased) was an artist, learning designer, and developmental evaluator at the Kennedy Center. He left an indelible mark on the field of arts education through his work focusing on the integration of inclusive arts curriculum design and developmental evaluation strategies into the ongoing professional development of educators. He was especially passionate about equitable access to arts learning opportunities. This article is dedicated to his memory and to his surviving wife and two children.
Author Note
The content of this paper does not necessarily represent the policy of the U.S. Department of Education. You should not assume endorsement by the federal government.
Data Availability Statement
The de-identified data that support the findings of this study are available upon reasonable request.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the U.S. Department of Education under grant number: U351F150001.
IRB Protocol Numbers and Name of Research Study
Student Engagement in Kennedy Center Performances, Events and Tours. IntegReview IRB, Austin, TX [2016-3.2]
