Sage Journals: Discover world-class research

Abstract

This study examined how school and district leaders access, value, and use research. From a representative sample of school districts across the United States, we surveyed 733 school and district leaders as part of an effort to develop understanding of the prevalence of research use, the nature of leaders’ attitudes toward research, and individual and organizational correlates of research use. School and district leaders alike reported frequent use of research use and generally positive attitudes toward research. Leaders reported accessing research primarily through their professional networks. Those in certain roles, those pursuing or holding an advanced degree, and those who reported a strong organizational culture of evidence use reported higher levels of research use. These findings suggest that policy efforts to promote evidence use among education leaders will be welcomed but that policy makers need to take into account the prevalence of various types of research use in designing supports for evidence use.

Keywords

educational policy leadership research utilization survey research

Despite widespread attention to national and state reforms, educational policy making remains a largely local affair. School and district leaders not only interpret and respond to state and federal policies (Hightower, Knapp, Marsh, & McLaughlin, 2002; Meyer, Scott, & Strang, 1987) but also lead the design and implementation of instructional policies and programs (Burch & Spillane, 2005; Datnow & Honig, 2008; Hightower et al., 2002; Rorrer, Skrla, & Scheurich, 2008; Spillane, 1996; Spillane & Thompson, 1997). Central office leaders adopt or design initiatives to improve teaching and learning, create structures and processes to implement these initiatives at scale, and provide system-wide management, oversight, and leadership (Rorrer et al., 2008). It is at the local level, as led by district and school leaders, that educational improvement efforts take root.

At the same time, local education leaders face pressure from federal policies to find and use research evidence to inform their decision making. The Every Student Succeeds Act (Public Law 114-95) calls on leaders to select, adopt, and implement “evidence based” programs and strategies that have demonstrated improved student outcomes or the potential for improved student outcomes. This act follows nearly two decades of federal policy making focused on increasing leaders’ opportunities to access and use high-quality research evidence (Haskins & Margolis, 2015; Haskins, Paxson, & Brooks-Gunn, 2009). Major initiatives funded by the U.S. Department of Education (e.g., What Works Clearinghouse) likewise aim to support leaders’ use of research when making decisions about programs and practices to adopt or design.

At present, there is limited information about how often and where local education leaders access research and the purposes for which they use it. A review of studies of school and district leaders’ research use indicate that research is difficult for leaders to access, it is rarely used, and it is only sometimes consulted for decision making about program adoption (Coburn, Honig, & Stein, 2009). Even within longer-term research-practice partnerships—in which researchers work closely with educators to investigate and search for solutions to persistent educational problems—the level of research use varies within and across districts (Honig, Venkateswaran, McNeil, & Twitchell, 2014; Hubbard, 2010; Penuel, Farrell, Allen, Toyama, & Coburn, 2016).

A limitation of the scholarship to date is that we do not know the extent to which the findings generalize to the larger population of school and district leaders in the United States. Because prior studies have focused on particular contexts, the measures of research use employed cannot generalize to leaders across districts nor capture variation in individuals’ research use. If we want to understand the potential for federal policies such as Every Student Succeeds Act to influence research use, it is important to establish a baseline understanding of education leaders’ research use from a nationally representative sample, using measures designed to capture a broad range of potential uses of research.

This article presents results of a survey of research use administered to a nationally representative sample of school and district leaders. We address three questions:

Research Question 1: How frequently do school and district leaders report that they use research and for what purposes?

Research Question 2: Where do school and district leaders access research?

Research Question 3: What individual and organizational characteristics are associated with research use?

Conceptual Framework

We used findings from three streams of scholarship on research use in education to inform the design of our survey instrument. First, in designing queries about leaders’ purposes for research use, we followed the categories first identified by Weiss and Bucuvalas (1980) and then applied more recently in studies of research use by education leaders (Coburn et al., 2009; Coburn & Talbert, 2006; Coburn, Toure, & Yamashita, 2009; Farley-Ripple, 2012). These categories suggest that research use by leaders is multifaceted. Second, findings from empirical studies have suggested the importance of considering the ways in which education leaders access research and the types of research that they find useful (Coburn, 2010; Hubbard, 2010). Finally, our survey’s focus on user characteristics and the organizational conditions that support evidence use was motivated by recent research on the correlates of research use across a range of fields, including education (see Contandriopoulos, Lemire, Denis, & Tremblay, 2010, for a review). We explain each stream of research in detail.

The Multifaceted Nature of Research Use

When policy makers encourage education leaders to use research to inform their decision making, they implicitly invoke a theory of action in which evidence from research findings directly shapes decisions related to policy or practice (Johnson, 1999; Sharkey & Murnane, 2006; Weiss, 1980). This type of use is one that Weiss (1980) calls instrumental use, because it is in the service of a particular decision. In addition to instrumental use, Weiss and Bucuvalas (1980) identified three other ways in which research was commonly used: conceptual use, which occurs when research changes the way that a person views a problem or the possible solution spaces for a problem; symbolic (or political) use, when research is used to validate a preference for a particular decision or to justify a decision already made; and imposed use, when use is mandated by law or policy.

Weiss and Bucuvalas’s (1980) typology underscores that research use is not a single process, serving a single purpose. Studies of research use in education support this idea. Not only do educators use research directly to make decisions, but they also use ideas and tools from research to gain “conceptual handles” for diagnosing educational problems and designing solutions to them (Datnow & Park, 2010; Ikemoto & Honig, 2010; Penuel, Briggs, et al., 2016). In addition, leaders use research symbolically within contentious debates over local policies, where research findings are invoked to justify particular positions and discredit others (Asen, Gurke, Solomon, Conners, & Gumm, 2011). Though often contrasted unfavorably with instrumental use, Weiss and Buculavas (1980) have argued that research can be used symbolically in responsible ways, such as when it gives “decision makers confidence and [strengthens] the case that the research itself supports” (p. 11). In some cases, imposed use has been successful in reducing the use of popular policies and programs when evidence suggests that those policies do not work (see, e.g., Weiss, Murphy-Graham, & Birkeland, 2005).

Studies of research use in school districts suggest that among these types of use, instrumental use—the type most emphasized in policies that encourage use of research evidence—is relatively uncommon. For example, in a study of how central office leaders in 16 districts made decisions, only two of 14 decisions analyzed involved the use of research evidence to directly inform those decisions (Kennedy, 1982). Similarly, David’s (1981) analysis of how leaders made decisions regarding use of Title I funds in 15 districts indicated that only one quarter of decisions were based on use of evaluation data. Other studies found a similar pattern of limited instrumental use of research in decision making (Corcoran, Fuhrman, & Belcher, 2001; Weiss et al., 2005; see also Coburn et al., 2009, for a review).

On the basis of these findings, we conjectured the following:

Conjecture 1a: We expect to find evidence not only of leaders’ instrumental use of research emphasized by policies but also conceptual, symbolic, and imposed uses of research.

Conjecture 1b: We expect instrumental use of research to be less common than other forms of research use.

Access to Research and Types of Research Used

The extent to which leaders differ in their levels of access to research may help to explain why some leaders use research more than others as part of their decision-making processes. School and district leaders both report that they have limited access to research findings that are timely and that address their immediate needs and questions (Corcoran et al., 2001; David, 1981; West & Rhoton, 1994). Previous studies have found that connections to outside sources, such as university faculty, research intermediaries, consultants, and libraries, can facilitate increased access to research (Coburn, 2010; Hubbard, 2010) and, under some conditions, facilitate its use in decision making (Aarons, Wells, Zagursky, Fettes, & Palinkas, 2009; Bickel & Cooley, 1985; Hubbard, 2010). On the basis of past research, we conjectured the following:

Conjecture 2: We expect leaders’ access to original research articles to be limited and that leaders access research findings through other means.

Individual Characteristics Related to Research Use

Research suggests that several individual characteristics may be associated with frequency of research use: the willingness to expend the necessary effort to acquire research (Landry, Lamari, & Amara, 2003), attitudes toward research (Johnson et al., 2009), and some preparation (e.g., an advanced degree) to read and interpret research.

Acquisition effort and attitudes toward research

Landry et al. (2003) linked what they called “acquisition effort” to research use, finding that individuals who used research were more likely to make pointed efforts to acquire research relevant to particular problems and to establish relationships with researchers. Likewise, Johnson and colleagues (2009) found that people with more positive judgments about the quality of research with regard to its relevance, usefulness, and credibility or “trustworthiness” were more likely to use it. Because these studies found that individuals differ in taking initiative to acquire research and in the degree to which they see research as providing relevant, valuable, and credible information to guide their decision making, we expected that education leaders’ attitudes toward research and acquisition efforts would be related to their use of research. Specifically, we conjectured the following:

Conjecture 3: We expect leaders’ attitudes toward research to be mixed, with participants expressing a range of positive and negative views of education research.

Conjecture 4a: We expect acquisition effort and attitudes toward research to be correlated with research use.

Formal preparation to interpret research

Individuals’ ability to evaluate research designs and interpret conclusions from research may explain some of the variation in their research use. Although there is little research on the relationship between knowledge about research methods and research use, a number of studies of data use showed that school and district leaders’ skills in posing questions about and making sense of patterns in achievement data can either support or impede the use of data in decision making (Makar & Confrey, 2005; Means, Padilla, DeBarger, & Bakia, 2009). We hypothesized that use of research would likewise be related to preparation to interpret research, as indicated by advanced coursework in which students are exposed to statistics and to qualitative and quantitative research designs:

Conjecture 4b: We expect formal preparation to use research to be correlated with research use.

Organizational Correlates of Research Use

In addition to individual characteristics, research suggests that organizational conditions—such as leaders’ roles in their districts, regular opportunities for discussing research, and expectations around research use within district or department cultures—may be related to patterns of research use (Contandriopoulos et al., 2010; Honig, Copland, Rainey, Lorton, & Newton, 2010; Ikemoto & Honig, 2010).

District size and professional roles

Different roles yield different kinds of opportunities to use research for decision making and other purposes. Large districts, for example, have highly complex and departmentalized organizational structures. Decision making related to instruction is often stretched across multiple units in the central office and across levels of the system. District subunits have employees with different disciplinary backgrounds and connections to external sources of research (Spillane, 1998), which may result in attitudes toward research use that vary systematically by division and level (Coburn & Talbert, 2006).

Regular occasions for talking about research

When district-level staff regularly facilitate and support others in making sense of evidence, whether through book studies or discussions of research, they create supportive conditions for using research (Kochanek & Clifford, 2014). The presence and frequency of regular meetings can play an important role in influencing when and how evidence enters into decision-making deliberations (Corcoran et al., 2001; Ikemoto & Honig, 2010; Spillane, Parise, & Sherer, 2011). While there is limited research on how research use in meetings might enable or constrain individuals’ engagement with research, findings on data use provide a compelling argument for its potential import (Anderson, Leithwood, & Strauss, 2010; Halverson, Grigg, Prichett, & Thomas, 2007; Kerr, Marsh, Ikemoto, Darilek, & Barney, 2006; Little, 2012; Park & Datnow, 2009).

Culture of research use

A number of scholars posit that a “culture of research use” is an important condition for research use and evidence-based policy making at the local level. A culture of research use is one in which organization members value research as a resource for decision making (Helmsley-Brown & Sharp, 2003), select strategies based on evidence (Dwyer, Millett, & Payne, 2006), remain open to change in light of evidence (Gerrish & Clayton, 2004), and enact multiple social supports and norms promoting evidence use (Fitzsimmons & Cooper, 2012). From these studies of organizational correlates of research use, we conjectured the following:

Conjecture 5: We expect the organizational characteristics that we measure to be correlated with research use, including the frequency with which research is discussed in meetings and organizational cultures that promote use.

Table 1 summarizes the set of initial conjectures that we developed to guide our study, based in this conceptual framework. These conjectures were limited in number, in part because national studies of research use in education have not been conducted. Nonetheless, they provided us with a way to focus the analyses presented here.

Table 1

Conjectures Guiding Analysis of Research Questions

Conjecture 1a	We expected to find evidence not only of leaders’ instrumental use of research emphasized by policies but also conceptual, symbolic, and imposed uses of research.
Conjecture 1b	We expected instrumental uses of research to be less common than other forms of research.
Conjecture 2	We expected that leaders’ access to original research articles would be limited and that leaders would access research findings through other means.
Conjecture 3	We expected leaders’ attitudes toward research to be mixed, with participants expressing a range of positive and negative views of education research.
Conjecture 4a	We expected acquisition effort and attitudes toward research to be correlated with research use.
Conjecture 4b	We expected formal preparation to interpret research to be correlated with research use.
Conjecture 5	We also expected the organizational characteristics that we measured to be correlated with research use, including the frequency with which research is discussed in meetings and organizational cultures that promote use.

Methods

We designed this study to find out where local education leaders access research and what types of research they use, how often they use research for different purposes, how leaders perceive the value of research, and what individual and organizational characteristics are related to its use. To do so, we surveyed a nationally representative sample of school and district central office leaders from mid- and large-size U.S. school districts using sets of items that we developed and refined through multiple rounds of piloting. To investigate the multifaceted nature of research use (Research Question 1), we developed scales representing the frequency with which leaders engaged in different types of research use for different purposes. We also developed items related to where leaders accessed research (Research Question 2). Finally, we developed scales for the individual and organizational characteristics associated with research use that were identified in our conceptual framework (Research Question 3). In what follows, we describe the development process for survey items and scales, the target population and procedures for sampling, and the methods of data collection and analysis.

Survey Development

We began survey development by bringing together project staff, scholars, and education leaders with interest and expertise in the area of research use. We refined our survey constructs and developed items associated with each construct to test in two sets of cognitive interviews with a total of 40 education leaders. We transcribed responses, systematically analyzed issues relating to items, and revised items based on whole-team discussions. We also solicited expert feedback from two sets of advisors through a formal survey and discussion.

Next, we pilot tested the revised instrument with a convenience sample of 265 education leaders from our target population (described in the next section) who responded to our invitation to take the survey. The sample was obtained from a list curated by MDR, an educational market research firm. We used this pilot test to generate initial scale reliabilities and likely distribution of responses to specific items, as well as to identify additional issues with the survey content. On the basis of our analyses, we created additional items with the intent of improving the internal consistency of selected scales, made revisions to items to improve clarity, and shortened the survey. We also made decisions about which sets of items would be asked of all respondents and which sets of items would be asked of a randomly chosen subsample of respondents. Finally, we chose to include a definition of research (“an activity in which people employ systematic, empirical methods to answer a specific question”) on all pages of the final survey instrument to ensure that respondents had a standard frame of reference for a term that could otherwise be understood very differently across respondents.

Items related to type of use (Research Question 1)

To identify instrumental use of research, we asked whether respondents had been involved in any of eight educational decision-making activities (e.g., curriculum adoption, scaling up a pilot program, designing professional development). If respondents indicated that they were involved, they were asked how often they had used research as part of that activity. To elicit evidence of conceptual use, we asked all leaders about the extent to which research informed their ways of looking at or solving a problem in their school or district (depending on their role). The six-item scale included such questions as “How often have you encountered research that changed the way you look at problems facing your school/district?” and “How often have you encountered research that suggested alternative solutions to a school/district problem?” To elicit evidence of symbolic use, we used a four-item scale that asked respondents to report their engagement in activities, such as using research to mobilize support for important issues or selectively using research to support a decision. In addition, we captured imposed use through three items that asked respondents to indicate how often they were required to use research for purposes such as choosing curricula and justifying plans. For all items related to types of research use, response choices were never, sometimes, frequently, and all of the time.

Items related to where leaders access research and what research they use (Research Question 2)

To find out where leaders accessed research, we asked respondents to identify how often in the past 12 months they had consulted 14 possible sources, ranging from university researchers to peer networks (e.g., professional associations) and the media. Item response choices were never, rarely, sometimes, often, and all of the time. We also asked a random subsample of respondents if they had access to a university library (yes or no).

Variables related to individual characteristics (Research Question 3)

Variables related to individual characteristics included a five-item scale of acquisition effort that asked leaders to indicate how often they made an effort to acquire research or develop relationships with researchers. We also included three subscales of attitudes (a total of 20 Likert-style items) that gauged respondents’ agreement with statements about the relevance, value, and credibility of research as it related to their own work. Together, these variables and items allowed us to explore associations between frequency of research use and individual characteristics related to acquisition effort and attitudes. As an indicator of preparation to interpret research, we asked a random subsample of respondents to report whether they held an advanced degree or were working toward one.

Items related to organizational conditions (Research Question 3)

To look at associations between roles and types of use, we asked the respondents to indicate one of several categories that captured their professional role per our sampling frame (see categories in the next section), and we corroborated these with the categories of roles listed in MDR records. We operationalized regular occasions for discussing research by asking about the frequency with which respondents were involved in various types of meetings and how often research was brought up in those meetings. To examine whether participants were part of a school district/department with a strong culture of research use, we provided the following statements for their response: “Research is seen as a useful source of information,” and “We are genuinely encouraged to use research as part of our ongoing work.” Because of time constraints, a random subsample of 372 respondents received the four items related to culture of use.

Population and Sample

The target population of education leaders consisted of principals and central office leaders from mid- and large-sized U.S. urban districts who were likely to be involved in instructional decision making. We focused on principals and central office leaders because they make the majority of decisions regarding what programs and interventions to adopt in schools.

To focus on individuals with instructional decision-making responsibilities across district subunits and levels, we targeted the following seven categories of roles: deputy, associate, and network superintendents; curriculum supervisors; special education supervisors; accountability, assessment, and research coordinators; directors of federal, bilingual, and English-as-a-second-language programs; “multirole” central office leaders, classified in more than one of the aforementioned roles; and elementary, middle school, and K–8 principals. We chose K–8 principals because there is more research available on effective programs and interventions at these grade levels and because more variety exists in the curricular materials, assessments, and other instructional programs that districts may adopt. Because smaller districts may not staff many of the positions included in our sampling frame, we focused on the 1,000 largest U.S. school districts, which each served >9,000 students according to Common Core data from the National Center for Education Statistics.

We identified a set of >41,000 school and district leaders from a sampling frame purchased from MDR. Because there are far more schools than school districts, the majority of people in the sampling frame (80%) were school principals. We reduced the target population to 14,276 by taking a random sample of 10 principals for any school district with >10 principals in the district. (The 21,852 principals excluded from the target population were used instead as part of a pool of candidates that included district leaders who received a pilot test survey prior to the field test described in this report.) We then created 14 strata by crossing our seven role categories with two groups representing above- and below-median district enrollment (i.e., above and below 17,860 students). Anticipating a 60% response rate, we established two stratified random samples from our reduced target population of 14,276: a primary field test sample and a reservoir field test sample, each containing 168 potential respondents by role or 84 for each role by size stratum. Additional cases were pulled from the reservoir sample, either because of lower-than-anticipated response rates or because we were not able to obtain up-to-date contact information for some members of the primary target sample. Once we identified individuals in our sample, we searched district websites and contacted districts by phone to confirm our roster and to acquire email addresses for respondents or their replacements. In addition, our survey included items asking respondents to indicate their role in the district so that we could assess the accuracy of the classifications of leaders into different role categories.

Following a series of follow-up requests to participate, our final field test sample consisted of 733 people from 487 school districts across 423 cities and 45 states. This represents districts that include roughly 13.8 million of the 50 million students in elementary and secondary students in the United States. The mean number of unique respondents coming from the same school district was 1.5, and the median was 1. The largest number of respondents coming from the same school district was 5.

The breakdown of our sample by decision-making role is shown in Table 2 (for more detail, see Penuel, Briggs, et al., 2016). The overall response rate was 51.5% but varied by role, from lows of about 35% for principals and deputy, associate, and network superintendents to highs of 66% for assessment directors and 71% for special education directors. Although this overall response rate is lower than our target, recent research suggests that lowered response rate may not result in nonrepresentative samples (see Johnson & Wislar, 2012, for a review).

Table 2

Respondents by Role

Role	n	%
Deputy, associate, and network superintendents	90	12
Curriculum supervisors	115	16
Special education supervisors	102	14
Accountability, assessment, and research coordinators	91	12
Elementary, middle school, and K–8 principals	138	19
Directors of federal, bilingual, and ESL programs	89	12
Multiple roles	108	15
Total	733	100

Note. ESL = English as a second language.

We compared survey respondents and nonrespondents with respect to their districts’ total student enrollment and racial/ethnic demographics. In general, education leaders from districts with smaller student enrollments were more likely to respond to the survey (see Table 3). Education leaders who responded to the survey came from school districts with enrollments ranging from roughly 12,400 to 27,000 students, with a median size of 16,431. In contrast, student enrollment in districts associated with survey nonrespondents ranged from roughly 13,300 to 31,400, with a median size of 17,672.

Table 3

Student Enrollment in Districts of Respondents and Nonrespondents

	Percentiles of distribution
	Min	25	50	75	Max
Respondents	167	12,423	16,431	26,899	989,012
Nonrespondents	283	13,329	17,672	31,402	989,012

Note. Unit of analysis is school district. n = 487 school districts for 733 survey respondents; n = 452 school districts for 689 survey nonrespondents.

As shown in Table 4, survey respondents also tended to come from districts with larger proportions of White students and lower proportions of Black and Hispanic students relative to nonrespondents. However, the differences were rather small. For example, the percentage of White students in the school districts associated with survey respondents (46.0%) was 4.5 points higher than the percentage in districts associated with nonrespondents (41.5%). In summary, with respect to some key observable characteristics of the districts in which they were employed, survey respondents were generally comparable to survey nonrespondents, although we cannot rule out the possibility that the participants in the two groups differed more significantly in other respects.

Table 4

Student Demographics of Districts With Respondents Versus Nonrespondents (in Percentages)

	Respondents		Nonrespondents
Demographic group	M	SD	M	SD
Asian/Pacific Islander	6.1	9.0	6.0	8.7
Black	15.9	18.0	17.1	19.1
Hispanic	27.0	24.8	30.7	27.6
White	46.0	25.9	41.5	26.3

Because our sampling frame focused on leaders’ roles and district size, we used sampling weights to adjust the proportions of survey respondents in our role by district size strata to reflect the population proportions. We did this in two ways, one in which we specified a single combined target population of education leaders and another in which we specified two target populations of school principals and central office leaders. We found that use of our sampling weights had no impact on the descriptive statistics and correlations that we present in the sections that follow. In other words, neither district enrollment nor the distinction between leaders’ roles as principals and central office leaders (taken as a group) was significantly correlated with the variables measured.

Reliability of Survey Scales

We defined a total of eight variables as a function of related sets of items on the survey. Each of these variables was created as the mean of anywhere from four to eight discrete items, with scores ranging from 1 to 4 or 1 to 5 within a given set. Reliability was estimated with Cronbach’s alpha (Cronbach, 1951). The scales for research use generally had high reliability coefficients, with the exception of the Imposed Use Scale, which consisted of three items (instrumental, α = .93; conceptual, α = .88; symbolic, α = .81; imposed, α = .72). For the remaining scales, reliability coefficients ranged from .67 to .87 (acquisition effort, α = .79; regular occasions for discussing research, α = .71; culture of research use, α = .87; attitude toward relevance of research, α = .67; attitude toward value of research value, α = .82; attitude toward credibility of research, α = .74).

Analytic Approach

This study is a descriptive and correlational study, focused on characterizing research use, access to research use, and correlates of different types of research use. For each type of research use, we created a simple index that is the sum of responses aggregated across items associated with each scale. We present means, standard deviations, and a distribution of responses for each scale related to research use. For correlates of research use, we report simple bivariate correlations between the use scales and indices for individual and organizational characteristics that we created in the same manner as scales for research use.

Results

Overall, our findings suggest that research was accessed, used more often, and valued more by education leaders than what past research led us to expect. Education leaders in the study accessed research through a variety of sources, especially through professional associations and peers. Where past studies reported that instrumental use was relatively rare, leaders reported using research often for this purpose. They also reported using research frequently for conceptual and symbolic purposes. Despite a conventional wisdom of research as being removed from concerns of practice, education leaders tended to positively endorse statements related to the value, credibility, and relevance of educational research. These attitudes and other individual characteristics were also associated with levels of research use. Among the organizational characteristics that we examined, regular occasions for discussing research and a culture of research use had the strongest positive associations with research use.

How Frequently Do Leaders Report Using Research for Different Purposes?

The four histograms in Figure 1 provide a visual inspection and comparison of reported research use across our four research use scales. Each plot shows the distribution of reported frequency of use as well as overall summary statistics (mean and standard deviation). Recall that each scale is based on the mean of anywhere from three to eight discrete items with Response Categories 1 and 2 indicating infrequent research use (never and sometimes) and Categories 3 and 4 indicating frequent research use (frequent and all of the time). Uses for instrumental purposes were reported with the greatest frequency (M = 3.1, SEM = 0.03), followed by use for imposed purposes (M = 2.7, SEM = 0.03) and then for conceptual and symbolic purposes (for each: M = 2.5, SEM = 0.02). Item-by-item frequency distributions for each scale are available in the online supplementary materials for this article.

Figure 1.

Mean frequency of research use by purpose (n = 733).

Instrumental uses of research were common among leaders involved in a variety of decisions, including those targeted by federal policies intended to promote research use among education leaders, such as decisions related to choosing curricula, directing resources to programs, adopting programs, eliminating programs and designing professional development for teachers. Among education leaders who reported involvement in these kinds of activities, roughly ≥80% reported that they had used research frequently or all of the time in support of these activities. Imposed use was most common for decisions regarding curriculum adoption, for which leaders were required to choose from among research-based programs. About 60% of respondents reported using research frequently or all of the time for this purpose.

While less common than reports of instrumental use, leaders widely reported using research for conceptual and symbolic purposes. Of the possible ways in which leaders might engage in conceptual use, they were most likely to say that they frequently or all of the time encountered research that had expanded their understanding of an issue (71%). Other items endorsed at high levels included those indicating that research provided a common language and set of ideas (57%) or a framework for guiding reform efforts (52%) in their schools and districts. At the same time, leaders were less likely to endorse items representing the claims that research had brought a new issue to their attention (36%) or that research changed the way that they looked at a problem (35%). With respect to use for symbolic purposes, leaders most frequently reported making rhetorical efforts to convince others of a particular point of view on an issue and using research selectively to support a particular decision: 68% and 67%, respectively, said that they did these frequently or all of the time. It was much less common for leaders to say that they used research to discredit a program (21%).

These results underscore the multifaceted nature of research use as described in our conceptual framework. Our results support this understanding of research use and our conjecture that leaders use research in a variety of ways, not just for instrumental purposes

How Do Leaders Access Research?

Although leaders reported accessing research through a variety of sources, their professional connections were the most common, with 55% reporting that they accessed research often or all of the time in this way. Leaders also accessed research commonly through conferences and from state departments of education. Just under 20% of leaders said they accessed research through the What Works Clearinghouse either often or all of the time, slightly more often than they got research directly from university researchers. Leaders accessed research least often from sources outside education, such as the media (see Table 5).

Table 5

Sources Used to Access Research (in Percentages)

Scale: Source of research	Never	Rarely	Sometimes	Often	All the time	Missing
Formal sources
What Works Clearinghouse	36	20	24	13	4	3
University researchers	12	33	36	13	2	3
National Center for Education Statistics	28	27	29	11	2	3
Regional Education Laboratories	27	31	27	11	1	4
Professional networks
Professional associations	2	5	36	40	13	3
Conferences	4	10	43	35	5	3
State department of education	10	16	38	26	7	3
People in other districts	5	15	38	29	10	3
County office of education	28	28	24	14	3	4
Translators of research
Newspaper or magazine	16	24	39	14	3	3
Social media	38	25	21	9	3	3
Vendors	25	35	30	6	3	1
Wikipedia	61	25	9	2	0	4

The pattern in Table 4 underscores the relative strength of social connections in shaping leaders’ access to research. Although half of leaders reported access to a university library (198 of a random subsample of 396 leaders asked), the frequency with which leaders reported accessing research through their professional networks suggests that these networks played an important role in shaping what research leaders actually see and use. This finding is consistent with recent research examining the role of trust in central office leaders’ access to and use of research, which underscores the importance of personal regard and respect in the choice of what research to use (Kochanek & Clifford, 2014). At the same time, our findings do not support our conjecture that research is not widely accessible to leaders. Leaders reported that their professional networks, in addition to other sources (e.g., What Works Clearinghouse), provide access to findings from selected research studies.

Associations of Individual and Organizational Characteristics With Research Use

Table 6 presents a summary of associations that we found between types of research use and the individual and organizational characteristics that we hypothesized would be related to use. In this section, we discuss strong and weak associations between types of research use and the correlates that we measured with our survey.

Table 6

Correlations With Research Use

	Instrumental	Conceptual	Symbolic	Imposed
Individual characteristics
Acquisition effort	.259^***	.469^***	.330^***	.142
Attitudes: relevance	.170^***	.285^***	.139^***	.080^*
Attitudes: value	.227^***	.519^***	.274^***	.139^***
Attitudes: credibility	.068	.284^***	.069	.035
Doctorate/working toward	−.0002	.116^*	.005	−.016
Organizational characteristics
District size (log of total enrollment)	−.004	−.005	−.003	.015
Regular occasions for discussing research	.615^***	.393^***	.520^***	.239^**
Culture of research use (n = 130)	.370^***	.435^***	.479^***	.237^***

Note. n = 733 unless otherwise stated.

p < .05. ^**p < .01. ^***p < .001.

Acquisition effort

The Acquisition Effort Scale was based on responses to five items that asked education leaders how often they made efforts to acquire educational research from different sources (e.g., authors of articles, researchers they know, researchers they do not know, published studies). For example, one item stated, “I contact researchers to find out more about articles they have written.” These items are adapted from ones that previous researchers (e.g., Amara, Ouimet, & Landry, 2004) have used to study this construct. Respondents rated each item on a scale from 1 (never) to 5 (all of the time).

The histogram in Figure 2 shows the distribution of this variable (M = 2.5, SEM = 0.03). This indicates that the average respondent rarely or sometimes sought out research, but this masks an important distinction. That is, while education leaders typically responded that they sometimes or often looked for relevant research studies and found this to be valuable, it was rare for them to contact study authors or researchers to acquire information about the studies. This is consistent with leaders’ reports of where they accessed research: Leaders were more likely to reach out to researchers with whom they were connected than those they did not know.

Figure 2.

Effort expended to acquire research (n = 733).

Acquisition effort was most strongly and positively correlated with conceptual use (r = .469), but it was also significantly correlated with instrumental (r = .259) and symbolic (r = .330) use. This finding is consistent with earlier studies of research use among government agency officials in Canada (e.g., Amara et al., 2004), which were central to the conceptualization of this construct in our framework. The fact that acquisition effort was uncorrelated with imposed use bears further inquiry, especially given that mandates to use research might be expected to result in more active search for research evidence on the part of district leaders.

Attitudes toward research

The survey provided insights into education leaders’ attitudes about the value, relevance, and credibility of education research by rating 20 statements with Likert-style categories ranging from strongly disagree (1) to strongly agree (4). The statements were developed collaboratively with project staff and those with experience working in school districts to correspond to actual language that education leaders would be likely to use. The items were written intentionally to map onto three dimensions, though they were mixed in the survey. Unlike some of our other scales (e.g., instrumental, conceptual, symbolic, and applications of research use) that have a purely formative meaning (i.e., the items operationally define the scale), we had conceptualized our attitude items as having a reflective meaning, written to elicit evidence with respect to three latent constructs. To evaluate whether we were successful in this endeavor, we conducted additional factor analyses on the attitude items as part of our pilot test and field test of the survey.¹ Results from exploratory factor analyses with pilot test data supported a three-factor solution, and this was confirmed after an exploratory factor analysis was conducted with the field test data. With a few exceptions, statements loaded solely onto factors consistent with their intended design of three attitude subscales. Statements illustrative of the value, relevance, and credibility dimensions are, respectively, “Research addresses questions that help make better decisions,” “Educational researchers live in an ivory tower isolated from practice,” and “The claims that research studies make are trustworthy.” Scores on each dimension were computed by taking the mean of their underlying items, with all negatively worded items reverse scored. Item-by-item response frequencies are available in the supplementary materials associated with this article.

Figure 3 provides a visual comparison of the distribution of means by attitude subscale. In general, a majority of education leaders tended to agree or strongly agree with statements about the value of educational research (M = 3.2, SE = .01). Leaders also reported positive perspectives about the relevance (M = 2.8, SE = 0.02) and credibility of research (M = 2.7, SE = 0.01), but here there were some statements that drew mixed reactions. For example, on the Relevance subscale, 52% of respondents agreed or strongly agreed with the statement “By time research is published it is no longer useful to me,” and on the Credibility subscale, only 51% of respondents agreed or strongly agreed with “Educational researchers are unbiased.” In other words, although education leaders tended to endorse statements about the relevance and credibility of research in a positive direction, there are clearly contexts in which leaders question the relevance and credibility of research. Overall, however, we found more limited support for the broader conjecture that we formulated that leaders’ attitudes toward research would be mostly mixed.

Figure 3.

Attitudes toward value, relevance, and credibility of research (n = 733).

The correlations between subscales of attitudes and conceptual use were strong among all research uses (Table 5). Particularly strong was the positive association between attitudes toward the value of research and conceptual uses of research (r = .519). In other words, those who reported using research to expand their understanding of issues often claimed that research was highly valuable for their work. Across the board, leaders’ attitudes about the value of research were positively associated with all types of research use, though more weakly associated with imposed use. This latter finding is not surprising: Requiring a leader to use research does not mean that the leader will value it.

Somewhat surprising, given our conceptual framework, was the lack of a relationship between attitudes regarding the credibility of research and three types of research use: instrumental, symbolic, and imposed use. We had hypothesized that when leaders perceived research as a credible source of information, they would be more likely to use it in decision making. Leaders’ responses indicated that this was only the case for conceptual uses of research, however. This finding merits further study in future research.

Role

As Table 7 indicates, for most role groups surveyed, levels of all types of research use were similar, but there were significant overall differences across role for conceptual, symbolic, and imposed use. Federal program leaders reported higher levels of conceptual use than district leaders (Bonferroni-corrected p = .04) and principals (p = .01) and higher levels of symbolic use than assessment leaders (p = .01). Special education leaders were also more likely to use research symbolically than were assessment leaders (p = .01). Imposed use among federal programs leaders was higher than that for curriculum and instruction leaders (p = .02) and for principals (p = .01). We suspect that these uses were higher for these groups in part because of policies and laws that mandate the use of evidence-based programs. These results were consistent with our expectation that role would be related to research use and with our conceptual framework positing that use would differ by department, which is at least partly reflected in the role groups that we identified and compared.

Table 7

Reported Research Use by Professional Role

	Instrumental			Conceptual			Symbolic			Imposed
Professional role	n	M	SD	n	M	SD	n	M	SD	n	M	SD
District leadership	84	3.00	0.66	90	2.46	2.44	90	2.51	0.52	90	2.65	0.70
Curriculum and instruction	114	3.08	0.71	115	2.55	0.49	115	2.50	0.63	114	2.60	0.83
Special education	98	3.12	0.65	102	2.52	0.49	102	2.60	0.63	100	2.62	0.81
Assessment	89	2.99	0.67	91	2.49	0.53	91	2.29	0.64	89	2.79	0.65
Principals	123	3.04	0.72	138	2.46	0.52	137	2.40	0.63	136	2.59	0.82
Federal programs	85	3.26	0.75	89	2.69	0.58	89	2.62	0.68	88	2.97	0.81
Multirole	107	3.19	0.65	107	2.55	0.41	108	2.56	0.63	108	2.79	0.75
F statistic		1.96			2.53			3.39			3.17
p value		.069			.020			.003			.004

Preparation to interpret research

Of the random subsample of 376 leaders who were asked, 137 (36%) either held or were working toward an advanced degree, which we took to be indicative of preparation to use research as described in our conceptual framework. When those who held or were working toward a degree (M = 2.61, SD = 0.54, n = 137) were compared with other respondents (M = 2.5, SD = 0.44, n = 241), the mean levels of use were higher for conceptual use by nearly one quarter of a standard deviation (ES = 0.233). The correlations with other forms of use were not significant, however. Thus, our survey results were consistent with our conjecture about preparation to use research with respect to conceptual use, but findings did not support our conjecture about preparation to use research instrumentally and symbolically.

Regular occasions for discussing research

We asked about the frequency with which research was discussed in regular meetings within leaders’ organizations. Research was brought up most commonly in meetings related to instruction (76% said that they did so often or all of the time) as well as those focused on designing or adapting programs (67%, often or all of the time). Just under two thirds (65%) of leaders said that research came up often or more in meetings related to selecting curricula or interventions (Table 8). The frequency with which research was discussed in meetings was strongly associated with instrumental use (r = .615) and symbolic use (r = .520). We suspect that these are higher because both are likely to occur in the deliberative contexts of meetings where research comes into play in decision making, either to inform a decision that has not yet been made (instrumental) or to support a position with regard to a decision being made or defended post hoc (symbolic).

Table 8

How Often Research Came Up in Different Types of Meetings (in Percentages)

	n	Never	Occasionally	Often	All of the time
Instruction	721	1	22	48	28
Designing or adapting programs	699	2	25	46	21
Strategic planning	712	5	33	40	19
Parent or community issues	698	11	44	30	9
How well a program was implemented	701	6	34	41	14
Selecting curricula or interventions	685	2	26	42	23

Organizational culture of research use

Four questions relevant to a district’s or department’s culture related to the use of research were answered by a subset of respondents. A majority of respondents indicated that research is viewed as being useful in their organization (87%) and that they are encouraged to use research as part of their work (76%). It was less common for respondents to indicate that their districts or departments frequently conduct studies on the programs that they implement (54%) or that staff are expected to back up claims with research evidence (44%; see Table 9).

Table 9

Culture of Research Use in Leaders’ Organizations (in Percentages)

	Never	Occasionally	Often	All of the time
Research is seen as a useful source of information.	0	13	40	47
We are genuinely encouraged to use research as part of our ongoing work.	3	20	36	40
We conduct studies on programs we select and implement to see how they work.	10	36	34	20
It is expected that if you make a claim at a meeting, you will be able to cite research evidence to back it up.	16	40	30	14

As shown in Table 9, a perceived culture that supported research use in leaders’ organizations was positively associated with all types of research use. Leaders reporting that their organizational culture valued and encouraged research use were also more likely to report higher conceptual (r = .44) and symbolic (r = .48) use. Although the association with instrumental use was less strong for a culture of use (r = .37) than it was for regular use in meetings (r = .62), a clear and positive relationship existed. Overall, organizational conditions related to valuing, encouraging, and offering regular opportunities for research use were more strongly associated with instrumental, symbolic, and imposed uses of research than were individual characteristics related to acquisition effort, attitudes toward research, and preparation to interpret research, while individual characteristics were more strongly associated with conceptual use. Taken together, these associations indicate that individual characteristics and organizational conditions both mattered for leaders’ use of research for various purposes.

Summary and Discussion

In our study, leaders reported high levels of research use of all four types identified by Weiss and Bucuvalas (1980). Leaders reported frequent use of research to make decisions, to expand their thinking about issues, and to persuade others of particular points of view, and they reported frequently being required to use research to choose curricula, justify programs, and fulfill grant requirements. Our findings contrast with past observational studies that found limited use of research by school and district leaders, especially for instrumental purposes (Coburn et al., 2009). Because studies that found higher rates of conceptual use drew more on direct observations of leaders rather than on survey responses, it is important to consider whether differences in our finding might reflect differences in localized practices as opposed to a nationally representative sample or if they might reflect the potential for interview, observation, and survey methods to depict different aspects of research use. Our own view is that it is likely a method effect: Our own ongoing mixed methods studies of research use that employ surveys and interviews suggest that our results are actually largely consistent with prior research in this regard (Penuel, Farrell, et al., 2016).

In addition, for school and district leaders, access to research may be less of an issue than what past research has found (Coburn et al., 2009) and conventional wisdom has suggested. Contrary to images of inaccessible research in ivory towers, education leaders in our sample reported that they leveraged their affiliations with professional associations and peer networks to access research. They also perceived research to be valuable to their decision making, in contrast to the widely circulating view that educators do not view research as being useful to their work. With respect to the latter finding, it is possible that this is an artifact of focusing on school and district leaders, rather than on teachers. Teachers may be more likely to encounter research in the form of tools for practice (e.g., curriculum materials that reflect research on learning) and messages given in professional development (Ikemoto & Honig, 2010; Kennedy, 2005). They might therefore not view research per se as useful or relevant, and they might see colleagues as a more valuable source of new knowledge (cf., Kennedy, 2005; Smylie, 1989). There is little published research in recent years comparable to our own on teachers’ views and use of research, however. We found modest associations between a number of individual characteristics and research use scales but stronger associations between two organizational conditions and research use scales. People who made more effort to acquire research and who had more positive attitudes toward research (especially with regard to the value and relevance of research) were more likely to say that they used research in instrumental, conceptual, and symbolic ways. These results are consistent with past research examining associations between individual characteristics and levels of research use (Landry et al., 2003). In addition, it seems likely that concrete efforts to acquire research lead to use of that research, for whatever purposes. They could signal an underlying orientation or disposition to use research on the part of individuals, but that conjecture requires further investigation.

What is particularly striking in our findings is the strong correlation between social and cultural aspects of district departments and central offices and different types of use. Having a strong district or departmental culture of research use was significantly associated with all types of research use, as was attendance in meetings where leaders reported research was discussed. The potential role of culture in supporting research use is supported by research outside education (e.g., Helmsley-Brown & Sharp, 2003) but, to date, has not been documented in educational research. Our study points to the need for further study of the organizational conditions related to research use. To this end, our current comparative case study research in four large school districts is considering aspects of organizational routines (Feldman & Pentland, 2003) that influence research use in decision making related to literacy and mathematics instruction.

Our findings regarding the strength of some associations but not others suggest a refinement of our conjectures about the relationship between individual and organizational characteristics, on one hand, and research use, on the other. For example, the stronger associations between having regular occasions for discussing research and instrumental and symbolic use suggest the need for a conceptual framework to account for differences in the social context of each type of use. Conceptual use may occur in social settings and in moments of individual reflection, whereas instrumental and symbolic uses may occur primarily in the context of social deliberation. More mixed methods studies that include surveys, interviews, and observations are needed to explore this possibility.

Study Limitations

One limitation of our study is that its findings may not apply to leaders in smaller districts in the United States. Our study focused on the largest school districts in the United States—that is, those with roughly ≥9,000 students. There are many more school districts that have fewer students, especially in rural areas. We focused on larger districts because we wanted to understand the link between role in the central office and research use; specifically, larger districts tend to have larger central offices with more variation in role. But our choice means that we cannot draw inferences about how leaders use research in smaller districts.

Another limitation is that our study relied on self-report. Self-report on surveys involving socially desirable behaviors like ours are subject to bias (Burstein et al., 1995). We suspect that the Instrumental Use Scale is more likely to be subject to social desirability bias than the other scales for this reason. In particular, we suspect that one reason why our findings are different from past studies is that there is a strong cultural bias toward using research evidence in decision making, and that bias may be reflected in leaders’ survey responses. This bias may be due to policy shifts that have taken place in recent decades, since early survey studies of research use in education found leaders to be more ambivalent about research use (West & Rhoton, 1994). Or, it is possible that research use indeed has increased with these policy shifts. In addition, self-report makes it difficult to distinguish between symbolic or political uses of research that are responsible (i.e., consistent with what research findings actually support) and those that are selective (elaborated later). We attempted to mitigate self-report bias through anonymous surveys, a condition that can reduce self-report bias as past research in education has shown (Mullens & Gayler, 1999).

It is important to recognize that self-report provides only a partial (albeit informative) picture of the interactive processes involved in research use. For one, researchers and educators can have very different definitions of what counts as research (Penuel, Farrell, et al., 2016). Therefore, it is possible that leaders interpreted our questions more broadly than intended, when reporting the role that research played in different work activities. Complementary methods of data collection are needed to understand these processes better. These include observations of the actual processes of deliberation and decision making in which research comes into play, as well as interviews of leaders focused on the role that research evidence plays in different work activities.

Conclusions and Implications

Our study is the first national survey of its kind to shed light on research use among a critical group of policy makers: local education leaders. No doubt, school and district leaders’ positive reports about the value of research in their work may be surprising and met with skepticism. Indeed, other studies show that research evidence is one of a number of considerations that a school or district leader does and must take into account when making decisions (Asen et al., 2011; Penuel, 2010). Moreover, because we did not study leaders’ actual processes of using research, our survey study cannot speak to the quality of research use.

Our study is consistent with prior research in that the purposes for which school and district leaders reported using research are broader than the instrumental uses that current policies encourage. Leaders report using research often for instrumental purposes, but they also report using research to expand their conceptions of problems and to persuade others of particular points of view. Conceptual uses in particular merit stronger attention in policies to support leaders in interpreting and using research to get new ideas, challenge preconceptions about problems and their solutions, and guide design efforts for new initiatives in their schools and districts.

We recognize that not all policy makers will be encouraged by the finding that symbolic use is common among school and district leaders. In our view, this finding merits more careful interpretation and consideration of the different functions symbolic use might play. It is important to note that the use of research to persuade others is not in and of itself problematic. Weiss (1979) wrote,

Only distortion and misinterpretation of findings are illegitimate. To the extent that the research, accurately interpreted, supports the position of one group, it gives the advocates of that position confidence, reduces their uncertainties, and provides them an edge in the continuing debate. (p. 429)

We cannot tell from our survey results whether leaders distorted or misinterpreted findings of research when using them symbolically. What is more likely—based on other studies of research use in politically contentious debates—is that research evidence invoked in deliberations is given only cursory treatment, invoked as broad claims about what “research says” without attention to details of particular studies (Asen et al., 2011). Our pattern of results, moreover, suggests that a culture of research use can exist side by side with symbolic uses of research. If policy makers desire more substantive engagement with research, then a better understanding is needed of the contexts where deliberation allows for consideration of the applicability of study findings to particular decisions.

We recognize that current policies—especially the Every Student Succeeds Act—point leaders toward instrumental use of research evidence. The good news is that school and district leaders have a strong appetite for research. Building on this are actionable opportunities to better integrate research and practice. Based on our findings, joining professional associations and attending conferences where education leaders are accessing research may be an important strategy for researchers to pursue (for one strategy, see Penuel, Bell, Bevan, Buffington, & Falk, 2016). Connecting directly to leaders through long-term research-practice partnerships is another strategy that has shown some promise for improving the connections between research and practice (see Coburn & Penuel, 2016, for reviews; Contandriopoulos et al., 2010; National Research Council, 2012). While partnerships require more intensive involvement on the part of researchers and education leaders, they are finding support in burgeoning commitments to understanding and investing in research that is closely connected to practice. In addition, they may be a valuable means to developing the trust that others (e.g., Kochanek & Clifford, 2014) have found to be an important condition for accessing and using research.

There are also potential implications for how and where we disseminate research. At present, university tenure policies privilege publication in peer-reviewed journal articles and presentations to academic audiences. In addition, emphasis is placed on communicating findings to practitioners in a clear, jargon-free way that focuses on the implications for practice (e.g., Dynarski & Kisker, 2014). But our finding that professional associations are a key source of research among school and district leaders suggests that a different approach might be needed. Attending, presenting, and even joining professional associations of leaders could be a more viable strategy for increasing research use. Not only does becoming part of such associations help researchers gain a better sense of the problems that school and district leaders are facing, it also builds trusting relationships that become the basis for leaders turning to researchers for advice (Penuel, Bell, et al., 2016). Intentional efforts to promote research access and use through professional associations should be studied and compared with other approaches, such as traditional web-based dissemination.

More broadly, research is needed on intentional collaborative strategies to promote use and their effects, such as research-practice partnerships and sustained engagements with professional associations. To date, the field has based policies about evidence use on theories of action that have poorly reflected the breadth of research use documented by Weiss and colleagues beginning in the early 1980s. A body of research on how best to promote different types of research for different leaders and to address different kinds of problems of practice is long overdue. For instance, research might explore how a book study among a group of administrators might support an in-depth interpretation of key ideas from research and their application to district reform efforts. Our study aims to provide an initial foundation for developing such a knowledge base.

Given our findings and the prevailing literature on research use, the language of policies related to research use should be broadened. Recognizing the multiple ways in which research informs the work of education leaders, policies could encourage evidence use across leaders’ various activities, not just those related to adoption decisions but also those that are more common, such as the design of professional development. Rather than solely focusing on impact studies of programs, policies might encourage leaders to turn to a methodologically broader range of sound research to inform their thinking or to support a strategic direction in their districts. Of course, expanding policies in this way creates new learning challenges for educators and the field more broadly. We do not yet know how best to support these different uses of research or how best to support research use within different activities, making the development of an evidence base for interventions to promote research use all the more important.

Footnotes

Funding

This study was funded by the Institute of Education Sciences (10.13039/100005246, R305C140008). Publication of this article was funded by the University of Colorado Boulder Libraries Open Access Fund.

Notes

Authors

WILLIAM R. PENUEL is professor of learning sciences and human development in the School of Education at the University of Colorado Boulder. He studies alternate models for relating research and practice and the implementation of reforms in science and mathematics education.

DEREK C. BRIGGS is professor of research and evaluation methodology in the School of Education at the University of Colorado Boulder. His research focuses on technical and theoretical issues in the development of measurement models.

KRISTEN L. DAVIDSON is a postdoctoral researcher in the Center for Research in Policy and Practice in the School of Education at the University of Colorado Boulder. Her research focuses on parent choice, democratic theory and approaches to education, and use of assessment data in educational decision making.

CORINNE HERLIHY is the director of research operations for the Center for Education Policy Research at the Harvard Graduate School of Education. Her research has included evaluation studies of small schools as well as mathematics interventions and reading interventions.

DAVID SHERER is a doctoral student at the Harvard Graduate School of Education. He specializes in research on policy implementation and the social dynamics of K–12 school reform.

HEATHER C. HILL is the Jerome T. Murphy Professor in Education at the Harvard Graduate School of Education. Her primary work focuses on teacher and teaching quality and the effects of policies aimed at improving both.

CAITLIN FARRELL is director of the Center for Research in Policy and Practice in the School of Education at the University of Colorado Boulder. Farrell studies the use of research and data among district leaders and models of district reform.

ANNA-RUTH ALLEN is a research associate in the School of Education at the University of Colorado Boulder. Her research interests include research-practice partnerships, research use, and youth literacies in and out of school.

References

Aarons

G. A.

Wells

R. S.

Zagursky

Fettes

D. L.

Palinkas

L. A.

(2009). Implementing evidence-based practice in community mental health agencies: A multiple stakeholder analysis. American Journal of Public Health, 99(11), 2087–2095.

Amara

Ouimet

Landry

(2004). New evidence on instrumental, conceptual, and symbolic utilization of university research in government agencies. Science Communication, 26(1), 75–106.

Anderson

Leithwood

Strauss

(2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 293–327.

Asen

Gurke

Solomon

Conners

Gumm

(2011). “The research says”: Definitions and uses of a key policy term in federal law and local school board deliberations. Argumentation and Advocacy, 47, 195–213.

Bickel

W. E.

Cooley

W. W.

(1985). Decision-oriented educational research in school districts: The role of dissemination processes. Studies in Educational Evaluation, 11(2), 183–203.

Burch

Spillane

J. P.

(2005). How subjects matter in district office practice: Instructionally relevant policy in urban school district redesign. Journal of Educational Change, 6(1), 51–76.

Burstein

McDonnell

Van Winkle

Ormseth

Mirocha

Guiton

(1995). Validating national curriculum indicators. Santa Monica, CA: RAND.

Coburn

C. E.

(2010). Partnership for district reform: The challenges of evidence use in a major urban district. In Coburn

C. E.

Stein

M. K.

(Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 167–182). New York, NY: Rowman & Littlefield.

Coburn

C. E.

Honig

M. I.

Stein

M. K.

(2009). What’s the evidence on districts’ use of evidence? In Bransford

J. D.

Stipek

D. J.

Vye

N. J.

Gomez

L. M.

Lam

(Eds.), The role of research in educational improvement (pp. 67–87). Cambridge, MA: Harvard Education Press.

10.

Coburn

C. E.

Penuel

W. R.

(2016). Research-practice partnerships in education: Outcomes, dynamics, and open questions Educational Researcher, 45(1), 48–54.

11.

Coburn

C. E.

Talbert

J. E.

(2006). Conceptions of evidence use in school districts: Mapping the terrain. American Journal of Education, 112, 469–495.

12.

Coburn

C. E.

Toure

Yamashita

(2009). Evidence, interpretation, and persuasion: Instructional decision making at the district central office. Teachers College Record, 111(4), 1115–1161.

13.

Contandriopoulos

Lemire

Denis

J.-L.

Tremblay

(2010). Knowledge exchange processes in organizations and policy arenas: A narrative systematic review of the literature. Milbank Quarterly, 88(4), 444–483.

14.

Corcoran

T. B.

Fuhrman

Belcher

C. L.

(2001). The district role in instructional improvement. Phi Delta Kappan, 83(1), 78–84.

15.

Cronbach

L. J.

(1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334.

16.

Datnow

Honig

M. I.

(2008). Introduction to the special issue on scaling up teaching and learning improvement in urban districts: The promises and pitfalls of external assistance providers. Peabody Journal of Education, 83(3), 323–327.

17.

Datnow

Park

(2010). Success for All: Using tools to transport research-based practices to the classroom. In Coburn

C. E.

Stein

M. K.

(Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 77–91). Lanham, MD: Rowman & Littlefield.

18.

David

J. L.

(1981). Local uses of Title I evaluations. Educational Evaluation and Policy Analysis, 3(1), 27–39.

19.

Dwyer

C. A.

Millett

C. M.

Payne

D. G.

(2006). A culture of evidence: Postsecondary assessment and learning outcomes. recommendations to policymakers and the higher education community. Princeton, NJ: Educational Testing Service.

20.

Dynarski

Kisker

(2014). Going public: Writing about research in everyday language (Report No. REL-2014-051). Washington, DC: Department of Education.

21.

Farley-Ripple

(2012). Research use in school district central office decision making: A case study. Educational Management, Administration, & Leadership, 40(6), 786–806.

22.

Feldman

M. S.

Pentland

B. T.

(2003). Reconceptualizing organizational routines as a source of flexibility and change. Administrative Science Quarterly, 48(1), 94–118.

23.

Fitzsimmons

Cooper

(2012). Embedding a culture of evidence-based practice. Nursing Management, 19(7), 14–19.

24.

Gerrish

Clayton

(2004). Promoting evidence-based practice: An organizational approach. Journal of Nursing Management, 12(2), 114–123.

25.

Halverson

Grigg

Prichett

Thomas

(2007). The new instructional leadership: Creating data-driven instructional systems in school. Journal of School Leadership, 17(2), 159–194.

26.

Haskins

Margolis

(2015). Show me the evidence: Obama’s fight for rigor and results in social policy. Washington, DC: Brookings Institution.

27.

Haskins

Paxson

Brooks-Gunn

(2009). Social science rising: A tale of evidence shaping public policy. Princeton, NJ: The Future of Children.

28.

Helmsley-Brown

Sharp

(2003). The use of research to improve professional practice: A systematic review of the literature. Oxford Review of Education, 29(4), 449–470.

29.

Hightower

A. M.

Knapp

M. S.

Marsh

J. A.

McLaughlin

M. W.

(2002). School districts and instructional renewal. New York, NY: Teachers College Press.

30.

Honig

M. I.

Copland

Rainey

L. R.

Lorton

J. A.

Newton

(2010). Central office transformation for district-wide teaching and learning improvement. Seattle, WA: Center for the Study of Teaching and Policy.

31.

Honig

M. I.

Venkateswaran

McNeil

Twitchell

J. M.

(2014). Leaders’ use of research for fundamental change in school district central offices: Processes and challenges. In Finnigan

K. S.

Daly

A. J.

(Eds.), Using research evidence in education: From the schoolhouse door to Capitol Hill (pp. 33–52). New York, NY: Springer.

32.

Hubbard

(2010). Research to practice: The case of Boston Public Schools, Education Matters and the Boston Plan for Excellence. In Coburn

C. E.

Stein

M. K.

(Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 55–72). Lanham, MD: Rowman & Littlefield.

33.

Ikemoto

G. S.

Honig

M. I.

(2010). Tools to deepen practitioners’ engagement with research: The case of the Institute for Learning. In Coburn

C. E.

Honig

M. I.

(Eds.), Research and practice in education: Building alliances, bridging the divide (pp. 93–108). Lanham, MD: Rowman & Littlefield.

34.

Johnson

B. L.

Jr. (1999). The politics of research-information use in the education policy arena. Educational Policy, 13(1), 23–36.

35.

Johnson

Greenseid

L. O.

Toal

S. A.

King

J. A.

Lawrenz

Volkov

(2009). Research on evaluation use: A review of the empirical literature from 1986 to 2005. American Journal of Evaluation, 30(3), 377–410.

36.

Johnson

T. P.

Wislar

J. S.

(2012). Response rates and nonresponse errors in surveys. Journal of the American Medical Association, 307(17), 1805–1806.

37.

Kennedy

M. M.

(1982). Evidence and decision. In Kennedy

M. M.

(Ed.), Working knowledge and other essays (pp. 59–103). Cambridge, MA: Huron Institute.

38.

Kennedy

M. M.

(2005). Inside teaching: How classroom life undermines reform. Cambridge, MA: Harvard University Press.

39.

Kerr

K. A.

Marsh

J. A.

Ikemoto

G. S.

Darilek

Barney

(2006). Strategies to promote data use for instructional improvement: Actions, outcomes, and lessons from three urban districts. American Journal of Education, 112(4), 496–520.

40.

Kochanek

J. R.

Clifford

(2014). Trust in districts: The role of relationships in policymaking for school improvement. In Van Maele

Forsyth

P. B.

Houtte

M. V.

(Eds.), Trust and school life: The role of trust for learning, teaching, leading, and bridging (pp. 313–334): Springer.

41.

Landry

Lamari

Amara

(2003). The extent and determinants of the utilization of university research in government agencies. Public Administration Review, 63(2), 192–205.

42.

Little

J. W.

(2012). Understanding data use practice among teachers: The contribution of micro-process studies. American Journal of Education, 118(2), 143–166.

43.

Makar

Confrey

(2005). Secondary teachers’ statistical reasoning in comparing two groups. In Ben-Zvi

Garfield

(Eds.), The challenge of developing statistical literacy, reasoning, and thinking (pp. 353–374). New York, NY: Kluwer.

44.

Means

Padilla

DeBarger

Bakia

(2009). Implementing data-informed decision making in schools—Teacher access, supports and use. Washington, DC: U.S. Department of Education.

45.

Meyer

J. W.

Scott

W. R.

Strang

(1987). Centralization, fragmentation, and school district complexity. Administrative Science Quarterly, 32, 186–201.

46.

Mullens

J. E.

Gayler

(1999). Measuring classroom instructional processes: Using survey and case study field test results to improve item construction. Washington, DC: U.S. Department of Education.

47.

National Research Council. (2012). Using science as evidence in public policy. Washington, DC: National Academies Press.

48.

Park

Datnow

(2009). Co-constructing distributed leadership: District and school connections in data-driven decision-making. School Leadership and Management, 29(5), 477–494.

49.

Penuel

W. R.

(2010). A dialogical epistemology for educational evaluation. National Society for the Study of Education Yearbook, 109(1), 128–143.

50.

Penuel

W. R.

Bell

Bevan

Buffington

Falk

(2016). Enhancing use of learning sciences research in planning for and supporting educational change: Leveraging and building social networks. Journal of Educational Change, 17(2), 251–278.

51.

Penuel

W. R.

Briggs

D. C.

Davidson

K. L.

Herlihy

Sherer

Hill

H. C.

. . . Allen

A.-R.

(2016). Findings from a national survey of research use among school and district leaders. Boulder, CO: National Center for Research in Policy and Practice.

52.

Penuel

W. R.

Farrell

C. C.

Allen

A.-R.

Toyama

Coburn

C. E.

(2016). What research district leaders find useful. Educational Policy. Advance online publication.

53.

Revelle

(2016). Psych: Procedures for personality and psychological research (Version 1.6.9). Evanston, IL: Northwestern University. Retrieved from https://CRAN.R-project.org/package=psych

54.

Rorrer

A. K.

Skrla

Scheurich

J. J.

(2008). Districts as institutional actors in educational reform. Educational Administration Quarterly, 44(3), 307–357.

55.

Sharkey

N. S.

Murnane

R. J.

(2006). Tough choices in designing a formative assessment system. American Journal of Education, 112(4), 572–588.

56.

Smylie

M. A.

(1989). Teachers’ views of the effectiveness of sources of learning to teach. Elementary School Journal, 89(5), 543–558.

57.

Spillane

J. P

. (1996). School districts matter: Local educational authorities and state instructional policy. Educational Policy, 10(1), 63–87.

58.

Spillane

J. P.

(1998). State policy and the non-monolithic nature of the local school district: Organizational and professional considerations. American Educational Research Journal, 35(1), 33–63.

59.

Spillane

J. P.

Thompson

C. L.

(1997). Reconstructing conceptions of local capacity: The local education agency’s capacity for ambitious instructional reform. Educational Evaluation and Policy Analysis, 19(2), 185–203.

60.

Spillane

J. P.

Parise

L. M.

Sherer

J. Z.

(2011). Organizational routines as coupling mechanisms: Policy, school administration, and the technical core. American Educational Research Journal, 48(3), 586–619.

61.

Weiss

C. H.

(1979). The many meanings of research utilization. Public Administration Review, 39, 426–431.

62.

Weiss

C. H.

(1980). Knowledge creep and decision accretion. Knowledge: Creation, Diffusion, Utilization, 1(3), 381–404.

63.

Weiss

C. H.

Bucuvalas

M. J.

(1980). Social science research and decision-making. New York, NY: Columbia University Press.

64.

Weiss

C. H.

Murphy-Graham

Birkeland

(2005). An alternate route to policy influence: How evaluations affect D.A.R.E. American Journal of Evaluation, 26(1), 12–30.

65.

West

R. F.

Rhoton

(1994). School district administrators’ perceptions of educational research and barriers to research utilization. ERS Spectrum, 12(1), 23–30.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.02 MB

How School and District Leaders Access,Perceive,and Use Research

Abstract

Keywords

Conceptual Framework

The Multifaceted Nature of Research Use

Access to Research and Types of Research Used

Individual Characteristics Related to Research Use

Acquisition effort and attitudes toward research

Formal preparation to interpret research

Organizational Correlates of Research Use

District size and professional roles

Regular occasions for talking about research

Culture of research use

Methods

Survey Development

Items related to type of use (Research Question 1)

Items related to where leaders access research and what research they use (Research Question 2)

Variables related to individual characteristics (Research Question 3)

Items related to organizational conditions (Research Question 3)

Population and Sample

Reliability of Survey Scales

Analytic Approach

Results

How Frequently Do Leaders Report Using Research for Different Purposes?

How Do Leaders Access Research?

Associations of Individual and Organizational Characteristics With Research Use

Acquisition effort

Attitudes toward research

Role

Preparation to interpret research

Regular occasions for discussing research

Organizational culture of research use

Summary and Discussion

Study Limitations

Conclusions and Implications

Footnotes

Funding

Notes

Authors

References

Supplementary Material