Teacher Implementation of College- and Career-Readiness Standards: Links Among Policy,Instruction,Challenges,and Resources

Abstract

Using state-representative teacher surveys in three states—Texas, Ohio, and Kentucky—we examine teachers’ implementation of college- and career-readiness (CCR) standards. What do teachers report about the specificity, authority, consistency, power, and stability of their standards environment? How does their policy environment predict standards-emphasized instruction? Do these relationships differ for those who teach different subjects (math and English Language Arts [ELA]), different grades (elementary or high school), different populations (English Language Learners [ELLs], students with disabilities [SWDs]), and in different areas (rural, urban, or suburban)? We found elementary math teachers taught significantly more standards-emphasized content than elementary ELA teachers, whereas secondary ELA teachers taught significantly more standards-emphasized content than secondary math teachers. Teachers of SWDs and rural teachers taught significantly less of the emphasized content. In all three states, we found greater buy-in (authority) predicted increased emphasized content coverage among ELA teachers but not among math teachers.

Keywords

standards-based reform Common Core instruction state policy college- and career-readiness standards

Introduction

Since the passage of the Every Student Succeeds Act (ESSA) in December 2015, K–12 standards-based reform is in the midst of a period of growing decentralization as states revise their standards, assessments, and associated policies (Polikoff, 2017). The resulting cross-state variation in standards, and the growing within-state differences in teacher supports and political rhetoric, suggest there may be wide variations in how teachers experience standards-based policy. Or, we may be overstating the extent to which state policy contexts are substantively different from teachers’ perspectives. To understand if and how standards-based reform improves student learning, we first must understand how teachers view and experience these policies and whether their perceptions of their policy environments have relationships to what teachers teach. Standards-emphasized instruction sits at the core of the theory of change of standards-based reform (M. S. Smith & O’Day, 1991). If teachers do not change what they are teaching to match state standards, standards-based policies may be much ado about nothing.

Despite backlash to the Common Core State Standards (CCSS) and the declaration by education secretary Betsy DeVos that “Common Core is dead” (U.S. Department of Education, 2018), college- and career-readiness (CCR) standards remain viable because of a series of institutional logics (DiMaggio & Powell, 1983) in both the political and professional spheres. The general public expects there to be some standards in K–12 public education and some level of accountability regardless of the name attached to the standards themselves—or their specific content (Bovens, 2007). In their 2015 book, Supovitz and Spillane (2015) identify seven reasons for supporting standards, and this study focuses on their second rationale: influencing core classroom interactions between teachers and students. Policymakers continue to set academic expectations for teachers in the hopes that teachers will teach to them, and this trend is arguably centuries old (Gamson, 2015).

Importance of the Study

Our study is novel in its efforts to measure how teachers experience policy and whether these perceptions relate to instruction. According to other surveys, teachers have made major instructional shifts in response to standards (Kane, Owens, Marinell, Thal, & Staiger, 2016)—but this conclusion relies on reported change rather than reported instructional content. We advance the field in this regard by measuring standards-emphasized content items while still accounting for differences in teacher professional development, classroom resources, and school-level random effects.

Successful standards implementation requires policymakers to think carefully about teacher learning and interpretation of the policy’s message (Spillane, Reiser, & Reimer, 2002). Previous studies of standards-based reform efforts have described state and district initiatives as well as how teachers perceive policy (e.g., Goertz, 2005; Spillane, 1999). This implementation research has chronicled the successes and failures of standards without proposing a unified framework to consider what is necessary for effective implementation (Hamilton, Stecher, & Yuan, 2008). There continues to be unevenness in professional development opportunities across and within schools in the implementation of standards according to Supovitiz, Fink, and Newman (2016). Other studies have established that much more alignment work needs to be done, particularly in the realms of instruction and system incentives (Massell & Perrault, 2014). This study considers all of these implementation issues in their totality, rather than focusing on a single initiative or intervention, a holistic approach that better captures the multiplicity of messages that teachers receive from policymakers. We consider whether teachers listen to these complex policy messages in deciding on their own instruction—or, as the decades-old loose-coupling argument goes, ignore them altogether (Weick, 1976).

Rather than examining standards policies “as written,” we ground our study in the idea that it is educators’ interpretations and beliefs about the policies that shape their reactions (e.g., Desimone, 2002). The best-designed standards policy matters little if its primary implementers, teachers, have wholly different interpretations (H. C. Hill, 2001; Spillane, 2009). Contemporaneous surveys show that even several years after adoption, most teachers felt that they still had not received adequate professional development on how to teach the standards (Troia & Graham, 2016). It is therefore of critical importance to examine teacher perspectives on the resources and supports they receive related to implementing standards, the instructional challenges they face, and whether these factors influence their use of standards in the classroom.

We build on prior work by detailing teachers’ implementation of college- and career-readiness standards and examine whether instruction is shifting to match the standards. We take an inclusive approach, unusual in the standards-based reform literature, by comparing and contrasting teachers of general education students, students with disabilities (SWDs), and English language learners (ELLs). Using state-representative surveys administered across three states in 2015–2016, we describe how teachers perceive policies related to their state standards. Our study allows several important comparisons by comparing the experiences of teachers across three states along three dimensions: geography (rural, suburban, and urban), grade level (elementary or secondary), and subject area (teachers of math, English Language arts [ELA], SWDs, and ELLs). In the following, we describe the importance of these particular contrasts and their implications for more deeply understanding how teachers and students are experiencing one of our nation’s major and ongoing education reforms.

First, there has been limited work on standards-based policy for SWDs and ELLs. Some describe standards-based reform as experienced by teachers of SWDs (e.g., Bacon, 2015) and ELLs (e.g., Figueroa-Murphy & Haller, 2015) by documenting how teachers struggle to differentiate instruction while still meeting accountability system demands. Others consider how ELLs might be better represented in the design of common standards (Flores & Schissel, 2014). But we are unaware of studies that directly examine how SWD and ELL teachers experience policies in comparison with general educators and how their use of standards in the classroom differs from other teachers. We also compare urban, suburban, and rural teachers, given economic and geographic differences that may influence the policy environment (Johnson & Howley, 2015). For example, rural districts face unique administrative challenges in standards implementation; they enact policy across wider geographic areas with fewer staff and resources (Yettick, Baker, Wickersham, & Hupfield, 2014). Thus, we would expect the policy system to be different in rural compared to other environments. Finally, we expect there to be significant grade-level differences as elementary schools have historically been more receptive and amenable to reform (Schueler, Goodman, & Deming, 2017).

State Background

The three states in this analysis—Kentucky, Ohio, and Texas—are diverse both geographically and in terms of their approach to CCR standards. These three states include both former CCSS adopters and a non-CCSS state. Kentucky and Ohio were early adopters of CCSS, while Texas did not adopt CCSS, instead choosing to develop state-specific CCR standards. Further policy differences are described in Table 1, including the relative rigor of each state’s standards according to Carmichael, Martino, Porter-Maggee, and Wilson (2010), where Texas is considered to have the highest rigor in both subjects. Ultimately, these differences between state standards are quite small as there is a core curriculum across all three states (Porter, McMaken, Hwang, & Yang, 2011). Our outcome of interest—standards-emphasized instruction—measures the difference in the current standards from the state’s prior iteration to identify which content areas should have shifted the most.

Table 1

Demographic and Policy Information on Three Partner States

Demographic Information or Policy Information	Kentucky	Ohio	Texas
PreK–12 enrollments	690,634	1,754,191	4,934,366
Number of school districts	173	955	1,239
Number of public schools	1,449	3,758	8,732
Number of public school teachers	41,588	109,282	334,997
Unadjusted education spending per student	$10,426	$11,224	$8,788
Percentage of minority students	21.0	25.8	68.8
Percentage of students eligible for free or reduced-price lunch	60.0	42.5	50.1
Percentage of students with disabilities	13.3	14.8	9.0
Percentage of English language learners	3.6	2.1	15.0
Percentage of students in city schools	21.5	18.9	42.3
Percentage of students in suburban and town schools	43.2	59.0	42.1
Percentage of students in rural schools	35.3	22.2	15.6
Rigor of ELA/math standards	3 (ELA) 2 (Math)	4 (ELA) 3 (Math)	6 (ELA) 4 (Math)
Adoption and implementation of CCR standards	Kentucky Academic Standards were adopted in 2010 and fully implemented in the 2011–2012 school year.	Ohio’s Learning Standards were adopted in 2010 and fully implemented in the 2013–2014 school year.	Texas Essential Knowledge and Skills (TEKS) were first adopted in 1997. The original TEKS in ELA and math were fully implemented in the 1998–1999 school year. The most recent revisions to the ELA TEKS were first implemented in the 2009–2010 school year. The most recent revisions to the Math TEKS were fully implemented in the 2014–2015 school year.
Statewide administration of CCR-aligned assessment	Kentucky Performance Rating for Educational Progress (K-PREP) was first administered in spring 2012.	The Ohio Achievement Assessment was first administered in spring 2015. Ohio originally was in the PARCC consortium—it withdrew in 2015.	The Texas Assessment of Knowledge and Skills (TAKS) was first administered in 2003. TAKS was phased out beginning in 2012 and was replaced by the State of Texas Assessment of Academic Readiness (STAAR). By 2015, all of the students in the state of Texas were taking the STAAR.

Note. ELA = English language arts; CCR = college- and career-readiness.

We focus on ELA and math because they are the two subjects for which all states have standards, the only subjects covered by CCSS, and the most tested subjects; thus, we would expect them to be most influenced by standards-based policy. We use a state-representative sample in both elementary and high schools. This design allows for grade-level instructional contrasts and enables us to highlight the differences between these two extremes as high schools are less prevalent in standards implementation studies (Polikoff, 2017). Resource constraints prevented us from including middle school; we consider this an important area to add to future research.

It is important also to note the inherent difficulty in studying standards, which may often be loosely coupled from classroom instruction (Weick, 1976). Polikoff (2017) notes the lack of causal evidence concerning the CCSS and their effectiveness; the most compelling causal studies address earlier eras of standards-based reform centered on No Child Left Behind (e.g., Dee & Jacob, 2011). To address these empirical concerns, we employ a well-tested theoretical framework, described in the following.

The Policy Attributes

In designing the teacher survey, we used the policy attributes framework and its five dimensions—specificity, authority, consistency, power, and stability—as well as resources, challenges, and professional development to quantify aspects of the policy environment (Porter, Floden, Freeman, Schmidt, & Schwille, 1988). The policy attributes framework (Porter, 1994; Porter et al., 1988) has been used for decades in education policy research to analyze both systemic reform efforts (Clune, 1993) and comprehensive school reforms (Berends, Bodilly, & Kirby, 2002; Desimone, 2002; Polikoff, 2012). Studies of standards efforts in the past decade have described state and district initiatives as well as how teachers perceive policy (Spillane, 1999; Stecher et al., 2008), but few have provided systematic data to compare experiences across states or linked policy perceptions and key inputs to teachers’ reported instructional changes. The policy attribute theory is particularly well-suited to examining the complexity of standards-based policy simultaneously, reflecting each key component of reform that must work in concert to achieve change (M. S. Smith & O’Day, 1991). Examining these components simultaneously is crucial to capturing the reality of how teachers experience an intricate web of policies originating from federal, state, and local governing bodies (Coburn, 2016; Marsh & Wohlstetter, 2013; Spillane, 2009). Recent studies have also deployed the attributes to understand complex systems such as research-practice partnerships, including how to develop and evaluate appropriate outcomes (Desimone, Wolford, & Hill, 2016) and how to evaluate school turnaround (K. Hill, Desimone, Wolford, & Reitano, 2017), which shows the continuing relevance of the theory to analyzing policy implementation.

Using this theory, we measure the policy environment by describing how specific, authoritative, consistent, powerful, and stable it is. Specificity describes how detailed or prescriptive a policy is. Authority reflects a policy’s legitimacy and status, which can be achieved through rules or law, historical practice, or charismatic leaders. Consistency is the extent to which policies are aligned and how policies relate to and support each other. Power reflects how policies are reinforced and enacted through systems of rewards and sanctions. Policies that have power include incentives for compliance. Stability is the extent to which policies change or remain constant over time.

To better contextualize these attributes, we describe examples of the most relevant standards policy issues of the day and how the attributes relate to them based on the survey items in the Appendix. Specificity is most salient in current debates over developing a coherent curriculum program (see Kaufman et al., 2018); it addresses the extent to which districts provide teachers with a prescriptive curriculum and accompanying resources. Consistency refers to alignment, addressing a longstanding concern in standards-based reform that tests and curricula are poorly aligned. Authority refers to the intrinsic belief that teachers hold concerning standards, whereas power refers to extrinsic motivators enacted through accountability policies such as teacher evaluations (Firestone, 2014). Stability is perhaps the simplest to explain, answering the question of whether teachers think the standards and associated policies will last. All of the attributes must work in concert to achieve the difficult task of changing the technical core of teaching over the long term (Spillane, Parise, & Sherer, 2011).

It is important also to consider whether the desire to change teacher instruction is the right focus. Polikoff and Porter (2014) argue that instructional alignment to standards is weakly associated with teacher quality, while raising teacher quality has been the subject of much recent policy research (e.g., Desimone, Hochberg, & McMaken, 2016; Stecher et al., 2018). This study is a departure from these approaches and is agnostic about relationships between standards-emphasized instruction and student outcomes—though clearly all of these efforts are aimed to help students. Rather, we ask more fundamentally whether standards are associated with teacher self-reported instruction as this is the first step toward finding any future relationships to or effects on student outcomes. Without the presence of these relationships, the entire theory of standards-based reform as enacted over the past 25 years seems on shaky ground (Mehta, 2013).

We readily acknowledge the many tradeoffs of different approaches to measuring policy and our primary outcome of interest, standards-emphasized instruction. Document analysis reflects “true” policies as written; however, we believe teachers understanding and “lived” experiences with policy are the conditions that are most potentially influential in changing behavior (Coburn, 2016; Desimone, 2002; H. C. Hill, 2001; Spillane, 2009). Further, we understand validity threats to teacher self-report (Burstein et al., 1995; Desimone, 2006, 2013; Desimone, Smith & Frisvold, 2010) and interpret our data with this in mind. But, we also constructed our measures to ensure teachers were not aware of the desired reports and were reporting on behaviors (as opposed to evaluating their own instruction). Here the attributes helped create reliable constructs without alerting teachers to the desired responses. We also administered the surveys in a confidential and nonevaluative environment. Under these conditions, teacher self-report data have been shown to be valid and reliable (Fowler, 2013; Mayer, 1999). Observation can measure quality in ways limited by survey data, but it is labor intensive, time-limited, and subject to rater bias. Our study, while correlational, begins to address the question of whether CCR standards might be shifting instruction, and it examines which elements of the policy environment play the most significant role in predicting these shifts.

Resources, Challenges, and Professional Development

In addition to the attributes, we measure three key factors in teachers’ policy environment: resources, challenges, and professional development (PD). In designing the survey, we identified the challenges and resources that are most often cited in the literature. Instructional resources play a critical role in moderating teachers’ use of standards in the classroom by providing aligned curricula, scope-and-sequence documents, and other resources for lesson planning (Ball & Cohen, 1996; Raudenbush, Rowan, & Cheong, 1992). While relationships between student achievement and resources can be inconsistent or weak (Hanushek, 1997), standards themselves may be boosting resource investments, as they did during prior periods of reform (Dee, Jacob, & Schwartz, 2013).

But even in schools where teachers receive robust resources, challenging working conditions can stymie teachers’ best efforts to implement standards-based reform. Challenges—such as student absenteeism and mobility (Allensworth & Easton, 2007), a wide range of student abilities presenting challenges for differentiation (Fuchs & Fuchs, 2006), a lack of administrative or parental support, and frequent changes in policies and school priorities (Coburn, 2006)—can interfere with teachers’ ability to leverage resources and professional development. We considered all of these challenges in our survey design.

Finally, professional development remains the central mechanism for fostering standards-emphasized instruction. Ongoing, interactive, content-focused, collaborative, and coherent forms of PD (Desimone, 2009) mobilize teachers to learn and refine ways of using available resources that best meet the needs of their students. Coaching, one prominent form of professional development, has been shown in many circumstances to be effective at raising achievement (Desimone & Pak, 2017; Kraft, Blazar, & Hogan, 2018). PD provides teachers with the sense-making opportunity to engage with the standards and their available resources in their zones of enactment, which are the spaces in which “reform initiatives are encountered by the world of practitioners and ‘practice,’ delineating that zone in which teachers notice, construe, construct and operationalize the instructional ideas advocated by reforms” (Spillane, 1999, p. 144). While links between PD and instruction are mixed (Desimone, Smith & Phillips, 2013; Kennedy, 2016), we expect that high-quality PD focused on content standards would boost teachers’ standards-emphasized instruction (Penuel, Fishman, Yamaguchi, & Gallagher, 2007).

The framework in Figure 1 illustrates the overarching model that guides our work. It depicts the relationship between the policy attributes, resources, challenges, professional development, and instruction. The policy attributes both determine and are determined by resources and challenges, as shown in the bidirectional arrows. State policy contexts and geography influence all of the elements of the framework, so we estimate each state analysis separately rather than pooling across states, which leverages our stratified sampling design. The outcome, what we label standards-emphasized instruction, is the critical objective of any attempt at standards-based reform. Ultimately, if policy does not shift instruction for students, the standards will not matter much, and the vision of standards-based reform as laid out by its earliest advocates will not be realized. Standards-based policy by design is meant to foster organizational coherence and strengthen the technical core of instruction (Spillane et al., 2011).

Figure 1.

Framework for standards-emphasized instructional shifts.

We examine the following research questions:

Research Question 1: How do teachers perceive their policy environment? That is, how do they perceive the policy attributes of specificity, consistency, authority, power, and stability and their resources, challenges, and PD related to CCR standards? Specifically, how are they similar or different across states; for teachers of ELA, math, SWDs, and ELLs; and for teachers in urban, suburban, and rural schools?

Research Question 2: To what extent are teachers covering content emphasized in their state’s CCR standards, and how does coverage of this standards-emphasized content differ for teachers of ELA, math, SWDs, and ELLs and rural, suburban, and urban teachers?

Research Question 3: To what extent do teachers’ policy environments predict the degree of standards-emphasized instruction?

The null hypotheses we are testing is that there are no significant differences between states and teacher types and no relationships between policy attributes, resources, challenges, and PD to the degree of standards-emphasized instruction. Rejecting the null in any of these instances—that is, finding significant relationships between states, teachers, policy attributes, resources, challenges, PD, and instruction—will have broad relevance to future standards-based reform implementation efforts.

Hypothesized Policy Differences Among States, Districts, and Teacher Types

Our first research question assumes that different teachers—separated by state, subject, grade level, and urbanicity—experience policy differently. In addition to distinct state policy environments, we look at each subject area differently because of the level of change demanded by the new standards, which varies by subject, as well as past research showing different outcomes for math compared to English (Dee & Jacob, 2011; Porter et al., 2011). Elementary math teachers may also react differently to directives asking them to cover fewer topics more rigorously than secondary ELA teachers, who may be asked to abandon long-taught units centered on favorite works of fiction (Achieve, 2013).

We expected PD in rural areas to be of lesser quality because of resource, personnel, and distance constraints, which would lead to less standards-emphasized instruction (Glover et al., 2016). We test whether policy perceptions hold constant across rural, suburban, and urban districts, and we use measures of teacher judgments of the usefulness of their PD when quantifying teacher professional development. We also expected policy differences to emerge when comparing general education teachers to teachers of SWDs or ELLs. Prior studies of reform documented teachers of SWDs as skeptical of the applicability of common standards to their students (Dorn, Fuchs, & Fuchs, 1996), and qualitative studies have found skepticism concerning the CCSS for both SWDs and ELLs (e.g., Figueroa-Murphy & Haller, 2015).

Data

We selected states to achieve variation in the type of standards they adopted and include both Common Core and state-specific standards. Kentucky is our smallest state by population, with fewer than 1 million students enrolled compared to nearly 2 million students in Ohio and 5 million students in Texas. Kentucky also has far fewer districts, only 173, compared to Texas’s 1,239. It is also a far more rural state, with over a third of its P–12 population in rural areas. Texas is a more urban state with a much higher population of English language learners and non-White students. It spends far less per student, with Ohio spending the most of the three states. These demographic details are described in depth in Table 1.

College- and career-readiness standards implementation deadlines were also quite different across the three states, with Texas implementing its own standards first, while both Ohio and Kentucky adopted and then dropped or revised the CCSS on different timelines. Kentucky fully implemented its standards far more quickly than Ohio. Considering these substantial state policy differences, any similar findings across all three states would be all the more notable.

The study has a multistage sampling design, with districts selected with probability proportional to the square root of student enrollment size (PPS), schools selected with equal probability within the elementary school group and the high school group, and teachers selected with equal probability within each teacher group. We employed a stratified random sampling technique designed to ensure the sample was representative of districts in Texas, Ohio, and Kentucky. We included 42 Texas districts, 42 Ohio districts, and 89 Kentucky districts in the sample (there was a larger number in Kentucky because of integration with ongoing state survey data collection). In each district, we sampled up to two elementary schools and two high schools, making sure to capture representative samples of traditional public and charter schools based on demographics. In each elementary school, we sampled two fifth-grade math teachers, two fourth-grade ELA teachers, one teacher of SWDs, and one teacher of ELLs. In each high school participating in the study, we sampled two ELA teachers and one teacher in each of the following specialties or subjects: SWD, ELL, Algebra I, Algebra II, and geometry. We chose the three math subjects because they are the most common high school math courses in order to maximize the number of high school target course responses we obtained. Further, we wanted to identify math classes enrolling students who were likely to be required to take the state mathematics assessment. Of the eligible teachers, in Ohio, 417 of 654 sampled teachers responded (conditional response rate: 64.8%); in Texas, 603 of 1,089 (55.3%); in Kentucky, 740 of 1,890 (39.2%).

We examined patterns of nonresponse in each state for areas of concern—district size, district type, urbanicity, school type (elementary, high school, charter, and non-charter), student demographics (free and reduced-price lunch, race, ethnicity, and subject area)—and we found between 0% and 8% difference in respondent teachers’ school and district characteristics compared to the selected sample along each of these dimensions. In most cases, there were no significant differences between the target sample and the respondent percentages (the largest difference being in Ohio, where districts with only one high school were 8% less likely to respond). To correct for remaining response bias, we used a technique in Stata known as jackknife resampling, which systematically leaves out each observation and calculates the averages to make the estimates more accurate. We also weighted each district accordingly and report robust standard errors obtained from this procedure. For more information on our sampling size and methods, please refer to the Center on Standards, Alignment, Instruction, and Learning (C-SAIL) at c-sail.org.

Methods

Using survey jackknife procedures, we weighted all survey results based on state demographics (i.e., each response is weighted by district to accurately represent the population of the state). To compare teacher reports of their policy environment (Research Question 1), we used Wald tests of significance across teacher subgroups (by subject and geography) within the same state to establish significant differences. To compare across states, we used an ANOVA test of state summary data with a Tukey post hoc correction. We constructed the policy attributes measures by averaging the items, and we report the internal reliability for each attribute using Cronbach’s α. We used multi-item composites for each of our measures to increase reliability and validity (Mayer, 1999). We borrowed items on already validated national surveys where possible, and when we needed to create or adapt items, we went through a cycle of development, expert review, cognitive interview, and review (Desimone & LeFloch, 2004). To analyze geographic differences, we coded each district as rural, suburban, or urban using the district database from the National Center for Education Statistics, and we combined rural and town into a single category.

To determine the association between policy and aligned instruction (Research Question 3), we examined the variance between schools, which was significant for three out of the five attributes, indicating that we should use a hierarchical linear model (HLM) rather than an ordinary least squares regression. We use an HLM approach with school-level random effects, where y is the standards-emphasized content (explained in more detail in the following), α is the constant, and ε is the residual:

\begin{array}{l} y = α + β_{1} (S p e c i f i c i t y) + β_{2} (A u t h o r i t y) + β_{3} (C o n s i s t e n c y) + β_{4} (P o w e r) + β_{5} (S t a b i l i t y) \\ + β_{6} (P D U s e f u l n e s s \times P D h o u r s) + β_{7} (R e s o u r c e s) + β_{8} (C h a l l e n g e s) + ε . \end{array}

Measures

All of our measures of the policy attributes consisted of composite averages of multiple items listed in the Appendix. To measure specificity, we asked teachers their level of agreement with statements related to how detailed guidance from the district was around standards implementation, where 1 = strongly disagree, 2 = somewhat disagree, 3 = somewhat agree, and 4 = strongly agree. For specificity, α values ranged from 0.74 to 0.92, depending on the state. For authority, we asked teachers the extent of their agreement with statements that reflected their buy-in to the standards, such as if they thought the standards made learning relevant, if the standards were appropriate for their students, and if they gave them the flexibility they needed to help students below grade level (α range, 0.84–0.89). For consistency, we asked the degree to which teachers believed curricula, assessments, professional development, evaluations, and other policies were aligned (α range, 0.87–0.90). For power, we asked teachers about positive and negative repercussions for implementing the standards (α range, 0.65–0.66). For stability, we asked them to predict how long the standards and assessments would last in each state (α range, 0.77–0.86). These constructs were not highly correlated (the strongest correlation was only 0.32 between specificity and consistency), and these constructs have been significantly predictive in prior work (e.g., Polikoff, Porter, & Smithson, 2011).

The full list of the resources, challenges, and professional development survey items can be found in Table 2. To measure challenges, we provided a list of 10 school, classroom, and student challenges, generated from the literature, including lack of support from parents (Fan & Chen, 2001), large class size (Hanushek & Woessmann, 2017), and lack of planning time built into the school day (Dever & Lash, 2013), to name a few. We asked teachers to rate them as either not a challenge, a minor challenge, a moderate challenge, or a major challenge (on a scale of 1–4). To measure resources, we created a composite measure of the instructional resources teachers used and found useful. We selected these items based on those we thought would most influence the instructional core and teachers’ interpretation of standards-based policy (Spillane, 2009). To determine the dosage of standards-focused professional development, we averaged the five types of PD in Table 2 related to content standards as well as their usefulness as reported on a 4-point Likert scale (1 = not at all useful, 2 = somewhat useful, 3 = useful, 4 = very useful). We then multiplied this average (from 1 to 4) by the number of types of standards-based PD teachers reported receiving, which was organized into increments as a continuous variable (on a 1–5 scale, where 1 = 1–10 hours, 2 = 11–20 hours, 3 = 21–40 hours, 4 = 41–80 hours, and 5 = 81 or more hours). The result is the independent variable called simply PD, which captures both the usefulness and the dosage of standards-focused PD received (a multiplier of two categorical variables). Thus, a teacher who reported 41 to 80 or more hours of PD that was not at all useful would have a PD value of 4 (4 × 1), whereas a teacher who reported 1 to 10 hours of very useful PD would also have a PD value of 4 (1 × 4).

Table 2

Resources, Challenges, and Professional Development Survey Items

Resources (0–4 scale)	Challenges (1–4 scale)	PD (0–20 scale)
Textbooks aligned to CCR standards	Inadequate student preparation in prior grades	Content of the CCR Standards
Curriculum resources aligned to CCR standards	Lack of support from parents	Instructional shifts associated with the CCR standards
Formative or diagnostic assessments aligned to CCR standards	Student absenteeism and tardiness	Test preparation strategies for the CCR standards
Digital tools (e.g., online textbooks, webinars, videos, online communities, applications)	Insufficient class time to cover all the content	Instructional strategies for teaching to the CCR standards for SWDs
Information about how CCR standards change what students are expected to learn	Wide range of student abilities	Instructional strategies for teaching to the CCR standards for ELLs
Information about how CCR standards change what is expected of teachers’ instructional practice	Large class size
Professional development on CCR standards	Inadequate instructional resources (e.g., textbooks)
	Frequent changes in school priorities or leadership (e.g., principal turnover)
	Lack of school resources to provide extra help for students
	Lack of planning time built into the school day

Note. PD is a categorical variable representing the number of hours of professional development received multiplied by its usefulness, with a maximum possible score of 20 (indicating more than 80 hours of very useful professional development). CCR = college and career readiness; SWDs = students with disabilities; ELLs = English language learners.

Standards-Emphasized Instruction

To measure standards-emphasized instruction, we started with content analyses of state standards in our partner states prior to the implementation of No Child Left Behind; we refer to this as pre-CCR standards. Trained content-expert raters coded the pre-CCR standards as covering topics and cognitive demands; the results were what teachers in these states should have been teaching before the CCR standards. We compared these pre-CCR results to content analyses of the current CCR standards in each state. We determined the particular content that saw the greatest average proportional increase at each grade level from pre-CCR to CCR standards, and we called this the emphasized instructional content. The de-emphasized instructional content was the content that saw the greatest decrease in coverage from pre-CCR to CCR standards. The list of emphasized and de-emphasized content by subject area can be found at c-sail.org.

This approach is known as a modified version of the surveys of enacted curriculum (SEC) approach (Porter, 2002). We asked teachers to report on the topics and cognitive demands that they covered. We then mapped content onto the topics (e.g., adding fractions) and cognitive demands (e.g., memorize, problem solve) from each state’s standards using trained content experts. This technique has been used and shown to be valid and reliable in multiple studies (Blank, 2004; Osthoff, 2007; Polikoff et al., 2011; Rowan, Camburn, & Correntti, 2004; Schmidt et al., 2001; Webb, 2002, 2007).

Findings

Research Question 1: How Do Teachers Perceive Their Policy Environment (Specificity, Consistency, Authority, Power, Stability, Resources, Challenges, and PD) Related to CCR Standards?

As shown in Table 3, scores closer to 2 (somewhat disagree on the majority of the items) versus 3 (somewhat agree) indicate perception gaps that should concern policymakers and states looking to bolster the strength of those attributes. We describe the significant differences in specificity, consistency, and authority between states, followed by the nonsignificant differences for power and stability.

Table 3

Teacher Policy Attribute Means Across States, by Subject and Geography

Attribute	Texas	Mean	Ohio	Mean	Kentucky	Mean	F Value	Significance
Attribute	n	Mean	n	Mean	n	Mean	F Value	Significance
Specificity	585	3.14** (0.07)	405	2.38** (0.13)	436	2.75** (0.04)	21.02	.00
Authority	583	2.56** (0.05)	402	2.30** (0.04)	430	2.78** (0.03)	26.59	.00
Consistency	564	2.82(0.04)	379	2.71* (0.04)	390	2.91(0.03)	3.86	.00
Power	586	2.68(0.07)	405	2.50(0.06)	428	2.56(0.03)	2.45	.09
Stability	579	2.51(0.07)	398	2.44(0.06)	400	2.29(0.06)	2.85	.06
Texas	ELA	Mean		Math	Mean	SWD	Mean	ELL	Mean
Texas	n	Mean		n	Mean	n	Mean	n	Mean
Specificity	207	3.16(0.12.)		250	3.24(0.09)	78	2.87* (0.13)	49	3.20(0.13)
Authority	207	2.70(0.09)		248	2.52(0.05)	79	2.29** (0.12)	49	2.80(0.10)
Consistency	202	2.85(0.09)		243	2.81(0.04)	71	2.80(0.07)	48	2.78(0.12)
Power	207	2.73(0.10)		250	2.65(0.09)	79	2.61(0.07)	50	2.67(0.08)
Stability	203	2.43(0.13)		249	2.61(0.07)	78	2.45(0.20)	49	2.50(0.19)
Ohio	ELA	Mean		Math	Mean	SWD	Mean	ELL	Mean
Ohio	n	Mean		n	Mean	n	Mean	n	Mean
Specificity	157	2.38(0.21)		166	2.40(0.14)	67	2.35(0.14)	15	2.23(0.41)
Authority	157	2.43(0.08)		163	2.24(0.06)	66	2.17 * (0.10)	16	2.60(0.36)
Consistency	146	2.80(0.05)		158	2.57* (0.07)	59	2.83(0.10)	16	2.70(0.39)
Power	157	2.50(0.08)		166	2.44(0.07)	67	2.57(0.10)	15	2.68(0.28)
Stability	153	2.44(0.08)		166	2.47(0.12)	64	2.39(0.13)	15	3.09* (0.29)
Kentucky	ELA	Mean		Math		Mean	SWD		Mean
Kentucky	n	Mean		n		Mean	n		Mean
Specificity	185	2.65(0.04)		150		2.70(0.08)	95		2.95** (0.09)
Authority	185	2.90(0.04)		147		2.63** (0.04)	92		2.74(0.05)
Consistency	167	2.92(0.04)		129		2.81** (0.05)	91		3.02(0.05)
Power	180	2.52(0.04)		148		2.49* (0.05)	95		2.69(0.07)
Stability	169	2.33(0.09)		133		2.15(0.07)	93		2.29(0.12)
Texas	Rural	Mean		Suburban		Mean	Urban		Mean
Texas	n	Mean		n		Mean	n		Mean
Specificity	102	2.92* (0.22)		134		3.39(0.08)	349		3.14(0.10)
Authority	100	2.55(0.12)		133		2.58(0.11)	350		2.60(0.04)
Consistency	97	2.87(0.14)		130		2.82(0.08)	337		2.79(0.04)
Power	102	2.55* (0.09)		134		2.60(0.15)	350		2.78(0.09)
Stability	100	2.38(0.14)		134		2.61(0.22)	345		2.55(0.05)
Ohio	Rural	Mean		Suburban		Mean	Urban		Mean
Ohio	n	Mean		n		Mean	n		Mean
Specificity	119	2.10(0.18)		183		2.40(0.18)	103		2.77(0.30)
Authority	118	2.14* (0.07)		182		2.36(0.06)	102		2.41(0.08)
Consistency	110	2.61** (0.06)		173		2.83(0.05)	96		2.63(0.08)
Power	119	2.29** (0.07)		183		2.58(0.06)	103		2.66(0.15)
Stability	118	2.33(0.17)		180		2.50(0.06)	100		2.49(0.09)
Kentucky	Rural	Mean		Suburban		Mean	Urban		Mean
Kentucky	n	Mean		n		Mean	n		Mean
Specificity	294	3.07(0.11)		78		2.55** (0.09)	64		2.68(0.05)
Authority	291	2.76(0.07)		77		2.75(0.05)	62		2.79(0.03)
Consistency	267	2.81* (0.07)		77		2.86(0.05)	62		2.97(0.03)
Power	289	2.63(0.06)		75		2.38* (0.10)	64		2.58(0.04)
Stability	273	2.40(0.12)		68		2.33(0.12)	59		2.23(0.05)

Note. Numbers in parentheses are standard errors. Significant differences indicate if the lowest mean was statistically significant with the highest group in the row. If all groups are marked as significant, all comparisons are significant. ELA = English language arts; ELL = English language learner; SWD = students with disabilities. *p < .05. **p < .01.

Teachers across all three states differed significantly in reporting the specificity of their standards environment, with Texas being highest (3.14), followed by Kentucky (2.75), and then Ohio (2.38). Ohio’s lower scores indicate that teachers there are significantly less likely to agree that districts have provided guidance on how much time to spend on content areas, the order they should teach content, and whether the standards clearly indicate what content to teach.

The states significantly differed on authority as well, with Kentucky being the highest (2.78), followed by Texas (2.56), and then Ohio (2.30). Kentucky teachers on average believed that the standards were somewhat appropriate, rigorous, and flexible for all students, whereas Texas and Ohio teachers were more ambivalent.

For consistency, Kentucky again was the highest at (2.91), followed by Texas (2.82), and then Ohio (2.71). But only Ohio’s and Kentucky’s consistency scores were significant, a difference of only 0.20. Consequently, we do not interpret this difference as particularly meaningful. All three states’ teachers perceive curricula, assessments, professional development, and administrative feedback to be aligned to their states’ CCR standards.

We found no statistically significant differences in how teachers rated the power of their policy environments, with Texas at 2.68, Kentucky at 2.56, and Ohio at 2.50. It may be the case that the frozen nature of accountability systems in 2016, with districts and teachers held harmless during the implementation of new assessments and systems until the approval of new ESSA accountability plans, kept power scores hovering around 2.5. Respondents were between somewhat agree and somewhat disagree in their opinion that they were being punished or rewarded for implementing the standards.

We found no statistically significant across-state differences in stability either. Stability scores were highest in Texas (2.51), followed by Ohio (2.44), and then Kentucky (2.29), with a score of 2 representing 3 years and a score of 3 representing 4 years. Most respondents expected the standards, tests, and current proficiency cut scores to last between 3 and 4 years.

We next tested resources, challenges, and professional development across states and include these results in Table 4. Texas teachers reported significantly higher useful resources (2.02), followed by Ohio (1.86), and then Kentucky (1.80). As for challenges, we found no statistically significant differences across states in Table 4 and thus focus on the average for the top five challenges in all three states, shown in Figure 2. The largest challenge by far was a wide range of student abilities, which 71% of teachers reported as a major or moderate challenge, followed by inadequate student preparation in prior grades (62%) and a lack of support from parents (59%).

Table 4

Resources, Challenges, and Professional Development (PD) by Teachers’ Geography

	Resources n	Mean	Challenges n	Mean	PD n	PD Mean
Texas rural	98	1.96(0.06)	102	2.20(0.10)	88	6.20(0.64)
Texas suburban	130	2.16(0.08)	136	2.26(0.09)	123	6.46(0.35)
Texas urban	335	2.00(0.06)	351	2.42(0.06)	313	5.70(0.39)
Texas all	563	2.03(0.04)	589	2.32(0.04)	524	6.02(0.25)
Ohio rural	109	1.75* (0.07)	119	2.34(0.06)	87	4.73(0.45)
Ohio suburban	169	1.96(0.05)	185	2.19(0.08)	145	4.88(0.43)
Ohio urban	96	1.84(0.09)	103	2.50* (0.04)	83	5.03(0.32)
Ohio all	374	1.86(0.04)	407	2.31(0.04)	315	4.87(0.28)
Kentucky rural	59	1.66* (0.10)	65	2.51** (0.07)	41	4.16(0.64)
Kentucky suburban	66	1.73(0.08)	77	2.26(0.06)	54	3.84(0.41)
Kentucky urban	248	1.99(0.04)	296	2.25(0.03)	203	3.95(0.17)
Kentucky all	373	1.80(0.04)	438	2.32(0.03)	298	3.98(0.23)

Note. Numbers in parentheses are standard errors. Significant differences indicate if the lowest mean was statistically significant with the highest of the remaining two groups. *p < .05. **p < .01.

Figure 2.

Cross-state percentages of moderate or major teacher challenges.

There were, however, significant and large differences in terms of the professional development teachers received across states. The largest possible PD score was 20 (5 types of standards PD × 4 = very useful), with Texas ranking the highest at 6.02, Ohio ranking second at 4.87, and Kentucky ranking last at 4.00. This finding suggests that Texas may have professional development practices worth emulating.

Research Question 1: To What Extent Are Teacher Perceptions of Their Policy Environment Similar or Different for Teachers of ELA, Math, SWDs, and ELLs?

We report unstandardized effect sizes to match the Likert scale—an effect size of 1.00 would mean a change from one category to the next (i.e., somewhat agree [3] becomes [4] strongly agree). In Table 3, only a few comparisons were significant with small effect sizes (ES). Specificity was significantly lower for SWD teachers than math teachers in Texas (2.87, ES = 0.37) but significantly higher for SWD teachers compared to math teachers in Kentucky (2.95, ES = 0.25). The authority scores for teachers of SWDs in Texas (2.29, ES = 0.41) and in Ohio (2.17, ES = 0.26) were significantly lower compared to ELA scores in those states. Math teachers experienced significantly lower consistency in Ohio (2.57, ES = 0.23) compared to ELA teachers. Scores below 2.5, which means responses skewed toward “disagreeing” that their policy environments were specific, consistent, authoritative, powerful, and/or stable, represent areas where state and district implementation efforts could be strengthened.

We found no statistically significant differences in the resources that different types of teachers received. Figure 3 details teacher subgroups that experienced significantly different challenges compared to general educators. Additional tables comparing challenges can be found at c-sail.org, but there were no clear patterns across the three states.

Figure 3.

Significantly different students with disabilities (SWD) and English language learner (ELL) challenges compared to general educators (math and English language arts [ELA]).

We also found a few statistically significant differences in the types of professional development that teachers received. More information on the professional development received by teachers of SWDs compared to general education teachers (who consistently received significantly less PD on how to help SWDs attain grade-level standards) can be found at (Edgerton, Fuchs, & Fuchs, in press).

Research Question 1: To What Extent Are Teachers’ Policy Environments Different Based on Whether They Teach at Urban, Suburban, or Rural Schools?

Next, we considered whether policy environments differed by urbanicity in Table 3. We hypothesized that rural teachers would report having lower levels of the policy attributes, and this was often the case. Among teachers within the same state, there were many more significant differences based on geography compared to subject-area differences, as shown in Tables 3 and 4 and described in the following. Specificity, authority, and power were all seen as significantly lower in rural areas in both Texas and Ohio compared to the highest group, which was either suburban or urban. In Kentucky, rural teachers reported significantly lower specificity than urban teachers. We report the significant results only, in order from highest to lowest.

In Texas, suburban teachers had significantly higher specificity (3.39), followed by urban (3.14), and then rural (2.92) teachers. Power was significantly higher among urban teachers (2.78), followed by suburban (2.60), and then rural (2.55).

In Ohio, urban teachers had significantly higher authority (2.39), followed by suburban (2.35), and rural (2.24); and power (2.66 for urban teachers, 2.58 for suburban, and 2.29 for rural). Suburban teachers in Ohio reported significantly higher consistency (2.83), followed by urban (2.63), and rural (2.61). In Kentucky, specificity was significantly higher among urban teachers (3.01), followed by rural (2.69), and then suburban (2.62).

Though we found no significant differences by teachers’ geography for professional development, we found significant differences in terms of resources and challenges. In Table 4, we demonstrate that in Ohio, rural teachers received statistically fewer useful resources than suburban teachers (ES = 0.21), and Kentucky rural teachers received statistically fewer useful resources than urban teachers (ES = 0.33). Also in Ohio, urban teachers faced significantly greater challenges than suburban teachers (ES = 0.31), while Kentucky rural teachers faced significantly greater challenges than both urban and suburban teachers (ES = 0.25–0.26).

Research Question 2: To What Extent Are Teachers Covering Content Emphasized in Their State’s CCR Standards? How Does Coverage of Standards-Emphasized Content Differ for Teachers of ELA, Math, SWDs, and ELLs and for Rural, Suburban, and Urban Teachers?

In an ideal policy environment, we would expect to see significantly more coverage of the standards-emphasized content when significance tested against the standards de-emphasized content. As Table 5 shows, we found that in all three of our study states, opposite of our expectations, ELA elementary teachers covered significantly less of the standards-emphasized content. At the high school level, the inverse was true: In Texas and Ohio, ELA high school teachers covered significantly more of the standards-emphasized content, though in Kentucky, there was no significant difference.

Table 5

Comparing Standards-Based Content Emphasis by Subject and Geography

Texas	ELA				Math				SWD				ELL
Texas	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference
ELA elementary	116	3.39(0.05)	3.68(0.03)	−.29**					24	3.09* (0.08)	3.48(0.01)	−.39*	18	3.44(0.11)	3.69(0.08)	−.25
ELA high school	92	3.59(0.06)	3.30(0.06)	.29**					16	3.19(0.17)	2.83(0.17)	−.39	23	3.54(0.17)	3.12(0.22)	.42
Math elementary					138	3.41(0.05)	2.95(0.03)	.46**	26	3.06(0.21)	2.79(0.11)		8	3.57(0.09)	2.97(0.11)	.60**
Math high school					52	3.35(0.08)	3.35(0.11)	.00	8	2.95* (0.25)	2.59* (0.19)	.36
		Rural			Suburban					Urban
ELA elementary	26	3.39(0.14)	3.73(0.06)	−.34*	38	3.32(0.09)	3.66(0.05)	−.34**	94	3.35(0.05)	3.62(0.04)	−.27
ELA high school	29	3.50(0.12)	3.19(0.11)	.31	30	3.71(0.10)	3.36(0.16)	.35	94	3.45(0.09)	3.15(0.13)	.30
Math elementary	21	3.49(0.12)	2.99(0.07)	.50**	41	3.37(0.16)	2.88(0.10)	.49	72	3.25(0.09)	2.89(0.04)	.36**
Math high school	11	2.95* (0.10)	2.87* (0.14)	.08	11	3.56(0.21)	3.38(0.19)	.18	72	3.33(0.12)	3.30(0.11)	.03
OH	ELA				Math				SWD				ELL
OH	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference
ELA elementary	62	3.41(0.08)	3.70(0.05)	−.29*					19	3.04* (0.11)	3.55(0.10)	−.41**
ELA high school	94	3.43(0.05)	3.08(0.05)	.35*					20	3.34(0.11)	3.03(0.11)	.31	13	3.45(0.13)	3.37(0.15)	.08
Math elementary					69	3.37(0.11)	2.57(0.04)	0.80**	10	2.50* (0.39)	2.08* (0.32)	.42
Math high school					49	3.12(0.06)	3.15(0.09)	−0.03	19	2.84(0.15)	2.73(0.20)	.11
		Rural				Suburban
ELA elementary	21	3.17* (0.08)	3.77(0.10)	−.60**	49	3.37(0.04)	3.63(0.04)	−.26**	15	3.33(0.09)	3.63(0.10)	−.30*
ELA high school	40	3.29* (0.07)	3.03(0.09)	.26*	48	3.55(0.08)	3.21(0.09)	.34**	39	3.41(0.07)	2.97(0.11)	.44**
Math elementary	21	3.30(0.21)	2.48(0.11)	.82**	41	3.16(0.16)	2.41(0.17)	.75**	17	3.11(0.23)	2.56(0.05)	.55**
Math high school	19	3.10	3.08	.02	29	2.96	2.98	−.02	30	3.12	3.02	.10
Math high school		(0.11)	(0.20)			(0.12)	(0.15)			(0.08)	(0.04)
KY	ELA				Math				SWD
KY	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference	N	Emphasized	De-Emphasized	Difference
ELA elementary	92	3.39(0.05)	3.49(0.04)	−.10					11	3.12(0.12)	3.60(0.07)	−.48*
ELA high school	102	3.20(0.05)	3.20(0.04)	.00					11	3.17(0.16)	3.29(0.18)	−.12
Math elementary					31	3.40(0.09)	2.68(0.09)	.72**	26	3.29(0.14)	2.76(0.12)	.53**
Math high school					105	3.07(0.05)	3.27(0.05)	−.20**	32	3.05(0.11)	3.07(0.12)	−.02
		Rural				Suburban				Urban
ELA elementary	19	3.20(0.14)	3.35(0.10)	−.15	30	3.23(0.13)	3.44(0.06)	−.21	55	3.44(0.05)	3.57(0.04)	−.13*
ELA high school	14	3.23(0.10)	3.28(0.09)	−.05	19	3.26(0.09)	3.25(0.07)	.01	80	3.12(0.05)	3.15(0.04)	−.03
Math elementary	15	3.42(0.17)	2.67(0.18)	−.75**	8	3.06(0.20)	2.39** (0.09)	.67**	39	3.39(0.12)	2.82(0.11)	.57**
Math high school	18	3.30(0.12)	3.42(0.12)	−.12	19	3.03(0.18)	3.08(0.23)	−.05	100	3.05(0.05)	3.21(0.05)	−.16*

Note. For n values <8, ELL teacher respondents are excluded. Numbers in parentheses are standard errors *Indicates significant differences for between emphasized/de-emphasized within subject (i.e., ELA teachers vs. SWD teachers on emphasized instruction), or between rural and at least one other geographic group, or for the difference between emphasized and de-emphasized in the “Difference” column. ELA = English language arts; ELL = English language learner; SWD = students with disabilities. *p < .05. **p < .01.

In every state, elementary math teachers covered significantly more emphasized content, which was the opposite of the ELA trend. At the high school level, only one state, Kentucky, showed any significant difference in math content coverage—and there, math teachers covered significantly less of the emphasized content.

In no state were differences between general education and ELL teachers significant, pointing to a promising trend of ensuring the same content for those students. The same was not true, however, for teachers of SWDs in Texas and Ohio, where teachers taught significantly less of the standards-emphasized content. In Texas, rural high school math teachers taught significantly less standards-emphasized content than suburban high school math teachers. In Ohio, rural ELA teachers taught significantly less standards-emphasized content at both the elementary and the high school levels compared to suburban teachers.

Research Question 3: To What Extent Do Teachers’ Policy Environments, Challenges, Resources, and PD Predict the Degree of Standards-Emphasized Instruction?

Incorporating the previous analyses as hypothesized predictors, Table 6 shows the results of our HLM. We find few significant predictors for any of our hypothesized independent variables, with the exception of authority. Overall, these findings suggest weak linkages between standards-emphasized instruction and four out of five policy attributes, resources, challenges, and professional development. However, authority is a positive significant predictor of emphasized content for ELA teachers in all three states (ES = 0.15, β = 0.23 in Texas; ES = 0.12, β = 0.20 in Ohio, and ES = 0.18, β = 0.24 in Kentucky). Resources also significantly and positively predicted standards-emphasized instruction among math teachers in Texas (ES = 0.26, β = 0.25).

Table 6

Results of Hierarchical Linear Regression Model for Standards-Emphasized Content by State and Subject Area

	Texas	Texas	Ohio	Ohio	Kentucky	Kentucky
	English Language Arts	Math	English Language Arts	Math	English Language Arts	Math
Elementary	−0.178** (0.052)	−0.027(0.103)	−0.151* (0.059)	0.245* (0.117)	0.133(0.083)	0.205(0.109)
Specificity	0.042(0.031)	0.049(0.064)	0.002(0.029)	−0.002(0.058)	−0.034(0.053)	−0.086(0.065)
Authority	0.147** (0.045)	0.001(0.069)	0.123* (0.049)	0.002(0.098)	0.182* (0.088)	0.034(0.107)
Consistency	0.036(0.049)	−0.007(0.087)	0.112(0.061)	−0.062(0.109)	0.088(0.092)	0.067(0.107)
Power	0.023(0.040)	0.117(0.063)	0.050(0.048)	−0.178(0.099)	0.008(0.059)	0.131(0.089)
Stability	0.002(0.025)	−0.077(0.043)	0.017(0.030)	0.103(0.056)	0.045(0.041)	0.011(0.051)
Resources	−0.014(0.047)	0.257** (0.086)	0.011(0.058)	0.063(0.096)	0.011(0.063)	−0.113(0.080)
Challenges	0.056(0.040)	0.140* (0.068)	−0.008(0.054)	−0.014(0.098)	0.047(0.074)	−0.024(0.085)
Professional development	0.012* (0.006)	0.008(0.010)	0.006(0.007)	−0.002(0.016)	0.022* (0.011)	0.006(0.018)
Constant	2.659** (0.228)	2.136** (0.395)	2.554** (0.279)	3.479** (0.512)	2.158** (0.397)	3.136** (0.432)
N	248	194	163	102	135	113

Note. Standard errors in parentheses after weights were applied. *p < .05. **p < .01.

Considerations in Interpreting the Results

As with all survey data and nonexperimental studies, there are important caveats to interpreting these results. Our survey items are self-report, and therefore, predictive relationships within them may contain endogeneity. It should be noted that in our survey we did not signify which content items were aligned with the standards; thus, teachers were not evaluating the alignment of their instruction. Further, teachers reported on a range of subject-specific content, and ELL and SWD teachers reported on both subjects, which should make our instructional conclusions more valid. Our method (surveys of enacted curriculum) has also been shown to be reliable and valid across multiple studies (Blank, 2004; Osthoff, 2007; Polikoff et al., 2011; Webb 2002, 2007).

Still, we do not use causal inference methods, so the relationships we find are predictive or correlational, not causal. Interpreting survey data should acknowledge the complexity of different ways of knowing. This includes the idea that quality of teaching can differ dramatically from observation and self-report (Cohen, 1990), while behaviorally based behaviors have reportedly high correlations with observations (Mayer, 1999). And as we noted previously, teacher quality and instructional content may not be closely related (Desimone et al., 2016; Polikoff & Porter, 2014). Triangulating our data with policy document analysis and classroom observation would undoubtedly add dimensions and insight to our findings.

Discussion

One of the main contributions of our study is the use of the policy attributes framework to quantify implementation and test for relationships to instruction. Using this framework, one of our key findings is that teacher reports of state standards policies and the alignment of teachers’ instruction to standards are remarkably similar despite state-level differences in written policy, as judged by studies of state policy during this same time period (Alexander, Jang, & Kankane, 2017; Polikoff, 2017). We find suggestive evidence in three states of a federalizing effect, where teachers are less sensitive and cognizant of changes in local policy and more attuned to national policy, mirroring trends toward the nationalization of education policy opinion (Levendusky, 2013; Polikoff, Hardaway, Marsh, & Plank, 2016; Supovitz, 2017). These similarities held for the two states that adopted the CCSS—Kentucky and Ohio—as well as Texas, the state in our study that developed its own standards. The existence of these similarities suggests that the common standards movement, even in its current state-by-state iteration, has been successful at shaping common policy perceptions among teachers. This finding by itself, absent any causal evidence of student outcomes, is a victory for those who have sought to standardize district-level education policies, even in states like Texas that have been notoriously resistant to federal intervention.

Despite many similarities, we found some important differences by subject, urbanicity, and special populations. Teachers of SWDs report lower authority (buy-in) for the standards and significantly less standards-emphasized instruction, which is consistent with previous work highlighting the tension between standardization and differentiations for SWDs (Dorn et al., 1996; Figueroa-Murphy & Haller, 2015). The persistence of these special education divides may not be surprising, but they remain of great practical relevance. In a related study, we note that while district administrators and principals in these states think the standards are appropriate for SWD, teachers disagree (Edgerton et al., in press). This study provides further evidence for the continuing exclusion of SWDs from standards-emphasized instruction based on teachers’ own self-report. This exclusion from the technical core remains deeply troubling—or, viewed more positively—suggests that teachers are in fact differentiating their instruction to meet the demands of individualized education plans.

Rural teachers as a whole experienced less robust policy environments, though this phenomenon was more pronounced in Texas and Ohio, which are more urban states overall than Kentucky. With an increased focus on rural America among federal policymakers, this finding highlights an important divide within the teaching profession itself in the implementation of policy. Our findings provide suggestive evidence of what Jimerson (2005) calls “placism”—the inherent bias toward urban areas in the construction of federal education policy. If SWD teachers and rural teachers do not perceive their policy environments to be as strong, we might anticipate declines in standards-emphasized instruction and worsening achievement gaps over time. In other words, students on individualized education plans and rural students may fall farther and farther behind if they do not make instructional shifts in line with their colleagues. We already find these teachers covering significantly less of the standards-emphasized content.

Another key difference we found is between math and ELA teachers. Standards-based policy seems to have taken root most significantly among elementary math teachers. In all three of our partner states, elementary math teachers are significantly more likely than elementary ELA teachers to cover more of the content emphasized by their state’s standards. This finding is consistent with previous work suggesting variation in outcomes for math and ELA (Dee & Jacob, 2011; Porter et al., 2011) and that ELA practices may be more difficult to change (Achieve, 2013).

Of the policy attributes, authority was the only one predictive of standards-emphasized instruction; this finding is consistent with prior work showing that authority was the strongest predictor of lasting change in the context of earlier school reform attempts (Desimone, 2002). Taken together, these instructional findings show that state policies may struggle to promote one type of content over another, as has been the case for much of the course of standards-based reform (Cohen, 1990). In all three states, authority and power were normally distributed; as a result, these variables were better predictors in our model. The other policy attributes were slightly skewed, which we would expect to result in them being less predictive. States seem to be showing much more success at aligning ELA instruction than math instruction through the policy attribute of authority, or buy-in, compared to rewards and sanctions (power) or specific curricular materials (specificity).

Implications

To return to a question raised in the introduction, is standards-based reform on shaky ground? We show that relationships between policy and instruction remain modest, similar to their relationships in previous reform environments (T. Smith, Desimone, & Ueno, 2005), but these relationships are similar across state lines. The use of the attributes in both constructing and analyzing these data reveals that teachers’ intrinsic belief in the appropriateness of the standards does relate to their instruction and that merely providing support (through resources and professional development) is not enough. We suggest that policy does have the ability to boost the technical core rather than remaining a loosely coupled system (Coburn, 2004). While we take no position concerning the ethics, relative importance, and effects of standards-based reform, we show that the maintenance of standards-based reform rests on teachers believing in the standards. We find that teachers in three states do find them useful and appropriate for planning instruction, with the exception of their students on individualized education plans. We suggest that while some may view this instructional gap as problematic, perhaps teachers are evincing their belief that they know best how to serve their students. A more pessimistic reading of our results suggests a continuing environment of lower expectations for students with disabilities (Edgerton et al., in press).

If teachers across three states report remarkably similar policy environments, what does this mean for the future of K–12 standards policy? State policy environments may have a marginal relationship to instruction, particularly in mathematics, another example of “tinkering towards utopia” (Tyack & Cuban, 1995). But viewed more positively, this study may be highlighting the relative success of CCSS in creating common policy perceptions six years after their adoption—even in a state like Texas that did not formally adopt them but instead echoed them in their own iteration (Porter et al., 2011). Other studies focus on highlighting differences in state laws and regulations (e.g., Alexander et al., 2017), but we find that these state policy differences do not manifest themselves in significantly different teacher perceptions that are meaningful across most of the policy attributes. Instead, we suggest that if the first goal of setting standards is to shift instruction, our significant authority finding implies that the first step is to cultivate teachers’ belief in seeing their state’s standards as useful, manageable, and appropriate. Developing a model with all of the attributes bolsters the notion that there is no silver bullet to change teacher behavior. Policymakers, administrators, and researchers must continue to work to convince teachers that changing their instructional habits benefits their students as well as provide evidence that a causal relationship exists. In the current policy environment, local actors—in this case, teachers—are the ones truly driving (or resisting) change (Marsh & Wohlstetter, 2013).

Coburn (2016) cautions against using policy research to learn the same lessons over and over again, and this study makes a new and important contribution to discussions of state accountability systems. Here we find that using the policy attributes theory reveals that authority is the only attribute with significant relationships to standards-emphasized instruction, even more so than power. Authority is the policy attribute that captures the “soft” persuasiveness of the standards rather than the “hard” power of rewards and sanctions. Policymakers often gravitate toward easy to quantify solutions, such as curricula (which provide specificity) and accountability (which provides power mechanisms for rewards and sanctions). Our findings suggest that facilitating teacher understanding, buy-in, and belief in the standards may be a more powerful leverage point.

While building up authority may be more difficult, expensive, and time-consuming, our results show how important these inherent teacher perceptions may be to shifting instruction, though we note again our results are correlational in nature. By examining how different teachers experience their standards policy environments and how those experiences are related to instruction, we aim to guide policymakers in identifying targets for increased support and attention. If policymakers wish to realize the goal of standards-based reform—a K–12 system where all students regardless of subgroup status receive standards-emphasized, grade-level instruction—we all must deepen our understanding of how we can shape policy in a manner that facilitates popular teacher support. This study demonstrates that many implementation gaps in instruction persist several years after the adoption of more rigorous standards and that the most well-funded policies for achieving instructional change (e.g., teacher evaluation and increased accountability) show insignificant relationships to instruction. Meanwhile, recent and developing studies find that achievement gains have slowed in this latest era (Alexander et al., 2017). Policymakers may be well advised to focus on cultivating teacher buy-in as they seek resources to support teachers’ standards implementation.

Footnotes

Appendix

The following appendix lists all of the survey items used to construct the policy attribute scales.

Acknowledgements

Many thanks, first and foremost, to the Institute of Education Sciences for funding this work. We thank our collaborators at the American Institutes for Research, specifically Michael Garet and Seth Brown, who led the survey data collection, and their team, as well as Rui Yang, who assisted with the Stata coding and cleaning of the survey data. We are also grateful for the participation of our partner states, Texas, Ohio, and Kentucky, who provided feedback on technical reports. Finally, a thanks to the implementation team at the University of Pennsylvania Graduate School of Education, as well as Andrew Porter, who commented on earlier drafts of this paper.

Authors’ Note

The content is solely the responsibility of the authors and does not necessarily represent the official views of the Institute of Education Sciences or the U.S. Department of Education.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported in part by Grant R305C150007 from the Institute of Education Sciences in the U.S. Department of Education to the University of Pennsylvania.

ORCID iD

Adam K. Edgerton

Authors

ADAM K. EDGERTON is a third-year PhD student in education policy at the University of Pennsylvania Graduate School of Education. His research interests include the politics of standards-based reform, teacher policy, and collective bargaining agreements.

LAURA M. DESIMONE is the associate dean of research at the University of Delaware’s College of Education and Human Development. She studies policy effects on teaching and learning.

References

Achieve. (2013). Achieving the Common Core. Retrieved from https://www.achieve.org/achieving-common-core

Alexander

N. A.

Jang

S. T.

Kankane

(2017). The performance cycle: The association between student achievement and state policies tying together teacher performance, student achievement, and accountability. American Journal of Education, 123(3), 413–446. doi:10.1086/691229

Allensworth

E. M.

Easton

J. Q.

(2007). What matters for staying on-track and graduating in Chicago public high schools: A close look at course grades, failures, and attendance in the freshman year. Research report. Retrieved from https://eric.ed.gov/?id=ED498350

Bacon

(2015). The impact of standards-based reform on special education and the creation of the ’dividual. Critical Studies in Education, 56(3), 366–383. doi:10.1080/17508487.2015.979845

Ball

D. L.

Cohen

D. K.

(1996). Reform by the book: What is—or might be—the role of curriculum materials in teacher learning and instructional reform? Educational Researcher, 25(9), 6–13. doi:10.3102/0013189X025009006

Berends

Bodilly

S. J.

Kirby

S. N.

(2002). Facing the challenges of whole-school reform. Santa Monica, CA: RAND Corporation.

Blank

R. K.

(2004). Findings on alignment of instruction using enacted curriculum data: Results from urban schools. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA. Retrieved from https://secure.wceruw.org/seconline/Reference/BlankAlignmentPaperAERA04.doc

Bovens

(2007). 8.1 The concept of public accountability. In Ferlie

Lynn

L. E.

Pollitt

(Eds.), The Oxford handbook of public management (pp. 182–211). New York, NY: Oxford University Press.

Burstein

McDonnell

L. M.

Van Winkle

Ormseth

Mirocha

Guiton

(1995). Validating national curriculum indicators. Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/monograph_reports/MR658.html

10.

Carmichael

S. B.

Martino

Porter-Magee

Wilson

W. S.

(2010). The state of the state standards—and the Common Core—in 2010. Washington, DC: Thomas B. Fordham Institute. Retrieved from https://files.eric.ed.gov/fulltext/ED516607.pdf

11.

Clune

W. H.

(1993). Systemic educational policy: A conceptual framework. In Fuhrman

S. H.

(Ed.), Designing coherent educational policy (pp. 125–140). San Francisco, CA: Jossey-Bass.

12.

Coburn

C. E.

(2004). Beyond decoupling: Rethinking the relationship between the institutional environment and the classroom. Sociology of Education, 77(3), 211–244. doi:10.1177/003804070407700302

13.

Coburn

C. E.

(2006). Framing the problem of reading instruction: Using frame analysis to uncover the microprocesses of policy implementation. American Educational Research Journal, 43(3), 343–349.

14.

Coburn

C. E.

(2016). What’s policy got to do with it? How the structure-agency debate can illuminate policy implementation. American Journal of Education, 122(3), 465–475. doi:10.1086/685847

15.

Cohen

D. K.

(1990). A revolution in one classroom: The case of Mrs. Oublier. Educational Evaluation and Policy Analysis, 12(3), 327–345. doi:10.3102/01623737012003311

16.

Dee

T. S.

Jacob

(2011). The impact of No Child Left Behind on student achievement. Journal of Policy Analysis and Management, 30(3), 418–446. doi:10.1002/pam.20586

17.

Dee

T. S.

Jacob

Schwartz

N. L.

(2013). The effects of NCLB on school resources and practices. Educational Evaluation and Policy Analysis, 35(2), 252–279. doi:10.3102/0162373712467080

18.

Desimone

(2002). How can comprehensive school reform models be successfully implemented? Review of Educational Research, 72(3), 433–479. doi:10.3102/00346543072003433

19.

Desimone

(2006). Consider the source: Response differences among teachers, principals and districts on survey questions about their education policy environment. Educational Policy, 20(4), 640–676. doi:10.1177/0895904805284056

20.

Desimone

(2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199. doi:10.3102/0013189X08331140

21.

Desimone

(2013). Teacher and administrator responses to standards-based reform. Teachers College Record, 115(8), 1–53.

22.

Desimone

Hochberg

McMaken

(2016). Teacher knowledge and instruction policy: Growth and linkages. Teachers College Record, 118(5), 1–54.

23.

Desimone

LeFloch

(2004). Are we asking the right questions? Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26(1), 1–22. doi:10.3102/01623737026001001

24.

Desimone

L. M.

Pak

(2017). Instructional coaching as high-quality professional development. Theory Into Practice, 56(1), 3–12. doi:10.1080/00405841.2016.1241947

25.

Desimone

Smith

Frisvold

(2010). Survey measures of classroom instruction: Comparing student and teacher reports. Educational Policy, 24(2), 267–329. doi:10.1177/0895904808330173

26.

Desimone

Smith

Phillips

(2013). Linking student achievement growth to professional development participation and changes in instruction: A longitudinal study of elementary students and teachers in Title I schools. Teachers College Record, 115(5), 1–46.

27.

Desimone

Wolford

Hill

(2016). Research-practice: A practical conceptual framework. AERA Open, 2(4). doi:10.1177/2332858416679599

28.

Dever

Lash

(2013). Using common planning time to foster professional learning. Middle School Journal, 45(1), 12–17. doi:10.1080/00940771.2013.11461877

29.

DiMaggio

Powell

(1983). The Iron Cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological Review, 48(2), 147–160. doi:10.2307/2095101

30.

Dorn

Fuchs

L. S.

(1996). A historical perspective on special education reform. Theory Into Practice, 35, 12–19. doi:10.1080/00405849609543696

31.

Edgerton

A. K.

Fuchs

(in press). New standards and old divides: Policy attitudes about college- and career-readiness standards for students with disabilities. Teachers College Record.

32.

Fan

Chen

(2001). Parental involvement and students’ academic achievement: A meta-analysis. Educational Psychology Review, 13(1), 1–22. doi:10.1023/A:1009048817385

33.

Figueroa-Murphy

Haller

(2015). Teachers’ perceptions of the implementation of the literacy Common Core State Standards for English language learners and students with disabilities. Journal of Research in Childhood Education, 29(4), 510–527. doi:10.1080/02568543.2015.1073200

34.

Firestone

W. A.

(2014). Teacher evaluation policy and conflicting theories of motivation. Educational Researcher, 32(2), 100–107. doi:10.3102/0013189X14521864

35.

Flores

Schissel

J. L.

(2014). Dynamic bilingualism as the norm: Envisioning a heteroglossic approach to standards-based reform. Tesol Quarterly, 48(3), 454–479. doi:10.1002/tesq.182

36.

Fowler

F. J.

Jr. (2013). Survey research methods. Thousand Oaks, CA: SAGE Publications.

37.

Fuchs

L. S.

(2006). Introduction to response to intervention: What, why, and how valid is it? Reading Research Quarterly, 41(1), 93–99. doi:10.1598/RRQ.41.1.4.

38.

Gamson

D. A.

(2015). Differences in common. In Supovitz

J. A.

Spillane

J. P.

(Eds.), Challenging standards: Navigating conflict and building capacity in the ear of the Common Core (pp. 133–144). Lanham, MD: Rowan & Littlefield.

39.

Glover

T. A.

Nugent

G. C.

Chumney

F. L.

Ihlo

Shapiro

E. S.

Guard

. . . Bovaird

. (2016). Investigating rural teachers’ professional development, instructional knowledge, and classroom practice. Journal of Research in Rural Education, 31(3), 1–16. Retrieved from https://files.eric.ed.gov/fulltext/EJ1101917.pdf

40.

Goertz

M. E.

(2005). Implementing the no child left behind act: Challenges for the states. Peabody Journal of Education, 80(2), 73–89. doi:10.1207/S15327930pje8002_5

41.

Hamilton

L. S.

Stecher

B. M.

Yuan

(2008). Standards-based reform in the United States: History, research, and future directions. Washington, DC: RAND Corporation. Retrieved from https://www.rand.org/content/dam/rand/pubs/reprints/2009/RAND_RP1384.pdf

42.

Hanushek

E. A.

(1997). Assessing the effects of school resources on student performance: An update. Educational evaluation and policy analysis, 19(2), 141–164. doi:10.3102/01623737019002141

43.

Hanushek

E. A.

Woessmann

(2017). School resources and student achievement: A review of cross-country economic research. In Rosén

Hansen

K. Y.

Wolff

(Eds.), Cognitive abilities and educational outcomes (pp. 149–171). Amsterdam: Springer.

44.

Hill

H. C.

(2001). Policy is not enough: Language and the interpretation of state standards. American Educational Research Journal, 38(2), 289–318. doi:10.3102/00028312038002289

45.

Hill

Desimone

L. D.

Wolford

Reitano

(2017). Assessing school turnaround: Using an integrative framework to identify levers for success. In Meyers

Darwin

(Eds.), Enduring myths that inhibit school turnaround (pp. 173–192). Charlotte, NC: Information Age Publishing.

46.

Jimerson

(2005). Placism in NCLB—How rural children are left behind. Equity & Excellence in Education, 38(3), 211–219. doi:10.1080/10665680591002588

47.

Johnson

Howley

C. B.

(2015). Contemporary federal education policy and rural schools: A critical policy analysis. Peabody Journal of Education, 90, 224–241. doi:10.1080/0161956X.2015.1022112

48.

Kane

T. J.

Owens

A. M.

Marinell

W. H.

Thal

D. R. C.

Staiger

D. O.

(2016). Teaching higher: Educators’ perspectives on Common Core implementation. Cambridge, MA: Center for Education Policy Research. Retrieved from http://cepr.harvard.edu/files/cepr/files/teaching-higher-report.pdf

49.

Kaufman

J. H.

Cannon

J. S.

Culbertson

Hannan

Hamilton

L. S.

Meyers

(2018). Raising the bar for student outcomes in Louisiana (RB-10024-BRAF). Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/research_briefs/RB10024.html

50.

Kennedy

(2016). How does professional development improve teaching? Review of Educational Research, 86(4), 945–980. doi:10.3102/0034654315626800

51.

Kraft

M. A.

Blazar

Hogan

(2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588. doi:10.3102/0034654318759268

52.

Levendusky

M. S.

(2013). Why do partisan media polarize viewers? American Journal of Political Science, 57(3), 611–623. doi:10.1111/ajps.12008

53.

Marsh

J. A.

Wohlstetter

(2013). Recent trends in intergovernmental relations: The resurgence of local actors in education policy. Educational Researcher, 42(5), 276–283. doi:10.3102/0013189X13492193

54.

Massell

Perrault

(2014). Alignment: Its role in standards-based reform and prospects for the common core. Theory Into Practice, 53(3), 196–203. doi:10.1080/00405841.2014.916956

55.

Mayer

D. P.

(1999). Measuring instructional practice: Can policymakers trust survey data? Educational Evaluation and Policy Analysis, 21(1), 29–45. doi:10.3102/01623737021001029

56.

Mehta

(2013). The allure of order: High hopes, dashed expectations and the troubled quest to remake American schooling. New York, NY: Oxford University Press.

57.

Osthoff

(2007). Using SEC data to improve curriculum content alignment in Oregon’s Canyon Falls School District: Initial steps and evolving challenges. Madison, WI: Center for Education Research. Retrieved from http://programs.ccsso.org/content/pdfs/Oregon%20SEC%20article%20021907%20Review.pdf

58.

Penuel

W. R.

Fishman

B. J.

Yamaguchi

Gallagher

L. P.

(2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44(4), 921–958. doi:10.3102/0002831207308221

59.

Polikoff

M. S.

(2012). The association of state policy attributes with teachers’ instructional alignment. Educational Evaluation and Policy Analysis, 34(3), 278–294. doi:10.3102/0162373711431302

60.

Polikoff

M. S.

(2017). Is Common Core “working”? And where does Common Core research go from here? AERA Open, 3(1). doi:10.1177/2332858417691749

61.

Polikoff

M. S.

Hardaway

Marsh

J. A.

Plank

D. N.

(2016). Who is opposed to Common Core and why? Educational Researcher, 45(4), 263–266.

62.

Polikoff

M. S.

Porter

A. C.

(2014). Instructional alignment as a measure of teaching quality. Educational Evaluation and Policy Analysis, 36(4), 399–416. doi:10.3102/0162373714531851

63.

Polikoff

Porter

Smithson

(2011). How well aligned are state assessments of student achievement with state content standards? American Educational Research Journal, 48(4), 965–995. doi:10.3102/0002831211410684

64.

Porter

A. C.

(1994). National standards and school improvement in the 1990s: Issues and promise. American Journal of Education, 102, 421–449. doi:10.1086/444081

65.

Porter

A. C.

(2002). Measuring the content of instruction: Uses in research and practice. Educational Researcher, 31(7), 3–14. doi:10.3102/0013189X031007003

66.

Porter

A. C.

Floden

Freeman

Schmidt

Schwille

(1988). Content determinants in elementary school mathematics. In Grouws

D. A.

Cooney

T. J.

(Eds.), Perspectives on research on effective mathematical teaching (pp. 96–113). Hillsdale, NJ: Lawrence Erlbaum Associates.

67.

Porter

McMaken

Hwang

Yang

(2011). Common Core Standards: The new U.S. intended curriculum. Educational Researcher, 40(3), 103–116. doi:10.3102/0013189X11405038

68.

Raudenbush

Rowan

Cheong

(1992). Contextual effects on the self-perceived efficacy of high school teachers. Sociology of Education, 65(2), 150–167. doi:10.2307/2112680

69.

Rowan

Camburn

Correnti

(2004). Using teacher logs to measure the enacted curriculum: A study of literacy teaching in third-grade classrooms. The Elementary School Journal, 105(1), 75–101. doi:10.1086/428803

70.

Schmidt

W. H.

McKnight

C. C.

Houang

R. T.

Wang

Wiley

D. E.

Cogan

L. S.

Wolf

R. G.

(2001). Why schools matter: A cross-national comparison of curriculum and learning. San Francisco, CA: Jossey-Bass.

71.

Schueler

B. E.

Goodman

J. S.

Deming

D. J.

(2017). Can states take over and turn around school districts? Evidence from Lawrence, Massachusetts. Educational Evaluation and Policy Analysis, 39(2), 311–332. doi:10.3102/0162373716685824

72.

Smith

M. S.

O’Day

(1991). Putting the pieces together: Systemic school reform (CPRE Policy Brief, RB-06-4/91). New Brunswick, NJ: Consortium for Policy Research in Education. Retrieved from http://www.cpre.org/sites/default/files/policybrief/847_rb06.pdf

73.

Smith

Desimone

Ueno

(2005). “Highly qualified” to do what? The relationship between NCLB teacher quality mandates and the use of reform-oriented instruction in middle school math. Educational Evaluation and Policy Analysis, 27(1), 75–109. doi:10.3102/01623737027001075

74.

Spillane

J. P.

(1999). State and local government relations in the era of standards-based reform: Standards, state policy Instruments, and local instructional policy making. Educational Policy, 13(4), 546–572. doi:10.1177/0895904899013004004

75.

Spillane

J. P.

(2009). Standards deviation: How schools misunderstand education policy. Cambridge, MA: Harvard University Press.

76.

Spillane

J. P.

Parise

L. M.

Sherer

J. Z.

(2011). Organizational routines as coupling mechanisms: Policy, school administration, and the technical core. American Educational Research Journal, 48(3), 586–619. doi:10.3102/0002831210385102.

77.

Spillane

Reiser

Reimer

(2002). Policy implementation and cognition. Reframing and refocusing implementation research. Review of Educational Research, 72(3), 387–431. doi:10.3102/00346543072003387

78.

Stecher

B. M.

Epstein

Hamilton

L. S.

Marsh

J. A.

Robyn

McCombs

J. S.

. . . Naftel

. (2008). Pain and gain: Implementing No Child Left Behind in three states, 2004—2006 (MG-784-NSF). Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/monographs/MG784.html

79.

Stecher

B. M.

Holtzman

D. J.

Garet

M. S.

Hamilton

L. S.

Engberg

Steiner

E. D.

. . . Chambers

. (2018). Improving teaching effectiveness: Final report: The Intensive Partnerships for Effective Teaching through 2015–2016 (RR-2242-BMGF). Santa Monica, CA: RAND Corporation. Retrieved from https://www.rand.org/pubs/research_reports/RR2242.html

80.

Supovitz

(2017). Social media is the new player in the politics of education. Phi Delta Kappan, 99(3), 50–55. doi:10.1177/0031721717739594

81.

Supovitz

Fink

Newman

(2016). From the inside in: Common Core knowledge and communication within schools. AERA Open, 2(3). doi:10.1177/2332858416653757

82.

Supovitz

J. A.

Spillane

(2015). Challenging standards: Navigating conflict and building capacity in the era of the Common Core. Lanham, MD: Rowman & Littlefield.

83.

Troia

G. A.

Graham

(2016). Common core writing and language standards and aligned state assessments: A national survey of teacher beliefs and attitudes. Reading and Writing, 29(9), 1719–1743. doi:10.1007/s11145-016-9650-z

84.

Tyack

D. B.

Cuban

(1995). Tinkering toward utopia. Cambridge, MA: Harvard University Press.

85.

U.S. Department of Education. (2018). Prepared remarks by U.S. Education Secretary Betsy DeVos to the American Enterprise Institute. Retrieved from https://www.ed.gov/news/speeches/prepared-remarks-us-education-secretary-betsy-devos-american-enterprise-institute

86.

Webb

N. L.

(2002). Alignment study of language arts, mathematics, science, and social studies of state standards and assessments in four states. Washington, DC: Council of Chief State School Officers. Retrieved from http://facstaff.wcer.wisc.edu/normw/AERA%202002/alignment%20analysis%20language%20arts%20%20four%20states%2031202.pdf

87.

Webb

N. L.

(2007). Issues related to judging the alignment of curriculum standards and assessments. Applied Measurement in Education, 20(1), 7–25. doi:10.1080/08957340709336728

88.

Weick

K. E.

(1976). Educational organizations as loosely coupled systems. Administrative Science Quarterly, 21(1), 1–19. doi:10.2307/2391875

89.

Yettick

H. H.

Baker

Wickersham

Hupfeld

(2014). Rural districts left behind? Rural districts and the challenges of administering the Elementary and Secondary Education Act. Journal of Research in Rural Education, 29(13), 1–15. Retrieved from http://jrre.psu.edu/wp-content/uploads/2015/01/29-13.pdf