Teaching to the data collection? (Un)intended consequences of online child assessment system,“Teaching Strategies GOLD”

Abstract

Drawing on a qualitative case study of four Head Start sites, this article analyzes a number of governing patterns that have emerged and grown in the field of early childhood education and care in the United States. This article specifically highlights a government’s enhanced oversight and management of Head Start programs by mandating a school readiness data system in their local programs: establishing a set of measurable school readiness goals, and collecting and analyzing ongoing child assessment data to monitor children’s progress in meeting these goals. The primary focus of this article is to illustrate the experience of Head Start teachers and children with a commercial online child assessment and record keeping system called the “Teaching Strategies Gold.” Research findings of this article suggest that seemingly “authentic, ongoing observational system,” Teaching Strategies Gold actually extends and intensifies the techniques and effects of power operating on both children and teachers on a more minute and everyday level. It (re)shapes the conditions of work and constructs new norms for teachers themselves as “ongoing” assessors/observers. Pressure to “collect” all the required observational evidence that would also determine children’s progress toward the next higher level has dramatically changed the nature of free play and small group activities into more “functional,” “hierarchical,” and “analytical” space.

Introduction

The standards movement of K-12 education in the United States is now extended to the early childhood education programs that serve 4-year-old or younger children (e.g. National Early Childhood Accountability Task Force, 2007; US Department of Education and US Department of Health and Human Services, 2011). Most states have developed child outcome standards that define general expectations for young children’s development and learning at a certain age. More importantly, concomitant with the increasing public investment in early childhood programs come various policies and reform efforts regarding documentation of systematic child outcome data to assess children’s progress toward achieving those standards and evaluate the quality and the effectiveness of programs. According to President Obama’s 2013 State of the Union address, states are required to “implement comprehensive data and assessment systems” to receive federal funding (The White House, 2013). Most states currently mandate all state pre-kindergarten programs to report child outcome data for various purposes that range from providing the appropriate professional development, training, or assistance to determining the need for corrective actions or sanctions (Ackerman and Coley, 2012).

Head Start programs, as the longest federal early childhood program for low-income children in the United States and a community partner of state-funded pre-kindergarten programs, are no exception to this trend (Bullough et al., 2012). It is mandated that each Head Start program tracks children’s progress toward their established school readiness goals in the five essential domains at an individual child level and a program-wide level at least three times per year. Furthermore, each Head Start program is mandated to use those data for ongoing program improvement, the failure of which will lead the program to a re-competition for a next funding cycle (Office of Head Start (OHS), n.d.).

Based on a qualitative case study of four Head Start sites, this article aims to examine a number of governing patterns embedded in the current child assessment policy and practice that regulate and control young children and teachers in the earliest stage of their schooling. Before I examine the experience of Head Start children and teachers with a computer-based, online child assessment tool, “Teaching Strategies Gold” (TS Gold) this article begins with a brief overview of the larger context of the rise and development of child assessment policies as a governing technology in the field of early childhood education.

In this article, I use the term technology in a double sense. First, drawn on the work of governmentality scholars, technologies refer broadly to the “mechanisms through which authorities of various sorts have sought to shape, normalize and instrumentalize the conduct, thought, decisions and aspirations of others in order to achieve the objectives they consider desirable” (Miller and Rose, 1990: 8). Second, in its narrowest and the most common sense, it indicates computer technology with certain digital characteristics and technical elements (e.g. “TS Gold”). In other words, by “technologies of government” (Miller and Rose, 1990: 1), I highlight current early childhood assessment policies as an important aspect of the changing nature of educational governance or “governance turn” (Ozga et al., 2011: 85) that has taken place through the reliance on the “almost taken-for-granted growth of data” (Lawn, 2011: 278) in tandem with technological advances in computing.

The audit culture in the field of early childhood education: “control of control”

The idea of audit, originally associated with financial accounting, extends into inspecting and managing professional conduct and performance in virtually every aspect of our work and life (e.g. operational/managerial audits, compliance audits, information system audits). The field of education is just one site among many where we can see an “explosion of audits” (Power, 1994: 1). In an era of accountability, there have been policy and reform efforts to make public sector institutions more accountable to taxpayers by improving efficiency, effectiveness, and transparency. Being accountable to the public is recast in terms of “performativity” in which “performances serve as measures of productivity or output, or displays of quality” (Ball, 2003: 216). Central to functioning of performativity is the rise and spread of measurable, predetermined standards or performance indicators by which “complex social processes and events” are transferred into “simple figures or categories of judgment” for comparisons (p. 217). The demands for transparency call for collecting, analyzing, and reporting performance data of various kinds at the child, teacher, and program levels that are relevant, accurate, and accessible to public interpretation. Particularly, child outcome data are taken to function as a powerful source from which performativity of teachers is also measured and by which effectiveness of education programs or services is eventually made visible and transparent to the public (Koyama, 2011; Roberts-Holmes, 2015).

However, the regulatory mechanism of contemporary audit practice is not solely based on the coercive imposition of external control from above. Rather, the new approach now requires that public institutions develop and maintain their own internal management systems including annual self-assessment and ongoing self-monitoring activities (Shore, 2008). In other words, new systems of audit in our contemporary society have become the “control of control” in Power’s (1994: 19) term. Data are an essential component in the control of control (Ozga, 2014). In order to provide evidence for the existence of such systems of internal management and prove their compliance with all laws, rules, and regulations, public institutions have involved producing tremendous amounts of documentation and reports of various kinds. More importantly, public sector institutions are required to prove the quality of their internal management system by demonstrating how they use data to guide their continuous program improvement. Simply collecting and reporting performance data or results are no longer acceptable to be held accountable. What is more important is to prove the existence of the system for data-driven decision making across all levels through which public institutions design and implement clear and specific intervention plans for improving their performances (Brawley and Stormont, 2014; Means et al., 2009; Riley-Ayers et al., 2011).

Child assessment as a governing technology: The case of Head Start

Head Start in the United States is the federally funded early childhood education program for low-income children and families. Every 5 years, the OHS conducts a comprehensive on-site inspection to measure Head Start program’s performance and compliance with federal regulations such as Head Start Program Performance Standards, Head Start Act, and other regulations. According to FY 2015 Monitoring Protocol, local Head Start program’s internal “management system” is one of six core performance areas that a federal on-site inspection team intensively examines (OHS, 2014: 3). Head Start program is required to demonstrate that it establishes effective ongoing monitoring systems and procedures to identify areas of concern and make immediate program corrections to ensure compliance. More specifically, Head Start program’s ongoing monitoring system is expected to

Use effective tools and procedures to ensure the program is in compliance and meets its goals and objectives; clearly define staff roles and responsibilities in program oversight; conducts frequent, ongoing monitoring activities; collect and use data for planning activities and to ensure compliance. (p. 16)

In addition, all Head Start programs are mandated to conduct a comprehensive, annual self-assessment:

Self-assessment (is) a process used to measure a Head Start program’s effectiveness and progress in meeting program goals and objectives and their implementation and compliance with regulatory requirements, and to identify areas of strength and for improvement, including school readiness of children. The Head Start program compiles the results of the Self-assessment into a Self-assessment summary report which includes recommendations for program improvement that are later used by the program to develop improvement plans and the program Training and Technical assistance plan. (The National Center on Program Management and Fiscal Operations, 2013: 1)

In this process, Head Start programs are required to demonstrate their capacity to manage and use data effectively and efficiently to make informed decisions. Particularly, Head Start programs are required to analyze and use child outcome data (e.g. school readiness data) in program self-assessment and continuous improvement. Since its inception in 1965, there has been ongoing controversy over the Head Start program’s effectiveness, mostly in terms of short- and long-term effects of the programs on child outcomes as the major indicator of Head Start program success or quality. Therefore, Head Start children’s learning and development have been under ongoing public scrutiny through various evaluation studies including federally sponsored, large-scale study such as The Family and Child Experiences Survey (FACES) (Zill et al., 2001, 2003) and The Head Start Impact Study (Puma et al., 2005) that assess the performance of nationally representative sample of Head Start children and their experience.

With the downward extension of child outcome–based school reform, public scrutiny is not limited to assessing a nationally representative sample only. All Head Start programs in the United States are mandated to assess each and every child birth to age 5. Following the re-authorization of Head Start in 1998, individual Head Start programs were required to assess children within each program with a focus on 13 legislatively mandated indicators in language, literacy, and mathematics and include those child outcome data in its program self-assessment process by 2003. Each Head Start program was allowed to select its own local assessment tools, as long as it measured progress in the legislatively mandated indicators. At the federal level, the OHS began to implement the National Reporting System (NRS) in 2003 for all Head Start programs that was the first nationwide, standardized assessment of all 4- and 5-year-old Head Start children (Paulsell et al., 2006). The OHS contracted with Westat, Inc. and its subcontractor XTRIA, LLC to develop the 15-minute NRS assessment battery. Children were pulled out of their classrooms and assessed twice a year in the four skill areas by a trained adult assessor they did not necessarily know: (1) English language screener, (2) 24 items assessing children’s receptive vocabulary, (3) naming 26 pairs of upper and lower case alphabet letters, and (4) early math skills assessment (e.g. shape recognition; relative size judgments and measures; and simple word problems involving reading graphs, counting, or basic addition and subtraction) (Vogel et al., 2008). Several months after each fall and spring administration, the OHS provided each program with a summary report of how children performed. For children who were assessed in both the fall and spring, a growth report was made available. Each local Head Start program could also download reference tables from the Computer-Based Reporting System to compare their average scores with the national averages, regional averages, and programs with similar characteristics. However, in response to a number of issues and concerns about developmentally and culturally inappropriate content and administration process of NRS (Government Accounting Office, 2005; Meisels and Atkins-Burnett, 2004; Moore and Yzaguirre, 2004; Zigler and Styfco, 2004), the Improving Head Start for School Readiness Act of 2007 suspended the controversial use of NRS.

While the NRS has been suspended, the demand for establishing such systems in local Head Start programs to ensure outcome-based accountability has not (Kallemeyn and DeStefano, 2009). In the Improving Head Start for School Readiness Act of 2007, measurable child outcomes remain a top priority as the key outcomes and indicators of Head Start program quality. As of 9 December 2011, all Head Start and Early Head Start programs must establish school readiness goals in the five essential child development and early learning domains (i.e. social-emotional, language and literacy, approach to learning, cognition and general knowledge, physical wellbeing and motor development). Then, they must create and implement a plan for achieving those goals, measure individual children’s progress, and analyze aggregate data at least three times per year. More importantly, Head Start programs are required to demonstrate that they use children’s school readiness outcome data to direct continuous program improvement related to the “effectiveness of curriculum, instruction, professional development, and program design or other program decisions” (OHS, 2014: 37). Finally, Head Start programs must “inform parents and the community of the program’s progress in achieving school readiness goals” (p. 37).

The decision to suspend top-down, centralized NRS and turn to the establishment of school readiness data system in each local Head Start program may appear initially to be a sign for increased local program autonomy, flexibility, and empowerment. However, this change exemplifies new governing patterns in which the government tightens its management of Head Start programs by incorporating local Head Start program’s self-management system into its overall control mechanism.

“Technology” of child assessment: “TS Gold”

In response to the legislative mandate for establishing school readiness data system, the TS Gold is currently consumed by more than half of all Head Start programs across the country (Isaacs et al., 2015). In addition, as of 2012, TS Gold has been the most common measure required to be used or approved for use by State-funded Pre-kindergarten programs (Schilder and Carolan, 2014). A criterion referenced, TS Gold is an observation-based assessment system for children from birth through kindergarten. Using color-coded, age-specific nine rating scales, teachers score each child’s knowledge, skills, and behaviors in 38 objectives across 10 areas of development and learning (e.g. total 66 assessment items in social-emotional, physical, language, cognitive, literacy, mathematics, science and technology, social studies, art and English language, and acquisition) at multiple checkpoints. These objectives are a set of predetermined standards of development and learning aligned with the Head Start Child Development and Early Learning Framework, Common Core State Standards, and early learning standards for each state. The unique feature of the TS Gold is that it uses computer data system that allows online data entry, storage of observation notes, and analysis. Teachers are required to upload adequate number of anecdotal records online to substantiate the numerical scores that they enter to rate each child’s knowledge, skills, and behaviors.

In contrast to a direct, standardized assessment of children at a single point-in-time on a limited set of academic skills (e.g. NRS), TS Gold is generally considered a more authentic early childhood assessment practice and therefore necessarily good for capturing a “true” picture of children in that (1) a variety of data from multiple sources are collected at multiple times, (2) all developmental and learning domains are assessed, (3) teachers (persons familiar to the child) observe children within the natural flow of their everyday classroom activity, and (4) this is curriculum-embedded assessment (Kostelnik et al., 2010). However, for much of the same reasons that it is considered a more authentic early childhood assessment practice rather than a direct, standardized testing, the TS Gold online is more likely to place both children and teachers in a web of continuous observation or even ongoing surveillance that accompanies normalizing judgment (Foucault, 1977). Particularly, with the use of computer technology in gathering, organizing, and analyzing child assessment data, the TS Gold extends and intensifies the techniques and effects of power operating on both children and teachers. I do not intend to argue in favor of using a direct, standardized testing in early childhood programs. Rather, my primary concern is with how seemingly authentic early childhood assessment practice actually works to regulate early childhood programs, children, teachers, and their pedagogical and curriculum choices on a more minute and everyday level.

The study: Taking a close look at four Head Start sites

The overarching goal of this study is to examine how current early childhood assessment practice regulates and controls children and teachers. I highlight one local Head Start program as a case that provides insights into the experience of young children and teachers with the legislative mandate for establishing a school readiness data system. I do not intend to provide a comprehensive evaluation of a local program’s school readiness data system or its overall performance. Rather, my primary focus is to explore and describe how local school readiness data system works to regulate children, teachers’ work, and their pedagogical and curriculum choices on a more minute and everyday level.

Setting

The Superstar ¹ is a community-based, private, non-profit Head Start program that serves over 1000 children in 17 locations in the Midwest state. During the year of the study (2011–2012), the Superstar Head Start program began to implement an online child assessment system called “TS Gold” in order to meet the legislative mandate to establish its own child outcome data system by 2011. Previously, the Superstar Head Start program used a paper version of the Creative Curriculum Developmental Continuum Assessment System that Teaching Strategies, Inc. eventually ceased to support in July 2011. In the same year, the Superstar Head Start program started participating in the state pre-kindergarten program as the local school district launched a new, universal 4-year-old pre-kindergarten program. The school-based, 4-year-old pre-kindergarten program also implemented the TS Gold.

In documenting the Superstar Head Start program’s experiences with the TS Gold during the first year of implementation, four different, half-day Head Start sites were selected as a main focus of this article. Three among four Head Start sites provided double session classes serving children aged 3–5 years (3.5 hours, 17 children per each session). Each site selected for this study varied in terms of its location, racial/ethnic demographics of the classroom, experience with a statewide 4K program and teachers’ educational background and years of teaching experience.

Data collection and analysis

Qualitative data were collected from multiple sources including observation in the four different classrooms and site-level meetings over the course of an academic year of 2011–2012, three in-depth individual interviews with teachers (N = 6) and one individual interview with their supervisors (N = 5), and the collection of observation records and assessment data for three to five randomly chosen children in each classroom. I observed each Head Start classroom twice per week (total 8 class observations per week) for 5 months. Each class observation lasted 3 hours 30 minutes. Free play time and small group time were the two main focus areas in my classroom observations to see strategies and techniques that teachers employed to facilitate and document ongoing child assessment. I also observed weekly team meetings to understand how use of TS Gold affected teachers’ pedagogical and curriculum choices.

All lead teachers and co-teachers of four Head Start classrooms (N = 6) participated in three individual interviews. In each semi-structured interview that lasted between 1 hour and 1.5 hours, teachers were asked to tell about (1) their individual experience with a Head Start program, (2) curriculum and instruction decision-making process, and (3) experience with TS Gold during the first year of implementation. By including interviews with child development specialists (N = 5), I aimed to understand broader policy issues at its local program level that affected participating Head Start teachers’ work in their individual classrooms. In one in-depth, semi-structured individual interview, each lasting between 1 hour and 1.5 hours, child development specialists were asked to tell about (1) their work and job responsibilities, (2) curriculum and instruction decision-making process at the local program level, and (3) experience with legislative mandate for establishing school readiness data system in their program.

I collected copies of teachers’ observation notes, selection of work samples, checklists, and other personal notes. With parental permission, each teacher provided documentation that they entered online to finalize checkpoint data for randomly chosen three to five children. Teachers also shared with me various kinds of TS Gold–generated reports. Collection of these artifacts helped me understand “inscription” processes (Latour, 1986: 20–22) by which complex learning processes and children’s development were reduced and fixed to simple figures or scores for “comparison,” “differentiation,” “hierarchization,” “homogenization,” and “exclusion” (Foucault, 1977: 182–183).

Analytical categories and related themes emerged and developed as I moved back and forth between theoretical framework and qualitative data collected. Particularly, in my analysis of governing technologies in various forms and types, a close reading of Foucault’s (1977) book Discipline & Punish provided useful concepts such as “functional,” “hierarchical,” and “analytical” to better understand and explain qualitative data from fieldwork (p. 148).

Findings

“These kids should all be blue”: Individualization and normalization

Highly individualized teaching is considered one of four integral elements of effective Head Start practice. According to the FY 2015 Monitoring Protocol, the level of Head Start program performance is assessed in terms of whether it uses information from ongoing assessment of a child’s progress to “individualize experiences, instructional strategies, and services” (OHS, 2014: 111). In the online teacher resources available at the OHS, individualization has been generally described as the “extra help needed to learn a specific skill or concept,” “adjustments,” or “just the right level of support” for “child outcomes and school readiness goals.” In addition, as the list of related materials provided by OHS implies, individualization has been often introduced or reviewed in association with Individualized Education Program (IEP)-related concepts such as “disabilities,” “intervention,” or “inclusion.” All children in Head Start programs are considered those with “specific learning needs.” For example, one child development specialist in this study explained,

Are you familiar with how public school works? If you have a child who has a special educational need, they have an IEP, and they identify goals in the IEP and that happens in the least restrictive environment of the regular education classrooms. That’s exactly it. Basically, what Head Start does is, say every child has an individualized education plan, and how are we going to accomplish that within the group setting? So I think, for teachers, that’s a big challenge. And for me in my role, that’s where I need to be most supportive.

Head Start teachers in this study were required to set up individualization goals for each and every child. The TS Gold became an integral part of individualization in Head Start programs. Teachers directly linked children’s individualization goals to the TS Gold, with heavy emphasis on the literacy objectives unless children had other social, emotional, and behavioral concerns or challenges. One of the most appealing features of the TS Gold is that it broke down children’s development and learning objectives into smaller, temporal components a lot more than ever before. It gave teachers the impression that it enabled them to accurately assess children’s current status in more specific terms, predict likely next steps, and eventually plan the most appropriate learning experience for each child. TS Gold–based individualization actually involves what Foucault (1977) refers to as “normalizing judgment” that defines and actually produces (ab)normal children or (un)typically developing and learning children (p. 170). Particularly, certain digital characteristics and technical elements of TS Gold serve to reinforce normalization processes by providing the means for “comparison,” “differentiation,” “hierarchization,” “homogenization,” and “exclusion” (Foucault, 1977: 182–183). As an “inscription device” (Latour, 1986: 18), TS Gold simply reduces the complexity of children’s developmental characteristics and learning experience to a fixed score. Particularly, in using the dropdown menu of an online reports tab, whoever has an access to TS Gold can “quickly and easily create” standard reports for various comparison purposes (TS Gold Reports, 2011: 14). “With just a few clicks of a mouse” (TS Gold touring guide, 2011: 11), children’s scores are compared to predetermined standards, to their previous performance, to the same age group in their own classroom or program, or to a nationally representative sample of children who have been assessed with TS Gold.

According to an education service director, Superstar Head Start program customizes those reports with the criteria they wish to use for comparison of teachers’ performance in specific TS Gold objectives or overall teaching quality:

You can do the overall, how our agency has done with infant toddler, preschool, three-year-olds, four-year-olds, so that’s the nice thing about this tool. You can kind of cut the report almost any way you want it … We can really dissect the report and try and figure out where the need is. We can pull it by teachers. Is it brand new teachers that are in their first year here? Is it long-term teachers? So every two weeks, this committee gets together and we look at both the data and then, what does it mean for professional development?

Some child development specialists in this study further articulated an interest in linking TS Gold child outcome data to Classroom Assessment Scoring System (CLASS) scores to compare individual or group of teachers’ classroom quality.

Teachers themselves began on their own or at least felt the necessity to learn the online report tools that they hope would assist their analysis of class and individual child data. For example, one teacher said,

Now, I’m more familiar with it, and know more about it … I would like more training, especially (on) the different reports. Because I know there’s even more that we can do with it than what I know right now.

A quote from one teacher below suggests that a visual display or distribution of children in a chart or a graph format led teachers to understand and describe children in certain ways:

It was helpful having them all in a report together … It shows with the color band, so you can see where all the three year olds are at, and four year olds, and then it’s helpful because you can kind of see, this is three or four but they’re not at the level they should be. So it’s just a different way of looking at the information.

While learning and reading TS Gold reports such as “class profile reports,” teachers quoted below used spatial, temporal, or directional terms to differentiate children from one another in terms of their position from the age-specific color-coded bands:

It does kind of help us individualize because here, the dark (color) is where they’re supposed to be. He is behind … The other one if you go into it, it’ll say these are the things they can do to move up so that can help us individualize to get them up to their age.

This one shows green is the three year olds. And the blue is four year olds. This shows us as a class where they should be. It shows that this child is below, and this child is actually higher than this. She is more like what the four year olds are. So we use this information for individualizing with kids.

Below is the teachers’ reference to numerical ratings (Not Yet to level 9) that they assigned to the children’s certain knowledge, skills, and behaviors, which indicated hierarchization of children:

This one is below. They’re supposed to be between a 3 and a 6. He’s a 2, but she’s a 7. These kids are supposed to be in the green. But then these are all higher than they’re supposed to be. They are now in blue.

A paradoxical nature of a TS Gold-driven individualization lies in the fact that a greater focus upon individual differences actually depends on the assumed sameness for all children. Age-specific, color-coded bands impose homogeneity in children’s development and learning by showing “where they’re supposed to be” or “where they should be” at a certain age. TS Gold establishes individual differences as deviation from universal norms of sameness and at the same time aims to eliminate these differences. One child development specialist explained,

The goal is to get there. So in (Teaching Strategies) gold, it has for a four-year-old. It’ll say “meets expectations for a four-year-old,” or it will say “below expectations for a four-year-old,” or it will say “above expectations for a four-year-old.” So that’s where you have to figure out. There’s a report … We look at that and find out how many of our four-year-olds or three-year-olds are below, meet, or exceed the expectation for that age group. And then, that tells us our next step (for) the children that are not getting it, going back and trying more activities.

In this process, certain groups of children are defined and excluded as not typically developing and learning. Teachers’ primary concern is to recognize children who might benefit from special help, screening, or further evaluation.

“Did we take the right observation to move them up?” Play under ongoing surveillance

TS Gold is marketed as offering user-friendly tools for teachers to save time with fast, efficient data collection, recording, storage, analysis, and sharing. The advanced computer technologies are generally considered to help teachers’ work if utilized properly by teachers who have strong commitment, motivation, and high capacity for technology use. Unlike its assertion that “Teaching Strategies Gold leaves teachers with more time to interact with children” (TS Gold touring guide, 2011: 3), it actually does not allow teachers to do so. Teachers in this study expressed their concern about the enormous amount of time and energy being spent on collecting anecdotal notes on 64–66 assessment items for total 34 children in both AM and PM sessions and entering those online during three checkpoint periods. In fact, some teachers reported that they ended up taking a lot more than 64–66 anecdotal notes per child each time because their assumption about irreversibly progressive child development led them to keep recording observation until they eventually found the “right” one:

If I had an observation that kept children in the same level, we try to get another observation to move them up … We look at winter (data) and say she was a level five … But if we only had observations for this spring that put her at a level four, we would think twice “ok, she was a level five, so what do we need to observe to see if she moved up to a level six?” … That’s why it takes me so long. People just take notes and score them wherever they’re at. (However) we want to see, did we take the right observation to move them up?

If I had him at a level 3 for counting (today), tomorrow, I’d want to keep assessing in that area even though there are fifty-some different (observations) because, maybe by May, he can count to 11. So I already did his assessment, you know, for this semester, but I end up wanting to keep doing that and seeing if they get higher. So then, I spend a lot of time on it because I want it to be right.

Being under pressure to collect all the required anecdotal notes that are also consistent with expected rates of development, teachers spent most of time monitoring and recording detailed information about various aspects of individuals. Free play time of 40–45 minutes functioned as an “observatory” of finer details of children’s activities and behaviors. In this process, a TS Gold booklet was considered as a thorough reference volume to “truly help teachers focus on what matters most” (TS Gold touring guide, 2011: 2). Therefore, teachers felt they should be able to easily locate or even memorize indicators and given examples to “know what to look and listen for as they observe and interact—and how to interpret what they see and hear” (p. 8). For example, one teacher said,

I have to use the (Teaching Strategies Gold) book and look up each thing and see where it can go. Once you memorize this, (and) if we keep doing this for years, which is what their plan is, it will get easier and it won’t take as much time.

For quick classroom reference, teachers made the TS Gold objectives easily accessible in various forms:

We have it posted on our teacher (bulletin) board by our lesson plans. Then we can write quickly the objective number or letter it goes to … It’s right there. Or maybe (we) even have it already written on a (observation) sheet so we have that with the (anecdotal) note.

In this context, not all children’s self-initiated play activities were considered equally important. The distinctions about children’s play were made. Teachers prioritized certain play types or playful experiences than others that yielded useful information about TS Gold objectives or helped children achieve those objectives. Therefore, teachers did not simply rely on the naturally occurring, spontaneous child observation opportunities. Instead, teachers intentionally organized play settings and activities for focused observation/assessment in one or more specific, TS Gold objectives:

Looking at the (Teaching Strategies) gold does help … How come we don’t have a lot of (observation) notes on this particular area? Is it because we’re not writing it down, or is it because we’re not doing a lot of activities around that subject? It helps to have our lesson plans more focused, maybe. We’re bringing in those objectives into our lesson planning. (We are) not teaching to the gold, but it’s giving us a little more focus … So it really makes us look at an activity from different ways.

In addition, free play time or child-initiated play was frequently interrupted as individual children were pulled aside to be directly assessed on certain things such as alphabet knowledge, number concepts, and operations.

“We need notes for the computer”: Functional, hierarchical, and analytical small group time

The legislative mandate for establishing a local school readiness data system has dramatically changed the nature and function of Head Start small group time as an intensively regulated pedagogical space in several ways. Small group activities are restricted to certain types that allow teachers to access children in one or more TS Gold objectives. An art project in and of itself is not considered an appropriate or an ideal task for a small group activity any more unless it provides a context for assessing TS Gold objectives or eventually leads to promoting TS Gold–related concept learning. One teacher said, “What is really driving lesson planning is, we need (observation) notes for the computer.” Teachers regularly monitor the number of documentation by objective/dimension for each and every child to “look at what areas need to be marked at.” One teacher’s description of the activity chosen for small group time indicates “teaching to the data collection” that TS Gold creates, not necessarily in the sense of teaching to the TS Gold but in the sense that ever-growing demands of TS Gold for a large collection of child assessment data drive decision making for certain activities and interactions:

It was like we needed to collect the data, because they told us to put it into the (Teaching Strategies) gold. So we entered all of this stuff into the (online) system … We were planning a classification activity because we needed notes on classification. The main driving force was, we hadn’t done an observation on classifying and sorting objects. So that was the motivator behind it … the only thing we were doing is these random lessons and sorting animals. It’s not tied to the whole curriculum. So it’s kind of like giving a child a test on something they’ve never done or experienced before, and then rating on it.

In order to better serve their needs to “take the anecdotal notes on all the students” at the same time, small group activities become far more regulated both spatially and temporally than before. Teachers set aside 10–20 minutes each day for a teacher-led small group activity in which all children participate at their regular assigned table or place. Teachers are in charge of a same, small group of five to six children for a week, a month, or even few months to focus their observation/assessment on just a few children in their own group. Along with the expressed imperative to “take the right observation” that determine children’s progression, one teacher described ideal small group time as “plan(ning) the next higher level (activity) to see what kids are there.” In order to facilitate the focused observation/assessment at an appropriate, “next higher” level, most of the teachers in this study broke up 3- to 5-year-old children by their age into three small groups with the assumption that older children are more advanced in their development and learning than younger children. In what follows, teachers seem to justify age-based grouping or segregation by referencing TS Gold data:

To move them around to find that good fit, I am looking more at some of the (Teaching Strategies) Gold and the anecdotal notes and really using them to help get that full picture as opposed to what you feel in your gut. (I am looking at) what the data is really telling you.

We took our Teaching Strategies Gold and grouped them. You know this group already knows how to count up to 20, and we can work on higher, but this group (not in my group) is only counting to 4 or 5, so let’s put those groups together.

Out of this spatial distribution of children, a “whole analytical pedagogy” is formed in which each stage of development and learning is hierarchized into small steps with increasing complexity (Foucault, 1977) as one teacher described below:

So for each group, we have a general idea, so we say we will work on numbers. And my group might be writing numbers, where the other group is counting to 5, or another group is just recognizing written numbers, so that’s kind of how we do it. (p. 159)

Therefore, individual children’s participation in certain tasks or activities during small group time is qualified or disqualified according to the ranks they occupy in a hierarchical space created by the TS Gold.

Final thoughts

This article demonstrates the centrality of TS Gold–based assessment practice in participating Head Start sites to the regulation of children, teachers, and their work on a more minute and everyday level. The choice of so-called research-based TS Gold seems to offer a “complete assessment solution” (TS Gold touring guide, 2011: 12) that helps Head Start programs ensure their compliance with child outcome data mandate and other Federal or State requirements: “By partnering with us, you can be confident that your program will meet Head Start’s high program standards” (Teaching Strategies Gold, Inc., n.d.-a: para 1). Each Head Start program is required to “choose child assessment instruments that are reliable and valid; developmentally, linguistically, and culturally appropriate for the population served; and aligned with the (Head Start Child Development and Learning) framework” (OHS, 2010: 4). With the most widely used “assessment solutions,” Head Start programs can avoid the extra work to justify their choice of an assessment tool on their own as it is believed to be already proven by “national early childhood experts” through “extensive research review and field testing”(Teaching Strategies Gold, Inc., 2011: 1–2). In addition, online data management and analysis tools that allow Head Start programs to easily pull information from the electronic system and look at child outcomes in aggregate appear to push more early childhood programs toward TS Gold.

However, the choice of TS Gold is not simply driven by the pressures of external demands for those data. It also works through teachers’ desire to maintain their focus on the whole child and defend deeply held, personal, and professional beliefs about play-based practices in early childhood education. TS Gold, unlike standardized testing on limited set of academic skills, is claimed to enable teachers to do so because it is based on ongoing observational assessment data in all areas that teacher themselves collect in their natural environments as children play or carry out playful activities. Ironically, the research findings in this article suggest that it actually extends and intensifies the techniques and effects of power operating on children by normalizing ongoing surveillance and control of children’s play and everyday classroom activities. Free play time and small group time are the two examples in this article that TS Gold dramatically changes to form a “functional,” “hierarchical,” and “analytical” space in Head Start classrooms (Foucault, 1977: 148).

Although, in principle, TS Gold implies a potential for teachers’ autonomy to exercise their professional judgment, the demands for a large collection of child assessment data placed upon teachers actually regulate and affect most noticeably on views teachers hold of themselves and their work. Particularly, it involves the process of reinventing new early childhood teacher identity by which teachers come to recognize themselves as those who should be ideally and potentially able to collect, analyze, and use child assessment data to better inform their practice.

Teachers in this article voiced their concerns about not being able to “fully use the tool to the extent of what could have or should have been (used)” during its first year of implementation. While teaching to the data collection, some teachers felt that they ended up planning certain classroom materials and activities “in order to” collect all the required assessment data rather than they necessarily modified their curriculum “based on” assessment results. For example, one teacher reflected upon her experience with TS Gold:

I feel like it was a big learning curve because the assessment was new. So a whole lot more time was spent on learning the tool, entering the data into the computer, and scoring and not really using a whole a lot of (data) … It was kind of survival, and you did what you had to do just to get it done … Instead of just looking at it (and saying), “we need anecdotal notes on this area to put in” … (We should ask ourselves) What do we have to provide?, what do we have to do to kick them up to the next level, or progress into where they should be developmentally?

Teachers in this study expressed their desire to know how to “use” assessment data to guide and individualize instruction in reference to the TS Gold “activity library” that recommends appropriate activities on the basis of a child’s preliminary or finalized checkpoint assessment levels entered in the system:

This is really incredible. They have planning activities, and you can search for them. They will say, to help teach letter recognition, these are games you can play, or these are things you can do. They tell you different ways you can individualize for your children.

A growing sense of urgency to make data-informed decisions rationalizes teachers’ increasing reliance on TS Gold tools that automatically link assessment data to certain curricular activities for quick, “effortless” curriculum solutions. For example, a new digital planning tool, “Gold plus,” promises

Creating engaging daily and weekly plans becomes effortless. You’ll have instant access to a wealth of rich resources for all the major parts of your day—so planning that used to take days or weeks can now be done in just minutes. (Teaching Strategies Gold, Inc., n.d.-b: para 3)

In this process, TS Gold has been increasingly taken for granted as necessarily telling truth about children’s development and learning, which in turn rationalizes shift and burdensome intensification of teachers’ work associated with collecting data on child outcome.

This article adds valuable description and insight into technologies of power or power of assessment technologies that presuppose and at the same time reinforce self-steering capacities of early childhood teachers to regulate curriculum and pedagogy in certain ways. This challenges the use of a simple, oppositional binary of power versus resistance or clear demarcation between compliance and resistance in policy studies because power does not always work against but through teachers’ desire to do the very best for children’s development and learning. In this article, although seemingly considered to liberate children from external standardized testing regime, “authentic, ongoing observational system” like TS Gold in fact exemplifies the new governing pattern by which children, teachers, and their work are regulated through predetermined, universal norms and standards for “all,” with constructions of sameness/otherness and (ab)normality.