Calls for evidence-based decision making have become increasingly prominent on the educational landscape. School district central offices increasingly experience these demands. Yet there are few empirical studies of evidence use at the district level. Furthermore, research on evidence use among policy makers in noneducation settings raises questions about the models of decision making promoted by evidence-use policies, suggesting that they do not take into account key features of the interpretive process or the organizational conditions that shape how decision making unfolds.
Purpose/Objective/Research Questions/Focus of Study
The central premise of this article is that only by understanding the patterns by which personnel in school district central offices actually use information, and the factors that affect this use, can we begin to understand the promise and possibilities of evidence use. We ask: What is the role of evidence in instructional decision making at the central office level? What factors shape how decision processes unfold?
Research Design
We draw on data from a longitudinal case study of one midsize urban district, which we followed from 2002 to 2005. We relied on in-depth interviewing, sustained observation, and document analysis. We identified 23 decisions related to instruction that were captured in our data over the 3 years and for which we had at least three independent sources of information. We analyzed each decision using a coding scheme that was developed from prior research and theory and elaborated through iterative coding. We then used matrices to compare across decisions and to surface and investigate emerging patterns.
Conclusions/Recommendations
We argue that decision making in complex organizations like school districts is centrally about interpretation, argumentation, and persuasion. These processes are shaped in crucial ways by preexisting working knowledge and practices that guide how people come to understand the nature of problems and possible avenues for solutions. They are also influenced by organizational and political factors, including the organizational structure of the central office, resource constraints, and leadership turnover. We close by suggesting implications for efforts to foster substantive and productive use of evidence at the central office level.
Get full access to this article
View all access options for this article.
References
1.
BabbS. (1996). “A true American system of finance”: Frame resonance in the U.S. labor movement, 1866 to 1886. American Sociological Review, 61, 1033—1052.
2.
BacharachS. (1988). Notes on a political theory of educational organizations. In WestobyA. (Ed.), Culture and power in educational organizations (pp. 277—288). Milton Keynes, England: Open University Press.
3.
BarleyS. R. (1990). Images of imaging: Notes on doing longitudinal field work. Organizational Science, 1, 220—247.
4.
BenfordR. D., & SnowD. A. (2000). Framing processes and social movements: An overview and assessment. Annual Review of Sociology, 26, 611—639.
5.
BinderA. J. (2002). Contentious curricula: Afrocentrism and creationism in American public schools. Princeton, NJ: Princeton University Press.
6.
BirkelandS.Murphy-GrahamE., & WeissC. (2005). Good reasons for ignoring good evaluation: The case of the Drug Abuse Resistance Education (D.A.R.E.) program. Evaluation and Program Planning, 28, 247—256.
7.
CelioM. B., & HarveyJ. (2005). Buried treasure: Developing a management guide from mountains of school data. Seattle: University of Washington, Center on Reinventing Public Education (CRPE), Daniel J. Evance School of Public Affairs.
8.
Coalition for Evidence-Based Policy. (2002). Bringing evidence-driven progress to education: A recommended strategy for the U.S. Department of Education. Washington, DC: Author.
9.
CoburnC. E. (2001). Collective sensemaking about reading: How teachers mediate reading policy in their professional communities. Educational Evaluation and Policy Analysis, 23, 145—170.
10.
CoburnC. E.HonigM. I., & SteinM. K. (2009). What is the evidence on districts’ use of evidence? In BransfordJ.StipekD.GomezL.LamD., & VyeN. (Eds.), The role of research in educational improvement. Cambridge, MA: Harvard Educational Press.
11.
CoburnC. E., & TalbertJ. E. (2006). Conceptions of evidence-based practice in school districts: Mapping the terrain. American Journal of Education, 112, 469—495.
12.
CohenM. D.MarchJ. G., & OlsenJ. P. (1988). The garbage can model of organizational choice. In MarchJ. G. (Ed.), Decisions and organizations (pp. 294—334). Oxford, England: Basil Blackwell.
13.
CooleyW. W. (1983). Improving the performance of an educational system. Educational Researcher, 126, 4—12.
14.
CressD. M., & SnowD. A. (2000). The outcomes of homeless mobilization: The influence of organization, disruption, political mediation, and framing. American Journal of Sociology, 105, 1063—1104.
15.
DavidJ. L. (1981). Local uses of Title I evaluations. Educational Evaluation and Policy Analysis, 3, 27—39.
16.
DornbuschS. M., & ScottW. R. (1975). Evaluation and the exercise of authority. San Francisco: Jossey-Bass.
17.
EisenhartM. A., & HoweK. R. (1992). Validity in educational research. In LeCompteM. D.MillroyW. L., & PreissleJ. (Eds.), The handbook of qualitative research in education (pp. 643—680). San Diego, CA: Academic Press.
18.
ElmoreR., & BurneyD. (1997). School variation and systemic instructional improvement in Community School District #2, New York City. Pittsburgh, PA: High Performance Learning Communities Project, Learning Research and Development Center, University of Pittsburgh.
19.
FeldmanM. S., & MarchJ. G. (1988). Information in organizations as signal and symbol. In MarchJ. G. (Ed.), Decisions and organizations (pp. 409—428). Oxford, England: Basil Blackwell.
20.
FennemaE., & FrankeM. L. (1992). Teachers’ knowledge and its impact. In GrouwsD. A. (Ed.), Handbook of research on mathematics teaching and learning (pp. 147—164). New York: Macmillan.
21.
FissP. C., & HirschP. M. (2005). The discourse of globalization: Framing and sensemaking of an emerging concept. American Sociological Review, 70, 29—52.
22.
FligsteinN. (2001). Social skill and the theory of fields. Sociological Theory, 19, 105—125.
23.
GerthH., & MillsC. M. (1946). From Max Weber: Essays in sociology. New York: Oxford University Press.
24.
HannawayJ. (1989). Managers managing: The workings of an administrative system. New York: Oxford University Press.
25.
HartleyJ. F. (1994). Case studies in organizational research. In CassellC., & SymonG. (Eds.), Qualitative methods in organizational research: A practical guide (pp. 208—229). Thousand Oaks, CA: Sage.
26.
HightowerA.KnappM. S.MarshJ. A., & McLaughlinM. W. (2002). School districts and institutional renewal. New York: Teachers College Press.
27.
JohnsonB. L.Jr. (1999). The politics of research-information use in the education policy arena. Educational Policy, 131, 23—36.
28.
KennedyM. M. (1982). Working knowledge and other essays. Cambridge, MA: The Huron Institute.
29.
KnappM.CoplandM., & TalbertJ. (2003). Leading for learning: Reflective tools for school and district leaders. Seattle: University of Washington, Center for the Study of Teaching Policy.
30.
LigonG., & JacksonE. (1986, April). Linking outcomes to organizational planning. Paper presented at the annual meeting of the American Educational Research Association, San Francisco, CA.
31.
MalenB. (2006). Revisiting policy implementation as a political phenomenon: The case of reconstitution policies. In HonigM. (Ed.), New directions in education policy implementation: Confronting complexity (pp. 83—104). Albany: State University of New York Press.
32.
MarchJ. G. (1988). Decisions and organizations. Cambridge, England: Basil Blackwell.
33.
MarchJ. G., & OlsenJ. P. (1989). Rediscovering institutions: The organizational basis of politics. New York: Free Press.
34.
MassellD. (2001). The theory and practice of using data to build capacity: State and local strategies and their effects. In FuhrmanS. H. (Ed.), From the capitol to the classroom: Standards-based reform in the states. One hundredth yearbook of the National Society for the Study of Education (pp. 148—169). Chicago: National Society for the Study of Education (NSSE) and University of Chicago Press.
35.
McAdamD., & ScottW. R. (2005). Organizations and movements. In DavisG. F.McAdamD.ScottW. R., & ZaldM. N. (Eds.), Social movements and organizational theory (pp. 4—40). Cambridge, England: Cambridge University Press.
36.
MerriamS. B. (1998). Qualitative research and case study applications in education (Rev. ed.). San Francisco: Jossey-Bass.
37.
MeyerJ. W., & ScottW. R. (1983). Organizational environments: Ritual and rationality. Beverly Hills, CA: Sage.
38.
MilesM. B., & HubermanA. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
39.
National Institute of Child Health and Human Development. (2000). Report of the National Reading Panel. Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: Reports of the subgroups. Washington, DC: U.S. Government Printing Office.
40.
PfefferJ. (1981). Power in organizations. Marshfield, MA: Pitman.
41.
PoracJ. F.ThomasH., & Baden-FullerC. (1989). Competitive groups as cognitive communities: The case of Scottish knitwear manufacturers. Journal of Management Studies, 26, 397—416.
42.
SchneiderA., & IngramH. (1993). Social construction of target populations: Implications for politics and policy. American Political Science Review, 87, 334—347.
43.
ScribnerJ. D.AlemanE., & MaxcyB. (2003). Emergence of the politics of education field: Making sense of the messy center. Educational Administration Quarterly, 39, 10—40.
44.
ShulmanL. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 151, 1—22.
45.
ShulmanL. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1—22.
46.
SlavinR. E. (1989). PET and the pendulum: Faddism in education and how to stop it. Phi Delta Kappan, 70, 752—758.
47.
SlavinR. E. (1998). Show me the evidence! Proven and promising programs for America's schools. Thousand Oaks, CA: Corwin Press.
48.
SnowD. A., & BenfordR. D. (1992). Master frames and cycles of protest. In MorrisA., & MuellerC. (Eds.), Frontiers in social movement theory (pp. 135—155). New Haven, CT: Yale University Press.
49.
SpillaneJ. P. (1998). State policy and the non-monolithic nature of the local school district: Organizational and professional considerations. American Educational Research Journal, 35, 33—63.
50.
SpillaneJ. P. (2000). Cognition and policy implementation: District policymakers and the reform of mathematics education. Cognition and Instruction, 18, 141—179.
51.
SpradleyJ. (1979). The ethnographic interview. New York: Holt, Rinehart and Winston.
52.
StarbuckW. H., & MillikenF. J. (1988). Executives’ perceptual filters: What they notice and how they make sense. In HambrickD. C. (Ed.), The executive effect: Concepts and methods for studying top managers (pp. 35—65). Greenwich, CT: JAI Press.
53.
SteinM. K., & NelsonB. S. (2003). Leadership content knowledge. Educational Evaluation and Policy Analysis, 25, 423—448.
54.
StoneD. A. (1988). Policy paradox and political reason. Glenview, IL: Scott Foresman.
55.
StraussA., & CorbinJ. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park, CA: Sage.
56.
ThompsonA. G. (1992). Teachers’ beliefs and conceptions: A synthesis of the research. In GrouwsD. A. (Ed.), Handbook on research on mathematics teaching and learning (pp. 127—146). New York: Macmillan.
