In this article, I outline how, with the appropriate mix of external requirements and internal motivation, structure and capacity, a school can promote and maintain a culture of evaluative practices, specifically data use, for continuous improvement. I draw upon qualitative data conducted in a study of an Edison Project school. The findings from this paper are the result of a larger 4-year study examining Comprehensive School Reform in 12 Title 1 schools in three states. Interviews were conducted with principals, teachers, other staff, parents and students, as well as with state officials, the superintendent, assistant superintendent and other high level administrators in the district. Observations were undertaken at the classroom level. Key findings include the importance of obtaining a multidimensional view of educational change. That is, we need to obtain a picture of the school reform environment (i.e., interconnections between the state, district, and school levels), and simultaneously investigate the interplay among critical school levers that are required to promote and maintain a culture of data use for school improvement. From this multidimensional view, we are better positioned to investigate and evaluate school change efforts.
Get full access to this article
View all access options for this article.
References
1.
Argyris, C. (1992). On organizational learning. Cambridge, MA: Blackwell Publishers.
2.
American Federation of Teachers (AFT). (2001). Student achievement in Edison Schools: Mixed results in an ongoing experiment. Accessed at: http://www.aft.org/research/edisonproject/preface.htm.
3.
Bruner, J. (1971). Toward a theory of instruction. Cambridge, MA: Harvard University Press.
4.
Chubb, J. E. (1998). The performance of privately managed schools: An early look at the Edison Project. In P. E. Peterson & B. C. Hassel (Eds.), Learning from school choice (pp. 213-248). Washington, DC: Brookings Institution Press.
5.
Costa, A. L., & Kallick, B. (1995). Through the lens of a critical friend. In A. L. Costa & B. Kallick (Eds.), Assessment in the learning organization: Shifting the paradigm (pp. 153-156). Alexandria, VA: Association for Supervision and Curriculum Development.
6.
Cousins, J. B. (1996). Understanding organizational learning for educational leadership and school reform. In K. Leithwood, J. Chapman, D. Corson, P. Hallinger, & A. Hart (Eds.), International handbook of educational leadership and administration (pp. 589-652). Boston: Kluwer Academic Press.
7.
Cousins, J. B., Brandon, P., Goh, S. C., & Heck, R. (November, 2003). A comparative study of organizational readiness for evaluation. Paper presented at the annual meeting of the American Evaluation Association, Reno, Nevada.
8.
Cousins, J. B., & Earl, L. M. (1995). The case for participatory evaluation: Theory, research, and practice. In J. B. Cousins & L. M. Earl (Eds.), Participatory evaluation in education: Studies in use and organizational learning (pp. 3-18). London: Falmer.
9.
Darling-Hammond, L., Hightower, A. M., Husbands, J. L., LaFors, J. R., & Young, V. M. (April, 2002). Building instructional quality: Inside-out, bottom-up, and top-down perspectives on San Diego’s School Reform. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, Louisiana.
10.
Datnow, A., & Castellano, M. (2001). Success for All: District and school leadership. In J. Murphy & A. Datnow (Eds.), Leadership lessons from Comprehensive School Reforms (pp. 187-208). Thousand Oaks, CA: Corwin Press, Inc.
11.
Datnow, A., & Kemper, E. (2002). From statehouse to schoolhouse: The implementation of Comprehensive School Reform in the Era of CSRD. Report prepared as a deliverable to OERI, US Department of Education for CRESPAR Project 4.2.
12.
Datnow, A., & Stringfield, S. (2000). Working together for reliable school reform. Journal of Education for Students Placed at Risk, 5, 183-204.
13.
Deal, T. E., & Peterson, K. D. (1999). Shaping school culture: The heat of leadership. San Francisco, CA: Jossey-Bass.
14.
Deci, E., & Ryan, R. (1985). Intrinsic motivation and self-determination in human behavior. New York: Plenum.
15.
Deci, E., & Ryan, R. (1987). The support of autonomy and control of behavior. Journal of Personality and Social Psychology, 53, 1024-1037.
16.
Desimone, L. (2002). How can comprehensive school reform models be successfully implemented?Review of Educational Research, 72, 433-479.
17.
Earl, L. (1998, July-August). Developing indicators: The call for accountability. Policy Options, 20-24.
18.
Earl, L., & Katz, S. (2002). Leading schools in a data rich world. In K. Leithwood, P. Hallinger, G. Furman, P. Grann, B. Mulford, & K. Riley (Eds.), The international handbook for leadership. Netherlands: Kluwer Academic Press.
19.
Feller, I. (2002). Performance measurement redux. American Journal of Evaluation, 23, 435-452.
20.
Finnan, C., & Meza, J. (2003). The accelerated schools: Can a leaders change the culture and embed reform? In J. Murphy & A. Datnow (Eds.), Leadership lessons from Comprehensive School Reforms (pp. 83-107). Thousand Oaks, CA: Corwin Press, Inc.
21.
Finnan, C., & Swanson, J. D. (2000). Accelerating the learning of all students: Cultivating culture change in schools, classrooms, and individuals. Boulder, CO: Westview Press.
22.
Forss, K., Cracknell, B., & Samset, K. (1994). Can evaluation help an organization to learn?Evaluation Review, 18, 574-591.
23.
Fullan, M. (1991). The new meaning of educational change. New York: Teachers College Press.
24.
Fullan, M. (1999). Change forces: The sequel. Philadelphia, PA: Falmer.
25.
Fullan, M. (2001). Leading in a culture of change. New York: Jossey-Bass, Incorporated Publishers.
26.
Guba, E. G., & Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.
27.
Hernandez, G., & Visher, M. (2001). Creating a culture of inquiry: Changing methods and on the use of evaluation in nonprofit organizations. Report on the Working on Workforce Development (WOW) Project. San Francisco, CA: The James Irvine Foundation.
28.
Huber, G. P. (1991). Organization all earning: The contributing processes and the literature. Organizational Science, 2, 88-115.
29.
Katz, S., Sutherland, S., & Earl, L. (2002). Developing an evaluation habit of mind. The Canadian Journal of Program Evaluation, 17, 103-119.
30.
Katz, S., Sutherland, S., & Earl, L. (2004). Towards an evaluation habit of mind: Mapping the journey. Teachers’ College Record.
31.
Louis, K. S. (1994). Beyond bureaucracy: Rethinking how schools change. School Effectiveness and School Improvement, 5, 2-24.
32.
Lusthaus, C., Anderson, G., & Murphy, E. (1995). Institutional assessment: A framework for strengthening organizational capacity for IDRC’s research partners. Ottawa: International Development Research Centre.
33.
MacBeath, J. (1998). I didn’t know he was ill: The role and value of the critical friend. In L. Stoll & K. Myers (Eds.), No quick fixes: Perspectives on schools in difficulty (pp. 118-132). London: Falmer.
34.
Miles, M. B., & Huberman, A. M. (1994). An expanded sourcebook: Qualitative data analysis. Thousand Oaks, CA: Sage.
35.
Miron, G., & Applegate, B. (2000). An evaluation of student achievement in Edison Schools opened 1995-96. Kalamazoo, MI: The Evaluation Center, Western Michigan University.
36.
Mislevy, R. J. (1996). Reading achievement test-score analysis 1996/97 King-Edison vs. control schools, Grades 1-3. Mount Clemens, MI: Mount Clemens Community Schools.
37.
Mislevy, R. J. (1997). Reading achievement test-score analysis: 1995/96 Dodge-Edison vs. control schools, Grades 1-3. Witchita, KS: Witchita Unified School District.
38.
Newman, F. M. (1996). Authentic achievement: Restructuring schools for intellectual quality. San Francisco, CA: Jossey-Bass.
39.
No Child Left Behind Act. (2001). Accessed at htp://www.ed.gov/policy/elsec/leg/esea02/107-110.pdf.
40.
Owens, R. G. (1991). Organizational behavior in education. Englewood Cliffs, NJ: Prentice-Hall.
41.
Owen, J. M., & Lambert, K. C. (1998). Evaluation and the information needs of organizational leaders. Evaluation Practice, 19, 355-365.
42.
Patton, M. Q. (1997). Utilization focused evaluation: The new century text. Thousand Oaks: Sage.
43.
Patton, M. Q. (1998). Discovering process use. Evaluation, 4, 225-233.
44.
Perrin, B. (1998). Effective use and misuse of performance measurement. American Journal of Evaluation, 19, 367-379.
45.
Preskill, H., & Torres, R. (1999). Evaluative inquiry for organizational learning. Thousand Oaks, CA: Sage.
46.
Preskill, H., Zuckerman, B., & Matthews, B. (2003). An exploratory study of process use: Findings and implications for future research. American Journal of Evaluation, 24, 423-442.
47.
RAND Corporation (in press). Accessed at http://www.RAND.org/child/projects/edison.html.
48.
Sarason, S. B. (1996). Revisiting the culture of the school and the problem of change. New York: Teachers College Press.
49.
Senge, P. (2000). A fifth discipline resource: Schools that learn. New York: Doubleday Dell Publishing Group Inc.
50.
Simon, H. A. (1991). Bounded rationality and organizational learning. Organizational Science, 2, 125-134.
51.
Sutherland, S., & Katz, S. (2003, November). Concept mapping methodology: A catalyst for organizational learning. Paper presented at the annual American Evaluation Association conference, Reno, Nevada.
