What If Only What Can be Counted Will Count? A Critical Examination of Making Educational Practice “Scientific”

Abstract

Background/Context

In recent decades, federal policymakers have pushed for education to be a more “scientific” endeavor. While scholars have considered the implications of this orientation for educational researchers, less attention has been given to its impact on educational practitioners.

Purpose/Focus of the Study

By focusing on the local interpretation and implementation of a multi-tiered system of supports (MTSS) model in one Midwestern school district, this study documents the translation of a comprehensive reform initiative meant to make educational practice more data-driven and scientific. With particular attention to interactions between district and building administrators, classroom teachers, and a group of outside consultants, we also consider the consequential effects of principal–agent relations in determining how learners (should) learn and teachers (should) teach.

Research Design

Using ethnographic methods over a period of five months, this study emerged from a larger project examining the work of educators in a rural district that includes 18 schools and serves approximately 7,600 students from racially, culturally, and linguistically diverse backgrounds. With MTSS as the unifying agenda across multiple interactions that involved a cross-section of the district's staff, administrative leaders, and outside consultants, we analyzed fieldnotes generated from participant observation during MTSS-specific meetings and semi-structured, individual interviews conducted with key implementation principals and agents. Other fieldnotes and interviews provided confirmation of our primary analysis, as well as supplementary perspectives from building and classroom contexts.

Findings

Through our analysis, we found that implementation leaders presumed the infallibility of the MTSS model; relied exclusively on certain forms of quantitative data; standardized the individual needs of learners, processes of learning, and roles of teachers; and insisted on fidelity of intervention as an end in itself.

Conclusions/Recommendations

Implementation leaders invoking research to inform practice can sometimes silence practitioners rather than foster their substantive involvement and understanding. This marginalizes certain types of knowledge that can contribute to understanding students’ needs, and it forces practitioners to be data-deferent rather than data-driven. The concept of implementation fidelity also needs to be reconsidered—not as an absolute good but with the necessary flexibility afforded to practitioners who are (1) educated in the essential components of available interventions, (2) able to become fluent through practice, and (3) allowed to exercise their professional expertise and judgment as appropriate.

Get full access to this article

View all access options for this article.

References

Agar

(2008). The professional stranger: An informal introduction to ethnography. Bingley, UK: Emerald Group Publishing.

Agar

(2013). The lively science: Remodeling human social research. Minneapolis, MN: Mill City Press, Inc.

Albers

C. A.

, & Martínez

R. S.

(2015). Promoting academic success with English language learners: Best practices for RTI. New York, NY: Guilford Publications.

Artiles

A. J.

, & Kozleski

E. B.

(2010). What counts as response and intervention in RTI? A sociocultural analysis. Psicothema, 22, 949—954.

Berends

, Bodilly

S. J.

, & Kirby

S. N.

(2002). Facing the challenges of whole-school reform. New American schools after a decade. Santa Monica, CA: RAND.

Berliner

D. C.

(2002). Educational research: The hardest science of all. Educational Researcher, 31(8), 18—20.

Biesta

(2007). Why “what works” won't work: Evidence-based practice and the democratic deficit in educational research. Educational Theory, 57(1), 1—22.

Borman

K. M.

, Carter

, Aladjem

D. K.

, & LeFlock

K. C.

(2004). Challenges for the future of comprehensive school reform. Washington, DC: The National Clearinghouse for Comprehensive School Reform.

Borman

G. D.

, Hewes

G. M.

, Overman

L. T.

, & Brown

(2003). Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research, 73(2), 125—230.

10.

Coburn

C. E.

, & Turner

E. O.

(2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99—111.

11.

Cross

C. T.

(2004). Political education: National policy comes of age. New York, NY: Teachers College Press.

12.

Daly

E. J.

, Hofstadter

K. I.

, Martinez

R. S.

, & Andersen

(2010). Selecting academic interventions for individual students. In Peacock

G. G.

, Ervin

, & Daly

E. J.

(Eds.), Practical handbook of school psychology: Effective practices for the 21st century (pp. 115—132). New York, NY: Guilford Publications.

13.

Desimone

(2002). How can comprehensive school reform models be implemented? Review of Educational Research, 72(3), 433—480.

14.

Donmoyer

(2014). What if educational inquiry were neither a social science nor a humanities field? Revisiting Joseph Schawb's “The Practical” in the aftermath of the science wars. Educational Policy Analysis Archives, 22(8).

15.

Eisenhart

, & Towne

(2003). Contestation and change in national policy on “scientifically based” education research. Educational Researcher, 32(7), 31—38.

16.

Erickson

(2014). Scaling down: A modest proposal for practice-based policy research in teaching. Education Policy Analysis Archives, 22(9). http://dx.doi.org/10.14507/epaa.v22n9.2014

17.

Erickson

, & Gutierrez

(2002). Culture, rigor, and science in educational research. Educational Researcher, 31(8), 21—24.

18.

Ferris

J. M.

(1992). School-based decision making: A principal-agent perspective. Educational Evaluation and Policy Analysis, 14(4), 333—346.

19.

Fluehr-Lobban

(1994). Informed consent in anthropological research: We are not exempt. Human Organization, 53(1), 1—10.

20.

Fuchs

, Mock

, Morgan

P. L.

, & Young

C. L.

(2003). Responsiveness to intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research and Practice, 18(3), 157—171.

21.

Geertz

(1976). From the native's point of view: On the nature of anthropological understanding. In Basso

, & Selby

(Eds.), Meaning in anthropology (pp. 221—237). Albuquerque, NM: University of New Mexico Press.

22.

Harn

, Parisi

, & Stoolmiller

(2013). Balancing fidelity with flexibility and fit: What do we really know about fidelity of implementation in schools? Exceptional Children, 79, 181—193.

23.

Honig

M. I.

, & Venkateswaran

(2012). School-central office relationships in evidence use: Understanding evidence use as a systems problem. American Journal of Education, 118(2), 199—222.

24.

Howe

K. R.

(2009). Isolating science from the humanities: The third dogma of educational research. Qualitative Inquiry, 15(4), 766—784.

25.

Kansas MTSS. (2008). Overview. Retrieved from www.kansasmtss.org/overview.htm

26.

Klingner

J. K.

, & Edwards

P. A.

(2006). Cultural considerations with response to intervention models. Reading Research Quarterly, 41(1), 108—117.

27.

Kretlow

A. G.

, & Helf

S. S.

(2013). Teacher implementation of evidence-based practices in Tier 1: A national survey. Teacher Education and Special Education, 36(3), 167—185.

28.

Labaree

(2004). The trouble with ed schools. New Haven, CT: Yale University Press.

29.

Lortie

(1975/2002). Schoolteacher: A sociological study. Chicago, IL: University of Chicago Press.

30.

Mandinach

E. B.

(2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71—85.

31.

Mandinach

E. B.

, & Gummer

E. S.

(2013). A systemic view of implementing data literacy in educator preparation. Educational Researcher, 42(1), 30—37.

32.

Martínez

R. S.

(2014). Best practices in instructional strategies for reading in general education. In Thomas

, & Harrison

(Eds.), Best practices in school psychology VI. Bethesda, MD: National Association of School Psychologists.

33.

Martínez

R. S.

, & Nellis

(2008). A school-wide approach for promoting academic wellness for all students. In Doll

, & Cummings

(Eds.), Transforming school mental health services (pp. 143—164). Thousand Oaks, CA: Corwin Press.

34.

McCook

J. E.

(2006). The RTI guide: Developing and implementing a model in your schools. Horsham, PA: LRP Publications.

35.

Merriam

S. B.

(2002). Qualitative research in practice: Examples for discussion and analysis. San Francisco, CA: Jossey-Bass.

36.

Merriam

S. B.

(2009). Qualitative research: A guide to design and implementation. San Francisco, CA: Jossey-Bass.

37.

Molloy

L. E.

, Moore

J. E.

, Trail

, Van Epps

J. J.

, & Hopfer

(2013). Understanding real-world implementation quality and “active ingredients” of PBIS. Prevention Science, 14, 593—605.

38.

Phillips

D. C.

(2014). Research in the hard sciences, and in very hard “softer” domains. Educational Researcher, 43(1), 9—11.

39.

Reinke

, Herman

K. C.

, & Stormont

(2013). Classroom-level behavior supports in schools implementing SW-PBIS: Identifying areas of enhancement. Journal of Positive Behavior Interventions, 15(1), 39—50.

40.

Roderick

(2012). Drowning in data but thirsty for analysis. Teachers College Record, 114, 1—9.

41.

Ross

S. M.

, & Gil

(2004). The past and future of comprehensive school reform: Perspectives from a researcher and practitioner. Washington, DC: The National Clearinghouse for Comprehensive School Reform.

42.

Rowan

, Barnes

, & Camburn

(2004). Benefiting from comprehensive school reform: A review of research on CSR implementation. Washington, DC: The National Clearinghouse for Comprehensive School Reform.

43.

Rubin

H. J.

, & Rubin

I. S.

(2012). Qualitative interviewing: The art of hearing data. Los Angeles, CA: SAGE.

44.

Ryan

G. W.

, & Bernard

H. R.

(2003). Techniques to identify themes. Field Methods, 15(1), 85—109.

45.

Sailor

(2009). Making RTI work: How smart schools are reforming education through schoolwide response-to-intervention. San Francisco, CA: John Wiley & Sons.

46.

Sanjek

(2014). Ethnography in today's world: Color full before color blind. Philadelphia, PA: University of Pennsylvania Press.

47.

Santoro

D. A.

(2011). Good teaching in difficult times: Demoralization in the pursuit of good work. American Journal of Education, 118(1), 1—23.

48.

Sassen

(1990). Economic restructuring and the American city. Annual Review of Sociology, 16, 465—490.

49.

Shavelson

R. J.

, & Towne

(Eds.). (2002). Scientific research in education. Washington, DC: National Academy Press.

50.

Simmons

D. C.

, Kame'enui

E. J.

, Harn

, Coyne

M. D.

, Stoolmiller

, Santoro

L. E.

, & Kaufman

N. K.

(2007). Attributes of effective and efficient kindergarten intervention: An examination of instructional time and design specificity. Journal of Learning Disabilities, 40, 331—347.

51.

Sink

C. A.

, & Ockerman

M. S.

(2016). School counselors and the multi-tiered system of supports: Cultivating systemic change and equitable outcomes. The Professional Counselor, 6(3), v—ix.

52.

Spillane

J. P.

(2012). Data in practice: Conceptualizing the data-based decision-making phenomena. American Journal of Education, 118(2), 113—141.

53.

St. Pierre

(2002). “Science” rejects post-modernism. Educational Researcher, 31(8), 25—27.

54.

Stringfield

(2000). Choosing success. Washington, DC: American Federation of Teachers.

55.

Stull

, & Ng

(2016). Majority educators in a United States minority/immigrant public school district: The case of Garden City, Kansas. Human Organization, 75(2), 181—191.

56.

Tyack

, & Cuban

(1997). Tinkering toward utopia: A century of public school reform. Cambridge, MA: Harvard University Press.

57.

Wohlstetter

, Datnow

, & Park

(2008). Creating a system for data-driven decision-making: Applying the principal-agent framework. School Effectiveness and School Improvement, 19(3), 239—259.