Improving Science Achievement—Is It Possible? Evaluating the Efficacy of a High School Chemistry and Physics Project-Based Learning Intervention

Abstract

Crafting Engaging Science Environments is a high school chemistry and physics project-based learning intervention that meets Next Generation Science Standards performance expectations. It was administered to a diverse group of over 4,000 students in a randomized control trial in California and Michigan. Results show that treatment students, on average, performed 0.20 standard deviations higher than control students on an independently developed summative science assessment. Mediation analyses show an indirect path between teacher- and student-reported participation in modeling practices and science achievement. Exploratory analyses indicate positive treatment effects for enhancing college ambitions. Overall, results show that improving secondary school science learning is achievable with a coherent system comprising teacher and student learning experiences, professional learning, and formative unit assessments that support students in “doing” science.

Keywords

achievement effect size evaluation experimental research hierarchical linear modeling learning environments program evaluation science education

Get full access to this article

View all access options for this article.

References

Bloom

(2005). Randomizing groups to evaluate place-based programs. MDRC.

Blumenfeld

P. C.

Soloway

Marx

R. W.

Krajcik

J. S.

Guzdial

Palincsar

(1991). Motivating project-based learning: sustaining the doing, supporting the learning. Educational Psychologist, 26(3–4), 369–398. https://doi.org/10.1080/00461520.1991.9653139

Chen

C.-H.

Yang

Y.-C.

(2019). Revisiting the effects of project-based learning on students’ academic achievement: A meta-analysis investigating moderators. Educational Research Review, 26(February), 71–81. https://doi.org/10.1016/j.edurev.2018.11.001

Cheng

Reckase

(2020). The effect of gender differences and similarities on science performance [Conference section]. American Educational Research Association Conference, San Francisco, CA, United States. https://www.aera.net/Events-Meetings/Annual-Meeting/Previous-Annual-Meetings/2020-Annual-Meeting (Conference canceled)

Cheung

Slavin

R. E.

Kim

Lake

(2016). Effective secondary science programs: A best-evidence synthesis. Journal of Research in Science Teaching, 54(1), 58–81. https://doi.org/10.1002/tea.21338

Condliffe

Quint

Visher

M. G.

Bangser

M. R.

Drohojowska

Saco

Nelson

(2017). Project-based learning: A literature review (Working Paper). MDRC. https://eric.ed.gov/?id=ED578933

Cooper

M. M.

(2020). The crosscutting concepts: Critical component or “third wheel” of three-dimensional learning? Journal of Chemical Education, 97(4), 903–909. https://doi.org/10.1021/acs.jchemed.9b01134

Darling-Hammond

Hyler

M. E.

Gardner

(2017, May). Effective teacher professional development [Research Brief]. Learning Policy Institute.

Dewey

(1938). Experience & education. Kappa Delta Pi.

10.

Engels

Miller

Squires

Jennewein

J. S.

Eitel

(2019). The confluence approach: Developing scientific literacy through project-based learning and place-based education in the context of NGSS. Electronic Journal of Science Education, 23(3), 33–58. https://files.eric.ed.gov/fulltext/EJ1228452.pdf

11.

Frank

K. A.

Maroulis

S. J.

Duong

M. Q.

Kelcey

B. M.

(2013). What would it take to change an inference? Using Rubin’s causal model to interpret the robustness of causal inferences. Educational Evaluation and Policy Analysis, 35(4), 437–460. https://doi.org/10.3102/0162373713493129

12.

Garet

M. S.

Porter

A. C.

Desimone

Birman

B. F.

Yoon

K. S.

(2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945. https://doi.org/10.3102/00028312038004915

13.

Harris

C. J.

Krajcik

J. S.

Pellegrino

J. W.

DeBarger

A. H.

(2019). Designing knowledge-in-use assessments to promote deeper learning. Educational Measurement: Issues and Practice, 38(2), 53–67. https://doi.org/10.1111/emip.12253

14.

Harris

C. J.

Penuel

D’Angelo

DeBarger

Gallagher

Kennedy

Cheng

Krajcik

(2015). Impact of project-based curriculum materials on student learning in science: Results of a randomized controlled trial. Journal of Research in Science Teaching, 52(10), 1362–1385. https://doi.org/10.1002/tea.21263

15.

Hinojosa

Rapaport

Jaciw

LiCalsi

Zacamy

(2016). Exploring the foundations of the future STEM workforce: K–12 indicators of postsecondary STEM success. U.S. Department of Education.

16.

Hyde

J. S.

Linn

M. C.

(2006). Gender similarities in mathematics and science. Science, 314(5799), 599–600. https://doi.org/10.1126/science.1132154

17.

Krajcik

(2014). The importance of viable models in the construction of professional development. In Wojnowski

B. S.

Pea

C. P.

(Eds.), Models and approaches to STEM professional development (pp. 33–44). NSTA Press. https://static.nsta.org/pdfs/samples/PB322Xf.pdf

18.

Krajcik

Czerniak

(2018). Teaching science in elementary and middle school: A project-based approach (5th ed.). Taylor & Francis.

19.

Krajcik

Shin

(2014). Project-based learning. In Sawyer

R. K.

(Ed.), The Cambridge handbook of the learning sciences (pp. 275–297). Cambridge University Press.

20.

Lee

Nasir

Pea

de Royston

(2020). Reconceptualizing learning: A critical task for knowledge-building and teaching. In Nasir

N. S.

Lee

C. D.

Pea

de Royston

M. M.

(Eds.), Handbook of the cultural foundations of learning (pp. xvii–xxxv). Routledge.

21.

Lorah

(2018). Effect size measures for multilevel models: Definition, interpretation, and TIMSS example. Large-scale Assessments in Education, 6(1), Article 8. https://doi.org/10.1186/s40536-018-0061-2

22.

Maestrales

Dezendorf

Tang

Salmela-Aro

Bartz

Juuti

Lavenon

Krajcik

Schneider

(2021). U.S. and Finnish high school science engagement during the COVID-19 pandemic. International Journal of Psychology. Advance online publication. https://doi.org/10.1002/ijop.12784

23.

Maestrales

Zhai

Touitou

Baker

Schneider

Krajcik

(2021). Using machine learning to evaluate multidimensional assessments of chemistry and physics. Journal of Science Education and Technology, 30(2), 239–254. https://doi.org/10.1007/s10956-020-09895-9

24.

National Academies of Sciences, Engineering, and Medicine. (2019). Science and engineering for Grades 6–12: Investigation and design at the center. The National Academies Press.

25.

National Assessment of Educational Progress. (2021). Results from the 2019 Science Assessment. U.S. Department of Education and the Institute of Education Sciences.

26.

National Center for Education Statistics. (2017). The condition of education 2017. Author. https://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2017144

27.

National Research Council. (2000). How people learn: Brain, mind, experience, and school (Expanded ed.). National Academies Press.

28.

National Research Council. (2012). A framework for K–12 science education: Practices, crosscutting concepts, and core ideas. National Academies Press.

29.

National Science Board. (2019). The skilled technical workforce: Crafting America’s science and engineering enterprise. https://www.nsf.gov/nsb/publications/2019/nsb201923.pdf

30.

National Science Board. (2020). The state of U.S. science and engineering 2020. https://ncses.nsf.gov/pubs/nsb20201

31.

National Science Teacher Association. (2017). NGSS hub. https://ngss.nsta.org/

32.

NGSS Lead States. (2013). Next generation science standards: For states, by states. National Academies Press. https://epsc.wustl.edu/seismology/book/presentations/2014_Promotion/NGSS_2013.pdf

33.

Organisation for Economic Co-operation and Development. (2020). Science performance (PISA)—indicator. https://data.oecd.org/pisa/science-performance-pisa.htm

34.

Pinkard

Erete

Martin

McKinney de Royston

(2017). Digital Youth Divas: Exploring narrative-driven curriculum to spark middle school girls’ interest in computational activities. Journal of the Learning Sciences, 26(3), 477–516. https://doi.org/10.1080/10508406.2017.1307199

35.

Pituch

K. A.

Murphy

D. L.

Tate

R. L.

(2009). Three-level models for indirect effects in school- and class-randomized experiments in education. Journal of Experimental Education, 78(1), 60–95. https://doi.org/10.1080/00220970903224685

36.

Polikoff

M. S.

Campbell

S. E.

Koedel

Q. T.

Haraway

Gasparian

(2018). The formalized processes districts use to evaluate textbooks [University of Southern California Working Paper].

37.

Raudenbush

S. W.

(1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173–185. https://doi.org/10.1037/1082-989X.2.2.173

38.

Raudenbush

S. W.

Bryk

(2002). Hierarchical linear models (2nd ed.). Sage.

39.

Riegle-Crumb

King

(2010). Questioning a White male advantage in STEM: Examining disparities in college major by gender and race/ethnicity. Educational Researcher, 39(9), 656–664. https://doi.org/10.3102/0013189X10391657

40.

Riegle-Crumb

King

Irizarry

(2019). Does STEM stand out? Examining racial/ethnic gaps in persistence across postsecondary fields. Educational Researcher, 48(3), 133–144. https://doi.org/10.3102/0013189X19831006

41.

Riegle-Crumb

Moore

Ramos-Wada

(2011). Who wants to have a career in science or math? Exploring adolescents’ future aspirations by gender and race/ethnicity. Science Education, 95(3), 458–476. https://doi.org/10.1002/sce.20431

42.

Rosebery

A. S.

Warren

Tucker-Raymond

(2016). Developing interpretive power in science teaching. Journal of Research in Science Teaching, 53(10), 1571–1600. https://doi.org/10.1002/tea.21267

43.

Sasson

Yehuda

Malkinson

(2018). Fostering the skills of critical thinking and question-posing in a project-based learning environment. Thinking Skills and Creativity, 29(September), 203–212. https://doi.org/10.1016/J.TSC.2018.08.001

44.

Schneider

Klager

Chen

I.-C.

Burns

(2016). Transitioning into adulthood: Striking a balance between support and independence. Policy Insights From the Behavioral and Brain Sciences, 3(1), 106–113. https://doi.org/10.1177/2372732215624932

45.

Schneider

Krajcik

Lavonen

Salmela-Aro

(2020). Learning science: The value of crafting engagement in science environments. Yale University Press. https://doi.org/10.2307/j.ctvwcjfk1

46.

Sinatra

G. M.

Heddy

B. C.

Lombardi

(2015). The challenges of defining and measuring student engagement in science. Educational Psychologist, 50(1), 1–13. https://doi.org/10.1080/00461520.2014.1002924

47.

Spybrook

Bloom

Congdon

Hill

Martinez

Raudenbush

S. W.

(2011). Optimal design plus empirical evidence: Documentation for the “optimal design.” http://www.hlmsoft.net/od/od-manual-20111016-v300.pdf

48.

Spybrook

Westine

C. D.

Taylor

J. A.

(2016). Design parameters for impact research in science education: A multistate analysis. AERA Open, 2(1). https://doi.org/10.1177/2332858415625975

49.

Tipton

(2014). How generalizable is your experiment? An index for comparing experimental samples and populations. Journal of Educational and Behavioral Statistics, 39(6), 478–501. https://doi.org/10.3102/1076998614558486

50.

van den Bergh

Ros

Beijaard

(2015). Teacher learning in the context of a continuing professional development programme: A case study. Teaching and Teacher Education, 47(April), 142–150. https://doi.org/10.1016/j.tate.2015.01.002

51.

What Works Clearinghouse. (2020). What Works Clearinghouse™: Standards handbook, version 4.1. Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/wwc/Docs/referenceresources/WWC-Standards-Handbook-v4-1-508.pdf

52.

Wooldridge

J. M.

(2010). Econometric analysis of cross section and panel data. MIT Press.

53.

Zell

Krizan

Teeter

S. R.

(2015). Evaluating gender similarities and differences using metasynthesis. American Psychologist, 70(1), 10–20. https://doi.org/10.1037/a0038208

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.45 MB