The Development and Validation of a Measure of Student Attitudes Toward Science,Technology,Engineering,and Math (S-STEM)

Abstract

Using an iterative design along with multiple methodological approaches and a large representative sample, this study presents reliability, validity, and fairness evidence for two surveys measuring student attitudes toward science, technology, engineering, and math (S-STEM) and interest in STEM careers for (a) 4th- through 5th-grade students (Upper Elementary S-STEM) and (b) 6th- through 12th-grade students (Middle/High S-STEM). Findings from exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) suggested the use of a four-factor structure to measure student attitudes toward science, math, engineering/technology, and 21st century skills. Subject matter experts and literature reviews provided evidence of content validity. Reliability levels were high for both versions. Furthermore, both the Upper Elementary S-STEM and Middle/High S-STEM Surveys demonstrated evidence of configural, metric, and scalar invariance across grade levels, races/ethnicities, and genders. The findings support the validity of interpretations and inferences made from scores on the instruments’ items and subscales.

Keywords

K-12 STEM measurement scale development measurement invariance attitudes

Get full access to this article

View all access options for this article.

References

Bandura

(1986). Social foundations of thoughts and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.

Brown

T. A.

Moore

M. T.

(2012). Confirmatory factor analysis. In Hoyle

R. H.

(Ed.), Handbook of structural equation modeling (pp. 361-379). New York, NY: Guilford Press.

Bureau of Labor Statistics. (2011). Occupational outlook handbook (2010-11 ed.). Washington, DC: U.S. Department of Labor. Retrieved from http://www.bls.gov/ooh/

Business-Higher Education Forum. (2010). Increasing the number of STEM graduates: Insights from the U.S. STEM education modeling project. Washington, DC: Author.

Byars-Winston

Estrada

Howard

Davis

Zalapa

(2010). Influence of social cognitive and ethnic variables on academic goals of underrepresented students in science and engineering: A multiple-groups analysis. Journal of Counseling Psychology, 57, 205-218. doi:10.1037/a0018608

Carnevale

A. P.

Smith

Melton

(2011). STEM: Science, technology, engineering, mathematics. Washington, DC: Georgetown University Center on Education and the Workforce.

Chen

Gully

S. M.

Whiteman

J. A

Kilcullen

R. N.

(2000). Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. The Journal of Applied Psychology, 85, 835-847.

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. doi:10.1207/S15328007SEM0902_5

Eccles

J. S.

Wigfield

(2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109-132.

10.

Enders

C. K.

(2013). Dealing with missing data in developmental research. Child Development Perspectives, 7, 27-31. doi:10.1111/cdep.12008

11.

Erkut

Marx

(2005). 4 schools for WIE (Evaluation report). Wellesley, MA: Wellesley College, Center for Research on Women. Retrieved from http://www.coe.neu.edu/Groups/stemteams/evaluation.pdf

12.

Fan

(2011). Social influences, school motivation and gender differences: An application of the expectancy-value theory. Educational Psychology, 31, 157-175. doi:10.1080/01443410.2010.53652

13.

Fraser

B. J.

(1978). Development of a test of science-related attitudes. Science Education, 62, 509-515.

14.

Fraser

B. J.

(1981). TOSRA: Test of science-related attitudes handbook. Hawthorn, Victoria: Australian Council for Educational Research.

15.

Fredricks

McColskey

Meli

Mordica

Montrosse

Mooney

(2011). Measuring student engagement in upper elementary through high school: A description of 21 instruments (Issues & Answers Report, REL 2011-No. 098). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Southeast. Retrieved from http://ies.ed.gov/ncee/edlabs

16.

Friday Institute. (2010). Governor Perdue’s North Carolina Student Learning Conditions Survey (SLCS): Survey Implementation Study. Raleigh, NC: Author.

17.

Gorsuch

R. L.

(1983). Factor analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

18.

Hair

J. F.

Black

W. C.

Babin

B. J.

Anderson

R. E.

Tatham

R. L.

(2006). Multivariate data analysis (6th ed.). Upper Saddle River, NJ: Prentice Hall.

19.

Horn

J. L.

McArdle

J. J.

(1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117-144.

20.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.

21.

Kaiser

H. F.

(1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141-151. doi:10.1177/001316446002000116

22.

Khalili

K. Y.

(1987). A cross-cultural validation of a test of science related attitudes. Journal of Research in Science Teaching, 24, 127-136.

23.

Kidd

Naylor

(1991). The predictive power of measured interests in tertiary course choice: The case of science. Australian Journal of Education, 35, 261-272. doi:10.1177/000494419103500304

24.

Kline

R. B.

(2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford Press.

25.

Lawshe

C. H.

(1975). A quantitative approach to content validity. Personnel Psychology, 28, 563-575. doi:10.1111/j.1744-6570.1975.tb01393.x

26.

Lent

R. W.

Brown

S. D.

(1996). Social cognitive approach to career development: An overview. Career Development Quarterly, 44, 310-321.

27.

Lent

R. W.

Brown

S. D.

(2006). Integrating person and situation perspectives on work satisfaction: A social-cognitive view. Journal of Vocational Behavior, 69, 236-247. doi:10.1016/j.jvb.2006.02.006

28.

Lent

R. W.

Brown

S. D.

Hackett

(2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of Counseling Psychology, 47, 36-49. doi:10.1037//0022-0167.47.1.36

29.

Lent

R. W.

Lopez

A. M.

Lopez

F. G.

Sheu

(2008). Social cognitive career theory and the prediction of interests and choice goals in the computing disciplines. Journal of Vocational Behavior, 73, 52-62. doi:10.1016/j.jvb.2008.01.002

30.

Lent

R. W.

Sheu

Gloster

C. S.

Wilkins

(2010). Longitudinal test of the social cognitive model of choice in engineering students at historically Black universities. Journal of Vocational Behavior, 76, 387-394. doi:10.1016/j.jvb.2009.09.002

31.

Maltese

A. V.

Tai

R. H.

(2011). Pipeline persistence: Examining the association of educational experiences with earned degrees in STEM among U.S. students. Science Education, 95, 877-907. doi:10.1002/sce.20441

32.

Marsh

H. W.

Hau

K.-T.

Wen

(2004). In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) Findings. Structural Equation Modeling, 11, 320-341.

33.

Meade

A. W.

Johnson

E. C.

Braddy

P. W.

(2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568-592. doi:10.1037/0021-9010.93.3.568

34.

Miller

R. B.

Greene

B. A.

Montalvo

G. P.

Ravindran

Nichols

J. D.

(1996). Engagement in academic work: The role of learning goals, future consequences, pleasing others, and perceived ability. Contemporary Educational Psychology, 21(4), 388-422. doi:10.1006/ceps.1996.0028

35.

Minner

Ericson

Martinez

(2012). Compendium of STEM student instruments PART II: Measuring students’ content knowledge, reasoning skills, and psychological attributes. Washington, DC: Abt Associates.

36.

National Academy of Engineering. (2008). Grand challenges for engineering. Washington, DC: National Academies Press.

37.

National Academy of Engineering and the National Research Council. (2014). STEM integration in K-12 education: Status, prospects, and an agenda for research. Washington, DC: National Academies Press.

38.

National Research Council. (2010). Exploring the intersection of science education and 21st century skills: A workshop summary. Washington, DC: National Academies Press.

39.

National Research Council. (2011). A framework for K-12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

40.

PTCS, Partnership for 21st Century Skills. (2004). Retrieved from http://www.p21.org/

41.

President’s Committee of Advisors on Science and Technology. (2010). Prepare and inspire: K-12 education in Science, Technology, Engineering, and Math (STEM) for America’s future. Washington, DC: Executive Office of the President.

42.

Rothwell

(2013). The hidden STEM economy: The metropolitan policy program at Brookings. Washington, DC: Brookings Institution.

43.

Rynearson

A. M.

Douglas

K. A.

Diefes-Dux

H. A.

(2014, June). “Engineering teaches problem solving”: Teachers’ perceptions of student learning through engineering lessons. Presented at the American Society for Engineering Education Annual Conference and Exposition, Indianapolis, IN.

44.

Sadler

P. M.

Sonnert

Hazari

Tai

(2012). Stability and volatility of STEM career interest in high school: A gender study. Science Education, 96, 411-427. doi:10.1002/sce.21007

45.

Scott

Mallinckrodt

(2005). Parental emotional support, science self-efficacy, and choice of science major in undergraduate women. Career Development Quarterly, 53, 263-273.

46.

Simpkins

S. D.

Davis-Kean

P. E.

Eccles

J. S.

(2006). Math and science motivation: A longitudinal examination of the links between choices and beliefs. Developmental Psychology, 42, 70-83. doi:10.1037/0012-1649.42.1.70

47.

Truxillo

(2005). Maximum likelihood parameter estimation with incomplete data. In Proceedings of the Thirtieth Annual SAS® Users Group International Conference (paper 111-30). Cary, NC: SAS Institute Inc.

48.

U.S. Department of Commerce. (2012). The competitiveness and innovative capacity of the United States. Washington, DC: Author.

49.

Vandenberg

R. J.

Lance

C. E.

(2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69. doi:10.1177/109442810031002

50.

Wang

(2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. American Educational Research Journal, 50, 1081-1121. doi:10.3102/0002831213488622

51.

Wigfield

Eccles

J. S.

(2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68-68.

52.

Williams

Kurtek

Sampson

(2011). The affective elements of science learning: A questionnaire to assess-and improve-student attitudes toward science. Science Teacher, 78(1), 40-45.

53.

Wolff

H.-G.

Preising

(2005). Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS. Behavior Research Methods, 37, 48-58.