This study presents an exemplar for psychometric evaluation and modification of established measures when applied to new populations. Specifically, we describe the use of two subscales (Career Decision-Making Self-Efficacy Scale and Math/Science Self-Efficacy Scale) from the Middle School Self-Efficacy Scale (MSSES) as outcome measures in an intervention study of high school students. Several researchers have utilized the MSSES with high school students since it was developed by Fouad, Smith, and Enochs 20 years ago, but few studies have examined it for construct validity with a high school sample, even though the measure was designed for a middle school population. Our findings demonstrated that the MSSES required modification for high school students in order to meet the standards of reliability and validity in a counseling intervention study. The discussion focuses on implications for career counseling and research, limitations of the findings, and suggestions for future research.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2.
ArulmaniG.Van LaarD.EastonS. (2001). Career planning orientation of disadvantaged high school boys: A study of socioeconomic and social cognitive variables. Journal of the Indian Academy of Applied Psychology, 27, 7–17.
3.
BanduraA. (1977). Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review, 84, 191–215. doi:10.1037/0033-295X.84.2.191
4.
BanduraA. (1986). Social foundations of thought and action: A social cognitive theory. Upper Saddle River, New Jersey: Prentice Hall.
5.
BanduraA. (1997). Self-efficacy: The exercise of control. New York, NY: Freeman.
6.
BanduraA.BarbaranelliC.CapraraG. V.PastorelliC. (2001). Self-efficacy beliefs as shapers of children’s aspirations and career trajectories. Child Development, 72, 187–206.
7.
BergeronL. M.RomanoJ. L. (1994). The relationships among career decision-making self-efficacy, educational indecision, vocational indecision, and gender. Journal of College Student Development, 35, 19–24.
8.
BetzN. E. (2000). Self-efficacy theory as a basis for career assessment. Journal of Career Assessment, 8, 205–222. doi:10.1177/106907270000800301
9.
BetzN. E. (2004). Contributions of self-efficacy theory to career counseling: A personal perspective. The Career Development Quarterly, 52, 340–353. doi:10.1002/j.2161-0045.2004.tb00950.x
10.
BetzN. E.HackettG. (1981). The relationship of career-related self-efficacy expectations to perceived career options in college women and men. Journal of Counseling Psychology, 28, 399. doi:10.1037/0022-0167.28.5.399
11.
BetzN. E.HackettG. (1983). The relationship of mathematics self-efficacy expectations to the selection of science-based college majors. Journal of Vocational behavior, 23, 329–345. doi:10.1016/0001-8791(83)90046-5
12.
BetzN. E.HackettG. (2006). Career self-efficacy theory: Back to the future. Journal of Career Assessment, 14, 3–11. doi:10.1177/1069072705281347
13.
BetzN. E.KleinK. L.TaylorK. M. (1996). Evaluation of a short form of the career decision-making self-efficacy scale. Journal of Career Assessment, 4, 47–57. doi:10.1177/106907279600400103
14.
BetzN. E.LuzzoD. A. (1996). Career assessment and the career decision-making self-efficacy scale. Journal of Career Assessment, 4, 413–428. doi:10.1177/106907279600400405
15.
BetzN. E.SchifanoR. S. (2000). Evaluation of an intervention to increase realistic self-efficacy and interests in college women. Journal of Vocational Behavior, 56, 35–52. doi:10.1006/jvbe.1999.1690
16.
BohrnstedtG. W. (2010). Measurement models for survey research. In MarsdenP. V.WrightJ. D. (Eds.), Handbook of survey research (2nd ed., pp. 347–404). Somerville, MA: Emerald Group.
17.
BongM. (2001). Between-and within-domain relations of academic motivation among middle and high school students: Self-efficacy, task value, and achievement goals. Journal of Educational Psychology, 93, 23. doi:10.1037/0022-0663.93.1.23
18.
BrowneM. W.CudeckR. (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230–258. doi:10.1177/0049124192021002005
19.
ByrneB. M. (2012). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York, NY: Routledge.
20.
ByrneB. M.ShavelsonR. J.MuthénB. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456. doi:10.1037/0033-2909.105.3.456
CreedP.TilburyC.BuysN.CrawfordM. (2011). The career aspirations and action behaviours of Australian adolescents in out-of-home-care. Children and Youth Services Review, 33, 1720–1729.
23.
CronbachL. J.MeehlP. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, 281–302. doi:10.1037/h0040957
24.
EcclesJ. S.WigfieldA.MidgleyC.ReumanD.IverD. M.FeldlauferH. (1993). Negative effects of traditional middle schools on students’ motivation. The Elementary School Journal, 93, 553–574. doi:10.1086/461740
25.
FalcoL.D.SummersJ. J. (2017). Improving career decision self-efficacy and STEM self-efficacy in high school girls: Evaluation of an intervention. Journal of Career Development. doi:10.1177/0894845317721651
26.
FerryT. R.FouadN. A.SmithP. L. (2000). The role of family context in a social cognitive model for career-related choice behavior: A math and science perspective. Journal of Vocational Behavior, 57, 348–364. doi:10.1006/jvbe.1999.1743
27.
FouadN. A.GuillenA. (2006). Outcome expectations: Looking to the past and potential future. Journal of Career Assessment, 14, 130–142. doi:10.1177/1069072705281370
28.
FouadN. A.SmithP. L.EnochsL. (1997). Reliability and validity evidence for the middle school self-efficacy scale. Measurement and Evaluation in Counseling and Development, 30, 17–31.
29.
GarciaA. (2012). Is the social cognitive career theory model plausible for Hispanic inner city high school biology students?(Unpublished doctoral dissertation). California State University, Fullerton.
30.
GullicksonA.R.HowardB.B. (2009). The personnel evaluation standards: How to assess systems for evaluating educators (2nd ed.). Thousand Oaks, CA: Corwin Press.
31.
HackettG.BetzN. E. (1981). A self-efficacy approach to the career development of women. Journal of Vocational Behavior, 18, 326–339. doi:10.1016/0001-8791(81)90019-1
32.
HackettG.BetzN. E. (1992). Self-efficacy expectations in the career choices of college students. In SchunkD.MeeseJ. (Eds.), Student perceptions in the classroom: Causes and consequences (pp. 229–246). Hillsdale, NJ: Erlbaum.
33.
HamelJ. (2014). Career camp: Elevating expectations for college-going and career self-efficacy in urban middle school students(Unpublished doctoral dissertation). Kansas State University, Manhattan.
34.
HowardK. A.WendtA.HagnessS.CramerS.DiestelmannJ.HuangT. L. (2012, 10). Work in progress: Grand challenges for engineering in the middle school classroom: Preliminary results. Proceedings of the Frontiers in Education Conference (FIE), Seattle, WA. doi: 10.1109/FIE.2012.6462362
35.
HuL. T.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6, 1–55. doi:10.1016/0001-8791(81)90019-1
36.
JacksonK. D. M. (2014). Understanding Latina adolescents’ science identities: A mixed methods study of socialization practices across contexts(Unpublished doctoral dissertation). University of Texas at Austin, Austin.
37.
KellerB. K.WhistonS. C. (2008). The role of parental influences on young adolescents’ career development. Journal of Career Assessment, 16, 198–217.
38.
KimberlinC. L.WintersteinA. G. (2008). Validity and reliability of measurement instruments used in research. American Journal of Health-System Pharmacy, 65, 2276–2284.
39.
KlassenR. M.UsherE. L. (2010). Self-efficacy in educational settings: Recent research and emerging directions. In UrdanT. C.KarabenickS. A. (Eds.), The decade ahead: Theoretical perspectives on motivation and achievement (pp. 1–33). Bingley, England: Emerald Group.
40.
KlineR. B. (2015). Principles and practice of structural equation modeling. New York, NY: Guilford.
41.
LentR. W.BrownS. D. (2006). On conceptualizing and assessing social cognitive constructs in career research: A measurement guide. Journal of Career Assessment, 14, 12–35.
42.
LentR. W.BrownS. D.HackettG. (1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45, 79–122. doi:10.1006/jvbe.1994.1027
43.
LentR. W.BrownS. D.HackettG. (2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of Counseling Psychology, 47, 36–49. doi:10.1037/0022-0167.47.1.36
44.
LentR. W.LopezF. G.BieschkeK. J. (1993). Predicting mathematics-related choice and success behaviors: Test of an expanded social cognitive model. Journal of Vocational Behavior, 42, 223–236. doi:10.1006/jvbe.1993.1016
45.
LuzzoD. A.HasperP.AlbertK. A.BibbyM. A.MartinelliE. A.Jr. (1999). Effects of self-efficacy-enhancing interventions on the math/science self-efficacy and career interests, goals, and actions of career undecided college students. Journal of Counseling Psychology, 46, 233. doi:10.1037/0022-0167.46.2.233
46.
Macht JantzerA.StalidesD. J.RottinghausP. J. (2009). An exploration of social cognitive mechanisms, gender, and vocational identity among eighth graders. Journal of Career Development, 36, 114–138. doi:10.1177/0894845309345841
47.
MarshH. W.CravenR. G.DebusR. (1991). Self-concepts of young children 5 to 8 years of age: Measurement and multidimensional structure. Journal of Educational Psychology, 83, 377.
48.
MarshH. W.HauK.WenZ. (2004). In search of golden rules: Comment on hypothesis testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler’s (1999) findings. Structural Equation Modeling, 11, 320–341. doi:10.1207/s15328007sem1103_2
49.
MessickS. (1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741.
50.
MuellerC. E.HallA. L.MiroD. Z. (2015). Testing an adapted model of social cognitive career theory: Findings and implications for a self-selected, diverse middle-school sample. Journal of Research in STEM Education, 1, 142–155.
51.
MuthénL. K.MuthénB. O. (2012). Mplus user’s guide (7th ed.). Los Angeles, CA: Author.
52.
NavarroR. L.FloresL. Y.WorthingtonR. L. (2007). Mexican American middle school students’ goal intentions in mathematics and science: A test of social cognitive career theory. Journal of Counseling Psychology, 54, 320–335. doi:10.1037/0022-0167.54.3.320
53.
OjedaL.Piña-WatsonB.CastilloL. G.CastilloR.KhanN.LeighJ. (2012). Acculturation, enculturation, ethnic identity, and conscientiousness as predictors of Latino boys’ and girls’ career decision self-efficacy. Journal of Career Development, 39, 208–228. doi:10.1177/0894845311405321
54.
OliverL. W. (1979). Outcome measurement in career counseling research. Journal of Counseling Psychology, 26, 217.
55.
OlleC. D.FouadN. A. (2015). Parental support, critical consciousness, and agency in career decision making for urban students. Journal of Career Assessment, 23, 533–544. doi:10.1177/1069072714553074
56.
PajaresF. (1996). Self-efficacy beliefs and mathematical problem-solving of gifted students. Contemporary Educational Psychology, 21, 325–344. doi:10.1006/ceps.1996.0025
57.
SassD. A. (2011). Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. Journal of Psychoeducational Assessment, 29, 347–363. doi:10.1177/0734282911406661
58.
SawitriD. R.CreedP. A.Zimmer-GembeckM. J. (2014). Parental influences and adolescent career behaviours in a collectivist cultural setting. International Journal for Educational and Vocational Guidance, 14, 161–180. doi:10.1007/s10775-013-9247-x
59.
SechrestL. (2005). Validity of measures is no simple matter. Health Services Research, 40, 1584–1604.
60.
SickingerP. H. (2013). Social cognitive career theory and middle school student career exploration (Unpublished doctoral dissertation).Regent University, Virginia Beach, VA.
61.
SongH. D. (2004). The effects of goal-oriented contexts and peer group composition on intrinsic motivation and problem solving (Unpublished doctoral dissertation).The Pennsylvania State University, College Station.
62.
StewartA. L.ThrasherA. D.GoldbergJ.SheaJ. A. (2012). A framework for understanding modifications to measures for diverse populations. Journal of Aging and Health, 24, 992–1017.
63.
StewartA. L.WareJ. E. (Eds.). (1992). Measuring functioning and well-being: the medical outcomes study approach. Durham, NC: Duke University Press.
64.
StipekD.IverD. M. (1989). Developmental change in children’s assessment of intellectual competence. Child Development, 60, 521–538.
65.
SullivanK. R.MahalikJ. R. (2000). Increasing career self-efficacy for women: Evaluating a group intervention. Journal of Counseling & Development, 78, 54–62. doi:0.1002/j.1556-6676.2000.tb02560.x
66.
SwitzerG. E.WisniewskiS. R.BelleS. H.DewM. A.SchultzR. (1999). Selecting, developing, and evaluating research instruments. Social Psychiatry and Psychiatric Epidemiology, 34, 399–409.
67.
TanakaJ. S. (1993). Multifaceted conceptions of fit in structural equation models. Sage focus editions, 154, 10.
68.
TaylorK. M.BetzN. E. (1983). Applications of self-efficacy theory to the understanding and treatment of career indecision. Journal of Vocational Behavior, 22, 63–81. doi:10.1016/0001-8791(83)90006-4
69.
TurnerS. L.Alliman-BrissettA.LapanR. T.UdipiS.ErgunD. (2003). The career-related parent support scale. Measurement and Evaluation in Counseling and Development, 36, 83–95.
70.
TurnerS. L.LapanR. T. (2005). Evaluation of an intervention to increase non-traditional career interests and career-related self-efficacy among middle-school adolescents. Journal of Vocational Behavior, 66, 516–531. doi:10.1016/j.jvb.2004.02.005
71.
UrdanT.PajaresF. (Eds.). (2006). Self-efficacy beliefs of adolescents. Greenwich, CT: Information Age Publishing.
72.
Van de SchootR.LugtigP.HoxJ. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 486–492. doi:10.1080/17405629.2012.686740
73.
WhistonS. C.QuinbyR. F. (2009). Review of school counseling outcome research. Psychology in the Schools, 46, 267–272. doi:10.1002/pits.20372
74.
WidamanK. F.ReiseS. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In BryantK. J.WindleM. E.WestS. G. (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281–324). Washington, DC: American Psychological Association.
75.
WorthingtonR. L.WhittakerT. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806–838. doi:10.1177/0011000006288127
76.
YarbroughD.B.ShulaL.M.HopsonR.K.CaruthersF.A. (2010). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Corwin Press.
77.
ZumboB. D. (2006). 3 Validity: Foundational issues and statistical methodology. Handbook of Statistics, 26, 45–79. doi:10.1016/S0169-7161(06)26003-6
78.
ZytowskiD. G.BetzE. L. (1972). Measurement in counseling research: A review. The Counseling Psychologist, 3, 72–81.
