Colleges and universities are being pressed to seek innovative ways to measure student learning outcomes and identify the conditions that lead to their development. Understanding how students group according to a multidimensional set of learning outcomes provides information on the extent to which institutions are meeting goals. This study develops a typology based on engineering undergraduates’ array of outcomes. The study also demonstrates variation in personal and educational experiences across outcomes-based student groupings, thus providing insight into “what works” for programs who seek to graduate students who have developed an array of engineering-related outcomes. This outcomes-based approach is readily applicable to any set of student learning outcomes that programs or institutions seek to cultivate.
Abu El-HajT. R. A. (2003). Practicing for equity from the standpoint of the particular: Exploring the work of one urban teacher network. Teachers College Record, 105, 817–845. doi:http://www.tcrecord.org/11136
3.
AdamsR. S.TurnsJ.AtmanC. J. (2003). Educating effective engineering designers: The role of reflective practice. Design Studies, 24, 275–294. doi:10.1016/S0142-694X(02)00056
4.
AlonS. (2009). The evolution of class inequality in higher education: Competition, exclusion, and adaptation. American Sociological Review, 74, 731–755. doi:10.1177/000312240907400503
5.
American Association of Colleges of Nursing. (2009). Standards for accreditation of baccalaureate and graduate degree nursing programs. Commission on Collegiate Nursing Accreditation. Retrieved from http://www.aacn.nche.edu/ccne-accreditation/standards09.pdf
6.
AmmonB. V.BowmanJ.MouradR. (2008). Who are our students?: Cluster analysis as a tool for understanding community college student populations. Journal of Applied Research in the Community College, 16, 32–44.
7.
AnayaG. (1999). College impact on student learning: Comparing the use of self-reported gains, standardized test scores, and college grades. Research in Higher Education, 40, 499–526. doi:10.1023/A:1018744326915
8.
ArmorD. J. (1974). Theta reliability and factor scaling. In CostnerH. (Ed.), Sociological methodology: 1973-1974 (pp. 17–50). San Francisco, CA: Jossey-Bass.
9.
ArumR.RoksaJ. (2011). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.
10.
Association of American Colleges and Universities (AAC&U). (2012). Liberal Education and America’s Promise (LEAP) essential learning outcomes. Retrieved from http://www.aacu.org/leap/vision.cfm
AstinA. W. (1991). Assessment for excellence: The philosophy and practice of assessment and evaluation in higher education. New York, NY: Macmillan.
13.
AstinA. W. (1993). What matters in college? Four critical years revisited. San Francisco, CA: Jossey-Bass.
14.
BahrP. R. (2010). The bird’s eye view of community colleges: A behavioral typology of first-time students based on cluster analytic classification. Research in Higher Education, 51, 724–749. doi:10.1007/s11162-010-9180-5
15.
BarberJ. (2012). Integration of learning: A grounded theory analysis of college students’ learning. American Educational Research Journal, 49, 590–617. doi:10.3102/0002831212437854
16.
BaruchY. (1999). Response rate in academic studies—A comparative analysis. Human Relations, 52, 421–438. doi:10.1177/001872679905200401
17.
Besterfield-SacreM.ShumanL. J.WolfeH. (2002). Modeling undergraduate engineering outcomes. International Journal of Engineering Education, 18, 128–139.
18.
BjorklundS. A.ParenteJ. M.SathianathanD. (2004). Effects of faculty interaction and feedback on gains in student skills. Journal of Engineering Education, 93, 153–160. doi:10.1002/j.2168-9830.2004.tb00799.x
19.
Boix MansillaV. B.DuraisingE. D. (2007). Targeted assessment of students’ interdisciplinary work: An empirically grounded framework proposed. Journal of Higher Education, 78, 215–237. doi:10.1353/jhe.2007.0008
20.
BoughanK. (2000). The role of academic process in student achievement: An application of structural equations modeling and cluster analysis to community college longitudinal data. AIR Professional File, 74, 1–18.
21.
BowmanN. A. (2010). Can 1st-year college students accurately report their learning and development?American Educational Research Journal, 47, 466–496. doi:10.3102/0002831209353595
22.
BrotmanJ. S.MooreF. M. (2008). Girls and science: A review of four themes in the science education literature. Journal of Research in Science Teaching, 45, 971–1002. doi:10.1002/tea.20241
23.
CampbellC. M.CabreraA. F. (2011). How sound is NSEE? Investigating the psychometric properties of NSEE at a public, research-extensive institution. Review of Higher Education, 35, 77–104. doi:10.1353/rhe.2011.0035
24.
CampbellP. B.JollyE.HoeyL.PerlmanL. K. (2002). Upping the numbers: Using research based decision making to increase diversity in the quantitative disciplines. GE Foundation Report. Retrieved from http://www.campbell-kibler.com/upping_the_numbers.pdf
25.
CorriganM. E. (2003). Beyond access: Persistence challenges and the diversity of low-income students. New Directions for Higher Education, 121, 25–34. doi:10.1002/he.99
26.
CoxB. E.McIntoshK. L.ReasonR. D.TerenziniP. T. (2010, November). When doing the “right” thing feels wrong: A primer and example of missing data imputation. Paper presented at the annual conference of the Association for the Study of Higher Education, Indianapolis, IN.
27.
DempsterA. P.LairdN. M.RubinD. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B, 39, 1–38. doi:10.1.1.133.4884
28.
DesJardinsS. (2001). A comment on interpreting odds-ratios when logistic regression coefficients are negative. AIR Professional File81(Fall), 1–7.
29.
GrahamJ. W. (2009). Missing data: Making it work in the real world. Annual Review of Psychology, 60, 549–576. doi:10.1146/annurev.psych.58.110405.085530
30.
HagedornL. S.PratherG. (2005, May). The community college solar system: If university students are from Venus community college students must be from Mars. Paper presented at the 2005 Annual Forum of the Association for Institutional Research, San Diego, CA.
31.
HarperB. J.LambertA. D.LattucaL. R. (2006, November). The interaction of student race and experience: Critical influences on learning outcomes in engineering. Paper presented at the meeting of the Association for the Study of Higher Education, Anaheim, CA.
32.
HaüsslerP.HoffmannL. (2002). An intervention study to enhance girls’ interest, self-concept, and achievement in physics classes. Journal of Research in Science Teaching, 39, 870–888. doi:10.1002/tea.10048
33.
HuS.McCormickA. C. (2012). An engagement-based student typology and its relationship to college outcomes. Research in Higher Education, 53, 738–754. doi:10.1007/s11162-012-9254-7
JohnsonD. W.JohnsonR. (1983). The socialization and achievement crisis: Are cooperative learning experiences the solution?Applied Social Psychology Annual, 4, 119–164.
36.
KaltonG. (1983). Introduction to survey sampling (Vol. 35). Newbury Park, CA: SAGE.
37.
KnightD. B. (2011, June). Educating broad thinkers: A quantitative analysis of curricular and pedagogical techniques used to promote interdisciplinary skills. Proceedings of the 118th Annual Conference of the American Society for Engineering Education, Vancouver, BC, Canada.
38.
KnightD. B.LattucaL. R.YinA. C.KremerG.YorkT.RoH. K. (2012). An exploration of gender diversity in engineering programs: A curriculum and instruction-based perspective. Journal of Women and Minorities in Science and Engineering, 18, 55–78. doi:10.1615/JWomenMinorScienEng.2012003702
39.
KuhG. D. (2004). The national survey of student engagement: Conceptual framework and overview of psychometric properties. Bloomington: Indiana University Center for Postsecondary Research and Planning.
40.
KuhG. D. (2005). Imagine asking the client: Using student and alumni surveys for accountability in higher education. In BurkeJ. C. (Ed.), Achieving accountability in higher education: Balancing public, academic, and market demands (pp. 148–172). San Francisco, CA: Jossey-Bass.
41.
KuhG. D.HuS.VesperN. (2000). They shall be known by what they do: An activities based typology of college students. Journal of College Student Development, 41, 228–244. doi:10.1007/s 11162-004-1599-0
42.
KuhG. D.IkenberryS. (2009). More than you think, less than we need: Learning outcomes assessment in American higher education: National Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomeassessment.org/NILOAsurveyresults09.htm
43.
KuhG. D.KinzieJ.SchuhJ. H.WhittE. J., & Associates (2005). Student success in college: Creating conditions that matter. San Francisco, CA: Jossey-Bass.
44.
KuncelN. R.CredeM.ThomasL. L. (2005). The validity of self-reported grade point averages, class ranks, and test scores: A meta-analysis and review of the literature. Review of Educational Research, 75, 63–82. doi:10.3102/00346543075001063
45.
LaingJ.SwayerR.NobleJ. (1989). Accuracy of self-reported activities and accomplishments of college-bound seniors. Journal of College Student Development, 29, 362–368.
46.
LattucaL. R.KnightD. B.BergomI. (2013). Developing a measure of interdisciplinary competence. International Journal of Engineering Education, 29, 726–739.
47.
LattucaL. R.StarkJ. S. (2009). Shaping the college curriculum: Academic plans in context. San Francisco, CA: Jossey-Bass.
48.
LattucaL. R.TerenziniP. T.VolkweinJ. F. (2006). Engineering change: A study of the impact of EC2000. Baltimore, MD: ABET.
49.
LattucaL. R.TrautvetterL. C.CoddS.KnightD. B. (2011, June). Working as a team: Enhancing interdisciplinarity for the engineer of (2020). Proceedings of the 118th Annual Conference of the American Society for Engineering Education, Vancouver, BC, Canada.
50.
LevinJ.WyckoffJ. (1990). Identification of student characteristics that predict persistence and success in an engineering college at the end of the sophomore year: Informing the practice of academic advising (Division of Undergraduate Studies Report No. 1990.1). University Park: Pennsylvania State University.
51.
Liaison Committee on Medical Education. (2011). Functions and structures of a medical school: Standards for accreditation of medical education programs leading to the M.D. degree. Liaison Committee on Medical Education. Retrieved from http://www.lcme.org/functions.pdf
52.
LongJ. S.FreeseJ. (2006). Regression models for categorical dependent variables using Stata (2nd ed.). College Station, TX: Stata Press.
53.
McGrathJ. E.MartinJ.KulkaR. A. (1982). Judgment calls in research. Beverly Hills, CA: SAGE.
54.
MentkowskiM. (1991). Creating a context where institutional assessment yields educational improvement. Journal of General Education, 40, 255–283.
55.
National Academy of Engineering. (2004). The engineer of 2020: Visions of engineering in the new century. Washington, DC: National Academies Press.
56.
National Academy of Sciences, Committee on Science, Engineering & Public Policy. (2007). Rising above the gathering storm: Energizing and employing America for a brighter economic future. Washington, DC: National Academies Press.
57.
PaceC. R. (1985). The credibility of student self-reports. Los Angeles: Center for the Study of Evaluation, University of California at Los Angeles.
58.
PalmerE.TerenziniP. T.HarperB. J.McKennaA.MersonD. S. (2011, June). Design in context: Where do the engineers of 2020 learn this skill?Proceedings of the American Society for Engineering Education, Vancouver, Canada.
59.
PascarellaE. T.TerenziniP. T. (1991). How college affects students: Findings and insights from twenty years of research. San Francisco, CA: Jossey-Bass.
60.
PascarellaE. T.TerenziniP. T. (2005). How college affects students: Vol. 2. A third decade of research. San Francisco, CA: Jossey-Bass.
61.
PawleyA. L. (2004, June). The feminist engineering classroom: A vision for future educational innovations. Proceedings of the 2004 American Society for Engineering Education Annual Conference & Exposition, Salt Lake City, UT.
62.
PikeG. R. (1995). The relationship between self-reports of college experiences and achievement test scores. Research in Higher Education, 36, 1–22.
63.
PorterS. R. (2005). What can multilevel models add to institutional research? In CoughlinM. A. (Ed.), Applications of intermediate/advanced statistics in institutional research (Resources in Institutional Research, No. 16, pp. 110–131). Tallahassee, FL: Association for Institutional Research.
64.
PorterS. R. (2011). Do college student surveys have any validity?Review of Higher Education, 35, 45–76. doi:10.1353/rhe.2011.0034
65.
PorterS. R.UmbachP. D. (2001, June). What works best? Collecting alumni data with multiple technologies. Paper presented at the annual meeting of the Association for Institutional Research, Long Beach, CA.
66.
PorterS. R.UmbachP. D. (2006). Student survey response rates across institutions: Why do they vary?Research in Higher Education, 47, 229–247.
67.
QuadeD. (1967). Rank analysis of covariance. Journal of the American Statistical Association, 62, 1187–1200.
68.
RaudenbushS. W.BrykA. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: SAGE.
69.
ReasonR. D.CoxB. E.Lutovsky-QuayeB. R.TerenziniP. T. (2010). Faculty and institutional factors that encourage faculty to promote student encounters with difference in first-year courses. Review of Higher Education, 33, 391–414. doi:10.1353/rhe.0.0137
70.
ReasonR. D.TerenziniP. T.DomingoR. J. (2006). First things first: Developing academic competence in the first year of college. Research in Higher Education, 47, 149–175. doi:10.1007/s11162-005-8884-4
71.
ReasonR. D.TerenziniP. T.DomingoR. J. (2007). Developing social and personal competence in the first year of college. Review of Higher Education, 30, 271–299.
72.
RoH. K.MersonD.LattucaL. R.TerenziniP. T. (2012, June). Measuring engineering students’ contextual competence. Proceedings of the 119th Annual Conference of the American Society for Engineering Education, San Antonio, TX.
73.
SeymourE. (1995). The loss of women from science, mathematics, and engineering undergraduate majors: An explanatory account. Science Education, 79, 437–473. doi:10.1002/sce.3730790406
74.
ShadishW. R.CookT. D.CampbellD. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
75.
ShavelsonR. J. (2010). Measuring college learning responsibility: Accountability in a new era. Palo Alto, CA: Stanford University Press.
76.
SheppardS.GilmartinS.ChenH. L.DonaldsonK.LichtensteinG.ErisÖ.. . .ToyeG. (2010). Exploring the engineering student experience: Findings from the Academic Pathways of People Learning Engineering Survey (APPLES, TR-10-01). Seattle, WA: Center for the Advancement for Engineering Education.
77.
ShumanL. J.Besterfield-SacreM.McGourtyJ. (2005). The ABET “professional skills”—Can they be taught? Can they be assessed?Journal of Engineering Education, 94, 41–55. doi:10.1002/j.2168-9830.2005.tb00828.x
78.
SteinleyD. (2006). K-means clustering: A half-century synthesis. British Journal of Mathematical and Statistical Psychology, 59, 1–34. doi:10.1348/000711005X48266
79.
SteinleyD. (2007). Initializing K-means batch clustering: A critical evaluation of several techniques. Journal of Classification, 24, 99–121. doi:10.1007/s00357-007-0003-0
80.
StraussL. C.TerenziniP. T. (2007). The effects of students’ in-and out-of-class experiences on their analytical and group skills: A study of engineering education. Research in Higher Education, 48, 967–992. doi:10.1007/s11162-007-9057-4
81.
TabachnickB. G.FidellL. S. (2001). Using multivariate statistics (4th ed.). Boston, MA: Allyn & Bacon.
82.
TerenziniP. T.ReasonR. D. (2005, November). Parsing the first-year of college: A conceptual framework for studying college impacts. Paper presented at the annual conference of the Association for the Study of Higher Education, Philadelphia, PA.
83.
TerenziniP. T.ReasonR. D. (2010, June). Toward a more comprehensive understanding of college effects on student learning. Paper presented at the annual conference of the Consortium of Higher Education Researchers (CHER), Oslo, Norway.
84.
U.S. Government Accountability Office. (2006). Science, technology, engineering and mathematics trends and the role of federal programs: Statement of Cornelia M. Ashby, Director, Education, Workforce, and Income Security Issues (GAO-06-702T). Washington, DC: Author.
85.
VanDerLindenK. (2002). Credit student analysis: 1999 and 2000. Annapolis Junction, MD: Community College Press, American Association of Community Colleges.
86.
Van HornP. S.GreenK. E.MartinussenM. (2009). Survey response rates and survey administration in counseling and clinical psychology: A meta-analysis. Educational and Psychological Measurement, 69, 389–403. doi:10.1177/0013164408324462
87.
VolkweinJ. F.LattucaL. R.HarperB. J.DomingoR. J. (2007). Measuring the impact of professional accreditation on student experiences and learning outcomes. Research in Higher Education, 48, 251–282. doi:10.1007/s11162-006-9039-y
88.
WardJ. H. (1963). Hierarchical grouping to optimize function. Journal of the American Statistical Association, 58, 236–244.
WhittE. J.PascarellaE. T.NeisheimB. S. E.MarthB. P.PiersonC. T. (2003). Differences between women and men in objectively measured outcomes, and the factors that influence those outcomes, in the first three years of college. Journal of College Student Development, 44, 587–610. doi:10.1353/csd.2003.0060
91.
ZhangG.AndersonT. J.OhlandM. W.ThorndykeB. R. (2004). Identifying factors influencing engineering student graduation: A longitudinal and cross-institutional study. Journal of Engineering Education, 93, 313–320. doi:10.1002/j.2168-9830.2004.tb00820.x
