Abstract
Elective curriculums require undergraduates to choose from a large roster of courses for enrollment each term. It has proven difficult to characterize this fateful choice process because it remains largely unobserved. Using digital trace data to observe this process at scale at a private research university, together with qualitative student interviews, we provide a novel empirical study of course consideration as an important component of course selection. Clickstream logs from a course exploration platform used by most undergraduates at the case university reveal that students consider on average nine courses for enrollment for their first fall term (<2% of available courses) and these courses predict which academic major students declare two years later. Twenty-nine interviews confirm that students experience consideration as complex and reveal variation in consideration strategies that may influence how consideration unfolds. Consideration presents a promising site for intervention in problems of equity, career funneling, and college completion.
You come in, you think you know what you want to do, then you explore, you realize you don’t know what you want to do. And then you eventually figure out a path.
Each academic term, millions of U.S. undergraduates like Isabelle consider a variety of courses for possible enrollment. In contrast to their high school curriculum, which offered only a narrow set of pathways (McFarland, 2006), college curriculums offer a large number of courses to choose from, which can be especially daunting to first-year students. The character of college course consideration is fateful. Too wide a range of options can be overwhelming to students, especially when they can access only minimal information and advice for navigating complex curriculums (Bailey et al., 2015; Rosenbaum et al., 2006). Too narrow a range can limit academic options later in college in ways that can be hard for students to recognize early on (Chambliss & Takacs, 2014). To the extent that pursuing a field of study is contingent upon students considering it, understanding this antecedent of the academic choice process can provide tractable insight and guide interventions for helping students informedly navigate their academic careers.
A systematic understanding of undergraduate course consideration can serve administrators, educators, and students by helping them plan ahead, make informed decisions, and manage complexity. Knowledge of consideration patterns can help administrators forecast demand for courses and majors, educators assess course and program visibility, and students discover blind spots in their own exploration. This research builds a foundation for further systematic inquiry by developing a conceptual framework to situate consideration within the larger academic choice process, by presenting a novel measurement approach, and by conducting an empirical investigation that addresses two major research questions.
First, what is the size and composition of consideration sets in relation to all available courses early in the college career? While students can hardly consider all possible academic options available to them, educators might hope that students consider enough courses to ensure breadth in exploration, but not so many as to be overwhelmed by their options (Schwartz, 2004). Second, how is course consideration at the start of college related to subsequent major declaration? As early course experiences are likely to influence the eventual selection of a major, educators might hope that students consider a spectrum of courses before making this decision.
To date, course consideration has only been minimally studied systematically because it is difficult to observe. With a few exceptions (Arcidiacono et al., 2010; Baker, 2017; Galotti, 1999), education researchers have inferred consideration either in terms of revealed preference on evidence of enrollment, or by valuation surveys that assess students’ preferences for different fields of study in the abstract (Baker & Orona, 2020). Observing consideration where and when it occurs in specific academic contexts has long been intractable, because consideration has left few empirical traces. Registrars keep detailed records of course offerings and student enrollments, but these records alone cannot distinguish between courses that students consider and choose, and those they consider but do not choose. By themselves, these data can tell us little about the character of the search processes that precede course choice and academic major selection.
To gain empirical insight on academic search, we created a novel web-based platform for course exploration and implemented it at scale to collect detailed records of students’ course considerations as they build their schedules each term. The platform, called Carta, is designed to enable students to search for and view detailed information on specific courses. Carta supports informed course consideration for undergraduates while simultaneously addressing the tractability problem described above. For each course, the platform provides information such as historical grade distributions and course sequences (derived from transcript data), estimated homework time (from prior student reviews), and basic scheduling information (from the registrar). Launched in August 2016, Carta now receives regular usage by more than 90% of current undergraduates at our case school and provides uniquely detailed data for observing consideration across entire cohorts of undergraduates navigating an elective curriculum.
For the present study, we examine two entering undergraduate cohorts in a data set that combines over 1 million timestamped course views, 3,336 student transcripts documenting over 67,000 course enrollments, and detailed information on 8,500 courses over two academic years. These data represent student experience at the micro level via individual-level information on platform interactions, and at the macro level through institutional-level information on course offerings, grade distributions, and course evaluations (Fischer et al., 2020). With a data set of this nature and magnitude, the study of course consideration becomes a worthy subject of education data science.
Our empirical study shows that entering first-year students, who are officially encouraged to explore a variety of courses before selecting a major in their third year, on average consider only nine courses prior to enrolling, which is fewer than 2% of the approximately 500 courses in which least one first-year student enrolls during the term of observation. Moreover, the composition of considered courses in the first term is a strong predictor of students’ eventual major 2 years later. Interviews with 29 mostly first-year students enrich our scientific understanding of these quantitative findings. Although students consider only a small fraction of all available courses, they nevertheless experience consideration as a complex, expansive, multistage process. Students bring different orientations and aspirations to their search process; in particular, reflections from first-generation college students and those pursuing preprofessional tracks indicate more constrained approaches to consideration.
We recognize that our inquiry is a case study of a single school. Yet elective curriculums are central to the academic organization of many U.S. colleges and universities, and digital platforms for creatively repurposing institutional data in the service of educational improvement are rapidly becoming ubiquitous (Piety et al., 2014). For these reasons our conceptualization of course consideration, and means of instrumenting its observation, offer portable utility for researchers and educators beyond our case university.
Background
Ideals of academic exploration and choice are hard-wired into undergraduate education in the United States. Colleges and universities often oblige undergraduates to choose many of their courses and place only modest restrictions on how choices must be made. This system of elective choice has strong partisans and detractors alike. Liberal-arts ideals embody a faith in the value of broad intellectual exposure to explore and integrate multiple realms of knowledge (e.g., Delbanco, 2012). Higher education leaders have long pitched broad education as preparing graduates for a world characterized by complexity, diversity, and change (Association of American Colleges and Universities, 2002). Yet many have critiqued the elective model, pointing out its costs and risks to students who navigate a complex academic landscape with little guidance (Goldrick-Rab, 2006; Rosenbaum et al., 2006). Such risks are especially substantial at modestly resourced community colleges with “cafeteria style” curriculums, and where students are provided with minimal information about the sequential relationships between particular courses, paths of study, and occupational destinations (Bailey et al., 2015; Crosta, 2014; Scott-Clayton, 2015). Even at selective 4-year colleges with ample advising and high completion rates, students may fail to consider entire fields of study on the basis of negative stereotypes or a single bad experience in an introductory-level course (Chambliss & Takacs, 2014). By investigating the mechanisms underlying students’ search processes, we can illuminate some of the nuances of elective search in ways that might benefit students, curriculum planners, and administrators.
Despite the centrality of the elective process to the organization of U.S. undergraduate education, there is a surprising scarcity of systematic empirical inquiry into how students navigate and select among the often wide range of courses on offer at a given institution. While the increasing availability of educational data has allowed for more in-depth exploration of student behavior in a variety of contexts (Fischer et al., 2020; Reich, 2020), course consideration remains relatively poorly instrumented and understood. Scott-Clayton (2015) provides a compelling summary of the challenges to students inherent in course consideration and choice under an elective curriculum: A large number of options, incomplete information, incommensurate alternatives, and students’ limited prior experience with academic choice-making. Yet empirical research on how students respond to academic choice in the wild remains limited.
Ethnographies of student life on 4-year residential campuses report that students’ awareness of available academic options is shaped substantially by parental expectations, prior knowledge and conceptions of college life, and flows of information among known peers (Armstrong & Hamilton, 2013; Nathan, 2005; Mullen, 2011). Other prior research includes exploring the impact of course evaluations on future demand for courses, yielding mixed results (e.g., Borgida & Nisbett, 1977; Brown & Kosovich, 2015; Wilhelm, 2004), inquiries utilizing machine-learning techniques to predict course enrollment for institutional planning purposes (e.g., Kardan et al., 2013; Ognjanovic et al., 2016), and recommendation systems that suggest courses in light of students’ prior course-taking (Jiang et al., 2019). Yet with few exceptions (Arcidiacono et al., 2010; Baker, 2017; Galotti, 1999), researchers have focused on the outcomes of academic consideration—enrolled courses—rather than consideration itself. None of the above studies have a direct and scalable proxy for consideration at the course level, leaving a significant gap in knowledge.
Conceptual Framework
Unlike high school curriculums, which afford relatively few and often highly circumscribed academic pathways (McFarland, 2006), college curricular offerings are famously complex—whether at community colleges (Bailey et al., 2015; Baker, 2017; Rosenbaum et al., 2006) or at comprehensive research universities (Abbott, 2001). If U.S. higher education is a “marketplace of ideas” (Menand, 2010), its undergraduate curricular offerings are marketplaces of courses. Students begin their college careers with limited (and in many cases, quite financially costly) personal stores of academic credit that they exchange in a diversified intramural course marketplace. This marketplace is highly structured: Each course is a contract, in which clearly specified units of academic credit are exchanged for tuition and demonstrated academic performance. Instructors and programs vie for student spending via course enrollments, to which material institutional resources (funding, faculty appointments) and organizational influence and prestige are often directly tied (Abbott, 2001, 2002; Armstrong & Hamilton, 2013). Commitment to course contracts typically takes a regimented temporal order, with specifically delineated “registration” periods and detailed guidelines for early contract exits (in the form of course drops or withdrawals, for example).
At any given moment in time, credit available for expenditure in the intramural academic marketplace is constrained: There are only so many students, possessing so many credits to spend each term. For course providers, this means that any enrollment in another course is an ineligible enrollment for one’s own course. For course consumers—students, the core subject of inquiry here—this means that any particular enrollment comes at the opportunity cost of nonenrollment elsewhere. Normative and financial incentives encourage students to limit expenditure of credits to 4 or fewer academic years. The constrained character of credit availability during any given enrollment period leads us to expect that students will invest time, energy, and intent on course selection.
Course selection in the intramural curricular marketplace is a version of a generic choice problem under bounded rationality that recurs each academic term (Simon, 1955). Each term, students confront potentially fateful academic choices with only partial information about the range of options available to them and the potential downstream consequences of those choices. Given that students are making these choices at young ages (particularly, those in the early years of undergraduate programs, when many are still teenagers), choices made at these stages are unlikely to adhere to standard rational decision models (Casey et al., 2011; Thaler & Mullainathan, 2008). Fresh arrivals at college are, almost by definition, new to the college environment and navigate the curricular marketplace with limited prior experience and limited assets of awareness of their options or time to consider them (Simon, 1982).
Consideration, an important part of any choice process, entails deriving a subset of all available academic options about which one makes further inquiries (Figure 1). While not yet explicitly modeled in research on course selection in higher education, consideration is recognized as a generic feature of search and choice phenomena in a wide range of social science literatures (see Bruch & Feinberg, 2017, for a review). Figure 1 depicts a funnel model in which courses that students do not consider and do not select are filtered out. Similar funnel models have been studied in education (e.g., to model attrition in massive open online courses; Clow, 2013) and marketing (e.g., to model online consumer purchasing behavior; Hoban & Bucklin, 2015).
Conceptual model of academic choice showing previously unobservable consideration.
Consideration is typically crucial to choice outcomes because it excludes many potential choice candidates from close scrutiny by decision makers. Given the complexity of the course marketplace under elective curriculums, we expect that consideration will be a fateful state in the course selection, yet traditionally it has left few data traces. Students may make lists of courses they are considering or exchange communication with peers about them, but considerations that precede selections typically are unseen by institutions or researchers. Working with survey data from community college students pertaining to major choice, Baker and Orona (2020) demonstrate substantial variation in the composition of students’ awareness and consideration sets early in their academic careers. Our own inquiry seeks to extend this line of inquiry by observing consideration among undergraduates at the course level.
When course search and selection are digitally mediated, researchers can treat digital traces of students’ searching and viewing activities as a proxy for consideration. This strategy is commonly used for decision research in other contexts such as online dating and e-commerce (Hitsch et al., 2010; Wedel & Kannan, 2016). Here we treat course views on the Carta platform as a proxy for consideration, but any digital tool for course search and exploration could be instrumented for this purpose and different proxy measures could be considered, including ones that assess strength of consideration by course view frequency or time spent on the page.
We acknowledge that course viewing on an exploration platform is an imperfect proxy for consideration, since students almost certainly consider courses that they do not view. To at least partially correct for this, we posit that students must have considered courses that they enroll in even if they do not view them, and we examine the nature of unviewed enrollments in relation to several course characteristics (whether the course serves as a major requirement, for example) to test the validity of views as a proxy for consideration. Missing data about unviewed considered courses that students do not enroll in remains an empirical limitation, but no more so than other methods of data collection, since students may also underreport considered courses in surveys and interviews. Yet unlike existing methods, our mechanism for proxying consideration is scalable, unobtrusive, and yields a baseline measure of courses students consider during any particular academic term.
Given a mechanism for collecting an empirical proxy for consideration sets, we can measure such features as their size (relative to available and enrolled courses) and their composition (relative to enrolled courses and subsequent majors). The size of consideration sets is not limited by organizational constraints. In the intramural academic marketplace, students are free to consider as many courses as they like. However, the time and attention entailed in considering courses are search costs that may limit the size of consideration sets. In addition to the size of consideration sets, we would like to know about set composition in terms of the disciplinary area of courses. In particular, we wish to observe how the composition of considered courses early in the academic career is related to overall academic trajectories. This is especially important in elective curriculums, in which students are encouraged to explore a variety of subject areas before committing to a major. We therefore pose the following research question:
As researchers we are agnostic about the proper breadth of consideration under such systems, yet we acknowledge ongoing discussion in higher education about the risks of exploration that is too broad (potentially delaying completion [Bailey et al., 2015]), or too narrow (funneling into a small range of majors [Binder et al., 2016; Deresiewicz, 2014; Guinier, 2016]). A first step toward understanding these phenomena is to characterize the relationship between early course consideration and subsequent major declaration. Hence,
Finally, we recognize that digitally captured interactions with web-based platforms tend to be neither complete nor transparent representations of phenomena of scientific interest (Salganik, 2018). In an effort to augment our computational analyses of undergraduate course consideration, we conduct close readings of verbatim transcripts of interviews with 29 undergraduates at our case university on their strategies for course consideration and selection during their first years in college.
Method
Sample and Context
The study site is an admissions-selective private university in the United States. The university offers courses during three 10-week terms of each academic year and during a summer term. The total number of undergraduates during the study period was roughly 7,000, with approximately equal numbers of women and men. The university’s undergraduate curriculum is largely elective: Students are obliged to fulfill certain academic requirements, but they retain discretion over which particular courses they choose in order to meet those requirements. Official campus literature advocates wide exploration of the curriculum, especially in the first and second years. Students are encouraged to declare majors (one or more) before the beginning of their third year.
For the present study, we restrict observation to those students who were admitted to the university in fall of 2016 and fall 2017 (N = 3,336). All students explicitly consent to participate in our research study when they elect to use the Carta platform. Ninety-seven percent of these students logged in to Carta at least once in their first year of college, and 92% between the opening of course registration and the university deadline for dropping and adding courses in the first term. Engagement with Carta is frequent; on average, students in our sample visited Carta on 27 days during the academic year (median = 21 days). Our focus here is on students’ early course consideration in their first fall term, how it relates to course choice in the fall term, and major declaration 2 years later.
To further contextualize our quantitative inquiry, we conducted 29 semistructured interviews with undergraduates enrolled in academic years 2016–2018. We recruited interviewees through the Carta platform and provided compensation for participants’ time. Our interview cohort comprised 18 students who identified as female, 10 who identified as male, and one student who identified with they/them pronouns. At the time of the interview, 12 of our participants were first-year students; we additionally had nine sophomores, four juniors, and four seniors in our sample. Interviews were semistructured and lasted between 25 and 45 minutes. Students were asked to reflect on their course selection processes, their typical Carta usage, and how they manage the various demands on their time. Interviews were transcribed verbatim and analyzed using an inductive coding process (Charmaz, 2006).
Platform
We empirically observe course consideration using the Carta platform for course exploration that is widely used by undergraduates at the case university (Chaturapruek et al., 2018). Carta is a student information service developed by an interdisciplinary team of faculty and students. It is explicitly presented to the university community as a voluntary free service and opt-in research project. The platform aggregates de-identified data from the university’s registrar and other official sources and presents it to students in a user-friendly interface. Available course information includes common course pairings, written student reviews, and histograms of previous students’ grades, reported time commitment, and instructor ratings. Carta also provides a suite of tools to simplify course comparison and planning (Appendix Figure A1). The platform continuously captures data describing the courses students view, search for, and “pin.” Linking this use data with institutional information on enrollments and major declaration provides unprecedented insight into how students navigate their academic careers.
Carta is one part of a campus information ecosystem that includes a searchable online catalogue, static informational websites, in-person advising, and peer networks through which students glean information about courses, course sequences, and instructors. Carta was specifically designed to augment these other information sources, not replace them. Qualitative data reported below indicate that students integrated Carta usage into diversified search strategies. Yet Carta is the only information service that systematically retains information about student search and consideration. Its routine and near-universal usage recommend it as a useful proxy for observing consideration at scale.
Measures
Course availability and enrollments are observed through official institutional data. We define available courses as all courses in a given term that enroll at least one first-year student, excluding overseas studies, independent-study, and one-unit courses. This definition yields a conservative estimate by excluding courses that were considered by first-year students but received no enrollments from them. Note that we only use the size of the availability set as a denominator to put consideration and enrollment sets in perspective. Enrolled courses exclude those that students eventually dropped from their schedules because they were not chosen in the end. While our research team did not have access to records for dropped courses, others may find these data useful in characterizing consideration. We operationalize consideration sets as the union of courses which students clicked to view on Carta before enrolling, and courses in which students enrolled without viewing them in advance. Temporally, we operationalize consideration as occurring at an intermediary period: commencing on the date when the university posts courses as available for registration, and concluding on the date when courses can no longer be dropped from the student transcript during any given term.
Following students longitudinally, we identify their declared majors at the beginning of their third academic year. We group courses of study into five academic areas: Engineering (12 % of all courses), Humanities & Arts (33%), Natural Sciences (9% including mathematics courses), Social Sciences (15%), and Other (32%). The latter includes first-year requirements such as required writing courses and interdisciplinary courses that straddle different departments such as education and human biology. We also group declared majors into five academic areas: Engineering (16 fields, 39% of students), Humanities & Arts (32, 13%), Natural Sciences (5, 8%), and Social Sciences (7, 13%), and Other (34, 26%). The bulk of observations in the latter category is made up of interdisciplinary majors such as international relations and science and technology studies. Less than 2% had not yet declared a major at the beginning of their third year and are therefore excluded in analyses that involve declared major.
We define a number of course characteristics based on institutional data for our analysis. First, we compute the number of pre- and postrequisites for each course. We define prerequisites as required courses to enroll in a given course c; we define postrequisites as courses that list course c as a prerequisite to enroll. In addition, we manually coded whether course c was a required or elective prerequisite for a select set of majors: Computer Science, Economics, Human Biology, International Relations, and an interdisciplinary major combining principles from Computer Science, Linguistics, and Psychology. These majors were chosen because they constitute about 20% of undergraduates and because they span a range of academic disciplines. As measures of course difficulty, we compute the average grade point average (GPA) received by students in the course as well as the course level (coded as indicator variables for 100-level, 200-level, 300-level, and 400-level or above). Finally, as a measure of course popularity, we compute the log-transformed number of enrollments each course.
Results
Size of Consideration Sets
We find that first-year course consideration sets account for less than 2% of available courses, leaving the vast majority of courses available for enrollment unconsidered. This is despite our conservative definition of available courses as those in a given term that ultimately enroll at least one first-year student (excluding single-unit, study abroad, and independent study courses), resulting in a much smaller number of courses than the total listed in the course catalogue. Students typically consider only approximately 9 (SE = 0.4) out of the 531 available courses for the fall term registration period (Figure 2). However, the size of consideration sets varies substantially across students, with the majority of students considering between four and 24 courses (the 25th and 75th percentiles) for enrollment in the fall term. A median first-year student at the case university ultimately enrolls in three courses in the first quarter (not counting single-unit, study abroad, and independent-study courses). These findings indicate that despite the breadth of courses available, few if any restrictions on consideration, and official encouragement from faculty and administrators to explore a wide variety of academic offerings, students seek information within the platform about only a small fraction of courses.
Enrollment and consideration set sizes in first-year students’ fall terms 2016 and 2017.
These results comport with classic theoretical insights on human decision making in complex organizations (Cohen & March, 1974; Simon, 1982), and market contexts in which buyers confront large numbers of choices (Schwartz, 2004). To the extent that the campus information ecosystem provides students with many different sources of information, the course catalogue presents a very wide array of options, and time for consideration is limited, students simplify the choice process by excluding the majority of available candidates from consideration.
In light of the high variance in consideration set sizes, we examine how set size varies across student curriculums. We find the size of consideration sets to be homogeneous across students with different subsequent majors (Breusch-Pagan test of heteroskedasticity: χ2 = 3.07, df = 3, P = .38), even though majors vary in terms of the balance between required and elective coursework (Figure 3). Formally, a robust Wald-test yields no statistical evidence for differences in consideration set size across major areas (F = 2.42, df1 = 3, df2 = 2,416, P = .064). To the extent that there are differences in consideration between major areas, such differences do not implicate the overall number of considered courses.
Consideration set size in first-year students’ fall terms 2016 and 2017 by domain of declared major at the beginning of the third year of college.
We evaluate the robustness of our proxy for consideration by examining a possible source of selection bias, namely courses that students enroll in without first viewing them. On average, approximately 6% of considered courses were not viewed on Carta prior to enrollment. To test for selection bias in terms of course characteristics, we fitted a linear probability model to predict which courses students enroll in without first viewing them on Carta (i.e., cases where consideration was inferred by enrollment only) using a variety of course characteristics as predictors. Consistent with our other analyses, we use 2 years of course view and enrollment data, from fall 2016 to spring 2018, which encompasses the first 2 years of study for the cohort entering in 2016 and the first year of study for the cohort entering in 2017. The two models summarized in Table 1 include student and course department fixed effects. Model 1 is fitted on data for all majors and Model 2 only for five large-enrollment majors for which we manually coded prerequisite courses. Either way, after accounting for student- and department-level variation, we find minimal evidence of selection bias in terms of course characteristics, based on the small proportion of variance explained by these features beyond the fixed effects in both models (within-strata R2 < .15%). Thus, courses that students enroll in without prior viewing are mostly similar to ones that they view before enrolling.
Linear Regression Results Predicting Likelihood of Enrolled Courses Not Being Viewed
Note. Standard errors in parentheses. FE = fixed effects.
p < .05. **p < .01. ***p < .001.
Nevertheless, a few course characteristics are related to whether a course will be considered without prior viewing. Across all majors, the number of postrequisites is a significant predictor of students’ likelihood of enrolling in a course without first viewing it (Table 1). Moreover, course enrollment and difficulty measured by the average grade received by students in a course are also significant predictors. Courses with a smaller enrollment and a higher average grade are more likely to be considered but not viewed. Courses at the 100 and 400 levels are also more likely to be considered but not viewed than courses below the 100 level (coded as the reference group in the regression). However, as previously noted, the marginal effect associated with each of these course characteristics is negligible empirically. A close look at the specific courses that are frequently considered but not viewed reveals two types: orientation courses for first-year students on topics such as how to succeed in college and IT resources, and large introductory course sequences such as in chemistry, computer science, and mathematics.
Consideration Set Composition and Subsequent Major
We also are interested in the relationship between the composition of consideration sets early in the college careers and subsequent majors. If early consideration sets are strongly predictive of students’ subsequent majors, then concerns about constrained major choice might be more fully understood and potentially addressed by interventions encouraging students to diversify the range of courses they consider in the first year.
Here we examine the disciplinary composition of consideration sets in terms of the fraction of considered courses that belong to each academic area, for example, the fractions of considered courses offered by Social Science departments and by Natural Science departments. At the student level, the composition of considered courses can be directly compared with the composition of enrolled courses for each academic area. We find strong correlations between first-term consideration set composition and enrollment set composition: the average Pearson correlation coefficient across the four academic areas is .77 (scatterplots provided in Appendix Figure A2). This demonstrates that 59% of the composition in course enrollments is predicted by consideration set composition alone. This means that observing consideration provides a strong early indicator of eventual enrollment patterns at the level of the individual student. Building on the result that consideration set composition is a strong predictor of course enrollments in the same term, we next investigated whether first-term consideration set composition predicts major declaration up to 2 years later. Figure 4 shows the average composition of consideration sets and enrolled courses in students’ first term, separated by the academic area of students’ majors at the beginning of their third year of college. The compositional patterns not only emphasize the similarity between consideration and enrollment set composition, they also reveal variation in consideration set composition by subsequent major. Students who ultimately major in a particular subject domain considered proportionally more courses in that domain at the very beginning of their college careers. In other words, consideration set composition during the first term of college is also indicative of subsequent major; the exception are Social Sciences majors whose consideration sets are more equally distributed across topical domains. Eventual Humanities and Arts majors, for example, consider on average 38% Humanities and Arts courses but only 11% Engineering courses, 12% Natural Sciences courses, and 14% Social Science courses (plus 25% “Other” courses, which include, e.g., required writing courses).
Consideration and enrolled course composition in first-year students’ fall terms 2016 and 2017 by domain of declared major at the beginning of their third year of college.
To examine the extent to which the composition of enrollment and consideration early in the undergraduate career is predictive of students’ subsequent major, we train multinomial logistic and random forest classifiers to predict a student’s major area for students in four areas: Engineering (53.6% of students), Social Sciences (17.6%), Humanities and Arts (17.7%), and Natural Sciences (11.1%). As predictors, we use each student’s first-term number of considered and enrolled courses in each of the five academic areas (same as major areas plus an “other” course category). We train the classifiers on 80% of the data (reserving 20% as a test set) with fivefold cross-validation and 500 trees. To balance major popularity in the training set, we randomly sample students for each major to match the smallest one, Natural Sciences. We then evaluate the classifiers’ balanced prediction accuracy (average of its recall and specificity) for each major to account for imbalance in the test set (Table 2). If students explored courses completely at random, the balanced accuracy would be 50%, but if they explored courses with a strong preference for an intended major, the balanced accuracy would be 80% or higher. What we find is that the balanced accuracy is between 63% and 69%, with minimal difference between the logistic and random forest classifiers. This confirms that even a simple characterization of the composition of a student’s consideration set in the first term can predict their declared major 2 years later.
Performance on the Test Set for a Multinomial Logistic and Random Forest Classifiers Predicting Students’ Eventual Major Based on First-Term Consideration and Enrollment Set Composition Features
Nevertheless, there is considerable breadth in early consideration, with a majority of courses considered in their first term falling outside of the area in which they will eventually major, a pattern that is more pronounced in consideration sets than enrollment sets. Students’ early academic exploration foreshadows their eventual major, even while they simultaneously investigate a variety of course options. While constraints of space and data availability prevent us from exploring this here, future inquiries should take advantage of administrative data to test for demographic variation in how the relationship between early consideration and subsequent major differs by gender, race, and household characteristics. The analyses presented here provide tractable strategies for these types of investigation.
Course Consideration in Context
We began our inquiry with the premise that course consideration is a large cognitive task. Especially at comprehensive universities with elective curriculums, students face many more potential courses of study than they can thoroughly consider—a case of decision making under bounded rationality. Borrowing classic insights from organizational science and cognitive psychology (Gigerenzer & Goldstein, 1996; Simon, 1955), we theorized that students would manage consideration by narrowing their rosters of considered courses. Quantitative data of Carta usage provided strong evidence that this is the case. Yet by themselves these data tell us little about how students experience the search process, or about the heuristics they use to navigate search. To at least begin to investigate these aspects of course consideration, we collected qualitative data from semistructured interviews and focus groups with 29 undergraduate students at the case university. We asked them about how they gather information about possible courses of study and how they go about the process of course selection. Qualitative findings comport with our theoretical priors but also offer insight about search “in the wild” that encourage us to amend at least one aspect of inherited theory for our case.
Consonant with prior theory, first- and second-year students report that they experience search and consideration processes as complex and extensive. While our quantitative analysis demonstrated that students generally view or enroll in approximately nine courses per term, interviews indicate that students cognize search as expansive. Consider the following reflections from Jack (all quotes are attributed to pseudonyms), a first-year student: A lot of times, I’ll pin a bunch of courses or write down a bunch of courses, or usually some combination of the two. Because one thing with searching for courses, is usually I’ll look for one thing, and then find a bunch more that seem interesting. So I don’t necessarily go in intending to look up all these different courses, but it just kind of ends up that I have a big list of them.
Jack doesn’t quantify his “big list,” nor does he specify what makes for “a bunch more” courses he finds interesting, but his choice of words suggests a large imagined consideration set.
Marie, also a first-year student, provides a detailed description of how she considers courses, indicating a multistage iterative process in which a list of “10 possible courses” is an intermediate outcome between search and selection: I’ll start out on my four-year plan. I’ll just see generally the classes that I probably want to take and then I’ll have like a list of ten possible classes that will probably work. Oh, and then I also look at [introductory seminars] and the [university distribution requirements] website. Also the Engineering Handbook . . . And I’ll make this list of all these classes that I want to take and write down the times for them and I’ll figure out, okay, I definitely need to take this class that’s required for my major and this class because it’s also required for my major and only offered in fall [term]. I’ll write those down as “definitely” classes that I need to take and then I’ll cross out classes that are at the same time as those, and then from the remaining classes, I’ll start off on [the university’s central course catalog], look at the description of it, and then go back to previous years because you can usually see the syllabus so I feel like that’s helpful in seeing like, what sorts of assignments are and stuff, and then I’ll go to Carta and see like if the grade distribution is reasonable, time commitment, and if people like the class.
Reflections like Marie’s suggest that students experience course consideration as a complicated process, even if the number of courses they consider is a very small proportion of all courses available to them in any given term.
Also comporting with prior theory is that students rely on heuristics to guide their search in the academic marketplace. In describing their searches to us, some students invoked a heuristic of exploration which comports with the university’s official narrative of how search should unfold. For example, Octavia, a second-year student, entered college thinking that she would pursue an English major. After taking courses in other disciplines out of curiosity, however, she switched gears and ultimately declared a computer science major: I came in intending to be an English major and, freshman fall, I was like “I’m going to [take a Computer Science] class and a drawing class and an engineering class.” Now I’m a [Computer Science] major. Never would’ve happened and I’m so glad I didn’t just dive into doing the core. That’s one of the things my freshman year I just super do not regret.
Other students, however, explicitly distinguished their own approach to search from the “freedom” they associated with broad exploration. Veronica, a first-year first-generation college student, spoke candidly about some of the differing expectations for those who are the first in their family to pursue higher education: There’s this big push to do things—to do majors, especially—like really heavily in tech, or things that just position you to be very career ready when you’re out of college . . . like doctors, engineers, computer scientists, that kind of thing, just because it’s like, you are very secure in having a job and a well paying job at the end . . . [Our university] has this mentality or like this push for intellectual vitality—like, do what you love and study what you love and what you’re passionate about. And I think that’s something that’s really challenging for kids that are first gen and/or lower income, because you’re not primed with this freedom to do what you love.
Veronica’s reflections indicate a more instrumental approach to course consideration that is explicitly keyed to the postcollege job market. Such an approach comports with prior research indicating that first-generation college students tend to be oriented toward securing earnings after graduation (Phillips et al., 2020; Tibbetts et al., 2016). Her remarks suggest that for some students (and unlike with Jack, quoted above), merely finding something interesting may not be a sufficient criterion for early course consideration or selection.
Clear professional goals can also shape early consideration strategies. For Aaron, an undeclared first-year student pursuing a premed track, maintaining a high GPA, and thus (as he sees it) increasing his chances of medical school admission, is a paramount consideration. Below is how Aaron narrated his consideration process. His account invokes the “grade distribution” and “time commitment” features of the Carta platform, which offer concise visual representations of aggregated grades, (self-reported) time investments, and reviews of prior students in each course: The first thing I would look at is the grade distribution. If it was good, then I would look at the time commitment. If that was also good, I would look through the reviews. And if the reviews were pretty good about the teachers, like the class in general, I would be like “okay, like, this is the class I want to take, I’ll just put this on my schedule.”
Aaron went on to describe “automatically” viewing the grade information on Carta, evaluating a particular course’s grade distribution as “pretty good” with 23% of prior students receiving grades of “A.” Optimizing course selection on the basis of grades received by prior students may not be what many educators might hope for, but it is certainly an evergreen strategy (Becker & Hughes, 1995).
While the above findings comport with inherited theory, interviews also provided evidence that encourages us to amend our conceptual framework in at least one sense. Specifically, while our model of the choice funnel depicted in Figure 1 implied a clear distinction between considered and enrolled courses, interviews indicated that students experience this distinction as fuzzy. Specifically, enrollment in a course does not necessarily imply commitment to actually take the course. Students may enroll in many more courses than they ultimately plan to keep in their schedules. Isabelle, a fellow first-year student, echoed Jack’s sentiments while also noting that enrolling in courses can be quite provisional: It’s way easier to sign up for a bunch of courses and then drop, than it is to try and add a course that’s already full, or maybe regret not having taken the opportunity. [Our university is] great in that you have three quarters to take more classes, but you’re still going to feel like you’re missing out on the opportunity to take a really cool course. So you might as well check it out in person and then decide if you’re going to keep it or drop it.
This strategy enables Isabelle to reserve spots in courses she thinks might fill up, while she sorts out other commitments she has for the coming term: Sometimes starting a quarter, I’m not quite sure how committed I am going to be to extracurricular stuff, and that can eat up a lot of time. So if I’m not sure if I’m going to be working that quarter or if I’m not sure I’m going to be in a theatre production, then I’m going to add more courses. And then when I decide oh, yeah, looks like I’m going to audition for that, then I’m gonna drop [some courses].
Rather than the clean distinction between consideration and choice implied by our initial model of academic decision-making in Figure 1, the boundary between consideration and enrollment is blurry, with at least some enrollments serving a reservation function rather than a sign of commitment.
This finding confirms the conservative intuition behind our definition of choice and consideration sets as comprising only those courses in which students remained enrolled subsequent to the case university’s course add/drop period. It also lends credence to the idea that the intramural academic marketplace is competitive. Students recognize that spots in some courses are scarce resources, and they use the university’s generous registration window to hoard options.
Discussion
We conceptualized the elective curriculum as an academic marketplace, in which undergraduates are obliged to make choices about how to spend their limited and often financially costly academic credits each term. Borrowing from decision theory, we posited course selection as a multistage process in which students transit from course availability to course choice via an intermediary stage—consideration—that has so far eluded systematic observation. Unlike any prior studies of academic decision making, our research platform enabled us to estimate the size and composition of students’ consideration sets, providing a novel window into a crucial stage in the decision process. We found that for entering first-year students at our case school in 2016 and 2017, the size of consideration sets was quite stable, with students considering approximately nine candidate courses for the courses they would ultimately choose in their first term. This indicates a substantial narrowing in the range of potential academic options at the earliest stage in students’ undergraduate careers. We found also that the composition of considered courses in the first term predicts the substantive domains of the majors they choose later in college. Students who ultimately major in engineering, humanities, natural science or social-science domains were considering relatively larger proportions of courses in those domains when they entered college. At the same time there is evidence that our case university’s elective curriculum is enabling academic exploration, because there is plurality in consideration set compositions of entering first-year students regardless of their subsequent declared majors. We note that our findings are robust to extending the period of observing consideration sets from the first fall term to the entire first academic year.
The near-universal use of the Carta platform among undergraduates at our case university provides prima facie evidence that students invest time and energy sourcing information when considering courses. At the same time, qualitative interviews indicate that course consideration, and the intramural academic marketplace in which it occurs, are substantially more complex than any single data source can fully capture. Students at our case university source information about courses and majors all over the place: not only on Carta but also on several other web-based platforms; from family, friends and acquaintances; from human advisors; and through sheer serendipity.
Despite this variety of information sourcing, our quantitative and qualitative evidence is clear that consideration sets are much more circumscribed than the population of courses available for enrollment each term. Students have limited information about the full range of alternatives available to them and limited cognitive capacity to comprehend all alternatives and the downstream consequences entailed by each one (Malhotra, 1982; Schwartz, 2004). The sheer scale and variety of the intramural academic marketplace can seem overwhelming. The academic calendar and registration period give students limited time to make choices. Popular classes may fill up quickly and demand more prompt decision making to secure spots. Taken together, these features of academic consideration under elective curriculums make global consideration of all available options intractable. As is common in other fateful choice scenarios like buying a car (Stigler, 1961), seeking a romantic partner (Miller & Todd, 1998), or selecting a school or residential neighborhood (Lareau & Goyette, 2014), students rely on whatever information sources they have to hand, and apply heuristics to simplify the selection problem.
Our findings corroborate more general models of how people navigate difficult choice dilemmas (Bruch & Feinberg, 2017). Such models presume a sequential reduction of alternatives as the choice process proceeds. Via active and passive engagement in the choice environment, including search activities, social interactions, and general observation, students develop an awareness of potential alternatives to consider. Awareness is dynamic, shaped by myriad environmental contingencies and conscious and unconscious actions over time, and not naturally observable. Consideration may be limited by ignorance or naivete as well as by active discretion (Roberts & Lattin, 1991). By limiting one’s search for a new pair of jeans to a single mall, or for a new home to a few neighborhoods, people substantially constrain their range of potential final choices in order to tame the scale and complexity of the task (Gigerenzer & Goldstein, 1996).
We believe that this complexity is precisely why educators should better instrument academic consideration in the service of both academic consideration and systematic empirical inquiry. The academic choices students make in college matter for their subsequent career trajectories (Carnevale et al., 2015; Roksa & Levey, 2010). While the intramural academic marketplace has long been opaque to scaled observation and intervention, the evolution of digital media gives educators powerful new tools for positively informing course consideration and choice.
Tools like Carta can offer a significant leap toward being able to observe and shape course consideration at scale, but we acknowledge that this type of educational data science is in an early stage. For instance, how the set of available and considered courses should be operationalized is an open question and this may need to be adjusted to different institutional practices such as the timeline for releasing the list of upcoming courses and course add/drop deadlines. Our robustness check suggests that only a small number of considered courses are unobserved on Carta. Likely sources of this measurement error are college orientation courses and popular STEM (science, technology, engineering, mathematics) gateway courses, which students may not feel the need to look up on Carta.
Conclusion
We conclude by offering a few examples of how pressing academic problems might be better understood and addressed through digital consideration platforms. First, to the extent that educators worry about the “funneling” of students into a small number of fields of study associated with highly compensated occupations (Binder et al., 2016; Deresiewicz, 2014; Guinier, 2016), the digital instrumentation of academic consideration can enable them to observe whether and when such funneling begins, and potentially intervene. While our own platform does not specifically encourage students to broaden consideration sets, it could easily be instrumented to do so. Just as early warning systems alert students and instructors to hazards of academic failure, course consideration systems could help students monitor characteristics of their consideration process. For example, first-year students might be notified that they “are narrowing their options to [names of fields]” and encouraged to consider courses in other substantive domains.
Second, academic consideration platforms might be of service to educators who are concerned about rates of entry and persistence of women and historically underrepresented groups in STEM fields. For example, there is now substantial evidence men and women navigate college with different understandings of what academic domains most suit their gender identities (Charles & Bradley, 2009; Hamilton, 2014) and, in highly quantitative fields especially, tend to interpret the same academic feedback different ways (Correll, 2014; Goldin, 2015). Digital consideration platforms would enable educators to observe for gendered variation in consideration of STEM courses, conversion of consideration to selection, and iterative change in consideration across academic terms in light of earned grades. Targeted information interventions might encourage students at hazard for STEM exit to “keep their options open” or “try another math course.”
Finally, digital consideration platforms might enable novel avenues of understanding and intervention into what may well be the most pressing problem of U.S. postsecondary education: exit before degree completion, or dropout. Nationwide, approximately 40% of those who enter a 4-year postsecondary program fail to complete a degree within 6 years (NCES, 2019). This problem is a wicked one, with myriad dimensions including K–12 preparation, college costs, financial-aid qualification, food insecurity and family and child-care obligations among many others (e.g., Bettinger, 2015; Grodsky & Jackson, 2009; Jack, 2016; Tyson et al., 2007). Yet field experts agree that at least part of the problem is the sheer complexity of intramural academic marketplaces: Colleges often present students with a cacophony of options unmatched by information and guidance to navigate it successfully (Bailey et al., 2015; Rosenbaum et al., 2006). To the extent that an accurate, reliable, free-for-use consideration scaffold such as Carta were made available on campuses where early college exit is an existential challenge, educators and researchers would have fresh mechanisms for observing and influencing how students make the choices that accumulate to successful college pathways.
Footnotes
Appendix
Acknowledgements
We thank Michael Bernstein, Sara Cina, David Lang, Arik Lifschitz, John Mitchell, Andreas Paepcke, student members of the Carta team, and our three anonymous reviewers for helpful input.
Authors
SORATHAN CHATURAPRUEK is a PhD graduate in Computer Science at Stanford University. His research interests include quantitative finance and computational social science, with a focus on instrumenting and supporting college course-selection decision-making processes.
TOBIAS DALBERG is a Wallenberg postdoctoral fellow at Stanford University. His main research interests lie in the sociology of education and the sociology of science, with a focus on social stratification and mobility, educational and career pathways, and the evolution of disciplines.
MARISSA E. THOMPSON is a PhD candidate in Sociology of Education and Education Policy at Stanford University. Her research focuses on inequality in access and returns to education by race and socioeconomic status.
SONIA GIEBEL is a PhD candidate in Higher Education and Sociology of Education at Stanford University. Her research interests include higher education admissions, gendered pathways through college, and how postsecondary institutions depict racial diversity.
MONIQUE H. HARRISON is a PhD candidate in Higher Education and Sociology of Education at Stanford University. She studies student pathways through college with an emphasis on gender, race, and first-generation college students.
RAMESH JOHARI is a professor in the Department of Management Science and Engineering, and (by courtesy) in the Departments of Computer Science and Electrical Engineering at Stanford University. He is broadly interested in the design, economic analysis, and operation of online platforms, as well as statistical and machine learning techniques used by these platforms.
MITCHELL L. STEVENS is a professor of education and (by courtesy) sociology at Stanford University. He studies postsecondary pathways and the political economy of higher education.
RENE F. KIZILCEC is an assistant professor of information science at Cornell University; email: