Sage Journals: Discover world-class research

Abstract

Prior research demonstrates the important role that financial considerations play in prospective students’ decision making when applying to and enrolling in graduate school. Racially/ethnically minoritized students, in particular, face persistent challenges during the graduate application and enrollment process. Capitalizing on a natural experiment, we identify the effects of introducing a PhD fellowship on the composition of applicants and enrolling students in PhD programs at a large public university’s graduate school of education. Using administrative data from 9 years of applications, we use difference-in-differences and event study analyses to show that the fellowship increased the number of applicants overall, as well as the share of Black applicants and enrollees in impacted cohorts, with no significant effects on academic preparation. To better understand why and how a PhD fellowship might impact students’ application behaviors and experiences once in graduate school, we supplement our primary findings with survey responses from current PhD students at the graduate school of education.

Keywords

graduate student access financial aid graduate school admissions difference-in-differences enrollment management diversity

Recent research on graduate education access has focused on the role of financial considerations in shaping prospective students’ graduate school decision making (e.g., Belasco et al., 2014; Denecke et al., 2016). Responding to long-standing concerns that financial barriers may discourage students from entering graduate school (e.g., Baum & Steele, 2018; Bauman, 2020; Malcom & Dowd, 2012), a growing number of institutions have announced policies aimed at increasing the financial appeal of their PhD programs. Several institutions, for instance, have announced plans to raise their minimum PhD stipends, including Emory University and the University at Buffalo (Emory University, 2018; Moyen & Gorny, 2019). Other institutions have sought to reduce restrictions on already-existing funding programs; the University of Chicago, for example, guaranteed humanities and social sciences PhD students funding for the full length of their program beginning in the 2020–2021 academic year, lifting a 6-year limit previously in place (Flaherty, 2019). This trend has also been seen in graduate programs outside of PhD programs, with one notable example coming from New York University’s (NYU) medical school, which announced that it would begin offering tuition-free graduate education to all admitted students moving forward (Jaschik, 2019). As a result of this program, NYU saw a 47% overall increase in applicants and a 102% increase in applicants who self-identify as a member of an underrepresented group in medicine, with the largest increase among those who identified as African American, Black, or Afro-Caribbean (Jaschik, 2019).

Indeed, a long-standing body of research has noted differential levels of graduate school access and graduate degree attainment by ascriptive characteristics such as race/ethnicity (e.g., Posselt & Grodsky, 2017). For example, in science, technology, engineering, and mathematics fields in 2016, Black/African American, Latinx/Hispanic, and Native American students accounted for only 22% of bachelor degree recipients, 13% of master degree recipients, and 9% of doctoral degree recipients (National Science Foundation, 2019). Motivated by such gaps in representation, a growing body of research has focused on efforts to increase the racial/ethnic diversity of graduate students (e.g., Garces, 2014; Griffin et al., 2012; Hurtado et al., 2010).

The types of funding opportunities described above—increased stipend amounts, guaranteed duration of support, and tuition waivers—have clear implications for addressing issues of equity in access to graduate education, particularly for racially/ethnically minoritized¹ students who have been consistently underrepresented in graduate programs. Still, there has been relatively limited research on the impact of these types of financial incentives on graduate school applicants’ behaviors. Prior research has frequently included descriptive analyses focusing on the association between various student and institutional characteristics and the likelihood of graduate school enrollment (e.g., Ethington & Smart, 1986; Mullen et al., 2003; Perna, 2004; Tienda & Zhao, 2017; Zhang, 2005). However, such studies have only occasionally focused on PhD program enrollment (with master’s programs receiving a greater share of attention) and are typically unable to isolate the impact of specific forms of financial support.

A specific focus on PhD enrollment is especially policy-relevant because PhD credentials serve as prerequisites for faculty careers, shape the questions and methods used in the next generation of research, and provide training for leaders both within and outside academia.

In this study, we seek to contribute to the literature by identifying the effects of introducing a new, comprehensive fellowship to support PhD students in a school of education at a large public research university. We focus in particular on the fellowship program’s potential effects on the volume of applications and the characteristics of applying and enrolling cohorts of students at impacted programs. This study seeks to address two primary research questions:

To what extent did the introduction of a fellowship program (which provided a stipend, tuition waiver, and subsidized health care) change the volume of applicants to PhD programs in a school of education?

To what extent did the introduction of a fellowship program lead to changes in the characteristics of PhD applicant and enrolling student cohorts, including the racial/ethnic composition and average academic preparation of students in those programs?

Our findings offer insights relevant to policy and practice on several dimensions. First, this study is among the first to offer quasi-experimental evidence on the effects of introducing a graduate financial aid package that includes a multiyear guaranteed stipend, tuition waiver, and subsidized health care. Second, there are conceptual reasons to believe that prospective students may be particularly responsive to the introduction of a fellowship program in graduate schools of education. Specifically, education is the field of study with the highest proportion of PhD students who primarily fund their studies through their own resources (46%), as well as the highest average graduate debt among research doctorate recipients ($28,600; Survey of Earned Doctorates, 2018). Third, because we observe characteristics of all applicants and matriculants to PhD programs across an entire university, the results of this study speak to the consequences of such initiatives on the composition of the cohort as a whole, offering a perspective that may be especially salient for institutions and departments considering the implementation of such initiatives.

The article proceeds as follows. First, we present a conceptual framework rooted in prior research on the college enrollment decision-making process for racially/ethnically minoritized students to help motivate why and how a fellowship may impact prospective students’ decisions on where to apply and enroll. We discuss relevant research on graduate student enrollment and the role of financial aid in student enrollment processes. We then share details about the fellowship program under study, along with context about the institution where the fellowship was offered. After a discussion of institutional data and the difference-in-differences and event study approaches we employ, we present evidence that the fellowship program increased applications to the affected PhD programs, while also increasing the racial and ethnic diversity of both the applicant and enrollee cohorts, with particularly strong evidence of an increase in the proportion of Black applicants. At the same time, we find no clear evidence of changes in academic preparation as measured by standardized tests. We also present exploratory evidence from a student survey that aims to uncover potential mechanisms through which the scholarship program may have been impactful, especially for racially and ethnically minoritized students. We conclude with a discussion of why these findings are particularly relevant given the ongoing drive to bring greater diversity and representation into doctoral education, and discuss implications for policy and future research.

Conceptual Framework

To guide our analysis of the fellowship program, we use a conceptual framework rooted in prior research on racially/ethnically minoritized students’ financial decision making regarding college enrollment. This literature helps shape the choices that racially/ethnically minoritized prospective students make in their application and enrollment processes for higher education.

Two tenets in particular undergird our conceptual framework, together contributing to several hypotheses about the outcomes a fellowship program is likely to produce. First, there are long-standing differences in family wealth by racial/ethnic background (Urban Institute, 2017). For instance, in 2019, compared with the median White family, typical Black and Latinx² families had one eighth and one fifth as much wealth, respectively (Bhutta et al., 2020). These stark disparities in financial resources potentially increase the salience of graduate program expenses and availability of stipends in the decisions that Black and Latinx students make regarding applying to and enrolling in doctoral education.

Second, prior research has demonstrated differential levels of student loan borrowing by racial/ethnic group membership at the undergraduate level (Hillman, 2015). One factor contributing to these differential levels is loan aversion, a phenomenon in which students avoid borrowing for educational expenses, even when the loan is likely to yield positive long-term returns. Prior research has found loan aversion to be especially high among Latinx individuals (Boatman et al., 2017), suggesting yet another factor that might discourage Latinx students from enrolling in graduate programs for which it would be necessary to borrow to cover educational expenses.

Together, these two tenets—differences in family wealth and loan aversion—may result in a scenario in which financial factors are especially relevant in the application decisions of prospective Black and Latinx students. By providing a tuition waiver, competitive stipend, and health insurance, a graduate fellowship program may therefore prove especially attractive to Black and Latinx applicants. When such a fellowship program includes a multiyear guarantee of funding, thereby further reducing uncertainty, the effects seem likely to be even stronger. The effects are also likely to be more noticeable in the field of education, which includes an especially large number of self-funded doctoral students and a comparatively high representation of Black and Latinx students (Survey of Earned Doctorates, 2018). Fellowship programs with these characteristics would appear to be poised to attract students who would otherwise attend a different institution for their PhD in education, as well as some subset of students who might not have otherwise entered a PhD in education.

Literature Review

Prior research has examined a variety of factors associated with graduate school application and enrollment. A number of these studies have relied on nationally representative data sets, finding relationships between graduate school enrollment and dimensions such as race/ethnicity (English & Umbach, 2016; Perna, 2004; Tienda & Zhao, 2017), undergraduate field of study (English & Umbach, 2016), undergraduate academic performance (Ethington & Smart, 1986; Mullen et al., 2003), undergraduate borrowing (Malcom & Dowd, 2012; Millett, 2003), and receipt of financial support (Ethington & Smart, 1986). While these studies have made valuable contributions to the understanding of graduate enrollment broadly, there are two primary constraints to these strands of research. First, most rely on data from baccalaureate recipients from 2001 or earlier, which leaves open the possibility that the observed relationships may have changed in recent decades. Second, this earlier work on graduate school choice uses disparate definitions of graduate education, sometimes combining enrollment in multiple degree programs (e.g., master’s, PhD, and professional) and occasionally omitting PhD programs altogether.

Beyond the overall decision of whether to attend graduate school in general, a somewhat narrower pool of research has explored prospective PhD students’ specific application and enrollment decisions. However, some consistent patterns have emerged with respect to factors associated with PhD application and enrollment decisions. Perhaps intuitively, PhD applicants rely on a number of factors that align with the undergraduate application process, such as institutions’ reputations and costs (Kallio, 1995). Additionally, several distinct considerations do appear to be closely related to application and enrollment decision-making at the PhD level, including financial support (both type and amount), familial responsibilities, job opportunities for spouses and partners, program flexibility, and geographic location (Allen et al., 1996; Olson & King, 1985; Talbot et al., 1996). Evidence also suggests that the criteria used by prospective students when deciding where to apply may also differ in the criteria used in making enrollment decisions (Bar et al., 2007). Furthermore, there appear to be differences by racial/ethnic group at the enrollment stage (Bersola et al., 2014). For instance, Black and Latinx students place greater emphasis on their perceived sense of inclusion at the graduate institution than their White and Asian counterparts (Bar et al., 2007). Additionally, the relative prioritization of considerations may vary along such dimensions as race/ethnicity, gender, student age, and applicants’ intended full-time/part-time enrollment status (Poock & Love, 2001).

Among the factors related to graduate school application and matriculation, this study builds on the modest but growing literature regarding the role of financial support in graduate education application and enrollment decision-making. To date, studies of graduate student financial support have primarily focused on its relationship with student outcomes, such as persistence (Ampaw & Jaeger, 2012; Ehrenberg et al., 2007), completion (de Valero, 2001; Ehrenberg & Mavros, 1995; Girves & Wemmerus, 1988), time-to-degree (Abedi & Benkin, 1987), and research performance (Horta et al., 2018). Some work has also illuminated variation in the rates of receiving various types of graduate financial support (e.g., assistantships, fellowships, grants, loans) by race/ethnicity (Millett & MacKenzie, 1995). While certain studies have highlighted the use of financial support as a recruitment strategy among master’s and professional students (Agrawal et al., 2005; Porter et al., 2014), comparatively less work has centered on the role of financial support in shaping application and enrollment decisions at the PhD level.

A small body of research has focused on the relationship between financial support and PhD application and matriculation. Descriptively, Malaney (1984) found that PhD students at the Ohio State University identified financial aid as their second most important reason for applying to the institution, behind only the departmental reputation, underscoring its potential salience in graduate school application and enrollment decision-making. Similarly, Bersola et al. (2014) found that students offered financial support were more likely to enroll at an institution than those without an aid offer, though only two types of support—fellowships and teaching assistantships—appeared to be related to higher levels of matriculation. A study of psychology PhD programs found that PhD students and faculty alike identified financial aid packages as instrumental in successful recruitment of racially/ethnically minoritized students (Rogers & Molina, 2006). These results suggest that shifts toward a fellowship-based model may prove attractive to PhD applicants, especially racially/ethnically minoritized applicants, though the limited research base calls for additional investigation, which the present study pursues.

Context and Program Description

Champion University (pseudonym) is classified as a large doctoral university: very high research activity and is located in the Northeastern United States. The approximate costs of graduate in-state and out-of-state tuition in the academic year 2017–2018 were $15,000 and $35,000, respectively (National Science Foundation, 2019). The School of Education (SOE) at Champion University is well ranked among graduate schools of education according to U.S. News and World Report. The SOE has nearly 25 degree programs and offers bachelor, masters, sixth year, and doctoral degrees. In fall 2017, just over 70% of the graduate enrollment were women and approximately 20% were racially/ethnically minoritized students.

A few years prior to the launch of the PhD fellowship program (PFP; pseudonym), the SOE provided about 14 similarly compensated doctoral fellowships to provide tuition waivers and stipends for assistantships. These fellowships functioned differently from the PFP in that the funding was assigned by Department faculty and was used to support admitted students, but was not marketed in advance as a way to recruit applicants. Thus, prior to the PFP, fellowships with stipends were available to some PhD students, but students could not reliably anticipate whether they would receive a stipend at the time they applied. These initial fellowship funds ceased soon before the start of the PFP, which was largely funded through a large donation to the SOE.

The PFP was launched in Autumn 2014 “to recruit exceptional doctoral students to the [SOE] to help . . . with [its] strategic initiatives, the preparation of outstanding professionals, and promote the positive culture of the [SOE]—locally, nationally, and internationally” (PFP Committee Chair, personal communication, September 10, 2015). The SOE-wide PFP committee was composed of faculty from all academic departments and an ex officio member from the Dean’s Office (PFP Committee Member, personal communication, December 8, 2020). This committee led the PFP selection process and recommendations. At the time, SOE competitor schools did not offer funding opportunities similar to the PFP both in terms of the scope of compensation and size of program. The SOE made no considerable changes in the PhD admissions criteria with the incorporation of the PFP.

The PFP guaranteed selected students a 4-year award that included full-time tuition expenses and offered a competitive stipend that slightly varied by year but was between $22,000 and $24,000 annually (nominal U.S. dollars). In some years, a modest summer stipend was offered, as was financial support to attend academic conferences. Further, as part of the PFP, students were eligible to access a “highly subsidized health insurance” plan (institutional document). Recipients were expected to enroll full-time for the duration of the PFP and gain research experience working as a graduate/research assistant with full-time faculty for 20 hours per week during the academic year. At the start of the PFP, existing SOE PhD students’ funding sources closely aligned with the Survey of Earned Doctorates (2015) with about half of students receiving a research or teaching assistantship and the remaining half of students paying out-of-pocket (PFP committee member, personal communication, December 8, 2020).

SOE administrators and faculty marketed the program through various outlets including Education Week, alumni, campus partners, professional conferences, and individual efforts by faculty. Beginning with the 2015–2016 academic year, marketing (e.g., website, social media) included materials with images predominantly of racially/ethnically minoritized students. Since the marketing campaign’s emphasis on racial/ethnic diversity and messaging about making PhD study more affordable may have impacted applicants’ decision making, we consider the marketing program a potentially key component of the PFP.

The first cohort of PFP students began in the academic year 2015–2016 and the second in the academic year 2016–2017. In the first year of the PFP, 10 offers were made and eight were accepted, of which five were racially/ethnically minoritized students (none were international). In the second year, six offers were made and five were accepted of which two were racially/ethnically minoritized students (another student was international). The PFP was suspended for academic years 2017–2018 and 2018–2019, which resulted in a halt of recruitment efforts for the PFP. The suspension was primarily attributed to financial constraints, since the program was mainly resourced with SOE funds. The suspension of the program meant that no incoming PhD students were awarded a PFP, though continuing PFP recipients continued to receive their awards. Between the two initial cohorts, the SOE made a total of 13 PFP awards.

Data and Method

Administrative Data

To explore the causal effects of this intervention, we analyzed administrative records of doctoral and master’s program applicants and enrollees across Champion University from the nine application cycles spanning the years 2010–2018. This approach enabled us to estimate deviations from prior trends in the application and enrollment patterns of PhD student cohorts, both in programs affected (i.e., those in SOE) and those unaffected by the PFP. Specifically, we employed both generalized difference-in-differences and event study analyses to estimate whether the introduction and cessation of the PFP in SOE changed the volume or profile of program applicants and enrollees to SOE PhD programs, relative to the profile of PhD applicants and enrollees in College of Arts & Sciences (CAS) programs at Champion University, which include programs in the natural sciences, social sciences, and other liberal arts disciplines.

By accounting for trends in PhD program applications at the university outside the affected programs, we are able to control for secular trends that could have affected the entire university in specific application cycles (e.g., labor market conditions). In addition to leveraging PhD programs and cohorts in CAS as one plausible counterfactual, we also examine the robustness of our findings to two additional comparison groups. First, given that PhD applicants and enrollees in social science programs may be especially similar in terms of the types of skills developed and career goals of PhD students in the SOE, we construct a comparison group that only includes PhD applicant and enrollee cohorts in social science programs (e.g., anthropology, communication, economics, and sociology). This has the benefit of not forcing a comparison between education students and students in the natural sciences (for example), who may be different in many unobserved ways. As an additional comparison group, we compare PhD applicant and enrollee year cohorts in the SOE to master’s applicant and enrollee year cohorts within the same school (SOE). This comparison protects against concerns that any detected effects might be attributable to other (non-PFP) changes in the SOE more broadly, or to secular trends within the field of education.

Data come from administrative records maintained by Champion University’s office of graduate admissions, which compiles information on applicants and matriculants to all master’s and doctoral programs. These records include information on gender, race/ethnicity, undergraduate institution, GRE scores, and ZIP code associated with all applicants to Champion University in the 2010–2018 admissions cycles. The data also include information on the degree type and program to which someone applied, and indicators of whether prospective students were admitted and whether they enrolled at Champion University. In Table 1, we present descriptive statistics for the PhD applicants and enrollees at the SOE and for our primary comparison, the CAS, during the years in study.

Table 1

Descriptive Statistics for School of Education (SOE) and College of Arts and Sciences (CAS) PhD Applicants and Enrollees

Characteristic	SOE PhD applicants	SOE PhD enrollees	CAS PhD applicants	CAS PhD enrollees
White	0.45	0.57	0.42	0.46
Black	0.06	0.11	0.03	0.02
Latinx/Hispanic	0.04	0.07	0.03	0.03
Asian	0.03	0.04	0.03	0.03
Multiple or other races	0.09	0.13	0.09	0.09
International	0.33	0.09	0.40	0.36
Female	0.75	0.72	0.54	0.52
In-region	0.47	0.75	0.32	0.43
In-state	0.28	0.60	0.10	0.23
Distance from Champion (miles)	417.70	235.66	646.40	450.53
Age (years)	29.56	32.25	25.06	25.59
GRE—Verbal (170-point scale)	154.18	157.24	155.57	155.69
GRE—Quantitative (170-point scale)	152.60	152.43	156.38	156.07
Number of students observed	1,228	304	19,367	2,146

Note. Includes all students applying to enter into either the SOE or CAS from the 2010–2011 to 2018–2019 cycles. GRE = Graduate Record Examination.

In our analyses, we use data aggregated to the school-by-cohort/year level, which allows for analyses of changes in the composition of schools’ cohorts over time. Results should be interpreted as the impacts of changes in funding on the characteristics of applicant and enrollee cohorts.

In our first analytic approach, we use a generalized difference-in-differences (DD) approach that maximizes statistical power and assumes that treatment effects are constant over time:

Y_{s t} = α + β_{1} T R E A T_{s} + β_{2} P O S T_{t} + β_{3} (T R E A T_{s} * P O S T_{t}) + ϵ_{s t} .

Following this classical DD model, $T R E A T_{s}$ represents an indicator for individual students in school s who applied to or enrolled in the treatment school (SOE), $P O S T_{t}$ is an indicator equal to 1 in the years during which the policy was in effect, and $T R E A T_{s} * P O S T_{t}$ is the interaction between these two focal indicators. The estimate of $β_{3}$ is the coefficient of interest, representing the causal effect of policy adoption on the outcomes of interest. Here, $β_{3}$ represents the degree to which the outcomes of interest (e.g., the proportion of students who are Black, or average Graduate Record Examination [GRE] scores) at SOE change after the introduction of the PFP, relative to the comparison College, CAS. Across 7 years (five pretreatment years and two treatment years) of applicant data and two schools (SOE is treated and CAS is comparison), we operationally use 14 data points in this model from which we make our inference.³,⁴ The strength of our findings is grounded in clear impacts, robustness to multiple analytic approaches, as well as to additional specification checks. All standard errors reported, here and throughout the article, are clustered by entry cycle. For our first specification, we use Huber–White robust standard errors to relax the homoskedasticity assumption (Huber, 1967; White, 1980). To account for the small numbers of clusters and small sample size, we also present p values from wild bootstrapping (see MacKinnon & Webb, 2018; Roodman et al., 2019). We draw 1,000 bootstrap replications using Webb (2013) weights to increase the number of potential draws. In all results tables, we report nonbootstrapped standard errors and significance levels, as well as bootstrapped p values. As an additional robustness check, we also employ a randomization inference (RI) process, in which we resample, or permute, the variable of interest (here, the interaction between treated programs of study during the treatment periods) 1,000 times to obtain a stronger estimate of the likelihood of finding significant results by chance (Heß, 2017; Imbens & Rubin, 2015; Rosenbaum, 2010). This process mainly results in similar significance levels to those from bootstrapping.

Our analytic approach can be thought to identify causal impacts of this initiative as long as two key assumptions are met. First, we rely on the assumption that there were no differential trends between the treatment and control school prior to the introduction of treatment. Figure 2 provides evidence that the trendlines for key characteristics at SOE and CAS were largely parallel until the first treated cohort (in the 2015 entry cycle), at which point SOE deviates from these parallel trends. Second, we must be confident that there are no other concurrent policy changes or reforms that might otherwise explain the difference in outcomes. Our familiarity with the setting, as well as interviews with the university administration make us confident that there were no other relevant changes that might have differentially impacted the treated and comparison groups.

In Table A1 in the appendix, we include a robustness check with results from a specification with data at the program of study level that adds fixed effects for program of study. We adopt the model without program fixed effects as our preferred model because (1) treatment was offered at the school (rather than program) level, (2) the small sizes of many individual PhD program-year cohorts (including in CAS, where several specific programs do not have any applicants in each entry cycle) means that fixed effects introduce the potential for substantial statistical noise, and (3) the goal of the PFP was to impact the overall student body at SOE rather than in specific programs. Regardless, estimates in Table A1 demonstrate that our key conclusions remain generally similar even after including program fixed effects, bolstering these results.

Our second approach allows for variation in treatment effects over time and fits an event study model in which we estimate the difference between the treated and comparison group programs in each admissions cycle. This approach highlights whether these differences vary at different points in the cycle before and after the program was enacted. Formally, we specify this event study model as follows:

Y_{s t} = α + β_{1} T R E A T_{s} + δ \sum_{t = - 5}^{t = 2} Y E A R_{t} + γ \sum_{t = - 5}^{t = 2} (T R E A T_{s} * Y E A R_{t}) + ϵ_{s t} .

In the event study specification, $Y_{s t}$ is an outcome of interest for students in school s, in year t (with years recentered as an indicator for the number of years before or after the PFP was first implemented), and $ϵ_{s t}$ is a mean zero random error term. $T R E A T_{s}$ is an indicator set to 1 for applicants or enrollees to the ever-treated school (SOE), with applicants/enrollees to the never-treated college (i.e., CAS) set to 0. $δ$ represents a vector associated with relative time before or after the policy change, which we operationalize as a set of binary indicators for the years prior to the implementation of the PFP (−5 or more to −1) as well as postintroduction (0 to 2) to capture year-specific effects on the outcome of interest. The $γ$ vector contains the parameters of interest, providing the estimated year-specific treatment effects via the interaction of treatment status and years before and after treatment $(T R E A T_{s} * Y E A R_{t})$ . In addition to uncovering any nonlinear trends such as fluctuations in the demand for PhD programs due to factors such as changing economic conditions, this approach also helps identify any pretreatment trends and test the assumptions of the parametric DD, while also allowing us to consider whether any impact of the PFP went away following the suspension of the initiative.

In these analyses, key outcomes of interest include the number of applicants, number of enrollees, and the proportion of applicants in a program cohort who identify as White, Black, Latinx/Hispanic, Asian, multiple or other races, international, from in-state, in-region (within the Northeastern United States), distance from Champion, along with age and average GRE scores (verbal and quantitative). These outcomes allow us to assess the extent to which the PFP changed the profile of applicants and enrollee characteristics among programs in the SOE.

Student Survey

In order to better understand the institutional effects of the PFP on applicant/enrollee numbers and overall program characteristics, it is also valuable to explore potential mechanisms through which the PFP may have influenced individual students. To better understand why and how the PFP may have impacted individual students, we surveyed enrolled PhD students at Champion University’s SOE to ask about the factors that influenced where they chose to apply for their PhD studies, how Champion University compared with other options when deciding where to enroll, and how they perceived their experience as a student once enrolled in Champion University.

All enrolled PhD students in the SOE received four emails over a 4-week period in fall 2019 soliciting their participation in a survey about their application and enrollment decision-making and graduate student experiences. Participation in the survey was strictly voluntary, anonymous, and without compensation. Out of the 156 potential respondents (identified by SOE), 41 participated in the survey, for a response rate of 26.3%. While the small sample size and modest response rate are clear limitations, the pool of respondents was largely representative of Champion University’s SOE PhD student population on observable characteristics we could match to administrative data, with 65% of respondents identifying as female (compared with 73% of the full population of PhD SOE students at Champion), 25% of respondents identifying with racially or ethnically minoritized student groups (compared with 20% of the full Champion PhD population), and 94% of respondents identifying as U.S. residents (compared with 91% of the full population).

Due to the limited number of survey responses, we are unable to separately consider students by individual racial/ethnic populations when analyzing the survey data, as we did with the analyses of administrative data. Instead, we jointly consider racially/ethnically minoritized students. Within the context of Champion University and the stated diversity goals of the PFP, this includes students identifying as Black, Latinx, and/or American Indian. We grappled with this decision, and acknowledge limitations in grouping multiple racial/ethnic groups into a single category. In particular, we are clear this categorization cannot fully capture the lived experience of any individual student. However, given that the PFP seemed to have a goal to racially/ethnically diversify SOE doctoral programs, we believe that it is a worthwhile effort to explore the possibility for differential impact among racially/ethnically minoritized students.

Results From Administrative Data

Descriptively, Figure 1 illustrates that during the fall 2015 and fall 2016 entry cycles (the years in which PFP funding was available to new SOE PhD students), the number of applicants and enrollees to SOE PhD programs increased and appeared to deviate from the trend both before and after the 2-year treatment period. Moreover, the increases in SOE PhD program sizes were more pronounced than for the CAS, especially in applications. In the first year of the program, for example, applications to SOE PhD programs increased from 133 in the previous year to 170 (a 28% increase), while applications to CAS PhD programs actually decreased from 2,175 to 1,920 (a 12% decrease). After the PFP was suspended, applications for the SOE declined, while CAS applications remained steady.

Figure 1.

Descriptive trends in numbers of applications and enrollments for SOE and CAS PhD programs.

Figure 2 highlights descriptive trends for selected characteristics that may be of interest to policy makers or institutional actors seeking to evaluate the extent to which the PFP may have changed the characteristics of the applicant and enrollee pool. Especially notable is the increase in the proportion of SOE PhD program applicants who identify as Black, growing from 4.5% of all applicants to 11.2% of all applicants in the first year of the scholarship (a 149% increase). This contrasts with the relatively stable proportion of applicants to CAS programs who identify as Black. The proportion of SOE program applicants who identify as Black decreases after the treatment period ends, suggesting that the deviation from the trend may plausibly be attributable to the PFP. In the period before the introduction of the PFP, there are no clearly observable pretreatment trends in the percent of applicants identifying as Black, providing evidence to support the parallel trends assumption. We also observe a strong uptick during the first treatment year of the percent of SOE enrollees who identify as Black, and again see a relatively flat trend among the CAS comparison programs. Figure 2 also demonstrates a small uptick in the portion of SOE program applicants/enrollees identifying as Latinx, and modest increases in average quantitative GRE scores during the treatment years, compared with no clear changes for the CAS cohorts.

Figure 2.

Descriptive trends in select characteristics of applicants and enrollees for SOE and CAS PhD programs.

Turning to the DD estimates, Table 2 displays the coefficients on the interaction term from the nonparametric DD model for each of several terms of interest. These represent the amount by which SOE PhD program applicant and enrollee cohorts changed relative to the average from the five preceding years, above and beyond any changes that occurred in the comparison group (CAS PhD programs). Columns I and IV display the estimates, standard errors, and significance levels of the DD treatment effect from the observed (nonbootstrapped) data for applicant and enrollee cohorts, while Columns II and V display p values associated with those same estimates from the bootstrapping procedure, and Columns III and VI display from randomization inference (RI). While the bootstrapping and RI procedures are more conservative and some estimates lose significance, the main results largely hold.

Table 2

Difference-in-Differences Estimates on Characteristics of Interest

Characteristic	I	II	III	IV	V	VI
	Applicants			Enrollees
	Observed coefficient (SE)	Bootstrap p value	RI − β p value	Observed coefficient (SE)	Bootstrap p value	RI − β p value
White (proportion)	−0.094^†(0.043)	.105	.108	−0.186*(0.069)	.101	.012*
Black (proportion)	0.046**(0.012)	.030*	.080^†	0.085*(0.030)	.074^†	.307
Latinx/Hispanic (proportion)	0.025^†(0.011)	.123	.042*	0.019(0.015)	.227	.387
Asian (proportion)	−0.004(0.005)	.410	0.438	−0.007(0.016)	.642	.648
Multiple or Other races (proportion)	0.054(0.030)	.135	.090^†	0.090^†(0.041)	.128	.051
International (proportion)	−0.026(0.023)	.280	.760	−0.001(0.048)	.989	.826
Female (proportion)	0.020(0.028)	.501	.882	−0.025(0.057)	.686	.671
In-region (proportion)	0.034(0.037)	.352	.686	0.039(0.062)	.575	.792
In-state (proportion)	−0.007(0.037)	.842	.886	−0.056(0.099)	.578	.606
Distance from Champion (miles)	4.051(41.633)	.913	.883	−137.273(112.531)	.455	.571
Age (years)	0.028(0.441)	.939	.878	−1.351(0.749)	.147	.673
GRE—Verbal (170-point scale)	−0.932^†(0.390)	.102	.632	−1.457(1.024)	.198	.595
GRE—Quantitative (170-point scale)	−0.807(0.882)	.587	.768	0.940(0.723)	.213	.562
Cluster-robust standard errors	×			×
Observations (school/entry cycles)	14			14

Note. Difference for School of Education PhD program students during PFP application cycles versus Pre-PFP cycles compared with difference for College of Arts and Sciences PhD program students. Each row represents a unique difference-in-differences estimate, in which the estimate represents a difference in the indicated student characteristics. Treatment period includes those applying to enter during the 2015–2016 cycle and 2016–2017 cycle. Pretreatment period includes those applying to enter during the 2010–2011, 2011–2012, 2012–2013, 2013–2014, and 2014–2015 entry cycles. Columns I and IV display the observed DD estimate and standard error for applicants and enrollees. Columns II and V are p values for each DD estimate as calculated from 1,000 samples drawn via wild bootstrapping using a Webb (2013) weight distribution. Columns III and VI are the randomization inference p values for each DD estimate, calculated by randomization inference through Stata’s ritest command using 1,000 random permutations (Heß, 2017). All standard errors are clustered by entry cycle and are reported in parentheses, with stars representing statistical significance. PFP = PhD fellowship program; DD = difference-in-differences; RI = randomization inference; GRE = Graduate Record Examination.

†

p < .10. *p < .05. **p < .01.

Estimates in Table 2 indicate that the most notable effects were on the proportion of applicants and enrollees who were Black (a 4.6 percentage point increase and 8.5 percentage point increase, respectively) above and beyond any changes in comparison programs. We also find a suggestively positive estimate on the percent of applicants who identify as Latinx (2.5 percentage points) and the proportion of enrollees identifying with multiple or other races (9.0 percentage points). Correspondingly, Table 2 shows that the increase in the proportion of applicants and enrollees who were Black (and the suggestive increase in Latinx and multiple/other race representation) is largely offset by a drop in the proportion of applicants and enrollees who were White (by 9.4 and 18.6 percentage points, respectively, though the statistical significance of these findings varies slightly across the bootstrapping and RI procedures). We also find no evidence of meaningfully reduced average quantitative or verbal GRE scores of applicants.

Estimates for select characteristics from our nonparametric event study model are presented in Figure 3 (estimates from additional characteristics can be found in Figures A1 and A2 in the appendix). As with the DD results, we find the clearest evidence of impact when considering the racial/ethnic diversity of the applicant and enrollee pools. Figure 3 shows that in the first year of the funding, the proportion of applicants who identified as Black grew by 6 percentage points more than the comparison group. Figure 3 also highlights a 9 and 11 percentage point increase (both significant at the .05 levels) in the proportion of applicants who identified with any racially or ethnically minoritized identity during the 2 years of the PFP. Again, we find little other evidence of effects for other student characteristics, such as GRE scores. Appendix Figures A3 and A4 show that these results are robust to all three of the potential comparison groups we tested. The consistency of results across all three potential comparisons lends particularly strong credibility to the conclusion that observed changes are attributable to the PFP and cannot be driven by any factors also affecting CAS PhD program students, social science program PhD students, or master’s program students within the CAS.

Figure 3.

Event study estimates for select characteristics of applicants and enrollees: PhD program students in CAS as comparison group.

Finally, in Table 3 we use a DD approach, similar to that presented in Table 2, to explore changes among racially/ethnically minoritized applicants and enrollees. When looking at only the characteristics of racially/ethnically minoritized applicant and enrollee cohorts, we find no evidence to suggest changes in the gender, age, or location profiles of racially minoritized applicant and enrollee cohorts in the treatment years. Moreover, we find no evidence of substantial impact on the average verbal or quantitative GRE scores of racially/ethnically minoritized SOE PhD applicants and enrollees during the PFP treatment period relative to CAS. While small sample sizes limit the precision of these estimates, these results hold after undergoing bootstrapping and suggest that the PFP was unlikely to have induced substantial decreases in the average GRE scores of racially/ethnically minoritized applicants and enrollees. This, alongside the DD results in Table 2, suggests that, if the introduction of the PFP increased the racial/ethnic diversity of the applicant and enrollee pools, it did not lower student academic preparation as proxied by the GRE scores. In Appendix Table A2, we take a similar approach to explore changes among Black applicants and enrollees, as they are the racial/ethnic group that appears most responsive to the introduction of the PFP (as evidenced by both DD and event study results). Here, we similarly find no noticeable evidence that the introduction of the PFP resulted in meaningful declines in academic preparation among Black students.

Table 3

Difference-in-Differences Estimates on Characteristics of Interest Among Racially/Ethnically Minoritized Applicants and Enrollees

Characteristic	I	II	III	IV	V	VI
	Applicants			Enrollees
	Observed coefficient (SE)	Bootstrap p value	RI − β p value	Observed coefficient (SE)	Bootstrap p value	RI − β p value
Female (proportion)	0.018(0.028)	.560	.882	−0.040(0.058)	.523	.788
In-region (proportion)	0.029(0.036)	.443	.686	0.026(0.061)	.685	.662
In-state (proportion)	−0.012(0.036)	.772	.886	−0.083(0.100)	.502	.794
Distance from Champion (miles)	6.169(40.271)	.887	.958	−99.459(117.112)	.581	.840
Age (years)	0.067(0.416)	.863	.978	−1.268(0.816)	.178	.836
GRE—Verbal (170-point scale)	−0.972^*(0.386)	.123	.632	−1.267(1.183)	.306	.644
GRE—Quantitative (170-point scale)	−0.779(0.827)	.624	.768	1.153(0.707)	.152	.736
Cluster-robust standard errors	×			×
Observations (school/entry cycles)	14			14

Note. Difference for racially/ethnically minoritized School of Education PhD program students during PFP application cycles versus pre-PFP cycles compared with racially/ethnically minoritized College of Arts and Sciences PhD program students. Each row represents a unique difference-in-differences estimate, in which the estimate represents a change in the indicated student characteristics. Sample only includes racially/ethnically minoritized applicants(Columns I and II) and enrollees (Columns III and IV). This includes applicants and enrollees identifying as Black, Latinx/Hispanic, some other race, or multiple races. Treatment period includes those applying to enter during the 2015–2016 cycle and 2016–2017 cycle. Pretreatment period includes those applying to enter during the 2010–2011, 2011–2012, 2012–2013, 2013–2014, and 2014–2015 entry cycles. Columns I and IV display the observed DD estimate and standard error for applicants and enrollees. Columns II and V are p values for each DD estimate as calculated from 1,000 samples drawn via wild bootstrapping using a Webb (2013) weight distribution. Columns III and VI are the randomization inference p values for each DD estimate, calculated by randomization inference through Stata’s ritest command using 1,000 random permutations (Heß, 2017). All standard errors are clustered by entry cycle and are reported in parentheses, with stars representing statistical significance. PFP = PhD fellowship program; DD = difference-in-differences; GRE = Graduate Record Examination; RI = randomization inference.

†

p < .10. ^*p < .05. ^**p < .01.

Results From Student Survey

Considerations When Applying to Doctoral Studies

Respondents first answered a series of questions about the relative importance of various factors that influenced the institutions where they had decided to apply to PhD programs. Respondents rated each factor on a scale of 1 (not a factor) to 4 (major factor). Table 4 displays the factors, from the highest overall rating to the lowest. Financial considerations weighed heavily on students’ minds when deciding where to apply, accounting for 3 of the 5 most highly rated factors. Given that the goal of the fellowship was to attract new students to SOE, Table 4 provides reason to believe that providing more and stronger scholarships and stipends would be particularly meaningful in inducing students to apply, given the importance respondents placed on financial considerations.

Table 4

Importance Placed on Various Factors When Considering Where to Apply

Factor	I	II	III	IV	V
Factor	Category	Overall importance rating (out of 4)	Rating among racially/ethnically minoritized students	Rating among nonracially/ethnically minoritized students	Difference for racially/ethnically minoritized students
Availability of stipends	Financial	3.75(0.65)	3.89(0.33)	3.68(0.75)	0.20(.43)
Fit with research interests	Academic	3.58(0.55)	3.56(0.73)	3.60(0.50)	−0.04(.84)
Reputation of department	Academic	3.47(0.81)	3.56(0.53)	3.52(0.87)	0.04(.91)
Stipend amount	Financial	3.42(1.00)	3.67(0.71)	3.28(1.10)	0.39(.34)
Scholarships	Financial	3.44(1.05)	3.78(0.67)	3.28(1.17)	0.50(.24)
Research opportunities	Academic	3.33(0.86)	3.44(1.01)	3.28(0.84)	0.16(.64)
Location	Personal	3.31(0.92)	3.56(0.73)	3.32(0.90)	0.24(.48)
Reputation of institution	Academic	3.19(0.79)	3.22(0.97)	3.24(0.72)	−0.02(.95)
Availability/quality of health insurance	Financial	2.89(1.21)	2.88(1.36)	2.88(1.20)	−0.01(.99)
Cost of tuition	Financial	2.53(1.23)	2.67(1.22)	2.56(1.26)	0.11(.83)
Teaching opportunities	Academic	2.22(1.10)	2.33(1.12)	2.24(1.13)	0.09(.83)
Number of students observed		36	9	25

Note. Categories are sorted by overall importance, as rated by students, with the most important categories listed first. In Columns II to IV, standard deviations are reported in parentheses. Column V reports the difference between racially/ethnically minoritized students and nonracially/ethnically minoritized students, with p values from two-sided t tests reported in parentheses.

Moreover, Table 4 also provides some suggestive evidence that racially/ethnically minoritized students applying to doctoral programs may be even more responsive to the introduction of funding inducements such as the PFP. While the small sample size lacks power to find statistical significance, we do observe suggestive differences in the weight that racially/ethnically minoritized students place on financial considerations. Most notably, racially/ethnically minoritized students placed greater importance on the stipend amount (a rating 0.39 points higher) and the availability of scholarships (a rating 0.50 points higher) than their nonracially/ethnically minoritized peers, the two highest differences of any factors. If the PFP aimed to increase the racial/ethnic diversity of applicants, this may offer one explanation of why—racially/ethnically minoritized prospective students may have been disproportionately attracted by the offer of funding, offering a potential mechanism through which the PFP may have helped attract a more racially/ethnically diverse pool of applicants.

Considerations When Deciding on Program to Enroll

The survey next asked students to think back to when they were deciding where to enroll in graduate school and to compare Champion University with another institution they most seriously considered attending; respondents rated Champion University on a scale of 1 (Champion was much less appealing than other program) to 5 (Champion was much more appealing than other program). In Table 5, we order these from most to least favorable for Champion. Overall, Table 5 highlights that students viewed Champion to be most competitive in terms of financial considerations, compared with other institutions vying for their enrollment, with Champion’s four most highly ranked factors all financial in nature. As might be expected, those who were offered funding from Champion viewed the financial factors more favorably than students who did not receive funding. Conversely, students who received funding actually viewed the reputation of Champion and their academic department less favorably than those who did not receive funding. Since these students all decided to enroll, this suggests that the financial advantages for students who received funding may have helped Champion overcome less favorable ratings in some academic and personal criteria.

Table 5

How Champion University Compared With Other Institutions When Deciding Where to Enroll

Factor	I	II	III	IV	V
Factor	Category	Overall Champion University rating (out of 5)	Rating among those offered funding	Rating among those not offered funding	Advantage for those offered funding
Stipend amount	Financial	4.41(0.96)	4.45(1.04)	4.36(0.92)	0.09(.83)
Availability of stipends	Financial	4.23(1.02)	4.55(0.82)	3.91(1.14)	0.64(.15)
Availability/quality of health insurance	Financial	4.05(0.90)	4.18(0.87)	3.91(0.94)	0.27(.49)
Scholarships	Financial	3.95(1.09)	4.45(0.82)	3.45(1.13)	1.00(.03)
Fit with research interests	Academic	3.86(0.94)	3.82(1.08)	3.91(0.83)	−0.09(.83)
Research opportunities	Academic	3.77(0.87)	3.73(0.90)	3.82(0.87)	−0.09(.81)
Reputation of department	Academic	3.77(0.97)	3.55(1.13)	4.00(0.77)	−0.46(.28)
Location	Personal	3.73(1.28)	3.64(1.21)	3.82(1.40)	−0.18(.75)
Cost of tuition	Financial	3.36(1.09)	3.64(1.21)	3.09(0.94)	0.55(.25)
Reputation of institution	Academic	3.36(1.00)	3.09(0.94)	3.64(1.03)	−0.55(.21)
Teaching opportunities	Academic	3.05(0.86)	2.90(0.88)	3.18(0.87)	−0.28(.47)
Number of students observed		34	16	18

Note. Categories are sorted by overall rating, as rated by students, with the most important categories listed first. In Columns II to IV, standard deviations are reported in parentheses. Column V reports the difference between students offered funding and students not offered funding, with p values from two-sided t tests reported in parentheses.

Experience in Program

While an implicit goal of the PFP was to attract and enroll more racially/ethnically minoritized applicants, the survey also asked a series of questions about students’ experiences once in graduate school at Champion to see how the PFP may have impacted their experience in graduate school. Table 6 displays students’ level of satisfaction (from 1 to 5) on a variety of factors, from least satisfied to most satisfied. Table 6 highlights that students who received funding were, in fact, more satisfied with their financial experience in graduate school. Students who received funding reported that they were more satisfied with their availability of health care and mental health services (both offered and highly subsidized as part of the PFP) and their overall financial situation. For additional context, it is worth noting that students who entered in the PFP cohorts (2015–2016 and 2016–2017) make up the majority of survey respondents who received funding, whereas those cohorts comprise slightly less than half of survey respondents who did not receive funding.

Table 6

Student Satisfaction Ratings of Aspects of Experience in Graduate School

Factor	I	II	III	IV	V
Factor	Category	Overall Champion University satisfaction rating (out of 5)	Rating among those offered funding	Rating among those not offered funding	Advantage for those offered funding
Availability/quality of health care	Financial	4.44(0.79)	4.69(0.48)	4.22(0.94)	0.47(.08)
Skill development	Academic	4.29(0.75)	4.31(0.70)	4.26(0.81)	0.05(.85)
Satisfied with decision to enroll	Personal	4.17(0.95)	4.06(1.06)	4.26(0.87)	−0.20(.54)
Mentorship from advisor	Academic	3.94(1.33)	3.75(1.57)	4.11(1.10)	−0.36(.44)
Mental health resources	Financial	3.76(1.28)	4.31(0.95)	3.28(1.36)	1.04(.02)
Mentorship from faculty	Academic	3.66(1.21)	3.38(1.36)	3.89(1.05)	−0.52(.21)
Overall experience	General	3.71(1.13)	3.75(1.18)	3.68(1.11)	0.07(.87)
Student groups and activities	General	3.48(1.20)	3.50(1.15)	3.47(1.28)	0.03(.94)
Teaching/research opportunities	Academic	3.40(1.42)	3.56(1.50)	3.26(1.37)	0.30(.54)
Preparation for job market	Academic	3.33(1.19)	3.06(1.18)	3.59(1.18)	−0.53(.21)
Finances in graduate school	Financial	3.31(1.23)	3.50(1.10)	3.16(1.34)	0.34(.42)
Students’ self-reported experiences while in PhD program
Average hours worked per week (tied to grad program)		16.41(7.20)	17.80(5.89)	15.18(8.16)	2.62(.31)
Average hours worked per week (not tied to grad program)		8.91(14.44)	5.00(5.77)	12.21(18.47)	−7.21(.14)
Average hours worked per week in Summer		21.83(15.65)	17.13(11.99)	25.79(17.50)	−8.66(.10)
Number of students observed		35	16	19

Note. Categories are sorted by overall satisfaction, as rated by students, with the most important categories listed first. In Columns II to IV, standard deviations are reported in parentheses. Column V reports the difference between students offered funding and students not offered funding, with p values from two-sided t tests reported in parentheses.

Students also responded to questions about how many hours per week they worked while in graduate school to provide a tangible sense of how funding may have changed their experience. As Table 6 shows, respondents with funding averaged 2.62 more hours per week on work related to their degree. Their outside-of-school workload, though, was substantially less: Students with funding worked 7.21 hours per week less in jobs not related to their program, and 8.66 hours per week less during the summer. Interestingly, students who received funding were not necessarily more satisfied with all other aspects of their graduate experience, suggesting that a fellowship, while important in the decision to apply and enroll, may not necessarily be a panacea that improves all other aspects of the graduate student experience.

Limitations

As with many studies focusing on a policy implemented at a single institution, our study faces a number of limitations. First, the number of students who receive the PFP is relatively small and the number of eligible cohorts is only two, which means that we are limited in our sample size and thus statistical power. Analyses of institutional policies at a school level such as the PFP always face limitations regarding statistical power, yet such institutional policies can be important to study nonetheless. We attempt to alleviate concerns about small sample sizes through our use of wild bootstrapping and randomization inference, as has become increasingly common in studies with smaller sample sizes and fewer treated units. Moreover, we find the existence of statistical significance for key results even with these limitations to statistical power to be especially compelling.

Second, data from a single institution may raise questions about generalizability; however, given that Champion is a large, public, research intensive university in a racially and ethnically diverse state, we believe that Champion offers several advantages as a relatively representative setting for a study of this nature.

Finally, the PFP as a policy incorporated multiple components, including tuition waivers, health care, paid assistantships, and an accompanying marketing campaign to spread the message about the PFP. As it is impossible for us to disentangle the component parts, we caution against generalizing to programs that may not incorporate all the aspects of the PFP; it may be plausible, for example, that particular aspects such as the marketing or the health care benefits played a key role in the policy’s success, and that the PFP could not have had the same outcomes without the marketing.

Discussion

Taken together, these findings provide evidence supporting the hypothesis that PhD stipends and tuition waivers can increase the number and racial/ethnic diversity of both program applicants and enrollees. The growth in the number of program applicants during the treatment period is consistent with a broad body of literature on the role of financial aid and debt in shaping graduate school decision-making (e.g., English & Umbach, 2016; Kallio, 1995; Weiler, 1994).

By increasing the size of the applicant pool, the funding initiative had the potential to aid graduate admissions committee members in their efforts to find talented students whose research interests were well-suited for the program. For instance, this might occur if the funding initiative increased awareness of the program among students with research interests in education but who might have otherwise applied to programs in related social science and human services fields (e.g., sociology, public administration, social work). Alternatively, if the applicants induced to apply as a result of the funding initiative were not well aligned with institutional priorities, such a rise in applications may principally reflect an increased burden for graduate admission committees. These factors underscore the importance of understanding shifts in the composition of program applicant and enrollee cohorts during the years of the funding initiative.

In particular, this type of intervention appears to have its strongest impact on inducing Black students to both apply and enroll, with more modest increases in applications from Latinx/Hispanic students. Consequently, this study highlights a potential opportunity to reach racially and ethnically minoritized populations that have historically been and continue to be underrepresented within doctoral education. Notably, the elevated rates of application and enrollment subsided in the years when the fellowship program was discontinued. Such responsiveness to the availability of the fellowship suggests that maintaining the suite of financial resources (e.g., stipend, tuition waiver, subsidized health care) is necessary to attract students. While the literature and our student survey both support the implication that the availability of such financial resources was meaningful for potential students, particularly Black and Latinx/Hispanic students, it is possible that another program attribute—the marketing campaign associated with the PFP—also played a role. However, despite the continued use of SOE marketing that highlighted racially/ethnically minoritized students even after the PFP ended, the event study estimates shown in Figure 3 reveal that the significant, positive estimates do not persist past the termination of the PFP, suggesting that such marketing efforts alone were not effective at producing the same outcomes as those achieved under the PFP.

Given the strong reliance on GRE scores in graduate admissions for many PhD programs (Posselt, 2016), we also examined the effects of the financial initiative on GRE scores of applicants and enrollees. At the same time, we acknowledge the limitations of GRE scores as proxies for student academic preparation and the growing GRE-optional movement in graduate admissions (Owens et al., 2020). We find little evidence of changes in GRE scores during the treatment period among enrollees. Thus, these findings suggest that during the years of the PFP, Champion University’s SOE increased racial/ethnic diversity while maintaining fairly consistent average GRE scores for incoming PhD students. Moreover, when only looking at the GRE scores of racially/ethnically minoritized applicants and enrollees, we detect no significant decrease in GRE scores. This finding runs counter to misperceptions that efforts to increase diversity necessarily lead to decreasing admissions standards (e.g., Sander & Taylor, 2012). Instead, these results provide encouraging evidence that Champion University was able to increase student diversity among its PhD applicant pool and student population while maintaining GRE scores of admitted and entering students.

Evidence from a survey of current doctoral students at Champion University’s SOE offers some insight into the mechanisms by which these shifts occurred. For instance, relative to other institutions considered during their PhD application process, students rated Champion University more highly on financial dimensions, but not on other factors. This finding suggests that the financial initiative may have played a strong role in attracting students who otherwise would not necessarily have viewed Champion University as favorably as other options for doctoral study. The survey results also indicate that racially/ethnically minoritized students placed greater weight on scholarships and stipends when considering where to apply than their nonracially/nonethnically minoritized peers. The increased importance placed on scholarships and stipends may help account for the elevated responsiveness the PFP observed for racially/ethnically minoritized students, and Black students in particular, in DD and event study analyses.

Finally, while the focus of this study was on application and enrollment behavior, it is notable that students receiving funding shared different levels of satisfaction and different lived experiences once in graduate school from those who did not receive funding. This finding is especially relevant for prospective students, who base their application and enrollment decisions partially on their assumptions about the experiences that they anticipate in graduate school. These results help us to better understand students’ decision making regarding where to apply and attend graduate school, in spite of imperfect information and limited capacity to fully weigh all relevant factors in their decisions. Since financial factors are likelier more clearly measurable and allow for cleaner comparisons with other potential school options than other considerations (e.g., “fit” or “prestige”), prospective students may be particularly reliant on financial considerations when deciding where to apply and enroll. While outside the scope of this study, these survey results also raise important questions motivating additional studies on the impact of funding on PhD student experiences and success once in their program.

Implications

These findings offer several implications for policy makers, researchers, and practitioners alike. Importantly, the results suggest that comprehensive financial aid packages for PhD programs that include stipend support, tuition waivers, and subsidized health care have the potential to increase overall application volume of applications, as well as applications specifically from racially/ethnically minoritized students, especially Black students. These findings point to PFP-type awards for PhD students as one potential mechanism for meeting federal, state, and institutional goals of diversifying doctoral education, especially along racial/ethnic dimensions. The results of this study may be particularly relevant for public institutions in states with affirmative action bans, where prior research has shown declining enrollment of racially/ethnically minoritized graduate students following the adoption of such bans (Garces, 2012).

At the same time, it is clear that further research is necessary along several dimensions. First, given that this study only examines PhD study in the field of education, it would be valuable to examine outcomes for similar initiatives in other types of doctoral programs. Types of programs that could be of particular interest include those in STEM fields, where problems of racial/ethnic underrepresentation are particularly acute, and the humanities, where fully funded programs are often scarce. Second, while this study focuses on a suite of financial inducements (e.g., stipend, tuition waiver, subsidized health care), there is likely value in seeking to separately assess the relative contribution of each component of the aid package. Such an examination may identify ways in which financially constrained institutions can most efficiently target their resources. Alternatively, closer examinations may confirm that students are most responsive to the combination of multiple forms of financial support, echoing findings from studies of undergraduate programs like the City University of New York’s ASAP program (Scrivener et al., 2015). Third, building on previous research regarding differences in graduate school decision-making by race/ethnicity (e.g., Perna, 2004), it would be valuable for future work to examine additional nonfinancial factors that play a role in the application and enrollment choices of prospective graduate students. Research in this vein may help identify methods for increasing the application and enrollment of student populations for whom we detected less clear impacts in response to PFP.

Finally, this study offers several lessons and considerations with respect to institutional practices. First, while our findings largely focus on admissions and enrollment decisions, it is also important for universities to ensure that they provide supportive, inclusive climates for the students that PFP-type programs attract, particularly, those from racially/ethnically minoritized backgrounds. Thus, before initiating a PFP-type program, institutions would be wise to examine other aspects of department culture and practices, to ensure that the environment is one in which the success of racially/ethnically minoritized students is well supported. The survey findings that PFP recipients were no more satisfied with nonfinancial aspects of their graduate experience than non-PFP recipients highlight that financial support alone is not enough to build an environment that fosters student success, reinforcing the importance of institutional practices above and beyond financial support. Second, the results from Champion University highlight the importance of sustained funding for financial aid initiatives. As is evident in Figures 1 and 2, increases in applicants were highly responsive to the availability of PFP awards, with steep declines in the year the PFP was discontinued. These results suggest that a short-term provision of PFP benefits will only yield short-term changes in applicant behaviors. However, for institutions hoping to leverage financial support programs to increase the size of their applicant pool and diversity of their PhD student populations, these results provide reason for optimism.

Footnotes

Appendix

Table A3

Difference-in-Differences Estimates (Posttreatment Period as Comparison)

Characteristic	I	II	III	IV	IV	VI
	Applicants			Enrollees
	Observed coefficient (SE)	Bootstrap p value	RI − β p value	Observed coefficient (SE)	Bootstrap p value	RI − β p value
White (proportion)	0.032(0.077)	.626	.796	−0.126(0.183)	.530	.153
Black (proportion)	0.030(0.014)	.093^†	.338	0.119(0.058)	.063^†	.131
Latinx/Hispanic (proportion)	0.012^†(0.004)	.055^†	.458	−0.051*(0.016)	.043*	.331
Asian (proportion)	−0.010(0.009)	.353	.124	0.001(0.014)	.916	.685
Multiple or Other races (proportion)	0.020(0.057)	.688	.372	−0.004(0.100)	.902	.671
International (proportion)	−0.084*(0.015)	.051^†	.434	0.062(0.053)	.273	.599
Female (proportion)	0.023(0.035)	.591	.880	−0.108(0.064)	.061^†	.445
In-region (proportion)	0.092*(0.024)	.049*	.194	−0.060(0.027)	.108	.610
In-state (proportion)	0.035(0.047)	.432	.658	−0.166(0.105)	.185	.467
Distance from Champion (miles)	−17.614(60.872)	.750	.962	24.741(129.884)	.719	.678
Age (years)	0.992^†(0.386)	.063^†	.708	0.920(3.062)	.602	.689
GRE–Verbal (170-point scale)	0.805(0.672)	.186	.726	−1.351(1.304)	.471	.575
GRE—Quantitative (170-point scale)	0.212(1.218)	.811	.926	2.942(1.299)	.065^†	.373
Cluster-robust standard errors	×			×
Observations (school/entry cycles)	8			8

Note. Difference for School of Education PhD program students during PFP application cycles versus post-PFP cycles compared with difference for College of Arts and Sciences (CAS) PhD program students. Each row represents a unique difference-in-differences estimate, in which the estimate represents a change in the indicated student characteristics. Treatment period includes those applying to enter during the 2015–2016 cycle and 2016–2017 cycle. Posttreatment period includes those applying to enter during the 2017–2018 and 2018–2019 entry cycles. Columns I and IV display the observed DD estimate and standard error for applicants and enrollees. Columns II and V are p values for each DD estimate as calculated from 1,000 samples drawn via wild bootstrapping using a Webb (2013) weight distribution. Columns III and VI are the randomization inference p values for each DD estimate, calculated by randomization inference through Stata’s ritest command using 1,000 random permutations (Heß, 2017). All standard errors are clustered by entry cycle and are reported in parentheses, with stars representing statistical significance. PFP = PhD fellowship program; DD = difference-in-differences; RI = randomization inference ; GRE = Graduate Record Examination.

†

p < .10. *p < .05. **p < .01.

ORCID iDs

Walter G. Ecton

Christopher T. Bennett

Notes

Authors

WALTER G. ECTON is a PhD candidate in the Department of Leadership, Policy, and Organizations at Vanderbilt University. His research focuses on the intersections between high school, higher education, and the workforce, with a specific focus on Career and Technical Education (CTE) in both the K–12 and higher education spaces.

CHRISTOPHER T. BENNETT is a PhD candidate in the Department of Leadership, Policy, and Organizations at Vanderbilt University. His research examines the ways that higher education policies can influence the trajectories of individuals and institutions, with a particular emphasis on students who have been historically underrepresented in higher education.

H. KENNY NIENHUSSER is an associate professor in the Higher Education & Student Affairs Program in the Neag School of Education at the University of Connecticut. His research examines the origins of public policies and their implementation environments that affect the postsecondary education access of minoritized youth in the United States.

MILAGROS CASTILLO-MONTOYA is an associate professor of Higher Education and Student Affairs in the Educational Leadership Department at the Neag School of Education in the University of Connecticut. Her research focuses on educational equity for first-generation college students, as well as racially minoritized students.

SHAUN M. DOUGHERTY is an associate professor of Public Policy & Education at Vanderbilt’s Peabody College. His research emphasizes the use of quantitative research methods to evaluate the impact of educational policies and programs, including understanding how the requirements, incentives, and behaviors that those policies produce develop human capital.

References

Abedi

Benkin

(1987). The effects of students’ academic, financial, and demographic variables on time to the doctorate. Research in Higher Education, 27(1), 3–14. https://doi.org/10.1007/BF00992302

Agrawal

J. R.

Vlaicu

Carrasquillo

(2005). Progress and pitfalls in underrepresented minority recruitment: Perspectives from the medical schools. Journal of the National Medical Association, 97(9), 1226–1231. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2594778/

Allen

Coccari

Webb

(1996). Master’s programs: What factors attract students. Journal of Marketing for Higher Education, 7, 73–85. https://doi.org/10.1300/J050v07n04_06

Ampaw

F. D.

Jaeger

A. J.

(2012). Completing the three stages of doctoral education: An event history analysis. Research in Higher Education, 53(6), 640–660. https://doi.org/10.1007/s11162-011-9250-3

Bar

D. A.

Wanat

Gonzalez

(2007). Racial and ethnic differences in students’ selection of a doctoral program to attend from those offering admission: The case of biomedical sciences. Journal of Women and Minorities in Science and Engineering, 13(1). https://doi.org/10.1615/JWomenMinorScienEng.v13.i1.20

Baum

Steele

(2018). Financing graduate and professional education: How students pay [AccessLex Institute Research Paper 18-01]. https://doi.org/10.2139/ssrn.3103241

Bauman

(2020, March 2). Do graduate assistants earn a living wage? Not in these cities. Chronicle of Higher Education. https://www.chronicle.com/article/Do-Graduate-Assistants-Earn-a/248169?cid=wcontentlist

Belasco

A. S.

Trivette

M. J.

Webber

K. L.

(2014). Advanced degrees of debt: Analyzing the patterns and determinants of graduate student borrowing. Review of Higher Education, 37(4), 469–497. https://doi.org/10.1353/rhe.2014.0030

Bersola

S. H.

Stolzenberg

E. B.

Love

Fosnacht

(2014). Understanding admitted doctoral students’ institutional choices: Student experiences versus faculty and staff perceptions. American Journal of Education, 120(4), 515–543. https://doi.org/10.1086/676923

10.

Bhutta

Chang

A. C.

Dettling

L. J.

Hsu

J. W.

Hewitt

(2020, September 28). Disparities in wealth by race and ethnicity in the 2019 Survey of Consumer Finances [FEDS Notes]. Board of Governors of the Federal Reserve System. https://doi.org/10.17016/2380-7172.2797

11.

Boatman

Evans

B. J.

Soliz

(2017). Understanding loan aversion in education: Evidence from high school seniors, community college students, and adults. AERA Open, 3(1). https://doi.org/10.1177/2332858416683649

12.

de Valero

Y. F.

(2001). Departmental factors affecting time-to-degree and completion rates of doctoral students at one land-grant research institution. Journal of Higher Education, 72(3), 341–367. https://doi.org/10.2307/2649335

13.

Denecke

Feaster

Okahana

Allum

Stone

(2016). Financial education: Developing high impact programs for graduate and undergraduate students. Council of Graduate Schools.

14.

Ehrenberg

R. G.

Jakubson

G. H.

Groen

J. A.

Price

(2007). Inside the black box of doctoral education: What program characteristics influence doctoral students’ attrition and graduation probabilities? Educational Evaluation and Policy Analysis, 29(2), 134–150. https://doi.org/10.3102/0162373707301707

15.

Ehrenberg

R. G.

Mavros

(1995). Do doctoral students’ financial support patterns affect their times-to-degree and completion probabilities? Journal of Human Resources, 30(3), 581–609. https://doi.org/10.2307/146036

16.

Emory University. (2018). Emory to increase stipends for Ph.D. students in Laney Graduate School. https://news.emory.edu/stories/2018/10/upress_lgs_announcement/campus.html

17.

English

Umbach

P. D.

(2016). Graduate school choice: An examination of individual and institutional effects. Review of Higher Education, 39(2), 173–211. https://doi.org/10.1353/rhe.2016.0001

18.

Ethington

C. A.

Smart

J. C.

(1986). Persistence to graduate education. Research in Higher Education, 24(3), 287–303. https://doi.org/10.1007/BF00992076

19.

Flaherty

(2019). Bold move in graduate education. Inside Higher Ed. https://www.insidehighered.com/news/2019/10/09/university-chicago-will-guarantee-full-funding-all-humanities-and-social-sciences

20.

Garces

L. M.

(2012). The impact of affirmative action bans in graduate education. The Civil Rights Project. https://files.eric.ed.gov/fulltext/ED533648.pdf

21.

Garces

L. M.

(2014). Aligning diversity, quality, and equity: The implications of legal and public policy developments for promoting racial diversity in graduate studies. American Journal of Education, 120(4), 457–480. https://doi.org/10.1086/676909

22.

Gillborn

(2005). Education policy as an act of White supremacy: Whiteness, critical race theory and education reform. Journal of Education Policy, 20(4), 485–505. https://doi.org/10.1080/02680930500132346

23.

Girves

J. E.

Wemmerus

(1988). Developing models of graduate student degree progress. Journal of Higher Education, 59(2), 163–189. https://doi.org/10.2307/1981691

24.

Griffin

K. A.

Muñiz

M. M.

Espinosa

(2012). The influence of campus racial climate on diversity in graduate education. Review of Higher Education, 35(4), 535–566. https://doi.org/10.1353/rhe.2012.0031

25.

Heß

(2017). Randomization inference with Stata: A guide and software. Stata Journal, 17(3), 630–651. https://doi.org/10.1177/1536867X1701700306

26.

Hillman

N. W.

(2015). Borrowing and repaying federal student loans. Journal of Student Financial Aid, 45(3), Article 5. https://files.eric.ed.gov/fulltext/EJ1080469.pdf

27.

Horta

Cattaneo

Meoli

(2018). Ph.D. funding as a determinant of Ph.D. and career research performance. Studies in Higher Education, 43(3), 542–570. https://doi.org/10.1080/03075079.2016.1185406

28.

Huber

P. J.

(1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1(1), 221–233.

29.

Hurtado

Newman

C. B.

Tran

M. C.

Chang

M. J.

(2010). Improving the rate of success for underrepresented racial minorities in STEM fields: Insights from a national project. New Directions for Institutional Research, 2010(148), 5–15. https://doi.org/10.1002/ir.357

30.

Imbens

G. W.

Rubin

D. B.

(2015). Causal inference for statistics, social, and biomedical sciences: An introduction. Cambridge University Press.

31.

Jaschik

(2019, January 14). Admissions surge after NYU Med goes tuition-free: Applications from underrepresented minority students double. Inside Higher Ed. https://www.insidehighered.com/admissions/article/2019/01/14/nyu-medical-school-sees-surge-applications-after-going-tuition-free

32.

Kallio

R. E.

(1995). Factors influencing the college choice decisions of graduate students. Research in Higher Education, 36(1), 109–124. https://doi.org/10.1007/BF02207769

33.

MacKinnon

J. G.

Webb

M. D.

(2018). The wild bootstrap for few (treated) clusters. Econometrics Journal, 21(2), 114–135. https://doi.org/10.1111/ectj.12107

34.

Malcom

L. E.

Dowd

A. C.

(2012). The impact of undergraduate debt on the graduate school enrollment of STEM baccalaureates. Review of Higher Education, 35(2), 265–305. https://doi.org/10.1353/rhe.2012.0007

35.

Malaney

G. D.

(1984). An analysis of financial aid in the recruitment of graduate students at the Ohio State University. Journal of Student Financial Aid, 14(1), 11–19.

36.

Millett

C. M.

(2003). How undergraduate loan debt affects application and enrollment in graduate or first professional school. Journal of Higher Education, 74(4), 386–427. https://doi.org/10.1353/jhe.2003.0030

37.

Millett

C. M.

MacKenzie

(1995). An exploratory study of the role of financial aid in minority doctoral education [ASHE Annual Meeting Paper].

38.

Moyen

Gorny

(2019, October 3). UB extends full Ph.D. Stipend funding for additional year. UB Spectrum. https://www.ubspectrum.com/article/2019/10/ub-extends-full-ph-d-stipend-funding-for-additional-year

39.

Mullen

A. L.

Goyette

K. A.

Soares

J. A.

(2003). Who goes to graduate school? Social and academic correlates of educational continuation after college. Sociology of Education, 76(2), 143–169. https://doi.org/10.2307/3090274

40.

National Science Foundation. (2019). Women, minorities, and persons with disabilities in science and engineering: 2017. https://ncses.nsf.gov/pubs/nsf19304/digest/field-of-degree-women#mathematics-and-statistics

41.

Noe-Bustamante

Mora

Hugo Lopez

(2020). About one-in-four U.S. Hispanics have heard of Latinx, but just 3% use it. Pew Research Center: Hispanic Trends. https://www.pewresearch.org/hispanic/2020/08/11/about-one-in-four-u-s-hispanics-have-heard-of-latinx-but-just-3-use-it/

42.

Olson

King

M. A.

(1985). A preliminary analysis of the decision process of graduate students in college choice. College and University, 60(4), 304–315.

43.

Owens

Zwickl

Miller

(2020). “Not required,” “No required minimum,” and “optional” general and physics GRE requirements: The impact on prospective graduate students. Bulletin of the American Physical Society, 65(1). https://meetings.aps.org/Meeting/MAR20/Session/G07.3

44.

Perna

L. W.

(2004). Understanding the decision to enroll in graduate school: Sex and racial/ethnic group differences. Journal of Higher Education, 75(5), 487–527. https://doi.org/10.1353/jhe.2004.0032

45.

Poock

M. C.

Love

P. G.

(2001). Factors influencing the program choice of doctoral students in higher education administration. NASPA Journal, 38(2), 203–223. https://doi.org/10.2202/0027-6014.1136

46.

Porter

Yang

Hwang

McMaken

Rorison

(2014). The effects of scholarship amount on yield and success for master’s students in education. Journal of Research on Educational Effectiveness, 7(2), 166–182. https://doi.org/10.1080/19345747.2013.836764

47.

Posselt

J. R.

(2016). Inside graduate admissions: Merit, diversity, and faculty gatekeeping. Harvard University Press. https://doi.org/10.4159/9780674915640

48.

Posselt

J. R.

Grodsky

(2017). Graduate education and social stratification. Annual Review of Sociology, 43, 353–378. https://doi.org/10.1146/annurev-soc-081715-074324

49.

Rogers

M. R.

Molina

L. E.

(2006). Exemplary efforts in psychology to recruit and retain graduate students of color. American Psychologist, 61(2), 143–156. https://doi.org/10.1037/0003-066X.61.2.143

50.

Roodman

Nielsen

M. Ø.

MacKinnon

J. G.

Webb

M. D.

(2019). Fast and wild: Bootstrap inference in Stata using boottest. Stata Journal, 19(1), 4–60. https://doi.org/10.1177%2F1536867X19830877

51.

Rosenbaum

P. R.

(2010). Design of observational studies. Springer.

52.

Sander

R. H.

Taylor

(2012). Mismatch: How affirmative action hurts students it’s intended to help, and why universities won’t admit it. Basic Books.

53.

Scrivener

Weiss

M. J.

Ratledge

Rudd

Sommo

Fresques

(2015). Doubling graduation rates: Three-year effects of CUNY’s Accelerated Study in Associate Programs (ASAP) for developmental education students. MDRC. https://doi.org/10.1257/rct.1637

54.

Stewart

D. L.

(2013). Racially minoritized students at U.S. four-year institutions. Journal of Negro Education, 8 2(2), 184–197. https://doi.org/10.7709/jnegroeducation.82.2.0184

55.

Survey of Earned Doctorates. (2018). Data tables 35 & 38. https://www.nsf.gov/statistics/2018/ nsf18304/data.cfm

56.

Talbot

D. M.

Maier

Rushlau

(1996). Guess who’s coming to doctoral programs: Factors influencing potential students’ choices of doctoral programs in student affairs. College Student Affairs Journal, 16(1), 5–15.

57.

Tienda

Zhao

(2017). Institutional and ethnic variations in postgraduate enrollment and completion. Journal of Higher Education, 88(4), 561–592. https://doi.org/10.1080/00221546.2016.1272332

58.

Urban Institute. (2017, October 5). Nine charts about wealth inequality in America (Updated). https://apps.urban.org/features/wealth-inequality-charts/

59.

Webb

M. D.

(2013). Reworking wild bootstrap based inference for clustered errors [Working Paper No. 1315]. Queen’s Economics Department. https://www.econ.queensu.ca/research/working-papers/1315

60.

Weiler

W. C.

(1994). Expectations, undergraduate debt and the decision to attend graduate school: A simultaneous model of student choice. Economics of Education Review, 13(1), 29–41. https://doi.org/10.1016/0272-7757(94)90021-3

61.

White

(1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48(4), 817–838. https://doi.org/10.2307/1912934

62.

Zhang

(2005). Advance to graduate education: The effect of college quality and undergraduate majors. Review of Higher Education, 28(3), 313–338. https://doi.org/10.1353/rhe.2005.0030

If You Fund Them,Will They Come? Implications From a PhD Fellowship Program on Racial/Ethnic Student Diversity

Abstract

Keywords

Conceptual Framework

Literature Review

Context and Program Description

Data and Method

Administrative Data

Student Survey

Results From Administrative Data

Results From Student Survey

Considerations When Applying to Doctoral Studies

Considerations When Deciding on Program to Enroll

Experience in Program

Limitations

Discussion

Implications

Footnotes

Appendix

ORCID iDs

Notes

Authors

References