Failed Warnings

Abstract

Background:

Academic probation is a nearly universal but underresearched policy practiced at most postsecondary institutions.

Objectives:

To evaluate the impact of probation warning letters on students’ academic performance.

Research design:

Employing the inferentially strong regression discontinuity design, we evaluated the impact of two versions of warning letters (U.S. mail and email), noting their impact on next semester grades.

Subjects:

Probation and nonprobation students at a large, Midwestern college enrolled during two, successive fall–winter semester pairs (n > 17,000, for each pair).

Measures:

Fall and winter grade point averages (GPAs) were identified for each individual student in the study sample.

Results:

Using both parametric and nonparametric analyses, we found that neither delivery method, paper or electronic, had a consistent, significant impact on subsequent GPA or odds of a GPA ≥ 2.0 during the next semester. Four of the eight measures of effect for GPA were small and positive (0.02–0.15; one significant positive outcome), and four were negative (−0.02 to −0.08; one significant negative result). A separate set of analyses that excluded students who took a single course led to further inconsistency in results. Supplementary analyses that excluded students scoring far from the cut-point yielded results consistent with a no-difference conclusion. Our findings also indicated that, after being placed on probation, only a small percentage of students were able to avoid movement to the next stage of academic sanction.

Conclusions:

Warning letters notifying students of probation status, lacking a staff-focused intervention, had little impact on academic performance.