Sage Journals: Discover world-class research

Abstract

Previous research investigating the relationship between peer-assisted study sessions (also called supplemental instruction or peer-assisted learning) and academic performance has a number of concerns. These include the lack of inclusion of important variables such as academic motivation and personality. This study (N = 233) investigated how motivation, personality, and control variables (prior subject attempts, number of university semesters completed, prior academic achievement) have an impact on the relationship between peer-assisted study sessions attendance and academic performance for psychology students. The results indicated that peer-assisted study sessions attendance predicted academic performance when controlling for academic motivation, personality, and control variables; however, the magnitude of the relationship was almost halved (r = .27 to ß = .13). Peer-assisted study sessions attendance mediated the relationship between neuroticism (κ²= .04) and prior academic achievement (κ²= .05) and academic performance, indicating that participants with these characteristics benefit from the sessions. Finally, adjunct peer-assisted study sessions focused on assessment items appear to be a large part of its efficacy in this sample.

Keywords

Peer-assisted study sessions supplemental instruction academic performance motivation personality

The academic support program entitled peer-assisted study sessions (PASS) in Australia, supplemental instruction in the USA, and peer-assisted learning in the UK, is used in over 1500 universities in over 29 countries (Martin, 2008). PASS have a number of characteristics which make them distinct from other academic interventions: they are attached to subjects that have high difficulty and/or lower grades; they are led by a student who has previously done well in the subject; the PASS leader’s goal is to facilitate learning activities rather than teach (i.e. by encouraging group learning activities and having students answer each others’ questions, rather than the PASS leader presenting content); they are voluntary; and, they are offered to all students, not just at-risk students (Dawson, van der Meer, Skalicky, & Cowley, 2014).

Widely cited articles from the 1990s (e.g. Matin & Arendale, 1993) suggest that PASS is effective at increasing grades, decreasing fail and withdraw rates, and increasing graduation rates, even after controlling for prior academic achievement and ethnicity (Dawson et al., 2014; Hurley & Gilbert, 2008; van der Meer, Wass, Scott, & Kokaua, 2017). However, a number of researchers have noted theoretical and methodological concerns with previous research. These include the way that PASS attendance has been operationalized, and the lack of inclusion of important control variables, particularly academic motivation (Dawson et al., 2014; McCarthy, Smuts, & Cosser, 1997; Paloyo, 2015), but also, perhaps, personality, and the lack of attention to who benefits from PASS attendance. Furthermore, the lack of reporting of effect sizes makes the interpretation of the efficacy of PASS problematic (Dawson et al., 2014), particularly when much of the research in this area consists of whole university cohorts as samples, so even small effect sizes can be found to be statistically significant. The aim of the current study was to address these previous limitations by investigating whether PASS attendance was related to academic performance after controlling for personality and motivation.

The first limitation with previous studies is not including psychological variables that could, potentially, confound the issue. There is a large body of literature that links psychological factors to learning and learning outcomes (see Richardson, Abraham, & Bond, 2012 for a review). Academic motivation is often recognized as having a potential effect on both PASS attendance and academic performance; however, few studies have directly investigated it. Some studies use prior academic performance as a proxy for motivation (Fayowski & MacMillan, 2008; Jones, 2013). However, academic performance is not an appropriate proxy for motivation, as it could be considered a consequence of motivation and other factors (e.g. academic ability, personality, etc., Richardson et al., 2012).

Only five studies have directly attempted to measure or control for academic motivation in relation to PASS attendance and academic performance, with inconsistent findings. For example, Hodges, Dochen, and Joy (2001) found that the voluntary group (attended one PASS session or more) had significantly higher motivation than the mandatory (PASS integrated into course structure) and no attendance groups. The mandatory and voluntary groups both had higher final course grades than the no attendance group. However, this analysis of variance did not control for motivation, making it difficult to interpret the impact of motivation on the PASS attendance–academic performance relationship. Conversely, Malm, Bryngfors, and Mörner (2011) used a single-item measure of motivation (“I am very motivated to study”) and found that although motivation was related to PASS attendance (attending three sessions or more), it was not related to exam pass/fail rates. However, these analyses were included in the discussion section of the study, and it appears as though a secondary sample was used, and motivation was not controlled for in the primary analyses. Gattis (2002) measured intention to attend PASS at the beginning of the semester. Those who wanted to attend, but could not due to the times they were offered were placed in a motivation control and were compared with those who attended four or more PASS, one to three PASS, those who did not attend PASS but attended a drop-in service, and those who did not attend PASS or the drop-in service and were not part of the PASS motivation control. They conducted an analysis of covariance with an admission index as a control and found a significant effect of group on academic performance. However, no post-hoc tests were conducted, making it difficult to interpret which groups were significantly different from each other. These studies suggest motivation may be associated with both attendance and performance, but the designs of the studies do not allow firm conclusions to be drawn. Other studies have not found any effect of motivation.

Two studies found no differences in motivation between attendees and non-attendees and no effect of motivation on performance. Terrion and Daoust (2011) matched 92 participants in their PASS group (attended at least twice, were in first year of study) to 92 participants with the same demographics as for the non-PASS group. They found no significant differences in motivation between the groups, and no significant differences between the groups in course grades. Similarly, Malm, Bryngfors, and Mörner (2015) found no differences in motivation between PASS attendees (0, 2–10, 12–20, > 20 contact hours) and non-attendees. However, they only included participants who submitted a follow-up survey at what appeared to be the beginning of the next semester, which raises concerns over attrition (i.e. those who were unmotivated dropped out). As such, it is currently unclear whether motivation is related to PASS attendance and outcomes as few studies have attempted to control for academic motivation in a rigorous or systematic way. Further, none of the previously mentioned studies controlled for academic motivation at the same time as other important variables such as academic ability or prior academic achievement.

Although academic motivation is the variable most often recognized as having a potential impact on the PASS attendance–academic performance relationship, there is reason to believe that other psychological variables, such as personality, would similarly have an impact on this relationship. There is an extensive literature linking personality with academic behaviors and performance (see Poropat, 2016). For instance, conscientiousness predicts academic performance (Poropat, 2009), and part of the reason for this may be due to increased effort (Corker, Oswald, & Donnellan, 2012). Conversely, self-rated neuroticism is negatively correlated with academic performance (Poropat, 2014), and associated with academic withdrawal (Komarraju & Karau, 2005). It is likely that these same factors may have an impact on PASS attendance. As such, we consider the role that personality may have on the PASS attendance–academic performance relationship.

A second limitation of most previous research is the operationalization of variables. Many previous studies dichotomize or categorize continuous variables such as PASS attendance and academic performance. In the case of PASS attendance, researchers have often used attendance at one PASS session at least, as the criteria for PASS “attendance” (e.g. Hodges & White, 2001; Okun, Berlin, Hanrahan, Lewis, & Johnson, 2013). It seems unlikely that attendance at one session would have a notable effect on performance, unless the treatment effect is very strong. Just as problematic, if the treatment effect is that strong, those who attend many sessions should have a distinct advantage over those who only attend one. Assuming a linear relationship, those who attend only one session likely have more in common with those attending no sessions, compared with those attending many.

Even those studies which have chosen different criteria for classifying PASS attendance (e.g. ≥ 5 sessions, Hughes, 2011) are problematic. The dichotomization of a continuous variable can cause a loss in power or spurious results, due to an unnecessary decrease in available data (MacCallum, Zhang, Preacher, & Rucker, 2002). This problem also exists for those studies which have categorized PASS attendance based on discrete categories (e.g. 0, 1–3, ≥ 4, Bruno et al., 2016), as they are eliminating data by forcing a continuous variable into categories. Additionally, the way PASS attendance is categorized is inconsistent across studies, with researchers often choosing their own criteria for “attendance,” making it difficult to compare results across studies. This problem also exists for the operationalization of academic performance. Many studies exploring PASS have used final grade bands (e.g. 0–4, Bowles & Jones, 2003) or course pass/fail and/or withdraw (e.g. Cheng & Walters, 2009), as their measure of academic performance, although this may be due to data access considerations.

As a further note, the inclusion of students who have withdrawn or not submitted all of their assessments raises an additional limitation (cf. Jones, 2013). Oja (2012) notes that there may be many external reasons for both withdrawing (e.g. family commitments, a job offer, not liking the program) and failing (e.g. ostensibly “withdrawing” by not attending and submitting assessments, but not officially withdrawing). These issues would affect both PASS attendance and course grades and, consequently, confound the relationship. For instance, if someone decides they want to do a different degree, they may withdraw from the course or not complete assessments, and also not attend PASS. As such, it is not PASS causing decreased failure/withdrawal rates, but rather external variables affecting both PASS attendance and withdrawals/non-submissions, thus, inflating the true relationship between PASS attendance and academic performance.

Finally, although many studies compare characteristics of participants in PASS with those of non-attendees, fewer investigate how these characteristics, along with attendance, relate to academic performance. There is a growing trend in examining the comparative efficacy of PASS attendance across participant categories: Prior achievement (e.g. Malm et al., 2015; van der Meer, et al., 2017); ethnicity or minority status (e.g. Okun et al., 2013; van der Meer et al, 2017); or gender (e.g. Fayowski & MacMillan, 2008). These studies are commendable; however, they are, in essence, investigating an interaction effect to see whether the relationship between PASS attendance and academic performance is stronger for participants with certain characteristics, and can only tell us who may benefit from PASS (if they attend). To understand who is actually benefiting from PASS attendance, we need to investigate the relationship between participant characteristics, attendance, and academic achievement through a mediation analysis, and to our knowledge, no studies have done this.

The current study

Our study sought to examine the efficacy of PASS on academic performance by addressing some of the concerns outlined in the previous section. Consequently, there were two main aims of this research. First, we sought to investigate whether PASS attendance was related to academic performance after controlling for personality and motivation, along with other control variables (previous course attempts, prior academic achievement, number of semesters of university completed). Note, in Australia, academic ability assessments are a very uncommon method of university entrance, so we were unable to access academic ability data.

The second purpose of this study was to see whether PASS mediated the relationship between personality, motivation, or control variables, and academic performance. We did not plan mediation analyses for specific variables a priori and, instead, investigated mediation effects for variables which had a significant relationship with both PASS attendance and academic performance.

Method

Participants

There were 233 psychology students in the study and they made up 53.44% of the total number of students enrolled on the course with the associated PASS program. Participants could elect to receive half a credit point toward the research participation component of the course for participation. Course marks for seven participants were unavailable, indicating withdrawal. A further nine participants did not complete any assessments, and an additional nine did not submit at least one major assessment piece. These participants were removed from further analysis, leaving 208 participants. The age range was 16–61 years (M = 21.53 years, SD = 7.23 years). One hundred and fifty-two were female (73.08%), 55 were male, and one indicated other. One hundred and fifty-two (73.08%) had not completed any semesters of university, with the range being 0–16 semesters (M = 0.88, SD = 2.04 semesters). Ten had attempted the course before.

Measures

Academic motivation

Academic motivation was measured by the Academic Motivation Scale (AMS) (Vallerand et al., 1992). The AMS measures academic motivation with three broad factors: amotivation (α = .86); extrinsic motivation (with facets of external regulation (α = .81), introjected regulation (α = .86), and identified regulation (α = .76)); and intrinsic motivation (with facets of motivation to know (α = .84), to accomplish (α = .83), and to experience stimulation (α = .87)). Each subscale contained four items, and was measured on a 7-point scale, ranging from 1 = Does not correspond at all, to 7 = Corresponds exactly. In order to calculate a composite measure of academic motivation, a self-determination index was calculated (Vallerand, 1997) from the subscales. Higher scores on the self-determination index indicate greater self-determined academic motivation.

Personality

Personality was measured using the 50-item version of the IPIP NEO-FFI (Goldberg, 1999). The scales measured extraversion, neuroticism, agreeableness, conscientiousness, and openness to experience. Items were rated on a 5-point scale ranging from 1 = Very inaccurate to 5 = Very accurate. One item for extraversion was not included due to a technical issue. The IPIP NEO-FFI has good internal consistency and concurrent validity (Gow, Whiteman, Pattie, & Deary, 2005) and has been used previously in studies investigating personality and academic performance (Richardson et al, 2012).

PASS attendance

PASS in this study was connected to an undergraduate introductory psychology course, offered in the first semester of the first year. Although the PASS were voluntary, they were timetabled, and students had to enroll for a specific PASS session when completing their enrollment for their classes. They were then informed that they did not have to attend and could attend different sessions to the one they had enrolled for, commonly referred to an opt-out rather than a traditional opt-in method. There were 10 regular PASS of one-hour duration, in which the PASS leader asked revision questions, directed group activities, discussions, and games, and provided worksheets and explained answers. However, the program at the current university also included three additional assessment-targeted sessions, including two practice exams (one before the midsemester exam and one before the final exam) and a session on the laboratory report assignment. The practice exams were written by PASS leaders and completed by students in a two-hour session in which they were exposed to “exam conditions.” PASS leaders then facilitated discussions around exams, studying effectively, and content, which the students found challenging. In the assignment session, the PASS leader revised the structure of the laboratory report, using a different example to the actual assignment, and facilitated discussion. The addition of these assessment-targeted sessions allowed us to also investigate whether attendance at these special sessions was a better predictor of academic performance than attendance at the regular sessions.

PASS attendance was recorded in each PASS session by the PASS leaders. The main PASS attendance variable was measured on a continuous scale of how many PASS students attended, ranging from 0 to 13. One hundred and seventy participants (81.73%) attended at least one PASS session (M = 4.87, SD = 3.76). Attendance at the assessment-targeted PASS sessions was a true dichotomy (i.e. attended versus did not). Forty participants attended the midsemester exam session, 32 participants attended the assignment session, and 35 attended the final exam session. When examining the impact of specific special sessions on academic performance compared with regular sessions, we counted PASS attendance (not including previous special sessions) up until the point of that assessment. Therefore, PASS attendance up until the midsemester session ranged from 0 to 4, for the assignment session from 0 to 6, and for the final exam session from 0 to 10.

Academic performance

There were three major assessment pieces: a midsemester exam (worth 20% of overall grade) a laboratory report (worth 25%); and a final exam (worth 45%). In addition, there were two minor assessment pieces: an in-class assignment preparatory exercise (5%); and completion of five credit points of research participation (5%). As the PASS content was not related to the in-class exercise or research participation and, therefore, there is limited reason to expect an association except due to extraneous variables, academic performance in this study was operationalized as the sum of the three major assessment pieces (a mark out of 90). Only those participants who attempted all assessments were included.

Prior academic achievement

In Australia, the primary method of university entrance is via prior academic achievement, which is converted to an entry rank. For those just leaving high school, this is a high school rank (currently called Overall Position in Queensland and the Australian Tertiary Admissions Rank (ATAR) in other states). For those who have completed some tertiary education, entrance is usually based on their university grade point average (GPA). There are other methods for those who may not have formal academic qualifications, including work experience, bridging courses, and an academic aptitude assessment called the Special Tertiary Admissions Test.

We asked students their entry method, and their relevant score if available. Twenty applied through ATAR, 119 through Overall Position, six through university studies (less than one year full time equivalent), 13 through university studies (more than eight subjects), six through the Special Tertiary Admissions Test, and 26 indicated other (e.g. work experience, overseas students, bridging courses). As ATAR is equivalent to a percentile score, we used it as our measure of academic performance, and converted Overall Position and prior university study to ATAR using official conversion tables. One participant who scored a low Overall Position score (23) was unable to be converted to an ATAR using the conversion table. For those who indicated they entered through less than one year of full-time university study, we converted their ATAR as though they were studying four subjects (one semester full time equivalent). Four of the participants’ scores, from those who indicated “other,” were also able to be converted to ATAR. We did not convert students’ scores using the Special Tertiary Admissions Test to rank entry because, unlike the other methods, it is a measure of academic aptitude rather than prior academic achievement, and so would contaminate the measure. In total, we were able to convert the scores of 161 participants. ATAR ranged from 56.55 to 99.85 (M = 75.67, SD = 11.27, possible range is ≤30 to 99.95).

Procedure

The survey was hosted on an online survey platform. Students enrolled on the target course were sent an email before the semester started, informing them of the study, and that participation was voluntary. All participants gave informed consent. The survey was available throughout the semester, although 110 (49%) participants completed the survey before PASS started. After the semester ended, survey data were compiled with course marks and PASS attendance.

Results

The IBM Statistical Package for Social Sciences (SPSS) Version 25 was used to analyze the data. All assumptions for multiple regression were met (i.e. normally distributed residuals, homoscedasticity, multicollinearity). One participant indicated that they had completed semesters of university before, but did not provide a number; therefore, this person was not included in analyses including this variable. Reliability for scales and correlations between major study variables is presented in Table 1. The correlations had .80 power to detect a ß of .19 or greater. Surprisingly, conscientiousness was not related to academic performance (p = .058, 95% CI [−.006, .26]). In order to assess whether PASS attendance affected academic performance after controlling for study variables, a stepwise multiple regression was conducted, with PASS attendance entered as the second step. Given the sample size (161, due to the smaller number of participants who had ATAR scores), the multiple regression could detect ß of .22 with .80 power. The results from the regression analyses can be found in Table 2. As can be seen, PASS attendance remains a marginally significant predictor (p = .046), although it explained only a small amount of the variance in academic performance.

Table 1.

Reliability and correlation matrix for major study variables.

	M	SD	α	1	2	3	4	5	6	7	8	9
1. Self-determination index	0.06	6.05		–
2. Openness	3.69	0.52	0.71	.26***	–
3. Neuroticism	2.90	0.78	0.86	–.18**	0.09	–
4. Conscientiousness	3.46	0.61	0.82	.26***	0.04	–.34***	–
5. Extraversion	3.21	0.66	0.83	.16*	0.1	–.36***	.23***	–
6. Agreeableness	3.74	0.56	0.79	.23***	0.13	–.22**	.33***	0.04	–
7. Academic performance	57.21	10.55		.14*	.17*	–.16*	0.13	0.07	0.02	–
8. PASS attendance	4.87	3.76		0.11	0.04	–.16*	0.13	0.07	–0.01	.27***	–
9. Australian Tertiary Admissions Rank^a	75.67	11.27		.18*	.20*	–.19*	0.15	.16*	–0.11	.57***	.20**	–
10. Week of semester when survey was taken	4.50	4.77		–.28***	–0.07	–0.11	–.17*	0	–0.06	–0	–0.1	–0.05

Note. ^an = 161.

p < .05. **p < .01. ***p < .001.

Table 2.

Regression coefficients for study variables in predicting academic performance.

	B	SE B	β	R² Δ
Step 1	15.86	10.86		.369
Self-determination index	−0.04	0.13	−0.02
Openness	1.13	1.42	0.06
Neuroticism	−0.71	1.03	−0.05
Conscientiousness	0.01	1.29	0.00
Extraversion	−0.36	1.15	−0.02
Agreeableness	0.83	1.38	0.04
Previous subject attempt	−8.90	3.36	−0.18**
Total semesters of university	0.27	0.39	0.05
Australian Tertiary Admissions Rank	0.49	0.07	0.53***
Step 2	14.89	10.76		.016*
Self-determination index	−0.04	0.12	−0.02
Openness	1.09	1.40	0.05
Neuroticism	−0.48	1.03	−0.04
Conscientiousness	−0.13	1.28	−0.01
Extraversion	−0.35	1.13	−0.02
Agreeableness	0.96	1.37	0.05
Previous subject attempt	−7.98	3.36	−0.16*
Total semesters of university	0.27	0.38	0.05
Australian Tertiary Admissions Rank	0.48	0.07	0.51***
PASS attendance	0.38	0.19	0.13*

Note. n = 161.

p < .05. **p < .01. ***p < .001.

We considered it a possibility that PASS attendance may increase academic motivation. In order to investigate this possibility, we conducted a post-hoc moderation analysis using time of semester, in which participants completed the survey as a moderating variable for the relationship between PASS attendance and the self-determination index. The survey completion date was recoded to week of semester, such that if participants completed the survey before PASS started, it would be recoded as week 1. If PASS attendance increased academic motivation, we would expect a stronger relationship between PASS attendance and motivation for those who completed the survey later in the semester (when there was the opportunity for PASS attendance to have had an effect on motivation), compared with early in the semester/before PASS started. The interaction term was not significant (p = .77).

To investigate who benefits from PASS attendance, we conducted mediation analyses for variables which had bivariate correlations with PASS attendance and academic performance (neuroticism and ATAR), with PASS attendance as the mediator. Mediation analyses were conducted in SPSS through PROCESS macros using bootstrapping with 10,000 samples (Hayes, 2013). In both cases, the completely standardized indirect effect was significant, as the 95% bootstrap confidence interval did not pass through zero (neuroticism, −0.04, 95% CI [−0.10, −0.01] and ATAR, 0.04, 95% Ci [0.01, 0.09]). Using Preacher and Kelley’s (2011) κ², as a measure of effect size, both had small-to-medium effect sizes (neuroticism, κ²= .04 95% CI [.01, .10], and ATAR, κ²= .05, 95% CI [.01, .11]). The results of these analyses are presented in Figures 1 and 2.

Figure 1.

Standardized regression coefficients for the relationship between neuroticism and academic performance, as mediated by PASS attendance. Standardized regression coefficient between neuroticism and academic performance while controlling for the mediator in parentheses.

Figure 2.

Standardized regression coefficients for the relationship between the ATAR and academic performance, as mediated by PASS attendance. Standardized regression coefficient between ATAR and academic performance while controlling for the mediator in parentheses.

Finally, in order to investigate whether the regular or assessment-targeted sessions of PASS had more effect on academic performance, we conducted four regression analyses—one for each assessment piece, and one for overall academic performance—while separating the assessment-targeted PASS from the regular sessions. The results are presented in Table 3 and suggest that attendance at the exam-targeted special sessions is a better predictor of exam performance, and attendance at the special sessions overall is a better predictor of overall performance than attendance at the regular PASS.

Table 3.

Regression coefficients for regular and special PASS in predicting individual assessment pieces, and the combination of major assessment pieces.

	Midsemester exam			Laboratory report			Final exam			Major assessments combined
	B	SE B	β	B	SE B	β	B	SE B	β	B	SE B	β
Regular sessions	0.13	0.12	0.08	0.26	0.25	0.08	0.32	0.14	0.16*	0.41	0.25	0.13
Special session/s	1.25	0.47	0.19**	0.53	1.47	0.03	3.68	1.25	0.22**	2.82	0.98	0.22**

Note. Special sessions were practice exams for the midsemester exam, and a questions session for the laboratory report. For each analysis of the individual assessment pieces, only the relevant special session was included. The regular sessions in each analysis included all regular (i.e. non-special) PASS sessions up to that point in the semester. For the major assessments combined, attendance at the special sessions was summed to form a variable in the range 0 to 3.

p < .05. **p < .01.

Discussion

There were two main aims of this research: To investigate whether PASS attendance was related to academic performance after controlling for relevant psychological and demographic variables; and to investigate whether PASS attendance mediated the relationship between variables of interest and academic performance. Because of the design of the PASS program offered, we were also able to investigate which aspects of PASS were beneficial.

With regard to our first aim, we found that PASS attendance was still a significant predictor of academic performance after controlling for motivation, personality, prior attempts at the subject, number of university semesters completed, and prior academic achievement. However, after controlling for these variables, PASS attendance explained only a small amount of the variance in academic performance, and the beta weight was half the size of the bivariate correlation. This indicates that although PASS attendance may predict academic performance, future research may benefit from more extensive control variables, and a more formal consideration of the implications of any effect sizes after controlling for relevant variables, something which has been inconsistent in previous research (Dawson et al., 2014).

We would caution against interpreting these results to suggest that motivation and personality are not worth controlling for when investigating PASS attendance. Although PASS attendance was still a significant predictor of academic performance after controlling for these variables, we did find significant correlations between personality and PASS attendance, whereas the lack of relationship between academic motivation and PASS attendance may have been due to lack of power to detect small effect sizes. We expect that these relationships are underestimated in our study (compared with other studies) due to our high PASS participation rates (most studies report 20% to 50% of the sample to have attended one or more PASS sessions (e.g. Ning & Downing, 2010; Okun et al., 2013)), and the fact that we eliminated participants who withdrew or did not submit assessments (to remove potential influences on PASS attendance and performance relationships that could confound the issue).

With regard to our second aim, we found that PASS attendance mediated the relationship between neuroticism and ATAR, and academic performance. Despite the high initial PASS attendance observed in this study, people who had a high ATAR and low neuroticism were the ones who could be considered as seeing the benefits of PASS attendance. Again, we would anticipate that these results would be stronger in other studies that did not have our high rates of PASS attendance. To a certain extent, finding that some people benefit from PASS more than others is likely unavoidable, as PASS is a voluntary program and, therefore, those who are not avoidant and have a higher history of academic success may be more likely to attend. However, it does highlight the necessity of investigating who benefits from PASS attendance, and of considering adapting outreach as appropriate.

Our PASS program had additional sessions devoted to practice exams and the assignment that are a common adjunct to programs (i.e. see van der Meer et al., 2017). As such, we were able to separate these aspects of the PASS program from the rest. Our results indicated that the practice exam sessions were more effective than the regular PASS sessions. Regular PASS attendance was not a significant predictor for the midsemester exam or final marks when controlling for attendance at the assessment-targeted sessions. Although the regular sessions were still a significant predictor of performance in the final exam when controlling for the special session, the lower magnitude of the regular sessions is particularly striking when it is considered that there were 10 regular sessions compared with one special practice exam session. Research has indicated that practice testing is one of the most effective learning strategies (Dunlosky, Rawson, Marsh, Nathan, & Willingham, 2013) and, as such, these results may not be overly surprising.

Limitations and future directions

We note a number of limitations to our study. First, we found it surprising that there was not a significant bivariate correlation between our measure of academic motivation and PASS attendance. There are two related explanations for this. As previously mentioned, we had a high level of PASS attendance compared with previous studies, likely due to PASS being opt-out, rather than opt-in. As PASS attendance was opt-out, this likely reduces the effect of academic motivation; that is, everyone is enrolled in PASS, not just those who are motivated to enroll. Second, given the likely reduced relationship between PASS attendance and academic motivation, our study may not have had enough power to detect this smaller relationship. On a related note, it is also surprising that conscientiousness did not have a significant bivariate correlation with academic performance, particularly considering it is the strongest personality predictor of tertiary academic performance (Poropat, 2009). This is, perhaps, more surprising, as our study had a sufficient sample size to detect a true score correlation of .19 with .80 power, which is close to meta-analytically derived estimates of uncorrected correlations between personality and academic performance (.18, Poropat, 2009). Regardless, a correlation of .18 is well within the 95% confidence interval of our non-sigificant correlation of .13, and the lack of significant finding may have simply been a matter of random sampling. Although interesting, this does not have an impact on the conculsions drawn.

Third, recruitment for the study continued throughout the semester, so it is possible that PASS attendance had an effect on academic motivation, rather than the other way round. However, if this was the case, we would expect a stronger relationship between PASS attendance and motivation later in the semester (when PASS attendance was able to have an effect), compared with earlier in the semester. A post-hoc moderation analysis revealed no moderation of PASS attendance and the self-determination index by the time the survey was taken, indicating that this is unlikely to be the case.

Fourth, although this study advanced previous research by controlling for academic motivation and personality, there are a number of further extraneous variables which may affect the PASS attendance–academic performance relationship. Research has indicated that PASS students attend more lectures (Parkinson, 2009), and successful PASS students (those who received an A or B compared with a D or F), attend more lectures, do more extra credit work, attend more help sessions, and attend more staff consults (Moore & LeDee, 2006). As such, the PASS attendance–academic performance relationship may be confounded with general academic engagement, help-seeking behaviors, and access to academic resources. Future research may benefit from controlling for these academic engagement behaviors.

Fifth, our measure of prior academic achievement was self-reported and, consequently, should be interpreted with a degree of caution. Despite this, we expect there would be little reason for the participants to intentionally enter misleading information on this measure and, consequently, we would not expect it to be distorted in any consistent manner which would affect the interpretation of the results.

Finally, we would not suggest that the results of this study indicate that PASS may be less effective than previously thought in all circumstances. It is important to note that this study investigated PASS only in one subject and, consequently, these results may not be applicable to other subjects or content areas. Furthermore, the focus of this research was on academic performance as an outcome of PASS attendance. It may be that PASS attendance has further positive, non-academic outcomes, such as increased student identity, peer relationships, and sense of belonging, which may have an impact on academic motivation and student retention.

Conclusion

This study was the first to investigate the relationship between PASS attendance and academic performance while controlling for academic motivation and personality. Although we still found a significant relationship between PASS attendance and academic performance, it decreased in magnitude after including control variables, suggesting that future research may benefit from including motivation and personality as control variables. Furthermore, mediation analyses suggested PASS was a significant mediator between individual characteristics (personality and prior academic achievement) and academic performance, suggesting that PASS may be benefiting students who are already academically engaged or capable. Finally, we found that special PASS involving practice testing were better predictors of performance in exams and overall marks than attendance at the regular PASS. Further research on the mechanisms of PASS efficacy may lead to a more effective and efficient intervention.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Daniel J. Cummings, PhD (organizational psychology), is a former teacher of undergraduate personality, statistics, and psychology at the Griffith University School of Applied Psychology in Brisbane, Queensland, Australia. He currently works in the private sector as a research psychologist and psychometrician. His research interests include the relationship between personality, emotions, cognitions, and important individual outcomes, and the proper implementation and evaluation of educational interventions, particularly those that utilize technology.

Nicola Sheeran, PhD (clinical psychology), is a lecturer at the Griffith University School of Applied Psychology in Brisbane, Queensland, Australia. She teaches first year, honors, and postgraduate psychology and counseling students and is interested in helping students successfully transition to university. Her research interests include the use of Facebook in university settings, interprofessional learning and education, helping people during transitions (i.e. to parenthood and to university), communication in health settings and between health professionals, and improving health services for minority groups.

References

Bowles

T. J.

Jones

(2003) An analysis of the effectiveness of supplemental instruction: The problem of selection bias and limited dependent variables. Journal of College Student Retention: Research, Theory & Practice 5(2): 235–243.

Bruno

P. A.

Love Green

J. K.

Illerbrun

S. L.

Holness

D. A.

Illerbrun

S. J.

Haus

K. A.

Sveinson

K. L.

(2016) Students helping students: Evaluating a pilot program of peer teaching for an undergraduate course in human anatomy. Anatomical Sciences Education 9(2): 132–142.

Cheng

Walters

(2009) Peer-assisted learning in mathematics: An observational study of student success. Journal of Peer Learning 2(3): 23–39.

Corker

K. S.

Oswald

F. L.

Donnellan

M. B.

(2012) Conscientiousness in the classroom: A process explanation. Journal of Personality 80(4): 995–1028.

Dawson

van der Meer

Skalicky

Cowley

(2014) On the effectiveness of supplemental instruction: A systematic review of supplemental instruction and peer-assisted study sessions literature between 2001 and 2010. Review of Educational Research 84(4): 609–639.

Dunlosky

Rawson

K. A.

Marsh

E. J.

Nathan

M. J.

Willingham

D. T.

(2013) Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest 14(1): 4–58.

Fayowski

MacMillan

P. D.

(2008) An evaluation of the supplemental instruction programme in a first year calculus course. International Journal of Mathematical Education in Science and Technology 39(7): 843–855.

Gattis

K. W.

(2002) Responding to self-selection bias in assessments of academic support programs: A motivational control study of supplemental instruction. Learning Assistance Review 7(2): 23–36.

Goldberg

L. R.

(1999) A broad-bandwidth, public-domain personality inventory measuring the lower-level facets of several five-factor models. In: Mervielde

Deary

De Fruyt

Ostendorf

(eds) Personality psychology in Europe, Tilburg, the Netherlands: Tilburg University Press, pp. 7–28.

10.

Gow

A. J.

Whiteman

M. C.

Pattie

Deary

I. J.

(2005) Goldberg’s “IPIP” big-five factor markers: Internal consistency and concurrent validation in Scotland. Personality and Individual Differences 39(2): 317–329.

11.

Hayes

A. F.

(2013) Introduction to mediation, moderation, and conditional process analysis: A regression-based approach, New York, NY: Guilford Press.

12.

Hodges

Dochen

C. W.

Joy

(2001) Increasing students’ success: When supplemental instruction becomes mandatory. Journal of College Reading and Learning 31(2): 143–156.

13.

Hodges

White

W. G.

(2001) Encouraging high-risk student participation in tutoring and supplemental instruction. Journal of Developmental Education 24(3): 2–10.

14.

Hughes

K. S.

(2011) Peer-assisted learning strategies in human anatomy and physiology. The American Biology Teacher 73(3): 144–147.

15.

Hurley, M., & Gilbert, M. (2008). Basic supplemental instruction model. In M. E. Stone & G. Jacobs (Eds.), (3rd ed., pp. 1–10). Columbia, SC: University of South Carolina, National Resource Center for the First-Year Experience and Students in Transition.

16.

Jones

J. P.

(2013) The impact of the supplemental instruction leader on student performance in introductory accounting. American Journal of Business Education 6(2): 247–254.

17.

Komarraju

Karau

S. J.

(2005) The relationship between the big five personality traits and academic motivation. Personality and Individual Differences 39(3): 557–567.

18.

MacCallum

R. C.

Zhang

Preacher

K. J.

Rucker

D. D.

(2002) On the practice of dichotomization of quantitative variables. Psychological Methods 7(1): 19–40.

19.

McCarthy

Smuts

Cosser

(1997) Assessing the effectiveness of supplemental instruction: A critique and a case study. Studies in Higher Education 22(2): 221–231.

20.

Malm

Bryngfors

Mörner

L.-L.

(2011) Supplemental instruction: Whom does it serve? International Journal of Teaching and Learning in Higher Education 23(3): 282–291.

21.

Malm

Bryngfors

Mörner

L.-L.

(2015) The potential of supplemental instruction in engineering education—helping new students to adjust to and succeed in university studies. European Journal of Engineering Education 40(4): 347–365.

22.

Martin

(2008) Foreword. Journal of Peer Learning 1(1): 3–5.

23.

Matin

D. C.

Arendale

D. R.

(1993) Supplemental instruction: Improving first-year student success in high-risk courses. 2nd ed, Colombia, SC: University of South Carolina, Center for the Study of the Freshman Year Experience.

24.

Moore

LeDee

(2006) Supplemental instruction and the performance of developmental education students in an introductory biology course. Journal of College Reading and Learning 36(2): 9–20.

25.

Ning

H. K.

Downing

(2010) The impact of supplemental instruction on learning competence and academic performance. Studies in Higher Education 35(8): 921–939.

26.

Oja

(2012) Supplemental instruction improves grades but not persistence. College Student Journal 46(2): 344–349.

27.

Okun

M. A.

Berlin

Hanrahan

Lewis

Johnson

(2013) Reducing the grade disparities between American Indians and Euro-American students in introduction to psychology through small-group, peer-mentored supplemental instruction. Educational Psychology 35(2): 176–191.

28.

Paloyo

A. R.

(2015) A note on evaluating supplemental instruction. Journal of Peer Learning 8(1): 1–4.

29.

Parkinson

(2009) The effect of peer-assisted learning support (PALS) on performance in mathematics and chemistry. Innovations in Education and Teaching International 46(4): 381–392.

30.

Poropat

A. E.

(2009) A meta-analysis of the five-factor model of personality and academic performance. Psychological Bulletin 135(2): 322–338.

31.

Poropat

A. E.

(2014) Other-rated personality and academic performance: Evidence and implications. Learning and Individual Differences 34: 24–32.

32.

Poropat, A. E. (2016). Beyond the shadow: The role of personality and temperament in learning. In L. Corno & E. M. Anderman (Eds.), Handbook of educational psychology (3rd ed., pp. 172–186). New York, NY: Routledge.

33.

Preacher

K. J.

Kelley

(2011) Effect size measures for mediation models: Quantitative strategies for communicating indirect effects. Psychological Methods 16(2): 93–115.

34.

Richardson

Abraham

Bond

(2012) Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin 138(2): 353–387.

35.

Terrion

J. L.

Daoust

J.-L.

(2011) Assessing the impact of supplemental instruction on the retention of undergraduate students after controlling for motivation. Journal of College Student Retention: Research, Theory and Practice 13(3): 311–327.

36.

van der Meer, J., Wass, R., Scott, S., & Kokaua, J. (2017). Entry characteristics and participation in a peer learning program as predictors of first-year students’ achievement, retention, and degree completion. AERA Open, 3(3), 1–13.

37.

Vallerand

R. J.

(1997) Toward a hierarchical model of intrinsic and extrinsic motivation. In: Zanna

M. P.

(ed) Advances in experimental social psychology Vol. 29 San Diego, CA: Academic Press, pp. 271–360.

38.

Vallerand

R. J.

Pelletier

L. G.

Blais

M. R.

Briere

N. M.

Senecal

Vallieres

E. F.

(1992) The Academic Motivation Scale: A measure of intrinsic, extrinsic, and amotivation in education. Educational and Psychological Measurement 52(4): 1003–1017.

Do academic motivation and personality influence which students benefit the most from peer-assisted study sessions?