Abstract
Background
Limited research exists on the prevalence of undergraduate research participant pools (URPs) in psychology departments or guidelines surrounding their use.
Objective
The aim of the current study was to replicate older studies on the prevalence of URPs in the United States and the procedures and guidelines related to those requirements.
Method
Psychology department chairs in the U.S. were recruited by email and via the Association of Heads of Departments of Psychology listserv. A total of 604 department chairs participated in the online study by completing a series of assessments related to URP use in their department.
Results
Most 4-year institutions (67.7%), but few 2-year institutions (6.3%) had a URP. Of the 4-year institutions with a URP, 83.7% had a research requirement for students enrolled in their introduction to psychology courses. Most departments (84.0%) did not assess the educational value of research participation.
Conclusion
Most psychology departments have a URP, but few examined the educational value of research participation.
Teaching Implications
Psychology departments should identify learning objectives for research participation, assess outcomes, and identify practices that optimize achieving those objectives. Research requirements should be consistent with ethical principles and empirical evidence of learning.
Keywords
Introduction
Many psychology departments rely on undergraduate research pools of students (URPs) enrolled in psychology courses for research conducted by faculty and students. However, we found no recently published studies examining the prevalence and nature of such URPs in the United States. Miller (1981) conducted a survey of the 100 most cited universities in the United States, Canada, and the United Kingdom. Of 76 responses, 70 had a URP. In 1989, Sieber and Saks found that 74% of graduate-degree-granting psychology departments in the U.S. had a URP. They also found that 93.4% of departments recruited students from introduction to psychology courses. Landrum and Chastain (1999) found that 32.7% of undergraduate-only departments had URPs, with public institutions reporting a higher incidence than private institutions. The most common approach was volunteer-based URPs (34.7% of departments), with 24.8% offering extra credit for participation.
There have long been concerns regarding ethicality of URPs, questioning the educational value for participating students and the use of debriefing and assessment procedures (e.g., Jung, 1969; Miller, 1981; Sieber, 1999). Sales & Folkman (2000) argue there must be an educational benefit to undergraduate students required to participate in research, but Zannella et al. (2020) found that 44% of students reported learning nothing new about the research process by participating in research. While 87% of undergraduates report that participating in research studies is worthwhile (Miller & Kreiner, 2008), it is unclear whether self-reported learning correlates to educators’ goals and expectations for the experience. Zannella et al. (2020) also gave students a quiz assessing knowledge of the research process and most reported learning the information in class lecture rather than through research participation. Similarly, Darling et al. (2007) found that only 22.1% of students reported better understanding how to conduct an experiment, while 18.3% reported learning nothing. Overall, there is little data regarding the prevalence and manner of assessment of the educational value of research participation.
Current Study
The purpose of the current study was to replicate and extend previous studies on the prevalence and practices related to URPs in the United States. We examined whether certain university characteristics (e.g., public vs. private) related to likelihood of having a URP. We also examined the prevalence of applied research requirements for students in introduction to psychology courses and practices related to such requirements. We investigated the prevalence of offering extra credit for research participation, debriefing requirements, and whether and how departments assessed the educational outcomes of research participation.
Method
Participants
The participants were 604 psychology department chairs in the United States, of whom 86.8% (n = 524) reported employment at 4-year institutions and 13.2% (n = 80) at 2-year institutions. See Supplementary Table A in the Open Science Framework (OSF; Flynn & Rocheleau, 2021) for geographic location. Among 2-year institutions, 97.5% (n = 78) were public and 2.5% were private (n = 2). Among 4-year institutions, 42% (n = 220) were public and 58% (n = 304) were private. Only one 2-year institution had a psychology graduate program; of 4-year institutions, 42.4% (n = 144) of departments had graduate programs (see Supplementary Table B in OSF, Flynn & Rocheleau, 2021, for the prevalence of different types of graduate programs).
Sampling
The local Institutional Review Board (IRB) approved this study, and our department chair emailed the Association of Heads of Departments of Psychology through listserv requesting their participation. The first author also sent initial and follow-up emails to 1718 psychology department chairs at higher education institutions across the country, identified through Wikipedia, requesting participation. An exact response rate cannot be calculated because we are unable to separate responses recruited from the listserv versus email.
Materials and Procedure
The entire survey is available on OSF (Flynn & Rocheleau, 2021). After providing informed consent, psychology department chairs answered items regarding their institution type (2-year vs. 4-year; public vs. private), regional location (based on the U.S. Census Bureau’s (2010) designations), existence of a graduate program, and existence and size of a URP. If participants reported no department URP, the survey ended. Those with URPs proceeded to the remaining items.
Next, an item assessed whether the department’s introduction to psychology course required research participation. If so, a multiple-answer item assessed what options were available to students to fulfill the research requirement (i.e., participate in studies, read articles and do writing assignments about them, attend lectures, participate in virtual/simulated studies, and other). If participants selected “other,” they were asked to specify the other option(s). An open-ended item assessed how the research requirement factored into course grades, and participants were asked the number of hours required to fulfill the requirement. An item assessed whether students enrolled in other departmental classes were allowed to participate in research studies for extra credit. Again, respondents were invited to provide additional information.
Respondents were asked whether researchers had departmental debriefing requirements beyond IRB regulations, and if so, to describe those requirements. An item asked whether their department assessed the educational value of student research participation and, if so, to describe the assessment process.
Results
The raw data can be found on OSF (Flynn & Rocheleau, 2021).
Psychology Programs and Undergraduate Research Pools
Only five respondents from 2-year institutions (6.3%) reported having URPs. Due to prohibitively small sample size, all subsequent analyses use data exclusively from 4-year institutions. Throughout, unless otherwise specified, “respondents” refers to department chairs from 4-year institutions who reported having a URP.
URPs were reported in 67.7% (n = 355) of departments. See Supplementary Table C in OSF (Flynn & Rocheleau, 2021) for frequency data regarding the size of the URPs from these institutions. Having a URP did not differ based on geographic location, X2 (3, N = 523) = 6.93, p = .07 (69.8% (n = 88) in the Northeast, 67.6% (n = 96) in the Midwest, 61.96% (n = 112) in the South, 78.4% (n = 58) in the West). Results of a chi-square test of independence showed that public universities had more psychology departments with a URP (80.9%; (n = 178) than did private universities (58.2%; (n = 177), X 2 (1, N = 524) = 30.06, p < .001, Cramer’s V = .24. More respondents from graduate-degree-granting departments reported having a URP (80.2%, n = 178), as compared to respondents from undergraduate-only departments (58.8%; (n = 177), X 2 (1, N = 523) = 26.78, p < .001, Cramer’s V = .23.
Introduction to Psychology Courses and Research Requirements
Approximately 83.7% 1 (n = 288) of respondents indicated that introduction to psychology classes had a research requirement. The proportion of departments with a research requirement did not differ by geographic location, X2(3, N = 343) = 2.98, p = .40 (82.8% (n = 72) in the Northeast, 79.1% (n = 72) in the Midwest, 88.1% (n = 96) in the South, 83.9% (n = 47) in the West). The prevalence of a research requirement was similar for public (85.4%; n = 146) and private (82.1%; n = 142) institutions, X 2 (1, N = 344) = .69, p = .41. The existence of a graduate program in the department was unrelated to an introduction to psychology research requirement; approximately 87.2% (n = 150) of respondents from graduate-granting departments and 80.2% (n = 138) of respondents from undergraduate-only departments reported a research requirement, X 2 (1, N = 344) = 3.07, p = .08.
Research requirement options for students enrolled in an introduction to psychology course at 4-year institutions.
Note. All 288 participants from 4-year institutions with an undergraduate pool and research requirements for students enrolled in their introduction to psychology courses responded to this item.
Other options to complete introduction to psychology research requirement at 4-year institutions.
Note. Approximately 33 out of 34 participants from 4-year institutions with an undergraduate pool and research requirements for students enrolled in their introduction to psychology courses, who selected “other” for a multiple answer question regarding options students have to choose from to complete their research requirement, responded to this item.
Participants were also asked how the requirement is factored into students’ grades. The most common responses were “5% of the total grade for the course” (n = 56) and “10% of the total grade” (n = 33). Several respondents (n = 37) reported that completing the research requirement is required to pass/earn a grade in the course, and/or that failure to complete the requirement results in earning an incomplete. Supplementary Table D in OSF (Flynn & Rocheleau, 2021) illustrates the wide variability in responses to this item. Responses also varied widely on an item asking how many research hours are required of students enrolled in introduction to psychology courses (see Supplementary Table E in OSF; Flynn & Rocheleau, 2021). The highest frequency responses were 3 hours (n = 45), 2 hours (n = 34), 5 hours (n = 32), and 6 hours (n = 25). Approximately 92.18% (n = 224) of respondents providing a specific number of hours reported a requirement of 6 hours or less.
Extra Credit
Approximately 77.4% 2 (n = 263) of respondents reported that students enrolled in other psychology classes were permitted to participate in research studies for extra credit. Allowing extra credit for research participation did not significantly differ based on geographic location, X 2 (3, N = 339) = 8.02, p = .05, Cramer’s V = .15 (69.8% (n = 60) in the Northeast, 82.0% (n = 73) in the Midwest, 83.3% (n = 90 in the South, 69.6% (n = 39) in the West). Public universities (84.0%; n = 142) were more likely than private institutions (70.8%; n = 121) to permit students in psychology classes other than introduction to psychology to earn extra credit for participating in research studies, X 2 (1, N = 340) = 8.54, p < .01, Cramer’s V = .16. Similarly, graduate-degree-granting psychology departments (83.4%; n = 141) were more likely to allow students to earn extra credit for participating in research studies than undergraduate-only departments (71.3%; n = 122)), X 2 (1, N = 340) = 7.09, p < .01, Cramer’s V = .14.
Debriefing
Debriefing requirements for researchers running studies in the department, beyond those required for IRB approval, were reported by 37.8% 3 (n = 128) of respondents. Approximately 85.16% (n = 109) of participants with departmental debriefing requirements responded to an open-ended item describing these requirements (see Supplementary Table F in OSF (Flynn & Rocheleau, 2021) for all responses and frequencies 4 ). The highest frequency responses were “a debriefing” (with no further specifications listed; n = 8), “students must be provided a relevant reference” (n = 8), and “students must be told the purpose of the study” (n = 8).
Educational Value
Participant responses regarding how their department assesses the educational value of research participation.
Note. Approximately 44 of 54 of the participants from 4-year institutions with an undergraduate research pool, who stated their department assesses the educational value of research participation responded to this item.
Discussion
The purpose of the current study was to replicate and extend dated studies on the prevalence and practices of URPs in the U.S. Approximately 6.3% of 2-year and 67.7% of 4-year institutions had a URP in the psychology department. Regarding 4-year institutions, and consistent with Landrum and Chastain (1999), more public institutions had departments with a URP than private institutions. Of the psychology departments at 4-year institutions with a URP, 83.7% had a research requirement for students in their introduction to psychology courses, representing a substantial increase over the results reported by Landrum and Chastain (1999), when a plurality of programs relied on volunteer-based pools. Additionally, replicating Sieber and Saks (1989), more graduate–degree-granting departments had a research requirement than undergraduate-only programs. Specifically, 80.2% of psychology departments with a graduate program had URPs, slightly higher than Sieber and Saks observed in their sample (74%).
While we cannot be certain why use of URPs increased during this period, their growth has paralleled the trend to weight scholarship more heavily in faculty evaluation processes for promotion and tenure. Schimanski and Alperin (2018) observed this trend across all types of higher education institutions, but it is most pronounced in research-oriented institutions. This is consistent with the current findings. URPs were more common in departments with graduate programs and in 4-year institutions, which tend to be more research-oriented than their comparators (cf. Schimanski & Alperin, 2018). Furthermore, research experience is increasingly critical to psychology graduate school admissions (Nguyen Littleford et al., 2018). These expectations and evaluation processes exert pressure on faculty members and the students they mentor, to produce research. This, in turn, increases demand for research participants, which has likely contributed to the growth of URPs. Departments do not generally attribute the development of URPs to this motivation, instead justifying URPs by arguing that they provide educational value to the participants. However, given that only 16% of departments assessed the educational value of research participation, it is largely unknown whether this is warranted. Without empirical support that participation in a URP has educational merit, the case for their ethical use is undermined.
Also of ethical concern is whether student participants are genuinely capable of voluntary consent and withdrawal of consent, or whether they are unduly coerced to participate. While not recognized as a vulnerable population, college students 7 in a URP may be subject to many of the same dynamics that resulted in these populations being conferred special protections. For example, URP participants may perceive limited autonomy when faced with dual relationships with a faculty member posing as both a researcher and an instructor with control over the student’s course grade, particularly when the research requirement represents a significant portion of that grade and/or there are punitive consequences for choosing not to participate (e.g., inability to pass the course). This may be particularly true when any alternative assignment is more onerous, less engaging, and/or more time-consuming than research participation. These concerns are exacerbated when the student is minimally debriefed and when the degree of educational value derived from their participation is unclear. URP participants’ perceptions of autonomy, pressure to participate, and freedom to withdraw are important areas for empirical investigation.
There is some emerging evidence that participation may result in self-reported educational benefits for some students (Zannella et al., 2020). However, psychology researchers should bring their skills to bear on assumptions (e.g., research participation has educational value) that have been taken for granted or used as justifications for URPs. These require the same rigorous testing as any other pedagogical technique. Our call is to focus attention, administratively and empirically, on making participating in research a better personal and learning experience for students.
Based on the current data, we make the following recommendations for further inquiry, to guide best practices in implementing URPs. 1. Psychology departments should identify student learning objectives for research participation (and alternative assignments), communicate these in course syllabi, and assess these. 2. Learning resulting from research participation is largely untested. A few studies assessed student perceptions of the educational nature of research participation (e.g., Cromer et al., 2013; Darling et al., 2007; VanWormer et al., 2014). Future research should evaluate the efficacy of research participation in meeting identified learning objectives and strategies to optimize the educational value of research participation, using objective measures and sound research designs aligned with the scholarship of teaching and learning. Strategies might include having researchers do short presentations about their studies in psychology classes, using debriefings as an educational tool, and creating exams for students based on their research participation (cf. Sharpe & Poets, 2017). 3. Departments should be able to logically defend the research hours required (Sharpe & Poets, 2017). No published research has evaluated the impact of the number of required research hours on learning outcomes. Unless and until supported by empirical data, we recommend setting a conservative number of required hours. 4. How research participation factors into students’ overall grade for the course needs to be appropriate and justifiable. When research participation is weighted too heavily, it raises ethical concerns regarding coercion and respect for persons. We recommend against tying research participation to a student’s ability to pass the course or receiving an incomplete.
URPs are ubiquitous and serve as a primary source of data in psychological research; however, few, if any, best practices exist to guide departmental policies regarding expectations of student hours required, grading, debriefing, and assessment. Departments with URPs should heed the decades of calls to action in the psychology scholarly literature and implement evidence-supported practices that effectively promote student learning while secondarily supporting broad-based research agendas.
Footnotes
Author Note
Maureen K. Flynn, Department of Psychological Sciences, Metropolitan State University of Denver; Courtney Rocheleau, Metropolitan State University of Denver
Acknowledgments
The authors acknowledge and thank Anne Rocheleau for assistance with the preparation of this manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
