Undergraduate Research Pools in Psychology Departments: Prevalence and Practices

Abstract

Background

Limited research exists on the prevalence of undergraduate research participant pools (URPs) in psychology departments or guidelines surrounding their use.

Objective

The aim of the current study was to replicate older studies on the prevalence of URPs in the United States and the procedures and guidelines related to those requirements.

Method

Psychology department chairs in the U.S. were recruited by email and via the Association of Heads of Departments of Psychology listserv. A total of 604 department chairs participated in the online study by completing a series of assessments related to URP use in their department.

Results

Most 4-year institutions (67.7%), but few 2-year institutions (6.3%) had a URP. Of the 4-year institutions with a URP, 83.7% had a research requirement for students enrolled in their introduction to psychology courses. Most departments (84.0%) did not assess the educational value of research participation.

Conclusion

Most psychology departments have a URP, but few examined the educational value of research participation.

Teaching Implications

Psychology departments should identify learning objectives for research participation, assess outcomes, and identify practices that optimize achieving those objectives. Research requirements should be consistent with ethical principles and empirical evidence of learning.

Introduction

Many psychology departments rely on undergraduate research pools of students (URPs) enrolled in psychology courses for research conducted by faculty and students. However, we found no recently published studies examining the prevalence and nature of such URPs in the United States. Miller (1981) conducted a survey of the 100 most cited universities in the United States, Canada, and the United Kingdom. Of 76 responses, 70 had a URP. In 1989, Sieber and Saks found that 74% of graduate-degree-granting psychology departments in the U.S. had a URP. They also found that 93.4% of departments recruited students from introduction to psychology courses. Landrum and Chastain (1999) found that 32.7% of undergraduate-only departments had URPs, with public institutions reporting a higher incidence than private institutions. The most common approach was volunteer-based URPs (34.7% of departments), with 24.8% offering extra credit for participation.

There have long been concerns regarding ethicality of URPs, questioning the educational value for participating students and the use of debriefing and assessment procedures (e.g., Jung, 1969; Miller, 1981; Sieber, 1999). Sales & Folkman (2000) argue there must be an educational benefit to undergraduate students required to participate in research, but Zannella et al. (2020) found that 44% of students reported learning nothing new about the research process by participating in research. While 87% of undergraduates report that participating in research studies is worthwhile (Miller & Kreiner, 2008), it is unclear whether self-reported learning correlates to educators’ goals and expectations for the experience. Zannella et al. (2020) also gave students a quiz assessing knowledge of the research process and most reported learning the information in class lecture rather than through research participation. Similarly, Darling et al. (2007) found that only 22.1% of students reported better understanding how to conduct an experiment, while 18.3% reported learning nothing. Overall, there is little data regarding the prevalence and manner of assessment of the educational value of research participation.

Current Study

The purpose of the current study was to replicate and extend previous studies on the prevalence and practices related to URPs in the United States. We examined whether certain university characteristics (e.g., public vs. private) related to likelihood of having a URP. We also examined the prevalence of applied research requirements for students in introduction to psychology courses and practices related to such requirements. We investigated the prevalence of offering extra credit for research participation, debriefing requirements, and whether and how departments assessed the educational outcomes of research participation.

Method

Participants

The participants were 604 psychology department chairs in the United States, of whom 86.8% (n = 524) reported employment at 4-year institutions and 13.2% (n = 80) at 2-year institutions. See Supplementary Table A in the Open Science Framework (OSF; Flynn & Rocheleau, 2021) for geographic location. Among 2-year institutions, 97.5% (n = 78) were public and 2.5% were private (n = 2). Among 4-year institutions, 42% (n = 220) were public and 58% (n = 304) were private. Only one 2-year institution had a psychology graduate program; of 4-year institutions, 42.4% (n = 144) of departments had graduate programs (see Supplementary Table B in OSF, Flynn & Rocheleau, 2021, for the prevalence of different types of graduate programs).

Sampling

The local Institutional Review Board (IRB) approved this study, and our department chair emailed the Association of Heads of Departments of Psychology through listserv requesting their participation. The first author also sent initial and follow-up emails to 1718 psychology department chairs at higher education institutions across the country, identified through Wikipedia, requesting participation. An exact response rate cannot be calculated because we are unable to separate responses recruited from the listserv versus email.

Materials and Procedure

The entire survey is available on OSF (Flynn & Rocheleau, 2021). After providing informed consent, psychology department chairs answered items regarding their institution type (2-year vs. 4-year; public vs. private), regional location (based on the U.S. Census Bureau’s (2010) designations), existence of a graduate program, and existence and size of a URP. If participants reported no department URP, the survey ended. Those with URPs proceeded to the remaining items.

Next, an item assessed whether the department’s introduction to psychology course required research participation. If so, a multiple-answer item assessed what options were available to students to fulfill the research requirement (i.e., participate in studies, read articles and do writing assignments about them, attend lectures, participate in virtual/simulated studies, and other). If participants selected “other,” they were asked to specify the other option(s). An open-ended item assessed how the research requirement factored into course grades, and participants were asked the number of hours required to fulfill the requirement. An item assessed whether students enrolled in other departmental classes were allowed to participate in research studies for extra credit. Again, respondents were invited to provide additional information.

Respondents were asked whether researchers had departmental debriefing requirements beyond IRB regulations, and if so, to describe those requirements. An item asked whether their department assessed the educational value of student research participation and, if so, to describe the assessment process.

Results

The raw data can be found on OSF (Flynn & Rocheleau, 2021).

Psychology Programs and Undergraduate Research Pools

Only five respondents from 2-year institutions (6.3%) reported having URPs. Due to prohibitively small sample size, all subsequent analyses use data exclusively from 4-year institutions. Throughout, unless otherwise specified, “respondents” refers to department chairs from 4-year institutions who reported having a URP.

URPs were reported in 67.7% (n = 355) of departments. See Supplementary Table C in OSF (Flynn & Rocheleau, 2021) for frequency data regarding the size of the URPs from these institutions. Having a URP did not differ based on geographic location, X² (3, N = 523) = 6.93, p = .07 (69.8% (n = 88) in the Northeast, 67.6% (n = 96) in the Midwest, 61.96% (n = 112) in the South, 78.4% (n = 58) in the West). Results of a chi-square test of independence showed that public universities had more psychology departments with a URP (80.9%; (n = 178) than did private universities (58.2%; (n = 177), X ² (1, N = 524) = 30.06, p < .001, Cramer’s V = .24. More respondents from graduate-degree-granting departments reported having a URP (80.2%, n = 178), as compared to respondents from undergraduate-only departments (58.8%; (n = 177), X ² (1, N = 523) = 26.78, p < .001, Cramer’s V = .23.

Introduction to Psychology Courses and Research Requirements

Approximately 83.7% ¹ (n = 288) of respondents indicated that introduction to psychology classes had a research requirement. The proportion of departments with a research requirement did not differ by geographic location, X²(3, N = 343) = 2.98, p = .40 (82.8% (n = 72) in the Northeast, 79.1% (n = 72) in the Midwest, 88.1% (n = 96) in the South, 83.9% (n = 47) in the West). The prevalence of a research requirement was similar for public (85.4%; n = 146) and private (82.1%; n = 142) institutions, X ² (1, N = 344) = .69, p = .41. The existence of a graduate program in the department was unrelated to an introduction to psychology research requirement; approximately 87.2% (n = 150) of respondents from graduate-granting departments and 80.2% (n = 138) of respondents from undergraduate-only departments reported a research requirement, X ² (1, N = 344) = 3.07, p = .08.

Participants reporting introduction to psychology research requirements responded to a multiple-answer item concerning options available to students to fulfill the requirement. The most frequent responses were “participating in research studies’ (n = 276) and ‘read articles and do writing assignment about them” (n = 260). See Table 1 for the frequency data for all options. Participants were invited to list additional alternatives provided to students (see Table 2). The most common responses were “instructors can choose additional options” (n = 11), “complete an online ethics training” (n = 6), and “attend and write a summary about the department or university-sponsored research symposium/poster session” (n = 6). Respondents presented other options, such as attending psychological or research events, writing papers, and engaging in experiential learning opportunities.

Table 1.

Research requirement options for students enrolled in an introduction to psychology course at 4-year institutions.

Options	N (%)
Participate in research studies conducted in the psychology department	276 (95.8%)
Read articles and do writing assignments about them	260 (90.3%)
Attend lectures	46 (15.97%)
Participate in virtual/simulated studies	39 (13.54%)
Other	34 (11.81%)
Not applicable (do not have a research requirement)	1 (0.003%)

Note. All 288 participants from 4-year institutions with an undergraduate pool and research requirements for students enrolled in their introduction to psychology courses responded to this item.

Table 2.

Other options to complete introduction to psychology research requirement at 4-year institutions.

Participant Responses	n (%)
Instructors can choose additional options	11 (33.33%)
Online ethics training	6 (18.18%)
Attend and write a summary about the department or university-sponsored research symposium/poster session	6 (18.18%)
Watch approved video and write a summary about it or take a quiz on the video	3 (9.10%)
Attend training sessions organized by the counseling center	1 (3.03%)
Participate as a practice client for graduate students learning psychological assessment	1 (3.03%)
Attend an educational activity that provides information about how research is conducted	1 (3.03%)
Summarize a chapter in a textbook not covered in class	1 (3.03%)
Serve as a “client” in a mock interview	1 (3.03%)
Attend psychology club meetings	1 (3.03%)
Interview a senior completing an honor’s thesis	1 (3.03%)
Attend a talk on campus that has a psychological component and write a paper summary	1 (3.03%)
Attend community or university events	1 (3.03%)
Volunteer	1 (3.03%)
Attend a self-care event (e.g., yoga, meditation)	1 (3.03%)
Write a short paper on a psychological topic of their choice	1 (3.03%)
Write a paper	1 (3.03%)
Attend lectures on how to learn and do well in classes	1 (3.03%)
Join a psychology learning community	1 (3.03%)
Go to an advising meeting	1 (3.03%)
Conduct a research project in small teams	1 (3.03%)

Note. Approximately 33 out of 34 participants from 4-year institutions with an undergraduate pool and research requirements for students enrolled in their introduction to psychology courses, who selected “other” for a multiple answer question regarding options students have to choose from to complete their research requirement, responded to this item.

Participants were also asked how the requirement is factored into students’ grades. The most common responses were “5% of the total grade for the course” (n = 56) and “10% of the total grade” (n = 33). Several respondents (n = 37) reported that completing the research requirement is required to pass/earn a grade in the course, and/or that failure to complete the requirement results in earning an incomplete. Supplementary Table D in OSF (Flynn & Rocheleau, 2021) illustrates the wide variability in responses to this item. Responses also varied widely on an item asking how many research hours are required of students enrolled in introduction to psychology courses (see Supplementary Table E in OSF; Flynn & Rocheleau, 2021). The highest frequency responses were 3 hours (n = 45), 2 hours (n = 34), 5 hours (n = 32), and 6 hours (n = 25). Approximately 92.18% (n = 224) of respondents providing a specific number of hours reported a requirement of 6 hours or less.

Extra Credit

Approximately 77.4% ² (n = 263) of respondents reported that students enrolled in other psychology classes were permitted to participate in research studies for extra credit. Allowing extra credit for research participation did not significantly differ based on geographic location, X ² (3, N = 339) = 8.02, p = .05, Cramer’s V = .15 (69.8% (n = 60) in the Northeast, 82.0% (n = 73) in the Midwest, 83.3% (n = 90 in the South, 69.6% (n = 39) in the West). Public universities (84.0%; n = 142) were more likely than private institutions (70.8%; n = 121) to permit students in psychology classes other than introduction to psychology to earn extra credit for participating in research studies, X ² (1, N = 340) = 8.54, p < .01, Cramer’s V = .16. Similarly, graduate-degree-granting psychology departments (83.4%; n = 141) were more likely to allow students to earn extra credit for participating in research studies than undergraduate-only departments (71.3%; n = 122)), X ² (1, N = 340) = 7.09, p < .01, Cramer’s V = .14.

Debriefing

Debriefing requirements for researchers running studies in the department, beyond those required for IRB approval, were reported by 37.8% ³ (n = 128) of respondents. Approximately 85.16% (n = 109) of participants with departmental debriefing requirements responded to an open-ended item describing these requirements (see Supplementary Table F in OSF (Flynn & Rocheleau, 2021) for all responses and frequencies ⁴ ). The highest frequency responses were “a debriefing” (with no further specifications listed; n = 8), “students must be provided a relevant reference” (n = 8), and “students must be told the purpose of the study” (n = 8).

Educational Value

Departmental assessment of the educational value of research participation was reported by 16.0% ⁵ (n = 54) of respondents. Of these, 81.48% (n = 44) responded to an open-ended item about how the educational value of research participation was assessed (see Table 3 for responses and frequencies ⁶ ). The two responses with the highest frequency were “student surveys at the end of the semester” (n = 5) and “as part of the program assessment activities” (n = 4). Other responses were provided, such as student surveys/interviews at varying timepoints, faculty reviewing studies, and discussion.

Table 3.

Participant responses regarding how their department assesses the educational value of research participation.

Participant Responses	n (%)
Student surveys at the end of the semester	4 (9.10%)
As part of the program assessment activities	4 (9.10%)
Surveys	3 (6.12%)
Course evaluations	2 (4.55%)
Survey given to students right after they complete the experience	2 (4.55%)
Discussed in class or written in handouts	2 (4.55%)
Department meeting discussion	2 (4.55%)
Via additional assignment tied to the participation (what did you learn, etc.)	1 (2.27%)
Participants are required to write a very brief statement about the studies in which they participate	1 (2.27%)
Students participate in a discussion board at the end of each term where they explain what they have learned about the research process, how it might differ from what they expected, and then their responses are aggregated for program assessment purposes	1 (2.27%)
We use studies that are recently completed as examples in class. This connects their participation to the content of their courses	1 (2.27%)
We administer a survey asking students about their experiences with undergraduate research	1 (2.27%)
Participants are asked to give written feedback on participation	1 (2.27%)
Exit interviews with graduating seniors	1 (2.27%)
Instructors conduct qualitative and quantitative evaluations of the course	1 (2.27%)
With a short quiz (in the past)	1 (2.27%)
Knowledge base on ACAT assessment test given annually	1 (2.27%)
Optional extra questions on the course evaluations asking students to comment on their experiences of research participation	1 (2.27%)
Senior survey	1 (2.27%)
Student interviews	1 (2.27%)
Use a 30-question assessment that addresses research concepts and all general psychology students must take it. Department looks at performance rates once a year	1 (2.27%)
Feedback from students	1 (2.27%)
Alumni survey	1 (2.27%)
Informally they noticed that a good number of the students who get involved in research later in their undergraduate careers had participated and asked questions about research when they participated in our studies	1 (2.27%)
Research coordinator speaks to all introduction to psychology classes on the educational value	1 (2.27%)
Research professor screens each proposal before approving it	1 (2.27%)
Through assessment of the debriefing	1 (2.27%)
All student majors complete an independent research project and the issue of ethics is covered in multiple courses	1 (2.27%)
Learning objective outcomes identified in rubric	1 (2.27%)
It is part of our annual quality improvement plan	1 (2.27%)
The faculty for the course and the subject pool decide on the educational value for a particular class	1 (2.27%)
Subject pool coordinator reviews studies	1 (2.27%)
Evaluate scientific value of the study to ensure it meets standards	1 (2.27%)
We assess the introduction to psychology course and the research credit is part of the course that gets assessed	1 (2.27%)
IDEA course evaluation feedback	1 (2.27%)

Note. Approximately 44 of 54 of the participants from 4-year institutions with an undergraduate research pool, who stated their department assesses the educational value of research participation responded to this item.

Discussion

The purpose of the current study was to replicate and extend dated studies on the prevalence and practices of URPs in the U.S. Approximately 6.3% of 2-year and 67.7% of 4-year institutions had a URP in the psychology department. Regarding 4-year institutions, and consistent with Landrum and Chastain (1999), more public institutions had departments with a URP than private institutions. Of the psychology departments at 4-year institutions with a URP, 83.7% had a research requirement for students in their introduction to psychology courses, representing a substantial increase over the results reported by Landrum and Chastain (1999), when a plurality of programs relied on volunteer-based pools. Additionally, replicating Sieber and Saks (1989), more graduate–degree-granting departments had a research requirement than undergraduate-only programs. Specifically, 80.2% of psychology departments with a graduate program had URPs, slightly higher than Sieber and Saks observed in their sample (74%).

While we cannot be certain why use of URPs increased during this period, their growth has paralleled the trend to weight scholarship more heavily in faculty evaluation processes for promotion and tenure. Schimanski and Alperin (2018) observed this trend across all types of higher education institutions, but it is most pronounced in research-oriented institutions. This is consistent with the current findings. URPs were more common in departments with graduate programs and in 4-year institutions, which tend to be more research-oriented than their comparators (cf. Schimanski & Alperin, 2018). Furthermore, research experience is increasingly critical to psychology graduate school admissions (Nguyen Littleford et al., 2018). These expectations and evaluation processes exert pressure on faculty members and the students they mentor, to produce research. This, in turn, increases demand for research participants, which has likely contributed to the growth of URPs. Departments do not generally attribute the development of URPs to this motivation, instead justifying URPs by arguing that they provide educational value to the participants. However, given that only 16% of departments assessed the educational value of research participation, it is largely unknown whether this is warranted. Without empirical support that participation in a URP has educational merit, the case for their ethical use is undermined.

Also of ethical concern is whether student participants are genuinely capable of voluntary consent and withdrawal of consent, or whether they are unduly coerced to participate. While not recognized as a vulnerable population, college students ⁷ in a URP may be subject to many of the same dynamics that resulted in these populations being conferred special protections. For example, URP participants may perceive limited autonomy when faced with dual relationships with a faculty member posing as both a researcher and an instructor with control over the student’s course grade, particularly when the research requirement represents a significant portion of that grade and/or there are punitive consequences for choosing not to participate (e.g., inability to pass the course). This may be particularly true when any alternative assignment is more onerous, less engaging, and/or more time-consuming than research participation. These concerns are exacerbated when the student is minimally debriefed and when the degree of educational value derived from their participation is unclear. URP participants’ perceptions of autonomy, pressure to participate, and freedom to withdraw are important areas for empirical investigation.

There is some emerging evidence that participation may result in self-reported educational benefits for some students (Zannella et al., 2020). However, psychology researchers should bring their skills to bear on assumptions (e.g., research participation has educational value) that have been taken for granted or used as justifications for URPs. These require the same rigorous testing as any other pedagogical technique. Our call is to focus attention, administratively and empirically, on making participating in research a better personal and learning experience for students.

Based on the current data, we make the following recommendations for further inquiry, to guide best practices in implementing URPs.

1. Psychology departments should identify student learning objectives for research participation (and alternative assignments), communicate these in course syllabi, and assess these.

2. Learning resulting from research participation is largely untested. A few studies assessed student perceptions of the educational nature of research participation (e.g., Cromer et al., 2013; Darling et al., 2007; VanWormer et al., 2014). Future research should evaluate the efficacy of research participation in meeting identified learning objectives and strategies to optimize the educational value of research participation, using objective measures and sound research designs aligned with the scholarship of teaching and learning. Strategies might include having researchers do short presentations about their studies in psychology classes, using debriefings as an educational tool, and creating exams for students based on their research participation (cf. Sharpe & Poets, 2017).

3. Departments should be able to logically defend the research hours required (Sharpe & Poets, 2017). No published research has evaluated the impact of the number of required research hours on learning outcomes. Unless and until supported by empirical data, we recommend setting a conservative number of required hours.

4. How research participation factors into students’ overall grade for the course needs to be appropriate and justifiable. When research participation is weighted too heavily, it raises ethical concerns regarding coercion and respect for persons. We recommend against tying research participation to a student’s ability to pass the course or receiving an incomplete.

URPs are ubiquitous and serve as a primary source of data in psychological research; however, few, if any, best practices exist to guide departmental policies regarding expectations of student hours required, grading, debriefing, and assessment. Departments with URPs should heed the decades of calls to action in the psychology scholarly literature and implement evidence-supported practices that effectively promote student learning while secondarily supporting broad-based research agendas.