Research Associates Program: Expanding clinical research productivity with undergraduate students

Abstract

Objectives:

Clinical research is often time-consuming and difficult to conduct in busy academic institutions. Previous studies have proposed methods to integrate undergraduate students as a means to increase research productivity. The authors aimed to describe the possibility to enhance emergency department research productivity at an academic emergency department in the United States, using undergraduate students in an Emergency Medicine Research Associates Program.

Methods:

The authors described the Emergency Medicine Research Associates Program curriculum and its implementation. We also conducted a retrospective study at a university-based emergency department from January 2005 to December 2014 to demonstrate the benefit of having an established Emergency Medicine Research Associates Program. The primary outcomes were number of Emergency Medicine Research Associates Program–related studies, number of enrolled patients, extramural/intramural funding, abstract presentations, and peer-reviewed publications. The authors analyzed the data using descriptive statistics.

Results:

Over the 10-year period, 110 Emergency Medicine Research Associates Program–assisted research studies were conducted, with research associates enrolling 46,219 patients. These studies yielded a total of 31 peer-reviewed publications and 77 abstract presentations (13 international, 27 national, 37 state/regional). The Emergency Medicine Research Associates Program–related studies were used as pilot studies to obtain US$1,751,036 in extramural grant funding and US$31,047 in intramural grant funding.

Conclusion:

The implementation of Emergency Medicine Research Associates Program can enhance emergency department clinical research productivity, and the inclusion of supplemental academic programs enhanced the undergraduate students’ research experience.

Introduction

Since the formation of academic emergency medicine (EM) as a specialty in the 1960s, emergency departments (ED) in the United States have struggled with conducting patient-oriented research simultaneously with fulfilling clinical responsibilities.^1–4 Despite the recent increase in the number of EM journals, ⁵ the success of academic EM research programs continues to be hindered by high patient volume, lack of protected research time, weak research infrastructure, and little federal research funding.^2,6–11 Compared to other specialties, academic EM historically has received the lowest amount of funding per faculty member and less than 1% of National Institutes of Health (NIH)-sponsored extramural funding. ⁸ Consequently, many academic EM research efforts faced inadequate funding and required alternative approaches.^{1,12, 13}

One alternative approach focused on the recruitment of undergraduate college students as trained research associates (RAs) to conduct clinical research.^6,14–17 Implementing this strategy in nursing and the ophthalmology department resulted in greater numbers of prospective clinical studies, patient enrollments, and publications among faculty and residents.^14–20

The Emergency Medicine Research Associates Program (EMRAP) was first implemented in 1997. EMRAP consisted of both a clinical research component and an academic curriculum for up to 25 undergraduate students per quarter. Students have learned clinical research methodology, prepared grant proposals and manuscripts, and planned for careers in healthcare.

Our study described the impact of EMRAP on research productivity in the ED. We have outlined a cost-efficient, structured student research program that enhances ED research productivity and provides a solid clinical research foundation for undergraduate students.

Materials and methods

Study design

We described the EMRAP curriculum and its implementation at our institution. We also conducted a retrospective analysis of the research productivity, utility, and benefits of the EMRAP program from January 2005 to December 2014. The following resources provided the data presented in this analysis: weekly meeting notes, faculty physician curriculum vitae (CV), the Undergraduate Research Opportunities Program (UROP) database, ED research funding records, and the PubMed journal database. The study received institutional review board (IRB) exemption as non-human subject research.

Data analysis

Our primary outcomes were the following: number of EMRAP-related studies, number of enrolled patients, extramural/intramural funding, abstract presentations (regional/state, national, international), and peer-reviewed publications. We analyzed the data using descriptive statistics in Microsoft Excel Version 15.13.1 and GraphPad Prism 7.0 software (GraphPad Software, La Jolla, CA).

EMRAP

Program development and leadership

Since 1997, EMRAP has trained undergraduate RAs to enroll patients and collect data for multiple ED studies while educating students on clinical and epidemiologic research methodology. A faculty director supervises the program, the interactions between RAs, and communication with principal investigators (PI). The faculty director selects two RAs to become program coordinators and one RA to become an academic coordinator. Program coordinators manage the administrative branch of EMRAP and lead weekly meetings with the RAs to discuss research productivity. The program coordinators work with the academic coordinator to formally train RAs in conducting clinical research. Upon completion of training, RAs have the opportunity to serve as a research coordinator (called EMRAP liaison) for specific research studies. Under the supervision of the PI, liaisons are responsible for the coordination and operational oversight of the research project in the ED (Figure 1).

Figure 1.

Research program organization and leadership.

EMRAP academic curriculum

In 2012, EMRAP launched an academic-based curriculum led by the faculty director and academic coordinator to supplement clinical research education for the RAs. The EMRAP academic curriculum consists of faculty-led lectures and workshops, techniques on writing manuscripts, and oral and poster presentations. Through the academic program, RAs have gained a more comprehensive understanding of the research process, while acquiring skills in writing grant proposals and conducting basic statistical analyses. To assess the RAs’ understanding of ongoing research studies—including study design, inclusion/exclusion criteria, enrollment procedures, and sample size calculations—a written examination is administered to all RAs at the start of the academic year. With the support of UROP, RAs have learned to write grant proposals, receive funding, and present their findings at the annual UROP symposium. RAs also present abstracts at other regional and national conferences and submit peer-reviewed manuscripts under direct supervision of the PI (Figure 2).

Figure 2.

Research program training and academic curriculum.

RA training requirements

RAs complete annual training modules certified by the university, including Health Insurance Portability and Accountability Act (HIPAA) training tutorials and the Collaborative IRB Training Initiative (CITI program). Additionally, RAs receive annual IRB-compliance training from the institutional compliance officer. Once certified, incoming RAs undergo a training period for approximately 2 months under the direct supervision of veteran RAs and the program coordinators. RAs also receive training in the Research Electronic Data Capture (REDCap) database to store enrollment data and log ED shift productivity. The program coordinators use the REDCap tool to account for EMRAP productivity.

RA responsibilities

Enrollment of patients

RAs are assigned to an average of two 4-h ED shifts per week from 8 a.m. to midnight, 7 days a week. Shifts comprised screening, approaching, and enrolling eligible patients into active studies. Up to three RAs cover each shift, for a total of 28 shifts per week, to maximize patient enrollment. EMRAP can accommodate patient enrollment from midnight to 8 a.m. for specific studies on a case-by-case basis. RAs assist with all research projects within the department, including prospective studies, retrospective studies, randomized controlled trials (RCT), and pharmaceutical trials. RAs are able to initiate the consent process; however, the faculty researchers and departmental research staff must finalize consent for subject participation, for each study.

Research coordinator (liaison)

RAs can pursue leadership opportunities as research coordinators (liaisons). The coordinator’s role is to facilitate communication between the PI and the RAs, supervise patient screening, verify data collection, and work with the PI to facilitate study completion. Coordinators have the opportunity to assist in the preparation of a research abstract, present findings at conferences, and prepare manuscripts for publication.

Program sustainability: time and cost

EMRAP consists of up to 25 RAs quarterly, providing a 1-year program commitment, who each dedicate 10–12 h weekly to research. The undergraduate students participate in the program for academic credit. The program director contributes approximately 2 h/week to manage the program with a budget of US$2000 yearly for administrative expenses.

Results

Clinical studies and patient enrollments

From 2005 to 2014, EMRAP assisted in 110 ED studies, in which RAs participated in enrolling 46,219 subjects. The number of yearly clinical research projects increased from three studies in 2005 to 20 studies by 2014 with an average of 11 studies per year (Table 1). RAs’ annual patient enrollments increased from 441 subjects in 2005 to 4297 subjects in 2014 with an average of 4622 enrollments per year (Table 1, Figure 3).

Table 1.

Variable totals and averages per year (2005–2014).

	Average (per year)	Total
EMRAP-assisted studies a
Active ED studies	11	110
Patients enrolled	4622	46,219
Peer-reviewed publications	3	31
Abstract presentations
State/regional	4	37
National	3	27
International	1	13
Funding
Extramural grants	2	15
Extramural grant amount awarded	US$175,103.63	US$1,751,036.25
Intramural UROP grant proposals	9	86
Intramural UROP grant amount awarded	US$3104.70	US$31,047
Studies attending UROP symposiums	8	83
Intramural SURP amount	US$620	US$6200
ED studies participating in UROP	13	129

EMRAP: Emergency Medicine Research Associates Program; ED: Emergency Department; UROP: Undergraduate Research Opportunities Program; SURP: Summer Undergraduate Research Program.

a

Numeric values rounded up to the nearest whole number.

Figure 3.

Number of EMRAP-related (a) active clinical studies and (b) enrolled subjects from 2005 to 2014.

Published manuscripts and abstract presentations

From 2005 to 2014, EMRAP-assisted research studies had one published manuscript in 2005 and seven manuscripts in 2014, for a total of 31 publications. The number of abstract presentations at the state/regional, national, and international meetings averaged eight abstracts presented annually (Figure 4).

Figure 4.

Number of EMRAP-related (a) published manuscripts and (b) abstract presentations from 2005 to 2014.

Extramural and intramural funding

From 2005 to 2014, EM faculty obtained over US$1.75 million in extramural funding through EMRAP-assisted studies, either by using EMRAP-assisted study results as pilot study data or by listing EMRAP as one of the institutional resources available to ensure data enrollment feasibility. Extramural funding constitutes all funding received from external institutions and agencies to conduct EM research within EMRAP.

Intramural funding constitutes all funding received from the university’s UROP program. RAs obtained a total of US$31,047 intramural, UROP grant funding under their PI supervision. There was an increase in UROP grant proposals and funding during this duration. While no proposals were submitted in 2005, US$4350 in UROP grant funding was obtained in 2014 through 13 undergraduate grant proposals, with an average of US$3105 awarded each year. The highest amount of funding awarded was US$4700 in 2013. The lowest amount of funding awarded was US$1975 in 2007, with the exception of US$0 awarded in 2005 (Figure 5).

Figure 5.

Number of EMRAP-related studies receiving (a) extramural and (b) intramural funding and (c) submitted UROP proposals from 2005 to 2014.

Discussion

In EMRAP-assisted studies, RAs were able to enroll 46,219 subjects over a 10-year period. ED faculty published 34 manuscripts, presented 77 abstracts (state/regional, national, and international), and received over US$1.78 million in extramural/intramural funding. Implementation of EMRAP enhances ED clinical research productivity.

In comparison, Hollander and Singer ¹⁶ reported an increase in publication rate from 2 to 20 manuscripts and abstracts per year using a similarly structured research program. However, it is unclear whether these publications are from studies only conducted by their RAs program or from the entire ED. In contrast, the studies and resulting publications reported in our study are all from original research and only include those conducted through EMRAP.

RAs receive graded academic units in place of hourly compensation; thus, lowering the cost of research conduction. Clinical-oriented research has significant costs to maintain a robust patient enrollment, as one study reported an annual cost of US$52,833 using 10 college-level paid RAs. ⁶ In contrast to other RA programs, which require external funding for hiring full-time managers and staff,^6,18 EMRAP remains both cost and time efficient, requiring 25 RAs with 10 h/week, at a cost saving of US$130,000 annually.

A significant proportion of enrollments were from retrospective studies, suggesting that low-risk, minimal intervention research studies may be optimal for college-level RAs in a university-level research program. ¹⁸ These studies do not require any active patient screening or consent and reduce the time needed per enrollment. Subsequently, we observed higher enrollments during years with higher proportions of retrospective studies. In 2010, over 15,000 of the 17,800 enrolled patients were from a single retrospective study. Similarly, in 2011, 30% of the studies were retrospective (7215 enrollments), while only 11%–15% of the studies were retrospective from 2012 to 2014 (2184–4297 enrollments).

Supplementing the EMRAP clinical research with an academic component prepares EMRAP for submitting grant proposals and encourages RAs to obtain intramural funding. Through the academic curriculum, RAs were trained in grant proposal planning, manuscript writing, and delivering oral presentations.

Similar programs report involvement of RAs in authored publications and presentations. ¹⁵ RAs also received valuable clinical and research experience that helped guide their decisions to pursue careers in healthcare and postgraduate school.^14,18,20 Participation in RA programs may potentially be attributed to the higher rates of acceptances to medical schools (79% compared to Association of American Medical Colleges (AAMC) national average of 49%) observed in student RAs. ²¹ Thus, many of the observed benefits to RAs arise from early undergraduate clinical research exposure, which in turn may lead to greater interest in health careers that include patient outcomes research.^2,22

These results are not exclusive to an academic EM setting. The implementation of an undergraduate research program can offer other clinical departments and specialties a method to conduct original research projects and maintain clinical-duty time, costs, and overall productivity, as previous studies have shown.^14–20 This study contributes to the existing literature on the effectiveness of undergraduate research programs for both the faculty and students.

Limitations

The reported data relied on the accuracy of faculty CVs, the UROP database, and archived meeting notes. Missing data, such as missing meeting notes or out-of-date CVs, may result in underreported values for each variable. A prospective evaluation of EMRAP would potentially yield more accurate results, as described in previous studies.^6,14,15,20 Additionally, there are no data on the total number of patients screened for each study.

RAs are not assigned from midnight to 8 a.m. due to low patient volume during these hours. Consequently, the enrollments may represent a convenience sampling for EMRAP-related studies. The department should provide the additional coverage for these hours to capture consecutive samples.

While our ED experienced a robust increase in research productivity from 2005 to 2014, the increase may be a result of other factors that occurred during this time period aside from the implementation of EMRAP, such as hiring new faculty and staff.

Furthermore, there is no method available to objectively evaluate the quality of data obtained by the RAs in addition to controlling for variation in productivity between the RAs. There were no control or comparison groups for this study. An association between the implementation of EMRAP and an increase in ED research productivity cannot be made as baseline information is not available.

Conclusion

Over the past 10 years, we observed an increase in EMRAP-assisted active research studies and subjects enrolled per year. Consistent with previous findings on the effectiveness of using undergraduate students as RAs, we support the implementation of a low-cost, well-trained undergraduate RAs program in conducting clinical research in the ED. Additional specialties may consider the adoption of a program similar to EMRAP to incorporate students early on in the healthcare setting while not compromising the physician’s focus on providing care for patients. Supplemental academic programs could enhance RAs’ research experience and research program funding.