Sage Journals: Discover world-class research

Abstract

OBJECTIVES

Instilling the principles of ethical and responsible medical research is critical for educating the next generation of clinical researchers. We developed a responsible conduct of research (RCR) workshop and associated curriculum for undergraduate trainees in a quantitative clinical research program.

METHODS

Topics in this 7-module RCR workshop are relevant to undergraduate trainees in quantitative fields, many of whom are learning about these concepts for the first time. These topics include: (1) research collaboration and mentoring, (2) human subjects and privacy, (3) rigor, reproducibility, and responsibility in data analysis, (4) data safety, security, and sharing, (5) responsible authorship, peer reviews, and conflicts of interest, (6) research misconduct and whistleblowing, and (7) social responsibility and contemporary ethical issues. The workshop adopts an active learning approach and is intended to encourage peer-to-peer learning through a critical review of case studies and contemporary media reports. Workshop materials and discussions are designed to address experiences relevant to undergraduate trainees. For example, for the discussion on human subject protection, trainees are asked to review their own university's policies for engaging students as research subjects. Student perceived knowledge and understanding of research ethics were obtained from pre- and post-workshop surveys, with responses ranging from 1 (low level of understanding) to 5 (high level of understanding).

RESULTS

Survey responses from 38 trainees across 4 years demonstrated marked increase in perceived knowledge and understanding of research ethics, from 64% in pre- to 96% in post-workshop responses with scores of 3 and up. Trainees noted the effectiveness of the active learning format of the workshop and relevance of the topics on their own research projects.

CONCLUSION

Intentional instruction is vital to help trainees discern between good and bad science in research. The presented RCR curriculum can be adapted by other programs designed for trainees engaged in medical research and education.

Keywords

scientific ethics and integrity rigor reproducibility human subjects research training bioethics medical research curriculum development

Introduction

Research ethics and the practice of experimentation with integrity are foundational to good scientific and biomedical investigation. Therefore, education in the responsible conduct of research (RCR) is a fundamental component of research training. Federal agencies and academic societies such as the US National Institutes of Health (NIH),¹ the US National Science Foundation,² and the European Federation of Academies of Sciences and Humanities (ALLEA),³ advocate for the integration of instruction in ethics within research training programs and have provided recommendations for effective training. In particular, the NIH recently updated its guidance for instruction in RCR for research trainees, recommending specific formats, frequency, timing, and subject matter.⁴

Formal training in RCR is essential across all educational stages, including the undergraduate level. In the last decade, the number of undergraduate students exposed to research has increased, accompanied by a growing number of undergraduate research programs and course-based undergraduate research experiences^5,6 available to students. Undergraduate students greatly benefit from participation in hands-on research. The Boyer Commission Report, which is a blueprint for undergraduate education at US research universities, advocated interdisciplinary and research-based education at the undergraduate level to prepare students early for success in graduate school and the workplace.^7,8 In these research-based programs, undergraduate trainees learn the research process and gain a deep understanding of the research process and the principles guiding scientific decision-making, gain valuable disciplinary knowledge, hone their intellectual skills in inquiry and data analysis, and appreciate the formal research mentoring process.^9,10

Despite the growing demand for undergraduate research experience, many participants in research programs arrive with limited or no training in RCR. For many undergraduate trainees participating in summer internship and training programs in the quantitative sciences (henceforth referred to as undergraduate trainees), these programs may be their initial introduction to hands-on research and the first exposure to the norms, principles and regulations relating to the practice of human subjects research. Quantitative science refers broadly to a field of study that focuses on the applications of mathematical, statistical, and computational approaches to analyze and interpret data. This includes measuring variables, creating models, testing hypotheses, and making predictions using data. Quantitative sciences are essential in numerous disciplines, including biology, economics, engineers, medicine, and social sciences, among others. Key areas of quantitative sciences include statistics, biostatistics, epidemiology, health policy, health economics, bioinformatics, biomathematics, quantitative life science, data science, health informatics, and computational biology. Students from the quantitative sciences may pursue data-related careers such as biostatistician, epidemiologist, and physician-scientist. The need to implement RCR training with these research programs is evident: these core concepts not only guide their summer research activities, but they also form a solid foundation for ethical research behaviors as the trainees continue their medical research careers whether as physician-scientists or as data scientists. However, formal instruction of research ethics is generally limited in the traditional undergraduate educational system. This paper introduces elements of an RCR curriculum designed for undergraduate trainees engaged in quantitative cancer research and describes steps to implement the curriculum using diverse materials. The proposed curriculum is ideal for summer undergraduate research programs but can be adapted for longer programs or course-based medical research experiences.

Methods

The RCR workshop detailed below was developed in 2017 for trainees participating in the National Cancer Institute-funded Quantitative Sciences Undergraduate Research Experience (QSURE) summer program at Memorial Sloan Kettering Cancer Center. QSURE is a comprehensive 10-week summer research program designed to provide hands-on research experience for undergraduate students to encourage them to pursue higher degrees or careers in cancer-related quantitative sciences. Every year, completed applications are reviewed by the program steering committee to select a cohort of up to 10 students. In addition to conducting an independent mentored quantitative research project with application in cancer, trainees participate in activities to enhance their knowledge, technical proficiency, and professional skills. The RCR workshop is a mandatory core component of the curriculum for all QSURE trainees. Details regarding the QSURE program and characteristics of the trainees in the first 4 years have been previously published.^11,12

The RCR workshop was modeled on a well-established course for federally funded trainees at the pre-doctoral and post-doctoral levels, with unit themes and content from the text, On Being a Scientist: A Guide to Responsible Conduct in Research, Third Edition, published by the National Academy of Sciences,¹³ and aligns with the key areas of RCR emphasized by the NIH Office of Research Integrity.¹⁴ Workshop sessions and content were adapted for an undergraduate audience studying quantitative sciences, with cases and discussions relevant to scenarios they might reasonably encounter in their current or future research.

Workshop Format

The QSURE RCR workshop is a series of facilitated discussions over the course of 7 modules (see Supplemental Material for the sample syllabus), ideally integrated within a short-term or summer research experience program. All workshop facilitators should have received didactic and experiential training in leading RCR case discussions through the institution's Division of Research and Technology Management. Workshop facilitators should have direct and substantive involvement in research such as design, implementation, analysis, or reporting. Within the QSURE program, the RCR workshop is facilitated by 2 program directors. In some cases, a guest discussant with specific expertise or experience on a topic may enhance certain modules. Departing from conventional lecturing, the RCR curriculum adopts an enquiry-based learning approach, emphasizing active participation, case-based discussions, and activities, and a blend of individual and group exercises, with examples and case studies from cancer research.

Workshop Materials

The trainees receive the workshop syllabus at the beginning of the program (see Supplemental Materials). The syllabus outlines the plan for each week, including learning objectives, relevant chapter(s) from the book, “On Being a Scientist,”¹³ and additional reading materials drawn from scientific journals and lay press. Trainees receive workshop materials for each week at least 1 week in advance to complete the readings and prepare for engaging discussions. All QSURE trainees also received a copy of “The Immortal Life of Henrietta Lacks,”¹⁵ by Rebecca Skloot, to supplement their RCR learning, particularly on the special consideration of research with underrepresented populations.

Workshop Curriculum

The RCR curriculum consists of topics relevant to human subjects research, appropriate for an undergraduate level audience (Figure 1). While the concepts are interconnected, each module is designed as a standalone component. This flexibility allows the curriculum to be adapted and tailored to meet programmatic needs and resource constraints—for example, modules can be rearranged or a subset selected. For example, in the 10-week QSURE internship program, the 7 RCR modules were presented weekly in the order below in the first 7 weeks of the program to accommodate program-specific activities such as final presentations of student research. Throughout the workshop, trainees are prompted to reflect upon how the week's topic may be relevant to their individual research projects. Trainees are especially encouraged to identify potential ethical concerns or questionable research practices that could arise in the research that they are engaged in over the summer. Throughout the course, if a student expressed an incorrect understanding of a key term or concept, facilitators were instructed to clarify and correct during the session and to follow up with the student at the next session to confirm understanding and retention of the term or concept. In the QSURE program, each module was designed to be 1-h long; however, other programs may consider longer durations to dedicate time to additional topics within the module. Additional details on the materials for each module are summarized in the sample syllabus (see Supplemental Materials).

Figure 1.

Alluvial plots of survey responses from workshop participants on perceived level of knowledge and understanding in research ethics at the beginning (baseline) and at the end of the program (end-of-program survey). Survey responses were categorized as Low (if the response was 1 or 2), Medium (if the response was 3), and High (if the response was 4 or 5). The proportion of trainees in each category is presented in each box; the width of the stream reflects the proportion of trainees in each pre-post pair of response categories.

Module 1: Research collaboration and mentoring

As trainees embark on their first research experiences, it is critical for them to grasp the significance of research collaboration, mentoring, and self-advocacy in career development. Mentoring, seen as a collaborative process, involves reciprocal engagement between mentors and mentees. The lessons derived from this module will equip trainees with essential skills to foster mentor–mentee relationships throughout their research projects and beyond. The learning objectives in this module are to (1) understand the role of mentoring in education and career development, (2) analyze challenges from both mentee and mentor perspectives, and (3) evaluate traits and actions contributing to an effective mentor–mentee relationship.

Interactive idea: in small groups, trainees are asked to reflect on the reading and generate a list addressing each of the prompts below:

“What qualities do you seek in a mentor? What is the role of the research mentor for your particular research project?”

“What are qualities and actions of a good mentee?”

Similar to best practices for effective undergraduate mentors,¹⁶ the goal of this exercise is to compile a list of salient practices for undergraduate mentees (see Supplemental materials for examples).

For RCR curriculum integrated within a summer research program, introducing this module at the beginning of the program equips trainees with a solid foundation to navigate mentor–mentee relationships effectively and set appropriate expectations for the remainder of the research program.

Module 2: Human subjects, privacy, and Health Insurance Portability and Accountability Act (HIPAA)

In this module, trainees discuss ethical principles and regulations for the protection of human subjects. This module provides the historical background that motivated these regulations, contextualizing the significance of RCR through cases that highlight unethical treatment of human subjects.¹⁷ The learning objectives in this module are to (1) recognize the ethical principles and regulations relevant to research with human subjects, (2) examine the elements of informed consent, and (3) distinguish types of protected health information (PHI). Discussions and analysis of the following key documents, in addition to other related publications, can facilitate achieving these learning objectives: the Nuremberg Code, the Helsinki Declaration, and the Belmont Report. Additionally, intentional discussions involve the topic of special considerations of human subjects research among underrepresented populations. For example, in the United States, the issues of medical ethics and systemic racism historically faced by African Americans were illustrated through landmark cases such as the Tuskegee Syphilis Study and the story of Henrietta Lacks.

Interactive idea: trainees are asked to apply these readings to the following exercises for in-person discussion:

“Reflect on an experience you have had while participating as a study subject (school survey on alcohol use, or psychology and behavioral study, etc). How was that experience for you as a human subject? What information was provided to you, were your responses anonymous? Did you give informed consent?” and

“Perform an online search to find out what your university's policies are regarding human subjects research; in particular, any mention of students serving as research subjects. See example from the Penn State IRB”¹⁸

Module 3: Rigor, reproducibility, and responsibility in data analysis

This module focuses on rigor, reproducibility, and responsibility in scientific research. The “3R” core norms of good scientific practice are fundamental across disciplines from bench science to public health research. The learning objectives are to (1) define concepts of rigor and reproducibility in research and (2) examine the quantitative scientists’ role and responsibility in research. Trainees review the ethical guidelines of the American Statistical Association,¹⁹ and a Nature Human Behavior paper²⁰ describing practical steps in the study design and analysis to ensure rigor and reproducibility, along with a textbook chapter on data treatment.

Interactive idea: To enrich discussions in this module, we recommend including an experienced biostatistician or quantitative scientist as a guest facilitator. The guest can share insights into challenging practical scenarios such as being asked to perform an inappropriate analysis (for example, to remove some data points in order to see statistically significant results or to keep adding patients until significance is attained). The guest can share real-world examples of how they successfully managed such situations and proactively implemented preventive strategies for the future. Example discussion prompts include, “What comes to mind when you hear the phrase rigor, reproducibility, and responsible data analysis?” “How should we balance between ‘rigor’ and ‘practicality’?” “Who is responsible for data and analysis integrity in an interdisciplinary team?”

Module 4: Data safety, security, and sharing

This module is devoted to the protection of the privacy, security, and confidentiality of data produced and analyzed in the context of human subjects research. Since the trainees are working with data as part of their research, it is crucial for them to be cognizant of the proper processes for data collection, storage, and sharing. The learning objectives are to (1) recognize the importance of data safety and security and (2) interpret policies and debates on data sharing. Trainees analyze the pros and cons of data sharing by reading and discussing published articles. They also discuss an online article about breach of data safety when a laptop containing data was stolen.

Interactive idea: Reflecting on the reading,

Each trainee describes (either in small groups or within the larger cohort) the type of data they are working with in their research (eg, highly sensitive data, restricted use data, open public records, public use data, confidential, anonymous, de-identified^21,22) and identify hypothetical scenarios and concerns that may arise in the collection, storage, and sharing of the data.

In small groups, the trainees brainstorm scenarios in data collection, storage and sharing and classify them within 3 zones: red zone (violates the principles of research data security), yellow zone (determination depends on the context), and green zone (aligns with the principles of research data security). Each group then shares key examples from the red and yellow zones to engage the larger group in discussion. Examples may include: “Trainee A is working with patient data that includes patient date of surgery and date of death. In order to continue working on the project while at home, the trainee emails a copy of the data to their personal email account.” “Trainee B is intrigued by the gaining popularity of ChatGPT Advanced Data Analysis feature.²³ The trainee decides to upload data collected from a recent survey to the site to test the capabilities of the artificial intelligence tool.”

Module 5: Responsible authorship, peer reviews, and conflicts of interest

In this module, the trainees focus on considerations surrounding responsible authorship and peer review being a trusted form of scientific communication. The learning objectives are to (1) articulate principles of responsible authorship, (2) examine the peer review process, its benefits, and possible biases, and (3) evaluate potential conflicts of interest in scientific research. The facilitator engages trainees in a discussion on challenges surrounding authorship conversation, particularly for those currently engaged in mentored research leading to publications. The group also discusses a paper by Kelly and colleague²⁴ on the benefits and critiques of peer review in scientific publications, addressing questions such as, “What is peer review?” “What are its benefits and limitations?” “Why is it essential for the scientific community?” Discussions also include a reading assignment regarding controversies surrounding a lung cancer screening trial.²⁵ In this trial, financial conflicts of interest and the use of tobacco company money to fund research came together in a highly unusual way, complicating the interpretation of results.

Interactive idea: The session facilitator can guide trainees through the peer review process using real examples from journals that employ a transparent peer review system. For example, BMC Medicine²⁶ publishes the reviewer reports alongside accepted articles. This exposure often serves as the initial introduction for undergraduate trainees to understand authorship, peer review processes, and the significance of the CRediT statement.²⁷ Examples of discussion prompts include: “Would you prefer single-blind/double-blind/open-peer/transparent peer review as an author versus a reviewer?” “How does the peer review process relate to RCR? Could/how would it impact the rigor of human subjects research?” “What are some criticisms of the peer review process?”

Module 6: Research misconduct and whistleblowing

As early career researchers, undergraduate trainees may not have had direct experience with research misconduct. Distinguishing scientific misconduct from genuine mistakes should increase the trainee's knowledge and awareness to detect and avoid research misconduct. The learning objectives are to (1) define scientific misconduct, (2) evaluate how misconduct differs from honest error, negligence, and difference of scientific opinion, and (3) analyze potential risks to and protections for whistleblowers. Trainees discuss a case study where 2 scientists falsified data to secure federal funding. The study's statistician, who detected this fraud, was terminated and their career came to an end. The statistician later won an appeal. From this case study, trainees learn about protection for whistleblowers.

Interactive idea: In small groups, trainees are asked to:

Recall the student code of conduct from their universities or colleges and generate examples of actions that would fall under academic misconduct. Then,

Identify from the initial list any actions that may also apply to research misconduct and add other scenarios that would apply to research misconduct. Lastly,

Consider scenarios in statistical analysis that could fall under “gray” areas of research integrity versus misconduct. For example, using statistical methods to identify outliers and removing them in an analysis, performing 10 separate subgroup analyses but only reporting 1 in the publication, or applying 2 different statistical methods on the same dataset to pick the one with significant results. Trainees then present these scenarios to engage the larger group in discussion.

Module 7: Social responsibility and contemporary ethical issues in cancer research

In addition to the topics addressed thus far, such as informed consent and research misconduct, it is essential for trainees to grasp how these ethical considerations and issues are intricately woven into today's cultural and social environment. The learning objectives are to (1) recognize ethical issues that arise in modern society and (2) identify the roles of quantitative scientists in addressing these issues, with a special focus on applications in the field of cancer research.

Interactive idea: prior to the workshop, trainees are assigned to groups to prepare a 20-min segment on 1 of 3 broad topics:

Genomic research using existing data (eg, biobank/biorepository)

Translating research into healthcare policymaking and resource allocation

Biases in statistical interpretations and predictions

Each group will define the research area and describe relevant ethical issues incorporating considerations of equity, justice, diversity, and inclusion in cancer research. Groups can present in any manner they choose. Each presentation is then followed by a discussion led by the group members, focusing on ethical issues that are relevant to the topic. Through the activity, trainees also learn to collaborate respectfully and professionally beyond the classroom. The activity can also be modified from a group-based activity to one that focuses on RCR related topics specific to each trainee's research projects.

Evaluation and Statistical Analysis

In this prospective survey-based study, we evaluated student perceived level of knowledge and understanding in research ethics using self-reported data from trainees in the Quantitative Sciences Undergraduate Research Experience (QSURE) program.¹¹ As the RCR workshop is mandatory for all QSURE trainees, the study population for the survey was a convenience sample of the cohort of trainees. Hence, the inclusion and exclusion criteria of the survey respondents are the same as the selection criteria used to select the final cohort of trainees each year.

Trainees were asked to complete surveys at the beginning of the QSURE Program (Week 1, pre-program survey) and at the end of the program (Week 10, post-program survey). Trainees were informed on the web-based survey that their responses would be linked by survey ID in order to connect their pre-program survey responses to their post-program survey responses, and that all reporting of responses will be anonymized. The surveys assessed perceived proficiency and knowledge across 2 skill domains (analytical and communication) and 2 knowledge domains (statistical and other scientific knowledge). One of the domains assessed is research ethics, which is the primary outcome of interest for this study. Details of the complete pre- and post-program surveys can be found in Tan et al.¹¹ Trainees report their current perceived level of knowledge and understanding in research ethics at both timepoints using a Likert scale²⁸ ranging from 1 (indicating low level of understanding) to 5 (indicating a high level of understanding) (see Supplemental Materials). Pre- and post-workshop responses are summarized using alluvial plots to reflect Low (response = 1 or 2), Medium (response = 3), and High (response = 4 or 5) perceived knowledge and understanding categories. Comparison of proportions from pre- to post-workshop was conducted using a 2-sided Bhapkar test. Descriptive summaries are obtained for qualitative feedback from open-ended question about the RCR workshop from the post-program surveys. These open-ended feedbacks were grouped using thematic content analysis. The reporting of this study conforms to the Consensus-Based Checklist for Reporting of Survey Studies (CROSS) statement²⁹ (see Supplemental Materials).

Results

The 7-module RCR curriculum was implemented for 4 consecutive summers between 2018 and 2021, involving a total of 38 trainees across all these years. All 38 trainees in the QSURE program participated in the RCR Workshop and all provided responses in pre- and post-surveys. Details regarding the characteristics of the trainees in the first 4 years have been previously published.^11,12 Briefly, the majority of trainees were female and had completed their junior year of undergraduate in the United States. In general, the trainees’ perceived level of knowledge and understanding increased at the end of the program, compared to their starting level (Figure 1). At the beginning of the program, 36% reported a low level of knowledge and understanding in research ethics, 36% reported a medium level, and 28% reported a high level. At the end of the program, the proportion of trainees who reported a high level significantly increased to 83% (Bhapkar test P < .001). Medium and high levels combined formed 96% of responses in the post-program survey. Year-by-year results also demonstrated similar increases in proportions of trainees with perceived knowledge and understanding in research ethics by the end of the program across all 4 years (Supplemental Figure). Of note, the RCR workshop was conducted online in 2020 and 2021 due to institution-wide restrictions on in-person activities.

Most trainees provided positive feedback about the overall RCR workshop in the post-program survey (Table 1). Trainees noted that the active learning format of the workshop was effective and provided a welcomed break from lectures and research. While many appreciated the pre-workshop reading materials, some found it hard to complete the readings while balancing their own research projects. The trainees particularly enjoyed insights from the facilitators about their personal experiences in navigating these issues. A few trainees acknowledged the relevance of these topics to politics and current culture and appreciated learning about the historical context of many of these issues. Overall, trainees expressed that the workshop was thought-provoking and a necessary component of their research experience.

Table 1.

Selection of student open-ended feedback on the responsible conducts of research workshop from the post-program evaluation.

THEME	SAMPLE FEEDBACK
Workshop format and materials (N = 18)	“I enjoy these seminars and find them quite informative. I like the discussion-based format and find it to be a nice break from lectures and research. Since the group is just QSURE folks and maybe a guest speaker, I feel more comfortable asking questions.” “Sometimes, it was not easy to finish all the readings when we were busy working on our projects, but most often, the reading level was appropriate.” “I had general knowledge of those topic but being able to read current paper/report and to discuss in a small group setting helped me to have deeper understanding and to form my own argument/opinion on those.” “The RCR seminars have been great. I enjoy the discussions on topics that are not commonly touched upon in school/research settings. I also appreciate the readings that are assigned to us every week.”
Real-world examples (N = 10)	“I learned a lot from the discussion and real-world cases. The seminars make me more aware of the ethical issues in my daily life and research.” “They were very interesting! The conversations are important to have, and it was eye-opening to see what my peers see/ feel. Moreover, having the facilitator share experiences that were personal, real and challenging made it easier to understand - in terms of what responsible/ irresponsible research conduct looks like.”
Exposure to RCR early in career (N = 4)	“I think that these workshops are extremely necessary for fledgling professionals like us. They've led to interesting conversations about ethics, responsibilities, data sharing, authorship, and so much more.” “I enjoyed the well-developed, frank conversations. It was thought-provoking and important to have earlier in our careers.”
The value beyond research (N = 8)	“I really enjoy these seminars. As scientists, especially as statisticians, we are responsible for ensuring that our research is done ethically. I really like how these seminars talk about how political research truly can be and how to navigate some of these issues.” “I really enjoy the RCR workshops. To be honest, I switched to research path unexpectedly; and so, I didn't really pay attention to these ethical issues before that. These RCR seminars really filled in that gap and helped me think more critically and conduct any analysis more responsibly. It's interesting in a way that these issues are related to politics, cultures, and so on.”

The responses are grouped by relevant themes. Minor edits in italics for grammar and clarity. N refers to the number of responders who provided feedback in the theme.

Discussion

We demonstrate the successful development and implementation of an RCR workshop designed for undergraduate summer trainees engaged in quantitative science research at Memorial Sloan Kettering Cancer Center. The central goal of the 7-module workshop is to foster a profound understanding and heightened appreciation of ethical and responsible research through active learning. The workshop aims to raise awareness of these considerations, enabling trainees to seamlessly integrate the topics learned into their own research. Trainees learn that while some scenarios are governed by bright lines and regulations that cannot be crossed, other circumstances may present ambiguities that require consideration of the broader cultural, economic, and political context. Through active learning, workshop participants reflect on their own experiences, examine multiple sides of a problem, and recognize that solutions to issues are often neither obvious nor straightforward. Through these reflections, participants are guided to assess the benefits and harm, pros and cons, and individual and societal perspectives of a given issue. The diverse materials such as case studies and activities reflect the challenges the trainees will likely face in future professional careers.

As a QSURE program requirement, trainees were required to complete training in human subjects research offered by the Collaborative Institutional Training Initiative (CITI Program) during Week 1 of the summer research program. The CITI Basic Course is conducted online and completed at the trainee's own pace. The modules are designed to be completed within 25 to 35 min each and consist of detailed written content, graphics, case studies. Each module also includes a quiz, and the trainee must score a minimum of 80% on each quiz in order to proceed to the next module. While the CITI training helped trainees to contextualize the workshop materials and discussions, it is not a requirement for the implementation of the RCR curriculum nor does CITI training alone meet the RCR training requirement for NIH funded training programs. However, we consider the CITI Basic Course training to be complementary to the proposed RCR Workshop: The CITI basic course focuses on technical knowledge, while the RCR workshop engages the student in active learning through in-person activities and discussions. Similar to a flipped classroom learning format, the technical knowledge gained from the CITI course can enhance the quality of the workshop, and the students are better prepared to engage in discussions during the workshop. Similarly, the active learning format of the workshop can help solidify the technical knowledge gained through the CITI course.

The topic of this RCR curriculum is relevant to graduate quantitative science trainees as well. The modular nature of the RCR curriculum enables modifications by other programs to adapt to program and trainee needs. For example, as the research projects within QSURE are human-subject-focused, the QSURE RCR curriculum does not include a module on laboratory safety, biosafety, and animal research subjects. However, these modules can be developed and incorporated into the existing curriculum. Similarly, the format of instruction may be modified to meet the needs of the program. For example, when the summer research program had to be shifted online at the height of the COVID pandemic, this RCR curriculum was successfully implemented as a virtual workshop.¹² The degree of improvement in perceived knowledge and understanding of research ethics during the virtual program was similar to what was observed among trainees of the in-person workshop.

In the field of RCR education, several groups have proposed creative approaches to teach undergraduate students research ethics, such as watching an interactive theatrical presentation of several research misconduct scenarios,³⁰ role play exercises,³¹ and single-player choice-based interactive narrative video game.³² These alternative methods of delivery have been shown to effectively engage undergraduate students in active learning. We believe there is value in formal instruction of research ethics, especially for the students who are concurrently conducting medical research, many for the first time. Depending on programmatic flexibility, these innovative approaches can be incorporated into the formal RCR curriculum.

This paper addresses the training of student mentees engaged in summer cancer research. However, it can be argued that beyond formal instruction, the informal instruction of responsible conduct of research permeates the entire research training experience. For trainees, their research mentor or supervisor plays a pivotal role in the informal training. Resources like the NIH Office of Research Integrity's “5 ways supervisors can promote research integrity³³” can be integrated into existing mentor training courses to help establish a culture that promotes integrity.

Because this RCR workshop is embedded within a 10-week summer research experience program, the format and duration were chosen to accommodate programmatic requirements. Furthermore, the pre- and post-program evaluations included assessments of other domains related to the summer program. In order to balance the tradeoff between response burden and defining a metric for success, there were only 2 questions on the survey about the workshop: the trainee's self-reported level of research ethics knowledge and an open-ended feedback question regarding the workshop. Both of these components focus on the trainee's perception of the workshop rather than formal assessment of learning, behavioral changes, or direct results. We acknowledge that perceived knowledge may not directly correlate with cognitive learning; however, based on trainee overall feedback, we consider the workshop to be an engaging introduction of research ethics considerations. We recognize the limitations of the current evaluation metric and are in the process of updating the evaluation process to more formally assess student knowledge gained from the workshop and emphasize various Kirkpatrick outcomes.³⁴ Programs that are interested in adapting this curriculum may consider utilizing formal assessments, such as a case-study rubric³⁵ or Hirsch's survey,³⁶ which includes 30 content-based questions reflecting the 9 RCR areas of focus defined by the NIH Office of Research Integrity.³⁷ Decision Procedure Checklist (DPC)³⁸ scores can also be used to summarize the trainee's ability to critically assess their own research projects to identify where ethical issues may arise, stakeholders involved, alternative courses of action and responses, and the short- and long-term consequences of proposed solutions. Finally, as this study utilized all respondents who were already participating as trainees in the QSURE summer research program, no formal power calculation was conducted. Furthermore, as all workshop participants were part of a highly competitive research program, the positive experiences and improvement in perceived knowledge and understanding of research ethics may not be generalizable to the general undergraduate student population.

While the curriculum was developed in 2017, it has since been updated each year to include current topics and feedback from trainees. For example, to address the feedback that some found it challenging to complete the readings while balancing their own research projects, QSURE program curriculum team routinely assess the pre-workshop reading materials for brevity, relevance, and broad interest. Additionally, during the first module, the facilitator sets the tone and frames the workshop as a crucial component of their learning: that this is just as important as their hands-on research as it trains them not just to be researchers, but responsible researchers. With a shift in mindset, students place greater value on RCR and the workshop. Lastly, the RCR curriculum should be continuously updated to reflect new federal guidelines and contemporary ethical issues in cancer research. For example, in the summer 2020 program, Module #2 human subjects and HIPAA coincided with the Shut Down Academia and Black Lives Matter movement.³⁹ The workshop was adapted to focus the conversation about race, ethnicities, and cultures in the academic research world, from the perspective of both a researcher and a research participant, guided by relevant articles and news insights.^40–43 Similarly, as the use of big data and artificial intelligence become commonplace in cancer research, additional focus may be placed on relevant ethical issues, such as algorithmic^44–46 bias in clinical research. Efforts are currently underway to incorporate equity, diversity, and inclusion components throughout this RCR curriculum.

Conclusions

In addition to technical skills, the training of the next generation of medical researchers should incorporate early training in the responsible conduct of research. Intentional exposures and instruction are vital to help trainees discern between good and bad science, navigate ethical dilemmas, and hone their moral reasoning skills. The QSURE RCR curriculum offers formal training in responsible conduct of research to undergraduate trainees engaged in quantitative clinical research.

Supplemental Material

sj-pdf-1-mde-10.1177_23821205241293515 - Supplemental material for Implementing a Responsible Conduct of Research Curriculum for an Undergraduate Summer Clinical Research Program in Quantitative Sciences

Supplemental material, sj-pdf-1-mde-10.1177_23821205241293515 for Implementing a Responsible Conduct of Research Curriculum for an Undergraduate Summer Clinical Research Program in Quantitative Sciences by Kay See Tan, Mengmeng Du, Jaya Satagopan and Elena Elkin in Journal of Medical Education and Curricular Development

Footnotes

Acknowledgments

The authors wish to thank Christy Rajcoomar for their editorial support.

Author Contributions

All authors contributed to the study conception and design. Material preparation and data collection were performed by all authors. Jaya Satagopan and Kay See Tan contributed to data analysis. The first draft of the manuscript was written by Kay See Tan, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript and agree to be accountable for all aspects of the work.

DECLARATION OF CONFLICTING INTERESTS

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

FUNDING

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The QSURE program is supported by the National Cancer Institute of the National Health Institutes (R25 CA214255). This work was also supported by the National Institutes of Health/National Cancer Institute Cancer Center Support Grant P30 CA008748 (to Memorial Sloan Kettering Cancer Center). These funding organizations did not play any roles in the survey's design, implementation, and analysis.

ORCID iDs

Kay See Tan

Jaya Satagopan

Supplemental material

Supplemental material for this article is available online.

References

National Academies of Sciences E, Medicine, Policy, Global A, Committee on Science EM, Public P, Committee on Responsible S. Fostering Integrity in Research. National Academies Press (US); 2017. Copyright 2017 by the National Academy of Sciences. All rights reserved. 2017.

OIG Review of Institutions’ Implementation of NSF’s Responsible Conduct of Research Requirements.Office of Inspector General. 2017.

ALLEA. The European Code of Conduct for Research Integrity – Revised Edition 2023. ALLEA; 2023.

NOT-OD-10-019 Update on the Requirement for Instruction in the Responsible Conduct of Research; 2011.

Jones

Lerner

. Implementing a course-based undergraduate research experience to grow the quantity and quality of undergraduate research in an animal science curriculum1. J Anim Sci. 2019;97(11):4691-4697. doi:https://doi.org/10.1093/jas/skz319

Krim

Cote

Schwartz

, et al. Models and Impacts of science research experiences: A review of the literature of CUREs, UREs, and TREs. CBE Life Sci Educ. Dec 2019;18(4): ar65, 1–14. doi:https://doi.org/10.1187/cbe.19-03-0069

University BCoEUitR. Reinventing Undergraduate Education: A Blueprint for America's Research Universities. State University of New York at Stony Brook; 1998.

Boyer

. Reinventing Undergraduate Education: Three Years after the Boyer Report. The Boyer Commission on Educating Undergraduates in the Research University; 2002.

Hunter

Laursen

Seymour

. Becoming a scientist: the role of undergraduate research in students’ cognitive, personal, and professional development. Sci Educ. 2007;91(1):36-74.

10.

Kardash

. Evaluation of undergraduate research experience: perceptions of undergraduate interns and their faculty mentors. J Educ Psychol. 2000;92(1):191.

11.

Tan

Elkin

Satagopan

. A model for an undergraduate research experience program in quantitative sciences. J Stat Data Sci Educ. 2022;30(1):65-74.

12.

Tan

Koppenaal

Lewis

Begg

. Adapting an undergraduate summer internship to a virtual format: implementing a mentored cancer research experience to meet rising demand for flexible learning environments. J Cancer Educ. Apr 2023;38(2):600-607. doi:https://doi.org/10.1007/s13187-022-02160-0

13.

National Academy of Sciences NAoE, Institute of Medicine Committee on Science E, Public P. On Being a Scientist: A Guide to Responsible Conduct in Research: Third Edition. National Academies Press (US) Copyright 2009 by the National Academy of Sciences; 2009. All rights reserved.

14.

Steneck

. ORI Introduction to the Responsible Conduct of Research. Dept. of Health and Human Services, Office of the Secretary, Office of Public Health and Science. Office of Research Integrity. 2003; p. 138.

15.

Skloot

. The Immortal Life of Henrietta Lacks. Broadway Paperbacks. CROWN; 2017.

16.

Shanahan

Ackley-Holbrook

Hall

Stewart

Walkington

. Ten salient practices of undergraduate research mentors: a review of the literature. Mentor Tutor: Partnership Learning. 2015;23(5):359-376.

17.

Chronology of Cases Involving Unethical Treatment of Human Subjects. University of New Hampshire. https://ori.hhs.gov/education/products/unh_round1/www.unh.edu/rcr/HumSubj-GoToChronology.htm

18.

IRB Guideline V - Research using Penn State students as participants. https://www.research.psu.edu/irb/policies/guideline5

19.

Ethical guidelines for statistical practice. https://www.amstat.org/your-career/ethical-guidelines-for-statistical-practice

20.

Munafò

Nosek

Bishop

, et al. A manifesto for reproducible science. Nature Human Behaviour. 2017;1(1):1-9. doi:https://doi.org/10.1038/s41562-016-0021

21.

Research Ethics & Compliance. https://research-compliance.umich.edu/data-security-guidelines

22.

Research data security. https://ria.princeton.edu/research-data-security

23.

How to Use ChatGPT’s Advanced Data Analysis Feature - MIT Sloan Teaching & Learning Technologies. https://mitsloanedtech.mit.edu/ai/tools/data-analysis/how-to-use-chatgpts-advanced-data-analysis-feature/

24.

Kelly

Sadeghieh

Adeli

. Peer review in scientific publications: benefits, critiques, & a survival guide. Ejifcc. 2014;25(3):227.

25.

Twombly

. Lung Cancer Screening Trial Financed by Tobacco-Funded Foundation, Sparks Debate. Oxford University Press; 2008.

26.

BMC Medicine. BioMed Central. https://bmcmedicine.biomedcentral.com/submission-guidelines/peer-review-policy

27.

CRediT author statement. https://www.elsevier.com/researcher/author/policies-and-guidelines/credit-author-statement. Accessed on March 28, 2024.

28.

Likert

. A technique for the measurement of attitudes. Archives Psychol. 1932;22:5-55.

29.

Sharma

Minh Duc

Luu Lam Thang

, et al. A consensus-based checklist for reporting of survey studies (CROSS). J Gen Intern Med. 2021 Oct;36(10):3179-3187. doi:https://doi.org/10.1007/s11606-021-06737-1

30.

Chen

K-C

Hester

. A dramatized method for teaching undergraduate students responsible research conduct. Account Res. 2023;30(3):176-198.

31.

Kraus

. You must participate: violating research ethical principles through role-play. College Teaching. 2008;56(3):131-136. doi:https://doi.org/10.3200/CTCH.56.3.131-136

32.

Grasse

Melcer

Kreminski

Junius

Wardrip-Fruin

. Improving undergraduate attitudes towards responsible conduct of research through an interactive storytelling game. 2021:1-8.

33.

New infographic: 5 Ways Supervisors Can Promote Research Integrity. https://ori.hhs.gov/blog/new-infographic-5-ways-supervisors-can-promote-research-integrity. Accessed on March 28, 2024.

34.

Kirkpatrick

. Evaluating Training Programs: The Four Levels. 3rd ed. Berrett-Koehler Publication; 2006.

35.

Loui

Revelo

. Cooperative learning and assessment of ethics sessions in a summer undergraduate research program. CURQ. 2015;36(1):4-10.

36.

Hirsch

Linsenmeier

Smith

Walker

. Enhancing core competency learning in an integrated summer research experience for bioengineers. J Eng Educ. 2005;94(4):391-401.

37.

Steneck

. ORI Introduction to the Responsible Conduct of Research. Department of Health and Human Services. Office of the Secretary; 2003.

38.

Keefer

Wilson

Dankowicz

Loui

. The importance of formative assessment in science and engineering ethics education: some evidence and practical advice. Sci Eng Ethics. 2014;20:249-260.

39.

Chen

. Researchers around the world prepare to# ShutDownSTEM and ‘strike for black lives’. Science. 2020:9.

40.

Stein

. Troubling history in medical research still fresh for black Americans. Natl Public Radio, All Things Consid. 2017.

41.

Geneviève

Martani

Shaw

Elger

Wangmo

. Structural racism in precision medicine: leaving no one behind. BMC Med Ethics. 2020;21:1-13. doi:https://doi.org/10.1186/s12910-020-0457-8

42.

Kirch

Addressing racism and mistreatment in academic medicine. AAMC; 2022. Available: https://www.aamc.org/news/addressing-racism-and-mistreatment-academic-medicine. Accessed on March 28, 2024.

43.

Harmon

. Indian tribe wins fight to limit research of its DNA. New York Times. 2010;21:2010.

44.

Obermeyer

Powers

Vogeli

Mullainathan

. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447-453.

45.

Vyas

Eisenstein

Jones

. Hidden in plain sight—reconsidering the use of race correction in clinical algorithms. Mass Medical Soc. 2020;383(9):874-882.

46.

Jain

Brooks

Alford

, et al. Awareness of racial and ethnic bias and potential solutions to address bias with use of health care algorithms. Am Med Assoc. 2023;4(6):e231197-e231197.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.30 MB