Sage Journals: Discover world-class research

Abstract

Objective:

We aimed to conduct a systematic review to identify curricular and educational interventions to build research competency among Canadian psychiatry residents and fellows transitioning to the competency-by-design framework.

Methods:

The PRISMA guidelines were followed, searching five databases from their inception to February 2023 for relevant evaluation-type studies exploring research competency among psychiatry residents and fellows. We appraised thestudy’s quality using the Joanna Briggs Institute’s risk of bias tool for observational designs.

Results:

Overall, 36 original articles met our inclusion criteria. Surveys (n = 10) showed that participation in scholarly research, quality improvement, or educational projects relevant to psychiatry is needed in most residency programs. However, these vary significantly across programs; few need direct research experience for residency completion. The interventions spanned four categories: externally funded comprehensive research training programs (n = 5); resident research tracks (n = 11); workshops and seminars (n = 7); and specific modules (n = 3). Reported outcomes included overall program ratings, research output, and career trajectory. The quality of most studies was low because of the lack of controls or validated metrics for evaluating outcomes.

Conclusions:

While many studies have explored best practices in research curricula, the current literature does not inform competency-based models for Canadian psychiatry residency programs incorporating research training requirements. Further description is needed from Canadian psychiatric training bodies regarding appropriate curricula, milestones, and metrics for evaluating research competencies.

Keywords

Internship and residency quality improvement curriculum psychiatry Canada education and training

Introduction

In 2001, the National Institute of Mental Health (NIMH) noted decreasing numbers of clinician scientists in psychiatry relative to other areas of medicine.^1–3 As a result, the NIMH commissioned the Institute of Medicine to investigate the decline of research training within psychiatry residency programs in the United States.^4–6 Consequently, research training has become an increasingly mandated component of the psychiatry residency experience in the United States to address the critical shortage of research psychiatrists.

In recent years, the landscape of psychiatric residency training in Canada has undergone a significant transformation with the adoption of competency by design (CBD). The Royal College of Physicians and Surgeons of Canada (RCPSC) has emphasized the importance of research, dedicating a CanMEDS role to scholarly development. It has specifically outlined requirements during psychiatric residency training (Appendix 1). Within the CanMEDS Scholar role, psychiatry residents must understand the scientific principles of research and scholarly inquiry; contribute to the creation and dissemination of knowledge and procedures applicable to health; describe the principles of research, scholarly inquiry, and research ethics; pose a scholarly question; conduct a systematic search for evidence; select and apply appropriate methods to address the question; and disseminate the findings of a study.

As of 1 July 2020, Canadian psychiatry residency programs shifted to CBD,⁷ which aims to prepare psychiatrists for practice by orienting training to competencies derived from the needs of society and patients.⁸ To operationalize CBD, the RCPSC has adopted “entrustable professional activities” (EPAs),⁹ which are units of professional practice entrusted to trainees demonstrating the necessary competence to execute the activity independently.¹⁰ Working groups in psychiatry have defined 20 EPAs across four stages of residency training, with some for specific psychiatric subspecialties.^11–34 Although the RCPSC has described the research requirements and emphasized research as an essential component of psychiatric training,^35,36 the only EPA related to research in the current CBD framework is early in training within the Foundations of Discipline (FoD) stage (EPA #5), which specifies that two direct observations from at least two different observers are required of the resident to “perform a critical appraisal and presenting psychiatric literature.” Key features of this EPA include: (1) “A focus on critical appraisal of literature to make appropriate clinical decisions and to encourage lifelong learning and acquisition of new knowledge and skills in the specialty”; (2) “Includes posing a clinically relevant question, performing a literature search, critically appraising the literature, and presenting in a group setting”; and (3) “Includes presentations such as grand rounds, journal club, case conference, morbidity and mortality rounds or quality improvement rounds.”

As CBD has progressed in Canadian psychiatry residency training, different program directors and competency chairs have noted issues with some of the EPAs and have been providing feedback to the Royal College. As the original cohort moves to different stages of their training, there is mounting pressure on programs to ensure that their residents achieve the current EPAs. However, the curriculum has become even more packed with the Royal College exam moving to the end of PGY4. Consequently, it may be even more difficult to address research competencies adequately. As EPAs tend to focus more on clinical skills that all generalist psychiatrists need, it is unlikely that there will be an expansion to have more research EPAs. However, other EPAs in the current CBD curriculum for psychiatry residents inherently touch on research competencies. For example, in the Core of Discipline stage, another EPA focuses on “teaching for students, residents, the public and other health care professionals.” A key feature of this EPA includes “critical appraisal of relevant literature,” but it is more applicable to presenting, where they also talk about adapting language and material to the audience’s needs and effective presentation skills.

Regrettably, the research-related EPAs provide little description or guidance on what meets the standard for individual evaluations on critical appraisal and presenting findings. Furthermore, there are no standards set for how preceptors who will evaluate this EPA must themselves have recognized skills in critical appraisal and conducting research, which could lead to unreliable assessments across sites and preceptors. There is also no description of what formal critical appraisal skills need to be demonstrated and what level of sophistication is required to meet the standard. Thus, the requirements of the EPA appear to fall short of what the RCPSC requires.^37,38

Despite the RCPSC’s recognition of the significance of research within CBD, the implementation and evaluation of research competencies in psychiatry residency programs remain complex and multifaceted. To address this, we conducted a systematic review of educational interventions related to research for psychiatry residents and fellows. The primary goal of this review is to clarify the requirements and standards for critical appraisal and presentation skills within the CBD framework. By evaluating and synthesizing existing educational interventions, we aim to provide guidance and support to Canadian psychiatric residency training programs, helping them develop a more consistent and effective approach to meeting the CanMEDS Scholar role’s research-related expectations. This systematic review, therefore, plays a crucial role in enhancing the competency of psychiatry residents in research skills within the evolving CBD framework.

Methods

Protocol and registration

To provide a comprehensive synthesis of educational interventions related to research for psychiatry residents and fellows, we conducted a systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines.³⁹ Our systematic review protocol was registered with PROSPERO (CRD42021244124). We have included a checklist with the relevant PRISMA items in Appendix 2.

Eligibility criteria

Eligibility criteria for the included studies were established based on the population-intervention-comparison-outcome-study design (PICOS) framework, which guided our selection process⁴⁰:

Population: Psychiatry residents and fellows.

Intervention: Research-oriented educational interventions, such as courses, curricula, modules, self-assessment tools, or competency frameworks.

Comparator: Not required.

Outcomes: Psychiatry research competencies as per the RCPSC guidelines (Appendix 1).

Study design: We considered any evaluative study (e.g., pre-post, randomized controlled trials, surveys); we excluded commentaries, editorials, and review articles.

Search strategy

After meeting with an experienced research librarian, we developed a systematic review protocol involving the following four databases on 16 March 2021, with an updated search on 14 February 2023, with no articles found in the next interval: EMBASE, MEDLINE, PsycINFO, and ERIC (Appendix 3). In addition, we supplemented the search by examining the reference lists of the eligible studies.

Selection of studies

Following the removal of duplicates, one coauthor (AB) completed two independent rounds of screening (first by title and abstract and then by full-text review).

Data extraction and management

We used Cochrane’s Covidence, a web-based systematic review manager, to extract information from each study. We contacted the study authors for data confirmation or clarification where necessary; we resolved disagreements by consensus. We pulled the following data items, aligned with the PICOS framework: study author, study design, year and location of study, sample size and demographic characteristics (e.g., age, sex, and stage of training), research aspect explored, nature and type of intervention, other outcome measures, and summary of the study’s findings.

Assessment of risk of bias in the included studies

Recent reviews have indicated that serious concerns arise from applying scales and instruments that have not been validated to evaluate the risk of bias in observational studies’ systematic reviews.^41–44 Therefore, we assessed the study’s quality using the Joanna Briggs Institute’s (JBI) risk of bias tool for observational studies.^45,46 In brief, the JBI checklist appraised eight domains of study quality in observational designs, including the description of study participants, study eligibility criteria, methods for measuring and defining exposure and outcome variables, accounting for confounding factors, and choice of analytic techniques. Each of these eight domains receives a 0 or 1, with higher scores indicating better quality. Studies receiving 6 or more points were designated “High quality”; otherwise, they were “Low quality.”

Synthesis of the findings

We undertook a descriptive synthesis as a meta-analysis was not feasible due to excessive heterogeneity in the study design and outcome reporting, including overall program ratings (e.g., participant satisfaction, confidence with a skill), research output (e.g., publications, awards, grant funding, citations), and research career trajectory (e.g., transition or matriculation to further research training, academic positions, affiliation with professional bodies). However, we identified themes from surveys on psychiatry residents that provided a context for attitudes and perceived barriers to research. In addition, we identified potential strategies that residency programs could employ to enhance research competency with their residents from intervention-type studies.

Results

Study selection

From 318 records, 36 original articles met our inclusion criteria (Figure 1). Ten surveyed general attitudes, experiences, and reflections on research training during residency (Table 1).^47–56 The remaining 26 studies evaluated interventions designed to enhance research competencies or skills in specific techniques (Table 2).^57–82

Figure 1.

PRISMA study flow diagram.

Table 1.

Characteristics of the survey studies (n = 10).

Study	Design and sample	Summary of findings
Hershberg et al.⁴⁷	A national survey of Canadian psychiatric residents (n = 190, response rate not reported)	Residents identified research as an essential part of psychiatry, and most welcomed the idea of formal training. However, two-thirds felt that their departments were underproductive in terms of research
Strauss et al.⁴⁸	A national survey of U.S. psychiatry residents (n = 115, 50% response)	Although most programs offered a research activity (e.g., journal club), minimal research resources were available to residents. The availability of research-specific teaching was highly variable, and few programs were budgeting for resident research training. Few programs require any direct research experience for residency completion
Andrews et al.⁴⁹	A national survey of Australian psychiatry training directors (n = 20, 87% response)	Most directors reported offering formal research seminars, lectures, and journal clubs for residents, but only two programs formally required trainees’ participation in research
Balon and Singh⁵⁰	A national survey of Canadian and U.S. psychiatry residency programs (n = 70, 59% response)	There were few research training opportunities across programs. Research didactics were relatively short, with trainees reporting a lack of education on research designs. In addition, trainees were rarely required to submit a research protocol or start a project for residency completion
Fitz-Gerald et al.⁵¹	A national survey of the American Association of Directors of Psychiatry Residency Training and chief residents (n = 399, 49% response)	The major finding of the survey pointed to a discrepancy between attitudes toward research, the need for research training requirements, and the availability of research resources. For example, 98% of respondents indicated that research training should be provided to residents, but only 32% thought that research should be required
Winter et al.⁵²	A regional survey of Wessex psychiatry specialist registrars (n = 26, 77% response)	Most (77%) trainees developed research training plans with an identified mentor, and most (91%) received protected research time. Of the 24 trainees undertaking research, 88% felt adequately supported, 83% were interested in their research, 67% enjoyed it, and 88% gained research knowledge/skills
Silberman et al.⁵³	A national survey of U.S. psychiatry senior residents from programs with class sizes of at least 10 (n = 127, 67% response)	Compared to residents with a low or medium research interest, residents with higher interest were more likely to be male, have graduate degrees, have previous research training experience, have greater research activity in residency, and carry lower debt
Shanmugalingam et al.⁵⁴	A national survey of Canadian resident representatives (n = 15, 88% response)	Most programs implemented scholarly activity for residency completion and created guidelines on resident research requirements. However, the specific requirements varied widely, as did research resources (e.g., protected time, mentorship, funding)
Laliberté et al.⁵⁵	A national survey of Canadian psychiatry residents (n = 207, 24% response)	Research interest decreased by 76% by the 5th year of residency across programs. There was no association between research track participation and improved research interests
MacMaster et al.⁵⁶	A national survey of Canadian psychiatry residents and directors (n = 119, 13% response)	Nearly all (87%) rated research as essential in informing clinical practice, but only 29% reported program satisfaction. In addition, only 22% were enthusiastic or very enthusiastic about participating in research

U.S.: United States.

Table 2.

Characteristics of the evaluation studies (n = 26).

Study	Design and sample	Intervention	Summary of findings
Fisher and Bender⁵⁷	Evaluation of a NIMH-funded research training program for senior Boston University Medical Centre psychiatry residents (n = 19)	The authors describe a 2-year program available to senior psychiatry residents, emphasizing basic and clinical research skills training, research activity, supervision, projects, seminars, workshops, and presentations	Program participants remained active in research, published extensively, and assumed substantial teaching responsibilities in medical schools
Tuma et al.⁵⁸	Evaluation of research mentorship workshops for select U.S. child psychiatry fellows (n = 34)	The authors describe three NIMH-funded 3-day workshops for fostering research among selected U.S. child psychiatry trainees. The workshops provided intensive mentorship and discussions with senior investigators in the field	Nearly 50% of trainees completed formal research training and remained actively involved in research at least one day a week. Trainee-led publications increased over time
Belmont et al.⁵⁹	Evaluation of a model for research training for U.S. child psychiatry fellows (sample size not reported)	The model involved a 4-month research rotation and a year of biweekly research meetings. During the research rotation, residents learned research methods and participated in a long-term research project	The training encouraged residents’ future research activities. As a result, program participants remained active in research, published extensively, and assumed substantial teaching responsibilities in medical schools
Balon and Kuhn⁶⁰	Evaluation of two innovative research training programs for Wayne State University psychiatry trainees (n = 9)	The authors developed, implemented, and evaluated a summer psychiatry research fellowship and a 2-year Master of Science in Psychiatry Program, which provided protected time for graduate coursework, work on a project, mentorship, and a graduate degree	Program participants remained active in research, published extensively, and assumed substantial teaching responsibilities in medical schools. They were also successful in terms of grants, awards, and citations
Halpain et al.⁶¹	Evaluation of a summer research training program for University of California San Diego psychiatry trainees (n = 123)	An annual week-long research training program followed by ongoing mentorship for selected trainees. Sessions covered scientific integrity, participant recruitment, grantsmanship, and work-life balance	Nearly 90% of attendees presented and published articles within a year of attendance. Over 50% received grant funding within 18 months. Almost 80% remained in academic positions
Gilbert et al.⁶²	Evaluation of a research mentorship track for psychiatry residents at the University of Pittsburgh (n = 33)	The university developed a research track for its residents to address the critical need for training physician-scientists in psychiatry, including protected time and mentorship from faculty members	Trainees received 50%–75% protected research time. As a result, nearly 50% became research postdocs or full-time grant-funded researchers in academic positions
Kunik et al.⁶³	Evaluation of a regional research training program for psychiatry residents at the South-Central Mental Illness Research, Education, and Clinical Center (n = 26)	A low-cost, innovative regional program to train psychiatry residents in research and attract them to academic careers, providing mentorship and exposure to seasoned researchers, didactic coursework, and a stipend to cover educational expenses	Program highlights included improved research enthusiasm, the increased representation of minority groups, six publications, 28 academic presentations, and 50% retention in academic research careers
Löwe et al.⁶⁴	Controlled evaluation of a 1-year clinical research training for German residents (n = 15); controls were 22 residents not participating in the training program	The training program included a weekly class in clinical research methods, a research project, and mentorship. Outcomes included methodological research knowledge (multiple-choice progress test), self-assessed research competence, progress on publications and grant applications, and quantitative-qualitative program evaluations	Methodological knowledge, self-assessed research competence, and publication activity improved significantly more in the intervention group compared to the control group. The intervention subjects evaluated the training program as valuable for becoming independent researchers
Roane et al.⁶⁵	Evaluation of the PART program for Beth Israel Medical Centre psychiatry residents (n = 89)	The PART program addressed the shortage of research psychiatrists by providing funding, protected research time, and mentorship to residents using an experiential research model	Following program inception, scholarly activities increased over a 5-year observation period (e.g., 32 publications, 56 presentations, and NIMH awards)
Dervic et al.⁶⁶	Evaluation of a research skills training program for Austrian child and adolescent psychiatry residents (sample size not reported)	The program provided 11 seminars during the academic year, including research/educational career planning, research methods, skills acquirement, ethics committee applications, and grant proposals. As well they offered mentorship and project opportunities	The first responses from the residents were positive, indicating overall satisfaction with the program. However, more detailed feedback or outcomes were not available
Back et al.⁶⁷	Evaluation of the DART program Medical University of South Carolina psychiatry residents (sample size not reported)	DART, an NIH-funded research track, provides funding, training, and mentorship to train residents in all phases of research—didactic seminars covering skill development, dissemination, responsible conduct, conferences, and grantsmanship	Most DART graduates reported high satisfaction with the program components and pursued academic careers to continue research activities
Balon et al.⁶⁸	Evaluation of a research colloquium for junior psychiatry researchers from the United States and abroad (n = 377)	The research colloquium, funded by the APA Committee on Research Training, offers a 2-day research conference, pairing trainees with mentors and providing a forum for research presentations. They also provide the trainees with longitudinal follow-up and a stipend to help offset travel costs	Broadly, colloquium participants reported a positive overall satisfaction with the program, high transition to research-based careers, extramural funding from various sources, and high numbers of papers and presentations
Duthie et al.⁶⁹	Evaluation of an annual research symposium for Scottish psychiatric trainees (n = 22)	Authors surveyed symposium attendants, compiled a list of all presentations to the symposium from 2007 through 2009, and searched major research databases for participants’ publication activity	Nearly 50% of research projects presented at the symposium achieved eventual publication. In addition, feedback from symposia attendees was almost universally positive
Hipolito et al.⁷⁰	Evaluation of a summer research training program for Dartmouth and Howard Universities’ psychiatry residents (n = 77)	The program received a 5-year grant to explore mental health treatment and research cultural factors. The authors conducted questionnaires to evaluate participant experiences	Participants reported overall satisfaction and enhanced confidence in dealing with minority patients and the diverse factors affecting their research treatment
Mezzacappa et al.⁷¹	Evaluation of a research scholarship program for U.S. child and adolescent fellow (n = 52)	The program included protected time for scholarly pursuits and a mentorship program to support academic interests	After implementing the program changes, there was a tenfold increase in the number of residents engaged in scholarly pursuits
Tsai et al.⁷²	Evaluation of a research training program for University of California psychiatry residents (n = 48)	Program components include mentorship, individualized training plans, clinical and research experiences, protected research time, and funding	More than 80% matriculated to postdoctoral research fellowships, irrespective of previous training
Zisook et al.⁷³	Evaluation of AADPRT-led research training conferences (n = 284)	AADPRT directors led several full-day seminars to increase residents’ competence in evidence-based medicine, research literacy, and knowledge translation	Feedback from the program’s first 5 years documented excellent attendance, perceived satisfaction, and material usefulness
Bhat et al.⁷⁴	Description and informal evaluation of three Canadian RT programs (n = 44)	Three RT programs at the University of British Columbia, the University of Toronto, and McGill University provided participants with protected time, seminars, opportunities to pursue graduate degrees, and funding for research	Although formal outcomes were not reported, this study provides the only published research on Canadian RT programs. In addition, the authors outline some of the issues with psychiatry resident research documentation in the literature
Goldstein et al.⁷⁵	Evaluation of a NIMH-funded research training program for U.S. psychiatry residents from minority populations (n = 99)	The PMRTP aimed to support psychiatric investigators from underrepresented populations in developing and maintaining research careers	Nearly 70% reported publishing their research findings, and 64% received post-training research grants; most pursued academic careers
Posporelis et al.⁷⁶	Evaluation of two research training programs for Johns Hopkins psychiatry residents (n = 11)	The two training curricula separately targeted first-year and 2nd-to-4th-year psychiatry residents, involving protected research time, mentorship, and seminars	Overall, participants reported satisfaction with both programs and increased interest in academic research careers after exposure to the curricula
Himelhoch et al.⁷⁷	Evaluation of a systematic review and meta-analysis course for University of Maryland psychiatry residents (n = 54)	The authors developed, implemented, and evaluated an interactive, web-enhanced course to provide tools for conducting and reporting systematic reviews and meta-analyses	There was overall participant satisfaction and improved confidence with reviews. Academic productivity included 11 presentations, four publications, and two awards
Aftab et al.⁷⁸	Evaluation of a resident-led research newsletter for Cleveland psychiatry residents (n = 29)	The authors developed, implemented, and evaluated a resident-led psychiatry research newsletter (“Research Watch”) to showcase resident research activity	Nearly 50% reported improved interest in scholarly activities, current psychiatric research, and participation in research projects
Besterman et al.⁷⁹	Evaluation of a biennial research retreat for California psychiatry residents (n = 65)	The authors developed, implemented, and evaluated a biennial psychiatry resident research retreat to enhance research networking and mentorship	Overall participant satisfaction and improvement over consecutive years of the retreat, particularly for networking sessions
Campbell et al.⁸⁰	Evaluation of the DART program for Medical University of South Carolina psychiatry residents (n = 132)	DART, an NIH-funded research track, provides funding, training, and mentorship. Didactic seminars covered research skills, manuscript writing, dissemination, ethics, and grantsmanship	Most DART graduates reported high satisfaction with the program components and pursued academic careers to continue research activities
Viswanath et al.⁸¹	Evaluation of an ethics training module for the Indian NIMH psychiatry residents (n = 25)	The authors developed, implemented, and evaluated a new research ethics training module for psychiatry residents (“The Five-Tier Approach”)	Participants demonstrated an adequate understanding of ethical concepts, including autonomy, benefits, and justice
Calhoun et al.⁸²	Controlled evaluation of an integrated research training program for Yale child psychiatry trainees (n = 36)	The AJSP aimed to prepare child psychiatry clinician scientists by providing dedicated funding, protected research time, and mentorship to trainee participants	Compared to controls, AJSP participants were more likely to hold AACAP certification and have more awards, publications, and grants

AACAP: American academy of child and adolescent psychiatry; AADPRT: American association of directors of psychiatric residency training; AJSP: Albert J. Solnit integrated training program; APA: American psychiatric association; DART: drug abuse research training; NIMH: national institute of mental health; PART: psychiatrists acquiring research training; PMRTP: program for minority research training program in psychiatry; RRTP: resident research training program; RT: research track; U.S.: United States.

Study quality

Using the JBI instrument, 34 of the 36 studies received low-quality ratings (Table 3). Most intervention-type studies reported numerous positive outcomes (e.g., grants, publications, matriculation to research roles) based on retrospective analyses but without any formal comparison to baseline productivity before implementing the program or to an external comparison group; only two of the studies^64,82 included a control group.

Table 3.

Joanna Briggs Institute’s risk of bias assessments.

Study	Were the criteria for inclusion in the sample clearly defined?	Were the study subjects and the setting described in detail?	Was the exposure measured validly and reliably?	Were objective, standard criteria used for the measurement of the condition?	Were confounding factors identified?	Were strategies to deal with confounding factors stated?	Were the outcomes measured validly and reliably?	Overall appraisal
Hershberg et al.⁴⁷	Yes	Yes	No	N/A	Yes	No	No	Low
Strauss et al.⁴⁸	Yes	Yes	No	N/A	Yes	No	No	Low
Andrews et al.⁴⁹	Yes	Yes	No	N/A	Yes	No	No	Low
Balon and Singh⁵⁰	Yes	Yes	No	N/A	Yes	No	No	Low
Fitz-Gerald et al.⁵¹	Yes	Yes	No	N/A	Yes	Yes	No	Low
Winter et al.⁵²	Yes	Yes	No	N/A	Yes	No	No	Low
Silberman et al.⁵³	Yes	Yes	No	N/A	Yes	No	No	Low
Shanmugalingam et al.⁵⁴	Yes	Yes	No	N/A	Yes	No	No	Low
Laliberté et al.⁵⁵	Yes	Yes	No	N/A	Yes	Yes	No	Low
MacMaster et al.⁵⁶	Yes	Yes	No	N/A	Yes	No	No	Low
Fisher and Bender⁵⁷	Yes	Yes	No	N/A	Yes	No	No	Low
Tuma et al.⁵⁸	Yes	Yes	No	N/A	Yes	No	No	Low
Belmont et al.⁵⁹	Yes	Yes	No	N/A	Yes	No	No	Low
Balon and Kuhn⁶⁰	Yes	Yes	No	N/A	Yes	No	No	Low
Halpain et al.⁶¹	Yes	Yes	No	N/A	Yes	No	No	Low
Gilbert et al.⁶²	Yes	Yes	No	N/A	Yes	No	No	Low
Kunik et al.⁶³	Yes	Yes	No	N/A	Yes	No	No	Low
Löwe et al.⁶⁴	Yes	Yes	Yes	N/A	Yes	Yes	Yes	High
Roane et al.⁶⁵	Yes	Yes	No	N/A	Yes	No	No	Low
Dervic et al.⁶⁶	Yes	Yes	No	N/A	Yes	No	No	Low
Back et al.⁶⁷	Yes	Yes	No	N/A	Yes	No	No	Low
Balon et al.⁶⁸	Yes	Yes	No	N/A	Yes	No	No	Low
Duthie et al.⁶⁹	Yes	Yes	No	N/A	Yes	No	No	Low
Hipolito et al.⁷⁰	Yes	Yes	No	N/A	Yes	No	No	Low
Mezzacappa et al.⁷¹	Yes	Yes	No	N/A	Yes	No	No	Low
Tsai et al.⁷²	Yes	Yes	No	N/A	Yes	No	No	Low
Zisook et al.⁷³	Yes	Yes	No	N/A	Yes	No	No	Low
Bhat et al.⁷⁴	Yes	Yes	No	N/A	Yes	No	No	Low
Goldstein et al.⁷⁵	Yes	Yes	No	N/A	Yes	No	No	Low
Posporelis et al.⁷⁶	Yes	Yes	No	N/A	Yes	No	No	Low
Himelhoch et al.⁷⁷	Yes	Yes	No	N/A	Yes	No	No	Low
Aftab et al.⁷⁸	Yes	Yes	No	N/A	Yes	No	No	Low
Besterman et al.⁷⁹	Yes	Yes	No	N/A	Yes	No	No	Low
Campbell et al.⁸⁰	Yes	Yes	No	N/A	Yes	No	No	Low
Viswanath et al.⁸¹	Yes	Yes	No	N/A	Yes	No	No	Low
Calhoun et al.⁸²	Yes	Yes	Yes	N/A	Yes	Yes	Yes	High

Synthesis of findings from the survey studies

While participation in scholarly research, quality improvement, or educational projects relevant to psychiatry is required in most residency programs, requirements vary significantly. Few require any direct research experience for residency completion. For example, a 2014 survey of representatives of all 17 accredited Canadian residency programs found that among the 11 programs mandating research requirements, qualifying activities ranged from fully independent resident-led research projects to quality improvement initiatives, assisting with faculty research, or simply presenting research.⁵⁴ However, none required publication of the resident’s final work.⁵⁴

Although most programs offer some form of research activity (e.g., at least presenting an article at a journal club), earlier survey studies demonstrated minimal research resources were available to residents.^47,48,50,51 The research infrastructure for psychiatry residents—exposure to psychiatry research, availability of resources, training, protected time, and dedicated funding—varied substantially across programs.^54–56 Epidemiological methods, critical appraisal skills, ethical issues, and research design were the most commonly taught topics in didactic resident seminars.⁴⁹ Residents and training directors consistently identified the need for protected research time and support as the most valued resources.^54–56 Agreeing on a research training plan with a mentor and regularly reviewing support and supervision through a newly developed mentoring process was associated with high trainee satisfaction levels.⁵² While early exposure to research was initially thought to promote future research interest,⁵⁵ two national surveys of Canadian psychiatry residents have challenged this assumption, suggesting that most residents lose interest in research by the time they graduate and that participation in a research track does not appear to enhance research interest.^55,56

In summary, survey studies indicated that while most programs offered some research activity, the availability of resources (e.g., protected time), the content of didactic seminars, and the extent of research requirements varied substantially across programs. There were also challenges with enhancing research interest among residents, particularly those who lack pre-residency research training experience.

Synthesis of findings from the intervention studies

Across studies, several identified various mechanisms by which psychiatry residents could receive additional research training. At the highest end, we identified several externally funded, comprehensive research training programs which offer formal training opportunities (i.e., graduate degrees and fellowships) integrated with psychiatry residency training. Most of these higher-end programs are available to U.S.-based psychiatry residents and are funded through the National Institutes of Health (NIH) or the National Institutes of Mental Health (NIMH). These programs generally demonstrated good outcomes and appeared to improve the scientific workforce, promote future research involvement, and generate awards, publications, and grants.^{57,67,75,80,82} For example, Yale’s Albert J. Solnit Integrated Training Program (AJSP), a 15-year NIMH-funded initiative, improved future professional board affiliation and generated more awards, publications, and grant funding than a control group.⁸² In addition, Canadian psychiatry residents can apply for a similar offering through the RCPSC’s Clinician Investigator Program (CIP), which provides at least 2 years of protected research time and salary support, enabling residents to pursue a thesis-based graduate degree alongside their clinical training.⁸³ The CIP has been very successful, and most graduates who have gone through the program have become quite involved in research—although not all.^84,85

In contrast to these formal programs, several locally developed research tracks offered more flexible, individualized, and innovative options to facilitate resident entry into research careers.^{59,60,62–66,71,72,74,76} For instance, shorter research training opportunities, such as intensive summer research institutes or symposia, targeted residents with less formal research interests, with attendants reporting high satisfaction.^{58,61,68–70,73,79} Similarly, 3-day NIMH-funded psychiatry research conferences, which included morning plenary sessions and afternoon small-group teaching sessions, appeared to consolidate knowledge and provide practical research skills for resident participants.⁸⁶ For child and adolescent psychiatry trainees, designated research workshops encouraged child psychiatrists who wished to pursue a research career and effectively encouraged trainee research.⁸⁷

There were also three miscellaneous research training interventions. The first was a novel online research ethics training module for psychiatry residents, which incorporated five modules for teaching psychiatry residents about the basics of ethical concepts, including autonomy, benefits, and justice.⁸¹ The second was a systematic review and meta-analysis course, developed using an interactive, web-enhanced module that gave trainees an overview of the essential tools for conducting and reporting systematic reviews and meta-analyses, overall participant satisfaction, and improved confidence with reviews.⁷⁷ Finally, a resident-led research newsletter showcasing resident research activity was associated with a nearly 50% improvement in research interest and participation in research projects among resident participants.⁷⁸

Given the lack of evidence in the literature, we felt that including some expert opinion could be worthwhile, and consequently, offering some practical advice could be helpful. Based on these findings and our own experience, we compiled a list of resources, strategies, and “tips” for psychiatry residents interested in research (Appendix 4).

Discussion

Summary of findings

The present study is the first comprehensive review of research training for psychiatry residents. Our review found 36 studies describing the state of psychiatry resident research opportunities and evaluating diverse research training methods. Formal research training programs enhanced the scientific workforce consistently, promoted future research involvement, and generated awards, publications, and grants. However, as these programs might simply sustain the scholarship of residents with previous research experience and interest, they may not appeal to trainees with less research interest or those who foresee non-research-based careers.

While there is insufficient evidence to inform guidelines for organizing research training programs concerning national requirements, we offer suggestions based on the available information.⁷⁴ Different intervention approaches studied may help inform how Canadian residency training programs can develop a more uniform approach to meeting the CanMEDS Scholar role for research. To help set up a reliable standard, a pragmatic recommendation would be for all training programs to teach critical appraisal and presentation skills. As this EPA occurs early in training within the FoD stage, relevant teaching to achieve this EPA should co-occur during the corresponding phase of training (i.e., PGY1&2). For example, in-person research training workshops or journal clubs⁸⁸ or online training (i.e., the United Kingdom’s Critical Appraisal Skills Program) could address critical appraisal content using checklists to help evaluate critical appraisal skill competence when residents present.⁸⁹

In parallel, all Canadian psychiatry residency training programs should have a research requirement, yet only 11 of the 17 accredited programs reported including research requirements.⁵⁴ At a minimum, Canadian psychiatry residents should complete a review of a major psychiatric topic applicable to their future practice and present it formally at grand rounds. The proposed requirement would demonstrate expertise in an area residents plan to practice and distinguish them from their peers for practice placement.

In a parallel vein, it is interesting to contrast the training of psychiatry residents with related mental health professionals’ training. For example, a previous study compared the training of psychologists and psychiatrists, concluding that the latter have little formal training in studying human behavior and little contact with problems and methods in science. In contrast, the former has a more extensive background in studying human behavior and formal training in science.⁹⁰ To that end, another study suggested that there may be mutual benefit from academic collaboration across institutions to achieve their respective educational and training missions.⁹¹

Strengths and limitations

While augmenting the current research-oriented EPAs may be beyond the scope of this article, we believe this article has several strengths. First, based on existing literature, it describes the components and sophistication of critical appraisal and presentation skills that psychiatry residents need to meet for entrusting the related EPA. These descriptions provide Canadian programs with tangible suggestions for base program research requirements. Second, our review’s findings identify the importance of distinguishing research capability and development levels for different residents. Different tracks could be available (i.e., basic for the general clinician, enhanced for the academic clinician, and advanced for the clinician-scientist/research track academic).

However, a few limitations call for further discussion. First, as our search strategy focused on publications that focus on psychiatry, we excluded reviews on more general approaches. To address this, we included additional data on the RCPSC’s CIP from a 2011 review published by Hayward et al. (performed by the RCPSC 13 years after the institution of the CIP in 1995) and a 2010/2011 survey on Canadian CIP programs by Hayward et al.,⁸⁵ as these indirectly provided some coverage of the psychiatry programs that were involved. Finally, as the CIP has been a considerable success and some coauthors have had personal experience with this excellent RCPSC program, we discussed it in our review.

Second, despite the breadth of findings, the included studies’ quality was poor due to few controls and standardized measurements. Given the uncontrolled designs of most studies, the presented findings are highly susceptible to selection bias. Most research tracks and formal training programs cater to motivated trainees.⁵³ Even the two high-quality studies^64,82 had severe methodological limitations in selecting controls (i.e., these groups were not randomly allocated).

Third, no intervention studies or surveys addressed critical appraisal and presentation skills to inform Canadian psychiatry residency training programs about a standard to meet this EPA in the current CBD framework. There were few Canadian-based intervention studies, and most recommendations were extrapolated to Canadians.

A fourth limitation was the high heterogeneity in reported outcomes (precluding quantitative synthesis) and methods across studies—some studies not fully delineating the feasibility or effectiveness of their interventions and few measuring sustained competence over a long-term follow-up period were another limitation of our review.

Conclusions

In conclusion, our systematic review has shed light on the challenges of integrating research training within the CBD framework in psychiatry. The interventions described in the included studies primarily focused on cultivating productive, independent researchers, while CBD emphasizes the development of research basics, such as critical appraisal and research fundamentals, alongside clinical excellence.

Drawing insights from other successful CBD models, we propose that psychiatry research training could incorporate one or two general EPAs applicable to all residents, emphasizing foundational research competencies. Additionally, specialized or enhanced EPAs could be designed for residents pursuing research tracks or fellowships, allowing them to deepen their research expertise.

Establishing core metrics for evaluating research competency across Canadian psychiatry residency training programs is crucial. By doing so, the RCPSC and the Canadian Psychiatric Association can promote a standardized and comprehensive approach to research training, aligned with the principles of CBD.

In light of the limited evidence available in the literature, we hope that our compilation of resources, strategies, and tips for psychiatry residents interested in research will serve as a valuable, practical element of this article. These resources aim to empower aspiring researchers and contribute to the overall enhancement of research training within psychiatry residency programs.

While integrating research training within CBD presents challenges, we believe that our recommendations and the proposed framework will foster a culture of research excellence among psychiatry residents and contribute to advancing the field of psychiatric research in Canada.

Supplemental Material

sj-docx-1-smo-10.1177_20503121231216846 – Supplemental material for Assessing research competency development in Canadian psychiatry residency programs: A systematic review and future directions

Supplemental material, sj-docx-1-smo-10.1177_20503121231216846 for Assessing research competency development in Canadian psychiatry residency programs: A systematic review and future directions by Anees Bahji, Marlon Danilewitz, David Crockford, Nicholas Delva and Benjamin Rusak in SAGE Open Medicine

Footnotes

Acknowledgements

We would like to recognize that our work takes place on historical and contemporary Indigenous lands, including the territories of Treaties 6, 7, and 8 and the homeland of the Métis. We also acknowledge the many Indigenous communities forged in Alberta’s urban centers.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Anees Bahji receives a small honorarium for teaching undergraduate and postgraduate medical trainees in the Cumming School of Medicine at the University of Calgary. In addition, Anees Bahji is an unpaid member of the Canadian Network for Mood and Anxiety Treatments (CANMAT) editorial committee, the International Society of Addiction Journal Editors (ISAJE), the Canadian Society of Addiction Medicine (CSAM) policy committee, and the Addiction Psychiatry section of the Canadian Psychiatric Association (CPA). Anees Bahji is also an unpaid associate editor of the Canadian Journal of Addiction (CJA) and a mental health educator for TED-Ed, where he receives a small honorarium for supporting online educational content. Finally, Anees Bahji does not report royalties, licenses, consulting fees, payment, or honoraria for lectures or presentations, speaker’s bureaus, article writing, expert testimony, patents, or participation on other boards. Marlon Danilewitz reports personal fees from Eisai Ltd and Winterlight Labs outside the submitted work. The remaining coauthors have no relevant disclosures.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors did not receive specific funding for conducting the present article. Anees Bahji has received grants to support addiction research training from the National Institutes of Health (NIH) and the National Institute on Drug Abuse (NIDA) through the International Collaborative Addiction Medicine Research Fellowship (R25-DA037756) and the Research in Addiction Medicine Scholars Program (R25-DA033211), respectively. In addition, Anees Bahji is a recipient of the 2020 Friends of Matt Newell Endowment from the University of Calgary Cumming School of Medicine. Anees Bahji also received financial support from a 2020 Research Grant on the Impact of COVID-19 on Psychiatry from the American Psychiatric Association and the American Psychiatric Association Foundation. Furthermore, Anees Bahji has been awarded doctoral studies research funding from the Canadian Institutes of Health Research (CIHR) Fellowship and the Harley N. Hotchkiss Graduate Scholarship in Neuroscience from the University of Calgary. Furthermore, Anees Bahji has received research funding through the Calgary Health Trust. However, the content is solely the authors’ responsibility and does not represent the official views of NIDA, the University of Calgary, the CIHR, or the Calgary Health Trust.

Ethical considerations

As a systematic review of previously published data, the University of Calgary Research Ethics Board waived the need for ethics clearance.

Informed consent

Not applicable.

ORCID iDs

Anees Bahji

Benjamin Rusak

Supplemental material

Supplemental material for this article is available online.

References

Fenton

James

Insel

Psychiatry residency training, the physician-scientist, and the future of psychiatry. Acad Psychiatry 2004; 28: 263–266.

Yager

Greden

Abrams

, et al The Institute of Medicine’s report on “Research Training in Psychiatry Residency: Strategies for Reform”— background, results, and follow up. Acad Psychiatry 2004; 28: 267–274.

Roane

Inan

Haeri

, et al Ensuring research competency in psychiatric residency training. Acad Psychiatry 2009; 33: 215–220.

Hamoda

Bauer

DeMaso

, et al A competency-based model for research training during psychiatry residency. Harv Rev Psychiatry 2011; 19: 78–85.

Batson

Peters

, et al Collaboration with clinical and translational sciences to improve psychiatry training in research. Acad Psychiatry 2013; 37: 426–428.

Institute of Medicine (US) Committee on Incorporating Research into Psychiatry Residency Training; Abrams

Patchan

, et al (eds.) Research training in psychiatry residency: strategies for reform. Washington, DC: National Academies Press (US), 2003.

Frank

Snell

Cate

Holmboe

, et al Competency-based medical education: theory to practice. Med Teach 2010; 32: 638–645.

Frank

Mungroo

Ahmad

, et al Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach 2010; 32: 631–637.

Royal College of Physicians and Surgeons of Canada. Competence by design: reshaping Canadian medical education. 2014, p. 141. Royal College of Physicians and Surgeons of Canada.

10.

Ten Cate

Chen

Hoff

, et al Curriculum development for the workplace using Entrustable Professional Activities (EPAs): AMEE Guide No. 99. Med Teach 2015; 37: 983–1002.

11.

Andrew

Shea

Geriatric psychiatry education in Canada: the pathway to subspecialty recognition. Int Psychogeriatr 2010; 22: 919–926.

12.

Benjamin

Educating psychiatry residents in neuropsychiatry and neuroscience. Int Rev Psychiatry 2013; 25: 265–275.

13.

Benjamin

Travis

Cooper

, et al Neuropsychiatry and neuroscience education of psychiatry trainees: attitudes and barriers. Acad Psychiatry 2014; 38: 135–140.

14.

Benjamin

Widge

Shaw

Neuropsychiatry and neuroscience milestones for general psychiatry trainees. Acad Psychiatry 2014; 38: 275–282.

15.

Bode

Jackson

JS.

The current emergency medicine residency curriculum: missing psychiatry. Am J Emerg Med 2017; 35: 1771–1772.

16.

Boland

Becker

Levenson

, et al The milestones for psychosomatic medicine subspecialty training. Psychosomatics 2015; 56: 153–167.

17.

Brasch

Ferencz

JC.

Training issues in emergency psychiatry. Psychiatr Clin North Am 1999; 22: 941–954.

18.

Chung

Insel

TR.

Mind the gap: neuroscience literacy and the next generation of psychiatrists. Acad Psychiatry 2014; 38: 121–123.

19.

Cookey

Butterfield

Robichaud

, et al Integrating clinical neurosciences in a psychiatry residency training program: a brief report with pilot data. Acad Psychiatry 2018; 42: 217–221.

20.

Cordon

Geriatric psychiatry and fellowship programs in the United States. Care Manag J 2008; 9: 180.

21.

Danilewitz

Ainsworth

Liu

, et al Towards competency-based medical education in neurostimulation. Acad Psychiatry 2020; 44(6): 775–778.

22.

Dingle

Sexson

SB.

A practical approach to implementing the core competencies in a child and adolescent psychiatry residency program. Acad Psychiatry 2007; 31: 228–244.

23.

Fage

Abadir

Boyle

, et al Competency-based medical education: objectives for a foundational emergency psychiatry experience. Acad Psychiatry 2018; 42: 519–522.

24.

Lofchy

Boyles

Delwo

Emergency psychiatry: clinical and training approaches. Can J Psychiatry 2015; 60: 1–7.

25.

Mezzacappa

Hamoda

DeMaso

DR.

Promoting scholarship during child and adolescent psychiatry residency. Acad Psychiatry 2012; 36: 443–447.

26.

Mian

Al-Mateen

Cerda

Training child, and adolescent psychiatrists to be culturally competent. Child Adolesc Psychiatr Clin N Am 2010; 19: 815–831.

27.

Rej

Laliberté

Rapoport

, et al What makes residents interested in geriatric psychiatry? A Pan-Canadian online survey of psychiatry residents. Am J Geriatr Psychiatry 2015; 23: 735–743.

28.

Sadhu

Lee

Stewart

, et al Lessons from the launch: program directors reflect on implementing the child and adolescent psychiatry milestones. Acad Psychiatry 2018; 42: 510–516.

29.

Sargent

Sexson

Cuffe

, et al Assessment of competency in child and adolescent psychiatry training. Acad Psychiatry 2004; 28: 18–26.

30.

Sexson

Sargent

Zima

, et al Sample core competencies in child and adolescent psychiatry training. Acad Psychiatry 2001; 25: 201–213.

31.

Sexson

SB.

Directing child and adolescent psychiatry training for residents. Child Adolesc Psychiatr Clin N Am 2010; 19: 31–46.

32.

Sexson

Thomas

Pope

Models of integrated training in psychiatry and child and adolescent psychiatry. Acad Psychiatry 2008; 32: 377–385.

33.

Swantek

Maixner

Llorente

, et al The road ahead in education: milestones for geriatric psychiatry subspecialty training. Am J Geriatr Psychiatry 2016; 24: 675–689.

34.

Townsend

MH.

Emergency psychiatry training for third-year medical students as reported by directors of medical student education in psychiatry. Teach Learn Med 2004; 16: 247–249.

35.

Royal College of Physicians and Surgeons of Canada. Specialty training requirements in psychiatry. Canada: Royal College of Physicians and Surgeons of Canada, 2015.

36.

Royal College of Physicians and Surgeons of Canada. Psychiatry competencies. Royal College of Physicians and Surgeons of Canada, https://www.royalcollege.ca/rcsite/residents-e (2019, accessed 30 November 2023).

37.

Galanter

Kaufman

Schnoll

, et al Postgraduate medical fellowship training in alcoholism and drug abuse: national consensus standards. Am J Drug Alcohol Abuse 1991; 17: 1–12.

38.

Dove

HW.

Postgraduate education and training in addiction disorders: defining core competencies. Psychiatr Clin North Am 1999; 22: 481–488.

39.

Liberati

Altman

Tetzlaff

, et al The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009; 6: e1000100.

40.

Methley

Campbell

Chew-Graham

, et al PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res 2014; 14: 579.

41.

Bero

Chartres

Diong

, et al The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures. Syst Rev 2018; 7: 242.

42.

L-L

Wang

Y-Y

Yang

Z-H

, et al

Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better?

Mil Med Res 2020; 7: 7.

43.

Page

McKenzie

Higgins

JPT

. Tools for assessing risk of reporting biases in studies and syntheses of studies: a systematic review. BMJ Open 2018; 8: e019703.

44.

Luchini

Stubbs

Solmi

, et al Assessing the quality of studies in meta-analyses: advantages and limitations of the Newcastle Ottawa Scale. World J Meta-Anal 2017; 5: 80–84.

45.

Joanna Briggs Institute. Critical appraisal checklist for analytical cross sectional studies, https://jbi.global/sites/default/files/2020-08/Checklist_for_Analytical_Cross_Sectional_Studies.pdf (2021, 11 February 2011).

46.

Moola

Munn

Tufanaru

, et al Chapter 7: systematic reviews of etiology and risk. In: Aromataris

Munn

(eds.) JBI manual for evidence synthesis. Adelaide, Australia: JBI, 2020.

47.

Hershberg

Harper

Leichner

Research in psychiatry: residents’ attitudes and use. Acad Med 1980; 55: 540–541.

48.

Strauss

Yager

Offer

Research training in psychiatry: a survey of current practices. Am J Psychiatry 1980; 137: 727–729.

49.

Andrews

Coverdale

Turbott

Training directors’ and registrars’ views on research training in Australasian psychiatry. Aus Psychiatry 2001; 9: 20–24.

50.

Balon

Singh

Status of research training in psychiatry. Acad Psychiatry 2001; 25: 34–41.

51.

Fitz-Gerald

Kablinger

Manno

, et al Psychiatry residents’ participation in research: a survey of attitudes and experience. Acad Psychiatry 2001; 25: 42–47.

52.

Winter

Rathod

Gregoire

Satisfaction with a research mentoring scheme for specialist registrars. Psychiatr Bull 2006; 30: 348–351.

53.

Silberman

Belitsky

Bernstein

, et al Recruiting researchers in psychiatry: the influence of residency versus early motivation. Acad Psychiatry 2012; 36: 85–90.

54.

Shanmugalingam

Ferreria

Norman

RMG

, et al Research experience in psychiatry residency programs across Canada: current status. Can J Psychiatry 2014; 59: 586–590.

55.

Laliberté

Rapoport

Andrew

, et al

Career interests of Canadian psychiatry residents: what makes residents choose a research career?

Can J Psychiatry 2016; 61: 86–92.

56.

MacMaster

Cohen

Waheed

, et al The psychiatry resident research experience. BMC Res Notes 2016; 9: 486.

57.

Fisher

Bender

SK.

A program of research training in psychiatry: ten-year evaluation and follow-up. Am J Psychiatry 1975; 132: 821–824.

58.

Tuma

Mitchell

Brunstetter

RW.

Toward a manpower base for research in child psychiatry: report of three National Institute of Mental Health Workshops. J Am Acad Child Adolesc Psychiatry 1987; 26: 281–285.

59.

Belmont

Swift

Schwartz

JL.

A model for research training in a child psychiatry residency. Acad Psychiatry 1989; 13: 159–163.

60.

Balon

Kuhn

TW.

Structure versus Laissez-Faire. Acad Psychiatry 2001; 25: 61–67.

61.

Halpain

Jeste

Katz

, et al Summer Research Institute. Acad Psychiatry 2001; 25: 48–56.

62.

Gilbert

Tew

Reynolds

, et al A developmental model for enhancing research training during psychiatry residency. Acad Psychiatry 2006; 30: 55–62.

63.

Kunik

Hudson

Schubert

, et al Growing our own: a regional approach to encourage psychiatric residents to enter research. Acad Psychiatry 2008; 32: 236–240.

64.

Löwe

Hartmann

Wild

, et al Effectiveness of a 1-year resident training program in clinical research: a controlled before-and-after study. J Gen Intern Med 2008; 23: 122–128.

65.

Roane

Inan

Haeri

, et al Ensuring research competency in psychiatric residency training. Acad Psychiatry 2009; 33: 215–220.

66.

Dervic

Lenz

Friedrich

MH.

Research skills training for child psychiatry residents. Eur Child Adolesc Psychiatry 2010; 19: 469–470.

67.

Back

Book

Santos

, et al Training physician-scientists: a model for integrating research into psychiatry residency. Acad Psychiatry 2011; 35: 40–45.

68.

Balon

Guerra

Meador-Woodruff

, et al Innovative approach to research training: research colloquium for junior investigators. Acad Psychiatry 2011; 35: 11–14.

69.

Duthie

Shand

Anderson

, et al Experience of an inter-regional research symposium for higher psychiatric trainees in Scotland. Ir J Psychol Med 2012; 29: 122–124.

70.

Hipolito

MMS

Malik

Carpenter-Song

, et al Capacity-building for African American mental health training and research: lessons from the Howard-Dartmouth Collaborative summer school. Acad Psychiatry 2012; 36: 47–50.

71.

Mezzacappa

Hamoda

DeMaso

DR.

Promoting scholarship during child and adolescent psychiatry residency. Acad Psychiatry 2012; 36: 443–447.

72.

Tsai

Ordóñez

Reus

, et al Eleven-year outcomes from an integrated residency program to train research psychiatrists. Acad Med 2013; 88: 983–988.

73.

Zisook

Boland

Cowley

, et al Teaching scholarly activity in psychiatric training: years 6 and 7. Acad Psychiatry 2013; 37: 82–86.

74.

Bhat

Leong

Lee

, et al Research-track programs for residents in psychiatry: a review of literature and a report of 3 Canadian experiences. Can J Psychiatry 2014; 59: 268–275.

75.

Goldstein

Guerra

Regier

Targeted research training: developing minority psychiatric investigators. Acad Psychiatry 2014; 38: 191–197.

76.

Posporelis

Sawa

Smith

, et al Promoting careers in academic research to psychiatry residents. Acad Psychiatry 2014; 38: 185–190.

77.

Himelhoch

Edwards

Ehrenreich

, et al Teaching lifelong research skills in residency: implementation and outcome of a systematic review and meta-analysis course. J Grad Med Educ 2015; 7: 445–450.

78.

Aftab

Lackamp

Cerny

Measuring perceived educational impact of a resident-led research newsletter. Acad Psychiatry 2017; 41: 373–36.

79.

Besterman

Williams

Reus

, et al The role of regional conferences in research resident career development: the California psychiatry research resident retreat. Acad Psychiatry 2017; 41: 272–277.

80.

Campbell

ANC

Back

Ostroff

, et al Addiction research training programs: four case studies and recommendations for evaluation. J Addict Med 2017; 11: 333–338.

81.

Viswanath

Jayarajan

Chandra

, et al Supplementing research ethics training in psychiatry residents: a five-tier approach. Asian J Psychiatry 2018; 34: 54–56.

82.

Calhoun

Bloch

Stubbe

, et al Integrating clinical and research training in child psychiatry: fifteen-year outcomes of a federally supported program. Child Adolesc Psychiatry Ment Health 2020; 14: 1–10.

83.

The Royal College of Physicians and Surgeons of Canada. Special Program Training requirements for the Clinician Investigator Program (CIP), https://www.royalcollege.ca/rcsite/documents/ibd/clinician_investigator_program_str_e (2015, accessed 16 December 2020).

84.

Appleton

Belrose

Ward

, et al Canadian clinician investigator training in the 21st century. Clin Invest Med 2013; 36: E163–E169.

85.

Hayward

Danoff

Kennedy

, et al Clinician investigator training in Canada: a review. Clin Invest Med 2011; 34: E192–E201.

86.

Zisook

Boland

Cowley

, et al Teaching scholarly activity in psychiatric training: years 6 and 7. Acad Psychiatry 2013; 37: 82–86.

87.

Duthie

Shand

Anderson

, et al Experience of an inter-regional research symposium for higher psychiatric trainees in Scotland. Ir J Psychol Med 2012; 29: 122–124.

88.

Horsley

Hyde

Santesso

, et al Teaching critical appraisal skills in healthcare settings. Cochrane Database Syst Rev 2011; 2011: CD001270.

89.

Critical Appraisal Skills Program. Critical appraisal skills program, https://casp-uk.net/ (2021, accessed 9 July 2021).

90.

Kiesler

CA.

The training of psychiatrists and psychologists. Am Psychol 1977; 32: 107–108.

91.

Scaturo

Huszonek

JJ.

Collaborative academic training of psychiatrists and psychologists in VA and medical school settings. Acad Psychiatry 2009; 33: 4–12.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.07 MB