Development,Implementation,and Initial Participant Feedback of an Online Anatomy and Radiology Contouring Bootcamp in Radiation Oncology

Abstract

BACKGROUND

The Anatomy and Radiology Contouring (ARC) Bootcamp was a face-to-face (F2F) course designed to ensure radiation oncology residents were equipped with the knowledge and skillset to use radiation therapy techniques properly. The ARC Bootcamp was proven to be a useful educational intervention for improving learners’ knowledge of anatomy and radiology and contouring ability. An online version of the course was created to increase accessibility to the ARC Bootcamp and provide a flexible, self-paced learning environment. This study aimed to describe the instructional design model used to create the online offering and report participants’ motivation to enroll in the course and the online ARC Bootcamp's strengths and improvement areas.

METHODS

The creation of the online course followed the analysis, design, development, implementation, and evaluation (ADDIE) framework. The course was structured in a linear progression of locked modules consisting of radiology and contouring lectures, anatomy labs, and integrated evaluations.

RESULTS

The online course launched on the platform Teachable in November 2019, and by January 2021, 140 participants had enrolled in the course, with 27 participants completing all course components. The course had broad geographic participation with learners from 19 different countries. Of the participants enrolled, 34% were female, and most were radiation oncology residents (56%), followed by other programs (24%), such as medical physics residents or medical students. The primary motivator for participants to enroll was to improve their subject knowledge/skill (44%). The most common strength identified by participants was the course's quality (41%), and the most common improvement area was to incorporate more course content (41%).

CONCLUSIONS

The creation of the online ARC Bootcamp using the ADDIE framework was feasible. The course is accessible to diverse geographic regions and programs and provides a flexible learning environment; however, the course completion rate was low. Participants’ feedback regarding their experiences will inform future offerings of the online course.

Keywords

online learning radiation oncology development implementation evaluation anatomy education students’ feedback

Background and Need for the ARC Bootcamp

Overview

This paper describes the creation of an online Anatomy and Radiology Contouring (ARC) Bootcamp designed after a face-to-face (F2F) version of the course. An online version was created to increase accessibility to the ARC Bootcamp and provide a flexible learning environment. The initial F2F ARC Bootcamp was created in 2013 to supplement radiation oncology (RO) education and ensure radiation oncologists were equipped with the knowledge and skillsets to use appropriately modern radiation therapy techniques.¹ The F2F ARC Bootcamp was proven to be a useful educational intervention that allowed instructor–learner interactions and provided a kinesthetic learning environment¹; however, it was costly and not easily accessible for learners located outside of Ontario, Canada. The online model offers flexible self-paced and independent learning, encourages self-assessment using evaluations that provide instant formative feedback, and is a more affordable option for learners.² The purpose of this study was to investigate participant motivation to enroll in the online ARC Bootcamp and their feedback regarding their experience to identify the course's strengths and areas for improvement to inform future offerings. Future studies will evaluate the impact of the online Bootcamp.

RO education

RO has been impacted by significant technological advancements leading to a new standard of patient care.³ Techniques have shifted from 2-dimensional radiographs to methods that rely on 3-dimensional, multimodality imaging for target and organs-at-risk (OAR) delineation.^4,6 For instance, the highly conformal technique, intensity-modulated radiation therapy, delivers an adequate dose to the target and minimizes toxicity to normal tissue and critical organs.⁴ This technique's success is based on accurate contouring of the target volume and OAR. Inaccurate delineation of these structures can lead to significant adverse clinical consequences and patient harm.^4,6 Therefore, radiation oncologists must have foundational and comprehensive knowledge in anatomy⁷ and radiology to outline structures with precision and accuracy; however, instruction in these subjects tends to be minimal during RO training.⁸

F2F ARC bootcamp description

To address the lack of adaptations in RO postgraduate medical education curricula,⁸ a multidisciplinary team of physicians and educators designed and developed the ARC Bootcamp, a 3-day F2F educational intervention.¹ It was designed for RO residents and medical physicists to gain additional integrated instruction in anatomy, radiology, and contouring for 5 major anatomic sites: head and neck, thorax, abdomen, male pelvis, and female pelvis. In 2011 to 2012, the ARC Bootcamp was piloted at a single institution, Western University,^8,9 and the first iteration was launched in 2013 allowing participants from any institution to enroll and attend. Each iteration has incorporated modifications from participants’ feedback, which has ensured that the content is useful and relevant to learners.¹

The ARC Bootcamp consisted of didactic radiology lectures led by diagnostic radiologists and radiation oncologists. The RO sessions were primarily case-based, and participants were encouraged to practice contouring in real time using the software EduCase on their personal devices. Small groups of participants interacted with anatomists to review anatomy concepts using cadaveric prosections tailored to oncological clinical correlations. The F2F ARC Bootcamp was proven to be a useful educational intervention for improving RO residents’ knowledge of anatomy and radiology and enhancing their confidence and accuracy in contouring.¹

Goal of the online arc bootcamp

Acknowledging the importance of increasing access to the ARC Bootcamp and providing flexible and self-paced learning, an online version of the course was developed to offer similar content to the in-person offering. This study aims to describe the instructional design model used to create the ARC Bootcamp's online version and report learners’ feedback of their experience in the course. The study objectives are to identify the participants’ motivation to enroll in the course and identify the ARC Bootcamp's strengths and areas for improvement.

Creation of the Online ARC Bootcamp Using the ADDIE Model

Participants

All learners who enrolled in the online ARC Bootcamp and provided consent between November 2019 and January 2021 were included in the study. The research protocol was approved by the University of Western Ontario Non-Medical Research Ethics Board (Project ID: 114207).

Overview of the addie design model

The design of the online ARC Bootcamp followed the framework of the analysis, design, development, implementation, and evaluation (ADDIE) model, which includes 5 clear and defined phases: analysis, design, development, implementation, and evaluation. The analysis phase is when educators identify learners’ needs and determine what needs to be taught to accomplish the educational goals. In the design phase, the educators decide how to deliver the material to meet the learning needs effectively. The development phase includes the creation and assembly of the course content. In the implementation phase, the educators provide the instruction of material. Lastly, during the evaluation phase, the educators obtain feedback regarding the course and make any necessary modifications. Although the ADDIE model is described as consecutive phases, movement from one phase to the next does not have to be exclusively linear.¹⁰ Figure 1 provides an overview of how the ADDIE instructional design model was used to create the online ARC Bootcamp.

Figure 1.

Overview of how the ADDIE instructional design model was used as a framework to create the online ARC Bootcamp to provide a similar learning experience to the F2F offering and to continuously monitor the online course to inform course modifications. Abbreviations: ADDIE, analysis, design, development, implementation, evaluation; ARC, anatomy and radiology contouring; F2F, face-to-face; LOs, learning objectives.

Online ARC bootcamp analysis

Learners’ needs were identified by the multidisciplinary team based on F2F participants’ feedback from past iterations. This information was used to design and develop the online course's lectures and labs. Analysis of all anatomical structures discussed during the F2F anatomy sessions occurred to ensure similar content was taught during the online course. For instance, the F2F head and neck anatomy sessions taught the sites, nasal cavity and oral cavity, despite the anatomy of these sites not being included in the course manual. The anatomical structures of these sites were taught since several of these structures were identified on annotated radiologic images within the course manual. The online course formally developed anatomy lab videos for these sites. To maintain consistent course learning expectations for F2F and online participants, the F2F course learning objectives were used for the online version.

Online ARC bootcamp design

The multidisciplinary team transformed the F2F Bootcamp into an online version using the model previously described. The online course was organized into 11 locked modules, which were structured linearly. Module components unlocked after a period of 7 days regardless of whether prior modules were fully complete. The first module introduced learners to the course, describing the reason for the ARC Bootcamp, the course's goals, and what learners can expect to gain. Two modules were dedicated to preintervention and postintervention evaluations. The remaining 8 modules consisted of content related to the 5 major anatomic sites covered in the F2F offering and previously mentioned in the introduction. The number of modules per anatomic site correlated to the amount of content for each site (head and neck: 4 modules; thorax: 1 module; abdomen: 1 module; male/female pelvis: 2 modules). Each of the 8 content modules included radiology and contouring lectures, taught by a diagnostic radiologist or radiation oncologist, and anatomy labs led by an anatomist. A few of the modules included interactive activities such as a discussion forum, a quiz, and contouring exercises using the contouring software EduCase (RadOnc eLearning Center Inc., Jackson, WY, USA) (Supplemental Table 1). The learning activities, learning objectives, and evaluation methods were intentionally aligned using the course design approach, constructive alignment, which is used for formulating learning activities and evaluation tasks that directly address the intended learning objectives.¹¹ Aligned courses lead to greater student learning, and it is more likely learners will achieve the course's goals and objectives.¹²

Online ARC bootcamp content development

An online course manual was developed based on the F2F ARC Bootcamp manual. The manual included 5 learning modules for each anatomic site. The modules addressed the learning objectives created by the multidisciplinary team.

The online radiology and contouring lectures were developed based on the F2F ARC Bootcamp lectures. The radiologists’ and radiation oncologists’ PowerPoint (PPT) slides used during the F2F Bootcamp were edited to focus on the essential concepts, which decreased the lecture length from 60 min to a more manageable 20 to 40 min. The screen capture software Camtasia (TechSmith Camtasia, Okemos, MI, USA) was used to record the lecturer's content and edit the videos. The online ARC Bootcamp consisted of 15 lectures, each focused on one of the significant anatomic sites (Supplemental Table 2). All 8 content modules included 1 lecture, with 4 of those modules including multiple lectures.

The anatomy labs were newly created to replace the in-person kinesthetic gross anatomy sessions. Instead of using an existing computer-based anatomy resource, such as 3-dimensional anatomy simulation, we created anatomy lab videos using the same cadaveric specimens used during the F2F course. The anatomy lab videos ranged from 10 to 15 min in length. Anatomists were recorded while they highlighted anatomical structures using cadaveric prosections and discussed spatial relationships. The anatomists annotated PPT slides that included cadaveric atlas images for structures that could not be visualized well with a cadaveric prosection. Using Camtasia's screen recorder function, the anatomist identified the necessary anatomical structures. The anatomy labs were edited using the software Camtasia. The ARC Bootcamp consisted of 12 anatomy labs, each focused on 1 of 5 anatomic sites (Supplemental Table 3). Six of the content modules included at least 1 anatomy lab, with 4 of those modules including multiple labs.

Three discussion forums were embedded in the course equally spaced apart, allowing participants to discuss concepts with their peers. A discussion forum was inserted following the head and neck content, thorax and abdomen content, and pelvis content. Two multiple-choice quizzes were embedded into the course to test the learner's knowledge of the content and provide instant feedback. Quiz 1 evaluated head and neck material, and quiz 2 evaluated thorax, abdomen, and pelvis material.

Online ARC bootcamp implementation

The online ARC Bootcamp was implemented using the course building platform Teachable (Teachable Inc.). Teachable was chosen because it allowed for multiple team members to develop sections of the course, granted an unlimited number of participants to register for the course, accepted international payments in different currencies, included built-in features, such as quizzes that could be integrated to provide instant feedback to learners, and was a secure platform that provided 24/7 monitoring. The online Bootcamp launched in November 2019. The online course cost $199 for 1 year of unlimited access.

Online ARC bootcamp evaluation

A preintervention and postintervention evaluation was embedded at the beginning and end of the ARC Bootcamp, respectively, and administered using Qualtrics (Qualtrics). The preintervention evaluation consisted of demographic questions, a self-confidence survey, an anatomy and radiology knowledge quiz, and a contouring test. The demographic questions identified learners’ characteristics and motivation to register for the ARC Bootcamp (Supplemental Table 4). The self-confidence survey asked learners to rate their self-confidence for concepts related to radiology, anatomy, and contouring using a 5-point Likert scale (1 = Not confident at all to 5 = Extremely confident) (Supplemental Table 5). The anatomy and radiology quiz consisted of 40 multiple choice questions to examine the course's impact on the learner's knowledge of anatomy and radiology (Supplemental Table 6). Learners were provided with 30 min to complete the quiz. The contouring test examined the ARC Bootcamp's impact on the learner's contouring accuracy. Using the software EduCase, learners had 30 min to contour 15 predetermined structures (Supplemental Table 7). Each knowledge quiz and contouring test question was mapped to a course learning objective and to the learning activity (ie, lecture or anatomy lab) that taught the concept.

The postintervention evaluation consisted of the same self-confidence survey, anatomy and radiology knowledge quiz, and contouring test and included a course satisfaction survey. The course satisfaction survey asked learners to provide feedback regarding their experience in the online ARC Bootcamp (Supplemental Table 8). Course feedback will be used to make modifications to future course offerings.

Statistical analysis

Qualitative preintervention and postintervention evaluation open-ended questions were coded independently by 2 researchers using NVivo software (QSR International Version 12; Melbourne, Australia). The first coder developed the codes, themes, and codebooks. Following the initial coding of the responses, the researchers redefined certain codes and reorganized themes. Using the new codebooks, researchers coded all participant responses. Using the coding data output from NVivo, the number of participants that reported each of the themes for participants’ enrollment motivation, the course's strengths, and the course's improvement areas were calculated based on the average between the 2 coders. Intraclass correlation coefficients (ICCs) were used to assess the interrater reliability of codes. The following ICC intervals were used to define the magnitude of reliability: poor (<0.40); fair (0.40-0.59); good (0.60-0.74); excellent (>0.74).¹³ Assessment of the relationship between independent groups and enrollment motivation was assessed with the Kruskal–Wallis test with an alpha ( $α$ ) level of .05. This was followed by Mann–Whitney U tests with an alpha ( $α$ ) level of .0167 (Bonferroni correction) to determine which relationships were significant, and P values subjected to the Bonferroni correction will be denoted as p_bonf. Analysis was performed using SPSS (IBM SPSS Version 26).

Table 2.

Motivation to Enroll in the Course Reported as Average Number of Participants and Grouped Into Themes.

Enrollment motivation themes	Number of participantsn = 140
Development—n(%)	19 (14%)
Improve patient care	5 (4%)
Personal improvement	2 (1%)
Professional improvement	13 (9%)
Helpful learning tool—n(%)	15 (11%)
Helpful course in general	5 (4%)
Exam preparation	6 (4%)
Radiation oncology career preparation	4 (3%)
Improve subject knowledge or skill—n(%)	62 (44%)
Anatomy	36 (26%)
Confidence	2 (1%)
Contouring	37 (26%)
Improve in general	3 (2%)
Improve in general knowledge or skill	4 (3%)
Radiology	13 (9%)
Teaching	1 (1%)
Learn subject knowledge or skill—n(%)	35 (25%)
Anatomy	12 (9%)
Anatomy-radiology correlations	6 (4%)
Contouring	11 (8%)
Learn in general	12 (9%)
Radiology	5 (4%)
Radiation oncology interest—n(%)	3 (2%)
Interested in career	3 (2%)
Refresh subject knowledge—n(%)	9 (6%)
Anatomy	4 (3%)
Contouring	1 (1%)
Update knowledge in general	4 (3%)
Radiology	1 (1%)
Requirement—n(%)	13 (9%)
Curriculum requirement	13 (9%)

Table 3.

ARC Bootcamp Strengths Reported as Average Number of Participants and Grouped Into Themes.

ARC Bootcamp strength themes	Number of participantsn = 27
Environment—n(%)	1 (4%)
Positive learning environment	1 (4%)
Interactive components—n(%)	4 (15%)
Cadaveric prosections	2 (7%)
Real-time contouring	3 (11%)
Learning—n(%)	8 (30%)
Integrative learning	4 (15%)
Multidisciplinary approach	4 (15%)
Pace of learning—n(%)	5 (19%)
Pace	3 (11%)
Review of course material	2 (7%)
Quality of course—n(%)	11 (41%)
Course design	3 (11%)
Course material	4 (15%)
Instructors	3 (11%)
Lectures	1 (4%)
Overall good course	2 (7%)

Table 4.

ARC Bootcamp Areas of Improvement Reported as Average Number of Participants and Grouped Into Themes.

ARC Bootcamp areas of improvement themes	Number of participantsn = 27
Contour instruction and feedback—n(%)	2 (7%)
Increase contouring feedback	1 (4%)
Visualization of instructor contouring	1 (4%)
Decrease amount of content—n(%)	1 (4%)
Anatomy lectures	1 (4%)
Increase amount of content—n(%)	11 (41%)
Breast content	2 (7%)
Content areas in general	2 (7%)
Course manual	1 (4%)
Contouring exercises	2 (7%)
Head and neck	1 (4%)
Neuroanatomy	2 (7%)
Quizzes	1 (4%)
Radiology	1 (4%)
Relevant pathology	2 (7%)
Increase content time—n(%)	3 (11%)
Abdomen and pelvis sections	2 (7%)
Nodal regions	2 (7%)
Pace of learning—n(%)	1 (4%)
Slow pace	1 (4%)
Restructure course—n(%)	2 (7%)
Discipline-specific module	2 (7%)

The participants were organized into 3 independent groups based on educational program: RO resident, professional (radiation oncologist and medical physicist), or other program (ie, medical student or medical physics resident). The quantitative analysis was based on group to determine similarities and differences between the groups’ motivation to enroll in the ARC Bootcamp, and the qualitative analysis was not based on group.

Online ARC Bootcamp Participants’ Initial Feedback

Demographic characteristics

From November 2019 to January 2021, 140 individuals registered for the ARC Bootcamp and completed the preintervention evaluation. Of the 140 participants, 27 (19%) completed all Bootcamp components and were therefore included in this study. The demographics of participants who registered for the Bootcamp versus completed the Bootcamp are identified in Table 1. Of participants who registered, 34% were female, and of participants who completed the course, 37% were female. Participants were located globally, reaching 6 continents and spanning 19 different countries. Most learners were located in North America (registered participants: 47%). The majority of registered learners were RO residents (56%), followed by other programs (24%), such as medical physics students or medical students. Participants who completed the course were mainly in the other program category (52%), followed by RO residents (41%). The majority of RO residents that enrolled in and completed the Bootcamp were junior residents (postgraduate year [PGY] 1-3) (registered: 60%; completed: 82%) compared to senior residents (PGY 4-5).

Table 1.

Preintervention Versus Postintervention Demographic Characteristics.

Characteristic	Registered for Bootcamp(n = 140)	Completed Bootcamp(n = 27)
Gender—n(%)
Female	48 (34)	10 (37)
Male	92 (66)	17 (63)
Location—n(%)
North America	66 (47)	15 (56)
South America	7 (5)	2 (7)
Europe	15 (11)	1 (4)
Asia	7 (5)	0 (0)
Africa	8 (6)	2 (7)
Australia	7 (5)	0 (0)
N/A	30 (21)	7 (26)
Position—n(%)
Radiation oncology resident	79 (56)	11 (41)
Radiation oncologist	22 (16)	2 (7)
Medical physicist	3 (2)	0 (0)
Other program	34 (24)	14 (52)
N/A	2 (2)	0 (0)
Postgraduate year—n(%)	(n = 79)	(n = 11)
1	9 (11)	0 (0)
2	17 (22)	4 (36)
3	21 (27)	5 (46)
4	14 (18)	1 (9)
5	16 (20)	1 (9)
N/A	2 (2)	0 (0)

Abbreviation: N/A, not answered.

Participants motivation to enroll in the online ARC bootcamp

Enrollment motivation themes were identified from participants’ answers to the open-ended questions in the preintervention evaluation. Table 2 highlights the participants’ motivation to enroll in the course. The primary motivator for participants to enroll in the course was to improve their knowledge or skill in a subject area. Specifically, learners wanted to improve their anatomy knowledge and their contouring abilities. The second most common motivator among participants was to learn subject knowledge or a skill, specifically in anatomy and contouring. The ICC indicated excellent reliability between the 2 coders of the coding data for participants’ enrollment motivation (ICC value: 0.88).

Significant differences between the independent groups’ motivation to enroll in the Online ARC Bootcamp were identified for 4 of the 7 motivation themes: improve knowledge or skill in a subject area (X²(2) = 10.452, P = .005), interest in RO (X²(2) = 8.870, P = .012), refresh subject knowledge (X²(2) = 8.852, P = .012), and requirement (X²(2) = 29.671, P < .001). RO residents more frequently stated that their motivation to enroll in the ARC Bootcamp was to improve their knowledge or skill in a subject area (p_bonf = 0.005) or refresh their subject knowledge (p_bonf = 0.009) compared to the other program group. Participants from the other program group more frequently stated that their motivation to enroll in the ARC Bootcamp was because they were interested in the field of RO compared to the RO resident group (p_bonf = 0.013) and more frequently stated that the online course was a curriculum requirement compared to the RO resident and professional groups (p_bonf< 0.001).

Online ARC bootcamp strengths and areas for improvement

Major themes were identified from the answers to the open-ended questions from the postintervention evaluation. Tables 3 and 4 highlights the ARC Bootcamp's strengths and areas of improvement identified by participants. The course's quality, specifically related to the design, material, and instructors, were major strengths. Additionally, the integration of anatomy, radiology, and contouring concepts was well received by the participants. Lastly, participants appreciated learning the material at their own pace as it provided a flexible learning environment. Regarding areas of improvement, participants stated increasing the amount of course content, such as adding a breast section to the course or including more contouring exercises would be beneficial. The ICC indicated excellent reliability between the 2 coders of the coding data for the course's strengths and improvement areas (ICC values: strengths: 0.96; areas of improvement: 0.99).

Discussion

The benefits and disadvantages of creating an online ARC bootcamp

We demonstrated the feasibility of creating an online version of the ARC Bootcamp using the ADDIE model. The model provided 5 defined phases that the multidisciplinary team followed to ensure the learning experiences developed met the learner's needs and that the course was continuously monitored to make modifications.

The main drivers for the online course development were to increase access to the ARC Bootcamp and provide a flexible, self-paced learning environment for participants. The online course was accessible, with 140 enrolled learners located in 19 different countries and from various programs such as RO and medical physics, with a handful of medical student participants interested in learning about a RO career. The online ARC Bootcamp is available to an unlimited number of learners compared to the F2F Bootcamp, which was generally limited to 40 participants to maintain opportunities for hands-on learning in small groups.¹ The online offering was also more affordable than the F2F course, especially for international learners.

In addition to increased accessibility, the online Bootcamp provided participants with a self-paced and flexible learning environment, which was identified as a strength of the course. Online participants were provided with 1 year to complete the course from the date of registration. Although the F2F Bootcamp was scheduled yearly to avoid competing clinical duties and vacations,¹ the online participants could complete the course on their schedule and could review course material. Since the labs and lectures were asynchronous, participants could control their learning pace by pausing, rewinding, and fast-forwarding the recordings.

Online learners were able to test their knowledge acquisition with the integration of formative knowledge quizzes. Participants were provided with immediate corrective feedback allowing learners to revisit concepts before continuing. Formative assessments were incorporated because they are intended to promote participants’ learning^14,15 and could result in improved learning outcomes.¹⁵ Although the 2 quizzes tested all anatomic sites, it may have been more beneficial to incorporate a quiz directly following the content of each anatomic site. This would have ensured knowledge was tested after each section, and corrective feedback was provided at each stage of the learning process. Contouring exercises were also integrated into the course; however, immediate feedback was not provided. Increasing the contouring feedback was an area identified by the participants for improving the course.

Although there were benefits to creating the online version, we did identify some disadvantages of an online ARC Bootcamp during the design and development phase of the course. The creation of the online radiology and contouring lectures was relatively simple; however, difficulties arose when developing the anatomy component of the course. The F2F anatomy sessions included kinesthetic learning experiences with cadaveric prosections and bony skulls; therefore, the online anatomy labs had to be newly created. Rather than incorporating active learning strategies through computer-based interactive resources, which is the common trend in online anatomy education,^16,17 we developed the online anatomy labs to focus on identifying anatomical structures using cadaveric prosections and the discussion of spatial relationships. The reasoning was to ensure that F2F and online participants received a similar learning experience to determine the online Bootcamp's effectiveness on the learner's knowledge acquisition by comparing to the F2F Bootcamp.

In addition to the lack of hands-on learning, the online Bootcamp also lacked immediate interactions. Discussion forums were integrated to allow learners to discuss concepts with other participants and the instructors. Although the discussion forums were meant to promote student–student and student–instructor communication, participation tended to be minimal. It remains unknown why the discussion forums lacked participation; however, factors such as instructor participation and discussion forum structure could have influenced the participation rates.¹⁸ It has been stated that instructors can encourage the learner's discussion forum participation by responding to student's questions/posts promptly.¹⁹ Also, learners are more likely to partake in discussions with a clear purpose²⁰ rather than discussions with no guidance or structure. By developing structured discussion forums with consistent instructor feedback, learner participation rates may increase.

Ultimately, the initial experiences with the online ARC Bootcamp have been similar to the experiences of other online courses designed for different medical residency specialties, such as obstetrics and gynecology (OB/GYN)²¹ and family medicine.²² Similarities were noted with the course design, specifically with both courses consisting of asynchronous learning modules designed by a multidisciplinary team and pretests and posttests to assess knowledge/understanding.^21,22 Lastly, one of the courses also included discussion forums monitored consistently by an instructor.²¹ Similar to the online ARC Bootcamp findings, the online course for OB/GYN residents reported a high drop-out rate after enrollment (35%), stating that the majority of these participants were older or in their final year of residency.²¹ The online ARC Bootcamp experienced a similar trend with senior residents (PGY 4-5) less likely to complete the course compared to junior residents (PGY 1-3). Lastly, the OB/GYN residents reported high satisfaction with the online course, specifically with the multidisciplinary instruction and the opportunity for a flexible learning environment,²¹ which were also components of the online ARC Bootcamp that were identified as strengths of the online course.

Motivation to enroll in the online arc bootcamp

The ARC Bootcamp was designed to provide high-yield knowledge of anatomy and radiology, with clinical correlates and contouring instruction. Therefore, this course was well suited for the participants’ enrolled to improve/refresh their subject knowledge in anatomy, radiology, and contouring and to learn anatomy-radiology correlates. The individuals enrolled to learn knowledge in these subject areas may have struggled with the course material. For instance, some learners recommended discipline-specific or beginner modules to be developed and added to the course. Therefore, it is highly recommended that learners have background knowledge in these 3 subject areas to gain the most from the online ARC Bootcamp.

To enhance professional development and patient care were also enrollment motivators identified by learners. Although better contouring and adherence to guidelines have been shown to lead to improved quality care,^23,24 it is unknown whether knowledge gained from the ARC Bootcamp results in measurable benefits for patients, which warrants the need for additional research. A handful of participants also used the ARC Bootcamp to prepare for their role as a radiation oncologist or upcoming examinations. The use of bootcamps to supplement both undergraduate and postgraduate medical education curricula has become common among medical trainees.²⁵ Lastly, some learners stated that the online course was a mandatory requirement of their program, such as the Commission on Accreditation of Medical Physics Education Programs (CAMPEP). In the past, CAMPEP students have attended the F2F ARC Bootcamp. With the online course development, the program made the ARC Bootcamp a curriculum requirement since the course was accessible to all learners and could be incorporated into the curriculum with ease.

Modifications to the online ARC bootcamp based on participants’ feedback

Although several participants acknowledged the course's quality as a strength, course modifications can be made to ensure the content provided is helpful to learners. The majority of participants’ feedback regarding course improvement areas focused on increasing the amount of course content. Most notably, learners expressed interest in incorporating a breast section and neuroanatomy section, which currently do not exist. Based on this feedback, new modules are currently under development. In addition to increasing the amount of course content, participants requested more time to be spent on specific topics, such as the abdomen and pelvis sections and nodal concepts. Since this is an asynchronous online course, participants can control their learning pace and revisit the sections and concepts they wish to spend more time. It is better to use our resources to create new sections that currently do not exist rather than recreate existing sections.

Limitations

Completion of the online ARC Bootcamp was measured by finishing all course content, including the postintervention evaluation. Based on this metric, a low number of participants (19%) completed all the course components compared to the number of participants who registered for the course. The average percentage of the course completed by participants was unknown; therefore, participants may have learned all of the course content but did not submit the postintervention evaluation. From the date of registration, participants were provided with 1 year to complete the course, which may have contributed to the low completion rates. A study including 129 online courses reported a negative correlation between course length and course completion rates stating that longer courses resulted in a lower course completion rate among students.²⁸ Therefore, decreasing the availability length of the online ARC Bootcamp could impact the course completion rates.

Future directions

Future studies of the ARC Bootcamp will assess the impact the online version has on learners’ anatomy and radiology knowledge, contouring ability, and anatomy, radiology, and contouring confidence by comparing to the F2F version of the Bootcamp. Additionally, predictors of participants’ success in the ARC Bootcamp will be determined. This will include identifying correlations between the predictor, such as gender, visuospatial ability, and current career position, to participants’ anatomy and radiology knowledge, contouring ability, and confidence. Data from the preintervention and postintervention evaluations will be used to conduct future studies.

Conclusion

This study demonstrated the feasibility of creating an online course in RO using the instructional design model ADDIE. The online ARC Bootcamp offers similar content to the F2F Bootcamp by providing additional integrated anatomy, radiology, and contouring instruction. The online offering is more accessible to learners resulting in enrollments from a diverse cohort of participants, provides a self-paced, flexible learning environment, and allows learners to assess their knowledge using integrated assessments. Several participants recognized the course's exceptional quality; however, the course will be continuously monitored using the integrated evaluations and participants’ feedback to make any necessary course modifications to ensure learners’ needs are met.

Footnotes

Acknowledgements

The authors would like to acknowledge all individuals who volunteered their time and contributed to creating the online ARC Bootcamp. The authors also recognize the participants who completed the evaluations and provided their valuable feedback. Finally, they thank Haley Linklater, Body Bequeathal Program Supervisor, and Kevin Walker, Body Bequeathal Program Technician, at the University of Western Ontario.

DECLARATION OF CONFLICTING INTERESTS

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

FUNDING

The author received no financial support for the research, authorship, and/or publication of this article.

Supplemental Material

Supplemental material for this article is available online.

ORCID iD

Paige Eansor

References

Jaswal

D’Souza

Johnson

, et al. Evaluating the impact of a Canadian national anatomy and radiology contouring boot camp for radiation oncology residents. Int J Radiat Oncol Biology Phys. 2015;91:701-707. doi:10.1016/j.ijrobp.2014.11.009

Panigrahi

Srivastava

Sharma

. Online learning: adoption, continuance, and learning outcome—A review of literature. Int J Inform Manage. 2018;43:1-14. doi:10.1016/j.ijinfomgt.2018.05.005

Debenham

Banerjee

Fairchild

Dundas

Trotter

Yee

. Canadian Radiation oncology resident survey. Int J Radiat Oncol Biology Phys. 2012;82:1326-1331. doi:10.1016/j.ijrobp.2011.04.030

Bucci

Bevan

Roach

. Advances in radiation therapy: conventional to 3D, to IMRT, to 4D, and beyond. Ca Cancer J Clin. 2005;55:117-134. doi:10.3322/canjclin.55.2.117

Garibaldi

Jereczek-Fossa

Marvaso

, et al. Recent advances in radiation oncology. Ecancermedicalscience. 2017;11:785. doi:10.3332/ecancer.2017.785

Nakamura

Sasaki

Ohga

, et al. Recent advances in radiation oncology: intensity-modulated radiotherapy, a clinical perspective. Int J Clin Oncol. 2014;19:564-569. doi:10.1007/s10147-014-0718-y

Chino

Doyle

Marks

. The anatomy of radiation oncology residency training. Int J Radiat Oncol Biology Phys. 2014;88:3-4. doi:10.1016/j.ijrobp.2013.09.039

D’Souza

Jaswal

Chan

, et al. Evaluating the impact of an integrated multidisciplinary head & neck competency-based anatomy & radiology teaching approach in radiation oncology: a prospective cohort study. Bmc Med Educ. 2014;14:124. doi:10.1186/1472-6920-14-124

Labranche

Johnson

Palma

D’Souza

Jaswal

. Integrating anatomy training into radiation oncology residency: considerations for developing a multidisciplinary, interactive learning module for adult learners. Anat Sci Educ. 2015;8:158-165. doi:10.1002/ase.1472

10.

Cheung

. Using the ADDIE model of instructional design to teach chest radiograph interpretation. J Biomed Educ. 2016;2016:1-6. doi:10.1155/2016/9502572

11.

Biggs

. Enhancing teaching through constructive alignment. High Educ. 1996;32:347-364. doi:10.1007/bf00138871

12.

Blumberg

. Maximizing learning through course alignment and experience with different types of knowledge. Innovative High Educ. 2009;34:93-103. doi:10.1007/s10755-009-9095-2

13.

Fleiss

Levin

Paik

. Statistical methods for rates and proportions. Published online 2018. John Wiley & Sons. doi:10.1002/0471445428

14.

Rolfe

McPherson

. Formative assessment: how am I doing? The Lancet. 1995;345:837-839.

15.

Velan

Jones

McNeil

Kumar

. Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning. Bmc Med Educ. 2008;8:52. doi:10.1186/1472-6920-8-52

16.

Trelease

. From chalkboard, slides, and paper to e-learning: how computing technologies have transformed anatomical sciences education. Anat Sci Educ. 2016;9:583-602. doi:10.1002/ase.1620

17.

Clunie

Morris

Joynes

VCT

Pickering

. How comprehensive are research studies investigating the efficacy of technology-enhanced learning resources in anatomy education? A systematic review. Anat Sci Educ. 2018;11:303-319. doi:10.1002/ase.1762

18.

Fehrman

Watson

. A systematic review of asynchronous online discussions in online higher education. Am J Distance Educ. 2020;34:1-14. doi:10.1080/08923647.2020.1858705

19.

Dailey-Hebert

. Maximizing interactivity In online learning: moving beyond discussion boards. Journal of Educators Online. 2018;3:65-90. doi:10.9743/jeo.2018.15.3.8

20.

Delahunty

. Connecting to learn, learning to connect: thinking together in asynchronous forum discussion. Linguist Educ. 2018;46:12-22. doi:10.1016/j.linged.2018.05.003

21.

Vieira

TCSB

Nakamura

Silva

I da

, et al. Experience of an online course on sexuality during pregnancy for residents. Sex Reprod Healthc. 2017;12:76-81. doi:10.1016/j.srhc.2017.03.004

22.

Skye

Wimsatt

Master-Hunter

Locke

. Developing online learning modules in a family medicine residency.pdf. Fam Med. 2011;43:185–192.

23.

Kong

Ying

, et al. Patterns of local-regional failure after primary intensity modulated radiotherapy for nasopharyngeal carcinoma. Radiat Oncol. 2014;9:60. doi:10.1186/1748-717x-9-60

24.

Furman

Whalen

Shah

Kadish

. Gastric perforation following stereotactic body radiation therapy of hepatic metastasis from colon cancer. Pract Radiat Oncol. 2013;3:40-44. doi:10.1016/j.prro.2012.03.005

25.

Davidson

Barker

Egro

Krajewski

Janis

Nguyen

. A national curriculum of fundamental skills for plastic surgery residency. Ann Plas Surg. 2017;78:121-126. doi:10.1097/sap.0000000000000977

26.

Duffy

Nawoor-Quinn

Burlacu

. “Rapid sequence induction”—an anaesthesiology boot camp. Ir J Medical Sci 1971. 2020;189:1047-1051. doi:10.1007/s11845-019-02146-w

27.

Khan

Bakhiet

. Implementing a unique immersive near-peer-led clinical skills educational bootcamp for early-year clinical medical students. Postgrad Med J. 2021;0:1-5. doi:10.1136/postgradmedj-2020-139233

28.

Jordan

. Massive open online course completion rates revisited: assessment, length and attrition. Int Rev Res Open Distributed Learn. 2015;16:341–358. 10.19173/irrodl.v16i3.2112

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.24 MB