Abstract
Objectives
To explore the efficacy of a novel teaching model based on the Monaco planning system in radiation oncology education.
Methods
Forty clinical undergraduate medical interns were randomized and equally divided into a control group and an experimental group. The experimental group underwent the Monaco planning system-based teaching model, while the control group received conventional clinical teaching activities. Upon completion of the internship, both groups were simultaneously assessed mini clinical evaluation exercise (Mini-CEX), direct observation of procedural skills (DOPS), and end-of-rotation theoretical and clinical skills assessment. A satisfaction survey was also conducted. The assessment scores were compared to evaluate the teaching effectiveness.
Results
The experimental group demonstrated significantly better scores than the control group according to the 7 core evaluation indicators of Mini-CEX, DOPS assessment results, final departmental exam performance, and satisfaction scores, with statistically significant differences (P < .05).
Conclusion
The Monaco planning system-based teaching model significantly enhances clinical competency and learner satisfaction in radiation oncology education compared to traditional methods.
Keywords
Introduction
In the context of the swiftly advancing field of radiation therapy, the discipline of radiation oncology emerges as a critical component of undergraduate oncology internship education. Given its highly specialized nature, intricate knowledge framework, and extensive scope of expertise encompassing clinical oncology, radiophysics, radiobiology, medical imaging, as well as systemic and regional anatomy, the execution of clinical undergraduate practical teaching presents a considerable challenge. 1 The conventional approach of teaching, such as lecture-based learning (LBL) and case-based learning (CBL), places the initiative predominantly with the instructor. The instructor selects a topic and then passes on knowledge centered around this theme, primarily utilizing PowerPoint (PPT) presentations. This method focuses more on theoretical learning than practical application, often overlooking the critical importance of hands-on experience.2,3 Given its highly practical nature, radiation oncology faces significant challenges in achieving effective clinical internship outcomes under this traditional teaching paradigm due to its monotony, lack of engagement, and absence of encouragement for creative thinking. 4 Furthermore, China experiences a notable scarcity of systematic and efficient internship teaching methodologies in radiation oncology, highlighting the pressing need for the development of an accessible, engaging, and innovative educational model in this field.
As artificial intelligence (AI) continues to weave its way into various application scenarios, its judicious integration has the potential to substantially enhance medical education. 5 The Monaco planning system stands at the forefront of this integration, serving as an advanced oncology radiotherapy planning tool that harnesses the precision of the Monte Carlo dose algorithm and the finesse of biological optimization. This system is good at supporting a comprehensive range of contemporary radiotherapy techniques, including 3-dimensional conformal radiation therapy (3D-CRT), intensity-modulated radiation therapy, volumetric modulated arc therapy, and stereotactic body radiation therapy plans. The simulation teaching method, utilizing the Monaco planning system, effectively recreates a realistic radiotherapy process, encompassing functions like automatic image fusion, automatic delineation of target volumes, and the identification of organs at risk. The teaching model based on the Monaco planning system offers the following advantages. Firstly, the system's capacity to store extensive radiation oncology case data enables interns to access diverse cancer case studies for repeated review, allowing students to engage in self-directed learning during their free time. Secondly, it integrates patient computed tomography (CT) or magnetic resonance imaging (MRI) images into 3D models, enabling trainees to observe anatomical structures from multiple planes and angles—such as coronal, sagittal, and transverse sections—providing a more comprehensive and accurate understanding of spatial anatomical relationships compared to traditional 2-dimensional (2D) imaging. Furthermore, its automated target delineation and radiation dose optimization functionalities enhance teaching efficiency and quality while improving students’ learning experience and practical skills.6-8 By leveraging advanced features such as 3D dose clouds, real-time multiplanar reconstruction, and dynamic dose painting, the Monaco planning system transforms traditional teaching into interactive 3D visualization training. Therefore, the Monaco planning system's superior performance in radiotherapy demonstrates its potential for developing an immersive radiotherapy internship teaching mode, which based on these advantages, this study hypothesizes that internship teaching models utilizing the Monaco planning system outperform traditional pedagogical approaches.
The mini-clinical evaluation exercise (Mini-CEX) and the direct observation of procedural skills (DOPS) serve as multifaceted assessment scales that fulfill both educational and evaluative roles. These tools are increasingly recognized as integral components of formative assessment systems within medical education. Their widespread adoption is a testament to their effectiveness in shaping the development of medical professionals.9-11 Research has indicated that both Mini-CEX and DOPS are valuable tools for formative assessment in clinical internship education, including in specialized fields like radiation oncology, due to their clear benefits.12,13 In this study, we explored the effectiveness of an innovative teaching model based on the Monaco planning system in radiation therapy internship training and evaluated its teaching outcomes using Mini-CEX combined with DOPS. The study aimed to explore a new teaching approach to enhance practical training in radiation therapy, thereby improving students’ understanding and skills in this field.
Methods
The reporting of this study conforms to the CONSORT statement 14 (Supplemental Material 1).
General Information
Between January and March 2023, 40 clinical undergraduate internship students were recruited from the Radiotherapy Center of the Third Affiliated Hospital of Chongqing Medical University (Chongqing, China) to participate in this study. Inclusion criteria included the following: (1) Full-time undergraduate interns in clinical medicine; (2) Completion of 4 years of basic medical and clinical medicine coursework and entry into the clinical internship phase; (3) Voluntary participation in this study; (4) No prior internship experience in radiation oncology; (5) Commitment to full participation in the study throughout the internship period. Exclusion criteria included the following: (1) Prior training or clinical work experience in radiation oncology (eg, individuals returning to school after clinical work experience); (2) Anticipated inability to complete required follow-ups or evaluations (eg, withdrawing from the internship midway or taking extended leave); (3) Severe physical or psychological conditions that may affect internship performance or evaluation outcomes. This study confirmed through baseline surveys that the experimental group had not received systematic lectures, workshops, or targeted training related to the research topic (including but not limited to theoretical courses or pedagogical instruction) prior to the intervention. Participants in both groups shared identical professional backgrounds and 4-year undergraduate education experiences. All 40 interns met the inclusion criteria and were randomly assigned to either the experimental group, which utilized the Monaco planning system-based teaching model, or the control group, which adhered to the traditional LBL and CBL teaching methods. Each group comprised 20 students, and all interns spent 4 weeks practicing in the oncology radiotherapy center.
Study Design
As shown in Figure 1, the experimental group received radiotherapy practice teaching facilitated by the Monaco planning system (Elekta, version 5.11.03). The comprehensive details are outlined as follows: Upon entering the radiotherapy center for practical experience, students are guided by an instructor through the following steps: (1) They embark on an observation of the radiotherapy mould-making process; (2) Subsequently, they visit the radiotherapy simulation positioning procedure, gaining insight into patient positioning for treatment; (3) Under the guidance of the instructor, the students utilized the Monaco planning system to outline the OAR and the radiotherapy target area, and subsequently compared the resulting image with the target area outlined by the instructor to identify any discrepancies; (4) They then observe the physicist's target area plan design and doctors’ radiotherapy plan verification process, deepening their understanding of the technical aspects; (5) Finally, students actively participate in the implementation of a patient's initial radiotherapy plan, gaining hands-on experience in the delivery of the treatment plan.

Flowchart of this research.
Under the guidance of their practice teachers, students had the opportunity to engage hands-on with various radiotherapy processes using the Monaco planning system. This included image fusion, target delineation, the identification of organs at risk, and radiotherapy plan design. The Monaco planning system converts CT and MRI tomographic images into 3D, volumetric, and dynamic educational models. With a simple mouse click, students can study and observe patient imaging data from multiple anatomical planes—including coronal, sagittal, and axial views—enabling a comprehensive understanding of the critical steps in radiotherapy workflows (Figure 2).

The Monaco planning system reconstructs CT scans into three-dimensional images. (A) CT images in axial, coronal, and sagittal planes; (B) 3D-reconstructed images viewable from multiple angles using the Monaco planning system.
For the control group, a conventional internship teaching approach was employed, encompassing LBL and CBL. The practice teachers utilized LBL and CBL to present the interns with the fundamental steps of tumor radiotherapy through PowerPoint presentations. This encompassed patient preparation before radiotherapy, mold making, target area delineation, plan verification, and radiotherapy implementation. In this teaching model, the practice teachers primarily relied on lecture methods to explain the radiotherapy process, guiding students to apply basic theories in analyzing and solving problems, thereby fulfilling the teaching objectives.
After the 4-week internship, both the control group and the experimental group underwent a formative evaluation encompassing Mini-CEX and DOPS assessments, as well as a summative assessment of theoretical knowledge and practical skills. Additionally, satisfaction surveys were conducted among the participants. Firstly, the specific procedure for the Mini-CEX assessment is as follows: With informed patient consent, conduct the initial consultation for newly admitted patients with malignant tumors (eg, stomach cancer, esophageal cancer, colorectal cancer, and lung cancer) on the same day. The Mini-CEX checklist evaluates 7 aspects: consultation skills, physical examination, specialized knowledge, clinical diagnosis, communication skills, humanistic care, and comprehensive ability. The scoring criteria are as follows: 1-3: Unsatisfactory, 4-6: Satisfactory, 7-9: Excellent. Secondly, the DOPS assessment method is as follows: It assesses the mastery of interns in the skills related to radiation oncology. It includes 11 items: knowledge reserve level, obtaining informed consent, preparation before operation, appropriate pain relief and sedation, clinical operations and technical capabilities, aseptic technique, seeking help in an appropriate time, related disposal after operation, communication skills, passionate care, and comprehensive performance. Each item is worth a maximum of 9 points, with 1-3 points indicating needs improvement, 4-6 points indicating satisfactory, and 7-9 points indicating excellent. The Mini-CEX and DOPS checklists are provided in Supplemental Material 2. Thirdly, at the conclusion of the internship, all interns are required to undergo a comprehensive after-department examination, encompassing both theoretical and skill assessments. The final rotation assessment content was strictly aligned with the pre-defined internship syllabus: the theoretical examination consisted of 25 multiple-choice questions randomly selected from the question bank (4 points per question, totaling 100 points), while the skill assessment evaluated trainees’ basic operational competencies in tumor radiotherapy using a standardized scoring rubric (total score: 100 points). Additionally, each internship student was expected to complete the internship satisfaction questionnaire, which was developed by the department and utilized a hundred-mark scoring system. A higher score indicated a higher level of satisfaction. To minimize assessment bias, evaluators were blinded to group allocation throughout the study period.
Statistical Methods
The statistical analysis was conducted using SPSS 27.0 software. The data were presented in the format of mean ± standard deviation (M ± SD), and an independent sample t-test was utilized to compare the differences between groups. The counting data were represented as case numbers or rates, and the comparison between groups was made using the X2 test. Differences were deemed statistically significant if the P-value was less than .05.
Results
Comparison of Examination Results between 2 Distinct Groups by Utilizing the Mini-CEX and DOPS Evaluations
The baseline demographic characteristics, including gender and age, were comparable between the 2 groups, showing no significant differences (P > .05, Table 1). Upon completion of the 4-week internship period, both groups were administered a series of assessment tests. In the Mini-CEX, the experimental group surpassed the control group in achieving higher scores across 7 distinct items: consultation skills, physical examination, specialized knowledge, clinical diagnosis, communication skills, humanistic care, and comprehensive ability. Notably, statistical differences were observed between each of these items, indicating a significant advantage for the experimental group (P < .05, Table 2). Similarly, in the DOPS, the experimental group demonstrated superior performance over the control group in 11 key items, including but not limited to knowledge reserve level, obtaining informed consent, preparation before operation, appropriate pain relief and sedation, with the observed differences being statistically significant (P < .05, Table 3).
General Information of 2 Groups of Interns.
Abbreviations: N, number; M, mean; SD, standard deviation.
Comparison of Mini-CEX Scores between 2 Groups of Internship Students upon Completion of the Internship (scores, M ± SD).
Abbreviations: N, number; M, mean; SD, standard deviation.
Comparison of DOPS Scores between 2 Groups of Internship Students upon Completion of the Internship (Scores, M ± SD).
Abbreviations: N: number; M: mean; SD: standard deviation.
Comparison of After-Department Examination Scores and Satisfaction Levels between the 2 Groups
Upon completion of the 4-week internship program, both groups of interns underwent comprehensive evaluations comprising a radiotherapy theory examination, skill examination, and satisfaction survey. The findings revealed that the experimental group outperformed the control group in all metrics, including theory examination scores (83.9 ± 1.37 vs. 77.40 ± 1.90, t = 12.38, P < .001), skill examination scores (80.00 ± 1.72 vs. 73.00 ± 3.00, t = 9.79, P < .001), and satisfaction scores (93.65 ± 1.46 vs. 86.60 ± 2.52, t = 10.81, P < .001), with these differences being statistically significant (Table 4).
Comparison of end-of-rotation Theoretical and Clinical Skills Assessment Scores between the 2 Groups (Scores, M ± SD).
Abbreviations: N, number; M, mean; SD, standard deviation.
Discussion
Radiation oncology serves as a pivotal aspect of internship teaching for clinical oncology undergraduate students, with approximately 60% of cancer patients undergoing radiotherapy or palliative radiotherapy throughout their course of disease. 15 Radiation oncology stands out as a comprehensive and specialized discipline amidst other medical fields, with its technology advancing rapidly. Although previous studies have indicated that both LBL and CBL possess certain advantages when it comes to radiation oncology education, 16 these traditional teaching models often require teachers to devote considerable time to face-to-face lectures. However, this approach tends to render students’ passive learners in practice, often unable to exercise their subjective initiative in the learning process. To proficiently grasp radiation therapy, it is imperative to acquaint oneself with a diverse array of knowledge encompassing clinical oncology, radiophysics, radiobiology, medical imaging, as well as systemic and local anatomy. 1 For undergraduate internship students newly entering the clinical internship, mastering the intricate process of tumor radiotherapy can be a challenging task. Hence, it is imperative for teachers to actively delve into enhanced teaching models in radiation oncology, enabling them to swiftly grasp the essence of this discipline within the limited time of their internship.
AI empowers various industries with its efficiency and precision. In clinical radiotherapy practice, AI-powered platforms enable radiotherapy planning systems to autonomously perform delineation of normal tissues and organs, thereby enhancing the efficiency and accuracy of cancer radiotherapy.17,18 Meanwhile, researchers are actively exploring AI applications in radiation oncology education. Studies have demonstrated that compared to traditional teaching methods, AI-integrated approaches provide students with expanded educational resources, allowing repeated virtual target delineation practice, and facilitating participation in radiotherapy plan development. These innovations significantly improve pedagogical outcomes in radiation oncology training. 18
The Treatment Planning System is a radiotherapy planning platform developed collaboratively by radiation physicists and computer specialists. 19 Utilizing CT imaging as its foundation, it integrates computational power, intelligent target delineation, and radiotherapy plan generation to ensure precision and efficiency throughout cancer radiotherapy. Clinically prevalent systems include Varian Eclipse, Pinnacle, and Monaco planning system.20,21 The Monaco system employs Monte Carlo algorithms to simulate particle trajectories in human tissues and calculate dose distribution. It incorporates radiobiological optimization models for direct biological effect integration, supports multimodal image fusion for targeting accuracy, and combines computer science with clinical oncology to deliver tumor-specific solutions. Statistical analysis and quality control tools ensure therapeutic safety. 6,22-26 The Monaco planning system reconstructs 3D images from 2D medical data, converting conventional imaging and irradiation field designs into interactive 3D teaching maps. Students can dynamically visualize radiotherapy targets across coronal, sagittal, and axial planes to assess spatial relationships between tumors and critical structures, enhancing teaching interactivity. Collaborative exchanges with instructors and peers further consolidate learning experiences. 27,28-30,31 Moreover, the Monaco planning system enables students to rapidly grasp multiple key aspects of cancer radiotherapy. Its rich and engaging content accelerates learning through simulated clinical cases, enhancing students’ ability to recognize anatomical structures and delineate targets, and deepening their understanding of the entire radiotherapy process. Since acquiring the Monaco planning system in 2019, our radiation oncology center has attained a high level of proficiency in utilizing the platform, and significantly enhanced the efficiency of our radiotherapy procedures.
Formative evaluation serves as a pivotal tool in the clinical internship teaching process, emphasizing the process rather than the result. It enables teachers to gain insights into students’ learning progress, allowing for timely adjustments in teaching strategies and fostering students’ continuous improvement. 32 It has been reported that the integration of the Mini-CEX and DOPS evaluation systems can effectively complement each other's strengths, thereby enhancing clinical teaching outcomes. The DOPS can address the operational skill limitations of the Mini-CEX, while the Mini-CEX offers a more comprehensive assessment of the operator's qualities, including humanistic aspects, that are not captured by the DOPS. 12 Liu Hongzhi et al successfully implemented a clinical teaching evaluation approach in oncology that combined the Mini-CEX with DOPS. Their findings revealed that this integrated application offered the advantages of simplicity, comprehensiveness, direct observation, and an overall superior clinical teaching effect. 33
This study incorporated the Monaco planning system into clinical oncology internships, using Mini-CEX and DOPS for formative evaluation. The experimental group outperformed the traditional group across all key assessment domains. Firstly, guided by instructors, the experimental group utilized the Monaco planning system to perform clinical tasks (eg, target delineation, plan verification) throughout the radiotherapy workflow. This immersive approach enhanced understanding of treatment processes and allowed instructors to reallocate saved time to communication skills training. This dynamic engagement empowered students to seek timely consultation during radiotherapy procedures, thus developing superior consultation abilities compared to the traditional group. Secondly, during their internship, the experimental group utilized the Monaco planning system to holistically engage in the radiotherapy process and study medical imaging. Access to its rich patient case library enhanced their understanding of physical examination-imaging correlations. Through direct patient interaction and active participation in pivotal stages—including target delineation, radiotherapy verification, and treatment execution—this immersive experience improved their knowledge reserve level, specialized knowledge acquisition, physical examination skills, and clinical diagnostic abilities. Consequently, they obtained informed consent more smoothly, demonstrated superior preprocedural preparation and postprocedural management, and exhibited enhanced clinical operations and technical capabilities. Thirdly, the use of the Monaco planning system cultivated heightened responsibility and compassion in the experimental group beyond traditional methods. This fostered a robust patient-centered philosophy and profound understanding of patient suffering, enabling the provision of appropriate pain relief and sedation. Furthermore, their enhanced awareness of tumor patients’ immunocompromised status led to rigorous emphasis on aseptic technique during radiotherapy. Collectively, these competencies resulted in superior humanistic care literacy and elevated comprehensive ability relative to the control group. Moreover, the incorporation of the Monaco planning system-based teaching mode by the experimental group showed vivid and authentic learning experience. It not only empowered interns to comprehensively grasp the pertinent theoretical and practical aspects of radiotherapy but also ignited their passion for the field. Consequently, the experimental group surpassed the control group in both theory examination and skill examination, earning not just higher scores but also securing a higher satisfaction rating from interns. Thus, the Monaco planning system-centered teaching approach proves to be an effective means of enhancing interns’ knowledge mastery and elevating overall teaching satisfaction.
Consequently, this study effectively demonstrated the superior benefits of the Monaco planning system-integrated radiation oncology teaching model for intern training, using Mini-CEX and DOPS assessments for preliminary validation. The present study represents a novel exploration and endeavor aimed at enhancing the quality of radiation oncology teaching. This innovative teaching mode offers interns vivid and interesting content, effectively breaking away from traditional indoctrination-based methods. By actively engaging students in the teaching process, it better caters to the specific needs of radiation oncology teaching, thereby improving its overall effectiveness and contributing to the cultivation of proficient and passionate young doctors. Nevertheless, despite the progress made, there are still certain limitations to be addressed. Firstly, due to the limited number of undergraduates assigned to our department for radiation oncology internships during the study period (total 40 students), data saturation was not achieved in this study. While our data provided valuable insights, the lack of saturation may limit the completeness of the findings. In future research, we will expand the sample size to further investigate and confirm the conclusions of this study. Secondly, no existing studies have incorporated the Monaco planning system into radiation oncology teaching. The conclusions of this study need further validation in other radiation oncology centers. Addressing these challenges will be a priority for future improvements.
Conclusion
Within our institutional context, the Monaco planning system-based teaching model demonstrated significant improvements in radiation oncology education compared to local traditional teaching methods. The integrated Mini-CEX/DOPS assessment framework proved effective in enhancing clinical skill development, increasing student engagement, and elevating clinical teaching quality. While our department has validated this model's feasibility for radiation oncology internships, broader implementation requires further validation across diverse educational settings.
Supplemental Material
sj-doc-1-mde-10.1177_23821205251372109 - Supplemental material for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study
Supplemental material, sj-doc-1-mde-10.1177_23821205251372109 for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study by Hongbo Zou, Qichao Xie and Bijing Mao in Journal of Medical Education and Curricular Development
Supplemental Material
sj-pdf-2-mde-10.1177_23821205251372109 - Supplemental material for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study
Supplemental material, sj-pdf-2-mde-10.1177_23821205251372109 for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study by Hongbo Zou, Qichao Xie and Bijing Mao in Journal of Medical Education and Curricular Development
Supplemental Material
sj-pdf-3-mde-10.1177_23821205251372109 - Supplemental material for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study
Supplemental material, sj-pdf-3-mde-10.1177_23821205251372109 for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study by Hongbo Zou, Qichao Xie and Bijing Mao in Journal of Medical Education and Curricular Development
Supplemental Material
sj-pdf-4-mde-10.1177_23821205251372109 - Supplemental material for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study
Supplemental material, sj-pdf-4-mde-10.1177_23821205251372109 for Practice and Evaluation of Competence in the Application of Monaco Planning System in Undergraduate Radiotherapy Internship Teaching: A Randomized Controlled Study by Hongbo Zou, Qichao Xie and Bijing Mao in Journal of Medical Education and Curricular Development
Footnotes
Acknowledgments
The authors wish to thank the Chongqing Medical University for providing student's data.
Ethics Approval and Consent to Participate
Ethics Committee of the Third Affiliated Hospital of Chongqing Medical University has confirmed that no ethical approval is required (Supplemental Material 3). All methods were carried out in accordance with relevant guidelines and regulations. Informed consent was obtained from all subjects and/or their legal guardians
Authors’ Contributions
ZHB and MBJ carried out the studies and participated in collecting data. MBJ drafted the manuscript. ZHB performed the statistical analysis. ZHB, MBJ, and XQC participated in the acquisition, analysis, or interpretation of data. All authors read and approved the final manuscript.
Funding
This study was funded by the institutional fund project of The Third Affiliated Hospital of Chongqing Medical University (Project No. KY23071 and KY24040) and 2022 Research Project of Postgraduate Education and Teaching Reform of Chongqing Medical University (No. xyjg220222) to Mao bijing.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Declarations Availability of Data and Materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
