Abstract
Objectives:
The aim of this study was to assess the predictive value of the Vandenberg and Kuse Mental Rotation Test (MRT) on performance of novice medical students for manipulation of a nasal endoscope on a cadaveric model.
Material and Method:
We randomly selected 39 medical students who had never handled a nasal endoscope and subjected them to the MRT. General information including experience in manual, technical, or surgical activities and testing of anatomical knowledge were collected to exclude possible confounding factors. They were then asked to perform series of cadaveric model exercises using a nasal endoscope. Their cadaver performance was evaluated by 2 blinded observers, using a standardized scale.
Results:
We found that medical students with higher mental rotation skills had significantly increased endoscopic sinus performance (P = .0002 using multivariate regression adjusted for specialty choice, previous surgical exposure, and anatomy knowledge). Higher anatomy knowledge was also associated with better endoscopic sinus performance (P = .0141). Other parameters had no impact on endoscopic sinus performance measured by the endoscopic scale (P > .05).
Conclusion:
The score obtained on the MRT was correlated with the practical performance of manipulating the nasal endoscope in cadaver. It could therefore be a useful spatial ability tool for directing targeted training in rhinology.
Keywords
Introduction
Practical training in endoscopic sinus surgery mainly relies on companionship, anatomic knowledge, and complex coordination of hands movements with deported endoscopic visualization on a screen in a compact 3-dimensional space. 1 Advances have been made in the development and validation of educational tools to assess surgical skills in Otolaryngology–Head & Neck Surgery.2-4 The objective structured assessment of technical skills proposed by Reznick et al is the main reference in that field and many similar assessment tools have been developed and validated for operative procedures in other surgical specialties.2,3 Indeed, objective assessment of a trainee’s abilities and progress in learning surgical skills would allow to focus on specific areas of needed improvement and remediation, thus improving teaching methods.2-5 Regarding spatial abilities, Vandenberg and Kuse’s Mental Rotation Test (MRT) is one of the most widely used tools.6-9 It conventionally consists of 24 points representing 2-dimensional drawings of 3-dimensional geometrical figures. These drawings were adapted by Vandenberg and Kuse from the original studies of Shepard and Metzler7,8 on mental rotation. Each point consists of a row including a reference figure in the left position followed by 4 figures: two of them are rotated reproductions of the left figure and 2 are figures of different structure (Figure 1). The patient’s aim is to identify correct reproductions of the reference figure. 6 This test has already proven its interest in different fields such as aeronautics 10 or different medical fields including anatomy learning, abdominal endoscopic surgery, urology, robotic surgery, anesthesiology, and health simulation.11-17 In 2018, Vajsbaher et al showed that spatial cognition was found to play an important role in minimal invasive surgery, with mental rotation showing a specific significance. 18 Numerous references in the literature also report relation between space performance and gender.6-9,19-21 To our knowledge, there are no studies evaluating the applicability of this tool in the evaluation of performance in nasal endoscopy. The aim of this study was therefore to demonstrate the predictive interest of the MRT on the practical performance of novice students on a cadaveric model using a standardized scale. 1
Examples of items with drawings of 3-dimensional geometrical figures. Each Mental Rotation Test (MRT) item consists of a row of 5-line drawings, including a target figure in the left position followed by 4 choice figures. The patient’s task is to indicate which of the 4 response-choice figures are rotated reproductions of the left figure.
Material and Method
This prospective cohort study evaluated mental rotation skills of 39 inexperienced medical students and their endoscopic sinus performance in the cadaver lab of our University Hospital.
We recruited randomly volunteers who had never handled a nasal endoscope. Informed consent to the study purpose and the terms contained in a General Data Protection Regulation form was obtained from each patient. They were then summoned for a session where they were firstly submitted to the MRT and then were asked to perform simple exercises using a nasal endoscope on a cadaveric model. Before participating in this session, they were subjected to a survey aimed at defining their general profile: age, sex, prospective surgical orientation for residency, previous surgical clerkship, endoscopic general surgery exposure, sinus surgery exposure, endoscope handling experience, technical procedure experience, last year of medical school, manual hobby, job as student tutors, scrub into operative cases experience, surgical simulator experience, and videogame use. In addition, they were tested for rhinosinusal anatomy to complete the exclusion of possible confounding factors.
The cadaver performance was measured with a standardized scale to evaluate movement in the 3 planes of space, precision of gesture, and dexterity. We used an adapted version of the 7-point scale established by Lindquist et al 1 , adapted from Laeeq et al.1-3 This is a standardized Global Rating Scale (GRS) using a 5-point Likert-type scale, adapted from Laeeq et al, and published in 2019 in the Ear, Nose and Throat journal for the evaluation of Simulator-Based Teaching Methods of endoscopic sinus surgery. This scale includes different dimensions allowing for the measurement of the installation and the setting up of the equipment, the control of the camera manipulations, the quality of the exposure, the tremors, the control of hands movements, the overall quality of the manipulations, and the completion of the requested task. The overall score is calculated as the sum of the scores from the 7 subdimensions. This scale was tailored to correspond to the clinical imperatives we are confronted with in our daily reality. Tasks to perform on cadavers were basic to photograph the piriform aperture with the camera, to photograph the nasopharynx, to touch the middle turbinate’s head with a surgical instrument, to touch the lower turbinate’s head with a surgical instrument, to place a swab in the nasopharynx and to remove of it, to place a swab in the middle meatus and to remove it, to photograph the maxillary sinus, and to place a swab in the maxillary sinus and to remove it. Performance was assessed by 2 senior rhinologist surgeons blinded to the MRT results and to each other.
Regarding the data analysis, first of all interrater agreement was assessed by 2-way mixed intraclass correlation coefficients (ICC) for the 7 dimensions and total scores of the Lindquist’s scales. To facilitate the evaluation, sessions were video recorded for each participant. The distribution of quantitative variables was investigated numerically and graphically. Shapiro-Wilk normality test completed this normality investigation. Quantitative variables were summarized by using median and interquartile range (P25-P75), whereas qualitative variables were described with frequency and percentage. In order to detect potential confounding factor, association between student’s characteristics and MRT were tested using Spearman test for quantitative characteristics and using Mann-Whitney U test for qualitative characteristics. In a first step, association between MRT and sinus endoscopic performance was tested using Spearman’s correlation test. In a second step, univariate tests—Spearman’s correlation coefficient for quantitative variable and Mann-Whitney U test for qualitative variable—were performed between student’s characteristics, mental rotation skills, and anatomic knowledge and endoscopic sinus performance score by Lindquist’s scale. Finally, multivariate regressions included significant factors in univariate analysis were applied to determine parameters that influence endoscopic sinus performance scored by the Lindquist’s scale and mental rotation skills. Statistical analysis was conducted using SAS 9.4 software. Result was considered significant at the uncertainty level of 5% (P < .05).
In this study, we made the hypotheses that MRT is significantly correlated with the 7 dimensions of the scale and that the total score/MRT is a significant predictor of the 7 dimensions and the total scores adjusted for other confounding factors.
Results
Interrater Agreement
Two independent observers evaluated the performance of the students based on the 7 dimensions of the Lindquist scale. Two-way mixed ICCs absolute agreement, revealed almost excellent interrater agreement for all of the dimensions (ICCs > 0.9), except for the task completion (ICC = 0.886). The 2 judges were also strongly in agreement with each other regarding the total scores, which were the sum of the 7 dimensions (Table 1). On this basis, the final scores of performances were calculated for each student by taking the mean of the scores given by the 2 observers.
Interrater Agreement by 2-Way Mixed Intraclass Correlation Coefficients (ICCs) for the 7 Dimensions and Total Scores of the Lindquist’s Scale.
Univariate Analysis
Endoscopic sinus skills scored by Lindquist scale, MRT results, demographics, education, and training history are summarized in Tables 2 and 3. In order to uncover potential confounding factors, association between each baseline parameter and MRT (Table 3) and association between each baseline parameter and endoscopic performance scored by Lindquist scale (Table 4) were screened by univariate analysis. None of the potential confounding factors did reach significance to influence MRT scores. Mental Rotation Test scores were homogeneously distributed across participants’ baseline characteristics (P > .0500).
Endoscopic Sinus Skills Scored by Lindquist Scale and Results of the Mental Rotation Test (MRT).
Demographics, Education, and Training History of the Participants.a,b
Abbreviation: MRT, Mental Rotation Test.
a n = 39.
b In order to uncover potential confounding factors, association between the results of the Mental Rotation Test and each of these parameters is presented by the P value obtained by univariate analysis. None of these potential confounding factors did reach significance. Mental Rotation Test scores were homogeneously distributed across participants’ baseline characteristics (P > .05).
Univariate Analysis Screening for the Association Between Baseline Characteristics of the Participants and Their Endoscopic Performance on the Total Lindquist Score.a
a n = 39.
Mental Rotation Test scores were significantly associated with the endoscopic performance on a cadaveric model (P = .0251). Thus, the higher the mental rotation capacity, the higher the endoscopic score tended to be (r = 0.3580). However, higher anatomy knowledge, surgical orientation, surgical clerkship, and sinus surgery experience were also associated with better endoscopic sinus performance on the Lindquist scale (Table 4). These significant factors for endoscopic performance were introduced into a multivariate regression model modeling the overall score.
Multivariate Analysis
Medical students with higher mental rotation skills had significantly increased endoscopic sinus performance measured by the Lindquist scale (R2 = 0.797; P = .0002, using multivariate regression adjusted for specialty choice, previous surgical exposure, and anatomy knowledge). Higher anatomy knowledge was also associated with better endoscopic sinus performance (P = .0141). Other parameters had no impact on endoscopic sinus performance measured by the Lindquist scale (P > .0500, Table 5).
Multivariate Regression Modeling the Total Lindquist Score as a Function of the Univariate Significant Parameters—R2 = 0.797.a
a n = 23.
Subanalysis of the 7 subdimensions of the Lindquist scale is presented in Table 6. Results of the MRT were significantly associated with some of the subdimensions of the Lindquist scale. Thus, the higher the score on the MRT, the higher the score obtained for the quality of exposure (r = 0.516, P = .0008). The same was found for the scores obtained for the motion flow (r = 0.395, P = .0128) and for the completion of the task (r = 0.444, P = .0047). The scores obtained for the equipment setup, for the camera handling, for the control of tremors, and for the quality of the manipulations were not significantly associated with the results of the MRT (P > .05).
Multivariate Correlation Between the Mental Rotation Test Results and the Total Lindquist Score as Well as the Individual Dimensions of the Lindquist’s Scale.a
a n = 39.
Discussion
Increased mental rotation ability was associated with enhanced technical performance in a sinus endoscopy cadaver test, even after multivariate regression adjusted for specialty choice, previous surgical exposure, and anatomy knowledge. Although our study did not identify confounding parameters influencing the MRT result, anatomical knowledge was the other parameter associated with better endoscopic score. Anatomical knowledge therefore appeared to be as important as spatial orientation skills.
This study is contemporary with increased pressure on doctors and hospitals to prove the safety and competence of practitioners at different levels of patient care. It is in this context that various stakeholders such as accreditation and licensing bodies, as well as the public body that funds medical trainees in particular, are increasingly demanding objectivity in terms of measuring competence, especially procedural performance, before allowing procedures on patients. This study identified a tool that can be used to assess psychomotor skills performance. So far, available evidences concerned 2 main tools: the 7-item GRS and the Procedure-Based Assessment. 13 Some authors reported outcomes from a redrawn version of the MRT and for alternate versions of the test.7-10 Some studies showed high variability within medical disciplines and provided evidence for differences in profiles between surgical and medical specialties.22-25 Our findings were in line with previous reports demonstrating that mental rotation skills may predict novice performance of surgical tasks.14-17 The findings did predict initial endoscopic sinus skills; however, they did not predict the ability to learn endoscopic skills. This is paramount as many skills are teachable and the MRT test alone cannot be used as a screen for an applicant’s ability to perform endoscopic sinus surgery.
In the literature, it also appears that males perform better than females on MRT, and physical sciences students perform better than social sciences students. 9 However, training has a significant effect on the evolution of test results and seems to erase this gender difference. 9 Our study did not find this gender difference, which may be due to the small number of participants which is the main limitation of our study. Our study was limited to naive medical students and their endoscopic performance; therefore, our findings cannot directly be generalized to more experienced surgeons. The lack of correlation with almost all of the other variables except for the MRT also deserves more deliberate discussion and consideration in future study designs. It would therefore be useful to conduct this study on a larger sample by employing an experimental design on a surgical learning model in order to measure changes in MRT scores and other variables. We hypothesize that several variables would predict and identify possible learning difficulties and could allow for closer companionship, including adapted surgical training in targeted surgical trainees.
Conclusion
The score obtained on the MRT was significantly correlated with the practical performance of manipulating the nasal endoscope in surgery. Mental Rotation Test was significantly correlated with the 7 dimensions of the endoscopic scale even when adjusted for confounding factors. It could, therefore, be a useful tool for educational purposes for candidates specializing in rhinology, helping to predict initial endoscopic sinus skills and to identify possible learning difficulties. It would allow for closer companionship, including adapted surgical training in targeted surgical trainees.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
