Abstract
Classroom examinations and a passing score on a sonography certification examination are means to assess a student’s retention of material presented during didactic education and further studying. Typically, however, clinical competencies are graded based on actual observation of scanning skills and the ability to identify specific organs and associated abnormalities. We are reporting an analysis of the data collected from students’ completed abdominal competency evaluations over the past four years to determine what organ was the easiest to learn to scan, what organ was the most difficult to learn to scan, and in what organ(s) was pathology most often recognized or missed.
Introduction
Sonography is an imaging modality that is highly operator dependent. Acquiring the scanning skills to competently perform a diagnostic ultrasound examination takes time, didactic education, and hands-on clinical scanning instruction.1-3 Clinical or technical competence may be defined as possessing the knowledge, skills, and training to consistently perform a particular action at a professionally acceptable level. 4 Although national board certification is the standard of practice in ultrasound, it does not necessarily assure clinical competence. Sonography education programs accredited by the Commission on Accreditation of Allied Health Education Programs (CAAHEP) must have an evaluation tool to assess the clinical competence of their students. 5 These evaluation tools, which are referred to by a variety of titles such as clinical competency, examination competency, or test out, are unique to each sonography program. They should, however, incorporate protocol guidelines established by national professional organizations and recommendations from the Scope of Practice for the Diagnostic Ultrasound Professional.6,7 The clinical competency forms used at our institution in the CAAHEP-accredited sonography program have been in place for many years and serve as an effective tool to evaluate a student’s clinical competence. In addition to this evaluation function, the scored data from these competencies offer an opportunity for programmatic self-analysis, a retrospective means to look for strengths and weaknesses in our students’ clinical performance, and the opportunity to improve the effectiveness of our clinical education.
The purpose of this study was to analyze data collected from completed abdominal competency evaluations over the past four years to determine what organ was the easiest to learn to scan, what organ was the most difficult to learn to scan, and in what organ(s) was pathology most often recognized or missed.
Materials and Methods
Institutional review board approval for exempt status was granted for this educational research project. Data were retrospectively collected from all students’ Abdominal Competency Evaluation Forms for the past four years. Only competency forms referable to an abdominal ultrasound examination completed by the student while being evaluated by a registered sonographer to formally assess the student’s knowledge, skills, and training to perform the examination as a competent entry-level sonographer were included. A total of 29 students graduated from the bachelor of science program between 2010 and 2013 (7 in 2010, 7 in 2011, 7 in 2012, and 8 in 2013). Each student was assessed on three abdominal competencies for a total of 87 competencies. Abdominal Competency Evaluations were performed in the spring/second semester of a 12-month, 3-semester program. All students were exposed to the same fall semester course curriculum and clinical rotations were equivalent in terms of hours and clinical site placement. The abdominal examination competencies were performed in the ultrasound department at a 627 acute-care bed facility providing solid organ and bone marrow transplantation services. Examinations were performed using a Philips HDI 5000, Philips IU22 (Philips Healthcare, Bothell, Washington, USA), or Toshiba Aplio 300 (Toshiba America Medical Systems, Inc., Tustin, California, USA) ultrasound system. Competencies were performed only on outpatients with an indication of liver disease or increased liver function tests (LFTs). Students were randomly assigned a patient and an evaluator by the clinical coordinator. Only American Registry for Diagnostic Medical Sonography (ARDMS–Abdomen) registered sonographers performed the evaluations; their level of scanning experience ranged from 2 to 22 years, calculated by subtracting their registry date from the year the evaluation was done. The evaluator was with the student for the duration of the abdominal examination competency including taking a medical history, performing the sonogram, and presenting the case to the radiologist. The evaluating sonographer also rescanned the patient before the case was presented to the radiologist. Evaluation forms were completed by the evaluating sonographer after the scan and then submitted to the program director.
Information was collected from two sections on the competency form: the normal anatomy section and the pathology section. Normal anatomy data used for study analysis included the ability to take a complete series of diagnostic images of the organs that are part of the American College of Radiology’s complete upper abdominal examination: kidneys, liver, gallbladder/biliary system, pancreas, and spleen. 6 Evaluators scored the student by selecting “yes” if the organ was thoroughly evaluated with a complete series of diagnostic images, “yes with qualifications” if the series of images was missing anatomical information or landmarks, “no” if the images taken were not diagnostic, and “NA” or not applicable if the organ was not present (Figure 1). The frequency and percentage of “yes” values were used to determine which organs were the easiest and most difficult to learn to image. Total possible yes values varied per organ based on the number of scan planes, the number of organs or lobes included in a category, and whether the organ was present. Data totals for the kidney included both the right and left kidneys. Data totals for the liver included both the right and left lobes. For this study, the total yes values possible per organ were kidneys, n = 348; liver, n = 348; spleen, n = 170; pancreas, n = 169; common bile duct, n = 86; and gallbladder, n = 68.
Abdomen competency form used for student evaluations for scoring the ability to image normal anatomy.
Recognition of abdominal pathology was also analyzed. A “yes” was scored if all pathology was found by the student. A “yes with qualifications” indicated that the student found some pathology but not all of the pathology was present, a “no” was scored if the student missed all of the pathology, and “NA” indicated that no pathology was present. Pathology was confirmed by the evaluating sonographer during the scan and by the radiologist’s report. The credentialed sonographer was required to record on the competency form all pathology that was found and/or missed (Figure 2).
Abdomen competency form used for student evaluations for scoring the ability to recognize and image pathology.
Results
Overall, the organ receiving the highest frequency of yes scores indicating a complete series of diagnostic images was the kidney with 87.9% (right kidney 89%, left kidney 88%). Sonography students received the lowest percentage of yes scores for the common bile duct (CBD) at 62.8%. Percentages of yes scores for the other abdominal organs evaluated included the spleen 84%, pancreas 79%, gallbladder 72%, and liver 65% (right lobe 56%, left lobe 74%) (Figure 3). The higher scores in the kidney and spleen, indicating that these organs are easier to learn to scan, may be the result of the ease with which they are swept through once an adequate sonographic window is acquired and that there is less need to adjust equipment controls (such as time gain compensation, focus, and depth) once a technique is set. Feedback from the sonographers for those students scoring a yes with qualifications or a no was primarily related to suboptimal images caused by a rib shadow in the image or not repositioning the patient to find a better scanning window.
Results for each organ for the percentage of “yes” grades given by the evaluators, indicating complete anatomic evaluation of the organ. CBD, common bile duct; GB, gallbladder.
The liver and the CBD proved to be the organs with the lowest percentage of yes scores and the most difficult to learn to scan. The lower scores in the liver could be related to a number of factors. The liver is the largest organ in the abdomen and complete sonographic assessment requires evaluation of more anatomic landmarks and vasculature than that of the kidney or spleen. Difficulty was reported in obtaining a complete series of diagnostic images of the liver dome, the posterior segment of the right lobe, and the lateral segment of the left lobe. The liver also requires more adjustments of instrument settings. Depth and focal zone must be adjusted in transition from the deeper position of the right lobe to the more anterior left lobe. Attenuation of the sound beam in the posterior segment of the right lobe requires more manipulation of controls related to increasing penetration and gain compensation. This was especially true in the patient population used for these evaluations, whose indications were increased liver function tests or liver disease with related hepatic steatosis or cirrhosis. Also, multiple scan windows, subcostal and intercostal, are needed to completely assess all hepatic lobes and segments. Students did score a higher percentage of yes on imaging the left lobe of the liver (73.6%) versus the right lobe (56.3%), which may be due to the left lobe’s more anterior position in the body. Because of its size and location, imaging of the CBD is also complex and requires extensive knowledge of relational anatomy to the porta hepatis and the head of the pancreas, advanced technical skills, and adjustment of patient position to work around bowel gas. Comments from the competency forms cited inability to locate the CBD or failure to elongate as the most common reasons for not receiving a yes value. It was also noted that students sometimes failed to use left lateral decubitus (LLD) or left posterior oblique (LPO) positions to improve the image.
It was noted that the pancreas scored higher for a complete evaluation than the gallbladder. Due to overlying gas in the stomach and bowel, pancreatic imaging can be challenging even for experienced sonographers. The evaluators’ ability or inability to image the pancreas may have affected scoring for this organ. Comments for improvement in this area focused on better use of vascular landmarks, more compressive pressure on the abdomen by the transducer, and more transducer manipulation for demonstrating the entire pancreas. Whereas the gallbladder was scored low for a complete evaluation (72.1%), it was the organ that scored the highest for recognition of pathology. Remarks for improvement were primarily related to the student forgetting a positional view, the majority being prone, or no image showing the entire organ from neck to fundus.
Pathology was present on 86.0% (75/87) of the abdominal competencies; 71% (53/75) had multiple pathologic findings by sonography. Pathology frequency distributions were fairly consistent over the three required competencies with percentages at 86.2%, 82.8%, and 89.7%, respectively, providing a good test for the assessment of clinical skills (Table 1). Recognition of pathology was highest in the gallbladder, with a 92% success rate. Pathology missed included adenomyomatosis and wall thickening. The pancreas had the lowest percentage of pathology recognition at 20.0%, conflicting with its higher rank in complete organ evaluation. Failure to recognize abnormal echogenicity was the pancreatic pathology most often missed. Common bile duct pathology also scored low at 33%, but this was consistent with the comments related to the students’ inability to adequately scan this structure. It should also be noted that the number of patients with these particular types of pathology was low. Liver pathology was present in more than 50.0% of the patients scanned during the competencies, but the pathology was detected in only 69.4% of the cases. Students performed well in the recognition of parenchymal changes related to chronic liver disease: nodularity, fatty infiltration, and heterogeneity. Recognition of para-umbilical varices proved difficult in many cases. The kidney and spleen scored more favorably for pathology detection at 81.0% and 74.1%, respectively (Figure 4). Renal cysts were the most common pathology and were documented in all but two of the cases. Splenomegaly was the most common splenic pathology and was identified by the student in all competencies. Evaluator comments indicated splenules, perisplenic fluid, and hilar varices as the pathologies not recognized during the examinations.
Number of Patients in Each Competency Found to Have Pathology.
Results for each organ for the percentage of “yes” grades given by the evaluators, indicating the students’ ability to completely identify pathology. BD, common bile duct; GB, gallbladder.
Discussion
Acquiring the skills to competently perform an abdominal sonogram requires significant education, hands-on training, and scanning experience. In addition to knowledge about anatomy and pathology, the sonography student must also incorporate the ability to optimize equipment controls and maneuver the transducer to thoroughly evaluate an organ and produce quality images. Based on scoring from the normal anatomy and pathology sections of the abdominal clinical competencies, students and sonographer evaluators found that some abdominal organs are easier to learn to scan than others, the kidney and spleen being the least difficult, and the liver and common bile duct the most difficult. For the most part, this was also reflected in their ability to detect pathology in these organs. Based on these results, curricular modifications in the students’ scanning laboratories can be recommended related to the organs that are more difficult to learn to scan. Additional laboratories for the liver should focus on equipment optimization for better penetration, adjustment of transducer and patient positioning for obtaining better scan windows, and more emphasis on sonographic landmarks to ensure a complete evaluation of all segments and lobes. Improvements in CBD and pancreatic imaging could be achieved with added practice on how to image structures that are affected by overlying bowel gas and more attention to understanding relational anatomy. Improvements in these scanning skills should also improve the students’ ability to detect pathology.
Although a literature search found no other studies similar to ours, it is expected that other CAAHEP-accredited programs would have similar findings to these. Clinical instructors may also recognize this pattern of clinical learning anecdotally. It would be beneficial for these results to be reported to sonographers and clinical instructors at the clinical sites so they could provide feedback to the students while scanning actual patients.
There are some potential limitations to this study. Foremost is the subjectivity associated with scoring and grading clinical competency examinations. 8 Variations in patient body habitus and pathology type make some examinations more difficult than others. Given that recognition and documentation of pathology is the ultimate goal of the abdominal sonogram, to better evaluate the students’ ability to perform an abdominal sonogram, only patients with an indication of increased LFTs or liver disease were scanned for a competency. Although patients were assigned randomly, 3 of the 29 students had pathology present in only one of their competencies; 20 of the students had pathology in all three of their competencies. Sonographer/clinical instructor experience also may have influenced how the students were scored on the competencies, with higher scores coming from the less experienced sonographers. This scoring discrepancy could be addressed by providing an annual in-service to all sonographers/clinical instructors who perform competencies concerning grading criteria for each organ. Another recommendation would be the use of standardized patients for competency evaluations whenever possible.
In conclusion, diagnostic sonography is an operator-dependent imaging modality that requires technical skill and knowledge of anatomy and pathology in order to provide a complete examination leading to the correct interpretation and diagnostic results. The inability to thoroughly and completely evaluate an organ can lead to an incomplete examination and incorrect diagnosis, affecting patient outcomes. Based on findings from our study, organ-specific improvements in scanning laboratory goals and assignments can be made to better meet the clinical educational needs of sonography students enrolled in sonography programs.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.