Comparison and Accuracy of Carotid Plaque Analysis Between Two- and Three-Dimensional Ultrasound Imaging

Abstract

Plaque characterization using traditional two-dimensional (2D) imaging and/or three-dimensional (3D) ultrasonographic (US) techniques is a new method for evaluating artery wall morphology and plaque risk stratification. The purpose of this study was to assess and compare 2D and 3D US, measuring the interobservation differences for specific plaque-imaging analyses. Phantoms that simulated various types of atherosclerotic plaque pathology were imaged and findings reported by three experienced sonographers. Interobservation agreement and subanalyses were created. For each type of plaque pathology, agreement was moderate; however, conformity increased with the application of 3D US versus 2D US alone. Agreement was best for the identification of fissures, ulcerations, and irregular plaque surface. Advanced sonographic techniques for carotid plaque imaging provide a reproducible method in the analysis and morphologic assessment of simulated carotid atheromatous lesions, with superior interobserver variability. Three-dimensional US improves visualization of some pathologies and may provide additional information in the evaluation and risk stratification of vulnerable carotid plaque.

Characterization of carotid atherosclerotic plaque morphology by sonography has shown great potential to increase the amount of diagnostic information traditionally gathered with routine carotid Doppler examinations. Sonographic imaging has shown the ability to visualize plaque directly and quantify certain features such as surface morphology, plaque geometry, and echotexture, using two-dimensional (2D) gray-scale B-mode imaging. Improved methods are being investigated to identify potential markers of high-risk plaques, in the hope of identifying specific imaging parameters to help describe a patient’s overall vascular risk. These modalities include the use of high-resolution sonography using newer high-frequency probes and volumetric three-dimensional (3D) imaging. Studies have reported that 3D sonography used as a complementary imaging technique may provide additional information in the evaluation and risk stratification of vulnerable carotid plaque beyond traditional 2D imaging.^1–5

Carotid 2D and 3D plaque imaging is emerging as the predominant approach in identifying and evaluating the progression of carotid atheromas. ¹ To our knowledge, no published studies have measured the interobserver analysis of various plaque pathologies and assess the value of 3D sonography, in addition to the routinely used 2D plaque imaging. The purpose of this study was to evaluate the utility of advanced sonographic plaque characterization using high-resolution 2D and 3D imaging for morphologic assessment of simulated diseased carotid artery segments. In addition, the interobservational differences of 2D and 3D sonography, respectively, were assessed for specific simulated plaque pathologies.

Materials and Methods

Phantoms

Twelve tissue-like phantoms, with five simulated carotid specimens each, were created using a previously published method to mimic various types of diseased atherosclerotic plaque segments. ⁶ To simulate the internal echotexture of the plaque, a number of items were tested including whole corn kernels, olives, modeling compound (Play-Doh, Hasbro, Pawtucket, RI), and compound putty, in an attempt to re-create the specific appearance of internal carotid artery atheromatous lesions. We determined that a segment of animal meat protein with internal coagulated milk casein provided the best tissue-mimicking echolucency within the model. Mustard seeds were added to some specimens to simulate calcifications. Each phantom specimen was shaped and precisely cut to represent specific pathologies within types of atherosclerotic plaque, with model characteristics including fissured, ulcerated, irregularly surfaced, internally echolucent, and calcified segments. Each prepared specimen was placed on the small finger of an examination glove and held in place using adhesive gel. Once the adhesive had been allowed to dry, the glove was turned inside out, and the finger with the specimen was filled with mineral oil to create a tubular structure that simulated a blood vessel, within the distal portion of the finger (Figure 1). Special care was taken when filling the glove finger with mineral oil to minimize the amount of trapped air and/or small air bubbles. The remainder of the glove was then cut off, resulting in a model with a representative sized (1.0-1.5 cm) segment simulating an atherosclerotic plaque.

Figure 1.

A powder-free examination glove, with the detailed pathologic specimen placed inside the small finger of the glove, was turned inside out and then filled with mineral oil to simulate a vessel.

These models of atherosclerotic plaque were then suspended in tissue-mimicking molds made from a mixture of gelatin, water, and psyllium hydrophilic mucilloid fiber to provide echogenicity comparable to soft tissue (Figure 2). ⁶

Figure 2.

The concoction of powdered mixture is congealed at various intervals to trap the fluid-filled gloves and form layers to mimic the surrounding soft tissue.

Previous analysis of these phantoms for image quality, plaque representation, and reporting of findings were assessed and showed minimal variability between sonographers and interpreters, with appropriate representation of vessel anatomy and/or pathology by sonography. ⁶

Sonographic Examination and Analysis

Detailed sonographic examinations were performed on the phantoms using an iU22 ultrasound system (Philips Healthcare, Bothel, WA) equipped with a linear high-resolution, small-parts probe (L-12 MhZ) for 2D sonographic imaging and a volumetric mechanical, high-resolution linear-array transducer (VL-13 MhZ) for 3D imaging. The assessment began with a systematic overview of the carotid artery phantom specimen using high-resolution 2D imaging. Each sample was evaluated for specific morphologic features: internal echolucency, presence of fissure(s), presence of surface ulceration(s), surface irregularity, internal calcification(s), and no abnormality detected (NAD). All of the phantoms were of fairly equal size, thus the lead sonographer preset the unit’s primary imaging parameters (depth, focus, resolution, gain) to maintain consistency between scans. Time gain compensation was not preset but was adjusted manually by each sonographer to optimize image quality for each specimen.

Each of three registered vascular sonographers, blinded to the phantom’s simulated pathology (if any) and any other sonographic data, independently performed a high-resolution 2D evaluation of the specimens. All three sonographers were trained and experienced in carotid duplex sonography; however, the sonographers varied in their level of experience in the sonographic evaluation of carotid plaque morphology. The senior sonographer had more than 10 years of experience, the second sonographer had 5 years of experience, and the third was a novice sonographer with less than 1 year of experience. Each sonographer reported their 2D findings independently on a worksheet.

The imaging process was repeated using the volumetric 3D probe. On switching to the 3D volumetric imaging mode, scanning parameters such as volume angle, acoustic output power, gain compensation, focus, zoom, and imaging depth were preset for each case by the lead sonographer prior to the acquisition of the 3D image. The 3D probe swept the region of interest of the phantom mechanically with a fixed frame rate of approximately 10 per second, resulting in a final 25-slice volume set. At the end of the scan, the acquired 3D sequential volume-rendered images were stored digitally for further analysis by 3D automated software that was routinely available on the system. Each sonographer evaluated the phantom by rotating the x, y, and z axes in the 3D volume view as needed. To further evaluate the internal echotexture(s), the sonographers could also select precise slices of the volumetric data set. Constructed 3D plaque images were assessed independently and evaluated using the same morphologic features as for the 2D images, and the findings were reported on the worksheet.

Based on a previously published literature review of carotid plaque characterization and earlier experimentation with the phantom model,^1,6 specific imaging criteria were adopted to quantify the pathology of each specimen. The phantom was considered to have NAD or regular when no variations in the phantom’s surface contour larger than 0.3 mm were observed and/or no alterations in the internal echotexture or echogenicity were noted. Internal echolucency was identified as a focal hypoechoic or echolucent internal area within the plaque structure (Figures 3a and 3b). An irregular surface contour was defined as an observed variation greater than 0.3 mm (Figures 4a and 4b). Surface ulceration in the phantom was defined by a focal irregularity or break in the surface of the phantom of greater than or equal to a depth of 3 mm (Figures 5a and 5b). A fissure was considered to be present when there was evidence of a linear break in the surface of the specimen (Figures 6a and 6b). Regions of focal, echogenic foci with the presence of acoustic shadowing were defined as calcifications (Figures 7a and 7b).

Figure 3.

(a) Cross-sectional two-dimensional gray-scale image of a plaque phantom with a simulated internal echolucency (arrows). (b) The same phantom showing the internal echolucency displayed using a single sagittal slice through a reformatted three-dimensional image (arrows).

Figure 4.

(a) Sagittal two-dimensional gray-scale image of a plaque phantom with simulated surface irregularity (arrows). (b) The same phantom showing the surface irregularity displayed using a reformatted three-dimensional image (arrows).

Figure 5.

(a) Sagittal two-dimensional gray-scale image of a plaque phantom with a simulated ulceration (arrow). (b) The same phantom showing the surface ulceration displayed using a reformatted three-dimensional image (arrow).

Figure 6.

(a) Sagittal two-dimensional gray-scale image of a plaque phantom with a simulated fissure (arrow). (b) The same phantom showing the fissure displayed using a reformatted three-dimensional image (arrows).

Figure 7.

(a) Two-dimensional gray-scale image of a plaque phantom with a simulated calcification (arrow) with acoustic shadowing. (b) The same phantom displayed using a single sagittal slice through a reformatted three-dimensional image, in this case showing two small calcifications (arrows).

Statistical Analysis

The degree of agreement among the three sonographers was measured by assessing the concordance or inter-rater agreement between each sonographer’s specific findings. Inter-rater reliability was determined by a joint probability of agreement using the intraclass correlation coefficient (ICC) and Cohen’s kappa (κ) coefficient. Inter-rater agreement or reliability was rated on the following scale: an ICC or κ value of 0 to 0.20 indicated poor agreement; 0.20 to 0.40, fair agreement; 0.40 to 0.60, moderate agreement; 0.60 to 0.80, substantial agreement; and 0.80 to 1.00, very good agreement. Inter-rater agreement was evaluated for each of the six morphologic features (NAD, echolucency, surface irregularity, ulceration, fissure, and calcification) for all levels of experience for both 2D and 3D techniques, followed by subanalyses across levels of sonographer experience (senior, experienced, and novice) and technique (2D versus 3D). In addition, overall sensitivity, specificity, and accuracy were assessed for each morphologic feature and for each technique and level of sonographer experience, using the phantom’s simulated pathology as a reference standard. Statistical analyses were performed using MedCalc Professional statistics software version 12.1.1 (MedCalc Software, Mariakerke, Belgium).

Results

All 60 samples included for the study were evaluated and included in the analysis. All sample data were used to calculate the agreement between 2D and 3D sonography and among the three levels of sonographer experience. Figure 8 shows the agreement (ICC scores) for 2D and 3D scans for specific simulated plaque morphologies; and Table 1 shows the overall strength of agreement among sonographers for the simulated plaque characteristics.

Figure 8.

Level of agreement (intraclass correlation coefficient) for two-dimensional (2D) and three-dimensional (3D) imaging techniques in the identification of simulated pathologic features of atherosclerotic plaque.

Table 1.

Overall Inter-rater Agreement Scores (Cohen’s κ Coefficient) for No Apparent Disease (NAD) and for Each of the Five Types of Simulated Atherosclerotic Plaque Pathology.

		Inter-rater Agreement
Assessment Criterion	Agreement	Cohen κ	95% Confidence Interval
Calcification	Substantial	0.69	0.60-0.78
NAD	Very good	0.85	0.80-0.90
Echolucency	Very good	0.81	0.74-0.86
Ulceration	Moderate/substantial	0.60	0.49-0.70
Fissure	Substantial	0.64	0.54-0.73
Irregular surface	Substantial	0.61	0.50-0.71

In the regular or NAD specimens, the agreement between the 2D and 3D methods was perfect for the novice sonographer (κ = 1.00, 95% confidence interval CI 1.00-1.00) but was substantially lower for the senior and experienced sonographers (κ = 0.65, 95% CI 0.20-1.00).

Overall agreement in the identification of internal echolucency was very good across all levels of sonographer experience (ICC = 0.81, 95% CI 0.74-0.86); however, the agreement increased when sonographers applied 3D sonography (ICC = 0.89, 95% CI 0.83-0.93) compared with 2D alone (ICC = 0.80, 95% CI 0.71-0.87). In this subcategory, the senior sonographer had the best agreement (κ = 0.83, 95% CI 0.69-0.97) compared with the experienced and novice sonographers (κ = 0.76, 95% CI 0.60-0.93).

Observed agreement in the identification of fissures between 2D and 3D was at the lower end of substantial (ICC = 0.64, 95% CI 0.54-0.73) across all sonographers. The level of agreement was increased for 3D alone (ICC = 0.74, 95% CI 0.63-0.82) versus moderate for 2D alone (ICC = 0.59, 95% CI 0.45-0.71). The concordance was highest for the experienced sonographer (κ = 0.836, 95% CI 0.72-0.9) and lowest for the novice (κ = 0.43, 95% CI 0.19-0.67).

Plaque surface irregularity agreement was just in the substantial range (ICC = 0.61, 95% CI 0.50-0.71) across all levels of experience. There was improved agreement with the application of 3D sonography (ICC = 0.66, 95% CI 0.54-0.77) compared with only moderate agreement for 2D alone (ICC = 0.49, 95% CI 0.34-0.64). The senior and novice sonographer yielded very good agreement (κ = 0.88, 95% CI 0.74-1.00; κ = 0.91, 95% CI 0.78-1.0, respectively) compared with the experienced sonographer (κ = 0.53, 95% CI 0.29-0.77).

The observed agreement for calcifications was substantial (ICC = 0.69, 95% CI 0.60-0.78), and accordance was substantial among sonographers for each 2D and 3D technique independently (ICC = 0.78, 95% CI 0.68-0.85; ICC = 0.70, 95% CI 0.59-0.80, respectively). The senior sonographer had the highest agreement (κ = 0.83, 95% 0.69-0.97) compared with the experienced (κ = 0.65, 95% CI 0.45-0.84) and novice (κ = 0.60, 95% CI 0.40-0.81) sonographers.

For phantoms depicting plaque ulceration, agreement across all levels of experience was found to be borderline between moderate and substantial (ICC = 0.60, 95% CI 0.49-0.70). The 3D sonography had substantial agreement for ulceration (ICC = 0.67, 95% CI 0.55-0.77) compared with moderate agreement for 2D sonography (ICC = 0.53, 95% CI 0.38-0.66). The senior sonographer had the highest agreement (κ = 0.89, 95% CI 0.76-1.00) regarding ulceration between 2D and 3D scans compared with the experienced (κ = 0.54, 95% CI 0.31-0.77) and novice (κ = 0.63, 95% CI 0.42-0.84) sonographers.

Discussion

Cerebral thromboembolisms are increasingly being recognized as the source of ischemic strokes, as the emboli typically originate from a carotid atheromatous plaque or from a thrombus originating from the site of plaque rupture. ⁴ Emerging data suggest that the severity of carotid plaque morphology can significantly affect cerebrovascular prognosis. Specific plaque characteristics are associated with neurologic events including a thin or fissured capsule, large lipid core(s), active inflammation, severe stenosis, or a combination of these findings.^5–7 The identification of early morphologic features and characteristics of vulnerable carotid plaque have opened up new possibilities in the discipline of carotid artery imaging for stroke prevention. Imaging modalities that offer potential identification of these features include magnetic resonance imaging, computed tomography, and sonography. Sonography is commonly the first-line imaging modality for the initial evaluation of asymptomatic patients at risk for stroke, transient ischemic attacks, or stroke symptoms. Traditionally, carotid duplex imaging is limited to grading specific severities of stenosis of lesions within the carotid artery. With the advancement of high-resolution transducers and 3D imaging capabilities, sonographic plaque characterization investigations present a new tomographic imaging method in the evaluation of plaque morphology by identifying internal components and the structure of specific plaque appearances and possibly stratifying the risk. Specific markers of possible clinical significance seen with sonography include echotexture, the presence of echolucency, and surface alterations or ulcerations. Besides traditional 2D imaging, the use of 3D sonography imaging techniques may provide additional information in the evaluation and risk stratification of vulnerable carotid plaque.

The purpose of this study was to analyze the ability to recognize morphologic features in simulated carotid artery atherosclerotic plaques using high-resolution 2D and 3D sonography and to demonstrate whether 3D imaging improved the visualization of these characteristics. In addition, the inter-rater reliabilities of the morphologic assessment were assessed using both 2D and 3D sonography. Sixty tissue-like carotid phantoms simulating various pathologic plaque features were created, which provided good sonographic representation of a carotid artery segment. The two minor limitations were noted in creating these phantoms. The first was the occasional damage to the specimen when reversing the glove prior to filling with mineral oil. The second resulting simulated vessel diameter was slightly larger (average 1.2 cm) than a standard carotid artery, which could create trivial increased resolution in imaging. The phantoms were noted otherwise to be good replicas of carotid atheromatous lesions and were easily used for measuring inter-rater variability. Given that the phantoms were used for validity and inter-rater agreement only, we did not measure the speed of sound in the embedding media, scattering coefficients, or attenuation coefficients to determine how closely they truly mimicked soft tissue. All phantoms were scanned within 3 days of creation; however, based on previous experience with these samples, phantoms can last for several weeks when properly refrigerated.^6,8

The senior sonographer’s results yielded the highest inter-rater reliability when all six ratings were considered simultaneously; reliability decreased with fewer years of experience with characterizing plaque morphology. The areas lacking agreement may be due to the lack of understanding and training, specifically in sonographic plaque characterization; it would be of interest to repeat this study after standardized training of sonographers. Based on the type of plaque pathology simulated by the phantoms, the use of 2D imaging alone was not reliably high in identifying each specific pathology. In each case, 3D demonstrated a higher reliability, and consistency was substantially higher in the agreement and type of plaque pathology, irrespective of the sonographer’s experience in characterizing plaque morphology. The identification of surface irregularity and ulceration showed the weakest reliability; however, substantial agreement increased with the application of 3D sonography for both parameters. This remained a consistent finding with the other factors as well when comparing 3D sonography versus 2D sonography alone. While high-resolution 2D sonography was able to identify correctly most vulnerable plaque characteristics, volumetric 3D imaging quantified plaque morphology further, increasing the proper recognition of the pathologic type. In addition, the combined use of rendered 3D imaging and postprocessing volumetric slice selection used as a complement to high-resolution 2D sonography reduced the inter-rater variability, which in turn increased the sensitivity and specificity for each plaque pathology type. Using a combination of 2D and 3D sonography, the identification of internal echolucency had the highest reliability (0.81) while ulceration had the poorest (0.60). The 3D inter-rater reliabilities were significantly higher when using 3D alone versus 2D alone. This was especially evident in the presence of fissures, irregular surfaces, and ulcerations. The 3D method was not as sensitive in identifying calcifications; however, the precision slicing of volumetric display may augment the imaging and diagnostic assessment of the internal content or echotexture of the plaque. The above data indicate that a combination of both 2D and 3D would provide the best results in identifying vulnerable plaque characteristics.

The novel findings from this study are as follows: (1) 3D sonography improves recognition of a plaque’s echographic parameters and may help improve assessment of the vascular risk in all types of atheromatous lesions, thus possibly improving stroke prevention; (2) the use of high-resolution 2D and 3D sonography for advanced imaging of simulated carotid plaque pathology provides a reproducible method for the analysis and morphologic assessment of disease with excellent inter-rater reliabilities; and (3) we hypothesize that combining the severity of stenosis with specific plaque morphology using high-resolution 2D and 3D imaging may help increase the assessment of total risk based on plaque vulnerability and could possibly be of use as a basis for predicting future ischemic events.

These results showed that combined 2D and 3D sonographic imaging can demonstrate high accuracy in plaque morphology assessment; however, this must be further investigated for true atheromatous lesions in human subjects and in all types of plaques. Inter-rater manipulation was not assessed, which is an important variable in the operator dependency of sonography; thus, we cannot make any definitive conclusions about the inter-rater variability and consistency. Further studies are required to evaluate these findings.

Conclusion

Advanced 2D and 3D sonographic imaging of simulated plaque morphology provides a reproducible and reliable diagnostic method for assessing characteristic morphologic changes in carotid plaque atheromas, with excellent inter-rater variability. This prospective phantom study suggests a paradigm shift in the standard protocol in carotid duplex sonographic imaging; however, larger prospective human studies are indicated to corroborate these findings.