Abstract
Purpose
To investigate the intra- and inter-observer reliability of the coronal curvature asymmetry of children with adolescent idiopathic scoliosis (AIS) using the center of lamina (COL) method on ultrasound (US) images.
Methods
A cadaver spinal column phantom which was manipulated to present 30 scoliotic curves of varying severity of scoliotic deformities was scanned using both the US and laser scanner (LS) systems. Three observers of varying experience and skill measured the coronal curvature using the Cobb method on the LS images and the COL method on the US images. All of the measurements were performed twice, with a 1-week interval to reduce memory bias. The intra-class correlation coefficient (ICC), the mean absolute differences (MAD), and the error index (EI) were calculated to determine the agreement on selecting the end vertebrae. In addition, five AIS subjects were scanned using the US system. One observer measured the coronal curvature on the US images twice, and the measurements were compared with the Cobb angle reported in the clinical records.
Results
In the phantom study, the COL method showed high intra- and inter-observer reliabilities, with all ICC values >0.88. The maximum MAD of the COL measurements between different sessions among all observers was <4.1°. The EI values of the US method had similar end-vertebra selections as the LS method. The results of the pilot study showed a high intra-reliability for the US measurements. The measured difference between the Cobb and COL methods was 0.7° ± 0.5°.
Conclusions
The COL method using US images appears to be a very reliable method for measuring the coronal curvature in AIS without the need to expose the patient to radiation.
Keywords
Introduction
Adolescent idiopathic scoliosis (AIS) is a three-dimensional (3D) spinal deformity that is first detected during adolescence with no known causes. The Cobb angle [1] measured from the standing posteroanterior (PA) radiograph is the standard method to evaluate the severity of scoliosis. The Cobb angle is not only used to quantify the magnitude of the curvature but is also used to estimate the risk of progression, decide on treatment options, and evaluate treatment outcomes [2–4]. Therefore, it is crucial that the Cobb angle measurement is a reliable index. Many studies have focused on the accuracy and intra- and inter-observer reliabilities of the Cobb angle measurement, with the measurement errors of the intra- and inter-observer reliabilities reported to be in the range of 3°–8° [5–7]. As digital radiographs become more widely used, the error due to using different measurement tools, such as pencils and protractors, on traditional films has been reduced [7–9]. In addition, computer-aided methods have been developed to reduce the measurement variability and error of the Cobb angle measurements [10–12]. The results of these studies indicate that computer-aided measurements have the ability to decrease the variability by minimizing the subjective factors, such as selection of end-vertebrae and skill level of the observer. Since the reliability and accuracy of any measurement is important in the clinical decision-making process, any new proposed method to assess the severity of the scoliotic spine must be evaluated before clinical acceptance can even be contemplated.
There is growing concern from families and health professionals that patients, especially growing children, are exposed to too much ionizing radiation. The radiation accumulated by AIS patients from long-term follow-up may increase the risk of breast cancer [13, 14]. Thus, alternative non-radiation methods are desirable to reduce the duration of radiation exposure. Ultrasound (US) imaging, which uses a non-ionizing radiation and is a non-invasive diagnostic method, has been applied to the study of scoliosis. The application of US to image vertebrae was proposed more than two decades ago by Suzuki et al. who used this technology to identify the lamina and spinous process [15]. These authors measured vertebral rotation and found a linear correlation with the Cobb angle in untreated patients. US has more recently been applied to improve the orthotic (brace) treatment by scanning and tracing the spinous process angle (SPA) during the brace fitting procedure [16]. In this study, the pre-brace SPA was first measured prior to putting on the brace, and then the in-brace SPAs were calculated when the major pressure pad inside the brace was adjusted to different positions. The optimal location of the pressure pad was determined at the spot where the best curvature correction was achieved according to the differences between the pre-brace SPA and the in-brace SPA measurements. As a result, 62 % of patients in this study benefitted from the use of US imaging. In one study, US was used to detect vertebral dislocation in spondylolisthesis, and the result was only a 1.3-mm (average) difference from that determined with roentgenography [17].
These studies show a positive impact from using US to evaluate spinal deformity. Other studies using cadaver vertebrae and AIS patients have revealed that the laminae are good US reflectors or landmarks to study spinal curvature [18, 19]. However, studies using US to diagnose scoliosis are still limited and their accuracy in coronal measurement is uncertain. In their study, Li et al. [16] only reported the intra-rater reliability analysis of the coronal curvature assessment by US.
The aim of our study was to investigate the reliability of using US to measure coronal curvature of the spine. The experiments were performed on a cadaver spinal phantom. The coronal view images of the spinal phantom were captured by both a laser scanner (LS) system and an US system. The center of lamina method (COL) method was used on the US images and the Cobb method was used on the LS images; the results were then compared. The intra- and inter-observer reliabilities of the coronal curvature measurements were determined using the proposed COL method. We also performed a pilot study on five AIS subjects who were scanned using the US system and whose coronal curvature measurements from the US images were compared to the Cobb angle recorded on the same day.
Materials and methods
Validation of the LS measurements and the X-ray measurements
Prior to the study, a sawbones spine phantom including the vertebrae from T2 to T12 was configured to mimic a typical scoliotic curve. A PA radiograph and a LS image of this sawbones spine phantom were obtained. Two observers measured the slope of the lower endplate of each vertebra from T3 to T12 on this phantom from both of the images.
The spinal phantom
A cadaver spinal phantom (Fig. 1) is composed of seven cervical vertebrae (C1–C7), 12 thoracic vertebrae (T1–T12), and five lumbar vertebrae (L1–L5). A 4-mm-diameter metal wire was threaded through the neural canal of the phantom from the cervical region to the lumbar region. The metal wire acted as a support and allowed us to place the phantom in positions that mimicked a series of severities of spinal deformity. Foam filled the gap between the wire and the specimen to fix the position of each vertebra along the wire. One end of the wire was fixed to an aluminum base which was used to position the phantom at the same location. Colored stickers were pasted on vertebrae to indicate the lamina positions. Alternate red and green colors were used for better identification of the vertebrae. Only the vertebrae from T1 to L5, which spanned 408 mm, were scanned in the study.
The cadaver spinal column phantom.T Thoracic,L lumbar
LS system
The LS system consisted of a laser camera (Minolta non-contact 3D Digitizer model VIVID 910; Konica Minolta, Tokyo, Japan) and a desktop computer pre-installed with Polygon Editing Tool (Konica Minolta) software. The wide light-receiving lenses of the camera were used to capture the 3D data, at an accuracy of ±1.40, ±1.04, and ±0.04 mm on the left–right, cranial–caudal, and posterior–anterior directions, respectively. The coronal view image was captured with the spinal phantom placed inside a custom frame at 1.5 m from the camera. The scanner was set to a “FINE” imaging mode, which was able to capture the image with the resolution of 640 × 480 pixels.
US system
The US system consisted of an Olympus TomoScan Focus LT™ Phased Array instrument (Olympus NDT Inc., Waltham, MA) equipped with a 5-MHz 128-element transducer and a mini-wheel encoder. A laptop pre-installed with the TomoView™ software (ver. 2.9 R6; Olympus NDT Inc.) was connected to the phased array equipment. TomoView™ was used to set up the parameters for data acquisition and to export data for further post-acquisition analysis. The dimensions of the 128-element transducer were 64 (length) × 10 mm (elevation) and the pitch was 0.5 mm (Fig. 2). Fourteen elements were used for depth focusing, and the groups were separated by a pitch interval. A total of 115 groups were formed to capture a 57.5-mm wide image in a single scan. The dynamic depth focusing (DDF) feature was also applied to achieve a better lateral resolution over an extended depth. The encoder was used to record the position of the transducer during data acquisition. The transducer and the encoder were mounted inside a holder, and a custom aluminum frame was built to control the movement of the holder (Fig. 3).
The effective width of the transducer array
The experimental set-up of the ultrasound (US) system.A top crossbar,B vertical bar,C water tank,D horizontal arm
Data acquisition procedures
A total of 15 different curvature configurations were captured, ranging from mild to severe cases (range of coronal curvature angle 10°–50° Cobb angles). At each configuration, a double ‘S’-shaped curve was created, and three procedures were performed to acquire the images: (1) using the LS system; (2) using the US system, and (3) using the LS system a second time to capture a second image. Among the 15 configurations, there were 11 major right thoracic and left lumbar curves and four left thoracic and major right lumbar curves.
Procedure 1
After a spinal curvature configuration was set, the phantom was placed on the top of a table located at the center of the frame. The base of the phantom was aligned with four markers at the corner and the coronal view of the phantom was placed parallel with the XY plane of the frame. The upper end of the 4-mm metal wire was secured to a metal bar which was mounted at the top of the frame. The anterior side of the spine was oriented to face toward the camera. The LS image was acquired and designated as the LS1 data set.
Procedure 2
The spinal phantom was carefully removed from the table. A water tank made of plexiglass was placed on top of the table. The phantom was put inside the tank and placed, as closely as possible, at the same location based on the four corner markers, with the base parallel with the wall of the water tank. The posterior side of the spine faced toward the transducer. The tank was filled with water until the top of vertebra T1 was immersed (Fig. 3). US gel was applied between the surface of the water tank and the transducer to ensure good coupling was obtained during data acquisition. The width of the captured image was assessed to determine if it captured the entire horizontal deviation of the spinal curvature. If a single scan could not cover the curvature, two scans with a 25-mm overlap were performed. The scanning speed of the transducer was kept under 36 mm/s so that a good quality of the image could be obtained.
Procedure 3
After the US image was captured, the phantom was removed from the water tank. The water tank was then removed from the table, and the phantom was placed back to the same position as described in procedure 1. The spinal curvature configuration was not altered during these movements. A second laser scan was performed (LS2). Data set LS2 was compared with data set LS1 to determine if there was any change in the curvature configuration due to movement.
Data processing
The laser scan images (LS1 and LS2) were exported into TIFF format, and the image contrast was optimized by an experienced LS operator using ImageJ software (ver. 1.45; NIH, Bethesda, MD) on a monitor with a maximum resolution of 1,280 × 800 pixels. The 30 LS image files, from both LS1 and LS2, were each assigned a code by using random number-generated software (www.random.org). The 3D US raw data (TomoView™ data format) was transformed to a format that could be imported into MATLAB and transferred to a PC. A custom MATLAB program (ver. R2011a; MathWorks, Natick, MA) was developed to extract the coronal image of the spinal phantom. This data set required two scans to be merged before the program could be run. To export the image, the US data was first projected onto a XZ-plane (sagittal view) using the maximum intensity projection (MIP) method (Fig. 4). Two lines were then drawn along the X-direction on the XZ-plane image. The range between the lines was determined based on the projected positions of the laminae. Based on knowledge of the anatomy, the laminae were defined to be at deeper positions than the spinous process and transverse process along the Z-direction. Finally, the XY-plane (coronal view) image was formed from the 3D data set among the two lines by the MIP method. Fifteen US images were exported at an image ratio of 1:1 and the images were also labeled with random codes to reduce memory bias. To ensure consistent vertebrae naming, the T12 vertebra was labeled on the US images by collaborative consensus of the three observers together prior to the measurements.
The XZ-plane of the US image. SP Spinous process, TP transverse process, L lamina
Observers
Three observers measured the coronal curvatures from both the LS and US images twice with a 1-week interval to reduce recall bias. Observer 1 (O1) had experience reviewing the US images (2 years) and making Cobb method measurements (1.5 years); observer 2 (O2) had experience examining US images and in making Cobb method measurements (1.5 years); observer 3 (O3) had no experience with either LS and US images. Prior to the study, O3 was trained by an expert who had 15 years of experience measuring Cobb angles on radiographs; during this training O3 practiced making measurements on ten radiographs. O3 was also trained by O1 to measure US images and practiced on ten US images prior to the study.
Measurement methods
The LS and US images were measured during two different sessions separated by 3 days in order to obtain blind assessments. The ImageJ software was used to measure the curvature angles. The Cobb method was used on the LS images and the COL method was used on the US images. For the Cobb method measurements, the end vertebrae of each curve (the vertebrae most tilted relative to the horizontal at the cephalic and caudal ends of a curve) were determined and recorded. Lines were drawn along the superior endplate of the upper end-vertebra (UEV) and inferior endplate of the lower end-vertebra (LEV). The angle between these lines formed the Cobb angle (Fig. 5a). For the COL measurement, the most tilted end vertebrae along a curve were selected. Lines were drawn through the centers of the laminae at the end-vertebrae. The angle formed by these two lines was defined as the angle measured by the COL method (Fig. 5b). When the spine had two curves, the bottom end-vertebra of the upper curve would act as the top end-vertebra of the second curve, keeping with the standard method used clinically to describe Cobb angles.
The measurement methods on the cadaver spinal phantom.
Analysis methods
The intra- and inter-observer measurement differences of each method were evaluated using the mean absolute difference (MAD) and standard deviations (SD). The intra-class correlation coefficient (ICC) (two-way random and absolute agreement) was used to analyze the reliability between the two sessions of each observer and to compare the measurements from different observers on each method [20, 21]. The Currier criteria for evaluating ICC values was adopted: very reliable (0.80–1.0), moderately reliable (0.60–0.79), and questioned reliable (≤0.60) [22]. The end-vertebra selections were also analyzed using the error index (EI) [6, 23], with a lower value (close to 0) indicating less variation. The equation used to calculate the EI was
Pilot study
A pilot study was performed after the in vitro experiment to further validate the proposed COL method for AIS subjects. The inclusion criteria were: (1) patients diagnosed with AIS who had not received surgical treatment, (2) the largest Cobb angle was <40°, and (3) the patient had no in-brace radiographs taken on the study day. Approval from the local health research ethics committee was granted, and the recruited subjects signed consent forms. Five subjects were scanned using the same US system. Each subject wore a gown with the back opened to the operator during the scanning process. Positioning was standardized by instructing the subject to sit straight inside the positioning frame and to look forward with their arms crossing on a metal bar in front of their chests to minimize the shoulder blades sticking out. The C7 was located by palpation and labeled as the start point. Ultrasound gel was applied to ensure good coupling between the transducer and the skin. The operator carried out the scan from C7 to L4 vertically. It took approximately 30 s to scan each subject. Observer 1 performed the measurements using the proposed COL method twice with a 1-week interval. The intra-observer measurements were compared, and the COL method was compared with the Cobb angle from the clinical record measured on the same day. The Cobb angle was measured during the routine clinical assessment by experienced clinic staff.
Results
Validation of the LS measurements and the X-ray measurements
The MAD between the measurements (10 vertebrae × 2 observers) from the radiographs and LS images was 0.54° ± 0.38°, and the range of the absolute difference was from 0.03° to 1.07°. This result implied that the Cobb angle measured from the LS images would be similar to that measured from the radiographs.
Motion effect and COL accuracy
It is always possible that the curve configuration will change during movement of the spinal phantom between the LS and US procedures. The average of the Cobb measurements from LS1 to LS2 was used to reduce potential error introduced due to any change of curve configuration to compare with the COL measurements. Based on the average Cobb value, the range of the MAD between the Cobb value from LS1 to LS2 was 2.8°, 3.6°, and 2.8° for the three observers, respectively. The average difference between the Cobb and COL for the three observers was 4.8°, 6.7°, and 5.7°, respectively.
Intra-observer reliability
The intra-observer reliability of both the Cobb and COL measurements of the three observers is shown in Table 1. The average Cobb values from each observer were smaller than their corresponding COL values. The ICC values from both methods among all observers were >0.968, indicating that the intra-observer reliability was very high. In addition, the minimum and maximum MAD ± SD values of the Cobb and COL measurements from session 1 versus session 2 among all observers were 1.23° ± 1.09° and 2.27° ± 1.15° for the COB measurements and 1.98° ± 1.61° and 3.9° ± 2.92° for the COL measurements, respectively. There was no significant difference between the Cobb and COL values, and all the MAD values were <4°, which is similar to the usual acceptable error of the Cobb measurement based on radiographs (5°).
The intra-observer reliability of all three observers
LS laser scanner,ICC intra-class correlation coefficient,MAD mean absolute difference, intra-class correlation coefficient,COL center of lamina,US ultrasound
See Section Observers for more detail
Data are presented as the mean ± standard deviation (SD)
Table 2 shows the intra-observer end-vertebra differences of the Cobb and COL methods. A total of 60 curvatures were measured by the Cobb method (LS1 and LS2) and 30 curvatures by the COL method. The UEV and LEV were investigated separately. In both methods, the error indices were increased from O1 to O3. O1 was the most reliable examiner among the three in terms of identifying the same end-vertebra along the curve, and O2 was better than the O3 in the COL method.
Intra-observer end-vertebra differences between the Cobb method using the LS system and the COL method using the US system
UEV upper end-vertebra,LEV lower-end vertebra
Inter-observer reliability
To analyze the inter-observer reliability, we used the average values of the measurements taken during each of the two sessions using each method. The results are shown in Table 3. Although the ICC values of the inter-observer reliability were lower for the COL method (0.877, 0.977, 0.866, and 0.877; see Table 3) than for the Cobb method (0.941, 0.985, 0.964, and 0.964; see Table 3), the overall values all exceeded the Currier criteria threshold of 0.80, indicating very good agreement. This meant that the inter-observer reliability in all comparison pairs for both methods was very high. The minimum and maximum MAD ± SD values among all four comparisons showed that the Cobb method was more reliable and had less variation than the COL method between observers. Furthermore, the EI results of the inter-observer comparisons varied in the range of 0.91–1.07 for the Cobb method and 0.94–1.26 for the COL method. The EI of the two methods was not significantly different.
The inter-observer reliability of all three observers
Data are presented as the mean ± standard deviation (SD)
Pilot study results
Five subjects (4 females, 1 male; mean age 13.6 ± 1.1 years; Cobb 25° ± 8.6°) participated in this study. Figure 6 shows the US image and the corresponding PA radiograph of one AIS subject. Only the major curve was measured. Table 4 shows the results of two measurements taken by O1 (COL1 and COL2, respectively) 1 week apart. The MAD ± SD of the coronal curvature angle between the average COL method and the Cobb method was 0.7° ± 0.5° (range 0.2°–1.4°).
Measurement on the radiograph (
Comparison of the coronal curvature angle determined using the COL and Cobb method
COL1, COL2, Two COL measurements taken 1 week apart; COL_Avg, measurement using the average COL method
The MAD was also calculated
Discussion
The ICC statistic is commonly used to assess the reliability of the Cobb method. ICC values for Cobb measurements reported in the literature range from 0.87 to 0.99 for intra-observer reliability and from 0.87 to 0.98 for inter-observer reliability [9, 24–26]. In our study, the intra- and inter-reliabilities of the COL method were found to be as high as those for the Cobb measurements reported in these studies [9, 24–26]. In terms of Cobb angle measurements, the end-vertebrae selection is considered by some authors to be a major source of error [6, 9, 23]; in these studies, the higher the EI value, the more varied the end-vertebra selection. In our study, both the intra- and inter-observer EIs were comparable with those reported previously [6, 23] and the intra-observer EI values of the COL method were similar to those of the Cobb method for all three observers. Perfect agreement gives an EI of 0. A score of 1 could be achieved if the end-vertebrae selected differed by one vertebrae level for 50 % of the selections or by two levels for 12 % of the selections. Selecting end-vertebrae on US images is similar to selecting end-vertebrae from radiographs. We also found that the EI and MAD values of the COL method measurements from O1 were lower than those from O2 to O3, which possibly indicates that US experience is important when making COL measurements.
The results of the pilot study also showed a small variation in the intra-observer comparison and a high agreement between the COL method and the Cobb method. However, we found that cadaveric laminae were much easier to identify on the US images than the laminae from AIS subjects. The thickness of the soft tissue may affect the image quality. Signal processing may be required to improve the signal-to-noise ratio on the US images so that subjects’ laminae can be easy recognized.
Although the results of our study suggest that the COL method is as reliable as the Cobb method, there are some limitations to our study. The motion effect of the phantom could not be avoided during the in vitro experiments; consequently, moving the phantom in and out of the water tank may have affected the curve configuration. Also, only five subjects and one observer were included in the pilot study. We therefore cannot make a conclusive statement from this small sample size. More studies and clinical trials should be considered to further validate the proposed US method before it can be used as the primary method to diagnose scoliosis. However, if the results of such studies are successful, this method can truly reduce radiation exposure to children who have scoliosis.
Conclusions
Our phantom study shows that the COL method is as reliable as the traditional Cobb method in terms of measuring the severity of scoliosis. The selection of the end-vertebrae of the curve from US images is similar to that from radiographs. The measurement error of the COL method is within the acceptable accuracy for a scoliosis clinic. Providing US training to the operator may help reduce variations in COL measurements. To further validate the proposed COL method, a large clinical trial is needed.
Footnotes
Acknowledgments
This work was supported by the Women and Children's Health Research Institute, the China Scholarship Council, and the Natural Sciences and Engineering Research Council of Canada.
None.