Abstract
Objective
To test the accuracy of an artificial intelligence (AI) algorithm in predicting the favorability of maxillary canine impaction as compared to the conventional manual tracing method using orthopantomograms.
Materials and Methods
Orthopantomograms of 437 canine impactions were included in this study. Six parameters (sector classification, three angular, and two linear) on orthopantomograms were used to assess the severity of the maxillary canine impaction. The most advanced convolutional neural network model implemented using MATLAB program was used in assessing the favorability of maxillary canine impaction. The outcome of the parameters assessed by the implemented AI model and conventional manual tracing were compared. Receiver operating characteristic (ROC) curves, diagnostic accuracy, sensitivity, specificity, and positive and negative predictive values were used to assess the performance of AI model in comparison to manual tracing for the angular and sector parameters. Paired t-tests were used for linear measurements.
Results
The overall clinical performance exceeded 90% for all the angular parameters including sector classification except for the angle between long axis of canine with lateral incisor which had a specificity score of 55%. The value and the area under the ROC curve were more than 0.9 for all the parameters. The distance from the canine cusp tip to the occlusal plane and midline was statistically significant between the groups (p = .000).
Conclusion
The proposed AI algorithm had higher accuracy in predicting the favorability of eruption in maxillary canine impactions compared to conventional manual tracing.
Introduction
Eruption of canine normally starts at a higher level in the maxilla, thereby having the longest path of eruption. After mandibular or maxillary third molar, maxillary canines are considered the second most commonly impacted teeth with incidence rate of 0.9–2.2%.1, 2 There are various local, systemic, and genetic factors that attribute to the etiology of maxillary canine. 3 The primary goal in diagnosis and treatment planning of an impacted maxillary canine is to identify whether the canine is impacted and to predict its subsequent eruption and outcome of orthodontic intervention. 4
Several radiographic aids have been advocated for the diagnosis of maxillary canine impaction. 5 Ericson and Kurol found that early diagnosis could be made with a series of measurements on panoramic film such as the angle between long axis of canine and midline, linear distances from the canine cusp tip to the occlusal plane, and the location of the canine in different sectors with relation to adjacent teeth. 6 Previous literature reported that locating the impacted canine based on sector classification is more accurate and reliable than tooth angulation for predicting impaction. 7
Computer-aided diagnosis (CAD) has evolved drastically due to the increased availability of digital data, computational ability, and advances in artificial intelligence (AI) technology. AI can be classified as symbolic AI and machine learning. Symbolic AI is a collection of techniques that works on a set of rules, whereas in machine learning, the model learns either by supervised or unsupervised learning from examples and adapts based on the available data. The input data and the correct output data (referred to as labels) should be available before starting the training phase of supervised learning algorithms. The input data are given to the model which then estimates the output values. The output values are then compared with the labels, and the difference between them is referred to as errors. These errors can be minimized by repetitive training and validating the model unless the errors are minimal.
In dentistry, radiographic records and their interpretation have a very important role in diagnosing, treatment planning, and monitoring dental health periodically. 8 Processing of radiographic images has become an area of interest for automation. This automated solution will improve the accuracy in clinical decision-making in routine practice and thereby reduce the stress and fatigue of dentists. 8 In orthodontics, the usefulness of AI has been studied for cephalometric landmark detection, teeth segmentation, and teeth classification.9 –11 The purpose behind the use of AI is to assist in making more accurate and unbiased decisions, thus increasing the efficiency of systems.12, 13
According to the available literature evidence, there has been no study that evaluated the diagnostic accuracy of an AI model for predicting the favorability of maxillary canine impaction using orthopantomograms (OPGs). Thus, the aim of this study was to test the accuracy of AI algorithm in predicting the favorability of maxillary canine impaction as compared to the conventional manual tracing method using OPGs.
Materials and Methods
Ethical Approval and Study Design
OPGs of patients with impacted maxillary canine were collected retrospectively from the archives of the department. The sample for the study was selected based on the following criteria.
Inclusion Criteria
Diagnostic OPGs of patients above 13 years of age with unilateral/bilateral impacted maxillary canine.
Patients with buccal/palatal canine impaction.
Patients with retained/exfoliated deciduous maxillary canine with unerupted permanent maxillary canine.
OPGs with good image characteristics.
No gender restrictions.
Exclusion Criteria
Patients who have previously undergone orthodontic treatment.
Patients with craniofacial anomalies associated with/without eruption abnormalities.
Patients with a history of traumatic injury and/or surgery in the head and neck region.
Patients with musculoskeletal or bone disorders.
Measurements
The parameters on OPGs that were used to assess the severity of the canine impaction were selected based on the conclusions of the systematic review by Ravi et al. 14 and they are as follows.
Sector Classification 6
The position of the impacted maxillary canine was designated to one of the five sectors categorized based on the position of the canine in relation to the midline of the maxillary dentition, long axes of central and lateral incisors, and lines passing through mesial surface of the first premolar. The impaction was decided as unfavorable for spontaneous eruption if the cusp tip of the maxillary canine crossed sector 2.
Angle between the Long Axis of the Maxillary Canine to the Midline15, 16
The midline was determined as line passing from the anterior nasal spine through the interincisal contact of the permanent maxillary central incisors. Angulation exceeding 31° was indicative of unfavorable canine impaction.
Angle between the Long Axis of the Maxillary Canine to the Lateral Incisor 17
Angulation of the long axis of the maxillary canine exceeding 51.47° relative to the lateral incisor was indicative of unfavorable canine impaction.
Angle between the Long Axis of the Maxillary Canine to the Occlusal Plane 18
The occlusal plane was constructed as a horizontal line passing through the incisal edge of the permanent maxillary central incisor and the mesiobuccal cusp tip of the first permanent maxillary molar on the given side. The impaction was considered to be unfavorable if the angulation was above 132°.
Perpendicular Distance between the Maxillary Canine Tip to the Occlusal Plane and Midline 19
It was measured by projecting a perpendicular line from the occlusal plane and midline to the cuspal tip of the impacted maxillary canine, respectively.
Development of AI Model
The model developing process was conducted on computer equipment at the Department of Computer Science Engineering. The most advanced convolutional neural network (CNN) model was used in assessing the favorability of maxillary canine impaction. Three layers were used to make up the CNN architecture: a fully connected classification layer and two convolutional pooling layers that follow one another. The two convolutional pooling layers each have 128 and 64 neurons and also utilize the same fixed 8 × 8 convolutional kernel and 2 × 2 pooling kernel. The remaining neurons in the last layer were connected to the fully connected classification layer that determines whether the input image has been classified or not. The proposed model is implemented in MATLAB program (R2022b v 9.13.0. The MathWorks, Inc., USA). In the initial phase, the size of all the photos was reduced to 80 × 215 pixels in order to optimize the data for model training. Then, using the associated landmarks from each training image, the training process was implemented on the whole target dataset. In a nutshell, the deep CNN model was used to build the proposed model in stages. The batch size was 329, the learning rate was 0.01, and there were 150 epochs in the image during the training phase. The trained model recognized the favorability in the test OPGs automatically.
Input Dataset
The input dataset for the AI model included OPG images of 437 maxillary canine impactions along with normative values for the sector classification and angular parameters. The images were distributed into:
Training group: 329 images
Validation group: 11 images
Testing group: 97 images
The distribution of samples is depicted in Table 1. The training group was used to train input image for automatic prediction for the favorability of the canine impaction. The validation and testing group was used for the evaluation of the performance of the software.
Distribution of Samples for Manual Tracing and Artificial Intelligence (AI) Model.
The overall model building process involving the training, validation, and testing phases is depicted in Figure 1.
Flowchart Depicting Model Building Process.
Training Phase
OPG images along with the normative values for the sector classification and angular parameters were given as an input for the AI model. The coordinates in the training process were as follows:
Point for Cropping Orthopantomogram.
Cropped Image of Orthopantomogram.
Points for Generation of Midline.
Generation of Midline.
Generation of the Sector Lines.
Generation of the Canine Outline.
Generation of Occlusal Plane.
Final Artificial Intelligence (AI) Prediction of Impacted Maxillary Canine.
The system generated the output values as the angles made by the long axis of the impacted maxillary canine to the midline, occlusal plane, lateral incisor, and the linear distances from the canine cusp tip to occlusal plane and midline. The final layer showed the measurements of all the six parameters which were stored as a single image in PNG format (Figure 8). Based on the measurements, AI model automatically predicted the favorability of maxillary canine impaction, and the final measurements along with the prediction were reported in an Excel format.
Validation Phase
The AI model was validated using 11 randomly selected images. The errors that occurred during the training phase were immediately rectified till the point of minimum error, after which the iterative learning with the training data was stopped.
Testing Phase
A fresh set of 97 impacted maxillary canine images were used for testing. These images were traced manually which forms Group 1 (Manual Tracing) and tested using the developed AI model which forms Group 2 (AI Model). All the abovementioned parameters were evaluated and compared between the two groups.
Manual Tracing of OPGs
The radiographic images of 97 impacted teeth were annotated using acetate tracing paper of 0.003” thickness and a sharp 3H drawing pencil. The outlines along with the long axes of the impacted canine and incisors of that side, midline, and occlusal plane were drawn on the OPGs as mentioned previously. This was followed by the measurement of selected angular and linear parameters using a ruler and protractor.
The angular and linear parameters and sector classification selected for assessment of maxillary canine impaction using OPGs were calculated for all the samples and compared with that of the data obtained from AI algorithm and then subjected to statistical analysis.
The reproducibility of measurements made with the AI model and manual tracing was evaluated by repeating the measurements on 20 randomly selected samples by the same investigator at an interval of 30 days.
Statistical Analysis
The following statistical analysis was carried out using Statistical Package for the Social Sciences (SPSS), version 16.0 (SPSS Inc., Chicago, IL). The intraexaminer reliability of linear and angular measurements was evaluated with the intraclass correlation coefficient (ICC). Mean and standard deviation were calculated for both manual and AI diagnostic results. To assess the performance of AI for the angular and sector parameters, diagnostic accuracy, sensitivity, specificity, and positive and negative predictive values were calculated by using the following formula through confusion matrices data (Table 2).
Description of Confusion Matrix.
where, TP: true positive; TN: true negative; FP: false Positive; FN: False Negative.
A comparison of mean linear measurements between both groups was performed using paired t-test. The overall diagnostic performance of the AI algorithm was performed using receiver operating characteristic (ROC) curve.
The ROC curves and area under the curve (AUC) with 95% confidence intervals were performed to assess the overall diagnostic performance of the AI algorithm.
Results
The ICC values were above 0.9, indicating excellent intraexaminer reliability of the measurements (Table 3).
Assessment of Intraexaminer Reliability of Measurements.
ICC values: >0.75 = excellent agreement, 0.4–0.75 = fair agreement, <0.4 = poor agreement.
Descriptive statistics with the mean and standard deviation of all the linear and angular parameters are shown in Table 4.
Descriptive Statistics with Mean and Standard Deviation (SD) of all Linear and Angular Parameters.
Angular Parameters and Sector Classification
The sensitivity, specificity, accuracy, and positive and negative predictive values for angular parameters and sector classification are provided in Table 5.
Performance of Artificial Intelligence Algorithm versus Manual Tracing for Predicting Favorability of Maxillary Canine Impaction.
Sensitivity
The angle between long axis of canine with midline, occlusal plane, and lateral incisor had the highest sensitivity value of 100%. The sector classification had sensitivity value of 98.5% (93.9–100).
Specificity
The highest specificity values of 100% were found for angle between long axis of canine with occlusal plane and sector classification followed by angle between long axis of canine with midline with a specificity score of 97 (95.3–100). The least specificity value of 55% (0.6–100) was found for the angle between long axis of canine with lateral incisor.
Accuracy
The angle between long axis of canine with the occlusal plane had an accuracy of 100%. The angle between long axis of canine with midline and sector classification had a similar accuracy score of 98.96 (98–100). The angle between long axis of canine with lateral incisor had the least accuracy values of 90.72% (82–100).
Positive Predictive Value
The angle between long axis of canine with occlusal plane and sector classification had the highest positive predictive value of 100% followed by 98% for angle between long axis of canine with midline. The least positive predictive value of 89.5% was found for angle between long axis of canine with lateral incisor.
Negative Predictive Value
The angle between long axis of canine with midline, occlusal plane, and lateral incisor had the highest negative predictive value of 100% followed by sector classification with a negative predictive value of 96.7%.
ROC Curves
The values of the ROC curves with 95% confidence intervals were far exceeding 0.9, which indicated that the proposed AI system is able to accurately predict maxillary canine impaction (Table 6 and Figures 10–13).
Area Under the Curve (AUC) and Confidence Interval (CI) for Angular Measurements and Sector Classification for Artificial Intelligence (AI) versus Manual Tracing.
Receiver Operating Characteristic (ROC) Curve for Angle between Long Axis of Canine with Midline (ML).
Receiver Operating Characteristic (ROC) Curve for Angle between Long Axis of Canine with Lateral Incisor (LI).
Receiver Operating Characteristic (ROC) Curve for Angle between Long Axis of Canine with Occlusal Plane (OP).
Receiver Operating Characteristic (ROC) Curve for Sector Classification.
Linear Parameters
The t-test values showed a statistically significant difference for both distance from the canine cusp tip to occlusal plane and midline (p = .000) (Table 7).
Paired t-test for Comparison of Linear Parameters between Artificial Intelligence (AI) versus Manual Tracing.
SD, standard deviation; OP, occlusal plane; ML, midline.
Discussion
AI has been widely used in the medical field for image analysis. Commonly used models include artificial neural network (ANN),20, 21 CNN,22, 23 random forest, 24 support vector machine, etc.25, 26 The CNNs are the most commonly used AI model. This study used the CNN architecture to test the accuracy of AI model developed using OPGs to predict the favorability of maxillary canine impaction as compared to manual prediction.
Several applications of AI for diagnosing and treatment planning in orthodontics have been published earlier such as cephalometric landmark identification, decisions for orthodontic extractions, anchorage patterns, growth maturation status, and orthognathic surgeries.9, 21, 24, 27 Most of the studies use radiographic diagnostic tools such as periapical radiographs, occlusal radiographs, lateral cephalograms, OPGs, and cone-beam computed tomography (CBCT).
Accuracy and reliability of AI have been investigated on OPGs for various purposes such as tooth detection which reported sensitivity of 99.41% and precision of 99.45%, tooth numbering with sensitivity and specificity of 98% and 99.94%, respectively, 8 caries and restoration detection with an accuracy of 93.6%, 28 identification and classification of dental implants system with an accuracy of 67%. 29 Margot et al. reported that logistic regression model is a useful tool to predict maxillary canine impaction on OPGs. 30
The results of the present study showed that the performance of the AI model when compared to manual tracing was excellent for all the parameters except for sector classification which had a sensitivity score of 98.5 (93.9–100) and negative predictive value of 96.7%. This is because one of the samples which was actually diagnosed as sector 2 (favorable) in manual tracing was diagnosed as sector 5 (not favorable) in the AI prediction.
With regard to the prediction power for non-favorable canine impactions, the angle between long axis of canine to the lateral incisor had a specificity score of 55 (0.6–100) and a positive predictive value of 89.5% which shows that the parameter is least effective in predicting the actual non-favorable canine impactions as non-favorable. This could be because the image of the lateral incisor was often unclear on OPGs due to superimposition of the image of surrounding structures. Despite the variation in the prediction power among the individual parameters, the overall performance of the proposed AI model was excellent, which was clearly evident in the ROC curves showing high AUC index of above 0.9 for all the angular parameters and sector classification.
The decision-making for the impacted maxillary canine using the linear parameters could not be performed because of lack of normative values in the literature. In addition, linear parameters were used to predict the favorability based on the change in the magnitude of the canine cusp tip in OPGs taken at sequential intervals. 31 Paired t-test was used to check the correlation of the linear parameters between AI model and manual tracing and it was found to be statistically significant indicating a weak correlation between the actual manual tracing and the AI prediction. This could be due to the magnification errors in OPGs that affect the linear measurements. 32
One major limitation of this study is that the AI model used OPG which is a 2D imaging modality. Future studies can be planned with three-dimensional imaging modalities such as CBCTs collected from various clinicians, which will aid in model training for improved performance. The application of AI algorithm was at the level of decision-making regarding the favorability of eruption in maxillary canine impaction but not at the level of detection of impaction. This was due to the difficulty in defining the region of interest and locating the impacted maxillary canine when it was not present in its usual erupted position. In addition, it was also difficult to define a complete outline including the root of the impacted maxillary canine because of similar gray scales on the OPG. 28 Newer AI models developed to identify impacted maxillary canines in OPGs blended with the current model to predict the favorability will provide a complete AI solution to diagnose and plan the treatment for maxillary canine impaction in the near future.
Conclusion
The proposed AI model had higher accuracy in predicting the favorability of eruption in maxillary canine impactions. The performance was excellent for sector classification, angle between long axis of canine with midline and occlusal plane, whereas it was slightly lower but acceptable for angle between long axis of the canine and lateral incisor. Due to lack of normative values, decision-making on favorability could not be performed with linear parameters.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Ethical Approval
The proposal of the study was approved by the Institutional Ethics Committee (CSP/21/DEC/103/609).
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research received IOS research grant from the Indian Orthodontic Society Research Foundation (IOSRF) for the academic year 2022.
Informed Consent
Not applicable.
