Abstract
Objective
This pilot study explored the predictive capabilities of individual components and composite scores of the Asher-McDade Esthetic Index (AMEI) for determining the need for nasolabial revision surgery in patients with repaired complete unilateral cleft lip and palate (UCLP). Receiver operating characteristic (ROC) analysis and Youden's index were employed to assess predictive power and identify optimal thresholds, respectively.
Design
Single-center retrospective cross-sectional pilot study
Setting
Cleft and craniofacial center
Participants
Thirty-two pre-adolescent patients with complete UCLP who underwent primary repair.
Interventions
Standardized frontal and lateral facial photographs were rated by three calibrated plastic surgeons using the AMEI. Components assessed included nasal form, nasal symmetry, vermilion border, and nasolabial profile. A separate panel of three plastic surgeons determined the need for nose and/or lip surgical revision by majority vote, based on the same photographs. ROC analysis evaluated the predictive ability of individual AMEI components and composite scores, with cutoffs identified via Youden's Index.
Main Outcome Measures
The predictive ability of AMEI for determining the need for nasolabial revision, using ROC analysis and Youden's Index.
Results
Nasal symmetry demonstrated the highest predictive accuracy for nose revision, and vermilion border was the strongest predictor for lip revision. Although composite scores showed promising predictive potential, they did not surpass the predictive power of the strongest individual AMEI components.
Conclusion
This pilot study provides preliminary evidence that the AMEI can serve as a useful objective tool to predict the need for revision surgery in patients with repaired UCLP.
Introduction
Patients with repaired cleft lip and palate frequently present with compromised nasolabial esthetics and altered craniofacial morphology, resulting from both the congenital defect and deformities secondary to primary surgical repairs.1–3 The management of cleft lip and palate is an inherently complex process and requires a coordinated multidisciplinary approach. Treatment protocols vary substantially across centers, influenced by clinician expertise, surgical techniques, timing, and institutional preferences. 4
Despite recent advances in primary cleft management, optimal functional and esthetic outcomes remain challenging for clinicians. 2 Patients with unilateral cleft lip and palate (UCLP) often present with secondary deformities, including nasal asymmetry, upper lip scarring, midface hypoplasia, and concave facial profile.5–7 These deformities often cause esthetic dissatisfaction among patients and their parents and lead them to pursue surgical revision procedures, with lip and nose revisions being the most frequently performed. 8 Secondary lip and nose revisions significantly improve not just esthetics but also psychosocial outcomes, including confidence, self-esteem, and social interactions. 9 However, the decision to proceed with revision surgery often lacks objective criteria and relies heavily on clinicians’ subjective judgment.10,11 Perceptions of the revision varied considerably among clinicians, influenced by their training, experience, and individual surgeon thresholds. 12 This resulted in a significant variation in revision rates across cleft centers globally.13,14 The variation was also attributed to differences in treatment protocols and initial repair quality. 15 In addition, revision surgery remains a common burden for many patients and could potentially be influenced by health literacy, socioeconomic status, and access to healthcare services. 16
To better delineate the influence of surgical techniques and timing on treatment outcomes, several intercenter studies, such as the Eurocleft, Americleft, and UK Clinical Standards Advisory Group studies, investigated the impacts of different primary surgical protocols on nasolabial esthetic outcomes.10,17,18 The Asher-McDade Esthetic Index (AMEI) is among the most widely used tools for standardized assessment of nasolabial appearance in patients with repaired UCLP. This index evaluates nasal form, nasal symmetry, nasolabial profile, and vermilion border on a 5-point ordinal scale. 19 The higher scores indicate poorer appearance and generally reflect greater esthetic dissatisfaction as well as increased demand for revision surgery.10,15 In addition, the AMEI demonstrated good reliability, particularly when assessments are conducted by three or more raters.20–22
In publicly funded healthcare systems, such as the National Health Service (UK) and the National Health Security Office (Thailand), financial coverage for secondary revisions is typically approved only when justified by significant functional or esthetic impairment deemed medically necessary. 23 To reduce subjectivity for surgical revision needs and promote equitable access to care, the use of objective assessment tools is required. The UK Index of Orthodontic Treatment Need (IOTN) exemplified how standardized criteria can guide treatment eligibility and help prioritize individuals with greater orthodontic treatment need based on their malocclusion severity and esthetic ratings. 24 Similarly, the AMEI, a commonly adopted esthetic index with a reproducible method to objectively assess nasolabial appearance for patients with repaired UCLP, could provide an objective criterion for determining the need for nasolabial surgical revision for patients with oral clefts. However, the utility of the AMEI in predicting the need for revision surgery has not yet been investigated. Therefore, this pilot study aimed to evaluate the predictive capabilities of the AMEI as an objective tool for determining the need for nose and/or lip revision surgery in patients with repaired complete UCLP using receiver operating characteristic (ROC) analysis and Youden's index.
Methods
Subjects
Patients with complete UCLP born between February 1997 and January 2002 at a cleft and craniofacial center were reviewed. Inclusion criteria were patients in their mixed dentition stage with facial photographs and dental models taken at birth before the initiation of nasoalveolar molding (NAM), as well as frontal and lateral profile photographs taken as pre-orthodontic records. Patients with Simonart's band, other craniofacial anomalies and associated syndromes, a history of revision surgery, alveolar bone graft surgery, orthodontics, and treatments other than NAM and primary cleft surgical repairs were excluded. All patients received their primary management protocol, which included presurgical NAM, followed by cheiloplasty using a modified Millard technique and primary rhinoplasty at 3 to 5 months of age. Palatoplasty was subsequently performed at 12 or 18 months using either the Bardach or von Langenbeck technique.
Nasolabial Esthetics Rating
Frontal and lateral profile photographs were obtained and cropped to eliminate surrounding facial features, leaving only the nasolabial area visible. This included the inner canthus, nasal bridge, nostrils, philtrum, and upper lip. All photographs were standardized with a uniform background color and imported into a PowerPoint presentation (Microsoft Corporation. Microsoft PowerPoint. Version 16. Redmond, WA: Microsoft Corporation; 2024). Each presentation slide simultaneously displayed both the frontal and lateral views of an individual patient (Figure 1). Slides were randomized and anonymized. Assessments were conducted by three experienced craniofacial plastic surgeons specializing in cleft care (one female and two males). None of the raters was involved in the patients’ primary surgeries. The AMEI was used to rate the nasolabial esthetic appearance. All raters were familiarized and calibrated before the actual rating sessions. 19 Nasal form, nasal symmetry, and vermilion border were evaluated using frontal photographs, while the nasolabial profile was assessed using lateral profile photographs. Each nasolabial component was scored on a 5-point ordinal scale, where 1 indicates in very good appearance and 5 represents a very poor appearance. There was no time limit imposed during the rating sessions. Raters completed their assessments independently, without consultation or discussion. All patients were rated twice, with a 2-week interval between sessions. The slide order was re-randomized to minimize rater memory bias. Intrarater and interrater reliability of these ordinal data was evaluated using the intraclass correlation coefficient (ICC). The interpretation of the ICC values was based on Koo and Li. 25
An example of a presentation slide used for the Asher-McDade Esthetic Index assessments and decisions to perform the nasolabial revision surgery in this study.
Scores from both sessions were tabulated as individual AMEI component scores and composite scores. Due to non-normal distribution, the median scores were used for all further analyses. 26 Composite scores were computed in both unweighted and weighted methods. The unweighted score was calculated from the median of the four components. The weighted score was performed according to Kantar et al. 27 They demonstrated that the components contributed unequally to overall nasolabial esthetics and proposed the following weights for each component score: nasal form (1.6), vermilion border (0.4), and nasal symmetry and nasolabial profile (1).
Nose and Lip Revision Surgery Decisions
A separate panel of three experienced craniofacial plastic surgeons was recruited. The panel also consisted of one female and two males, none of whom had been involved in the primary surgeries of the included patients. The same set of cropped frontal and lateral profile photographs was reordered and displayed to evaluate the decision to perform revision surgery on the lip and/or nose (Figure 1). The decision in favor of revision surgery was counted when at least two of the three raters concurred that such intervention was necessary. Interrater reliability for these categorical data was evaluated using Cohen's Kappa statistic. The interpretation of the Kappa coefficients followed the guidelines proposed by Landis and Koch. 28
Statistical Methods
ROC analysis was performed to evaluate the predictive capabilities of individual AMEI components and composite scores in determining the need for nasolabial revision surgery. Specifically, nasal form, nasal symmetry, and nasolabial profile were assessed for their ability to predict the need for nose revision, while the nasolabial profile and vermilion border were evaluated for their predictive ability in determining the need for lip revision. Both unweighted and weighted composite scores were also analyzed for their predictive performance in nose and lip revision (See Figure 2). ROC curves were generated by plotting the true positive rate (sensitivity) on the y-axis and (1 – specificity) on the x-axis at various scores of each AMEI component and composite scores. 29 Greater curvature toward the upper-left corner of the plot reflects stronger discriminative performance, whereas curves approaching the diagonal line indicate poorer predictive ability. The area under the ROC curve (AUC) represents the degree of predictive power with values ranging from 0 to 1. The AUC can be interpreted as follows: 0.60-0.70 indicated poor discrimination, 0.70-0.80 fair, 0.80-0.90 good, and 0.90-1.00 excellent. 30 In this study, the AUC and corresponding 95% confidence intervals were used to evaluate the ability of the AMEI to distinguish between patients who required revision surgery and those who did not. In addition, the optimal cutoff score for each predictor was determined using the Youden Index (J), which identified the cutoff value that maximized sensitivity and specificity. 31 The statistically significant level was set at P < .05.
Individual component and composite scores of the Asher-McDade Esthetic Index were assessed for their predictive capabilities regarding the need for the nasolabial revision surgery.
All statistical analyses were performed using the Statistical Package for the Social Sciences (SPSS) (IBM Corporation. IBM SPSS Statistics for Macintosh. Version 29. Armonk, NY: IBM Corporation; 2023).
Results
Participants
Frontal and lateral profile facial photographs of 32 patients with repaired complete UCLP, taken as part of pre-orthodontic treatment records, were collected for the analysis. The mean age at the time photographs were taken was 8.08 years (range, 6.67-10.34 years). The subjects comprised 23 males and 9 females, with 34.4% identifying as Caucasian and 65.6% as Hispanic. Regarding cleft laterality, 23 patients presented with the left side, and the remaining with the right.
Reliability
For AMEI ratings of nasolabial esthetics, the vermilion border showed the highest consistency, followed by nasal form, nasal symmetry, and nasolabial profile. Intrarater and interrater reliability were good, with mean ICCs of 0.788 and 0.871, respectively. 25 Reliability for nose revision showed fair to moderate agreement, 28 with mean kappa values of 0.307 for intrarater and 0.455 for interrater assessments. For lip revisions, intrarater and interrater reliability were slight to fair, 28 with mean kappa values of 0.255 and 0.328, respectively. Detailed results are provided in Supplementary Table.
Predictive Capabilities of the Asher-McDade Esthetic Index for Nasolabial Revision Surgery
For the prediction of nose revision surgery, nasal symmetry demonstrated the highest predictive capability (AUC = 0.837, P < .001), followed by the unweighted composite score (AUC = 0.831) and nasal form (AUC = 0.825). At a cutoff score of 2.25, nasal symmetry achieved 85.7% sensitivity and 77.8% specificity. The unweighted composite score yielded the same sensitivity but slightly lower specificity (72.2%) at a cutoff score of 2.125. Nasal form offered the highest sensitivity (92.9%) at a 2.75 cutoff score with only 72.2% specificity. The weighted composite score, despite a lower AUC (0.796), achieved the highest specificity (88.9%) at a cutoff of 2.625, but the sensitivity dropped to 64.3%. Nasolabial profile had the weakest predictive ability (AUC = 0.692, P = .043), with 71.4% sensitivity and 61.1% specificity.
For the prediction of lip revision surgery, the vermilion border demonstrated the highest predictive ability (AUC = 0.856, P = .001), with 100.0% specificity and 60.0% sensitivity at a cutoff score of 3.5. Nasolabial profile also exhibited a good predictive value (AUC = 0.774, P = .001), achieving 100.0% sensitivity but lower specificity (55.6%) at a cutoff score of 2.25. Both unweighted and weighted composite scores showed comparable predictive performance, with AUCs of 0.833 (P = .007) and 0.841 (P < .001), respectively. At an identical cutoff score of 2.875, both scores yielded 80.0% sensitivity and 85.2%. Results are summarized in Figure 3 and Table 1.
ROC curves for the predictive performance of the Asher-McDade Esthetic Index for determining decisions for nasolabial revision: (A) nasal form, nasal symmetry, nasolabial profile, unweighted composite, and weighted composite scores predicting nose revision; (B) nasolabial profile, vermilion border, unweighted composite, and weighted composite scores predicting lip revision. ROC, receiver operating characteristic.
ROC Analysis of Asher-McDade Esthetic Index for Predicting Nasolabial Revision Surgery.
Abbreviations: AUC, area under the ROC curve; CI, confidence interval; ROC, receiver operating characteristic.
Discussion
Nasolabial esthetics is one of the principal outcomes in the management of cleft lip and palate. 32 However, perceptions of appearance vary not only among individual patients but also among clinicians. 2 While the need for nose and/or lip revision surgery may be apparent in certain patients, the decision to perform such surgeries often lacked consensus within the cleft team.10,33 Our study indicated low agreement among surgeons for decisions to perform both nose and lip surgical revisions, 28 similar to previous findings by Trotman and coworkers. 33 They reported poor agreement among eight plastic surgeons when subjectively evaluating the need for lip revision. The low interobserver agreement is expected as revision decisions are often influenced by surgeons’ training and experience. 12 These results reflected the subjective nature of clinical judgment and reinforced the need to reduce subjectivity in the decision-making process when managing patients with repaired UCLP. To date, no standardized objective tools have been implemented to support revision surgery decisions. The AMEI has demonstrated good reliability in evaluating nasolabial esthetics following primary repair, 22 and our results also confirmed good intrarater and interrater agreement, consistent with previous studies.20,21
For the decisions regarding nose revision surgery, our findings revealed that nasal symmetry demonstrated the highest predictive capability, with an AUC of 0.837. Youden's index identified an optimal cutoff score of 2.250 for nasal symmetry, achieving a sensitivity of 85.7%. High sensitivity is crucial to ensure that patients who would benefit from revision surgery are not overlooked, particularly in cleft care, where undertreatment of nasal deformities can result in persistent functional and psychosocial burdens. 34 Marcusson and associates 8 reported that 58% of treated adults remained dissatisfied with their surgical outcomes, with 38% specifically seeking rhinoplasty, emphasizing the long-term importance of accurate revision assessment for nasal deformities. This study also identified nasal form as another significant AMEI component predictor for nasal revision (P < .001). This suggests that greater attention should be given to the evaluation of nasal symmetry and form in routine clinical assessments. Nasal symmetry evaluates the degree of deviation of the nose from an imaginary vertical midline drawn between the inner canthi, assessing the balance and alignment of the nasal tip and alar bases. And nasal form appraises the overall shape and contour, including deformities such as broadness or flattening. Our results demonstrated that nasal symmetry had superior predictive performance and higher sensitivity in identifying the need for nose revision. In contrast, Kantar et al 27 placed greater emphasis on nasal form over symmetry in their overall assessment of nasolabial appearance.
In regard to predicting the need for lip revision surgery, the vermilion border emerged as the strongest individual predictor (AUC = 0.856, P = .001), indicating a good level of predictive capability. 30 At the optimal cutoff score of 3.5, it achieved 100.0% specificity, reliably identifying patients who do not require revision. This finding is particularly valuable in clinical practice, where false positives could result in unnecessary surgical revision procedures. However, the sensitivity was relatively modest at 60.0%, suggesting a considerable proportion of patients who may benefit from revision could be underidentified. The nasolabial profile showed fair predictive capability with an AUC of 0.774 (P = .001). At a cutoff score of 2.250, it achieved 100.0% sensitivity, effectively identifying all patients who do need surgery. Nonetheless, its specificity was only 55.6%, reflecting lower precision in ruling out those who do not require revision. Thus, while the vermilion border is more effective at accurately ruling out patients who do not require surgery, the nasolabial profile is more sensitive in identifying all patients who do require revision. This concluded the importance of incorporating both frontal (vermilion border) and lateral (nasolabial profile) views to guide clinical decision-making in lip revision surgery.
In this study, composite scores exhibited lower predictive performance compared with certain individual components. For nose revision, the composite scores achieved an AUC of 0.831 and 0.796 for the unweighted and weighted scores, respectively. The scores for the lip revision (0.833 for the unweighted and 0.841 for the weighted scores) performed comparably. Although these results indicate reasonable predictive power, the composite scores did not outperform the strongest individual components, which target anatomically specific deformities. This likely explains why individual component ratings are more clinically relevant for guiding revision decisions, whereas composite scores may be better suited for providing an overall assessment of nasolabial appearance outcomes.
This study relied solely on professional assessments to determine the need for revision surgery. However, previous research by Mani and colleagues 2 demonstrated that laypersons tend to assign poorer esthetic ratings than professionals, indicating a potential discrepancy in perceived outcomes. Such differences in nasolabial esthetics evaluations between professionals and laypeople could directly impact revision decisions. Furthermore, Mulder et al 35 emphasized the importance of aligning surgical plans with patient expectations, perceptions, and satisfaction. Incorporating patient-reported outcomes such as satisfaction, function, and quality of life could elevate the AMEI from an objective outcome evaluation to a more holistic guide for nasolabial revision decisions. Integrating clinician assessments with patient perspectives will promote a more patient-centered approach to cleft care.
Our findings provide preliminary evidence supporting the predictive value of the AMEI in guiding revision surgery decisions. While the AMEI has traditionally been used to assess nasolabial esthetics, our results indicate that it may also serve as a useful tool for identifying patients who are more likely to be considered for revision surgery. This extends the clinical relevance of the AMEI beyond esthetic assessment, supporting its potential role in anticipating revision needs and potentially contributing to more equitable care for patients with repaired UCLP.
This study is not without limitations. Although the size of this single-center pilot study was relatively small, it provides preliminary data to perform power analyses for future full-scale investigations on the potential use of AMEI in predicting nasolabial revision surgery. There was also an imbalance of samples in both gender and cleft laterality. These disparities reflect the prevalence of cleft lip and palate, affecting more males and the left side at approximately 2:1 ratio. 36 Although patients across all severity levels (AMEI scores 1-5) were included, the distribution of scores concentrated around the scores of 2 and 3. Future studies should incorporate a larger sample size with balanced distributions of gender, cleft laterality, and severity to strengthen study design. Moreover, only right-sided lateral photographs were assessed, which may underrepresent left-sided cleft deformities. Therefore, the low predictive capability for both nose and lip revision surgery could potentially be attributed to this underrepresentation. Future studies should include lateral views from the cleft-affected side to improve accuracy and consistency. While three-dimensional imaging demonstrates higher efficacy in reliably assessing nasolabial esthetics, 37 its effectiveness is hindered by cost, training requirements, and restricted accessibility.38,39 As such, two-dimensional photographs remain the most cost-effective and practical option for multicenter comparisons at present.
Conclusion
The AMEI has proven to be a useful tool capable of predicting the need for nose and/or lip revision surgery. Among its components, nasal symmetry and vermilion border exhibited the highest predictive capability for nose and lip revisions, respectively.
Supplemental Material
sj-docx-1-cpc-10.1177_10556656251415230 - Supplemental material for Predictive Capabilities of the Asher-McDade Esthetic Index for Nasolabial Revision Surgery: A Pilot Study
Supplemental material, sj-docx-1-cpc-10.1177_10556656251415230 for Predictive Capabilities of the Asher-McDade Esthetic Index for Nasolabial Revision Surgery: A Pilot Study by Fonthip Tatiyanupanwong, Supatchai Boonpratham, Natchalee Srimaneekarn, Chaiyapol Chaweewannakorn, Yodhathai Satravaha and Supakit Peanchitlertkajorn in The Cleft Palate Craniofacial Journal
Footnotes
Acknowledgements
This research project is supported by the Department of Orthodontics, Faculty of Dentistry, Mahidol University.
ORCID iDs
Ethical Considerations
This study was approved by the Institutional Review Board of the Faculty of Dentistry/Faculty of Pharmacy, Mahidol University (COA No. MU-DT/PY-IRB 2024/058.1909), Bangkok, Thailand, which complies with international guidelines for human research protection, including the Declaration of Helsinki.
Consent to Participate
Written informed consents were obtained from the legal guardian for research purposes.
Consent for Publication
Written informed consent for publication was obtained from the legal guardian for anonymized patient information to be published in this article.
Author Contributions
FT contributed to investigation, data collection, data analysis, visualization, and writing—original draft. SB contributed to formal analysis, resources, and supervision. NS contributed to data analysis and interpretation and validation. CC contributed to data analysis. YS contributed to project administration. SP contributed to conceptualization, data curation, investigation, methodology, validation, supervision, and writing—review and editing.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Data Availability
The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
