To develop and validate a Belgian Dutch outcome tool for the perceptual evaluation of speech in patients with cleft palate.
Setting:
Cleft palate team in a tertiary university hospital.
Methods:
The tool was based on the Cleft Audit Protocol for Speech–Augmented (John et al., 2006; Sell et al., 2009), with adaptations to some of the speech variables and the structured listening protocol. Following a preliminary listening experiment in phase 1, the tool was optimized. In the second phase, a listening experiment with 4 experienced listeners was set up to assess face validity, inter- and intrarater reliability and criterion validity.
Results:
Results of phase 1 indicated good to very good inter- and intrarater reliability for the majority of the speech variables, good discriminant validity, and varying sensitivity and specificity based on a comparison with nasalance values and the Nasality Severity Index 2.0 (criterion validity). Results of phase 2 showed good to very good interrater reliability for 5 of the 14 variables and good intrarater reliability in 3 of the 4 experienced listeners. Sensitivity and specificity were sufficient, except the specificity of the hypernasality judgments in comparison with the nasalance values of the oral text. Overall, listeners positively judged the face validity of the tool.
Conclusion:
The 2-phase evaluation indicated varying validity and reliability results. Future studies will aim to optimize validity and reliability of the developed tool based on adaptations to the listening protocol, the addition of speech variables, and the inclusion of a more elaborate training.
AlloriACKelleyTMearaJGAlbertABonanthayaKChapmanKCunninghamMDaskalogiannakisJde GierHHeggieAA, et al.A standard set of outcome measures for the comprehensive appraisal of cleft care. Cleft Palate Craniofac J. 2017;54(5):540–554.
2.
AltmanDG. Practical Statistics for Medical Research. Boca Raton, FL: CRC Press; 1990.
3.
BaylisAChapmanKWhitehillTLGroupTA. Validity and reliability of visual analog scaling for assessment of hypernasality and audible nasal emission in children with repaired cleft palate. Cleft Palate Craniofac J. 2015;52(6):660–670.
4.
BettensKBruneelLMarynYDe BodtMLuytenAVan LierdeKM. Perceptual evaluation of hypernasality, audible nasal airflow and speech understandability using ordinal and visual analogue scaling and their relation with nasalance scores. J Commun Disord. 2018;76:11–20.
5.
BettensKVan LierdeKMCorthalsPLuytenAWuytsFL. The Nasality Severity Index 2.0: revision of an objective multiparametric approach to hypernasality. Cleft Palate Craniofac J. 2016;53(3):e60–e70.
6.
BettensKWuytsFLJonckheereLPlatbroodSVan LierdeK. Influence of gender and age on the Nasality Severity Index 2.0 in Dutch-speaking Flemish children and adults. Logoped Phoniatr Vocol. 2017;42(3):133–140.
7.
BeukelmanDRYorkstonKM. Influence of passage familiarity on intelligibility estimates of dysarthric speech. J Commun Disord. 1980;13(1):33–41.
BorgGBorgE.A new generation of scaling methods: Level-anchored ratio scaling. Psychologica. 2001;28(1):15–45.
10.
BrancampTULewisKEWattersonT. The relationship between nasalance scores and nasality ratings obtained with equal appearing interval and direct magnitude estimation scaling methods. Cleft Palate Craniofac J. 2010;47(6):631–637.
11.
BreulsMSellDMandersEBouletEVander PoortenV. SISL (ScreeningsInstrument Schisis Leuven): assessment of cleft palate speech, resonance and myofunction. B ENT. 2006;2(suppl 4):71–84.
12.
BrittonLAlberyLBowdenMHarding-BellAPhippenGSellD. A cross-sectional cohort study of speech in five-year-olds with cleft palate±lip to support development of national audit standards: benchmarking speech standards in the United Kingdom. Cleft Palate Craniofac J. 2014;51(4):431–451.
13.
BrondstedKGrunwellPHenningssonGJansoniusKKarlingJMeijerMOrdingUSellDVermeij-zieverinkEWyattR. A phonetic framework for the cross-linguistic analysis of cleft palate speech. Clin Linguist Phon. 1994;8(2):109–125.
14.
BruneelLAlighieriCDe SmetSBettensKDe BodtMVan LierdeK. Health-related quality of life in patients with cleft palate: reproducibility, responsiveness and construct validity of the Dutch version of the VELO questionnaire. Int J Pediatr Otorhinolaryngol. 2019a;119:141–146.
15.
BruneelLBettensKVan LierdeK. The relationship between health-related quality of life and speech in patients with cleft palate. Int J Pediatr Otorhinolaryngol. 2019b;120:112–117.
16.
BruneelLVan LierdeKBettensKCorthalsPVan PoelEDe GrooteEKepplerH. Health-related quality of life in patients with cleft palate: validity and reliability of the VPI Effects on Life Outcomes (VELO) questionnaire translated to Dutch. Int J Pediatr Otorhinolaryngol. 2017;98:91–96.
17.
BrunnegårdKLohmanderAvan DoornJ. Untrained listeners’ ratings of speech disorders in a group with cleft palate: a comparison with speech and language pathologists’ ratings. Int J Lang Commun Disord. 2009;44(5):656–674.
18.
BrunnegårdKLohmanderAvan DoornJ. Comparison between perceptual assessments of nasality and nasalance scores. Int J Lang Commun Disord. 2012;47(5):556–566.
19.
CastickSKnightRASellD. Perceptual judgments of resonance, nasal airflow, understandability, and acceptability in speakers with cleft palate: ordinal versus visual analogue scaling. Cleft Palate Craniofac J. 2017;54(1):19–31.
20.
ChapmanKLBaylisATrost-CardamoneJCorderoKNDixonADobbelsteynCThurmesAWilsonKHarding-BellASweeneyT, et al.The Americleft Speech Project: a training and reliability study. Cleft Palate Craniofac J. 2016;53(1):93–108.
21.
ChapmanKLHardin-JonesMAGoldsteinJAHalterKAHavlikRJSchulteJ. Timing of palatal surgery and speech outcome. Cleft Palate Craniofac J. 2008;45(3):297–308.
22.
EcksteinDAWuRLAkinbiyiTSilverLTaubPJ. Measuring quality of life in cleft lip and palate patients: currently available patient-reported outcomes measures. Plast Reconstr Surg. 2011;128(5):518e–526e.
23.
FleissJL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76(5):378–382.
24.
GoochJLHardin-JonesMChapmanKLTrost-CardamoneJESussmanJ. Reliability of listener transcriptions of compensatory articulations. Cleft Palate Craniofac J. 2001;38(1):59–67.
25.
HallgrenKA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23–34.
26.
HardingAGrunwellP. Active versus passive cleft-type speech characteristics. Int J Lang Commun Disord. 1998;33(3):329–352.
27.
HardingAHarlandKRazzellR. Cleft Audit Protocol for Speech (CAPS). Broomfield, CO: St Andrew’s Plastic Surgery Centre; 1997.
28.
HenningssonGHuttersB. Perceptual Assessment of Cleft Palate Speech, with Special Reference to Minimum Standards for Inter-Centre Comparisons of Speech Outcome. Paper presented at the Transactions 8th International Congress on Cleft Palate and Related Anomalies; 1997.
29.
HenningssonGKuehnDPSellDSweeneyTTrost-CardamoneJEWhitehillTL; Speech Parameters Group. Universal parameters for reporting speech outcomes in individuals with cleft palate. Cleft Palate Craniofac J. 2008;45(1):1–17.
30.
HuttersBHenningssonG. Speech outcome following treatment in cross-linguistic cleft palate studies: methodological implications. Cleft Palate Craniofac J. 2004;41(5):544–549.
31.
JohnASellDSweeneyTHarding-BellAWilliamsA. The cleft audit protocol for speech-augmented: a validated and reliable measure for auditing cleft speech. Cleft Palate Craniofac J. 2006;43(3):272–288.
32.
KentRD. Hearing and believing: some limits to the auditory-perceptual assessment of speech and voice disorders. Am J Speech Lang Pathol. 1996;5(3):7–23.
33.
KeuningKHWienekeGHDejonckerePH. Correlation between the perceptual rating of speech in Dutch patients with velopharyngeal insufficiency and composite measures derived from mean nasalance scores. Folia Phoniatr Logop. 2004;56(3):157–164.
34.
KlassenAFTsangarisEForrestCRWongKWPusicALCanoSJSyedIDuaMKainthSJohnsonJ, et al.Quality of life of children treated for cleft lip and/or palate: a systematic review. J Plast Reconstr Aesthet Surg. 2012;65(5):547–557.
35.
KlintöKSalamehEKSvenssonHLohmanderA. The impact of speech material on speech judgement in children with and without cleft palate. Int J Lang Commun Disord. 2011;46(3):348–360.
36.
KlintöKSvenssonHElanderALohmanderA. Speech and phonology in Swedish-speaking 3-year-olds with unilateral complete cleft lip and palate following different methods for primary palatal surgery. Cleft Palate Craniofac J. 2014;51(3):274–282.
37.
KuehnDPMollerKT. Speech and language issues in the cleft palate population: the state of the art. Cleft Palate Craniofac J. 2000;37(4):348–348.
38.
KuehnDPMoonJB. Velopharyngeal closure force and levator veli palatini activation levels in varying phonetic contexts. J Speech Lang Hear Res. 1998;41(1):51–62.
39.
LandisJRKochGG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174.
40.
LohmanderA. Universal parameters for reporting speech outcomes in individuals with cleft palate [Letter to the editor]. Cleft Palate Craniofac J. 2008;45(4):452–453.
41.
LohmanderAHagbergEPerssonCWilladsenELundeborgIDaviesJHavstamCBoersMKisling-MøllerMAlaluusuaS, et al.Validity of auditory perceptual assessment of velopharyngeal function and dysfunction—the VPC-Sum and the VPC-Rate. Clin Linguist Phon. 2017a;31(7-9):589–597.
42.
LohmanderAOlssonM. Methodology for perceptual assessment of speech in patients with cleft palate: a critical review of the literature. Cleft Palate Craniofac J. 2004;41(1):64–70.
43.
LohmanderAPerssonC. A longitudinal study of speech production in Swedish children with unilateral cleft lip and palate and two-stage palatal repair. Cleft Palate Craniofac J. 2008;45(1):32–41.
44.
LohmanderAPerssonCWilladsenELundeborgIAlaluusuaSAuknerRBauABoersMBowdenMDaviesJEmborgBHavstamC, et al.Scandcleft randomised trials of primary surgery for unilateral cleft lip and palate: 4. Speech outcomes in 5-year-olds-velopharyngeal competency and hypernasality. J Plast Surg Hand Surg. 2017b;51(1):27–37.
45.
LohmanderAWilladsenEPerssonCHenningssonGBowdenMHuttersB. Methodology for speech assessment in the Scandcleft project—an international randomized clinical trial on palatal surgery: experiences from a pilot study. Cleft Palate Craniofac J. 2009;46(4):347–362.
46.
MalmbornJOBeckerMKlintöK. Problems with reliability of speech variables for use in quality registries for cleft lip and palate—experiences from the Swedish Cleft Lip and Palate Registry. Cleft Palate Craniofac J. 2018;55(8):1051–1059. doi:10.1177/1055665618765777.
47.
McLeodSHarrisonLJMcCormackJ. The intelligibility in context scale: validity and reliability of a subjective rating measure. J Speech Lang Hear Res. 2012;55(2):648–656.
48.
MoonJBKuehnDPHuismanJJ. Measurement of velopharyngeal closure force during vowel production. Cleft Palate Craniofac J. 1994;31(5):356–363.
49.
MorrisHOzanneA. Phonetic, phonological, and language skills of children with a cleft palate. Cleft Palate Craniofac J. 2003;40(5):460–470.
50.
MosseyPALittleJMungerRGDixonMJShawWC. Cleft lip and palate. The Lancet. 2009;374(9703):1773–1785.
51.
NormanGStreinerD. Elements of statistical inference. In: Biostatistics: The Bare Essentials. NC, USA: PMPH USA; 2008;46–62.
52.
PereiraVJSellDTuomainenJ. Effect of maxillary osteotomy on speech in cleft lip and palate: perceptual outcomes of velopharyngeal function. Int J Lang Commun Disord. 2013;48(6):640–650.
53.
Peterson-Falzone. The Clinician’s Guide to Treating Cleft Palate Speech (Vol. 1). Maryland Heights, MO: Mosby; 2006.
54.
Peterson-FalzoneSJHardin-JonesMAKarnellMPMcWilliamsBJ. Cleft Palate Speech. St Louis, MO: Mosby; 2001.
55.
SamoyKHensGVerdonckASchoenaersJDormaarTBreulsMVander PoortenV. Surgery for velopharyngeal insufficiency: the outcomes of the University Hospitals Leuven. Int J Pediatr Otorhinolaryngol. 2015;79(12):2213–2220.
56.
SellD. Issues in perceptual speech analysis in cleft palate and related disorders: a review. Int J Lang Commun Disord. 2005;40(2):103–121.
57.
SellDHardingAGrunwellP. GOS. SP. ASS.’98: an assessment for speech disorders associated with cleft palate and/or velopharyngeal dysfunction (revised). Int J Lang Commun Disord. 1999;34(1):17–33.
58.
SellDJohnAHarding-BellASweeneyTHegartyFFreemanJ. Cleft audit protocol for speech (CAPS-A): a comprehensive training package for speech analysis. Int J Lang Commun Disord. 2009;44(4):529–548.
59.
SellDMildinhallSAlberyLWillsAKSandyJRNessAR. The Cleft Care UK study. Part 4: perceptual speech outcomes. Orthod Craniofac Res. 2015;18(suppl 2):36–46.
60.
SellDSouthbyLWrenYWillsAKHallAMahmoudOWaylenASandyJRNessAR. Centre-level variation in speech outcome and interventions, and factors associated with poor speech outcomes in 5-year-old children with non-syndromic unilateral cleft lip and palate: The Cleft Care UK study. Part 4. Orthod Craniofac Res. 2017;20(suppl 2):27–39.
61.
ShribergLDAustinDLewisBAMcSweenyJLWilsonDL. The percentage of consonants correct (PCC) metric: extensions and reliability data. J Speech Lang Hear Res. 1997;40(4):708–722.
62.
ShribergLDKwiatkowskiJ. Phonological disorders III: a procedure for assessing severity of involvement. J Speech Hear Disord. 1982;47(3):256–270.
63.
ShribergLDKwiatkowskiJHoffmannK. A procedure for phonetic transcription by consensus. J Speech Hear Res. 1984;27(3):456–465.
64.
ShribergLDLofGL. Reliability studies in broad and narrow phonetic transcription. Clin Linguist Phon. 1991;5(3):225–279.
65.
SitzmanTJAlloriACThorburnG. Measuring outcomes in cleft lip and palate treatment. Clin Plast Surg. 2014;41(2):311–319.
66.
SpruijtNEBeenakkerMVerbeekMHeinzeZCMBreugemCCMink van der MolenAB. Reliability of the Dutch cleft speech evaluation test and conversion to the proposed universal scale. J Craniofac Surg. 2018;29(2):390–395.
67.
StevensS.Laws that govern behavior (Book Reviews: Psychophysics. Introduction to Its Perceptual, Neural, and Social Prospects). Science. 1975;188(4):827–829.
68.
SweeneyTSellD. Relationship between perceptual ratings of nasality and nasometry in children/adolescents with cleft palate and/or velopharyngeal dysfunction. Int J Lang Commun Disord. 2008;43(3):265–282.
69.
Van DemarkDR. Misarticulations and listener judgements of the speech of individuals with cleft palates. Cleft Palate J. 1964;1(2):232–245.
70.
Van de WeijerJSlisI. Nasaliteitsmeting met de nasometer. Logopedie en Foniatrie. 1991;63:97–101.
71.
Vander PoortenVOstynFVan KerckhovenWWellensWBreulsMVerdonckAVergalleCSchoenaersJH. The Leuven staged supraperiosteal retropositioning repair: long-term velopharyngeal function in non-syndromic cleft palate. B-ENT. 2006;2(suppl 4):35–43.
72.
VerhoevenJ. Belgian standard Dutch. JIPA. 2005;35(2):243–247.
73.
WatsonASellDGrunwellP. Management of Cleft Lip and Palate. Hoboken, NJ: John Wiley & Sons Incorporated; 2001.
74.
WattersonTMcFarlaneSCWrightDS. The relationship between nasalance and nasality in children with cleft palate. J Commun Disord. 1993;26(1):13–28.
75.
WhitehillTL. Assessing intelligibility in speakers with cleft palate: a critical review of the literature. Cleft Palate Craniofac J. 2002;39(1):50–58.
76.
WildDGroveAMartinMEremencoSMcElroySVerjee-LorenzAEriksonP; ISPOR Task Force for Translation and Cultural Adaptation. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8(2):94–104.
77.
WilladsenELohmanderAPerssonCLundeborgIAlaluusuaSAuknerRBauABoersMBowdenMDaviesJ, et al.Scandcleft randomised trials of primary surgery for unilateral cleft lip and palate: 5. Speech outcomes in 5-year-olds-consonant proficiency and errors. J Plast Surg Hand Surg. 2017;51(1):38–51.
78.
Wong RiffKWTsangarisEGoodacreTForrestCRPusicALCanoSJKlassenAF. International multiphase mixed methods study protocol to develop a cross-cultural patient-reported outcome instrument for children and young adults with cleft lip and/or palate (CLEFT-Q). BMJ open. 2017;7(1):e015467.
79.
YamashitaRPBorgEGranqvistSLohmanderA. Reliability of hypernasality rating: comparison of 3 different methods for perceptual assessment. Cleft Palate Craniofac J. 2018. doi:10.1177/1055665618767116.
