Response to Letter to the Editor Titled Remote Fiberoptic Nasopharyngolaryngoscopy in the Pediatric Population: Methodological Issues on Reliability and Accuracy

We appreciate reviewing Dr Sabour’s letter to the editor regarding our paper published in April 2020, “Remote Fiberoptic Nasopharyngolaryngoscopy in the Pediatric Population: Methodological Issues on Reliability and Accuracy.”

In his letter, Dr Sabour posits the Cohen κ statistic that we utilize in our manuscript is limited in scope due to the inability to account for prevalence. In response to his concerns regarding methodology and choice of statistics used, there are different methods that can be used to assess agreement based on the “type” of outcome variable. We chose to use Cohen’s κ given that the outcomes of our study were binary with only 2 raters, an accepted use of Cohen κ. ¹ Although we agree there are limitations to Cohen κ with respect to prevalence and concordance, we followed the guidance of McHugh et al ² and used Cohen κ along with a measure of “percentage agreement.” In using this method, we diminish the assumption for the possibility of “chance agreement” as both raters were considered to be well-trained experts in the field and unlikely to “guess” at the outcome. ² We still support our choice of analysis as the other tests mentioned are better suited for different situations as noted by Ranganathan et al. ¹ In doing so, we feel our results have not changed. We cannot support our use of the term “accuracy” implying validity specifically but instead agree that our results relate to “reliability” or agreement between our 2 raters. Our use of the phrase, “strong diagnostic accuracy” can be misinterpreted, and the authors here would reinforce that it truly showed “strong agreement” instead.

Finally, the data entry in Dr Sabour’s Table 1 (with attention to situation b) is mathematically incorrect—the number 5 should be where the numbers 1 is for in both of the yes/no categories to obtain the kappa he has calculated.