Analyzing the Readability of Facial Paralysis Patient-Reported Outcome Measures

Key Messages

• Future patient-reported outcome measures should be developed within recommended readability levels

• Clinicians should be cognizant of how higher than recommended readability levels of existing patient-reported outcome measures may impact how certain patient populations complete them

Introduction

Facial paralysis (FP) can significantly impact quality of life for patients in different ways, and treatment should be guided by patients’ symptoms and experiences. Therefore, it is critical to understand their experiences using patient-reported outcome measures (PROMs).

PROMs provide patients with an opportunity to report health experiences while minimizing external influence on their responses. Health care providers should ensure that disseminated patient materials, including PROMs, are at an appropriate reading level so patients can understand and accurately complete them. ¹ The National Institutes of Health and American Medical Association recommend health education materials to be written at or below the sixth to seventh grade level. ²

The readability of commonly used FP PROMs has not been studied. This research evaluates the readability of the Facial Disability Index (FDI), Facial Clinimetric Evaluation Scale (FaCE), Synkinesis Assessment Questionnaire (SAQ), and FACE-Q Craniofacial Module (FACE-Q).^{3 -6} We hypothesize these commonly used FP PROMs have higher than recommended readability levels.

Materials and Methods

This study did not require institutional review board approval. PROMs were converted to text-based documents and reviewed for accuracy. Readable software (Added Bytes, East Sussex, United Kingdom) was used to evaluate readability levels. Three readability formulas were used to analyze each PROM: Gunning Fog, SMOG, and FORCAST.^{7 -13}

The Gunning Fog score considers complex or polysyllabic words and is best utilized to assess research writing. ¹⁴ This score provides an estimated grade level correlating to the readability level required to understand texts. The SMOG Index includes polysyllabic words and approximates the number of years of education necessary to understand writing materials. ¹⁵ Due to its ability to assess complete comprehension, SMOG is frequently used in healthcare. ¹⁵ PROMs are often presented as surveys, which are best analyzed by the FORCAST formula. ¹⁶ Unlike the Gunning FOG and SMOG, FORCAST does not require complete sentences for accurate analysis, lending itself useful for survey and questionnaire analysis. ¹⁶

Gunning Fog (GF) Formula:

$$\mathrm{Reading}\mathrm{level}=0.4*\left(\begin{array}{l}\left(\mathrm{average}\mathrm{sentence}\mathrm{length}\right)+\\ \left(\%\mathrm{polysyllable}\mathrm{words}\right)\end{array}\right)$$

Simple Measure of Gobbledygook (SMOG) Formula:

$$\mathrm{Reading}\mathrm{level}=3=+\sqrt{\mathrm{polysyllable}\mathrm{count}}$$

Polysyllable count is calculated from a 30-sentence sampling. The algorithm is adjusted for narratives less than 30 sentences.

FORCAST Formula:

$$\mathrm{Reading}\mathrm{level}=20-(\mathrm{N}\backslash 10)$$

where N = number of monosyllabic words in a 150-word sample.

The readability level for each PROM was determined using the formulas above. Descriptive statistics were generated using Microsoft Excel (Microsoft, Redmond, WA).

Results

Average readability levels for Gunning Fog, SMOG, and FORCAST across all PROMs studied were grade levels 6.4, 7.6, and 8.3, respectively (Figure 1 and Table 1). Mean readability levels of the SAQ, FDI, FaCE, and FACE-Q across readability formulas were 5.4, 10.5, 7.7, and 6.2, respectively (Figure 2). The SAQ was the only PROM that met the recommended sixth-grade readability level.

Figure 1.

Readability for facial paralysis PROMs by formula.

Table 1.

Readability Scores for Facial Paralysis PROMs.

PROM	Gunning fog	SMOG	FORCAST	Mean
FDI	11.2	11.5	8.8	10.5
FaCE	6.2	7.7	9.1	7.7
SAQ	3.8	5.0	7.3	5.4
FACE-Q	4.3	6.2	8	6.2
Mean Readability of All PROMs	6.4	7.6	8.3	7.4

Abbreviations: FaCE, Facial Clinimetric Evaluation; FACE-Q, FACE-Q Craniofacial Module; FDI, Facial Disability Index; PROMs, patient-reported outcome measures; SAQ, Synkinesis Assessment Questionnaire; SMOG, Simple Measure of Gobbledygook.

Figure 2.

Facial paralysis PROM readability by PROM.

Discussion

Three out of 4 commonly-used FP PROMs are written above the recommended readability level. The high readability level of FP PROMs is similar to PROMs in other otolaryngology subspecialities and to FP education materials.^{10 -13,17}

Using the FORCAST algorithm, no PROMs met recommended readability levels. The Gunning Fog and SMOG algorithms are utilized for analyzing narrative text, which is not typical of PROM questionnaire-based formats, and may subsequently lead to underestimation of readability levels. ⁹ Further, none of the algorithms accounted for difficulties of interpreting medical terminology, which may also result in underestimation of readability levels. ¹⁸ These findings suggest the true readability level of FP PROMs is likely higher than this research predicts.

Future FP PROMs should be developed to optimize patient understanding to thereby minimizing communication errors and improving patient-centered care. While current FP PROMs have been validated and are an important part of patient care, clinicians should keep in mind that they exceed recommended reading levels, which may impact how some patients complete them. Future FP PROMs should be developed to optimize patient understanding to thereby minimizing communication errors and improving patient-centered care. In the interim, continued use of these well-established PROMs is encouraged for outcomes tracking, recognizing that further assistance and support may be required by clinic personnel to facilitate completion.

Conclusion

Facial paralysis PROMs have been written above the recommended sixth-grade readability level. Future PROMs should be developed within recommended readability levels, and clinicians should keep in mind how the higher-than-recommended readability levels of existing PROMs may impact how certain populations complete them.