Abstract
Objective
Patients commonly use the Internet to obtain their health-related information. The purpose of this study was to investigate the quality, accuracy, and readability of online patient resources for the management of articular cartilage defects.
Design
Three search terms (“cartilage defect,” “cartilage damage,” “cartilage injury”) were entered into 3 Internet search engines (Google, Bing, Yahoo). The first 25 websites from each search were collected and reviewed. The quality and accuracy of online information were independently evaluated by 3 reviewers using predetermined scoring criteria. The readability was evaluated using the Flesch-Kincaid (FK) grade score.
Results
Fifty-three unique websites were evaluated. Quality ratings were significantly higher in websites with a FK score >11 compared to those with a score of ≤11 (P = 0.021). Only 10 websites (19%) differentiated between focal cartilage defects and diffuse osteoarthritis. Of these, 7 (70%) were elicited using the search term “cartilage defect” (P = 0.038). The average accuracy of the websites was high (11.7 out of maximum 12), and the average FK grade level (13.4) was several grades higher than the recommended level for readable patient education material (eighth grade level).
Conclusions
The quality and readability of online patient resources for articular cartilage defects favor those with a higher level of education. Additionally, the majority of these websites do not distinguish between focal chondral defects and diffuse osteoarthritis, which can fail to provide appropriate patient education and guidance for available treatment. Clinicians should help guide patients toward high-quality, accurate, and readable online patient education material.
Introduction
Patient health literacy has been proven to be one of the most important indicators of health status.1-3 As the Internet becomes more accessible for people of all ages, a large number of orthopaedic patients rely on online resources as their primary source of health care information. However, many of the orthopaedic resources on the Internet are poor quality, inaccurate, and/or written at an inappropriately high reading level.1,4-9 Additionally, the overwhelming volume of information available on the Internet, of which very little is reviewed or regulated, can be difficult to manage for patients that independently seek to learn more about their conditions. Thus, there is a need for physicians to critically evaluate these resources and help guide patients toward high-quality, accurate, and readable information.
Cartilage repair and restoration procedures have garnered a significant amount of attention over the past few decades given the increased prevalence of isolated chondral injuries in young, active patients.10-12 A recent population based analysis revealed a 5% annual incidence growth in cartilage procedures performed in the United States over the past 8 years. 13 There are several well-established surgical options for isolated cartilage defects, and novel cartilage restoration techniques demonstrate promising results.14-16 However, the myriad of treatment options for articular cartilage defects and their respective indications can be confusing for patients, particularly for those who do not understand the distinction between focal articular cartilage defects and generalized osteoarthritis. Patients who are hesitant about undergoing a joint replacement are naturally drawn toward less invasive cartilage repair strategies, and they often seek independent resources to determine if they may be candidates for alternative procedures. Therefore, it is important that online patient resources regarding the treatment options for articular cartilage defects make this important distinction in an effort to properly educate patients as they attempt to make informed treatment decisions.
The purpose of our study was to evaluate the quality, accuracy, and readability of online patient resources for articular cartilage defects. We hypothesized that the quality of these resources would depend on the readability level and that the average readability level would be greater than the recommended eighth grade level for readable patient education material. 17 Additionally, we hypothesized that the majority of websites on articular cartilage defects would not distinguish between focal chondral defects and generalized osteoarthritis.
Methods
Three search terms were used to search for websites pertaining to articular cartilage defects: “cartilage defect,” “cartilage damage,” and “cartilage injury.” Each search term was entered as written without the addition of any other words 3 three search engines: Google, Bing, and Yahoo. These search engines were chosen because they represent more than 95% of searches performed on the Internet. 18 A total of nine searches were completed in July 2015. All browser cookies were cleared before each search. The first 25 results of each search were collected, and after duplicates were eliminated, 94 unique websites were evaluated (Figure 1). Forty-one websites were excluded because of irrelevant content (e.g., meniscus tear), broken links, login requirement, or they contained material intended for peer review. Ultimately, 53 websites met the inclusion criteria and were included for evaluation.

Search algorithm for online patient resources for the management of articular cartilage defects. After applying exclusion criteria, 53 websites were identified for review.
The quality and accuracy of the websites were assessed by methods similar to those used by previously published studies in the orthopaedic literature.6,9,19,20 The websites were gathered and sorted by a single researcher (R.H.J.) who was not involved in website scoring. Websites were categorized according to search term, highest search result order in any search engine (i.e., 1-5, 6-10, 11-15, 16-20, 21-25), authorship, commercial bias, and Flesh-Kincaid (FK) grade. Authorship was subdivided into health care provider (physician, nurse, or physical therapist with state credentials) and non–health care provider (blogs, boards, or personal web sites without explicitly stated credentials). A website was presumed to have commercial bias if it contained advertisements related to a product for profit. The websites were then randomized and independently assessed by 3 orthopaedic surgery chief residents (D.W., N.L.L., M.P.L.) blinded to search term and search result order. The 3 evaluators were trained by the senior author (K.J.J.) in the evaluation of quality and accuracy of the online content.
Website quality was determined according to grading criteria composed of 25 items encompassing the elements of anatomy, diagnosis, treatment options, and rehabilitation based on guidelines written by the American Academy of Orthopaedic Surgeons (AAOS) (Table 1). 21 Quality was scored by awarding one point per criteria contained in the website, with a maximum score of 25 points. Additionally, websites that differentiated between focal cartilage defects and diffuse osteoarthritis were recorded. Quality scores from the reviewers were then averaged to generate a mean quality score (maximum score of 25) for each website.
Quality Scoring Criteria.
Website accuracy was determined using a previously established scoring system.6,9,19,20 Each reviewer assigned an accuracy score of 1 if they agreed with <25% of the website content, 2 for agreement with 26% to 50% of the website content, 3 for agreement with 51% to 75% of the website content, and 4 for agreement with >75% of the website content. Accuracy scores were then added to create a composite accuracy score for each website (maximum score of 12).
Website readability was determined using the FK method.1,7,22 FK grade level indicates the maximum level of education a patient must possess to be able to read and comprehend the material. Thus, a higher FK grade level indicates material that is more difficult to comprehend. The FK level of each website was evaluated using the readability statistics function in Microsoft Word (Microsoft Corp, Redmond, WA). 23 The standard formula for FK grade level is = (0.39 × average number of words per sentence) + (11.8 × average number of syllables per word) – 15.59. 7 Websites were sorted by those with FK grade level >11 and those ≤11.
Statistics
One-way analysis of variance tests (for normally distributed data) and Kruskal-Wallis tests (for nonnormally distributed data) with post hoc pairwise comparisons utilizing Tukey’s tests (for normally distributed data) and Dunn’s tests (for nonnormally distributed data) were used to determine any differences in quality, accuracy, and readability between search term and highest search result order. Independent sample t tests (for normally distributed data) and Mann-Whitney U tests (for nonnormally distributed data) were used to determine any differences in quality, accuracy, and readability between authorship, commercial bias, and FK grade level. Chi-square test was used to determine any differences in the proportion of websites that differentiated between focal cartilage defects and diffuse arthritis among search terms. Interrater reliability for quality and accuracy ratings was evaluated using the intraclass correlation coefficient. Significance was set at the P < 0.05 level. Statistical analysis was conducted with GraphPad Prism 5 for Mac OS (GraphPad Software Inc, La Jolla, CA).
Results
Of the 53 unique websites, 19 were categorized under “cartilage defect,” 16 were categorized under “cartilage damage,” and 18 were categorized under “cartilage injury.” When sorted by highest search order, 13 websites appeared between 1 and 5, 8 websites appeared between 6 and 10, 12 websites appeared between 11 and 15, 13 websites appeared between 16 and 20, and 7 websites appeared between 21 and 25. Thirty-three websites (62%) were authored by health care providers, and 15 websites contained commercial bias (28%).
The average quality rating of all websites was 7.4 ± 4.4 out of a maximum of 25 points (range, 0-15.7) (Table 2). The quality rating was highly reliable among the reviewers with an intraclass correlation coefficient of 0.85. Websites that appeared between the search order of 11 to 15 demonstrated a higher mean quality score than those that appeared between 15 and 20 (9.4 ± 3.3 vs. 4.6 ± 3.9, P = 0.021). Websites with a FK level of more than 11 demonstrated a higher mean quality score than those with a FK level of 11 or less (8.2 ± 4.1 vs. 4.8 ± 4.3, P = 0.010). There were no significant differences among the mean quality scores of websites categorized by search term, authorship, or commercial bias. Only 10 websites (19%) differentiated between focal cartilage defects and diffuse osteoarthritis. Of these, 7 (70%) were elicited using the search term “cartilage defect” (P = 0.038). Websites that made this distinction had a higher mean quality rating compared to those that did not (10.2 ± 3.0 vs. 6.8 ± 4.4, P = 0.022). Twenty-five websites (47%) mentioned at least one well-established surgical treatment option (e.g., microfracture, osteochondral autograft transfer system [OATs], osteochondral allograft, autologous chondrocyte implantation).
Summary of Quality Results.
P = 0.021; †P = 0.010.
The average accuracy rating of all websites was 11.7 ± 0.6 out of a maximum of 12 points (range, 10-12). The accuracy rating was highly reliable among the reviewers with an intraclass correlation coefficient of 0.78. Forty-one websites (73%) received a maximum score of 12. There were no significant differences among the mean accuracy scores of websites within any category.
The average FK grade score of all websites was 13.4 ± 8.0 (range, 7-67) (Table 3). Only one website had an FK grade score that was below the eighth grade level, which is the recommended threshold for readable patient education material and the average reading level of the United States population. 17 Forty-one websites (73%) had a FK grade level of more than 11. Websites with commercial bias had a lower mean FK grade score than those without bias (11.4 ± 2.0 vs. 14.2 ± 9.2, P = 0.028).
Summary of Flesch-Kincaid (FK) Grade Results.
P = 0.028.
Discussion
With the ever-increasing ease of access to the Internet, more patients are relying on online resources as their primary source of health care information. Furthermore, with the advent of blogs and social media, the potential for the rapid circulation of online patient education material is immense. A significant amount of the orthopaedic resources available through the Internet are poor quality and/or inaccurate.4-6 Prior studies evaluating websites for low back pain, 5 scoliosis, 20 and hallux valgus 9 found that they are of limited quality and poor educational value. Other studies determined that the quality and accuracy of online resources for shoulder instability, 6 lateral epicondylitis, 19 distal radius fractures, 24 and developmental hip dysplasia 25 vary tremendously with the search term used. Thus, it is important to critically evaluate these resources and help guide patients toward high-quality, accurate, and readable information. In this study, we found that despite being accurate, online resources for articular cartilage defects are variable in quality and written at levels that far surpass the average reading level of the U.S. population.
Focal articular cartilage defects are common in young, active populations. 12 When left untreated, these lesions have the potential to progress in size and lead to the development of generalized arthritis. 26 Appropriate treatment of cartilage defects strongly depends on lesion characteristics and individual patient factors. Among the general population, there is a lack of sufficient understanding regarding the difference between a focal chondral defect and diffuse cartilage loss, and currently available online patient education material does not clarify the important difference between these 2 entities. We found that only 19% of websites on articular cartilage defects attempted to explain this important distinction, despite the very different implications for available treatment. Many patients with diffuse cartilage degeneration who want to avoid the morbidity of a joint replacement seek out less invasive cartilage repair strategies. General interest has increased in recent years due to widespread media coverage regarding novel stem cell and tissue engineering techniques.14-16 However, we found the majority of online resources for articular cartilage defects do not adequately define the indications for cartilage restoration surgery. Additionally, many websites list cartilage regeneration procedures as acceptable treatment options for patients with generalized osteoarthritis.27,28 As a result, patients with diffuse cartilage loss or advanced chondral pathology often need to be re-educated, thereby leading to inefficient time devoted to patient counseling.
Evaluation of the quality of websites revealed a bias that favors patients with higher levels of education. Quality ratings were significantly higher in websites with an FK grade score greater than 11. Furthermore, the average readability of the websites (13.4) was at a collegiate level and five grades higher than the recommended eighth grade level.1,17,29 This finding is particularly worrisome given that a patient’s health literacy has been proven to be one of the most important indicators of his or her health status.1-3 Interestingly, we noted the FK grade score of websites that reviewed the management of articular cartilage defects was higher than those reported for other orthopaedic subspecialties, including arthroplasty, foot and ankle, hand, oncology and spine, which range from 8.3 to 11.3.1,7,8,22,23,30-33 This finding may be due to the inherent nature of the topic, as an adequate review of cartilage pathology requires a review of basic science concepts and the utilization of associated terminology in its descriptions. Ultimately, the lack of comprehension by readers may further compound the confusion between focal cartilage lesions and diffuse arthritic disease, thus demonstrating a clear need to develop more readable online patient resources on articular cartilage defects.
Other studies regarding online patient education material report that the accuracy of orthopaedic resources is poor and can largely vary depending on authorship (i.e., a health care provider or non–health care provider).6,19,25 Garcia et al. 6 reported an overall accuracy of 8.6 out of 12 for websites on shoulder instability, indicating a 26% to 75% level of reviewer agreement with the online content. Similarly, Mathur et al. 20 reported that the majority of online resources on scoliosis were in the 0% to 25% accuracy range. In contrast, we found that online resources for articular cartilage defects are generally very accurate. This may be due to a greater emphasis on basic science in the discussion of cartilage pathology and restorative treatment. In our review of the online resources, we noted a cellular focus on the pathologic descriptions of cartilage injury and the mechanisms of surgical techniques, including microfracture, autologous chondrocyte implantation, and tissue engineering techniques. Because of the inherent biologic focus of the topic, there is likely less ambiguity when translating factual scientific information to online resources for patients. In contrast, patient resources regarding the treatment of other orthopaedic conditions that have been studied in a similar fashion (e.g. back pain, shoulder instability) are more dependent on the interpretation of clinical signs and symptoms, and as such, there may be an increased likelihood of introducing inaccurate information into online resources.
There is significant potential for the Internet to capably educate patients with appropriate physician guidance.34,35 However, the overwhelming volume of information available on the Internet, which is largely unreviewed and unregulated can be misleading for patients seeking to learn more about their conditions. Ultimately, this can lead to inefficient interactions between the doctor and patient, as a significant amount of time is required to re-educate the patient and clarify misleading information. Therefore, physicians should endeavor to direct patients to high quality and readable online resources. Based on our comprehensive review of the available websites, we developed a list of the top 3 websites on articular cartilage defects with the highest overall quality rating in our study (Table 4). We realize that patient resources on the Internet are fluid, but we hope this will serve as a guide for physicians to help patients learn more about articular cartilage defects and make informed decisions regarding management of their condition, thereby improving the patient-physician relationship.
Recommended Websites on the Management of Articular Cartilage Defects for Patients.
AAOS = American Academy of Orthopaedic Surgeons; URL = uniform resource locator.
There are several limitations of this study. First, our comprehensive search was a single snapshot of available websites on the Internet and does not accurately demonstrate the relative fluidity of available information on the Internet. The order of search results may change on a daily basis, and thus, there may be other quality websites that were missed at the time of our search. Second, our search was limited by the search terms used. We chose to avoid the use of “chondral,” “lesion,” or other medical terminology that may be unfamiliar to the general public in order to replicate the most common searches performed by patients. However, using such technical terms may have elicited more websites that contain high quality and accurate information. Next, our study only included patient resources in English and does not evaluate online resources in other languages that may contain more suitable content for the general population. An additional limitation includes the fact that all of the website evaluators were trained at the same institution, there was a potential for bias in the evaluation of website accuracy. Next, some institution- or surgeon-authored websites that were presumed to be without commercial bias may indeed gain monetary benefits from highlighting specific treatments to cater to their patient populations. Finally, although the use of the FK grade score is reproducible and well supported in the literature,7,8,23,30,31 scores are based on word length and sentence length, which are unreliable proxies for semantic and syntactic complexity. 36
In conclusion, the quality and readability of online patient resources for articular cartilage defects favor those with a higher level of education. This finding is consistent with previous studies that examined the quality and readability of orthopaedic patient education materials on the Internet. Additionally, the majority of these websites did not distinguish between focal chondral defects and diffuse osteoarthritis, which can be a significant source of confusion for patients who seek alternative solutions to joint arthroplasty. We hope that the present work highlights the limitations of readily available educational material for patients regarding the management of cartilage lesions and may provide insight into patients’ decision making and clinical expectations. Given the role of health literacy in a patient’s ability to make informed decisions regarding their health care outcomes, as well as the increasing prevalence of Internet use among patients, a significant amount of work is needed to improve the quality and readability of online patient resources for articular cartilage defects.
Footnotes
Authors’ Note
The work reported in this article was completed at the Department of Orthopaedic Surgery, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, California.
Acknowledgments and Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
