Abstract
The Internet is capable of providing an unprecedented amount of information to both physicians and patients interested in headache. To assess the status of headache information on the Internet (as of January 2000), a search for ‘headache’ was performed using 10 leading Internet search engines. The number of web pages identified ranged from 4419 (WebCrawler) to 506 426 (Northern Light). The ‘average’ search yielded nearly 150 000 web page listings for ‘headache’. The content was then reviewed of the top 10 listed web pages for each search (i.e. a total of 100 page listings). The results demonstrate that, at the present time, Internet-based information on headache is extensive but poorly organized. Editorial review of this potential valuable resource is required in order to maximize its utility in headache education and management.
Introduction
The Internet is revolutionizing all aspects of communication. The medical field is a major part of this massive social change. Medical information has been reported to be the second most common search topic on the Internet. It is estimated that as many as 85 million Americans searched for medical information on the Internet in 1999, an increase from an estimated 17 million searchers in 1998. Clearly, the use of the Internet for patient information needs will continue to expand as Internet access becomes more readily available.
The advantages on Internet-based medical information for patients are obvious (1). The Internet offers widespread, low-cost and easy access to extensive information on an unprecedented scale. The amount of medical information is already voluminous and continues to grow exponentially. Over a billion web pages now exist as of early 2000, a number which is expected to double within the next year or two. At present, there are an estimated 38 new web pages being added to the Internet every second.
In order to facilitate the utility of this information, the Internet community has developed a number of ‘search engines’ that allow for the identification of web pages containing certain keywords. The ability to search the Internet using ‘search engines’ has become indispensable for anyone looking for information on the Internet. However, the results of most web searches consist of a listing of thousands of web pages for most common medical terms. Extensive lists are provided of possible web sites of interest, with little or no evaluation of content quality (2, 3).
In theory, the availability of extensive information on headache could lead to better headache education and, ultimately, patient management. However, finding useful headache information on the Internet is becoming increasingly difficult as the number of web pages grows and the ability to evaluate the content of all available pages decreases (4, 5).
By contrast, it is possible to find useful information in the medical literature, which also consists of millions of pages of data. The medical literature is well organized as a result of the peer-reviewed system. Dates of publications, authorship and study sponsorship are expected components of medical publications. Moreover, the peer-review process provides an invaluable screening service that, in general, allows the reader to be able to assess the quality and relative importance of the data presented based on the journal in which the data were published, author(s), sponsors, etc.
The present study was designed to assess the relevance and quality of headache information available to patients on the Internet.
Methods
Analysis of ‘headache’ web pages
In an attempt to assess the status of headache information on the Internet (as of January 2000), a search for ‘headache’ was performed using 10 leading Internet search engines. The content of the top 10 listed web pages for each search (i.e. a total of 100 page listings) was then reviewed.
Rating criteria
The content of the 100 identified web pages in the above Internet searches was evaluated. The criteria described below are a prototypical attempt to combine traditional peer-review criteria with elements of review related to the unique nature of Internet-based information.
Basic web site characteristics (20 possible total points)
Five points each were awarded to a web page for:
the listing of author(s)
date of page creation or of last edit
sponsor of the web page
disclaimer to patients stating that the provided information was for informational purposes only and
that a physician should be consulted for a complete medical evaluation.
Clinical content (20 possible total points)
The clinical content was evaluated as to its relevance to a general patient population. The relevance of the site was considered higher if it contained primary data vs. referrals to other sources. In general, the amount of information that was provided on a web site was positively associated with the clinical content rating. An emphasis was also placed on the appropriateness of the material for patient, as opposed to healthcare provider, usefulness. The following rating system was used to assess the clinical content of each web site: 0=none; 5=poor; 10=average; 15=good; 20=excellent.
Accuracy of information (20 possible total points)
Web page content accuracy and data validity were also assessed using the rating scale described above.
References (20 possible total points)
Both references to primary medical sources as well as links to other relevant web sites were evaluated using the rating scale described above.
Web site function and design (20 possible total points)
In addition to the more ‘standard’ peer-review type of criteria listed above, a number of features that are unique to the Internet were assessed. For example, the web site design, ease of use, presence of an overview and/or site map on the home page of the web site, etc., were evaluated. Once again, the 0–20 point rating scale described above was used in the evaluation of each web site.
Therefore, a total of 100 possible points could be awarded per web site.
Results
Search engine analysis for ‘headache’
As shown in Table 1, the number of web pages identified ranged from 4419 (WebCrawler) to 506 426 (Northern Light). The ‘average’ search yielded nearly 150 000 web page listings for ‘headache’. The contents of these pages ranged widely from personal web pages to major pharmaceutical company web sites. Two of the 100 listed web pages were no longer functioning. Thirty-nine web pages appeared only once in the top 10 list in any of the 10 independent searches. Fourteen web pages were listed in the top 10 list of two to five of the 10 independent searches. Two web pages were listed in seven of the 10 independent Internet searches.
Number of web pages identified by a search for ‘headache’
Web site appearance in the ‘top 10’ list from of 10 different search engines
The content of each site listed in the ‘top 10’ of each search engine result was then reviewed and categorized. Out of the 100 sites ranked in the ‘top 10’ of results by the 10 different search engines, a total of 51 different web pages and four non-functioning sites were identified, as shown in Table 2.
Web site appearance in the ‘top 10’ list from of 10 different search engines
Evaluation of the ‘top 10’ Internet ‘headache’ sites from 10 different search engines
As stated above, 51 unique ‘headache’ web pages were identified in the present study. Each of the sites was then analysed for its relevance as a source of patient information on headache using the review criteria described in Methods. The results of this analysis are shown in Table 3.
Evaluation of ‘top 10’ web sites based on the criteria listed in Methods
A total of five sites were rated as being ‘good to excellent’ (defined as having received 76–100 points in the evaluation), whereas as an additional five sites were rated as ‘average to good’ (defined as having received 51–75 points in the evaluation). The remaining 41 sites received less than 51 points in the evaluation.
Search engine detection of highest rated headache sites for patients
The presence of the five highest rated headache web sites in the ‘top 10’ list from each of the 10 different search engines was then determined. The ‘top 10’ list of the 10 different search engines contained between 0 and 3 of the five highest rated web sites. Only 19% of the sites contained in the top 10 lists from the 10 different search engines were evaluated as being ‘good to excellent’ as a source of patient information.
Discussion
The major finding of the present study is that the Internet contains an extensive and excellent amount of patient-related headache information but that existing search methods are incapable of providing a concise, quality-based assessment of the available data. Given that an average of 150 000 web page ‘hits’ for headache are provided by existing search engines, no patient can be expected to be able to evaluate the educational utility of each of these sites. Indeed, the clinical quality of the vast majority of web sites is extremely poor if evaluated on the basis of traditional medical criteria. Even if patients were to limit their analysis to the ‘top 10’ sites listed by a given search engine, then the data in the present study suggest that as many as 80% of the web sites would be considered less than optimal by most healthcare providers.
Therefore, this study demonstrates the need for health care providers to provide their patients with guidance on how to gather useful medical information on the Internet. The review criteria used in the present were developed as a prototype in order to initiate a discussion within the medical community that would allow for a long-term solution to the problem of Internet information filtering for patients (6–8). The criteria used in the study are by no means offered as being definitive. Indeed, any criteria used for Internet site evaluations must be validated and refined before the general medical community could accept them.
Increasingly easy access to the Internet provides health care providers with an unprecedented resource for patient education and management. High-quality medical information on the Internet exists today, yet a simple and reliable method for locating such sources of information does not yet exist. Establishing a clear and objective criteria set for web sites seems to be the first step needed to maximize the utility of the Internet for patients. Once established, a central listing of quality-based evaluations of medical web sites would appear to be an invaluable resource for both health care providers and their patients.
Footnotes
Acknowledgements
I thank Dr Darlene M. Dreon for her thoughtful comments on this manuscript.