Abstract
Falls among older people are a major public health issue. Increasing numbers of older people are accessing the internet for health-related information, including information on falls risk and prevention. However, we are aware of no study that has assessed the quality of such websites. Using techniques for conducting systematic literature reviews, we evaluated English-language websites offering falls-related advice to members of the public. Forty-two websites were identified using popular search engines; these were assessed using evidence-based guidelines and codes of conduct on coverage of falls-related information, credibility and senior friendliness. Overall, scores were poor for coverage of falls information and credibility, although they were higher for senior friendliness. Few of the websites had been recently updated and none provided individually-tailored advice. We conclude that websites have fallen short of their potential to provide accessible, evidence-based information on the risks of falls and their prevention.
Introduction
Rapid increases in life expectancy and declines in fertility rates observed during the twentieth century have led to an increase in the percentage of older people in today’s societies. Associated with this is the issue of whether health can be preserved in later years so that older people can retain their independence and quality of life. Central to this issue is the problem of falls among older people, which has now reached global recognition. 1 Some 30% of community-dwelling people over 65 fall each year. About 20% of falls require medical attention2, 3 and around 5% result in fractures, 4 which poses substantial economic costs.5, 6 Falls are also associated with social and psychological consequences, with individuals restricting their activity owing to loss of confidence and fear of further falls. 7 As well as leading to isolation, this reduction in activity can contribute to further deconditioning, weakness and abnormal gait, and so eventually increase the risk of future falls.4, 8
Providing information to older people and their families on falls and their prevention is an important step in helping them change their behaviour and environment to reduce their risk of falling. One way of doing this is via the internet. The internet can be an effective and efficient way of making information available to the general population9–11 to facilitate health promotion and self-management of long-term conditions. Older people are increasingly accessing the internet: in 2010, 40% of UK adults aged 65 and older reported having ever used it, 12 a huge increase on the corresponding figure of 10% reported in 2000. 13 The proportion of older Americans reporting using the internet in 2010 was similar, at 42%. 14 People are also increasingly using the internet to access health information for themselves and their families: 15 in 2010, 36% of UK internet users aged 65 and older and 44% of those aged 55–64 had looked for health information online. 12 It is unlikely that internet users currently in their 50s will give up their ‘wired ways’ as they get older and retire, 16 so the internet will become even more important as a source of information for older people over the coming decades.
The unregulated nature of the internet, however, means that the quality of advice offered is difficult to monitor. Information may be incomplete, inaccurate or misleading. 17 Furthermore, problems such as small font sizes or inappropriate use of multimedia may seriously hinder older people’s internet use. 11 Although information concerning falls risk and prevention is widely available to the public via the internet, we have found no study assessing the quality and credibility of online information, and usability of falls-related websites. Studies evaluating websites in other health-related areas, however, have generally found them to be of poor quality.18–23 The aim of this study, therefore, was to identify websites offering advice on falls risk and prevention to older members of the public and to evaluate their coverage of evidence-based advice, credibility and usability.
Methods
Using techniques drawn from systematic literature review methodology 24, 25 we searched the internet for websites that provide older members of the public and their families with advice on falls risk and prevention.
Search strategy
Nielsen NetRatings monitors and measures more than 90% of global internet activity and provides insights about the online environment, including the search behaviour of internet users. The top three search engines at the time we conducted our searches (August 2010) were Google (65%), MSN/Windows/Bing (14%) and Yahoo! (13%). 26 These three search engines were used to perform 18 searches each, giving a total of 54 searches. Six keywords were used: ‘falls’, ‘preventing falls’, ‘falls prevention’, ‘reducing falls’, ‘improving balance’ and ‘balance training’. These six terms were used in isolation and then in combination with ‘older people’ and ‘elderly’. The keywords reflect the fact that although many websites target their audience using a falls-prevention message, research suggests that older people reject the notion that they are personally at risk of falls but respond to messages about improving balance. 27 We also sought advice on our search strategy from three user representatives who were all aged over 65 and had experience of using the internet to check that our search terms reflected those that might be used by this age group. This advice took the form of face-to-face discussions and observations of the user representatives’ own searching behaviour.
The total number of websites returned for each search was recorded. However, as research shows that internet users explore only the first few links on search engines,28,29 only the first 20 hits identified by each search were examined. Duplicate sites and those that did not satisfy the inclusion criteria were removed to provide the sample for evaluation. Inclusion and exclusion criteria are shown in Table 1. Both websites offering generic advice on falls risk and prevention and those allowing the user to complete questions on their own risk of falling and offering individually-tailored advice in response to the answers provided were acceptable for inclusion.Links to internal pages that met the inclusion criteria were followed and included in the review. Links to external sites were not followed. A random selection of 10% of the websites was assessed independently by two reviewers (FB, SW) to confirm their inclusion/exclusion status. Any disagreement was resolved by discussion, using a third reviewer (SN) where necessary, until mutual agreement was reached. The remaining websites were then assessed by a single reviewer (FB).
Website inclusion and exclusion criteria
Assessment of websites
In line with systematic review techniques and evaluation of other health-related information, we assessed the sites using clear assessment criteria.24, 25, 30, 31 We designed an evaluation form (Appendix 1) to record coverage of falls information, credibility and the senior friendliness of each site. This form was also used to collect general information on each site.
Coverage of falls information
Coverage of advice related to falls risk and prevention was assessed by comparing information on the websites with that provided in the joint American Geriatrics Society and British Geriatrics Society Guideline for the Prevention of Falls in Older Persons 32 and National Institute of Clinical Excellence (NICE) Guidelines on the Assessment and Prevention of Falls in Older People. 33 For each item, a summary statement was prepared detailing what information the website should present based on best current evidence contained within these two sets of guidelines. If information was included accurately for each item, one point was scored, whereas if the information was included inaccurately, was insufficient or incomplete, or was not mentioned at all, zero points were scored. Based on the strength of the evidence to support the conclusions made within the two sets of guidelines, information was categorised into three grades: A = good evidence for strong recommendations (3 items); B = fair evidence for recommendations (6 items); and C = fair evidence but no clear recommendations (11 items). This gave a total score range of 0–20 points across the three categories.
Credibility
Credibility of the websites was assessed using the Health on the Net Code of Conduct for Medical and Health Websites (HONCode). 34 The code consists of eight principles assessing: (1) authority of information; (2) clarity of purpose; (3) treatment of confidential information; (4) referencing and dating of information provided; (5) justification of claims regarding commercial products; (6) provision of contact details; (7) disclosure of funding sources; and, (8) disclosure of advertising policy. For each principle one point was allocated if the criterion was satisfied and zero points were allocated if the criterion was not satisfied. However, it was noted that criterion five, relating to justification of claims regarding commercial products, was not applicable to any of the websites that we assessed and so this item was not used. The score range for credibility was therefore 0–7.
‘Senior friendliness’
‘Senior friendliness’, or usability, was assessed against guidelines developed by the National Institute on Aging and the National Library of Medicine. 35 The aim of this checklist is to provide research-based design guidelines that will make websites more accessible to all adults. Guidance is provided on 40 items across five categories of organising information (8 items), writing online text (11 items), designing readable online text (9 items), making information easy to find (9 items) and including other media (3 items). For each item, one point was scored if the requirement was satisfied, and no points were scored if the requirement was not satisfied, giving a score range of 0–40. The total possible score across all three categories (coverage of falls information, credibility, and senior friendliness) was therefore 0–67.
Reviewer agreement and statistical analysis
All sites were assessed independently by two raters (SW and SN for coverage of falls information, SW and FB for credibility and senior friendliness). For each website, scores were compared across each of the three assessment categories. For coverage of falls information, scores were deemed to be in suitable agreement if they were within two points of one another. For credibility, only a one-point difference was deemed to be acceptable. For senior friendliness, a difference of four points was allowable. This effectively represented an allowable difference of as close as possible to 10% between the two raters for each of the assessment categories. Where scores differed by more than these agreed limits, discussion took place between the raters to resolve differences of opinion.
The two raters’ mean scores were calculated and intra-class correlations (ICC) were performed for each of the three assessment categories, as well as for the overall score. A two-way random effects model was used, using an absolute agreement definition. The single measure ICC is reported along with their 95% confidence intervals (CIs). After checking that scores were normally distributed, mean scores and standard deviations were calculated for total scores, and for each of coverage of falls information, credibility and senior friendliness, as well as for the sub-categories within these. Analyses were initially carried out both across all websites and comparing scores according to organisation type and country of origin of the website. We hypothesised that scores might vary here, as previous reviews of health-related websites have found such differences to exist.21–23 However, exploratory analyses showed there to be no significant differences in scores according to either organisation or country of origin and so scores are presented for all websites together. All statistical analyses were carried out using SPSS v.16.0.
Results
The searches each returned between 124,000 and 885,000,000 websites (mean =53,634,963). Based on the first 20 websites returned by each of the 54 searches, 1,080 websites were examined, 1,038 were either duplicates or did not fulfil the inclusion criteria, leaving 42 sites to be assessed more fully. Figure 1 shows how studies were identified and reasons for exclusion.
Flow diagram of identification and retention of sites for review
The characteristics of the 42 websites retained for assessment (Appendix 2) are summarised above (Table 2). The majority of websites originated from the USA (48%) or the UK (31%), with a leaning towards sites provided by government (41%) or commercial organisations (38%).
Frequency (%) of reviewed websites according to country of origin and type of organisation
NZ = New Zealand.
Eight websites (19%) were updated during 2010 and seven (17%) were updated during 2009. A further seven sites (17%) were updated between 2005 and 2008. Sixteen (38%) did not state when the last update took place. Only three of the 42 websites were dedicated falls websites, with the vast majority of websites covering a variety of health topics. All of the websites offered static and generic advice on falls risk and prevention; none provided the opportunity for individuals to complete questions relating to their own risk of falling or provided advice tailored to the individual on the basis of his or her answers to such questions.
Agreement between the raters was high for falls-related information (ICC = 0.97, 95% CI = 0.94–0.98, p < .001),and for credibility (ICC = 0.90, 95% CI = 0.79–0.95, p < .001). It was slightly lower for senior friendliness, although still at a more than acceptable level (ICC = 0.81, 95% CI = 0.68–0.90, p < .001). Overall agreement for the total score was high (ICC = 0.92, 95% CI = 0.83–0.96, p < .001).
Scores for coverage of falls information, agreement with the HONCode and senior friendliness are presented in Table 3. Scores for coverage of falls information were generally poor, with a mean total score of only 3.92 (SD 3.60) out of a possible 20. Further investigation revealed that scores were poor across all categories of evidence and 5 of the 42 websites failed to score any points at all.
Website scores for coverage of falls-related information, credibility and senior friendliness
Assessed against NICE Guidelines and American Geriatric Society/British Geriatric Society Guidelines: Grade A = good evidence for strong recommendations; Grade B = fair evidence for recommendations; Grade C = fair evidence but no clear recommendations.32, 33
Assessed against the Health on the Net Code of Conduct for Medical and Health Websites (HONCode). 34
Assessed against National Institute on Aging and the National Library of Medicine Guidelines. 35
Compliance with the HON Code was slightly better, although scores were still fairly poor overall. The mean score across all websites was 3.86 (SD 1.65) out of a possible 7. Six of the sites assessed stated that they were HON Code compliant and a Mann Whitney U-test showed that mean scores were significantly better among these websites than those that did not state they were HONCode compliant (U = 36.00. p = .008).
Senior friendliness scores were reasonably high. The mean score was 31.69 (SD 2.69) out of a possible 40, and scores were high across all of the categories, with the exception of including other media. It was noted, however, that few of the websites we assessed incorporated other media; for these websites, this, in effect, gives a maximum possible score of 37. This should be kept in mind when considering the overall scores.
The mean score overall was 39.46 (SD 5.40) out of a maximum possible 67. To give a further indication of the overall quality of the websites, total scores were converted into percentages. So that each of the categories of coverage of falls-related information, credibility and senior friendliness were equally represented in the overall percentage score, scores for each of these categories were first converted into an individual percentage before a total average percentage was calculated. The mean percentage score across all websites was 51.30 (SD 10.64).
Websites were given an overall grade based on their percentage score. Websites were categorised as being poor overall if they scored less than 50%, fair if they scored 50–74.99% and excellent if they scored 75% or greater. Twenty-two of the 42 websites were classified as being poor overall and 19 were classified as being fair. Only one of the websites was rated as being excellent. This was a USA-based website originating from a government department.
Discussion
The purpose of this study was to systematically identify and evaluate websites offering advice on falls risk and prevention to older members of the public and their families. Forty-two websites were identified that fulfilled the search criteria and these were evaluated in terms of coverage of falls-related information, credibility and senior friendliness.
Scores for coverage of falls-related information were generally very poor, and there was no single area where websites scored particularly well. This suggests that providers of web-based information need to work towards providing more comprehensive information on all the evidence-based risk factors and interventions. Reasons for the poor quality of information could include competing interests of the authors, lack of peer review and failure to update information, 36 and, indeed, 38% of the websites did not provide a date of the most recent revision. Information on many of the websites was also very scant, with many of the falls-related topics either not being covered at all or being afforded only a short statement. Depth of coverage for key topics has also been found to be minimal in a review of smoking cessation websites. 37 It is important that enough information is provided so that members of the public are able to be pro-active in making positive changes to their lifestyles and their environments. However, too much information could prove overwhelming, 38 leading to users missing important content 37 or looking elsewhere for information that, whilst incomplete or inaccurate, may prove to be more readable. Bock et al. 37 suggest that it may be useful to design websites in such a way that users can drill down to their desired level of detail on any given topic. Qualitative research similar to that of Yardley et al. 38 would be helpful in examining the needs and requirements of older internet users so that the amount of falls-related information provided can be pitched at an appropriate level.
It is also possible that websites might score poorly on coverage of falls-related information owing to the author not having suitable knowledge or qualifications to provide such information. The HONCode was developed to avoid such problems and websites that are HONCode-compliant should provide details on the training of the author and where the information provided comes from. Compliance with the HONCode was fairly poor overall, however, and only six of the sites assessed stated that they were HONCode-compliant. It may be the case that many providers of health-related information on the internet are still unaware of available standards to demonstrate the credibility of their websites. With so much information now available on the internet, and with very limited opportunities for policing of this information, members of the public need to know that the information they are accessing is of good quality. It is vitally important, therefore, that health professionals with relevant expertise are involved in the production of health-related information online. 39 In addition, evidence-based information should be clearly visible online and easily identifiable as reliable and from a reputable source. A key challenge, however, is to raise awareness of such standards as the HONCode to both website providers and members of the public. Research has shown, for example, that when accessing health information on the internet, users do not attempt to find out who the authors or owners of the site are, nor do they read disclaimers or disclosure statements. 28 Unless awareness of standards for demonstrating and assessing the credibility of websites increases, the internet may continue to be saturated with poor quality information provided by individuals and organisations with a lack of sufficient authority and knowledge. Members of the public, meanwhile, may continue to accept such poor quality information, possibly jeopardising their own health in the process.
One of the main advantages of the internet is its ability to instantly provide individually-tailored feedback and advice,10, 37, 40 and it was disappointing that no website provided any form of tailoring. A recent review of web-assisted tobacco interventions similarly found that only a minority of websites provided interactive features or personalised treatment. 37 However, support exists for tailoring falls prevention advice to older people from a website promoting strength and balance training. 41 A pooled analysis of two experiments also showed that compared with a generic control, tailored advice was perceived to be more personally relevant and older people reported greater intention to act on the advice in response to the tailored version. 42 Tailoring could therefore be a cost-effective way of encouraging positive behaviour change and yet it appears that it is being under-utilised in current falls and health-related web provision. 10
It was promising to note, however, that the websites scored generally well in terms of senior friendliness. Now that the World Wide Web is well established, it seems likely that good design principles will have become more entrenched. Older members of the public may well have different requirements, however, and it is encouraging that the websites reviewed seem to be, for the most part, attending to these requirements.
There are several limitations to this study. Owing to the sheer size and dynamic nature of the World Wide Web, it is impossible to identify all of the falls-related websites that are available, and so good quality websites may have been missed. Even amongst those websites that were identified, the internet moves so rapidly that websites can change or even disappear. 43 The search terms were also designed to reflect those that might be used by members of the public and so websites using more medical or specialist keywords may have been missed. However, the searches and evaluations were carried out by academic researchers and so we cannot draw conclusions about what members of the public themselves encounter when they search for falls-related information or how well they are able to interpret the information they find. 18 Only English language websites were reviewed and so the findings may not be reflective of websites written in other languages. We are aware, for example, of a Dutch-language website offering falls prevention advice to older members of the public 44 that is theoretically informed, provides evidence-based information and that was developed in consultation with both users and health professionals. 45 Unfortunately, as this website is not written in English, we were unable to assess it. The quality of falls-prevention websites therefore needs to be evaluated across a number of languages to assess whether it is similarly poor in languages other than English. Currently, there is no accepted and validated way of measuring the quality of health-related websites 20 and so an evaluation sheet was developed specifically for this study, following techniques used in previous research.19, 21–23 The evaluation sheet has not been tested extensively and was, perhaps, not able to capture more subtle differences in the level of detail of information provided by the various websites.
Despite these limitations, this is the first study, to our knowledge, to have examined the quality of falls-related websites aimed at members of the public. Other reviews have examined websites in areas such as back pain, 19 breastfeeding, 23 fever in children, 22 treatment of depression 21 and across various health-related topics, 18 and have, on the whole, found the information provided to be poor and inconsistent.18, 20 There is clearly a need for better evidence-based information on the internet and for greater exploitation of the capacity of the internet to use interactive features and to personalise information. Standardised methods should also be developed for assessing the quality of health-related websites and for demonstrating to members of the public that the information provided is reliable, comprehensive and up to date.
Future research should evaluate falls-prevention websites on other quality indicators in addition to those included in this study. For example, websites have been evaluated on the basis of readability, relevance of content to terms entered into search engines and accuracy using an international expert panel. 18 Further research with older people should also investigate how they search for websites, appraise advice and its credibility, and which features of online content and presentation style motivate positive health behaviour change. This will enable websites to be designed in the most useful and appropriate manner for their intended target audiences. Finally, research should explore whether brief training with older people can enhance their ability to search for, and evaluate, online health advice.
Given that falls are associated with frailty and predicted by a range of risk factors, 46 enabling older people to address these risk factors and take up evidence-based preventive strategies will also enable them to prevent/reduce the symptoms of chronic illnesses and improve their general health and wellbeing.47, 48
Conclusions
Our review of websites offering falls-related advice to older members of the public found that although much of the information was presented in a user friendly manner, coverage of falls-related information was generally inadequate and lacking in credibility, and information was frequently out of date. Furthermore, none of the websites took advantage of the interactive nature of the web to offer self-assessment of an individual’s risk of falling or personally tailored advice. The danger is that older people and their carers are currently accessing and interpreting inaccurate information to the detriment of their own health.
Footnotes
Appendix 1.
Appendix 2
This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.