Increasing response rates to lifestyle surveys: A pragmatic evidence review

Abstract

Aims: Lifestyle surveys are often a key component of a local Joint Strategic Needs Assessment (JSNA), undertaken to inform public health planning. They are usually administered to a large number of people in order to provide a comprehensive profile of population health. However, declining response rates coupled with the under-representation of certain population groups in lifestyle survey data has led to doubts concerning the reliability of findings. In order to inform the design of their own lifestyle survey, NHS Calderdale commissioned an evidence-based review of the methodological literature relating to the administration of lifestyle surveys, with the specific aim of identifying practical and resource-efficient strategies shown to be effective for maximizing whole-population response rates.

Methods: A pragmatic review of the published literature was undertaken, specifically to explore the most practical and resource-efficient ways to maximize lifestyle survey response rates to the most commonly used methods (postal surveys, face-to-face interviews, telephone interviews and electronic surveys). Electronic databases including MEDLINE, CINAHL, DARE, EMBASE and PsychINFO were searched. Empirical evidence published in the last 10 years was identified and citation tracking performed on all retrieved articles. An internet search for ‘grey literature’ was also conducted.

Results: The postal questionnaire remains an important lifestyle survey tool, but reported response rates have decreased rapidly in recent years. Interviews and telephone surveys are recommended in order to supplement data from postal questionnaires and increase response rates in some population groups, but costs may be prohibitive. Electronic surveys are a cheaper alternative, but the empirical evidence on effectiveness is inconclusive. Careful planning and tailoring of survey design to the characteristics of target populations can increase response rates and representativeness of lifestyle survey data.

Conclusions: The results of this pragmatic review could provide a valuable resource for those involved in the design and administration of lifestyle surveys.

Keywords

lifestyle survey mixed-methods literature review response rates

Get full access to this article

View all access options for this article.

References

Edwards P. , Roberts I. , Clarke M. , DiGuiseppi C. , Pratap S. , Wentz R. , Kwan I. Increasing response rates to postal questionnaires: Systematic review. British Medical Journal 2002; 324(7347): 1183

Hill A. , Roberts J. , Ewings P. , Gunnell D. Non-response bias in a lifestyle survey. Journal of Public Health Medicine 1997; 19(2): 203-207

Curtin R. , Presser S. , Singer E. Changes in telephone survey non-response over the past quarter century. Public Opinion Quarterly 2005; 69: 87-98

Biemer PP , Lyberg LE Introduction to Survey Quality. Hoboken, NJ: Wiley 2003

Groves RM , Presser S. , Dipko S. The role of topic interest in survey participation decisions. Public Opinion Quarterly 2004; 68(1): 2-31

Hunt S. , Bhopal R. Self-reports in research with non-English speakers. British Medical Journal 2003; 327(7411): 352-353

Free C. Breaking down language barriers. Some ethnic groups may have problems in getting as far as a consultation. British Medical Journal 1998; 317(7161): 816-817

Chaturvedi N. , McKeigue PM Methods for epidemiological surveys of ethnic minority groups. Journal of Epidemiology and Community Health 1994; 48: 107-111

Allison T. , Ahmad T. , Brammah T. , Symmons D. , Urwin M.

Can findings from postal questionnaires be combined with interview results to improve the response rate among ethnic minority populations?

Ethnicity & Health 2003; 8(1): 63-69

10.

Lloyd CE , Johnson MR , Mughal S. , Sturt JA , Collins GS , Roy T. , Bibi R. , Barnett AH Securing recruitment and obtaining informed consent in minority ethnic groups in the UK. BMC Health Services Research 2008 ; 8: 68. doi: 10.1186/1472-6963-8-68

11.

Kiezebrink K. , Crombie IK , Irvine L. , Swanson V. , Power K. , Wrieden WL , Slane PW Strategies for achieving a high response rate in a home interview survey. BMC Medical Research Methodology 2009; 9: 46

12.

Edwards PJ , Roberts I. , Clarke MJ , Diguiseppi C. , Wentz R. , Kwan I. , Cooper R. , Felix LM , Pratap S. Methods to increase response to postal and electronic questionnaires. Cochrane Database of Systematic Reviews 2009(3): MR000008

13.

Katja Lozar M. , Michael B. , Jernej B. , Iris H. , Vasja V. Web surveys versus other survey modes: A meta-analysis comparing response rates. International Journal of Market Research 2008; 50(1): 79-104

14.

Cook C. , Heath F. , Thompson RL A meta-analysis of response rates in web- or internet-based surveys. Educational and Psychological Measurement 2000; 60: 821-836

15.

Fricker S. , Galesic M. , Tourangeau R. , Yan T. An experimental comparison of web and telephone surveys. Public Opinion Quarterly 2005; 69: 370-392

16.

Kaplowitz MD , Hadlock TD , Levine R. A comparison of web and mail survey response rates. Public Opinion Quarterly 2004; 68(1): 94-101

17.

Kwak N. , Radler B. A comparison between mail and web surveys: Response pattern, respondent profile and data quality. Journal of Official Statistics 2002; 18: 257-273

18.

Dillman DA , Smyth JD , Christian LM Internet, mail and mixed-mode surveys: The tailored design method. Hoboken, NJ: Wiley, 2009

19.

Kroth PJ , McPherson L. , Leverence R. , Pace W. , Daniels E. , Rhyne RL , Williams RL , Prime Net Consortium. Combining web-based and mail surveys improves response rates: A PBRN study from PRIME Net. Annals of Family Medicine 2009; 7: 245-248

20.

Couper MP , Miller PV Web survey methods: Introduction. Public Opinion Quarterly 2008; 72: 831-835

21.

Sills SJ , Song C. Innovations in survey research: An application of web surveys. Social Science Computer Review 2002; 20: 22-30

22.

Collins GS , Johnson MR Addressing Ethnic Diversity in Health Outcome Measurement: A Systematic and Critical Review of the Literature 2004. Warwick: Warwick University, 2004

23.

Vettini A. , Bhopal R. , Hunt S. , Wiebe S. , Hanna L. , Amos A. Measurement of Risk Factors for Cancer in Ethnicity and Health Research: A Case Study of Tobacco and Alcohol. Report to the Scottish Cancer Group of the Scottish Executive. Edinburgh: Section of Public Health Sciences, Edinburgh University, 2002

24.

Fischbacher CM , Bhopal R. , Unwin N. , White M. , Alberti KG The performance of the Rose angina questionnaire in South Asian and European origin populations: A comparative study in Newcastle, UK. International Journal of Epidemiology 2001; 30: 1009-1016.

25.

Greenhalgh T. , Collard A. , Begum N. Sharing stories: Complex intervention for diabetes education in minority ethnic groups who do not speak English. British Medical Journal 2005; 330(7492): 628

26.

Roberts PJ , Roberts C. , Sibbald B. , Torgerson DJ The effect of a direct payment or a lottery on questionnaire response rates: A randomized controlled trial. Journal of Epidemiology and Community Health 2000; 54(1): 71-72

27.

Heerwegh D. , Loosveldt G. Face-to-face versus web surveying in a high-internet-coverage population: Differences in response quality. Public Opinion Quarterly 2008; 72: 836-846

28.

Malhotra N. Completion time and response order effects in web surveys. Public Opinion Quarterly 2008 ; 72: 914-934

29.

Eaker S. , Bergstrom R. , Bergstrom A. , Adami HO , Nyren O. Response rate to mailed epidemiologic questionnaires: A population-based randomized trial of variations in design and mailing routines. American Journal of Epidemiology 1998; 147(1): 74-82

30.

Scott P. , Edwards P. Personally addressed hand-signed letters increase questionnaire response: A meta-analysis of randomized controlled trials. BMC Health Services Research 2006; 6: 111

31.

McCluskey S. , Topping AE Increasing Response Rates to Lifestyle Surveys: A Review of Methodology and ‘Good Practice’. The National Joint Strategic Needs Assessment Dataset Project: University of Huddersfield, 2009