The Practicalities of SMS Research

Abstract

This paper describes a survey based on SMS messaging, presenting details of the tests and pilots undertaken, the practical difficulties found and overcome, as well as an examination of the differences found between the CATI and SMS elements of the survey. It also describes pilot surveys undertaken to test mixed-mode methods: SMS-to-online and SMS-to-IVR (interactive voice response). As the paper describes, Network Research has a long-standing relationship with a top service provider, which interacts with its customers through high-street outlets, by phone and online. For the past 15 years and more Network Research has conducted customer satisfaction surveys to cover all those interactions, mainly using computer-assisted telephone interviewing (CATI). Throughout the relationship with this client, Network Research has periodically investigated different means of data collection as an alternative or adjunct to CATI. Its primary motivation has been to seek more cost-efficient ways of collecting timely data (proximate to a specific event) from the customer base. Cost saving is not the only criterion, however. Consideration has also to be given to impact on the customer; attribution, confidentiality and data protection issues; potential biases leading to skewed and unrepresentative scores (non-response, age, gender, geographic); representing outlet and organisation structure; and scalability. Considerable time and effort has gone into refining the CATI approach to render it as cost- and methodologically efficient as possible. Nonetheless CATI is still, relative to other options – like online or other self-completion methods – a higher-cost approach. However, it is also the ‘gold standard’ on each of the non-cost criteria above. Pure online surveys have limited application for this client: among all but digital customers, very few customer email addresses are available. Where online research is conducted, the surveys generally suffer from low response rates. In 2015, Network Research started to supplement CATI data for the largest of the customer satisfaction surveys, among high-street customers, with a survey based on SMS messaging (or ‘text messaging’ via mobile phone). This paper will be of interest to anyone who is contemplating using SMS methodology.