Surveys are a common method for assessing patient and clinician perceptions, attitudes and outcomes of telehealth. However, inadequacies in both the conduct and reporting of survey studies are common in telehealth research. This article provides clinicians and researchers with practical guidance on the appropriate selection, use and reporting of survey tools for telehealth research. We identify common survey outcomes and instruments used in telehealth research, and methods to assess the validity and psychometric properties of survey tools. Enhancing the quality and reporting of telehealth research is important to improve our understanding of which telehealth-supported models of care improve outcomes and for which patient groups.
WhittenPJohannessenLKSoerensenTet al.A systematic review of research methodology in telemedicine studies. J Telemed Telecare2007; 13: 230–235.
2.
MasinoCLamTC. Choice of rating scale labels: implication for minimizing patient satisfaction response ceiling effect in telemedicine surveys. Telemed J E Health2014; 20: 1150–1155.
3.
Allemann IseliMKunzRBlozikE. Instruments to assess patient satisfaction after teleconsultation and triage: A systematic review. Patient Prefer Adherence2014; 8: 893–907.
4.
McCollEJacobyAThomasLet al.Design and use of questionnaires: A review of best practice applicable to surveys of health service staff and patients. Health Technol Assess2002; 5: 1–256.
5.
DillmanDA. Mail and telephone surveys: the total design method, New York: Wiley, 1978.
6.
CafferyLJTaylorMLucasKet al.Substitution rates of video consultations for traditional consultations at a tertiary public hospital. J Telemed Telecare2016; 22: 453–458.
7.
YipMPChangAMChanJet al.Development of the Telemedicine Satisfaction Questionnaire to evaluate patient satisfaction with telemedicine: A preliminary study. J Telemed Telecare2003; 9: 46–50.
8.
BakkenSGrullon-FigueboaLIzquierdoRet al.Development, validation, and use of English and Spanish versions of the Telemedicine Satisfaction and Usefulness Questionnaire. J Am Med Inform Assn2006; 13: 660–667.
9.
MarshallGNHaysRD. The patient satisfaction questionnaire short-form (PSQ-18), Rand Santa Monica, CA, 1994.
10.
Carr-HillRA. The measurement of patient satisfaction. J Public Health Med1992; 14: 236–249.
11.
SitziaJWoodN. Patient satisfaction: A review of issues and concepts. Soc Sci Med1997; 45: 1829–1843.
12.
ManaryMPBouldingWStaelinRet al.The patient experience and health outcomes. New Engl J Med2013; 368: 201–203.
13.
WilliamsTLMayCREsmailA. Limitations of patient satisfaction studies in telehealthcare: A systematic review of the literature. Telemed J E Health2001; 7: 293–316.
14.
WhittenPLoveB. Patient and provider satisfaction with the use of telemedicine: Overview and rationale for cautious enthusiasm. J Postgrad Med2005; 51: 294–300.
15.
EpsteinRMFranksPFiscellaKet al.Measuring patient-centered communication in patient-physician consultations: Theoretical and practical issues. Soc Sci Med2005; 61: 1516–1528.
16.
BlackNVaraganumMHutchingsA. Relationship between patient reported experience (PREMs) and patient reported outcomes (PROMs) in elective surgery. BMJ Qual Saf2014; 23: 534–542.
17.
WadeVAEliottJAHillerJE. Clinician acceptance is the key factor for sustainable telehealth services. Qual Health Res2014; 24: 682–694.
18.
StahlJEDixonRF. Acceptability and willingness to pay for primary care videoconferencing: A randomized controlled trial. J Telemed Telecare2010; 16: 147–151.
19.
CafferyLJSmithAC. Investigating the quality of video consultations performed using fourth generation (4G) mobile telecommunications. J Telemed Telecare2015; 21: 348–354.
20.
TaylorAMorrisGPechJet al.Home telehealth video conferencing: Perceptions and performance. Jmir Mhealth Uhealth2015; 3.
21.
HuPJChauPYKShengORLet al.Examining the technology acceptance model using physician acceptance of telemedicine technology. J Manage Inform Syst1999; 16: 91–112.
22.
HoldenRJKarshBT. The technology acceptance model: Its past and its future in health care. J Biomed Inform2010; 43: 159–172.
23.
RussellTGGillespieNHartleyNet al.Exploring the predictors of home telehealth uptake by elderly Australian healthcare consumers. J Telemed Telecare2015; 21: 485–489.
24.
FatehiFGrayLCRussellAW. A clinimetric study of outpatient diabetes consultations: The potential for telemedicine substitution. Diabetes Technol Ther2014; 16: 8–14.
25.
LuxtonDDPruittLDOsenbachJE. Best practices for remote psychological assessment via telehealth technologies. Prof Psychol-Res Pr2014; 45: 27–35.
26.
LudzikJWitkowskiAMRoterman-KoniecznaIet al.Improving diagnostic accuracy of dermoscopically equivocal pink cutaneous lesions with reflectance confocal microscopy in telemedicine settings: Double reader concordance evaluation of 316 cases. Plos One2016; 11.
27.
SchillingLMKozakKLundahlKet al.Inaccessible novel questionnaires in published medical research: Hidden methods, hidden costs. Am J Epidemiol2006; 164: 1141–1144.
28.
BoyntonPMGreenhalghT. Selecting, designing, and developing your questionnaire. BMJ2004; 328: 1312–1315.
29.
HinkinTRSchriesheimCA. Development and application of new scales to measure the French and Raven (1959) bases of social power. J Appl Psychol1989; 74: 561–567.
30.
LandyFJ. Stamp collecting versus science – validation as hypothesis-testing. Am Psychol1986; 41: 1183–1192.
31.
LangbeckerDJandaM. Quality and readability of information materials for people with brain tumours and their families. J Cancer Educ2012; 27: 738–743.
32.
CalderonJLMoralesLSLiuHet al.Variation in the readability of items within surveys. Am J Med Qual2006; 21: 49–56.
33.
Microsoft. Test your document s readability. 2017.
34.
JamiesonS. Likert scales: How to (ab)use them. Med Educ2004; 38: 1217–1218.
35.
EdwardsMLSmithBC. The effects of the neutral response option on the extremeness of participant responses. Incite2011; 6: 177–192.
36.
van de MortelT. Faking it: Social desirability response bias in self-report research. Aust J Adv Nurs2008; 25: 40–48.
37.
KellermanSEHeroldJ. Physician response to surveys. Am J Prevent Med2001; 20: 61–67.
38.
GalesicMBosnjakM. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opin Quart2009; 73: 349–360.
39.
DillmanD. Mail and telephone surveys: The total design method, New York: John Wiley and Sons, 1978.
40.
NunnallyJC. Psychometric theory, 2nd ed. New York: McGraw-Hill, 1978. p.xv, 701 pp.
41.
CronbachLJMeehlPE. Construct validity in psychological tests. Psychol Bull1955; 52: 281–302.
42.
KelleyKClarkBBrownVet al.Good practice in the conduct and reporting of survey research. Int J Qual Health Care2003; 15: 261–266.
43.
EdwardsP. Questionnaires in clinical trials: Guidelines for optimal design and administration. Trials2010; 11: 2–2.
44.
ReaLMParkerRA. Designing and Conducting Survey Research: A Comprehensive Guide, 4th ed. San Fransisco: Jossey-Bass, 2014.
45.
BMJ Publishing Group. Statistics Notes. 2017.
46.
RobbKAGattingLWardleJ. What impact do questionnaire length and monetary incentives have on mailed health psychology survey response?Br J Health Psychol2017.
47.
MedwayRLTourangeauR. Response quality in telephone surveys. Do prepaid cash incentives make a difference?Public Opin Quart2015; 79: 524–543.
48.
LitmanLRobinsonJRosenzweigC. The relationship between motivation, monetary compensation, and data quality among US- and India-based workers on Mechanical Turk. Behav Res Methods2015; 47: 519–528.
49.
SingerECouperMP. Do incentives exert undue influence on survey participation? Experimental evidence. J Empir Res Hum Res Ethics2008; 3: 49–56.
50.
Marcano BelisarioJSJamsekJHuckvaleKet al.Comparison of self-administered survey questionnaire responses collected using mobile apps versus other methods. Cochrane Database Syst Rev2015, pp. MR000042–MR000042.
51.
HershWRHelfandMWallaceJet al.Clinical outcomes resulting from telemedicine interventions: A systematic review. BMC Med Inform Decision Making2001; 1: 5–5.
52.
RadhakrishnanKJacelonC. Impact of telehealth on patient self-management of heart failure: A review of literature. J Cardiovasc Nurs2012; 27: 33–43.
53.
ChiN-CDemirisG. A systematic review of telehealth tools and interventions to support family caregivers. J Telemed Telecare2015; 21: 37–44.
54.
CafferyLJBradfordNKWickramasingheSIet al.Outcomes of using telehealth for the provision of healthcare to Aboriginal and Torres Strait Islander people: A systematic review. Aust N Z J Public Health2017; 41: 48–53.
55.
BennettCKhanguraSBrehautJCet al.Reporting guidelines for survey research: An analysis of published guidance and reporting practices. PLoS Med2010; 8: e1001069–e1001069.
56.
BurnsKEDuffettMKhoMEet al.A guide for the design and conduct of self-administered surveys of clinicians. CMAJ2008; 179: 245–252.
57.
The EQUATOR Network and UK EQUATOR Centre for Statistics in Medicine NDoO, Rheumatology and Musculoskeletal Sciences (NDORMS), University of Oxford Botnar Research Centre. The EQUATOR Home page. Oxford, UK.
58.
GrigsbyJBregaAGDevorePA. The evaluation of telemedicine and health services research. Telemed J E Health2005; 11: 317–328.
59.
KahnJMHillNSLillyCMet al.The research agenda in ICU telemedicine: A statement from the Critical Care Societies Collaborative. Chest2011; 140: 230–238.
60.
ParmantoBNelson LewisAGrahamKMet al.Development of the Telehealth Usability Questionnaire (TUQ). Int J Telerehabil2016; 8: 3–10.
61.
LillyCMFisherKARiesMet al.A national ICU telemedicine survey: Validation and results. Chest2012; 142: 40–47.
62.
AghaZSchapiraRMLaudPWet al.Patient satisfaction with physician-patient communication during telemedicine. Telemed J E Health2009; 15: 830–839.
63.
DemirisGSpeedieSFinkelsteinS. A questionnaire for the assessment of patients' impressions of the risks and benefits of home telecare. J Telemed Telecare2000; 6: 278–284.
64.
HiraniSPRixonLBeynonMet al.Quantifying beliefs regarding telehealth: Development of the Whole Systems Demonstrator Service User Technology Acceptability Questionnaire. J Telemed Telecare2016; 23: 460–469.
65.
BowerPCartwrightMHiraniSPet al.A comprehensive evaluation of the impact of telemonitoring in patients with long-term conditions and social care needs: Protocol for the whole systems demonstrator cluster randomised trial. BMC Health Serv Res2011; 11: 184–184.
66.
ZhangY. Development and validation of an internet use attitude scale. Compute Educ2007; 49: 243–253.
67.
DavisFD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart1989; 13: 319–340.
68.
TeoTLeeCBChaiCSet al.Assessing the intention to use technology among pre-service teachers in Singapore and Malaysia: A multigroup invariance analysis of the Technology Acceptance Model (TAM). Compute Educ2009; 53: 1000–1009.
