This project examined test-retest reliability and survey mode administration across single-item and multi-item measures among adolescent and young adult (AYA) cancer survivors.
Methods:
Forty-six AYAs randomly assigned to survey mode (phone, online, and paper) completed the survey and were invited to complete the survey again 1 week later.
Results:
Mode effects were found on 6% of single-items and 25% of multi-item scores. Reliability was low for 52% of single-items and 8% of multi-item scores.
Conclusion:
Multi-item measures should generally be used over single-item measures due to better reliability, but single-item measures may be preferable when mode effects are large.
de BoerAGEM, van LanschotJJB, StalmeierPFM, et al.Is a single-item visual analogue scale as valid, reliable and responsive as multi-item scales in measuring quality of life? Qual Life Res, 2004; 13(2):311–320.
2.
DiamantopoulosA, SarstedtM, FuchsC, et al.Guidelines for choosing between multi-item and single-item scales for construct measurement: A predictive validity perspective. J of the Acad Mark Sci, 2012; 40(3):434–449.
3.
SchnittkerJ, BacakV. The increasing predictive validity of self-rated health. PLoS One, 2014; 9(1):e84933.
4.
ReichA, ChatzigeorkidisE, ZeidlerC, et al.Tailoring the cut-off values of the visual analogue scale and numeric rating scale in itch assessment. Acta Derm Venereol, 2017; 97(6):759–760.
5.
Angulo-BrunetA, ViladrichC, PallarèsS, et al.Can multi-item measures and single-item measures be trusted to assess self-determination theory constructs in the elderly? Psicothema, 2020; 32(4):583–589.
6.
SarstedtM, WilczynskiP. More for less? A comparison of single-item and multi-item measures. Die Betriebswirtschaft, 2009; 69(2):211.
7.
CarrièreJS, Donayre PimentelS, Bou SabaS, et al.Recovery expectations can be assessed with single-item measures: Findings of a systematic review and meta-analysis on the role of recovery expectations on return-to-work outcomes after musculoskeletal pain conditions. Pain, 2023; 164(4):e190–e206.
8.
KwonH, TrailG. The feasibility of single-item measures in sport loyalty research. Sport Management Review, 2005; 8(1):69–89.
9.
LooR. A caveat on using single‐item versus multiple‐item scales. Journal of Managerial Psychology, 2002; 17(1):68–75.
10.
GardnerDG, DunhamRB, CummingsLL, PierceJL. Focus of attention at work: Construct definition and empirical validation. Journal of Occupational Psychology, 1989; 62(1):61–77.
11.
AaronsonN, Alonso J Fau- BurnamA, Burnam A Fau- LohrKN, et al.Assessing health status and quality-of-life instruments: Attributes and review criteria. Qual Life Res, 2002; 11(3):193–205.
12.
RobertsC. Mixing modes of data collection in surveys: A methodological review. NCRM Methods Review Papers Volume NCRM/008. 2007.
13.
Gelles-WatnickR. Americans’ Use of Mobile Technology and Home Broadband. Pew Research Center; 2024.
14.
AntounC, KatzJ, ArguetaJ, WangL. Design Heuristics for effective smartphone questionnaires. Social Science Computer Review, 2018; 36(5):557–574.
15.
SmithAW, KeeganT, HamiltonA, et al.Understanding care and outcomes in adolescents and young adult with Cancer: A review of the AYA HOPE study. Pediatr Blood Cancer, 2019; 66(1):e27486.
16.
JacobsenPB, RansomS. Implementation of NCCN distress management guidelines by member institutions. J Natl Compr Canc Netw, 2007; 5(1):99–103.
17.
ByromB, DollH, MuehlhausenW, et al.Measurement equivalence of patient-reported outcome measure response scale types collected using bring your own device compared to paper and a provisioned device: Results of a randomized equivalence trial. Value Health, 2018; 21(5):581–589.
18.
MauzE, HoffmannR, HoubenR, et al.Mode equivalence of health indicators between data collection modes and mixed-mode survey designs in population-based health interview surveys for children and adolescents: Methodological study. J Med Internet Res, 2018; 20(3):e64.
19.
NicholsHB, WernliKJ, ChawlaN, et al.Challenges and opportunities of epidemiological studies to reduce the burden of cancers in young adults. Curr Epidemiol Rep, 2023; 10(3):115–124.
20.
DillmanDA, PhelpsG, TortoraR, et al.Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc Sci Res, 2009; 38(1):1–18.
21.
JonesS, DuYX, PanattoniL, HenriksonN. Development of a questionnaire to assess worry about affording healthcare in an international sample. Qual Life Res, 2019; 28:S54. S.
22.
Agency for Health care Research and Quality. Medical Expenditure Panel Survey. Rockville, MD: Agency for Health care Research and Quality; 2014. Available from: http://meps.ahrq.gov/mepsweb/
23.
ReeveBB, HaysRD, BjornerJB, et al.; PROMIS Cooperative Group. Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care, 2007; 45(5 Suppl 1):S22–S31.
24.
CellaDF, TulskyDS, GrayG, et al.The Functional Assessment of Cancer Therapy scale: Development and validation of the general measure. J Clin Oncol, 1993; 11(3):570–579.
25.
ParkER, KirchhoffAC, NippRD, et al.Assessing health insurance coverage characteristics and impact on health care cost, worry, and access: A report from the childhood cancer survivor study. JAMA Intern Med, 2017; 177(12):1855–1858.
TourangeauR, SmithTW. Asking sensitive questions: The impact of data collection mode, question format, and question context. The Public Opinion Quarterly, 1996; 60(2):275–304.
30.
PresserS, StinsonL. Data collection mode and social desirability bias in self-reported religious attendance. Am Sociol Rev, 1998; 63(1):137–145.
31.
LyonsRA, WarehamK, Fau- LucasM, et al.SF-36 scores vary by method of administration: Implications for study design. J Public Health Med, 1999; 21(1):41–45.
32.
BuchananT. Chapter 5 - Potential of the internet for personality research. In: BirnbaumMH., editor. Psychological Experiments on the Internet. Academic Press: San Diego; 2000. pp. 121–140.
33.
BaronJ, SiepmannM. Chapter 10 - Techniques for creating and using web questionnaires in research and teaching. In: BirnbaumMH., editor. Psychological Experiments on the Internet. Academic Press: San Diego; 2000. pp. 235–265.
34.
ChoH, LaroseR. Privacy issues in internet surveys. Social Science Computer Review, 1999; 17(4):421–434.
35.
ChristmannA, Van AelstS. Robust estimation of Cronbach’s alpha. J Multivar Anal, 2006; 97(7):1660–1674.
36.
TangW, CuiY, BabenkoO., editors. Internal Consistency: Do We Really Know What It Is and How to Assess It? Journal of Psychology & Behavioral Science; 2013.
37.
HaysRD, SchaletBD, SpritzerKL, CellaD. Two-item PROMIS(R) global physical and mental health scales. J Patient Rep Outcomes, 2017; 1(1):2.
38.
BreenS, RitchieD, SchofieldP, et al.The Patient Remote Intervention and Symptom Management System (PRISMS)—a Telehealth- mediated intervention enabling real-time monitoring of chemotherapy side-effects in patients with haematological malignancies: Study protocol for a randomised controlled trial. Trials, 2015; 16:472.
39.
SmithGT, McCarthyDM, AndersonKG. On the sins of short-form development. Psychol Assess, 2000; 12(1):102–111.
40.
SteptoeA, GibsonEL, HamerM, WardleJ. Neuroendocrine and cardiovascular correlates of positive affect measured by ecological momentary assessment and by questionnaire. Psychoneuroendocrinology, 2007; 32(1):56–64.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
