A systematic review of articles using qualitative methods to generate questionnaire items identified in MEDLINE and PsycINFO from 2000 to 2014 was carried out. Articles were analyzed for (a) year of publication and journal domain, (b) qualitative data collection methods, (c) method of data content analysis, (d) professional experts’ input in item generation, and (e) debriefing of the newly developed items. In total, 371 articles were included and results showed (a) an acceleration of published articles, (b) individual interviews and focus groups were common ways of generating items and no emergent approach was identified, (c) the content analysis was usually not described (43% of articles), (d) experts were involved in eliciting concepts in less than a third of articles, (e) 61% of articles involved a step of further submission of newly developed items to the population of interest. This review showed an insufficient reporting of qualitative methods used to generate new questionnaires despite previous recommendations.
BeattyP. C.WillisG. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71, 287–311. doi:10.1093/poq/nfm006
2.
BoyerL.Baumstarck-BarrauK.CanoN.ZendjidjianX.BelzeauxR.LimousinS.. . . AuquierP. (2009). Assessment of psychiatric inpatient satisfaction: A systematic review of self-reported instruments. European Psychiatry, 24, 540–549. doi:10.1016/j.eurpsy.2009.05.011
3.
BrédartA.MarrelA.Abetz-WebbL.LaschK.AcquadroC. (2014). Interviewing to develop Patient-Reported Outcome (PRO) measures for clinical research: Eliciting patients’ experience. Health and Quality of Life Outcomes, 12, Article 15. doi:10.1186/1477-7525-12-15
4.
BrodM.HøjbjerreL.AdalsteinssonJ. E.RasmussenM. H. (2014). Assessing the impact of growth hormone deficiency and treatment in adults: Development of a new disease-specific measure. The Journal of Clinical Endocrinology & Metabolism, 99, 1204–1212. doi:10.1210/jc.2013-3438
5.
BrodM.TeslerL. E.ChristensenT. L. (2009). Qualitative research and content validity: Developing best practices based on science and experience. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation, 18, 1263–1278. doi:10.1007/s11136-009-9540-9
6.
CarterJ. D.CroweM.CarlyleD.FramptonC. M.JordanJ.McIntoshV. V. W.. . . JoyceP. R. (2012). Patient change processes in psychotherapy: Development of a new scale. Psychotherapy Research, 22, 115–126. doi:10.1080/10503307.2011.631195
7.
CopeD. G. (2014). Computer-assisted qualitative data analysis software. Oncology Nursing Forum, 41, 322–323. doi:10.1188/14.ONF.322-323
8.
DowardL. C.McKennaS. P.MeadsD. M.TwissJ.EckertB. J. (2009). The development of patient-reported outcome indices for multiple sclerosis (PRIMUS). Multiple Sclerosis Journal, 15, 1092–1102. doi:10.1177/1352458509106513
9.
HansonJ. L.BalmerD. F.GiardinoA. P. (2011). Qualitative research methods for medical educators. Academic Pediatrics, 11, 375–386. doi:10.1016/j.acap.2011.05.001
10.
JapT.TiatriS.JayaE. S.SutejaM. S. (2013). The development of Indonesian online game addiction questionnaire. PLoS ONE, 8(4), e61098. doi:10.1371/journal.pone.0061098
11.
KitzingerJ. (1995). Qualitative research: Introducing focus groups. British Medical Journal, 311(7000), 299–302. doi:10.1136/bmj.311.7000.299
12.
LaschK. E.MarquisP.VigneuxM.AbetzL.ArnouldB.BaylissM.. . . RosaK. (2010). PRO development: Rigorous qualitative research as the crucial foundation. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care & Rehabilitation, 19, 1087–1096. doi:10.1007/s11136-010-9677-6
13.
LiberatiA.AltmanD. G.TetzlaffJ.MulrowC.GøtzscheP. C.IoannidisJ. P. A.. . . MoherD. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: Explanation and elaboration. British Medical Journal, 339, Article b2700. doi:10.1136/bmj.b2700
14.
MacKenzieS. B.PodsakoffP. M.PodsakoffN. P. (2011). Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Quarterly, 35, 293–334. doi:10.2307/23044045
15.
MaysN.PopeC. (1995). Qualitative research: Rigour and qualitative research. British Medical Journal, 311(6997), 109–112. doi:10.1136/bmj.311.6997.109
16.
MaysN.PopeC. (2000). Qualitative research in health care. Assessing quality in qualitative research. British Medical Journal (Clinical Research ed.), 320(7226), 50–52. doi:10.1136/bmj.320.7226.50
17.
McKennaS. P.CookS. A.WhalleyD.DowardL. C.RichardsH. L.GriffithsC. E. M.van AsscheD. (2003). Development of the PSORIQoL, a psoriasis-specific measure of quality of life designed for use in clinical practice and trials. British Journal of Dermatology, 149, 323–331. doi:10.1046/j.1365-2133.2003.05492.x
18.
McKennaS. P.DowardL. C.WhalleyD.TennantA.EmeryP.VealeD. J. (2004). Development of the PsAQoL: A quality of life instrument specific to psoriatic arthritis. Annals of the Rheumatic Diseases, 63, 162–169. doi:10.1136/ard.2003.006296
19.
McKennaS. P.MeadsD. M.DowardL. C.TwissJ.PokrzywinskiR.RevickiD.. . . GlendenningG. A. (2011). Development and validation of the living with chronic obstructive pulmonary disease questionnaire. Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, 20, 1043–1052. doi:10.1007/s11136-011-9850-6
20.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Journal of Clinical Epidemiology, 62, 1006–1012. doi:10.1016/j.jclinepi.2009.06.005
21.
MorseJ. M. (1995). The significance of saturation. Qualitative Health Research, 5, 147–149. doi:10.1177/104973239500500201
22.
PatrickD. L.BurkeL. B.GwaltneyC. J.LeidyN. K.MartinM. L.MolsenE.RingL. (2011a). Content validity-establishing and reporting the evidence in newly developed Patient-Reported Outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 1-eliciting concepts for a new PRO instrument. Value in Health, 14, 967–977. doi:10.1016/j.jval.2011.06.014
23.
PatrickD. L.BurkeL. B.GwaltneyC. J.LeidyN. K.MartinM. L.MolsenE.RingL. (2011b). Content validity-establishing and reporting the evidence in newly developed Patient-Reported Outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: Part 2-assessing respondent understanding. Value in Health, 14, 978–988. doi:10.1016/j.jval.2011.06.013
24.
PolitD. F.BeckC. T.OwenS. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30, 459–467. doi:10.1002/nur.20199
25.
PopeC.ZieblandS.MaysN. (2000). Qualitative research in health care: Analysing qualitative data. British Medical Journal (Clinical Research ed.), 320(7227), 114–116. doi:10.1136/bmj.320.7227.114
26.
RatA.-C.PouchotJ.GuilleminF.BaumannM.Retel-RudeN.SpitzE.CosteJ. (2007). Content of quality-of-life instruments is affected by item-generation methods. International Journal for Quality in Health Care, 19, 390–398. doi:10.1093/intqhc/mzm040
27.
SerpellL.TeasdaleJ. D.TroopN. A.TreasureJ. (2004). The development of the P-CAN, a measure to operationalize the pros and cons of anorexia nervosa. The International Journal of Eating Disorders, 36, 416–433. doi:10.1002/eat.20040
28.
SkulmoskiG. J.HartmanF. T.KrahnJ. (2007). The Delphi method for graduate research. Journal of Information Technology Education: Research, 6, 1–21. doi:10.28945/199
29.
TacconelliE. (2010). Systematic reviews: CRD’s guidance for undertaking reviews in health care. The Lancet Infectious Diseases, 10, 226. doi:10.1016/S1473-3099(10)70065-7
30.
TolW. A.KomproeI. H.JordansM. J. D.SusantyD.de JongJ. T. V. M. (2011). Developing a function impairment measure for children affected by political violence: A mixed methods approach in Indonesia. International Journal for Quality in Health Care, 23, 375–383. doi:10.1093/intqhc/mzr032
31.
WuH.-S.McSweeneyM. (2004). Assessing fatigue in persons with cancer. Cancer, 101, 1685–1695. doi:10.1002/cncr.20540
32.
YuehB.McDowellJ. A.CollinsM.SouzaP. E.LoovisC. F.DeyoR. A. (2005). Development and validation of the effectiveness of [corrected] auditory rehabilitation scale. Archives of Otolaryngology–Head & Neck Surgery, 131, 851–856. doi:10.1001/archotol.131.10.851
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
