Despite the research benefits and cost savings pilot studies offer, such studies have been the exception rather than the rule in social science research. This article provides a primer and guide to pilot studies and posits the case that their systematic use is critical to rigorous social science research. Pilot study benefits include refinement of research questions, establishing goals, and providing a rehearsal study to (a) identify problems, (b) determine and implement potential solutions, and (c) improve the main study. Beyond their utility to improve scientific rigor, pilot studies have practical benefits, including cost savings and reductions in time and work required to complete a study. Human participants bring complexity, diversity, and environmental embeddedness to scientific research, qualities that create variability and sensitivity not fully recognized during the design stage. When confounding elements are not identified and managed at the outset, the integrity of the main study findings is at risk.
AaltioE. (2023). Evaluating the effectiveness of the systemic practice model of children’s social care–a pilot study on child-and family-level outcomes. Children and Youth Services Review, 144, Article 106732. https://doi.org/10.1016/j.childyouth.2022.106732
2.
AveryK. N. L.WilliamsonP. R.GambleC.O'Connell FrancischettoE.MetcalfeC.DavidsonP.WilliamsH.BlazebyJ. M. (2017). Informing efficient randomised controlled trials: Exploration of challenges in developing progression criteria for internal pilot studies. BMJ Open, 7(2), Article e013537. https://doi.org/10.1136/bmjopen-2016-013537
3.
BaileyL. (2018). The essentials of research design and methodology. Sage Publications.
4.
BalzaJ. S.CusatisR.McDonnellS. M.BasirM. A.FlynnK. E. (2022). Effective questionnaire design: How to use cognitive interviews to refine questionnaire items. Journal of Neonatal-Perinatal Medicine, 15(2), 345–349. https://doi.org/10.3233/NPM-210848
5.
BeattyP.WillisG. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287–311. https://doi.org/10.1093/poq/nfm006
6.
BeerO. W.McInroyL. B.HefnerJ.BeerS. E. (2024). Examining the feasibility and acceptability of an online mindfulness-based stress reduction program for frontline child protection social workers in England. Journal of Public Child Welfare, 19(1), 190–210. https://doi.org/10.1080/15548732.2024.2312838
7.
Bermejo-CajaC. J.KoatzD.OrregoC.Perestelo-PérezL.González-GonzálezA. I.BallesterM.Martínez-MarcosM.Pacheco-HuergoV.del Rey-GranadoY.Muñoz-BalsaM.Ramírez- PuertaA. B.Canellas-CriadoY.Pérez-RivasF. J.Toledo-ChávarriA.Martínez- MarcosM. (2019). Acceptability and feasibility of a virtual community of practice to primary care professionals regarding patient empowerment: A qualitative pilot study. BMC Health Services Research, 19(1), Article 403. https://doi.org/10.1186/s12913-019-4185-z
BillinghamS. A.WhiteheadA. L.JuliousS. A. (2013). An audit of sample sizes for pilot and feasibility trials being undertaken in the United Kingdom registered in the United Kingdom clinical research network database. BMC Medical Research Methodology, 13(1), Article 104. https://doi.org/10.1186/1471-2288-13-104
10.
BoatengG. O.NeilandsT. B.FrongilloE. A.Melgar-QuiñonezH. R.YoungS. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6(149), 1–18. https://doi.org/10.3389/fpubh.2018.00149
11.
BondC.LancasterG. A.CampbellM.ChanC.EddyS.HopewellS.MellorK.ThabaneL.EdridgeS. (2023). Pilot and feasibility studies: Extending the conceptual framework. Pilot and Feasibility Studies, 9(1), Article 24. https://doi.org/10.1186/s40814-023-01233-1
BowenD. J.KreuterM.SpringB.Cofta-WoerpelL.LinnanL.WeinerD.BakkenS.KaplanC. P.SquiersL.FabrizioC.FernandezM. (2009). How we design feasibility studies. American Journal of Preventive Medicine, 36(5), 452–457. https://doi.org/10.1016/j.amepre.2009.02.002
14.
BrowneR. H. (1995). On the use of a pilot sample for sample size determination. Statistics in Medicine, 14(17), 1933–1940. https://doi.org/10.1002/sim.4780141709
15.
BrownsonR. C.ColditzG. A.ProctorE. K. (Eds.). (2017). Dissemination and implementation research in health: Translating science to practice. Oxford University Press.
16.
CalderónJ. L.BakerR. S.FabregaH.CondeJ. G.HaysR. D.FlemingE.NorrisK. (2006). An ethno-medical perspective on research participation: A qualitative pilot study. Medscape General Medicine, 8(2), Article 23. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1785211/
CastañedaE.SmithC. (2022). Conducting research with marginalized populations: Methodological, ethical, and IRB considerations. Journal of Applied Social Science, 17(1), 111–131. https://doi.org/10.1177/19367244221141326
19.
Castillo-MontoyaM. (2016). Preparing for interview research: The interview protocol refinement framework. The Qualitative Report, 21(5), 811–831. https://doi.org/10.46743/2160-3715/2016.2337
20.
ChowP. I. (2020). Developing mental or behavioral health mobile apps for pilot studies by leveraging survey platforms: A do-it-yourself process. JMIR mHealth and UHealth, 8(4), Article e15561. https://doi.org/10.2196/15561
21.
DeVellisR. F. (2016). Scale development: Theory and applications (4th ed.). Sage publications.
DouglasB. D.EwellP. J.BrauerM. (2023). Data quality in online human-subjects research: Comparisons between MTurk, prolific, CloudResearch, qualtrics, and SONA. PLOS ONE, 18(3), Article e0279720. https://doi.org/10.1371/journal.pone.0279720
24.
GlasgowR. E.LichtensteinE.MarcusA. C. (1999). Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. American Journal of Public Health, 89(9), 1261–1267. https://doi.org/10.2105/AJPH.89.9.1322
25.
GrantS.Mayo-WilsonE.MontgomeryP.MacdonaldG.MichieS.HopewellS.MoherD., &, on behalf of the CONSORT-SPI Group. (2018). CONSORT-SPI 2018 explanation and elaboration: Guidance for reporting social and psychological intervention trials. Trials, 19(406), 1–18. https://doi.org/10.1186/s13063-018-2735-z
26.
HansonJ. D.LivengoodA.Kulbacki-FabisiakC.HardcastleK.BuchwaldD.McDonellM. (2023). Implementation of a pilot project program to expand research on alcohol use disorders in American Indian and Alaska native communities. The Journal of Research Administration, 54(3), Article 82.
27.
HarrisonS.HendersonJ.AlderdiceF.QuigleyM. A. (2019). Methods to increase response rates to a population-based maternity survey: A comparison of two pilot studies. BMC Medical Research Methodology, 19(1), Article 65. https://doi.org/10.1186/s12874-019-0702-3
28.
HerekG. M. (2010). Developing a theoretical framework and rationale for a research proposal. In PequegnatW.StoverE.BoyceC. (Eds.), How to write a successful research grant application (pp. 95–112). Springer. https://doi.org/10.1007/978-1-4419-1454-5_12
IsraelB. A.EngE.SchulzA. J.ParkerE. A. (2010). Methods for community-based participatory research for health (2nd ed.). Jossey-Bass.
31.
JamadinN.NoordinF. (2018). Pilot study: A lesson for future research. [Conference Presentation]. Proceedings of the International Conference on Business, Management, Accounting, Tourism & Hospitality (ICBMATH).
32.
JenkinsC.FaganH. B.PassarellaJ.FournakisN.BurshellD. (2020). Training academic and community investigator teams for community-engaged research: Program development, implementation, evaluation, and replication. Progress in Community Health Partnerships: Research, Education, and Action, 14(2), 229–242. https://doi.org/10.1353/cpr.2020.0019
33.
KaittilaA.IsoniemiH.ViitasaloK.MoisioM.RaijasA.ToikkaE.HakovirtaM. (2024). A pilot randomized controlled trial of intervention for social work clients with children facing complex financial problems in Finland: A study protocol. Journal of Evidence-Based Social Work, 21(1), 32–49. https://doi.org/10.1080/26408066.2023.2257174
34.
KallioH.PietiläA.-M.JohnsonM.KangasniemiM. (2016). Systematic methodological review: Developing a framework for a qualitative semi-structured interview guide. Journal of Advanced Nursing, 72(12), 2954–2965. https://doi.org/10.1111/jan.13031
35.
KazdinA. E. (2017). Research design in clinical psychology (5th ed.). Pearson.
36.
KimY. (2011). The pilot study in qualitative inquiry: Identifying issues and learning lessons for culturally competent research. Qualitative Social Work, 10(2), 190–206. https://doi.org/10.1177/1473325010362001
37.
KistinC.SilversteinM. (2015). Pilot studies: A critical but potentially misused component of interventional research. Journal of the American Medical Association, 314(15), 1561–1562. https://doi.org/10.1001/jama.2015.10962
38.
KraemerH. C. (2014). Evaluating medical tests: Objective and quantitative guidelines. Sage Publications.
LancasterG. A.DoddS.WilliamsonP. R. (2004). Design and analysis of pilot studies: Recommendations for good practice. Journal of Evaluation in Clinical Practice, 10(2), 307–312. https://doi.org/10.1111/j..2002.384.doc.x
41.
LeeE. K. O.BarrettC. (2007). Integrating spirituality, faith, and social justice in social work practice and education: A pilot study. Journal of Religion & Spirituality in Social Work: Social Thought, 26(2), 1–21. https://doi.org/10.1300/J377v26n02_01
42.
MajidM. A. A.OthmanM.MohamadS. F.LimS. A. H.YusofA. (2017). Piloting for interviews in qualitative research: Operationalization and lessons learnt. International Journal of Academic Research in Business and Social Sciences, 7(4), 1073–1080. https://doi.org/10.6007/IJARBSS/v7-i4/2916
43.
MalmqvistJ.HellbergK.MöllåsG.RoseR.ShevlinM. (2019). Conducting the pilot study: A neglected part of the research process? Methodological findings supporting the importance of piloting in qualitative research studies. International Journal of Qualitative Methods, 18(1), 1–11. https://doi.org/10.1177/1609406919878341
44.
MaxwellJ. A. (2013). Qualitative research design: An interactive approach (3rd ed.). Sage Publications.
45.
MellorK.AlburyC.DuttonS. J.EldridgeS.HopewellS. (2023). Recommendations for progression criteria during external randomised pilot trial design, conduct, analysis and reporting. Pilot and Feasibility Studies,9(1), 59. https://doi.org/10.1186/s40814-023-01291-5
46.
MinklerM.WallersteinN. (2011). Community-based participatory research for health: Advancing social and health equity (2nd ed.). Jossey-Bass.
47.
MrazekP. J.HaggertyR. J. (Eds.). (1994). Reducing risks for mental disorders: Frontiers for preventive intervention research. National Academies Press.
NestelD.GrayK.SimmonsM.PritchardS. A.IslamR.EngW. Q.DornanT. (2014). Community perceptions of a rural medical school: A pilot qualitative study. Advances in Medical Education and Practice, 7(5), 407–413. https://doi.org/10.2147/AMEP.S70876
50.
NiklesJ.MitchellG. K.SchluterP.GoodP.HardyJ.RowettD.Shelby-JamesT.VohraS.CurrowD. (2011). Aggregating single patient (n-of-1) trials in populations where recruitment and retention was difficult: The case of palliative care. Journal of Clinical Epidemiology, 64(5), 471–480. https://doi.org/10.1016/j.jclinepi.2010.05.009
51.
OnkenL. S.CarrollK. M.ShohamV.CuthbertB. N.RiddleM. (2014). Reenvisioning clinical science: Unifying the discipline to improve the public health. Clinical Psychological Science, 2(1), 22–34. https://doi.org/10.1177/2167702613497932
52.
OrsmondG. I.CohnE. S. (2015). The distinctive features of a feasibility study: Objectives and guiding questions. OTJR: Occupation, Participation and Health, 35(3), 169–177. https://doi.org/10.1177/1539449215578649
53.
PearsonN.NaylorP.-J.AsheM. C.FernandezM.YoongS. L.WolfendenL. (2020). Guidance for conducting feasibility and pilot studies for implementation trials. Pilot and Feasibility Studies, 6(1), Article 167. https://doi.org/10.1186/s40814-020-00634-w
54.
PetersonA. L.RoacheJ. D.RajJ.Young-McCaughanS. (2013). The need for expanded monitoring of adverse events in behavioral health clinical trials. Contemporary Clinical Trials, 34(1), 152–154. https://doi.org/10.1016/j.cct.2012.10.009
55.
PfleddererC. D.von KlinggraeffL.BurkartS.da Silva BandeiraA.LubansD. R.JagoR.OkelyA. D.van SluijsE. M. F.IoannidisJ. P. A.ThrasherJ. F.LiX.BeetsM. W. (2024). Consolidated guidance for behavioral intervention pilot and feasibility studies. Pilot and Feasibility Studies, 10(1), 57. https://doi.org/10.1186/s40814-024-01485-5
56.
ProctorE. K.LandsverkJ.AaronsG.ChambersD.GlissonC.MittmanB. (2011). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 24–34. https://doi.org/10.1007/s10488-008-0197-4
57.
SaddourI. (2020). Methodological considerations when piloting an interview protocol: The example of Syrian asylum seekers in France. Journal of French Language Studies, 30(2), 211–238. https://doi.org/10.1017/s0959269520000101
58.
SimJ. (2021). Distinctive aspects of consent in pilot and feasibility studies. Journal of Evaluation in Clinical Practice, 27(3), 657–664. https://doi.org/10.1111/jep.13556
59.
SimJ.LewisM. (2012). The size of a pilot study for a clinical trial should be calculated in relation to considerations of precision and efficiency. Journal of Clinical Epidemiology, 65(3), 301–308. https://doi.org/10.1016/j.jclinepi.2011.07.011
60.
SmithM. J.BornheimerL. A.LiJ.BlajeskiS.HiltzB.FischerD. J.CheckK.RuffoloM. (2021). Computerized clinical training simulations with virtual clients abusing alcohol: Initial feasibility, acceptability, and effectiveness. Clinical Social Work Journal, 49(2), 184–196. https://doi.org/10.1007/s10615-020-00779-4
SolomonP.HanrahanN. P.HurfordM.DeCesarisM.JoseyL. (2014). Lessons learned from implementing a pilot RCT of transitional care model for individuals with serious mental illness. Archives of Psychiatric Nursing, 28(4), 250–255. https://doi.org/10.1016/j.apnu.2014.03.005
63.
SternbergK. J.LambM. E.EsplinP. W.BaradaranL. P. (1999). Using a scripted protocol in investigative interviews: A pilot study. Applied Developmental Science, 3(2), 70–76. https://doi.org/10.1207/s1532480xads0302_1
64.
TeresiJ. A.YuX.StewartA. L.HaysR. D. (2022). Guidelines for designing and evaluating feasibility pilot studies. Medical Care, 60(1), 95–103. https://doi.org/10.1097/MLR.0000000000001664
65.
TerweeC. B.BotS. D. M.de BoerM. R.van der WindtD. A. W. M.KnolD. L.DekkerJ.de VetH. C. W.BouterL. M. (2007). Quality criteria were proposed for measurement properties of health status questionnaires. Journal of Clinical Epidemiology, 60(1), 34–42. https://doi.org/10.1016/j.jclinepi.2006.03.012
66.
ThabaneL.HopewellS.LancasterG. A.BondC. M.ColemanC. L.CampbellM. J.EldridgeS. M. (2016). Methods and processes for development of a CONSORT extension for reporting pilot randomized controlled trials. Pilot and Feasibility Studies, 2(1), Article 25. https://doi.org/10.1186/s40814-016-0065-z
67.
ThabaneL.MaJ.ChuR.ChengJ.IsmailaA.RiosL. P.Robson R. ThabaneM.GiangregorioL.GoldsmithC. H. (2010). A tutorial on pilot studies: The what, why, and how. BMC Medical Research Methodology, 10(1), Article 1. https://doi.org/10.1186/1471-2288-10-1
68.
ThyerB. A. (2023). Experimental research designs in social work: Theory and applications. Columbia University Press.
69.
TolW. A.AugustinaviciusJ.CarswellK.BrownF. L.AdakuA.LekuM. R.García-MorenoC.VentevogelP.WhiteR. G.van OmmerenM. (2018). Translation, adaptation, and pilot of a guided self-help intervention to reduce psychological distress in South Sudanese refugees in Uganda. Global Mental Health, 5, Article e25. https://doi.org/10.1017/gmh.2018.14
70.
TrickettE. J.BeehlerS.DeutschC.GreenL. W.HaweP.McleroyK.MillerR. L.RapkinB. D.SchensulJ. J.SchulzA. J.TrimbleJ. E. (2011). Advancing the science of community-level interventions. American Journal of Public Health, 101(8), 1410–1419. https://doi.org/10.2105/AJPH.2010.300113
van TeijlingenE. R.RennieA. M.HundleyV.GrahamW. (2001). The importance of conducting and reporting pilot studies: The example of the Scottish births survey. Journal of Advanced Nursing, 34(3), 289–295. https://doi.org/10.1046/j.1365-2648.2001.01757.x
73.
van WijkE.HarrisonT. (2013). Managing ethical problems in qualitative research involving vulnerable populations, using a pilot study. International Journal of Qualitative Methods, 12(1), 570–586. https://doi.org/10.1177/160940691301200130
74.
VogelS.Draper-RodiJ. (2017). The importance of pilot studies, how to write them and what they mean. International Journal of Osteopathic Medicine, 23, 2–3. https://doi.org/10.1016/j.ijosm.2017.02.001
75.
WillisG. B. (2004). Cognitive interviewing: A tool for improving questionnaire design. Sage Publications.
76.
WrayJ.ArchibongU.WaltonS. (2017). Why undertake a pilot in a qualitative PhD study? Lessons learned to promote success. Nurse Researcher, 24(3), 31–35. https://doi.org/10.7748/nr.2017.e1416
77.
YeongM. L.IsmailR.IsmailN. H.HamzahM. I. (2018). Interview protocol refinement: Fine-tuning qualitative research interview questions for multi-racial populations in Malaysia. The Qualitative Report, 23(11), 2700–2713. https://doi.org/10.46743/2160- 3715/2018.3412
