Restricted accessResearch articleFirst published online 2022-10
The Voices of Stakeholders Involved in Precision Medicine: The Co-Design and Evaluation of Qualitative Indicators of Intervention Acceptability,Fidelity and Context in PRecISion Medicine for Children With Cancer in Australia
We report a novel approach of amalgamating implementation outcomes of acceptability and fidelity alongside context as a new way of qualitatively evaluating implementation outcomes and context of a precision medicine intervention. A rapid qualitative online proforma was co-designed with stakeholders and sent to a purposive sample of healthcare professionals involved in an early-phase clinical trial intervention. Data were analysed using Framework Analysis. A total of 24 out of 68 proformas were returned. Although some participants raised concerns about drug medication access issues, the main intervention was well accepted and understood across professional groups. Comprehension was enhanced through exposure to specialist multidisciplinary meeting arrangements. In conclusion, a rapid data collection tool and framework are now available to assess readily measurable, qualitative indicators of acceptability, fidelity of receipt and contextual fit within the dynamic precision medicine context.
AndersonN. C.KestenJ. M.AyresR.HickmanM.AmlôtR.MichieS.LorencattoF. (2021). Acceptability of, and barriers and facilitators to, a pilot physical health service for people who inject drugs: A qualitative study with service users and providers. International Journal of Drug Policy, 99(Suppl_5), 103437. https://doi.org/10.1016/j.drugpo.2021.103437
BellgA.BorrelliB.ResnickB.HechtJ.MinicucciD.OryM.OgedegbeG.OrwigD.ErnstD.CzajkowskiS. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychology: Official Journal of the Division of Health Psychology, American Psychological Association, 23(5), 443–451. https://doi.org/10.1037/0278-6133.23.5.443
5.
BestS.BrownH.LunkeS.PatelC.PinnerJ.BarnettC. P.WilsonM.SandaraduraS. A.McClarenB.BrettG. R.BraithwaiteJ.StarkZ. (2021). Learning from scaling up ultra-rapid genomic testing for critically ill children to a national level. NPJ Genomic Medicine, 6(1), 5. https://doi.org/10.1038/s41525-020-00168-3
6.
BestS.LongJ. C.BraithwaiteJ.TaylorN. (2022). Standardizing variation: Scaling up clinical genomics in Australia. Genetics in Medicine, S1098–S3600(22), 00019–00023. https://doi.org/10.1016/j.gim.2022.01.004
7.
BierbaumM.McMahonC. M.HughesS.BoisvertI.LauA. Y. S.BraithwaiteJ.RapportF. (2020). Barriers and facilitators to cochlear implant uptake in Australia and the United Kingdom. Ear and Hearing, 41(2), 374–385. https://doi.org/10.1097/AUD.0000000000000762
8.
BonevskiB.MaginP.HortonG.FosterM.GirgisA. (2011). Response rates in GP surveys: Trialling two recruitment strategies. Australian Family Physician, 40(6), 427–430.
9.
BorrelliB.SepinwallD.ErnstD.BellgA.CzajkowskiS.BregerR.DefrancescoC.LevesqueC.SharpD.OgedegbeG.ResnickB.OrwigD. (2005). A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. Journal of Consulting and Clinical Psychology, 73(5), 852–860. https://doi.org/10.1037/0022-006X.73.5.852
10.
BradleyF.WilesR.KinmonthA. L.MantD.GantleyM. (1999). Development and evaluation of complex interventions in health services research: Case study of the southampton heart integrated care project (SHIP). The SHIP collaborative group. BMJ (Clinical Research ed.), 318(7185), 711–715. https://doi.org/10.1136/bmj.318.7185.711
11.
BraithwaiteJ. (2018). Changing how we think about healthcare improvement. Bmj: British Medical Journal, 361. https://doi.org/10.1136/bmj.k2014.
12.
BraithwaiteJ.MarksD.TaylorN. (2014). Harnessing implementation science to improve care quality and patient safety: A systematic review of targeted literature. International Journal for Quality in Health Care, 26(3), 321–329. https://doi.org/10.1093/intqhc/mzu047
CarrollC.PattersonM.WoodS.BoothA.RickJ.BalainS. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(1), 40. https://doi.org/10.1186/1748-5908-2-40
16.
ChambersD. A. (2020). Considering the intersection between implementation science and COVID-19. Implementation Research and Practice, 1, 0020764020925994. https://doi.org/10.1177/0020764020925994.
17.
ChambersD. A.GlasgowR. E.StangeK. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science, 8(1), 117. https://doi.org/10.1186/1748-5908-8-117
CunninghamC. T.QuanH.HemmelgarnB.NoseworthyT.BeckC. A.DixonE.SamuelS.GhaliW. A.SykesL. L.JettéN. (2015). Exploring physician specialist response rates to web-based surveys. BMC Medical Research Methodology, 15(1), 32. https://doi.org/10.1186/s12874-015-0016-z
20.
CurranG. M. (2020). Implementation science made too simple: A teaching tool. Implementation Science Communications, 1(1), 27. https://doi.org/10.1186/s43058-020-00001-z
21.
DamschroderL. J.AronD. C.KeithR. E.KirshS. R.AlexanderJ. A.LoweryJ. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50
22.
DamschroderL. J.ReardonC. M.Opra WiderquistM. A.LoweryJ. (2022). Conceptualizing outcomes for use with the consolidated framework for implementation research (CFIR): The CFIR outcomes addendum. Implementation Science, 17(1), 1–10. https://doi.org/10.1186/s13012-021-01181-5
23.
EvansW. E.PuiC. H.YangJ. J. (2020). The promise and the reality of genomics to guide precision medicine in pediatric oncology: The decade ahead. Clinical Pharmacology and Therapeutics, 107(1), 176–180. https://doi.org/10.1002/cpt.1660
GinsburgL. R.HobenM.EasterbrookA.AndersonR. A.EstabrooksC. A.NortonP. G. (2021). Fidelity is not easy! Challenges and guidelines for assessing fidelity in complex interventions. [Trials Electronic Resource], 22(1), 372. https://doi.org/10.1186/s13063-021-05322-5
26.
GouldN. J.LorencattoF.StanworthS. J.MichieS.PriorM. E.GlidewellL.GrimshawJ. M.FrancisJ. J. (2014). Application of theory to enhance audit and feedback interventions to increase the uptake of evidence-based transfusion practice: An intervention development protocol. Implement Sci, 9(1), 92. https://doi.org/10.1186/s13012-014-0092-1
27.
GreenhalghT.PapoutsiC. (2018). Studying complexity in health services research: Desperately seeking an overdue paradigm shift. BMC Medicine, 16(1), 95. https://doi.org/10.1186/s12916-018-1089-4
28.
Holen-RabbersvikE.UllebustB.ReeE.SchibevaagL.Hurup-ThomsenL.StrømmeT.AaseK.AaseI.EllisL.WiigS. (2020). How to deal with context? Evaluation of the SAFE-LEAD context tool for quality and safety in nursing home and homecare services. Tidsskrift for Omsorgsforskning, 6(2), 87–102. https://doi.org/10.18261/issn.2387-5984-2020-02-07
LevatiS.CampbellP.FrostR.DougallN.WellsM.DonaldsonC.HagenS. (2016). Optimisation of complex health interventions prior to a randomised controlled trial: A scoping review of strategies used. Pilot and Feasibility Studies, 2(1), 17. https://doi.org/10.1186/s40814-016-0058-y
31.
LevyK. D.BlakeK.Fletcher-HoppeC.FranciosiJ.GotoD.HicksJ. K.HolmesA. M.KanuriS. H.MaddenE. B.MustyM. D.OrlandoL.PrattV. M.RamosM.WuR.GinsburgG. S. (2019). Opportunities to implement a sustainable genomic medicine program: Lessons learned from the IGNITE network. Genetics in Medicine, 21(3), 743–747. https://doi.org/10.1038/s41436-018-0080-y
32.
ManolioT. A.ChisholmR. L.OzenbergerB.RodenD. M.WilliamsM. S.WilsonR.BickD.BottingerE. P.BrilliantM. H.EngC.FrazerK. A.KorfB.LedbetterD. H.LupskiJ. R.MarshC.MrazekD.MurrayM. F.O’DonnellP. H.RaderD. J.… GinsburgG. S. (2013). Implementing genomic medicine in the clinic: The future is here. Genetics in Medicine, 15(4), 258–267. https://doi.org/10.1038/gim.2012.157
33.
ManolioT. A.RowleyR.WilliamsM. S.RodenD.GinsburgG. S.BultC.ChisholmR. L.DeverkaP. A.McLeodH. L.MensahG. A.RellingM. V.RodriguezL. L.TamburroC.GreenE. D. (2019). Opportunities, resources, and techniques for implementing genomics in clinical care. The Lancet, 394(10197), 511–520. https://doi.org/10.1016/s0140-6736(19)31140-7
34.
McDonaldK. M. (2013). Considering context in quality improvement interventions and implementation: Concepts, frameworks, and application. Academic Pediatrics, 13(6), S45–S53. https://doi.org/10.1016/j.acap.2013.04.013
35.
McGillB. C.WakefieldC. E.HetheringtonK.MunroL. J.WarbyM.LauL.TyrrellV.ZieglerD. S.O’BrienT. A.MarshallG. M.MalkinD.HansfordJ. R.TuckerK. M.VetschJ. (2020). “Balancing expectations with actual realities”: Conversations with clinicians and scientists in the first year of a high-risk childhood cancer precision medicine trial. Journal of Personalized Medicine, 10(1), 9. https://doi.org/10.3390/jpm10010009
36.
McGuierE. A.KolkoD. J.KlemM. L.FeldmanJ.KinklerG.DiabesM. A.WeingartL. R.WolkC. B. (2021). Team functioning and implementation of innovations in healthcare and human service settings: A systematic review protocol. Systematic Reviews, 10(1), 189. https://doi.org/10.1186/s13643-021-01747-w
37.
MichieS.JohnstonM. (2012). Theories and techniques of behaviour change: Developing a cumulative science of behaviour change. Health Psychology Review, 6(1), 1–6. https://doi.org/10.1080/17437199.2012.654964
38.
MilasanL. (2020). Recovery snapshots: A photographic exploration of Romanian mental health service users’ experience of recovery. Lancaster University.
39.
MorrowA.ChanP.TuckerK. M.TaylorN. (2021). The design, implementation, and effectiveness of intervention strategies aimed at improving genetic referral practices: A systematic review of the literature. Genetics in Medicine, 23(12), 2239–2249. https://doi.org/10.1038/s41436-021-01272-0
40.
NevedalA. L.ReardonC. M.Opra WiderquistM. A.JacksonG. L.CutronaS. L.WhiteB. S.DamschroderL. J. (2021). Rapid versus traditional qualitative analysis using the consolidated framework for implementation research (CFIR). Implementation Science, 16(1), 67. https://doi.org/10.1186/s13012-021-01111-5
41.
NunnsH. (2009). Responding to the demand for quicker evaluation findings. Social Policy Journal of New Zealand, 34(34), 89–99. ■■■
42.
PalsolaM.RenkoE.KostamoK.LorencattoF.HankonenN. (2020). Thematic analysis of acceptability and fidelity of engagement for behaviour change interventions: The Let’s Move it intervention interview study. British Journal of Health Psychology, 25(3), 772–789. https://doi.org/10.1111/bjhp.12433
43.
PriceA.SmithJ.DakinH.KangS.EibichP.CookJ.GrayA.HarrisK.MiddletonR.GibbonsE.BenedettoE.SmithS.DawsonJ.FitzpatrickR.SayersA.MillerL.MarquesE.Gooberman-HillR.BlomA.… BeardD. (2019). The arthroplasty candidacy help engine tool to select candidates for hip and knee replacement surgery. Health Technology Assessment, 623(32), 1–216. https://doi.org/10.3310/hta23320
44.
ProctorE.SilmereH.RaghavanR.HovmandP.AaronsG.BungerA.GriffeyR.HensleyM. (2011). Outcomes for implementation research: Conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7
45.
QSR (2019). NVivo qualitative data analysis version 12 Plus. QSR International Pty Ltd.
46.
RankinN. M.LaiM.MillerD.BealeP.SpigelmanA.PrestG.TurleyK.SimesJ. (2018). Cancer multidisciplinary team meetings in practice: Results from a multi‐institutional quantitative survey and implications for policy change. Asia‐Pacific Journal of Clinical Oncology, 14(1), 74–83. https://doi.org/10.1111/ajco.12765
47.
RapportF.Francis-AutonE.CartmillJ.RyderT.BraithwaiteJ.Clay-WilliamsR. (2020). A mobile methods pilot study of surgical spaces: ‘Fit for purpose? Organisational productivity and workforce wellbeing in workspaces in hospital’ (FLOURISH). BMC Health Services Research, 20(1), 78. https://doi.org/10.1186/s12913-020-4938-8
48.
RapportF.SmithJ.O’BrienT. A.TyrrellV. J.MouldE. V.LongJ. C.GulH.BraithwaiteJ. (2020). Development of an implementation and evaluation strategy for the Australian ‘Zero childhood cancer’ (Zero) program: A study protocol. BMJ Open, 10(6), e034522. https://doi.org/10.1136/bmjopen-2019-034522
49.
RiggsE.MuyeenS.BrownS.DawsonW.PetschelP.TardiffW.NormanF.VanpraagD.SzwarcJ.YellandJ. (2017). Cultural safety and belonging for refugee background women attending group pregnancy care: An Australian qualitative study. Birth, 44(2), 145–152. https://doi.org/10.1111/birt.12272
50.
RiggsE.YellandJ.MensahF. K.GoldL.SzwarcJ.KaplanI.SmallR.MiddletonP.KrastevA.McDonaldE.EastC.HomerC.NesvadbaN.BiggsL.BraithwaiteJ.BrownS. J. (2021). Group pregnancy care for refugee background women: A codesigned, multimethod evaluation protocol applying a community engagement framework and an interrupted time series design. BMJ Open, 11(7), e048271. https://doi.org/10.1136/bmjopen-2020-048271
51.
RitchieJ.SpencerL. (1994). Qualitative data analysis for applied policy research. In BrymanA.BurgessR. G. (Eds.), Analysing qualitative data (pp. 173–194). Routledge.
52.
RixonL.BaronJ.McGaleN.LorencattoF.FrancisJ.DaviesA. (2016). Methods used to address fidelity of receipt in health intervention research: A citation analysis and systematic review. BMC Health Services Research, 16(1), 663–663. https://doi.org/10.1186/s12913-016-1904-6
53.
RolfoC.MancaP.SalgadoR.Van DamP.DendoovenA.Ferri GandiaJ.RuttenA.LybaertW.VermeijJ.GevaertT.WeynC.LefebureA.MetsuS.Van LaereS.PeetersM.PauwelsP.Machado CoelhoA. (2018). Multidisciplinary molecular tumour board: A tool to improve clinical practice and selection accrual for clinical trials in patients with cancer. ESMO Open, 3(5), e000398. https://doi.org/10.1136/esmoopen-2018-000398
54.
RushforthA.GreenhalghT. (2020). Personalised medicine, disruptive innovation and ‘trailblazer’guidelines: Case study and theorization of an unsuccessful change effort. Accepted Milbank Quarterly, 98(2), 581–617. https://doi.org/10.1111/1468-0009.12455
55.
SekhonM.CartwrightM.FrancisJ. J. (2017). Acceptability of healthcare interventions: An overview of reviews and development of a theoretical framework. BMC Health Services Research, 17(1), 88. https://doi.org/10.1186/s12913-017-2031-8
56.
SharmaS.WaltonM.ManningS. (2021). Social determinants of health influencing the New Zealand COVID-19 response and recovery: A scoping review and causal loop diagram. Systems, 9(3), 52. https://doi.org/10.3390/systems9030052
57.
SmithJ. (2022). Sentiment analysis for use within rapid implementation research. How far and fast can we go? Methodology and methods of implementation science. In RapportF.Clay-WilliamsR.BraithwaiteJ. (Eds.), Implementation science (1st ed.). Taylor & Francis Group. https://doi.org/10.4324/9781003109945-34
58.
SmithJ.BraithwaiteJ.O’BrienT.SmithS.TyrrellV.MouldE.LongJ.RapportF. (2022). Re-imagining the data collection and analysis research process by proposing a rapid qualitative data collection and analytic roadmap applied to the dynamic context of precision medicine. International Journal of Qualitative Methods, 21, 160940692211030. https://doi.org/10.1177/16094069221103097.
59.
SmithJ.LeeM. D.EllisL. A.IjazK.YinK. (2021). Developing a novel psychographic-behavioral qualitative mapping method for exergames. International Journal of Serious Games, 8(2), 87–107. https://doi.org/10.17083/ijsg.v8i2.422
60.
SmithJ.RapportF.O’BrienT. A.SmithS.TyrrellV. J.MouldE. V. A.LongJ. C.GulH.CullisJ.BraithwaiteJ. (2020). The rise of rapid implementation: A worked example of solving an existing problem with a new method by combining concept analysis with a systematic integrative review. BMC Health Services Research, 20(1), 449. https://doi.org/10.1186/s12913-020-05289-0
61.
SmithS.AlvandA.LocockL.RyanS.SmithJ.BaylissL.WilsonH.PriceA. (2020). Partial or total knee replacement? Identifying patients’ information needs on knee replacement surgery: A qualitative study to inform a decision aid. Quality of Life Research, 29(4), 999–1011. https://doi.org/10.1007/s11136-019-02381-9
62.
Tran SmithB.PadgettD. K.Choy-BrownM.HenwoodB. F. (2015). Rebuilding lives and identities: The role of place in recovery among persons with complex needs. Health Place, 33, 109–117. https://doi.org/10.1016/j.healthplace.2015.03.002.
63.
Vindrola-PadrosC. (2021). Can we re-imagine research so it is timely, relevant and responsive? Comment on “experience of health leadership in partnering with university-based researchers in Canada: A call to ‘re-imagine’ research”. International Journal of Health Policy and Management, 10(3), 172–175. https://doi.org/10.34172/ijhpm.2020.43
64.
Vindrola-PadrosC.ChisnallG.CooperS.DowrickA.DjellouliN.SymmonsS. M.MartinS.SingletonG.VanderslottS.VeraN.JohnsonG. A. (2020). Carrying out rapid qualitative research during a pandemic: Emerging lessons from COVID-19. Qualitative Health Research, 30(14), 2192–2204. https://doi.org/10.1177/1049732320951526
65.
WaltonH.SpectorA.TomborI.MichieS. (2017). Measures of fidelity of delivery of, and engagement with, complex, face‐to‐face health behaviour change interventions: A systematic review of measure quality. British Journal of Health Psychology, 22(4), 872–903. https://doi.org/10.1111/bjhp.12260
66.
WandersmanA.ChienV. H.KatzJ. (2012). Toward an evidence-based system for innovation support for implementing innovations with quality: Tools, training, technical assistance, and quality assurance/quality improvement. American Journal of Community Psychology, 50(3–4), 445–459. https://doi.org/10.1007/s10464-012-9509-7
67.
WeinerB. J.LewisC. C.StanickC.PowellB. J.DorseyC. N.ClaryA. S.BoyntonM. H.HalkoH. (2017). Psychometric assessment of three newly developed implementation outcome measures. Implementation Science, 12(1), 108. https://doi.org/10.1186/s13012-017-0635-3
68.
WiigS.AaseK.JohannessenT.Holen-RabbersvikE.ThomsenL. H.van de BovenkampH.BalR.ReeE. (2019). How to deal with context? A context-mapping tool for quality and safety in nursing homes and homecare (SAFE-LEAD context). BMC Research Notes, 12(1), 259. https://doi.org/10.1186/s13104-019-4291-3
69.
WillemsenA. E.KrauszS.LigtenbergM. J.GrünbergK.GroenH. J.VoestE. E.CuppenE. P.van LaarhovenH. W.van HerpenC. M. (2019). Molecular tumour boards and molecular diagnostics for patients with cancer in The Netherlands: Experiences, challenges, and aspirations. British Journal of Cancer, 121(1), 34–36. https://doi.org/10.1038/s41416-019-0489-3
70.
WongM.MayohC.LauL. M. S.Khuong-QuangD.-A.PineseM.KumarA.BarahonaP.WilkieE. E.SullivanP.Bowen-JamesR.SyedM.MartincorenaI.AbascalF.SherstyukA.BolanosN. A.BaberJ.PriestleyP.DolmanM. E. M.FleurenE. D. G.… CowleyM. J. (2020). Whole genome, transcriptome and methylome profiling enhances actionable target discovery in high-risk pediatric cancer. Nature Medicine, 26(11), 1742–1753. https://doi.org/10.1038/s41591-020-1072-4
71.
ZebrowskiA. M.EllisD. E.BargF. K.SperberN. R.BernhardtB. A.DennyJ. C.DexterP. R.GinsburgG. S.HorowitzC. R.JohnsonJ. A.LevyM. A.OrlandoL. A.PollinT. I.SkaarT. C.KimmelS. E. (2019). Qualitative study of system-level factors related to genomic implementation. Genetics in Medicine, 21(7), 1534–1540. https://doi.org/10.1038/s41436-018-0378-9
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
