Advance care planning (ACP) can improve end-of-life care, yet practices in U.S. nursing homes (NHs)—where nearly 30% of older adults die—remain inconsistent. This study aimed to revise the Advance Care Planning Implementation Quality Assessment Tool (ACP-QAT) to measure NH-level ACP implementation quality through Delphi method. In Round 1, five focus groups provided feedback on each item, followed by iterative revisions and Delphi surveys. Panelists rated items as endorse, endorse with modification, or not endorse. Deductive content analysis summarized revisions, and consensus was defined as interquartile range = 0, median = 1, and ≥75% agreement. Eighteen interdisciplinary clinical experts (mean experience = 23 years; 83% female; 89% White) participated. Six main categories for revision were identified (e.g., defining ACP scope, specifying processes, clarifying team collaboration, specific training, and improving documentation) resulted in 10 revisions to items and all achieved consensus. ACP-QAT is the first tool to assess ACP implementation quality in NHs and may guide policy and practice.
AasmulI.HuseboB. S.SampsonE. L.FloE. (2018). Advance care planning in nursing homes – Improving the communication among patient, family, and staff: Results from a cluster randomized controlled trial (COSMOS). Frontiers in Psychology, 9, 2284. https://doi.org/10.3389/fpsyg.2018.02284
2.
CarpenterJ. G.BerryP. H.ErsekM. (2017). Nursing home care trajectories for older adults following in-hospital palliative care consultation. Geriatric Nursing, 38(6), 531–536. https://doi.org/10.1016/j.gerinurse.2017.03.016
3.
CDC. (2020). QuickStats: Percentage of deaths, by place of death — National vital statistics system, United States, 2000–2018. MMWR. Morbidity and Mortality Weekly Report, 69(19), 611. https://doi.org/10.15585/mmwr.mm6919a4
ChiarchiaroJ.BuddadhumarukP.ArnoldR. M.WhiteD. B. (2015). Prior advance care planning is associated with less decisional conflict among surrogates for critically ill patients. Annals of the American Thoracic Society, 12(10), 1528–1533. https://doi.org/10.1513/AnnalsATS.201504-253OC
DiamondI. R.GrantR. C.FeldmanB. M.PencharzP. B.LingS. C.MooreA. M.WalesP. W. (2014). Defining consensus: A systematic review recommends methodologic criteria for reporting of Delphi studies. Journal of Clinical Epidemiology, 67(4), 401–409. https://doi.org/10.1016/j.jclinepi.2013.12.002
8.
ElwynG.O’ConnorA.StaceyD.VolkR.EdwardsA.CoulterA.CollaborationI. P. D. A. S.BarryM.BernsteinS.ButowP.ClarkeA.EntwistleV.Feldman-StewartD.Holmes-RovnerM.Llewellyn-ThomasH.MoumjidN.MulleyA.RulandC.International Patient Decision Aids Standards IPDAS Collaboration (2006). Developing a quality criteria framework for patient decision aids: Online international Delphi consensus process. BMJ (Clinical Research), 333(7565), 417–419. https://doi.org/10.1136/bmj.38926.629329.AE
9.
EngelsY.CampbellS.DautzenbergM.van den HomberghP.BrinkmannH.SzécsényiJ.FalcoffH.SeuntjensL.KuenziB.GrolR.EPA Working Party (2005). Developing a framework of, and quality indicators for, general practice management in Europe. Family Practice, 22(2), 215–222. https://doi.org/10.1093/fampra/cmi002
10.
ErsekM.UnroeK. T.CarpenterJ. G.CagleJ. G.StephensC. E.StevensonD. G. (2022). High quality nursing home and palliative care – one and the same. Journal of the American Medical Directors Association, 23(2), 247–252. https://doi.org/10.1016/j.jamda.2021.11.027
11.
FloE.HuseboB. S.BruusgaardP.GjerbergE.ThoresenL.LillemoenL.PedersenR. (2016). A review of the implementation and research strategies of advance care planning in nursing homes. BMC Geriatrics, 16(1), 24. https://doi.org/10.1186/s12877-016-0179-4
12.
FriedT. R.ReddingC. A.RobbinsM. L.PaivaA.O’LearyJ. R.IannoneL. (2010). Stages of change for the component behaviors of advance care planning. Journal of the American Geriatrics Society, 58(12), 2329–2336. https://doi.org/10.1111/j.1532-5415.2010.03184.x
13.
FulmerT.EscobedoM.BermanA.KorenM. J.HernándezS.HultA. (2018). Physicians’ views on advance care planning and End-of-Life care conversations. Journal of the American Geriatrics Society, 66(6), 1201–1205. https://doi.org/10.1111/jgs.15374
14.
GelfmanL. P.BarnesD. E.GoldsteinN.VolowA. M.ShiY.LiB.SudoreR. L. (2023). Quality and satisfaction with advance care planning conversations among English- and Spanish-speaking older adults. Journal of Palliative Medicine, 26(10), 1380–1385. https://doi.org/10.1089/jpm.2022.0565
15.
HickmanS. E.MitchellS. L.HansonL. C.TuW.StumpT. E.UnroeK. T. (2022). The design and conduct of a pragmatic cluster randomized trial of an advance care planning program for nursing home residents with dementia. Clinical Trials, 19(6), 623–635. https://doi.org/10.1177/17407745221108992
16.
InoueM.IharaE.TerrillionA. (2019). Making your wishes known: Who completes an advance directive and shares it with their health care team or loved ones?Journal of Applied Gerontology: The Official Journal of the Southern Gerontological Society, 38(12), 1746–1762. https://doi.org/10.1177/0733464817748778
17.
JüngerS.PayneS. A.BrineJ.RadbruchL.BrearleyS. G. (2017). Guidance on conducting and Reporting Delphi studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine, 31(8), 684–706. https://doi.org/10.1177/0269216317690685
LakinJ. R.IsaacsE.SullivanE.HarrisH. A.McMahanR. D.SudoreR. L. (2016). Emergency physicians’ experience with advance care planning documentation in the electronic medical record: Useful, needed, and elusive. Journal of Palliative Medicine, 19(6), 632–638. https://doi.org/10.1089/jpm.2015.0486
20.
LamasD.PanarielloN.HenrichN.HammesB.HansonL. C.MeierD. E.GuinnN.CorriganJ.HubberS.Luetke-StahlmanH.BlockS. (2018). Advance care planning documentation in electronic health records: Current challenges and recommendations for change. Journal of Palliative Medicine, 21(4), 522–528. https://doi.org/10.1089/jpm.2017.0451
21.
MartinR. S.HayesB.GregorevicK.LimW. K. (2016). The effects of advance care planning interventions on nursing home residents: A systematic review. Journal of the American Medical Directors Association, 17(4), 284–293. https://doi.org/10.1016/j.jamda.2015.12.017
22.
McMahanR. D.TellezI.SudoreR. L. (2021). Deconstructing the complexities of advance care planning outcomes: What do We know and Where do We go? A scoping review. Journal of the American Geriatrics Society, 69(1), 234–244. https://doi.org/10.1111/jgs.16801
23.
MohanD.SacksO. A.O’MalleyJ.RudolphM.BynumJ.MurphyM.BarnatoA. E. (2021). A new standard for advance care planning (ACP) conversations in the hospital: Results from a Delphi panel. Journal of General Internal Medicine, 36(1), 69–76. https://doi.org/10.1007/s11606-020-06150-0
24.
MorV.VolandesA. E.GutmanR.GatsonisC.MitchellS. L. (2017). Pragmatic trial Of video education in nursing homes: The design and rationale for a pragmatic cluster randomized trial in the nursing home setting. Clinical Trials, 14(2), 140–151. https://doi.org/10.1177/1740774516685298
25.
MoriM.ChanH. Y. L.LinC.-P.KimS.-H.Ng Han LipR.MartinaD.YuenK. K.ChengS.-Y.TakenouchiS.SuhS.-Y.MenonS.KimJ.ChenP.-J.IwataF.TashiroS.KwokO. L. A.PengJ.-K.HuangH.-L.MoritaT.KizawaY.RietjensJ. A. C. (2025). Definition and recommendations of advance care planning: A Delphi study in five Asian sectors. Palliative Medicine, 39(1), 99–112. https://doi.org/10.1177/02692163241284088
26.
NasaP.JainR.JunejaD. (2021). Delphi methodology in healthcare research: How to decide its appropriateness. World Journal of Methodology, 11(4), 116–129. https://doi.org/10.5662/wjm.v11.i4.116
27.
PalmerJ. A.ParkerV. A.BarreL. R.MorV.VolandesA. E.BelangerE.LoomerL.McCreedyE.MitchellS. L. (2019). Understanding implementation fidelity in a pragmatic randomized clinical trial in the nursing home setting: A mixed-methods examination. Trials, 20(1), 656. https://doi.org/10.1186/s13063-019-3725-5
28.
PersaiD.PandaR.KumarR.Mc.EwenA. (2016). A Delphi study for setting up tobacco research and practice network in India. Tobacco Induced Diseases, 14, 4. https://doi.org/10.1186/s12971-016-0067-x
29.
PivodicL.SmetsT.Van den NoortgateN.Onwuteaka-PhilipsenB. D.EngelsY.SzczerbińskaK.Finne-SoveriH.FroggattK.GambassiG.DeliensL.Van den BlockL. (2018). Quality of dying and quality of End-of-Life care of nursing home residents in six countries: An epidemiological study. Palliative Medicine, 32(10), 1584–1595. https://doi.org/10.1177/0269216318800610
30.
Platts-MillsT. F.RichmondN. L.LeFebvreE. M.MangipudiS. A.HollowellA. G.TraversD.BieseK.HansonL. C.VolandesA. E. (2017). Availability of advance care planning documentation for older emergency department patients: A cross-sectional study. Journal of Palliative Medicine, 20(1), 74–78. https://doi.org/10.1089/jpm.2016.0243
31.
RayensM. K.HahnE. J. (2000). Building consensus using the policy Delphi method. Policy, Politics, & Nursing Practice, 1(4), 308–315. https://doi.org/10.1177/152715440000100409
32.
RietjensJ. A. C.SudoreR. L.ConnollyM.van DeldenJ. J.DrickamerM. A.DrogerM.van der HeideA.HeylandD. K.HouttekierD.JanssenD. J. A.OrsiL.PayneS.SeymourJ.JoxR. J.KorfageI. J.European Association for Palliative Care. (2017). Definition and recommendations for advance care planning: An international consensus supported by the European association for palliative care. The Lancet. Oncology, 18(9), e543–e551. https://doi.org/10.1016/S1470-2045(17)30582-X
33.
SingerM.DeutschmanC. S.SeymourC. W.Shankar-HariM.AnnaneD.BauerM.BellomoR.BernardG. R.ChicheJ.-D.CoopersmithC. M.HotchkissR. S.LevyM. M.MarshallJ. C.MartinG. S.OpalS. M.RubenfeldG. D.Van Der PollT.VincentJ.-L.AngusD. C. (2016). The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA, 315(8), 801–810. https://doi.org/10.1001/jama.2016.0287
34.
SongM.-K.HigginsM. K.WardS. E.LeeH.NooraniN.HappM. B. (2024). Measures of patient and surrogate preparedness for End-of-Life decision-making. Journal of Pain and Symptom Management, 67(5), 429–440.e2. https://doi.org/10.1016/j.jpainsymman.2024.02.005
35.
SprangerJ.HombergA.SonnbergerM.NiederbergerM. (2022). Reporting guidelines for Delphi techniques in health sciences: A methodological review. Zeitschrift fur Evidenz, Fortbildung und Qualitat im Gesundheitswesen, 172, 1–11. https://doi.org/10.1016/j.zefq.2022.04.025
StephensC. E.HuntL. J.BuiN.HalifaxE.RitchieC. S.LeeS. J. (2018). Palliative care eligibility, symptom burden, and Quality-of-Life ratings in nursing home residents. JAMA Internal Medicine, 178(1), 141–142. https://doi.org/10.1001/jamainternmed.2017.6299
38.
SudoreR. L.HeylandD. K.BarnesD. E.HowardM.FassbenderK.RobinsonC. A.BoscardinJ.YouJ. J. (2017). Measuring advance care planning: Optimizing the advance care planning engagement survey. Journal of Pain and Symptom Management, 53(4), 669–681. https://doi.org/10.1016/j.jpainsymman.2016.10.367
39.
SudoreR. L.HeylandD. K.LumH. D.RietjensJ. A. C.KorfageI. J.RitchieC. S.HansonL. C.MeierD. E.PantilatS. Z.LorenzK.HowardM.GreenM. J.SimonJ. E.FeuzM. A.YouJ. J. (2018). Outcomes that define successful advance care planning: A Delphi panel consensus. Journal of Pain and Symptom Management, 55(2), 245–255.e8. https://doi.org/10.1016/j.jpainsymman.2017.08.025
40.
SudoreR. L.LumH. D.YouJ. J.HansonL. C.MeierD. E.PantilatS. Z.MatlockD. D.RietjensJ. A. C.KorfageI. J.RitchieC. S.KutnerJ. S.TenoJ. M.ThomasJ.McMahanR. D.HeylandD. K. (2017). Defining advance care planning for adults: A consensus definition from a multidisciplinary Delphi panel. Journal of Pain and Symptom Management, 53(5), 821–832. https://doi.org/10.1016/j.jpainsymman.2016.12.331
41.
SudoreR. L.WallingA. M.GibbsL.RahimiM.WengerN. S.UC Health Care Planning Study Team. (2023). Implementation challenges for a multisite advance care planning pragmatic trial: Lessons learned. Journal of Pain and Symptom Management, 66(2), e265–e273. https://doi.org/10.1016/j.jpainsymman.2023.04.022
42.
TongA.SainsburyP.CraigJ. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042
43.
TowsleyG. L.HirschmanK. B.MaddenC. (2015). Conversations about End of Life: Perspectives of nursing home residents, family, and staff. Journal of Palliative Medicine, 18(5), 421–428. https://doi.org/10.1089/jpm.2014.0316
44.
UnroeK. T.LumH. D.HickmanS. E. (2024). Perspectives of nursing home staff in advance care planning conversations: Experiences from the approaches project. Journal of the American Medical Directors Association, 25(11), Article 105265. https://doi.org/10.1016/j.jamda.2024.105265
45.
Van Der SteenJ. T.Van Den BlockL.NakanishiM.Harrison DeningK.ParkerD.LarkinP.GiulioP. D.Der SchmittenJ.SudoreR. L.MimicaN.HolmerovaI.Martins PereiraS.KorfageI. J.European Association for Palliative Care EAPC (2025). Optimizing advance care planning in dementia: Recommendations from a 33-Country Delphi study. Journal of Pain and Symptom Management, 69(6), e755–e772. https://doi.org/10.1016/j.jpainsymman.2025.02.471
46.
van der SteenJ. T.van Soest-PoortvlietM. C.Hallie-HeiermanM.Onwuteaka-PhilipsenB. D.DeliensL.de BoerM. E.Van den BlockL.van UdenN.HertoghC. M. P. M.de VetH. C. W. (2014). Factors associated with initiation of advance care planning in dementia: A systematic review. Journal of Alzheimer’s Disease, 40(3), 743–757. https://doi.org/10.3233/JAD-131967
47.
van Soest-PoortvlietM. C.van der SteenJ. T.ZimmermanS.CohenL. W.ReedD.AchterbergW. P.RibbeM. W.de VetH. C. W. (2013). Selecting the best instruments to measure quality of End-of-Life care and quality of dying in long term care. Journal of the American Medical Directors Association, 14(3), 179–186. https://doi.org/10.1016/j.jamda.2012.09.019
48.
ZhangP.CagleJ. G. (2022). When living wills go missing: Associations with hospice use and hospital death using national longitudinal data. Journal of Applied Gerontology: The Official Journal of the Southern Gerontological Society, 42(5), 07334648221146774. https://doi.org/10.1177/07334648221146774
49.
ZhangP.SunF.HirschJ. (2023). Perceived barriers and social cultural factors associated with advance care planning conversations among Chinese American older adults. Journal of Applied Gerontology: The Official Journal of the Southern Gerontological Society, 42(10), 2110–2118. https://doi.org/10.1177/07334648231176142
50.
ZimmermanS.SloaneP. D.WretmanC. J.CaoK.SilbersackJ.CarderP.ThomasK. S.AllenJ.ButrumK.ChicotelT.GiorgioP.HernandezM.KalesH.KatzP.KlingerJ. H.KunzeM.LaxtonC.McNealleyV.MeeksS.Van HaitsmaK. (2022). Recommendations for medical and mental health care in assisted living based on an expert Delphi consensus panel: A consensus statement. JAMA Network Open, 5(9), Article e2233872. https://doi.org/10.1001/jamanetworkopen.2022.33872
51.
ZupancS. N.LakinJ. R.VolandesA. E.Paasche-OrlowM. K.MoseleyE. T.GundersenD. A.DasS.PenumarthyA.Martins-WelchD.BurnsE. A.CarneyM. T.IttyJ. E.EmmertK.TulskyJ. A.LindvallC. (2023). Forms or free-text? Measuring advance care planning activity using electronic health records. Journal of Pain and Symptom Management, 66(5), e615–e624. https://doi.org/10.1016/j.jpainsymman.2023.07.016
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
