Although online patient reviews (OPRs) are ubiquitous, there has been debate about whether OPRs are associated with measures of health care outcomes. The majority of patients search for online information about a hospital or provider before scheduling an encounter, therefore, the extent to which OPRs are valid information is important. We conducted a systematic review of the literature to examine the extent to which OPRs are associated with measures of patient experience, clinical quality, and other outcomes. We searched Medline, CINAHL, Web of Science, and PsycInfo, which yielded 32 studies. There were consistent positive correlations between OPRs and patient experience at both the organization and individual provider levels of analysis. However, associations between OPRs and quality measures were mixed. Organizational level OPRs may be more reliable than individual provider OPRs. In addition, the strength of association could be affected by the type of encounter setting, specialty, and specific measures.
*AdelhardtT.EmmertM.SanderU.WambachV.LindenthalJ. (2015). Can patients rely on results of physician rating websites when selecting a physician? A cross-sectional study assessing the association between online ratings and structural and quality of care measures from two German physician rating websites. Value in Health, 18(7), A335-A766. https://doi.org/10.1016/j.jval.2015.09.1732
Anhang-PriceR.ElliottM. N.ZaslavskyA. M.HaysR. D.LehrmanW. G.RybowskiL.Edgman-LevitanS.ClearyP. D. (2014). Examining the role of patient experience surveys in measuring health care quality. Medical Care Research and Review, 71(5), 522-554. https://doi.org/10.1177/1077558714541480
5.
BardachN. S. (2018a). Letter to the editor in response to “online physician ratings fail to predict actual performance on measures of quality, value, and peer review.”Journal of the American Medical Informatics Association, 25(6), 744-745. https://doi.org/10.1093/jamia/ocx143
*BardachN. S.Asteria-PeñalozaR.BoscardinW. J.DudleyR. A. (2012). The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Quality & Safety, 22(3), 194-202. http://dx.doi.org/10.1136/bmjqs-2012-001360
BurkleC. M.KeeganM. T. (2015). Popularity of internet physician rating sites and their apparent influence on patients’ choices of physicians. BMC Health Services Research, 15, Article 416. https://doi.org/10.1186/s12913-015-1099-2
11.
*CampbellL.LiY. (2018). Are Facebook user ratings associated with hospital cost, quality and patient satisfaction? A cross-sectional analysis of hospitals in New York State. BMJ Quality Safety, 27(2), 119-129. https://doi.org/10.1136/bmjqs-2016-006291
12.
*ChenJ.PressonA.ZhangC.RayD.FinlaysonS.GlasgowR. (2018). Online physician review websites poorly correlate to a validated metric of patient satisfaction. Journal of Surgical Research, 227, 1-6. https://doi.org/10.1016/j.jss.2018.01.037
CropanzanoR.MitchellM. (2005). Social exchange theory: An interdisciplinary review. Journal of Management, 31(6), 874-900. https://doi.org/10.1177/0149206305279602
15.
*DaskivichT. J.HoumanJ.FullerG.BlackJ. T.KimH. L.SpiegelB. (2018). Online physician ratings fail to predict actual performance on measures of quality, value, and peer review. Journal of the American Medical Informatics Association, 25(4), 401-407. https://doi.org/10.1093/jamia/ocx083
16.
DaskivichT. J.SpiegelB.KimH. L. (2017). Online ratings systems for physicians and institutions: Limitations of the current state of the art. European Urology, 71(3), 311-312. https://doi.org/10.1016/j.eururo.2016.08.025
17.
EllimoottilC.LeichtleS. W.WrightC. J.FakhroA.ArringtonA. K.ChirichellaT. J.WardW. H. (2013). Online physician reviews: The good, the bad and the ugly. Bulletin of the American College of Surgeons, 98(9), 34-39. https://bulletin.facs.org/2013/09/online-physician-reviews/
18.
ElvinsR.GreenJ. (2008). The conceptualization and measurement of therapeutic alliance: An empirical review. Clinical Psychology Review, 28(7), 1167-1187. https://doi.org/10.1016/j.cpr.2008.04.002
19.
*EmmertM.AdelhardtT.SanderU.WambachV.LindenthalJ. (2015). A cross-sectional study assessing the association between online ratings and structural and quality of care measures: Results from two German physician rating websites. BMC Health Services Research, 15, Article 414. https://doi.org/10.1186/s12913-015-1051-5
20.
*EmmertM.MeszmerN.SanderU. (2016). Do health care providers use online patient ratings to improve the quality of care? Results from an online-based cross-sectional study. Journal of Medical Internet Research, 18(9), Article e254. https://doi.org/10.2196/jmir.5889
21.
*EmmertM.MeszmerN.SchlesingerM. (2018). A cross-sectional study assessing the association between online ratings and clinical quality of care measures for US hospitals: Results from an observational study. BMC Health Services Research, 18(1), Article 82. https://doi.org/10.1186/s12913-018-2886-3
22.
EmmertM.SchlesingerM. (2017). Hospital quality reporting in the United States: Does report card design and incorporation of patient narrative comments affect hospital choice?Health Services Research, 52(3), 933-958. https://doi.org/10.1111/1475-6773.12519
*Gaudet-HefeleJ.LiY.CampbellL.BarooahA.WangJ. (2018). Nursing home Facebook reviews: Who has them, and how do they relate to other measures of quality and experience?BMJ Quality & Safety, 27(2), 130-139. https://doi.org/10.1136/bmjqs-2017-006492
25.
GiordanoL. A.ElliottM. N.GoldsteinE.LehrmanW. G.SpencerP. A. (2010). Development, implementation, and public reporting of the HCAHPS survey. Medical Care Research and Review, 67(1), 27-37. https://doi.org/10.1177/1077558709341065
26.
Grabner-KräuterS.WaigunyM. K. (2015). Insights into the impact of online physician reviews on patients’ decision making: Randomized experiment. Journal of Medical Internet Research, 17(4), Article e93. https://doi.org/10.2196/jmir.3991
27.
*GreavesF.PapeU. J.KingD.DarziA.MajeedA.WachterR. M.MillettC. (2012). Associations between internet-based patient ratings and conventional surveys of patient experience in the English NHS: An observational study. BMJ Quality & Safety, 21(7), 600-605. https://doi.org/10.1136/bmjqs-2012-000906
28.
*GreavesF.PapeU. J.LeeH.SmithD. M.DarziA.MajeedA.MillettC. (2012). Patients’ ratings of family physician practices on the internet: Usage and associations with conventional measures of quality in the English National Health Service. Journal of Medical Internet Research, 14(5), Article e146. https://doi.org/10.2196/jmir.2280
29.
*GreavesF.Ramirez-CanoD.MillettC.DarziA.DonaldsonL. (2013). Use of sentiment analysis for capturing patient experience from free-text comments posted online. Journal of Medical Internet Research, 15(11), Article e239. https://doi.org/10.2196/jmir.2721
30.
*GriffithsA.LeaverM. P. (2018). Wisdom of patients: Predicting the quality of care using aggregated patient feedback. BMJ Quality & Safety, 27(2), 110-118. https://doi.org/10.1136/bmjqs-2017-006847
31.
HanX.YaraghiN.GopalR. (2018). Winning at all costs: Analysis of inflation in nursing homes’ rating system. Production and Operations Management, 27(2), 215-233. https://doi.org/10.1111/poms.12804
32.
HanX.YaraghiN.GopalR. (2019). Catching them red-handed: Optimizing the nursing homes’ rating system. ACM Transactions on Management Information Systems, 10(2), 1-26. https://doi.org/10.1145/3325522
33.
HanauerD. A.ZhengK.SingerD. C.GebremariamA.DavisM. M. (2014). Public awareness, perception, and use of online physician rating sites. Journal of the American Medical Association, 311(7), 734-735. https://doi.org/10.1001/jama.2013.283194
34.
*HaskinsI. N.KrpataD. M.RosenM. J.PerezA. J.TastaldiL.ButlerR. S.RosenblattS.PrabhuA.S. (2017). Online surgeon ratings and outcomes in hernia surgery: An Americas hernia society quality collaborative analysis. Journal of American College of Surgeons, 225(5), 582-589. https://doi.org/10.1016/j.jamcollsurg.2017.08.007
35.
*HawkinsJ. B.BrownsteinJ. S.TuliG.RunelsT.BroeckerK.NsoesieE. O.McIverD. J.RozenblumR.WrightA.BourgeoisF. T.GreavesF. (2016). Measuring patient-perceived quality of care in US hospitals using Twitter. BMJ Quality & Safety, 25(6), 404-413. https://doi.org/10.1136/bmjqs-2015-004309
HollidayA. M.KachaliaA.MeyerG. S.SequistT. D. (2017). Physician and patient views on public physician rating websites: A cross-sectional study. Journal of General Internal Medicine, 32(6), 626-631. https://doi.org/10.1007/s11606-017-3982-5
38.
HongY.LiangC.RadcliffT.WigfallL.StreetR. (2019). What do patients say about doctors online? A systematic review of studies on patient online reviews. Journal of Medical Internet Research, 21(4), Article e12521. https://doi.org/10.2196/12521
39.
*HuppertzJ. W.OttoP. (2018). Predicting HCAHPS scores from hospitals’ social media pages: A sentiment analysis. Health Care Management Review, 43(4), 359-367. https://doi.org/10.1097/HMR.0000000000000154
40.
*JohariK.KelloggC.VazquezK.IrvineK.RahmanA.EnguidanosS. (2018). Ratings game: an analysis of nursing home compare and Yelp ratings. BMJ Quality & Safety, 27(8), 619-624. https://doi.org/10.1136/bmjqs-2017-007301
41.
*KoolR. B.KleefstraS. M.BorghansI.AtsmaF.Van de BeltT. H. (2016). Influence of intensified supervision by health care inspectorates on online patient ratings of hospitals: A multilevel study of more than 43,000 online ratings. Journal of Medical Internet Research, 18(7), e198. https://doi.org/10.2196/jmir.5884
42.
LaguT.MetayerK.MoranM.OrtizL.PriyaA.GoffS. L.LindenauerP. K. (2017). Website characteristics and physician reviews on commercial physician-rating websites. Journal of the American Medical Association, 317(7), 766-768. https://doi.org/10.1001/jama.2016.18553
LeeV. S. (2017a). Transparency and trust: Online patient reviews of physicians. New England Journal of Medicine, 376(3), 197-199. https://doi.org/10.1056/NEJMp1610136
*LiY.CaiX.WangM. (2019). Social media ratings of nursing homes associated with experience of care and “Nursing Home Compare” quality measures. BMC Health Services Research, 19(1), Article 260. https://doi.org/10.1186/s12913-019-4100-7
47.
*LiuJ. J.MatelskiJ.CramP.UrbachD. R.BellC. M. (2016). Association between online physician ratings and cardiac surgery mortality. Circulation: Cardiovascular Quality and Outcomes, 9(6), 788-791. https://doi.org/10.1161/CIRCOUTCOMES.116.003016
48.
*LuS. F.RuiH. (2018). Can we trust online physician ratings? Evidence from cardiac surgeons in Florida. Management Science, 64(6), 2557-2573. https://doi.org/10.1287/mnsc.2017.2741
*McGrathR. J.PriestlyJ.ZhouY.CulliganP. J. (2018). The validity of online patient ratings of physicians: Analysis of physician peer reviews and patient ratings. Interactive Journal of Medical Research, 7(1), e8. https://doi.org/10.2196/ijmr.9350
51.
MohanC. S.RotterJ. S.TanH. J.KentE.BjurlinM. A.BaschE.SamuelC.NielsenM.SmithA. B. (2020). The association between patient experience and healthcare outcomes using SEER-CAHPS patient experience and outcomes among cancer survivors. Journal of Geriatric Oncology. Advance online publication. https://doi.org/10.1016/j.jgo.2020.11.006
52.
MoherD.LiberatiA.TetzlaffJ.AltmanD. G., & The PRISMA Group. (2009). Preferred Reporting Items for Systematic Reviews and Meta-analyses: The PRISMA statement. PLOS MEDICINE, 6(7), e1000097. https://doi.org/10.1371/journal.pmed1000097
53.
*MurphyG. P.AwadM. A.OsterbergE. C.GaitherT. W.ChumnarnsongkhrohT.WashingtonS. L.BreyerB. N. (2017). Web-based physician ratings for California physicians on probation. Journal of Medical Internet Research, 19(8), Article e254. https://doi.org/10.2196/jmir.7488
54.
MurphyG. P.RadadiaK. D.BreyerB. N. (2019). Online physician reviews: Is there a place for them?Risk Management and Healthcare Policy, 12, 85-89. https://doi.org/10.2147/RMHP.S170381
55.
*OkikeK.Peter-BibbT. K.XieK. C.OkikeO. N. (2016). Association between physician online rating and quality of care. Journal of Medical Internet Research, 18(12), Article e324. https://doi.org/10.2196/jmir.6612
56.
*OkikeK.UhrN. R.ShinS.XieK. C.KimC. Y.FunahashiT. T.KanterM. H. (2019). A comparison of online physician ratings and internal patient-submitted ratings from a large healthcare system. Journal of General Internal Medicine, 34(11), 2575-2579. https://doi.org/10.1007/s11606-019-05265-3
57.
*RanardB. L.WernerR. M.AntanaviciusT.SchwartzA. H.SmithR. J.MeiselZ. F.AschD. A.UngarL. H.MerchantR. M. (2016). What can Yelp teach us about measuring hospital quality?Health Affairs (Millwood), 35(4), 697-705. https://doi.org/10.1377/hlthaff.2015.1030
58.
*RotmanL. E.AlfordE. N.ShankC. D.DalgoC.StetlerW. R. (2019). Is there an association between physician review websites and press ganey survey results in a neurosurgical outpatient clinic?World Neurosurgery, 132(December), e891-e899. https://doi.org/10.1016/j.wneu.2019.07.193
59.
*SaifeeD. H.ZhengZ. (Eric)BardhanI. R.LahiriA. (2020). Are online reviews of physicians reliable indicators of clinical outcomes? A focus on chronic disease management. Information Systems Research, 31(4), 1037-1492. https://doi.org/10.1287/isre.2020.0945
60.
SchlesingerM.GrobR.ShallerD.MartinoS. C.ParkerA. M.FinucaneM. L.CerullyJ. L.RybowskiL. (2015). Taking patients’ narratives about clinicians from anecdote to science. New England Journal of Medicine, 373(7), 675-679. https://doi.org/10.1056/NEJMsb1502361
61.
SchmockerR. K.StaffordL. M. C.WinslowE. R. (2019). Satisfaction with surgeon care as measured by the Surgery-CAHPS survey is not related to NSQIP outcomes. Surgery, 165(3), 510-515. https://doi.org/10.1016/j.surg.2018.08.028
62.
SchwarzeM. L.BradleyC. T.BraselK. J. (2010). Surgical “buy-in”: The contractual relationship between surgeons and patients that influences decisions regarding life-supporting therapy. Critical Care Medicine, 38(3), 843-848. https://doi.org/10.1097/CCM.0b013e3181cc466b
63.
*SegalJ.SacopulosM.SheetsV.ThurstonI.BrooksK.PucciaR. (2012). Online doctor reviews: Do they track surgeon volume, a proxy for quality of care?Journal of Medical Internet Research, 14(2), Article e50. https://doi.org/10.2196/jmir.2005
64.
ShahA. M.YanX.ShahS. A. A.ShahS. J.MamirkulovaG. (2019). Exploring the impact of online information signals in leveraging the economic returns of physicians. Journal of Biomedical Informatics, 98(October), 103272. https://doi.org/10.1016/j.jbi.2019.103272
65.
StilesR. A.MickS. S.WiseC. G. (2001). The logic of transaction cost economics in health care organization theory. Health Care Management Review, 26(2), 85-92. https://doi.org/10.1097/00004010-200104000-00010
66.
TerlutterR.BidmonS.RöttlJ. (2014). Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. Journal of Medical Internet Research, 16(3), Article e97. https://doi.org/10.2196/jmir.3145
67.
*TrehanS. K.NguyenJ. T.MarxR.CrossM. B.TingJ.PanT. J.DaluiskiA.LymanS. (2018). Online patient ratings are not correlated with total knee replacement surgeon-specific outcomesHSS Journal: The Musculoskeletal Journal of Hospital for Special Surgery, 14(2), 177-180. https://doi.org/10.1007/s11420-017-9600-6
68.
VerhoefL. M.Van de BeltT. H.EngelenL. J.SchoonhovenL.KoolR. B. (2014). Social media and rating sites as tools to understanding quality of care: A scoping review. Journal of Medical Internet Research, 16(2), Article e56. https://doi.org/10.2196/jmir.3024
69.
*YangA.FinkelsteinM.PaleseM. (2019). Online patient ratings of urologists are correlated with short-term outcomes following major urological surgeries. Journal of Urology, 201(Suppl. 4), e303-e304. https://doi.org/10.1097/01.JU.0000555557.53309.fa
70.
YaraghiN.WangW.GaoG.AgarwalR. (2018). How online quality ratings influence patients’ choice of medical providers: Controlled experimental survey study. Journal of Medical Internet Research, 20(3), Article e99. https://doi.org/10.2196/jmir.8986
71.
*WidmerR. J.MaurerM. J.NayarV. R.AaseL. A.WaldJ. T.KotsenasA. L.TimmiF. K.HarperC. M.PruthiS. (2018). Online physician reviews do not reflect patient satisfaction survey responses. Mayo Clinic Proceedings, 93(4), 453-457. https://doi.org/10.1016/j.mayocp.2018.01.021
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
