Little is known concerning the way that Licensed Masters-Level social workers are evaluating outcomes.
Methods
Licensed providers (n = 111) completed a mixed-methods web-based survey regarding ways they evaluate practice outcomes. Content analysis was used to identify ways in which social workers reported evaluating practice outcomes.
Results
Evaluation methods identified were (1) not consistently evaluating client outcomes, (2) completion of follow-up appointments, (3) clinical judgment/completion of treatment plan, (4) satisfaction surveys, (5) client-reported symptom remission, (6) judgment of key informants, and (7) standardized objective measures. Objective evaluation methods were used most by individuals reporting frequent outcome evaluation, those reporting less frequent outcome evaluation used client self-report or relied on feedback from key informants to gauge client’s progress. Frequency of outcome evaluation was inversely related to perceived difficulty of evaluating outcomes.
Discussion
Social workers frequently rely on non-standardized, subjective evaluation methods. More research in this area is currently indicated.
Arnd-CaddiganM. (2012). Clinical social workers’ response to a state policy on the use of evidence-based practice: How attitudes, behaviors, and competing values affect actual practice. Smith College Studies in Social Work, 82(1), 19–38. https://doi.org/10.1080/00377317.2012.638892
2.
BakerD. B.BenjaminL. T.Jr. (2000). The affirmation of the scientist-practitioner: A look back at Boulder. American Psychologist, 55(2), 241–247. https://doi.org/10.1037/0003-066X.55.2.241
3.
BakerL. R.StephensF.HitchcockL. (2010). Social work practitioners and practice evaluation How are we doing?Journal of Human Behavior in the Social Environment, 20(8), 963–973. https://doi.org/10.1080/15433714.2010.498669
4.
BeckA. T.EpsteinN.BrownG.SteerR. A. (1988). An inventory for measuring clinical anxiety: Psychometric properties. Journal of Consulting and Clinical Psychology, 56(6), 893–897. http://dx.doi.org/10.1037/0022-006X.56.6.893
5.
BeidasR. S.KendallP. C. (2010). Training therapists in evidence‐based practice: A critical review of studies from a systems‐contextual perspective. Clinical Psychology: Science and Practice, 17(1), 1–30. https://doi.org/10.1111/j.1468-2850.2009.01187.x
6.
BeidasR. S.StewartR. E.WalshL.LucasS.DowneyM. M.JacksonK.MandellD. S. (2015). Free, brief, and validated: Standardized instruments for low-resource mental health settings. Cognitive and Behavioral Practice, 22(1), 5–19. https://doi.org/10.1016/j.cbpra.2014.02.002
7.
BellamyJ. L.ParishD. E. (2020). Practical implementation in social work practice: A guide to engaging in evidence-based practice. Oxford University Press.
8.
BenderK.AltschulI.YoderJ.ParrishD.NickelsS. J. (2014). Training social work graduate students in the evidence-based practice process. Research on Social Work Practice, 24(3), 339–348. https://doi.org/10.1177/1049731513506614
9.
BoswellJ. F.KrausD. R.MillerS. D.LambertM. J. (2015). Implementing routine outcome monitoring in clinical practice: Benefits, challenges, and solutions. Psychotherapy Research, 25(1), 6–19. https://doi.org/10.1080/10503307.2013.817696
CollinsE.DalyE. (2011). Decision making and social work in Scotland: The role of evidence and practice wisdom. The institute for research and innovation in social services (IRISS). http://www.iriss.org.uk/sites/default/files/decision-making-wisdom-iriss-2011.pdf.
12.
CookB. G.TankersleyM.Harjusola-WebbS. (2008). Evidence-based special education and professional wisdom: Putting it all together. Intervention in School and Clinic, 44(2), 105–111. https://doi.org/10.1177%2F1053451208321566
CreeV.JainS.HillenD. P. (2018). Evaluating effectiveness in social work: Sharing dilemmas in practice. European Journal of Social Work, 22(4), 599–610. https://doi.org/10.1080/13691457.2018.1441136
15.
Crits-ChristophP.Ring-KurtzS.LambertM. J.GallopR.McClureB.KulagaA.RotrosenJ. (2012). A preliminary study of the effects of individual patient-level feedback in outpatient substance abuse treatment programs. Journal of Substance Abuse Treatment, 42(3), 301–309. https://doi.org/10.1016/j.jsat.2011.09.003
16.
EdmondT.MegivernD.WilliamsC.RochmanE.HowardM. (2006). Integrating evidence-based practice and social work field education. Journal of Social Work Education, 42(2), 377–396. https://doi.org/10.5175/JSWE.2006.200404115
GambrillE. (2019). The promotion of avoidable ignorance in the British Journal of Social Work. Research on Social Work Practice, 29(4), 455–469. https://doi.org/10.1177%2F1049731518763953
GarlandA. F.KruseM.AaronsG. A. (2003). Clinicians and outcome measurement: What’s the use?The Journal of Behavioral Health Services & Research, 30(4), 393–405. https://doi.org/10.1007/BF02287427
21.
GromoskeA. N.BergerL. K. (2017). Replication of a continuing education workshop in the evidence-based practice process. Research on Social Work Practice, 27(6), 676–682. https://doi.org/10.1177%2F1049731515597477
22.
HatfieldD.McCulloughL.FrantzS. H.KriegerK. (2010). Do we know when our clients get worse? An investigation of therapists’ ability to detect negative client change. Clinical Psychology & Psychotherapy: An International Journal of Theory & Practice, 17(1), 25–32. https://doi.org/10.1002/cpp.656
23.
HoloskoM. J. (2010). What types of designs are we using in social work research and evaluation?Research on Social Work Practice, 20(6), 665–673. https://doi.org/10.1177/1049731509339586
24.
HsiehH.-F.ShannonS. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
25.
KarverM.HandelsmanJ.FieldsS.BickmanL. (2006). Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clinical Psychology Review, 26(1), 50–65. https://doi.org/10.1016/j.cpr.2005.09.001
26.
KazdinA. E. (2008). Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist, 63(3), 146–159. https://doi.org/10.1037/0003066X.63.3.146
27.
KirkS.A.ReidW.J. (2002). Science and social work: A critical appraisal. Columbia University Press.
28.
LoveS. M.KoobJ. J.HillL. E. (2007). Meeting the challenges of evidence-based practice: Can mental health therapists evaluate their practice?Brief Treatment and Crisis Intervention, 7(3), 184–193. https://doi.org/10.1093/brief-treatment/mhm008
29.
MartinD. J.GarskeJ. P.DavisM. K. (2000). Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. Journal of Consulting and Clinical Psychology, 68(3), 438–450. https://psycnet.apa.org/doi/10.1037/0022-006X.68.3.438
30.
McNeeceC. A.ThyerB. A. (2004). Evidence-based practice and social work. Journal of Evidence-Based Social Work, 1(1), 7–25. https://doi.org/10.1300/J394v01n01_02
31.
MullenE.J.BaconW. (2004). Implementation of practice guidelines and evidence-based treatment. In RobertsA. R.YeagerK. R. (Eds.), Evidence-based practice manual: Research and outcome measures in health and human services (pp. 210-218). Oxford University Press.
32.
MullenE. J.ShlonskyA.BledsoeS. E.BellamyJ. L. (2005). From concept to implementation: Challenges facing evidence-based social work. Evidence & Policy: A Journal of Research, Debate and Practice, 1(1), 61–84. https://doi.org/10.1332/1744264052703159.
33.
MyersL. L.ThyerB. A. (1997). Should social work clients have the right to effective treatment?Social Work, 42(3), 288–298. https://doi.org/10.1093/sw/42.3.288
NelsonT. D.SteeleR. G.MizeJ. A. (2006). Practitioner attitudes toward evidence-based practice: Themes and challenges. Administration and Policy in Mental Health and Mental Health Services Research, 33(3), 398–409. https://doi.org/10.1007/s10488-006-0044-4
36.
OxhandlerH. K.GiardinaT. D. (2017). Social workers’ perceived barriers to and sources of support for integrating clients’ religion and spirituality in practice. Social Work, 62(4), 323–332. https://doi.org/10.1093/sw/swx036
37.
ParrishD. E.OxhandlerH. K. (2015). Social work field instructors’ views and implementation of evidence-based practice. Journal of Social Work Education, 51(2), 270–286. https://doi.org/ 10.1080/10437797.2015.1012943
38.
ParrishD. E.RubinA. (2011). Validation of the evidence-based practice process assessment scale-short version. Research on Social Work Practice, 21(2), 200–211. https://doi.org/10.1177%2F1049731510389193
39.
ParrishD. E.RubinA. (2012). Social workers’ orientations toward the evidence-based practice process: A comparison with psychologists and licensed marriage and family therapists. Social Work, 57(3), 201–210. https://doi.org/10.1093/sw/sws016
40.
ParrishD.SpringerD.FranklinC. (2015). Standardized assessment measures and computer-assisted technologies for evidence-based practice. In JordanC.J.FranklinC (Eds.), Clinical assessment for Social Workers: Quantitative and qualitative methods (4th ed., pp. 81-120). Lyceum Books.
41.
PattonM. Q. (2014). Qualitative research & evaluation methods (3rd ed.). Saint Paul, MN: Sage Publications.
42.
PignottiM.ThyerB. A. (2009). Use of novel unsupported and empirically supported therapies by licensed clinical social workers: An exploratory study. Social Work Research, 33(1), 5–17. https://doi.org/10.1093/swr/33.1.5
43.
PopeN. D.RollinsL.ChaumbaJ.RislerE. (2011). Evidence-based practice knowledge and utilization among social workers. Journal of Evidence-Based Social Work, 8(4), 349–368. https://doi.org/10.1080/15433710903269149
44.
ProctorE.SilmereH.RaghavanR.HovmandP.AaronsG.BungerA.HensleyM. (2011). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research, 38(2), 65–76. https://doi.org/10.1007/s10488-010-0319-7
45.
RandolphJ. J. (2005, October 12–15). Free-marginal multirater Kappa (multirater K [free]): An alternative to Fleiss' fixed-marginal multirater Kappa.[Paper presentation]. Joensuu, Finland: Joensuu Learning and Instruction Symposium 2005. https://files.eric.ed.gov/fulltext/ED490661.pdf
46.
RichmondM. E. (1917). Social diagnosis. New York, NY: Russell Sage Foundation.
47.
RobertsA. R.YeagerK. R. (2004). Evidence-based practice manual: Research and outcome measures in health and human services. USA: Oxford University Press.
48.
RosenA. (2003). Evidence-based social work practice: Challenges and promise. Social Work Research, 27(4), 197-208. https://doi.org/10.1093/swr/27.4.197.
RubinA.BellamyJ. (2012). Practitioner’s guide to using research for evidence-based practice (2nd ed.). Wiley.
51.
RubinA.ParrishD. (2007). Challenges to the future of evidence-based practice in social work education. Journal of Social Work Education, 43(3), 405–428. https://doi.org/10.5175/JSWE.2007.200600612
52.
RubinA.ParrishD. E. (2010). Development and validation of the evidence-based practice process assessment scale: Preliminary findings. Research on Social Work Practice, 20(6), 629–640. https://doi.org/10.1177%2F1049731508329420
53.
RubinA.ParrishD. E.WashburnM. (2016). Outcome benchmarks for adaptations of research-supported treatments for adult traumatic stress. Research on Social Work Practice, 26(3), 243-259. https://doi.org/10.1177%2F1049731514547906.
54.
SackettD. L.RosenbergW. M.GrayJ. M.HaynesR. B.RichardsonW. S. (1996). Evidence based medicine: What it is and what it isn’t. BMJ, 312(7023), 71–72. https://doi.org/10.1136/bmj.312.7023.71
55.
ShawI. (2011). Evaluating in practice. Ashgate Publishing, Ltd.
56.
ShimokawaK.LambertM. J.SmartD. W. (2010). Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system. Journal of Consulting and Clinical Psychology, 78(3), 298–311. https://doi.org/10.1037/a0019247
57.
ShirkS. R.KarverM. (2003). Prediction of treatment outcome from relationship variables in child and adolescent therapy: A meta-analytic review. Journal of Consulting and Clinical Psychology, 71(3), 452–464. https://doi.org/10.1037/0022-006X.71.3.452
58.
SimonW.LambertM. J.HarrisM. W.BusathG.VazquezA. (2012). Providing patient progress information and clinical support tools to therapists: Effects on patients at risk of treatment failure. Psychotherapy Research, 22(6), 638–647. https://doi.org/10.1080/10503307.2012.698918
TeaterB.ChonodyJ. M. (2018). Identifying as an evidence-based social worker: The influence of attitudes, access, confidence, and education. Social Work Education, 37(4), 442–457. https://doi.org/10.1080/02615479.2017.1421161
61.
WalfishS.McAlisterB.O'DonnellP.LambertM. J. (2012). An investigation of self-assessment bias in mental health providers. Psychological Reports, 110(2), 639–644. https://doi.org/10.2466%2F02.07.17.PR0.110.2.639-644
62.
WarrensM. J. (2010). Inequalities between multi-rater Kappas. Advances in Data Analysis and Classification, 4(4), 271–286. https://doi.org/10.1007/s11634-010-0073-4
63.
WashburnM.ParrishD. E.OxhandlerH. K.GarrisonBMaA (2021). Licensed Master of Social Workers’ engagement in the process of evidence-based practice: Barriers and facilitators. Journal of Evidence Based Social Work, 1-17. http://dx.doi.org/10.1080/26408066.2021.1918598.
64.
WashburnM.RubinA.ZhouS. (2018). Benchmarks for outpatient dialectical behavioral therapy in adults with borderline personality disorder. Research on Social Work Practice, 28(8), 895–906. https://doi.org/10.1177/1049731516659363
65.
YunongH.FengzhiM. (2009). A reflection on reasons, preconditions, and effects of implementing evidence-based practice in social work. Social Work, 54(2), 177–181. https://doi.org/10.1093/SW%2F54.2.177
