Assessing Statistical Change Indices in Selected Social Work Intervention Research Studies

Abstract

Objectives:

This study examined how evaluation and intervention research (IR) studies assessed statistical change to ascertain effectiveness.

Methods:

Studies from six core social work journals (2009–2013) were reviewed (N = 1,380). Fifty-two evaluation (n = 27) and intervention (n = 25) studies met the inclusion criteria. These studies were categorized by the statistical change indices and effect sizes reported. Only intervention studies were included in the final analysis.

Results:

Multivariate analysis of variance (28%) was the most frequently used primary change statistic and χ² (43%) was the most frequent secondary indicator. About 68% of studies reported effect sizes with Cohen’s d (28%) being the most frequent.

Conclusion:

In addition to assessing statistical significance, the extent of rigor lies in determining the size of any change or effect in evaluation and IR. Implications for social work practice and research are discussed.

Keywords

intervention research primary and secondary statistical change indices effect sizes

Get full access to this article

View all access options for this article.

References

Amico

K. R.

(2009). Percent total attrition: A poor metric for study rigor in hosted intervention designs. American Journal of Public Health, 99, 1567–1575. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=47134697&site=eds-live

Barry

A. E.

Chaney

B. H.

Piazza-Gardner

Chavarria

E. A.

(2014). Validity and reliability reporting practices in the field of health education and behavior: A review of seven journals. Health Education & Behavior, 41, 12–18. Retrieved from http://ezproxy.gsu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ1020524&site=eds-live; http://dx.doi.org/10.1177/1090198113483139

Bledsoe-Mansori

(2013). Agency–University partnerships for evidence-based practice: A national survey of schools of social work. Social Work Research, 37, 179–193. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=pbh&AN=93601201&site=eds-live

Brink

Louw

Q. A.

(2012). Clinical instruments: Reliability and validity critical appraisal. Journal of Evaluation in Clinical Practice, 18, 1126–1132. doi:10.1111/j.1365-2753.2011.01707.x

Cohen

B. J.

(2011). Design-based practice: A new perspective for social work. Social Work, 56, 337–346. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=swh&AN=81807&site=eds-live

Council of Social Work Education, Inc. (2008). Educational policy and accreditation standards. Retrieved from http://www.cswe.org/File.aspx?id=41861

Davis

Gervin

White

Williams

Taylor

McGriff

(2013). Bridging the gap between research, evaluation, and evidence-based practice. Journal of Social Work Education, 49, 16–29. doi:10.1080/10437797.2013.755099

Drisko

(2014). Research evidence and social work practice: The place of evidence-based practice. Clinical Social Work Journal, 42, 123–133. doi:10.1007/s10615-013-0459-9

Fong

Pomeroy

E. C.

(2011). Translating research to practice. Social Work, 56, 5–7. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=a9h&AN=63700653&site=eds-live

10.

Fraser

M. W.

(2004). Intervention research in social work: Recent advances and continuing challenges. Research on Social Work Practice, 14, 210–222. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ807764&site=eds-live; http://dx.doi.org/10.1177/1049731503262150

11.

Fraser

M. W.

Galinsky

M. J.

(2010). Steps in intervention research: Designing and developing social programs. Research on Social Work Practice, 20, 459–466. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ893094&site=eds-live; http://dx.doi.org/10.1177/1049731509358424

12.

Herie

Martin

G. W.

(2002). Knowledge diffusion in social work: A new approach to bridging the gap. Social Work, 47, 85–95. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=swh&AN=71405&site=eds-live

13.

Holosko

M. J.

(2010). What types of designs are we using in social work research and evaluation? Research on Social Work Practice, 20, 665–673. doi:10.1177/1049731509339586

14.

Holosko

M. J.

Leslie

(1998). Obstacles to conducting empirically-based practice. In Wodarski

Thyer

B. A.

(Eds.), Handbook of empirical social work: Vol 2, social problems and practice issues (pp. 433–451). New York, NY: John Wiley.

15.

Holosko

M. J.

Thyer

B. A.

(2011). Pocket glossary for commonly used research terms. Thousand Oaks, CA: Sage.

16.

Holosko

M. J.

Thyer

B. A.

Danner

(2009). Ethical guidelines for designing and conducting evaluations of social work practice. Journal of Evidence-Based Social Work, 6, 348–360. doi:10.1080/15433710903126778

17.

Howard

M. O.

McMillen

C. J.

Pollio

D. E.

(2003). Teaching evidence-based practice: Toward a new paradigm for social work education. Research on Social Work Practice, 13, 234–259. Retrieved from http://ezproxy.gsu.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true&db=rzh&AN=2003108879&site=eds-live

18.

Ligon

Cobb

Thyer

(2012). Academic affiliations of social work journal article authors, 2004–2008. Journal of Social Work Education, 48, 613–622.

19.

Ligon

Jackson

D. L.

Thyer

(2007). Academic affiliations of social work journal article authors from 1999–2003: A productivity analysis spanning 25 years of social work scholarship. Journal of Social Service Research, 33, 13–20.

20.

Muhlhausen

D. B.

(2012). Evaluating federal social programs: Finding out what works and what does not. Research on Social Work Practice, 22, 100–107. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=eric&AN=EJ952595&site=eds-live; http://dx.doi.org/10.1177/1049731511420267

21.

Mullen

E. J.

Shuluk

(2011). Outcomes of social work intervention in the context of evidence-based practice. Journal of Social Work, 11, 49–63. doi:10.1177/1468017310381309

22.

National Association of Social Workers. (2013). Code of ethics of the National Association Of Social Workers. Retrieved from http://www.socialworkers.org/pubs/code/code.asp?print=1&

23.

Okpych

N. J.

J. L.

(2014). A historical analysis of evidence- based practice in social work: The unfinished journey toward an empirically grounded profession. Social Service Review, 88, 3–58. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=94885811&site=eds-live

24.

Patton

M. Q.

(2010). Utilization-focused evaluation for social services and social work. Social Work Review/Revista De Asistenta Sociala, 4, 11–27. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=sih&AN=55466879&site=eds-live

25.

Rubin

Parrish

(2007). Problematic phrases in the conclusions of published outcome studies: Implications for evidence-based practice. Research on Social Work Practice, 17, 334–347. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=swh&AN=56812&site=eds-live

26.

Secret

Abell

M. L.

Berlin

(2011). The promise and challenge of practice-research collaborations: Guiding principles and strategies for initiating, designing, and implementing program evaluation research. Social Work, 56, 9–20. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=swh&AN=80288&site=eds-live

27.

Shardlow

S. M.

Harlow

(2010). Academic affiliations of social work journal article authors from 1999–2003: A productivity analysis from the United Kingdom. Journal of Social Service Research, 36, 377–384.

28.

Smeeton

Goda

(2003). Conducting and presenting social work research: Some basic statistical considerations. British Journal of Social Work, 33, 567–573.

29.

Soydan

(2002). Formulating research problems in practitioner-researcher partnerships. Social Work Education, 21, 297–304.

30.

Thomson Reuters. (2014). 2014 journal citation reports social science edition. Retrieved from http://admin-apps.webofknowledge.com.proxy-remote.galib.uga.edu/JCR/JCR?PointOfEntry=Home&SID=3DQJyYZZH3CGLxP9cth

31.

Thyer

B. A.

Bentley

K. J.

(1986). Academic affiliations of social work authors: A citation analysis of six major journals. Journal of Social Work Education, 22, 67–73.

32.

Valentine

J. C.

McHugh

C. M.

(2007). The effects of attrition on baseline comparability in randomized experiments in education: A meta-analysis. Psychological Methods, 12, 268–282. doi:10.1037/1082-989X.12.3.268

33.

Wodarski

J. S.

(2011). The social work practice research conundrum. Journal of Human Behavior in the Social Environment, 21, 577–600. doi:10.1080/10911359.2011.580249