Sage Journals: Discover world-class research

Abstract

Purpose: Group-administered and shared facilitator treatments can induce nested data in a treatment arm that is not present in the control arm. Failure to accommodate these partially nested data structures produces study design inefficiencies, biased parameter estimates, and inaccurate inferences. This work introduces partially nested data structures. Method: We began by describing the features of partially nested data then discuss best practices and guidelines for study planning and analysis through examples commonly found in social work research. Results: The totality of this work provides social work researchers with the knowledge and tools to accommodate partially nested data in study planning and analysis including integration of comprehensive effects (i.e., mediation and moderation). Discussion: Improved understanding of partially nested data structures help researchers avoid the detrimental effects associated with disregarding them. Broadly, these methodological advances increase the capacity and quality of research in the field of social work.

Keywords

partial nesting partially nested design individual randomized trial cluster randomized trial multilevel modeling

Get full access to this article

View all access options for this article.

References

Ataneka

Kelcey

Dong

Bulus

Bai

(2023). PowerUp R Shiny App (v. 0.9).

Baldwin

Bauer

Stice

Rohde

(2011). Evaluating models for partially clustered designs. Psychological Methods, 16(2), 149–165. https://doi.org/10.1037/a0023464

Bates

Mächler

Bolker

Walker

(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67(1), 1–48. https://doi.org/10.18637/jss.v067.i01

Bauer

Sterba

S. K.

Hallfors

(2008). Evaluating group-based interventions when control participants are ungrouped. Multivariate Behavioral Research, 43(2), 210–236. https://doi.org/10.1080/00273170802034810

Browne

Darnell

Savage

T. E.

Brown

(2015). Social workers as patient navigators: A review of the literature. Social Work Research, 39, 158–166. https://doi.org/10.1093/swr/svv017

Bulus

Dong

Kelcey

Spybrook

(2021). PowerUpR: Power Analysis Tools for Multilevel Randomized Experiments. R package version 1.1.0, https://CRAN.R-project.org/package=PowerUpR

Candlish

Teare

Dimairo

Flight

Mandefield

Walters

(2018). Appropriate statistical methods for analysing partially nested randomised controlled trials with continuous outcomes: A simulation study. BMC Medical Research Methodology, 18(105), 1–17. https://doi.org/10.1186/s12874-018-0559-x

Chu

(2013). Bayesian mediation analysis for partially clustered designs (Master of Science in Statistics). University of Texas at Austin. http://repositories.lib.utexas.edu/handle/2152/22571

Cox

Kelcey

(2019). Optimal sample allocation in group randomized mediation studies with a group-level mediator. The Journal of Experimental Education, 87(4), 616–640. https://doi.org/10.1080/00220973.2018.1496060

10.

Cox

Kelcey

(2023). Statistical power for detecting moderation in partially nested designs. American Journal of Evaluation, 44(1), 133–152. https://doi.org/10.1177/1098214020977692

11.

Cox

Kelcey

Luce

(2022). Power to detect moderated effects in three-level partially nested designs. Journal of Experimental Education, 1–24. https://doi.org/10.1080/00220973.2022.2130130.

12.

Dong

Kelcey

Spybrook

(2018). Power analyses for moderator effects in three-level cluster randomized trials. The Journal of Experimental Education, 86(3), 489–514. https://doi.org/10.1080/00220973.2017.1315714

13.

Dong

Kelcey

Spybrook

(2021a). Design considerations in multisite randomized trials probing moderated treatment effects. Journal of Educational and Behavioral Statistics, 46(5), 527–559. https://doi.org/10.3102/1076998620961492

14.

Dong

Reinke

W. M.

Herman

K. C.

Bradshaw

C. P.

Murray

D. W.

(2016). Meaningful effect sizes, intraclass correlations, and proportions of variance explained by covariates for planning two- and three-level cluster randomized trials of social and behavioral outcomes. Evaluation Review, 40(4), 334–377. https://doi.org/10.1177/0193841X16671283

15.

Dong

Spybrook

Kelcey

Bulus

(2021b). Power analyses for moderator effects with (non)randomly varying slopes in cluster randomized trials. Methodology, 17(2), 92–110. https://doi.org/10.5964/meth.4003

16.

Fritz

M. S.

MacKinnon

D. P.

(2007). Required sample size to detect the mediated effect. Psychological Science, 18(3), 233–239. https://doi.org/10.1111/j.1467-9280.2007.01882.x

17.

Garland

E. L.

(2013). Mindfulness research in social work: Conceptual and methodological recommendations. Social Work Research, 37, 439–448. https://doi.org/10.1093/swr/svt038

18.

Hayes

A. F.

(2013). Introduction to mediation, moderation, and conditional process Analysis: A regression-based approach. Guilford Press.

19.

Hedges

Citkowicz

(2015). Estimating effect size when there is clustering in one treatment group. Behavior Research Methods, 47, 1295–1308. https://doi.org/10.3758/s13428-014-0538-z

20.

Kazdin

A. E.

(2007). Mediators and mechanisms of change in psychotherapy research. Annual Review of Clinical Psychology, 3(1), 1–27. https://doi.org/10.1146/annurev.clinpsy.3.022806.091432

21.

Kelcey

Bai

Xie

(2020). Statistical power in partially nested designs probing multilevel mediation. Psychotherapy Research, 30(8), 1061–1074. https://doi.org/10.1080/10503307.2020.1717012

22.

Kelcey

Dong

Spybrook

Cox

(2017). Statistical power for causally-defined indirect effects in group-randomized trials with individual-level mediators. Journal of Educational and Behavioral Statistics, 42(5), 499–530. https://doi.org/10.3102/1076998617695506

23.

Kiley

K. A.

Sehgal

A. R.

Neth

Dolata

Pike

Spilsbury

J. C.

Albert

J. M.

(2018). The effectiveness of guided imagery in treating compassion fatigue and anxiety of mental health workers. Social Work Research, 42(5), 33–43. https://doi.org/10.1093/swr/svx026

24.

Korendijk

Maas

Hox

Moerbeek

(2012). The robustness of the parameter and standard error estimates in trials with partially nested data: A simulation study. In Korendijk

(Ed.), Robustness and optimal design issues for cluster randomized trials (pp. 59–94). Dissertation, Utrecht University.

25.

Lachowicz

M. J.

Sterba

S. K.

Preacher

K. J.

(2015). Investigating multilevel mediation with fully or partially nested data. Group Processes & Intergroup Relations, 18(3), 274–289. https://doi.org/10.1177/1368430214550343

26.

Lai

Kwok

O.-M.

(2014). Standardized mean differences in two-level cross-classified random effects models. Journal of Educational and Behavioral Statistics, 39(4), 282–302. https://doi.org/10.3102/1076998614532950

27.

Lander

L. R.

Gurka

K. K.

Marshalek

P. J.

Riffon

Sullivan

C. R.

(2015). A comparison of pregnancy-only versus mixed-gender group therapy among pregnant women with opioid use disorder. Social Work Research, 39(4), 235–244. https://doi.org/10.1093/swr/svv029

28.

Lee

K. J.

Thompson

S. G.

(2005). The use of random effects models to allow for clustering in individually randomized trials. Clinical Trials, 2(2), 163–173. https://doi.org/10.1191/1740774505cn082oa

29.

Lee

M. Y.

Wang

Liu

Raheim

Tebb

(2018). Outcome literature review of integrative body-mind-spirit practices for mental health conditions. Social Work Research, 42(3), 251–265. https://doi.org/10.1093/swr/svy018

30.

Lohr

Schochet

P. Z.

Sanders

(2014). Partially nested randomized controlled trials in education research: A guide to design and analysis (NCER 2014-2000). National Center for Education Research Institute of Education Sciences, U.S. Department of Education.

31.

Lundahl

B. W.

Kunz

Brownell

Tollefson

Burke

B. L.

(2010). A meta-analysis of motivational interviewing: Twenty-five years of empirical studies. Research on Social Work Practice, 20(2), 137–160. https://doi.org/10.1177/1049731509347850

32.

MacKinnon

D. P.

(2008). Introduction to statistical mediation analysis. Erlbaum.

33.

MacKinnon

D. P.

(2011). Integrating mediators and moderators in research design. Research on Social Work Practice, 21(6), 675–681. https://doi.org/10.1177/1049731511414148

34.

Magill

(2011). Moderators and mediators in social work research: Toward a more ecologically valid evidence base for practice. Journal of Social Work, 11(4), 387–401. https://doi.org/10.1177/1468017310379930

35.

Marsiglia

F. F.

Yabiku

S. T.

Kulis

Nieri

Lewin

(2010). Influences of school Latino composition and linguistic acculturation on a prevention program for youths. Social Work Research, 34(1), 6–19. https://doi.org/10.1093/swr/34.1.6

36.

Mehta

P. D.

(2013). N-level structural equation modeling. In Petscher

Schatschneider

Compton

(Eds.), Applied quantitative analysis in education and the social sciences (pp. 329–361). Routledge.

37.

Mehta

P. D.

Neale

M. C.

(2005). People are variables too: Multilevel structural equations modeling. Psychological Methods, 10(3), 259–284. https://doi.org/10.1037/1082-989X.10.3.259

38.

Moerbeek

Wong

W. K.

(2008). Sample size formulae for trials comparing group and individual treatments in a multilevel model. Statistics in Medicine, 27(15), 2850–2864. https://doi.org/10.1002/sim.3115

39.

Newlin

Webber

Morris

Howarth

(2015). Social participation interventions for adults with mental health problems: A review and narrative synthesis. Social Work Research, 39(4), 167–180. https://doi.org/10.1093/swr/svv015

40.

Petscher

Schatschneider

(2019). Using n-level structural equation models for causal modeling in fully nested, partially nested, and cross-classified randomized controlled trials. Educational and Psychological Measurement, 79(6), 1075–1102. https://doi.org/10.1177/0013164419840071

41.

Raudenbush

S. W.

(1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173–185. https://doi.org/10.1037/1082-989X.2.2.173

42.

Raudenbush

S. W.

Martinez

Spybrook

(2007). Strategies for improving precision in group randomized experiments. Educational Evaluation and Policy Analysis, 29(1), 5–29. https://doi.org/10.3102/0162373707299460

43.

R Core Team (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. URL https://www.R-project.org/.

44.

Roberts

S. A.

(2005). Design and analysis of clinical trials with clustering effects due to treatment. Clinical Trials, 2(2), 152–162. https://doi.org/10.1191/1740774505cn076oa

45.

Roberts

Walwyn

(2013). Design and analysis of non- pharmacological treatment trials with multiple therapists per patient. Statistics in Medicine, 32(1), 81–98. https://doi.org/10.1002/sim.5521

46.

Rose

R. A.

Bowen

G. L.

(2009). Power analysis in social work intervention research: Designing cluster-randomized trials. Social Work Research, 33(1), 43–52. https://doi.org/10.1093/swr/33.1.43

47.

Rosen

Proctor

E. K.

Staudt

M. M.

(1999). Social work research and the quest for effective practice. Social Work Research, 23(1), 4–14. https://doi.org/10.1093/swr/23.1.4

48.

Rubin

Parrish

(2007, January 12). Examination of a sample of published outcome studies from 2000–2005: Implications for evidence-based practice. Paper presented at 10th Annual Conference of the Society for Social Work and Research, San Francisco.

49.

Sanders

E. A.

(2011). Multilevel analysis methods for partially nested cluster randomized trials (No. 3452760). ProQuest LLC.

50.

Schweig

Pane

(2016). Intention-to-treat analysis in partially nested randomized controlled trials with real-world complexity. International Journal of Research & Method in Education: SPECIAL ISSUE (PART 1): Is the Educational “What Works” Agenda Working? Critical Methodological Developments, 39(3), 268–286. https://doi.org/10.1080/1743727X.2016.1170800.

51.

Shadish

W. R.

Cook

T. D.

Campbell

D. T.

(2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton, Mifflin and Company.

52.

Spybrook

Kelcey

Dong

(2016). Power for detecting treatment by moderator effects in two- and three-level cluster randomized trials. Journal of Educational and Behavioral Statistics, 41(6), 605–627. https://doi.org/10.3102/1076998616655442

53.

Sterba

S. K.

(2017). Partially nested designs in psychotherapy trials: A review of modeling developments. Psychotherapy Research, 27(4), 425–436. https://doi.org/10.1080/10503307.2015.1114688

54.

Sterba

S. K.

Preacher

K. J.

Hardcastle

E. J.

Forehand

Cole

D. A.

Compas

B. E.

(2014). Structural equation modeling approaches for analyzing partially nested data. Multivariate Behavioral Research, 49(2), 93–118. https://doi.org/10.1080/00273171.2014.882253

55.

Thyer

(2023). Experimental research designs in social work: Theory and applications. Columbia University Press. https://doi.org/10.7312/thye20116.

56.

Thyer

B. A.

(2015). A bibliography of randomized controlled experiments in social work (1949–2013): Solvitur Ambulando. Research on Social Work Practice, 25(7), 753–793. https://doi.org/10.1177/1049731515599174

57.

Walwyn

Roberts

(2010). Therapist variation within randomised trials of psychotherapy: Implications for precision, internal, and external validity. Statistical Methods in Medical Research, 19(3), 291–315. https://doi.org/10.1177/0962280209105017

58.

Wampold

B. E.

Serlin

R. C.

(2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods, 5(4), 425–433. https://doi.org/10.1037/1082-989X.5.4.425

59.

Wanzek

Petscher

Al Otaiba

Rivas

Jones

F.,

Kent

, … Mehta

(2017). Effects of a year-long supplemental Reading intervention for students with Reading difficulties in fourth grade. Journal of Educational Psychology, 109(8), 1103–1119. https://doi.org/10.1037/edu0000184

60.

Williams

J. H.

(2016). Grand challenges for social work: Research, practice, and education. Social Work Research, 40(2), 67–70. https://doi.org/10.1093/swr/svw007

61.

Willy-Gravley

Beauchemin

Pirie

Gomes

Klein

(2021). A randomized controlled trial of yoga with incarcerated females: Impacts on emotion regulation, body dissociation, and warnings of substance relapse. Social Work Research, 45(1), 19–29. https://doi.org/10.1093/swr/svaa023

62.

Yang

Bear

G. G.

May

(2018). Multilevel associations between school-wide social–emotional learning approach and student engagement across elementary, middle, and high schools. School Psychology Review, 47(1), 45–61. https://doi.org/10.17105/SPR-2017-0003.V47-1

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB

Partially Nested Designs in Social Work Research: Principles and Practices

Abstract

Keywords

Get full access to this article

References

Supplementary Material