Quality Indicators of Secondary Data Analyses in Special Education Research: A Preregistration Guide

Abstract

Secondary data analyses occur when new analyses are proposed for existing data. Although they are prevalent in special education research, there is little guidance on how to prepare secondary data analyses studies. Preregistration of secondary data analyses studies provides a nice opportunity and structure for fellow researchers to share innovative questions and analytic approaches to existing data sets as well as increase transparency. In this manuscript, we (a) describe quality indicators of secondary data analyses consistent with open science practices and (b) provide applied examples of these indicators from a sampling of published studies based on two iterations of data from the National Longitudinal Transition Study (NLTS2 and NLTS2012) with the overall goals to provide guidance to authors and peer reviewers and promote collaboration among fellow researchers engaged in secondary analyses for a range of purposes.

Get full access to this article

View all access options for this article.

References

Baldwin

J. R.

Pingault

J. B.

Schoeler

Sallis

H. M.

Munafò

M. R

. (2022). Protecting against researcher bias in secondary data analysis: Challenges and potential solutions. European Journal of Epidemiology, 37, 1–10. https://doi.org/10.1007/s10654-021-00839-0

Cook

B. G.

Fleming

J. I.

Hart

S. A.

Lane

K. L.

Therrien

W. J.

van Dijk

Wilson

S. E

. (2022). A how-to guide for open-science practices in special education research. Remedial and Special Education, 43(4), 270–280.

Croon

(2002). Using predicted latent scores in general latent structure models. In Marcoulides

Moustaki

(Eds.), Latent variable and latent structure models (pp. 195–223). Lawrence Erlbaum.

Cruz

R. A.

Lee

J. H.

Aylward

A. G.

Voulgarides

C. K

. (2022). The effect of school funding on opportunity gaps for students with disabilities: Policy and context in a diverse urban district. Journal of Disability Policy Studies, 33(1), 3–24. https://doi.org/10.1177/1044207320970545

Dong

Maynard

. (2013). PowerUp! A tool for calculating minimum detectable effect sizes and minimum required sample sizes for experimental and quasi-experimental design studies. Journal of Research on Educational Effectiveness, 6(1), 24–67. https://doi.org/10.1080/19345747.2012.673143

Devlieger

Talloen

Rosseel

. (2019). New developments in factor score regression: Fit indices and a model comparison test. Educational and Psychological Measurement, 79(6), 1017–1037. https://doi.org/10.1177/0013164419844552

Enders

C. K.

(2010). Applied missing data analysis. Guilford Press.

Fleming

J. I.

Cook

B. G

. (2022). Open access in special education: A review of journal and publisher policies. Remedial and Special Education, 43(1), 3–14. https://doi.org/10.1177/0741932521996461

Gelman

Hill

(2006). Data analysis using regression and multilevel/hierarchical models. Cambridge University Press.

10.

Grigal M.

Hart

& Migliore

, A. (2011) Comparing the transition planning, postsecondary education, and employment outcomes of students with intellectual and other disabilities. Career Development for Exceptional Individuals, 34(1), 4–17. https://doi.org/10.1177/0885728811399091

11.

Hall-Mills

. (2019). A comparison of the prevalence rates of language impairment before and after response-to-intervention implementation. Language, Speech, and Hearing Services in Schools, 50(4), 703–709. https://doi.org/10.1044/2019_LSHSS-18-0144

12.

Johnson

A. H.

Cook

B. G

. (2019). Preregistration in single-case design research. Exceptional Children, 86(1), 95–112. https://doi.org/10.1177/0014402919868529

13.

Kerr

N. L

. (1998). HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3), 196–217. https://doi.org/10.1207/s15327957pspr0203_4

14.

Kline

R. B.

(2015). Principles and practice of structural equation modeling. Guilford Press.

15.

Lang

K. M.

. (2017). A comparison of methods for creating multiple imputations of nominal variables. Multivariate Behavioral Research, 52(3), 290–304. https://doi.org/10.1080/00273171.2017.1289360

16.

Leppo

Cawthon

Bond

. (2013). Including deaf and hard-of-hearing students with co-occurring disabilities in the accommodations discussion. Journal of Deaf Studies and Deaf Education, 19(2), 189–202. https://doi.org/10.1093/deafed/ent029

17.

Little

T. D.

Jorgensen

T. D.

Lang

K. M.

Moore

E. W. G

. (2014). On the joys of missing data. Journal of Pediatric Psychology, 39(2), 151–162. https://doi.org/10.1093/jpepsy/jst048

18.

Logan

. (2020) A practical, iterative framework for secondary data analysis in educational research. Australian Educational Researcher, 47, 129–148. https://doi.org/10.1007/s13384-019-00329-z

19.

Lombardi

Rifenbark

Hicks

Taconet

A. V.

Challenger

(2021). College and career readiness support for youth with and without disabilities based on the National Longitudinal Transition Study 2012. [Stage-1 Registered Report] https://doi.org/10.17605/OSF.IO/C9D4P

20.

Lombardi

A. R.

Rifenbark

G. G.

Hicks

T. A.

Taconet

Challenger

. (2022). College and career readiness support for youth with and without disabilities based on the National Longitudinal Transition Study 2012. Exceptional Children, 89(1), 5–21. https://doi.org/10.1177/00144029221088940

21.

Maas

C. J.

Hox

J. J

. (2005). Sufficient sample sizes for multilevel modeling. Methodology, 1(3), 86–92. https://doi.org/10.1027/1614-2241.1.3.86

22.

Mazzotti

V. L.

Rowe

D. A.

Sinclair

Poppen

Woods

W. E.

Shearer

M. L

. (2016). Predictors of post-school success: A systematic review of NLTS2 secondary analyses. Career Development and Transition for Exceptional Individuals, 39(4), 196–215. https://doi.org/10.1177/2165143415588047

23.

McCoach

D. B

., Peters

., Long

., & Siegle

. (2022). Examining teacher variability in teacher rating scales for gifted identification [Stage 1 registered report]. https://osf.io/ykv5h/

24.

McCoach

D. B.

Rifenbark

G. G.

Newton

S. D.

Kooken

Yomtov

Gambino

A. J.

Bellara

. (2018). Does the package matter? A comparison of five common multilevel modeling software packages. Journal of Educational and Behavioral Statistics, 43(5), 594–627. https://doi.org/10.3102/1076998618776348

25.

McNeish

Stapleton

L. M.

Silverman

R. D

. (2017). On the unnecessary ubiquity of hierarchical linear modeling. Psychological Methods, 22(1), 114. https://doi.org/10.1037/met0000078

26.

Morgan

Farkas

Hillemeier

M. M.

Maczuga

. (2017). Replicated evidence of racial and ethnic disparities in disability identification in U.S. schools. Educational Researcher, 46(6), 305–322. https://doi.org/10.3102/0013189X17726282

27.

Nosek

B. A.

Ebersole

C. R.

DeHaven

A. C.

& Mellor

D. T.

(2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114

28.

Nosek

B. A.

Lakens

. (2014). A method to increase the credibility of published results. Social Psychology, 45(3), 137–141. https://doi.org/10.1027/1864-9335/a000192

29.

Raudenbush

S. W.

Spybrook

Congdon

Liu

Martinez

Bloom

Hill

(2011). Optimal design software for multi-level and longitudinal research (Version 3.01) [Computer software].

30.

Rubin

D. B

. (1976). Inference and missing data. Biometrika, 63(3), 581–592. https://doi.org/10.1093/biomet/63.3.581

31.

Rubin

D. B.

(1987). Multiple imputation for nonresponse in surveys. Wiley.

32.

Schuck

Emerson

R. W.

Kim

D. S.

Nelson

N. W

. (2018). An approach to using orientation and mobility (O&M) variables from the second National Longitudinal Transition Study. Journal of Visual Impairment & Blindness, 112(2), 203–208 https://doi.org/10.1177/0145482X1811200208

33.

Shogren

K. A.

Kennedy

Dowsett

Garnier Villarreal

Little

T.D.

(2014). Exploring essential characteristics of self-determination for diverse students using data from NLTS2. Career Development and Transition for Exceptional Individuals, 37(3), 168–176. https://doi.org/10.1177/2165143413486927

34.

Shogren

K. A.

Pace

J. R.

Wittenburg

D. C.

Raley

S. K.

Hicks

T. A.

Rifenbark

G. G.

Lane

K. L.

Anderson

M. H

. (2022). Self-report and administrative data on disability and IEP status: Examining differences and impacts on intervention outcomes. Journal of Disability Policy Studies. https://doi.org/10.1177/10442073221094811

35.

Talbott

De Los Reyes

Kearns

D. M.

Mancilla-Martinez

Cook

C. R.

Wang

. (2023). Evidence based assessment in special education research: Advancing the use of evidence in methods, tools, and empirical processes. Exceptional Children, 89(4).

36.

Test

D. W.

Mazzotti

V. L.

Mustian

A. L.

Fowler

C. H.

Kortering

Kohler

. (2009). Evidence-based secondary transition predictors for improving postschool outcomes for students with disabilities. Career Development for Exceptional Individuals, 32(3), 160–181. https://doi.org/10.1177/0885728809346960

37.

van den Akker

O. R

., Weston

., Campbell

., Chopik

., Damian

., Davis-Kean

Hall

Kosie

Kruse

Olsen

Ritchie

Valentine

van't Veer

Bakker

. (2021). Preregistration of secondary data analysis: A template and tutorial. Meta-Psychology, 5. https://doi.org/10.15626/MP.2020.2625

38.

Van Buuren

Groothuis-Oudshoorn

. (2011). mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45(3), 1–67.

39.

White House. (2022). Breakthroughs for all: Delivering equitable access to America’s research . https://www.whitehouse.gov/ostp/news-updates/2022/08/25/breakthroughs-for-alldelivering-equitable-access-to-americas-research/

40.

Zablocki

Krezmien

M. P

. (2013). Drop-out predictors among students with high-incidence disabilities: A National Longitudinal and Transitional Study 2 analysis. Journal of Disability Policy Studies, 24(1), 53–64. https://doi.org/10.1177/1044207311427726

41.

Zhou

Griffin-Shirley

Kelley

Banda

D. R.

Lan

W. Y.

Parker

A. T.

Smith

D. W

. (2012). The relationship between computer and internet use and performance on standardized tests by secondary school students with visual impairments. Journal of Visual Impairment & Blindness, 106(10), 609–621. https://doi.org/10.1177/0145482X1210601005

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.06 MB