School Counselors as Critical Consumers of Research: Guidelines for Evaluating Quantitative Research by Recognizing Quality Indicators

Abstract

The purpose of this article is to provide a resource for school counselors to recognize key indicators of quality quantitative research and make informed decisions about incorporating scholarly works into school counseling practice and comprehensive program development. We describe quality indicators of quantitative research by walking readers through common sections of an article, including the literature review, method, results, and discussion sections, and considerations for critical, multicultural, and antiracist appraisal when reading quantitative research. We also address implications for school counseling practitioners, training programs, and students.

Keywords

quantitative research research guidelines quality indicators school counseling antiracism

Get full access to this article

View all access options for this article.

References

Aisenberg

(2008). Evidence-based mental health care to ethnic minority communities: Has its practice fallen short of its evidence? Social Work, 53(4), 297–306. https://www.jstor.org/stable/23718873.

American Psychological Association (2019). Journal article reporting standards (JARS). https://apastyle.apa.org/jars/quantitative.

American Psychological Association (2020). Publication manual of the American Psychological Association: The official guide to APA style (7th ed.).

American School Counselor Association (n.d.a). Empirical research studies supporting the value of school counseling. https://www.schoolcounselor.org/getmedia/7d00dcff-40a6-4316-ab6c-8f3ffd7941c2/Effectiveness.pdf.

American School Counselor Association (2019). ASCA National model: A framework for school counseling programs (4th ed.).

American School Counselor Association (2021a). Current issues in education: Critical race theory. https://www.schoolcounselor.org/getmedia/3e2ae0cf-42ad-418e-8f4f-c56300657a84/cie-crt.pdf.

American School Counselor Association (2021b). The school counselor and anti-racist practices. ASCA Position Statements. https://www.schoolcounselor.org/Standards-Positions/Position-Statements/ASCA-Position-Statements/The-School-Counselor-and-Anti-Racist-Practices.

American School Counselor Association (2021c). The school counselor and cultural diversity. ASCA Position Statements. https://www.schoolcounselor.org/Standards-Positions/Position-Statements/ASCA-Position-Statements/The-School-Counselor-and-Cultural-Diversity.

American School Counselor Association (n.d.b). Standards in practice: Eliminating racism and bias in schools: The school counselor’s role. https://www.schoolcounselor.org/getmedia/542b085a-7eda-48ba-906e-24cd3f08a03f/SIP-Racism-Bias.pdf.

10.

Appelbaum

Cooper

Kline

R. B.

Mayo-Wilson

Nezu

A. M.

Rao

S. M.

(2018). Journal article reporting standards for quantitative research in psychology: The APA publications and communications board task force report. American Psychologist, 73(1), 3–25. https://doi.org/10.1037/amp0000191.

11.

Baker

C. A.

(2019). A QuantCrit approach: Using critical race theory as a means to evaluate if rate my professor assessments are racially biased. Journal of Underrepresented & Minority Progress, 3(1), 1–22. https://doi.org/10.32674/jump.v3i1.1012.

12.

Balkin

R. S.

Sheperis

C. J.

(2011). Evaluating and reporting statistical power in counseling research. Journal of Counseling & Development, 89(3), 268–272. https://doi.org/10.1002/j.1556-6678.2011.tb00088.x.

13.

Bauman

(2004). School counselors and research revisited. Professional School Counseling, 7(3), 141–151.

14.

Belser

Mason

(2021, March 13–15). School counselors’ professional reading behaviors: Implications for evidence-based practice and school counselor training [Conference presentation]. The 2021 Virtual Evidence-Based School Counseling Conference.

15.

Brigman

(2006). Research methods in school counseling: A summary for the practitioner. Professional School Counseling, 9(4), 421–425. https://doi.org/10.1177/2156759X0500900412.

16.

Chen

C.-C.

Lau

J. M.

Richardson

G. B.

Dai

C.-L.

(2020). Measurement invariance testing in counseling. Journal of Professional Counseling: Practice, Theory & Research, 47(2), 89–104. https://doi.org/10.1080/15566382.2020.1795806.

17.

Cook

A. L.

Levy

Whitehouse

(2020). Exploring youth participatory action research in urban schools: Advancing social justice and equity-based counseling practices. Journal for Social Action in Counseling & Psychology, 12(1), 27–43. https://doi.org/10.33043/JSACP.12.1.27-43.

18.

Coughlan

Cronin

Ryan

(2007). Step-by-step guide to critiquing research. Part 1: Quantitative research. British Journal of Nursing, 16(11), 658–663. http://www.tara.tcd.ie/bitstream/handle/2262/69919/Quant%20Article%20Final.pdf;jsessionid=3B65BB9B98E7C61EE576CFEAE85D89DB?sequence=1.

19.

Council for Accreditation of Counseling and Related Educational Programs (2015). 2016 CACREP standards. http://www.cacrep.org/wp-content/uploads/2017/08/2016-Standards-with-citations.pdf.

20.

Creswell

J. W.

Guetterman

T. C.

(2019). Educational research: Planning, conducting, and evaluating quantitative and qualitative research (6th ed.). Pearson.

21.

Dale

J. C.

(2005). Critiquing research for use in practice. Journal of Pediatric Health Care, 19(3), 183–186. https://www.jpedhc.org/article/S0891-5245(05)00106-9/fulltext.

22.

Dei

G. J. S.

(2005). Critical issues in anti-racist research methodologies: An introduction. In Dei

G. J. S.

Johal

G. S.

(Eds.), Critical issues in anti-racist research methodologies (pp. 1–27). Peter Lang.

23.

Dimmitt

Carey

J. C.

Hatch

(2007). Evidence-based school counseling: Making a difference with data-driven practices. Corwin.

24.

Dimmitt

Zyromski

(2020). Evidence-based school counseling: Expanding the existing paradigm. Professional School Counseling, 23(1), 1–5. https://doi.org/10.1177/2156759X20904501.

25.

Edirmanasinghe

Blaginin

(2019). Demystifying the research process: A career intervention with Latinas. Professional School Counseling, 22(1b), 1–12. https://doi.org/10.1177%2F2156759X19834433.

26.

Faul

Erdfelder

Lang

A.-G.

Buchner

(2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146.

27.

Fischer

Poortinga

(2018). Addressing methodological challenges in culture-comparative research. Journal of Cross-Cultural Psychology, 49(5), 691–712. https://doi.org/10.1177/0022022117738086.

28.

Fraenkel

J. R.

Wallen

N. E.

Hyun

H. H.

(2015). How to design and evaluate research in education (9th ed.). McGraw Hill Education.

29.

Frambach

J. M.

van der Vleuten

C. P.

Durning

S. J.

(2013). Quality criteria in qualitative and quantitative research. Academic Medicine: Journal of the Association of American Medical Colleges, 88(4), 552. https://journals.lww.com/academicmedicine/Citation/2013/04000/AM_Last_Page__Quality_Criteria_in_Qualitative_and.29.aspx.

30.

Garcia

N. M.

López

Vélez

V. N.

(2018). QuantCrit: Rectifying quantitative methods through critical race theory. Race Ethnicity and Education, 21(2), 149–157. https://doi.org/10.1080/13613324.2017.1377675.

31.

Gillborn

(2010). The colour of numbers: Surveys, statistics and deficit-thinking about race and class. Journal of Education Policy, 25(2), 253–276. https://doi.org/10.1080/02680930903460740.

32.

Gillborn

Warmington

Demack

(2018). QuantCrit: Education policy, ‘Big Data,’ and principles for a critical race theory of statistics. Race Ethnicity and Education, 21(2), 158–179. https://doi.org/10.1080/13613324.2017.1377417.

33.

Giordano

A. L.

Schmit

M. K.

Schmit

E. L.

(2021). Best practice guidelines for publishing rigorous research in counseling. Journal of Counseling & Development, 99(2), 123–133. https://doi.org/10.1002/jcad.12360.

34.

Griffith

Mariani

McMahon

H. G.

Zyromski

Greenspan

S. B.

(2020). School counseling intervention research: A 10-year content analysis of ASCA- and ACA-affiliated journals. Professional School Counseling, 23(1), 1–12. https://doi.org/10.1177/2156759X19878700.

35.

Gruman

D. H.

Purgason

L. L.

(2019). A brief experiential school counseling site supervisor training approach. The Clinical Supervisor, 38(2), 243–261. https://doi.org/10.1080/07325223.2019.1635061.

36.

Holcomb-McCoy

Gonzalez

Johnston

(2009). School counselor dispositions as predictors of data usage. Professional School Counseling, 12(5), 343–351. https://doi.org/10.1177%2F2156759X0901200504.

37.

Howe

Eisenhart

(1990). Standards for qualitative (and quantitative) research: A prolegomenon. Educational Researcher, 19(4), 2–9. https://doi.org/10.3102/0013189X019004002.

38.

Ieva

K. P.

Beasley

Steen

(2021). Equipping school counselors for antiracist healing centered groups: A critical examination for preparation, connected curricula, professional practice and oversight. Teaching and Supervision in Counseling, 3(2), Article 7. https://doi.org/10.7290/tsc030207.

39.

Johnson

(2021). Introduction to QuantCrit for counseling researchers [webinar]. Association for Counselor Education and Supervision Diversity and Inclusion Research and Publication Subcommittee. https://www.youtube.com/watch?v=KkMB7EKQaUo.

40.

Johnson

T. P.

O’Rourke

Burris

Owens

(2002). Culture and survey nonresponse. In Groves

R. M.

Dillman

D. A.

Eltinge

J. L.

(Eds.), Survey nonresponse (pp. 55–70). Wiley.

41.

Kataoka

Novins

D. J.

Santiago

C. D.

(2010). The practice of evidence-based treatments for ethnic minority youth. Child and Adolescent Psychiatric Clinics of North America, 19(4), 775–789. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4547560/.

42.

Kuyper

B. J.

(1991). Bringing up scientists in the art of critiquing research. BioScience, 41(4), 248–250. http://www.sfu.ca/biology/courses/bisc830/Kuyper_1991.pdf.

43.

Ladson-Billings

(1999). Just what is critical race theory, and what’s it doing in a nice field like education? In Parker

Deyhle

Villenas

(Eds.), Race is...race isn’t: Critical race theory & qualitative studies in education (pp. 7–30). Avalon Publishing. https://doi.org/10.4324/9780429503504-2.

44.

Machi

L. A.

McEvoy

B. T.

(2016). The literature review: Six steps to success (3rd ed.). Sage.

45.

Matsumoto

Kim

J. J.

Grissom

R. J.

Dinnel

D. L.

(2010). Effect sizes in cross-cultural research. In Matsumoto

van de Vijver

F. J. R.

(Eds.), Cross-cultural research methods in Psychology (pp. 244–272). Cambridge University Press.

46.

Milfont

Klein

(2018). Replication and reproducibility in cross-cultural psychology. Journal of Cross-Cultural Psychology, 49(5), 735–750. https://doi.org/10.1177/0022022117744892.

47.

Mills

G. E.

Gay

L. R.

(2019). Educational research: Competencies for analysis and applications (12th ed.). Pearson.

48.

Moher

Hopewell

Schulz

K. F.

Montori

Gotzsche

P. C.

Devereaux

P. J.

Elbourne

Egger

Altman

D. G.

(2012). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. International Journal of Surgery, 10(1), 28–55. https://doi.org/10.1016/j.ijsu.2011.10.001.

49.

Mullen

P. R.

Stevens

Chae

(2019). School counselors’ attitudes toward evidence-based practices. Professional School Counseling, 22(1), 1–11. https://doi.org/10.1177/2156759X18823690.

50.

O’Hara

Chang

C. Y.

Giordano

A. L.

(2021). Multicultural competence in counseling research: The cornerstone of scholarship. Journal of Counseling and Development, 99(2), 200–209. https://doi.org/10.1002/jcad.12367.

51.

Povall

(2018, September). Data has power, so why do we fear it? New Jersey School Counselor Association Counselor News. https://www.schoolcounselor.org/Newsletters/October-2018/Data-Has-Power,-So-Why-Do-We-Fear-It?st=NJ

52.

Poynton

T. A.

DeFouw

E. R.

Morizio

L. J.

(2019). A systematic review of online response rates in four counseling journals. Journal of Counseling & Development, 97(1), 33–42. https://doi.org/10.1002/jcad.12233.

53.

Prosek

E. A.

Gibson

D. M.

(2021). Promoting rigorous research by examining lived experiences: A review of four qualitative traditions. Journal of Counseling and Development, 99(2), 167–177. https://doi.org/10.1002/jcad.12364.

54.

Ratts

M. J.

Singh

Nassar-McMillan

Butler

S. K.

McCullough

J. R.

(2016). Multicultural and social justice counseling competencies: Guidelines for the counseling profession. Journal of Multicultural Counseling and Development, 44(1), 24–48. https://doi.org/10.1002/jmcd.12035.

55.

Sablan

J. R.

(2019). Can you really measure that? Combining critical race theory and quantitative methods. American Educational Research Journal, 56(1), 178–203. https://doi.org/10.3102/0002831218798325.

56.

Sallavanti

M. I.

Berkoski

K. L.

Muse-Burke

J. L.

(2015). From idea to acceptance: Strategies for successful publishing in counseling. Journal of the Pennsylvania Counseling Association, 14(2), 55–61. http://pacounseling.org/aws/PACA/asset_manager/get_file/113396?ver=70.

57.

Sink

C. A.

Stroh

H. R.

(2006). Practical significance: The use of effect sizes in school counseling research. Professional School Counseling, 9(4), 401–411. https://doi.org/10.1177/2156759X0500900406.

58.

Smith

Beck

Bernstein

Dashtguard

(2014). Youth participatory action research and school counseling practice: A school-wide framework for student well-being. Journal of School Counseling, 12(21), 31. http://jsc.montana.edu/articles/v12n21.pdf.

59.

Solórzano

D. G.

Yosso

T. J.

(2002). Critical race methodology: Counter-storytelling as an analytical framework for education research. Qualitative Inquiry, 8(1), 23–44. https://doi.org/10.1177%2F107780040200800103.

60.

Suzuki

Morris

S. L.

Johnson

S. K.

(2021). Using QuantCrit to advance an anti-racist developmental science: Applications to mixture modeling. Journal of Adolescent Research, 36(5), 535–560. https://doi.org/10.1177/07435584211028229.

61.

Trusty

(2011). Quantitative articles: Developing studies for publication in counseling journals. Journal of Counseling and Development, 89(3), 261–267. https://doi.org/10.1002/j.1556-6678.2011.tb00087.x.

62.

Wester

K. L.

Wachter Morris

C. A.

Trustey

C. E.

Cory

J. S.

Grossman

L. M.

(2021). Promoting rigorous research using innovative qualitative approaches. Journal of Counseling and Development, 99(2), 189–199. https://doi.org/10.1002/jcad.12366.

63.

Zyromski

Dimmitt

Mariani

Griffith

(2018). Evidence-based school counseling: Models for integrated practice and school counselor education. Professional School Counseling, 21(1), 1–12. https://doi.org/10.1177/2156759X18801847.

64.

Zyromski

Mariani

M. A.

(2016). Facilitating evidence-based, data-driven school counseling: A manual for practice. Corwin.