Student Learning Outcome Assessment in NASPAA Programs: A Review of Validity and Reliability

Abstract

Developments in regional and professional accreditation have brought profound changes to how public affairs programs measure student learning outcomes (SLOs). There is little doubt that these developments have resulted in an increase in the frequency of SLO assessment as well as an increase in the variety of assessment mechanisms used. As programs continue to focus resources and attention on assessment, it is instructive to examine how programs are engaging in the assessment of SLOs and what best practices have emerged from these assessment approaches. This study reviews the assessment strategies and instruments utilized by programs in the 2012–2013 NASPAA accreditation cohort, focusing on the types of validation and reliability methods used by programs.

Get full access to this article

View all access options for this article.

References

Arinto

(2013). A framework for developing competencies with open and distance learning. International Review of Research in Open and Distance Learning, 14(1), 167–185.

Aristegueta

Gomes

(2006). Assessing performance in NASPAA graduate programs. Journal of Public Affairs Education, 12(1), 1–18.

Armstrong

Gosling

Weinmann

Marteau

(1997). The place of interrater reliability in qualitative research: An empirical study. Sociology, 31(3), 597–606.

Ewell

P. T.

(1994). The self-regarding institution: Information for excellence. Boulder, CO: National Center for Higher Education Management Systems.

Getha

Hummert

Nalbandian

Silva

(2013). Competency model design and assessment: Findings and future directions. Journal of Public Affairs Education, 19(1), 141–171.

Green

(1999). Building robust competencies: Linking human resources systems to organizational strategies. San Francisco, CA: Jossey-Bass.

Heise

(1969). Separating reliability and stability in test-retest correlation. American Sociological Review, 34(1), 93–101.

Jennings

(1989). Accountability, program quality, outcomes assessment, and graduate education for public affairs and administration. Public Administration Review, 42(5), 438–446.

Messick

(1990). Validity of test interpretation and use. Princeton, NJ: Education Testing Services.

10.

O’Sullivan

Rassel

Berner

(2008). Research methods for public administrators. New York, NY: Pearson.

11.

Rissi

Gelmon

(2014). Development, implementation, and assessment of a competency model for a graduate public affairs program in health administration. Journal of Public Affairs Education, 20(3), 335–332.

12.

Saint-Germain

Powell

D. C.

(2012). An analysis of standard 5 in 2012 self-studies. NASPAA white paper.

13.

Shrout

(1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7(3), 301–317.

14.

Streiner

(2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80(1), 99–103.

15.

Tompkins

Laslovish

Greene

(1996). Developing a competency-based MPA curriculum. Journal of Public Affairs Education, 2(2), 117–130.

16.

Voorhees

(2002). Competency-based learning models: A necessary future. New Directions for Institutional Research, 2001(110), 5–13.

17.

Weiss

Cosbey

Habel

Hanson

Larsen

(2002). Improving the assessment of student learning: Advancing a research agenda in sociology. Teaching Sociology, 30(1), 63–79.