Does My Program Really Make a Difference? Program Evaluation Utilizing Aggregate Single-Subject Data

Abstract

In the current climate of increasing fiscal and clinical accountability, information is required about overall program effectiveness using clinical data. These requests present a challenge for programs utilizing single-subject data due to the use of highly individualized behavior plans and behavioral monitoring. Subsequently, the diversity of the client-specific programming does not yield data that are easily aggregated to inform program evaluations for effectiveness. This issue was addressed in one New England community mental health center providing school-based treatment for youth with significant emotional and behavioral disorders. After developing a behavior categorization system, behavior analysts reported client-specific behavioral progress into a main database. Aggregate data were then analyzed to provide information about overall program effectiveness using two reporting measures. The utility of aggregated single-subject data to inform clinical practice, clients, and program stakeholders is discussed.

Keywords

program evaluation single-subject design applied behavior analysis

Get full access to this article

View all access options for this article.

References

Achenbach

T. M.

(2009). The Achenbach system of empirically based assessment (ASEBA): Development, findings, theory, and applications. Burlington: University of Vermont Research Center for Children, Youth and Families.

Cooper

Heron

Heward

(1987). Applied behavior analysis. Pennington, NJ: Merrill.

Fisher

W. W.

Piazza

C. C.

Roane

H. S.

(Eds.). (2011). Handbook of applied behavior analysis. New York, NY: Guilford Press.

Hendricks

Plantz

M. C.

Pritchard

K. J.

(2008). Measuring outcomes of united way–funded programs: Expectations and reality. In Carman

J. G.

Fredericks

K. A.

(Eds.), Nonprofits and evaluation. New directions for evaluation (Vol. 119, pp. 13–35). San Francisco, CA: Jossey-Bass.

Honoré

Simoes

Moonesinghe

Kirbey

Renner

(2004). Applying principles for outcomes-based contracting in a public health program. Journal of Public Health Management Practices, 10, 451–457.

Kazdin

(2011). Single-case research designs (2nd ed.). New York, NY: Oxford University Press.

Matson

J. L.

(Ed.). (2009). Applied behavior analysis for children with autism spectrum disorders. New York, NY: Springer Science and Business Media.

Rosenbaum

(2011). The patient protection and affordable care act: Implications for public health policy and practice. Public Health Report, 126, 130–135.

U. S. Congress. The patient protection and affordable care act (PPACA). Pub. L. No. 111-148, 124 stat. 119. Retrieved September 2013 from http://www.gpo.gov/fdsys/pkg/PLAW-111publ148/pdf/PLAW-111publ148.pdf

10.

U. S. Congress. The Paul Wellstone and Pete Domenici mental health and addiction equity act of 2008. H. R. 1424-117. Retrieved from https://www.cms.gov/Regulations-and-Guidance/Health-Insurance-Reform/HealthInsReformforConsume/downloads/MHPAEA.pdf

11.

Van Biervliet

(1979). Program evaluation research; its importance for applied behavior analysis and its basic components. The Behavior Therapist, 2, 3–7.