Stakeholder Collaboration in the Design and Implementation of a Family Literacy Portfolio Assessment

Abstract

Collaborative, participatory, and empowerment forms of evaluation advocate the inclusion of stakeholders in decision-making roles in the evaluation process; however; little in the literature describes the involvement of stakeholders in the design and implementation of evaluative tools for data collection. This case study describes the collaborative process and the lessons learned when the staff of a family literacy program and an evaluator collaborated to design and implement a portfolio assessment that was used to collect program evaluation information over a two-year period. The program, Even Start, provides integrated education and human services to meet the literacy needs of participants. The evaluation offered opportunities to collaborate with stakeholders-the program coordinator and family educators-in each step of the creation and implementation of the portfolio system.

Get full access to this article

View all access options for this article.

References

Advanced Systems in Measurement and Evaluation (1993). Kentucky Instructional Results Information system: 1991-1992 Technical Report. Author.

Barrick, C. & Cogliano, J. (1993). Stakeholder involvement: Mythology or methodology? Evaluation Practice, 14(1), 33-37.

Coons, C. , Gay, E. , Fandal, A. , Ker, C. & Frankenburg, W. (1981). The Home Screening Questionnaire reference manual. J.F.K. Child Development Center: Denver.

Cousins, J. , Donohue, J. , & Bloom, G. (1996). Collaborative evaluation in North America: Evaluators’ self-reported opinions, practices, and consequences, Evaluation Practice, 17(3), 207-226.

Fetterman, D. M. (1994). Empowerment evaluation. Evaluation Practice, 15(1), 1-15.

Green, B. , Mulvey, L. , Fisher, H. , & Woratschek, F. (1996). Integrating program and evaluation values: A family support approach to program evaluation. Evaluation Practice, 17(3), 261-272.

Greene, J. (1987). Stakeholder participation in evaluation design: Is it worth the effort? Evaluation and Program Planning, 10, 379-394.

Impara, J. , Plake, B. , & Fager, J. (1993). Teachers’ assessment background and attitudes toward testing. Theory into Practice, 32(2) 113-117.

Johnson, R. , Willeke, M. , Bergman, T. , & Steiner, D. (1996). Collaboration to create a family literacy portfolio system to assess program impact: Year two. Paper presented at the annual meeting of the American Evaluation Association, Atlanta.

10.

Johnson, R. , Willeke, M. , Bergman, T. & Steiner, D. (1997). Family literacy portfolios: Development and implementation. Window on the World of Family Literacy, 2(2), 10-17.

11.

Joint Committee on Standards for Educational Evaluation (1994). The program evaluation standards (2nd ed.). Thousand Oaks, CA: Sage.

12.

Knapp, M. (1995). How shall we study comprehensive, collaborative services for children and families? Educational Researcher, 24(4), 5-16.

13.

Meisels, S. (1993, July). Remaking classroom assessment with the Work Sampling System. Young Children, pp. 34-40.

14.

Moss, P. , Beck, J. , Ebbs, C. , Matson, B. , Muchmore, J. , Steele, D. , Taylor, C. , & Herter, R. (1992). Portfolios, accountability, and an interpretive approach to validity, Educational Measurement: Issues and Practice, 11 (3), 12-21.

15.

National Association for the Education of Young Children (1991). Early Childhood Classroom Observation: National Academy of Early Childhood Programs. Washington, DC.

16.

O’Sullivan, R. (1995). From judges to collaborators: Evaluators’ roles in science curriculum reform. In R.G. O’Sullivan (Ed.), Emerging Roles of Evaluation in Science Education Reform, New Directions for Program Evaluation, no. 65, 19-29. Jossey-Bass: San Francisco.

17.

Popp, R. (1992). Family Portfolios: Documenting Change in Parent-Child Relationships. Louisville, KY: National Center for Family Literacy.

18.

Rockwell, S. & Buck, J. (1995). An interdisciplinary and interagency evaluation team: Benefits for other-discipline specialists. Evaluation Practice, 16(3), 239-245.

19.

St. Pierre, R. , Swartz, J. , Gamse, B. , Murray, S. , Deck, D. & Nickel, P. (1995). National evaluation of the Even Start Family Literacy Program: Final report. Cambridge, MA: Abt Associates.

20.

St. Pierre, R. , Swartz, J. , Murray, S. , Deck, D. & Nickel, P. (1993). National evaluation of the Even Start Family Literacy Program: Report on effectiveness. Cambridge, MA: Abt Associates.

21.

Van Horn, B. , & Guaraldi, R. (1994). Alternative assessment: Use of portfolio assessment in a work-place literacy program. In B.L. Hayes & K. Camperell (Eds.) Yearbook of the American Reading Forum, Volume XIV. pp. 140-151. Logan, UT: American Reading Forum.