Evaluability assessment (EA) has potential as a design option for thesis and dissertation studies, serving as a practical training experience for both technical and nontechnical evaluation skills. Based on a content review of a sample of EA theses and dissertations from graduate professional degree programs, the authors of this article found that some technical skills, such as understanding the knowledge base of evaluation, evaluation design, qualitative methods, identifying data sources, data collection, and data analysis, were demonstrated through the EA thesis and dissertation studies. However, the review also indicated a lack of understanding of EA and its essential elements and application of standards of quality evaluation. Recommendations are offered to enhance the quality of EA theses and dissertations, the learning experiences of students, and, ultimately, evaluation capacity building.
References marked with an asterisk indicate dissertations and theses included in the review.
2.
AlkinM. C.ChristieC. A. (2004). An evaluation theory tree. In AlkinM. C. (Ed.), Evaluation roots: Tracing theorists’ views and influences (pp. 12–65). Thousand Oaks, CA: Sage.
3.
AltschuldJ. W. (1995). Developing an evaluation program: Challenges in the teaching of evaluation. Evaluation and Program Planning, 18, 481–493.
4.
AltschuldJ. W. (1999). The case for a voluntary system for credentialing evaluators. American Journal of Evaluation, 20, 507–517.
5.
American Evaluation Association. (2004). American Evaluation Association guiding principles for evaluators. Retrieved fromhttp://www.eval.org
6.
American Evaluation Association. (2011). American Evaluation Association statement on cultural competence in evaluation. Retrieved fromhttp://www.eval.org
7.
AuerbachS. (2011). “It’s not just going to collect dust on a shelf”: Faculty perceptions of the applied dissertation in the new California State University (CSU) Ed.D. programs leadership education from within a feminist ethos. Journal of Research on Leadership Education, 6, 59–82.
8.
*BajwaM. (1999). An evaluability assessment of program 5C of the Winnipeg development agreement (Master’s thesis). Retrieved from WorldCat.
9.
*BellD. R. (2004). An evaluability assessment of the parent-teen initiative at Winnipeg child and family services (Master’s thesis). Retrieved from WorldCat.
10.
*Cardoso PereiraE. (2011). An evaluability assessment of a program to combat stigma to mental illness in the workplace (Master’s thesis). Retrieved from WorldCat.
11.
*CheadleT. (2003). An evaluation of the coordinated community capacity building committee: Evaluability assessment, outcome and process evaluation (Master’s thesis). Retrieved from WorldCat.
12.
CicchettiD. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Special section: Normative assessment. Psychological Assessment, 6, 284–290.
13.
DattaL. (2006). The practice of evaluation: Challenges and new directions. In ShawI. F.GreeneJ. C.MarkM. M. (Eds.), The Sage handbook of evaluation (pp. 419–438). Thousand Oaks, CA: Sage.
14.
DaviesR. (2013). Planning evaluability assessments: A synthesis of the literature and recommendations. (Report of a study commissioned by the Department of International Development, U.K. Working Paper 40), https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/248656/wp40-planning-eval-assessments.pdf
15.
DeweyJ. D.MontrosseB. E.SchroterD. C.SullinsC. D.MattoxJ. R.II (2008). Evaluator competencies: What’s taught versus what’s sought. American Journal of Evaluation, 29, 268–287.
16.
*DucharmeK. (2004). An evaluability assessment of the substance misuse initiative at Winnipeg child and family services (Master’s thesis). Retrieved from WorldCat.
17.
FitzpatrickJ. L.SandersJ. R.WorthenB. R. (2011). Program evaluation: Alternative approaches and practical guidelines (4th ed.). Upper Saddle River, NJ: Pearson.
18.
*GolumR. H. (2003). An evaluability assessment of a school-based group counseling program for African-American female students living in an Urban area of predominantly low-income status (Doctoral dissertation). Retrieved from WorldCat.
19.
*Grant-DeFiniJ. L., & Rutgers University. (2010). An evaluability assessment of an elementary school giftedness program for third through fifth grade students (Doctoral dissertation). Retrieved from WorldCat.
20.
HerrK. G.AndersonG. L. (2005). The action research dissertation: A guide for students and faculty. Thousand Oaks, CA: Sage.
21.
*JonesG. J. (2013). Integrating evaluability assessment in after school and youth programming research (Master’s thesis). Retrieved from WorldCat.
22.
JungS. M.SchubertJ. G. (1983). Evaluability assessment: A two-year retrospective. Educational Evaluation and Policy Analysis, 5, 435–444.
23.
KrathwohlD. R.SmithN. L. (2005). How to prepare a dissertation proposal: Suggestions for students in education and the social and behavioral sciences. Syracuse, NY: Syracuse University Press.
24.
*LaceyS. (2012). Implementing a sustainable program evaluation component at a large university counseling center (Doctoral dissertation). Retrieved from WorldCat.
25.
LevitonL. C.CollinsC. B.LairdB. L.KrattP. P. (1998). Teaching evaluation using evaluability assessment. Evaluation, 4, 389–409.
26.
LevitonL. C.Kettel KhanL.RogD.DawkinsN.CottonD. (2010). Evaluability assessment to improve public health policies, programs, and practices. Annual Review of Public Health, 31, 213–233.
27.
*LuchsS. C. (1999). Moving the client from Z to A: Rediscovering the utility of evaluability assessment a case study of a Girl Scout Environmental Education Center (Master’s thesis). Retrieved from WorldCat.
28.
MertensD. (1994). Training evaluators: Unique skills and knowledge. In AltschuldJ. W.EngleM. (Eds.), The preparation of professional evaluators: Issues, perspectives and programs (New Directions for Program Evaluation, 62, pp. 17–28). San Francisco, CA: Jossey-Bass.
29.
MorrisM. (1994). The role of single evaluation courses in evaluation training. In AltschuldJ. W.EngleM. (Eds.), The preparation of professional evaluators: Issues, perspectives, and programs. (New Directions for Program Evaluation, 62, pp. 51–60). San Francisco, CA: Jossey-Bass.
30.
*NaughtonM. A. (2003). The Governor’s summer reading challenge: An evaluability assessment (Master’s thesis). Retrieved from WorldCat.
31.
OwenJ. M. (2007). Program evaluation: Forms and approaches (3rd ed.). New York, NY: Guilford Press.
32.
PattonM. Q. (1990). The challenge of being a profession. Evaluation Practice, 11, 45–51.
33.
*PenroseD. (2003). An evaluability assessment of the child protection mediation pilot project (Master’s thesis). Retrieved from WorldCat.
34.
RogD. J. (1985). A methodological analysis of evaluability assessment. Dissertation Abstracts International, 46, 8B.
35.
*RufleE. A. (2009). An evaluability assessment of psychological services provided by a small rural Kansas school district (Master’s thesis). Retrieved from WorldCat.
SchmidtR. E.ScanlonJ. W.BellJ. B. (1979). Evaluability assessment: Making public programs work better (Human Services Monograph Series, No. 14). Washington, DC: Urban Institute.
38.
SmithM. F. (1989). Evaluability assessment: A practical approach. Clemson, SC: Kluwer Academic.
39.
SmithM. F. (2005). Evaluability assessment. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 136–139). Thousand Oaks, CA: Sage.
40.
StevahnL.KingJ. A.GhereG.MinnemaJ. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26, 43–59.
41.
ThurstonW. E.PotvinL. (2003). Evaluability assessment: A tool for incorporating evaluation in social change programmes. Evaluation, 9, 453–469.
42.
TrevisanM. S. (2002). Evaluation capacity in K-12 school counseling programs. American Journal of Evaluation, 23, 291–305.
43.
TrevisanM. S. (2004). Practical training in evaluation: A review of the literature. American Journal of Evaluation, 25, 255–272.
44.
TrevisanM. S. (2007). Evaluability assessment from 1986 to 2006. American Journal of Evaluation, 28, 290–303.
45.
TrevisanM. S.WalserT. M. (2014). Evaluability assessment: Improving evaluation quality and use. Thousand Oaks, CA: Sage.
46.
van der MeulenE. (2011). Participatory and action-oriented dissertations: The challenges and importance of community-engaged graduate research. The Qualitative Report, 16, 1291–1303.
47.
WholeyJ. S. (1979). Evaluation: Promise and performance. Washington, DC: The Urban Institute.
48.
WholeyJ. S. (2004). Using evaluation to improve performance and support policy decision making. In AlkinM. C. (Ed.), Evaluation roots: Tracing theorists’ views and influences (pp. 267–275). Thousand Oaks, CA: Sage.
49.
WillisJ.InmanD.ValentiR. (2010). Completing a professional practice dissertation: A guide for doctoral students and faculty. Charlotte, NC: Information Age Publishing, Inc.
50.
YarbroughD. B.ShulhaL. M.HopsonR. K.CaruthersF. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users (3rd ed.). Thousand Oaks, CA: Sage.
51.
ZamboD. (2011). Action research as a signature pedagogy in an education doctorate program: The reality and hope. Innovative Higher Education, 36, 261–271. doi:10.1007/s10755-010-9171-7
52.
Zuber-SkerrittO.FletcherM. (2007). The quality of an action research thesis in the social sciences. Quality Assurance in Education, 15, 413–436.
53.
Zuber-SkerrittO.PerryC. (2002). Action research within organisations and university thesis writing. The Learning Organization, 9, 171–179.
