The Office of Management and Budget has recommended the termination of numerous federal programs, citing a lack of program results as the primary reason for this decision. In response to this recommendation, several federal agencies have turned to logic modeling to demonstrate that programs are on the path to results accountability. However, approaches to logic modeling in some agencies have not followed the strategies that evaluators recommend as necessary to lead to a high quality logic model. Models of poor quality are unlikely to contribute to improved program accountability or better program results. In this article, the author assesses the quality of the logic-modeling approach taken by one agency to illustrate how a flawed approach to logic modeling may lead to incorrect conclusions about programs and about the benefits of logic models. As a result of this analysis, the author questions the conditions under which capacity building should be considered and whether the field of evaluation needs to be more judicious when mainstreaming methodologies.
Azer, S. A., Simmons, D., & Elliott, S. L. (2001). Rural training and the state of rural health services: Effect of rural background on the perception and attitude of first-year medical students at the university of Melbourne. Australian Journal of Rural Health, 9, 178-185.
2.
Bush, G. W. (2005, February 7). State of the union address. Available at http://www.whitehouse.gov/news/releases/2005/02/20050202-11.html
3.
Chen, H. T. (1989). The conceptual framework of the theory-driven perspective. Evaluation and Program Planning, 12, 391-396.
4.
Chen, H. T., & Rossi, P. H. (1983). Evaluating with sense: The theory-driven approach. Evaluation Review, 7(3), 283-302.
5.
den Hayer, M. (2002). The temporal logic model concept. Canadian Journal of Program Evaluation, 17(2), 27-47.
6.
Gale, J., Loux, S., & Coburn, A. (2006). Creating program logic models: A toolkit for state Flex programs. Portland: University of Southern Maine, Flex Monitoring Team.
7.
Goertzen, J. R., Fahlman, S. A., Hampton, M. R., & Jeffery, B. L. (2003). Creating logic models using grounded theory: A case example demonstrating a unique approach to logic model development. Canadian Journal of Program Evaluation, 18(2), 115-138.
8.
Greene, L. W., & Kreuter, M. W. (1999). Health promotion planning: An educational and ecological approach (3rd ed.). New York: McGraw-Hill.
9.
Health Resource and Services Administration. (2005). Bureau of health professions: First all-grantee conference. Performance Measurement System. Washington, DC: Department of Health and Human Services.
10.
Huntington, C., & Renger, R. (2003, April-June). NAO'sresponse to the OMB—PART report. NAO News of the National AHEC Organization. Retrieved from http://www.nationalahec.org/Publications/documents/Response%20to%20the%20OMB%20Performance%20and%20Management%20Assessment%20of%20the%20Health%20Professions%20Program%202003.pdf%23search=%22huntington%20renger%22.
11.
Joint Committee on Standards for Educational Evaluation (1994). The program evaluation standards: How to assess evaluations of educational programs (2nd ed.). Thousand Oaks, CA: Sage.
12.
Lynch, D. C., Teplin, S. E., Willis, S. E., Pathman, D. E., Larsen, L. C., Steiner, B. D., et al. (2001). Interim evaluation of the rural scholars program. Teaching and Learning in Medicine, 13(1), 36-42.
13.
Mark, M. M., Henry, G. T., & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving policies and programs. San Francisco, CA: Jossey-Bass.
14.
McLaughlin, J. A., & Jordan, G. B. (1999). Logic models: A tool for telling your program'sperformance story. Evaluation and Program Planning, 22, 65-72.
15.
Millar, A., Simeone, R. S., & Carnevale, J. T. (2001). Logic models: A systems tool for performance management. Evaluation and Program Planning, 24, 73-81.
16.
Office of Management and Budget. (2004a). Program: Health professions. Available at http://www.whitehouse.gov/omb/budget/fy2004/pma/healthprofessions.pdf
17.
Office of Management and Budget. (2004b). Rating the performance of federal programs. Available at http://www.gpoaccess.gov/usbudget/fy04/pdf/budget/performance.pdf
18.
Office of Management and Budget. (2005). Budget Procedures Memorandum No. 861. Available at http://www.whitehouse.gov/omb/budget/fy2005/pdf/bpm861.pdf
19.
Office of Management and Budget. (2006). Performing programs. Available at http://www.whitehouse.gov/omb/expectmore/perform.html
20.
Renger, R. (2003a). The need for a deeper level of accountability—And how we might get there. National AHEC Bulletin, 19(2), 10-13.
21.
Renger, R. (2003b). A standardized tool (vendor) for collecting data. National AHEC Bulletin, 19(2), 16-17.
22.
Renger, R. (2003c). Strategic directions for improving documentation of AHEC/HETC accomplishments. National AHEC Bulletin, 19(2), 14-16.
23.
Renger, R. (2005a). Application of logic models to the Hawaii AHEC. Honolulu, HI.
24.
Renger, R. (2005b). Logic modeling. State Offices of Rural Health (SORH) pre-meeting. Washington, DC: Health and Resources Services Administration, Office of Rural Health Policy, all programs meeting.
25.
Renger, R. (2005c). Logic modeling for Moloka'i Substance Abuse Program. Kaunakaka, HI.
26.
Renger, R. (2005d). National Health Outcomes Working Group. Presentation at the Health Resources and Services Administration all grantee meeting, Washington, DC.
27.
Renger, R., & Bourdeau, B. (2004). Strategies for values inquiry: An exploratory case study. American Journal of Evaluation, 25, 39-49.
28.
Renger, R., & Hurley, C. (2006). From theory to practice: Lessons learned in the application of the ATM approach to developing logic models. Evaluation and Program Planning, 29(2), 106-119.
29.
Renger, R., & Titcomb, A. (2002). A three-step approach to teaching logic models. American Journal of Evaluation, 23, 493-503.
30.
Spath, P. (2003). Don't get caught in the activity trap. Hospital Peer Review, 28(9), 130-132.
31.
Trochim, W.M.K. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12, 1-16.
32.
United Way of America. (1996). Measuring program outcomes: A practical approach. Alexandria, VA: Author.
33.
Weiss, C. H. (1997). How can theory-based evaluation make even greater headway?Evaluation Review, 21(4), 501-524.
34.
W. K. Kellogg Foundation (2001). Logic model development guide. Available at http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
35.
Xu, G., & Veloski, J. J. (1998). Debt and primary care physicians? career satisfaction. Academic Medicine, 73(2), 119-124.
