Reasons for unintended consequences of program action are explained. Although this topic has been dealt with extensively in the field of planning, that literature provides little guidance for how evaluation methodologies can be improved. Implications for evaluation are provided by drawing the distinction between consequences that are unforeseen and consequences that are unforeseeable and then recommending tactics for dealing with each.
Adner, R. (2004). A demand-based perspective on technology life cycles. Business Strategy Over the Industry Life Cycle: Advances in Strategic Management, 21, 25-43.
2.
Behrens, T. R., & Gray, D. O. (2001). Unintended consequences of cooperative research: Impact of industry sponsorship on climate for academic freedom & other graduate student outcome. Research Policy, 30(2), 179-199.
3.
Birkeland, S., Murphe-Grahm, E., & Weiss, C. (in press). Good reasons for ignoring good evaluation: The case of the Drug Abuse Resistance Education (D.A.R.E.) program. Evaluation and Program Planning.
4.
Boehm, B., & Egyed, A. (1999). Optimizing software product integrity through life-cycle process integration. Computer Standards and Interfaces, 21, 63-75.
5.
Courtney, M. E., Needell, B., & Wulczyn, F. (2004). Unintended consequences of the push for accountability: The case of national child welfare performance standards. Children and Youth Services Review, 26(12), 1141-1154.
6.
Dannemiller Tyson Associates. (2000). Whole-scale change: Unleashing the magic in organizations. San Francisco, CA: Berrett-Koehler.
7.
Detert, J. R., Schroeder, R. G., & Cudeck, R. (2003). The measurement of quality management culture in schools: Development and validation of the SQMCS. Journal of Operations Management, 21(3), 307-328.
8.
Dewar, J. A. (2002). Assumption based planning: A tool for reducing avoidable surprises. New York: Cambridge University Press.
9.
DiNardo, J., & Lemieux, T. (2001). Alcohol, marijuana, and American youth: The unintended consequences of government regulation. Journal of Health Economics, 20(6), 991-1010.
10.
Dorner, D. (1996). The logic of failure. New York: Henry Holt.
11.
Dreborg, K. H. (1996). The essence of backcasting. Futures, 28(9), 813-828.
12.
Flamholtz, E., & Hua, W. (2002). Strategic organizational development, growing pains & corporate financial performance: An empirical test. European Management Journal, 20(5), 527-536.
13.
Fetterman, D. M., & Wandersman, A. (2005). Empowerment evaluation principles in practice. New York: Guilford.
14.
Firestone, W. A., Schorr, R. Y., & Monfils, L. F. (2004). The ambiguity of teaching to the test. Mahwah, NJ: Lawrence Erlbaum.
15.
Fleischer, M., & Liker, J. K. (1997). Concurrent engineering effectiveness: Integrating product development across organizations. Cincinnati, OH: Hanser Gardner.
16.
Fry, M., & Polonsky, J. (2004). Examining the unintended consequences of marketing. Journal of Business Research, 57(11), 1303-1306.
17.
Fullerton, R. R., & McWatters, C. S. (2001). The production performance benefits from JIT implementation. Journal of Operations Management, 19(1), 81-96.
18.
Garver, M. S. (2003). Best practices in identifying customer-driven improvement opportunities. Industrial Marketing Management, 32(6), 455-466.
19.
Gilmore, W. S., & Camillus, J. C. (1996). Do your planning processes meet the reality test?Long Range Planning, 29(6), 869-879.
20.
Ginsburg, L. S. (2001). Total quality management in health care: A goal setting approach. Advances in Health Care Management, 2, 265-290.
21.
Godet, M. (2000). The art of scenarios & strategic planning: Tools and pitfalls. Technological Forecasting and Social Change, 65, 3-22.
22.
Gupta, U. G., & Clarke, R. E. (1996). Theory and applications of the Delphi technique: A bibliography (1975-1994). Technological Forecasting and Social Change, 53(2), 185-211.
23.
Gupta, Y. P., & Chin, D.C. W. (1993). Strategy making & environment: An organizational life cycle perspective. Technovation, 13(1), 27-44.
24.
Hoek, J. (2004). Tobacco promotion restrictions: Ironies and unintended consequences. Journal of Business Research, 57(11), 1250-1257.
25.
Huffman, D., & Kalnin, J. (2003). Collaborative inquiry to make data-based decisions in schools. Teaching and Teacher Education, 19(6), 569-580.
26.
Kaminski, M. (2001). Unintended consequences: Organizational practices and their impact on workplace safety and productivity. Journal of Occupational Health Psychology, 6(2), 127-138.
27.
Kaplan, S. A., & Garrett, K. E. (2005). The use of logic models by community-based initiatives. Evaluation and Program Planning, 28(2), 167-172.
28.
Kauffman, S. (1995). At home in the universe: The search for the laws of self organization and complexity. New York: Oxford University Press.
29.
Keeney, S., Hasson, F., & McKenna, H. P. (2001). A critical review of the Delphi technique as a research methodology for nursing. International Journal of Nursing Studies, 38(2), 195-200.
30.
Kildegaard, A. (2001). Fiscal reform, bank solvency, and the law of unintended consequences: A CGE analysis of Mexico. North American Journal of Economics and Finance, 12(1), 55-77.
31.
Koberg, C. E., Uhlenbruck, N., & Sarason, Y. (1996). Facilitators of organizational innovation: The role of life cycle stage. Journal of Business Venturing, 11, 133-149.
32.
Lambe, C. J., & Spekman, R. E. (1997). Alliances, external technology acquisition, and discontinuous technological change. Journal of Product Innovation Management, 14, 102-116.
33.
Lee, H. L., & Whang, S. (2005). Higher supply chain security with lower cost: Lessons from total quality management. International Journal of Production Economics, 96, 289-300.
34.
Lindahl, F. W., & Fanelli, R. (2002). Applying continuous improvement to teaching in another culture. Journal of Accounting Education, 20(4), 285-295.
35.
Lukas, B. A., & Menon, A. (2004). New product quality: Intended and unintended consequences of new product development speed. Journal of Business Research, 57(11), 1258-1264.
36.
Marion, R. (1999). The edge of organization: Chaos and complexity theories of formal social systems. Thousand Oaks, CA: Sage.
37.
Mathie, A., & Greene, J. C. (1997). Stakeholder participation in evaluation: How important is diversity?Evaluation and Program Planning, 20(3), 279-285.
38.
Morell, J. A. (2000). Internal evaluation: A synthesis of traditional methods and industrial engineering. American Journal of Evaluation, 1, 41-52.
39.
Morell, J. A. (2003). Evaluating the impact of an electronic business system in a complex organizational setting: The case of Central Contractor Registration. Evaluation and Program Planning, 26, 429-440.
40.
Morell, J. A. (2004, January-April). Evaluating the impact of electronic business systems: Lessons learned from three cases at the Defense Logistics Agency. Acquisition Review Quarterly, pp. 3-20.
41.
Morgan, G. (1997) Images of organization. Thousand Oaks, CA: Sage.
42.
Newby, P. (1999). Culture and quality in higher education. Higher Education Policy, 12(3), 261-275.
43.
O’Brien, F. A. (2003). Scenario planning—Lessons for practice from teaching and learning. European Journal of Operational Research, 152, 709-722.
Pillai, A. S., Joshi, A., & Rao, K. S. (2001). Performance measurement of R&D projects in a multi-project, concurrent engineering environment. International Journal of Project Management, 20, 165-177.
46.
Plott, R. C., & Chen, K. (2002). Information aggregation mechanisms: Concept, design and implementation for a sales forecasting problem (Social Sciences Working Paper 1131). Pasadena: California Institute of Technology, Division of Humanities and Social Sciences.
47.
Pugh, K. J., & Zhao, Y. (2003). Stories of teacher alienation: A look at the unintended consequences of efforts to empower teachers. Teaching and Teacher Education, 19(2), 187-201.
48.
Ryan, J., Stone, R. I., & Raynor, C. R. (2004). Using large data sets in long-term care to measure and improve quality. Nursing Outlook, 52(1), 38-44.
49.
Sherman, D. J., & Olsen, E. A. (1996). Stages in the project life cycle in R&D organizations and the differing relationship between organizational climate and performance. Journal of High Technology Management Research, 7(1), 79-90.
50.
Snowberg, E., Wolfers, J., & Zitzewitz, E. (2005). Information (in)efficiency in prediction markets In L. V. Williams (Ed.), Information efficiency in financial and betting markets (chap.15). New York: Cambridge University Press.
51.
Sternberg, R. J. (2004, October 27). Good intentions, bad results: A dozen reasons why the No Child Left Behind Act is failing our schools. Education Week. Retrieved from http://www.edweek.org/ew/articles/2004/10/27/09sternberg.h24.html
52.
Stiles, K. (2002). International support for NGOs in Bangladesh: Some unintended consequences. World Development, 30(5), 835-846.
53.
Surowiecki, J. (2004). The wisdom of crowds. New York: Doubleday.
54.
Sutherland, S. (2004). Creating a culture of data use for continuous improvement: A case study of an Edison Project school. American Journal of Evaluation, 25, 277-293.
55.
Tenner, E. (1996). Why things bite back. New York: Knopf.
56.
Tian, J. (1999). Measurement and continuous improvement of software reliability throughout software life-cycle. Journal of Systems and Software, 47(2-3), 189-195.
57.
Van de Ven, A. H., & Poole, S. M. (1995). Explaining development and change in organizations. Academy of Management Review, 20(3), 510-540.
58.
van Harten, W. H., Casparie, T. F., & Fisscher, O.A. M. (2000). Methodological considerations on the assessment of the implementation of quality management systems. Health Policy, 54(3), 187-200.
59.
Wolfers, J., & Zitzewitz, E. (2004). Prediction markets. Journal of Economic Perspectives, 18(2), 107-126.
