The “performance movement” has been a subject of enthusiasm and frustration for evaluators. Performance measurement, data analytics, and program evaluation have been treated as different tasks, and those addressing them speak their own languages in their own circles. We suggest that situating performance measurement and data analytics within the broader field of evaluation would be theoretically parsimonious and fruitful. Scholars and practitioners of performance measurement and analytics may profitably use an evaluation mind-set and frame their tasks within the multidisciplinary field of evaluation practice. With this change in mind-set, we discuss some implications of viewing measurement, analytics, and other evaluation-related capacities within public organizations as part of an integrated, evaluation mission-support function. Working with other mission-support functions, evaluation capacity could be used by operating units to improve learning, strategy, and performance and better accomplish the mission. We outline steps that could be considered to help forge a more strategic and comprehensive approach to evaluation in public and nonprofit organizations.
American Evaluation Association (AEA). (2013). An evaluation roadmap for a more effective government. Retrieved fromhttp://www.eval.org/p/cm/ld/fid=129
4.
BernsteinD. (1999). Comments on Perrin’s ‘Effective use and misuse of performance measurement.’American Journal of Evaluation, 20, 85–93.
5.
BjornholtB.LarsonF. (2014). The politics of performance measurement: Evaluation use as mediator for politics. Evaluation, 20, 400–411.
6.
BlalockA. (1999). Evaluation research and the performance management movement: From estrangement to useful integration?Evaluation, 5, 117–149.
7.
BlalockA.BarnowB. (2001). Is the new obsession with performance management masking the truth about social programs? In ForsytheD. (Ed.), Quicker better cheaper? Managing performance in American government (pp. 485–517). Albany, NY: Rockefeller Institute Press.
8.
BussT.BussN. (2011). Four controversies in evidence-based public management. In ShillabeerA.BussT.RousseauD. (Eds.), Evidence-based public management: Practices, issues, and prospects (pp. 17–45). Armonk, NY: M.E. Sharpe.
9.
CartwrightN. (2013). Knowing what we are talking about: Why evidence doesn’t always travel. Evidence & Policy, 9, 97–112.
10.
ChelimskyE. (1997). The coming transformations in evaluation. In ChelimskyE.ShadishW. (Eds.), Evaluation for the 21st century: A handbook (pp. 18–22). Thousand Oaks, CA: Sage.
11.
ConcatoJ.ShahN.HorwitzR. (2000). Randomized, controlled trials, observational studies, and the hierarchy of research designs. New England Journal of Medicine, 342, 1887–1892.
12.
ConcatoJ.LawlerE. V.LewR. A.GazianoJ. M.AslanM.HuangG. D. (2010). Observational methods in comparative effectiveness research. American Journal of Medicine, 123, e16–e23.
13.
ConcatoJ. (2012). Is it time for medicine-based evidence?Journal of the American Medical Association, 307, 1641–1643.
14.
Congressional Research Service. (2004). General management laws: Major themes and management policy options, Report RL32388. Publicly released on World Wide Web and Retrieved fromhttp://wlstorage.net/file/crs/RL32388.pdf
15.
Congressional Research Service. (2006). Congress and program evaluation: An overview of randomized controlled trials (RCTs) and related issues, Report RL33301. Publicly released on World Wide Web and Retrieved fromhttp://digital.library.unt.edu/ark:/67531/metacrs9145/
16.
Congressional Research Service. (2011). Obama administration agenda for government performance: Evolution and related issues for Congress, Memorandum. Publicly released on World Wide Web and Retrieved fromhttp://fas.org/sgp/crs/misc/govperf.pdf
17.
Congressional Research Service. (2012). Changes to the Government Performance and Results Act (GPRA): Overview of the new framework of products and processes, Report R42379. Publicly released on World Wide Web and Retrieved fromhttp://www.fas.org/sgp/crs/misc/R42379.pdf
18.
CousinsJ.BourgeoisI. (Eds.) (2014). New directions for evaluation: Organizational capacity to do and use evaluation, vol. 141. Hoboken, NJ: Wiley Periodicals.
Department of Labor, U.S. (2013). Office of the Assistant Secretary for Policy (OASP), Chief Evaluation Office. Retrieved fromhttp://www.dol.gov/asp/evaluation/
21.
DruckerP. (1964). Managing for results: Economic tasks and risk-taking decisions. New York, NY: Harper & Row.
22.
DubnickM.FredericksonH. (2011). Public accountability: Performance measurement, the extended state, and the search for trust. Washington, DC: The Kettering Foundation.
23.
EbrahimA. (2005). Accountability myopia: Losing sight of organizational learning. Nonprofit and Voluntary Sector Quarterly, 34, 56–87.
24.
EbrahimA. (2010). The many faces of nonprofit accountability (Working Paper 10-069). Boston, MA: Harvard Business School.
25.
FazarW. (1959). Federal statistical activities: Program evaluation and review technique. The American Statistician, 13, 9–12.
26.
FellerI. (2002). Performance measurement redux. American Journal of Evaluation, 23, 435–452.
27.
FournierD. (2005). Evaluation. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 139–140). Thousand Oaks, CA: Sage.
28.
GolubR.FontanarosaP. (2012). Comparative effectiveness research: Relative successes. Journal of the American Medical Association, 307, 1643–1645.
29.
Government Accountability Office. (2009). Program evaluation: A variety of rigorous methods can help identify effective interventions. GAO-10-30. Retrieved fromhttp://www.gao.gov
30.
GAO. (2010). Employment and training administration: Increased authority and accountability could improve research program. GAO-10-243. Retrieved fromhttp://www.gao.gov
31.
GAO. (2011a). Government auditing standards: 2011 revision. GAO-12-331G. Retrieved fromhttp://www.gao.gov/yellowbook
32.
GAO. (2011b). Performance measurement and evaluation: Definitions and relationships. GAO-11-646SP. Retrieved fromhttp://www.gao.gov
33.
GAO. (2011c). Program evaluation: Experienced agencies follow a similar model for prioritizing research. GAO-11-176. Retrieved fromhttp://www.gao.gov
GAO. (2013). Managing for results: Data-driven performance reviews show promise but agencies should explore how to involve other relevant agencies. GAO-13-228. Retrieved fromhttp://www.gao.gov
36.
GreeneJ. (1999). The inequality of performance measurements. Evaluation, 5, 160–172.
37.
HaberS. (1964). Efficiency and uplift: Scientific management in the progressive era, 1890-1920. Chicago, IL: University of Chicago Press.
HatryH. (2013). Sorting the relationships among performance measurement, program evaluation, and performance management. In NielsenS.HunterD. (Eds.), New directions for evaluation: Performance management and evaluation (Vol. 137, pp. 19–32). Hoboken, NJ: Wiley Periodicals.
41.
HatryH.WholeyJ.NewcomerK. (2010). Evaluation challenges, issues, and trends. In WholeyJ.HatryH.NewcomerK. (Eds.), Handbook of practical program evaluation (3rd ed., pp. 668–679). San Francisco, CA: Jossey-Bass.
42.
HeinrichC.MarschkeG. (2010). Incentives and their dynamics in public sector performance management systems. Journal of Policy Analysis and Management, 29, 183–208.
43.
HeinrichC.LynnL.MilwardH. (2010). A state of agents? Sharpening the debate and evidence over the extent and impact of the transformation of governance. Journal of Public Administration Research and Theory, 20, 3–19.
44.
HeinrichC. (1999). Do government bureaucrats make effective use of performance management information?Journal of Public Administration Research and Theory, 9, 363–393.
45.
HeinrichC. (2004a). Improving public-sector performance management: One step forward, two steps back?Public Finance and Management, 4, 317–351.
46.
HeinrichC. (2004b). Performance management as administrative reform: Is it improving government performance?Public Finance and Management, 4, 240–246.
47.
HeinrichC. (2007). Evidence-based policy and performance management: Challenges and prospects in two parallel movements. The American Review of Public Administration, 37, 255–277.
48.
HeinrichC. (2012). How credible is the evidence, and does it matter? An analysis of the program assessment rating tool. Public Administration Review, 72, 123–134.
49.
HoodC. (2006). Gaming in targetworld: The targets approach to managing public services. Public Administration Review, 66, 515–521.
50.
HoodC. (2012). Public management by numbers as a performance-enhancing drug: Two hypotheses. Public Administration Review, 72, 585–592.
51.
HunterD.NielsenS. (2013). Performance management and evaluation: Exploring complementarities. In NielsenS.HunterD. (Eds.), New directions for evaluation: Performance management and evaluation (Vol. 137, pp. 115–123). Hoboken, NJ: Wiley Periodicals.
52.
JacobsR.GoddardM. (2007). How do performance indicators add up? An examination of composite indicators in public services. Public Money & Management, 27, 103–110.
53.
KarolyL. (2012). Toward standardization of benefit-cost analysis of early childhood interventions. Journal of Benefit-Cost Analysis, 3, 1–43.
LabinS. (2008). Research synthesis: Toward broad-based evidence. In SmithN.BrandonP. (Eds.), Fundamental issues in evaluation (pp. 89–110). New York, NY: Guilford Press.
56.
MathisonS. (2011). Internal evaluation, historically speaking. In VolkovB.BaronM. (Eds.), New directions for evaluation: Internal evaluation in the 21st century (Vol. 132, pp. 13–23). Hoboken, NJ: Wiley Periodicals.
57.
MayneJ. (2007). Evaluating for accountability: Myth or reality. In Bemelmans-VidecM.LonsdaleJ.PerrinB. (Eds.), Making accountability work: Dilemmas for evaluation and for audit (pp. 63–84). New Brunswick, NJ: Transaction.
58.
McDavidJ.HawthornL. (2006). Program evaluation & performance measurement: An introduction to practice. Thousand Oaks, CA: Sage.
59.
Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI). (2012). Methodological standards and patient-centeredness in comparative effectiveness research: The PCORI perspective. Journal of the American Medical Association, 307, 1636–1640.
60.
MintzbergH. (1985). Of strategies, deliberate and emergent. Strategic Management Journal, 6, 257–272.
61.
MintzbergH. (1994). The rise and fall of strategic planning: Reconceiving roles for planning, plans, planners. New York, NY: Free Press.
62.
MoynihanD. (2008). The dynamics of performance management: Constructing information and reform. Washington, DC: Georgetown University Press.
63.
MoynihanD. (2009). How do public organizations learn? Bridging cultural and structural perspectives. Public Administration Review, 69, 1097–1105.
64.
MoynihanD. (2011). The big question for performance management: Why do managers use performance information?Journal of Public Administration Research and Theory, 20, 849–866.
65.
MoynihanD.LavertuS. (2012) Does involvement in performance management routines encourage performance information use? Evaluating GPRA and PART. Public Administration Review, 72, 592–602.
66.
NewcomerK. (Ed.) (1997). New directions for program evaluation: Using performance measurement to improve public and nonprofit programs, vol. 75. San Francisco, CA: Jossey-Bass.
67.
NewcomerK. (2007). How does program performance assessment affect program management in the federal government?Public Performance and Management Review, 30, 332–350.
68.
NewcomerK. (2010). Putting performance first: A new performance improvement and analysis framework. In BreulJ.NewcomerK.GoldmanJ. P.PosnerP. L.SchoonerS. (Eds.), Framing a public management research agenda (pp. 7–16). Washington, DC: IBM Center for the Business of Government. Retrieved fromhttp://www.businessofgovernment.org/report/framing-public-management-research-agenda
69.
NewcomerK.BrassC. (2013, 3). Reconceiving ‘performance management’: Situating performance measurement within program evaluation, and program evaluation as a mission-support function. Paper presented at the 2013 international symposium on The frontier of performance management: The interplay of theory and practice, American Society for Public Administration 2013 Annual Conference, New Orleans, LA, pp. 1–48.
70.
NewcomerK.CaudleS. (2011). Public performance management systems: Embedding practices for improved success. Public Performance and Management Review, 35, 108–132.
71.
NewcomerK.RedburnF. (2009). Driving improvements in federal policy outcomes: Lessons for a new president. In ShillabeerA.BussT.RousseauD. (Eds.), Evidence-based public management: Practices, issues, and prospects (pp. 259–268). Armonk, NY: M.E. Sharpe.
72.
NewcomerK. E.RedburnF. S. (2009). Achieving Real Improvement in Federal Policy and Program Outcomes: The Next Frontier (A Research Report). Washington, DC: National Academy of Public Administration.
73.
NielsenS.EjlerN. (2008). Improving performance? Exploring the complementarities between evaluation and performance management. Evaluation, 14, 171–192.
74.
NielsenS.HunterD. (Eds.) (2013a). New directions for evaluation: Performance management and evaluation, vol. 137. Hoboken, NJ: Wiley Periodicals.
75.
NielsenS.HunterD. (2013b). Challenges to and forms of complementarity between performance management and evaluation. In NielsenS.HunterD. (Eds.), New directions for evaluation: Performance management and evaluation (Vol. 137, pp. 115–123). Hoboken, NJ: Wiley Periodicals.
PattonM. (2011b). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York, NY: The Guilford Press.
85.
PawsonR. (2013). The science of evaluation: A realist manifesto. Thousand Oaks, CA: Sage.
86.
PerrinB. (1998). Effective use and misuse of performance measurement. American Journal of Evaluation, 19, 367–379.
87.
PerrinB. (1999). Performance measurement: Does the reality match the rhetoric? A rejoinder to Bernstein and Winston. American Journal of Evaluation, 20, 101–111.
88.
PerrinB. (2007). Towards a new view of accountability. In Bemelmans-VidecM.LonsdaleJ.PerrinB. (Eds.), Making accountability work: Dilemmas for evaluation and for audit (pp. 41–58). New Brunswick, NJ: Transaction.
89.
RadinB. (2006). Challenging the performance movement. Washington, DC: Georgetown University Press.
90.
RadinB. (2009). What can we expect from performance measurement activities?Journal of Policy Analysis and Management, 28, 505–512.
91.
RengerR.TitcombA. (2002). A three-step approach to teaching logic models. American Journal of Evaluation, 23, 493–503.
92.
RistR. (2006). The “E” in monitoring and evaluation—Using evaluative knowledge to support a results-based management system. In RistR.StameN. (Eds.), From studies to streams: Managing evaluative systems (pp. 3–21). New Brunswick, NJ: Transaction.
93.
RobertsA.YeagerK. (Eds.). (2004). Evidence-based practice manual: Research and outcome measures in health and human services. Oxford, England: Oxford University Press.
94.
RogersP. (2005). Logic model. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 232–235). Thousand Oaks, CA: Sage.
95.
SchwandtT. (2005). Values. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 443–444). Thousand Oaks, CA: Sage.
96.
ShadishW.LuellenJ. (2005). History of evaluation. In MathisonS. (Ed.), Encyclopedia of evaluation (pp. 183–186). Thousand Oaks, CA: Sage.
97.
SimonH. (1976). Administrative behavior: A study of decision-making processes in administrative organization. 3rd ed. New York, NY: Free Press.
98.
TalbotC. (2010). Theories of performance: Organizational and service improvement in the public domain. Oxford, England: Oxford University Press.
99.
Van DoorenW.BouckaertG.HalliganJ. (2010). Performance management in the public sector. London, England: Routledge.
100.
VolkovB.BaronM. (Eds.). (2011). New directions for evaluation: Internal evaluation in the 21st century, vol. 132. Hoboken, NJ: Wiley Periodicals.
101.
WeissC. (1997). Theory-based evaluation: Past, present, and future. In FournierD.RogD. J. (Eds.), New Directions for Evaluation: Progress and future directions in evaluation: Perspectives on theory, practice, and methods, vol. 76 (pp. 41–55). San Francisco, CA: Jossey-Bass.
102.
WeissC. (2004). Rooting for evaluation: A cliff notes version of my work. In AlkinM. (Ed.), Evaluation roots: Tracing theorists’ views and influences (pp. 153–168). Thousand Oaks, CA: Sage.
103.
WestW. (2011). Program budgeting and the performance movement: The elusive quest for efficiency in government. Washington, DC: Georgetown University Press.
104.
WilliamsD. (2002). Before performance measurement. Administrative Theory & Praxis, 24, 457–486.
105.
WinstonJ. (1999). Performance indicators—promises unmet: A response to Perrin. American Journal of Evaluation, 20, 95–99.
