A multistakeholder evaluation procedure is presented to address the many challenges in evaluating the performance of condominial sewer projects in Brazil. Condominial sewerage is a promising appropriate technology that is coproduced by users and public agencies, but little is known about project performance. This article shows that multistakeholder evaluation is more appropriate to the research setting than evaluations based on a single stakeholder’s perspective, is more reliable than the use of expert judgments alone, and provides more information than recommended sewer performance indicators. The author argues that the perspectives of multiple stake-holders should be included in project evaluations. The multistakeholder approach presented is a workable solution to the condominial sewer evaluation problem.
Abbott, J. (1996). Sharing the city: Community participation in urban management. London: Earthscan.
2.
American Society of Civil Engineers. (1994). Existing sewer evaluation and rehabilitation (2nd ed.). Reston, VA: Author.
3.
American Society of Civil Engineers. (1998). Report card for America’s infrastructure. Retrieved November 23, 2004, from http://www.asce.org/reportcard
4.
Bennett, A. (1999). Causal inference in case studies: From Mill’s methods to causal mechanisms. Retrieved November 19, 2004, from http://www.georgetown.edu/faculty/bennetta/APSA99.html
5.
Bernard, H. R. (1994). Research methods in anthropology: Qualitative and quantitative approaches (2nd ed.). Thousand Oaks, CA: Sage.
6.
Birckmayer, J. D., & Weiss, C. H. (2000). Theory-based evaluation in practice: What do we learn?Evaluation Review, 24(4), 407-431.
7.
Brown, K., & Coulter, P. B. (1983). Subjective and objective measures of police service delivery. Public Administration Review, 43(1), 50-58.
8.
Brudney, J. L., & England, R. E. (1982). Urban policy making and subjective service evaluations: Are they compatible?Public Administration Review, 42(2), 127-135.
9.
Chelimsky, E., & Shadish, W. (Eds.).(1997). Evaluation for the 21st century: A handbook. Thousand Oaks, CA: Sage.
10.
Fetterman, D. (1997). Empowerment evaluation: A response to Patton and Scriven. Evaluation Practice, 18(3), 253-266.
11.
George, A., & Bennett, A. (2005). Case studies and theory development in the social sciences. Cambridge, MA: MIT Press.
12.
Heller, L. (1999). Who really benefits from environmental sanitation services in the cities? An intra-urban analysis in Betim, Brazil. Environment and Urbanization, 11(1), 133-144.
13.
Hinton, P. R. (1995). Statistics explained: A guide for social science students. London: Routledge.
14.
Huck, S. W. (2000). Reading statistics and research (3rd ed.). New York: Addison Wesley Longman.
15.
Janssens, J. G., Pintelon, L., Cotton, A., & Gelders, L. (1996). Development of a framework for the assessment of operation and maintenance (O&M) performance of urban water supply and sanitation. Water Supply, 14(1), 21-33.
16.
Kelly, J. M., & Swindell, D. (2002a). A multiple-indicator approach to municipal service evaluation: Correlating performance measurement and citizen satisfaction across jurisdictions. Public Administration Review, 62(5), 610-621.
17.
Kelly, J. M., & Swindell, D. (2002b). Service quality variation across urban space: First steps toward a model of citizen satisfaction. Journal of Urban Affairs, 24(3), 271-288.
18.
Lineberry, R. L. (1977). Equality and urban policy: The distribution of municipal services. Beverly Hills, CA: Sage.
19.
Loomis, J. B. (2002). Integrated public lands management: Principles and applications to national forests, parks, wild-life refuges, and BLM lands. New York: Columbia University Press.
20.
Mainwaring, S. (1999). Rethinking party systems in the third wave of democratization: The case of Brazil. Stanford, CA: Stanford University Press.
21.
Mohr, L. B. (1999). The impact profile approach to policy merit: The case of research grants and the university. Evaluation Review, 23(2), 212-249.
22.
Nance, E. B. (2004). Putting participation in context: An evaluation of urban sanitation in Brazil. Dissertation Abstracts International, 65(04), 2035B. (UMI No. 3128354)
23.
National Council on Public Works Improvement. (1988). Report card on the nation’s public works. Retrieved November 23, 2004, from http://www.asce.org/reportcard
24.
Ostrom, E. (1973). The need for multiple indicators in measuring the output of public agencies. Policy Studies Journal, 2(1), 85-91.
25.
Ostrom, E. (1996). Crossing the great divide: Co-production, synergy, and development. World Development, 24(6), 1073-1088.
26.
Parks, R. B. (1984). Linking objective and subjective measures of performance. Public Administration Review, 44(2), 118-127.
27.
Percy, S. L. (1986). In defense of citizen evaluations as performance measures. Urban Affairs Quarterly, 22(1), 66-83.
28.
Reader, N. F., Tremey, G., Kempe, A., Friis, P., & Garcia-Poveda, M. (1983). Performance indicators. Water Supply, 1(2/3), SS21-1-SS21-4.
29.
Rosentraub, M. S., & Thompson, L. (1981). The use of surveys of satisfaction for evaluations. Policy Studies Journal, 9(7), 990-999.
30.
Schalock, R. L. (2001). Outcome-based evaluation (2nd ed.). New York: Kluwer Academic.
31.
Stipak, B. (1979). Citizen satisfaction with urban services: Potential misuse as a performance indicator. Public Administration Review, 39(1), 46-52.
32.
Tolosa, H. C. (1978). Causes of urban poverty in Brazil. World Development, 6(9/10), 1087-1101.
33.
U.K. Office of Water. (1998). Level of service indicators. Retrieved November 23, 2004, from http://www.ofwat.gov.uk/aptrix/ofwat/publish.nsf/Content/losindicators
34.
U.S. Environmental Protection Agency. (1977). Sewer system evaluation, rehabilitation, and new construction: A manual of practice (No. EPA-600/2-77-017d). Cincinnati, OH: Author.
35.
U.S. Environmental Protection Agency. (1991). Sewer system infrastructure analysis and rehabilitation (No. EPA/625/6-91/030). Cincinnati, OH: Author.
36.
U.S. Environmental Protection Agency. (2002). Decision-support tools for predicting the performance of water distribution and wastewater collection systems. Cincinnati, OH: Author.
37.
Van Ryzin, G. G. (2004). Expectations, performance, and citizen satisfaction with urban services. Journal of Policy Analysis and Management, 23(3), 433-448.
38.
Watson, G. (1995). Good sewers cheap?Agency-customer interactions in low-cost urban sanitation in Brazil. Washington, DC: World Bank.
39.
World Bank. (1994). World development report 1994: Infrastructure for development. Washington, DC: Oxford University Press.
40.
World Bank. (2004). Benchmarking water and sanitation utilities: Network of core indicator values. Retrieved October 18, 2004, from http://www.worldbank.org/html/fpd/water/topics/bench/bench_network_inddef_ac.html
41.
Yepes, G. (1993). Water and sanitation utilities: Operational indicators. Washington, DC: World Bank.
