Comparative performance evaluation has taken different forms depending upon the purposes of performance monitoring and the types of measures available. This paper investigates the different performance measurement systems in place in the social care setting, in particular for older people receiving community care services. In England, earlier systems to assist performance management within organizations have been eclipsed by national systems of regulation with top— down implementation of standards and measures. In Northern Ireland, by contrast, organizations have been compared descriptively without the use of national targets. Internationally, in Japan, organizations arranging similar services have had more local information available collected in a bottom—up fashion with greater employment of service user-level data. These differences in performance evaluation are located within an analytical framework permitting comparisons of system design and the use of measures. Conclusions are drawn concerning the breadth of evidence available for successfully monitoring service provision in this setting.
Adab, P., A.M. Rouse, M.A. Mohammed and T. Marshall ( 2002) ‘Performance League Tables: The NHS Deserves Better’ , British Medical Journal324: 95-8.
2.
Applebaum, R. and M. White (eds) (2000) Case Management around the Globe. San Francisco, CA: American Society on Aging.
3.
Appleby, J. ( 2005) Independent Review of Health and Social Care Services in Northern Ireland. London: Kings Fund.
4.
Arvidsson, G. ( 1986) ‘Performance Evaluation’, in F. X. Kaufman, G. Majone and V. Ostrom (eds) Guidance Control and Evaluation in the Public Sector, pp. 625-43.Berlin : Walter de Gruyter.
5.
Audit Commission (1996) Local Authority Performance Indicators 1994/95, vol. 1, Education, Social Services, Libraries and Expenditure. London: Audit Commission.
6.
Barnes, M. and N. Miller ( 1988) ‘Performance Measurement in Personal Social Services’, Research, Policy and Planning6: 1-47.
7.
Behn, R.D. ( 2003) ‘Why Measure Performance? Different Purposes Require Different Measures’, Public Administration Review63(5): 586-606.
8.
Benjamin, L.M. ( 2008) ‘Evaluator’s Role in Accountability Relationships: Measurement Technician Capacity Builder, or Risk Manager?’, Evaluation14(3): 323-43.
9.
Bloomfield, J. ( 2006) Controlling, Cajoling or Co-operating? Central Governments’ Policy Approaches towards Local Government on the Issues of Performance and Cost-Effectiveness. Birmingham: Institute of Local Government Studies, University of Birmingham, and Council of European Municipalities and Regions.
10.
Brignall, S. and S. Modell ( 2000) ‘An Institutional Perspective on Performance Measurement and Management in the "New Public Sector"’, Management Accounting Research11: 281-306.
11.
Burnham, D. ( 2006) Only Get Better? A Guide to Social Services Performance Measurement Processes for Frontline Staff. Lyme Regis: Russell House Publishing.
12.
Camp, R.C. ( 1989) Benchmarking: The Search for Industry Best Practices that Lead to Superior Performance. Milwaukee, WI: Quality Press.
13.
Campbell, J.C. ( 1992) How Policies Change: The Japanese Government and the Aging Society. Princeton, NJ: Princeton University Press.
14.
Campbell, J.C. and N. Ikegami ( 2003) ‘Japan’s Radical Reform of Long-Term Care’ , Social Policy and Administration37: 21-34.
15.
Challis, D., P. Clarkson, J. Williamson, J. Hughes, D. Venables, A. Burns and A. Weinberg ( 2004) ‘The Value of Specialist Clinical Assessment of Older People Prior to Entry to Care Homes’, Age and Ageing33: 25-34.
16.
Challis, D., P. Clarkson and R. Warburton ( 2006a) Performance Indicators in Social Care for Older People . Aldershot: Ashgate.
17.
Challis, D., K. Stewart, M. Donnelly, K. Weiner and J. Hughes ( 2006b) ‘Care Management for Older People: Does Integration Make a Difference?’, Journal of Interprofessional Care20(4): 335-48.
18.
Clarkson, P. and D. Challis ( 2003) ‘Quality Assurance Practices in Care Management: A Perspective from the United Kingdom’, Care Management Journals4(3): 142-51.
19.
Clarkson, P. and D. Challis ( 2006) ‘Performance Measurement in Social Care: A Comparison of Efficiency Measurement Methods’, Social Policy and Society5(4): 461-77.
20.
Clarkson, P., D. Venables, J. Hughes, A. Burns and D. Challis ( 2006) ‘Integrated Specialist Assessment of Older People and Predictors of Care Home Admission’, Psychological Medicine36: 1011-21.
21.
Clarkson, P., S. Crompton, D. Challis, M. Donnelly and R. Beech ( 2007) The Design and Use of Local Metrics to Evaluate Performance: A Comparative Analysis of Social Care Organisations. Manchester : Personal Social Services Research UnitURL (consulted July 2008): http://www.pssru.ac.uk/pdf/MCpdfs/DPM160.pdf.
22.
Commission for Social Care Inspection (2007 ) Social Services Performance Assessment Framework Indicators Adults 2006-07. London: Commission for Social Care Inspection.
23.
Community Care Policy Network (2001) Software for Analysis of Long-Term Care Insurance Performance. Tokyo: Community Care Policy Network.
24.
Davies, I. ( 1999) ‘Evaluation and Performance Management in Government’ , Evaluation5(2): 150-9.
25.
Day, P. and R. Klein ( 1987) Accountabilities: Five Public Services. London: Tavistock.
26.
Deming, W.E. ( 1982) Out of the Crisis. Cambridge, MA: Massachusetts Institute of Technology.
27.
Department for Communities and Local Government (DCLG) ( 2007) The New Performance Framework for Local Authorities and Local Authority Partnerships: Single Set of National Indicators. London: DCLG.
28.
Department of Health (1998) Modernising Social Services. Promoting Independence, Improving Protection, Raising Standards. Cm 4169. London: Stationery Office.
29.
Department of Health (2003a) Social Services Performance Assessment Framework Indicators 2002-2003. London: Dept of Health.
30.
Department of Health (2003b) Social Services Performance Star Ratings. London: Dept of Health.
31.
Department of Health, Social Services and Public Safety (DHSSPS) (2008) Proposals for Health and Social Care Reform. Belfast: DHSSPS. URL (consulted Oct. 2008): http://www.dhsspsni.gov.uk/consultation_package.pdf .
32.
Department of the Environment (1981) Local Government Annual Reports: A Code of Practice. London : HMSO.
33.
Goldstein, H. and D.J. Spiegelhalter ( 1996) ‘League Tables and their Limitations: Statistical Issues in Comparisons of Institutional Performance’, Journal of the Royal Statistical Society, Series A 159: 385-443.
34.
Heenan, D. and D. Burrell ( 2006) ‘The Integration of Health and Social Care: The Lessons from Northern Ireland’, Social Policy and Administration40(1): 47-66.
35.
Henwood, M. and G. Wistow ( 1999) With Respect to Old Age: Long Term Care - Rights and Responsibilities. Community Care and Informal Care Research, vol. 3. London : Stationery Office.
36.
Hood, C. ( 2007) ‘Public Service Management by Numbers: Why does it Vary? Where has it Come from? What are the Gaps and the Puzzles?’ , Public Money and Management27(2): 95-102.
37.
Hood, C., C. Scott, O. James, G. Jones and T. Travers ( 1999) Regulation Inside Government: Waste-Watchers, Quality Police and Sleaze-Busters. Oxford: Oxford University Press.
38.
Hood, C.,O. James , B. G. Peters and C. Scott (eds) (2004) Controlling Modern Government. Cheltenham: Edward Elgar.
39.
House of Commons Health Committee (2005) NHS Continuing Care: Sixth Report of Session 2004-05, vol. 1. London: Stationery Office.
40.
Huber, N. ( 1999) ‘"At Risk" Local Authorities Could Lose their Social Services, Warns Hutton’, Community Care (25 Nov.-1 Dec.): 2-3.
41.
Ikegami, N. ( 1997) ‘Public Long-Term Care Insurance in Japan’ , Journal of the American Medical Association278: 1310-14.
42.
Ikegami, N. ( 2007) ‘Rationale, Design and Sustainability of LTCI in Japan - In Retrospect’, Social Policy and Society6: 423-34.
43.
Institute of Health and Economic Programmes (IHEP) ( 2002) Study on Evaluation Methods of the Effectiveness of Long Term Care Insurance. Tokyo: IHEP .
44.
Jowett, P. and M. Rothwell ( 1988) Performance Indicators in the Public Sector. Basingstoke: Macmillan.
45.
Kondo, K. ( 2003) The Outline of Aichi Gerontological Evaluation Study (AGES) Project: Wave 2000 and 2003. Nagoya: Nihon Fukushi University, Research Promotion Centre for Community Care.
46.
Leeuw, F.L. and J.E. Furubo ( 2008) ‘Evaluation Systems: What are They and Why Study Them?’, Evaluation14(2): 157-69.
47.
Lijphart, A. ( 1999) Patterns of Democracy: Government Forms and Performance in Thirty-Six Countries. New Haven and London: Yale University Press.
48.
Llewellyn, S. ( 1996) ‘Performance Management in the Social Services: Its Meaning and Measurement’, in I. Lapsley and F. Mitchell (eds) Accounting and Performance Measurement: Issues in the Private and Public Sectors , pp. 166-79. London: Paul Chapman.
49.
Mannion, R., H. Davies and M. Marshall ( 2005) ‘Impact of Star Performance Ratings in English Acute Hospital Trusts’, Journal of Health Services Research and Policy10: 18-24.
50.
March, J.G. and R.I. Sutton ( 1997) ‘Organizational Performance as a Dependent Variable’ , Organization Science8(6): 698-706.
51.
Matsuda, S. and M. Yamamoto ( 2001) ‘Long-Term Care Insurance and Integrated Care for the Aged in Japan’, International Journal of Integrated Care1: 1-11.
52.
Milburn, A. ( 2001) ‘Speech to the Annual Social Services Conference’ , Harrogate, England, 19 Oct. URL (consulted July 2007): http://www.dh.gov.uk/en/News/Speeches/Speecheslist/DH_4000442
53.
Miller, N. ( 1986) ‘Management Information and Performance Measurement in Personal Social Services’, Social Services Research4(5): 7-55.
54.
Ministry of Health, Labour and Welfare (2002 ) Long-Term Care Insurance in Japan. Tokyo : Ministry of Health, Labour and Welfare.
55.
Ministry of Public Management, Home Affairs, Posts and Telecommunications (2001) Government Policy Evaluation Act. Tokyo: Ministry of Public Management, Home Affairs, Posts and Telecommunications.
56.
Morris, J.N., B.E. Fries, K. Steel, N. Ikegami, R. Bernabei, G.I. Carpenter, R. Gilgen, J.P. Hirdes and E. Topinkova E. ( 1997) ‘Comprehensive Clinical Assessment in Community Setting: Applicability of the MDS-HC’, Journal of American Geriatrics Society45(8): 1017-24.
57.
Nielsen, S.B. and N. Ejler ( 2008) ‘Improving Performance? Exploring the Complementarities between Evaluation and Performance Management’, Evaluation14(2): 171-92.
58.
Northern Ireland Statistics and Research Agency ( 2003) Key Indicators of Personal Social Services for Northern Ireland 2003. Belfast: Dept of Health Social Services and Public Safety.
59.
Northern Ireland Statistics and Research Agency ( 2006) Community Statistics 1 April 2005-31 March 2006. Belfast: Dept of Health Social Services and Public Safety.
60.
Northern Ireland Statistics and Research Agency ( 2007) Key Indicators of Personal Social Services for Northern Ireland 2006. Belfast: Dept of Health Social Services and Public Safety.
61.
Ouchi, W. ( 1981) Theory Z: How American Management can Meet the Japanese Challenge. Reading, MA: Addison-Wesley .
62.
Preston, R.W. ( 2000) Understanding Modern Japan: A Political Economy of Development, Culture and Global Power. London: SAGE.
63.
Reilly, S., D. Challis, A. Burns and J. Hughes ( 2003) ‘Does Integration Really Make a Difference? A Comparison of Old Age Psychiatry Services in England and Northern Ireland’, International Journal of Geriatric Psychiatry18: 887-93.
64.
Schacter, M. ( 2002) Not a Tool Kit: Practitioner’s Guide to Measuring the Performance of Public Programs. Ottawa: Institute on Governance.
65.
Shewhart, W.A. ( 1931) Economic Control of Quality of Manufactured Product. New York: D Van Norstrand.
66.
Tanaka, H. ( 2006) Current State of Performance Measurement Systems in Japan: Are they Properly Functioning?Hamamatsu City: Shizuoka University.
67.
Tanaka, H. and T. Ono ( 2002) ‘Performance Measurement Utilization in Japan: A Critical View’, paper presented at the Annual National Conference of the American Society for Public Administration, Phoenix, AZ, 26 March.
68.
Timonen, V. ( 2005) ‘Policy Paradigms and Long-Term Care: Convergence or Continuing Difference?’, in P. Taylor-Gooby (ed.) Ideas and Welfare State Reform in Western Europe , pp. 30-53. Basingstoke: Palgrave Macmillan.
69.
Tsutsui, T. and N. Muramatsu ( 2007) ‘Japan’s Universal Long-Term Care System Reform of 2005: Containing Costs and Realizing a Vision’, Journal of the American Geriatrics Society55: 1458-63.
70.
Van Thiel, S. and F. Leeuw ( 2002) ‘The Performance Paradox in the Public Sector’ , Public Performance and Management Review25(3): 267-81.
71.
Wiener, J.M., J. Tilly, A. Howe, C. Doyle, A.E. Cuellar, J. Campbell and N. Ikegami ( 2007) Quality Assurance for Long-Term Care: The Experiences of England, Australia, Germany and Japan. Washington, DC : AARP.
72.
Wilford, R., R. Ginty, M. Roger, L. Dowds and G. Robinson ( 2003) ‘Northern Ireland’s Devolved Institutions: A Triumph of Hope over Experience?’, Regional and Federal Studies13(1): 31-54.
