This paper examines problems of setting targets for health care performance in which
the centre sets a uniform set of targets and levels of performance. The case study
examined by the paper is from the system of performance assessment of
‘star ratings’ that was introduced from 2001 by the Department
of Health to give each organisation in the NHS in England a single summary score
from zero rating to three stars. Star ratings were directed at holding
trusts’ chief executives to account for the local delivery of national
priorities through a process of ‘naming and shaming’, in which
Chief Executives of zero-rated organisations were at risk of losing their jobs. This
paper outlines the underlying model and its three underlying assumptions: the centre
can determine a scoring system to prioritise what matters; failures of performance
that are not reflected in the scoring system do not matter; and the advantages of a
system of scoring on accountable targets outweigh the disadvantages of various types
of gaming responses. This paper examines the application of the model of star
ratings to a case study of Primary Care Trusts (PCTs), which are complex
organisations with three diverse sets of responsibilities: delivering primary care,
commissioning secondary care, and supervising public health. It outlines an
alternative model and the issues this raises for governance of health care performance.
Alvarez-Rosete, A.
,
Bevan, G.
,
Mays, N.
and
Dixon, J.
(2005),
‘Diverging policy across the UK NHS: what is the impact?’
, British Medical Journal, 331, pp.
946-950
.
2.
Anonymous
(2005),
‘A&E survey highlights dirt and waiting times’
, Health Service Journal, 24 February, p.
7-7
.
3.
Arrow, K. J.
(1963),
‘Uncertainty and the welfare economics of medical care’
, American Economic Review, 53, pp.
941-973
.
Berliner, J. S.
(1988), Soviet Industry from Stalin to Gorbachev,
Aldershot, Edward Elgar
.
6.
Bevan, G.
and
Hood, C.
(2004),
‘Targets, inspections, and transparency’
, British Medical Journal, 328, p.
598-598
. http://bmj.bmjjournals.com/cgi/reprint/328/7440/598?maxtoshow=&HITS=10&hits=10&RESULTFORMAT=&author1=Bevan&andorexactfulltext=and&searchid=1118748446953_5283&stored_search=&FIRSTINDEX=0&sortspec=relevance&volume=328&resourcetype=1.
7.
Bevan, G.
and
Hood, C.
(2006)
‘Have targets improved performance in the English NHS?’
, British Medical Journal, 332, pp.
419-422
.
8.
Bevan, G.
and
Hood, C.
(forthcoming), ‘What’s measured is what matters:
targets and gaming in the English public health care system’,
Public Administration.
9.
Bevan, G.
and
Robinson R.
(2005),
‘The interplay between economic and political logics: path
dependency in health care in England’
, Journal of Health Politics, Policy and Law, 30,
1-2, pp.
53-78
.
10.
Bird, S. M.
,
Cox, D.
,
Farewell, V. T.
,
Goldstein, H.
,
Holt T.
, and
Smith, P. C.
(2005),
‘Performance indicators: good, bad, and ugly’
, Journal of the Royal Statistical Society, Series A,
168, 1, pp.
1-27
.
11.
British Medical Association
(2005), BMA Survey of A&E Waiting Times,
London, British Medical Association
.
12.
Carter, N.
,
Klein, R.
and
Day, P.
(1995), How Organisations Measure Success. The Use of
Performance Indicators in Government,
London, Routledge
.
13.
Commission for Health Improvement
(2003a), NHS Performance Ratings. Acute Trusts, Specialist
Trusts, Ambulance Trusts 2002/03,
London, The Stationery Office
. www.chi.nhs.uk/eng/ratings.
14.
Commission for Health Improvement
(2003b), NHS Performance Ratings. Primary Care Trusts, Mental
Health Trusts, Learning Disability Trusts 2002/03,
London, The Stationery Office
. www.chi.nhs.uk/eng/ratings.
15.
Commission for Health Improvement
(2003c), What CHI Has Found in: Ambulance Trusts,
London, The Stationery Office
. http://www.healthcarecommission.org.uk/NationalFindings/NationalThemedReports/Ambulance/fs/en
16.
Commission for Health Improvement
(2004a), A commentary on star ratings 2002/2003, http://www.chi.nhs.uk/eng/ratings/ratings03.shtml.
17.
Commission for Health Improvement
(2004b), What CHI Has Found in: Acute Services,
London, The Stationery Office
. http://www.healthcarecommission.org.uk/NationalFindings/NationalThemedReports/AcuteAndSpecialist/fs/en.
18.
Commission for Health Improvement
(undated) A Guide to Clinical Governance Reviews,
London, Commission for Health Improvement
.
19.
Dawson, D.
,
Gravelle, H.
,
O’Mahony, M.
,
Street, A.
,
Weale, M.
,
Castelli, A.
,
Jacobs, R.
,
Kind, P.
,
Loveridge, O.
,
Martin, S.
,
Stevens, P.
and
Stokes, L.
(2005), Developing New Approaches to Measuring NHS Output and
Productivity,
York, York University and London
, National Institute of Economic and Social Research, http://www.niesr.ac.uk/pdf/nhsoutputsprod.pdf.
20.
Department of Health
. (2002), Improvement, Expansion and Reform: The Next 3 Years.
Priorities and Planning Framework 2003 - 2006,
London, Department of Health
. http://www.dh.gov.uk/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/PublicationsPolicyAndGuidanceArticle/fs/en?CONTENT_ID=4008430&chk=lXp8vH
21.
Department of Health
. (2005), Healthcare Output and Productivity: Accounting for
Quality Change,
London, Department of Health
. http://www.dh.gov.uk/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/PublicationsPolicyAndGuidanceArticler/fs.en?CONTENT_ID=4124266&chk+5QmbY7.
22.
Evans, R. G.
(1987),
‘Public health insurance: the collective purchase of individual care’
, Health Policy, 7, pp.
115-134
.
23.
Evans, R. G.
,
Barer, M. L.
and
C. Hertzman
. (1991),
‘The 20-year experiment: accounting for, explaining and
evaluating health care cost containment in Canada and the United States’
, American Review of Public Health, 12, pp.
481-518
.
24.
Giddens, A.
(1998), The Third Way,
Cambridge, Polity Press
.
25.
Hart, J. T.
(1971),
‘The inverse care law’
, Lancet, i, pp.
405-412
.
Healthcare Commission
(2005b), Assessment for Improvement. The Annual Health
Check.
London: Healthcare Commission
.
http://www.healthcarecommission.org.uk/InformationForServiceProviders/AnnualHealthCheck/fs/en?CONTENT_ID=4017483&chk=ub2qrx
(accessed 28 September 2005)
29.
Healthcare Commission
(2005c), Primary Care Trust: Survey of Patients 2005,
London, Healthcare Commission
. http://www.healthcarecommission.org.uk/assetRoot/04/01/93/73/04019373.pdf
Hood, C.
,
James, O.
, and
Scott, C.
(2000),
‘Regulation of government: has it increased, is it increasing,
should it be diminished?’Public Administration, 78, 2, pp.
283-304
.
32.
Hood, C.
,
James, O.
,
Scott, C.
,
Jones, G.
and
Travers, T.
(1999), Regulation Inside Government,
Oxford, Oxford University Press
.
33.
Hood, C.
and
Scott, C.
(2000), Regulating Government in a
‘Managerial’ Age: Towards a Cross-National
Perspective,
London, Centre for Analysis of Risk and
Regulation, LSE
.
34.
James, O.
(2004),
‘The UK core executive’s use of public service
agreements as a tool of governance’
, Public Administration, 82, 2, pp.
397-419
.
35.
Klein, R.
(1998),
‘Why Britain is reorganizing its national health
service—yet again’
, Health Affairs, 17, 4, pp.
111-125
.
36.
Kornai, J.
(1994), Overcentralisation in Economic Administration,
Oxford, Oxford University Press
.
37.
Leatherman, S.
and
Sutherland K.
(2003), The Quest for Quality in the NHS. A Mid-Term
Evaluation of the Ten-Year Quality Agenda,
London, The Stationery Office
.
38.
McGuire, A.
,
Henderson, J.
and
Mooney, G.
(1987), The Economics of Health Care,
London, Routledge and Kegan Paul
.
39.
Majone, G.
(1996), Regulating Europe,
London, Routledge
.
40.
Mayor, S.
(2003),
‘Hospitals take short term measures to meet targets’
, British Medical Journal, 326, p.
1054-1054
.
41.
National Audit Office
. (2001), Inappropriate Adjustments to NHS Waiting
Lists,
London, Stationery Office
. (HC 452). www.nao.gov.uk/publications/nao_reports/01-02/0102452.pdf
42.
Nove, A.
(1958),
‘The problem of success indicators in Soviet industry’
, Economica (New Series)25, 97, pp.
1-13
.
43.
O’Mahony, M.
(2006),
‘Outputs, inputs and productivity in the NHS’
, presentation to 4th NIESR Public Sector Performance
conference, British Academy.
44.
Power, M.
(1999), The Audit Society: Rituals of Verification,
Oxford, Oxford University Press
.
45.
Public Administration Select Committee
(2003), Fifth Report. On Target? Government by
Measurement (HC 62-I),
London, The Stationery Office
.
46.
Propper, C.
(1995),
‘Agency and incentives in the NHS internal market’
, Social Science and Medicine, 40, 12,
pp.
1683-1690
.
47.
Propper, C.
and
Wilson D.
(2003),
‘The use and usefulness of performance measures in the public sector’
, Oxford Review of Economic Policy, 19, pp.
250-267
.
48.
Secretaries of State for Health, Wales, Northern Ireland and Scotland
(1989), Working for Patients, CM 555,
London, HMSO
.
49.
Secretary of State for Health
(1997), The New NHS: Modern, Dependable, Cm 3807,
London, The Stationery Office
.
50.
Secretary of State for Health
(2000), The NHS Plan, Cm 4818-I,
London, The Stationery Office
.
51.
Secretary of State for Health
(2001), NHS Performance Ratings Acute Trusts 2000/01,
London, Department of Health
. http://www.dh.gov.uk/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/PublicationsPolicyAndGuidanceArticle/fs/en?CONTENT_ID=4003181&chk=wU4Zop
52.
Secretary of State for Health
(2002a), NHS Performance Ratings Acute Trusts, Specialist
Trusts, Ambulance Trusts, Mental Health Trusts 2001/02,
London, Department of Health
. http://www.dh.gov.uk/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/PublicationsPolicyAndGuidanceArticle/fs/en?CONTENT_ID=4002706&chk=dBD1wB
53.
Secretary of State for Health
(2002b), Learning from Bristol: The Department of
Health’s Response to the Report of the Public Inquiry into
Children’s Heart Surgery at the Bristol Royal Infirmary
1984-1995, Cm 5363,
London, Stationery Office
, http://www.dh.gov.uk/PublicationsAndStatistics/Publications/PublicationsPolicyAndGuidance/PublicationsPolicyAndGuidanceArticle/fs/en?CONTENT_ID=4002859&chk=CPhi%2BI.
54.
Secretary of State for Health
(2002c), Delivering the NHS Plan, Cm 5503,
London, Stationery Office
.
55.
Shaw, M.
,
Smith, G. D.
and
Dorling, D.
(2005),
‘Health inequalities and New Labour: how the promises compare
with real progress’
, British Medical Journal, 330, pp.
1016-1021
.
56.
Smith, P.
(1995),
‘On the unintended consequences of publishing performance data in
the public sector’
, International Journal of Public Administration,
18, pp.
277-310
.
57.
Stevens, S.
(2004),
‘Reform strategies for the English NHS’
, Health Affairs, 23, 3, pp.
41-44
.
58.
Timmins, N.
(2005),
‘Blair bemused over GP waiting times’
, Financial Times, April 30/May 1, p.
2-2
.
59.
Timmins, N.
(2006),
‘How the mighty came to fall’
, British Medical Journal, 332, p.
628-628
.
60.
Tuohy, C.
(1999a),
‘Dynamics of a changing health sphere: the United States, Britain
and Canada’
, Health Affairs, 18, 3, pp.
114-134
.
61.
Tuohy, C.
(1999b), Accidental Logics. The Dynamics of Change in the
Health Care Arena in the United States, Britain and Canada,
New York, Oxford University Press
.
62.
Tuohy, C.
(2003),
‘Agency, contract, and governance: shifting shapes of
accountability in the health care arena’
, Journal of Health Politics, Policy and Law, 28,
2-3, pp.
195-215
.
63.
UK Centre for the Measurement of Government Activity
(2006),
‘Public service productivity: health’
, Economic trends, 628, pp.
26-57
. http://www.statistics.gov.uk/downloads/theme_economy/ET628.pdf.
64.
Wintour, P.
(2006),
‘Blair faces inquiry into NHS crisis’
, The Guardian, 19 April. http://society.guardian.co.uk/publicfinances/story/0..1756465.00.html.
