Evaluation of Health Systems Performance of OECD Countries Using MOORA Method

Abstract

International comparison of health systems performance provides evidence to countries, policymakers and decision-makers. This study was conducted in order to rank and compare the health systems performance levels of the Organization for Economic Development and Cooperation (OECD) countries according to the determined health performance indicators. In the study, the data of the countries between the years of 2000–2017 were used and evaluated. The study consists of 36 member countries of the OECD. The data was analysed with the MOORA (Multi-Objective Optimization on the basis of Ratio Analysis) method, which is one of the multi-criteria decision-making methods, allowing the comparison of the performance of health systems across countries is compared. As a result of the health system performance evaluations of OECD countries. As a result of the health system performance evaluations of OECD countries, it has been concluded that countries such as Switzerland, Germany and Sweden have a high level of health systems performance, and Mexico, Colombia and Turkey have a lower performance compared to other countries. It is expected that countries with lower health system performance levels due to health system structures, demographic characteristics, education levels and technological development levels can still develop by selecting alternative countries suitable for their purpose and adopting the policies and strategies of such countries to their own systems.

Keywords

Health systems multi-criteria decision-making methods MOORA method OECD countries international comparison performance

Introduction

Policymakers must be responsible to individuals and ensure the effective and efficient use of resources provided to health systems (Paoli et al., 2019). Evaluation of the performance of health systems has become an important tool for the evidence-based approach (Blümel et al., 2020) and in many respects (Mahdiyan, 2020; Noto et al., 2019; Smith et al., 2009). Evaluation of health systems is important to inform policymakers and to highlight inequalities between different societies as well as to improve health systems. Comparing healthcare systems is challenging because the healthcare industry frequently provides various results (Schütte et al., 2018). Despite various structural differences and inequalities, many countries share common health system goals and face similar challenges (Fekri et al., 2018; WHO, 2000). At this point, international comparisons can be a reference source for countries to achieve the goals they set for the improvement of their health systems (Cylus et al., 2017; Nolte et al., 2006; Papanicolas & Smith, 2013; Suhrcke et al., 2006).

Over the past two decades, some international organisations have conducted various studies in order to assess the performance of healthcare systems and create international comparisons (Arah et al., 2006; European Commission, 2014; OECD, 2015; WHO, 2000). For example, the World Health Organization (WHO) published a ranking in 2000 to assess the performance of member nations’ health systems. The Organization for Economic Development and Cooperation (OECD) and the European Commission have developed a conceptual framework for assessing health system performance. In addition to international organisations, some countries have developed and compared conceptual frameworks for health system performance (Hurst & Jee-Hughes, 2001). Schieber et al. (1994) compared the United States’ healthcare system to those of other developed countries and developing countries. Anderson and Hussey (2001) used some indicators to examine the data on the performance of 29 nations’ health systems. Tchouaket et al. (2012) evaluated the health systems performance of some OECD countries and tried to distinguish the homogeneity of performance levels. Braithwaite et al. (2017) defined and compared the frameworks and performance indicators used in some OECD countries to measure and report the performance of health systems. Arah et al. (2003) examined the dynamics of the conceptual framework for health system performance, as well as its effectiveness and performance indicators. In a study conducted by Schoen et al. (2005), it was shown that negative patient experiences with healthcare systems put more pressure on decision-makers from other countries to seek alternative policy options.

Comparing the performance of the health system is a fundamental prerequisite for assessing progress and planning a way to proceed (Sharma et al., 2018). When making comparisons, a cautious approach should be adapted since efforts based on insufficient and biased policies can have a serious policy and political consequences (Papanicolas & Smith, 2013). In order to evaluate and compare the performance of countries, measurable and reliable indicators and methodologies derived from a competent information system are required (Hurst & Jee-Hughes, 2001). Performance indicators serve a variety of purposes, including facilitating measurability, frequently monitoring health systems, monitoring the behaviour of health professionals and organisations on a macro and micro scale, and directing policymakers (Arah et al., 2003; Kruk & Freedman, 2008; OECD, 2015; Smith et al., 2009). Traditionally, monitoring of developments in the health system and performance evaluation efforts have focused on such indicators (Jordans et al., 2016). While determining these indicators, it is necessary to take into account the effects on health service users and the structures, functions and outputs of health systems. Determined performance indicators should also have a great impact for health systems and should represent health systems performance (Arbour et al., 2009; Kruk & Freedman, 2008; Rowe et al., 2008). Despite the large number of national and international studies, methodological problems related to health system performance indicators continue to be discussed (Bankauskaite & Dargent, 2007).

Health systems performance has been limited by certain performance characteristics such as service quality, service access and service cost in various theoretical frameworks. Current methodologies and their limitations require a new search for assessing healthcare systems (Romaniuk et al., 2018). In particular, the lack of a comprehensive data collection tool and the lack of standard indicators that define the overall performance are among the main topics discussed (Fekri et al., 2018).

Performance measures can be used to guide consumer preferences, meet needs, and implement best practices for continuous improvement (Pransky et al., 2001). Factors such as the increase in the elderly population and the frequency of chronic diseases, new and developing medical technology, the increase in the demand and expectations of health services, and the more widespread use of information in today’s world have brought along the necessity of measuring and improving the performance of health systems (McLoughlin et al., 2002). Comparing the performance of healthcare systems may be an effective instrument for policymakers in accomplishing their objectives (Shari et al., 2017). It offers the opportunity to explore new options for mutual learning and formulating an effective national health policy (Fekri et al., 2018). Excluding the WHO (2000) study, literature which ranks nations’ health system performance is rare (Schütte et al., 2018; Sharma et al., 2018).

The aim of this study is to evaluate the health systems performance of 36 OECD member countries based on selected health indicators.

Methods

The study measures the performance with a Multi-Objective Optimization on the basis of Ratio Analysis (MOORA), one of the multi-criteria decision-making methods, and ranks the countries’ performance in order to investigate the relationship between indicators and the performance of healthcare systems.

Brauers and Zavadskas (2006) described their method as multi-objective optimisation of many alternatives. When compared to other multi-criteria decision-making methods, MOORA is said to have a high level of reliability (Brauers & Zavadskas, 2012; Chakraborty, 2011; İç, 2020; Kıyıcı et al., 2016). There are studies in the literature that use different approaches such as the ratio method, reference point approach, importance coefficient and full multiplicative form (Karande and Chakraborty, 2012) (Figure 1). In general, the ratio method and the reference point approach are used together, but some studies use just one method (Kalibatas & Turskis, 2008).

Figure1.

Diagram of MULTIMOORA (Brauers & Zavadskas, 2012).

In this study, the health system performance of OECD countries between 2000 and 2017 is ranked by using the ratio method and reference point approach together. Performance indicators indicate quantifiable value by taking into account several factors to assess the degree of performance (Parida & Kumar, 2006). There is no widely accepted standard evaluation method in the literature for assessing the criteria influencing the performance of health systems (Fekri et al., 2018; Schütte et al., 2018; Sharma et al., 2018). For this reason, studies that evaluate health system performance and the conceptual frameworks created are taken as reference when determining performance indicators (Arah et al. 2006; Blümel et al., 2020; European Commission, 2014; Noto et al., 2019; OECD, 2009; Romaniuk et al., 2020; WHO, 2000, 2007). Table 1 presents the indicators selected to evaluate health systems performance. In high-performing health systems, infant mortality rate is expected to be minimum and life expectancy at birth to be maximum (WHO, 2000). Taking these two indicators as reference; In order to determine the maximum or minimum desired values of other indicators, to determine their effects on each other, and to ensure that the research results are more accurate and reliable, correlation analysis was performed and objective functions were created according to the results.

Table 1.

Performance Indicators.

Performance Indicators		Aim
K1	Life expectancy at birth (years)	Maximum
K2	Health expenditure per capita (PPP, $)	Maximum
K3	Number of physicians per 1,000 people	Maximum
K4	Number of nurses and midwives per 1,000 persons	Maximum
K5	Infant mortality rate (per 1,000 live births)	Minimum
K6	Social protection (% of total population)	Maximum

Every year, the OECD gathers and frequently provides statistical data on its member and partner nations. The collection of data by the OECD according to certain standards allows comparative performance measurement studies (Afonso & Aubyn, 2005). Since the data standard and quality are thought to be high, the population of the study was determined as 37 OECD member countries. Luxembourg was not included in the study because there were missing values in the indicators used in the study. The study comprised 36 countries, and analyses and assessments were performed on these countries. There were some deficiencies in the data obtained from OECD member countries between the years of 2000–2017. In order to rectify this situation, in cases were information was unavailable for a specific year data from the most recent year was used. Although it is an undesirable situation, a solution can be produced this way in order to complete the deficiencies in the data set (Moran & Jacobs, 2013). The data on the health systems performance indicator variables used in the research were sourced from the data in the OECD database, which contains many data belonging to OECD members, and the World Development Indicators (WDI) databases of the World Bank, which allow countries to be compared. The number of physicians per 1,000 people and the number of nurses and midwives per 1,000 people were obtained from the WDI database because there was more missing data in the OECD database. It provides separate data for physicians (practitioners, etc.) and also provides data on the number of nurses and midwives in specific breakdowns. Data management and analysis were performed using Excel 2016.

Ratio Method

I = 1,2,3,…,m represents the number of alternatives, j = 1,2,3,…, n represents the number of criteria. The normalisation process (Equation 1) is performed by dividing the sum of the squares of each alternative by the square root of the criteria.

x_{i j}^{*} = \frac{x_{i j}}{\sqrt{\sum_{i}^{m} x_{i j}^{2}}}

(1)

x_ij is the response of alternative j to objective Ii, j = 1, 2,…,m; m the number of alternatives, i = 1,2,…, n; n the number of objectives.

Following this, performance criteria are established depending on the objective functions (maximum or minimum). For optimisation, these responses are added in case of maximisation and subtracted in case of minimisation. In other words, Equation (2) is applied to the criteria to maximise j = 1,2,…,g and to minimise j = g + 1, g + 2,…,n (Brauers & Ginevicius, 2010);

y_{i}^{*} = Σ_{j = 1}^{g} x_{i j}^{*} - Σ_{j = g + 1}^{n} x_{i j}^{*}

(2)

$y_{i}^{*}$ is the normalised assessment of alternative j with respect to all objectives.

According to the ratio method, performance ranking is obtained by ranking the $y_{i}^{*}$ ’s from maximum to minimum.

Table 2 shows an overview of the ranking of the health system performances obtained by using the steps of the ratio method of the data of the 36 OECD member countries for the year 2000. The ratio method summed the values for which the maximising of the normalised values obtained for the criteria was expected and subtracted the values for which the minimisation was intended from this total. The result obtained represents the performance score. The country with the highest score was determined to have the highest healthcare system performance, and a ranking was constructed from highest to lowest. It can be seen from the data in Table 2 that the top three countries are developed countries such as Iceland $(y_{i}^{*} = 0.9824)$ , Switzerland $(y_{i}^{*} = 0.9224)$ and Norway $(y_{i}^{*} = 0.8762)$ . The ranking of last three countries are Mexico $(y_{i}^{*} = 0.0876)$ , Colombia $(y_{i}^{*} = 0.0313)$ and Turkey $(y_{i}^{*} = - 0.0473)$ . For all other years (2001–2017), the same steps were applied again for each year (Appendix 1).

Table 2.

Ranking by Ratio Method (Year 2000).

Countries	Normalised Values of Indicators (Year 2000) ( $x_{i}^{*}$ )						$y_{i}^{*}$	S
Countries	K1(+)	K2(+)	K3(+)	K4(+)	K5(−)	K6(+)	$y_{i}^{*}$	S
Australia	0.1722	0.1713	0.1463	0.2009	0.0902	0.1788	0.7794	11
Austria	0.1699	0.2071	0.2252	0.1132	0.0832	0.1770	0.8093	8
Belgium	0.1690	0.1800	0.1650	0.1154	0.0832	0.1770	0.7232	18
Canada	0.1716	0.1856	0.1170	0.1989	0.0919	0.1788	0.7601	13
Chile	0.1659	0.1583	0.0527	0.1830	0.1543	0.1173	0.5229	32
Colombia	0.1583	0.1268	0.0702	0.0099	0.4334	0.0994	0.0313	35
Czech Republic	0.1631	0.1287	0.1968	0.1599	0.0711	0.1788	0.7562	14
Denmark	0.1670	0.1824	0.1704	0.1897	0.0919	0.1788	0.7965	9
Estonia	0.1542	0.1161	0.1828	0.2883	0.1456	0.1674	0.7631	12
Finland	0.1688	0.1607	0.1670	0.2181	0.0659	0.1788	0.8275	6
France	0.1720	0.2157	0.1963	0.1415	0.0780	0.1787	0.8261	7
Germany	0.1699	0.2225	0.1918	0.2047	0.0763	0.1620	0.8747	4
Greece	0.1707	0.1630	0.2495	0.0574	0.1023	0.1788	0.7171	19
Hungary	0.1562	0.1525	0.1757	0.1089	0.1595	0.1788	0.6127	29
Iceland	0.1731	0.2014	0.2020	0.2790	0.0520	0.1788	0.9824	1
Ireland	0.1664	0.1329	0.1305	0.2790	0.1075	0.1788	0.7801	10
Israel	0.1712	0.1531	0.2236	0.1198	0.0954	0.1788	0.7512	16
Italy	0.1736	0.1703	0.2024	0.1034	0.0745	0.1788	0.7539	15
Japan	0.1764	0.1609	0.1174	0.1721	0.0555	0.1788	0.7501	17
Korea	0.1651	0.0877	0.0761	0.0597	0.1110	0.1788	0.4564	33
Latvia	0.1527	0.1226	0.1996	0.0945	0.1786	0.1788	0.5697	30
Lithuania	0.1566	0.1393	0.1928	0.1595	0.1491	0.1626	0.6617	27
Mexico	0.1592	0.1001	0.1159	0.0178	0.3918	0.0864	0.0876	34
The Netherlands	0.1699	0.1734	0.1429	0.1591	0.0884	0.1153	0.6723	26
New Zealand	0.1703	0.1681	0.1287	0.1589	0.1092	0.1788	0.6957	23
Norway	0.1712	0.1735	0.1697	0.2506	0.0676	0.1788	0.8762	3
Poland	0.1603	0.1192	0.1290	0.1092	0.1404	0.1740	0.5513	31
Portugal	0.1670	0.1885	0.1874	0.0796	0.0954	0.1788	0.7060	21
Slovak Republic	0.1594	0.1193	0.1959	0.1476	0.1491	0.1767	0.6499	28
Slovenia	0.1653	0.1756	0.1260	0.1364	0.0849	0.1753	0.6936	24
Spain	0.1722	0.1530	0.1823	0.0699	0.0763	0.1769	0.6781	25
Sweden	0.1731	0.1658	0.1798	0.2079	0.0589	0.1788	0.8465	5
Switzerland	0.1736	0.2110	0.2065	0.2375	0.0849	0.1788	0.9224	2
Turkey	0.1544	0.1040	0.0789	0.0350	0.5444	0.1248	−0.0473	36
United Kingdom	0.1692	0.1639	0.1165	0.1710	0.0971	0.1788	0.7024	22
United States	0.1666	0.2822	0.1517	0.1882	0.1196	0.0402	0.7095	20

The rankings of some of the other years (2001–2017) evaluated according to the ratio method are given in Table 3. The entire table is presented in Suppl Appendix 1. Table 3 shows that Iceland had the best performance between 2000 and 2010. While Switzerland had the second-best performance until 2010, its health system has become the best country on a yearly basis since 2015. Iceland and Switzerland are followed by Norway, Germany, Switzerland and Finland. Determining the overall ranking, Iceland is the country with the best health performance. Switzerland is in the second place, Finland is in the third place and Sweden is in the fourth place. The USA ranks ²⁴th despite being the country with the highest health expenditure. Mexico, Turkey and Colombia make up the last three places.

Table 3.

Ranking of Health Systems Performance of OECD Countries by Ratio Method.

Countries	2000		2005		2010		2015		2016		2017		Ratio Method General Ranking
Countries	RM	R	RM	R	RM	R	RM	R	RM	R	RM	R	Ratio Method General Ranking
Australia	0.7794	11	0.7505	15	0.7563	14	0.7904	10	0.7888	11	0.7661	12	11
Austria	0.8093	8	0.8114	9	0.8021	9	0.8019	7	0.7923	9	0.7829	8	8
Belgium	0.7232	18	0.7946	11	0.7697	12	0.7730	14	0.7705	13	0.8458	4	12
Canada	0.7601	13	0.7238	19	0.6834	23	0.6866	20	0.6734	22	0.6615	24	19
Chile	0.5229	32	0.4796	33	0.4633	33	0.5169	33	0.5224	32	0.5207	31	33
Colombia	0.0313	35	0.0716	35	0.0580	36	0.0408	36	0.0363	36	0.1369	35	36
Czech Republic	0.7562	14	0.7565	12	0.7317	17	0.7247	18	0.7088	18	0.6966	18	15
Denmark	0.7965	9	0.8111	10	0.8196	8	0.7761	12	0.7889	10	0.7470	13	9
Estonia	0.7631	12	0.6110	27	0.6382	26	0.6357	26	0.6401	25	0.6296	26	26
Finland	0.8275	6	0.8671	4	0.8794	4	0.8759	4	0.8788	4	0.8483	3	3
France	0.8261	7	0.8246	8	0.7981	10	0.7736	13	0.7707	12	0.7439	15	10
Germany	0.8747	4	0.8590	6	0.8518	6	0.8420	6	0.8369	6	0.8217	7	6
Greece	0.7171	19	0.6968	21	0.6941	22	0.6543	23	0.6735	21	0.6782	23	22
Hungary	0.6127	29	0.6350	26	0.5911	29	0.5807	29	0.5903	29	0.5851	29	28
Iceland	0.9824	1	0.9796	1	0.9080	1	0.8829	3	0.9067	2	0.8340	6	1
Ireland	0.7801	10	0.8557	7	0.8920	3	0.7987	8	0.7947	7	0.7760	10	7
Israel	0.7512	16	0.7216	20	0.6555	25	0.6373	25	0.6328	26	0.6250	27	23
Italy	0.7539	15	0.7522	13	0.7322	16	0.7001	19	0.7000	19	0.6903	19	16
Japan	0.7501	17	0.7497	16	0.7578	13	0.7929	9	0.7942	8	0.7783	9	13
Korea	0.4564	33	0.5073	32	0.5544	30	0.5913	28	0.6066	28	0.5985	28	30
Latvia	0.5697	30	0.5434	30	0.5461	31	0.5395	31	0.5542	30	0.5187	32	31
Lithuania	0.6617	27	0.6038	28	0.6220	27	0.6448	24	0.6502	24	0.6834	22	27
Mexico	0.0876	34	0.1115	34	0.1366	34	0.1473	35	0.1351	35	0.0942	36	34
The Netherlands	0.6723	26	0.6702	24	0.7388	15	0.7791	11	0.7675	14	0.7446	14	18
New Zealand	0.6957	23	0.7514	14	0.7265	18	0.7410	16	0.7399	16	0.7169	17	17
Norway	0.8762	3	0.8594	5	0.8750	5	0.8903	2	0.8945	3	0.8732	2	5
Poland	0.5513	31	0.5380	31	0.5347	32	0.5210	32	0.5219	33	0.5026	33	32
Portugal	0.7060	21	0.7458	17	0.7735	11	0.7572	15	0.7565	15	0.7669	11	14
Slovak Republic	0.6499	28	0.5990	29	0.6068	28	0.5552	30	0.5444	31	0.5400	30	29
Slovenia	0.6936	24	0.6916	22	0.7107	21	0.7330	17	0.7364	17	0.7179	16	20
Spain	0.6781	25	0.6612	25	0.6805	24	0.6719	22	0.6935	20	0.6840	21	25
Sweden	0.8465	5	0.8826	3	0.8497	7	0.8705	5	0.8465	5	0.8436	5	4
Switzerland	0.9224	2	0.9358	2	0.9076	2	0.9194	1	0.9354	1	0.9175	1	2
Turkey	−0.0473	36	0.0336	36	0.1296	35	0.2256	34	0.2293	34	0.2270	34	35
United Kingdom	0.7024	22	0.7304	18	0.7129	20	0.6739	21	0.6683	23	0.6444	25	21
United States	0.7095	20	0.6867	23	0.7143	19	0.6153	27	0.6140	27	0.6844	20	24

Note: RM: ratio method score, R: ranking.

Reference Point Approach

In addition to the ratio method, the reference points (r_j) are determined for each criterion in the reference point approach. If the aim is maximisation, the maximum points, and if the aim is minimisation, the minimum points are determined. Equation (3) is used to measure the distances between these determined points for each $x_{i j}^{*}$ .

r_{j} - x_{i}^{*}

(3)

The ‘Tchebycheff Min-Max Metric’ (Equation 4) function is used on the newly created matrix to obtain the data for sorting.

{m i n}_{i} \{{m a k s}_{j} (|r_{j} - x_{i j}^{*}|)\}

(4)

Table 4 illustrates the ranking of health system performance for the year 2000 according to the reference point approach of 36 OECD member countries. From Table 4 it is seen that Switzerland (Max.= 0.0713), Iceland (Max.= 0.0808) and Germany (Max.= 0.0835) are the top three countries in the 2000 performance ranking. As in the ratio method, Mexico (Max.= 0.3398), Colombia (Max.= 0.3814) and Turkey (Max.= 0.4924) are in the last three places. The same steps were applied to the data between 2001 and 2017 and performance scores were obtained according to the reference point (Suppl Appendix 2).

Table 4.

Ranking by Reference Point Approach (Year 2000).

Countries	$\|r_{j} - x_{i j}^{*}\|$						Max.	Ranking
Countries	K1(+)	K2(+)	K3(+)	K4(+)	K5(−)	K6(+)	Max.	Ranking
Australia	0.0041	0.1109	0.1032	0.0874	0.0381	0.0000	0.1109	6
Austria	0.0065	0.0751	0.0243	0.1750	0.0312	0.0018	0.1750	24
Belgium	0.0074	0.1022	0.0845	0.1729	0.0312	0.0018	0.1729	23
Canada	0.0048	0.0966	0.1325	0.0894	0.0399	0.0000	0.1325	12
Chile	0.0104	0.1239	0.1968	0.1053	0.1023	0.0615	0.1968	29
Colombia	0.0180	0.1554	0.1793	0.2783	0.3814	0.0794	0.3814	35
Czech Republic	0.0132	0.1536	0.0527	0.1284	0.0191	0.0000	0.1536	19
Denmark	0.0093	0.0999	0.0791	0.0985	0.0399	0.0000	0.0999	4
Estonia	0.0222	0.1662	0.0667	0.0000	0.0936	0.0114	0.1662	21
Finland	0.0076	0.1216	0.0825	0.0701	0.0139	0.0000	0.1216	8
France	0.0043	0.0666	0.0532	0.1468	0.0260	0.0002	0.1468	16
Germany	0.0065	0.0597	0.0576	0.0835	0.0243	0.0168	0.0835	3
Greece	0.0056	0.1192	0.0000	0.2309	0.0503	0.0000	0.2309	33
Hungary	0.0202	0.1297	0.0738	0.1793	0.1075	0.0000	0.1793	26
Iceland	0.0033	0.0808	0.0475	0.0092	0.0000	0.0000	0.0808	2
Ireland	0.0100	0.1494	0.1190	0.0092	0.0555	0.0000	0.1494	17
Israel	0.0052	0.1292	0.0258	0.1684	0.0433	0.0000	0.1684	22
Italy	0.0028	0.1120	0.0471	0.1848	0.0225	0.0000	0.1848	27
Japan	0.0000	0.1213	0.1321	0.1161	0.0035	0.0000	0.1321	11
Korea	0.0113	0.1945	0.1734	0.2286	0.0589	0.0000	0.2286	32
Latvia	0.0237	0.1596	0.0499	0.1938	0.1266	0.0000	0.1938	28
Lithuania	0.0198	0.1429	0.0567	0.1288	0.0971	0.0163	0.1429	15
Mexico	0.0172	0.1821	0.1336	0.2704	0.3398	0.0925	0.3398	34
The Netherlands	0.0065	0.1088	0.1066	0.1291	0.0364	0.0635	0.1291	9
New Zealand	0.0061	0.1141	0.1208	0.1293	0.0572	0.0000	0.1293	10
Norway	0.0052	0.1087	0.0798	0.0377	0.0156	0.0000	0.1087	5
Poland	0.0161	0.1630	0.1204	0.1791	0.0884	0.0048	0.1791	25
Portugal	0.0093	0.0937	0.0621	0.2087	0.0433	0.0000	0.2087	30
Slovak Republic	0.0169	0.1629	0.0536	0.1406	0.0971	0.0021	0.1629	20
Slovenia	0.0111	0.1066	0.1235	0.1518	0.0329	0.0036	0.1518	18
Spain	0.0041	0.1292	0.0672	0.2183	0.0243	0.0020	0.2183	31
Sweden	0.0033	0.1164	0.0697	0.0804	0.0069	0.0000	0.1164	7
Switzerland	0.0028	0.0713	0.0429	0.0508	0.0329	0.0000	0.0713	1
Turkey	0.0219	0.1782	0.1706	0.2533	0.4924	0.0540	0.4924	36
United Kingdom	0.0072	0.1183	0.1330	0.1173	0.0451	0.0000	0.1330	13
United States	0.0098	0.0000	0.0977	0.1000	0.0676	0.1386	0.1386	14
Reference point ( $r_{j}$ )	0.1764	0.2822	0.2495	0.2883	0.0520	0.1788

The rankings of some of the other years (2001–2017) evaluated according to the reference point approach are given in Table 5. The entire table is presented in Suppl Appendix 2. Germany has become the country with the best performance between 2005 and 2014. It can be seen from Table 5 that Switzerland is in first place in the last three years. Switzerland has alternated between first and fourth place in the performance rankings. While Iceland was in first place in the early 2000s, it showed a serious decline after 2010. By 2017, Iceland (Max.=0.1580) regressed to ¹⁶th place in the ranking. As in the ratio method, Switzerland, Germany and Iceland are followed by developed and developing countries such as Denmark, Norway and Australia. When the changes of the countries between the years are examined, it can be said that there are no serious ranking changes.

Table 5.

Ranking of Health Systems Performance of OECD Countries by Reference Point Approach.

Countries	2000		2005		2010		2015		2016		2017		General Ranking
Countries	RPA	R	RPA	R	RPA	R	RPA	R	RPA	R	RPA	R	General Ranking
Australia	0.1109	6	0.1348	15	0.1467	15	0.1354	11	0.1427	12	0.1423	11	13
Austria	0.1750	24	0.1678	21	0.1797	22	0.1822	21	0.1830	21	0.1866	22	20
Belgium	0.1729	23	0.1089	3	0.1187	5	0.1152	4	0.1230	5	0.1204	4	5
Canada	0.1325	12	0.1267	8	0.1464	14	0.1351	10	0.1474	13	0.1417	10	15
Chile	0.1968	29	0.1902	27	0.1784	21	0.1541	15	0.1551	14	0.1448	14	24
Colombia	0.3814	35	0.2747	36	0.2854	36	0.2804	36	0.2797	36	0.2779	36	36
Czech Republic	0.1536	19	0.1676	20	0.1746	19	0.1735	20	0.1799	20	0.1783	19	19
Denmark	0.0999	4	0.1124	5	0.1187	4	0.1280	8	0.1280	7	0.1362	9	3
Estonia	0.1662	21	0.1946	29	0.1867	25	0.1933	25	0.1928	26	0.1954	26	27
Finland	0.1216	8	0.1293	11	0.1340	8	0.1295	9	0.1394	9	0.1427	12	6
France	0.1468	16	0.1302	12	0.1383	11	0.1220	6	0.1185	4	0.1226	5	11
Germany	0.0835	3	0.0875	1	0.0973	1	0.1016	2	0.1057	2	0.1027	2	1
Greece	0.2309	33	0.2199	33	0.2331	33	0.2412	33	0.2408	33	0.2408	33	33
Hungary	0.1793	26	0.1684	22	0.1849	23	0.1877	23	0.1893	25	0.1910	24	22
Iceland	0.0808	2	0.1112	4	0.1448	13	0.1572	17	0.1615	17	0.1580	16	8
Ireland	0.1494	17	0.1418	18	0.1083	2	0.1721	19	0.1751	19	0.1797	20	14
Israel	0.1684	22	0.1744	23	0.2076	29	0.2085	30	0.2070	30	0.2084	30	25
Italy	0.1848	27	0.1916	28	0.2014	28	0.2040	29	0.2028	28	0.2027	27	29
Japan	0.1321	11	0.1391	16	0.1375	9	0.1445	13	0.1426	11	0.1434	13	17
Korea	0.2286	32	0.2133	32	0.2173	32	0.1995	27	0.1842	22	0.1861	21	32
Latvia	0.1938	28	0.1872	26	0.2080	30	0.2187	32	0.2199	32	0.2233	32	30
Lithuania	0.1429	15	0.1827	25	0.1763	20	0.1872	22	0.1893	24	0.1923	25	21
Mexico	0.3398	34	0.2448	34	0.2575	34	0.2594	35	0.2582	35	0.2585	35	34
The Netherlands	0.1291	9	0.1266	7	0.1475	16	0.1195	5	0.1235	6	0.1267	7	10
New Zealand	0.1293	10	0.1290	10	0.1253	6	0.1355	12	0.1413	10	0.1457	15	7
Norway	0.1087	5	0.1279	9	0.1379	10	0.1275	7	0.1289	8	0.1236	6	12
Poland	0.1791	25	0.1793	24	0.1958	27	0.2029	28	0.2041	29	0.2083	29	26
Portugal	0.2087	30	0.1987	30	0.1946	26	0.1903	24	0.1890	23	0.1884	23	28
Slovak Republic	0.1629	20	0.1654	19	0.1854	24	0.1990	26	0.1989	27	0.2084	31	23
Slovenia	0.1518	18	0.1417	17	0.1519	18	0.1518	14	0.1555	15	0.1608	17	18
Spain	0.2183	31	0.2037	31	0.2088	31	0.2116	31	0.2084	31	0.2079	28	31
Sweden	0.1164	7	0.1306	14	0.1485	17	0.1078	3	0.1129	3	0.1133	3	4
Switzerland	0.0713	1	0.0879	2	0.1125	3	0.0979	1	0.0971	1	0.0932	1	2
Turkey	0.4924	36	0.2528	35	0.2598	35	0.2559	34	0.2572	34	0.2553	34	35
United Kingdom	0.1330	13	0.1237	6	0.1423	12	0.1607	18	0.1621	18	0.1687	18	16
United States	0.1386	14	0.1306	13	0.1253	7	0.1564	16	0.1567	16	0.1340	8	9

Notes: Changes from green to red indicate decreased performance. RPA: reference point approach score, R: rank.

In order to obtain the general ranking, the results obtained from the rankings of the countries between the years 2000–2017 were used and a new ranking was created in Table 6. As shown in Table 6 the overall ranking, Switzerland has the best healthcare system performance. Germany is ranked second, Denmark is ranked third and Sweden is ranked fourth. Mexico, Turkey and Colombia are in the last three in the ranking.

Table 6.

Health System Performance MOORA Overall Ranking (2000–2017).

Countries	Ratio Method	Reference Point Approach	MOORA Method
Australia	11	13	11
Austria	8	20	13
Belgium	12	5	7
Canada	19	15	17
Chile	33	24	30
Colombia	36	36	36
Czech Republic	15	19	18
Denmark	9	3	6
Estonia	26	27	27
Finland	3	6	4
France	10	11	9
Germany	6	1	2
Greece	22	33	28
Hungary	28	22	25
Iceland	1	8	5
Ireland	7	14	10
Israel	23	25	23
Italy	16	29	22
Japan	13	17	15
Korea	30	32	33
Latvia	31	30	32
Lithuania	27	21	24
Mexico	34	34	34
The Netherlands	18	10	14
New Zealand	17	7	12
Norway	5	12	8
Poland	32	26	31
Portugal	14	28	21
Slovak Republic	29	23	26
Slovenia	20	18	20
Spain	25	31	29
Sweden	4	4	3
Switzerland	2	2	1
Turkey	35	35	35
United Kingdom	21	16	19
United States	24	9	16

Note: Changes from green to red indicate decreased performance.

Discussion

The health level of developed countries is higher than that of developing or undeveloped countries. However, it is very difficult to infer which country’s health system has the highest performance. Performance needs to be evaluated according to worst and best under certain conditions (WHO, 2007). In this study, the performance of 36 OECD member countries’ health systems was assessed using several health indicators. With MOORA, one of the multi-criteria decision-making methods, the performance of health systems was measured and the performance of the countries were ranked. When studies evaluating health systems performance are examined, very few studies using multi-criteria decision-making methods have been found. The study differs from the studies conducted in the field of health systems performance with the method it uses. In this respect, it is thought that the study will contribute by expanding the existing literature.

The performance score of the health systems performance of 36 OECD member countries between 2000 and 2017 was obtained using the ratio method and reference point approach. When the health system performance rankings of the countries are examined, it is seen that the top 10 countries with the best health system performance are Switzerland, Germany, Sweden, Finland, Iceland, Denmark, Belgium, Norway, France and Ireland, respectively. It has been determined that Poland, Latvia, Korea, Mexico, Turkey and Colombia perform poorly in comparison to other countries. In reviewing the literature, studies comparing health systems using different techniques support our findings (Hejduková & Kureková, 2017; Mackenbach & McKee, 2013; Tchouaket et al. 2012). It is also seen that there is a similarity between the findings of our study and the Human Development Index (HDI, 2020) report which states that countries such as Norway, Ireland, Switzerland, Iceland, Sweden, Australia, The Netherlands and Denmark are among the top 10 countries. While obtaining the development ranking of the countries, indicators such as life expectancy at birth, years of education and income are used. These findings also support the concept that the performance of health systems should be assessed with the performance of other systems such as education and the economy, and that these systems might have an impact on each other directly or indirectly.

When the indicators of the countries with low performance are examined, it is seen that the infant mortality rate is high and the health workforce is below the average compared to other countries. There are several possible explanations for this result. According to WHO (2007), health manpower is the workforce that works efficiently to achieve the best health outcomes given the available resources and conditions. A sufficient number of qualified health workers is a critical tool for ensuring the performance of healthcare systems. In order to access quality health services, a sufficient and qualified health workforce is needed (Chen et al., 2004; Hongoro & McPake, 2004). Life expectancy at birth is longer in developed countries than in underdeveloped and developing countries. Among the most important reasons for this is the investment in health manpower. These investments both enable the use of advanced medical technology and reduce the number of people per healthcare worker (Beck & Webb, 2003). Anand and Bärnighausen (2004) found that the decrease in the infant mortality rate of the health workforce was significantly explained in their analysis. According to Speybroeck et al. (2006), countries with higher total health manpower had improved health outcomes. Or (2001) emphasises that the number of physicians per capita is the second most important variable for both perinatal and infant mortality. When considered in this context, it is thought that countries such as Mexico, Turkey and Colombia should develop policies that will increase their health workforce. Romaniuk et al. (2020) state that there is a correlation between per capita health expenditures and health outcomes. Countries allocate a significant share of their national income to health systems. The ratio of health expenditures to the total share in OECD countries is around 9% (OECD, 2021). In our study, although the per capita health expenditure of the USA is much higher than that of other countries, it was seen that the performance of health systems is at an average level (¹⁶th). In the study conducted by Squires and Anderson (2015), the health system performance of the USA was found to be low compared to other countries. In the report by the WHO (2000), which compared the health systems performance of 191 countries, it was reported that the performance of the USA was moderate. The present results are significant in some major respects. A possible explanation for this might be that this has been associated with the widespread use of medical technology and higher-cost healthcare delivery.

Conclusion and Recommendations

Evaluating and improving health system performance has become an important policy issue in many developed and developing countries. International comparisons have an important potential driver for health system improvement. Measuring and evaluating health systems performance provides countries, policymakers and decision-makers with evidence for improvement. When assessing the performance of healthcare systems, various indicators are considered. The purpose of the indicators is to change the behaviour of health professionals and organisations at both macro and micro levels and to guide policymakers.

These findings may help us understand that the effective and efficient use of investments in health may increase the performance of the health system. Health workforce, access to health services, health expenditures increase the health status of countries. At the macro level, the health system structure, demographic characteristics, education level, economic structure, technological development level, etc. of countries with low health system performance can develop as a result of adapting the policies and strategies implemented by those countries with high health system performance to their own systems. Examples of high performance in other countries are expected to encourage lower-income countries to make similar attempts and contribute to collaborative learning through the use of performance statistics. In this way, countries have the opportunity to develop strategies and policies based on evidence and statistical data.

In order to increase the performance of health systems, countries need to increase life expectancy at birth. To that end, it is recommended to expand activities such as healthy nutrition, counselling, education, public health training on the management of chronic diseases, professional support, home care services, cancer screenings, physical activities and similar activities to develop a healthy life culture. Improvements should be made to ensure adequate financing for health expenditures and to increase productivity. However, investments in health should be made in line with needs. The effectiveness of science and advisory boards should be increased, and the financial knowledge level of health managers should be increased. Cooperation should be made with health institutions and universities for health workforce, and improvements should be made regarding the working and living conditions of health personnel. In order to reduce the infant mortality rate, the number of health personnel should be sufficient. The quality of service care, the number of follow-ups for maternal and child health, and the number of community-level training on maternal and child health should be increased. The access of mothers and children to health services should be facilitated and the effectiveness of public health units should be increased.

Limitations and Future Studies

The study has several limitations. The generalisability of these results is subject to certain limitations. It is assumed that the variables used in the study represent the health systems performance of OECD member countries. It should not be forgotten that factors other than the determined health indicators can also affect the performance of health systems. When evaluating the results of the study, it should be taken into account that the results of the study may be affected by the fact that countries have different health systems, their economic levels, demographic characteristics and variances in socio-cultural structures. The findings of this study have a number of important implications for future practice. Future studies can expand the scope of the study by using different variables, examining the strategies and policies implemented by the countries.

Footnotes

Acknowledgement

This study is produced from Ferit Sevim’s master’s thesis titled ‘An Evaluation of OECD Countries Health System Performance’.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors received no financial support for the research, authorship and/or publication of this article.

Supplemental material

References

Afonso

, & Aubyn

(2005). Non-parametric approaches to education and health efficiency in OECD countries. Journal of Applied Economics , 8, 227–246.

Anand

, & Bärnighausen

(2004). Human resources and health outcomes: Cross-country econometric study. The Lancet , 364(9445), 1603–1609.

Anderson

, & Hussey

P. S.

(2001). Comparing health system performance in OECD countries. Organization for Economic Cooperation and Development. Health Affairs (Millwood) , 20(3), 219–232. doi:10.1377/hlthaff.20.3.21

Arah

O. A.

, Klazinga

N. S.

, Delnoij

D. M.

, Ten Asbroek

Ah.

, & Custers

(2003). Conceptual frameworks for health systems performance: A quest for effectiveness, quality, and improvement. International Journal for Quality in Health Care , 15(5), 377–398. doi:10.1093/intqhc/mzg04

Arah

O. A.

, Westert

G. P.

, Hurst

, & Klazinga

N. S.

(2006). A conceptual framework for the OECD health care quality indicators project. International Journal for Quality in Health Care , 18(suppl_1), 5–13.

Arbour.

, Melnikov

, Mcintosh

, Olsen

, Osborne

, & Vaktskjold

(2009). The current state of birth outcome and birth defect surveillance in northern regions of the world. International Journal of Circumpolar Health , 68(5),443–458.

Bankauskaite

, & Dargent

(2007). Health systems performance indicators: Methodological issues. Presupuesto y Gasto Publico , 49, 125–137.

Beck

, & Webb

(2003). Economic, demographic, and institutional determinants of life insurance consumption across countries. The World Bank Economic Review , 17(1), 51–88. https://doi.org/10.1093/wber/lhg011

Blümel

, Röttger

, Köppen

, Achstetter

, & Busse

(2020). Integrating the population perspective into health system performance assessment (IPHA): Study protocol for a cross-sectional study in Germany linking survey and claims data of statutorily and privately insured. International Journal of Health Policy and Management , 9(9), 370–379. https://doi.org/10.15171/ijhpm.2019.141

10.

Braithwaite

, Hibbert

, Blakely

, Plumb

, Hannaford

, Long

J. C.

, & Marks

(2017). Health system frameworks and performance indicators in eight countries: A comparative international analysis. SAGE Open Medicine , 5, 2050312116686516.

11.

Brauers

W. K. M.

, & Zavadskas

E. K.

(2006). The MOORA method and its application to privatization in a transition economy. Control and Cybernetics , 35(2), 445–469.

12.

Brauers

W. K. M.

, & Ginevicius

(2010). The economy of the Belgian regions tested with multimoora. Journal of Business Economics and Management , 11(2), 173–209.

13.

Brauers

W. K. M.

, & Zavadskas

E. K.

(2012). Robustness of MULTIMOORA: A method for multi-objective optimization. Informatica , 23(1), 1–25.

14.

Chakraborty

(2011). Applications of the MOORA method for decision making in manufacturing environment. The International Journal of Advanced Manufacturing Technology , 54, 1155–1166. https://doi.org/10.1007/s00170-010-2972-0

15.

Chen

, Evans

, Anand

, Boufford

J. I.

, Brown

, Chowdhury

, Cueto

, Dare

, Dussault

, Elzinga

, Fee

, Habte

, Hanvoravongchai

, Jacobs

, Kurowski

, Michael

, Pablos-Mendez

, Sewankambo

, Solimano

, Stilwell

, … Wibulpolprasert

. (2004). Human resources for health: Overcoming the crisis. The Lancet , 364(9449), 1984–1990 https://doi.org/10.1016/S0140-6736(04)17482-5.

16.

Cylus

, Papanicolas

, Smith

P. C.

, & Richardson

(Eds.). (2017). How to make sense of health system efficiency comparisons? European Observatory on Health Systems and Policies.

17.

European Commission. (2014). Communication from the commission: On effective, accessible and resilient health systems . European Commission.

18.

Fekri

, Macarayan

E. R.

, & Klazinga

(2018). Health system performance assessment in the WHO European Region: Which domains and indicators have been used by member states for its measurement? WHO Regional Office for Europe.

19.

HDI. (2020). Human development reports . http://hdr.undp.org/en/content/latest-human-development-index-ranking

20.

Hejduková

, & Kureková

(2017). Healthcare systems and performance evaluation: Comparison of performance indicators in v4 countries using models of composite indicators. E+M Ekonomie a Management , 20(3), 133–146.

21.

Hongoro

, & McPake

(2004). How to bridge the gap in human resources for health. The Lancet , 364(9443), 1451–1456.

22.

Hurst

, & Jee-Hughes

(2001). Performance measurement and performance management in OECD health systems (Vol. 47, pp. 68). OECD Publishing.

23.

İç

Y. T.

(2020). A multi-objective credit evaluation model using MOORA method and goal programming. Arabian Journal for Science and Engineering , 45, 2035–2048. https://doi.org/10.1007/s13369-019-03930-7

24.

Jordans

M. J.

, Chisholm

, Semrau

, Upadhaya

, Abdulmalik

, Ahuja

, Alem

, Hanlon

, Kigozi

, Mugisha

, Petersen

, Shidhaye

, Lund

, Thornicroft

, & Gureje

(2016). Indicators for routine monitoring of effective mental healthcare coverage in low- and middle-income settings: a Delphi study. Health Policy and Planning , 31(8), 1100–1106. https://doi.org/10.1093/heapol/czw040

25.

Kalibatas

, & Turskis

(2008). Multicriteria evaluation of inner climate by using MOORA method. Information Technology and Control , 37(1), 79–83.

26.

Karande

, & Chakraborty

(2012). Application of multi-objective optimization on the basis of ratio analysis (MOORA) method for materials selection. Materials and Design , 37, 317–324.

27.

Kıyıcı

, Korkmaz

, & Uygurtürk

(2016). Evaluation of comparative performances of Turkish individual pension funds with TOPSIS, VIKOR and MOORA methods [Papers on Social Sciences] (Vol. 2, pp. 1–16).

28.

Kruk

M. E.

, & Freedman

L. P.

(2008). Assessing health system performance in developing countries: A review of the literature. Health Policy (Amsterdam, The Netherlands) , 85(3), 263–276. https://doi.org/10.1016/j.healthpol.2007.09.003

29.

Mackenbach

J. P

, & Mckee

(2013). A comparative analysis of health policy performance in 43 European countries. The European Journal of Public Health , 23(2), 195–201.

30.

Mahdiyan

, Akbari

, Bozorgi

, & Amini-Rarani

(2020). Identifying and prioritizing challenges related to promoting the health system performance assessment in Iran. Journal of Education and Health Promotion , 9, 169. https://doi.org/10.4103/jehp.jehp_106_20

31.

Mcloughlin

, Leatherman

, Fletcher

, & Owen

(2002). Improving performance using indicators. Recent experiences in the United States, the United Kingdom, and Australia. International Journal for Quality in Health Care: Journal of the International Society for Quality in Health Care/ISQua , 13, 455–462.

32.

Moran

, & Jacobs

(2013). An international comparison of efficiency of inpatient mental health care systems. Health Policy , 112(1–2), 88–99.

33.

Nolte

, Wait

, & McKee

(2006). Investing in health. Benchmarking health systems . The Nuffield Trust.

34.

Noto

, Corazza

, Kļaviņa

, Lepiksone

, & Nuti

(2019). Health system performance assessment in small countries: The case study of Latvia. The International Journal of Health Planning and Management , 34(4), 1408–1422. https://doi.org/10.1002/hpm.2803

35.

OECD. (2009). Health at a Glance 2009. Health workers: Psychiatrists . Retrieved from website: http://www.oecd-ilibrary.org/docserver/download/fulltext/8109111ec030.pdf.

36.

OECD. (2015). Health at a Glance 2015 .

37.

OECD. (2021). Health data . https://stats.oecd.org/

38.

(2001). Exploring the effects of health care on mortality across OECD countries . doi:https://doi.org/10.1787/71647258570

39.

Paoli

, Schmidt

, Wigzell

, & Ryś

(2019). An EU approach to health system performance assessment: Building trust and learning from each other. Health Policy (Amsterdam, The Netherlands) , 123(4), 403–407. https://doi.org/10.1016/j.healthpol.2019.02.004

40.

Papanicolas

, & Smith

(2013). Health system performance comparison: An agenda for policy, information and research: An agenda for policy, information and research . McGraw-Hill Education.

41.

Parida

, & Kumar

(2006). Maintenance performance measurement (MPM): Issues and challenges. Journal of Quality in Maintenance Engineering , 12(3), 239–251.

42.

Pransky

, Benjamin

, & Dembe

A. E.

(2001). Performance and quality measurement in occupational health services: Current status and agenda for further research. American Journal of Industrial Medicine , 40(3), 295–306.

43.

Romaniuk

, Kaczmarek

, Syrkiewicz-Świtała

, Holecki

, & Szromek

A. R.

(2018). Health systems and their assessment: A methodological proposal of the synthetic outcome measure. Frontiers in Public Health , 6, 126. https://doi.org/10.3389/fpubh.2018.00126

44.

Romaniuk

, Poznańska

, Brukało

, & Holecki

(2020). Health system outcomes in BRICS countries and their association with the economic context. Frontiers in Public Health , 8, 80.

45.

Rowe

, Magee

, Quigley

, Heron

, Askham

, & Brocklehurst

(2008). Social and ethnic differences in attendance for antenatal care in England. Public Health , 122(12), 1363–1372.

46.

Schieber

, Poullier

, & Greenwald

(1994). Health system performance in OECD countries, 1980–1992. Organization for Economic Cooperation and Development. Health Affairs (Millwood) , 13(4), 100–112.

47.

Schoen

, Osborn

, Huynh

P. T

, Doty

, Zapert

, Peugh

, & Davis

(2005). Taking the pulse of health care systems: Experiences of patients with health problems in six countries. Health Affairs (Millwood), Suppl Web Exclusives , W5-509-525.

48.

Schütte

, Acevedo

, & Flahault

(2018). Health systems around the world – A comparison of existing health system rankings. Journal of Global Health , 8(1), 010407. https://doi.org/10.7189/jogh.08.010407

49.

Shari

, Tabibi

S. J.

, Nasiripour

, & Ghaffari

(2017). Health system performance evaluation framework with an emphasis on health promotion of vulnerable groups in Iran. Acta Medica Mediterranea , 33(2), 259–265.

50.

Sharma

, Prinja

, & Aggarwal

A. K.

(2018). Measurement of health system performance at district level: A study protocol. Journal of Public Health Research , 6(3), 917. https://doi.org/10.4081/jphr.2017.917.

51.

Smith

P. C.

, Mossialos

, & Papanicolas

(2010). Measurement for health system improvement: Experiences, challenges and prospects . Cambridge University Press. http://dx.doi.org/10.1017/CBO9780511711800

52.

Speybroeck

, Kinfu

, Dal Poz

M. R.

, & Evans

D. B.

(2006). Reassessing the relationship between human resources for health, intervention coverage and health outcomes . World Health Organization.

53.

Squires

, & Anderson

(2015). US health care from a global perspective: Spending, use of services, prices, and health in 13 countries. Issue Brief (Commonwealth Fund) , 15, 1–15.

54.

Suhrcke

, McKee

, Stuckler

, Arce

R. S.

, Tsolova

, & Mortensen

(2006). The contribution of health to the economy in the European Union. Public Health , 120(11), 994–1001.

55.

Tchouaket

E. N.

, Lamarche

P. A.

, Goulet

, & Contandrıopoulos

A. P.

(2012). Health care system performance of 27 OECD countries. International Journal of Health Planning and Management , 27(2), 104–129.

56.

WHO. (2000). The world health report: 2000: Health systems: Improving performance . World Health Organization.

57.

WHO. (2007). Everybody’s business-strengthening health systems to improve health outcomes: WHO’s framework for action .

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

1.53 MB

0.00 MB

1.44 MB

0.56 MB