Sage Journals: Discover world-class research

Abstract

This paper firstly reviews how issues of accountability and transparency have become some of the key drivers of government policy over recent years. It finds that the drive for greater higher education accountability and transparency has encouraged the growth of an international evaluation industry. It secondly discusses issues related to different types of policy and evaluation instruments across Europe, as well as the importance of rankings. It finds that traditional approaches have relied on collegiality, expert judgment, and peer review, and there is a growing focus on indicator-led systems in the belief that indicators are value-free and statistically robust so that leads to rankings have a significant part of development. The paper finally discusses and compares trends and developments of accountability and transparency in China’s higher education. It finds that China is now pursuing “World-class Universities” (WCUs) establishment and China’s experience reflect the challenges and tensions around quality and accountability.

Keywords

higher education evaluation ranking China

1 Introduction

Around the world, higher education is usually seen as serving the public interest because its benefits extend to the individual and society. People with a tertiary qualificationcan expect an earnings premium of more than 50% over people with only a secondary education (OECD, 2011). They are also likely to have better health, and to be more socially aware and civically engaged. These societal and personal benefits have reinforced an implicit “social contract” between higher education and society that balances public support through taxation and public policy in exchange for academic and institutional autonomy.

Yet, despite higher education’s significant positive impact on society and economy, universities and colleges today are often accused of being insufficiently accountable to society, and to students. There are growing concerns about student learning outcomes, graduate attributes and life-sustaining skills which are coming to dominate the discourse about higher education around the globe (Coates, 2017). These issues take slightly different forms in each country but essentially the questions being asked involve the degree of transparency and accountability about what higher education institutions (HEI), both public and private, are doing about these matters.

This paper reviews some of the issues and tensions driving the accountability and transparency agenda across Europe and the US, and asks if existing systems are (still) fit for the 21st century? After considering the issues and reviewing traditional forms of quality assurance and accreditation, as well as the role of rankings, the paper discusses the trends and developments in China. There are four main sections. Part 1 discusses some theoretical and policy concerns around accountability and transparency in the context of the massification, internationalisation and globalisation of higher education, and reviews issues and tensions driving the accountability and transparency of higher education across Europe and US. Part 2 discusses issues relating to quality, performance and accountability of higher education in Europe and US, while Part 3 discusses the relationship of rankings and quality assurance on an institutional and a system level. Finally, Part 4 situates developments in China within this context, and considers the extent to which China shares trends developing elsewhere.

2 The Accountability and Transparency Agenda

Over recent decades, the concepts of accountability and transparency have taken centre stage in public and policy discourse. The Marriam-Webster online Dictionary (2017) defines accountability as not simply accounting for or recording one’s activities, but actually acknowledging both the obligation and the responsibility to be accountable. For Bovens (2003), contemporary concerns about accountability “has moved beyond its bookkeeping origins and has become a symbol for good governance, both in the public and in the private sector.” Hence, there is an emphasis on being transparent—being responsive and answerable as well as being straightforward and truthful—for one’s actions. Bovens et al. (2014, p. 16) argue that these issues are associated with the “ever increasing complexity of governance” as well as broader concerns about elites and the misuse of public funds, “fuelled by scandal and perceived misuse of authority in both the private and public sectors” (Leveille, 2013, p. 6).

Others have tied accountability to the rise of neo-liberalism and new public management (NPM), and the adoption of private corporate mechanisms to public sector organizations, not just higher education (King, 2018, p. 42). A key factor has been the ideological and policy view that the market and competitive principles, rather than the state, are a more effective mechanism to bring about change and greater efficiency and benefit for customers and consumers, including students. This approach is commonly operationalized in terms of control and power, often with respect to using resource allocation to drive change (Morrell, 2009). In parallel, astrong evaluative culture has materialised, with an emphasis on measuring, assessing, comparing and benchmarking performance and productivity. Using a preponderance of quantitative indicators in a variety of “governance indices,” the aim is to drive, monitor and evaluate behaviour as well as focusing on/funding outputs, outcomes and impacts rather than inputs (Neave, 2012; Dahler-Larsen, 2011; Erkkila & Piironen, 2009).

Over recent decades, governments, especially in advanced democracies, have extracted themselves from direct control, ownership and/or management of (public) services. Governance had shifted from top-down intervention—in some cases micro-management—to an indirect softer, steering from a distance approach (Erkkilä, 2007). However, in the aftermath of the global financial crisis of 2008, concerns about the limits of the market in many domains, such as banking and financial services, provoked concerns about whether the pendulum had shifted too far the other way and there was now insufficient oversight. In response, many governments stepped up their role, endeavouring to (re)regulate in order to ensure a closer alignment between public and private services with societal and national objectives. The concept of “networked governance” or “value management” is often used to describe a new relationship between the state and its various agencies. It reflects a transformation in public attitudes towards public services. It acknowledges the necessity for public trust between different sectors of society (Ferlie et al., 2008), and public interest in the effective and efficient use of public resources and the contribution and value back to society. The legitimacy given to these issues helps explain why these issues command such attention and support for enhanced democratic governance and greater political accountability (Lijphart, 2012, p. 279).

Higher education has been caught up in these discussions, in many countries, because of concerns about educational relevance, graduate attributes, and the contribution of research. Universities are often accused of being disconnected from the communities in which they are located, and insufficiently interested in student learning or outcomes (Hazelkorn & Gibson, 2018a). These issues speak to concerns about holding higher education accountable and responsible

to the public for quality. It is about meeting the needs of students, society and government. It is about the effectiveness and performance of colleges and universities as well as their transparency of their efforts. Accountability is about higher education serving the public interest and about higher education as a public trust.

Eaton, 2016, p. 325

Higher education has traditionally relied on peer-review and self-reporting and has asked the public to trust this form of accountability. However, this approach no longer seems adequate when big questions are being asked about quality, performance and productivity. Questions are asked about adequacy of existing formats and/or the absence of independent or external verification mechanisms. In this fraught atmosphere, rankings have emerged to fill the gap; but their methodology is considered unsuitable, the indicators are insufficiently meaningful, and the data is unreliable. Their over-emphasis on research and elite universities has also made them educationally and politically problematic, and inappropriate for massified systems.

Issues of accountability and transparency have re-focused attention on the “social contract” between higher education and the “society of which it is a part” (Zumeta, 2011, p. 134), and the extent to which that bargain is being upheld and interests balanced. These issues reflect changing public and political attitudes, which are sometimes presented in terms of tensions between (university) autonomy vs. (societal) accountability (Scott, 1995; Estermann & Nokkala, 2009). The European University Association (EUA), for example, noted that the debate around “institutional autonomy” reflects the

constantly changing relations between the state and higher education institutions and the degree of control exerted by the state … in response to the demands of society and the changing understanding of public responsibility for higher education.

Estermann & Nokkala, 2009, p. 6

A survey for the US Association of College and University Governing Boards (AGB) also acknowledged this friction. About 57% of its members said they agreed/strongly agreed that public perceptions of higher education had declined over the past 10 years (Gallup, 2017). While people believe “some kind of postsecondary education or training” is important (Lederman, 2017), surveys and studies in the United States and United Kingdom highlight growing public concerns around credential relevance, career readiness and cost vs. price (Public Agenda, 2016; HEFCE, 2010; Immerwahr & Johnson, 2010; Lumina, 2013; Ipsos MORI, 2010). A report by the UK-based Higher Education Policy Institute (HEPI) (Neves & Hillman, 2018, p. 11) called attention to a “consistent decline in perceptions of value for money” between 2012-2017, with only a slight improvement in 2018. European students have said they would favour independent reports on the quality of universities and programmes to help them to decide where to study (Eurobarometer, 2009, p. 5). Ideological as well as deepening cultural divides, as evidenced by recent elections in the US, UK, France and other countries, have fed a narrative about higher education’s isolation from the communities and regions on which they rely and in which they reside (Pew Research Center, 2017; Inglehart & Norris, 2016).

These developments lie at the heart of an on-going public and policy debate about the role and value of higher education, particularly in the context of increasing interest in issues of value-for-money, return-on-investment, learning-gain, etc., and changing the relationship of higher education to the state (Hazelkorn, 2017, pp. 13-14; Dill & Beerkens, 2010, p. 4). These issues vary according to national context, but different experiences of the “American dream” are driving deep socio-political fissures in many advanced societies.

3 Emerging Issues in the Global Era

The previous part has discussed how issues of accountability and transparency have become some of the key drivers of government policy over recent years and finds that the drive for greater higher education accountability and transparency has encouraged the growth of an international evaluation industry. In this part, the paper discusses issues related to quality, performance and accountability of higher education across Europe, and analyses the issues related to a wide array of different types of policy and evaluation instruments.

Defining and maintaining quality, guided by norms of peer review, has been a cornerstone of the academy since the 17th century, underpinning academic-professional self-regulation and self-governance (Rowland, 2002, p. 248). University autonomy has been an important symbol of independence of thought and decision-making, enabling the academy to shape its curriculum and research, be the primary determinant of quality, and speak “truth to power,” even in politically challenging environments. University autonomy was re-affirmed in the Bologna Declaration with reference to the Magna Charta Universitatum (1988), and it continues to be recognised as a core principle in most national legislation as well as by the European Commission.

In Europe, these values were further strengthened by the Bologna Process and enshrined in quality assurance processes which are built around institutional ownership of quality with assessment mechanisms which aim to enhance rather than enforce quality. Since 2005, key components of institutional based and oriented quality assurance have been reinforced by adoption of qualifications frameworks at the European and national levels, recognition and the promotion of learning outcomes, and the paradigm shift towards student-centered learning and teaching. The objectives have been supported by a range of organisations, such as the European Association for Quality Assurance in Higher Education (ENQA) and the European Quality Assurance Register for Higher Education (EQAR), and formalised in the European Standards and Guidelines for Quality Assurance (ESG) which all operate across the European Higher Education Area (EHEA). 1 Together, these organisations have created a meta-structure (Maassen & Stensaker, 2011) which has effectively introduced new forms of governance, beyond the nation-state, which “stress […] transparency, accountability, and value-for-money for taxpayer-funded expenditure” (King, 2018).

Ensuring that qualifications are of high quality and internationally comparable and transferable is a precondition for participation in the global economy and for student and professional mobility. With the surge in the number and range of educational programmes, and educational providers, including for-profit and transnational/cross-border higher education, there are many concerns about standards, ethics, and consumer protection. While quality and pursuit of excellence are institutional strategic goals, they are also national and global goals—albeit with slightly different implications (Eaton, 2016). In this international environment, there has been growing dissatisfaction with the usefulness, robustness and comparability of traditional collegial mechanisms.

Four issues should be noted (Hazelkorn, Coates & McCormick, 2018).

First, quality is a complex term, and “despite the fact that the concept is used widely, there is no agreed-upon definition … or how it should be measured, much less improved. Everyone has their own perspective, as evidenced by the different approaches, methodologies, and choice of indicators” (Hazelkorn, Coates & McCormick, 2018; Valeikienė, 2017). Emphasis has primarily been on teaching and learning, and research, but increasingly quality extends beyond internal matters and reflects the capacity and capability of higher education to meet a variety of societal needs and demands. Collectively these different attributes matter because of the significance of (higher) education for national competitiveness and global positioning; hence there is increasingly, a geo-political aspect to quality. It is often used in association or conterminously with “excellence” as if to assert or emphasize the objective of quality. This means that quality is effectively shaped by who-ever decides, by the choice of methodology (qualitative and/or quantitative) and the indicators—rather than on the basis of standards. This means quality is variable, and hence a cause of great perplexity and unhappiness. As academics, we may understand why that is so and why context matters, but to others this seems to be just a(nother) form of obfuscation.

Second, quality assurance has been the mainstay of the academy, but the inability to provide comparability and to provide evidence in a usable and easily digestible format has become a major handicap. In the US, accreditation has come up against similar challenges. Without doubt, promotion of and embedding a quality culture is a vital first step, especially for countries and institutions with no history or understanding of these issues (Sursock, 2003). Academic self-reporting and self-governance requires taking meaningful ownership of quality management by placing responsibility onto higher education. Butthe emergence of a coterie of internationally mobile peer-reviewers—a quality “industry”—carries many of the same limitations inherent in peer-review for research. In addition, despite its observable virtues, quality assurance is often seen as being/becoming too process-oriented and insufficiently focused on real outcomes. Indeed, it often seems that the process of assessing and monitoring is just that—a process, which is arguably an inefficient use of public resources and people’s time, a system which benefits the academy (which has a proclivity towards process-oriented actions) more than students or society and is not scalable in any meaningful way.

Third, while quality-standards remain important, higher education is now being asked additional questions about performance and productivity which get to the heart of the matter. Performance involves questions of how well the institutions operate vis-à-vis their goals and those of society; hence, focus is on actual outcomes and outputs rather than simply the process. While quality assurance focused traditionally on individual institutions, performance-related deliberations have shifted attention onto academic and professional staff and students. There has been a long history of measuring research activity, but questions are now being asked about what academics produce through their teaching, and issues of academic outputs and outcomes, such as progression and graduate employment. This may be a welcome rejoinder to global rankings which overwhelmingly focus on research, but it speaks directly to public and political perceptions about what academics do all day or all year. Thus, what people want to know is how effectively students are learning, what they are achieving, and how personnel, institutions and the systems overall help students to succeed.

Fourth, assessing and evaluating performance is both a controversial and complex process. Traditional approaches have relied on collegiality, expert judgment, and peer review. More quantitative and externally-driven approaches have emerged in recent decades, including, inter alia, international as well as national rankings and bibliometric systems; multi-dimensional profiling and classification tools; teaching excellence assessments, learning gain initiatives, and wider usage of learning analytics; government databases and “score cards” alongside open-source websites; institution- and department- or field-based approaches; and national and international benchmarking frameworks such as that proposed by the OECD (Hazelkorn & Gibson, 2018b). At a national level, various countries are experimenting with re-constructing the “social contract” using a set of negotiations, such as performance agreements or compacts. While the latter enables both government and institutions to set goals, this approach can’t respond to wider demands for international comparability. Students have been an important part of the process. But, as our systems become even more diverse, participation of third-parties, including business and employers and civil society, becomes inevitable. Indeed, new technologies will make the participation of citizens easier than ever with the potential to by-pass the academy entirely.

4 Global Rankings

What all the above-mentioned issues and developments have in common is the on-going search to measure and compare quality and performance of higher education and research across national boundaries. In this environment, rankings have become an inevitable and influential tool in a globalised world. In contrast to traditional academic peer review processes, rankings are perceived as being independent and give the aura of scientific objectivity. Rankings have been around for over 100 years; there are now almost 20 global rankings and over 150 known national rankings. However, commentators and critics continually raise questions about their choice of indicators and methodology. Global rankings claim to “compare the world’s top universities” (Quacquarelli Symonds World University Rankings, 2019) 2 or “provide the definitive list of the world’s best universities evaluated across teaching, research, international outlook, reputation and more” (Times Higher Education, 2018), 3 but in truth, global rankings measure a very small sub-set of the total 18,000 higher education institutions (HEI) worldwide.

While teaching is the fundamental mission of higher education, rankings do not measure educational quality. In fact, too many of the indicators used by rankings focus on inputs which are strongly correlated to wealth (e.g. institutional age, tuition fees or endowments/philanthropy), as a proxy for educational quality (Pike, Kuh, McCormick, Ethington, & Smart, 2011). Rather, research and research-related indicators constitute approximately 70 percent of the total score for QS. Both the Academic Ranking of World Universities (ARWU) and Times Higher Education Global Rankings (THE) are 100 percent based on research/research-related indicators. The QS World University Rankings and THE rankings rely heavily on reputational indicators although this is controversial methodology due to the low response rates; they are also susceptible to bias and a self-perpetuating a view of quality. In other words, respondents can only identify universities about which s/he knows rather than offering an independent perspective. Table 1 provides an overview of what global rankings measure and do not measure. (Further discussion of these issues in Hazelkorn, 2015).

Table 1

What global rankings predominately measure

Despite these shortcomings, rankings have gained significant influence. They are used by students and parents (especially international students), governments and policymakers, businesses and the media—as well as by universities themselves. Rankings are regularly interpreted as an indicator of national and institutional competitiveness, reflecting a widespread understanding of the significant role universities play with respect to talent maximisation and knowledge production. There is a strong correlation between being highly-ranked universities and being a magnet for mobile capital and talent, including attracting international students. Accordingly, rankings have become a significant policy driver in many countries, and are often used to classify or categorise universities, to restructure the higher education system including encouraging mergers between universities to achieve greater critical mass, to allocate resources, as a means of accreditation, etc.

Because high-ranked universities are perceived to play a major role nationally and internationally, many countries have launched initiatives with the objective of boosting the ranked position of some universities. The strategy of creating world-class universities (WCU) is usually based upon replicating the characteristics of universities within the top-100 and thereby using rankings to define “excellence” (Salmi, 2017). Promoting and having world-class universities is seen as essential for ensuring success in the global economy; universities see this label as essential to its brand, especially internationally. The WCU strategy is dependent on investment in a limited number of elite universities as a result of a strategy of “selection and concentration” (Shin & Kehm, 2013, p. 11). Countries which have embarked on this approach include France, Germany, Russia, Spain, South Korea, Taiwan, Malaysia, Finland, India, Japan, Singapore, Vietnam and Latvia—as well as China.

While developed countries tend to use rankings to bolster their position or restructure their systems, emergent countries might choose to use rankings as a method to measure quality when external quality assurance (QA) systems are weak or non-existent and/or as a gauge and/or symbol of global competitiveness and engagement in/with world science. For example, Russia’s 5-100 programme aims to “target the growing gap in Russian research performance, by seeking to provide financial support for a limited number of institutions to become world-class universities” (Taradina & Yudkevich, 2017, p. 145).

Rankings can also encourage perverse behaviour by persuading universities to abandon their mission or values in favour of climbing higher in the rankings. There are however examples of universities providing misleading information about student performance in order to achieve a more favourable rank. Pursuing high-rankings can also be a costly strategy because rankings essentially reward continued high investment or resource-intensity, particularly around research. Thus, the evidence internationally shows that rankings have encouraged prestige-seeking: focusing on high-achieving students and elite researchers, on research in preference to teaching, on postgraduate students in preference to undergraduate students, and to prioritising global activity in preference to national/sub-regional societal engagement.

5 Experience in China

This section provides a brief overview of developments in China in light of the issues discussed above. To what extent does China’s experience reflect the challenges and tensions around quality and accountability that are evident in western countries, and what is the experience of rankings?

China experienced a dramatic expansion of higher education in the early 21st century. The gross enrolment ratio increased from 9.8 % in 1998 to 45.7% in 2018 (Ministry of Education, 1998-2018). In recent years, China has identified the specific goal to build up world-class universities (WCU). To achieve this, China has implemented several strategic funding initiatives such as the “211” Project and the “985” Project. After receiving large amounts of financial support from the central government, the selected institutions improved their research performance and competitiveness and narrowed the gap with other international universities (Deng et al., 2010). Table 2 illustrates the nine “flagship” universities from the Chinese Ivy league, “C9,” most notably Tsinghua University and Peking University, improving their positions in the three main global ranking: ARWU, THE and QS.

Table 2

a Table extracts from Liu, L. (2018). On the governance of ‘Newly-formed’ world-class universities: value, institution and action. Nanjing Agricultural University, Nanjing, China.

In 2015, Chinese State Council released another statement, “Coordinate Development of World-class Universities and First-class Disciplines Construction Overall Plan,” designed to lift the status and standing and international competitiveness of China’s higher education system. The statement set out an ambitious target for China to develop more WCUs and disciplines in the next three decades. In 2017, MOE, Ministry of Finance (MOF) and National Development and Reform Commission (NDRC) released detailed lists of universities and disciplines to be developed under China’s “Double First Class” (DBC) initiative. This went beyond the previous “211” Project and the “985” Project to support more leading institutions to become WCU (Liu, 2018). Under this initiative, 42 universities have been selected to develop WCU and 465 disciplines from 140 universities have identified to become world class (MOE, 2017).

Becoming an WCU requires an overall improvement in the quality of higher education sector. However, China faces similar challenges as other countries in terms of tensions around governance and maintaining or improving the quality of higher education with limited resources (Huang et al. 2014). To further enhance higher education’s global competitiveness, the Chinese government has strengthened its control over the quality of higher education and reinforced requirements for accountability of public funding.

Due to China’s unique political environment and cultural tradition, Chinese universities have a different governance structure compared with other western developed countries. As shown in chart 1, the public university in China is governed by three governing bodies—Party Committee of Chinese Communist Party (CCP), President Committee and Academic Council. These political and administrative factors dominate university governance. In this scenario, the Party Committee of CCP and President are in charge together because it is ruled by the Law of Higher Education (Jokila, 2015). The Law of Higher Education in 1998 ruled that the president leads the president committee to take overall responsibilities for supporting and enhancing the university’s overall operations, while the presidential responsibility sits under the leadership of the institutional party committee of CCP. The party committee of CCP is not only the paramount body of strategic decision-making; it is also responsible for supervising the daily operational work. Apart from the Party Committee of CCP and the President Committee, the Academic Council is the principal academic decision-making body of the university in China. Its role is to advise the President committee and the President on all matters related to the academic functioning of the university. It has delegated authority to make strategic decisions about the university’s academic aspect of policies and programs.

Chart 1

Chinese public university internal governance

Although there is still strict control from the central government and CCP, Chinese universities have experienced demands for more autonomy and accountability in recent years (Hong, 2018). In 2010, the Ministry of Education (MOE) issued the “Outline of national medium- and long-term educational reform and development plan (2010-2020 years),” providing that universities are required to establish a new type of management mechanism and modern system by reforming and improving organizational governance of universities. In order to respond the call of MOE (MOE, 2010), Peking University, Tsinghua University and Fudan University have recently established University Council or Board of Trustee to include individuals who represent diverse professions and areas of expertise. Compared with western countries, the University Council and Board of Trustee have relatively little influence on policy creation although they serve as the university’s advisory body which helps the President with mission setting, strategic planning and programs reviewing (Liu, 2018).

The Chinese Ministry of Education (MOE) has also initiated a series of institutional level quality assessment schemes to assure and enhance the quality of higher education. For example, the Quality Evaluation of Undergraduate Program scheme is one of most important schemes conducted by MOE since 2002. According to it, all HEIs providing undergraduate education are required to be compulsorily evaluated by Higher Education Evaluation Center of the Ministry of Education (HEECME) within a five-year period on a rolling basis (Liu, 2016). HEECME is also responsible for design criteria and to evaluate each institution which include following demission: faculty team construction, fundamental teaching facilities, teaching funds, subject construction, curriculum quality, quality control, learning atmosphere, quality of graduation thesis (Xiao and Chen, 2009). According to figures from HEECME, a total of 589 higher education institutions were evaluated in the first round of assessment and over the 90% of the higher education institutions were assessed excellent or good (HEECME, 2007).

The Quality Evaluation of Undergraduate Program faces significant challenges. Some researchers (Gao et al. 2006; Zhang & Xue 2009; Zou et al., 2012; Liu, 2016) have studied the effect and impact of the first round of evaluation on institution. Researchers find that the scheme will have significantly facilitated the enhancement of education quality as the evaluation results will impact on universities’ public funds allocation and use in the next finical year. However, they have recognized problems produced from the scheme such as limited teaching and learning activities assessment, universities have no authority to participate in the process of formulating the assessment scheme and use same indicators to benchmark all universities also have been recognized by these researchers.

In addition, the China Disciplinary Ranking (CDR) evaluates the disciplines of universities and colleges in Mainland China in accordance with the Discipline Catalogue of Degree Awarding and Talent Training approved by the Chinese Ministry of Education. This scheme is conducting by China Academic Degrees and Graduate Education Development Center (CDGDC) which is an administrative department directly under the Ministry of Education, operating under the joint leadership of the MOE and the Academic Degrees Committee of the State Council (ADCSC), and possessing the independent qualification of legal entity. Since its start in 2002, CDR has completed four rounds of evaluations between 2004 and 2016. In the latest evaluation, 7449 disciplines from 513 universities and colleges applied, including 94% of the disciplines awarding doctoral degrees. The evaluation data is obtained in applications from university and college (CDGDC, 2017). The evaluation results are presented in a “sub-file” manner. The specific method is to divide the first 70% subjects into 9 “classification of the quality category,” including: the first 2% (or the top 2) is A+, 2% to 5% is A (excluding 2%, the same below), 5% to 10% is A-, and 10% to 20% is B+. 20% to 30% is B, 30% to 40% is B-, 40% to 50% is C+, 50% to 60% is C, and 60% to 70% is C- (CDGDC, 2018). Through the evaluation of the effectiveness and quality of discipline construction, it helps universities and colleges to understand the advantages and disadvantages of the disciplines and the imbalances in the development process, promote the construction of disciplines, and improve the level of disciplines and the quality of personnel training (Zhang, 2009; Lin et al., 2010; Wang et al., 2016).

For these reasons, the disciplinary ranking has attracted nation-wide attention from government and public media. Many universities and colleges have even cited the ranking results in their newsletter, annual reports, strategic plan and promotional brochures to show their efforts to build international, world-class discipline. Compared with the Quality Evaluation of Undergraduate Program, CDR has some advantages around .

At the same time, the rise of CDR has not gone unchallenged as researchers have identified a number of shortcomings. Zhu and Yi (2004) have cast doubt on the transparency of data collection process; Chen et al. (2016) argue that the use of bibliometric and citation to measure the practices of different subjects is an inaccurate measures of research activity. Finally, Wang (2017) argues that the quality of each discipline depends on multiple factors inter alia, the number of faculty, pedagogical outcomes, laboratories and other facilities usage etc., yet CDR concentrates on the quality of research measurement.

6 Conclusion

This paper has discussed how issues of accountability and transparency have become some of the key drivers of government policy over recent years. Accordingly, a wide array of different types of policy and evaluation instruments, bi-lateral and international agreements, quality and qualification frameworks as well as different organisations have evolved over recent decades in an effort to individually and collectively identify ways to better measure and compare higher education performance in a global world (Gallagher, 2010; Salmi, 2015). These developments have transformed quality from something that was undertaken by individual faculty and led by institutions to something being driven and regulated by the nation-state, and increasingly propelled at an international level. Traditional approaches have relied on collegiality, expert judgment, and peer review. Quantitative and externally-driven approaches have grown in popularity in recent decades. For example, global rankings have been a significant development in recent years. Although global rankings have many shortcomings, they have gained huge influence because they are regularly interpreted as an indicator of national and institutional competitiveness. This reflects widespread understanding of the significant role universities play with respect to talent maximisation and knowledge production.

While national contexts differ, the overall experience in China is similar to what is happening elsewhere. China has a very different national and institutional governance system, but it is clear that the quality of higher education and research is becoming a major policy matter. Under the WCU construction process, the Chinese government has initiated a series of institutional level quality assessment schemes such as, the Quality Evaluation of Undergraduate Program scheme and China Disciplinary Ranking to assure and enhance the quality of higher education, however, the quality evaluations lead by MOE face significant challenges and researchers suggest it requires reform in the future.

In conclusion, higher education’s importance for national and personal prosperity, sustainability and competitiveness means that matters of education and research quality, and indeed the management and leadership of universities, is now a matter of national/government and public interest. As the paper illustrates, this is an international trend. A Rubicon has been crossed everywhere. The direction of travel is likely to involve continued government steerage and new forms of “social contract,” combined with increased use of big data.

Footnotes

1

The EHEA was established in 2010. It is one of the main achievements of the Bologna Process. Today, the EHEA consists of 48 countries that form an area with comparable and transferable higher education systems and qualifications, where mobility of staff and students is integral. The EHEA is not a product of, or similar in membership to, the European Union. Rather, it consists of voluntary member states who are not bound by legal mandates but rather a commitment to implement reforms based on common values.

2

https://www.topuniversities.com/university-rankings.

3

https://www.timeshighereducation.com/world-university-rankings.

References

Bovens, M. (2003). Public Accountability. In Paper for the EGPA annual conference, Oeiras Portugal September 3-6, 2003 to be presented in workshop 8 (Ethics and integrity of governance) (Vol. 8). Retrieved from http://doi.org/10.1093/oxfordhb/9780199226443.003.0009.

Bovens

, Goodin

R. E.

, Schillemans

, Bovens

, Schillemans

, & Goodin

R. E.

( 2014 ). Public Accountability. In Bovens

, Goodin

R. E.

, & Schillemans

(Eds.), The Oxford Handbook of Public Accountability (pp. 1-20). Oxford: Oxford University Press. http://doi.org/10.1093/oxfordhb/9780199641253.013.0012.

CDGDC. (2017, 2018). The Official Release of Chinese University Ranking by Disciplines. Retrieved from: http://www.cdgdc.edu.cn/xwyyjsjyxx/xkpgjg/283498.shtml (In Chinese).

Chen

X. F.

, Ye

Z. D.

, and Xiong

Q. N.

( 2016 ). Chinese discipline assessment: Problems and Solutions. Journal of Exploration and Controversy, 9: 59-74.

Coates

( 2017 ). The Market for Learning: Leading transparent higher education. Dordrecht: Springer.

Dahler-Larsen

( 2011 ). The Evaluation Society. Stanford: Stanford University Press.

DES. ( 2011 ). National Strategy for Higher Education to 2030. Report of the Strategy Group. Dublin: Department of Education and Skills.

DES. ( 2013 ). Higher Education System Performance Framework, 2014-2016. Dublin: Department of Education and Skills. Retrieved from http://hea.ie/assets/uploads/2017/06/DES-System-Performance-Framework.pdf.

DG Education. ( 2017 ). Quality and Relevance in Higher Education. Brussels: European Commission. Retrieved from https://ec.europa.eu/education/policy/higher-education/quality-relevance_en.

10.

Dill

D. D.

& Beerkens

(Eds.) ( 2010 ). Public Policy for Academic Quality. Analyses of Innovative Policy Instruments. Dordrecht: Springer.

11.

Eaton

( 2016 ). The question for quality and the role, impact and influence of supra-national organisations. In Hazelkorn

(Ed.) Global Rankings and the Geopolitics of Higher Education. Understanding the influence and impact of rankings on higher education, policy and society (pp. 324-338). London and New York: Routledge.

12.

Erkkilä

( 2007 ). Governance and accountability—a shift in conceptualisation. Public Administration Quarterly, 1-38.

13.

Erkkilä

, & Piironen

( 2009 ). The Iron Cage of Governance Indicators. In Cox

R. W.

(Ed.), Ethics and Integrity in Public Administration. Concepts and Cases (pp. 125-145). Armonk, New York, London, UK: M. E. Sharpe.

14.

Estermann

& Nokkala

( 2009 ). University Autonomy in Europe. Exploratory Study. Brussels: European University Association.

15.

Eurobarometer. ( 2009 ). Students and Higher Education Reform. Survey among students in higher education institutions, in the EU Member States, Croatia, Iceland, Norway and Turkey. Special Target Survey. Analytical Report. Brussels: DG Education and Culture. Retrieved from http://ec.europa.eu/public_opinion/flash/fl_260_en.pdf.

16.

Ferlie

, Musselin

, & Andresani

( 2008 ). The steering of higher education systems: a public management perspective. Higher Education, 56(3), 325-348. Retrieved from http://doi.org/10.1007/s10734-008-9125-5.

17.

Gallagher

( 2010 ). The Accountability for Quality Agenda in Higher Education. Canberra: Group of 8. Retrieved from https://go8.edu.au/publication/accountability-quality-agenda-higher-education.

18.

Gallup. ( 2017 ). The AGB 2017 Trustee Index. Trustees and Higher Education’s Proposition, Value. Washington DC: Association of Governing Boards of Universities and Colleges.

19.

Gao

, Zhang

, Chen

, Lan

, & Zhang

( 2006 ). The impact of the evaluation of the standards of undergraduate teaching work on teaching at higher education institutions: An investigative study. Journal of Higher Education, 27(11), 85-90 (In Chinese).

20.

Hazelkorn

( 2015 ). Rankings and the Reshaping of Higher Education. The Battle for World-class Excellence (2nd ed.). Basingstoke: Palgrave MacMillan.

21.

Hazelkorn

( 2018 ). Reshaping the world order of higher education: the role and impact of rankings on national and global systems. Policy Reviews in Higher Education, 2(1), 1-28.

22.

Hazelkorn

& Gibson

( 2018 a). Public policy and the public good: the changing nature of state-higher education relations. Higher Education. DOI: 10.1007/s10734-10018-0341-0343. Retrieved from https://link.springer.com/article/10.1007/s10734-018-0341-3?wt_mc=Internal.Event.1.SEM.ArticleAuthorOnlineFirst&utm_source=ArticleAuthorOnlineFirst&utm_medium=email&utm_content=AA_en_06082018&ArticleAuthorOnlineFirst_20181115.

23.

Hazelkorn

& Gibson

( 2018 b). The Impact and Influence of Rankings on the Quality, Performance and Accountability Agenda. In Hazelkorn

, Coates

& McCormick

A. C.

(Eds.) Research Handbook on Quality, Performance and Accountability in Higher Education, pgs 232-245. Cheltenham: Edward Elgar.

24.

Hazelkorn

, Coates

& McCormick

A. C.

( 2018 ). Quality, Performance, and Accountability: Emergent Challenges in the Global Era. In Hazelkorn

, Coates

& McCormick

A. C.

(Eds.) Research Handbook on Quality, Performance and Accountability in Higher Education, pgs 3-12. Cheltenham: Edward Elgar.

25.

HEFCE . ( 2010 ). Public Perceptions of the Benefits of Higher Education. Bristol and London: Higher Education Funding Council for England. Retrieved from http://www.hefce.ac.uk/media/hefce/content/pubs/2010/rd2410/rd24_10.pdf.

26.

Higher Education Evaluation Centre of the Ministry of Education. HEECME. (2007). The handbook of quality assessment of undergraduate education: Questions and answers. Beijing: Higher Education Evaluation Centre (In Chinese).

27.

Hong

( 2018 ). Public university governance in China and Australia: a comparative study. Higher Education, 3, 1-17.

28.

Huang

X. H.

, Adamson

, & Lee

C. K.

( 2014 ). The move to quality assurance in Chinese higher education: tensions between policy and practices. London Review of Education, 12(3), 286-299.

29.

Immerwahr

& Johnson

( 2010 ). Squeeze Play 2010: Continued Public Anxiety on Cost, Harsher Judgments on How Colleges Are Run. Washington D.C. and San Jose: Public Agenda and National Center for Public Policy and Higher Education.

30.

Inglehart, R. F. & Norris, P. (2016). Trump, Brexit, and the Rise of Populism: Economic Have-Nots and Cultural Backlash, HKS Working Paper No. RWP16-026. Retrieved from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2818659.

31.

Ipsos MORI Social Research Institute. ( 2010 ). Public perceptions of the benefits of higher education. Bristol: Higher Education Funding Council for England. Retrieved from http://www.hefce.ac.uk/media/hefce/content/pubs/2010/rd2410/rd24_10.pdf.

32.

Jokila

( 2015 ). The internationalization of higher education with Chinese characteristics: Appadurai’s ideas explored. Asia Pacific Journal of Education, 35(1), 125-139.

33.

King

( 2018 ). Challenges for Quality Assurance in Higher Education: The Regulatory Turn. In Hazelkorn

, Coates

& McCormick

A. C.

(Eds.) Research Handbook on Quality, Performance and Accountability in Higher Education. Cheltenham: Edward Elgar 30–41.

34.

Lederman

( 2017 , December 15). Is Higher Educaiton Really Losing the Public? Inside Higher Ed. Retrieved from https://www.insidehighered.com/news/2017/12/15/public-really-losing-faith-higher-education.

35.

Leveille

D. E.

( 2013 ). Accountability in Postsecondary Education Revisited. CSHE Research & Occasional Paper Series, 13(9).

36.

Lijphart

( 2012 ). Patterns of Democracy (Revised). New Haven, CT: Yale University Press.

37.

Lin

M. Q.

, Jiang

, and Ren

( 2010 ). The development and reform of discipline assessment. Journal of Chinese Higher Education, (21), 43-52. (In Chinese).

38.

Liu, L. (2018). On the governance of ‘Newly-formed’ world-class universities: value, institution and action. Nanjing Agricultural University, Nanjing, China.

39.

Liu, S. (2016). Higher Education Quality Assurance System in China. Quality Assurance and Institutional Transformation. Singapore: Springer.

40.

Lumina Foundation ( 2013 ). America’s Call for Higher Education Redesign. Study of the American Public’s Opinion on Higher Education. Indianapolis: Lumina Foundation. Retrieved from https://www.luminafoundation.org/files/resources/americas-call-for-higher-education-redesign.pdf.

41.

Maassen

& Stensaker

( 2011 ). The Knowledge Triangle, European Higher Education Policy Logics and Policy Implications. Higher Education, 61(6), 757-769.

42.

Magna Charta ( 1988 ). Magna Charta Universitatum. Retrieved from http://www.magna-charta.org/resources/files/the-magna-charta/english.

43.

Ministry of Education (MOE). (1998-2018). National education development statistics bulletins. Beijing: Ministry of education. Retrieved from http://www.moe.gov.cn/jyb_sjzl/ (In Chinese).

44.

Ministry of Education. (MOE). (2010). Outline of national medium and long term education reform and development 2010-2020. Retrieved from http://old.moe.gov.cn/publicfiles/business/htmlfiles/moe/info_list/201407/xxgk_171904.html. (In Chinese).

45.

Ministry of Education. (MOE). (2017). On the publication of world class universities and first-class disciplines to build universities and Colleges Notification of subject list. Retrieved from http://www.moe.gov.cn/srcsite/A22/moe_843/201709/t20170921_314942.html (In Chinese).

46.

Morrell

( 2009 ). Governance and the Public Good. Public Administration, 87(3), 538-556.

47.

Neave

( 2012 ). The Evaluative State, Institutional Autonomy and Re-Engineering Higher Education in Western Europe: The Prince and His Pleasure. Basingstoke: Palgrave Macmillan.

48.

Neves

, & Hillman

( 2018 ). Student Academic Experience Survey 2018. York and Oxford. Retrieved from http://www.hepi.ac.uk/478-2156/2013-Student-Academic-Experience-Survey.html.

49.

OECD . ( 2011 ). Education at a Glance. Paris: Organisation of Economic Co-operation and Development. Retrieved from http://www.oecd.org/education/skills-beyond-school/48630790.pdf.

50.

Pew Research Center. ( 2017 ). Sharp Partisan Divisions in Views of National Institutions. Washington DC: Pew Research Center. Retrieved from http://www.people-press.org/2017/07/10/sharp-partisan-divisions-in-views-of-national-institutions/.

51.

Pike

G. R.

, Kuh

G. D.

, McCormick

A. C.

, Ethington

C. A.

, & Smart

J. C.

( 2011 ). If and When Money Matters: The Relationships Among Educational Expenditures, Student Engagement and Students’ Learning Outcomes. Research in Higher Education, 52(1), 81-106. https://doi.org/10.1007/s11162-010-9183-2.

52.

Public Agenda. ( 2016 , 12 September). Public Opinion on Higher Education. Retrieved from https://www.publicagenda.org/pages/public-opinion-higher-education-2016.

53.

Rowland

( 2002 ). The Peer-Review Process. Learned Publishing, 15, 247-258.

54.

Salmi

( 2015 ). Is Big Brother Watching You? The Evolving Role of the State in Regulating and Conducting Quality Assurance. CHEA/CIQG Publication Series (Vol. 2).

55.

Scott

( 1998 ). Massification, Internationalization and Globalization. In Scott

(Ed.) The Globalization of Higher Education, pp. 108-129. Buckingham: Open University Press.

56.

Sursock

( 2003 ). Developing an Internal Quality Culture in European Universities. Report on the Quality Culture Project, 2002-2003. Brussels: European University Association.

57.

Valeikienė

Aurelija

. ( 2017 , November 17). The Politics of Quality Assurance in Higher Education. University World News. Retrieved from http://www.universityworldnews.com/article.php?story=2017111610182154.

58.

Wang

L. S.

( 2017 ). Reform and innovation of subject assessment in Universities under the background of “double first class”. Journal of Ningbo university, 39(01), 1-3 (In Chinese).

59.

Wang

, Cheng

H. P.

, and Yang

( 2016 ). Research on the Construction of Index System of Universities Discipline Evaluation. Chongqing higher education research, 5(01), 100-107.

60.

Xiao

X, A.

, & Chen

( 2009 ). The historical evolution of undergraduate education evaluation policy in China. Journal of National Academy of Education Administration, (02), 71-77 (In Chinese).

61.

Zhang

, & Xue

( 2009 ). The effectiveness of the feedback from the first round of the quality assessment of undergraduate education in China. Higher Education Exploration, 12, 47-53 (in Chinese).

62.

Zhang

X. H.

( 2009 ). Research on the distribution and construction of preponderant disciplines in Chinese Universities. China Higher Education Research, (04), 32-35 (In Chinese).

63.

Zhu

Y. W.

and Yi

K. G.

( 2004 ). The problems and Countermeasures in the evaluation of the first level discipline in China. China Higher Education Research, (12), 30-32.

64.

Zou

Y. H.

, Du

X. Y.

, and Rasmussen

( 2012 ). Quality of higher education: Organisational or educational? A content analysis of Chinese university self-evaluation reports. Quality in Higher Education, 18(2), 169-184.

65.

Zumeta

W. M.

( 2011 ). What Does It Mean to Be Accountable? Dimensions and Implications of Higher Education’s Public Accountability. The Review of Higher Education, 35(1), 131-148. http://doi.org/10.1353/rhe.2011.0037.

Developing Meaningful Higher Education Evaluation Systems: Putting China in Context

Abstract

Keywords

1 Introduction

2 The Accountability and Transparency Agenda

3 Emerging Issues in the Global Era

4 Global Rankings

Table 1

What global rankings predominately measure

Chinese public university internal governance

Footnotes

1

2

3

References