Evaluating Complex Higher Education Systems

Abstract

In this article we introduce the five papers published in this issue of the International Journal of Chinese Education (IJCE). We begin by discussing complexities shaping the analysis of education, then turn to each paper’s nature and contributions. The article concludes by introducing revised IJCE editorial arrangements.

1 Clarifying Evaluation Complexities

This issue of the International Journal of Chinese Education (IJCE) continues publication of the contributions made to the July 2018 conference held at Tsinghua University titled “Constructing Higher Education Evaluation Systems for the Global Era”. As such, it provides analysis from global thought leaders on prospects for improving the evaluation of higher education. The issue also includes a paper from IJCE’s regular publication schedule.

The four conference-related papers in this issue each tackle large questions. What, in essence, are the shortcomings of prevailing ways for understanding and reporting university characteristics and performance? What evaluation mechanisms would better help to progress universities and students, and thereby the contributions the sector makes to broader society?

Reviews of relevant higher education policy in recent decades (see: Hazelkorn, Coates & McCormick, 2018; Cantwell, Coates & King, 2018) reveal two distinct positions with respective to such inquiry. These positions are underpinned by substantial institutional, professional and commercial interests. They are characterised epistemologically for current purposes.

On the one hand, myriad ratings, reviews, rankings and classifications have been promulgated in recent decades, reflecting efforts to understand and lead a sector undergoing substantial transformation in both substance and scale. Such mechanisms have flourished by building reductive information architectures around very complex situations. They have filled a void and sated people’s need to make reasonably simple sense of out of what universities do and achieve.

On the other hand lie persistent arguments about the irreducibility of higher education. Higher education is indeed a ‘complex system’ (Cairney, 2012) which thwarts quick reductionism. Already on the second glance, for instance, even seemingly simple information relating to graduate employment unfolds into complexities around qualifications and skills, and the changing characteristics of contemporary professional work.

It is difficult or perhaps too soon to tell if such positions are anachronistic, enduring, conflicting or even relevant. They do help derive implications, however, which aid ongoing inquiry into higher education evaluation methodology. For instance, there are likely no simple or static answers to the large questions posed above. Hence there is value in ongoing reform of evaluation mechanisms. Often controversial simple indicators can play a powerful role in guiding practice. However, to ‘cut-through and shape practice’ evaluation mechanisms should resonate with both positions. The dialect revealed by these two positions carries important implications for policy and practice, hence for evaluation research.

2 Papers in This Issue

Each of the four papers on higher education in this issue of IJCE touches on such duelling complexities and prospects. In response to the above conference theme, which itself is underpinned by a national key research project (Shi, 2017) framed in response to major policies like the ‘Double World-Class’ Construction Project (State Council, 2015), the papers touch on conflicts between qualifications and skills, the shortcomings of prevailing ranking schemes, the need to re-orient evaluation practices towards education improvement, and the potential value of rebuilding evaluation systems around defined capabilities. Far from racing towards quick solutions, the papers reveal the value for evaluation hence higher education in ongoing methodological debate.

In the first paper, Van Damme tackles tensions between the homogenising trajectory of internationalisation and the need for more regionally nuanced conceptualisations of skills. Rather than focus policy on the production of qualifications by universities, economies need to focus on producing a “well-balanced, harmonious portfolio of knowledge and skills”. The specification of required skills needs to be read in relation to employment contexts pertinent to countries, regions or industries. Evaluation, consequently, needs to consider not just education processes and qualification outcomes, but also the extent to which universities function in broader innovation ecosystems to contribute required scientific and professional skills.

The next paper by Ramakrishna and Sachsenmeier adopts an institutional perspective to review examples of the kind of bibliometric reports which have sprouted over the last twenty years. It identifies limitations of such reports and discusses options for development. It argues for investment in information and reports on how universities work beyond campus walls to impact employers, and other institutions and countries. International connectivity, for instance, is articulated as a key measure of universities’ evaluation. In little over two decades bibliometrics has grown from being the quiet technical work of librarians into a multi-billion-dollar global enterprise carrying material implications for innovation systems. Similar growth is possible for the emerging mechanisms discussed in this paper.

Cantwell, next, looks at how evaluations might be designed to take account of disciplinary and organisational complexities inside the academy. He untethers higher education from prevailing status-based evaluations and instead examines prospects for rebuilding evaluation around capabilities. The paper gives life to this approach, looking at examples in terms of student learning, research performance and revenue generation, and unpacks the potential multilevel application within universities and within academic units in particular. The paper concludes by exploring the application of this methodology to create higher education systems that meet social demands.

The paper by McCormick delves into the need for evaluation systems to incentivise evidence-informed improvement efforts. Looking from a policy perspective, McCormick reviews the rise of performance indicators and accountability mechanisms. The paper reveals how these mechanisms run into complexities associated with inherent norms, structures and practices of higher education. As a result of such misalignment, the mechanisms have a limited capacity to promote performance improvement. McCormick advances an alternative perspective which re-positions evidence-informed improvement in a primary rather than subordinate or derivative role. Educational change requires improvement, not just the collection and reporting of information.

The final paper by Mok continues IJCE’s regular publication schedule. It explores lessons from reforms in mathematics teaching in Hong Kong. The analysis adopts the lens of large-scale international assessment studies, and of more traditional framings of education in Hong Kong including Confucian conceptions of learning. This final paper varies in terms of its practical and technical nature, but in a broader way is a useful juxtaposition to the earlier papers. For instance, it frames complexities and debates about education and learning using empirical sources which are only just starting to emerge in higher education.

The editors are very grateful to the authors for the contributions, and to the peers who reviewed and gave helpful comments on earlier drafts. Each paper clarifies a unique response to the large questions framed above. We hope that each paper contributes insight into the ongoing construction of effective evaluation mechanisms for future higher education.

3 Editorial Changes

We report changes in IJCE editorial arrangements to conclude this article. We welcome new colleagues. It is also timely to thank all of the experts and scholars who have served and contributed in previous years in the editorial office, on the editorial board, as authors, and as advisors and reviewers.

In 2018, Jinghuan Shi (Tsinghua University) assumed the role of IJCE’s Chief Editor. Jinghuan Shi is supported by Hamish Coates (Tsinghua University) as Executive Chief Editor, and by Lu Liu (Tsinghua University) as Managing Editor. Several Tsinghua University graduate students assist with editorial duties, keeping production moving while also learning valuable publication skills.

The IJCE Editorial Board has also been refreshed. It now includes (in alphabetical order by surname): Michael Apple (University of Wisconsin-Madison), Nicholas Dirks (University of Berkeley), Leo Goedegebuure (University of Melbourne), Ruth Hayhoe (University of Toronto), Futao Huang (Hiroshima University), Jeroen Huisman (University of Ghent), Wing On Lee (Zhengzhou University), Jing Lin (University of Maryland), Kelly Matthews (University of Queensland), Gary McCulloch (University College London), Joshua Ka Ho Mok (Lingnan University), Gerard A. Postiglione (The University of Hong Kong), Yingyi Qian (Tsinghua University), Zhongying Shi (Tsinghua University), Guo Shibao (University of Calgary), Jung Cheol Shin (Seoul National University), Bjørn Stensaker (University of Oslo), Olga Troitschanskaia (University of Mainz), Robert Wagenaar (University of Groningen), Lan Xue (Tsinghua University), Rui Yang (University of Hong Kong), and Qiang Zha (York University).

The editorial team has clarified IJCE’s scope and coverage. Specifically, IJCE aims to strengthen Chinese academic exchange and cooperation with other countries in order to improve education research and promote education development. The journal welcomes both empirical and theoretical studies on popular and policy issues. Articles can address all China-related education disciplines, education phenomena and education problems, including comparative studies. Articles should be of interest to both scholars and advanced students, specialists and informed readers, as well as policy makers. Authors should submit papers online (www.editorialmanager.com/ijce). Please contact the editorial office for any questions or problems regarding IJCE or a manuscript: [e-mail].

Please engage with IJCE. There are many options, including endorsing IJCE to authors, to readers and to subscribers, encouraging colleagues to submit articles, assisting with peer reviews, identifying topics for commissions and special editions, suggesting potential authors, and of course submitting your own papers, perhaps in collaboration with emerging scholars. We look forward to greater collaborations which advance education equity and excellence.