Identifying and coping with the dilemmatic goals of university performance measurement systems in Finland

Abstract

Performance measurement systems (PMSs) form an integral part of universities’ management processes, but their designs and applications have received criticism. As such systems are prevalent in universities, this study aimed to highlight the dilemmatic nature of university PMSs by illustrating their contradictory goals and discussing coping mechanisms to delineate the need for PMS development. The data used in this study were obtained from 23 interviews with administrative managers at 12 university departments of three universities in Finland. The data were analysed using a dilemma approach, which is considered useful for analysing problems that include difficult choices between opposing options. The analysis identified three dilemmas in the university PMSs: 1) measuring individual performance versus the quality of academic work, 2) rewarding good performance versus using scarce resources efficiently and 3) measuring performance in teaching versus research. This paper describes how the university departments have begun to deal with the identified dilemmas. It argues that by explicitly identifying the dilemmas revolving around PMSs, their interdependence can be observed, which affects coping with these dilemmas. The results of this study suggest that extending the role of PMSs from evaluating to incorporating different forms of rewards would help reconcile the observed dilemmas.

Introduction

Many countries have harnessed universities as drivers of innovation, competition and overall prosperity. As universities are increasingly important to the economy, they face pressures on the local and global levels from various stakeholders, and their role in society is becoming more instrumental (Bedggood and Donovan, 2012). A wealth of studies suggest that universities’ sense of independence, freedom and pluralism have decreased, while the higher education sector is increasingly taking economic interests into account (Mignot-Gérard et al., 2023).

Accordingly, universities are market-driven institutions and are thus accountable to their governments and funders (Akstinaite and Lewis, 2024; Edgar and Geare, 2013). Performance-based funding appears to be the chief instrument in steering and monitoring universities because it is awarded on the basis of concrete outcomes (Bazeley, 2010; Mathies et al., 2020). As an outcome, universities strive for excellence, goal attainment, quality and effectiveness in their operations (Hansen et al., 2019) and pursue international recognition (Banker and Bhal, 2020). Owing to the introduction of new management models, outcomes related to teaching, research and public service are now deemed definable and measurable (Kallio and Kallio, 2014), and university management is distinguishable from academic work (Mignot-Gérard et al., 2023). The higher education sector today is managed with the aid of data obtained through performance measurement systems (PMSs) (Bedggood and Donovan, 2012).

Performance measurement systems refer to performance planning, measures, targets and incentives. In theory, PMSs should bring transparency, consistency and objectivity to management processes (Bloch et al., 2022). There is fair agreement as to what good academic performance is and that is why subjective assessments, such as peer reviews, are done routinely in academia (e.g., Rushforth and De Rijcke, 2024). However, the operationalisation of that performance to define objective, effective performance measures is complicated because while publications are outputs created by a researcher, the value of those outputs is only realised in their effects on others’ subsequent outputs or outcomes and their impacts beyond academia (Penfield et al., 2014). In fact, research shows that quantification of performance – through university rankings, bibliometric indicators and defined performance metrics (Mignot-Gérard et al., 2023) – does not fit well with higher education institutions and may yield unintended, negative consequences (Graf et al., 2019). In line with the observation of Craig et al. (2014) that many features of PMSs in modern universities appear to be inconsistent with universities’ original goals, it has been found that the adoption of private sector-like PMSs in the higher education sector may generate counterproductive goals (Spence, 2019). The application of PMSs is technically challenging (Moore et al., 2019) and increases bureaucracy (Kallio et al., 2016). Performance measurement systems may cause gaming with performance indicators (Bazeley, 2010; Graf et al., 2019), teaching impairment (Jiménez, 2024) and decreased perceptions of collegiality (Mignot-Gérard et al., 2023) and work motivation (Kallio and Kallio, 2014) among academics.

Owing to the inconsistencies found in academic works and PMSs, this study used Hampden-Turner's (1990) dilemma approach as its theoretical framework. Accordingly, dilemmas are defined as two conflicting but equally desirable managerial and/or organisational goals. Instead of choosing one goal over another, the dilemma approach encourages a synthesis of those goals through organisational dialogue and exploration. Dilemmas have parallels with practical challenges and complicated issues that may not have ‘right’ answers (Kuoppakangas et al., 2020). The dysfunctional outcomes of university PMSs have been widely discussed recently (Kallio and Kallio, 2014; Oppi et al., 2022; Pilonato and Monfardini, 2020), but the dilemma approach brings a new perspective to the discussion owing to its intention to reconcile conflicting goals.

The consequences of PMSs are often investigated from the perspective of academics whose performance is being measured. Recent research has acknowledged that administrative employees’ perceptions of the topic are also important, as they are supposed to have the capacity to shape PMS practices (Pilonato and Monfardini, 2020; Seneviratne and Hoque, 2023). According to Oppi et al. (2022), ambiguity is a common problem in PMSs, and public managers should recognise and then manage it. This study follows the line of research that considers the viewpoint of administrative employees to add to our understanding of university PMSs. This perspective is in line with a dilemma approach that encourages organisational dialogue (Kuoppakangas et al., 2020). Accordingly, instead of criticising the problems associated with them, this study highlights the dilemmatic nature and contradictory goals of PMSs and sheds light on the different, still evolving, ways of coping with dilemmas to develop functioning PMSs at universities. The study posed the following research questions: (1) What kinds of dilemmas do PMSs present? (2) How do university administrations try to cope with these dilemmas? The data used in this study were obtained in two stages from semi-structured interviews with administrative managers at 12 departments of 3 Finnish multidisciplinary universities.

This paper contributes to the literature by adding dialogue from the administrative staff's viewpoint. They have been shown to be rule-oriented and focused on managing control problems rather than improving performance (Seneviratne and Hoque, 2023). However, in this study, we benefited from their knowledge in working between requirements from the national and university levels, and the academic faculty. We used the administrators’ ideas to find dilemmas and alleviations for them. Using the dilemma approach, this study developed a simple framework for developing universities’ PMSs. The study concludes that extending the function of PMSs from an ‘evaluation machine’ (Bedggood and Donovan, 2012; ter Bogt and Scapens, 2012) to incorporate the aspects of more innovative rewarding would remedy the observed problems. Adverse features of PMSs do not need to be taken as given as they can be improved through relatively small, albeit concrete, actions.

Literature review

Performance measurement systems in universities

New public management-induced reforms have applied private sector management principles to the public sector and have been introduced into universities to provide improved information for decision-making and, in many cases, to support the value-for-money ideology (Hansen et al., 2019). In performance-based funding, the assumption seems to be that behaviour at both the university and staff levels should respond to incentives (Mathies et al., 2020: 33). In universities, the ideology of PMSs facilitates people to think of university institutions as manufacturers of certain products, such as degrees and research outputs (Kallio et al., 2017; Spence, 2019). However, the division of inputs and outputs is difficult to measure (Moore et al., 2019) because in the production process, education, research and service to the community are intertwined. Craig et al. (2014: 2) pointed out that the measurement and assessment mania ‘attempts to construct a vocabulary of knowledge that legitimises managerial power at the expense of more traditional and collegial visions of a university’. Bedggood and Donovan (2012) referred to an ‘evaluation machine’ to illustrate the development in which performance is assessed merely as a box-ticking exercise, rather than as a means to support genuine improvement. Thus, a volume of previous work in the area suggests that current forms of PMS are probably not the best way to manage universities in the first place (Bazeley, 2010; Kallio and Kallio, 2014). Nevertheless, PMSs have been introduced to most universities in developed countries, and universities in less developed countries are also following this trend (Seneviratne and Hoque, 2023; see also Banker and Bhal, 2020). When viewing this matter at the international level, universities seem to have become increasingly central to national innovation and competition policies (Parker, 2012). On the basis of these considerations, it is hardly surprising that universities today suffer from goal ambiguity (Oppi et al., 2022; Pilonato and Monfardini, 2020).

Overall, it seems that the new style of university management is based on accountability and control mechanisms, as well as market-based competition (Kallio et al., 2016; Parker, 2012). Although simple measures to capture the impact of research in PMS have been criticised, they are actively used in assessments, and their usability is believed in (Rushforth and De Rijcke, 2024). The same applies to university rankings and accreditations (Spence, 2019). Accordingly, research is evaluated on the basis of the number of publications (considering journal rankings), citations and external funding. Teaching is evaluated on the basis of measures such as student feedback, number of courses taught, hours in class, number of students, theses supervised and degrees awarded. However, performance measures related to research tend to carry more weight (Cadez et al., 2017; ter Bogt and Scapens, 2012). Universities increasingly compete with each other both nationally and internationally on the basis of these performance criteria, which trickle down to the level of academics (Bedggood and Donovan, 2012; Clarke et al., 2012). Studies suggest that the development towards judgemental PMSs does not necessarily increase transparency and objectivity (ter Bogt and Scapens, 2012) but may result in adverse consequences (Graf et al., 2019; Jiménez, 2024). Focusing on quantitative metrics cannot remove subjectivity from performance assessments.

Dilemma approach to PMSs in universities

The study adopts the dilemma approach of Hampden-Turner (1990) and defines a dilemma as two equally desirable managerial and/or organisational goals that seem to be contrasting. As these conflicting goals are both related and pursued simultaneously, they create contradictions. Dilemmatic goals create practical challenges, as they may be attractive separately but are still mutually exclusive. The existing literature (Hampden-Turner, 1990, 2009; Hampden-Turner and Trompenaars, 2015; Kuoppakangas et al., 2020; Suomi et al., 2014) claims that the identified management dilemmas hinder managerial work. However, the dilemma approach also proposes that contradictory managerial and/or organisational goals are not binary but can create positive outcomes in organisations through the processes of dialogue and exploration, which combine different perspectives within organisations. These processes are designed to help managers identify underlying issues and develop solutions that respect both sides of the dilemma (Kuoppakangas et al., 2020). Possible reconciliations to core managerial and/or organisational dilemmas may generate innovative new solutions, thereby adding value to existing managerial and/or organisational models (Hampden-Turner, 1990, 2009; Hampden-Turner and Trompenaars, 2015). Accordingly, the dilemma approach is used here to obtain ‘both/and’ solutions instead of ‘either/or’ answers. These ‘both/and’ solutions reconcile the two conflicting managerial and/or organisational goals (Hampden-Turner, 1990; Hampden-Turner and Trompenaars, 2015), which are often presented in the forms of continuums (Suomi et al., 2014). Dilemma management is a context-specific action without off-the-shelf resolutions (Kuoppakangas et al., 2020). It should ideally lead to reconciliation, though in practice dilemmas are not always fully resolvable and may instead be alleviated.

It can be argued that the dilemma approach would reveal ambiguities that are obstacles to the functionality of PMSs in a university context (Pilonato and Monfardini, 2020). According to Oppi et al. (2022), ambiguity in PMSs is a problem that cannot be solved. It has been reported how unpleasantly academics perceive PMSs in their work because of publication pressures, discounted role of teaching, one-dimensional career systems, increased toying with performance indicators and the loss of collegiality (Kallio and Kallio, 2014; Kallio et al., 2016; ter Bogt and Scapens, 2012). To date, a few remedies have been proposed for the problems of PMSs in universities. One example is a suggestion by Spence (2019), according to which measurement should be more enlightened, holistic and reflective. Evaluation should be balanced, incorporating aspects of academic work that are often neglected in PMSs (Kallio et al., 2017). This is because current university PMSs tend to be judgmental and backward-looking, as they were previously perceived to be developmental and future oriented (ter Bogt and Scapens, 2012). Du et al. (2023) suggest that active change management is required to create a sense of trust and support among university staff.

Two professional hierarchies exist within a university, one for academic staff and one for administrative staff. In the organisational echelons of a university, the administrative staff must attempt to get academics to act according to the rules and procedures set out by the university administration but without any actual power over them. Most academic research on PMSs has concentrated on the perceptions of academic faculty or university management. Only more recently has the viewpoint of administrative staff been examined. Pilonato and Monfardini (2020) found that administrative staff perceived PMSs more positively than academic faculty. Seneviratne and Hoque (2023) suggested that university administrators are rule-oriented and that their view of others focuses on PMS-related planning and control problems rather than improving performance. In light of the abovementioned studies, many administrators view their role as executors of prevailing national and local university policies with preset requirements and procedures. Therefore, because performance measurement may be a sensitive topic (Edgar and Geare, 2013), by interviewing administrators, who are the mediators between external requirements and the academic faculty, whose performance is being measured, we can examine the ambiguities from another viewpoint to find alleviation and even reconciliation to conflicting goals of the PMSs. This is possible because the data achieved may not be as value-laden or emotionally charged as it might have been if the interviewees had belonged to academic faculty (Kallio et al., 2016).

Research design

Research setting

The specific setting of this study is Finland, which is among the countries that regulate higher education through unified national legislation. All universities are publicly funded via the performance criteria set by the Finnish Ministry of Education and Culture (Mathies et al., 2020), and the PMSs set by the ministry directly concern all Finnish universities. Since the 1990s, management by results has been used regularly in budgetary negotiations between the ministry and each university in Finland (Kuoppala, 2005). These negotiations determine the objectives and consequent financing for each university.

In the Finnish model, output targets are established for each university to steer them. Targets represent aspects of operational performance that the university can influence. According to the model, the output targets should be set primarily as indicators (i.e., presented numerically) and only secondarily as verbal targets. However, in practice, this model, imitated from the private sector, has not been very successful, mainly because of difficulties in deciding on solid performance targets (Kallio and Kallio, 2014). The funding scheme of universities, and hence the performance measurement output targets, has been reformed several times (Kuoppala, 2005; Ylijoki and Ursin, 2013). To put it bluntly, Finnish universities compete with each other for access to scarce public resources, as in a continuous zero-sum game.

The PMS currently applied in Finland has created conflicting goals and has led to unwanted consequences, such as suboptimization and short termism (Kallio et al., 2017). Previously, collegial university management has taken steps towards a managerial model that emphasises virtues such as scientific productivity, steering, monitoring and societal effectiveness and relevance (Ylijoki and Ursin, 2013). While the purpose of the nationwide university PMS in Finland was to operate at the institutional level, it moulded the internal performance measurements of Finnish universities and their departments (Kallio et al., 2017).

Data collection and analysis

This study applied the abductive line of reasoning, which starts by identifying and confirming an anomaly in the empiria and/or literature and then conceives ideas that help explain the anomaly (Sætre and Van de Ven, 2021). In the following, the abductive research process of this study is described using the framework of Pfister et al. (2023), who modelled the abductive research process as including and moving between three theoretical abstraction levels: descriptive, analytical and explanatory. The descriptive level concerns the pragmatic description of PMSs. At the analytical level, the described practices are analysed using the theoretical concepts and perspectives of PMSs. The explanatory level focuses on the theoretical motivation of the study to find explanations for PMS phenomena.

The study started from previous research findings that showed how PMSs imitated from the private sector are not well suited for universities. The qualitative approach is considered useful for investigating how performance measurement issues play out in different contexts (Hansen et al., 2019: 572). To gain a new perspective on the controversy, the data used in this study were gathered through semi-structured interviews with administrative managers from 12 university departments in Finland (Appendix). Administrative managers at Finnish universities are not academics but are part of the administrative staff. The administrative managers interviewed represented three Finnish universities (Universities A, B and C) and, consequently, four departments in each (Education, Math and Sciences, Business and Economics and Humanities). Originally, these universities were selected to ensure comparability for the purpose of qualitative research, but the departments also represented various academic disciplines to better enable theoretical generalisations. A total of 23 interviews were conducted in two rounds. The interviews lasted from 30 to 90 min. Nine interviews were conducted personally, 13 were conducted remotely (by phone or video call) and one was conducted via email at the interviewee's own request. The interviews were recorded and transcribed.

The interviewees were asked to describe the current PMS of their department, to relate it to the national universities’ funding model and to describe the PMS of their university. They were also asked about the major changes in these systems and how employees perceived them. As administrative managers execute PMSs as set out by university management, they are in a position to observe how the PMSs are actually applied. Therefore, they were also asked how the PMSs were actually used. Analysis of the data after the first round of interviews started at the descriptive level, describing the major traits of each department's performance evaluation, which were derived from the interview data. At the analytical level, these findings were examined on the basis of the earlier literature on PMSs, specifically in the university context. Data from the second round of interviews and supplementary data from internal documents were used to validate the findings. As the interviewees repeatedly discussed how accomplishing one goal would simultaneously jeopardise another desirable goal, contradictory targets were found that the PMSs tried to capture. Accordingly, at the explanatory level, we chose a dilemma approach to examine the contradictions more carefully. Subsequently, because the explanatory level included iterating and sharpening the research design, the theoretical explanations through which the findings were explained were constantly amended and sharpened. This process included specifying the role of the dilemma approach and continuously (re)positioning the findings in the earlier literature. The series of abductive reasoning finally resulted in a set of three core dilemma pairs, which are discussed in the next section. While the interviews were conducted in Finnish and the quotes to illustrate in the empirical analysis were translated into English, the empirical analysis was also validated using the supplementary data from the internal documents. The dilemmas were universal in the 12 departments studied, but there were differences in how the departments dealt with them.

Findings: dilemmas in university PMSs

Measuring individual performance versus the quality of academic work

The first dilemma pair (D1) deals with measuring individual performance versus the quality of work carried out in universities. This dilemma entails the fact that a PMS intervenes in the work of academic faculty – at the expense of academic freedom and autonomy – as expressed in the following:

Maybe for some, it [performance measurement] is a necessary evil… And it is strange. I often run into issues [where an employee thinks] why the employer should have the right or need to know what they are doing because they do it anyway. Such rather primitive issues we wrestle with sometimes. (A2)

We look at the management team where we are going and try to look at quality. It is terribly difficult; it is very difficult to find measures for it. (B3)

Universities are, by nature, knowledge-intensive and creative organisations. This means that the work is typically guided by certain degrees of academic freedom and that it is increasingly difficult to evaluate the value and quality of individual outputs or even the outcomes and impacts of individual work, which are finally defined by peers within each discipline and theme. Thus, the current PMSs have the tendency of undermining the academic freedom of university employees or require them to pursue something in their discipline that is considered secondary or simply irrelevant and, in this way, interfere with their work. One administrative manager raised an important point:

…because [our] university is a multidisciplinary university, there are very different faculties. It is always so that it [performance measurement] cuffs some on the ear and some benefit from it… (C2)

Administrative managers also noticed that PMSs entail the possibility of controlling and steering what is done:

Nowadays, it [PMS] has become common, and I believe that little by little, the employees will learn to see that it has its advantages and it also leads to actions. But, of course, there is always that when the evaluation is done and some issues are discovered, then we should be able to put in effect the changes quite quickly… (C1)

…I would say that at our faculty, the personnel are very conscious about the individual's significance, for example, in their own department's or faculty's performance and how it is related to the whole university… (C2)

They were also aware of the potential pitfall that the difficulties in measuring the value and quantity of ‘production’ might entail encouraging academic staff to prefer quantity. Measuring performance using quantitative methods alone may create contradictions between the aims of high-quality outcomes and the quantitative outputs of academic employees. As this dilemma is not new, university departments invented means to cope with it. First, because research published in (preferably top-tier) journals is one of the most important criteria for performance measurement in higher education, working periods with no set timetables may reduce contradiction, as suggested by one interviewee in this study:

…we must pay attention to it (PMS) in work allocation, and we aim to organise research periods and support them at the faculty level. (B4)

As sabbatical systems are not widely used in Finnish universities, departments’ attempts to create conditions for good research performance are perceived as notable advancements. Second, the use of PMSs, not only for evaluation but also for incentivisation and reward, could improve the system. In one case department, the research outcomes were genuinely rewarded on the basis of performance measurement evaluations, with possible career enhancements, research time and/or bonus salaries. If performance measures are used without a well-functioning reward system, a PMS seems to only partially fulfil its aim of motivating academic employees to conduct a decent amount of high-quality research. Only one department was seemingly successful in adopting such a balanced PMS with rewards.

…we look at publications where they have come out, and that affects the person's academic role… When there is an open position, a researcher can get to it, and then we look at publications, the amount and the quality. It is unconditional. It is not a scoring system. We send the applicants’ information to experts who evaluate the applicants […]. But we have had one before, and now a performance reward system has also been introduced at the university level. (C3)

Administrative managers in the other departments admitted that they still struggled with rewarding adherence to the PMS and high-quality university work, whether it consisted of research or teaching. In summary, when considering the opposing goals of PMS in connection with the nature of academic work, the findings indicate that a possible alleviation and reconciliation of this dilemma is the combination of monetary rewards, such as higher salaries, with other forms of reward, such as enhanced career opportunities and work periods with no set timetables.

Rewarding good performance versus using scarce resources efficiently

Combining monetary rewards with PMSs may encounter difficulties because of the second dilemma pair (D2), which deals with the idea of rewarding good performance versus the efficient use of scarce resources. One intention of a Finnish university PMS immediately upon its implementation was in accordance with a balanced PMS with monetary rewards connected: If an employee performed well according to the indicators of the PMS, they would be rewarded. However, as Finnish universities are publicly funded, one main problem is the zero-sum game, which leads to a limited capacity to reward good performance. Funds allocated to rewards reduce those available for other purposes. Therefore, no matter how well the personnel work according to the indicators, the performance does not always manifest in any way in the employee's compensation or the department's resources. The interviewees highlighted the second dilemma:

I have to answer that they [the faculty] are not satisfied with the system [PMS]. They are not satisfied with the fact that they have not earned any money according to it. The most important focus of criticism during the last couple of years has been that there have been no rewards for good performance. (A4)

At the faculty, we do not have a performance reward system on either the individual or departmental levels, but then, of course, a lot of things happen in more informal ways. (B1)

The scarce economic resources of publicly funded universities are a fact that was well recognised by the interviewees. Although there is a common union and collective agreement among Finnish university personnel and the performance evaluations are unified, the problem remains. University budgets are made so tight that despite the individual employee appraisal discussions held every other year in each university after a predetermined set of issues to consider, the actual outcomes in terms of, for instance, pay increases are in most cases disappointing. There are no budget allowances to share, as the quotes that follow illustrate:

The principle is that the better you do, the better your level of performance, but there are many steps before the level of salary is defined. Nowadays, it is the rector of the university who decides each individual's level of performance. (A3)

At our faculty, no performance money has been divided for years. It has been very unfortunate for departments and disciplines and those who perform well… (A4)

Nevertheless, the public sector faces continuous pressure to develop efficiency in its organisations, including universities. The initiation of the PMS may have good developmental intentions when it is fully implemented. The administrative managers also acknowledged that the PMS should have two roles that must be implemented and balanced in a transparent manner for a university to succeed. First, the PMS is intended to evaluate the faculty's individual performance using performance indicators. Second, its role is to motivate academic employees to achieve high-quality outcomes by rewarding them.

The resources, personnel and equipment, are targeted at all levels where there is the expectation of high-quality research-teaching, in other words, results. The consequence is that we get talented and motivated researchers, teachers and equipment for those that are capable of acquiring external research funding; hire more assistant professors and research fellows, who, in turn, teach and supervise others. (B2)

[PMS] is a double-sided thing… But, for example, our incentive pay system specifically aims at motivating people to work and to do good work, and that would have an effect… Because if you think universities in general, earlier you only got nothing but a pat on the back and some sort of medal at most, so with this, the goal is to encourage a bit passive ones to achieve better results. (C4)

However, the limited capacity to reward good results contradicts the PMS's important role in motivating and evaluating. As noted earlier, it seems that in many departments, an important part of the PMS, namely motivating and rewarding, seems to remain unimplemented. While there is an acknowledged possibility of reconciling this dilemma in performance appraisal discussions with supervisors, the actual monetary effects usually remain minor owing to scarce resources. Unfortunately, administrative managers have not implemented any effective solution to this dilemma, in addition to efforts to use resources ‘wisely’. Thus, at the core of this dilemma is the collision between rewarding itself and the scarce economic resources on offer. When a PMS is not developed in a way that includes rewarding both in a monetary sense and in terms of other forms, its originally designed features will be challenged.

Measuring performance in teaching versus research

Traditionally, there is a trade-off between research and teaching in the academic world in terms of one valuable resource that an academic has: time. It can be concluded from the interviews that in many departments, the esteem imbalance between teaching and research activities is also reflected in the PMS. Thus, measuring performance in teaching versus performance in research is the third dilemma (D3). Employees whose main task is teaching are especially put in an unequal position because they might be, in many cases, evaluated with the same criteria as those with research-related tasks. The problem is even further emphasised with employees who do not have ongoing work contracts or permanent positions. The following quote exemplifies this dilemma:

…what irritates me is that we say that we put effort into teaching and its role will be increased. But, at the same time, we do nothing to it. […] Those [departments] who have invested in the development of teaching are now clearly in the situation they have to put efforts into research to cope with financing. (A2)

The quote illustrates that activities related to teaching can dilute an employee's chances of conducting research. This dilemma is evident and well known in universities. Universities are expected to conduct research and offer research-based education at the highest international level. As members of academia largely gain their professional merits through research rather than through teaching, research is typically highly esteemed. Nevertheless, to fulfil their universities’ missions, academics dedicate a significant amount of their time to teaching and other activities, such as administration and a third mission.

It is our challenge to communicate that there can be different job titles, and they [employees] must and can do different things. (B3)

The question of how to divide working hours is a fundamental issue in this dilemma. Academic employees with similar working hours may have different work descriptions. Those who are researchers usually have fewer teaching hours than lecturers, university teachers or even professors. It goes without saying that lecturers and teachers have less time to conduct research than colleagues with fewer hours of teaching. The contradiction in this dilemma (D3) is generated because in 11 of the 12 departments, the PMS evaluates both the teaching staff and researchers with the same evaluation criteria and thus ignores one of the university's core missions:

It [the PMS] has an effect. It is monetary, so it is clear. It affects people positively, although some might see it negatively. Until now, it has been the case that research is appreciated more in the academic field, and there are people who succeed in teaching and perhaps do not feel that their work is much appreciated. (C3)

In theory, performance in one of the tasks could compensate for the other, thus reconciling this dilemma:

Not everyone is necessarily great at everything, so we have to look at it by discipline and by field of science. If someone is really successful, let's say that a person gets many people to graduate, then maybe they do not succeed as much on the research side, so we have to compensate a little. (C4)

However, this may not work in practice at the departmental level, where there are fewer people. In teaching, with its multiple forms and different demands, varying from basic bachelor's to master's courses, it is challenging to evaluate and build an equal reward base. Nonetheless, one case department in this study found a solid way to reward teaching merits by establishing a transparent merit system. First, teachers are rewarded on the basis of their pedagogical qualifications. The rationale here is that if the pedagogical skills of the teacher are at a good level, the student's learning outcomes could be predictably good as well. Second, as the supervision of master's theses is somewhat demanding, time consuming and connected to students’ graduation, such supervision is also rewarded:

We ended up thinking that we can assume that if the teacher is good, the learning outcomes will be good. And we thought how the teacher can be good; others are weaker by nature, but in the educational environment, we thought that it is the education that can improve the level of teaching, so we put in the criteria that pedagogical training by the personnel [is monetarily rewarded]. Then we thought that master's thesis supervision is also arduous, so if you supervise more than 10 master's theses per year, you get [a monetary reward]. (C3)

Overall, reconciling the dilemmas, even if they were identified, is also hindered by the pace at which the PMS has been introduced to universities and departments. The administrative managers acknowledged that the pace at which the PMS was introduced to universities and departments was too fast to adapt. As a result, it might be the case that neither the university employees have internalised the PMS system as a concrete part of their work nor have the administrators or government setting up the national PMS acknowledged the dilemmas that arose, as the following quote exemplifies:

…really, the speed we have proceeded [at our university] during this year does frighten me. In my opinion, we are moving forward with too little collective discussion, which can be more. Of course, it is up to the university management which direction they want to guide the university. However, in my opinion, university management should discuss more with the community when they are taking the university in some direction. (A1)

Thus, the findings suggest that it might well be that university employees could be more able to internalise the PMS as a concrete, practical part of their work if it is introduced over a longer period, with concrete benefits to the faculty, including consideration of teaching merits.

Concluding discussion

The findings of this study suggest that administrative managers perceived the current PMSs rather positively: Performance measurement systems were viewed as an effective tool to guide faculty behaviour. This does not prevent administrative managers from seeing the apparent flaws that the systems may contain. The analysis of the first dilemma pair illustrates that in regard to the quantity-quality problem (Kallio and Kallio, 2014), administrative managers endorse outputs of high quality. In the studied departments, some intentions to organise research time and provide monetary incentives or career enhancements from high-quality work were observed. The administrative staff considered that, despite the limitations of current PMSs, they are still able to recognise good results – even when these do not conform to the ideals of academic work. The second dilemma pair leads to a problem that is not susceptible to easy fixes. Using PMSs as rewards was considered increasingly difficult, as Finnish universities were perceived to be under-resourced and to lack financing. This is evidently a common problem in higher education. For instance, Pilonato and Monfardini (2020: 9) observed that study programmes suffer from severe financial constraints, which may be used as an excuse for undesirable performance. No signs of blaming financial constraints for performance were found, but the administrative managers highlighted the need to use resources as efficiently as possible. This may lead to a trade-off in developing capabilities related to whether to concentrate on research (publications and external funding) or teaching (e.g., completed study units and degrees), let alone the third mission (interaction with stakeholders). The third dilemma again deals with a classic problem in higher education (Jiménez, 2024), intimating that PMSs emphasise research activity at the expense of teaching merits. The analysis suggests that irrespective of job descriptions, academic staff are increasingly evaluated with the criteria that highlight research outputs. However, finding time to conduct research is challenging because, in addition to the regular teaching load, structural changes, curriculum planning and other teaching-related administration require considerable time. Alongside one-sided career systems (Kallio et al., 2016), the current PMSs seem to result in narrower job roles in which conducting research is of greatest importance. This finding is echoed in the study by Jiménez (2024), who argued that the teaching tradition is in danger of ‘disappearing’ because of the current goal attainment. Only one of the departments included in this study attempted to alleviate the lack of balance between teaching and research by rewarding pedagogical qualifications and supervision of master's theses, as they are relatively standardised accomplishments.

It can be argued that universities suffer from goal ambiguity at the organisational level (Kallio and Kallio, 2014). This ambiguity originates from recent structural changes (Hansen et al., 2019) and chronic uncertainty in the external environment (Oppi et al., 2022). To alleviate and reconcile the dilemmas identified in PMSs embedded in universities, there is value in pointing out the contradictory goals these dilemmas may contain. The analysis revealed a few measures for these dilemmas that might have alleviated the observed ambiguities. The findings of this study warrant the following contributions: The current form of PMS cannot connect individual-level performance evaluations to generate monetary or any other kind of reward. In light of this, it seems that PMSs are not fully harnessed, as they tend to be ‘evaluation machines’ (Bedggood and Donovan, 2012; ter Bogt and Scapens, 2012) that exclude rewards. However, the analysis also revealed that the universities suffered from scarce resources and had not invented or implemented cost-efficient ways to connect performance measurement and rewards, which could circumvent the lack of (monetary) resources.

It can be argued that existing PMSs can be designed to be more purposeful with the help of the dilemma approach, regardless of opinions about performance measurement itself. The dilemma reconciliation with PMSs must be an ongoing process, as the interviewed administrators noted that the chances in the landscape of higher education were so rapid that universities did not have enough time to find solutions to the dilemmas. After identifying the core dilemmas and their alleviations, we noticed that the dilemmas were intertwined, similar to Kuoppakangas et al. (2020). Thus, it is possible that academics may consider the current PMSs too negatively (cf. Kallio et al., 2016), even though the root causes could be a shortage of funding and a lack of vacant positions in universities. All things considered, performance-based funding and scarce economic resources can be toxic combinations, as they may increase competition without a clear sense of reward. From the management control theory, we know that well-functioning performance measurement requires incentives and rewards. The observed dilemmas may be challenging to reconcile fully but can be alleviated. The following table can be used in the development processes of PMSs in universities. This suggests that the key reconciliation is the extension of PMSs to incorporate different forms of rewarding.

While Table 1 provides practical guidelines for managers, the results of the present study provide other managerial implications. First, if more monetary rewards are not available, the non-monetary types of rewards could be developed to be more structured to offer more freedom, career enhancement or other types of intangible rewards that academics would value (see Kallio et al., 2017). Second, teaching performance should also be rewarded by, for example, designing career possibilities for teaching faculty, as degree programmes remain the key source of income for public universities. Third, the rules and procedures for rewarding must be explicit to academics. All that is required is the incorporation of creativity and open-mindedness into the development of PMSs (Spence, 2019). Even modest but practical actions would improve their functionality. Hopefully, the present study paves the way for a culture in which the dilemmas (and observed unfairnesses) in PMSs are actively voiced and alleviations are collectively examined. This may also include discussions of meaningful outcomes from the perspective of PMSs and yield prosperity.

Table 1.

Framework for the development of PMSs in universities

Stage	Action
1	Starting point: Current design of PMSs
2	Dilemma analysis: Recognition of possibly conflicting goals and ambiguities
3	Alleviation: Identification of concrete measures perceived useful
4	Reconciliation: Extension of the PMSs’ function toward rewarding (especially non-monetary elements)

Finally, the study had the following limitations: First, it was conducted in a country where universities are public, basic education is free and universities have limited financial autonomy. Nevertheless, the core dilemmas are probably alike elsewhere because of the (continuous) modernisation agendas of higher education in many countries. Second, we did not examine all the academic departments in the universities selected for this study. Future research should investigate how universities improved their PMSs from ambivalence towards multivalence by taking into account the discords between research, teaching, third mission and resource scarcity. In all, positive studies on the developmental processes of performance management should be welcomed to convey best practices in the higher education sector (see Du et al., 2023).