Sage Journals: Discover world-class research

Abstract

In studying the performance of academic systems, one often tries to find out—by inspecting some well-defined parameters—how specific changes in the legal system influence the volume and quality of desired outputs, such as publications, patents, or alumni. This mechanistic approach neglects the fact that the main actors in this spectacle are not passive, but have their own priorities, and actively self-organize in response to any changes in their professional environment. In this contribution, we investigate such collective social behavior using the data for academic promotions in Poland in the years 2011 to 2020. Focusing on degrees higher than the PhD, we analyzed over 12,500 habilitation (= higher doctorate) and over 3,000 professorship applications, extracting personal data on academic promotion panel membership along with official roles on the panels. These details were then linked with data on the personal make-up of three consecutive terms of the central body overseeing academic promotions in Poland, Centralna Komisja do Spraw Stopni i Tytułów (“Central Board for Degrees and Titles”). As a result, we were able to identify a privileged group of individuals who controlled a substantial majority of academic promotions beyond the PhD degree by virtue of being part of the Central Board. We found that this relatively small group of academics held dominant control over academic promotions by repeatedly serving on promotion panels. Thus, despite the well-meaning intentions of the reformists who had designed it, the Polish system of academic promotions in the past decade can hardly be regarded as based on pure merit.

Keywords

academic promotion academic degree higher education law power structures social order

Introduction

In this paper, we aim to determine how the national academic promotion system is appropriated by the so-called old boys, and how the mechanisms designed for academic evaluation are used to distribute power and prestige among those old boys. Meta-research and science-of-science studies benefit from using multiple databases and automatically harvested data from, among others, social networks and publication repositories (Bornmann & Haunschild, 2018; Kousha & Thelwall, 2020). Such new sources of data may shed new light on, or reveal hidden aspects of, social processes and organization of science by shifting from micro- and meso- levels to a macro perspective, covering the entire population or multiple aspects of multidimensional phenomena. In this study, we focus on the Polish academic promotion system, which is highly centralized and—to some extent—transparent. In our research questions we aimed to establish whether members of a central body concerned with academic promotions exerted control over the process by repeatedly serving on promotion panels. Further, we looked for patterns for the different panel roles across disciplines. In order to address our research questions, we collected data on academic promotions over the course of a decade.

Related Work and Rationale

Academic promotion systems for research staff vary across countries and regions (Azman et al., 2016; Cabrera et al., 2018; Dunn et al., 2020; Reymert, 2020; Zerem et al., 2021). On the one hand, there is the tenure system, in which “tenure” is an indefinite academic appointment. The rationale for such a system is to safeguard academic freedom (appointment can be terminated only under extraordinary circumstances) and to provide stable employment conditions. Tenure-based systems function, for instance, in the United States, Canada, and the United Kingdom. In contrast to tenure-based systems lie systems based on qualifications procedures that include research degrees beyond the PhD (most commonly “habilitation”) which virtually guarantee tenure, or are a prerequisite to securing tenure or professorship in many European countries. For instance, in Germany, habilitation is a type of a higher doctorate and may be presented in the form of a thesis or a series of publications (Enders, 2001). In France, habilitation offers one way to becoming a full professor (Musselin, 2014).¹ In Italy, the Italian National Scientific Qualification is required when applying for tenure in full-professor positions (Marini, 2014; Marzolla, 2016). Habilitation procedures can be regulated at the institutional level or at the national level; the latter is the case in Austria, Italy, or Poland, among others. In these cases, a habilitation degree is usually required for institutional promotion.

National and highly centralized systems foster the creation of privileged groups that not only hold symbolic authority, but also wield real power over entire communities of scholars and research areas. Such groups in power have been referred to as “old boy networks” (Travis & Collins, 1991), whose members can use their status to control the career, success, or promotion of others. This concept, stemming from the British elite school system (Dalu et al., 2018; Wennerås & Wold, 1997), has been found useful in studies of peer review procedures as well as various biases in academia, such as nepotism, cronyism, or sexism (Becher & Trowler, 2001; Bourdieu, 1984). The power of old boy networks lies in their capacity to influence and shape the whole landscape of academic promotions, and to change the behaviors and practices of individuals or institutions. Those with a say in grant applications, hiring decisions, or academic promotions also benefit the most from the Matthew effect of accumulated advantage (Merton, 1968): the rich get even richer, and those with effective social networks get more rewards and funds, which in turn allow them to sustain and strengthen their dominance. In our study, we use the term “old boys” with reference to members of the Centralna Komisja do Spraw Stopni i Tytułów (“Central Board for Degrees and Titles”; henceforth: CK), the national body responsible for managing and controlling academic promotions in Poland. Our “old boys,” while certainly senior in age and position, need not always be male, though male academics do dominate in the CK (this gender structure is reflective of that of senior Polish academics in general: Kosmulski, 2015b). CK members are elected by other senior academics, and are endowed by law with the power to decide on the selection of degree promotion panels, including the referees. Thus, old boys not only hold power themselves, but also confer it on those they deem worthy of joining the ranks of senior academics.

The direct motivation for our research was the news that a prominent Polish academic had completed a staggering 650 official reports (the number does not include extraordinary anonymous reviews) for academic degrees (Tadeusiewicz, 2020), with 46 such reports done in 2007 alone. We wondered how much of an exception this level of concentration of power would be, and what elements in the system of academic promotions would contribute to this phenomenon. Have the numerous attempts at reforming the system addressed the problematic issues adequately?

While these questions may, at first sight, seem to be of interest mostly to the local (Polish) readers, they are in fact questions of a more general nature, as Poland’s system of academic promotions is by no means an isolated example of a highly centralized and hierarchical academic system. In fact, most Eastern European countries share a similar organization of the state and society, and the academic system is just one of its components.

In 2011, the Polish system underwent a major overhaul. Since then, the scanned documentation for all promotion procedures has been made available online to ensure greater transparency and proper monitoring of the procedures by the community (Kulczycki, 2019). We draw on this substantial collection of documents to address our research questions. Before we get there, we owe the international reader a sketch of the Polish system, to which we turn immediately below in Section 3., before presenting our Methods (Section 4.) and Results (Section 6.), all leading up to Discussion (Section 7.).

The System of Academic Promotions in Poland

Polish Academic Degrees

The Polish system of academic degrees is highly centralized, and includes two degrees beyond the level of the Ph.D. (Achmatowicz, 2011; Brzeziński, 2017; Kierznowski, 2021; Kosmulski, 2015a; Lewandowska & Kulczycki, 2021; Stec, 2021; Wojtczak, 2019). The process of awarding these life-time higher degrees is regulated at the state level, and overseen by the CK (see above), or—for the most recent cases—by a newly established though similar body (with quite a few members sitting on both) termed Rada Doskonałości Naukowej (“Council for Research Excellence”). The lower of the state degrees is called habilitacja (“habilitation”) and is essentially a local flavor of a higher doctorate (as awarded, e.g., in Denmark or Germany). Currently, about 20% of all academic staff in Poland hold this degree (OPI, 2019, p. 72). A habilitation degree allows the holder to supervise PhD theses and act as referee for higher degrees by research. An application for this degree is handled by a panel consisting of seven members in four roles (a distinction which will be important below): three referees presenting written reports, a secretary, a chair presiding over the panel, and two ordinary members.

The higher of the two centrally regulated degrees is the state title of professor (Polish: tytuł profesora). Formally speaking, it is a title: it is awarded by the president of Poland in an (optional) ceremony. However, in a lot of ways, the professor title functions as a degree, and it is widely perceived as the top rung in a model academic career. In principle, anyone can apply for it (via their higher research institution), as long as they meet the statutory criteria, which are formulated in terms of research excellence. The professor title is currently held by some 10% of academic staff (OPI, 2019, p. 72). Holders of the professor title get promoted to the rank of full professor at their institution.

A professorship application is handled by a panel consisting of five referees, all submitting written reports, which are then sent to a special rapporteur who then returns an anonymous summary report. All these judges are appointed by the CK, which strongly favors Polish academics, to the almost complete exclusion of foreign experts.

The CK—in the years covered by our study—had oversight of degree promotions and could, when justified, limit, or revoke degree-granting powers. The CK also handled any appeals by unsuccessful doctoral and habilitation candidates. Members of the CK were elected by ballot from among those holding the title of professor, and likewise only professors were eligible to vote. Given that the average age of obtaining professorship is 57 years (Najwyższa Izba Kontroli, 2017, p. 46; Rodzik, 2016, p. 172), and the title is awarded on average 20 years after habilitation (Rodzik, 2016, pp. 172–173), the CK is a statutory old boys’ club. Candidates for CK membership were proposed by faculty councils of universities with degree-awarding rights (at least the PhD level) in the candidate’s field. This was a “one professor—one vote” system, and the votes were not weighted by institution size. As Polish universities vary in size substantially, this system favored big players, able to form majority coalitions and outvote candidates proposed by smaller institutions.

For each habilitation application, the CK appointed a panel composed of three referees, a secretary, two ordinary members, and the chair, all holding at least a habilitation degree. Of these seven panel members, three were pre-selected by the degree-granting university: the secretary, one of the referees, and one ordinary member. The CK selected the other four: the chair, two referees, and two ordinary members. In this way, the higher education institution picked three out of the seven panelists. While this was only a minority on the panel, the panel’s ruling was subject to approval by the faculty council of the granting university. Although the council would normally follow the panel’s conclusion, it could—and sometimes did—reverse it. The secretary was a crucial role, because, despite being a glorified paper-pusher, it was the secretary’s job to make sure the procedure ran smoothly and conformed with the relevant laws and regulations. That is why, from the administrative point of view, it was convenient for institutions to keep a standing pool of potential secretaries familiar with the technicalities of the procedure. The easiest jobs were those of the chair and ordinary members, who were just additional voting members of the panel. All panelists received financial compensation.

In the case of the professor title, five referees were selected by the CK, which was free to choose them out of a pool of 10 (or more) candidates suggested by the degree-granting university unit, as well as any others outside this pool. The remainder of the process was governed by internal by-laws of the CK.

Research Questions

Our primary aim was to establish whether relatively small groups of academics held dominant control over academic promotions in Poland by repeatedly serving on promotion panels (as revealed by the available official documentation). We soon realized that the names that started to emerge from our preliminary analysis had one common denominator: membership in the CK. We were able to identify academics whose exceptional levels of activity in habilitation panels aligned closely with their tenure in the CK, and we wanted to find out if this was an objectively verifiable pattern across the complete dataset. Thus, our primary concern was to see to what extent the observable concentration of influence in decisions related to academic promotions might be related to formal position. Our specific research questions were as follows:

Are CK members more likely to be appointed as habilitation panelists or referees for professorship applications?

Are there identifiable patterns for the different panel roles across disciplines?

Methods

We downloaded the relevant documents from the official website of the CK (www.ck.gov.pl/promotion.html), which serves as the public repository for academic promotions of Polish academics. The relevant documents were CK decisions appointing panel members for each application. We downloaded the complete available set of these documents using a custom-written scraping script. Apparently, the CK server had not been designed to facilitate automated processing, as the data held no explicit structural tagging, and the files were mostly low-quality graphic scans of the original paper documents with no textual layer present. We parsed the HTML (hypertext markup language) files hosting the documents, extracted links to the relevant scans, and downloaded them as PDF files. Subsequently, we converted the graphic images to text, and parsed it for the names of appointed referees and other panel members.

The main data set we worked with was downloaded on December 6th, 2020. It was supplemented by 447 records that we had mirrored from the CK server in 2015, in response to CK’s surprising announcement that it would remove the documents from the public web repository 6 months after publication (a decision soon to be reversed). The whole data set contains 18,161 entries for academic promotion applications submitted between 2011 and 2019, 12,741 of which were found to contain decisions appointing members to habilitation panels, whereas 3,138 documents named referees in applications for professorship. The remaining 2,282 entries were either incomplete (2,207 cases) or else included in error and thus irrelevant (75); they were excluded from further processing.

Each PDF file was converted into the PGM (portable graymap) format, and then piped to OCR (optical character recognition) software. We tested two open-source OCR software tools of high repute: GOCR (jocr.sourceforge.net) and Tesseract (github.com/tesseract-ocr), and chose the latter based on its considerably better support for Polish, the language of our corpus of documentation.

To extract the data from the text files generated by the OCR engine, we wrote a custom parser. The script leveraged the fact that the documents, even though produced over a period of 10 years, adhered to practically the same structural template. We extracted three pieces of information from each document: (1) the names of individuals appointed to the promotion panel; (2) their roles on the panel; and (3) the year of the decision.

OCR processing presented a number of challenges, mostly due to the poor quality of some of the document images. We took a number of steps to overcome them, including manual inspection and the creation of custom term lists. A detailed account of the OCR process is given in Supplemental Material S2.

To help estimate the completeness of the CK database, we include in Supplemental Tables S1 and S2, a detailed report on the annual figures of academic promotions in Poland between 2010 and 2020, as well as the corresponding numbers of documents that should have been made public and those that have actually been published by CK. The database for professorship applications turns out to be very nearly complete, whereas between 10% and 20% of the habilitation documents appear to be missing.

Results

Appointments

All Appointments

We started the analysis of the data from a coarse-grained perspective: we assumed all CK appointments—both to habilitation panels and as professorship referees—to be equivalent, and counted their number per each academic. We used these numbers to rank individual researchers, and then used these ranks to create the Zipf-style (Zipf, 1949) plot in Figure 1.

Figure 1.

A log-log plot of all panel appointments of individuals vs. their rank. Blue pluses represent CK members, filled dots—non-members. The dashed line is a guide to the eye (slope: 0.46). Inset: a semi-log plot of the same data split into CK members and non-members.

For ranks r in the range 10 ≤ r ≤ 3,000, the number of all appointments a follows an empirical power law

a \propto r^{- β}

(1)

with the exponent β ≈ .46. This equation does not hold for very small or very large r. In the former case, equation (1) overestimates the actual number of appointments. This is most likely related to the physical limitations on the largest possible number of appointments: if equation (1) were to hold for all r, then the number of appointments for r=1 would amount to ≈335, or one panel appointment per week for 6 years without a break. For large r (r≳ 3,000), the empirical number of appointments is also much lower than the value predicted by equation (1). Actually, this tail is described better by a logarithmic decay, as will be justified below. In Figure 1 we used different symbols for CK members and non-members, which reveals that the distribution of appointments is heavily biased in favor of the members. Being a CK member during any of the three terms of office covered by our study is thus a significant factor determining the distribution of appointments. With this in mind, we partitioned the researchers into two disjoint subsets: CK members and non-members, and repeated the analysis for each subset separately: see the inset in Figure 1. Seen from this perspective, the power-law is gone and instead we have two distributions that are logarithmic, to a good accuracy. This is interesting because in many quantitative sub-disciplines of social sciences, a power-law distribution in a Zipf-like plot is considered to be the hallmark of important social interaction between the agents (Adamic & Huberman, 2002; Gabaix, 1999, 2016; Gabaix et al., 2007; Mahanti et al., 2013). Here we have data that display good power-law behavior, but when split into two natural subsets, the power law no longer holds.

The tail of the distribution for all appointments is dominated by non-members, so it is also described by a logarithmic function. This tail is formed by individuals appointed at most six times, which roughly corresponds to taking part in at most one academic promotion process per year. This rate appears to be a threshold between “casual” and “regular” panelists.

Power-law probability distributions are believed to be a universal hallmark of self-organization in complex systems (Barabási, 2002). The idea that the social network of connections formed during academic promotions is an example of a so-called small world is attractive, but we did not investigate it in detail.

Referees in Professorship Applications

Next, we investigated how the distribution of appointments depends on the roles played in academic promotion procedures. We ranked academics in five categories defined through the total number of cases where they served as referees in professorship applications; or as chairs, referees, secretaries, or else ordinary members on habilitation panels. The distributions of appointments for each of these categories are shown in Figures 2 to 6, respectively.

Figure 2.

A linear-log plot of professorship referee appointments versus rank. Symbols as in Figure 1. Inset: a log-linear plot of the number of individuals with a given number of referee appointments for professorship applications.

Figure 3.

Habilitation referee appointments versus rank. Inset as in Figure 2.

Figure 4.

Habilitation panel chair appointments versus rank. The inset zooms in on the tail of the data, in a log-log plot.

Figure 5.

Habilitation panel secretary appointments versus rank.

Figure 6.

Habilitation panel ordinary member appointments versus rank. Inset as in Figure 2.

The relationship between professorship appointments and the rank of an academic is presented in Figure 2. Note that this time we use a semi-log (linear-log) data representation, and for this reason Figure 2 is not a typical Zipfian plot. While the original layout is a popular method for visualizing power-law relationships, linearity of a semi-log rank plot implies a logarithmic relationship of the plotted variable on the rank. We use this data representation as a convenient way to show the difference, if any, between CK members and non-members.

The same data can also be presented as a function of the number of academics (S) with a given number of appointments (p), as in the inset of Figure 2. In the Appendix we show that if the rank plot is approximately linear in linear-log axes, as seen in Figure 2, then the corresponding log-linear plot of S(p) is also linear, as in the inset of Figure 2. This implies that the number of academics with a given number of appointments can be modeled by a geometric sequence: S(1)/S(2) ≈S(2)/S(3) ≈S(3)/S(4) etc.

Interestingly, CK members do not seem to be strongly favored as referees in professorship applications. The fact that in the five “top referees” there are 2 out of 462 CK members and 3 out of over 10,000 non-members suggests a bias in favor of (a few) CK members, but it can be argued that perhaps at least some CK members are top experts in their fields, which is a far more important factor than being a CK member. There are also pragmatic considerations. First, the decision to accept or reject an application for professorship is the exclusive purview of the CK, so if their members wish to exercise control, they need not get involved as referees. Second, if some CK members wish to leverage their position to profit financially from panelist compensation, they have far easier ways of doing so. Third, the applicant’s institution submits a list of 10 or more suggested referee candidates, and although the CK is not bound strictly by this list, it is both easy and convenient for the CK to draw on it. For these reasons, we treat the data for professorship refereeing as baseline reference wherein any bias in favor of members (or otherwise) should be smallest.

The hypothesis that the number of academics with a given number of appointments can be modeled by a geometric sequence is a purely phenomenological one: we do not know why this happens so. Still, we can strengthen it by verifying whether it is applicable to any cases not included in the data it was based upon. Actually, there is one such case: the number of academics who were eligible to act as referees but did not. To examine this, we fitted the data shown in the inset of Figure 2 to

y_{j} = a \exp (- bj)

(2)

with y_j denoting the number of academics assigned j appointments, and a, b being fit parameters, respectively. Assuming that the uncertainty of y_j is the square root of y_j (as expected for the Bernoulli distribution), we obtained a = 5,800 ± 400 with the value 1.7 of the regression standard error, (χ²/dof)^1/2 (the square root of the chi-squared statistic per degree of freedom) being good given the simplicity of our model. If equation (2) is correct, the number of eligible referees with no appointments is y₀ = a, or ≈5,800. On the other hand, the mean number of full professors employed in Poland in higher education (HE) institutions in the years 2012 to 2018 was 10,300. Almost all our data for professorship referee appointments dates from the years 2015 to 2019. The influx of new professors in the years 2016 to 2019 was about 500 per year (see Supplemental Material), which yields about 2000 additional names. We assume the percentage of the number of full professors employed only in the government-run research institutions (with the total of ≈3,800 research staff) to be the same as for the HE sector overall, 10% (Radon, 2020), which would yield ≈400 additional eligible referees. Further, our data suggest that the participation of retired professors and foreign referees is marginal: we estimated the number to be on the order of 100. This yields a total of approximately 12,800 eligible referees. Subtracting the 7,000 individuals known to have served as referees at least once, we arrive at y₀ ≈ 5,800 ± 500. This agrees perfectly with the value of a obtained from equation (2).

We repeated the above reasoning for CK members (details not shown), and we obtained a = 135 ± 12 with the regression standard error ≈0.9, a signature of a good fit, and y₀ ≈ 161.

Habilitation Panels

Referees

The shape of the rank plot of habilitation referee appointments (Figure 3) resembles that for professorship referees (cf. Figure 2). Using log-linear axes, we get an approximately linear plot, except for several outliers representing a few top-ranking referees. As it turns out, the top eight referees come from just two fields: economics and management. This reveals that in some academic disciplines, criteria for referee selection are so unique as to distort the statistical properties of the complete data set. The same outliers stand out even more in the plot of counts versus appointments: note the flat line at bottom right of Figure 3 inset. Equation (2) yields a rather poor fit now, even if restricted to academics with low numbers of appointments; consequently, we did not use it for habilitation referees. This failure may be related to some other fields tending to appoint referees from limited pools of candidates, but we did not follow up this thread in detail.

The number of eligible academics who were never appointed, M₀, can be estimated as in the previous section. The number of university researchers holding the habilitation degree or the professorship title in 2015 to 2019 was about 29,000 (Radon, 2020). The corresponding number in the government sector can be estimated at around 1,200. The number of new habilitation degrees over this period was some 1,750 per year, which adds up to 7,000 additional academics. This leads to a rough estimate of 30,000 ≤ M₀ ≤ 37,000, with the true value likely closer to the upper limit. As per our data, the number of academics who were appointed at least once stands at 15,905. We thus conclude that the fraction of eligible referees who were not called upon even once in the 5-year period 2015 to 2019 lies between 50% and 60%. Interestingly, applying the same calculation to CK members only, minus the outliers, we obtained a very good fit (regression standard error ≈1.18), and a close estimate of those CK members who were never habilitation referees, 124 ± 15, whereas the true number was 96, or 23% of their total number.

Chairs

Ranking Polish academics with respect to the number of their appointments as habilitation panel chairs yielded a rather telling finding: no fewer than the top-100 places are taken exclusively by members of the CK (Figure 4)!

The top non-member was ranked a distant 103rd, with just three non-members in the top-200. The top number of appointments by a single individual is also shockingly high at 142, while the highest number of chair appointments in a single year was 38. Since habilitation panels take a summer break between July and mid-September, this corresponds to one chaired meeting every 8 days. For ranks over 200, the distribution of appointments develops a fat tail. An enlargement of that part is shown in the inset of Figure 4. This is a standard Zipfian plot with two log axes. The tail is filled mostly by non-members, and the distribution of chair appointments in this region can be described by a power law, whereas the top-200 ranks are occupied almost exclusively by CK members, and the distribution here is closer to geometric. This suggests there must have been some unofficial selection criteria at play that strongly favored CK members for the position of panel chair. Recall that the entity responsible for appointing panel chairs is no other than the CK itself.

Secretaries

A secretary of a habilitation panel must hold at least a habilitation degree, and gets appointed by the academic unit processing the application. To our knowledge, this panel role is assigned exclusively to the unit’s faculty member: a practice with a profound effect on the distribution of appointments (Figure 5).

The maximum number of all appointments, 36, as well as appointments in a single year, 13, are high, considering that the secretary is expected to be an expert in the field of the application and to have enough time to get acquainted with it. We conjecture that the distribution of secretary appointments is actually a superposition of at least two distributions, each reflecting a different pattern of social behavior by the academic unit’s staff. We will illustrate them with examples. In one academic unit, the secretary role was assigned 13 times (out of a total of 15) to the same individual, and so the distribution resembles Dirac’s delta. In another academic unit, the distribution was uniform: seven nominations were assigned to seven different faculty members. In many cases, we noted a superposition of these two extremes. For example, in an academic faculty with ≈40 eligible researchers, 58 of a total of 66 secretary roles were assigned to just 3 individuals, the remaining 7 to 8, and the 30 other eligible members of this unit’s academic faculty were never appointed to this role.

Ordinary Members

Each habilitation panel includes two ordinary members (that is members that are not tasked with any special assignments such as chairing, keeping minutes, or submitting a written evaluation). One ordinary member each is appointed by the CK and the applicant’s academic unit. This mixed origin produced a combined distribution of member appointments, as the academic units almost invariably selected their candidate from among their faculty, while the CK cast its net more widely. Our data (Figure 6) reveal three data points corresponding to individuals with most appointments as outliers in the distribution.

The three outlying individuals turn out to have all worked at the same university that appointed them, and held the posts of deputy deans. The maximum number of appointments in a single year was 23. Several other academics with top ranks were also appointed predominantly by their own academic units and held important academic posts, such as members of the CK or faculty deans. In some of these cases, academic superiors (e.g., deans) were directly involved in the promotion panels of their subordinates. We believe this was (and perhaps still is) a common practice in many academic units in Poland.

The number of academics with a given number of ordinary member appointments can be, for 12 or fewer appointments, approximated by the geometric distribution (Figure 6, inset), which was already seen for professorship referee appointments (Figure 2). We believe that a rather small deviation from the geometric distribution is mainly caused by the academic units where most of the “local” appointments go to just a few faculty members. However, our data do not hold the information needed to examine this hypothesis quantitatively.

Distribution of CK Participation

Our data (Table 1) indicate that habilitation panels very often include a CK member: on average, this happens in two out of three cases. Even more surprisingly, in 67 cases (≈0.6%), CK members formed the majority in the habilitation panel (four votes out of seven). Interestingly, all but one of these panels processed degree applications in economics, showing that practices varied across disciplines (see below). These numbers are qualitatively different from those we found for professorship referees, where 90% of the procedures did not include any referee from the CK, 9% had one such referee, and 0.7% had two out of the total of five.

Table 1.

Distribution of CK Member Participation in Habilitation Panels.

C	N	r (%)
0	4,335	34.0
1	7,830	61.5
2	425	3.4
3	80	0.6
4	67	0.5

Note. N and r denote the absolute and relative numbers of habilitation committees with C CK members, respectively.

Concentration of Power in Different Academic Disciplines

We have shown above that CK members have a dominant position in the procedures of academic promotions in Poland. However, does this dominance apply equally across all academic disciplines? To try to answer this question, we computed, for each discipline, the percentage r of habilitation panels with at least one CK member as a panelist. Detailed results for all 93 disciplines are given in the Supplemental Table S3. In a vast majority of disciplines, CK members had virtually total control over the procedure. This is true, for example, of such diverse disciplines as biochemistry (100%), electronic engineering (99%), chemistry (98%), biology (94%), economics (93%), and linguistics (91%). Among the disciplines with a relatively small share of CK members on habilitation panels were physics (8%), mathematics (8%), media studies (7%), and astronomy (2%). There seems to be no objective pattern behind these findings. One might expect a positive relationship between the share of CK members involved and the level of internationalization of a discipline; that is, a large CK share in the humanities and social sciences and minimal in the chemical, physical and biological sciences. No such relationship emerges from our data. We thus conclude that the share of CK members in the academic promotion procedure is a predominantly social factor.

The data for professorship referees (Supplemental Table S4) are qualitatively different. The maximum fraction of procedures with a CK member serving as a referee is 33% (pharmaceutical sciences), and for many disciplines, including biology, chemistry, and economics, it is below 5%, or even zero (visual arts).

Further, let us define a measure of the degree of concentration of power, K₅₀, as the minimum cardinality of the set of academics who served on at least half the panels. We found this number to be 118, of which 113 were CK members for habilitations, and 228 (including 27 CK members) for professorship applications. To put this in perspective, a select group of 118 academics participated in at least half of the 12,737 habilitation procedures. Since academia breaks down naturally into almost disjoint niches (for example, physicists do not take part in academic promotions of lawyers), such a low value of K₅₀ suggests that the actual concentration of power must have taken place at the level of individual disciplines or even subdisciplines. Between 2011 and 2018, habilitation applications were formally assigned to 1 of 104 disciplines. We identified several sizeable disciplines (a hundred or more applications) with K₅₀ as low as 1 (electronic engineering, food and nutrition technology, environmental engineering). Thus, in each of these disciplines, there was one individual who participated in at least half of all panels. At the other end of the spectrum there is medicine with 1,030 habilitation panels and K₅₀ = 27. Such a relatively high value of K₅₀ does not imply that medicine is free of power concentration. Medicine is special in that it is a mega-discipline with many subdisciplines almost as far apart as physics and law.

One problem with K₅₀ is that it does not correct for discipline size: a value of K₅₀ = 1 has a different significance for sound engineering (only nine procedures) than for environmental engineering (198 procedures). To overcome this problem, we computed the ratio ρ = S/2K₅₀, where S is the number of procedures, the factor 2 counterbalances taking into account half of the procedures, and ρ is rounded to the nearest integer. This adjustment immediately reveals disciplines with most extreme concentration of power (at the Polish habilitation level): linguistics (ρ = 108), environmental engineering (99), food & nutrition technology (87), political science (84), and history (83). This means that, for example, in linguistics, K₅₀ = 2 academics served on at least K₅₀ × ρ = 2 × 108 = 216 panels. Among large (S > 100) disciplines with low values of S/2K₅₀ (ρ ≤ 20) are: mathematics (ρ = 7), physics (ρ = 11), health sciences (ρ = 11), and law (ρ = 12). However, for mega-disciplines such as medicine, the value of ρ = 19 is inconclusive. The measures r, K₅₀, and ρ are only coarse indicators of power concentration, and ultimately each discipline deserves a separate detailed analysis, which goes beyond the scope of this paper.

We also computed the values of K₅₀ and ρ for professorship procedures, but did not detect any evidence of unusual concentration of power. Detailed values of S and K₅₀ for academic disciplines (for habilitations) and broader fields of study (relevant professorships) are listed in Supplemental Tables S3 and S4, respectively.

To interpret these numbers properly, one should keep in mind that although we analyzed documents issued in the years 2011 to 2020—a period spanning three terms of office of the CK (2011–2012, 2013–2016, and 2017–2020)—the documents were not distributed evenly across these three terms. For example, the number of decisions appointing habilitation panels issued during these three terms were 215, 4,875, and 7,647, respectively. Thus, the values of parameters r, K₅₀, and ρ given in this section are most reflective of the last term of office, 2017 to 2020. However, most CK members who served on panels most frequently in 2017 to 2020 were CK members for the second or third term running. There is no evidence that the previous CK terms were less power-hungry. In fact, since CK documentation became partially available to the public only after the 2010 reforms, and since before that time the CK had virtually total control over all habilitation and professorship procedures, routinely citing reports returned by somewhat mysterious anonymous special rapporteurs (contracted by the CK on top of, and following, the reports of the regularly appointed referees), the power concentration must have been even stronger.

Discussion

Our study focuses on the composition of the committees selected by the CK, the national body responsible for managing and controlling academic promotions in Poland between the years 2011 and 2019. The role of these committees was to assess whether candidates applying for higher academic degrees were sufficiently excellent to grant them promotion. The analysis provides a novel perspective on the inner workings of highly centralized academic systems, an approach that has became viable only very recently, with rapidly growing access to digitized public administration documents. From a more general viewpoint, the documents we have studied characterize a complex social system with unknown and rather unpredictable social interactions between the parties involved. As with most other social systems, this one is composed of many loosely coupled subsystems (disciplines and subdisciplines) of various sizes and traditions, yet—as it turns out—which exhibit quite regular behavior as a whole. This suggests that our study has quantitatively captured hallmarks of spontaneous social order on a national scale.

Our major finding is that the Polish system of academic promotion does not seem to be entirely based on merit;² rather, interpersonal connections appear to be much at play. Evidence pointing to such an interpretation is that—in some disciplines—only a handful of individuals wield control over academic promotions, and that the leaders are allowed to serve on dozens of promotion panels in a single year. This level of concentration of power is made possible by the high degree of centralization in the Polish system of academic promotions, with just a handful of influential players (or, in extreme cases, just a single individual) being in control of academic promotions in a given discipline. We have shown that being selected to a panel of judges is positively related to membership in a central administrative body.

Though alternative explanations of our results are possible, they seem rather unlikely. For example, one might argue that perhaps members of the CK were selected by their CK peers to the promotion panels based on their scientific merit. This, however, would in no way explain why they would so often take on the least absorbing yet most prestigious (and influential) role of chair, as opposed to that of referee. Another finding speaking against this alternative interpretation is that panel memberships by CK members decayed sharply as soon as their term was over.

One might also contend that it should be possible to verify whether CK members are more highly cited in prestigious databases (Scopus or WoS), and such an “objective” measure could perhaps explain their dominance in promotion panels. In our opinion, any such measure would be corrupted by at least two factors. First, international databases are inadequate for disciplines like law that are tightly bound to source language and culture. Second, we deal here with a social phenomenon for which the real motives may not show up in documents. For example, our results show that in too many cases candidates could try and predict, with a high success rate, the (partial) composition of their future panel, and start rewarding these potential panellists with citations as a calculated attempt at ingratiation. Another explanation of our results could be that CK members wanted to ensure due process in promotion procedures to reduce the number of appeals. If that were so, why would some disciplines, such as mathematics or physics, do perfectly well without such systematic “oversight,” and why in some other cases panels were arranged so that CK members had a majority of votes? Yet another explanation (one which cannot be confirmed or refuted through a document-oriented study like ours) might be related to the financial remuneration received by all panel members (Prost, 1997).

Despite a high degree of formal regulation and codification at the national level, clear differences exist between disciplines. This is apparent when comparing disciplines as close as physics and chemistry, and suggests that the forces of habit dominating a particular discipline, as championed by a handful of leading players in the field, may override formal regulation. A further fundamental difference was found between appointments to, on the one hand, habilitation panels—which obeys a power-law distribution—with the most active panellist having served on no less than 143 panels; and, on the other, professorship referee panels—following a geometric distribution—with 18 appointments at most. Professorship applications might be less affected by biases and pressures from interest groups because producing a substantial written report involves a non-trivial investment of time and effort. This might seem superfluous, given that the final decision was vested with CK members anyway. Thus, the CK were able to retain control without direct participation in the refereeing process proper. By contrast, selection of habilitation panellists was evidently based on different criteria, no doubt in close relation to the fact that in this case the final decision was transferred, in 2010, from the CK to academic units.

A striking feature of the Polish promotions system is its inherent conflict of interest, which constitutes a fundamental obstacle to any successful reform. Many members of the central body like to see themselves, not as overseers to the process, but rather they feel authorized and indeed obliged to participate directly and actively in the academic promotion process. This obligation is apparently motivated by the belief that their active involvement is needed to ensure that the requisite level of research excellence is maintained. There are two problems with this logic. First, centralized control is unnecessary in disciplines that are already competitive at the international level, such as chemistry or psychology. Second, in those disciplines that lag behind the world, it is unrealistic to expect that academic institutions would be willing (or able) to elect a central representation that would ensure an international level of excellence. Moreover, we have identified more than isolated cases of subordinate-superior professional relationship between degree applicants and panelists (data not shown), a recipe for cronyism or mobbing, with the centralized character of the process only deepening the conflict.

Limitations and Future Work

The practice of awarding academic degrees studied here is a temporal social process, of which we have presented a static snapshot. The practices of the academic community, particularly referees, that shaped the documents considered here, had been formed over decades, including periods of varying social and legal circumstances. One dynamic social factor that we did not cover here is gender (im-)balance.

A question might be asked to what extent the documents we have culled and analyzed are a faithful representation of the Polish academic promotion system. While the online records for professorship applications are practically complete, those for habilitations are not (see Supplemental Material). One systematic omission are applications withdrawn after applicants learned that their applications had received negative reports. It is impossible to assess how widespread this practice was, as such cases went unrecorded. Next, in our analysis, we have not considered additional—anonymous and behind-the-scenes—special rapporteur opinions in professorship applications, on which no data are available. Their impact, however, cannot be large, since the overall number of negative decisions is small (between 2% and 8% in 2015–2019, according to the CK annual reports), and anonymous rapporteurs would likely be tougher than would regularly appointed referees. Finally, we did not consider PhD degrees. Record-holders (Tadeusiewicz, 2020) have served as external examiners in as many as 300 PhD applications. Still, we believe that none of these factors affected the statistical properties of our data, and thus the conclusions we drew from it.

Our conclusions may be limited to a particular academic system that was in force in Poland between the 2010 and 2019 reforms. It would be desirable to conduct similar studies for other academic systems as the data become available. These might include national centralized systems (France, Italy, Ukraine, Russia), as well as systems with non-central higher doctorate degrees (Germany, Austria, Czechia, Slovakia). Comparing the two might offer insight into whether the phenomena we describe are specific to centralized systems. There may be some promise in analyzing the network of connections formed during academic promotions using a small world paradigm (Barabási, 2002).

To facilitate any comparative studies, we share the raw data used for the present study (https://github.com/zkoza/ck_data).

Conclusion and Recommendations

Centralization of academic promotions may serve a range of purposes, such as wielding ideological control or promoting meritocracy. For the past thirty years, Poland (along with other Eastern European countries) has struggled to transform its academic system into one more based on merit. A major obstacle in this transition seems to have been the continued combination of three branches of power in single hands: executive (acting as referees), overseeing (a central body), and judicial (de facto awarding the degrees, and handling appeals against decisions taken by academic units or indeed its own). We would recommend that these prerogatives be curtailed, so that members of the central bodies are prohibited from serving on panels for central degrees, except in appeals.

In countries with centralized academic promotions systems and tight state control, it is a matter of consistent public policy to maintain complete documentation of academic promotions. Such documentation should be publicly available, aiming for maximum transparency (insofar as this does not infringe third-party rights). Not only would this improve the efficiency of spending public funds, but it would also enable the state to pursue an informed policy in the fields of higher education and science. Naturally, in today’s world this can only mean a fully digital process with a publicly searchable database.

In centralized systems of academic promotions, there should be a statutory limit on the number of panels that a single individual is permitted to serve on. Explicit instruments should safeguard the elimination of any conflict of interest, such as having potential panelists declare that they have no family or professional relationship or joint publications with the candidate.

Supplemental Material

sj-docx-1-sgo-10.1177_21582440231177974 – Supplemental material for Who Controls the National Academic Promotion System: An Analysis of Power Distribution in Poland

Supplemental material, sj-docx-1-sgo-10.1177_21582440231177974 for Who Controls the National Academic Promotion System: An Analysis of Power Distribution in Poland by Zbigniew Koza, Robert Lew, Emanuel Kulczycki and Piotr Stec in SAGE Open

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Ethical Approval

Not applicable due to the nature of the research.

ORCID iDs

Zbigniew Koza

Robert Lew

Emanuel Kulczycki

Piotr Stec

Supplemental Material

Supplemental material for this article is available online.

Notes

References

Achmatowicz

(2011). Blaski i cienie awansu naukowego w Polsce [The highs and lows of academic promotion in Poland]. Rocznik Towarzystwa Naukowego Warszawskiego, 74, 5–26.

Adamic

L. A.

Huberman

B. A.

(2002). Zipf’s law and the Internet. Glottometrics, 3(1), 143–150.

Azman

Che Omar

Md Yunus

A. S.

Zain

A. N. M.

(2016). Academic promotion in Malaysian public universities: A critical look at issues and challenges. Oxford Review of Education, 42(1), 71–88. https://doi.org/10.1080/03054985.2015.1135114

Barabási

A.-L.

(2002). Linked: The new science of networks. Perseus.

Becher

Trowler

P. R.

(2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines (2nd ed.). The Society for Research into Higher Education & Open University Press.

Bornmann

Haunschild

(2018). Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000 Prime data. PLoS One, 13(5), 1–12. https://doi.org/10.1371/journal.pone.0197133

Bourdieu

(1984). Homo academicus. Stanford University Press.

Brzeziński

J. M.

(2017). O poprawianiu (ale też i o psuciu) systemu przeprowadzania awansów naukowych w Polsce w latach 1990-2017 [On improving (but also spoiling) the system of academic promotions in Poland in the years 1990-2017]. Nauka i Szkolnictwo Wyższe, 2(50), 147–172. https://doi.org/10.14746/nisw.2017.2.7

Cabrera

Roy

Chisolm

M. S.

(2018). Social media scholarship and alternative metrics for academic promotion and tenure. Journal of the American College of Radiology, 15(1), 135–141. https://doi.org/10.1016/j.jacr.2017.09.012

10.

Centralna Komisja do spraw Stopni i Tytułów. (n.d.). Sprawozdania z działalności. [Annual Reports] Retrieved 1 March 2022, from https://www.ck.gov.pl/articles/id/18.html

11.

Dalu

M. T.

Gunter

A. W.

Wasserman

R. J.

Dalu

(2018). Owning the lake, not just the rod: The continuing challenge of ‘the old boys’ in knowledge production. South African Journal of Science, 114(7–8), 11–12.

12.

Dunn

L. B.

Iglewicz

Zisook

(2020). How to build a national reputation for academic promotion. In Roberts

L. W.

(Ed.), Roberts academic medicine handbook (pp. 515–523). Springer International Publishing. https://doi.org/10.1007/978-3-030-31957-1_58

13.

Enders

(2001). A chair system in transition: Appointments, promotions, and gate-keeping in German higher education. Higher Education, 41(1–2), 3–25. https://doi.org/10.1023/A:1026790026117

14.

Gabaix

(1999). Zipf’s law for cities: An explanation. The Quarterly Journal of Economics, 114(3), 739–767.

15.

Gabaix

(2016). Power laws in economics: An introduction. The Journal of Economic Perspectives, 30(1), 185–205.

16.

Gabaix

Gopikrishnan

Plerou

Stanley

(2007). A unified econophysics explanation for the power-law exponents of stock market activity. Physica A: Statistical Mechanics and Its Applications, 382(1), 81–88. https://doi.org/10.1016/j.physa.2007.02.030

17.

Kierznowski

. (2021). Stopnie naukowe i stopnie w zakresie sztuki: Komentarz [Academic degrees and degrees in the fine arts: A commentary]. Wydawn. Difin. https://repozytorium.uwb.edu.pl/jspui/bitstream/11320/11238/3/L_Kierznowski_Stopnie_naukowe_Komentarz.pdf

18.

Kosmulski

(2015a). Careers of young Polish chemists. Scientometrics, 102(2), 1455–1465. https://doi.org/10.1007/s11192-014-1461-x

19.

Kosmulski

(2015b). Gender disparity in Polish science by year (1975–2014) and by discipline. Journal of Informetrics, 9(3), 658–666. https://doi.org/10.1016/j.joi.2015.07.010

20.

Kousha

Thelwall

(2020). COVID-19 publications: Database coverage, citations, readers, tweets, news, Facebook walls, Reddit posts. Quantitative Science Studies, 1, 1068–1091. https://doi.org/10.1162/qss_a_00066

21.

Kulczycki

(2019). Field patterns of scientometric indicators use for presenting research portfolio for assessment. Research Evaluation, 28(2), 169–181. https://doi.org/10.1093/reseval/rvy043

22.

Lewandowska

Kulczycki

(2021). Academic research evaluation in artistic disciplines: The case of Poland. Assessment & Evaluation in Higher Education, 47, 284–296. https://doi.org/10.1080/02602938.2021.1893651

23.

Mahanti

Carlsson

Mahanti

Arlitt

Williamson

(2013). A tale of the tails: Power-laws in internet measurements. IEEE Network, 27(1), 59–64.

24.

Marini

(2014). Italy’s new requirements for academic careers: The new habilitation and its worthiness (No. 201403; CERIS Working Paper, p. 30). Institute for Economic Research on Firms and Growth - Moncalieri (TO) ITALY -NOW- Research Institute on Sustainable Economic Growth - Moncalieri (TO) ITALY.

25.

Marzolla

(2016). Assessing evaluation procedures for individual researchers: The case of the Italian National Scientific Qualification. Journal of Informetrics, 10(2), 408–438. https://doi.org/10.1016/j.joi.2016.01.009

26.

Merton

R. K.

(1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. https://doi.org/10.1126/science.159.3810.56

27.

Musselin

(2014). Towards a European academic labour market? Some lessons drawn from empirical studies on academic mobility. Higher Education, 48(1), 55–78.

28.

Najwyższa Izba Kontroli. (2017). Informacja o Wynikach Kontroli. Rozwój Kadr Naukowych [Audit Results. Research Staff Development] (KNO.411.002.00.2016; p. 60). https://www.nik.gov.pl/plik/id,14767,vp,17235.pdf

29.

OPI. (2019). Szkolnictwo wyższe w Polsce w latach 2012–2018 [Higher education in Poland 2012-2018] (p. 412).

30.

Prost

E. K.

(1997). Centralna Komisja ds. Tytulu Naukowego i Stopni Naukowych—Nowa kadencja i impresje [Central board for degrees and titles: A new term and some thoughts]. Medycyna Weterynaryjna, 53(4), 202–203.

31.

Radon. (2020, December 25). Kadra uczelni w Polsce w latach 2012–2018 [The staff of universities in Poland in 2012–2018]. https://radon.nauka.gov.pl/raporty/Kadra

32.

Reymert

(2020). Bibliometrics in academic recruitment: A screening tool rather than a game changer. Minerva, 59, 53–78. https://doi.org/10.1007/s11024-020-09419-0

33.

Rodzik

(2016). Nadane stopnie i tytuły naukowe – czy coś się zmieniło? [Degrees and titles conferred: Has anything changed?] Nauka i Szkolnictwo Wyższe, 2(48), 139–174. https://doi.org/10.14746/nisw.2016.2.7

34.

Stec

(2021). Nowe ścieżki kariery akademickiej – habilitacja 2.0 [New academic career paths: Habilitation 2.0]. Państwo i Prawo, 2021(7), 3–18.

35.

Tadeusiewicz

(2020). Kształcenie kadr [Staff training]. https://www.cri.agh.edu.pl/uczelnia/tad/ksztalcenie_menu.php

36.

Travis

G. D. L.

Collins

H. M.

(1991). New light on old boys: Cognitive and institutional particularism in the peer review system. Science, Technology, & Human Values, 16(3), 322–341. https://doi.org/10.1177/016224399101600303

37.

Wennerås

Wold

(1997). Nepotism and sexism in peer-review. Nature, 387(6631), 341–343. https://doi.org/10.1038/387341a0

38.

Wojtczak

(2019). Habilitacja jako warunek awansu naukowego w Polsce w latach 1990–2018 [Habilitation as a condition of academic promotion in Poland in the years 1990–2018]. Studia Prawa Publicznego, 3(27), 29–72. https://doi.org/10.14746/spp.2019.3.27.2

39.

Zerem

Kunosić

Imširović

Kurtčehajić

(2021). Science metrics systems and academic promotion: Bosnian reality. Psychiatria Danubina, 33(3), S371–S377.

40.

Zipf

G. K.

(1949). Human behavior and the principle of least effort. Addison-Wesley.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.09 MB