Post by Loredana Bellantuono, Nicola Amoroso and Roberto Bellotti

Imagine a race in which cars with a wide range of different engine powers are admitted, and there is a single overall classification: those cars with the most powerful engines obviously tend to take the biggest prizes, which can then be reinvested. Foreseeing the results of the following races is not a difficult task, since competitors that were already endowed with the best means will be even more facilitated by the resources collected through their wins. Such a mechanism increases the gap between cars with different power, and gives no award to drivers whose performances, though comparatively poor, are far beyond the expectations due to the features of their ride. 

A similar unfair competition occurs in university rankings, where institutions from the most diversified contexts, in terms of both territory and educational offer, are compared, often with no consideration for the different starting conditions. Therefore, using rankings for funding allocation and policy design can be dangerous, as the resulting awarding mechanisms can increase the social and economic gaps instead of trying to bridge them. In addition, the constant quest for certificates of prestige and awarded funding generates academic conformism: universities, and even individual researchers, adapt their scientific paths and priorities to the evaluation criteria. Hence, breaking new grounds becomes discouraged in favor of research lines that are perceived as more rewarding on a score basis.

Furthermore, positive feedback between an institution’s prestige, certified by the achieved ranked score, and performance-based funding enhances existing gaps in third mission, internationalization, research, scholarships and even diffusion of scientific ideas. It was this aspect, specifically, that prompted us to start a research line aimed at measuring and removing structural biases that inhomogeneously affect the ranking outcomes of universities from diversified territorial and educational contexts.

So, what’s behind our paper? First of all, a deep awareness of territorial bias issues in academic life, directly affecting the context in which we work. Actually, the majority of the authors are based in a university in the South of Italy, historically characterized by a lower wealth level as compared to the northern part of the country. The ongoing struggle to improve the conditions of the South of Italy is witnessed by the fact that four of its regions are included in the Convergence Objective of the European Union, since their GDP per capita is lower than 75% of the European Union average.

The second motivating factor behind this paper is our background in the science of complex systems, which we had already applied to the problem of structural inequalities within rankings, in collaboration with Mr. Vincenzo Aquaro of the United Nations Department of Economic and Social Affairs (DESA). In a 2020 paper [1], we introduced a method for a fairer assessment of country performances in ratings concerning five pillars of social development: health, innovation, environment, gender gap and sustainability. The analysis revealed that the rating of some developing countries goes well beyond the expectation based on their conditions, and such an overperformance remains hidden in the original ranking. Considering this result, it was a natural step for us to extend the study to the academic environment, that is the system to which we are most directly concerned.

Identifying external biases that affect performances of universities is a complicated task, since classifications based on a small number of proxy parameters are subject to discretionality and are not able to capture the interrelations among different institutions. Guided by the complexity of these connections, we modeled the academic environment as a network, in which universities coincide with nodes, and similarities in their territories or educational mission determine the strength of the links. The analysis of biases in rankings, initially concerning Italian universities, was then extended to a worldwide scale, focusing on the ecosystem of institutions based in OECD countries and rated in the Times Higher Education ranking. Compared with traditional statistical methods, our proposal has a twofold advantage: on one hand, similarities are determined by more than one hundred certified indicators, that fully provide a multifaceted representation of the regions they are referred to; on the other hand, the tool of network community detection determines an unsupervised and data-driven way to categorize university ecosystems.

In our analysis we used complex networks to classify academic institutions not only in terms of the territorial features of the environment they operate in, but also according to their educational offer, which can nonetheless affect rating outcomes. We unearthed the existence of a territorial bias in both the international and the Italian rankings: the performance of universities in more developed areas is systematically boosted. Results referred to the territorial bias in the Times Higher Education ranking are displayed in the figure above. Interestingly, no similar effect induced by differences in educational offer can be observed.

Besides highlighting the problem, we also identified the possible solution: the machinery of complex networks and community detection provides a way to define a fairer rating, in which the drift effect due to territorial development is removed. Remarkably, universities achieving the largest placement improvements in the redefined rankings with respect to the original ones belong to comparatively disadvantaged territorial communities. On the other hand, top performers are still based in the wealthiest communities, indicating that the excellence of some academic institutions persists even after bias removal. The Italian case is particularly interesting, since all the considered universities operate in the same legislative framework. Here, we found a gap between the positive bias on universities in the center-north and the negative bias on the south, reflecting a century-long polarization between the two parts of the country.

We believe that the algorithmic tools for fair rating developed in our work can represent a step towards optimization of resource distribution in a competitive environment such as the academic one, along with results from parallel research lines such as the investigation of ranking dynamics [3]. We plan to dedicate future work to investigating the opposite side of the matter, that is how much the advantageous features of a territory are determined by the presence of outstanding universities. 

References

[1] Bellantuono, L., Monaco, A., Tangaro, S. et al. An equity-oriented rethink of global rankings with complex networks mapping development. Sci Rep 10, 18046 (2020). https://doi.org/10.1038/s41598-020-74964-3

[2] Bellantuono, L., Monaco, A., Amoroso, N. et al. Territorial bias in university rankings: a complex network approach. Sci Rep 12, 4995 (2022). https://doi.org/10.1038/s41598-022-08859-w

[3] Iñiguez, G., Pineda, C., Gershenson, C. et al. Dynamics of ranking. Nat Commun 13, 1646 (2022). https://doi.org/10.1038/s41467-022-29256-x