We all understand the importance of evaluation. Evaluation is a process that facilitates continuous improvement, allows us to critique our work, establish improvement plans, identify problems, and encourage progress. Evaluation utilizes various instruments to obtain metrics and assessments about the quality of products and/or services. Unfortunately, academic evaluation systems rarely employ consensus-based and/or validated evaluation instruments. Moreover, it is uncommon for these systems to adjust to the particularities of the object being evaluated, its contribution to the research field, its relevance for scientific advancement, its practical utility, the population that could benefit, and its ability to foster dialogue and critical analysis.
It is crucial that before implementing hiring or promotion policies for academic staff based on commercial metrics, funders—such as universities, research centers, and government agencies—consider the sociocultural and economic context of the region, reflect on the institutional mission and vision, and develop or adapt academic evaluation instruments that respond to the specific needs of the field of study, researchers, beneficiary population, among others. These practices can contribute to their staff developing high-quality, ethically responsible research with a high degree of creativity. The generation of a research product intrinsically depends on an inspiring creative process, resulting in a diversity of academic products as broad as inspiration itself.
Among the academic products that can be used to evaluate the activity of academics and researchers are monographs, book chapters, complete books, seminars, the design and development of databases, specialized or dissemination websites, videos, posters, Journal clubs, academic and social impact programs, curricular designs, evaluation instrument designs, conferences, software development, prototypes, critiques of educational programs, repository development, brochures, newspapers, notebooks, newsletters, technical reports, thesis supervision, articles, institutional linkage activities, participation in academic advisory to institutions, among others. All of them must prioritize quality over quantity, and comply with ethical research, publication, and dissemination practices of science and culture. Moreover, the evaluation of an academic or researcher should also include their contribution to human resources training, mentorship programs, and the direction of research groups.
It is essential to avoid falling into the erroneous policy of "publish or perish," which, far from fostering significant advances in science, can result in an abundance of publications of little value. To this end, institutions must overcome the fear of investing in large-scale projects that require years of work and collaboration from numerous research teams and undergraduate and graduate students. For these projects, it is essential to develop specific evaluation instruments that consider the individual contribution of each member, the stage of advancement of the project, and the contributions in their area of specialty.
Therefore, both researchers and institutions must strive to create evaluation instruments that are relevant to their field of specialization and consistent with their mission and vision, thus promoting the professional and personal development of their research teams to foster interdisciplinary, multidisciplinary, and transdisciplinary collaboration.
The responsibility of designing suitable academic evaluation instruments falls on us; to achieve this, we must establish contextualized and collegial criteria. Moreover, it is crucial not to rely exclusively on commercial metrics, which have different purposes than academic evaluation.
Promoting an integral evaluation system implies a commitment to developing methods that are both necessary and sufficient to achieve the desired objectives and outcomes. This can include the use of qualitative research methods, such as focus groups, to better understand the problem and then design evaluation tools that offer approaches to try to "measure" the quality and relevance of research products through rubrics, evaluation scales, and checklists, as well as the formation of committees for collegial evaluation.
The paradox of institutions that value the quantity of publications over quality underscores the need for a more integral and qualitative approach in academic evaluation. It is essential that we design comprehensive academic evaluation processes to foster quality science, add value to research, and build teaching and learning communities that promote the formation of innovative researchers and generate meaningful hypotheses to advance in science and improve society.
Please sign in or register for FREE
If you are a registered user on Research Communities by Springer Nature, please sign in