CREAF's recent adhesion to CoARA (Coalition for Advancing Research Assessment) seeks to contribute to transforming the procedures and indicators by which research is evaluated. More than 350 organizations from over 40 countries have become involved in this agreement, which adopts the commitment to promote fair and transparent practices to evaluate science, with the aim of reaching an international scope. ANECA, the network of CERCA centers and CSIC, among others, have already signed this transforming commitment. CoARA is an international initiative launched in 2022 and promoted by the European University Association (USA) and the European Commission, among others.

More than 350 organisations from over 40 countries have signed CoARA, an international agreement to promote fair and transparent practices in science assessment, with the aim of international coverage.

Quality indicators, peer review and the responsible use of quantitative metrics are the cornerstones of the agreement. In essence, CoARA proposes to recognize the results, practices and activities that maximize the quality and impact of research. The scientific community itself has traditionally questioned the current research evaluation system, due to limitations, biases and reliance on limited metrics. The evaluation procedure used up to now often ignores the diversity of scientific contributions, the value and impact of academic work, restricts innovation and conditions the advancement of knowledge.

Joan Pino, CREAF’s director, states that "from the center we share the need for change and we want to be present from the outset in the new paradigm to which the CoARA Agreement contributes". However, Joan Pino admits that this international agreement opens a complex but necessary path: "Until now, numerical indicators have the virtue of being simple, although this is usually the only one they have. We have the strong will to incorporate elements beyond academic impact, but it is necessary to develop the metrics and this is a shared task, in which CREAF will collaborate with internal and external research management teams". The adhesion implies CREAF's commitment to implement a plan over the next 5 years.

“From the center we share the need for change and we want to be present from the outset in the new paradigm to which the CoARA Agreement contributes"

JOAN PINO, CREAF's director.

"If we want diversity, we must go beyond the number of publications"

TERESA ROSAS, CREAF's responsible for Talent and Gender

"The objective is to include the generation of knowledge and the scientific, technological, economic, cultural and social impacts of research"

TERESA ROSAS, CREAF's responsible for Impact

Previous
Next

At CREAF, Teresa Rosas –Talent and Gender officer– and Anabel Sánchez –Impact officer– have especially encouraged adherence to CoARA, given their responsibility in areas related to indicators. To explain why a new look at scientific rankings is key, Teresa Rosas incorporates the concept of diversity: "If we want diversity, we must go beyond the number of publications", according to her, and points out that she considers it logical that the CoARA Agreement coexists with quantitative indexes. For Rosas, the change proposed by CoARA involves taking into consideration individual but also collective contributions, valuing open science and promoting responsible leadership, for example. However, she is fully aware of the great work involved: "It is a huge challenge, it is not easy and it might be why it has not been done so far, because numbers a priori seem more objective and easy. It is a slow process and, like any cultural change, it involves time, patience and many diverse minds thinking to propose the best solutions".

Quality indicators, peer review and the responsible use of quantitative metrics are the cornerstones of the agreement, which proposes to recognise results, practices and activities that maximise the quality and impact of research.

For Anabel Sánchez, it is a matter of broadening the criteria, both by the organizations and the scientific system itself. The objective is to "include the generation of knowledge and the scientific, technological, economic, cultural and social impacts of research". Therefore, she considers essential to provide value to activities such as "knowledge transfer, training and mentoring, contributions to a positive research culture, interaction with citizens and decision makers, commitment with key social actors and their inclusion in the research process", to highlight the essential ones.

The 10 commitments of the CoARA Agreement

1. Recognise the diversity of contributions to, and careers in, research in accordance with the needs and nature of the research.
2. Base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators.
3. Abandon inappropriate uses in research assessment of journal- and publicationbased metrics, in particular inappropriate uses of Journal Impact Factor (JIF) and h-index.
4. Avoid the use of rankings from research organisations in research assessment.
5. Commit resources to reforming research assessment as is needed to achieve the organisational changes committed to.
6. Review and develop research assessment criteria, tools and processes.
7. Raise awareness of research assessment reform and provide transparent communication, guidance, and training on assessment criteria and processes as well as their use.
8. Exchange practices and experiences to enable mutual learning within and beyond the Coalition.
9. Communicate progress made on adherence to the Principles and implementation of the Commitments.
10. Evaluate practices, criteria and tools based on solid evidence and the state-ofthe-art in research on research, and make data openly available for evidence gathering and research.

Among the 350 participating organizations are public and private research funding agencies, universities, research centers and institutes, associations and alliances, national and regional authorities, accreditation and evaluation bodies, scientific societies and professional associations, representative of the multiple points of view that CoARA seeks to accommodate.