After an initial institutional day dedicated to reviewing HAL’s progress, the fifth annual session of the HAL Partners’ Assembly continued on 13 November with a thematic study day exploring the relationship between open archives and research evaluation.
The landscape of research evaluation is undergoing profound changes that challenge established paradigms. Against this backdrop, open archives such as HAL are emerging as transformative tools, offering fresh approaches to research evaluation that transcend traditional metrics and providing channels for dissemination that circumvent conventional publishing models.
Replays are available here.
Rethinking research evaluation
Noémie Aubert-Bonn, a researcher in information and communication sciences at Hasselt University and the University of Manchester, opened the day by questioning the foundations of researcher evaluation and the need for in-depth reform.
Drawing on interviews with researchers, she identified a key conflict between the criteria for research success (quality, openness and innovation) and the factors that are valued for career advancement (number of publications, journal impact factor, etc.). Current evaluation practices, which focus on end products and bibliometric indicators, largely ignore the content of the work and research processes themselves.
This model has several harmful effects, including precariousness, the justification of poor scientific practices, a lack of diversity in profiles and even mental health issues. In response to these systemic issues, Noémie Aubert-Bonn presented initiatives to promote more responsible evaluation. These include adopting the DORA and CoARA principles, promoting narrative CVs, broadening the concept of impact and training evaluators. Coordinated action is required from all stakeholders in the scientific ecosystem to implement these changes.
<iframe src=”https://www.canal-u.tv/chaines/ccsd/embed/169803?t=0″ width=”560″ height=”315″ allowfullscreen=”allowfullscreen”></iframe>
Update on the French chapter of CoARA
Fabien Borget, Open Science Project Manager at Aix-Marseille University and co-leader of the French chapter of CoARA, presented this international coalition, which was founded in 2022 with the aim of reforming research evaluation. With over 800 signatories spanning 18 national chapters, CoARA is founded on four core principles: recognising career diversity, prioritising qualitative peer review, promoting open-access publishing and steering clear of inappropriate institutional rankings.
Launched in October 2023, the French chapter brings together 64 signatory organisations and 92 participants representing 46 institutions. Its objectives are to promote the sharing of experiences, support institutional experimentation, and coordinate French action at an international level.
Fabien Borget emphasised the potential of open archives, particularly HAL, in this reform. This includes managing researcher identifiers, creating links between scientific outputs, ensuring international interoperability, and promoting open publishing practices. These are all key factors in supporting the cultural change required for a more responsible evaluation of research.
Equipping ourselves and supporting the assessment reform
Valérie Legué, Open Science Project Manager at Clermont-Auvergne University, opened the session by presenting the strategy of the institution, which is a signatory to DORA and CoARA. Partly due to the scientific community’s dissatisfaction with the evaluation process, in 2020 UCA adopted a framework document incorporating five key principles, initiating a local evaluation reform driven in particular by more open practices:
- Evaluate all research outputs
- Abandon the impact factor
- Rely on HAL
- Recognise preprints
- Promote diamond open access publications
HAL plays a central role in this, acting as a tool for transparency and making scientific output available. Through the OSQUAR project (winner of the CoARA Boost call for proposals), UCA is continuing its efforts, notably by developing a work package to advance the integration of open science criteria for more qualitative evaluation.
In their presentation, Nantes University showed how the institution is revamping its analysis of scientific output using SoVisu+, the CRIS module that it is gradually implementing as part of the CRISalid consortium. Emmanuelle Paulet-Grandguillot, Head of the Information Systems and Research Support Department and an elected member of the CCSD Steering Committee, and Guillaume Godet, Head of Bibliometrics, presented an approach based on a knowledge graph — a veritable ‘digital twin’ of research activity — which links publications (particularly those from HAL) to the institution’s internal data, including structures, projects, researcher identities and related activities. As an open and modular tool, SoVisu+ enables researchers to enrich and consolidate their data, thereby improving the quality of metadata and consistency across the entire research information system.
This approach addresses a structural issue: traditional bibliometric tools do not accurately reflect scientific output in a highly multidisciplinary institution. Nantes University therefore relies on HAL, its official bibliography, which is governed by a clear institutional policy. However, the indicators derived from HAL are incomplete. This CRIS overcomes this limitation.
The role of open science and HAL in research evaluation
The morning concluded with a roundtable discussion, which brought together several representatives from institutions involved in transforming research assessment: The participants were Nathalie Fargier (CCSD), Amélie Church (Sorbonne University), Caroline Gaucher (University of Lorraine), Marion Cipriano (Inserm) and Sylvie Rousset (CNRS). The discussion was moderated by Mickaël Ménager (University Paris Cité).
The discussion revealed a broad consensus that open science must play a role in evaluation, but without being reduced to a single criterion or automatic indicator. The speakers emphasised the importance of diversifying the evaluated outputs, improving the quality of metadata, basing analysis on open sources such as HAL or OpenAlex, and better accounting for the variety of research activities (data, projects, expertise, mediation, etc.).
The importance of qualitative peer review was emphasised throughout, as a means of avoiding pitfalls such as overemphasising quantitative factors, dependence on proprietary databases, and the risks associated with the uncontrolled use of AI. HAL therefore seems to be a reliable, open support tool that does not become the sole focus of evaluation. Above all, transformation relies on acculturation, the training of evaluators, and the collective development of new, fairer and more consistent open science practices.
International insights
Following an exploration of local and national issues and initiatives, a valuable comparative insight was offered in an international session, thanks to feedback from the Swiss Federal Institute of Technology in Lausanne, presented by Pascale Bouton, Head of the Digital Repositories & Archives Unit. The institution has opted for an open, modular and interoperable system to promote and manage its scientific output, with Infoscience — an institutional repository created in 2004 and now based on DSpace-CRIS. The deposit of publications is mandatory and is made easier by the widespread use of persistent identifiers. Infoscience aggregates data from multiple sources, including Scopus, OpenAlex, Arxiv, Zenodo and DataCite, and then consolidates it through extensive curation work.
The platform supports a vast ecosystem, including the display of publications from laboratories, synchronised researcher profiles (ORCID), open bibliometric analyses, thematic exploration tools, and networking tools for project calls. EPFL is also involved in ambitious national initiatives aimed at strengthening interoperability and developing a consistent Swiss infrastructure for open science.
The session continued with a presentation by Mathijs Vleugel, Head of the Open Science Office at the Helmholtz Association, Germany’s largest research organisation. The Helmholtz Association brings together 18 independent centres covering six major areas, including energy, health and the environment. The association coordinates multi-year research programmes and is committed to open science, with an annual programme structured around publication, open data and software, sovereign infrastructures, and responsible research.
Helmholtz is also conducting in-depth work on evaluation reform. In order to reduce reliance on traditional indicators such as the impact factor and publication volume, the organisation is developing quality indicators for data and software based on the FAIR principles, which are enriched with additional dimensions such as curation and scientific robustness. A survey of 1,145 researchers highlights a discrepancy between the criteria that are perceived as important and the criteria that are desired: collaboration appears to be the dimension that needs to be promoted the most. This work informs Helmholtz’s participation in COARA’s activities in Germany.
This study day, organised by the Partners’ Assembly, demonstrated the central role that open archives, open infrastructures and open science approaches now play in understanding, promoting and evaluating research. A common trend is emerging from French initiatives to European feedback: more interoperable tools, better structured data, more qualitative indicators, and greater attention to researchers’ actual practices.
As a national resource, a repository and a foundation for emerging uses, such as management, visualisation, analysis and interconnection with identifiers and repositories, HAL is fully committed to this dynamic. However, this ongoing transformation can only be achieved by engaging with communities, consolidating data, and encouraging the necessary cultural change.