Measuring the openness of science


Written by Hélène Begnis

How can we measure the openness of science and the impact of the open science public policy that has been in place for several years now? This is the very purpose of the French Open Science Monitor (Baromètre français de la science ouverte – BSO) developed by the Ministry of Higher Education and Research, and the subject of our 6th webinar “Open Science talks” which brought together nearly 160 participants on 10th March 2023.

This webinar has helped to understand how HAL contributes to this measure, and how academic institutions can use the BSO. The latest developments of the BSO were also presented.

We were pleased to welcome: 

  • Laetitia Bracco, head of the “Atelier de la donnée” at Lorraine University and project leader of the French Open Science Monitor, 
  • Eric Jean Girard, Data scientist at French Ministry of higher education and research,
  • Anne-L’Hôte, engineer at French Ministry of higher education and research, 
  • Pierre Diaz, librarian, head of the research support service at IMT Mine-Albi.

Replay (french only):

The open science Monitor (Baromètre de la Science Ouverte – BSO) was launched by the first National Plan for open science in 2018. Its objective is to measure the evolution of open science in France using reliable, open and controlled data.

About the methodology

The contributions of HAL

This year, BSO takes into account all the articles with a HAL identifier, even if they do not have a CrossRef DOI identifier, contrary to what was done before. It is an option that can be activated for the local monitors. This enables to have a broader coverage of scientific production beyond CrossRef data, in particular for Human and Social Sciences.

The BSO also benefits from the authors’ affiliations managed in auréHAL. These affiliations feed the French publication database and facilitate the creation of local monitors which are based on these affiliations.

HAL is also a help for thematic classification of publication. If a publication is present in HAL and other sources, the classification proposed by HAL is taken into account in priority. The important metadata curation work is in fact a guarantee on the quality and reliability of the data collected.

The harvesting of HAL also enables to detect additional PhD theses when these have been self-archived.

What’s new at BSO 2023? Indicators for data and software codes!

Initially designed to measure the open access of publications, this latest version offers additional indicators – in beta version – on research data and software codes.

What are the objectives of this project devoted to measuring the openness of datasets and software? Be able to detect the datasets and software produced by the ESR as a whole because they are scientific productions in the same way as publications: to analyze them and produce monitoring and steering indicators. In order to guarantee transparency and facilitate the reuse of these results, it was decided to open the data, the approach, the algorithms and the results. It can be implemented for local monitors.

How are datasets and software identified? thanks to the text mining of open access publications which makes it possible to identify mentions relating to datasets and software!

The BSO is available at the local level

Organizations (university, research organization) can implement the BSO at their level. For this, they provide a list of publications and/or PhD theses in csv format. This step can be facilitated by providing the research structure and/or HAL collection identifier. Thanks to a “studio”, it is possible to visualize all the graphics produced from the data provided before their integration into the websites of the organizations.

The example of IMT Mines-Albi

IMT Mines-Albi is a school of the Institut Mines Télécom. The fields of research focus on health, industrial engineering, energy transition and aeronautics. Created in 2014, its HAL portal is managed by the documentation center to improve the visibility of the researchers’ publications and provide open access to their fulltext.

The dynamics in favor of open access is important for IMT Mines-Albi: creation of a documentalist position in charge of the scientific publications in 2017, referencing of all publications in HAL since 2018, use of an annual full-text deposit indicator extracted from HAL for the school dashboard.

IMT Mines-Albi took on very early the question of measuring the openness of science : production of the first “homemade” indicators in 2019, implementation of the methodology developed by the Lorraine University in 2020, and finally setting up a local version of the BSO according to the Ministry’s methodology in 2022.

After internal use of these indicators, it was decided to communicate them more widely, in particular through the creation of a dedicated web page.

The school benefits largely of this experience: management of the local open science policy and of its impact, monitoring of the impact of transformative agreements, monitoring of APCs. It is also beneficial for the librarians of the documentation center: they participate at the open science working group of the Mines Telecom group, and their expertise is well acknowledged at the Toulouse site level as at the national level.

A growing community

In order to support the ever-increasing number of local versions of the BSO, a user club was set up in March 2022. The goals are to federate skills, to share experiences, encourage initiatives, communicate on developments, collect user feedback, test functional advances.

To join the user club :

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.