The EOSCpilot Open Science Monitor Framework (EOSCpilot OSMF) aims to build a model and initial high-level specifications for providing useful analytics to researchers as well as enabling research performing and funding organizations within EOSC to monitor and gain insights about the OS movement, regardless of their service management systems and the technology behind them. Moreover, it aspires to become a dynamic tool in the future, with benefits both to the organisations using it to measure the OA levels of implementation and impact to their community and for the High-Level Stakeholders which are the European Union’s Legal Entities and Bodies, including the Member States and their respective Units. Gaps, implications and new ways of performing OS are among the elements that could be identified through EOSC OSM which could then be easily incorporated within the stakeholders’ scope and strategic planning for OS.
A prerequisite for the design of the framework is the definition of its objectives - goals and the identification of its core elements, i.e., what should be monitored and how. The methodology followed for deriving these concepts is presented and an overview is shown in the following Figure. The proposed approach is a methodological path to be used during the implementation of the framework, which can guide the initial design as well as any further refinements and extensions of the framework. It provides the steps that should be followed by an organization for deriving and choosing high level monitoring targets, identifying the OS elements to be measured, mapping monitoring targets to specific indicators and processes for data collection.
Figure 1: OS Monitor Methodological approach
The six core steps of the methodology are:
- Step 1. Identification of the Open Science Activities;
- Step 2. Policy-driven derivation of monitoring targets;
- Step 3. Identification of the main Open Science Resources and Indicators;
- Step 4. Design of monitoring processes, tasks and workflows;
- Step 5. Modelling and implementation of the framework;
- Step 6. Continuous validation of the monitoring targets;
Monitoring Targets and Dimensions of the EOSC OSM Framework
Monitoring targets capture high level aspects of the OS practices and trends and thus they are the main measurable dimensions of the OSM. Following the methodology, a taxonomy of targets and sub-targets, each of them addressing a different stakeholder group. Each target / sub-target is mapped to a set of indicators that provide quantifiable results (e.g., a score).
Aligned with most existing efforts, this categorization primarily considers aspects of openness, FAIRness, trustworthiness as well as the impact of OS in:
- research artefacts: mainly publications, research data and software;
- educational resources;
- research collaboration;
- citizen science practices.
The EOSC Open Science Monitor Framework provides a first taxonomy of the EOSC OSM Monitoring Targets, sub-targets and the associated dimensions that facilitate them. It should be noted that this categorization can be further extended and refined with more specific goals during the implementation of the framework addressing different stakeholder needs.
Figure 2: Monitoring Targets
The above figure consists of two circles, one containing basic elements/dimensions that characterise FAIR principles and the other concerning aspects of Open Access. Additional elements that are included in the figure deal with complementary issues, such as practices (e.g., peer review) and policies (e.g., skills). The middle part of the figure, the intersection of the two circles, highlights more prominent elements that accommodate common targets in both FAIR and OA spectrums. These common targets are displayed in bold and in the EOSC OSM Framework are the primary Monitoring Targets (MTs); more specifically these are: FAIR, Open (Access), Research Collaboration/Scholarly Communication, Trustworthiness, Skills and Readiness. The fact that, in the common circle, FAIR is separated by/per principle and is not represented as a whole, was purposely chosen to better reflect interconnection between dimensions (findability, accessibility, interoperability, reusability) and elements (PIDs, licenses etc) of FAIR and Open Access. Unique dimensions are kept outside of the common circle acknowledging disparities.
EOSC Monitoring Targets can be measured in the context of EOSC via a set of indicators, categorised per type of research artefact that they concern and enriched with interested stakeholders. The report presents the detailed mappings between targets and sub-targets, stakeholders and indicators in tabular form.